View previous topic :: View next topic |
Author |
Message |
Pse Apprentice
Joined: 01 Mar 2005 Posts: 188 Location: by the plate river
|
Posted: Thu Mar 06, 2008 4:39 am Post subject: |
|
|
vinboy wrote: | WAT THE HELL!!! I thought my HDD was going to die.
The HDD is brand new.
I formatted my exterhal HDD (500GB) connected through USB2.0.
With XFS (used settings in the first post):
-When writing to the HDD, the max speed was 20MB/s
-The HDD sounded like it was going to explode! The head was moving here and there all the time!
With EXT2:
-Mas speed 29MB/s <---- 50% improvement over XFS.
-The writing operation is so quiet, hardly notice anything.
please advise what was going on?
[/list] |
You could maybe try the "nobarrier" mount option. Be aware that doing that may put you data at risk in the events of a crash. Also, upgrading to the latest kernels may improve performance a bit. |
|
Back to top |
|
|
pathfinder l33t
Joined: 19 Jan 2006 Posts: 731 Location: Barcelona, Spain
|
Posted: Wed Apr 23, 2008 1:35 pm Post subject: |
|
|
i did the test of untarring the kernel (by ssh though)
well. first result, with reiserfs in /
real 3m41.672s
user 0m53.231s
sys 0m7.997s
Then, copying the tarball in the xfs partition mounted with barriers logbufs and noatime
real 3m58.457s
user 0m53.663s
sys 0m7.356s
then same thing with the nobarrier (it is not safe when power failure):
real 3m26.725s
user 0m49.515s
sys 0m6.824s
Is this test relevant? or are the kernel files small ones?
I expected something much more significative... |
|
Back to top |
|
|
pilla Bodhisattva
Joined: 07 Aug 2002 Posts: 7729 Location: Underworld
|
Posted: Wed Apr 23, 2008 4:15 pm Post subject: |
|
|
Moved from Other Things Gentoo to Documentation, Tips & Tricks. _________________ "I'm just very selective about the reality I choose to accept." -- Calvin |
|
Back to top |
|
|
rada Apprentice
Joined: 21 Oct 2005 Posts: 202 Location: Ottawa, Canada
|
Posted: Mon Apr 28, 2008 1:30 am Post subject: |
|
|
"Increasing the number of allocation groups will decrease the space available in each group. For most workloads, filesystem configurations with a very small or very large number of allocation groups should be avoided."
http://oss.sgi.com/projects/xfs/training/xfs_lab_02_mkfs.pdf
From this I gather that with a SMP system, more allocation groups are good. And according to it, too few is not good. Maybe the default of 16 is just fine? |
|
Back to top |
|
|
kanaric1 n00b
Joined: 02 May 2008 Posts: 7
|
Posted: Fri May 02, 2008 4:31 pm Post subject: |
|
|
On my next system i'm leaning towards using XFS due to this topic.
Someone earlier said that are using, on their 64 bit system
Code: | mkfs.xfs -b size=8192 -l internal,size=128m -d agcount=20 /dev/sdb1 |
Would this be good on a 750GB HD, also have a quad core processor. Or could I maybe tweak it another way? Any suggestions? |
|
Back to top |
|
|
rada Apprentice
Joined: 21 Oct 2005 Posts: 202 Location: Ottawa, Canada
|
Posted: Fri May 02, 2008 6:43 pm Post subject: |
|
|
Bigger blocksize will increase performance but you will not be able to use that file system on a non-64bit system, ever. Also if you have many small files (<8KiB) a lot of space will be wasted. For allocation groups, too many will increase cpu usage when the file system is really full. Too few wont optimize usage across processors. 20 seems fine. |
|
Back to top |
|
|
kanaric1 n00b
Joined: 02 May 2008 Posts: 7
|
Posted: Fri May 02, 2008 7:24 pm Post subject: |
|
|
rada wrote: | Bigger blocksize will increase performance but you will not be able to use that file system on a non-64bit system, ever. Also if you have many small files (<8KiB) a lot of space will be wasted. For allocation groups, too many will increase cpu usage when the file system is really full. Too few wont optimize usage across processors. 20 seems fine. |
Well for the block size i will only ever be using 64 bit OS to use it.
Is the small files issue enough of a problem so that I should use the default setting or something lower? What would you recommend? |
|
Back to top |
|
|
rada Apprentice
Joined: 21 Oct 2005 Posts: 202 Location: Ottawa, Canada
|
Posted: Fri May 02, 2008 7:55 pm Post subject: |
|
|
It depends what this partition is for. If its for /, use a 4k blocksize. If its for mostly large data files, 8k blocksize is fine. |
|
Back to top |
|
|
Taily Tux's lil' helper
Joined: 19 Jun 2004 Posts: 115
|
Posted: Fri Jun 06, 2008 6:01 pm Post subject: |
|
|
Great thread, I love XFS and seeing this thread warms my heart .
I was really into optimizing XFS a while back and found some good pointers from this article.
I'm going to play around with some ideas from this thread now, I live for optimization!
Cheers. |
|
Back to top |
|
|
pdw_hu Apprentice
Joined: 02 Jun 2008 Posts: 200 Location: Budapest, Hungary
|
Posted: Fri Jun 06, 2008 8:12 pm Post subject: |
|
|
Just a small addendum, I set up my XFS partitions with Code: | mkfs.xfs -l size=64m /dev/xyz | , so no further options and it put them on agcounts=4 by default, compared to what the initial post said. Otherwise it works just fine :)
I'm gonna experiment a bit with the nobarrier option, but I've been using xfs on lvm2 (which means no barriers) for a few months now, before gentoo on slackware and I did experience a few crashes here and there, but no corruption ever occured. (Or i just didn't notice :D) |
|
Back to top |
|
|
rada Apprentice
Joined: 21 Oct 2005 Posts: 202 Location: Ottawa, Canada
|
Posted: Sun Jun 08, 2008 3:09 pm Post subject: |
|
|
Are you able to set write barriers on LVM partitions, or Device-Mapper partitions in general? |
|
Back to top |
|
|
pdw_hu Apprentice
Joined: 02 Jun 2008 Posts: 200 Location: Budapest, Hungary
|
Posted: Sun Jun 08, 2008 8:22 pm Post subject: |
|
|
rada wrote: | Are you able to set write barriers on LVM partitions, or Device-Mapper partitions in general? |
Nope. I meant that when i used that LVM setup it didn't have barriers but i didn't lose any data either. |
|
Back to top |
|
|
brfsa Tux's lil' helper
Joined: 01 Aug 2005 Posts: 121 Location: Brazil
|
Posted: Tue Jul 01, 2008 7:41 am Post subject: |
|
|
this is how i have my XFS partition on fstab.
/dev/sdd1 /mnt/backup xfs logbufs=8,logbsize=262144,biosize=16,noatime,nodiratime 0 1
some info:
# xfs_info /mnt/backup/
meta-data=/dev/sdd1 isize=256 agcount=4, agsize=19535700 blks
= sectsz=512 attr=2
data = bsize=4096 blocks=78142797, imaxpct=25
= sunit=0 swidth=0 blks
naming = version 2 bsize=4096
log = internal bsize=4096 blocks=32768, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime = none extsz=4096 blocks=0, rtextents=0 |
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
Posted: Wed Jul 02, 2008 5:47 pm Post subject: |
|
|
xfs really seems to offer nice performance (efficient space-usage is another story , with reiser4 only half of the space would be used)
is it normal that it needs around 100 minutes extracting an 4.1 GB stage4-tarball (2 different kernels directories, everything else is openoffice, kde3, kde4, gnome, xfce4) ?
the options I used where Code: | noatime,nodiratime,biosize=16,logbufs=8 | (during mount)
and Code: | -l size=128m -b size=4096 -i size=512 | during creation,
Quote: | rootfs 20G 14G 5.7G 71% / |
kernel used during extract was 2.6.25
thanks _________________ https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa
Hardcore Gentoo Linux user since 2004 |
|
Back to top |
|
|
rada Apprentice
Joined: 21 Oct 2005 Posts: 202 Location: Ottawa, Canada
|
Posted: Wed Jul 02, 2008 7:53 pm Post subject: |
|
|
Whats the cpu usage? Is the 'rootfs 20G 14G 5.7G 71% /' before or after extraction? |
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
|
Back to top |
|
|
rada Apprentice
Joined: 21 Oct 2005 Posts: 202 Location: Ottawa, Canada
|
Posted: Wed Jul 02, 2008 9:46 pm Post subject: |
|
|
If cpu usage is at 100% then you need a faster cpu . It seems tar is only single threaded. XFS will start to use lots of cpu when it gets really full and fragments a lot, usually >85% usage. |
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
Posted: Thu Jul 03, 2008 9:36 am Post subject: |
|
|
rada wrote: | If cpu usage is at 100% then you need a faster cpu . It seems tar is only single threaded. XFS will start to use lots of cpu when it gets really full and fragments a lot, usually >85% usage. |
naa, my cpu is fast enough for that kind of task
yeah, it's a shame that tar + bzip2 aren't multi-threaded by default,
I just made the same extraction yesterday with reiserfs and it only took 20 minutes vs. 100 minutes (xfs), is there something I could tweak ?
since I often play back a stage4-tarball waiting 1.5 hours for the partition isn't really satisfying ...
thanks _________________ https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa
Hardcore Gentoo Linux user since 2004 |
|
Back to top |
|
|
brfsa Tux's lil' helper
Joined: 01 Aug 2005 Posts: 121 Location: Brazil
|
Posted: Thu Jul 03, 2008 2:00 pm Post subject: |
|
|
you saying that on the "same machine" it takes 20 minutes using reiserfs?
same hardisk and hardware?
bzip2 taks a long time to compress and decompress if you use high level of compression...
(try using "lzma -2" very fast and still high compression)
what is agcount of your FS?
it should be 4 if you have dual core cpu. (CPU cores x 2).
can be set on fs creation only I believe.
anyone knows if u can change agcount after creation? |
|
Back to top |
|
|
brfsa Tux's lil' helper
Joined: 01 Aug 2005 Posts: 121 Location: Brazil
|
Posted: Fri Jul 04, 2008 1:29 pm Post subject: cpu usage |
|
|
I also got very high CPU usage when I use XFS as root partition during emerges. (MAKEOPTS="-j2")
maybe because agcount=4 will use both CPU cores, thus using all cpu power...
It actually might be a good thing...
if setting MAKEOPTS="-j4" load reaches 4.45 |
|
Back to top |
|
|
Enlight Advocate
Joined: 28 Oct 2004 Posts: 3519 Location: Alsace (France)
|
Posted: Wed Jul 09, 2008 8:54 am Post subject: |
|
|
brfsa wrote: | ...
what is agcount of your FS?
it should be 4 if you have dual core cpu. (CPU cores x 2).
|
WTH???????????????????? |
|
Back to top |
|
|
exif n00b
Joined: 07 Jul 2008 Posts: 57 Location: Waterloo, ON, Canada
|
Posted: Wed Jul 09, 2008 4:14 pm Post subject: |
|
|
@brfsa: The CPU spike during emerges is probably due to the compiling more than the FS. |
|
Back to top |
|
|
prymitive Apprentice
Joined: 13 Jun 2004 Posts: 260
|
Posted: Thu Aug 07, 2008 10:22 pm Post subject: |
|
|
kernelOfTruth wrote: | xfs really seems to offer nice performance (efficient space-usage is another story , with reiser4 only half of the space would be used)
is it normal that it needs around 100 minutes extracting an 4.1 GB stage4-tarball (2 different kernels directories, everything else is openoffice, kde3, kde4, gnome, xfce4) ?
the options I used where Code: | noatime,nodiratime,biosize=16,logbufs=8 | (during mount)
and Code: | -l size=128m -b size=4096 -i size=512 | during creation,
Quote: | rootfs 20G 14G 5.7G 71% / |
kernel used during extract was 2.6.25
thanks |
You got write barriers by default, use mount -o nobarrier and You will see the difference. With barriers on (default), xfs doesn't really use much RAM for write buffers to be more power off safe. |
|
Back to top |
|
|
jsosic Guru
Joined: 02 Aug 2004 Posts: 510 Location: Split (Croatia)
|
Posted: Sun Aug 10, 2008 1:23 am Post subject: |
|
|
I'm glad to see that this thread is still alive and kickin' ! _________________ I avenge with darkness, the blood is the life
The Order of the Dragon, I feed on human life |
|
Back to top |
|
|
TSP__ n00b
Joined: 16 Sep 2008 Posts: 21
|
Posted: Tue Sep 16, 2008 8:24 pm Post subject: |
|
|
I been using XFS for a while, now...i am thinking in tweak a bit my fstab. i used mkfs.xfs without options to make my / and also for my /home. it's is safe to add
right now? i only use noatime in fstab for xfs in both partitions. Any other hint?
Cheers! |
|
Back to top |
|
|
|