Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
XFS on steroids
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4, 5  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
Pse
Apprentice
Apprentice


Joined: 01 Mar 2005
Posts: 188
Location: by the plate river

PostPosted: Thu Mar 06, 2008 4:39 am    Post subject: Reply with quote

vinboy wrote:
WAT THE HELL!!! I thought my HDD was going to die.
The HDD is brand new.

I formatted my exterhal HDD (500GB) connected through USB2.0.

With XFS (used settings in the first post):
-When writing to the HDD, the max speed was 20MB/s
-The HDD sounded like it was going to explode! The head was moving here and there all the time!

With EXT2:
-Mas speed 29MB/s <---- 50% improvement over XFS.
-The writing operation is so quiet, hardly notice anything.

please advise what was going on?
[/list]


You could maybe try the "nobarrier" mount option. Be aware that doing that may put you data at risk in the events of a crash. Also, upgrading to the latest kernels may improve performance a bit.
Back to top
View user's profile Send private message
pathfinder
l33t
l33t


Joined: 19 Jan 2006
Posts: 731
Location: Barcelona, Spain

PostPosted: Wed Apr 23, 2008 1:35 pm    Post subject: Reply with quote

i did the test of untarring the kernel (by ssh though)

well. first result, with reiserfs in /
real 3m41.672s
user 0m53.231s
sys 0m7.997s


Then, copying the tarball in the xfs partition mounted with barriers logbufs and noatime
real 3m58.457s
user 0m53.663s
sys 0m7.356s


then same thing with the nobarrier (it is not safe when power failure):
real 3m26.725s
user 0m49.515s
sys 0m6.824s


Is this test relevant? or are the kernel files small ones?
I expected something much more significative...
Back to top
View user's profile Send private message
pilla
Administrator
Administrator


Joined: 07 Aug 2002
Posts: 7214
Location: Pelotas, BR

PostPosted: Wed Apr 23, 2008 4:15 pm    Post subject: Reply with quote

Moved from Other Things Gentoo to Documentation, Tips & Tricks.
_________________
"I'm just very selective about the reality I choose to accept." -- Calvin
Back to top
View user's profile Send private message
rada
Apprentice
Apprentice


Joined: 21 Oct 2005
Posts: 202
Location: Ottawa, Canada

PostPosted: Mon Apr 28, 2008 1:30 am    Post subject: Reply with quote

"Increasing the number of allocation groups will decrease the space available in each group. For most workloads, filesystem configurations with a very small or very large number of allocation groups should be avoided."

http://oss.sgi.com/projects/xfs/training/xfs_lab_02_mkfs.pdf

From this I gather that with a SMP system, more allocation groups are good. And according to it, too few is not good. Maybe the default of 16 is just fine?
Back to top
View user's profile Send private message
kanaric1
n00b
n00b


Joined: 02 May 2008
Posts: 7

PostPosted: Fri May 02, 2008 4:31 pm    Post subject: Reply with quote

On my next system i'm leaning towards using XFS due to this topic.

Someone earlier said that are using, on their 64 bit system
Code:
mkfs.xfs -b size=8192 -l internal,size=128m -d agcount=20 /dev/sdb1


Would this be good on a 750GB HD, also have a quad core processor. Or could I maybe tweak it another way? Any suggestions?
Back to top
View user's profile Send private message
rada
Apprentice
Apprentice


Joined: 21 Oct 2005
Posts: 202
Location: Ottawa, Canada

PostPosted: Fri May 02, 2008 6:43 pm    Post subject: Reply with quote

Bigger blocksize will increase performance but you will not be able to use that file system on a non-64bit system, ever. Also if you have many small files (<8KiB) a lot of space will be wasted. For allocation groups, too many will increase cpu usage when the file system is really full. Too few wont optimize usage across processors. 20 seems fine.
Back to top
View user's profile Send private message
kanaric1
n00b
n00b


Joined: 02 May 2008
Posts: 7

PostPosted: Fri May 02, 2008 7:24 pm    Post subject: Reply with quote

rada wrote:
Bigger blocksize will increase performance but you will not be able to use that file system on a non-64bit system, ever. Also if you have many small files (<8KiB) a lot of space will be wasted. For allocation groups, too many will increase cpu usage when the file system is really full. Too few wont optimize usage across processors. 20 seems fine.


Well for the block size i will only ever be using 64 bit OS to use it.

Is the small files issue enough of a problem so that I should use the default setting or something lower? What would you recommend?
Back to top
View user's profile Send private message
rada
Apprentice
Apprentice


Joined: 21 Oct 2005
Posts: 202
Location: Ottawa, Canada

PostPosted: Fri May 02, 2008 7:55 pm    Post subject: Reply with quote

It depends what this partition is for. If its for /, use a 4k blocksize. If its for mostly large data files, 8k blocksize is fine.
Back to top
View user's profile Send private message
Taily
Tux's lil' helper
Tux's lil' helper


Joined: 19 Jun 2004
Posts: 115

PostPosted: Fri Jun 06, 2008 6:01 pm    Post subject: Reply with quote

Great thread, I love XFS and seeing this thread warms my heart :).
I was really into optimizing XFS a while back and found some good pointers from this article.

I'm going to play around with some ideas from this thread now, I live for optimization!

Cheers.
Back to top
View user's profile Send private message
pdw_hu
Apprentice
Apprentice


Joined: 02 Jun 2008
Posts: 200
Location: Budapest, Hungary

PostPosted: Fri Jun 06, 2008 8:12 pm    Post subject: Reply with quote

Just a small addendum, I set up my XFS partitions with
Code:
mkfs.xfs -l size=64m /dev/xyz
, so no further options and it put them on agcounts=4 by default, compared to what the initial post said. Otherwise it works just fine :)

I'm gonna experiment a bit with the nobarrier option, but I've been using xfs on lvm2 (which means no barriers) for a few months now, before gentoo on slackware and I did experience a few crashes here and there, but no corruption ever occured. (Or i just didn't notice :D)
Back to top
View user's profile Send private message
rada
Apprentice
Apprentice


Joined: 21 Oct 2005
Posts: 202
Location: Ottawa, Canada

PostPosted: Sun Jun 08, 2008 3:09 pm    Post subject: Reply with quote

Are you able to set write barriers on LVM partitions, or Device-Mapper partitions in general?
Back to top
View user's profile Send private message
pdw_hu
Apprentice
Apprentice


Joined: 02 Jun 2008
Posts: 200
Location: Budapest, Hungary

PostPosted: Sun Jun 08, 2008 8:22 pm    Post subject: Reply with quote

rada wrote:
Are you able to set write barriers on LVM partitions, or Device-Mapper partitions in general?


Nope. I meant that when i used that LVM setup it didn't have barriers but i didn't lose any data either.
Back to top
View user's profile Send private message
brfsa
Tux's lil' helper
Tux's lil' helper


Joined: 01 Aug 2005
Posts: 121
Location: Brazil

PostPosted: Tue Jul 01, 2008 7:41 am    Post subject: Reply with quote

this is how i have my XFS partition on fstab.

/dev/sdd1 /mnt/backup xfs logbufs=8,logbsize=262144,biosize=16,noatime,nodiratime 0 1


some info:

# xfs_info /mnt/backup/
meta-data=/dev/sdd1 isize=256 agcount=4, agsize=19535700 blks
= sectsz=512 attr=2
data = bsize=4096 blocks=78142797, imaxpct=25
= sunit=0 swidth=0 blks
naming = version 2 bsize=4096
log = internal bsize=4096 blocks=32768, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime = none extsz=4096 blocks=0, rtextents=0
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5721
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Wed Jul 02, 2008 5:47 pm    Post subject: Reply with quote

xfs really seems to offer nice performance (efficient space-usage is another story :lol: , with reiser4 only half of the space would be used)

is it normal that it needs around 100 minutes extracting an 4.1 GB stage4-tarball (2 different kernels directories, everything else is openoffice, kde3, kde4, gnome, xfce4) ? 8O

the options I used where
Code:
noatime,nodiratime,biosize=16,logbufs=8
(during mount)

and
Code:
-l size=128m -b size=4096 -i size=512
during creation,

Quote:
rootfs 20G 14G 5.7G 71% /


kernel used during extract was 2.6.25

thanks
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.3.0-r2
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
rada
Apprentice
Apprentice


Joined: 21 Oct 2005
Posts: 202
Location: Ottawa, Canada

PostPosted: Wed Jul 02, 2008 7:53 pm    Post subject: Reply with quote

Whats the cpu usage? Is the 'rootfs 20G 14G 5.7G 71% /' before or after extraction?
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5721
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Wed Jul 02, 2008 8:46 pm    Post subject: Reply with quote

rada wrote:
Whats the cpu usage? Is the 'rootfs 20G 14G 5.7G 71% /' before or after extraction?


cpu usage is around 100% for 1 core (core2 duo), bzip2 + tar,

the space-usage is after extraction and pretty on par with reiserfs
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.3.0-r2
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
rada
Apprentice
Apprentice


Joined: 21 Oct 2005
Posts: 202
Location: Ottawa, Canada

PostPosted: Wed Jul 02, 2008 9:46 pm    Post subject: Reply with quote

If cpu usage is at 100% then you need a faster cpu :P. It seems tar is only single threaded. XFS will start to use lots of cpu when it gets really full and fragments a lot, usually >85% usage.
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5721
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Thu Jul 03, 2008 9:36 am    Post subject: Reply with quote

rada wrote:
If cpu usage is at 100% then you need a faster cpu :P. It seems tar is only single threaded. XFS will start to use lots of cpu when it gets really full and fragments a lot, usually >85% usage.


naa, my cpu is fast enough for that kind of task :wink:

yeah, it's a shame that tar + bzip2 aren't multi-threaded by default,

I just made the same extraction yesterday with reiserfs and it only took 20 minutes vs. 100 minutes (xfs), is there something I could tweak ?

since I often play back a stage4-tarball waiting 1.5 hours for the partition isn't really satisfying ...

thanks
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.3.0-r2
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
brfsa
Tux's lil' helper
Tux's lil' helper


Joined: 01 Aug 2005
Posts: 121
Location: Brazil

PostPosted: Thu Jul 03, 2008 2:00 pm    Post subject: Reply with quote

you saying that on the "same machine" it takes 20 minutes using reiserfs?
same hardisk and hardware?

bzip2 taks a long time to compress and decompress if you use high level of compression...
(try using "lzma -2" very fast and still high compression)

what is agcount of your FS?

it should be 4 if you have dual core cpu. (CPU cores x 2).

can be set on fs creation only I believe.

anyone knows if u can change agcount after creation?
Back to top
View user's profile Send private message
brfsa
Tux's lil' helper
Tux's lil' helper


Joined: 01 Aug 2005
Posts: 121
Location: Brazil

PostPosted: Fri Jul 04, 2008 1:29 pm    Post subject: cpu usage Reply with quote

I also got very high CPU usage when I use XFS as root partition during emerges. (MAKEOPTS="-j2")
maybe because agcount=4 will use both CPU cores, thus using all cpu power...

It actually might be a good thing...

if setting MAKEOPTS="-j4" load reaches 4.45 8O
Back to top
View user's profile Send private message
Enlight
Advocate
Advocate


Joined: 28 Oct 2004
Posts: 3514
Location: Alsace (France)

PostPosted: Wed Jul 09, 2008 8:54 am    Post subject: Reply with quote

brfsa wrote:
...
what is agcount of your FS?

it should be 4 if you have dual core cpu. (CPU cores x 2).



WTH???????????????????? 8O
_________________
le "lol" est aux boulets ce que le ";" est au programmeurs
Back to top
View user's profile Send private message
exif
n00b
n00b


Joined: 07 Jul 2008
Posts: 57
Location: Waterloo, ON, Canada

PostPosted: Wed Jul 09, 2008 4:14 pm    Post subject: Reply with quote

@brfsa: The CPU spike during emerges is probably due to the compiling more than the FS.
Back to top
View user's profile Send private message
prymitive
Apprentice
Apprentice


Joined: 13 Jun 2004
Posts: 260

PostPosted: Thu Aug 07, 2008 10:22 pm    Post subject: Reply with quote

kernelOfTruth wrote:
xfs really seems to offer nice performance (efficient space-usage is another story :lol: , with reiser4 only half of the space would be used)

is it normal that it needs around 100 minutes extracting an 4.1 GB stage4-tarball (2 different kernels directories, everything else is openoffice, kde3, kde4, gnome, xfce4) ? 8O

the options I used where
Code:
noatime,nodiratime,biosize=16,logbufs=8
(during mount)

and
Code:
-l size=128m -b size=4096 -i size=512
during creation,

Quote:
rootfs 20G 14G 5.7G 71% /


kernel used during extract was 2.6.25

thanks


You got write barriers by default, use mount -o nobarrier and You will see the difference. With barriers on (default), xfs doesn't really use much RAM for write buffers to be more power off safe.
Back to top
View user's profile Send private message
jsosic
Guru
Guru


Joined: 02 Aug 2004
Posts: 510
Location: Split (Croatia)

PostPosted: Sun Aug 10, 2008 1:23 am    Post subject: Reply with quote

I'm glad to see that this thread is still alive and kickin' ! :)
_________________
I avenge with darkness, the blood is the life
The Order of the Dragon, I feed on human life
Back to top
View user's profile Send private message
TSP__
n00b
n00b


Joined: 16 Sep 2008
Posts: 21

PostPosted: Tue Sep 16, 2008 8:24 pm    Post subject: Reply with quote

I been using XFS for a while, now...i am thinking in tweak a bit my fstab. i used mkfs.xfs without options to make my / and also for my /home. it's is safe to add

Code:

logbufs=8


right now? i only use noatime in fstab for xfs in both partitions. Any other hint?

Cheers!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3, 4, 5  Next
Page 4 of 5

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum