The Filesystem choice thread

Message

phong · Post by **phong** » Tue Nov 05, 2002 1:45 pm

I'm making this the "master" filesystem choice discussion thread and moving it to Gentoo Chat. It's become more of a discussion topic than a support question, and I'd like to be able to concentrate all the "which filesystem should I use" discussion to one thread. I've changed the subject of the thread to reflect this, and also made it sticky.

For those who are reading this thread looking for Yet Another Opinion(tm), I'd like to plug Reiserfs. It's stable in current versions of the Kernel (note that Ext3 and XFS have also had corruption problems in the past, and apparently XFS has some problems currently), and it's very fast for large numbers of files. For example, manipulating hundreds of megs of e-mail in maildir format goes very very quickly (tens of thousands of messages can be inadvertantly deleted in seconds!)

Also, reiser4 looks like it will change the world.

eryvile · Post by **eryvile** » Tue Nov 05, 2002 8:33 pm

hook wrote:ok, this will sound very very stupid, but i will still make an attempt:

what are the main characteristics of all these systems? (reiser, xfs, ext3, etc. etc.) ...i mean most of the posts were related to users personal favour.

In the german magazine 'iX', they just (10/02) had a comparison of the different journaling file systems. Unfortunately that article isn't online, but the bottom line, as far as I remember it, is that compared to FreeBSD's Softupdates, fastest is Reiser, closely followed by XFS and then, with a little bit larger gap, ext3. Concerning functionality, all three imho offer pretty much the same.

To get back to the personal favour (which is kind of necessary in my opinion as there isn't much real difference among the journaling file systems for Linux), I have XFS on my laptop (it was the recommendation for Gentoo 1.2 according to the docs, no problems so far, neither with 1.2 or 1.4 rc 1, 2.4.18 nor 2.4.19 XFS-sources), ext3 on my servers (it was said to be the safest and for a two person household, performance isn't really an issue on a file- or database-server

) and reiserfs on my workstation (with Gentoo 1.4 rc1 the recommendation in the docs has changed a little and I wanted to try something new

). With neither of those choices I had any problems

. Performancewise, I can't really tell you any difference, as all computers have different harddisk performances, i.e.

Code: Select all

hdparm -t /dev/hda

gets me most of 16 MB/sec on this laptop while 45 to 50 MB/sec on my workstation

And my servers are old computers that don't even know what udma is

boyo · Post by **boyo** » Tue Nov 05, 2002 8:56 pm

45 to 50 MB/s?

For an EIDE drive? What kind of hdd/controller setup does your workstation have?

hook · Post by **hook** » Tue Nov 05, 2002 9:20 pm

ok, this will sound really dumb, but i think it's an importaint question:
is there a difference between these FSs in the point that some programs don't / won't work under certain FSs?

...i appologise myself again for the stupid question, but i had to ask

phong · Post by **phong** » Tue Nov 05, 2002 9:35 pm

Short answer - no. Long answer - maybe, but only if you're into kernel hacking or heavy-wizardry. For the end user, unless there's a bug, the only noticable difference will be performance.

eryvile · Post by **eryvile** » Tue Nov 05, 2002 9:45 pm

boyo wrote:45 to 50 MB/s? For an EIDE drive? What kind of hdd/controller setup does your workstation have?

It's one of those IBM ICxyz 40GB drives with 7200 rpms, the mobo is a MSI KT266A RU (the drive is connected to the standard ide, not the raid!). These numbers are what I remember (I'm currently on the road, so I can't check proof on my statmement above) I hope, I didn't exgerate, although, as seen here (escpecially when you look at the numbers of "squanto"), they don't seem to be really that far off. I've compiled all VIA-chipset-features into the kernel (latest gentoo-sources), and as I could see with dmesg UDMA-100 is set up at boot time. I will give proof (hopefully!) and copy the output of hdparm -t here at the weekend!

KiTaSuMbA · Post by **KiTaSuMbA** » Tue Nov 05, 2002 11:26 pm

For the file systems:
I use XFS for the / since I first installed gentoo 1.2 with no troubles at all (mind you, I have experienced inumerous power losses and brown-outs since then). Ext2 and ext3 are also worth their bucks in reliability but are quite more slow. I haven't tried reiserFS since the first versions of 2.4.x kernels (back then it was very unreliable) so I can't refer personally on its current state - people seem very happy with it though...
And now I have a question: do the current gentoo-sources support XFS? I've sticked with the xfs- branch but I would like to try preemptive. Any specific DONTs with using preemptive with ALSA and XFS?

For the hdparm "sub-issue":
My hdb (gentoo) is an ATA100 disk and goes at 34-38 Mb/s while hda (udma66 - windows and slack) goes at about 24-26. I know a faster drive shouldn't be a slave, but windows are rather picky on going for first disks... This will probably change as I'm in desperate need of extra disk space and think of "upgrading" to XP (installing it on a fresh hda, too many have had nightmares with drives containing non windows partitions).

rac · Post by **rac** » Tue Nov 05, 2002 11:36 pm

KiTaSuMbA wrote:do the current gentoo-sources support XFS?

Depends on how you define "current". For all intents and purposes, xfs-2.4.19-r2 and gentoo-2.4.19-r10 are the exact same kernel - you have to have USE="xfs" defined when emerging gentoo-r10 to get XFS support, however. gentoo-r10 appears to still be marked as "~x86", but I've been using it for almost two weeks now - I needed XFS too.

KiTaSuMbA · Post by **KiTaSuMbA** » Tue Nov 05, 2002 11:51 pm

So, the -r2 xfs sources have preemptive bundled... I'm using the -r1 right now and didn't notice the options (either there weren't any, or I'm getting blind). Since, I wouldn't like using unstable code on this machine, I guess emerge xfs-2.4.19-r2 is the prefered course of action here...
Thanx for the feedback

rac · Post by **rac** » Tue Nov 05, 2002 11:59 pm

KiTaSuMbA wrote:So, the -r2 xfs sources have preemptive bundled... I'm using the -r1 right now and didn't notice the options (either there weren't any, or I'm getting blind).

I don't think they're there. There's a world of difference between xfs-r1 and -r2.

Since, I wouldn't like using unstable code on this machine, I guess emerge xfs-2.4.19-r2 is the prefered course of action here... Thanx for the feedback

I hope the

means you realize that installing xfs-r2 is every bit as stable or unstable as gentoo-r10. You might be interested in the fine-grained patch control, so that you can decide for yourself which patches get included in recent lolo kernels. Read the ebuilds for details (look for KERNEL_EXCLUDE).

boyo · Post by **boyo** » Wed Nov 06, 2002 12:13 am

From what I've checked out it seems that 40-50 MB/s is about right for the upper performance limits of ATA-100 drives and controllers. I've only been able around 35 or so MB/s out of my 7200 rpm ATA-100 WD400BB (40GB w/ 2MB cache) on a Highpoint 370 controller. On paper the WD drive I'm using matches up evenly w/ the IBM Deskstar models, and I doubt my controller makes 5-15 MB/s worth of difference. I'm not doubting your earlier post (eryvile). I'd just love to match it.

Which hdparm flags are you using?

squanto · Post by **squanto** » Wed Nov 06, 2002 2:23 am

eryvile wrote:
boyo wrote:45 to 50 MB/s? For an EIDE drive? What kind of hdd/controller setup does your workstation have?
I hope, I didn't exgerate, although, as seen here (escpecially when you look at the numbers of "squanto"), they don't seem to be really that far off.

I got a plug in a thread I didn't even participate in

leet! I have special edition 80gig WD drive on ultra 100 bus on Epox 8KHA+ board. no raid

I can't afford it now

I user Reiser, the main advantage that I see is the -tail -notail options. When you use standard ext2 or ext3 partition type the smallest a file can be is either 1k, 2k or 4k with 4k being the standard. With Reiser, you can use the -tail option and any space left over in side of the small files can be used.
my example:

ext2 4k file size limit [ | | | | ] each | is a k
so say you pack in only two k into it, we get [ | | _ _ ] with two empty spots, not very efficient

With Reiser, in a 4k spot we can do this, take 2 different 2k files, put them in same spot, so instead of what we get for 2 files inext2 : [ | | _ _ ] and [ | | _ _ ]
we now get [ | | | | ] in reiser, thus saving 4k of space

Get it?
If not it is cause I am wierd

boyo · Post by **boyo** » Wed Nov 06, 2002 5:25 am

I'd expect a drive with a larger cache to deliver better performance so I'm not terribly surprised that you're getting the transfer rates you are. Especially since the WD special edition drives also support ATA-133. I'd still like to know how to go about getting 40-45 MB/s with an ATA-100 controller and a 7200 RPM 2MB cache hdd. I thought my hdd was hdparm'ed to it's fullest potential, but apparently not. any tips would be welcomed and appreciated.

Back to the actual subject of the thread...

Does the Reiserfs -notail option have an impact on performance at all? It's definitely a cool feature, but what's it cost?

phong · Post by **phong** » Wed Nov 06, 2002 5:34 am

Using the "tail" option is more space efficient, but a bit slower (not hugely apparently). The "notail" option is what people usually recommend because it's faster, and even with that, reiser tends to be more space effiecent than the others.

eryvile · Post by **eryvile** » Wed Nov 06, 2002 7:43 pm

boyo wrote:I'm not doubting your earlier post (eryvile). I'd just love to match it. Which hdparm flags are you using?

Boy, do I feel miserable, throwing out big numbers without been able to proof them

. I definitely had too much beer last night when writing those posts

I don't want to say that those numbers are completely wrong as I'm still really sure about having seen them when typing hdparm -t /dev/hdb on my workstation at home. But it might also just be some halucination (as I for sure would love to have those numbers). I promise to post the real ones as soon as I get back home. I'm on the road during the week, so that will be Saturday!!!

Again, please excuse my unreflected writing! Lesson learned: don't drink and post

hook · Post by **hook** » Wed Nov 06, 2002 7:53 pm

ok, ...so the conclusion would be that if you are using your linuxbox as an usual home workstation, you don't have to care about the difference, because you would not get any profit out of it?

boyo · Post by **boyo** » Wed Nov 06, 2002 8:15 pm

Honestly, hdparm stats are hardly anything to feel miserable about.

The intent of my question was curiosity, not to discredit anyone. If 45-50 MB/s was mis-read, no biggie. If not give me a hand tweaking hdparm for sure.

Anyway, phong, thanks for the earlier link to Reiser4. The plugin funtionality sounds awesome. Do you have any idea which ones will be availible upon realease? I'll definitely be inclined to give it a try.

squanto · Post by **squanto** » Wed Nov 06, 2002 9:42 pm

boyo wrote:Does the Reiserfs -notail option have an impact on performance at all? It's definitely a cool feature, but what's it cost?

From what I have read, about 5% loss when using -tail.
Although about 6% average gain in useable hd space, so there is a good advantage to using -tail.

boyo · Post by **boyo** » Wed Nov 06, 2002 10:16 pm

I'm guessing XFS has no such feature. I suppose I'll have to do some additional reading to adequately compare how it manages space in comparison to Reiserfs. I'd be willing to bet that it doesn't have the same degree of flexibility, but there's probably a distinct performance gap between Reiserfs w/ the -tail option and XFS. Do you see any noticeable differences in CPU usage between -tail and -notail?

Thanks for the good info.

squanto · Post by **squanto** » Thu Nov 07, 2002 3:00 am

The default is to run with the tail option enabled, I just found that out. So I have been running all the time with tail on, and writing to disk uses less than 1% of cpu cycles. So I would say that it is pretty good at not using the cpu

I have had no problems so far with reiser.

metalhedd · Post by **metalhedd** » Thu Nov 07, 2002 3:11 am

Ok I think It's generally understood that Reiser is the better FS for many small files, but nobody ever specified what is considered "small" are we talking about 5k config files? or 5MB MP3s?either could be considered small

boyo · Post by **boyo** » Thu Nov 07, 2002 3:53 am

The "small" range in question is pretty trivial - something like < 10k. Nothing signifigant over XFS with files above that size. Knowing that XFS was built by SGI from the ground up for handling larger media files (mp3, ogg, mpg, avi, images, etc.) is about the only reason it's something I'm still considering (I've read some great stuff about Reiserfs on this thread). Much of my space consuming data consists of those file types, and XFS seems to have the upper hand when dealing with them. phong posted a link to Reiser4 earlier in the thread. Hopefully when I've finished reading it I'll be convinced one way or the other. I have been using Reiserfs, but XFS has always been in the back of mind since I've been quite impressed by it as well. I've been keeping close track of this thread hoping for something distinctly convincing.

pilla · Post by **pilla** » Thu Nov 07, 2002 4:14 am

I first tried ReiserFS when it did not have quota support (too bad I just learned that after installing it).

I am using EXT3 in my notebook (cannot lost data!) and in my desktop, no problems at all until now. We are using it in some servers, too. About performance, I cannot say anything but that it is not slow enough to be a problem for me... I do not have another FS to compare, right now.

EXT3 has the advantage of being widelly supported, any grub or kernel usually can mount at least EXT2 filesystems (and you can mount EXT3 without jornal, which is just EXT2).

boyo · Post by **boyo** » Thu Nov 07, 2002 4:29 am

C'mon, Where's your sense of adventure?

ext3 has done well when I've used it, but that hasn't been as extensively as Reiserfs or XFS (both have been outstanding). Being the performance junkie I am if there's something faster that's not ridiculously unsafe I'm all about it. The mounting without a journal sounds handy though.

lupad · Post by **lupad** » Thu Nov 07, 2002 11:22 am

boyo wrote:Honestly, hdparm stats are hardly anything to feel miserable about. The intent of my question was curiosity, not to discredit anyone. If 45-50 MB/s was mis-read, no biggie. If not give me a hand tweaking hdparm for sure.

Umm...

Code: Select all

sahara root # hdparm /dev/hda

/dev/hda:
 multcount    = 16 (on)
 IO_support   =  0 (default 16-bit)
 unmaskirq    =  0 (off)
 using_dma    =  0 (off)
 keepsettings =  0 (off)
 readonly     =  0 (off)
 readahead    =  8 (on)
 geometry     = 3738/255/63, sectors = 60058656, start = 0

sahara root # hdparm -t /dev/hda

/dev/hda:
 Timing buffered disk reads:  64 MB in 20.78 seconds =  3.08 MB/sec

sahara root # hdparm -d 1 -c 1 -X 69 /dev/hda

/dev/hda:
 setting 32-bit IO_support flag to 1
 setting using_dma to 1 (on)
 setting xfermode to 69 (UltraDMA mode5)
 IO_support   =  1 (32-bit)
 using_dma    =  1 (on)

sahara root # hdparm -t /dev/hda

/dev/hda:
 Timing buffered disk reads:  64 MB in  1.28 seconds = 50.00 MB/sec
sahara root # hdparm /dev/hda

/dev/hda:
 multcount    = 16 (on)
 IO_support   =  1 (32-bit)
 unmaskirq    =  0 (off)
 using_dma    =  1 (on)
 keepsettings =  0 (off)
 readonly     =  0 (off)
 readahead    =  8 (on)
 geometry     = 3738/255/63, sectors = 60058656, start = 0

sahara root # cat /proc/ide/hda/model
Maxtor 6E030L0

Best drive I ever had!