| View previous topic :: View next topic |
| Author |
Message |
XenoTerraCide
Veteran
Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA
|
 Posted: Mon Dec 19, 2005 1:39 am Post subject: |
 |
|
| Code: | | Filesystem features: has_journal dir_index filetype needs_recovery sparse_super |
what do the last 2 features mean? |
|
| Back to top |
|
 |
codergeek42
Bodhisattva
Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)
|
| Posted: Mon Dec 19, 2005 2:23 am Post subject: |
|
|
needs_recovery means that the filesystem is marked "dirty". If it is currently mounted, then this is perfectly normal behavior, as it will be marked "clean" when it is unmounted normally. This is to tell the startup scripts that the partition needs to be fsck'ed before it can be used again if, for example, a power outage occurs and the partition is not unmounted correctly.
sparse_super means that the filesystem stores less backup copies of the superblock, which increases the space available to the standard filesystem semantics, and is usually very safe since there are still many backup superblock copies anyway.
Edit: Some minor wording corrections.
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF
Last edited by codergeek42 on Mon Dec 19, 2005 2:31 am; edited 1 time in total |
|
| Back to top |
|
|
XenoTerraCide
Veteran
Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA
|
| Posted: Mon Dec 19, 2005 2:29 am Post subject: |
|
|
| yeah it's mounted... and otherwise says clean... thanks... |
|
| Back to top |
|
|
jedsen
Apprentice
Joined: 15 Oct 2004
Posts: 276
Location: Sacramento, California, USA
|
| Posted: Tue Dec 20, 2005 10:10 pm Post subject: |
|
|
My reiserfs box still beats out my ext3 box when emerge syncing by about 2-3x, even after these tweaks. And no problems yet (knock on wood).
Last edited by jedsen on Thu Dec 22, 2005 10:44 am; edited 1 time in total |
|
| Back to top |
|
|
BeteNoire
Veteran
Joined: 25 Sep 2005
Posts: 1827
|
| Posted: Thu Dec 22, 2005 10:42 am Post subject: |
|
|
Ok, there is a nice discussion here but I've got a question: what ext3 options do you recommend for a large filesystem (data storage) about 65 GB, which will store 10000-12000 files size of 3-15 MB?
I had ext3 on this partition once, created with default options and was very unhappy with its performance. Then I switched to xfs and noticed better performance, but now I want to give ext3 another try.
I want to give it maximum read performance as writes will not occur very often - this is to be only storage filesystem. |
|
| Back to top |
|
|
enderandrew
l33t
Joined: 25 Oct 2005
Posts: 731
|
|
| Back to top |
|
|
itsr0y
Tux's lil' helper
Joined: 22 Dec 2002
Posts: 81
|
| Posted: Fri Dec 23, 2005 5:52 am Post subject: |
|
|
Well, in case anyone wants some actual (if meaningless) numbers, here they are:
| Code: | +----------+------+----------+
| | ext3 | reiserfs |
+----------+------+----------+
| bootup | 45 | 50 |
| login | 10 | 12 |
| sync | 506 | 93 |
| updatedb | 662 | 90 |
| ctags | 24 | 55 |
+----------+------+----------+
|
Notes:
all numbers are in seconds
reiserfs mounted using noatime,notail
ext3 mounted using noatime and using the dir_index option (not the writeback thingie)
bootup is grub to gdm login
login is password entry in grub to finishing loading PekWM and my desktop apps (conky, etc)
sync is an "emerge sync"
updatedb is just that
ctags is running "exuberant-ctags -R" on usr include
Background:
I had a reiserfs partition of about 8 gigs at the end of my disk that I needed to make bigger. So I wiped off my windows partition (30 gigs) and made a new ext3 partition at the front of the drive. Then I did a "cp -ax" to copy all files from the old reiserfs to the new ext3 partition. Before copying, I did a "tune2fs -O dir_index /dev/hda3". Then, I booted up into each and ran the above commands. Each partition was in the same exact state when I ran them. If you really care for more details, let me know.
Conclusion:
DAMN! ext3 is SLOW when trying to access a lot of files. I mean, it is like 500% slower doing emerge sync and updatedb! Interestingly enough, booting and ctags were a little faster. Maybe this has to do with the dir_index option. If I can disable it and try running the test again, I will. Can I just run "tune2fs -O ^dirindex /dev/hda3" and will that get rid of all dir indexes? Also, maybe that writeback thing can speed things up. Does this mean you put the journal on its own partition? How do you set that up?
Hopefully someone might find this interesting or useful. If you'd like me to try anything else, or you think you can help me fix that SUPER SLOW emerge problem, please tell me! |
|
| Back to top |
|
|
ExZombie
Apprentice
Joined: 29 May 2004
Posts: 170
|
| Posted: Fri Dec 23, 2005 7:55 am Post subject: |
|
|
Definitely try writeback, i think (but am not sure) reiserfs uses it as well. It would be interesting to see if it is really any faster. I am too lazy to do any king of benchmarking  .
As for putting the journal to a separate partition, it's useless, or even harmful to performance because the head has to reach a completely different part of the disk to write to the journal. On the other hand, putting it to a separate drive should boost performance significantly, especially in full data journaling mode. |
|
| Back to top |
|
|
pv
Tux's lil' helper
Joined: 25 Mar 2005
Posts: 103
Location: Russia, Yaroslavl
|
| Posted: Fri Dec 23, 2005 9:23 am Post subject: |
|
|
itsr0y, please, try data=journal while mounting ext3/reiserfs or journal_data with tune2fs.
I run some tests including unpacking portage-20050712, linux-2.6.13.4 and full my /home/ partition, the latter having about 4.5G of files. I did that with data=journal,noatime with both reiserfs and ext3 and with dir_index for ext3 and I had the results like your ones (except sync and updatedb). My results for the portage and the kernel are described above. As to 4.5Gb of /home/ files (KDE config, all the distfiles I use, my work files, etc), ext3 is 10-20% FASTER then reiserfs.
_________________
Nothing but perfection. |
|
| Back to top |
|
|
pv
Tux's lil' helper
Joined: 25 Mar 2005
Posts: 103
Location: Russia, Yaroslavl
|
| Posted: Fri Dec 23, 2005 9:32 am Post subject: |
|
|
I have just seen TWO kjournald in the process list. I guess it's due to my having two ext3 partitions mounted now.
Can I reduce the number of kjournalds to only ONE? I'm afraid TWO daemons eat too much resources in spite of that I don't really need both of them.
Why cannot a single daemon solve the disk access tasks?
_________________
Nothing but perfection. |
|
| Back to top |
|
|
itsr0y
Tux's lil' helper
Joined: 22 Dec 2002
Posts: 81
|
| Posted: Fri Dec 23, 2005 8:48 pm Post subject: ext3 stinks! |
|
|
Ok, I ran a few more tests and I found reiserfs to be WAY faster than ext3:
| Code: | +----------+-----+-----+-----+-----+-----+-----+
| | (1) | (2) | (3) | (4) | (5) | (6) | all times in seconds
+----------+-----+-----+-----+-----+-----+-----+ (1) original reiserfs (notail)
| bootup | 50 | 45 | 45 | | | 45 | (2) ext3 (dir_index)
| login | 12 | 10 | 10 | | | 10 | (3) ext3 ()
| sync | 93 | 506 | | | | 53 | (4) ext3 (journal_data)
| updatedb | 90 | 662 | 567 | 729 | 735 | 57 | (5) ext3 (writeback,dir_index)
| ctags | 55 | 24 | 23 | | | 16 | (6) reiserfs ()
+----------+-----+-----+-----+-----+-----+-----+
| space* | 7.6 | 8.0 | 7.0 | * space consumption in GB
+----------+-----+-----+-----+-----+-----+-----+
|
Setup:
I had an 8 gb reiserfs partition (1) using notail at the end of the drive. I created a new 22 gb partition at the beginning of the drive, first using ext3 (2-5) then finally switched over to reiserfs with tail (6), copying the data from the original partition using cp -ax. I tried using the options listed above, and they only seemed to slow down the system. The blanks indicate a test I didn't perform.
Conclusion:
I'm back on reiserfs, and I'm never going to ext3 again. It was rediculously slow and used up a heck of a lot more space. I think any speedup gained from it (such as the faster ctags and bootup time) was simply because it was defragged by copying the data to a new partition. Additionally, I will no longer use the notail since it just uses up space and seems slower anyway.
I realize my test results are far from definitive, but there is no way I can live with the 10-12x slowdown in emerge sync and updatedb! |
|
| Back to top |
|
|
XenoTerraCide
Veteran
Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA
|
| Posted: Fri Dec 23, 2005 9:00 pm Post subject: |
|
|
| well I stopped using reiser cause it caused issues... and recovery wasn't as good. |
|
| Back to top |
|
|
wrc1944
Advocate
Joined: 15 Aug 2002
Posts: 3435
Location: Gainesville, Florida
|
| Posted: Sat Dec 24, 2005 3:12 pm Post subject: |
|
|
itsr0y,
I notice you didn't list ext3 with BOTH dir_index and journal_data enabled for any of your tests. Apparently, that was the combination that supposedly is the best for desktop interactive response. I've used that combination for 2+weeks now (coming from reiserfs, and notice no slowdown in emerge sync, and updatedb on 4 different Gentoo installations. I'm still looking for some more reports on writeback vs. journal_data mode as to which is the best overall.
Your better reiserfs performance without notail is very curious, as it seems to contradict most info I've ever seen.
_________________
Main box- AsRock x370 Gaming K4
Ryzen 7 3700x, 3.6GHz, 16GB GSkill Flare DDR4 3200mhz
Samsung SATA 1000GB, Radeon HD R7 350 2GB DDR5
OpenRC Gentoo ~amd64 plasma, glibc-2.36-r7, gcc-13.2.1_p20230304
kernel-6.8.4 USE=experimental python3_11 |
|
| Back to top |
|
|
andrewd18
Guru
Joined: 11 Apr 2004
Posts: 364
Location: Wisconsin, USA
|
| Posted: Sat Dec 24, 2005 7:26 pm Post subject: |
|
|
codergeek42 -
Thank you for your excellent documentation. I've been using ext3 for a while - reiserfs on SuSE was iffy at best with recovery, and Partition Magic recognizes ext3, so it made my life easier to switch over. Little tweaks like these continue to solidify my growing affection for ext3. Thanks much.
~~ Andrew D.
_________________
Keep Your Toolchain Stable! - emwrap.sh
There's no place like ::1 |
|
| Back to top |
|
|
codergeek42
Bodhisattva
Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)
|
| Posted: Sat Dec 24, 2005 7:28 pm Post subject: |
|
|
andrew18, Happy to help. 
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF |
|
| Back to top |
|
|
neenee
Veteran
Joined: 20 Jul 2003
Posts: 1786
|
| Posted: Sun Dec 25, 2005 12:33 pm Post subject: |
|
|
as i was running out of space on my separate root partition,
i had to increase its size - i could not shrink my /home par-
tition to make room for it (it was xfs), so i had to make new
partitions anyway and put back the files.
so i moved to ext3
everything seems to be working fine, no problems at boot
and i like that my kernel is a bit smaller because i could re-
move xfs support.
portage seems a bit faster than on xfs, which is good too.
thanks for the nice guide / tips |
|
| Back to top |
|
|
BeteNoire
Veteran
Joined: 25 Sep 2005
Posts: 1827
|
| Posted: Sun Dec 25, 2005 12:44 pm Post subject: |
|
|
| I wonder why, oh why, everyone ignores my previous post in this thread? |
|
| Back to top |
|
|
enderandrew
l33t
Joined: 25 Oct 2005
Posts: 731
|
| Posted: Sun Dec 25, 2005 12:56 pm Post subject: |
|
|
It has been stated a few times (including a few posts above you) that dir_index and journal_data seem to offer the best results for ext3.
Depending on your kernel version, you can also try the ext3 kernel patches I posted above. They claim to offer improved performance, but they were designed for 2.6.11 and don't work with the latest kernels.
_________________
Nihilism makes me smile. |
|
| Back to top |
|
|
ruben
Guru
Joined: 04 Jul 2003
Posts: 462
|
| Posted: Sun Dec 25, 2005 1:31 pm Post subject: |
|
|
| BeteNoire wrote: | Ok, there is a nice discussion here but I've got a question: what ext3 options do you recommend for a large filesystem (data storage) about 65 GB, which will store 10000-12000 files size of 3-15 MB?
I had ext3 on this partition once, created with default options and was very unhappy with its performance. Then I switched to xfs and noticed better performance, but now I want to give ext3 another try.
I want to give it maximum read performance as writes will not occur very often - this is to be only storage filesystem. |
One should always try to use the right file system for the job. Maybe xfs is better for the kind of usage you describe. If you go with ext3, you should take a large block size and definitely use the dir_index optimisation. I do think however that with a size of 65GB, mkfs.ext3 will automatically chose a large block size.
Here's what i'd use:
| Code: | | mkfs.ext3 /dev/hdxn -b 4096 -m 1 -O dir_index |
Large block size, and only 1% reserved for root (instead of default 5%).
I would mount it with data=writeback and commit=600. I believe this will give you good performance and is in line with the usage you describe. |
|
| Back to top |
|
|
pv
Tux's lil' helper
Joined: 25 Mar 2005
Posts: 103
Location: Russia, Yaroslavl
|
| Posted: Sun Dec 25, 2005 7:42 pm Post subject: |
|
|
| ruben wrote: | | I would mount it with data=writeback and commit=600. I believe this will give you good performance and is in line with the usage you describe. |
It was said too much about large values for commit= option so I don't think it's good to make it being more than 10-20 seconds. I HAVE TRIED that even with 5 seconds the system sometimes (rarely) become very unresponsible for 1-2 seconds.
As to data=writeback, I live in city where nobody knows when the electricity may be turned off so I prefer data=journal.
| BeteNoire wrote: | | I wonder why, oh why, everyone ignores my previous post in this thread? |
As to me, unfortunatelly I cannot give you a piece of advice except those given above in this forum. You can also read
| Code: | man tune2fs
man mke2fs
|
_________________
Nothing but perfection. |
|
| Back to top |
|
|
itsr0y
Tux's lil' helper
Joined: 22 Dec 2002
Posts: 81
|
| Posted: Tue Dec 27, 2005 6:33 am Post subject: |
|
|
| I don't understand why the method of writing the journal would have anything to do with read speed. ven if I turned off journaling (which should give the biggest performance boost), I can't imagine that it would have any affect on the read speed, and certainly not the 10x speedup required to beat reiserfs. As for dir_index, enabling it should improve read access, but in my tests it clearly did not. |
|
| Back to top |
|
|
ruben
Guru
Joined: 04 Jul 2003
Posts: 462
|
| Posted: Tue Dec 27, 2005 10:00 am Post subject: |
|
|
| itsr0y wrote: | Conclusion:
DAMN! ext3 is SLOW when trying to access a lot of files. I mean, it is like 500% slower doing emerge sync and updatedb! Interestingly enough, booting and ctags were a little faster. Maybe this has to do with the dir_index option. If I can disable it and try running the test again, I will. Can I just run "tune2fs -O ^dirindex /dev/hda3" and will that get rid of all dir indexes? Also, maybe that writeback thing can speed things up. Does this mean you put the journal on its own partition? How do you set that up?
Hopefully someone might find this interesting or useful. If you'd like me to try anything else, or you think you can help me fix that SUPER SLOW emerge problem, please tell me! |
These results don't really surprise me. Especially not the results of the "sync" test. If you'd like to know why, check the on-disk size of /usr/portage (excluding the distfiles directory). You would see that on the reiserfs system, this is a *lot* smaller than on the ext3 system. The original reiserfs partition is 8Gb in size, i wonder how big the block size is. The ext3 partition is 22Gb in size, i guess that block size will be 4k, which means you'll have a lot of waste, since /usr/portage contains lots of tiny files. If you'd like to use ext3 for /usr/portage (excluding distfiles), i'd just stick to ext2 with dir_index with a block size of 1k because journalling isn't important for this, it's faster, and lots of tiny files, thus take a small block size for less waste.
The difference between the 2 reiserfs timings is not very surprising either. There's two issues here: fragmentation and 'notail'. It seems you don't really know what "notail" actually means, since you say "notail just uses up space and seems slower anyway". Of course, "notail" is using up more space, since it actually disables a space optimisation. When you don't use the "notail" option, the data which would fill a block only partially, is stored in the tree itself. This means that very small files and the end of files is stored in the tree itself. If you use "notail", the small files and the end of a file always consume a whole block, e.g. a 4096 byte block, even if only 100 bytes are used. So, you gain space by doing this optimisation. This also means that your "sync" is gonna be faster, since it needs to access less disk space. So, "notail" is not necessarily faster, it can be faster since it uses less cpu time, but it can be slower since you need to access less blocks on disk for accessing the same amount of data. The other factor is fragmentation: believe me, reiserfs gets fragmented over time much more than ext2/3. So, the difference in timings between the 2 reiserfs measurements is also caused by fragmentation (although i don't know how old the 8Gb partition is in terms of disk activity). But i've seen that fragmentation can have a dramatic effect on performance for a reiserfs partition.
The "sync" and "updatedb" in the ext3 system is worse than in the reiserfs system. Lots of tiny files is the strong point of reiserfs, so that explains the "sync" performance. I don't know an explanation for the "updatedb", it would mean that reiserfs is faster in reading the directory entries than ext3. You did mount the ext3 with dir_index *before* copying the files to the ext3, right? I never measured "updatedb" performance myself though. I've done some tests with "sync" performance and "gnome" startup time. Also, reiserfs uses a hash function to find files in directories.. maybe that has something to do with it too.
Another thing is that you should keep in mind that by default ext3 does more journalling than reiserfs does. To get that to the same, you'd need to use "writeback" mode for the journalling. In addition, by default, ext3 commits the journal each 5 seconds, while i've read that reiserfs does that much less frequently. Finally, you do not use the option "noatime", which means that for each read, you also do a write to update the access time of the file or directory. I'm not sure whether this kind of write is also journalled or not, but this can also be something that has influence.
As for me, i still have some reiserfs partitions on my desktop, but for my laptop, i only use ext3. Performance degradation due to fragmentation was really bad with reiserfs. For /usr/portage, i used a small partition with ext2 to (a) limit any fragmentation to that partition, to (b) performance and (c) small block size. However, nowadays i use Debian on my laptop and i have only ext3 partitions now. As a final remark, i cared more about startup time of gnome than about "sync" or "updatedb" performance. Ext3 uses less cpu time than reiserfs, and that kind of applications can be run in the background without dragging the whole system down. |
|
| Back to top |
|
|
ruben
Guru
Joined: 04 Jul 2003
Posts: 462
|
| Posted: Tue Dec 27, 2005 10:06 am Post subject: |
|
|
| itsr0y wrote: | | I don't understand why the method of writing the journal would have anything to do with read speed. ven if I turned off journaling (which should give the biggest performance boost), I can't imagine that it would have any affect on the read speed, and certainly not the 10x speedup required to beat reiserfs. As for dir_index, enabling it should improve read access, but in my tests it clearly did not. |
It depends on what is put in the journal. I'm not sure, but i wouldn't be surprised if an "ok" entry is added to the journal every 5 seconds even though you only do reads. And as i mentioned in the other post, there is a write for updating the access time. About "dir_index", it should be enabled before files are put on the system, otherwise it only has effect for files copied to the file system later on. Also, "dir_index" improves the time needed to find the location of the file on disk, it does not improve the time needed to read the file itself, but it is part of the total read speed of course. Its effect will be most visible in directories with lots of files. "dir_index" improves read access compared to ext2/3 without "dir_index". |
|
| Back to top |
|
|
BeteNoire
Veteran
Joined: 25 Sep 2005
Posts: 1827
|
| Posted: Tue Dec 27, 2005 11:01 am Post subject: |
|
|
I keep track this thread since few weeks trying to find best solution for me, but there is always one question which concerns me: if ext3 is the best linux filesystem (as some say) then why it doesn't use its best features by default? Why do I have to read manuals, make test or search for reliable opinions, to get its best features to work?
I switched to reiserfs some time ago, because I was unhappy with ext3 performance. I noticed best performance and overall activity of reiserfs without any tuning. Never had any corruption and never lost any data stored on my r-fs partitions. So I stayed with it.
But now it's time to re-think my solutions, and if someone can explain why do I have to do some special procedures to get ext3's best features, why it doesn't use them by default - I'd be very thankful. |
|
| Back to top |
|
|
alexlm78
Veteran
Joined: 08 Dec 2003
Posts: 1265
Location: Guatemala,Guatemala
|
| Posted: Tue Dec 27, 2005 3:28 pm Post subject: |
|
|
Very usefull, thanks a lot.
_________________
"This is a different kind of world, you need a different kind of software"
Linux User# 315201
100% Chapin hecho en Guatemala |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Documentation, Tips & Tricks
