| View previous topic :: View next topic |
| Author |
Message |
codergeek42
Bodhisattva
Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)
|
 Posted: Wed Mar 08, 2006 3:43 am Post subject: Re: ext3/reiser on 24/7 |
 |
|
| Special K wrote: | when using ext3 on a server which will reboot let's say every 200days only, should I run fsck sometimes?
Is there anything wich sould be done so ext3 is not defragmenting? collecting errors until crash? |
As mentioned, Ext3 does not suffer any serious performance loss due to fragmentation until it (the filesystem) becomes near-fully consumed with regards to block/space usage, so you should be fine (as Ext3 reserves about 5% of the available blocks by default to help prevent this). ReiserFS I don't know much about, sorry...
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF |
|
| Back to top |
|
 |
pv
Tux's lil' helper
Joined: 25 Mar 2005
Posts: 103
Location: Russia, Yaroslavl
|
| Posted: Sun Mar 12, 2006 8:33 pm Post subject: |
|
|
Hi, again!
I've just seen that while creating a filesystem e2fsprogs-1.38 set the default journal size of ext3 partitions to 32K blocks (that's 128M) instead of 8K blocks (that's 32M) as of e2fsprogs-1.37.
Is such a large journal is intended for using with large partitions? What journal size may be the best for my 8G partitions?
_________________
Nothing but perfection. |
|
| Back to top |
|
|
wrex
n00b
Joined: 01 May 2003
Posts: 26
Location: San Jose, CA
|
| Posted: Tue Mar 14, 2006 3:58 am Post subject: |
|
|
| codergeek42 wrote: | | XenoTerraCide wrote: | | is there a way to resize ext2/3 partitions? |
Yes. You can add the option while creating the fileystem: | Code: | | # /sbin/mkfs.ext3 -O dir_index -E resize=max_online_resize /dev/hdXY |
This allows the filesystem to grow to a size of max_online_resize blocks. (Check the mke2fs(8) man page for more information.)
Alternatively, you can use something like LVM to manage the partitioning. I think parted can do it too, but I'm not certain. |
Better yet, try going to google, typing "ext2 resize", clicking "I'm Feeling Lucky", chuckling with pleasure, typing "emerge --search ext2resize", gasping with joy, then typing "emerge ext2resize" and smiling with smug satisfaction. :-)
md, lvm2, and ext2resize make for a quite feature-rich storage management solution.
Good question, and great thread - thanks CoderGeek! |
|
| Back to top |
|
|
wrex
n00b
Joined: 01 May 2003
Posts: 26
Location: San Jose, CA
|
| Posted: Tue Mar 14, 2006 5:15 am Post subject: Has anyone experimented with device=external-journal? |
|
|
Has anyone experimented with the journal on a separate physical device? I would expect this to provide a pretty big performance win at the cost of a little extra complexity.
I'm deploying a new server with four physical SATA drives configured as a pair of mirrors. I've got the resources to dedicate a pair of drives just to journal my busiest filesystem (it's a database server). This will obviously waste a lot of space, but keeping all the sequential journal writes to one pair of spindles, with the random I/O to the other pair of spindles . I've never actually tried this with ext3, though.
Has anyone else experimented with a separate journal device?
TIA |
|
| Back to top |
|
|
codergeek42
Bodhisattva
Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)
|
| Posted: Tue Mar 14, 2006 5:40 am Post subject: |
|
|
| wrex wrote: | Better yet, try going to google, typing "ext2 resize", clicking "I'm Feeling Lucky", chuckling with pleasure, typing "emerge --search ext2resize", gasping with joy, then typing "emerge ext2resize" and smiling with smug satisfaction.  |
That's cool! Thanks.
| wrex wrote: | | Good question, and great thread - thanks CoderGeek! |
Glad to be of help.
| pv wrote: | | Is such a large journal is intended for using with large partitions? What journal size may be the best for my 8G partitions? |
From what I've read in the man pages, a larger journal is supposed to help increase sustained throughput; but I'm not certain about this, nor have I actually played around with changing the journal size, sorry. I'll play with that later this week.
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF |
|
| Back to top |
|
|
XenoTerraCide
Veteran
Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA
|
| Posted: Tue Mar 14, 2006 10:30 am Post subject: |
|
|
I ended up using gparted. It was easier for me to do the resizing that way because I could see it. However I was aware of the CLI option that ext3 supports natively, although I'm not sure I was at the time I asked that question... That was a while ago.
_________________
I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help. |
|
| Back to top |
|
|
wrc1944
Advocate
Joined: 15 Aug 2002
Posts: 3435
Location: Gainesville, Florida
|
| Posted: Wed Apr 12, 2006 11:08 pm Post subject: |
|
|
Here's a post on our subject I made to the nesl247 guide for advanced gcc-4.1 Gentoo installs (I've migrated all my Gentoo boxes to this method- works perfectly, no problems with the latest version, and nesl247 recommends tuned ext3). Their forum is new and very small, so perhaps I'll get more feedback here. I think this is on point to the ext3 discussion.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
In the wiki text I quoted, they didn't mention it at all, so apparently they don't know about nodiratime either. It isn't even in the linux kernel Documentation (linux-2.6.16-beyond1/Documentation/filesystems/ext3.txt). I ran across it about 6 months ago while googling for filesystems info, after Codergeek42 on the Gentoo forum ext3 tips thread sold me on switching to ext3 tuned from reiserfs.
Previously, I had also played around with the ext3 commit= value (change from the default of every 5 seconds to a higher number). I became convinced that in order to assure a pretty current journal being written, something between 60 and 180 seconds was preferable to the wildly high numbers (like 9999) recommended by some. I hadn't done this on my last few installs, and will go back to it right away. (This is also good for saving power on laptops)
Since this is a "performance" ext3 install guide, maybe this tweak could also be considered for those not absolutely requiring the default every 5 seconds journal commit.
| Quote: | From kernel Documentation files:
commit=nrsec (*) Ext3 can be told to sync all its data and metadata
every 'nrsec' seconds. The default value is 5 seconds.
This means that if you lose your power, you will lose
as much as the latest 5 seconds of work (your
filesystem will not be damaged though, thanks to the
journaling). This default value (or any low value)
will hurt performance, but it's good for data-safety.
Setting it to 0 will have the same effect as leaving
it at the default (5 seconds).
Setting it to very large values will improve
performance. |
This is a mount option added to the fstab partition lines, and IIRC, you also have to add rootflags=commit=60 (or whatever number you choose) to the grub kernel line for the / partition. I guess /usr/portage, /var, and /tmp could safely be set way higher, or even just use ext2 with dir_index for those three.
EDIT: Had to refresh my memory. It seems that /boot can't recognize the commit=n option, so don't use it on the fstab /boot partition. I've gone ahead and set / and /home to =60, and /var, /tmp, /mnt/data, /mnt/data2, and /mnt/portage to=600. I'll see how that goes.
Any feedback on commit=n settings, pro and con, about intervals for specific partitions related to Gentoo usage is greatly appreciated. My own thinking is that / and /home need to be reliably journaled, while /var and /tmp are OK with less frequent commits, and basically "storage" partitions like /mnt/data that are rarely touched need commit even less frequently. Apparently, /boot must remain the ext3 default of 5 secs.
I'm also reconsidering using data=writeback instead of data=journal. The LinuxGazette filesystems tests author (Justin Piszcz) was kind enough to let me use his script, and I ran all his tests on my own system, comparing all ext3 modes to reiserfs, and data=writeback values came out ahead of all other ext3 modes across the board, and on most tests beat reiserfs (also reiser4, XFS, and JFS). One caveat- this is only on my system- YMMV. Of course with writeback mode, journalling is less reliable. I'm not sure how his battery of tests specifically relates to Gentoo usage, but they do give a good general idea of how the modes compare in performance.
To simplify, I'm also considering putting the portage directory in /var instead of /usr/portage or /mnt/portage (I already have distfiles directory there). In other words, things that tend to fragment need to go into the /var directory, so one operation can defrag when needed, and it keeps / relatively clean. This is probably Gentoo (and other source distro) specific, but in two weeks my ext3 tuned /var partition has gone from 0.0% non-contiguous, to 21.0% non-contiguous- an alarming observation, that convinced me this is a big problem, even with ext3. I had to copy all /var directories to another partition, and restore in order to defrag. In only two days (and two emerge --syncs), /var has already gone from 0.0% to 0.4% fragmented. I think this really clarifies the importance of getting /var and portage off of /, and dispells the myth that Linux filesystems don't fragment. 
I recall when running windows systems defragmenter, where it shows the graphical representation of clusters and fragmentation, even a 2-4% partition fragmentation level appears as horribly fragmented- so much so that you wonder how the hard drive even functions. I can't imagine how 21% or higher would appear, but this is what is happening on my gentoo /var partition. and to a lesser extent, /tmp and /home.
I've installed many hundreds of Linux systems in the last 5 years, and my conclusion is that for all future Gentoo installations, I will have /var on it's own partition (at least 10GB), with the distfiles and portage directories also on /var (requires changing the /etc/make.profile symlink to point to /var/portage). I will also always have a spare /data partion in which to cp -a the /var (and other directory) contents to, and restore from, when defragging is needed.
After much hard experience, here's my recommended basic minimal default Gentoo specific partitioning scheme (I'll leave the filesystem choices to the user, but my current recommendation is ext3, tuned):
/boot 80mb ext2
/swap 512mb-1GB
/ 10GB
/var 10-12GB (put portage and distfiles directories here, edit /etc/make.conf appropriately, change /etc/make.profile symlink)
/tmp 2-3GB
/home 5-10GB (or more, user's choice)
/data 20-30GB (or more, for backups, misc. tmp stuff, etc., and copy/restore operations (defragging) on /var and /tmp.
Freespace- Room for expanding /home, adding more /data partitions, or another distro installation.
I don't put /usr on it's own partition, because without portage there, fragmentation potential is much smaller.
/boot, /, and swap should be large enough for most users.
_________________
Main box- AsRock x370 Gaming K4
Ryzen 7 3700x, 3.6GHz, 16GB GSkill Flare DDR4 3200mhz
Samsung SATA 1000GB, Radeon HD R7 350 2GB DDR5
OpenRC Gentoo ~amd64 plasma, glibc-2.36-r7, gcc-13.2.1_p20230304
kernel-6.8.4 USE=experimental python3_11
Last edited by wrc1944 on Thu Apr 13, 2006 8:11 pm; edited 1 time in total |
|
| Back to top |
|
|
Massimo B.
Veteran
Joined: 09 Feb 2005
Posts: 1776
Location: PB, Germany
|
| Posted: Thu Apr 13, 2006 7:43 am Post subject: |
|
|
I fully agree with your suggestion about where to place the portage tree, eventhough I preferred a loop file for /var. Especially /var runs better on reiserfs in my opinion, but I can't approve that by benchmarks.
Nice information about the commit= option, didn't know that.
Concerning data=writeback or journal I think a determining factor is the cpu power. For my 600Mhz journal was just too much load whereas writeback is not.
_________________
HP ZBook Power 15.6" G8 i7-11800H|HP EliteDesk 800G1 i7-4790|HP Compaq Pro 6300 i7-3770 |
|
| Back to top |
|
|
ruben
Guru
Joined: 04 Jul 2003
Posts: 462
|
| Posted: Fri Apr 14, 2006 12:10 pm Post subject: |
|
|
| wrc1944 wrote: | | I'm also reconsidering using data=writeback instead of data=journal. The LinuxGazette filesystems tests author (Justin Piszcz) was kind enough to let me use his script, and I ran all his tests on my own system, comparing all ext3 modes to reiserfs, and data=writeback values came out ahead of all other ext3 modes across the board, and on most tests beat reiserfs (also reiser4, XFS, and JFS). One caveat- this is only on my system- YMMV. Of course with writeback mode, journalling is less reliable. I'm not sure how his battery of tests specifically relates to Gentoo usage, but they do give a good general idea of how the modes compare in performance. |
You're really doing serious investigation on those file systems I think I've seen the LinuxGazette filesystem tests before, but those are quite old, isn't it? So I think it'd be great if you could post the results if it's not too much work. I'd appreciate it to see how the different configs compare now.
As for the high fragmentation in /var... one of the reasons is probably "/var/tmp/portage" where compiles are done and the ccache cache for emerge.
About the "commit=" value: on my desktop I've chosen 30 seconds, but didn't do any real tests or anything. I just thought it'd increase throughput when copying a lot of data, while still I could only potentially lose 30 seconds of work. On my laptop, I want to disk to be spinned down as much as possible, and I use laptop-mode-tools which delays writes up to half an hour, and for that i also let it remount my partitions with a commit of 1800 seconds. I'm comfortable with this on my laptop, since my laptop is very stable and "can't" shut down because of losing power: the battery works fine, and when my laptop is starting to run out of battery, it simply remounts my partitions with a commit value of 5 seconds, and it goes into sleep when the battery has only 5 minutes left. So, in that sense, I consider it safe. It's happened before that the electricity is going down, and that makes me stay with the 30 seconds on the desktop. I never measured the influence of the commit value on performance though. |
|
| Back to top |
|
|
wrc1944
Advocate
Joined: 15 Aug 2002
Posts: 3435
Location: Gainesville, Florida
|
| Posted: Fri Apr 14, 2006 1:25 pm Post subject: |
|
|
ruben wrote: | Quote: | | I think I've seen the LinuxGazette filesystem tests before, but those are quite old, isn't it? So I think it'd be great if you could post the results if it's not too much work. I'd appreciate it to see how the different configs compare now. |
Justin Piszcz recently did another run updating his tests using a 2.6.14 kernel, and they are also on LinuxGazette. After I saw them, I contacted him and he sent me his script, but was very adamant about not releasing it on a public forum. IIRC, I believe I emailed him back an asked if I could post any results here, but never heard back. In any case, I feel I must abide by his wishes, as he was extremely helpful when I had some questions about his script. I did send him my results, in hopes he would write another LinuxGazette article, complete with the charts.
What would really be interesting from a Gentoo perspective would be to devise a script that would run multiple tests of things like emerge --sync, revdep-rebuild, and other gentoo related operations, and compare the filesystem mode performance.
I can say that data=writeback times were about 2-5% faster than data=journal, and both were considerably faster than the ext3 defaults. However, I really think that adding dir_index is the big factor and makes the most difference, compared to defaults. I ran the ext3 defaults, reiserfs, and then the other modes with dir_index, all on the same empty partition (reformatting for each run).
Very interesting comments on your commit= settings. I just switched back to using commt=60 and 600 as mentioned above. My subjective impression is that windows generally open slightly faster than they did, maybe due to the fact that with the default 5 seconds, there is a better chance that a commit is being written when opening a window, or doing something else. I guess this could also be affected by the choice of schedulers in kernel config.
Hmmmm.... I just realized about /boot mentioned above in an edit. The reason it didn't recognize commit=n is because I forgot it's an ext2 partition. 
_________________
Main box- AsRock x370 Gaming K4
Ryzen 7 3700x, 3.6GHz, 16GB GSkill Flare DDR4 3200mhz
Samsung SATA 1000GB, Radeon HD R7 350 2GB DDR5
OpenRC Gentoo ~amd64 plasma, glibc-2.36-r7, gcc-13.2.1_p20230304
kernel-6.8.4 USE=experimental python3_11 |
|
| Back to top |
|
|
skyfolly
Apprentice
Joined: 16 Jul 2003
Posts: 245
Location: Dongguan & Hong Kong, PRC
|
| Posted: Sat May 06, 2006 3:54 pm Post subject: |
|
|
too lazy to read the post through, but I am wondering if small block size can help performance? I believe small block size helps to save harddisk space. But other than that, I am not sure.
_________________
I am the only being whose doom
No tongue would ask no eye would mourn
I never caused a thought of gloom
A smile of joy since I was born.
emily bronte |
|
| Back to top |
|
|
codergeek42
Bodhisattva
Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)
|
| Posted: Sat May 06, 2006 7:03 pm Post subject: |
|
|
| skyfolly wrote: | | too lazy to read the post through, but I am wondering if small block size can help performance? I believe small block size helps to save harddisk space. But other than that, I am not sure. |
As I understand it, the block size argument is a trade off between performance and space usage, as a larger block size means that the VFS layer can read or write data in large chunks, instead of a series of smaller I/O calls; but it also means that each used block will be larger, so even if your file is, for example, 780 bytes, it could still consume (by not using the remainder of its reserved space) 4 kilobytes of disk space.
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF |
|
| Back to top |
|
|
micr0c0sm
Tux's lil' helper
Joined: 29 Oct 2005
Posts: 148
Location: New York
|
| Posted: Tue May 23, 2006 7:03 am Post subject: soft updates? |
|
|
| A friend of mine recommended using soft-updates but I am having trouble finding any information on the topic, can anyone explain to me what that is? |
|
| Back to top |
|
|
codergeek42
Bodhisattva
Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)
|
| Posted: Tue May 23, 2006 3:27 pm Post subject: Re: soft updates? |
|
|
| micr0c0sm wrote: | | A friend of mine recommended using soft-updates but I am having trouble finding any information on the topic, can anyone explain to me what that is? |
Soft updates is different from journalling in that it does not keep backup metadata/object copies of transactions, but simply maintains metadata writes in the proper order. As far as I know, this is really only implemented in the UFS incarnations (*BSD).
As I've only played with FreeBSD in a qemu VM instance, I can't say with certainty how soft updates compares to journalling in terms of performance (though journalling as I explained here does gaurantee consistent file data as well, which may be very important).
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF |
|
| Back to top |
|
|
PaulBredbury
Watchman
Joined: 14 Jul 2005
Posts: 7310
|
| Posted: Thu Jun 22, 2006 1:35 am Post subject: |
|
|
| Here's another ext3 performance tip: Upgrade to kernel 2.6.17 |
|
| Back to top |
|
|
codergeek42
Bodhisattva
Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)
|
| Posted: Thu Jun 22, 2006 3:36 am Post subject: |
|
|
| PaulBredbury wrote: | | Here's another ext3 performance tip: Upgrade to kernel 2.6.17 |
Cool! Thanks! 
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF |
|
| Back to top |
|
|
DarkMind
Guru
Joined: 18 Dec 2003
Posts: 525
Location: Santiago, Chile
|
| Posted: Thu Jun 22, 2006 4:49 pm Post subject: Re: Some ext3 Filesystem Tips |
|
|
| codergeek42 wrote: | Copyright (c) 2005 Peter Gordon
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.2
or any later version published by the Free Software Foundation;
with no Invariant Sections, no Front-Cover Texts, and no Back-Cover
Texts. A copy of the license can be found here.
Overview
I'm a big fan of the Third Extended ("ext3") filesystem. It's in-kernel and userspace code has been tried, tested, fixed, and improved upon more than almost every other Linux-compatible filesystem. It's simple, robust, and extensible. In this article I intend to explain some tips that can improve both the performance and the reliability of the filesystem.
In the document, /dev/hdXY will be used as a generic partition. You should replace this with the actual device node for your partition, such as /dev/hdb1 for the first partition of the primary slave disk or /dev/sda2 for the second partition of your first SCSI or Serial ATA disk.
I: Using The tune2fs and e2fsck Utilities
Before we begin, we need to make sure you are comfortable with using the tune2fs utility to alter the filesystem options of an ext2 or ext3 partition. Please make sure to read the tune2fs man page:It's generally a good idea to run a filesystem check using the e2fsck utility after you've completed the alterations you wish to make on your filesystem. This will verify that your filesystem is clean and fix it if needed. You should also read the manual page for the e2fsck utility if you have not yet done so:
WARNING: Make sure any filesystems are cleanly unmounted before altering them with the tune2fs or e2fsck utilities! (Boot from a LiveCD such as Knoppix if you need to.) Altering or tuning a filesystem while it is mounted can cause severe corruption! You have been warned!
II: Using Directory Indexing
This feature improves file access in large directories or directories containing many files by using hashed binary trees to store the directory information. It's perfectly safe to use, and it provides a fairly substantial improvement in most cases; so it's a good idea to enable it:
| Code: | | # tune2fs -O dir_index /dev/hdXY |
This will only take effect with directories created on that filesystem after tune2fs is run. In order to apply this to currently existing directories, we must run the e2fsck utility to optimize and reindex the directories on the filesystem: | Code: | # e2fsck -D /dev/hdXY
|
 Note: This should work with both ext2 and ext3 filesystems. Depending on the size of your filesystem, this could take a long time. Perhaps you should go get some coffee 
III: Enable Full Journaling
By default, ext3 partitions mount with the 'ordered' data mode. In this mode, all data is written to the main filesystem and its metadata is committed to the journal, whose blocks are logically grouped into transactions to decrease disk I/O. This tends to be a good default for most people. However, I've found a method that increases both reliability and performance (in some situations): journaling everything, including the file data itself (known as 'journal' data mode). Normally, one would think that journaling all data would decrease performance, because the data is written to disk twice: once to the journal then later committed to the main filesystem, but this does not seem to be the case. I've enabled it on all nine of my partitions and have only seen a minor performance loss in deleting large files. In fact, doing this can actually improve performance on a filesystem where much reading and writing is to be done simultaneously. See this article written by Daniel Robbins on IBM's website for more information:
http://www-106.ibm.com/developerworks/linux/library/l-fs8.html#4
In fact, putting /usr/portage on its own ext3 partition with journal data mode seems to have decreased the time it takes to run emerge --sync significantly. I've also seen slight improvements in compile time.
There are two different ways to activate journal data mode. The first is by adding data=journal as a mount option in /etc/fstab. If you do it this way and want your root filesystem to also use it, you should also pass rootflags=data=journal as a kernel parameter in your bootloader's configuration. In the second method, you will use tune2fs to modify the default mount options in the filesystem's superblock: | Code: | | # tune2fs -O has_journal -o journal_data /dev/hdXY |
Please note that the second method may not work for older kernels. Especially Linux 2.4.20 and below will likely disregard the default mount options on the superblock. If you're feeling adventurous you may also want to tweak the journal size. (I've left the journal size at the default.) A larger journal may give you better performance (at the cost of more disk space and longer recovery times). Please be sure to read the relevant section of the tune2fs manual before doing so: | Code: | | # tune2fs -J size=$SIZE /dev/hdXY |
IV: Disable Lengthy Boot-Time Checks
WARNING: Only do this on a journalling filesystem such as ext3. This may or may not work on other journalling filesystems such as ReiserFS or XFS, but has not been tested. Doing so may damage or otherwise corrupt other filesystems. You do this AT YOUR OWN RISK.
Hmm..It seems that our ext3 filesystems are still being checked every 30 mounts or so. This is a good default for many because it helps prevent filesystem corruption when you have hardware issues, such as bad IDE/SATA/SCSI cabling, power supply failures, etc. One of the driving forces for creating journalling filesystems was that the filesystem could easily be returned to a consistent state by recovering and replaying the needed journalled transactions. Therefore, we can safely disable these mount-count- and time-dependent checks if we are certain the filesystem will be quickly checked to recover the journal if needed to restore filesystem and data consistency. Before you do this please make sure your filesystem entry in /etc/fstab has a positive integer in its 6th field (pass) so that it is checked at boot time automatically. You may do so using the following command: | Code: | | # tune2fs -c 0 -i 0 /dev/hdXY |
V: Checking The Filesystem Options Using tune2fs
Well, now that we've tweaked our filesystem, we want to make sure those tweaks are applied, right? Surprisingly, we can do this options iusing the tune2fs utility quite easily. To list all the contents of the filesystem's superblock, we can pass the "-l" (lowercase "L") option to tune2fs: | Code: | | # tune2fs -l /dev/hdXY |
Unlike the other tune2fs calls, this can be run on a mounted filesystem without harm, since it doesn't access or attempt to change the filesystem at such a low level.
This will give you a lot of information about the filesystem, including the block/inode information, as well as the filesystem features and default mount options, which we are looking for. If all goes well, the relevant part of the output should include "dir_index" and "has_journal" flags in the Filesystem features listing, and should show a default mount option of "journal_data".
This concludes this filesystem tweaking guide for now. Happy hacking!  |
or use reiserfs?? |
|
| Back to top |
|
|
synss
Apprentice
Joined: 08 Mar 2006
Posts: 282
Location: Dijon > Berlin > Tokyo > Nürnberg > München
|
| Posted: Thu Jun 22, 2006 5:33 pm Post subject: Re: Some ext3 Filesystem Tips |
|
|
| DarkMind wrote: | | or use reiserfs?? |
Oh my God... Let us not have this rather interesting thread degenerate into a flame war, please...
Just use reiserfs if it makes you happy for all I care. 
_________________
Compress portage tree
Elog viewer
Autodetect swap |
|
| Back to top |
|
|
codergeek42
Bodhisattva
Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)
|
| Posted: Thu Jun 22, 2006 5:41 pm Post subject: |
|
|
Thank you, synss.
DarkMind: This is meant to be a thread about Ext3 tweaking. If you use ReiserFS and want to encourage its use or discuss its tweaking capabilites, then by all means go ahead: But not in this thread.
Thanks.
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF |
|
| Back to top |
|
|
ppurka
Advocate
Joined: 26 Dec 2004
Posts: 3256
|
| Posted: Thu Jun 22, 2006 6:21 pm Post subject: |
|
|
<edited> (some stupid crap I had posted)
Last edited by ppurka on Tue Jun 27, 2006 5:51 am; edited 1 time in total |
|
| Back to top |
|
|
Doogman
Apprentice
Joined: 24 Sep 2004
Posts: 242
|
| Posted: Sun Jun 25, 2006 7:34 pm Post subject: |
|
|
Just a heads-up on being careful with setting ext3 with full data journaling. I've used it in the past without any noticable speed problems, but I've noticed that lately I've been getting a big slowdown with journal_data. It was most apparent this weekend when I installed a new 500G SATA HD and I couldn't figure out why the drive benchmarks were so slow. After spending a few hours going down the wrong patch assuming it was a problem with libsata, I swithced to using the default ext3 settings with dir_index and drive performance returned to normal.
| Code: |
With full data journal
Version 1.93c ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
ghidorah 1G 19430 11 15519 8 74570 17 163.2 2
Latency 1920ms 1555ms 39679us 281ms
And now without
Version 1.93c ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
ghidorah 1G 39035 18 24111 9 74431 16 203.2 4
Latency 3253ms 211ms 36911us 509ms
|
Write performance was cut in half with full data journaling!
I would recommend if you get away from "mkfs.ext3 -O dir_index" that you run a few benchmarks. |
|
| Back to top |
|
|
XenoTerraCide
Veteran
Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA
|
| Posted: Mon Jun 26, 2006 2:01 pm Post subject: |
|
|
on your data_journal partition did you have dir_index turned on? it is recommended. how did you benchmark it? just out of curiousity.
_________________
I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help. |
|
| Back to top |
|
|
Doogman
Apprentice
Joined: 24 Sep 2004
Posts: 242
|
| Posted: Mon Jun 26, 2006 8:48 pm Post subject: |
|
|
| XenoTerraCide wrote: | | on your data_journal partition did you have dir_index turned on? it is recommended. how did you benchmark it? just out of curiousity. |
Yeah, I always use dir_index. The benchmarks I posted were from bonnie++.
An alternative to bonnie++ was to simple use "mc" to copy files into the data_journal_partition. On a big file (like a 700MB xvid), it will give you a MB/S reading which backed-up bonnie++'s results. |
|
| Back to top |
|
|
codergeek42
Bodhisattva
Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)
|
| Posted: Tue Jun 27, 2006 4:04 am Post subject: |
|
|
| Doogman wrote: | [...]
Write performance was cut in half with full data journaling! |
Those benchmarks are rather surprising. I'll have to try that out on my own. 
Thanks.
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF |
|
| Back to top |
|
|
XenoTerraCide
Veteran
Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA
|
| Posted: Tue Jun 27, 2006 5:23 am Post subject: |
|
|
one other important thing. what version of the kernel was this done on?
_________________
I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Documentation, Tips & Tricks
