Some ext3 Filesystem Tips

[codergeek42]

Copyright (c) 2005 Peter Gordon

Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.2
or any later version published by the Free Software Foundation;
with no Invariant Sections, no Front-Cover Texts, and no Back-Cover
Texts. A copy of the license can be found here.

Overview
I'm a big fan of the Third Extended ("ext3") filesystem. It's in-kernel and userspace code has been tried, tested, fixed, and improved upon more than almost every other Linux-compatible filesystem. It's simple, robust, and extensible. In this article I intend to explain some tips that can improve both the performance and the reliability of the filesystem.

In the document, /dev/hdXY will be used as a generic partition. You should replace this with the actual device node for your partition, such as /dev/hdb1 for the first partition of the primary slave disk or /dev/sda2 for the second partition of your first SCSI or Serial ATA disk.

I: Using The tune2fs and e2fsck Utilities

Before we begin, we need to make sure you are comfortable with using the tune2fs utility to alter the filesystem options of an ext2 or ext3 partition. Please make sure to read the tune2fs man page:

Code: Select all

$ man tune2fs 

It's generally a good idea to run a filesystem check using the e2fsck utility after you've completed the alterations you wish to make on your filesystem. This will verify that your filesystem is clean and fix it if needed. You should also read the manual page for the e2fsck utility if you have not yet done so:

Code: Select all

$ man e2fsck 

WARNING: Make sure any filesystems are cleanly unmounted before altering them with the tune2fs or e2fsck utilities! (Boot from a LiveCD such as Knoppix if you need to.) Altering or tuning a filesystem while it is mounted can cause severe corruption! You have been warned!

II: Using Directory Indexing

This feature improves file access in large directories or directories containing many files by using hashed binary trees to store the directory information. It's perfectly safe to use, and it provides a fairly substantial improvement in most cases; so it's a good idea to enable it:

Code: Select all

# tune2fs -O dir_index /dev/hdXY

This will only take effect with directories created on that filesystem after tune2fs is run. In order to apply this to currently existing directories, we must run the e2fsck utility to optimize and reindex the directories on the filesystem:

Code: Select all

# e2fsck -D /dev/hdXY

Note: This should work with both ext2 and ext3 filesystems. Depending on the size of your filesystem, this could take a long time. Perhaps you should go get some coffee

III: Enable Full Journaling

By default, ext3 partitions mount with the 'ordered' data mode. In this mode, all data is written to the main filesystem and its metadata is committed to the journal, whose blocks are logically grouped into transactions to decrease disk I/O. This tends to be a good default for most people. However, I've found a method that increases both reliability and performance (in some situations): journaling everything, including the file data itself (known as 'journal' data mode). Normally, one would think that journaling all data would decrease performance, because the data is written to disk twice: once to the journal then later committed to the main filesystem, but this does not seem to be the case. I've enabled it on all nine of my partitions and have only seen a minor performance loss in deleting large files. In fact, doing this can actually improve performance on a filesystem where much reading and writing is to be done simultaneously. See this article written by Daniel Robbins on IBM's website for more information:

http://www-106.ibm.com/developerworks/l ... fs8.html#4

In fact, putting /usr/portage on its own ext3 partition with journal data mode seems to have decreased the time it takes to run emerge --sync significantly. I've also seen slight improvements in compile time.

There are two different ways to activate journal data mode. The first is by adding data=journal as a mount option in /etc/fstab. If you do it this way and want your root filesystem to also use it, you should also pass rootflags=data=journal as a kernel parameter in your bootloader's configuration. In the second method, you will use tune2fs to modify the default mount options in the filesystem's superblock:

Code: Select all

# tune2fs -O has_journal -o journal_data /dev/hdXY

Please note that the second method may not work for older kernels. Especially Linux 2.4.20 and below will likely disregard the default mount options on the superblock. If you're feeling adventurous you may also want to tweak the journal size. (I've left the journal size at the default.) A larger journal may give you better performance (at the cost of more disk space and longer recovery times). Please be sure to read the relevant section of the tune2fs manual before doing so:

Code: Select all

# tune2fs -J size=$SIZE /dev/hdXY

IV: Disable Lengthy Boot-Time Checks

WARNING: Only do this on a journalling filesystem such as ext3. This may or may not work on other journalling filesystems such as ReiserFS or XFS, but has not been tested. Doing so may damage or otherwise corrupt other filesystems. You do this AT YOUR OWN RISK.

Hmm..It seems that our ext3 filesystems are still being checked every 30 mounts or so. This is a good default for many because it helps prevent filesystem corruption when you have hardware issues, such as bad IDE/SATA/SCSI cabling, power supply failures, etc. One of the driving forces for creating journalling filesystems was that the filesystem could easily be returned to a consistent state by recovering and replaying the needed journalled transactions. Therefore, we can safely disable these mount-count- and time-dependent checks if we are certain the filesystem will be quickly checked to recover the journal if needed to restore filesystem and data consistency. Before you do this please make sure your filesystem entry in /etc/fstab has a positive integer in its 6th field (pass) so that it is checked at boot time automatically. You may do so using the following command:

Code: Select all

# tune2fs -c 0 -i 0 /dev/hdXY

V: Checking The Filesystem Options Using tune2fs

Well, now that we've tweaked our filesystem, we want to make sure those tweaks are applied, right? Surprisingly, we can do this options iusing the tune2fs utility quite easily. To list all the contents of the filesystem's superblock, we can pass the "-l" (lowercase "L") option to tune2fs:

Code: Select all

# tune2fs -l /dev/hdXY

Unlike the other tune2fs calls, this can be run on a mounted filesystem without harm, since it doesn't access or attempt to change the filesystem at such a low level.

This will give you a lot of information about the filesystem, including the block/inode information, as well as the filesystem features and default mount options, which we are looking for. If all goes well, the relevant part of the output should include "dir_index" and "has_journal" flags in the Filesystem features listing, and should show a default mount option of "journal_data".

This concludes this filesystem tweaking guide for now. Happy hacking!

[i92guboj]

Nice thing!
Just one thing: woundn't it be:

Code: Select all

tune2fs -O dir_index,has_journal /dev/hdXY
tune2fs -o journal_data /dev/hdXY

in place of:

Code: Select all

tune2fs -O dir_index /dev/hdXY
tune2fs -o has_journal,journal_data /dev/hdXY

Editer for clarification: notice that 'has_journal' parameter is valid for '-O' (upper case 'o') modifier, not for '-o' (lower case 'o').

[codergeek42]

Gah! I can't believe I didn't see that. Thanks 6thpink...

[jetsaredim]

is there a way to list the options for a particular ext3 fs options that have been set?

I'm not sure if things are working correctly... I formatted the FS with mke2fs -j /dev/vg/ext3, but when I went to mount it it seems to be using the ext2 kernel module...

[saffy]

Thanks!

You are absolutely right about ext3 and full journal mode. I have several systems running a combination of XFS and ReiserFS and my main workstation has used both for various periods. Recently I backed up my workstation and switched to ext3 with full journal and have to report that the performance is nothing short of amazing! I also don't 'notice' disk access as much as I used to with XFS or ReiserFS, ext3 appears to be much smoother with disk access not interrupting normal workstation activity during heavy usage.

I would also recommend adding orlov and commit=9999 to you mount options.

As you can tell I am now a big fan of ext3 full journal mode

[i92guboj]

Gah! I can't believe I didn't see that. Thanks 6thpink...

Thanks to u, I just got an error and looked into man page Now my super-ext3-home reads smooth. The rest of my fs's are reiserfs, I think I'm gonna try, since I really prefer a hard-stable system, and ext3 if for me the best fs all over the world when it comes to reliability.

[jetsaredim]

So, if I were looking for the best performance, I would run the commands in the original post?

I'm trying to do some performance testing for some research I'm working on... Its not easy trying to find good recommendations of performance tuning options for any of the filesystems (ext3, reiserfs, etc...)

[i92guboj]

So, if I were looking for the best performance, I would run the commands in the original post?

I'm trying to do some performance testing for some research I'm working on... Its not easy trying to find good recommendations of performance tuning options for any of the filesystems (ext3, reiserfs, etc...)

Yes, codergeek corrected it.

[codergeek42]

is there a way to list the options for a particular ext3 fs options that have been set?

`tune2fs -l /dev/hdXY` will list the current contents of the filesystem's superblock.

[acasto]

There are two different ways to activate journal data mode. The first is by adding journal=data as a mount option in /etc/fstab. If you do it this way and want your root filesystem to also use it, you should also pass rootfsflags=data=journal as a kernel parameter in your bootloader's configuration.

Shouldn't that be data=journal for the option to /etc/fstab? Or does the order not matter?



- Adam

[codergeek42]

Shouldn't that be data=journal for the option to /etc/fstab? Or does the order not matter?

Your kernel has to mount your root filesystem before it can read the /etc/fstab file that it contains, so only putting it in there would effect everything _except_ your root partition . It's kind of like the chicken-and-egg problem...

[acasto]

Your kernel has to mount your root filesystem before it can read the /etc/fstab file that it contains, so only putting it in there would effect everything _except_ your root partition . It's kind of like the chicken-and-egg problem...

I thought that when it boots up and initially mounts the root filesystem as read-only, that it then looks at /etc/fstab and applies it upon remount to read-write. I just ask becaue I've been using data=journal in my /etc/fstab on the root filesystem for about the last year and it shows up as being mounted properly with that option.


- Adam

[codergeek42]

Hm.. perhaps since Gentoo's initscripts do remount the root filesystem that does apply. Good point, Adam. Thanks.

[OneOfMany]

I think the above post was pointing out that the above instructions shouldn't be "journal=data", but "data=journal" (for "/etc/fstab").

[codergeek42]

I think the above post was pointing out that the above instructions shouldn't be "journal=data", but "data=journal" (for "/etc/fstab").

I'm stupid. I didn't even notice that. Thanks OneOfMany

[barrct]

I _should_ be running all of my drives and partitions in ext3, but I seem to end up with corruptions regularly. Is there a way to view the journal, or see if it is functioning?

[codergeek42]

I _should_ be running all of my drives and partitions in ext3, but I seem to end up with corruptions regularly. Is there a way to view the journal, or see if it is functioning?

Malfunctioning? That's definitely something I don't expect ext3 to do. Perhaps your hardware is going dead? Do other filesystems (such as ReiserFS, XFS, or *shudder* FAT32) work ok on the same disk(s)?

[irondog]

Is there any way to do defragmentation on ext3? My experience is that havily used ext3 partitions become slower and slower while the amount of files in the filesystem and used diskspace don't significally change.

[codergeek42]

ext3 automagically handles defragmentation as you use it. The best thing I can think of is to try re-optimizing the filesystem structure by running the following:

Code: Select all

# e2fsck -D /dev/hdXY

(/dev/hdXY should be unmounted as explained in the first post in order to avoid filesystem corruption.)

[barrct]

I _should_ be running all of my drives and partitions in ext3, but I seem to end up with corruptions regularly. Is there a way to view the journal, or see if it is functioning?

Malfunctioning? That's definitely something I don't expect ext3 to do. Perhaps your hardware is going dead? Do other filesystems (such as ReiserFS, XFS, or *shudder* FAT32) work ok on the same disk(s)?

The hardware has seemed to be fine, I haven't used it much, well at all really in anything other than our main server since it required an HVD SCSI connection that isn't exactly a common thing.
As for the hardware going bad, well it should be, it seems to happen on every drive, and every drive is mirrored to another array. The setup in like this.
Server
||--- Array 1 A B C D D D D
|--- Array 2 A B C D D D D

Each array is on a different controller and different cables, and the drives are mirrored across the two arrays, PLUS the 4D drives are in a RAID 5 that is mirrored.

So if there is something going bad, I think that if ext3 can't catch it, then the RAID5 should and if it's the card or cable, the mirror should.

right?

So is there any way to test the journal? To view it? To view it's size?

[codergeek42]

That I'm not sure of sorry

[barrct]

Yea, it's got me fairly perplexed as well. Of course, I was makeing shure that my backups are running and I saw this.

Code: Select all

Apr 12 03:02:25 William kjournald starting. Commit interval 5 seconds
Apr 12 03:02:25 William EXT3 FS 2.4-0.9.19, 19 August 2002 on md(9,3), internal journal
Apr 12 03:02:25 William EXT3-fs: mounted filesystem with ordered data mode.

Apr 12 03:09:53 William kjournald starting. Commit interval 5 seconds
Apr 12 03:09:53 William EXT3-fs: mounted filesystem with ordered data mode.

So that leads me to think that the journal is running at least.

[c0bblers]

Hi,

After having used XFS for a LONG time I decided to switch totally to ext3 after playing with ubuntu on ext3. Here's what I noticed....disk head movement (or what sounds like it anyway). With XFS the disk is thrashed to within an inch of it's life at times compared to ext3, this probably isn't as noticable with some disks but my Barracudas are whisper quiet when the head isn't moving. It's particularly noticable when scrolling through a list of emails in a folder in Evolution, which for some reason causes a LOT of HD activity on XFS....ext3 is whisper quiet by comparison. I've noticed no slow down of any note, even with the full data journalling codergeek suggests. In fact, some apps seem to start up quicker...though that may be a placebo effect. All in all I'm very happy with ext3 so far, plus I have the warm fuzzy feeling of full data journalling keeping my precious data together. I'm officially a convert.

Cheers,
James

[codergeek42]

I'm officially a convert.

Excellent.

[Crazor]

I would also recommend adding orlov and commit=9999 to you mount options.

when does one benefit from using orlov?
and what exactly does commit=9999 mean?