Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Some ext3 Filesystem Tips
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, ... 13, 14, 15  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
codergeek42
Bodhisattva
Bodhisattva


Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)

PostPosted: Thu Apr 14, 2005 12:44 am    Post subject: Reply with quote

You do not need to use orlov as a mount option, since, according to the mount(8) man page, it is the default if neither oldalloc nor orlov is specified. This option would tell ext2/ext3 whether to use the old inode allocator or the new Orlov inode allocator.

I also highly recommend against using commit=9999. This mount option specifies how often (in second intervals) to sync the data to disk. Setting this too high may cause excessive usage of memory and possibly CPU/swap resources. This really is not needed (and from my experience) will not give you a large performance increase at all.

Edit: Disabled smiley.
_________________
~~ Peter: Brony, GNU/Linux geek, caffeine addict, and Free Software advocate.
Who am I? :: EFF & FSF


Last edited by codergeek42 on Thu Apr 14, 2005 2:01 am; edited 1 time in total
Back to top
View user's profile Send private message
i92guboj
Moderator
Moderator


Joined: 30 Nov 2004
Posts: 9810
Location: Córdoba (Spain)

PostPosted: Thu Apr 14, 2005 1:41 am    Post subject: Reply with quote

codergeek42 wrote:
You do not need to use orlov as a mount option, since, according to the mount(8) man page, it is the default if neither oldalloc nor orlov is specified. This option would tell ext2/ext3 whether to use the old inode allocator or the new Orlov inode allocator.

I also highly recommend against using commit=9999. This mount option specifies how often (in second intervals) to sync the data to disk. Setting this too high may cause excessive usage of memory and possibly CPU/swap resources. This really is not needed (and from my experience) will not give you a large performance increase at all.
I agree and also add that a so high commit raises the probability of data loss, and we dont want that (if we wanted we would be using reiser4 :twisted: )
_________________
Gentoo Handbook | My website
Back to top
View user's profile Send private message
Boris27
Guru
Guru


Joined: 05 Nov 2003
Posts: 562
Location: Almelo, The Netherlands

PostPosted: Sat Apr 16, 2005 5:05 pm    Post subject: Reply with quote

The doc by drobbins says its rootflags=data=journal instead of rootfsflags. I was kinda wondering why it wasn't working ;)

PS: I'm on dir_index now :) Nice stuff
_________________
we are microsoft, lower your firewalls and surrender your pc's. we will add your biological and technological distinctiveness to our own. your culture will adapt and service us. resistance is futile.
Back to top
View user's profile Send private message
codergeek42
Bodhisattva
Bodhisattva


Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)

PostPosted: Sun Apr 17, 2005 5:44 pm    Post subject: Reply with quote

Oops. :oops: Thanks Boris27. 8)
_________________
~~ Peter: Brony, GNU/Linux geek, caffeine addict, and Free Software advocate.
Who am I? :: EFF & FSF
Back to top
View user's profile Send private message
monkey89
Guru
Guru


Joined: 08 Mar 2004
Posts: 596

PostPosted: Sun Apr 17, 2005 8:28 pm    Post subject: Reply with quote

After each of the filesystems I'm mounting is listed in the boot scripts, Gentoo always says (check at next mount). Ubuntu isn't doing this and its dual-booting with the same partitions. Is this fixable?

I've done everything as said in the howto. Otherwise, thanks for the tips, hopefully it will speed things up.

Edit: Weird, but even though ubuntu and gentoo both have e2fsprogs 1.35, upgrading gentoo's to 1.37 seems to hide the warning.

-Monkey
Back to top
View user's profile Send private message
warrior
n00b
n00b


Joined: 27 Sep 2004
Posts: 23
Location: LJ, Slovenija

PostPosted: Tue Apr 26, 2005 12:25 pm    Post subject: Reply with quote

Hi, guys.

Does anybody tried to compare tuned ext3fs perfomance with reiserfs36?
I dont' sure, that namesys tests are reality...
_________________
warrior
Back to top
View user's profile Send private message
wing
Tux's lil' helper
Tux's lil' helper


Joined: 27 Aug 2002
Posts: 108
Location: Mountain View, CA

PostPosted: Thu Apr 28, 2005 2:15 am    Post subject: Reply with quote

warrior wrote:
Hi, guys.

Does anybody tried to compare tuned ext3fs perfomance with reiserfs36?
I dont' sure, that namesys tests are reality...


I know this isn't what you asked, you can ignore my post. I'm just going to recap my experiences with two systems and reiser3, reiser4 and ext3.

I'm reinstalling my system with ext3 from a previous install with reiser4, I've enabled ext3's b-trees and I honestly cannot notice the difference. emerge sync goes as fast as it ever did, as do my tar extractions and whatnot. Also, it is nice to use a FS that you know that you have a good chance of getting your data from in a worst case scenario. I'm using a SATA setup here though, that might effect things for either better or worse.

So I'll tell you my experience in the past. I've had horrible experiences with reiser3 on my x86 ATA system though, I've found that I'd install and literally three to four boots down the line my fs would be hosed. Their fsck tools are no guarantee for safety, either, they barely helped at all, at best they just chocked EVERYTHING in /lost+found. ext3 on the same system performed pretty much on the same level.

Honestly, I love that feeling you can get from being on the bleeding edge, but on a system like gentoo it's good know you're planning for the long term. Even though reiser3 has been around for a while, I think I like the contentment of stability better than than the excitement of the bleeding edge (though I can't help but be ~amd64 :)).

edit: missing words yay!

and thank you vvv :)


Last edited by wing on Thu Apr 28, 2005 8:26 am; edited 2 times in total
Back to top
View user's profile Send private message
warrior
n00b
n00b


Joined: 27 Sep 2004
Posts: 23
Location: LJ, Slovenija

PostPosted: Thu Apr 28, 2005 6:04 am    Post subject: Reply with quote

wing wrote:

I know this isn't what you asked, you can ignore my post. I'm just going to recap my experiences with two systems and reiser3, reiser4 and ext3.

Its really a goot post.
I try to deside for myself what do i want.
I dont have a SATA drive. I'm using notebook. Originally it has 4200rpm/8192Kb cache drive. It's slow one, much more slower than SATA, you know.. I've found another drive 5400rpm/16384Kb just for performance increasing. I also have a bad expirience with reiserfs, but it was few years in past.. May be current versions are more stable. And it will vexing situation when i'll upgrade my hardware but will use slow fs.

I understand, it's not a question, just speak out...
May be somebody was in the same situation and tell his experience..
_________________
warrior
Back to top
View user's profile Send private message
ruben
Guru
Guru


Joined: 04 Jul 2003
Posts: 462

PostPosted: Sat Apr 30, 2005 7:45 pm    Post subject: Reply with quote

I have an iBook with a 30Gb 4200rpm/2Mb drive, which is pretty slow. I had it running with the /-partition on Reiserfs and the /home-partition on ext3, but over time i just got the impression that things were just getting slower and slower on my laptop. And then mainly starting up applications. From gdm to a fully working gnome desktop took 45 seconds, while the cpu didn't seem busy at all during those 45 secs. Then i read this thread and also the optimisation to use B-tree for the directories and i decided to backup all my data, reformat and repartition the harddrive and copy everything back. I already suspected that my /-partition must have suffered a lot from fragmentation... the /-partition has always been reasonably filled, and i also think that a gentoo system is especially 'hard' on the filesystem. I run a "~ppc" on the iBook, and there is a frequent installing of new programs and removing of old programs, also a lot of creating temporary files during the compiles, i think all those operations are causing fragmentation on your partition. So i made 3 partitions, one for /home, one for /, and one partition that is used to store all ebuilds and as compile space for portage. I want my data in /home to be as secure as possible and i used ext3 for that. For the / is just took plain old ext2 with the B-tree optimisation stuff. And for the third partition, i also took ext2 and i made sure that the block size was small, since this partition will contain a lot of tiny files. I took ext2 for those partitions since it is faster than ext3, and i don't care that much if i'd lose something from those partitions (besides all data on the iBook is also put on backup).

Anyways, the important thing here is that defragmentation can deteriorate your performance, and a good way to 'defragment' is to backup the whole partition (make a tar.gz), reformat and copy your data back: all data which probably belongs together (in the same directory) will be copied right next to each other on the harddisk, and this can speed up application startup. My gdm to fully working gnome desktop went from 45 seconds (on reiserfs) to 20 seconds (on ext2). You can read in several places that ext2/3 does not need defragmenting... well... i'm still suspicious about that (a lot of emerging/unmerging on a nearly full partition just can't be good for the data layout on your harddisk), but i hope it'll do better than reiserfs in the long term. In any case, i also don't have the feeling that this ext2/3 is slower than reiserfs (like it was initially).
Back to top
View user's profile Send private message
acdispatcher
n00b
n00b


Joined: 26 Feb 2004
Posts: 42

PostPosted: Mon May 09, 2005 7:35 pm    Post subject: Reply with quote

codergeek42-

Thanks for the tip. I went from reiserfs to ext3 with no problems. No difference in speed either.

I ended up creating a new partition "rsync"ing it over to the new ext3 partition

modify fstab and stuff reboot and zoooom!!!

.
_________________
Budget PC -
DFI NF3 250Gb Socket 754
AMD Sempron 3100+
1.5 GB Crucial DDR 400(PC 3200)
SAMSUNG 160GB SATA
NVIDIA 6600
Back to top
View user's profile Send private message
codergeek42
Bodhisattva
Bodhisattva


Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)

PostPosted: Mon May 09, 2005 8:32 pm    Post subject: Reply with quote

ruben wrote:
You can read in several places that ext2/3 does not need defragmenting... well... i'm still suspicious about that (a lot of emerging/unmerging on a nearly full partition just can't be good for the data layout on your harddisk),
You don't need to defragment ext2/ext3 because as you use the filesystem file blocks and inodes are moved around and reallocated to keep the data nearly contiguous. It's not perfect, but it works fairly well and you should almost never see a performance degradation caused by the filesystem's fragmentation.
_________________
~~ Peter: Brony, GNU/Linux geek, caffeine addict, and Free Software advocate.
Who am I? :: EFF & FSF
Back to top
View user's profile Send private message
hbp4c
n00b
n00b


Joined: 17 Apr 2002
Posts: 46
Location: Charlottesville, Va

PostPosted: Thu May 12, 2005 10:25 pm    Post subject: Reply with quote

codergeek42 wrote:
ruben wrote:
You can read in several places that ext2/3 does not need defragmenting... well... i'm still suspicious about that (a lot of emerging/unmerging on a nearly full partition just can't be good for the data layout on your harddisk),
You don't need to defragment ext2/ext3 because as you use the filesystem file blocks and inodes are moved around and reallocated to keep the data nearly contiguous. It's not perfect, but it works fairly well and you should almost never see a performance degradation caused by the filesystem's fragmentation.

If the filesystem gets very full (lets say,<5% free space available), fragmentation goes to hell as one would expect. I fsck'ed a 99% full ext3 disk yesterday, and had all kinds of fragmented files and problems.
Back to top
View user's profile Send private message
codergeek42
Bodhisattva
Bodhisattva


Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)

PostPosted: Thu May 12, 2005 10:34 pm    Post subject: Reply with quote

hbp4c wrote:
If the filesystem gets very full (lets say,<5% free space available), fragmentation goes to hell as one would expect. I fsck'ed a 99% full ext3 disk yesterday, and had all kinds of fragmented files and problems.
That's a good point (and quite true, I might add). This is also why I recommend creating filesystems slightly larger than the size you'll expect to need. Thanks for bringing this to my attention. :)
_________________
~~ Peter: Brony, GNU/Linux geek, caffeine addict, and Free Software advocate.
Who am I? :: EFF & FSF
Back to top
View user's profile Send private message
prymitive
Apprentice
Apprentice


Joined: 13 Jun 2004
Posts: 260

PostPosted: Fri May 13, 2005 6:27 am    Post subject: Reply with quote

What about space usage? reiser 3.6 is very good at this and reiser4 is even little better (well it pack's data more on disk but reserves 5% of space at now for safety of commits so acually it will probably waste more space then save ;), I hope they will change that 5% to twice the ram size or whatever). I remember that I tested ext3 vs reiser3.6/4 vs xfs and using default settings ext3 had the lowest amount of free space (I used mostly small files like kernel sources and mp3's), xfs was a little better and the best one was reiser3.6 (as I said reiser4 had little lower amount of used space but formatted partition size was smaller due to that 5% he reserves). If I remember correctly the values were:

r3.6 - about 2GB
xfs - about 2,5GB
ext3 - about 2.6GB

this was on 10GB partition, all using default settings.

btw. reiser4 is realy fast for me and it made my hard drive much quiter, unfortunetly it's performance degradates with time and You need to use repacker (which is not ready yet) to resore it.
Back to top
View user's profile Send private message
ndbruin
n00b
n00b


Joined: 14 Jul 2003
Posts: 15

PostPosted: Fri May 13, 2005 10:57 am    Post subject: Re: Some ext3 Filesystem Tips Reply with quote

:!: WARNING: Make sure any filesystems are unmounted before altering them with the tune2fs or e2fsck utilities! (Boot from a LiveCD if you need to.) Altering or tuning a filesystem while it is mounted can cause severe corruption! You have been warned!

I just tried this out and you will get corruption, so be warned! (had to see it happen ;) )

Furthermore I am now also convinced of using ext3 instead of reiserfs, so hope to see some more performance options :D
Thanks for these tweaks
Back to top
View user's profile Send private message
codergeek42
Bodhisattva
Bodhisattva


Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)

PostPosted: Fri May 13, 2005 3:28 pm    Post subject: Reply with quote

@prymitive: You can adjust your block size and inode count when you initially create the filesystem. Setting it to use 1KB block sizes and 1 inode per 1KB should give you the most effecient space usage:
Code:
# /sbin/mkfs.ext3 -b 1024 -i 1024 /dev/hXY
Be warned though that decreased the block size and increasing the inode allocation in this manner can cause a significant performance decrease if the filesystem is store larger files as well (since it has to do more journalling, I/O, and resource allocation).

@ndbruin: I warned you about that, silly! :P Glad to have another Ext3 fan though. :)
_________________
~~ Peter: Brony, GNU/Linux geek, caffeine addict, and Free Software advocate.
Who am I? :: EFF & FSF
Back to top
View user's profile Send private message
prymitive
Apprentice
Apprentice


Joined: 13 Jun 2004
Posts: 260

PostPosted: Fri May 13, 2005 3:36 pm    Post subject: Reply with quote

codergeek42 wrote:
@prymitive: You can adjust your block size and inode count when you initially create the filesystem. Setting it to use 1KB block sizes and 1 inode per 1KB should give you the most effecient space usage:
Code:
# /sbin/mkfs.ext3 -b 1024 -i 1024 /dev/hXY
Be warned though that decreased the block size and increasing the inode allocation in this manner can cause a significant performance decrease if the filesystem is store larger files as well (since it has to do more journalling, I/O, and resource allocation).

@ndbruin: I warned you about that, silly! :P Glad to have another Ext3 fan though. :)


I know that You can change block size but as You said it costs us speed, reiserfs have 4kb blocks but it can pack few files into one cluster so theye are kept inside directory tree, ntfs can also do this but only for files that are 650B or smaller. So under reiserfs big files are being read with full speed while small files can use only needed space without low block size (at least in theory).
Back to top
View user's profile Send private message
hbp4c
n00b
n00b


Joined: 17 Apr 2002
Posts: 46
Location: Charlottesville, Va

PostPosted: Fri May 13, 2005 3:50 pm    Post subject: Reply with quote

prymitive wrote:

I know that You can change block size but as You said it costs us speed, reiserfs have 4kb blocks but it can pack few files into one cluster so theye are kept inside directory tree, ntfs can also do this but only for files that are 650B or smaller. So under reiserfs big files are being read with full speed while small files can use only needed space without low block size (at least in theory).

A lot of people who use reiserfs use the notail option, which doesn't pack the small files into a shared block of data. This is known to increase performance noticably, at the cost of space.

In order to compare oranges with oranges, you should compare sizes of files on a reiserfs filesystem with notail option, without notail option, and compare that to ext3. If you reference that against speed on all three filesystems, you'd be dangerously close to getting valid and useful results.
Back to top
View user's profile Send private message
regeya
Apprentice
Apprentice


Joined: 28 Jul 2002
Posts: 270
Location: Desoto, IL, USA

PostPosted: Mon May 16, 2005 5:55 pm    Post subject: Reply with quote

Wow, y'all just discovered dir_index? I had suggested it a few times before I went Ubuntu, and the response I usually got was "um, why don't you just use reiser?"

Um, because I like my data and loathe restoring from backups? ;-)
_________________
Why, yes, I am a bore.
Back to top
View user's profile Send private message
syrrus
n00b
n00b


Joined: 16 May 2005
Posts: 24
Location: College Station, TX

PostPosted: Mon May 16, 2005 7:55 pm    Post subject: Reply with quote

ext3 supports POSIX ACLs which is still fairly unused in the linux world. Check out these links

http://gentoo-wiki.com/HOWTO_Use_filesystem_ACLs
http://www.suse.de/~agruen/acl/linux-acls/online/
http://www.suse.de/~agruen/ea-acl-copy/
http://security.linux.com/article.pl?sid=04/07/28/1746258&tid=23&tid=35

Also this series is a good compilation of information.
http://www-106.ibm.com/developerworks/library/l-fs.html
http://www.linuxplanet.com/linuxplanet/reports/4136/2/
_________________
Gates Closed,
Windows Broken,
I Found the Source;
And It Was Open.
Back to top
View user's profile Send private message
jdgill0
Veteran
Veteran


Joined: 25 Mar 2003
Posts: 1366
Location: Lexington, Ky -- USA

PostPosted: Mon May 16, 2005 8:19 pm    Post subject: Reply with quote

The ext3 filesystem can make files immutable, where not even root can delete the file when the immutable bit is set. I have not run across any mention of this in the forums myself. It seems this could be a nice security feature for such things as config files.

Does anyone here have an opinion on immutable files? I posted this here, because it's my understanding that only ext2,3 have immutable file support.
_________________
Vim has excellent syntax highlighting for configuration files: emerge gentoo-syntax
Learn how to use Vim: vimtutor
Back to top
View user's profile Send private message
syrrus
n00b
n00b


Joined: 16 May 2005
Posts: 24
Location: College Station, TX

PostPosted: Tue May 17, 2005 1:29 am    Post subject: Reply with quote

No ReiserFS has support aswell.
_________________
Gates Closed,
Windows Broken,
I Found the Source;
And It Was Open.
Back to top
View user's profile Send private message
jdgill0
Veteran
Veteran


Joined: 25 Mar 2003
Posts: 1366
Location: Lexington, Ky -- USA

PostPosted: Tue May 17, 2005 2:15 am    Post subject: Reply with quote

syrrus,

What command do you use to set the immutable attribute under reiserfs? Man pages for chattr and lsattr indicate only functioning with ext2,3.
_________________
Vim has excellent syntax highlighting for configuration files: emerge gentoo-syntax
Learn how to use Vim: vimtutor
Back to top
View user's profile Send private message
syrrus
n00b
n00b


Joined: 16 May 2005
Posts: 24
Location: College Station, TX

PostPosted: Tue May 17, 2005 2:32 am    Post subject: Reply with quote

Code:
hera etc # grep /dev/hdc3 /etc/fstab
/dev/hdc3               /               reiserfs                noatime,notail,acl                              0 0
hera etc # chattr +i /etc/shadow
hera etc # lsattr /etc/shadow
----i-------- /etc/shadow
hera etc #

_________________
Gates Closed,
Windows Broken,
I Found the Source;
And It Was Open.
Back to top
View user's profile Send private message
syrrus
n00b
n00b


Joined: 16 May 2005
Posts: 24
Location: College Station, TX

PostPosted: Tue May 17, 2005 2:38 am    Post subject: Reply with quote

Another little tidbit that needs to be discussed when talking about these immutable bits is the following.

Now I can simply chattr -i /etc/shadow anytime I want as root and it'll be like it never happened.
However with seclvl (A linux implementation of BSD Secure Levels) the behavior mimics that of
BSD. So when using these attributes remember to echo "2" > /sys/seclvl/seclvl if you have this support
built into your kernel.

The hardened kernel series I know supports this and is always a great idea to use in any secure server
implementation.
_________________
Gates Closed,
Windows Broken,
I Found the Source;
And It Was Open.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3, ... 13, 14, 15  Next
Page 2 of 15

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum