Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Some ext3 Filesystem Tips
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3 ... 12, 13, 14, 15  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
neuron
Advocate
Advocate


Joined: 28 May 2002
Posts: 2371

PostPosted: Sun Dec 17, 2006 6:44 pm    Post subject: Re: Ext3 patch: mballoc Reply with quote

1bitmemory wrote:
Has anyone here played with the ext3 patch set mentioned here ( http://lkml.org/lkml/2006/6/6/65 ) ? Some more explanations here http://lwn.net/Articles/80285/



1bm


Looks extremely interesting, found this aswell:
http://www.ussg.iu.edu/hypermail/linux/kernel/0606.0/1580.html

//edit and this:
https://mail.clusterfs.com/pipermail/lustre-discuss/2006-June/001601.html

//edit and this showing performance gains:
http://www.bullopensource.org/ext4/lowmemory/index.html
Back to top
View user's profile Send private message
onesandzeros
Tux's lil' helper
Tux's lil' helper


Joined: 30 Sep 2002
Posts: 108

PostPosted: Wed Jan 31, 2007 9:16 am    Post subject: Reply with quote

I looked through this thread for this issue, but I may have missed it in the 13 pages, heh. When I applied the changes in the first post, I had some weirdness in KDE. Specifically, I often had kio_slaves crashes, and the occasional total Konq failure. I was running 3.5.5 and 3.5.6 during that period. Prior to the changes, I never had those troubles (numerous KDE versions) and now that I've undone them, 3.5.6 is as stable as I'd expect. Has anyone else had this trouble?
Back to top
View user's profile Send private message
adsmith
Veteran
Veteran


Joined: 26 Sep 2004
Posts: 1386
Location: NC, USA

PostPosted: Wed Jan 31, 2007 2:03 pm    Post subject: Reply with quote

That's interesting.
I've had some sporadic KDE crashing [Krashing?], but I just assumed it was KDE.

Have you done any experiments to determine which options are correlated with the wonkiness?
Back to top
View user's profile Send private message
onesandzeros
Tux's lil' helper
Tux's lil' helper


Joined: 30 Sep 2002
Posts: 108

PostPosted: Wed Jan 31, 2007 2:35 pm    Post subject: Reply with quote

No, unfortunately I didn't. I might in the future, but my gentoo box is my *only* box, and ext3's default settings are fine. The dir_index is the option that's supposed to speed things up, right? I didn't notice much of a difference at all. Maybe it's my hardware.

Also, I didn't mean to imply that I had other KDE problems (or any other problems at all). Only konq (sometimes) and kio_slaves (often) were wacky. I don't recall any other problems.
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3205

PostPosted: Wed Jan 31, 2007 8:18 pm    Post subject: Reply with quote

onesandzeros wrote:
No, unfortunately I didn't. I might in the future, but my gentoo box is my *only* box, and ext3's default settings are fine. The dir_index is the option that's supposed to speed things up, right? I didn't notice much of a difference at all. Maybe it's my hardware.

Also, I didn't mean to imply that I had other KDE problems (or any other problems at all). Only konq (sometimes) and kio_slaves (often) were wacky. I don't recall any other problems.
Do you have the problem of kopete just not responding all of a sudden (even in the middle of a chat)? And that the problem does not go away until you kill kopete, and then killall -9 dcopserver?
Back to top
View user's profile Send private message
onesandzeros
Tux's lil' helper
Tux's lil' helper


Joined: 30 Sep 2002
Posts: 108

PostPosted: Thu Feb 01, 2007 12:13 pm    Post subject: Reply with quote

ppurka wrote:
onesandzeros wrote:
No, unfortunately I didn't. I might in the future, but my gentoo box is my *only* box, and ext3's default settings are fine. The dir_index is the option that's supposed to speed things up, right? I didn't notice much of a difference at all. Maybe it's my hardware.

Also, I didn't mean to imply that I had other KDE problems (or any other problems at all). Only konq (sometimes) and kio_slaves (often) were wacky. I don't recall any other problems.
Do you have the problem of kopete just not responding all of a sudden (even in the middle of a chat)? And that the problem does not go away until you kill kopete, and then killall -9 dcopserver?

I don't use Kopete much (don't think I used it all while I had the ext3 mods in place), but I've got it going now. Seems ok. Feel free to contact me at my jabber address, heheh.
Back to top
View user's profile Send private message
carpman
Advocate
Advocate


Joined: 20 Jun 2002
Posts: 2202
Location: London - UK

PostPosted: Fri Feb 02, 2007 2:24 pm    Post subject: Reply with quote

Hello, ok i have 200 GB partition on my backup server that is only going to be used for backups.

I thinking that ext3 is going to be best for this but was wondering if that is correct and if so what are best settings to use on it?

cheers
_________________
Work Station - 64bit
Gigabyte GA X48-DQ6 Core2duo E8400
8GB GSkill DDR2-1066
SATA Areca 1210 Raid
BFG OC2 8800 GTS 640mb
--------------------------------
Notebook
Samsung Q45 7100 4gb
Back to top
View user's profile Send private message
vanten
n00b
n00b


Joined: 08 Jan 2006
Posts: 44
Location: Sweden

PostPosted: Sat Feb 03, 2007 4:23 am    Post subject: Reply with quote

Thanks there codergeek42, Your first post worked like a charm now.
( And thanks to all people thats brought up those things that made it that way. )
_________________
.: GNU/Linux :: Gentoo :: Fluxbox :.
.: BSD :: OpenBSD :: Fluxbox :.
Back to top
View user's profile Send private message
XenoTerraCide
Veteran
Veteran


Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA

PostPosted: Sat Feb 17, 2007 6:01 pm    Post subject: Reply with quote

hmm... I had an filesystem consistency problem on /home 2 directories had lost there permissions. so I rebooted and of course ext3 put things right... but I decide to run further checks, and I found out journal_data was no longer on any of my filesystems. this is the first thing I do after making ext3 in a new install and I remember doing it. so at some point it disappeared. no idea why how or when. but I would be interested to know if other's have had this problem? secondly. I don't really want to burn a recovery disk right now. and I'm not sure I actually have a good one. would enabling it before I reboot and then forcing an fsck on the next boot fix it? how do I force an fsck on 2 filesytems on reboot?
_________________
I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help.
Back to top
View user's profile Send private message
onesandzeros
Tux's lil' helper
Tux's lil' helper


Joined: 30 Sep 2002
Posts: 108

PostPosted: Sun Feb 18, 2007 4:23 am    Post subject: Reply with quote

XenoTerraCide wrote:
how do I force an fsck on 2 filesytems on reboot?

Well, I'm not sure about your other problems (it was good to post them, someone will probably know), but as for the fsck: shutdown from a console with 'shutdown -rF now'. The -r will reboot, the -F will force a fsck of any partition that mounts automatically at boot (that is, a partition with 'auto' in its fs_mntops field in fstab).

As I wrote in an earlier post in this thread, I was getting some real oddball behavior from konqueror and the kio daemon. Undoing the fs modifications that were listed in the first post of this thread seems to have taken care of those issues. Why those problems came about is still beyond me.
Back to top
View user's profile Send private message
Maf
Apprentice
Apprentice


Joined: 15 May 2005
Posts: 278

PostPosted: Wed Feb 28, 2007 6:10 pm    Post subject: Reply with quote

Guys, how can I check whether the system is mounted at the moment with 'journal' option to make me certain?
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2734
Location: Bay Area, CA

PostPosted: Wed Feb 28, 2007 6:14 pm    Post subject: Reply with quote

Maf wrote:
Guys, how can I check whether the system is mounted at the moment with 'journal' option to make me certain?
Code:
tune2fs -l <device_with_ext3_filesystem>
grep <device_with_ext3_filesystem> /etc/mtab
dmesg | grep "EXT3-fs: mounted filesystem"
PS: I forgot one....;-)

Last edited by devsk on Wed Feb 28, 2007 6:32 pm; edited 1 time in total
Back to top
View user's profile Send private message
XenoTerraCide
Veteran
Veteran


Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA

PostPosted: Wed Feb 28, 2007 6:18 pm    Post subject: Reply with quote

it's also in part v? of the tutorial on page 1. I had codegeeker add it ages ago.
_________________
I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help.
Back to top
View user's profile Send private message
Maf
Apprentice
Apprentice


Joined: 15 May 2005
Posts: 278

PostPosted: Wed Feb 28, 2007 6:23 pm    Post subject: Reply with quote

devsk wrote:
Maf wrote:
Guys, how can I check whether the system is mounted at the moment with 'journal' option to make me certain?
Code:
tune2fs -l <device_with_ext3_filesystem>
grep <device_with_ext3_filesystem> /etc/mtab

Well what I mean is, my mount says:
Code:

/dev/sda3 on /home/maf type ext3 (rw,noatime,data=journal)

And my kernel options in grub.conf:
Code:

kernel /boot/linux-2.6.20 root=/dev/sda1 rootflags=data=journal vga=792 video=vesafb:mtrr,ywrap verbose mce

But I kinda' can't believe the "journal-mode" is on ;) And unfortunatelly tune2fs -l doesn't contain this kind of information. Is there any other way?
Back to top
View user's profile Send private message
XenoTerraCide
Veteran
Veteran


Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA

PostPosted: Wed Feb 28, 2007 6:30 pm    Post subject: Reply with quote

part of tune2fs -l should say
Code:
Filesystem features:      has_journal dir_index filetype needs_recovery sparse_super
Default mount options:    journal_data
if you followed the tutorial. it may not doing journal data the way you are. I've never done it that way. has_journal will be in there though. dmesg should tell you how it's mounting it when it boots. that's how I cought that somehow a bunch of my filesystems on a box were in ordered mode.
_________________
I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help.
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2734
Location: Bay Area, CA

PostPosted: Wed Feb 28, 2007 6:33 pm    Post subject: Reply with quote

Maf wrote:
But I kinda' can't believe the "journal-mode" is on ;) And unfortunatelly tune2fs -l doesn't contain this kind of information. Is there any other way?

Code:
dmesg | grep "EXT3-fs: mounted filesystem"
Back to top
View user's profile Send private message
Maf
Apprentice
Apprentice


Joined: 15 May 2005
Posts: 278

PostPosted: Wed Feb 28, 2007 6:44 pm    Post subject: Reply with quote

devsk wrote:
Code:
dmesg | grep "EXT3-fs: mounted filesystem"

Sweet, thx :)
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2734
Location: Bay Area, CA

PostPosted: Fri Mar 30, 2007 9:15 pm    Post subject: Reply with quote

One of the most ignored fact about ext[23] is that its default number of inodes calculation is pretty conservative. It assumes the larger the partition, larger the number of files will be stored on it by the user. e.g. for a 150GB partition, it formats the partition with 23 million inodes. Now, I know I am never going to create that many files/dirs on this partition. The result of large number of inodes is that fragmentation of large files increases drastically because a large file's blocks can not use the blocks that are reserved for inodes and have to be jumped over.

So, what is the cure? Use -N option to specify roughly how many inodes you want on this partition. For example, if I am storing my mp3's on this 150GB partition, and everage size of an mp3 is 5MB, I would store about 30,000 files on it. Account for some small files and dirs, I multiply by 16 and pass "-N 480,000" to mke2fs. Even 1 (as opposed to default 23 in this case) million inodes will reduce your fragmentation by leaps and bound.

Now, the disadvantage of doing this: Because the inode blocks are marked during formatting, you can not change the number of inodes later. So, make a good guess depending upon your situation and multiply it by a factor of 2, 4, 8, or 16 to make sure that you are able to grow, but still keep the number much below the default used by mke2fs.
Back to top
View user's profile Send private message
XenoTerraCide
Veteran
Veteran


Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA

PostPosted: Wed Apr 04, 2007 12:03 am    Post subject: Reply with quote

in reply to devsk observation. anyone know a good way to find the average size and number of files in a directory?
_________________
I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help.
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2734
Location: Bay Area, CA

PostPosted: Wed Apr 04, 2007 2:55 am    Post subject: Reply with quote

XenoTerraCide wrote:
in reply to devsk observation. anyone know a good way to find the average size and number of files in a directory?
you mean in an already populated and mounted file system? if so, following should do:

Code:

#change dir to where you want count and avg size
sum=0
count=0
for i in `find .`;do sz=`ls -ald "$i"|awk '{print $5}'`;sum=$((sum+sz));count=$((count+1));done
echo "Count = $count Average Size = $((sum/count)) Bytes"
Back to top
View user's profile Send private message
XenoTerraCide
Veteran
Veteran


Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA

PostPosted: Wed Apr 04, 2007 3:54 am    Post subject: Reply with quote

yeah I meant populated. The best way to make a guess is to make it educated. In my case I wish to use my existing systems to get an idea what size and kind of files I have in each directory off /. I use reiserfs for things like ccache and portage.
_________________
I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help.
Back to top
View user's profile Send private message
rickrick
n00b
n00b


Joined: 27 Apr 2007
Posts: 5

PostPosted: Fri Apr 27, 2007 6:54 pm    Post subject: Reply with quote

I am using the tuned ext3 and am curious about the speed of seeing how big directories are. When I bring up the properties of say /usr it takes a while (couple minutes) to for ext3 to count up the size of it. On some other filesystems is was a lot quicker (i think it was xfs or ntfs). Anyway... I was just wondering if I was missing a setting... I did everything from the first post. I have a journal size of 128M and I commit=60 (I've used the defaults too). This isn't meant to be a this fs is better than that one... I'm just curious if anyone else notices this.
Back to top
View user's profile Send private message
mno
Apprentice
Apprentice


Joined: 29 Dec 2003
Posts: 285
Location: Toronto, Canada

PostPosted: Sat Jul 28, 2007 3:15 pm    Post subject: Reply with quote

Sorry, maybe this was discussed in the 13+ pages, but before I play around with my partitions, wanted to confirm:

I already have several years ago set up my partitions and formatted everything to use ext3 (except boot). If I switch the journaling mode now, is it safe? I am running a RAID1 setup using a 3ware 8506 card.

Thanks,
Max
_________________
"Hello and goodbye. As always." | You can't use &nbsp; here?? | Unanswered
Back to top
View user's profile Send private message
XenoTerraCide
Veteran
Veteran


Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA

PostPosted: Sat Jul 28, 2007 3:28 pm    Post subject: Reply with quote

yes. but you'll need to run a fsck afterwards.
_________________
I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help.
Back to top
View user's profile Send private message
mno
Apprentice
Apprentice


Joined: 29 Dec 2003
Posts: 285
Location: Toronto, Canada

PostPosted: Sat Jul 28, 2007 3:29 pm    Post subject: Reply with quote

ugh then i'd rather not. it's a server, and if fsck hangs..... :/
_________________
"Hello and goodbye. As always." | You can't use &nbsp; here?? | Unanswered
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3 ... 12, 13, 14, 15  Next
Page 13 of 15

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum