Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Some ext3 Filesystem Tips
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3 ... 11, 12, 13, 14, 15  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
enderandrew
l33t
l33t


Joined: 25 Oct 2005
Posts: 731

PostPosted: Tue Jun 27, 2006 7:39 am    Post subject: Reply with quote

I want to throw things right now.

I normally use Reiser4 all day long on all my boxes. I keep hearing the ext3 is more stable, and offers just as good performance when properly configured.

My first two boxes I put ext3 on, I had crashes and lost data within 2 weeks.

So I decided to go ahead on give ext3 one more try.

Brand new install. I just spent 4 days compiling everything. Now it won't boot, and it says /dev/root is corrupted. the fschk program spots the errors, attempts to fix them, but doesn't do any good. I know I am probably the exception to the rule, but 3 times out of 3, ext3 has taken a crap on me within 2 weeks of using it. I've never lost data with reiser4.

I'm *really* hoping I don't have to format and start over again, because my free time is next to nonexistant these days, but I thought ext3 was supposed to be absolutely stable as a rock.
_________________
Nihilism makes me smile.
Back to top
View user's profile Send private message
Doogman
Apprentice
Apprentice


Joined: 24 Sep 2004
Posts: 241

PostPosted: Tue Jun 27, 2006 8:05 pm    Post subject: Reply with quote

More info:

doug@ghidorah ~ $ uname -a
Linux ghidorah 2.6.16-ck12n #2 SMP Sat Jun 24 12:34:26 EDT 2006 i686 Dual Core AMD Opteron(tm) Processor 175 GNU/Linux

I seem to be having the same slow-write problem with a box running 2.6.16-gentoo-r7. I would like to see some other people's benchmarks! Bonnie++ isn't that hard to run. :)

Question enderandrew... why move from Reiser when you weren't having problems?

Obviously people who use ext3 don't have to reformat partitions every other week as I think you would see some red flags raised somewhere.

<Testimonial> I've been using ext3 since it's inception and my filesystems survived numerious hard crashes and such while trying out new kernels and hardware. Heck, my last system had a hardware problem that when the MB had high IO it would occasionally corrupt the disk writes. Nice. Even so, I never lost a filesystem and in fact only noticied it when I seen the errors corrected during the monthly fsck. Moving to a Tyan K8E fixed all those problems. </Testimonial>
Back to top
View user's profile Send private message
enderandrew
l33t
l33t


Joined: 25 Oct 2005
Posts: 731

PostPosted: Wed Jun 28, 2006 5:12 am    Post subject: Reply with quote

I was doing new installs on two more laptops.
_________________
Nihilism makes me smile.
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2836
Location: Bay Area, CA

PostPosted: Mon Jul 03, 2006 8:35 pm    Post subject: Reply with quote

PaulBredbury wrote:
Here's another ext3 performance tip: Upgrade to kernel 2.6.17 :)

not really, just see these comparisons I made using bonnie++:

Code:

Version 1.93c       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
2.6.15-r8    1800M   331  99   97570  39 73378  25   844    98   253583  47  2728  36
2.6.17-r1    1800M   304  99   96719  35 61476  16   777    97   184444  22  2490  28

These are done using a dual boot install of 2.6.15-r8 and 2.6.17-r1 on the same ext3 filesystem which resides on nvraid, RAID0 using the same bonnie package and same kernel config (make oldconfig using 2.6.15-r8 .config in 2.6.17-r1). Seq rewrite and read rates have fallen by as much as 15-25%. Rewrite may have suffered because of reduced seq. read performance. Upgrades are not always what they are made out to be.
Back to top
View user's profile Send private message
darklegion
Guru
Guru


Joined: 14 Nov 2004
Posts: 468

PostPosted: Thu Jul 06, 2006 5:05 am    Post subject: Reply with quote

devsk wrote:
PaulBredbury wrote:
Here's another ext3 performance tip: Upgrade to kernel 2.6.17 :)

not really, just see these comparisons I made using bonnie++:

Code:

Version 1.93c       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
2.6.15-r8    1800M   331  99   97570  39 73378  25   844    98   253583  47  2728  36
2.6.17-r1    1800M   304  99   96719  35 61476  16   777    97   184444  22  2490  28

These are done using a dual boot install of 2.6.15-r8 and 2.6.17-r1 on the same ext3 filesystem which resides on nvraid, RAID0 using the same bonnie package and same kernel config (make oldconfig using 2.6.15-r8 .config in 2.6.17-r1). Seq rewrite and read rates have fallen by as much as 15-25%. Rewrite may have suffered because of reduced seq. read performance. Upgrades are not always what they are made out to be.


Did you enable "CONFIG_ADAPTIVE_READAHEAD" while doing "make oldconfig"?. I think this option is needed to get the performance increase (I haven't tested the kernel as of yet).
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2836
Location: Bay Area, CA

PostPosted: Thu Jul 06, 2006 5:59 am    Post subject: Reply with quote

where does this config parameter lie? I can't find it.

PS: I found out why its not there. Its in a patch for mm kernel, which is not present in the latest gentoo-sources (at least that string CONFIG_ADAPTIVE doesn't figure in any of files under /usr/src/linux) that I am running. But the poster (and the link he posted) actually said it was in mainline kernel.
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2836
Location: Bay Area, CA

PostPosted: Fri Jul 07, 2006 1:18 am    Post subject: Reply with quote

darklegion wrote:

Did you enable "CONFIG_ADAPTIVE_READAHEAD" while doing "make oldconfig"?. I think this option is needed to get the performance increase (I haven't tested the kernel as of yet).
It seems that the article talks about a different patch than the one which contains the new enhanced ADAPTIVE_READAHEAD. This commit is different from the patch in mm kernel:

http://kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=89747d369d34e333b9b60f10f333a0b727b4e4e2
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2836
Location: Bay Area, CA

PostPosted: Fri Jul 07, 2006 3:18 pm    Post subject: Reply with quote

if it makes anybody happy knowing that they are not missing out on holy-grail, here it is:

Code:

Version 1.93c       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1         -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine           Size    K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
van-2.6.17     1800M   356  98 33834  13 28018  18   934  90 64460  23 186.2   6
mm6-2.6.17   1800M  495  98 33363  11 25761  13  1238  93 64946  18 276.4  13


no gains for adaptive readahead in mm6...:-(

Both tests done under a VM using same install with dual boot setup.
Back to top
View user's profile Send private message
Sedrik
l33t
l33t


Joined: 08 Apr 2005
Posts: 653
Location: Uppsala, Sweden

PostPosted: Wed Aug 02, 2006 8:37 am    Post subject: Reply with quote

How much improvement can one expect these tweaks(full journaling, indexing) to give?

Is it like 0.5% increase? more? less?

I'm a ext3 user already, have been since my friend pointed me to a post somewhere pointing out the faults of reiserfs (which I used before since it was the popular chooice).

Nice guide =)

*Edit
Please continue to add more tweaks and safe modes to run to the first post as time goes by =)
_________________
From Gentoo with love
Back to top
View user's profile Send private message
Sachankara
l33t
l33t


Joined: 11 Jun 2004
Posts: 696
Location: Stockholm, Sweden

PostPosted: Wed Aug 02, 2006 6:12 pm    Post subject: Reply with quote

Full journaling on Ext3 decreases the performance. People should try running with the default to see the speed difference.

Personally I've started to using XFS because it feels faster on everything but really large files. People should try XFS for their /usr/portage partition, then they might notice how fast it stays over time. :)
_________________
Gentoo Hardened Linux 2.6.21 + svorak (Swedish dvorak)
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2836
Location: Bay Area, CA

PostPosted: Wed Aug 02, 2006 6:39 pm    Post subject: Reply with quote

Sachankara wrote:
People should try running with the default to see the speed difference.
what is default? I thought with -j, the default was full (metadata as well as data) journalling.
Sachankara wrote:

Personally I've started to using XFS because it feels faster on everything but really large files. People should try XFS for their /usr/portage partition, then they might notice how fast it stays over time. :)
Tried it. Its freaky fast (mostly because of no data journalling) in benchmarks but my boot times increased on the same hardware in a dual boot setup, meaning real life read performance was worse than ext3. And this was with all the optimizations that I could find. Moreover, no data journalling kinda keeps me away from it.
Back to top
View user's profile Send private message
adsmith
Veteran
Veteran


Joined: 26 Sep 2004
Posts: 1386
Location: NC, USA

PostPosted: Wed Aug 02, 2006 6:56 pm    Post subject: Reply with quote

If you want real data, here are some good and generally accepted benchmarks:
http://linuxgazette.net/122/TWDT.html#piszcz

They are probably posted several pages back as well.

JFS and XFS are both excellent FS's, but as the previous posted said, neither support full-data journalling. With that fact and these benchmarks, EXT3 really shines.

By the way, for a while it was true that data journalling in EXT3 was somehow *faster* than just metadata journalling. See:
http://www-128.ibm.com/developerworks/linux/library/l-fs8.html
I'm not sure if this is still true.
Back to top
View user's profile Send private message
codergeek42
Bodhisattva
Bodhisattva


Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)

PostPosted: Thu Aug 03, 2006 1:36 am    Post subject: Reply with quote

Sedrik wrote:
How much improvement can one expect these tweaks(full journaling, indexing) to give?

Is it like 0.5% increase? more? less?
Honestly, I can't say for certain (though if you'd like, give me some time and I'm sure I could come up with some benchmarks). It certainly feels faster, or at least more responsive, though.

Sedrik wrote:
Nice guide =)
Thanks. :)

Sedrik wrote:
*Edit
Please continue to add more tweaks and safe modes to run to the first post as time goes by =)
If I think of any or find out anything, I'll surely add it. :)
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF
Back to top
View user's profile Send private message
mrcs
Tux's lil' helper
Tux's lil' helper


Joined: 10 Oct 2003
Posts: 137

PostPosted: Wed Aug 23, 2006 9:23 pm    Post subject: Reply with quote

Sedrik wrote:
HNice guide =)

I second this! Without this thread I would'nt even have looked at ext3 after always hearing how slow and basically useless it was, but you showed me the light :)
Back to top
View user's profile Send private message
codergeek42
Bodhisattva
Bodhisattva


Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)

PostPosted: Thu Aug 24, 2006 3:47 am    Post subject: Reply with quote

mrcs wrote:
Sedrik wrote:
HNice guide =)

I second this! Without this thread I would'nt even have looked at ext3 after always hearing how slow and basically useless it was, but you showed me the light :)
Wee. Another Ext3 convert! :D
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF
Back to top
View user's profile Send private message
mrcs
Tux's lil' helper
Tux's lil' helper


Joined: 10 Oct 2003
Posts: 137

PostPosted: Mon Aug 28, 2006 10:52 am    Post subject: Reply with quote

codergeek42 wrote:
Wee. Another Ext3 convert! :D

:)

Just one question though. When I set the size of ccache to 1G or greater, it's just a matter of time before ext3 breaks down, spits out tons of errors and remount / as read only.

If I set ccache to 512 or less, that never happens. What's up with that?
Back to top
View user's profile Send private message
neuron
Advocate
Advocate


Joined: 28 May 2002
Posts: 2371

PostPosted: Mon Aug 28, 2006 1:35 pm    Post subject: Reply with quote

mrcs wrote:
codergeek42 wrote:
Wee. Another Ext3 convert! :D

:)

Just one question though. When I set the size of ccache to 1G or greater, it's just a matter of time before ext3 breaks down, spits out tons of errors and remount / as read only.

If I set ccache to 512 or less, that never happens. What's up with that?


hardware error, and your simply not hitting the error when using a smaller ccache.
Back to top
View user's profile Send private message
mrcs
Tux's lil' helper
Tux's lil' helper


Joined: 10 Oct 2003
Posts: 137

PostPosted: Mon Aug 28, 2006 5:20 pm    Post subject: Reply with quote

neuron wrote:
hardware error, and your simply not hitting the error when using a smaller ccache.

Well, I have the same exact error on two machines, but I guess it could be the hardware. But shouldn't a hardware error pop up when doing other operations too? Like now, my laptop hardrive is 99% full and still no error like that.

EDIT: Um, actually it's nowhere near full, my bad, wrong drive, but the question remains.
Back to top
View user's profile Send private message
neuron
Advocate
Advocate


Joined: 28 May 2002
Posts: 2371

PostPosted: Mon Aug 28, 2006 9:05 pm    Post subject: Reply with quote

mrcs wrote:
neuron wrote:
hardware error, and your simply not hitting the error when using a smaller ccache.

Well, I have the same exact error on two machines, but I guess it could be the hardware. But shouldn't a hardware error pop up when doing other operations too? Like now, my laptop hardrive is 99% full and still no error like that.

EDIT: Um, actually it's nowhere near full, my bad, wrong drive, but the question remains.


it can be a lot of things, but all ccache does is stress the hardware, I very seriously doubt you'll find stability problems in ext3. (atleast without experimental patches) as it's being used on enterprise level all over the world.
Back to top
View user's profile Send private message
mrcs
Tux's lil' helper
Tux's lil' helper


Joined: 10 Oct 2003
Posts: 137

PostPosted: Mon Aug 28, 2006 11:26 pm    Post subject: Reply with quote

neuron wrote:
it can be a lot of things, but all ccache does is stress the hardware, I very seriously doubt you'll find stability problems in ext3. (atleast without experimental patches) as it's being used on enterprise level all over the world.

Yeah, I'm suspecting that my slow laptop hard drive can't cope with that many seeks when the cache is to big. It does work flawlessly when the cache is set to <512M so you're probably right. The other one's a SCSI disc but that one might actually being dying.
Back to top
View user's profile Send private message
neuron
Advocate
Advocate


Joined: 28 May 2002
Posts: 2371

PostPosted: Tue Aug 29, 2006 1:26 am    Post subject: Reply with quote

mrcs wrote:
neuron wrote:
it can be a lot of things, but all ccache does is stress the hardware, I very seriously doubt you'll find stability problems in ext3. (atleast without experimental patches) as it's being used on enterprise level all over the world.

Yeah, I'm suspecting that my slow laptop hard drive can't cope with that many seeks when the cache is to big. It does work flawlessly when the cache is set to <512M so you're probably right. The other one's a SCSI disc but that one might actually being dying.


you might wanna look into smartmontools and check dmesg.
Back to top
View user's profile Send private message
wrc1944
Advocate
Advocate


Joined: 15 Aug 2002
Posts: 3152
Location: Gainesville, Florida

PostPosted: Thu Sep 28, 2006 8:29 pm    Post subject: Reply with quote

adsmith wrote:
Quote:
By the way, for a while it was true that data journalling in EXT3 was somehow *faster* than just metadata journalling. See: http://www-128.ibm.com/developerworks/linux/library/l-fs8.html I'm not sure if this is still true.
Considering that the above article is from Dec 2001 and with 2.4 kernels, I've often wondered about this since I read page one of this thread, when I became an ext3 convert, although I went ahead and am still using data=journal to this day.

Seeing as how it was never entirely clear why this was the case (at least to me), and it simply goes against all logic (and the kernel fs docs), I'm changing all my ext3 partitions to data=writeback. I also ran Justin's LinuxGazette newer tests (with his script, and a 2.6.14 kernel) on my own system, comparing ext3 data=journal and data=writeback modes, and confirmed what logic predicts- writeback indeed produced better scores.

From the kernel Documentation/filesystems/ext3.txt file:
Quote:

Data Mode
---------
There are 3 different data modes:

* writeback mode
In data=writeback mode, ext3 does not journal data at all. This mode provides
a similar level of journaling as that of XFS, JFS, and ReiserFS in its default
mode - metadata journaling. A crash+recovery can cause incorrect data to
appear in files which were written shortly before the crash. This mode will
typically provide the best ext3 performance.

* ordered mode
In data=ordered mode, ext3 only officially journals metadata, but it logically
groups metadata and data blocks into a single unit called a transaction. When
it's time to write the new metadata out to disk, the associated data blocks
are written first. In general, this mode performs slightly slower than
writeback but significantly faster than journal mode.

* journal mode
data=journal mode provides full data and metadata journaling. All new data is
written to the journal first, and then to its final location.
In the event of a crash, the journal can be replayed, bringing both data and
metadata into a consistent state. This mode is the slowest except when data
needs to be read from and written to disk at the same time where it
outperforms all others modes.

_________________
Main box- AsRock x370 Gaming K4
Ryzen 1700, 3.0GHz, 16GB GSkill Flare DDR4 3200mhz
Samsung SATA 1000GB, Radeon HD R7 350 2GB DDR5
Gentoo ~amd64 plasma, glibc-2.26-r5, gcc-7.3.0 kernel-4.15.7-gentoo USE=experimental
Back to top
View user's profile Send private message
feld
Guru
Guru


Joined: 29 Aug 2004
Posts: 593
Location: WI, USA

PostPosted: Mon Oct 30, 2006 4:20 pm    Post subject: Reply with quote

but some of us are more worried about filesystem integrity than being a ricer.

it's not like full journaled mode annihilates our performance in any considerable way... :roll:
_________________
< bmg505> I think the first line in reiserfsck is

if (random(65535)< 65500) { hose(partition); for (i=0;i<100000000;i++) print_crap(); }
Back to top
View user's profile Send private message
XenoTerraCide
Veteran
Veteran


Joined: 18 Jan 2004
Posts: 1418
Location: MI, USA

PostPosted: Mon Oct 30, 2006 4:27 pm    Post subject: Reply with quote

yep integrity. writes are noted to be slower, but reads should be as fast normally and faster (than writeback) while writing. I don't know about others but I read data more than i write it. the only thing I have on reiserfs is portage because I don't need integrity and it handles small files better. the rest I need integrity on. /var might be a good candidate for writeback because logs are constantly being written. and aren't read as much.
_________________
I don't hang out here anymore, try asking on http://unix.stackexchange.com/ if you want my help.
Back to top
View user's profile Send private message
1bitmemory
n00b
n00b


Joined: 08 Jun 2004
Posts: 17

PostPosted: Sun Dec 17, 2006 5:45 pm    Post subject: Ext3 patch: mballoc Reply with quote

Has anyone here played with the ext3 patch set mentioned here ( http://lkml.org/lkml/2006/6/6/65 ) ? Some more explanations here http://lwn.net/Articles/80285/



1bm
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3 ... 11, 12, 13, 14, 15  Next
Page 12 of 15

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum