| View previous topic :: View next topic |
| Author |
Message |
Dark Foo l33t

Joined: 21 Nov 2008 Posts: 901 Location: 127.0.0.1
|
Posted: Sun Jan 24, 2010 3:28 pm Post subject: Is Ext4 a safe bet now? |
|
|
Or is it still an idea to use ext3 for a bit longer, i have managed to get some bits and pieces together and am about to install gentoo on it, so am working out what to use _________________ I know 43 ways to kill with a SKITTLE, so taste my rainbow bitch. |
|
| Back to top |
|
 |
mv Advocate


Joined: 20 Apr 2005 Posts: 2483
|
Posted: Sun Jan 24, 2010 4:27 pm Post subject: |
|
|
| If you plan to access to disks only with kernel-2.6.30 or newer: use ext4. |
|
| Back to top |
|
 |
gerard82 Veteran


Joined: 04 Jan 2004 Posts: 1855 Location: Netherlands
|
Posted: Sun Jan 24, 2010 4:41 pm Post subject: |
|
|
I've switched to ext4 ~1 year ago on my desktop.
Contrary to all the scary stories I haven' lost anything sofar.
Had a power outage abt a month ago,nothing lost.
And fsck is soooo fast!
I regularly make backups but did that on ext3 also.
Gerard. _________________ To install Gentoo I use sysrescuecd.Based on Gentoo,has Firefox to browse Gentoo docs and mc to browse (and edit) files.
The same disk can be used for 32 and 64 bit installs.
You can follow the Handbook verbatim.
http://www.sysresccd.org/Download |
|
| Back to top |
|
 |
Dark Foo l33t

Joined: 21 Nov 2008 Posts: 901 Location: 127.0.0.1
|
Posted: Sun Jan 24, 2010 4:52 pm Post subject: |
|
|
yes i will have back ups so i should be ok, thanks  _________________ I know 43 ways to kill with a SKITTLE, so taste my rainbow bitch. |
|
| Back to top |
|
 |
uaeb n00b

Joined: 28 May 2008 Posts: 27
|
|
| Back to top |
|
 |
d2_racing Moderator


Joined: 25 Apr 2005 Posts: 12849 Location: Ste-Foy,Canada
|
Posted: Sun Jan 24, 2010 9:00 pm Post subject: |
|
|
And for the migration thing, you can backup your partition with a Stage 4 or Stage 5, format it and then restore your tar.
It's clean and simple  _________________ Sysadmin of Funtoo-Québec.org
Wiki
Signature
IRC on Freenode : #funtoo-quebec |
|
| Back to top |
|
 |
cach0rr0 Moderator


Joined: 13 Nov 2008 Posts: 3592 Location: Houston, Republic of Texas
|
Posted: Sun Jan 24, 2010 10:18 pm Post subject: |
|
|
I've had no problems whatsoever with ext4 as my root fs, though, this is a laptop so I'm never going to experience the immediate power loss (it shuts itself down gracefully if battery is too low)
if you do go ext4, i would steer clear of 2.6.33 until we can make some sense of Phoronix' claims that ext4 has suffered a huge performance regression in 2.6.33
This isn't the fastest FS I've ever dealt with, but it's pretty damn stable, and "good enough" for my day to day use. _________________ Lost configuring your system?
dump lspci -n here | see Pappy's guide | Link Stash |
|
| Back to top |
|
 |
CaptainHero n00b


Joined: 27 Nov 2009 Posts: 41 Location: Germany
|
Posted: Sun Jan 24, 2010 10:19 pm Post subject: |
|
|
The only issue I had with an ext4 root partition was that like every 2 or 3 boots fsck told me there was data inconsistency and it had to be repaired.
The reason probably was that it wasn't properly unmounted when shutting my PC down. I got the hint to mount it with the option "journal_checksum", since then fsck never spoke to me again. |
|
| Back to top |
|
 |
kernelOfTruth Watchman


Joined: 20 Dec 2005 Posts: 5314 Location: Vienna, Austria; Germany; hello world :)
|
|
| Back to top |
|
 |
Shining Arcanine Veteran

Joined: 24 Sep 2009 Posts: 1110
|
Posted: Mon Jan 25, 2010 12:36 am Post subject: |
|
|
I have been running ext4 on my laptop and in virtual machines and I have not had a single issue with data corruption on it. It is as stable as a rock.
However, there is one minor issue. Ext4 performance has been dropping in newer kernel versions as the kernel developers have been prioritizing data integrity above performance:
http://www.phoronix.com/scan.php?page=article&item=linux_perf_regressions&num=1
Supposedly, this can be fixed by mount with nobarrier, as the commit in 2.6.32 that caused this issue turned barrier=1 on by default. However, it is my understanding that additional performance losses are in 2.6.33 and it is not known to me whether or not the commits that caused these can be disabled with the nobarrier mount flag:
http://www.phoronix.com/scan.php?page=article&item=ext4_then_now&num=1
From what I have read, the commit in 2.6.33, affects ext3 as well as ext4 because it makes barrier=1 the default mount option for both file systems, however, since the commit, ext3 performance has been higher than ext4 performance (with default mount options):
http://www.phoronix.com/scan.php?page=article&item=linux_2632_fs&num=1
nobarrier is not a recommended mount option unless either your system is battery-backed up or your drive has a way of flushing its cache in the event of a power failure. I have heard of some SSDs have capacitors that enable them to do this. Off the top of my head, the issue with nobarrier is that it allows changes to be made to a disk before they are logged to the journal so in the event of a power outage, the journal will not accurately reflect what has been written to the disk, which can cause data integrity issues, although someone might have to correct me on that, as I do not remember where I read that to be able to verify my memory and cite my source.
I am not trying to scare you away from ext4, but these are some things to keep in mind. Perhaps some other people here could add more information, such as whether or not the commits in 2.6.33 that negative affect ext4 performance are disabled by the nobarrier mount option, which is something I am eager to learn myself. |
|
| Back to top |
|
 |
kernelOfTruth Watchman


Joined: 20 Dec 2005 Posts: 5314 Location: Vienna, Austria; Germany; hello world :)
|
Posted: Mon Jan 25, 2010 1:30 am Post subject: |
|
|
| Shining Arcanine wrote: |
nobarrier is not a recommended mount option unless either your system is battery-backed up or your drive has a way of flushing its cache in the event of a power failure. I have heard of some SSDs have capacitors that enable them to do this. Off the top of my head, the issue with nobarrier is that it allows changes to be made to a disk before they are logged to the journal so in the event of a power outage, the journal will not accurately reflect what has been written to the disk, which can cause data integrity issues, although someone might have to correct me on that, as I do not remember where I read that to be able to verify my memory and cite my source.
I am not trying to scare you away from ext4, but these are some things to keep in mind. Perhaps some other people here could add more information, such as whether or not the commits in 2.6.33 that negative affect ext4 performance are disabled by the nobarrier mount option, which is something I am eager to learn myself. |
IMO some rather safe way of using nobarrier would be to mount it with
| Code: | | nobarrier,commit=5,data=journal |
that way it would be pretty similar to ext3 when not using with extents and some other ext4-specific stuff ...
I'd however highly recommend using barrier for data-safety reasons
keep in mind that barrier-support will degrade read-performance also because of strict ordering of write and read transactions _________________ Unofficial minimal livecd x86/amd64 w/reiser4+truecrypt (by Neo2)
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004  |
|
| Back to top |
|
 |
mv Advocate


Joined: 20 Apr 2005 Posts: 2483
|
Posted: Mon Jan 25, 2010 1:21 pm Post subject: |
|
|
| kernelOfTruth wrote: | IMO some rather safe way of using nobarrier would be to mount it with
| Code: | | nobarrier,commit=5,data=journal |
that way it would be pretty similar to ext3 |
Yes, similarly stupid as that ext3 default: Without barriers, journaling has practically lost its meaning (unless you disable all disk-internal cache in which case it makes no difference whether you use barrier or not). If really speed is more important to you than data safety, i.e. if you rely on non-crashing of system and non-power-loss anyway, you should switch off the journal as well, thus effectively running ext2 only (with some ext4 features like extents or 256 bit inode size).
Edit: I really cannot see a reason why anybody would care more about speed than data safety. Even if you have a multimedia partition in which you need low safety because you do not write often, it makes no sense: If you do not write often then barrier/journal does not matter concerning speed. |
|
| Back to top |
|
 |
kernelOfTruth Watchman


Joined: 20 Dec 2005 Posts: 5314 Location: Vienna, Austria; Germany; hello world :)
|
Posted: Mon Jan 25, 2010 2:54 pm Post subject: |
|
|
| mv wrote: |
Edit: I really cannot see a reason why anybody would care more about speed than data safety. Even if you have a multimedia partition in which you need low safety because you do not write often, it makes no sense: If you do not write often then barrier/journal does not matter concerning speed. |
you know - there ARE these kind of people who care only about speed and neglect the integrity of their data who after a data loss come back crying and shouting "why me ?" or pointing the finger at you why you haven't warned them
it's more of a learning process - in the end most or all users want to use barrier-support or integrity-ensuring settings ... _________________ Unofficial minimal livecd x86/amd64 w/reiser4+truecrypt (by Neo2)
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004  |
|
| Back to top |
|
 |
Dark Foo l33t

Joined: 21 Nov 2008 Posts: 901 Location: 127.0.0.1
|
Posted: Mon Jan 25, 2010 3:30 pm Post subject: |
|
|
well laptop would be first to have the make over, then the little box i managed to get together both only really play films or me surfing net and looking at email, films are backed up on disc (blu-ray takes a lot of space - mkv doesn't ) and email is imap, so i should lose anything i cant replace.
As to the kernel what is the default one that ships atm, i can always pull an older one from portage if i need to _________________ I know 43 ways to kill with a SKITTLE, so taste my rainbow bitch. |
|
| Back to top |
|
 |
mv Advocate


Joined: 20 Apr 2005 Posts: 2483
|
Posted: Mon Jan 25, 2010 4:09 pm Post subject: |
|
|
| Carps wrote: | | well laptop would be first to have the make over, then the little box i managed to get together both only really play films or me surfing net and looking at email |
So it is as in my example: No immense amounts of writes are expected (some exceptional situations like first installation do hardly count); there is no reason to omit the security provided by barriers. Perhaps some excessive benchmarks do not give optimal results then, but you will not see the difference on your system. Even when a reinstall after a filesystem crash should be fast, it is still some work which you should not risk for no reason: Go with the safe options (barriers=1). |
|
| Back to top |
|
 |
Dark Foo l33t

Joined: 21 Nov 2008 Posts: 901 Location: 127.0.0.1
|
Posted: Mon Jan 25, 2010 7:37 pm Post subject: |
|
|
so basically, do it as i always have, no special mount options required? Main system (this one might be different) that has a bit of a heavier disk usage, will cross that bridge when i come to it, speed it not that important to me, making sure my data is there is
edit: dual booting win7 and linux bites now as i have to use extended partitions ;-/ _________________ I know 43 ways to kill with a SKITTLE, so taste my rainbow bitch. |
|
| Back to top |
|
 |
Cyker Veteran

Joined: 15 Jun 2006 Posts: 1427
|
Posted: Mon Jan 25, 2010 11:02 pm Post subject: |
|
|
I'm still using ext4 in backward-compatibility mode so that I can use the ext3 or even ext2 filesystem driver to access the partitions if I need to.
The main one is to just mount it with the noextents option (Although be careful: This switch is broken on newer kernels; They assume that because the default is noextents the switch is useless which is dangerous and shortsighted. I'm now slightly terrified that if I ever have to mount an older kernel I will hose my main filesystem because older kernels NEED that switch otherwise they can, under certain circumstances, convert your FS without even asking you!)
The barriers thing is just awful; There must be a better way! It reduces performance to PIO4 levels on my system, and doesn't even work on my mdadm array anyway!
I'm all for data integrity (I use full journalling, and, IMHO, the killer feature of ext4 - Journal checksumming!) but not when it renders the filesystem unfit for purpose (Seriously... I could run stuff off a tape faster! And barriers completely subverts things like NCQ, delayed allocation, and all those other optimizing tricks to make fileystem access faster!) |
|
| Back to top |
|
 |
mv Advocate


Joined: 20 Apr 2005 Posts: 2483
|
Posted: Mon Jan 25, 2010 11:22 pm Post subject: |
|
|
| Cyker wrote: | | The barriers thing is just awful; There must be a better way! It reduces performance to PIO4 levels on my system, and doesn't even work on my mdadm array anyway! | That some hardware (and even lvm) does not properly respect it, is really bad. I hope they will at least fix it in the kernel.
| Quote: | | I'm all for data integrity (I use full journalling, and, IMHO, the killer feature of ext4 - Journal checksumming!) |
Without barrier, this is pure nonsense. It does not make your data more secure, maybe even quite the opposite than not using it. What does a journal help you if it happens to be written after the actual data is already stored and then might contain out-of-date things on the next boot in case of a crash?
| Quote: | | And barriers completely subverts things like NCQ, delayed allocation, and all those other optimizing tricks to make fileystem access faster!) |
Also this is pure nonsense. Delayed allocation is rather useless without extents. You intentionally switch off the main speed improvement of ext4 and then complain about a few percent you lose through barriers which are the most necessary thing for data integrity.  |
|
| Back to top |
|
 |
kernelOfTruth Watchman


Joined: 20 Dec 2005 Posts: 5314 Location: Vienna, Austria; Germany; hello world :)
|
Posted: Mon Jan 25, 2010 11:31 pm Post subject: |
|
|
| mv wrote: | | Cyker wrote: | | The barriers thing is just awful; There must be a better way! It reduces performance to PIO4 levels on my system, and doesn't even work on my mdadm array anyway! | That some hardware (and even lvm) does not properly respect it, is really bad. I hope they will at least fix it in the kernel.
|
this is a long known problem which seems obviously hard to fix:
some filesystems like reiser4 and btrfs (?) switch to sync-mode in that case in order to ensure data integrity/safety
(afaik - if it didn't got fixed already) barriers also don't work on partitions which are encrypted with LUKS / cryptsetup and using LVM2 _________________ Unofficial minimal livecd x86/amd64 w/reiser4+truecrypt (by Neo2)
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004  |
|
| Back to top |
|
 |
mv Advocate


Joined: 20 Apr 2005 Posts: 2483
|
Posted: Mon Jan 25, 2010 11:33 pm Post subject: |
|
|
| Carps wrote: | | so basically, do it as i always have, no special mount options required? :-) Main system (this one might be different) that has a bit of a heavier disk usage, will cross that bridge when i come to it, speed it not that important to me, making sure my data is there is :-) |
I am using noatime,data=ordered,journal_checksum,barrier=1 explicitly in my /etc/fstab: It is too insecure for me to remember in which kernel version which is the default, and who knows what will be the default in 2.6.34 or 2.6.35?
When formatting you might consider removing the huge_file option from /etc/mke2fs.conf unless you intend to put files in TB size there: Otherwise, not every x86 kernel will be able to mount it: It requires a special kernel option activated. (Since this option exists, I suppose it also has a mild speed/memory impact). |
|
| Back to top |
|
 |
mv Advocate


Joined: 20 Apr 2005 Posts: 2483
|
Posted: Mon Jan 25, 2010 11:35 pm Post subject: |
|
|
| kernelOfTruth wrote: | | (afaik - if it didn't got fixed already) barriers also don't work on partitions which are encrypted with LUKS / cryptsetup and using LVM2 |
Yes of course, because this is just a variant of lvm. I don't know whether there are current plans to fix this, but I hope it will be fixed some day... |
|
| Back to top |
|
 |
Cyker Veteran

Joined: 15 Jun 2006 Posts: 1427
|
Posted: Tue Jan 26, 2010 12:41 am Post subject: |
|
|
| mv wrote: | | Quote: | | And barriers completely subverts things like NCQ, delayed allocation, and all those other optimizing tricks to make fileystem access faster!) |
Also this is pure nonsense. Delayed allocation is rather useless without extents. You intentionally switch off the main speed improvement of ext4 and then complain about a few percent you lose through barriers which are the most necessary thing for data integrity.  |
If it was only a few percent I wouldn't mind, but when I tried it I found the penalty was more in the order of 30-50% (!!), and there's enough corollary floating around that I know it's not some quirk on my system.
From what I understand of the problem tho', with full journalling and journal checksum enabled, a lot of the potential for corruption by crashes during inopportune out-of-order writes is mitigated; There is still a danger, but it's low enough that I can be more worried about what happens with the mdadm partition and its RAID5 write hole problem
The problem with barriers for me is that they don't seem to be optimized at all and get triggered far too often (At least with the access patterns I'm usually dealing with). Performance is on par with unbuffered blocking IO! (Or, like using DOS without smartdrv).
If they batched the journal writes together it might work better, but that in turn would require booting out cached data for what is essentially a one-off write...
I wonder how NTFS goes about it, unless that also overlooks the atomic write for the journal data...?
I'm tempted to test barrier again, but I've not seen anything to indicate it has been sped up in some way... (And the point is largely moot since the most important partition on my system is now an unsupported mdadm RAID5 anyway!) |
|
| Back to top |
|
 |
Dark Foo l33t

Joined: 21 Nov 2008 Posts: 901 Location: 127.0.0.1
|
Posted: Tue Jan 26, 2010 7:55 am Post subject: |
|
|
| mv wrote: | | Carps wrote: | so basically, do it as i always have, no special mount options required? Main system (this one might be different) that has a bit of a heavier disk usage, will cross that bridge when i come to it, speed it not that important to me, making sure my data is there is  |
I am using noatime,data=ordered,journal_checksum,barrier=1 explicitly in my /etc/fstab: It is too insecure for me to remember in which kernel version which is the default, and who knows what will be the default in 2.6.34 or 2.6.35?
When formatting you might consider removing the huge_file option from /etc/mke2fs.conf unless you intend to put files in TB size there: Otherwise, not every x86 kernel will be able to mount it: It requires a special kernel option activated. (Since this option exists, I suppose it also has a mild speed/memory impact). |
Thanks
Yes i removed large files last time, the lagest files that might go on there will be blu-ray discs i own, and if you strip all the other lanuages out and stuff most are 25gb max.
That mount line above do you use that for all ext4 partitions? and would you advise ext4 for all partitions? normally i use ext2 for boot, and ext3 for the rest
edit: if i use gentoo-sources-2.6.31-r6 do i still need the nobarrier option _________________ I know 43 ways to kill with a SKITTLE, so taste my rainbow bitch. |
|
| Back to top |
|
 |
Shining Arcanine Veteran

Joined: 24 Sep 2009 Posts: 1110
|
Posted: Tue Jan 26, 2010 2:27 pm Post subject: |
|
|
| mv wrote: | | kernelOfTruth wrote: | IMO some rather safe way of using nobarrier would be to mount it with
| Code: | | nobarrier,commit=5,data=journal |
that way it would be pretty similar to ext3 |
Yes, similarly stupid as that ext3 default: Without barriers, journaling has practically lost its meaning (unless you disable all disk-internal cache in which case it makes no difference whether you use barrier or not). If really speed is more important to you than data safety, i.e. if you rely on non-crashing of system and non-power-loss anyway, you should switch off the journal as well, thus effectively running ext2 only (with some ext4 features like extents or 256 bit inode size).
Edit: I really cannot see a reason why anybody would care more about speed than data safety. Even if you have a multimedia partition in which you need low safety because you do not write often, it makes no sense: If you do not write often then barrier/journal does not matter concerning speed. |
You could have a SSD that has a built-in capacitor to allow the drive to flush its cache in the event of a power outage, in which case the journal is pointless and harmful to both your SSD's performance and its life span. |
|
| Back to top |
|
 |
mv Advocate


Joined: 20 Apr 2005 Posts: 2483
|
Posted: Tue Jan 26, 2010 5:35 pm Post subject: |
|
|
| Cyker wrote: | | If it was only a few percent I wouldn't mind, but when I tried it I found the penalty was more in the order of 30-50% (!!) |
Of course, because only due to barriers your journaling option actually takes effect. Probably without barriers, your "journal" was just modified in the disk cache and never written to disk, because it was changed immediately afterwards. As I told you: Journaling without barriers is just useless or even worse.
| Quote: | | ... with full journalling and journal checksum enabled, a lot of the potential for corruption by crashes during inopportune out-of-order writes is mitigated |
Not if the journal is not written or not at the correct time due to missing barriers. |
|
| Back to top |
|
 |
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|