| View previous topic :: View next topic |
| Author |
Message |
queen
Veteran
Joined: 19 Jul 2005
Posts: 1642
|
 Posted: Tue Sep 11, 2012 6:44 am Post subject: which filesystem to use? |
 |
|
| I want to install gentoo on a 1TB hd. I want to use the 3 classical partition. 100mb for boot, swap and the rest for /. What file system is recommended for such a large file system based on your experience? Currently I am considering ext3, ext4, jfs, xfs. |
|
| Back to top |
|
 |
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9677
Location: almost Mile High in the USA
|
| Posted: Tue Sep 11, 2012 6:54 am Post subject: |
|
|
This depends on what kinds of files are you going to put on it?
I recently just used ext3 for a 2TB disk for mythtv. Seems to work fine.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
yngwin
Retired Dev
Joined: 19 Dec 2002
Posts: 4572
Location: Suzhou, China
|
| Posted: Tue Sep 11, 2012 6:56 am Post subject: |
|
|
Personally I use and recommend JFS:
| Wikipedia wrote: | | According to reviews and benchmarks of the available filesystems for Linux, JFS is fast and reliable, with consistently good performance under different kinds of load, contrary to other filesystems that seem to perform better under particular usage patterns, for instance with small or large files. Another characteristic often mentioned, is that it's light and efficient with available system resources and even heavy disk activity is realized with low CPU usage. |
Others will recommend ext4, which is the default choice for most Linux distros.
XFS is known to be especially good with large files (think multimedia), but can be flaky w.r.t. data-loss on unclean shutdown, so should only be used with a UPS.
_________________
"Those who deny freedom to others deserve it not for themselves." - Abraham Lincoln
Free Culture | Defective by Design | EFF |
|
| Back to top |
|
|
queen
Veteran
Joined: 19 Jul 2005
Posts: 1642
|
| Posted: Tue Sep 11, 2012 7:38 am Post subject: |
|
|
| yngwin wrote: | Personally I use and recommend JFS:
| Wikipedia wrote: | | According to reviews and benchmarks of the available filesystems for Linux, JFS is fast and reliable, with consistently good performance under different kinds of load, contrary to other filesystems that seem to perform better under particular usage patterns, for instance with small or large files. Another characteristic often mentioned, is that it's light and efficient with available system resources and even heavy disk activity is realized with low CPU usage. |
Others will recommend ext4, which is the default choice for most Linux distros.
XFS is known to be especially good with large files (think multimedia), but can be flaky w.r.t. data-loss on unclean shutdown, so should only be used with a UPS. |
Thanks. So XFS is out for me, due to data loss on unclean shutdown. I can't risk data loss. Looks like JFS will be a good choice. I will have some multimedia files and other files around 200-500mb each. |
|
| Back to top |
|
|
tld
Veteran
Joined: 09 Dec 2003
Posts: 1816
|
| Posted: Tue Sep 11, 2012 1:43 pm Post subject: |
|
|
| eccerr0r wrote: | This depends on what kinds of files are you going to put on it?
I recently just used ext3 for a 2TB disk for mythtv. Seems to work fine. |
Just a note on this: The only downside of using ext3 for MythTV storage is that you have to enable the "delete files slowly" option in MythTV. That causes it to slowly truncate files rather than deleting them outright. This is because ext3 takes a LONG time to delete huge files and locks up the file system long enough to cause glitches in any recording that might be going on.
I moved to ext4 for MythTV a long time ago and it's been great. Huge files delete instantly. Note however that when you convert ext3 to ext4, existing files will not delete quickly...just newly created ones.
Tom |
|
| Back to top |
|
|
Hypnos
Advocate
Joined: 18 Jul 2002
Posts: 2889
Location: Omnipresent
|
| Posted: Tue Sep 11, 2012 3:24 pm Post subject: |
|
|
Another vote for ext4.
I have a number crunching machine with a 1TB ext4 partition. I routinely create and delete 200GB files without difficulty.
_________________
Personal overlay | Simple backup scheme |
|
| Back to top |
|
|
fhaddad78
n00b
Joined: 21 May 2012
Posts: 68
|
| Posted: Tue Sep 11, 2012 3:47 pm Post subject: |
|
|
For general desktop usage, I would argue that you will most likely not notice any differences between your file system partition choices. Many of the performance gains or losses, from the various file systems, are experienced in heavier workloads centered around servers. Now, if this is your situation, you should probably read up on the strengths as well as the shortcomings of the various file systems available.
With that said you may want to use ext2 for the boot partition (because it doesn't bring the added overhead of ext3/4) and then either JFS (which I also use and recommend for the same reasons mentioned in queen's post) or ext3/4. All will support your hard drives capacity without issue. The other choices, such as ReiserFS, I don't think have the heavy usage and support you will get from JFS (which was developed by IBM) and ext2/3/4. |
|
| Back to top |
|
|
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9677
Location: almost Mile High in the USA
|
| Posted: Tue Sep 11, 2012 6:34 pm Post subject: |
|
|
| tld wrote: | | eccerr0r wrote: | This depends on what kinds of files are you going to put on it?
I recently just used ext3 for a 2TB disk for mythtv. Seems to work fine. |
Just a note on this: The only downside of using ext3 for MythTV storage is that you have to enable the "delete files slowly" option in MythTV. That causes it to slowly truncate files rather than deleting them outright. This is because ext3 takes a LONG time to delete huge files and locks up the file system long enough to cause glitches in any recording that might be going on.
I moved to ext4 for MythTV a long time ago and it's been great. Huge files delete instantly. Note however that when you convert ext3 to ext4, existing files will not delete quickly...just newly created ones.
Tom |
Hmm.... Looks like this is default, I've noticed files deleting slowly on the machine and didn't notice too many hiccups... In any case, perhaps I should migrate to ext4 anyway, I just migrated my laptop ssd to ext4 after mistakingly used ext3 (since the kernel I had to bootstrap with didn't have ext4 in it).
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
queen
Veteran
Joined: 19 Jul 2005
Posts: 1642
|
| Posted: Wed Sep 12, 2012 12:02 pm Post subject: |
|
|
| Hypnos wrote: | Another vote for ext4.
I have a number crunching machine with a 1TB ext4 partition. I routinely create and delete 200GB files without difficulty. |
It will be a crunching numbers machine mainly + some multimedia (less). And performance in this case is important.
If you know some good links to compare between ext4 and jfs, I will appreciate. |
|
| Back to top |
|
|
Hypnos
Advocate
Joined: 18 Jul 2002
Posts: 2889
Location: Omnipresent
|
| Posted: Wed Sep 12, 2012 1:11 pm Post subject: |
|
|
The Arch wiki page on JFS is a good place to start.
Google has some interesting things to say about JFS --
* It only journals metadata (link), so I would only use it with a UPS and backups. (Same with XFS.)
* Moreover, it is no longer being actively maintained by IBM and has unfixed bugs (mailing list thread). (XFS wins here, as it has many active developers.)
Ext4 can journal both metadata and data (if enabled), so it's safer, but slower. Obviously it has the largest developer and user community of any Linux filesystem, so it's the best on that count.
Ext4 with hashed tree support ("dir_index" option) and extents gives you some of the performance niceties of JFS/XFS for big directories and large files, respectively.
I would use Ext4 unless you have a particular reason to use something different.
_________________
Personal overlay | Simple backup scheme |
|
| Back to top |
|
|
queen
Veteran
Joined: 19 Jul 2005
Posts: 1642
|
| Posted: Wed Sep 12, 2012 10:02 pm Post subject: |
|
|
Good links. It looks like I will choose ext4 based on the reasons mentioned here and some link I found via google. There it shows some benchmarks they did with various file systems. http://farbfinal.wordpress.com/2010/05/10/file-system-performance-benchmark-ext4-jfs-xfs-reiserfs/ ext4 was the best.
| Hypnos wrote: |
Ext4 can journal both metadata and data (if enabled), so it's safer, but slower. Obviously it has the largest developer and user community of any Linux filesystem, so it's the best on that count.
Ext4 with hashed tree support ("dir_index" option) and extents gives you some of the performance niceties of JFS/XFS for big directories and large files, respectively.
I would use Ext4 unless you have a particular reason to use something different. |
Can you please write how you enable data too? Or better, if you recall the whole command when you created the file system, it will be nice. /etc/fstab will be nice to have too. |
|
| Back to top |
|
|
Hypnos
Advocate
Joined: 18 Jul 2002
Posts: 2889
Location: Omnipresent
|
| Posted: Wed Sep 12, 2012 11:35 pm Post subject: |
|
|
Here is my fstab line:
| Code: | | /dev/sda1 / ext4 noatime,nodiratime 0 1 |
Data journaling is on by default; you would turn it off by adding the "data=writeback" mount option.
I don't recall the command I used to make the fs, but according to the options I have set as reported by tune2fs, the following should work:
| Code: | | # mke2fs -L <label> -t ext4 -O dir_index,extent,has_journal,large_file,spare_super,uninit_bg <device> |
_________________
Personal overlay | Simple backup scheme |
|
| Back to top |
|
|
dmpogo
Advocate
Joined: 02 Sep 2004
Posts: 3267
Location: Canada
|
| Posted: Thu Sep 13, 2012 1:14 am Post subject: |
|
|
| And to add to the mix - reiserfs3 worked flawlessly for me on servers and desktops that ran for 5-8 years. |
|
| Back to top |
|
|
Hypnos
Advocate
Joined: 18 Jul 2002
Posts: 2889
Location: Omnipresent
|
| Posted: Thu Sep 13, 2012 1:28 am Post subject: |
|
|
| dmpogo wrote: | | And to add to the mix - reiserfs3 worked flawlessly for me on servers and desktops that ran for 5-8 years. |
What kind of developer backing does ReiserFS currently have? BTW, it also only does metadata journaling.
_________________
Personal overlay | Simple backup scheme |
|
| Back to top |
|
|
dmpogo
Advocate
Joined: 02 Sep 2004
Posts: 3267
Location: Canada
|
| Posted: Thu Sep 13, 2012 4:13 am Post subject: |
|
|
| Hypnos wrote: | | dmpogo wrote: | | And to add to the mix - reiserfs3 worked flawlessly for me on servers and desktops that ran for 5-8 years. |
What kind of developer backing does ReiserFS currently have? BTW, it also only does metadata journaling. |
It was just a data point - I never lost data on 5 machines with reiserfs3 running over I'd say 6 years each on average, mostly on 24/7 - that is 30 computer years. Perhaps my luck,
And the current develpment ? As long as it is in the kernel and works, I am happy, are there any bugs to be worried about ?
It is not that I use reiserfs exclusively - I use xfs on scratch disks with big files, ext2 on some root partitions on SSD's and jfs on one machine.
The only one which stutters on portage update, BTW, so I plan to replace jfs to reiser3 (or xfs) on partition that carries portage.
Somehow, I never used ext4, so have no track record here. |
|
| Back to top |
|
|
Hypnos
Advocate
Joined: 18 Jul 2002
Posts: 2889
Location: Omnipresent
|
| Posted: Thu Sep 13, 2012 4:23 am Post subject: |
|
|
I think it's useful for a filesystem to have a large, active developer community in case of bugs and to prevent regressions on kernel upgrades. I don't know the current situation with ReiserFS, which is why i asked.
_________________
Personal overlay | Simple backup scheme |
|
| Back to top |
|
|
dmpogo
Advocate
Joined: 02 Sep 2004
Posts: 3267
Location: Canada
|
| Posted: Thu Sep 13, 2012 4:30 am Post subject: |
|
|
| Hypnos wrote: | | I think it's useful for a filesystem to have a large, active developer community in case of bugs and to prevent regressions on kernel upgrades. I don't know the current situation with ReiserFS, which is why i asked. |
I don't think there is any active work on reiserfs3 for quite a few years already but it is part of the standard kernel, so
unless it is dropped it should remain compartible with newer kernels. reiserfs4 development, on the other hand, has just been reactivated few days ago at
http://sourceforge.net/projects/reiser4/files/reiser4-for-linux-3.x/ |
|
| Back to top |
|
|
The Doctor
Moderator
Joined: 27 Jul 2010
Posts: 2678
|
| Posted: Thu Sep 13, 2012 4:34 am Post subject: |
|
|
| Hypnos wrote: | | I think it's useful for a filesystem to have a large, active developer community in case of bugs and to prevent regressions on kernel upgrades. I don't know the current situation with ReiserFS, which is why i asked. |
From the Arch wiki: | Quote: | | ReiserFS (V3) Hans Reiser's high-performance journaling FS uses a very interesting method of data throughput based on an unconventional and creative algorithm. ReiserFS is touted as very fast, especially when dealing with many small files. ReiserFS is fast at formatting, yet comparatively slow at mounting. Quite mature and stable. ReiserFS (V3) is not being actively developed at this time. Generally regarded as a good choice for /var. |
It does not look like prison is a good place to develop a filesystem from.
_________________
First things first, but not necessarily in that order.
Apologies if I take a while to respond. I'm currently working on the dematerialization circuit for my blue box. |
|
| Back to top |
|
|
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9677
Location: almost Mile High in the USA
|
| Posted: Thu Sep 13, 2012 5:42 am Post subject: |
|
|
I was using reiser3 for rootfs for some machines and was generally pleased with it, but with Hans behind bars and having disks large enough to not worry about wasting space (that tail packing saves), I've started to migrate back to the ext2/ext3/ext4 line of filesystems. Mainly because I hate forgetting to compile in filesystem modules :) Ext2 is pretty small and the smallest machines, to save RAM, try to use ext2.
Reiser3 saves a *lot* of disk space for the portage tree. It and portage seem like a match made in heaven (that, and if there's an innately compressing file or other tail packing system on the horizon.)
I still do have xfs as an alternate preferred filesystem (I have a RAID5 LVM that uses xfs) and it seems to work OK but may end up using ext4 if I have to redo it...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
dmpogo
Advocate
Joined: 02 Sep 2004
Posts: 3267
Location: Canada
|
| Posted: Thu Sep 13, 2012 5:58 am Post subject: |
|
|
| eccerr0r wrote: | I was using reiser3 for rootfs for some machines and was generally pleased with it, but with Hans behind bars and having disks large enough to not worry about wasting space (that tail packing saves), I've started to migrate back to the ext2/ext3/ext4 line of filesystems. Mainly because I hate forgetting to compile in filesystem modules  Ext2 is pretty small and the smallest machines, to save RAM, try to use ext2.
Reiser3 saves a *lot* of disk space for the portage tree. It and portage seem like a match made in heaven (that, and if there's an innately compressing file or other tail packing system on the horizon.)
I still do have xfs as an alternate preferred filesystem (I have a RAID5 LVM that uses xfs) and it seems to work OK but may end up using ext4 if I have to redo it... |
Yes, it is /var where I also put my protage tree on that I want to covert from jfs back to reiser3. But in order not to have 4 different filesystem concurrently (ext2 on root on SSD, xfs on big data disks, jfs on /home and /var - wanted to try it) I may end up converting the final jfs partition on the machine - /home to reiser or xfs as well |
|
| Back to top |
|
|
khayyam
Watchman
Joined: 07 Jun 2012
Posts: 6227
Location: Room 101
|
| Posted: Fri Sep 14, 2012 12:00 pm Post subject: |
|
|
| Hypnos wrote: | | Code: | | /dev/sda1 / ext4 noatime,nodiratime 0 1 |
|
Hypnos ... I was under the impression that nodiratime was a subset of noatime, and that the latter implies the former.
best ... khay |
|
| Back to top |
|
|
Hypnos
Advocate
Joined: 18 Jul 2002
Posts: 2889
Location: Omnipresent
|
| Posted: Fri Sep 14, 2012 12:18 pm Post subject: |
|
|
| khayyam wrote: |
Hypnos ... I was under the impression that nodiratime was a subset of noatime, and that the latter implies the former. |
It seems that you are correct
_________________
Personal overlay | Simple backup scheme |
|
| Back to top |
|
|
|
Installing Gentoo
