Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
which filesystem to use?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
queen
Veteran
Veteran


Joined: 19 Jul 2005
Posts: 1512

PostPosted: Tue Sep 11, 2012 6:44 am    Post subject: which filesystem to use? Reply with quote

I want to install gentoo on a 1TB hd. I want to use the 3 classical partition. 100mb for boot, swap and the rest for /. What file system is recommended for such a large file system based on your experience? Currently I am considering ext3, ext4, jfs, xfs.
Back to top
View user's profile Send private message
eccerr0r
Advocate
Advocate


Joined: 01 Jul 2004
Posts: 3598
Location: USA

PostPosted: Tue Sep 11, 2012 6:54 am    Post subject: Reply with quote

This depends on what kinds of files are you going to put on it?

I recently just used ext3 for a 2TB disk for mythtv. Seems to work fine.
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed to be advocating?
Back to top
View user's profile Send private message
yngwin
Developer
Developer


Joined: 19 Dec 2002
Posts: 4394
Location: Suzhou, China

PostPosted: Tue Sep 11, 2012 6:56 am    Post subject: Reply with quote

Personally I use and recommend JFS:

Wikipedia wrote:
According to reviews and benchmarks of the available filesystems for Linux, JFS is fast and reliable, with consistently good performance under different kinds of load, contrary to other filesystems that seem to perform better under particular usage patterns, for instance with small or large files. Another characteristic often mentioned, is that it's light and efficient with available system resources and even heavy disk activity is realized with low CPU usage.


Others will recommend ext4, which is the default choice for most Linux distros.

XFS is known to be especially good with large files (think multimedia), but can be flaky w.r.t. data-loss on unclean shutdown, so should only be used with a UPS.
_________________
"Those who deny freedom to others deserve it not for themselves." - Abraham Lincoln
Free Culture | Defective by Design | EFF
Back to top
View user's profile Send private message
queen
Veteran
Veteran


Joined: 19 Jul 2005
Posts: 1512

PostPosted: Tue Sep 11, 2012 7:38 am    Post subject: Reply with quote

yngwin wrote:
Personally I use and recommend JFS:

Wikipedia wrote:
According to reviews and benchmarks of the available filesystems for Linux, JFS is fast and reliable, with consistently good performance under different kinds of load, contrary to other filesystems that seem to perform better under particular usage patterns, for instance with small or large files. Another characteristic often mentioned, is that it's light and efficient with available system resources and even heavy disk activity is realized with low CPU usage.


Others will recommend ext4, which is the default choice for most Linux distros.

XFS is known to be especially good with large files (think multimedia), but can be flaky w.r.t. data-loss on unclean shutdown, so should only be used with a UPS.


Thanks. So XFS is out for me, due to data loss on unclean shutdown. I can't risk data loss. Looks like JFS will be a good choice. I will have some multimedia files and other files around 200-500mb each.
Back to top
View user's profile Send private message
tld
l33t
l33t


Joined: 09 Dec 2003
Posts: 773

PostPosted: Tue Sep 11, 2012 1:43 pm    Post subject: Reply with quote

eccerr0r wrote:
This depends on what kinds of files are you going to put on it?

I recently just used ext3 for a 2TB disk for mythtv. Seems to work fine.


Just a note on this: The only downside of using ext3 for MythTV storage is that you have to enable the "delete files slowly" option in MythTV. That causes it to slowly truncate files rather than deleting them outright. This is because ext3 takes a LONG time to delete huge files and locks up the file system long enough to cause glitches in any recording that might be going on.

I moved to ext4 for MythTV a long time ago and it's been great. Huge files delete instantly. Note however that when you convert ext3 to ext4, existing files will not delete quickly...just newly created ones.

Tom
Back to top
View user's profile Send private message
Hypnos
Advocate
Advocate


Joined: 18 Jul 2002
Posts: 2859
Location: Omnipresent

PostPosted: Tue Sep 11, 2012 3:24 pm    Post subject: Reply with quote

Another vote for ext4.

I have a number crunching machine with a 1TB ext4 partition. I routinely create and delete 200GB files without difficulty.
_________________
Personal overlay | Simple backup scheme
Back to top
View user's profile Send private message
fhaddad78
n00b
n00b


Joined: 21 May 2012
Posts: 68

PostPosted: Tue Sep 11, 2012 3:47 pm    Post subject: Reply with quote

For general desktop usage, I would argue that you will most likely not notice any differences between your file system partition choices. Many of the performance gains or losses, from the various file systems, are experienced in heavier workloads centered around servers. Now, if this is your situation, you should probably read up on the strengths as well as the shortcomings of the various file systems available.

With that said you may want to use ext2 for the boot partition (because it doesn't bring the added overhead of ext3/4) and then either JFS (which I also use and recommend for the same reasons mentioned in queen's post) or ext3/4. All will support your hard drives capacity without issue. The other choices, such as ReiserFS, I don't think have the heavy usage and support you will get from JFS (which was developed by IBM) and ext2/3/4.
Back to top
View user's profile Send private message
eccerr0r
Advocate
Advocate


Joined: 01 Jul 2004
Posts: 3598
Location: USA

PostPosted: Tue Sep 11, 2012 6:34 pm    Post subject: Reply with quote

tld wrote:
eccerr0r wrote:
This depends on what kinds of files are you going to put on it?

I recently just used ext3 for a 2TB disk for mythtv. Seems to work fine.


Just a note on this: The only downside of using ext3 for MythTV storage is that you have to enable the "delete files slowly" option in MythTV. That causes it to slowly truncate files rather than deleting them outright. This is because ext3 takes a LONG time to delete huge files and locks up the file system long enough to cause glitches in any recording that might be going on.

I moved to ext4 for MythTV a long time ago and it's been great. Huge files delete instantly. Note however that when you convert ext3 to ext4, existing files will not delete quickly...just newly created ones.

Tom

Hmm.... Looks like this is default, I've noticed files deleting slowly on the machine and didn't notice too many hiccups... In any case, perhaps I should migrate to ext4 anyway, I just migrated my laptop ssd to ext4 after mistakingly used ext3 (since the kernel I had to bootstrap with didn't have ext4 in it).
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed to be advocating?
Back to top
View user's profile Send private message
queen
Veteran
Veteran


Joined: 19 Jul 2005
Posts: 1512

PostPosted: Wed Sep 12, 2012 12:02 pm    Post subject: Reply with quote

Hypnos wrote:
Another vote for ext4.

I have a number crunching machine with a 1TB ext4 partition. I routinely create and delete 200GB files without difficulty.


It will be a crunching numbers machine mainly + some multimedia (less). And performance in this case is important.
If you know some good links to compare between ext4 and jfs, I will appreciate.
Back to top
View user's profile Send private message
Hypnos
Advocate
Advocate


Joined: 18 Jul 2002
Posts: 2859
Location: Omnipresent

PostPosted: Wed Sep 12, 2012 1:11 pm    Post subject: Reply with quote

The Arch wiki page on JFS is a good place to start.

Google has some interesting things to say about JFS --

* It only journals metadata (link), so I would only use it with a UPS and backups. (Same with XFS.)

* Moreover, it is no longer being actively maintained by IBM and has unfixed bugs (mailing list thread). (XFS wins here, as it has many active developers.)

Ext4 can journal both metadata and data (if enabled), so it's safer, but slower. Obviously it has the largest developer and user community of any Linux filesystem, so it's the best on that count.

Ext4 with hashed tree support ("dir_index" option) and extents gives you some of the performance niceties of JFS/XFS for big directories and large files, respectively.

I would use Ext4 unless you have a particular reason to use something different.
_________________
Personal overlay | Simple backup scheme
Back to top
View user's profile Send private message
queen
Veteran
Veteran


Joined: 19 Jul 2005
Posts: 1512

PostPosted: Wed Sep 12, 2012 10:02 pm    Post subject: Reply with quote

Good links. It looks like I will choose ext4 based on the reasons mentioned here and some link I found via google. There it shows some benchmarks they did with various file systems. http://farbfinal.wordpress.com/2010/05/10/file-system-performance-benchmark-ext4-jfs-xfs-reiserfs/ ext4 was the best.



Hypnos wrote:


Ext4 can journal both metadata and data (if enabled), so it's safer, but slower. Obviously it has the largest developer and user community of any Linux filesystem, so it's the best on that count.

Ext4 with hashed tree support ("dir_index" option) and extents gives you some of the performance niceties of JFS/XFS for big directories and large files, respectively.

I would use Ext4 unless you have a particular reason to use something different.


Can you please write how you enable data too? Or better, if you recall the whole command when you created the file system, it will be nice. /etc/fstab will be nice to have too.
Back to top
View user's profile Send private message
Hypnos
Advocate
Advocate


Joined: 18 Jul 2002
Posts: 2859
Location: Omnipresent

PostPosted: Wed Sep 12, 2012 11:35 pm    Post subject: Reply with quote

Here is my fstab line:

Code:
/dev/sda1      /      ext4 noatime,nodiratime   0 1


Data journaling is on by default; you would turn it off by adding the "data=writeback" mount option.

I don't recall the command I used to make the fs, but according to the options I have set as reported by tune2fs, the following should work:

Code:
# mke2fs -L <label> -t ext4 -O dir_index,extent,has_journal,large_file,spare_super,uninit_bg <device>

_________________
Personal overlay | Simple backup scheme
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 2276
Location: Canada

PostPosted: Thu Sep 13, 2012 1:14 am    Post subject: Reply with quote

And to add to the mix - reiserfs3 worked flawlessly for me on servers and desktops that ran for 5-8 years.
Back to top
View user's profile Send private message
Hypnos
Advocate
Advocate


Joined: 18 Jul 2002
Posts: 2859
Location: Omnipresent

PostPosted: Thu Sep 13, 2012 1:28 am    Post subject: Reply with quote

dmpogo wrote:
And to add to the mix - reiserfs3 worked flawlessly for me on servers and desktops that ran for 5-8 years.

What kind of developer backing does ReiserFS currently have? BTW, it also only does metadata journaling.
_________________
Personal overlay | Simple backup scheme
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 2276
Location: Canada

PostPosted: Thu Sep 13, 2012 4:13 am    Post subject: Reply with quote

Hypnos wrote:
dmpogo wrote:
And to add to the mix - reiserfs3 worked flawlessly for me on servers and desktops that ran for 5-8 years.

What kind of developer backing does ReiserFS currently have? BTW, it also only does metadata journaling.


It was just a data point - I never lost data on 5 machines with reiserfs3 running over I'd say 6 years each on average, mostly on 24/7 - that is 30 computer years. Perhaps my luck,

And the current develpment ? As long as it is in the kernel and works, I am happy, are there any bugs to be worried about ?

It is not that I use reiserfs exclusively - I use xfs on scratch disks with big files, ext2 on some root partitions on SSD's and jfs on one machine.
The only one which stutters on portage update, BTW, so I plan to replace jfs to reiser3 (or xfs) on partition that carries portage.

Somehow, I never used ext4, so have no track record here.
Back to top
View user's profile Send private message
Hypnos
Advocate
Advocate


Joined: 18 Jul 2002
Posts: 2859
Location: Omnipresent

PostPosted: Thu Sep 13, 2012 4:23 am    Post subject: Reply with quote

I think it's useful for a filesystem to have a large, active developer community in case of bugs and to prevent regressions on kernel upgrades. I don't know the current situation with ReiserFS, which is why i asked.
_________________
Personal overlay | Simple backup scheme
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 2276
Location: Canada

PostPosted: Thu Sep 13, 2012 4:30 am    Post subject: Reply with quote

Hypnos wrote:
I think it's useful for a filesystem to have a large, active developer community in case of bugs and to prevent regressions on kernel upgrades. I don't know the current situation with ReiserFS, which is why i asked.


I don't think there is any active work on reiserfs3 for quite a few years already but it is part of the standard kernel, so
unless it is dropped it should remain compartible with newer kernels. reiserfs4 development, on the other hand, has just been reactivated few days ago at

http://sourceforge.net/projects/reiser4/files/reiser4-for-linux-3.x/
Back to top
View user's profile Send private message
The Doctor
Veteran
Veteran


Joined: 27 Jul 2010
Posts: 1265

PostPosted: Thu Sep 13, 2012 4:34 am    Post subject: Reply with quote

Hypnos wrote:
I think it's useful for a filesystem to have a large, active developer community in case of bugs and to prevent regressions on kernel upgrades. I don't know the current situation with ReiserFS, which is why i asked.


From the Arch wiki:
Quote:
ReiserFS (V3) Hans Reiser's high-performance journaling FS uses a very interesting method of data throughput based on an unconventional and creative algorithm. ReiserFS is touted as very fast, especially when dealing with many small files. ReiserFS is fast at formatting, yet comparatively slow at mounting. Quite mature and stable. ReiserFS (V3) is not being actively developed at this time. Generally regarded as a good choice for /var.


It does not look like prison is a good place to develop a filesystem from.
_________________
First things first, but not necessarily in that order.
Back to top
View user's profile Send private message
eccerr0r
Advocate
Advocate


Joined: 01 Jul 2004
Posts: 3598
Location: USA

PostPosted: Thu Sep 13, 2012 5:42 am    Post subject: Reply with quote

I was using reiser3 for rootfs for some machines and was generally pleased with it, but with Hans behind bars and having disks large enough to not worry about wasting space (that tail packing saves), I've started to migrate back to the ext2/ext3/ext4 line of filesystems. Mainly because I hate forgetting to compile in filesystem modules :) Ext2 is pretty small and the smallest machines, to save RAM, try to use ext2.

Reiser3 saves a *lot* of disk space for the portage tree. It and portage seem like a match made in heaven (that, and if there's an innately compressing file or other tail packing system on the horizon.)

I still do have xfs as an alternate preferred filesystem (I have a RAID5 LVM that uses xfs) and it seems to work OK but may end up using ext4 if I have to redo it...
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed to be advocating?
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 2276
Location: Canada

PostPosted: Thu Sep 13, 2012 5:58 am    Post subject: Reply with quote

eccerr0r wrote:
I was using reiser3 for rootfs for some machines and was generally pleased with it, but with Hans behind bars and having disks large enough to not worry about wasting space (that tail packing saves), I've started to migrate back to the ext2/ext3/ext4 line of filesystems. Mainly because I hate forgetting to compile in filesystem modules :) Ext2 is pretty small and the smallest machines, to save RAM, try to use ext2.

Reiser3 saves a *lot* of disk space for the portage tree. It and portage seem like a match made in heaven (that, and if there's an innately compressing file or other tail packing system on the horizon.)

I still do have xfs as an alternate preferred filesystem (I have a RAID5 LVM that uses xfs) and it seems to work OK but may end up using ext4 if I have to redo it...


Yes, it is /var where I also put my protage tree on that I want to covert from jfs back to reiser3. But in order not to have 4 different filesystem concurrently (ext2 on root on SSD, xfs on big data disks, jfs on /home and /var - wanted to try it) I may end up converting the final jfs partition on the machine - /home to reiser or xfs as well
Back to top
View user's profile Send private message
khayyam
Veteran
Veteran


Joined: 07 Jun 2012
Posts: 1531

PostPosted: Fri Sep 14, 2012 12:00 pm    Post subject: Reply with quote

Hypnos wrote:
Code:
/dev/sda1      /      ext4 noatime,nodiratime   0 1

Hypnos ... I was under the impression that nodiratime was a subset of noatime, and that the latter implies the former.

best ... khay
Back to top
View user's profile Send private message
Hypnos
Advocate
Advocate


Joined: 18 Jul 2002
Posts: 2859
Location: Omnipresent

PostPosted: Fri Sep 14, 2012 12:18 pm    Post subject: Reply with quote

khayyam wrote:

Hypnos ... I was under the impression that nodiratime was a subset of noatime, and that the latter implies the former.

It seems that you are correct
_________________
Personal overlay | Simple backup scheme
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum