Which filesystem for Linux? A HD utilization question.

[Albert_Alligator]

I am very curious about the file systems for Linux and those used by other operating systems.

Which filesystem utilizes HD space the most efficiently? I know that FAT, FAT16 and FAT32 all have their deficiencies in utilizing space well. For example, my USB drive which is using a FAT filesystem looses quite a bit of space. The drive has 523,485,184 bytes of available space, yet with the FAT filesystem, only around 498,000,000 bytes is available for use. This means that there is around 25.5 MB that is lost due to a crappy file system. On hard drives, it is much the same, disk space goes unused due to the way a file system allocates usable space.

How do the Linux file systems stack up against each other, and against the FAT and NTFS file systems for HD and non-volatile memory utilization?

Another question is; has anyone had experience with changing the filesystem on USB non-volatile memory drives that utilize NAND or NOR flash? Just curious if anyone has changed the filesystem over to ReiserFS or JFS or ext3 or something like that.

Anyone?

Thanks much for the input guys,

Al
_________________
As Socrates once said "I drank what?"

[eelke]

You will allways loose space to administrative purposes. Filesystems like fat and ext2/3 allocate this beforehand while reiserfs for instance allocates this dynamically. However I have no actual numbers on how much each uses. This is also highly dependent on the average file size.

FAT/16/32 use a FAT and a backup copy. The space for this you loose as soon as you format the partition. The space for storing filenames, times, etc.. is allocated dynamically. The FAT is space where for each cluster on the disk a 32-bit value is reserved (on fat32). If this value is zero the cluster is free if it is non zero it is in use and contains the number of the next cluster of the file/directory. FAT filesystems suffer from high fragmentation.

NTFS, I do not know much about the internals. However in many ways I guess its more like the UNIX file systems so it will be more dynamic however my experience with it shows that performance is lacking and fragmentation is still a big problem.

ext2 is a relative simple filesystem which is reasonable efficient and performs well when memory and cpu power is limited. However because it splits the partition on formatting into an administrative part and a data part there is the problem that if one is full the disk is full while there might actually be space left.

ext3 is ext2 with journaling

Reiserfs is a journaling filesystem based on B* trees (index structure used by many databases) which makes reiserfs very efficient with large numbers of files in a single directory. Small files can be stored directly in the tree while large files are stored seperatly this saves a block read for small files. Also it doesn't allocate space in blocks of say 4 kB it just takes the space it needs. The downside is that it requires more CPU power and more memory however starting with pentium II class machines this is no problem.

JFS and XFS are both 64 bit filesystem and thus can handle very large partitions. Both are designed for multi cpu server environments where concurrent access is very important. For most people these filesystems are overkill.

There is a big difference between flashmemory drives and normal drives. While for normal drives the seek time is a large factor this is no issue for flashdrives. Because of this I do not expect much benefits from reiserfs on flashdrives while I normally would recommend it for harddisks. If you work on your flashdrive ext2/3 might be beneficial if you use it mostly to transfer files I guess FAT32 will do nice. If you have mainly large files on your flashdrive you should consider formatting it with a larger cluster size as this will reduce the size of the FAT's. The disadvantage of large cluster size on FAT (and ext2/3) is that you waste an average of half a cluster for each file.

[Ruzbeh]

Reliable --> ext2, ext3

Fast --> ReiserFS, Reiser4, XFS

Or, at least, that's how I look at the filesystems. Personally I use XFS for my partitions with big things on them, /opt, /mnt/octoginta (for music and movies). And I use Reiser4 with small config files and such, my / and everything else.

XFS is uberfast with big files, Reiser4 is uberfast with small files (<4kb).

[Archangel1]

I heard somewhere that Reiser4 takes up very little space on an empty system compared to reiser3. I can't remember where that was though so I can't say how accurate it is.

Yeah, FAT is appalling. Especially on bigger drives, as you found the space overhead gets really rough. And the lack of journalling is generally not good.
It generally gets chosen for compatibility though, there's not a lot of point having a flash drive if it won't read on anything.

As the name implies, SquashFS is particularly space-efficient. I think it may be read only though, so not appropriate for most cases.