View previous topic :: View next topic |
Author |
Message |
GENERiCfr n00b
Joined: 13 Feb 2012 Posts: 11
|
Posted: Wed Aug 22, 2012 8:28 pm Post subject: EXT4 and 9,5GB utilized space |
|
|
Hi, I recently bought a 1TB external HDD(Platinum myDrive, whatever it is) for storage and I decided to create 2 partitions, 300GB's in FAT32 and the rest in EXT4. After setting up reserved block count to 0, there is still 9,5GB of utilized space. Well the 9,5GB does matter, so I'll really appreciate any solutions.
Code: |
Sys. fich. Taille Util. Dispo Uti% Monté sur
rootfs 112G 103G 2,7G 98% /
/dev/root 112G 103G 2,7G 98% /
tmpfs 750M 264K 749M 1% /run
rc-svcdir 1,0M 92K 932K 9% /lib64/rc/init.d
cgroup_root 10M 0 10M 0% /sys/fs/cgroup
udev 10M 0 10M 0% /dev
shm 750M 0 750M 0% /dev/shm
/dev/sda1 124M 21M 98M 18% /boot
/dev/sdb2 631G 9,5G 622G 2% /media/DATA
|
|
|
Back to top |
|
|
BillWho Veteran
Joined: 03 Mar 2012 Posts: 1600 Location: US
|
Posted: Wed Aug 22, 2012 9:30 pm Post subject: |
|
|
GENERiCfr,
There's a good explanation for that here _________________ Good luck
Since installing gentoo, my life has become one long emerge |
|
Back to top |
|
|
gentoo_ram Guru
Joined: 25 Oct 2007 Posts: 474 Location: San Diego, California USA
|
Posted: Thu Aug 23, 2012 12:09 am Post subject: |
|
|
I bet a good amount of that 'used' space is actually the ext4 journal. |
|
Back to top |
|
|
dE_logics Advocate
Joined: 02 Jan 2009 Posts: 2253 Location: $TERM
|
Posted: Thu Aug 23, 2012 2:50 am Post subject: |
|
|
Why don't you use xfs? It's faster, more reliable and has better disaster recovery (it doesn't need a fsck). _________________ My blog |
|
Back to top |
|
|
GENERiCfr n00b
Joined: 13 Feb 2012 Posts: 11
|
Posted: Thu Aug 23, 2012 9:56 am Post subject: |
|
|
There will be a lot of big and small files. It should be fast and safety, so I've chosen EXT4. Are you sure that in this case XFS will be a better option? |
|
Back to top |
|
|
dE_logics Advocate
Joined: 02 Jan 2009 Posts: 2253 Location: $TERM
|
Posted: Thu Aug 23, 2012 1:03 pm Post subject: |
|
|
According to my benchmarks (real world tests), large file performance (300 MB+) is practically the same for ext4 and xfs, ext4 being slightly faster.
Copying small files (portage tree) to ext4 partition is twice as fast compared to XFS.
XFS is about twice as fast in searching for small files as compared to ext4.
Removing portage tree (many small files) was more than 10 times in ext4.
Both gave same performance when reading small files (copying).
But note, these tests were using a flash disk.
So take your pick. The major drawback with ext4 is it's bad search performance, considering searching is something commonly done by everyone, ext4 is a bad candidate, but if you need to copy small files and also delete them at bulk then ext4 is a good candidate.
By the end, I conclude that reiserfs was fantastic for small files (and reiser4 was a lot better, but the project is dead) and xfs was an all rounder. JFS ended last. _________________ My blog |
|
Back to top |
|
|
Small_Penguin Tux's lil' helper
Joined: 27 May 2005 Posts: 140
|
Posted: Fri Aug 24, 2012 9:30 am Post subject: |
|
|
Depending on how many files you're going to store on that partition, set the inode count using parameter -N when creating it using mkfs.ext4, and optionally the block size with -b.
For example, my /usr/portage partition which has a lot of small files uses
tune2fs -l /dev/sdb5:
Inode count: 200192 (set with -N 200000)
Block size: 1024 (set with -b 1024)
Free inodes: 32834
This can save much space. Note that the small block size might introduce a noticeable performance hit, so probably you shouldn't use it if you can avoid it. You will have to experiment to find out.
For my boot partition:
tune2fs -l /dev/sda5:
Inode count: 10048 (set with -N 10000)
Block size: 4096 (default)
Free inodes: 9980
I've created nearly all my partitions setting -N and -b parameters manually. There's also the -T option, but I've never used that.
Note that if you're setting inode count to big values (>300000), you won't see much differences. That means there will be no gain going from 600000 to 300000 inodes.
Also, be warned that if you're setting the inode count too low, you might run out of inodes obviously (dmesg will tell you no free inodes) and will be unable to create any new files. |
|
Back to top |
|
|
dusanc Apprentice
Joined: 19 Sep 2005 Posts: 248 Location: Serbia
|
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54234 Location: 56N 3W
|
Posted: Mon Sep 10, 2012 9:28 pm Post subject: |
|
|
dE_logics,
Did you use the btrees option for ext4 so that directory entries were stored in a binary tree data structure? _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
dE_logics Advocate
Joined: 02 Jan 2009 Posts: 2253 Location: $TERM
|
Posted: Wed Sep 12, 2012 2:18 am Post subject: |
|
|
NeddySeagoon wrote: | dE_logics,
Did you use the btrees option for ext4 so that directory entries were stored in a binary tree data structure? |
No. I'll surely try it out.
However, I didn't find any btress option in the man pages. _________________ My blog |
|
Back to top |
|
|
dE_logics Advocate
Joined: 02 Jan 2009 Posts: 2253 Location: $TERM
|
Posted: Wed Sep 12, 2012 2:26 am Post subject: |
|
|
Hay, that AFTER my post. So before that it was dead.
And it appears we have new contributers too. _________________ My blog |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54234 Location: 56N 3W
|
Posted: Wed Sep 12, 2012 5:46 pm Post subject: |
|
|
dE_logics,
The option is -O dir_index
Thej man page says
Code: | dir_index
Use hashed b-trees to speed up lookups in large
directories. |
_________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
sgarcia Apprentice
Joined: 21 May 2003 Posts: 254 Location: Bakersfield, CA
|
Posted: Wed Sep 12, 2012 11:23 pm Post subject: |
|
|
FWIW, the last time I made use of XFS it had *real* problems getting shut down dirty. You'd be almost guaranteed file system corruption. That's been a long time, maybe they've fixed that, but at the time you were recommended to have good UPS and auto-shutdown support if you used it. _________________ Ignorance killed the cat. Curiosity was framed. |
|
Back to top |
|
|
dE_logics Advocate
Joined: 02 Jan 2009 Posts: 2253 Location: $TERM
|
Posted: Fri Sep 14, 2012 2:36 am Post subject: |
|
|
NeddySeagoon wrote: | dE_logics,
The option is -O dir_index
Thej man page says
Code: | dir_index
Use hashed b-trees to speed up lookups in large
directories. |
|
I'll do benchmarks between ext4 and xfs again. _________________ My blog |
|
Back to top |
|
|
dE_logics Advocate
Joined: 02 Jan 2009 Posts: 2253 Location: $TERM
|
Posted: Fri Sep 14, 2012 3:20 am Post subject: |
|
|
I again did search benchmarks between ext4, xfs and reiserfs using portage tree, but on a spare and old internal HDD.
With tune2fs -O dir_index /dev/... xfs was ~69% faster than ext4, rather than double, whereas reiserfs was more than 10 folds faster than ext4 and more than 7 folds faster than xfs.
As of reliability, all of ext* FS have given me more problems than any other FS other than JFS (maybe). By that I don't mean you'll loose data quick, but you need more frequent fsck checks, and it takes time. _________________ My blog |
|
Back to top |
|
|
dE_logics Advocate
Joined: 02 Jan 2009 Posts: 2253 Location: $TERM
|
Posted: Fri Sep 14, 2012 4:06 am Post subject: |
|
|
I tried to compile with the patches, but --
Code: | CC fs/fs-writeback.o
fs/fs-writeback.c: In function 'generic_writeback_sb_inodes':
fs/fs-writeback.c:560:6: error: request for member 'nr_to_write' in something not a structure or union
fs/fs-writeback.c:561:6: error: request for member 'pages_skipped' in something not a structure or union
fs/fs-writeback.c:563:3: warning: passing argument 3 of 'writeback_single_inode' from incompatible pointer type [enabled by default]
fs/fs-writeback.c:344:1: note: expected 'struct writeback_control *' but argument is of type 'struct writeback_control **'
fs/fs-writeback.c:565:38: error: request for member 'nr_to_write' in something not a structure or union
fs/fs-writeback.c:566:29: error: request for member 'nr_to_write' in something not a structure or union
fs/fs-writeback.c:569:10: error: request for member 'pages_skipped' in something not a structure or union
make[1]: *** [fs/fs-writeback.o] Error 1
make: *** [fs] Error 2 |
_________________ My blog |
|
Back to top |
|
|
dE_logics Advocate
Joined: 02 Jan 2009 Posts: 2253 Location: $TERM
|
Posted: Thu Sep 20, 2012 4:01 am Post subject: |
|
|
dE_logics wrote: | I again did search benchmarks between ext4, xfs and reiserfs using portage tree, but on a spare and old internal HDD.
With tune2fs -O dir_index /dev/... xfs was ~69% faster than ext4, rather than double, whereas reiserfs was more than 10 folds faster than ext4 and more than 7 folds faster than xfs.
As of reliability, all of ext* FS have given me more problems than any other FS other than JFS (maybe). By that I don't mean you'll loose data quick, but you need more frequent fsck checks, and it takes time. |
Compressed reiser4 was almost 16 times faster than reiserfs, that implies, 150 times faster than ext4; these benchmarks are magnitudes apart. And reiser4 without compression delivers the same performance.
I think it's ironic to develop btrfs in front of such a fantastic FS which hosts such a long tail of features with a LOT better performance. _________________ My blog |
|
Back to top |
|
|
|