Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
EXT4 and 9,5GB utilized space
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
GENERiCfr
n00b
n00b


Joined: 13 Feb 2012
Posts: 11

PostPosted: Wed Aug 22, 2012 8:28 pm    Post subject: EXT4 and 9,5GB utilized space Reply with quote

Hi, I recently bought a 1TB external HDD(Platinum myDrive, whatever it is) for storage and I decided to create 2 partitions, 300GB's in FAT32 and the rest in EXT4. After setting up reserved block count to 0, there is still 9,5GB of utilized space. Well the 9,5GB does matter, so I'll really appreciate any solutions. ;)

Code:

Sys. fich.     Taille Util. Dispo Uti% Monté sur
rootfs           112G  103G  2,7G  98% /
/dev/root        112G  103G  2,7G  98% /
tmpfs            750M  264K  749M   1% /run
rc-svcdir        1,0M   92K  932K   9% /lib64/rc/init.d
cgroup_root       10M     0   10M   0% /sys/fs/cgroup
udev              10M     0   10M   0% /dev
shm              750M     0  750M   0% /dev/shm
/dev/sda1        124M   21M   98M  18% /boot
/dev/sdb2        631G  9,5G  622G   2% /media/DATA
Back to top
View user's profile Send private message
BillWho
Veteran
Veteran


Joined: 03 Mar 2012
Posts: 1600
Location: US

PostPosted: Wed Aug 22, 2012 9:30 pm    Post subject: Reply with quote

GENERiCfr,

There's a good explanation for that here :wink:
_________________
Good luck :wink:

Since installing gentoo, my life has become one long emerge :)
Back to top
View user's profile Send private message
gentoo_ram
Guru
Guru


Joined: 25 Oct 2007
Posts: 474
Location: San Diego, California USA

PostPosted: Thu Aug 23, 2012 12:09 am    Post subject: Reply with quote

I bet a good amount of that 'used' space is actually the ext4 journal.
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2253
Location: $TERM

PostPosted: Thu Aug 23, 2012 2:50 am    Post subject: Reply with quote

Why don't you use xfs? It's faster, more reliable and has better disaster recovery (it doesn't need a fsck).
_________________
My blog
Back to top
View user's profile Send private message
GENERiCfr
n00b
n00b


Joined: 13 Feb 2012
Posts: 11

PostPosted: Thu Aug 23, 2012 9:56 am    Post subject: Reply with quote

There will be a lot of big and small files. It should be fast and safety, so I've chosen EXT4. Are you sure that in this case XFS will be a better option?
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2253
Location: $TERM

PostPosted: Thu Aug 23, 2012 1:03 pm    Post subject: Reply with quote

According to my benchmarks (real world tests), large file performance (300 MB+) is practically the same for ext4 and xfs, ext4 being slightly faster.

Copying small files (portage tree) to ext4 partition is twice as fast compared to XFS.

XFS is about twice as fast in searching for small files as compared to ext4.

Removing portage tree (many small files) was more than 10 times in ext4.

Both gave same performance when reading small files (copying).

But note, these tests were using a flash disk.

So take your pick. The major drawback with ext4 is it's bad search performance, considering searching is something commonly done by everyone, ext4 is a bad candidate, but if you need to copy small files and also delete them at bulk then ext4 is a good candidate.

By the end, I conclude that reiserfs was fantastic for small files (and reiser4 was a lot better, but the project is dead) and xfs was an all rounder. JFS ended last.
_________________
My blog
Back to top
View user's profile Send private message
Small_Penguin
Tux's lil' helper
Tux's lil' helper


Joined: 27 May 2005
Posts: 140

PostPosted: Fri Aug 24, 2012 9:30 am    Post subject: Reply with quote

Depending on how many files you're going to store on that partition, set the inode count using parameter -N when creating it using mkfs.ext4, and optionally the block size with -b.

For example, my /usr/portage partition which has a lot of small files uses

tune2fs -l /dev/sdb5:
Inode count: 200192 (set with -N 200000)
Block size: 1024 (set with -b 1024)
Free inodes: 32834

This can save much space. Note that the small block size might introduce a noticeable performance hit, so probably you shouldn't use it if you can avoid it. You will have to experiment to find out.

For my boot partition:
tune2fs -l /dev/sda5:
Inode count: 10048 (set with -N 10000)
Block size: 4096 (default)
Free inodes: 9980

I've created nearly all my partitions setting -N and -b parameters manually. There's also the -T option, but I've never used that.

Note that if you're setting inode count to big values (>300000), you won't see much differences. That means there will be no gain going from 600000 to 300000 inodes.

Also, be warned that if you're setting the inode count too low, you might run out of inodes obviously (dmesg will tell you no free inodes) and will be unable to create any new files.
Back to top
View user's profile Send private message
dusanc
Apprentice
Apprentice


Joined: 19 Sep 2005
Posts: 248
Location: Serbia

PostPosted: Mon Sep 10, 2012 8:56 pm    Post subject: Reply with quote

dE_logics wrote:
.... (and reiser4 was a lot better, but the project is dead) ...


Dead project just released a patch for kernel 3.5.3 http://sourceforge.net/projects/reiser4/files/reiser4-for-linux-3.x/

:D
_________________
Reiser4 Gentoo FAQ [25Dec2016]
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54097
Location: 56N 3W

PostPosted: Mon Sep 10, 2012 9:28 pm    Post subject: Reply with quote

dE_logics,

Did you use the btrees option for ext4 so that directory entries were stored in a binary tree data structure?
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2253
Location: $TERM

PostPosted: Wed Sep 12, 2012 2:18 am    Post subject: Reply with quote

NeddySeagoon wrote:
dE_logics,

Did you use the btrees option for ext4 so that directory entries were stored in a binary tree data structure?


No. I'll surely try it out.

However, I didn't find any btress option in the man pages.
_________________
My blog
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2253
Location: $TERM

PostPosted: Wed Sep 12, 2012 2:26 am    Post subject: Reply with quote

dusanc wrote:
dE_logics wrote:
.... (and reiser4 was a lot better, but the project is dead) ...


Dead project just released a patch for kernel 3.5.3 http://sourceforge.net/projects/reiser4/files/reiser4-for-linux-3.x/

:D


Hay, that AFTER my post. So before that it was dead.

And it appears we have new contributers too.
_________________
My blog
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54097
Location: 56N 3W

PostPosted: Wed Sep 12, 2012 5:46 pm    Post subject: Reply with quote

dE_logics,

The option is -O dir_index
Thej man page says
Code:
                   dir_index
                          Use hashed b-trees to  speed  up  lookups  in  large
                          directories.

_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
sgarcia
Apprentice
Apprentice


Joined: 21 May 2003
Posts: 254
Location: Bakersfield, CA

PostPosted: Wed Sep 12, 2012 11:23 pm    Post subject: Reply with quote

FWIW, the last time I made use of XFS it had *real* problems getting shut down dirty. You'd be almost guaranteed file system corruption. That's been a long time, maybe they've fixed that, but at the time you were recommended to have good UPS and auto-shutdown support if you used it.
_________________
Ignorance killed the cat. Curiosity was framed.
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2253
Location: $TERM

PostPosted: Fri Sep 14, 2012 2:36 am    Post subject: Reply with quote

NeddySeagoon wrote:
dE_logics,

The option is -O dir_index
Thej man page says
Code:
                   dir_index
                          Use hashed b-trees to  speed  up  lookups  in  large
                          directories.


I'll do benchmarks between ext4 and xfs again.
_________________
My blog
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2253
Location: $TERM

PostPosted: Fri Sep 14, 2012 3:20 am    Post subject: Reply with quote

I again did search benchmarks between ext4, xfs and reiserfs using portage tree, but on a spare and old internal HDD.

With tune2fs -O dir_index /dev/... xfs was ~69% faster than ext4, rather than double, whereas reiserfs was more than 10 folds faster than ext4 and more than 7 folds faster than xfs.

As of reliability, all of ext* FS have given me more problems than any other FS other than JFS (maybe). By that I don't mean you'll loose data quick, but you need more frequent fsck checks, and it takes time.
_________________
My blog
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2253
Location: $TERM

PostPosted: Fri Sep 14, 2012 4:06 am    Post subject: Reply with quote

dusanc wrote:
dE_logics wrote:
.... (and reiser4 was a lot better, but the project is dead) ...


Dead project just released a patch for kernel 3.5.3 http://sourceforge.net/projects/reiser4/files/reiser4-for-linux-3.x/

:D


I tried to compile with the patches, but --

Code:
  CC      fs/fs-writeback.o
fs/fs-writeback.c: In function 'generic_writeback_sb_inodes':
fs/fs-writeback.c:560:6: error: request for member 'nr_to_write' in something not a structure or union
fs/fs-writeback.c:561:6: error: request for member 'pages_skipped' in something not a structure or union
fs/fs-writeback.c:563:3: warning: passing argument 3 of 'writeback_single_inode' from incompatible pointer type [enabled by default]
fs/fs-writeback.c:344:1: note: expected 'struct writeback_control *' but argument is of type 'struct writeback_control **'
fs/fs-writeback.c:565:38: error: request for member 'nr_to_write' in something not a structure or union
fs/fs-writeback.c:566:29: error: request for member 'nr_to_write' in something not a structure or union
fs/fs-writeback.c:569:10: error: request for member 'pages_skipped' in something not a structure or union
make[1]: *** [fs/fs-writeback.o] Error 1
make: *** [fs] Error 2

_________________
My blog
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2253
Location: $TERM

PostPosted: Thu Sep 20, 2012 4:01 am    Post subject: Reply with quote

dE_logics wrote:
I again did search benchmarks between ext4, xfs and reiserfs using portage tree, but on a spare and old internal HDD.

With tune2fs -O dir_index /dev/... xfs was ~69% faster than ext4, rather than double, whereas reiserfs was more than 10 folds faster than ext4 and more than 7 folds faster than xfs.

As of reliability, all of ext* FS have given me more problems than any other FS other than JFS (maybe). By that I don't mean you'll loose data quick, but you need more frequent fsck checks, and it takes time.


Compressed reiser4 was almost 16 times faster than reiserfs, that implies, 150 times faster than ext4; these benchmarks are magnitudes apart. And reiser4 without compression delivers the same performance.

I think it's ironic to develop btrfs in front of such a fantastic FS which hosts such a long tail of features with a LOT better performance.
_________________
My blog
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum