View previous topic :: View next topic |
Author |
Message |
Gentree Watchman
Joined: 01 Jul 2003 Posts: 5350 Location: France, Old Europe
|
Posted: Fri Aug 09, 2013 8:36 am Post subject: ext4 disk bloat |
|
|
HI,
I have been holding back on kernel updates in order to retain several partitions I have using Reiser4 but this implies holding back other packages and is not longer really practicable.
So I decide to migrate those partitions to ext4. However, having converted a couple of them I note a huge bloat of the used disk space for the same data.
I'm using /one as a temporary storage to copy the fs contents
Code: |
pwd
/tmpd
time cp -a * /one
real 4m50.358s
user 0m0.696s
sys 0m22.923s
df -h
/dev/hda16 reiser4 3.8G 3.5G 277M 93% /tmpd
/dev/hda11 ext4 13G 6.3G 5.7G 53% /one
|
So what reiser4 managed to store in 3.5GB ext4 is managing to bloat to 6.3GB.
WTF? That's not far off DOUBLE !
Do I need to tune the default options or something?
TIA, Gentree. _________________ Linux, because I'd rather own a free OS than steal one that's not worth paying for.
Gentoo because I'm a masochist
AthlonXP-M on A7N8X. Portage ~x86 |
|
Back to top |
|
|
morpheus2051 Tux's lil' helper
Joined: 07 May 2006 Posts: 95
|
|
Back to top |
|
|
Gentree Watchman
Joined: 01 Jul 2003 Posts: 5350 Location: France, Old Europe
|
Posted: Fri Aug 09, 2013 9:50 am Post subject: |
|
|
Thanks ,
-m 1 option may reduce the wastage a bit, but I can't find -C indicated in that link.
Quote: | using the mkfs -C option (requires e2fsprogs 1.42) |
I have 1.42.4 yet don't find -C documented in man mkfs.ext4
However, I'm running 2.6.32-hh1 maybe that kernel is too old.
_________________ Linux, because I'd rather own a free OS than steal one that's not worth paying for.
Gentoo because I'm a masochist
AthlonXP-M on A7N8X. Portage ~x86 |
|
Back to top |
|
|
morpheus2051 Tux's lil' helper
Joined: 07 May 2006 Posts: 95
|
Posted: Fri Aug 09, 2013 10:45 am Post subject: |
|
|
I have sys-fs/e2fsprogs-1.42.7 and the -C option is documented. But I am running a fairly new kernel, vanilla-sources-3.10.5. |
|
Back to top |
|
|
Gentree Watchman
Joined: 01 Jul 2003 Posts: 5350 Location: France, Old Europe
|
Posted: Fri Aug 09, 2013 10:53 am Post subject: |
|
|
I'm a bit confused here. I doubt that manpages is sniffing your kernel version so this does not make sense. Maybe an omission somewhere?
Could you snip me the relevant paragraph. I'll try the option to see whether it's active or not.
thx. _________________ Linux, because I'd rather own a free OS than steal one that's not worth paying for.
Gentoo because I'm a masochist
AthlonXP-M on A7N8X. Portage ~x86 |
|
Back to top |
|
|
morpheus2051 Tux's lil' helper
Joined: 07 May 2006 Posts: 95
|
Posted: Fri Aug 09, 2013 11:02 am Post subject: |
|
|
Code: | -C cluster-size
Specify the size of cluster in bytes for filesystems using the bigalloc feature. Valid cluster-size values are from 2048 to
256M bytes per cluster. By default (if bigalloc is enabled and no cluster size is otherwise specified using this option), the
cluster size will be 16 times the block size. |
Code: | -b block-size
Specify the size of blocks in bytes. Valid block-size values are 1024, 2048 and 4096 bytes per block. If omitted, block-size
is heuristically determined by the filesystem size and the expected usage of the filesystem (see the -T option). If block-size
is preceded by a negative sign ('-'), then mke2fs will use heuristics to determine the appropriate block size, with the con‐
straint that the block size will be at least block-size bytes. This is useful for certain hardware devices which require that
the blocksize be a multiple of 2k. |
|
|
Back to top |
|
|
morpheus2051 Tux's lil' helper
Joined: 07 May 2006 Posts: 95
|
Posted: Fri Aug 09, 2013 11:05 am Post subject: |
|
|
What linux-headers do you have? I have 3.7. Perhaps this option is enabled at build time of e2fsprogs. |
|
Back to top |
|
|
Gentree Watchman
Joined: 01 Jul 2003 Posts: 5350 Location: France, Old Europe
|
Posted: Fri Aug 09, 2013 12:04 pm Post subject: |
|
|
I tried -C and it did not complain .
I think I'll wait until I have the system backedup and updated since this drive is getting a bit flaky anyway. I just need to convert all these partitions to universally available formats.
This is going to cost some much time now , I think I'll go a fresh installation. This one's about ten years old since I first installed Gentoo.
Just as well, installing Gentoo is so frigging complicated, you don't want to do it more than every ten years. _________________ Linux, because I'd rather own a free OS than steal one that's not worth paying for.
Gentoo because I'm a masochist
AthlonXP-M on A7N8X. Portage ~x86 |
|
Back to top |
|
|
morpheus2051 Tux's lil' helper
Joined: 07 May 2006 Posts: 95
|
Posted: Fri Aug 09, 2013 5:41 pm Post subject: |
|
|
My oldest gentoo installation is eight years old and has seen different mobos and cpus. I never felt the need to make a new install. When I upgraded my installation to ext4, I used tar to back up my system. That goes much faster than a new install. If you worry about cruft in your system I found this tool to be working quite well: http://www.genoetigt.de/site/projects/gcruft. |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54237 Location: 56N 3W
|
Posted: Fri Aug 09, 2013 9:11 pm Post subject: |
|
|
Gentree,
The data does not occupy twice the space. The filesystems operate differently
ext4 reserves 5% for the superuser by defualt. It also allocates all of the space for its metadata (i-nodes) at filesystem create time.
The 5% (650Mb) can be reclaimed with tune2fs. The metadata needs the filesystem to be remade.
Look at this tells how may i-nodes you have. Each i-node needs 128B or 256B, depending on your options and you need one i-node per file. Running out of i-nodes produces disk full messages.
mke2fs -t ext4 tries to guess from the size of the volume what a good i-node to disk space ration will be.
For the portage tree, use 1k disk blocks and one i-node per block. For DVD rips one i-node per GB may be generous.
Hmm /dev/hda ... thats going to be a really painful update _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
Gentree Watchman
Joined: 01 Jul 2003 Posts: 5350 Location: France, Old Europe
|
Posted: Sat Aug 10, 2013 9:38 am Post subject: |
|
|
thanks I didn't know that option.
Quote: | Hmm /dev/hda ... thats going to be a really painful update |
Yes, that was the other reason I'm still on an older kernel. I'm pissed off about them pulling the PATA driver. It makes it a lot easier to identify and manage hardware if an IDE drive is identified as such and not pretending to be a sata device.
Apart from that I have a clone of the root partition on sda anyway as a fallback in case a portage update does something silly.
The main problem is that I've had to block some packages like udev that no longer recognise /dev/hda* and the knock-on effect means I started to have a chain other updates I had to block and the whole system has not had a proper update for nearly a year.
That is likely to produce such a rat's nest blockages once I update the portage tree that I think it's probably going to be easier to start afresh.
_________________ Linux, because I'd rather own a free OS than steal one that's not worth paying for.
Gentoo because I'm a masochist
AthlonXP-M on A7N8X. Portage ~x86 |
|
Back to top |
|
|
dmpogo Advocate
Joined: 02 Sep 2004 Posts: 3267 Location: Canada
|
Posted: Sat Aug 10, 2013 3:42 pm Post subject: |
|
|
Gentree wrote: |
thanks I didn't know that option.
Quote: | Hmm /dev/hda ... thats going to be a really painful update |
The main problem is that I've had to block some packages like udev that no longer recognise /dev/hda* and the knock-on effect means I started to have a chain other updates I had to block and the whole system has not had a proper update for nearly a year.
That is likely to produce such a rat's nest blockages once I update the portage tree that I think it's probably going to be easier to start afresh.
|
Why would not you just as a first step convert /dev/hda to a new driver using your old kernel, and then proceed from there ? |
|
Back to top |
|
|
Gentree Watchman
Joined: 01 Jul 2003 Posts: 5350 Location: France, Old Europe
|
Posted: Sun Aug 11, 2013 1:25 pm Post subject: |
|
|
Thanks , pretending IDE drives are SATA is not the real problem, it's just a PITA from a hardware maintenance standpoint.
The trouble is the mess I'm going to hit when I update the portage tree. Either I spend a day trying to unravel a rat's nest of dependency blockages or I do a fresh installation.
Either way looks like a day's work at least. Update will probably involve about 48 hours worth of compile time in itself since just about every package will be out of date. It's probably going to be like a stage 1 tarball installation plus the hassle with the tree.
Plus I've just had to deal with a mobo going flaky that needed swapping out.
I think I'd rather be at the beach.
_________________ Linux, because I'd rather own a free OS than steal one that's not worth paying for.
Gentoo because I'm a masochist
AthlonXP-M on A7N8X. Portage ~x86 |
|
Back to top |
|
|
|