| View previous topic :: View next topic |
| Author |
Message |
clytle374
Apprentice
Joined: 01 Aug 2006
Posts: 221
|
 Posted: Mon Oct 31, 2022 3:03 pm Post subject: Low disk space: recommend changes to wiki and DIY threads |
 |
|
I run an old computer with a small SSD drive, needed more space and found these pages.
Sticky on freeing up disk space
https://forums.gentoo.org/viewtopic-t-30547.html
Wiki on SSDs
https://wiki.gentoo.org/wiki/SSD
And a wiki on freeing up disk space
https://wiki.gentoo.org/wiki/Knowledge_Base:Freeing_disk_space
These should have using savedconfig for firmware included as firmware has gotten to nearly 1Gb in size now. I realized this when recently the firmware file was updated and I remembered the savedconfig.
I don't have edit privileges on the wiki or sticky FAQ subsection and thought it should probably be added.
Thanks
Cory
_________________
NooB since RedHat 6.1 |
|
| Back to top |
|
 |
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54244
Location: 56N 3W
|
| Posted: Mon Oct 31, 2022 7:14 pm Post subject: |
|
|
clytle374.
The Wiki Talk pages are open to all. Even if you can't fix a page, you can create a talk page entry.
The wiki will then email all contributors to the page ... who can fix it.
You post here is likely to be missed.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9679
Location: almost Mile High in the USA
|
| Posted: Mon Oct 31, 2022 7:28 pm Post subject: |
|
|
Surprised existing pages don't include more stuff that can be trimmed...
make.conf: FEATURES="nodoc noinfo" or you can delete a lot of the extra documentation stored in /usr/share/doc/* and /usr/share/info/* respectively. /usr/share/gtk-doc could also be trimmed.
If you're really manly you can try FEATURES="noman" to not install man pages. (hmm... this probably could be worded better...)
Then there's localization information - you can blow away the locales that you don't use in /usr/share/locale/ /usr/share/help ... come to think of it, a lot of /usr/share/ can be blown away if needed.
Making sure you go down to just one python/gcc/llvm/... version is also helpful.
Also do consider, do you really need to run portage/emerge on some of these embedded systems. That's yet another optimization, though you should think long and hard before omitting them, though it may make sense in some situations.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54244
Location: 56N 3W
|
| Posted: Mon Oct 31, 2022 8:06 pm Post subject: |
|
|
eccerr0r,
It easy to put the repos and distfiles on external media.
As you hint, the requirements for maintaining Gentoo and running Gentoo are quite different and the two cases can be separated.
Until a few years ago I had a Gentoo install running from an early 8G SSD. An Acer One Netbook with the 32 bit Intel Atom
Needless to say, it didn't build there.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
Goverp
Advocate
Joined: 07 Mar 2007
Posts: 2008
|
| Posted: Tue Nov 01, 2022 10:40 am Post subject: |
|
|
| clytle374 wrote: | ...
These should have using savedconfig for firmware included as firmware has gotten to nearly 1Gb in size now. I realized this when recently the firmware file was updated and I remembered the savedconfig.
... |
<ediit>I forgot about building firmware into the kernel</edit>
A bit of digging on the web, and a bit of experimentation, found the following useful command: to run at least 48 hours after your last update to /lib/firmware, and with a reboot in the last 24 hours, and/or if you build your firmware into the kernel, a kernel build in the last 24 hours:
| Code: | | find /lib/firmware -atime 2 |
It lists all firmware files accessed since the reboot. If you're building a list for savedconfig, don't forget to scan /etc for drivers (such as USB scanners) that might download firmware on demand. (The use of -atime 2 instead of -atime 1 is down to find rounding rules - see man find for details.)
This assumes you have not totally disabled atime updates by specifying noatime for your rootfs in /etc/fstab; there's not a lot of advantage in that nowadays; the kernel assumes lazyatime,relatime, which have negligible performance impact, and ensure atime gets bumped once a day for accessed files. (I wonder if the lazyatime feature is responsible for the delay when the OpenRc shutdown remounts / as R/O? IIUC that's when the kernel has to update the atime values when running with lazyatime.)
_________________
Greybeard |
|
| Back to top |
|
|
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9679
Location: almost Mile High in the USA
|
| Posted: Tue Nov 01, 2022 5:08 pm Post subject: |
|
|
atime updates don't cost too much in terms of slowing things down, but the main reason for not enabling them nowadays is due to flash memory wear... which might be a large percentage of total writes to a small embedded system. It's still useful to enable them temporarily to do the accounting, find out what you need, and then disable it later.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20067
|
| Posted: Thu Nov 03, 2022 3:00 am Post subject: |
|
|
| clytle374 wrote: | These should have using savedconfig for firmware included as firmware has gotten to nearly 1Gb in size now. I realized this when recently the firmware file was updated and I remembered the savedconfig.
I don't have edit privileges on the wiki or sticky FAQ subsection and thought it should probably be added. |
I've added a reference to the wiki page documentation for 'savedconfig'. Thanks.
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
clytle374
Apprentice
Joined: 01 Aug 2006
Posts: 221
|
| Posted: Sun Dec 11, 2022 3:08 am Post subject: |
|
|
Thanks everyone, I forgot about this post until I started running out of space again.
| eccerr0r wrote: | Surprised existing pages don't include more stuff that can be trimmed...
make.conf: FEATURES="nodoc noinfo" or you can delete a lot of the extra documentation stored in /usr/share/doc/* and /usr/share/info/* respectively. /usr/share/gtk-doc could also be trimmed.
If you're really manly you can try FEATURES="noman" to not install man pages. (hmm... this probably could be worded better...)
Then there's localization information - you can blow away the locales that you don't use in /usr/share/locale/ /usr/share/help ... come to think of it, a lot of /usr/share/ can be blown away if needed.
Making sure you go down to just one python/gcc/llvm/... version is also helpful.
Also do consider, do you really need to run portage/emerge on some of these embedded systems. That's yet another optimization, though you should think long and hard before omitting them, though it may make sense in some situations. |
The FEATURES= nodoc noinfo noman don't trigger rebuilds, will it take effect as things are rebuilt due to other changes?
This system has a small SSD drive that the system runs on. distfiles are on another drive and /tmp is a ram drive.
Thanks
Cory
_________________
NooB since RedHat 6.1 |
|
| Back to top |
|
|
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9679
Location: almost Mile High in the USA
|
| Posted: Sun Dec 11, 2022 7:15 am Post subject: |
|
|
correct, they are applied on package installation, so it doesn't affect the current system until the packages get re-merged.
You could also just manually delete /usr/share/doc/ if you feel lucky.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54244
Location: 56N 3W
|
| Posted: Sun Dec 11, 2022 12:06 pm Post subject: |
|
|
clytle374,
Put /usr/src on the external drive too. Its only needed to build kernels and external modules.
Likewise, the repos are only needed a build and install time, so can go with distfiles.
If you want to do binary package installs, the piece(s) of the repo to support that are required.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9679
Location: almost Mile High in the USA
|
| Posted: Sun Dec 11, 2022 6:58 pm Post subject: |
|
|
curious how big peoples' DE/GUI Gentoo installs are (with portage tree, but minus user data, distfiles, etc.)
Seems like around
i686 with XFCE/Firefox: 10GB
amd64 multilib with XFCE/Firefox: 16GB
These also aren't fully cleaned of superfluous linux-firmware files, so perhaps these can go down a little more.
I don't remember if these numbers are with kernel sources installed, which is about 1GB these days. Note these numbers do not leave any room to build additional packages so a 20GB HDD is not big enough for an amd64 multilib self-hosting (i.e. can emerge for itself for updates), though for i686 it may work with some file space juggling...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3137
|
| Posted: Sun Dec 11, 2022 7:32 pm Post subject: |
|
|
My amd64 nomultilib with lxde (and a lot of other things installed) is just under 20GB.
This does not include RAM/SWAP and temporary files required for a big build (like a web browser) to succeed.
Although I do have some headless servers running of 20GB drives, I wouldn't expect them to be able to compile GUI applications. System takes around 9Gb there, but being headless it has no need for GUI stuff. |
|
| Back to top |
|
|
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9679
Location: almost Mile High in the USA
|
| Posted: Sun Dec 11, 2022 8:56 pm Post subject: |
|
|
Just checked one of my amd64-multilib XFCE+Firefox installs with minimal extras, looks like 21GB used that includes 2 kernels in /usr/src and a 4GB swapfile. So yeah 16GB is possible, but need at least 32GB to be self hosting, if not more. I did not prune doc/info/manpages or /lib/firmware.
Been thinking about how to set up a very small setup with all the distcc compilers set up along with PXE boot... Would be nice to share the same NFS but things like /var and /run are probably going to be issues, but would be nice to share binaries, etc... hmm.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3137
|
| Posted: Sun Dec 11, 2022 11:41 pm Post subject: |
|
|
| Quote: | | Been thinking about how to set up a very small setup with all the distcc compilers set up along with PXE boot |
By far the best thing I came up with was a shared squashfs image copied to RAM at boot.
I had some scripts that would derive hostname from IP address and announce machine's presence using avahi. Distcc can use avahi.
Back in the day I had to patch distcc to let it use hosts discovered on ipv6 link-local, it's been a few years ago so might have been fixed upstream. Prioritizing ipv4 over ipv6 (or disabling it completely) might do the trick as well.
A single image makes sure you will have the same version of toolchain running everywhere, which takes care of gcc clashes, and copying it to RAM at boot saves a lot of bandwidth later, this makes a noticeable difference even on fast networks.
Also, this way satellites won't crash just because you're rebooting NFS, though this also means you might want to add a hook to mass-reboot all the satellites after creating a new image. |
|
| Back to top |
|
|
Goverp
Advocate
Joined: 07 Mar 2007
Posts: 2008
|
| Posted: Mon Dec 12, 2022 9:02 am Post subject: |
|
|
| NeddySeagoon wrote: | clytle374,
Put /usr/src on the external drive too. Its only needed to build kernels and external modules.
Likewise, the repos are only needed a build and install time, so can go with distfiles.
If you want to do binary package installs, the piece(s) of the repo to support that are required. |
Alternatively, if you want them, you can use squashfs images of both /usr/src and /var/db/repos/gentoo. To use the former, you need to build kernels outside /usr/src/linux, but the kernel guys have always recommended building in user space, not from root. I'll dig out my scripts for handling /usr/src (I'm on the wrong machine just now).
_________________
Greybeard |
|
| Back to top |
|
|
|
Documentation, Tips & Tricks
