| View previous topic :: View next topic |
| Author |
Message |
jenkler
Apprentice
Joined: 28 Apr 2003
Posts: 222
Location: Sweden - Stockholm
|
 Posted: Sat Aug 25, 2018 10:34 am Post subject: Any tips on minimizing the gentoo installation? |
 |
|
Optimize my build:
| Code: | CFLAGS="-march=corei7 -Os -pipe"
CHOST="x86_64-pc-linux-gnu"
CPU_FLAGS_X86="mmx sse sse2 sse3 ssse3 sse4_1 sse4_2"
CXXFLAGS="${CFLAGS}"
FEATURES="nodoc noinfo noman unmerge-orphans"
LINGUAS="en sv"
MAKEOPTS="-j9"
USE="
hardened
minimal mmx
nptl nptlonly
sse sse2 sse3 ssse3 sse4_1 sse4_2
threads
-X"
PORTDIR="/usr/portage"
DISTDIR="${PORTDIR}/distfiles"
PKGDIR="${PORTDIR}/packages"
|
After install cleanup
| Code: | function clean {
if [ -f "/build/$build/usr/bin/eselect" ]; then
chroot /build/$build eselect news read
chroot /build/$build eselect news purge
fi
if [ $build != 'gentoo-base-minimal' ] && [ $build != 'distcc' ]; then
emerge --color=n --nospinner --root="/build/$build" -Cqv app-portage/gentoolkit app-portage/portage-utils sys-devel/autoconf \
sys-devel/autoconf-wrapper sys-devel/autogen sys-devel/automake sys-devel/automake-wrapper sys-devel/bin86 sys-devel/binutils \
sys-devel/binutils-config sys-devel/bison sys-devel/flex sys-devel/gcc sys-devel/gcc-config sys-devel/gettext \
sys-devel/gnuconfig sys-devel/libtool sys-devel/llvm sys-devel/m4 sys-devel/make
fi
rm /build/$build/etc/group- &> /dev/null
rm -r /build/$build/etc/modprobe.d &> /dev/null
rm -r /build/$build/root/* &> /dev/null
rm -r /build/$build/root/.* &> /dev/null
rm -r /build/$build/tmp/* &> /dev/null
rm -r /build/$build/tmp/.* &> /dev/null
rm -r /build/$build/usr/local &> /dev/null
rm -r /build/$build/usr/portage &> /dev/null
rm -r /build/$build/usr/share/doc &> /dev/null
rm -r /build/$build/usr/share/info &> /dev/null
rm -r /build/$build/usr/share/man &> /dev/null
rm -r /build/$build/var/tmp/* &> /dev/null
rm -r /build/$build/var/tmp/.* &> /dev/null
rm -r /build/$build/var/log/* &> /dev/null
rm -r /build/$build/var/log/.* &> /dev/null
touch /build/$build/root/.keep /build/$build/tmp/.keep /build/$build/var/tmp/.keep /build/$build/var/log/.keep &> /dev/null
}
|
More stuff I can remove? Or other tips? More packages that is for compile time only, is there a list?
[Moderator edit: added [code] tags to preserve output layout. -Hu]
_________________
Hello from me: Jenkler IT AB (swedish) (use google translate). Check out my Linux manpages for web in english. |
|
| Back to top |
|
 |
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54232
Location: 56N 3W
|
| Posted: Sat Aug 25, 2018 11:13 am Post subject: |
|
|
jenkler,
Tell us the problem you want to solve rather than your solution.
e.g. | Code: | rm -r /build/$build/usr/share/doc &> /dev/null
rm -r /build/$build/usr/share/info &> /dev/null
rm -r /build/$build/usr/share/man &> /dev/null |
is the same as FEATURES="nodoc noinfo noman" so that they are not installed in the first place.
/usr/local should be empty.
If you have the RAM on the build host, /var/tmp/ and /var/log/ can be in RAM, so its gone after reboot.
If /etc/group- is a concern, so is /etc/passwd- and /etc/shadow-
My own use case is to fit Gentoo onto a 2008 Acer One netbook with a 8G SSD.
I do that by building on an external HDD, to avoid the SSD writes, with the HDD partitioned to leave things behind that I don't want when I copy the install over.
You don't need /usr/src, which in the kernel build location. That's about 1G per compiled kernel.
You don't need the toolchain either, unless you plan on having the capability to install more software.
With Mate, Xfce4 and the toolchain, my stripped down Gentoo is about 6G. That gives me 1.5G for swap, so I can hibernate the system. About 32M for swap and a little left over for /home.
If you have the RAM on the build host, /var/tmp/ and /var/log/ can be in RAM, so its gone
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
jenkler
Apprentice
Joined: 28 Apr 2003
Posts: 222
Location: Sweden - Stockholm
|
| Posted: Sat Aug 25, 2018 9:37 pm Post subject: |
|
|
Thanks for the input 
This is my build script for my docker files. The clean func runs after build time to remove unnecessary stuff. I build the system in a new root with root= arg to emerge. I want to be 100 sure of removal so, i can run rm on stuff that not installed.
"You don't need the toolchain either, unless you plan on having the capability to install more software. " <-- What pkg is that, do you have a list ?
Here is the updated version:
function clean {
if [ -f "$path/usr/bin/eselect" ]; then
chroot $path eselect news read
chroot $path eselect news purge
fi
if [ $build != 'gentoo-base-minimal' ] && [ $build != 'distcc' ]; then
emerge --color=n --nospinner --root="$path" -Cqv app-portage/gentoolkit app-portage/portage-utils sys-devel/autoconf \
sys-devel/autoconf-wrapper sys-devel/autogen sys-devel/automake sys-devel/automake-wrapper sys-devel/bin86 sys-devel/binutils \
sys-devel/binutils-config sys-devel/bison sys-devel/flex sys-devel/gcc sys-devel/gcc-config sys-devel/gettext \
sys-devel/gnuconfig sys-devel/libtool sys-devel/llvm sys-devel/m4 sys-devel/make
fi
rm $path/etc/group- $path/etc/passwd- $path/etc/shadow- &> /dev/null
rm -r $path/etc/modprobe.d &> /dev/null
rm -r $path/root/* $path/root/.* &> /dev/null
rm -r $path/tmp/* $path/tmp/.* &> /dev/null
rm -r $path/usr/local &> /dev/null
rm -r $path/usr/portage &> /dev/null
rm -r $path/usr/share/doc &> /dev/null
rm -r $path/usr/share/info &> /dev/null
rm -r $path/usr/share/man &> /dev/null
rm -r $path/var/tmp/* $path/var/tmp/.* &> /dev/null
rm -r $path/var/log/* $path/var/log/.* &> /dev/null
touch $path/root/.keep $path/tmp/.keep $path/var/tmp/.keep $path/var/log/.keep &> /dev/null
}
_________________
Hello from me: Jenkler IT AB (swedish) (use google translate). Check out my Linux manpages for web in english. |
|
| Back to top |
|
|
mike155
Advocate
Joined: 17 Sep 2010
Posts: 4438
Location: Frankfurt, Germany
|
| Posted: Sun Aug 26, 2018 12:44 am Post subject: |
|
|
| Quote: | USE="... mmx sse sse2 sse3 ssse3 sse4_1 sse4_2 ..."
|
Those atoms shouldn't be in your USE flags. Move them to CPU_FLAGS_X86. |
|
| Back to top |
|
|
jenkler
Apprentice
Joined: 28 Apr 2003
Posts: 222
Location: Sweden - Stockholm
|
| Posted: Sun Aug 26, 2018 8:29 am Post subject: |
|
|
| mike155 wrote: | | Quote: | USE="... mmx sse sse2 sse3 ssse3 sse4_1 sse4_2 ..."
|
Those atoms shouldn't be in your USE flags. Move them to CPU_FLAGS_X86. |
Thanks for the heads up. Found doc about it here: https://wiki.gentoo.org/wiki/CPU_FLAGS_X86
But as you can see I have them in both CPU_FLAGS_X86 and USE. I think it can do some good to have then on both places, NOTE: Any hardcore Gentoo devs that can confirm that those flags does something in the USE also.
Love the feedback people, anything else?
_________________
Hello from me: Jenkler IT AB (swedish) (use google translate). Check out my Linux manpages for web in english. |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54232
Location: 56N 3W
|
| Posted: Sun Aug 26, 2018 9:04 am Post subject: |
|
|
jenkler,
| Code: | | USE="... mmx sse sse2 sse3 ssse3 sse4_1 sse4_2 ..." |
and friends were moved to the CPU_FLAGS_X86 use expand in the main portage tree several years ago.
However, if you use overlays, its up to the overlay maintainer to do the migration.
The flags are harmless in both places. Note that they are not options passed to the compiler, rather they instruct build system to include specially optimised code that takes advantage of these instruction sets.
The toolchain and a few other packages are built by /usr/portage/scripts/bootstrap.sh
That's the script that does stage1 of of a stage1 install. It will give you some ideas of what else you can remove.
gcc and portage come to mind but not glibc.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
jenkler
Apprentice
Joined: 28 Apr 2003
Posts: 222
Location: Sweden - Stockholm
|
| Posted: Sun Aug 26, 2018 9:11 am Post subject: |
|
|
Wow, how could i forget sys-apps/portage. Thanks
Updated:
function clean {
if [ -f "$path/usr/bin/eselect" ]; then
chroot $path eselect news read
chroot $path eselect news purge
fi
if [ $build != 'gentoo-base-minimal' ] && [ $build != 'distcc' ]; then
emerge --color=n --nospinner --root="$path" -Cqv app-portage/gentoolkit app-portage/portage-utils sys-apps/portage sys-devel/autoconf \
sys-devel/autoconf-wrapper sys-devel/autogen sys-devel/automake sys-devel/automake-wrapper sys-devel/bin86 sys-devel/binutils \
sys-devel/binutils-config sys-devel/bison sys-devel/flex sys-devel/gcc sys-devel/gcc-config sys-devel/gettext \
sys-devel/gnuconfig sys-devel/libtool sys-devel/llvm sys-devel/m4 sys-devel/make virtual/*
fi
rm $path/etc/group- $path/etc/passwd- $path/etc/shadow- &> /dev/null
rm -r $path/etc/modprobe.d &> /dev/null
rm -r $path/root/* $path/root/.* &> /dev/null
rm -r $path/tmp/* $path/tmp/.* &> /dev/null
rm -r $path/usr/local &> /dev/null
rm -r $path/usr/portage &> /dev/null
rm -r $path/usr/share/doc &> /dev/null
rm -r $path/usr/share/info &> /dev/null
rm -r $path/usr/share/man &> /dev/null
rm -r $path/var/db &> /dev/null
rm -r $path/var/tmp/* $path/var/tmp/.* &> /dev/null
rm -r $path/var/log/* $path/var/log/.* &> /dev/null
touch $path/root/.keep $path/tmp/.keep $path/var/tmp/.keep $path/var/log/.keep &> /dev/null
}
/var/db must be safe to remove ?
gcc is removed but as you say not glibc  Fun to take that one away... My new brick system hehe
_________________
Hello from me: Jenkler IT AB (swedish) (use google translate). Check out my Linux manpages for web in english. |
|
| Back to top |
|
|
Marcih
Apprentice
Joined: 19 Feb 2018
Posts: 213
|
| Posted: Mon Aug 27, 2018 12:52 pm Post subject: |
|
|
Unimportant but worth considering:
_________________
| Bones McCracker wrote: | | It wouldn't be so bad, if it didn't suck. |
| NeddySeagoon wrote: | | The problem with leaving is that you can only do it once and it reduces your influence. |
|
|
| Back to top |
|
|
Anon-E-moose
Watchman
Joined: 23 May 2008
Posts: 6098
Location: Dallas area
|
| Posted: Mon Aug 27, 2018 1:06 pm Post subject: |
|
|
good advice on march native.
Re the jobs vs cores, even from the command line with me doing nothing but emerge I like to keep # of jobs = # of cores.
If I'm in X and doing something I might even run the # of jobs to half the # of cores, since I consider it background, I don't care if it takes a little longer.
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54232
Location: 56N 3W
|
| Posted: Mon Aug 27, 2018 2:05 pm Post subject: |
|
|
jenkler,
Rather than the top down approach of building too much and ripping things out, consider bottom up. Only putting into the container what is actually needed.
I wrote Installing Gentoo Using Packages on the liveDVD a long time ago, mostly because someone told me it wasn't possible. :)
The interesting bit for you is how little you need in the chroot to be able to chroot. See Section 5.
The method is to make or acquire some binary packages, then unpack them.
Add your app and anything that it needs and you are done.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
jenkler
Apprentice
Joined: 28 Apr 2003
Posts: 222
Location: Sweden - Stockholm
|
| Posted: Mon Aug 27, 2018 2:48 pm Post subject: |
|
|
Yes, indeed. But a lot of things are needed at gentoo buildtime. Those things need to be removed afterwards 
_________________
Hello from me: Jenkler IT AB (swedish) (use google translate). Check out my Linux manpages for web in english. |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54232
Location: 56N 3W
|
| Posted: Mon Aug 27, 2018 2:55 pm Post subject: |
|
|
jenkler,
With the bottom up approach you only untar the required binaries into your container filesystem space.
The Gentoo build system stays outside so the container gets only what is required.
There is nothing to remove.
You can have emerge do that directly. It understands several different $ROOT environmental variables, so you can keep your build system and container run time completely separate.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
jenkler
Apprentice
Joined: 28 Apr 2003
Posts: 222
Location: Sweden - Stockholm
|
| Posted: Mon Aug 27, 2018 3:11 pm Post subject: |
|
|
Thats what I am doing, see the code. Note the --root arg. This is just some extra cleanup :p
_________________
Hello from me: Jenkler IT AB (swedish) (use google translate). Check out my Linux manpages for web in english. |
|
| Back to top |
|
|
khayyam
Watchman
Joined: 07 Jun 2012
Posts: 6227
Location: Room 101
|
| Posted: Mon Aug 27, 2018 5:56 pm Post subject: |
|
|
| jenkler wrote: | | Thats what I am doing, see the code. Note the --root arg. This is just some extra cleanup :p |
jenkler ... I see no mention of it anywhere ... but anyhow, as you're getting buildtime deps, and have to then remove them, you probably should be using '--root-deps=rdeps' in conjunction with '--root'
best ... khay |
|
| Back to top |
|
|
jenkler
Apprentice
Joined: 28 Apr 2003
Posts: 222
Location: Sweden - Stockholm
|
| Posted: Mon Aug 27, 2018 6:04 pm Post subject: |
|
|
Wtf... That one was a great one Thanks man! Testing now!
SO this:
emerge --color=n --nospinner --root="$path" --root-deps=rdeps -qv dev-java/oracle-jre-bin
is better than
emerge --color=n --nospinner --root="$path" -qv dev-java/oracle-jre-bin
if I dont want extra files in my new root (--root). correct?
should i use SYSROOT also or what is EAPI 7?
-------------
--root-deps[=rdeps]
If no argument is given then build-time dependencies of packages for ROOT are installed to ROOT instead of /. If the rdeps argument is given then discard all build-time dependencies of packages for ROOT. This option is only meaningful when used together with ROOT and it should not be enabled under normal circumstances!
Does not affect EAPIs that support BDEPEND or HDEPEND. EAPI 7 introduces BDEPEND and experimental EAPI 5-hdepend features HDEPEND as a means to adjust installation into / and ROOT. Use the SYSROOT environment variable to control where DEPEND is installed to under EAPI 7.
When ebuilds with different EAPIs feature in the same emerge run, the appropriate behaviour for each EAPI is applied independently to each ebuild.
---------------
_________________
Hello from me: Jenkler IT AB (swedish) (use google translate). Check out my Linux manpages for web in english. |
|
| Back to top |
|
|
jenkler
Apprentice
Joined: 28 Apr 2003
Posts: 222
Location: Sweden - Stockholm
|
| Posted: Mon Aug 27, 2018 6:43 pm Post subject: |
|
|
Sorry but the --root-deps=rdeps option broke the build. Better to have deps, its more safe that way! I guess that some packages dont like this
Thanks anyway!
_________________
Hello from me: Jenkler IT AB (swedish) (use google translate). Check out my Linux manpages for web in english. |
|
| Back to top |
|
|
khayyam
Watchman
Joined: 07 Jun 2012
Posts: 6227
Location: Room 101
|
| Posted: Mon Aug 27, 2018 6:55 pm Post subject: |
|
|
| jenkler wrote: | | Sorry but the --root-deps=rdeps option broke the build. Better to have deps, its more safe that way! I guess that some packages dont like this :P |
jenkler ... that doesn't exclude (runtime) deps (rdeps), it excludes bdeps (build deps). You should probably add '--with-bdeps=y' to the emerge, as your failure may be due to their being an older bdep that satisfies the dependency, but that may need updated for the package to build (that would be my first assumption anyhow, but without the build.log its next to impossible to say what went wrong).
best ... khay |
|
| Back to top |
|
|
jenkler
Apprentice
Joined: 28 Apr 2003
Posts: 222
Location: Sweden - Stockholm
|
| Posted: Mon Aug 27, 2018 7:16 pm Post subject: |
|
|
Ok, thanks. Will lab with the settings
_________________
Hello from me: Jenkler IT AB (swedish) (use google translate). Check out my Linux manpages for web in english. |
|
| Back to top |
|
|
|
Installing Gentoo
