| View previous topic :: View next topic |
| Author |
Message |
Massimo B.
Veteran
Joined: 09 Feb 2005
Posts: 1776
Location: PB, Germany
|
 Posted: Wed Nov 09, 2022 10:23 am Post subject: How to share /etc/portage and world file |
 |
|
How to share /etc/portage and world file
I have to maintain several Gentoo installations and like to keep them as synchronized as possible. That means after extending overlays, /etc/portage/package* settings or installing new packages (world file changes) I usually merge /etc/portage/ and /var/lib/portage/ between the machines. Currently I still use dirdiff to merge, later I'm going to use git and etckeeper or something like this.
For easier merging of /etc/portage I separated local machine specific changes in *local files while keeping the other files in sync. That means when using dirdiff for instance I only merge files not ending on *local:
package.accept_keywords
| Code: | # ls /etc/portage/package.accept_keywords/
categories custom local temp
|
categories: Accept whole categories like xfce-*/* (shared)
custom: My own custom ~arch accepts (shared)
local: Machine specific accepts (not shared)
temp: Temporary accepts added by portage itself to get world built.. (shared)
package.use
| Code: | # ls /etc/portage/package.use/
abi_x86_32 custom expand expand.local global local portage |
abi_x86_32: All lazy 32bit uses like virtual/wine abi_x86_32 (shared)
custom: My own custom uses (shared)
local: Machine specific uses like */* -bluetooth (not shared)
expand: Expands only like */* LANG: -* en_US.UTF-8 (shared)
expand.local: Machine specific expands (not shared): | Code: | */* INPUT_DEVICES: -* evdev
*/* VIDEO_CARDS: -* intel i965 radeon r600
*/* GRUB_PLATFORMS: -* efi-64 pc
# echo "*/* $(cpuid2cpuflags)" >> /etc/portage/package.use/expand.local
*/* CPU_FLAGS_X86: -*
*/* CPU_FLAGS_X86: aes avx f16c mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3 |
global: Global uses (shared) | Code: | # ...
*/* -gnome
*/* -ipv6
*/* -kde
*/* -libass
*/* -systemd
*/* bash
*/* bash-completion
*/* cddb
*/* gnome-keyring
*/* gnome-online-accounts
*/* gstreamer
*/* hddtemp
*/* heif
# ... |
The files {expand*,global} are useful to get that kind of stuff out of make.conf in order to make it mergeable line-by-line.
package.mask
| Code: | # ls /etc/portage/package.mask/
custom repos temp |
custom: My custom masks (shared)
repos: (shared) Here I usually mask everything from all enabled repos because I don't want an enabled repo to insert any unwanted packages: | Code: | #...
*/*::akujin
*/*::awesome
*/*::bar
*/*::bgo-overlay
*/*::bombo82
#... |
temp: Temporary masks added by portage (shared)
package.unmask
| Code: | # ls /etc/portage/package.unmask/
custom portage temp |
custom: My custom unmasks (shared)
I usually unmask specific inofficial overlay packages here because I globally masked all from overlays before: | Code: | #...
acct-group/x2godesktopsharing::akujin
media-sound/freetube::zGentoo
media-sound/qpwgraph::flussence
#... |
make.conf
I made /etc/portage/make.conf a directory in order to have separated files for shared and a local parts
| Code: | ls /etc/portage/make.conf/
global local |
global: The usual make.conf that has only shared content (shared)
local: The machine specific part (not shared)
| Code: | CHOST="x86_64-pc-linux-gnu"
### CFLAGS #####################################
# native
CFLAGS="-march=native -mtune=native"
CFLAGS="${CFLAGS} -O2 -pipe -fomit-frame-pointer"
CXXFLAGS="${CFLAGS} -fvisibility-inlines-hidden"
LDFLAGS="${LDFLAGS} -Wl,--as-needed -Wl,-z,now -Wl,-z,relro"
################################################
MAKEOPTS="-j10 -l16" # locally |
distcc
For temporarilly using distcc I had to switch the make.conf/local part to a distcc configuration before, but now have a separate function doing that:
| /etc/bash/bashrc.d/my_emerge-distcc.sh: | function emerge-distcc () {
DISTCC_FEATURES="distcc"
DISTCC_MAKEOPTS="-j28 -l24"
### CFLAGS #####################################
# http://en.gentoo-wiki.com/wiki/Safe_Cflags/Intel
# https://blogs.gentoo.org/mgorny/2014/06/23/inlining-marchnative-for-distcc/
# gcc -v -E -x c -march=native -mtune=native - < /dev/null 2>&1 | grep cc1 | perl -pe 's/^.* - //g;'
# gcc -### -march=native -x c -
# resolve-march-native --keep-identical-mtune --keep-default-params
# explicitly for distcc
#DISTCC_CFLAGS="-march=ivybridge -mtune=ivybridge"
# explicitly for distcc
#DISTCC_CFLAGS="-march=ivybridge --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=8192 -mtune=ivybridge"
# explicitly for distcc (resolve-march-native --keep-identical-mtune)
DISTCC_CFLAGS="-march=ivybridge -maes -mavx -mcx16 -mf16c -mfsgsbase -mfxsr -mmmx -mpclmul -mpopcnt -mrdrnd -msahf -msse -msse2 -msse3 -msse4.1 -msse4.2 -mssse3 -mtune=ivybridge -mxsave -mxsaveopt --param=l1-cache-line-size=64 --param=l1-cache-size=32 --param=l2-cache-size=8192"
################################################
DISTCC_EMERGEARGS=""
CFLAGS="$DISTCC_CFLAGS" CXXFLAGS="${CFLAGS} -fvisibility-inlines-hidden" FEATURES="$FEATURES $DISTCC_FEATURES" MAKEOPTS="$MAKEOPTS $DISTCC_MAKEOPTS" emerge $DISTCC_EMERGEARGS "$@"
} |
I hope you got an idea how I'm going to separate those files. I also merge /etc/portage/repos.conf/ and some of the other parts...
World file and package sets
Next job is to share the world file. Previously I just merged /var/lib/portage/world between the machines and run emerge -auUDtv world --keep-going. Now I have separated that as well:
My world file is now only getting machine specific installations. This usually are firmwares for local hardware only.
I could put also intel-related stuff like sys-firmware/intel-microcode sys-power/intel-undervolt there. But because all my current machines are Intel based I'll put that to the shared as well. It always depends on how common or different your machines are.
Almost all packages go into the shared list as all of my machines are used for daily work and development.
/etc/portage/sets/machine_shared get's all shared packages that all machines have installed.
/var/lib/portage/world_sets always contains @machine_shared.
Currently I'm running the same architecture, profile and same global amd64 (stable) keyword on all machines.
Another approach discussed in the discussion thread would be to define a shared profile or to create my own overlay for the machines. Currently for my case I don't see any advantage for a profile or overlay.
~~~~~~~ END ~~~~~~~
Keep this thread clean and post to discussion thread: 8756631
_________________
HP ZBook Power 15.6" G8 i7-11800H|HP EliteDesk 800G1 i7-4790|HP Compaq Pro 6300 i7-3770
Last edited by Massimo B. on Thu Nov 17, 2022 9:19 am; edited 3 times in total |
|
| Back to top |
|
 |
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3149
|
| Posted: Wed Nov 09, 2022 1:26 pm Post subject: |
|
|
| Or just use ansible. |
|
| Back to top |
|
|
Massimo B.
Veteran
Joined: 09 Feb 2005
Posts: 1776
Location: PB, Germany
|
| Posted: Thu Nov 10, 2022 2:46 pm Post subject: |
|
|
I made /etc/portage/make.conf/ a directory now for having separate make.conf files without the need to source.
_________________
HP ZBook Power 15.6" G8 i7-11800H|HP EliteDesk 800G1 i7-4790|HP Compaq Pro 6300 i7-3770 |
|
| Back to top |
|
|
Massimo B.
Veteran
Joined: 09 Feb 2005
Posts: 1776
Location: PB, Germany
|
| Posted: Tue Nov 15, 2022 10:14 am Post subject: |
|
|
I learned that all interactive bash settings like PROMPT_COMMAND and user required aliases and function better go to /etc/bash/bashrc.d/ instead of /etc/profile.d/ so I moved the distcc function there.
_________________
HP ZBook Power 15.6" G8 i7-11800H|HP EliteDesk 800G1 i7-4790|HP Compaq Pro 6300 i7-3770 |
|
| Back to top |
|
|
Massimo B.
Veteran
Joined: 09 Feb 2005
Posts: 1776
Location: PB, Germany
|
| Posted: Thu Nov 17, 2022 9:21 am Post subject: |
|
|
Due to some magic replacing of portage (8759373) I needed to add CXXFLAGS to the emerge-distcc function.
_________________
HP ZBook Power 15.6" G8 i7-11800H|HP EliteDesk 800G1 i7-4790|HP Compaq Pro 6300 i7-3770 |
|
| Back to top |
|
|
|
Documentation, Tips & Tricks
