Creating binpkg for slow machines

[cogman]

I'm looking for tips on how to approach this.

I have an AMD server with some beefy hardware and a slower laptop with not so beefy hardware (Intel skylake). I'd like to use the AMD server to produce binary packages for my slower laptop with all the trimmings.

Reading over the binary package guide, it looks like I have 4 options.

The first, which is my least preferred, is I could weaken the optimizations down to a common target compatible for everyone. This is definitely the easiest thing to do.

The second, is a chroot environment. The main issue here (as I see it) is I'd be targeting skylake in this env, but I'd be running on an AMD box. that MAYBE fine as I believe my AMD CPU (zen3) has all the same instructions as skylake, but it also might not be if there's some oddball intel only instructions which get inserted into the output.

The 3rd option would be to (somehow) leverage crossdev to create binary packages specifically for my laptop. The issue here is that my laptop has ultimately the same architecture as my host system and cross dev doesn't appear to allow me to split the two up. (Maybe I'm missing something here). This would be my most preferred route, but IDK how to make it work.

The 4th option would be to use qemu (somehow) and emulate my laptop. This is probably doable but it's nasty. Ideally I'd want the host system to be able to build these packages on a schedule and a virtual machine just makes that annoying to deal with. I'd rather not do this.

Any suggestions here would be great. I'd also like to potentially extend this for other computers I own which run into similar problems. It'd be nice to have these packages compiled by my server for those computers while I'm sleeping.

[NeddySeagoon]

cogman,

The second, is a chroot environment. The main issue here (as I see it) is I'd be targeting skylake in this env, but I'd be running on an AMD box. that MAYBE fine as I believe my AMD CPU (zen3) has all the same instructions as skylake, but it also might not be if there's some oddball intel only instructions which get inserted into the output.

The build system emitting "oddball intel only instructions" is OK as long as you don't try to execute them during the build process too.
That would get you an Illegal Instruction Exception. dmesg will tell you all about that.

The Build In A Chroot is the way to go.
Crossdev can't be used.
You may need to set some per package CFLAGS in the chroot to avoid Illegal Instructions but you won't know until you try.

[mega_flow]

Hello

I'm using the same setup
My old computer is a intel haswell cpu and my laptop is a amd cpu
this is the make.conf for the build computer

Code: Select all

WARNING_FLAGS="-Werror=odr -Werror=lto-type-mismatch -Werror=strict-aliasing"
COMMON_FLAGS="-O2 -pipe -march=x86-64-v3 -mtune=znver3 -flto -fomit-frame-pointer ${WARNING_FLAGS}"
CFLAGS="${COMMON_FLAGS} -fno-plt -fexceptions"
CXXFLAGS="${CFLAGS}"
FCFLAGS="${COMMON_FLAGS} -fno-plt"
FFLAGS="${COMMON_FLAGS}"
RUSTFLAGS="-C target-cpu=x86-64-v3 -C link-arg=-Wl,-z,pack-relative-relocs"
CPU_FLAGS_X86="aes avx avx2 bmi1 bmi2 f16c fma3 mmx mmxext pclmul popcnt rdrand sse sse2 sse3 sse4_1 sse4_2 ssse3"

PORTAGE_ELOG_SYSTEM="save"
PORTAGE_ELOG_CLASSES="warn error info log qa"
PORTAGE_SCHEDULING_POLICY="idle"

FEATURES='${FEATURES} buildpkg binpkg-multi-instance -network-sandbox'
EMERGE_DEFAULT_OPTS="${EMERGE_DEFAULT_OPTS} --binpkg-respect-use=y --binpkg-changed-deps=y"
EMERGE_DEFAULT_OPTS="${EMERGE_DEFAULT_OPTS} --buildpkg-exclude='acct-group/* acct-user/* virtual/* app-eselect/* */*-bin'"

MAKEOPTS="-j4 -l5"

I have set the use flags in a custom profile (no-miltilib + gnome) so i dont need to edit all make.conf https://github.com/megaflow8/megaflow
The latop have some extra cpu_flags_x86 but only dev-libs/libgcrypt benefit from that 1
As for chroot i use systemd-nspawn and machinectl on a encrypt image
for the file server i use www-servers/lighttpd and rsync to sync portage and the custom overlay form my build computer to my laptop keep all in sync

The laptop can get to 4000 rpm in fans speed . so night build was is to loud for the people I live with
i'm using this setup for 2 year and i like it

[mid-kid]

...
Crossdev can't be used.
...

I've never tried this myself, but I'm not entirely certain why?
You can create a cross-toolchain for x86_64-mylaptop-linux-gnu, set CHOST=x86_64-mylaptop-linux-gnu on the target system's make.conf, and every build system will treat this as cross-compilation. Alternatively you can create a chroot where you change the CHOST to x86_64-mybuilder-linux-gnu and a crossdev to cross-compile to x86_64-pc-linux-gnu.
That said, crossdev brings its own slew of headaches, with many packages that can't be cross-compiled, or have issues, but in theory, I'm not sure why it wouldn't?

[NeddySeagoon]

mid-kid,

If you change the tuple as you suggest, it will probably work.
With the default x86_64-unknown-linux-gnu parts of the cross toolchain overwrite the host toolchain.

Using x86_64-mylaptop-linux-gnu as a tuple ensures that you cannot use binaries with any other tuple. You are very much on your own.

As you say, crossdev is a mixed blessing.