thought experiment... QEMU VM Gentoo updates...distcc?

[eccerr0r]

I have Gentoo running in Gentoo on a QEMU VM and need to update that VM.

So it's a regular virtual machine, I can emerge -uDU @world just fine... well, incomplete story. I allocated two of the 4 CPUs to the VM. As naturally I want the updates to go as fast as possible, I don't mind using all four cores but want to limit it during regular use, but the question is: Which is faster:

Do I increase the core count to 4 on the VM ...
or do I tell the VM to distcc to the host?

Well, there are some build processes that would indeed benefit from core count increases but for gcc, it seems a bit muddy as running a process in a VM has a speed penalty, but distcc runs full speed on the helper machine, which would be the host that running QEMU (among others...). Network activity also wastes CPU cycles so that too would be a negative for using distcc... But solution would result in a faster gcc time? Anyone did some experiments?
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[Hu]

In the general case, I'd say increasing the VM's core count would be better, since that can benefit any package you build, but distcc can only benefit the subset of packages that (1) are C/C++ and (2) are compatible with distcc. In the case of gcc, the latter part of its build involves running the just-built gcc that has not been installed. That phase is not compatible with distcc. Given the rise of projects that use expensive non-C compilers (Go, Rust), you may have packages that are expensive to build and completely incompatible with distcc, so the only way to make them go faster is to give them more cores.

Another option, depending on how similar the host and VM are, would be to use the host to build tbz2 packages, then have the guest install them via emerge --usepkgonly. This has the benefits of using all the host's cores, allowing use of more RAM than the VM might readily have access to, avoiding any virtualization overhead, and being compatible with all packages, regardless of size or compiler. It has the notable disadvantage that it requires the host to be able to build the package, which could require the host to install tools you otherwise don't need. (As a silly example, if the host is a headless server, and the guest runs Firefox over a forwarded X11 connection to a third system, then the host doesn't need gtk+ or rust for its own purposes, but building Firefox on the host will require them to be installed anyway.)

[eccerr0r]

A rough guess is that I'm taking at least around 20% speed penalty overall running in QEMU-KVM but it's hard to gauge in this case as the host is amd64 and the guest is x86. For packages like qtwebengine (luckily I don't need on the x86 VM) I'd think that distcc would greatly benefit compiling straight on bare metal -- minus the network overhead I didn't account for.

Indeed when hitting rust/go/and the plethora of programs that preprocess with python along with gcc itself, these things don't benefit from distcc. However most of core is still running C/C++ and surprisingly even clang distcc's fairly well though I think Gentoo gets distcc+gcc better than distcc+clang.

If only hotplugging CPUs on qemu was more transparent and automatic, even with libvirt it's kind of messy and risky - it would be nice if autodetect and autoundetect processors was automatic - though this issue is a real problem: the guest better stop using a CPU before it gets hot unplugged. Even hotplugging memory which usually works is still kind of clunky and sometimes doesn't work. I have an amd64 guest on the same machine and for some reason RAM can't be hotplugged.

Alas the x86 guest seem does balloon properly.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[Hu]

amd64 vs x86 makes things a bit more complicated, since you'd have to maintain an x86 chroot on the host to use the binary package approach.

For hotplugging, does it work any better if the guest keeps all its cores, but you use guest kernel features to limit how many cores are used by the guest processes, either cgroups or having the guest kernel offline the CPU?