[Solved] Portage with distcc and cross-compiling

solamour · Last edited by solamour on Tue Jun 19, 2018 12:10 am; edited 2 times in total

[Moderator note: this post, and the first 3 responses, were originally attached to the extremely long dead thread Compile on fast machine, run on slow machine?, last active October 2003. This thread has similar subject matter, but can stand on its own. -Hu]

Sorry to wake up a really, really old thread.

My slow machine is 32-bit with CHOST="i486-pc-linux-gnu", and the fast machine is 64-bit with CHOST="x86_64-pc-linux-gnu". When I ran distcc, the slow one complained that it couldn't link 64-bit objects. Do I need to change the fast one's CHOST to "i486-pc-linux-gnu" and rebuild gcc? If so, do I need to change the fast one's CHOST back to "x86_64-pc-linux-gnu" when I update the fast one itself?
__
sol

krinn · Watchman Joined: 02 May 2003 Posts: 7470

You need to cross compile.
As you use distcc refers to https://wiki.gentoo.org/wiki/Distcc/Cross-Compiling
But you should better not use distcc, and build binaries for the i486 host and provide them just ready made. See https://wiki.gentoo.org/wiki/Embedded_Handbook/General/Creating_a_cross-compiler and https://wiki.gentoo.org/wiki/Binary_package_guide

NeddySeagoon · Posted: Tue Jan 16, 2018 12:48 pm Post subject:

solamour,

What profile do you use on the 64 bit system?
What profile do you have on the 486?

What CFLAGS do you have on the 486?
You may need to use cross distcc, you may only need to fix the CFLAGS on the 486.

-- edit --

For something as old as a 486, I would create a 486 chroot on the 64 bit box and make a binary package server there.
Then have the 486 use the prebuilt binaries.

Heh. krinn said that already.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

solamour · Posted: Tue Jan 16, 2018 7:49 pm Post subject:

The profile and CFLAGS for the slow the the fast are as the following.

Hu · Moderator Joined: 06 Mar 2007 Posts: 21631

solamour

NeddySeagoon · Posted: Thu Jan 18, 2018 11:48 am Post subject:

solamour,

If you set up your cross compiler with

solamour · Posted: Thu Jan 18, 2018 9:41 pm Post subject:

NeddySeagoon · Posted: Thu Jan 18, 2018 9:54 pm Post subject:

solamour,

Its a geode, not a i486. It might even be a i686. Please post /proc/cpuinfo from the slow system.

When you install cross compilers with crossdev, gcc-config lists them like

solamour · Posted: Fri Jan 26, 2018 1:36 am Post subject:

solamour · Posted: Thu Jun 14, 2018 9:51 pm Post subject:

After much distraction by other tasks that had higher priorities, I finally got around setting up distcc with cross-compiling, so that I can start a build on a slow machine (x86) and send all the jobs to the fast machine (x64). The slow machine still does preprocessing and such, but at least compiling a non-trivial package doesn't take a few hours (or, more often, days).

Now the only remaining issue is, although I can build, say, vim from source code I manually downloaded from (https://github.com/vim/vim) and the final result runs on the slow machine just fine, portage (e.g. "emerge -v vim") always stays in the slow machine; the job is never sent to the fast machine. Do I need to issue a special command to distribute jobs?

gordonb3 · Apprentice Joined: 01 Jul 2015 Posts: 185

Nope. As the name already suggests, distcc distributes (g)cc jobs (and g++) to one or more other machines. Portage is based on python, so that will not be distributed.

solamour · Posted: Fri Jun 15, 2018 9:23 pm Post subject:

Hu · Moderator Joined: 06 Mar 2007 Posts: 21631

Yes, Portage can cause C and C++ compilation jobs to be run remotely rather than locally. Portage's own code (dependency resolution, mainly) would be nice to distribute, but is completely outside the scope of distcc. I think gordonb3 thought you wanted all CPU heavy tasks distributed, regardless of the program that needs the CPU time.

solamour · Posted: Sat Jun 16, 2018 5:54 am Post subject:

gordonb3 · Apprentice Joined: 01 Jul 2015 Posts: 185

Ah. Okay. Misinterpretation on my side. Since you want to helper machine to do cross-compiling, did you also create the wrapper script as described HERE?

As a personal note I find that building on the slow machine itself even with distcc never gets really quick. You get better results if you cross-build binary packages (FEATURES=buildpkg) on the fast machine. Shamefully this doesn't work with every package. e.g. a package can't do something like bootstrapping if the target code can't run on the host architecture. Package installers also often (or always) appear to ignore the ROOT environment setting and you end up with incorrect paths like 32 bit libraries in lib64

solamour · Posted: Tue Jun 19, 2018 12:10 am Post subject:

I finally got distcc and portage work correctly. Perhaps all the documentations are crystal clear to everyone else, but it certainly wasn't for me, so I'll summarize what I've found so far, as a future reference.

In short, I'd like to give all build jobs to the fast machine, because, well, the slow machine is just too slow.

* If, for whatever reasons, FAST (i.e. helper machine) is not available, all the jobs will remain in SLOW, so given enough time, I'd have the exactly same outcome; I just need to wait more. Way more.

* Even when FAST is performing all build jobs, SLOW still needs to do a lot of preprocessing and linking. In other words, simply using the "rule of thumb" for "J" value (i.e. "total cores * 2 + 1") might not be a good idea if SLOW is pitifully under-powered, because it would be too busy preparing the build jobs. Finding the optimal value is more of a trial-and-error process.

* When monitoring the status of SLOW, I need to set DISTCC_DIR.