cross compile for powerpc on x86_64, crossdev chroot?

[Pipeline79]

Hi there,

I have an imac-g5, so ppc64, running gentoo which is taking a really long time today to build firefox with optimization flags. Ridiculously long and using up lots of swap.

I was wondering if I could use distcc with portage to speed things up. My laptop is decent, with quite a speedy ivybridge cpu, but it's running slackware natively and gcc is series 5 at the moment.
GCC is 8.3 on the imac. They're both on the same LAN. If the laptop was running Gentoo then the instructions for crossdev to install a powerpc toolchain would be easy, but it's not.
Can I run crossdev in a gentoo chroot and then set that up as the distcc host/client for the imac target?

Or is the reason why all the info I've been able to google relates to qemu in a chroot for distcc because the remote compiler needs a particular ip address?

It seems to be the linking that's got stuck. I used the custom-optimization, custom-cflags & lto flags hoping it would build a faster firefox binary.

Cheers.

[NeddySeagoon]

Pipeline79,

Welcome to Gentoo.

The problem with Gentoo in a chroot on helpers is just as you describe. The system outside the chroot won't know about the compiler inside the chroot.
A Virtual Machine works though, as to all intends and purposes, its another system on your LAN, with its own IP.

Firefox is bad news for lots of reasons. It depends on rust, which is not a language that distcc can distribute. Its not all rust, fh. C/C++ will be sent to the helper(s)
With distcc, the configure, preprocess and link steps still take place on the weaker system.
Linking needs lots of RAM.

You could create a ppc64 chroot on your laptop and have your ivybridge CPU execute QEMU to run ppc64 code.
That has its drawbacks. Its slow, as the ppc64 is emulated in software but it can use all your RAM, so less or no swapping.
You would build your own ppc64 binaries, then install your binaries on your imac-g5.

[Pipeline79]

Pipeline79,

Welcome to Gentoo.

The problem with Gentoo in a chroot on helpers is just as you describe. The system outside the chroot won't know about the compiler inside the chroot.
A Virtual Machine works though, as to all intends and purposes, its another system on your LAN, with its own IP.

Firefox is bad news for lots of reasons. It depends on rust, which is not a language that distcc can distribute. Its not all rust, fh. C/C++ will be sent to the helper(s)
With distcc, the configure, preprocess and link steps still take place on the weaker system.
Linking needs lots of RAM.

You could create a ppc64 chroot on your laptop and have your ivybridge CPU execute QEMU to run ppc64 code.
That has its drawbacks. Its slow, as the ppc64 is emulated in software but it can use all your RAM, so less or no swapping.
You would build your own ppc64 binaries, then install your binaries on your imac-g5.

Thanks for your reply Neddy,

Before I read your reply I thought I'd be clever and try creating a gentoo LXC container on my laptop and see if I could get it to crossdev for ppc64 over distcc by combining your wiki instructions for the raspberry pi and this post here: viewtopic-t-1056580-start-0.html

Its networking is OK, I can ping and ssh into it from the imac target with an assigned static address and a route. I can emerge the crossdev, ccache and distcc packages inside the container and build a crossdev environment with the crossdev tool and a powerpc64-unknown-linux-gnu tuple. However, I've tried configuring the distccd on both machines to accept connections from each other and enabled distcc in portage and so far they don't seem to be sharing jobs. I don't know if what I'm trying is not actually possible, or I am just missing something in my configuration.

The setup that the poster I mentioned was using was actually for 2 x86 machines, and he used ubuntu containers with a gentoo target. In your article you're compiling for the pi from a system that is natively running gentoo. I don't know if the reason my idea won't work is because the container is x86_64 itself - I can't choose a ppc64 container architecture, the closest would be ppc64el, which I think means power9 or talos. Also, some crossdev guides on the wiki suggest changing the portage 'profile' for the build environment to the target arch, which may not be possible inside a container.

I'd be happy trying to build packages either using the method you described, or creating a minimal gentoo install on a new partition, if that would work better for most things. Like in your raspeberry pi example.

[NeddySeagoon]

Pipeline79,

It sounds like you are putting all the the pieces together, standing back a safe distance, then lighting the blue touch paper.
Gentoo does not respond well to that approach. Baby steps ... build on what you know works.

Step 1, on the imac-g5, install distcc, configure it to listen for pleas for help but to its own IP address.
This is not useful long tern but you can use this setup to test the imac-g5 distcc configuration.

Step 2. Do the same thing inside the LXC container.

When that works, you have distccc set up to help itself on both systems.
You don't use the cross compiler in the LXC container yet. You are really only testing the setup. That you will call a cross compile via distcc doesn't matter yet.

Step 3.
In the LXC container, set up your /usr/powerpc64-unknown-linux-gnu/etc/portage almost as it in the the imacs /etc/portage.
Take care with the make.profile symlink and the PACKAGES path in /usr/powerpc64-unknown-linux-gnu/etc/portage/make.conf
The CHOST remains unchanged too.

This step will make a binary package or two for the imac and you really don't want to overwrite your host packages.

Do

Code: Select all

emerge-wrapper -t powerpc64-unknown-linux-gnu --init
powerpc64-unknown-linux-gnu-emerge glibc

This will test your cross compiler. It will try to build glibc with the setting provided in /usr/powerpc64-unknown-linux-gnu/etc/portage/*
If it works glibc will be installed there and you will have a binary package you can install and execute on the imac too.

I chose glibc as an example because every time Linux is brought up on a new architecture, The kernel and glibc are cross compiled this way, so both are known to work.
Now you know your cross toolchain works too.

Step 4. Put the working pieces together.
On the imac, change the /etc/distcc/hosts file to point to the LXC container and stop distccd.
distccd is the bit that listens for cries for help, so its not needed here.
In the LXC container, set up /etc/conf.d/distccd to listen to the imac and restart distccd if it was running. Start it if it wasn't

Check that the imac has FEATURES=disctcc in its make.conf
Check that you have identical powerpc64-unknown-linux-gnu compiler versions both places.

On the imac, run

Code: Select all

emerge -1av glibc

Post any error messages.
Again, glibc is a good test case.

[Pipeline79]

Pipeline79,

It sounds like you are putting all the the pieces together, standing back a safe distance, then lighting the blue touch paper.
Gentoo does not respond well to that approach. Baby steps ... build on what you know works.

Yep, that's a pretty good description...

Thanks for giving me a roadmap to work to, I will follow your advice.

Much appreciated.

[Pipeline79]

Step 3.
In the LXC container, set up your /usr/powerpc64-unknown-linux-gnu/etc/portage almost as it in the the imacs /etc/portage.
Take care with the make.profile symlink and the PACKAGES path in /usr/powerpc64-unknown-linux-gnu/etc/portage/make.conf
The CHOST remains unchanged too.

So, I used the same USE flags, CFLAGS, etc.
The make.profile symlink was pointing to 'embedded' in /var/db/repos/gentoo/profiles , I presume that was what the crossdev setup (for ppc) had done, so I left it at that.
I have CHOST=powerpc64-unknown-linux-gnu and CBUILD=x86_64-pc-linux-gnu --- hope I didn't screw that up.

The next step builds glibc and some others for me successfully in overlay (/usr/powerpc64-unknown-linux-gnu/) root /packages.
I sftp the glibc.tbz2 to the imac and put it in my /usr/portage/packages directory (which didn't exist before, but was specified in the make.conf. Maybe should be in /var/cache/binpkgs/ now?

emerge --usepkgonly -1av =glibc-xxx

now installs the binary package, but has to generate 487 locales on the host. Perhaps should have brought over the built dependency packages too?

It's still generating them as I type this. So, that's where I've got to so far. The crossdev appears to be working, unless it's possible to emerge an 'incorrect' binary package and break the system.
I wanted to include the verbose information that portage reported about the glibc package, but it's gone off the screen now.

[NeddySeagoon]

Pipeline79,

You have different /etc/locale.gen on the I-mac and in the ppc target root
That files governs what locales will be built with glibc.

The embedded profile in the target root is in general, not useful. The /etc/portage/make.profile symlink on the I-mac needs to be copied to the ppc target root.
On the I-mac, its a relative link.
My amd64 has

Code: Select all

/etc/portage/make.profile -> ../../usr/portage/profiles/default/linux/amd64/17.0/no-multilib

If you want to keep it a relative link in the target root you either need to bind mount the hosts /usr/portage inside the target root or add ../../ to the front of the path.

You should fix that as the profile provides a set of default USE flays and the USE flags need to be the same in both places.

Crossdev sets the embedded profile for all arches and almost nobody want to use it.

Provided glibc works, your test passed. The /etc/locale.gen is a detail.

On the I-mac, stop distccd.
Point its /etc/distcc/hosts at the helper.
You don't need to include the I-mac here. It will be used as a fallback anyway.
Check that your I-mac gcc version is the same as the cross compiler version.

Enjoy cross distcc.

If you want to see it working, run the monitor on the I-mac.
Only compiles get distributed, so don't be disappointed to see nothing for a while.

-- edit --

Add distcc to FEATURES on the I-mac.

[NeddySeagoon]

Pipeline79,

Did you get my edit above?

Add distcc to FEATURES on the I-mac.
That's how portage knows to use distcc.

When the i-mac asks for help it will pass powerpc64-unknown-linux-gnu as part of its request.
Unfortunately, it can't ask for a specific version.

It uses whatever version is set in the helpers gcc-config an the active powerpc64-unknown-linux-gnu.

You can have lots of cross compilers and target roots on your helpers

Code: Select all

$ gcc-config -l
 [1] aarch64-unknown-linux-gnu-8.3.0
 [2] aarch64-unknown-linux-gnu-9.2.0 *

 [3] armv6j-hardfloat-linux-gnueabi-8.3.0
 [4] armv6j-hardfloat-linux-gnueabi-9.2.0 *

 [5] armv7a-hardfloat-linux-gnueabi-8.3.0
 [6] armv7a-hardfloat-linux-gnueabi-9.2.0 *

 [7] avr-8.3.0
 [8] avr-9.2.0 *

 [9] i586-pc-linux-gnu-8.3.0
 [10] i586-pc-linux-gnu-9.2.0 *

 [11] i686-pc-linux-gnu-8.3.0
 [12] i686-pc-linux-gnu-9.2.0 *

 [13] mips64el-unknown-linux-gnu-8.3.0
 [14] mips64el-unknown-linux-gnu-9.2.0 *

 [15] powerpc-unknown-linux-gnu-8.3.0
 [16] powerpc-unknown-linux-gnu-9.2.0 *

 [17] x86_64-pc-linux-gnu-8.3.0
 [18] x86_64-pc-linux-gnu-9.2.0 *

The version with the * will be used.
The native compiler has a green *. The cross compiler(s) have a cyan *.

[salfter]

You could create a ppc64 chroot on your laptop and have your ivybridge CPU execute QEMU to run ppc64 code.
That has its drawbacks. Its slow, as the ppc64 is emulated in software but it can use all your RAM, so less or no swapping.

I'm trying to do just that, only ppc instead of ppc64 as I'm looking to get my old G4 Mac mini running something more modern than the now 11-year-old Leopard. I have some experience with cross-architecture chroots as I have one up and running to build aarch64 code for the Raspberry Pi 3 and 4. Here's what I've done so far:

• I added qemu_user_targets_ppc to /etc/portage/package.use/qemu and rebuilt qemu...here's what /etc/portage/package.use/qemu currently has in it:

  Code: Select all

  app-emulation/qemu static-user qemu_user_targets_aarch64 qemu_user_targets_arm qemu_user_targets_ppc qemu_softmmu_targets_arm

• I unpacked the latest ppc stage3 tarball into the chroot directory (call it $CHROOT for reference)
• I installed qemu into the chroot:

  Code: Select all

  ROOT=$CHROOT emerge --usepkgonly --oneshot --nodeps qemu

• I created /etc/binfmt.d/qemu-ppc.conf with this configuration, which I think is correct, and restarted /etc/init.d/binfmt:

  Code: Select all

  :ppc:M::\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x14:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff:/usr/bin/qemu-ppc:

At this point, I should be able to

Code: Select all

chroot $CHROOT bash

, right? That works for the RPi chroot, but for this one I'm getting this:

Code: Select all

chroot: failed to run command ‘bash’: No such file or directory

bin/bash exists where you'd expect it to be. Running qemu-ppc bin/bash gets it to complain that /lib/ld.so.1 is missing, which is expected...it's in the chroot, but not the (AMD64) host. Is there something I'm missing with the chroot setup, or possibly another USE flag needed for qemu?

[Jaglover]

I hate to rain on your parade ... G5 is slow with a full featured browser, the main cause is not slow binary but bloated web sites which put too much load on the browser and the computer which is running it. If you are doing it mainly for fun then go ahead, but don't expect wonders. Only lightweight browser with less features can run faster on old hardware.

[salfter]

I hate to rain on your parade ... G5 is slow with a full featured browser, the main cause is not slow binary but bloated web sites which put too much load on the browser and the computer which is running it. If you are doing it mainly for fun then go ahead, but don't expect wonders. Only lightweight browser with less features can run faster on old hardware.

I'm not expecting miracles from hardware that's probably slower than a Raspberry Pi 3. I suppose it is more for sh*ts and grins than anything else.

I have an SSD on the way for it to replace the hard drive (a 240GB SATA M.2 stick in an adapter that'll convert it to a 2.5" PATA device...cost a whopping $40 for both). That should speed things up somewhat from eliminating seek time and drastically reducing latency, though the SATA-to-PATA conversion will necessarily leave lots of the drive's performance potential untapped (500+ MB/s vs. the 133 at which ATA-7 is maxed out (and the underlying PCI bus, too)). It's kinda like running a modern(ish) server hard drive in an Apple II...ask me how I know about that.

[Kaskad]

I'm not expecting miracles from hardware that's probably slower than a Raspberry Pi 3.

I agree with you on this point.

[schwarzygesetzlos]

@salfter Actually you can have a decent browsing experience on a G4 by using Wayfarer on MorphOS. Also Epiphany works fine on ppc32 Void Linux, though a lot slower. I did not succeed in building Epiphany on Gentoo as webkit-gtk fails to build there. Only browser on Gentoo working fine on ppc32 currently is Netsurf.

Besides that I find cross-compiling via distcc most convenient. Just do a crossdevt -t ppc on any of your more powerful boxes and setup distcc on this box. Build times are not too bad on my G4 DP using FEATURES="distcc".

@Jaglover Firefox works quite decent on my G5 11,2 since FF 82. On youtube you get massive frameskips when running anything >360p but that's pretty ok for hardware from 2005.

[salfter]

Pipeline79,
You could create a ppc64 chroot on your laptop and have your ivybridge CPU execute QEMU to run ppc64 code.
That has its drawbacks. Its slow, as the ppc64 is emulated in software but it can use all your RAM, so less or no swapping.
You would build your own ppc64 binaries, then install your binaries on your imac-g5.

I'm trying to do that, only with a Ryzen 7 3800X building code for a G4 Mac mini. I can qemu-chroot into a directory with the latest ppc stage3 just fine, but I can't build anything. gcc doesn't want to run:

Code: Select all

janeway / # gcc
bash: /usr/bin/gcc: cannot execute binary file: Exec format error

Comparing the gcc executable to a working executable, we get this:

Code: Select all

janeway / # file /bin/ls /usr/powerpc-unknown-linux-gnu/gcc-bin/11/powerpc-unknown-linux-gnu-gcc
/bin/ls: ELF 32-bit MSB pie executable, PowerPC or cisco 4500, version 1 (SYSV), dynamically linked, interpreter /lib/ld.so.1, for GNU/Linux 3.2.0, stripped
/usr/powerpc-unknown-linux-gnu/gcc-bin/11/powerpc-unknown-linux-gnu-gcc: ELF 32-bit MSB executable, PowerPC or cisco 4500, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld.so.1, for GNU/Linux 3.2.0, stripped

The only difference is the lack of PIE on the gcc binary.

(I see that I've posted in this topic before...had Gentoo running for a while, then set aside the computer for a bit. Now I'm trying to sell it and wanted to update, but the lack of EAPI 8 support on the old install made it more or less impossible to upgrade, so I'm reinstalling from scratch. It's been compiling for a couple or three days now.)