crossdev build chain

[eu47]

Trying to build the crossdev chain on amd64 for Raspberry Pi 4:

Code: Select all

crossdev -P -v -t aarch64-unknown-linux-gnu --init-target -oO /var/db/repos/rpi
crossdev --target aarch64-unknown-linux-gnu -oO /var/db/repos/rpi -s4

This fails at stage 3 (building glibc) with:

Code: Select all

emerge: there are no ebuilds to satisfy "virtual/awk"

The glibc ebuild states "app-alternatives/awk".
Why is this happening?
Where is that "virtual/awk" coming from?

[Moderator edit: added [code] tags to preserve output layout. -Hu]

[pingtoo]

You run into Introduction of app-alternatives ebuilds problem.

You can do after you finish step: "crossdev -P -v -t aarch64-unknown-linux-gnu --init-target -oO /var/db/repos/rpi"

Code: Select all

cd /usr/aarch64-unknown-linux-gnu/
cd etc/portage
mkdir package.use
cd package.use
cat<<_EOF>crossdev
app-alternatives/awk gawk
app-alternatives/bc gnu
app-alternatives/bzip2 reference
app-alternatives/cpio gnu
app-alternatives/gzip reference
app-alternatives/lex flex
app-alternatives/sh bash
app-alternatives/tar gnu
app-alternatives/yacc bison
_EOF

of cause you can change above packages alternative as long as it will let you finish compile.

Then continue to crossdev build.

I think this is a bug associate with the introduce the app-alternatives concept. both crossdev and catalyst suffer this problem.

[Hu]

This fails at stage 3 (building glibc) with:

Code: Select all

emerge: there are no ebuilds to satisfy "virtual/awk"

Please post the output leading up to this.

Why is this happening?
Where is that "virtual/awk" coming from?

From the command line or from a DEPEND, RDEPEND, BDEPEND, ... in an ebuild that is needed, transitively, by the command line. Seeing the full build log might tell us more.

[eu47]

After applying pingtoo's hint it still fails.

I did not find a way to attach the logs in the post, so I put shared them here:

https://drive.google.com/drive/folders/ ... sp=sharing

[Hu]

Logs can be pasted inline using [code] tags if they are short, or uploaded to a pastebin or other external service if they are long. Based on the Google Drive link you provided, you appear to have 5 log files. However, the page is not able to present them as copyable text, nor are any of them actual links, so I can only see how big they are. From your original report, I suspect we want the glibc log file, the exact name of which cannot be selected from the Google Drive page. I have seen Google fail before, but failing even to present selectable text is a new one for me.

[pingtoo]

After applying pingtoo's hint it still fails.

I did not find a way to attach the logs in the post, so I put shared them here:

https://drive.google.com/drive/folders/ ... sp=sharing

How did you populate ebuild repository "rpi"?

emerge: there are no ebuilds to satisfy "virtual/awk".
(dependency required by "cross-aarch64-unknown-linux-gnu/glibc-2.33-r14::rpi" [ebuild])
(dependency required by "cross-aarch64-unknown-linux-gnu/glibc" [argument])

The reference to "virtual/awk" came from the "rpi" ebuild repo.

[eu47]

/var/db/repos/rpi is initially an empty directory.

At the end of this command:

Code: Select all

crossdev -P -v -t aarch64-unknown-linux-gnu --init-target -oO /var/db/repos/rpi

it is populated with whatever the result it generates. That is automatic, without my intervention.

cross-aarch64-unknown-linux/binutils
cross-aarch64-unknown-linux/gcc
cross-aarch64-unknown-linux/gdb
cross-aarch64-unknown-linux/glibc
cross-aarch64-unknown-linux/linux headers
metadata/layout.conf
packages
profiles/categories
profiles/repo_name

All the files in cross-aarch64-unknown-linux are symlinks to /var/db/repos/gentoo, which is the machine's portage repo.

[eu47]

I should mention a few more things:

1. This started to happen a few months ago, probably around the time when app-alternatives concept was introduced.
2. I keep regular backups, so I have a (still) working crossdev chain based on gcc-12.2.0 and glibc-2.33-r14.
3. Each time I'm trying to update to a newer glibc version, the build fails. For example, I just updated to gcc-13 by using

Code: Select all

crossdev --target aarch64-unknown-linux-gnu -oO /var/db/repos/rpi --gcc 13 --libc 2.33-r14 -s4

If I remove "--libc 2.33-r14" from above command (which means I'm also trying to update glibc), the build fails.

[pingtoo]

I should mention a few more things:

1. This started to happen a few months ago, probably around the time when app-alternatives concept was introduced.
2. I keep regular backups, so I have a (still) working crossdev chain based on gcc-12.2.0 and glibc-2.33-r14.
3. Each time I'm trying to update to a newer glibc version, the build fails. For example, I just updated to gcc-13 by using

crossdev --target aarch64-unknown-linux-gnu -oO /var/db/repos/rpi --gcc 13 --libc 2.33-r14 -s4

If I remove "--libc 2.33-r14" from above command (which means I'm also trying to update glibc), the build fails.

Your "cross-aarch64-unknown-linux-gnu-glibc.log" only content very few lines of message. if the cross-aarch64-unknown-linux/glibc is symbolic point to glibc-2.33-r14.ebuild::gentoo than we need to understand why your ::gentoo repository have reference to virtual/awk. On my ::gentoo repository which is git base was synced on 04/16.2023 which does not have the reference to virtual/awk.

Do you update your ::gentoo recent?

Do you have to use glibc-2.33? can you use more recent version? i.e. 2-36-r7?

[eu47]

Your "cross-aarch64-unknown-linux-gnu-glibc.log" only content very few lines of message. if the cross-aarch64-unknown-linux/glibc is symbolic point to glibc-2.33-r14.ebuild::gentoo than we need to understand why your ::gentoo repository have reference to virtual/awk. On my ::gentoo repository which is git base was synced on 04/16.2023 which does not have the reference to virtual/awk.

Do you update your ::gentoo recent?

Do you have to use glibc-2.33? can you use more recent version? i.e. 2-36-r7?

Your answer made me purge /var/db/repos and update ::gentoo. Then I tried to rebuild the crossdev chain with:

Code: Select all

crossdev --target aarch64-unknown-linux-gnu -oO /var/db/repos/rpi -s4

which failed at cross-gcc-stage2, after successful build of glibc.
Then I stepped back and built successfully with

Code: Select all

crossdev --target aarch64-unknown-linux-gnu -oO /var/db/repos/rpi --libc 2.33-r14 -s4

At this point I made a backup, then I repeated the same as above with newer versions of glibc.
Any glibc version from 2.34 to 2.37 builds successfully, but afterwards cross-gcc-stage2 fails.

https://drive.google.com/drive/folders/ ... share_link

[pingtoo]

eu47,

You really should try app-text/wgetpaste. Using google drive is pain.

Not sure why you want to use gcc-13. since it is not stable.

cross-gcc-stage2 is using first stage to build final gcc with all the front end tool. it also a verify that stage1 build is really useful. I guess in your case it prove it is not stable yet. so it is not good to use for cross compile.

[eu47]

I understand.
But even gcc-12 fails to build at stage 2 with glibc > 2.33.
I can't figure out if it's a gcc problem or glibc one.

[Hu]

If you still need help, please pastebin the complete build log. As I stated above, and as pingtoo alluded to recently, Google Drive is not a proper way to share files, because it cannot properly share files or even their names.

[eu47]

I'll give up for now and stick to crossdev with gcc-12 and glibc-2.33, which works well.

I tried a pastebin.com free account but it only allows for 512kB pastes and the full log has 2.7MB.

A few extracts related to errors are shown below:

Code: Select all

Checking multilib configuration for libgomp...
mkdir -p -- aarch64-unknown-linux-gnu/libgomp
Configuring in aarch64-unknown-linux-gnu/libgomp
configure: creating cache ./config.cache
...
...
configure: error: Pthreads are required to build libgomp
make[1]: *** [Makefile:17133: configure-target-libgomp] Error 1
make[1]: Leaving directory '/tmp/alien/portage/cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1/work/build'
make[1]: *** Waiting for unfinished jobs....
make[1]: Entering directory '/tmp/alien/portage/cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1/work/build'
Checking multilib configuration for libbacktrace...
...
...
make[1]: Leaving directory '/tmp/alien/portage/cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1/work/build'
make[1]: Entering directory '/tmp/alien/portage/cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1/work/build'
Checking multilib configuration for libatomic...
mkdir -p -- aarch64-unknown-linux-gnu/libatomic
Configuring in aarch64-unknown-linux-gnu/libatomic
...
...
configure: error: Pthreads are required to build libatomic
make[1]: *** [Makefile:18098: configure-target-libatomic] Error 1
make[1]: Leaving directory '/tmp/alien/portage/cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1/work/build'
make: *** [Makefile:1041: all] Error 2
 * ERROR: cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1::rpi failed (compile phase):
 *   emake failed

All this is happening when crossdev uses any glibc >= 2.34.
glibc-2.33 works perfect

[Hu]

None of that output shows any errors that we can use. At best, it confirms to us that pthreads cannot be used, but does not say why. See config.log for details.

[eu47]

Wondering if I should report a bug on crossdev related to this.
Maybe the issue can be replicated by others who could find the problem.

[axl]

I'll give up for now and stick to crossdev with gcc-12 and glibc-2.33, which works well.

I tried a pastebin.com free account but it only allows for 512kB pastes and the full log has 2.7MB.

A few extracts related to errors are shown below:

Code: Select all

Checking multilib configuration for libgomp...
mkdir -p -- aarch64-unknown-linux-gnu/libgomp
Configuring in aarch64-unknown-linux-gnu/libgomp
configure: creating cache ./config.cache
...
...
configure: error: Pthreads are required to build libgomp
make[1]: *** [Makefile:17133: configure-target-libgomp] Error 1
make[1]: Leaving directory '/tmp/alien/portage/cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1/work/build'
make[1]: *** Waiting for unfinished jobs....
make[1]: Entering directory '/tmp/alien/portage/cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1/work/build'
Checking multilib configuration for libbacktrace...
...
...
make[1]: Leaving directory '/tmp/alien/portage/cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1/work/build'
make[1]: Entering directory '/tmp/alien/portage/cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1/work/build'
Checking multilib configuration for libatomic...
mkdir -p -- aarch64-unknown-linux-gnu/libatomic
Configuring in aarch64-unknown-linux-gnu/libatomic
...
...
configure: error: Pthreads are required to build libatomic
make[1]: *** [Makefile:18098: configure-target-libatomic] Error 1
make[1]: Leaving directory '/tmp/alien/portage/cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1/work/build'
make: *** [Makefile:1041: all] Error 2
 * ERROR: cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1::rpi failed (compile phase):
 *   emake failed

All this is happening when crossdev uses any glibc >= 2.34.
glibc-2.33 works perfect

I seen that pthread error before, usually due to lack of static-libs useflag on glibc.

[eu47]

I seen that pthread error before, usually due to lack of static-libs useflag on glibc.

glibc on host is built with static-libs.

For the target glibc, I put static-libs useflag in two places:
- /usr/aarch64-unknown-linux-gnu/etc/portage/package.use/glibc
- /etc/portage/package.use/aarch64-unknown-linux-gnu

The last one is overwritten at build time by crossdev, which removes that use flag and replaces it with:

Code: Select all

aarch64-unknown-linux-gnu/glibc -selinux -libraries -multilib

It does not seem to be helping.

[axl]

glibc on host is built with static-libs

Not on host. On guest. Now granted, there's this component on where the guest is. I'm used that the guest is on /usr/$ARCH. And even then, there are caveats. I use cross-emerge, not crossdev stage4. so there are caveats.

Again, if you want sys-devel/gcc with useflag cxx (on guest), then you will need glibc with static-libs on guest as well.

[axl]

I use mine as such:

Code: Select all

ARCH="arm64" CHOST=aarch64-unknown-linux-gnu CBUILD=aarch64-unknown-linux-gnu ROOT=/mnt/blanc/rootfs/pi/ PKG_CONFIG_PATH="${ROOT}/usr/lib/pkgconfig/" SYSROOT=${ROOT} cross-emerge -NuavD --with-bdeps=y -j4 @world

few things to note here.

first of all, I have 3 different linuces here. the one in /. the one in /usr/aarch64-unknown-linux-gnu and the one in /mnt/blanc/rootfs/pi/. 1 builds 2. 2 builds 3.

in my case 1 & 2 are both aarch64. but if there's another type of computer, like amd64 to arm64, thats' where it will be. an amd64 has a arm64 build in /usr/aarch64-unknown-linux-gnu. and that is the basis to construct a 3rd gentoo.

at least that's how I use it... I use cross emerge, not crossdev. I used crossdev when I started. Now I use cross-emerge.

PS just to cheatsheet, CBUILD is x86_64-pc-linux-gnu when you are on an intel (like) machine.

[eu47]

Interesting. You may have your reasons to have that third linux in /mnt/blanc/rootfs/pi.

I only need your second linux, that in /usr/aarch64-unknown-linux-gnu.
It is used with distcc to help with compilation on the stronger machine of ebuilds on the Raspberry Pi.
So, I do not need to build a full system with cross-emerge @world.

For now it works well with gcc-12.2.1 and glibc-2.36.
I was able to get here with:

Code: Select all

crossdev -P -v -t aarch64-unknown-linux-gnu --init-target -oO /var/db/repos/rpi
crossdev --target aarch64-unknown-linux-gnu -oO /var/db/repos/rpi --gcc 12 --libc 2.33-r14 -s4
crossdev --target aarch64-unknown-linux-gnu -oO /var/db/repos/rpi --gcc 12 --libc 2.36-r7 -s3

If I skip the second command and go directly to the third, the build fails at cross-gcc-stage2.

It's just frustrating that crossdev overwrites automatically at build time any useflags that are put in host's /etc/portage and fully ignores anything put in /usr/cross-aarch64-unknown-linux-gnu/etc/portage. I wonder what's the point of creating /etc/portage in target.

As such, crossdev developers decided to build everything in a certain way, precludes use of static-libs in the target and it's somewhat against Gentoo philosophy of giving the users full liberty to build their system as desired.

[Hu]

The error messages in this thread seem to match [Crossdev] Pthreads are required to build libgomp at cross-gcc-stage2.

From reading the crossdev script, it looks like there are ways to preserve USE, but yes, it does tend to reset things when used, presumably because some cross-targets are so fragile that this was the only way to get them working. Once the cross build has succeeded far enough for you to use emerge cross-arch..., you can customize the USE flags, cease use of crossdev, and maintain the packages via emerge.

[eu47]

I have a complete --stage4 cross build.
/usr/aarch64-unknown-linux-gnu/etc/portage/make.conf has

Code: Select all

CHOST=aarch64-unknown-linux-gnu
CBUILD=x86_64-pc-linux-gnu

This minimal toolchain works perfectly fine in combination with distcc for Raspberry Pi 4.

But any cross-emerge fails at configuration stage with:

Code: Select all

configure: error: C compiler cannot create executables

axl was able to build his third linux with cross-emerge because both his host and target (guest) are arm64 systems, as he mentioned.

In my case the host machine architecture is amd64 and the target toolchain is arm64 (aarch64).
Unless there's another explanation, I think that's why guest's gcc cannot create executables when run on the host with cross-emerge.
It creates executables just fine when running on RPI (arm64 system) with distcc: the RPI sends compilation tasks through distcc to the amd64 machine, which runs its aarch64-unknown-linux-gnu-gcc and its resulting machine code is then sent back to the RPI which links it into an aarch64 executable.

For me, there would be one reason to use cross-emerge on the amd64 host: building an aarch64 llvm/clang to help with clang+distcc compilation on the RPI. Unfortunately, it looks like I can't do that, so I'm limited to gcc+distcc.

[axl]

But any cross-emerge fails at configuration stage with:

Code: Select all

configure: error: C compiler cannot create executables

axl was able to build his third linux with cross-emerge because both his host and target (guest) are arm64 systems, as he mentioned.

In my case the host machine architecture is amd64 and the target toolchain is arm64 (aarch64).
Unless there's another explanation, I think that's why guest's gcc cannot create executables when run on the host with cross-emerge.
It creates executables just fine when running on RPI (arm64 system) with distcc: the RPI sends compilation tasks through distcc to the amd64 machine, which runs its aarch64-unknown-linux-gnu-gcc and its resulting machine code is then sent back to the RPI which links it into an aarch64 executable.

For me, there would be one reason to use cross-emerge on the amd64 host: building an aarch64 llvm/clang to help with clang+distcc compilation on the RPI. Unfortunately, it looks like I can't do that, so I'm limited to gcc+distcc.

I didn't start with both systems being arm. I started just like you, building for PI on intel. It should work. I assume there is something else at fault here. NOT the different arch. And even that is not a problem as you can run cross platform through qemu-static.

I suggest you take a look in config.log to see what the actual error was.

[eu47]

Thanks axl.
The error was setting -march=native, which was not recognized by aarch64-unknown-linux-gnu-gcc.