Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
crossdev build chain
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Gentoo on ARM
View previous topic :: View next topic  
Author Message
eu47
n00b
n00b


Joined: 06 May 2022
Posts: 16
Location: Edmonton, Canada

PostPosted: Sun Apr 30, 2023 3:05 pm    Post subject: crossdev build chain Reply with quote

Trying to build the crossdev chain on amd64 for Raspberry Pi 4:

Code:
crossdev -P -v -t aarch64-unknown-linux-gnu --init-target -oO /var/db/repos/rpi
crossdev --target aarch64-unknown-linux-gnu -oO /var/db/repos/rpi -s4


This fails at stage 3 (building glibc) with:

Code:
emerge: there are no ebuilds to satisfy "virtual/awk"


The glibc ebuild states "app-alternatives/awk".
Why is this happening?
Where is that "virtual/awk" coming from?

[Moderator edit: added [code] tags to preserve output layout. -Hu]
Back to top
View user's profile Send private message
pingtoo
l33t
l33t


Joined: 10 Sep 2021
Posts: 926
Location: Richmond Hill, Canada

PostPosted: Sun Apr 30, 2023 3:39 pm    Post subject: Reply with quote

You run into Introduction of app-alternatives ebuilds problem.

You can do after you finish step: "crossdev -P -v -t aarch64-unknown-linux-gnu --init-target -oO /var/db/repos/rpi"
Code:
cd /usr/aarch64-unknown-linux-gnu/
cd etc/portage
mkdir package.use
cd package.use
cat<<_EOF>crossdev
app-alternatives/awk gawk
app-alternatives/bc gnu
app-alternatives/bzip2 reference
app-alternatives/cpio gnu
app-alternatives/gzip reference
app-alternatives/lex flex
app-alternatives/sh bash
app-alternatives/tar gnu
app-alternatives/yacc bison
_EOF


of cause you can change above packages alternative as long as it will let you finish compile.

Then continue to crossdev build.

I think this is a bug associate with the introduce the app-alternatives concept. both crossdev and catalyst suffer this problem.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 21631

PostPosted: Sun Apr 30, 2023 4:12 pm    Post subject: Re: crossdev build chain Reply with quote

eu47 wrote:
This fails at stage 3 (building glibc) with:
Code:
emerge: there are no ebuilds to satisfy "virtual/awk"
Please post the output leading up to this.
eu47 wrote:
Why is this happening?
Where is that "virtual/awk" coming from?
From the command line or from a DEPEND, RDEPEND, BDEPEND, ... in an ebuild that is needed, transitively, by the command line. Seeing the full build log might tell us more.
Back to top
View user's profile Send private message
eu47
n00b
n00b


Joined: 06 May 2022
Posts: 16
Location: Edmonton, Canada

PostPosted: Sun Apr 30, 2023 4:58 pm    Post subject: Still failing Reply with quote

After applying pingtoo's hint it still fails.

I did not find a way to attach the logs in the post, so I put shared them here:

https://drive.google.com/drive/folders/1ig1PO85yLz_PGuzz_hgtI7KMXBDZpfv-?usp=sharing
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 21631

PostPosted: Sun Apr 30, 2023 5:26 pm    Post subject: Reply with quote

Logs can be pasted inline using [code] tags if they are short, or uploaded to a pastebin or other external service if they are long. Based on the Google Drive link you provided, you appear to have 5 log files. However, the page is not able to present them as copyable text, nor are any of them actual links, so I can only see how big they are. From your original report, I suspect we want the glibc log file, the exact name of which cannot be selected from the Google Drive page. I have seen Google fail before, but failing even to present selectable text is a new one for me.
Back to top
View user's profile Send private message
pingtoo
l33t
l33t


Joined: 10 Sep 2021
Posts: 926
Location: Richmond Hill, Canada

PostPosted: Sun Apr 30, 2023 5:28 pm    Post subject: Re: Still failing Reply with quote

eu47 wrote:
After applying pingtoo's hint it still fails.

I did not find a way to attach the logs in the post, so I put shared them here:

https://drive.google.com/drive/folders/1ig1PO85yLz_PGuzz_hgtI7KMXBDZpfv-?usp=sharing


How did you populate ebuild repository "rpi"?

cross-aarch64-unknown-linux-gnu-glibc.log wrote:
emerge: there are no ebuilds to satisfy "virtual/awk".
(dependency required by "cross-aarch64-unknown-linux-gnu/glibc-2.33-r14::rpi" [ebuild])
(dependency required by "cross-aarch64-unknown-linux-gnu/glibc" [argument])

The reference to "virtual/awk" came from the "rpi" ebuild repo.
Back to top
View user's profile Send private message
eu47
n00b
n00b


Joined: 06 May 2022
Posts: 16
Location: Edmonton, Canada

PostPosted: Sun Apr 30, 2023 5:51 pm    Post subject: /var/db/repos/rpi Reply with quote

/var/db/repos/rpi is initially an empty directory.

At the end of this command:

Code:
crossdev -P -v -t aarch64-unknown-linux-gnu --init-target -oO /var/db/repos/rpi


it is populated with whatever the result it generates. That is automatic, without my intervention.

cross-aarch64-unknown-linux/binutils
cross-aarch64-unknown-linux/gcc
cross-aarch64-unknown-linux/gdb
cross-aarch64-unknown-linux/glibc
cross-aarch64-unknown-linux/linux headers
metadata/layout.conf
packages
profiles/categories
profiles/repo_name

All the files in cross-aarch64-unknown-linux are symlinks to /var/db/repos/gentoo, which is the machine's portage repo.


Last edited by eu47 on Sun Apr 30, 2023 6:13 pm; edited 1 time in total
Back to top
View user's profile Send private message
eu47
n00b
n00b


Joined: 06 May 2022
Posts: 16
Location: Edmonton, Canada

PostPosted: Sun Apr 30, 2023 6:02 pm    Post subject: crossdev build chain Reply with quote

I should mention a few more things:

1. This started to happen a few months ago, probably around the time when app-alternatives concept was introduced.
2. I keep regular backups, so I have a (still) working crossdev chain based on gcc-12.2.0 and glibc-2.33-r14.
3. Each time I'm trying to update to a newer glibc version, the build fails. For example, I just updated to gcc-13 by using

Code:
crossdev --target aarch64-unknown-linux-gnu -oO /var/db/repos/rpi --gcc 13 --libc 2.33-r14 -s4


If I remove "--libc 2.33-r14" from above command (which means I'm also trying to update glibc), the build fails.
Back to top
View user's profile Send private message
pingtoo
l33t
l33t


Joined: 10 Sep 2021
Posts: 926
Location: Richmond Hill, Canada

PostPosted: Sun Apr 30, 2023 6:16 pm    Post subject: Re: crossdev build chain Reply with quote

eu47 wrote:
I should mention a few more things:

1. This started to happen a few months ago, probably around the time when app-alternatives concept was introduced.
2. I keep regular backups, so I have a (still) working crossdev chain based on gcc-12.2.0 and glibc-2.33-r14.
3. Each time I'm trying to update to a newer glibc version, the build fails. For example, I just updated to gcc-13 by using

crossdev --target aarch64-unknown-linux-gnu -oO /var/db/repos/rpi --gcc 13 --libc 2.33-r14 -s4

If I remove "--libc 2.33-r14" from above command (which means I'm also trying to update glibc), the build fails.


Your "cross-aarch64-unknown-linux-gnu-glibc.log" only content very few lines of message. if the cross-aarch64-unknown-linux/glibc is symbolic point to glibc-2.33-r14.ebuild::gentoo than we need to understand why your ::gentoo repository have reference to virtual/awk. On my ::gentoo repository which is git base was synced on 04/16.2023 which does not have the reference to virtual/awk.

Do you update your ::gentoo recent?

Do you have to use glibc-2.33? can you use more recent version? i.e. 2-36-r7?
Back to top
View user's profile Send private message
eu47
n00b
n00b


Joined: 06 May 2022
Posts: 16
Location: Edmonton, Canada

PostPosted: Sun Apr 30, 2023 8:29 pm    Post subject: Re: crossdev build chain Reply with quote

pingtoo wrote:

Your "cross-aarch64-unknown-linux-gnu-glibc.log" only content very few lines of message. if the cross-aarch64-unknown-linux/glibc is symbolic point to glibc-2.33-r14.ebuild::gentoo than we need to understand why your ::gentoo repository have reference to virtual/awk. On my ::gentoo repository which is git base was synced on 04/16.2023 which does not have the reference to virtual/awk.

Do you update your ::gentoo recent?

Do you have to use glibc-2.33? can you use more recent version? i.e. 2-36-r7?

Your answer made me purge /var/db/repos and update ::gentoo. Then I tried to rebuild the crossdev chain with:
Code:
crossdev --target aarch64-unknown-linux-gnu -oO /var/db/repos/rpi -s4

which failed at cross-gcc-stage2, after successful build of glibc.
Then I stepped back and built successfully with
Code:
crossdev --target aarch64-unknown-linux-gnu -oO /var/db/repos/rpi --libc 2.33-r14 -s4

At this point I made a backup, then I repeated the same as above with newer versions of glibc.
Any glibc version from 2.34 to 2.37 builds successfully, but afterwards cross-gcc-stage2 fails.

https://drive.google.com/drive/folders/1ig1PO85yLz_PGuzz_hgtI7KMXBDZpfv-?usp=share_link
Back to top
View user's profile Send private message
pingtoo
l33t
l33t


Joined: 10 Sep 2021
Posts: 926
Location: Richmond Hill, Canada

PostPosted: Sun Apr 30, 2023 9:17 pm    Post subject: Reply with quote

eu47,

You really should try app-text/wgetpaste. Using google drive is pain.

Not sure why you want to use gcc-13. since it is not stable.

cross-gcc-stage2 is using first stage to build final gcc with all the front end tool. it also a verify that stage1 build is really useful. I guess in your case it prove it is not stable yet. so it is not good to use for cross compile.
Back to top
View user's profile Send private message
eu47
n00b
n00b


Joined: 06 May 2022
Posts: 16
Location: Edmonton, Canada

PostPosted: Sun Apr 30, 2023 9:27 pm    Post subject: Reply with quote

I understand.
But even gcc-12 fails to build at stage 2 with glibc > 2.33.
I can't figure out if it's a gcc problem or glibc one.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 21631

PostPosted: Sun Apr 30, 2023 9:59 pm    Post subject: Reply with quote

If you still need help, please pastebin the complete build log. As I stated above, and as pingtoo alluded to recently, Google Drive is not a proper way to share files, because it cannot properly share files or even their names.
Back to top
View user's profile Send private message
eu47
n00b
n00b


Joined: 06 May 2022
Posts: 16
Location: Edmonton, Canada

PostPosted: Sun Apr 30, 2023 11:08 pm    Post subject: Reply with quote

I'll give up for now and stick to crossdev with gcc-12 and glibc-2.33, which works well.

I tried a pastebin.com free account but it only allows for 512kB pastes and the full log has 2.7MB.

A few extracts related to errors are shown below:

Code:
Checking multilib configuration for libgomp...
mkdir -p -- aarch64-unknown-linux-gnu/libgomp
Configuring in aarch64-unknown-linux-gnu/libgomp
configure: creating cache ./config.cache
...
...
configure: error: Pthreads are required to build libgomp
make[1]: *** [Makefile:17133: configure-target-libgomp] Error 1
make[1]: Leaving directory '/tmp/alien/portage/cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1/work/build'
make[1]: *** Waiting for unfinished jobs....
make[1]: Entering directory '/tmp/alien/portage/cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1/work/build'
Checking multilib configuration for libbacktrace...
...
...
make[1]: Leaving directory '/tmp/alien/portage/cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1/work/build'
make[1]: Entering directory '/tmp/alien/portage/cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1/work/build'
Checking multilib configuration for libatomic...
mkdir -p -- aarch64-unknown-linux-gnu/libatomic
Configuring in aarch64-unknown-linux-gnu/libatomic
...
...
configure: error: Pthreads are required to build libatomic
make[1]: *** [Makefile:18098: configure-target-libatomic] Error 1
make[1]: Leaving directory '/tmp/alien/portage/cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1/work/build'
make: *** [Makefile:1041: all] Error 2
 * ERROR: cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1::rpi failed (compile phase):
 *   emake failed

All this is happening when crossdev uses any glibc >= 2.34.
glibc-2.33 works perfect
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 21631

PostPosted: Mon May 01, 2023 1:01 am    Post subject: Reply with quote

None of that output shows any errors that we can use. At best, it confirms to us that pthreads cannot be used, but does not say why. See config.log for details.
Back to top
View user's profile Send private message
eu47
n00b
n00b


Joined: 06 May 2022
Posts: 16
Location: Edmonton, Canada

PostPosted: Mon May 01, 2023 1:28 am    Post subject: Reply with quote

Wondering if I should report a bug on crossdev related to this.
Maybe the issue can be replicated by others who could find the problem.
Back to top
View user's profile Send private message
axl
Veteran
Veteran


Joined: 11 Oct 2002
Posts: 1144
Location: Romania

PostPosted: Mon May 01, 2023 2:16 am    Post subject: Reply with quote

eu47 wrote:
I'll give up for now and stick to crossdev with gcc-12 and glibc-2.33, which works well.

I tried a pastebin.com free account but it only allows for 512kB pastes and the full log has 2.7MB.

A few extracts related to errors are shown below:

Code:
Checking multilib configuration for libgomp...
mkdir -p -- aarch64-unknown-linux-gnu/libgomp
Configuring in aarch64-unknown-linux-gnu/libgomp
configure: creating cache ./config.cache
...
...
configure: error: Pthreads are required to build libgomp
make[1]: *** [Makefile:17133: configure-target-libgomp] Error 1
make[1]: Leaving directory '/tmp/alien/portage/cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1/work/build'
make[1]: *** Waiting for unfinished jobs....
make[1]: Entering directory '/tmp/alien/portage/cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1/work/build'
Checking multilib configuration for libbacktrace...
...
...
make[1]: Leaving directory '/tmp/alien/portage/cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1/work/build'
make[1]: Entering directory '/tmp/alien/portage/cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1/work/build'
Checking multilib configuration for libatomic...
mkdir -p -- aarch64-unknown-linux-gnu/libatomic
Configuring in aarch64-unknown-linux-gnu/libatomic
...
...
configure: error: Pthreads are required to build libatomic
make[1]: *** [Makefile:18098: configure-target-libatomic] Error 1
make[1]: Leaving directory '/tmp/alien/portage/cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1/work/build'
make: *** [Makefile:1041: all] Error 2
 * ERROR: cross-aarch64-unknown-linux-gnu/gcc-13.1.0-r1::rpi failed (compile phase):
 *   emake failed

All this is happening when crossdev uses any glibc >= 2.34.
glibc-2.33 works perfect


I seen that pthread error before, usually due to lack of static-libs useflag on glibc.
Back to top
View user's profile Send private message
eu47
n00b
n00b


Joined: 06 May 2022
Posts: 16
Location: Edmonton, Canada

PostPosted: Mon May 01, 2023 7:03 am    Post subject: Reply with quote

axl wrote:

I seen that pthread error before, usually due to lack of static-libs useflag on glibc.

glibc on host is built with static-libs.

For the target glibc, I put static-libs useflag in two places:
- /usr/aarch64-unknown-linux-gnu/etc/portage/package.use/glibc
- /etc/portage/package.use/aarch64-unknown-linux-gnu

The last one is overwritten at build time by crossdev, which removes that use flag and replaces it with:
Code:
aarch64-unknown-linux-gnu/glibc -selinux -libraries -multilib

It does not seem to be helping.
Back to top
View user's profile Send private message
axl
Veteran
Veteran


Joined: 11 Oct 2002
Posts: 1144
Location: Romania

PostPosted: Mon May 01, 2023 6:40 pm    Post subject: Reply with quote

eu47 wrote:
glibc on host is built with static-libs


Not on host. On guest. Now granted, there's this component on where the guest is. I'm used that the guest is on /usr/$ARCH. And even then, there are caveats. I use cross-emerge, not crossdev stage4. so there are caveats.

Again, if you want sys-devel/gcc with useflag cxx (on guest), then you will need glibc with static-libs on guest as well.
Back to top
View user's profile Send private message
axl
Veteran
Veteran


Joined: 11 Oct 2002
Posts: 1144
Location: Romania

PostPosted: Mon May 01, 2023 6:49 pm    Post subject: Reply with quote

I use mine as such:

Code:
ARCH="arm64" CHOST=aarch64-unknown-linux-gnu CBUILD=aarch64-unknown-linux-gnu ROOT=/mnt/blanc/rootfs/pi/ PKG_CONFIG_PATH="${ROOT}/usr/lib/pkgconfig/" SYSROOT=${ROOT} cross-emerge -NuavD --with-bdeps=y -j4 @world


few things to note here.

first of all, I have 3 different linuces here. the one in /. the one in /usr/aarch64-unknown-linux-gnu and the one in /mnt/blanc/rootfs/pi/. 1 builds 2. 2 builds 3.

in my case 1 & 2 are both aarch64. but if there's another type of computer, like amd64 to arm64, thats' where it will be. an amd64 has a arm64 build in /usr/aarch64-unknown-linux-gnu. and that is the basis to construct a 3rd gentoo.

at least that's how I use it... I use cross emerge, not crossdev. I used crossdev when I started. Now I use cross-emerge.

PS just to cheatsheet, CBUILD is x86_64-pc-linux-gnu when you are on an intel (like) machine.
Back to top
View user's profile Send private message
eu47
n00b
n00b


Joined: 06 May 2022
Posts: 16
Location: Edmonton, Canada

PostPosted: Mon May 01, 2023 10:26 pm    Post subject: Reply with quote

Interesting. You may have your reasons to have that third linux in /mnt/blanc/rootfs/pi.

I only need your second linux, that in /usr/aarch64-unknown-linux-gnu.
It is used with distcc to help with compilation on the stronger machine of ebuilds on the Raspberry Pi.
So, I do not need to build a full system with cross-emerge @world.

For now it works well with gcc-12.2.1 and glibc-2.36.
I was able to get here with:
Code:
crossdev -P -v -t aarch64-unknown-linux-gnu --init-target -oO /var/db/repos/rpi
crossdev --target aarch64-unknown-linux-gnu -oO /var/db/repos/rpi --gcc 12 --libc 2.33-r14 -s4
crossdev --target aarch64-unknown-linux-gnu -oO /var/db/repos/rpi --gcc 12 --libc 2.36-r7 -s3

If I skip the second command and go directly to the third, the build fails at cross-gcc-stage2.

It's just frustrating that crossdev overwrites automatically at build time any useflags that are put in host's /etc/portage and fully ignores anything put in /usr/cross-aarch64-unknown-linux-gnu/etc/portage. I wonder what's the point of creating /etc/portage in target.

As such, crossdev developers decided to build everything in a certain way, precludes use of static-libs in the target and it's somewhat against Gentoo philosophy of giving the users full liberty to build their system as desired.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 21631

PostPosted: Tue May 02, 2023 1:02 am    Post subject: Reply with quote

The error messages in this thread seem to match [Crossdev] Pthreads are required to build libgomp at cross-gcc-stage2.

From reading the crossdev script, it looks like there are ways to preserve USE, but yes, it does tend to reset things when used, presumably because some cross-targets are so fragile that this was the only way to get them working. Once the cross build has succeeded far enough for you to use emerge cross-arch..., you can customize the USE flags, cease use of crossdev, and maintain the packages via emerge.
Back to top
View user's profile Send private message
eu47
n00b
n00b


Joined: 06 May 2022
Posts: 16
Location: Edmonton, Canada

PostPosted: Tue May 02, 2023 1:12 pm    Post subject: Reply with quote

I have a complete --stage4 cross build.
/usr/aarch64-unknown-linux-gnu/etc/portage/make.conf has
Code:
CHOST=aarch64-unknown-linux-gnu
CBUILD=x86_64-pc-linux-gnu

This minimal toolchain works perfectly fine in combination with distcc for Raspberry Pi 4.

But any cross-emerge fails at configuration stage with:
Code:
configure: error: C compiler cannot create executables

axl was able to build his third linux with cross-emerge because both his host and target (guest) are arm64 systems, as he mentioned.

In my case the host machine architecture is amd64 and the target toolchain is arm64 (aarch64).
Unless there's another explanation, I think that's why guest's gcc cannot create executables when run on the host with cross-emerge.
It creates executables just fine when running on RPI (arm64 system) with distcc: the RPI sends compilation tasks through distcc to the amd64 machine, which runs its aarch64-unknown-linux-gnu-gcc and its resulting machine code is then sent back to the RPI which links it into an aarch64 executable.

For me, there would be one reason to use cross-emerge on the amd64 host: building an aarch64 llvm/clang to help with clang+distcc compilation on the RPI. Unfortunately, it looks like I can't do that, so I'm limited to gcc+distcc.
Back to top
View user's profile Send private message
axl
Veteran
Veteran


Joined: 11 Oct 2002
Posts: 1144
Location: Romania

PostPosted: Wed May 03, 2023 2:14 am    Post subject: Reply with quote

eu47 wrote:
But any cross-emerge fails at configuration stage with:
Code:
configure: error: C compiler cannot create executables

axl was able to build his third linux with cross-emerge because both his host and target (guest) are arm64 systems, as he mentioned.

In my case the host machine architecture is amd64 and the target toolchain is arm64 (aarch64).
Unless there's another explanation, I think that's why guest's gcc cannot create executables when run on the host with cross-emerge.
It creates executables just fine when running on RPI (arm64 system) with distcc: the RPI sends compilation tasks through distcc to the amd64 machine, which runs its aarch64-unknown-linux-gnu-gcc and its resulting machine code is then sent back to the RPI which links it into an aarch64 executable.

For me, there would be one reason to use cross-emerge on the amd64 host: building an aarch64 llvm/clang to help with clang+distcc compilation on the RPI. Unfortunately, it looks like I can't do that, so I'm limited to gcc+distcc.


I didn't start with both systems being arm. I started just like you, building for PI on intel. It should work. I assume there is something else at fault here. NOT the different arch. And even that is not a problem as you can run cross platform through qemu-static.

I suggest you take a look in config.log to see what the actual error was.
Back to top
View user's profile Send private message
eu47
n00b
n00b


Joined: 06 May 2022
Posts: 16
Location: Edmonton, Canada

PostPosted: Wed May 03, 2023 7:06 am    Post subject: Reply with quote

Thanks axl.
The error was setting -march=native, which was not recognized by aarch64-unknown-linux-gnu-gcc.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo on ARM All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum