Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Configuring crossdev for PPC64
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Gentoo on PPC
View previous topic :: View next topic  
Author Message
oneghost
n00b
n00b


Joined: 23 May 2015
Posts: 13

PostPosted: Fri Oct 20, 2023 8:46 am    Post subject: Configuring crossdev for PPC64 Reply with quote

Hello. Im trying to make a Gentoo installation in Virtual Machine on M1, that would serve only one purpose - helping compiling stuff for the PS3 console (PowerPC64 Big Endian CPU).In order to do this, I made a bash script that installs and configures Gentoo on this Host machine (M1/Arm64 VM).It contains crossdev with overlay configured for PowerPC64, Distccd, and few other tools.Distcc works correctly already, but I wanted to do something more - create a crossdev configuration, which would contain the exact same make.conf as the PS3, and then compile all packages into binpkg packages, so that I can install them directly on the PS3 without compiling on the PS3. This includes only work for PPC64BE part of the CPU, no need to touch SPU's from the CELL CPU.
I have problem with compiling packages on the host. Most of packages works correctly, but unfortunately Im unable to compile everything in @ world, and wanted to see if this could be somehow overcome.
What I do after Gentoo is installed:

1 Create crossdev overlay files, according to gentoo tutorial at https://wiki.gentoo.org/wiki/Crossdev
Code:
/var/db/repos/crossdev/metadata/layout.conf
masters = gentoo
thin-manifests = true


Code:
/etc/portage/repos.conf/crossdev.conf
[crossdev]
location = /var/db/repos/crossdev
priority = 10
masters = gentoo
auto-sync = no


2 Create a crossdev environment on the host
Code:
crossdev --b '~2.40' --g '~13.2.1_p20230826' --k '~6.5' --l '~2.37' -t powerpc64-unknown-linux-gnu --abis altivec

3 Select profile inside crossdev:
Code:
PORTAGE_CONFIGROOT=/usr/powerpc64-unknown-linux-gnu eselect profile set 1

4 Copy the elements of PS3 make.conf and package.use, package.accept_keywords into crossdev
Code:
/usr/powerpc64-unknown-linux-gnu/etc/portage/make.conf

CHOST=powerpc64-unknown-linux-gnu
CBUILD=aarch64-unknown-linux-gnu
ROOT=/usr/${CHOST}/
ACCEPT_KEYWORDS="${ARCH}"
ACCEPT_LICENSE="*"
USE="ps3 zeroconf mdnsresponder-compat"
VIDEO_CARDS=""
COMMON_FLAGS="-O2 -pipe -mcpu=cell -mtune=cell -mabi=altivec -maltivec -mno-string -mno-update -mno-multiple"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}"
MAKEOPTS="-j6"
FEATURES="-collision-protect sandbox buildpkg"
PKGDIR=${ROOT}var/cache/binpkgs/
PORTAGE_TMPDIR=${ROOT}tmp/
PKG_CONFIG_PATH="${ROOT}usr/lib/pkgconfig/"


5 Install @system in crossdev
Code:
powerpc64-unknown-linux-gnu-emerge -uva --keep-going @system


6 Change profile to desktop
Code:
PORTAGE_CONFIGROOT=/usr/powerpc64-unknown-linux-gnu eselect profile set 2


7 Trying to recompile world with
Code:
powerpc64-unknown-linux-gnu-emerge --newuse --update --deep @world


Code:
* Error: circular dependencies:

(net-print/cups-2.4.7-r1:0/0::gentoo, ebuild scheduled for merge to '/usr/powerpc64-unknown-linux-gnu/') depends on
 (net-dns/avahi-0.8-r7:0/0::gentoo, ebuild scheduled for merge to '/usr/powerpc64-unknown-linux-gnu/') (buildtime)
  (x11-libs/gtk+-3.24.38:3/3::gentoo, ebuild scheduled for merge to '/usr/powerpc64-unknown-linux-gnu/') (buildtime)
   (net-print/cups-2.4.7-r1:0/0::gentoo, ebuild scheduled for merge to '/usr/powerpc64-unknown-linux-gnu/') (buildtime)

It might be possible to break this cycle
by applying any of the following changes:
- x11-libs/gtk+-3.24.38 (Change USE: -cups)
- net-print/cups-2.4.7-r1 (Change USE: -zeroconf)
- net-dns/avahi-0.8-r7 (Change USE: -gtk)

Note that this change can be reverted, once the package has been installed.

The following USE changes are necessary to proceed:
 (see "package.use" in the portage(5) man page for more details)
# required by x11-libs/gtk+-3.24.38::gentoo
# required by x11-themes/adwaita-icon-theme-44.0::gentoo
>=x11-libs/cairo-1.17.8 X
# required by media-libs/mesa-23.1.8::gentoo
# required by media-libs/libepoxy-1.5.10-r2::gentoo[egl]
# required by x11-libs/gtk+-3.24.38::gentoo
# required by x11-themes/adwaita-icon-theme-44.0::gentoo
>=media-libs/libglvnd-1.6.0 X
# required by x11-misc/xdg-utils-1.1.3_p20210805-r1::gentoo
# required by net-print/cups-2.4.7-r1::gentoo[X]
# required by app-text/ghostscript-gpl-10.02.0::gentoo[cups]
# required by net-print/cups-filters-1.28.17-r2::gentoo[postscript]
>=app-text/xmlto-0.0.28-r11 text


Notice that it asks me to add X to few packages, but the profile already contains X flag, so this should not be the issue. I think it might for some reason confuse the host profile and crossdev profile, because host does not use X. I can confirm that X is used with emerge-powerpc64-unknown-linux-gnu --info:
Code:
Portage 3.0.51 (python 3.11.6-final-0, default/linux/ppc64/17.0/desktop, gcc-13, glibc-2.37-r7, 6.1.57-gentoo-dist aarch64)
=================================================================
System uname: Linux-6.1.57-gentoo-dist-aarch64-with-glibc2.37
KiB Mem:     8127616 total,   1286644 free
KiB Swap:          0 total,         0 free
Timestamp of repository gentoo: Fri, 20 Oct 2023 07:30:01 +0000
Head commit of repository gentoo: 6704122c393dc08a9f62984be3238edb77c152a9
sh bash 5.1_p16-r6
ld GNU ld (Gentoo 2.40 p7) 2.40.0
distcc 3.4 aarch64-unknown-linux-gnu [disabled]
app-misc/pax-utils:        1.3.7::gentoo
app-shells/bash:           5.1_p16-r6::gentoo
dev-lang/python:           3.11.5::gentoo
sys-apps/baselayout:       2.14::gentoo
sys-apps/openrc:           0.48::gentoo
sys-apps/sandbox:          2.38::gentoo
sys-devel/binutils:        2.40-r5::gentoo
sys-devel/binutils-config: 5.5::gentoo
sys-devel/gcc:             13.2.1_p20230826::gentoo
sys-devel/gcc-config:      2.11::gentoo
sys-devel/make:            4.4.1-r1::gentoo
sys-kernel/linux-headers:  6.5-r1::gentoo (virtual/os-headers)
sys-libs/glibc:            2.37-r7::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000
    volatile: False
    sync-rsync-verify-metamanifest: yes
    sync-rsync-extra-opts:
    sync-rsync-verify-max-age: 24
    sync-rsync-verify-jobs: 1

ACCEPT_KEYWORDS="ppc64"
ACCEPT_LICENSE="*"
CBUILD="aarch64-unknown-linux-gnu"
CFLAGS="-O2 -pipe -mcpu=cell -mtune=cell -mabi=altivec -maltivec -mno-string -mno-update -mno-multiple"
CHOST="powerpc64-unknown-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe -mcpu=cell -mtune=cell -mabi=altivec -maltivec -mno-string -mno-update -mno-multiple"
DISTDIR="/var/cache/distfiles"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-O2 -pipe -mcpu=cell -mtune=cell -mabi=altivec -maltivec -mno-string -mno-update -mno-multiple"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe -mcpu=cell -mtune=cell -mabi=altivec -maltivec -mno-string -mno-update -mno-multiple"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LEX="flex"
MAKEOPTS="-j6"
PKGDIR="/usr/powerpc64-unknown-linux-gnu/var/cache/binpkgs/"
PORTAGE_CONFIGROOT="/usr/powerpc64-unknown-linux-gnu/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/usr/powerpc64-unknown-linux-gnu/tmp/"
SHELL="/bin/bash"
USE="X a52 aac acl alsa big-endian bluetooth branding bzip2 cairo cdda cdr cli crypt cups dbus dri dts dvd dvdr elogind encode exif flac fortran gdbm gif gpm gtk gui ibm iconv icu ipv6 jpeg lcms libnotify mad mdnsresponder-compat mng mp3 mp4 mpeg ncurses nls nptl ogg opengl openmp pam pango pcre pdf png policykit ppc64 ppds ps3 qt5 readline sdl seccomp sound spell split-usr ssl startup-notification svg test-rust tiff truetype udev udisks unicode upower usb vorbis vulkan wxwidgets x264 xattr xcb xft xml xv xvid zeroconf zlib" ADA_TARGET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_PPC="altivec" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-1" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_11" PYTHON_TARGETS="python3_11" RUBY_TARGETS="ruby31" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS

So I have no idea how to fix that. The same make.conf and profile on PS3 doesn't complain about these flags, so it is probably related to wrong usage or configuration of crossdev.


One more thing: Adding these flags:
Code:
# required by x11-libs/gtk+-3.24.38::gentoo
# required by x11-themes/adwaita-icon-theme-44.0::gentoo
>=x11-libs/cairo-1.17.8 X
# required by media-libs/mesa-23.1.8::gentoo
# required by media-libs/libepoxy-1.5.10-r2::gentoo[egl]
# required by x11-libs/gtk+-3.24.38::gentoo
# required by x11-themes/adwaita-icon-theme-44.0::gentoo
>=media-libs/libglvnd-1.6.0 X
# required by x11-misc/xdg-utils-1.1.3_p20210805-r1::gentoo
# required by net-print/cups-2.4.7-r1::gentoo[X]
# required by app-text/ghostscript-gpl-10.02.0::gentoo[cups]
# required by net-print/cups-filters-1.28.17-r2::gentoo[postscript]
>=app-text/xmlto-0.0.28-r11 text

To /usr/powerpc64-unknown-linux-gnu/etc/portage/package.use/env
Doesn't solve that warnings.
Strangely adding them to host configuration = /etc/portage gets rid of them, so I think, that for some reason, crossdev tries to read host portage configuration and not crossdev config. I tried adding PORTAGE_CONFIGROOT=/usr/powerpc64-unknown-linux-gnu to command, but it doesn't change anything.
Back to top
View user's profile Send private message
Chewi
Developer
Developer


Joined: 01 Sep 2003
Posts: 886
Location: Edinburgh, Scotland

PostPosted: Fri Oct 20, 2023 2:28 pm    Post subject: Reply with quote

You're right that this doesn't make sense. I don't tend to see things like this because my build system is a desktop. The only plausible explanations I can come up with is it's a poorly-specified BDEPEND somewhere (a dependency to install to the host) or a Portage bug. It's not that long ago that I fixed a Portage bug where it got confused between the dependencies needed in / vs ROOT.

If it's BDEPEND, seeing exactly which of these packages end up in / may give a clue.
Back to top
View user's profile Send private message
oneghost
n00b
n00b


Joined: 23 May 2015
Posts: 13

PostPosted: Fri Oct 20, 2023 3:44 pm    Post subject: Reply with quote

Chewi wrote:
You're right that this doesn't make sense. I don't tend to see things like this because my build system is a desktop. The only plausible explanations I can come up with is it's a poorly-specified BDEPEND somewhere (a dependency to install to the host) or a Portage bug. It's not that long ago that I fixed a Portage bug where it got confused between the dependencies needed in / vs ROOT.

If it's BDEPEND, seeing exactly which of these packages end up in / may give a clue.


Thank you for the replay. Im not sure, what do you mean by seeing which of packages ends up in /. You mean if I emerge something in crossdev and it gets installed in host instead?
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54247
Location: 56N 3W

PostPosted: Fri Oct 20, 2023 4:14 pm    Post subject: Reply with quote

oneghost,

Try
Code:
USE=X powerpc64-unknown-linux-gnu-emerge --newuse --update --deep --pretend --tree @world


The output will list packages to be installed to /usr/powerpc64-unknown-linux-gnu/ which is your target root and to / which is the build host root.
The latter are built time dependencies.
They are the packages of interest.

Put the command and all of the output onto a pastbin please.

For the circular dependencies

Code:
It might be possible to break this cycle
by applying any of the following changes:
- x11-libs/gtk+-3.24.38 (Change USE: -cups)
- net-print/cups-2.4.7-r1 (Change USE: -zeroconf)
- net-dns/avahi-0.8-r7 (Change USE: -gtk)



Code:
USE: -gtk powerpc64-unknown-linux-gnu-emerge -1av net-dns/avahi
may be a good choice.
Make sure that it will be installed into /usr/powerpc64-unknown-linux-gnu/ before you do it.

You are only a small step from building in a QEMU chroot too.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
oneghost
n00b
n00b


Joined: 23 May 2015
Posts: 13

PostPosted: Fri Oct 20, 2023 4:58 pm    Post subject: Reply with quote

Oh ok, I think Im starting to understand now. So the issue is that crossdev is not totally separated from host, in terms of BDEPEND dependencies, and these needs to be meet for the system to be able to cross-compile packages for my crossdev. Do I understand that correctly? So I should be able to solve these dependencies on host first and not on the crossdev environment?
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 21635

PostPosted: Fri Oct 20, 2023 5:02 pm    Post subject: Reply with quote

BDEPEND is explicitly for dependencies that must be present on the build machine (hence "B" DEPEND; therefore, yes, these are on the host), usually because the build system expects to run them as a part of its build. DEPEND is for dependencies that must be present for the target at build time, such as static libraries or header files (therefore, these are installed in the cross-root). RDEPEND is for dependencies that must be present on the target at runtime, such as programs it will run or shared libraries it will load.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54247
Location: 56N 3W

PostPosted: Fri Oct 20, 2023 5:14 pm    Post subject: Reply with quote

oneghost,

That's correct. Your target root is for an entirely different arch that your build host.
Build Time dependencies (BDEPEND), e.g. make must execute on the host so they need to be installed there. They may be in the target root too but executing any code there on the build host will end badly.

Chewi was suggesting that some of the packages that appear to be BDEPENDs may not be correctly specified in the ebuilds, or it may be a bug in portage.
That's a bug, if its correct. The ebuilds should be fixed.
Its not a frequently tested case. My build hast has a GUI, so, like Chewi, I would never see this issue.

Your system is presently in a state to trigger this odd behaviour.

If you are in a hurry, you can install the BDEPENDs and it will probably all just work.
If you have some time, we can investigate and understand the root cause. That's the first step in fixing it properly for everyone.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
oneghost
n00b
n00b


Joined: 23 May 2015
Posts: 13

PostPosted: Fri Oct 20, 2023 5:25 pm    Post subject: Reply with quote

Sure, I have an gentoo installer script, that easily restores my VM into that state exactly, so I can investigate more if you want to.
I can have separate VM's, one for investigation, and one where I'll try to just meet the dependencies for the host machine.
I might be able to try to move this installation to x86_64 as well and provide a tarball of that system. Right now Im using it on ARM64 as host, but that script can be modified easily for other architectures, just need to make sure all packages are installed correctly, and tinker with package.* flags if needed.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54247
Location: 56N 3W

PostPosted: Fri Oct 20, 2023 7:17 pm    Post subject: Reply with quote

oneghost,

You may need to fix the circular dependencies first.
If you put the command and output of
Code:
USE=X powerpc64-unknown-linux-gnu-emerge --newuse --update --deep --pretend --tree @world
onto a pastebin we would have a few pointers.

As you don't have a GUI, that's
Code:
wgetpaste -c "USE=X powerpc64-unknown-linux-gnu-emerge --newuse --update --deep --pretend --tree @world"
and post the link you get back.
Like everything else, you won't have wgetpaste until you install it.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo on PPC All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum