| View previous topic :: View next topic |
| Author |
Message |
vagran
n00b
Joined: 26 Jan 2019
Posts: 13
|
 Posted: Sat Jan 26, 2019 6:48 am Post subject: Problem with chrooted emerge [SOLVED] |
 |
|
Hello,
I am building a system for an embedded cross-target (arm64 on x86_64 host). I have mostly followed guides from here and here. I have managed to cross-compile most of base system, I have installed qemu and its binfmt handler. I can chroot there, and almost everything works except emerge.
| Code: |
alias emerge-chroot='ROOT=/ CBUILD=aarch64-unknown-linux-gnu HOSTCC=aarch64-unknown-linux-gnu emerge'
# emerge-chroot htop
Calculating dependencies... done!
>>> Verifying ebuild manifests
>>> Emerging (1 of 1) sys-process/htop-2.2.0::gentoo
qemu: qemu_thread_create: Invalid argument
* The ebuild phase 'die_hooks' has been aborted since PORTAGE_BUILDDIR
* does not exist: '/tmp/portage/sys-process/htop-2.2.0'
>>> Failed to emerge sys-process/htop-2.2.0
* Messages for package sys-process/htop-2.2.0:
|
"htop" is just for example, any package fails with the same error. As I understand, some build step silently fails and emerge is unable to find build output directory. I am pretty sure that the problem is not with the "/tmp/portage" directory itself, it's in place, and emerge is creating it if deleted. May be it is related to qemu message "qemu: qemu_thread_create: Invalid argument". However I am unable to reproduce it on any other program in this environment, and have found nothing useful about it. I also tried to manually build some downloaded source packages via configure/make in this environment. They are built and work without problems.
emerge.log:
| Code: |
1548483902: Started emerge on: Jan 26, 2019 06:25:02
1548483902: *** emerge htop
1548483911: >>> emerge (1 of 1) sys-process/htop-2.2.0 to /
1548483911: === (1 of 1) Cleaning (sys-process/htop-2.2.0::/usr/portage/sys-process/htop/htop-2.2.0.ebuild)
1548483913: *** Finished. Cleaning up...
1548483913: *** exiting unsuccessfully with status '1'.
1548483913: *** terminating.
|
protage/elog/summary.log is empty:
| Code: |
>>> Messages generated by process 19041 on 2019-01-26 06:25:12 -00 for package sys-process/htop-2.2.0:
|
So the question is - how to troubleshoot emerge in general? How to understand which exactly build step leaded to "die_hooks" phase and why? I tried to use --debug option but it only dumps dependency graph related information, nothing about what is done in ebuild.
Last edited by vagran on Mon Jan 28, 2019 3:21 pm; edited 1 time in total |
|
| Back to top |
|
 |
vagran
n00b
Joined: 26 Jan 2019
Posts: 13
|
| Posted: Sat Jan 26, 2019 7:05 am Post subject: |
|
|
Some update. Half an hour in the morning is more productive than half a night. I found that running ebuild command manually works and the package is successfully setup, configured, compiled and installed. The directory in /tmp/portage is created without problems. The failing step is "clean" (however it does it job and directory is deleted). Still thinking how to troubleshoot it.
| Code: |
# ROOT=/ CBUILD=aarch64-unknown-linux-gnu HOSTCC=aarch64-unknown-linux-gnu ebuild htop-2.2.0.ebuild clean
qemu: qemu_thread_create: Invalid argument
* The ebuild phase 'die_hooks' has been aborted since PORTAGE_BUILDDIR
* does not exist: '/tmp/portage/sys-process/htop-2.2.0'
|
Adding FEATURES="noclean" does not help with emerge for some reason.
Qemu message appears for other steps from time to time but seems does not affect them. |
|
| Back to top |
|
|
nick_gentoo
Tux's lil' helper
Joined: 07 Jan 2019
Posts: 140
|
| Posted: Sat Jan 26, 2019 7:17 am Post subject: |
|
|
| Quote: | | PORTAGE_BUILDDIR does not exist |
This looks important. PORTAGE_BUILDDIR should normally be /var/tmp/portage - check that it exists and is owned by portage.
(at least on x86, I am not so sure about the arm64 qemu environment) |
|
| Back to top |
|
|
vagran
n00b
Joined: 26 Jan 2019
Posts: 13
|
| Posted: Sat Jan 26, 2019 7:48 am Post subject: |
|
|
In make.conf I have:
PORTAGE_TMPDIR=${ROOT}tmp/
which was set by crossdev.
So it ends up in /tmp/portage which is accessible, and as I said, it works with ebuild.
I there any way to see at least which ebuild commands are being executed by emerge? |
|
| Back to top |
|
|
nick_gentoo
Tux's lil' helper
Joined: 07 Jan 2019
Posts: 140
|
| Posted: Sat Jan 26, 2019 8:33 am Post subject: |
|
|
I missed that, sorry! And now I realize that probably PORTAGE_TMPDIR=/tmp and PORTAGE_BUILDDIR=/tmp/portage.
Could you post the output of emerge --info? It might help to bring some other ideas. |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54232
Location: 56N 3W
|
| Posted: Sat Jan 26, 2019 9:09 am Post subject: |
|
|
vagran,
What are the permissions on the PORTAGE_TMPDIR target?
Portage itself starts as root then drops to portage:portage for all the phases it can. (ebuild won't do that)
Check that portage can read and write PORTAGE_TMPDIR as the portage user.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
vagran
n00b
Joined: 26 Jan 2019
Posts: 13
|
| Posted: Sat Jan 26, 2019 1:26 pm Post subject: |
|
|
| Code: | # emerge --info
Portage 2.3.58 (python 3.6.6-final-0, default/linux/arm64/17.0, gcc-8.2.0, unavailable, 4.14.83-gentoo aarch64)
=================================================================
System uname: Linux-4.14.83-gentoo-aarch64-with-gentoo-2.6
KiB Mem: 8158344 total, 515540 free
KiB Swap: 8388604 total, 8388596 free
Timestamp of repository gentoo: Fri, 25 Jan 2019 18:30:01 +0000
Head commit of repository gentoo: ac194697f14c2298a4b18ab43d811397a59e84d5
sh bash 5.0_p2
ld GNU ld (Gentoo 2.31.1 p5) 2.31.1
Repositories:
gentoo
location: /usr/portage
sync-type: rsync
sync-uri: rsync://rsync.gentoo.org/gentoo-portage
priority: -1000
sync-rsync-verify-metamanifest: yes
sync-rsync-verify-jobs: 1
sync-rsync-extra-opts:
sync-rsync-verify-max-age: 24
ACCEPT_KEYWORDS="arm64 ~arm64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=armv8-a+crc -mtune=cortex-a53 -O2 -pipe -fomit-frame-pointer -fno-stack-protector -U_FORTIFY_SOURCE"
CHOST="aarch64-unknown-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=armv8-a+crc -mtune=cortex-a53 -O2 -pipe -fomit-frame-pointer -fno-stack-protector -U_FORTIFY_SOURCE"
DISTDIR="/usr/portage/distfiles"
ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news noclean parallel-fetch pid-sandbox preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
PKGDIR="/usr/aarch64-unknown-linux-gnu/packages/"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/usr/aarch64-unknown-linux-gnu/tmp/"
USE="arm64 berkdb bzip2 cli crypt cxx dri fortran gdbm iconv ipv6 libtirpc multilib ncurses nls nptl openmp pcre readline seccomp ssl tcpd unicode xattr zlib" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_ARM="edsp neon thumb thumb2 v4 v5 v6 v7 v8 vfp vfp-d32 vfpv3 vfpv4" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6 php7-1" POSTGRES_TARGETS="postgres9_5 postgres10" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" RUBY_TARGETS="ruby21 ruby24" USERLAND="GNU" VIDEO_CARDS="fbdev dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset: CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LINGUAS, MAKEOPTS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
|
| Code: | # ls -l /tmp
total 4
drwxrwxr-x 4 portage portage 4096 Jan 26 13:13 portage
|
| Quote: | | Portage itself starts as root then drops to portage:portage for all the phases it can. (ebuild won't do that) |
BTW when I run ebuild under root it creates /tmp/portage with portage:portage owner.
| Code: | # ROOT=/ CBUILD=aarch64-unknown-linux-gnu HOSTCC=aarch64-unknown-linux-gnu ebuild htop-2.2.0.ebuild --debug clean
qemu: qemu_thread_create: Invalid argument
+ __dyn_clean
+ '[' -z /tmp/portage/sys-process/htop-2.2.0 ']'
+ '[' '!' -d /tmp/portage/sys-process/htop-2.2.0 ']'
+ return 0
+ set +x
* The ebuild phase 'die_hooks' has been aborted since PORTAGE_BUILDDIR
* does not exist: '/tmp/portage/sys-process/htop-2.2.0'
|
|
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54232
Location: 56N 3W
|
| Posted: Sat Jan 26, 2019 2:51 pm Post subject: |
|
|
vagran,
Is that --info from inside your aarch64 chroot?
If so, qemu emulates an arm64 CPU, so
CBUILD="x86_64-pc-linux-gnu" is incorrect.
PKGDIR="/usr/aarch64-unknown-linux-gnu/packages/" looks odd. That's at /usr/aarch64-unknown-linux-gnu/usr/aarch64-unknown-linux-gnu/packages/ from your real amd64 root.
You need a different make.conf when you chroot into a qemu arm64 emulated environment.
Compare https://paste.pound-python.org/show/o1yME8epzb0TMSwC2gBd/ and https://paste.pound-python.org/show/kUUyb5ArI0U7ovsKW54J/
That my two. I have a make.conf symlink to manage to choose between them.
Your gcc version is missing in
| Code: | | Portage 2.3.58 (python 3.6.6-final-0, default/linux/arm64/17.0, gcc-8.2.0, unavailable, 4.14.83-gentoo aarch64) |
I set up the chroot with
| Code: | #!/bin/bash
# run this script to prepare the arm64 chroot
# for QEMU building
# start the binfmt service on the host
# ignore the error if its already running
/etc/init.d/binfmt start
# tell the kernel about aarch64 - the above service should do it
# but it seems not to, sometimes.
echo ':aarch64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xb7:\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff:/usr/bin/qemu-aarch64:' > /proc/sys/fs/binfmt_misc/register
cd /usr/aarch64-unknown-linux-gnu/
# set the right make.conf for QEMU
rm make.conf
ln -s make.conf_qemu make.conf
mount --bind /usr/portage usr/portage
mount --bind /usr/portage/distfiles var/cache/distfiles
# not /usr/portage/packages as they are for the host!
mount --bind /proc proc
mount --bind /sys sys
mount --bind /dev dev
mount --bind /dev/pts dev/pts
# must be after the bind mount of dev
mount tmpfs -t tmpfs -o rw,nosuid,nodev,noexec dev/shm
# don't actually do the chroot |
-- edit -- you get extra points for spotting the bug in that script
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
vagran
n00b
Joined: 26 Jan 2019
Posts: 13
|
| Posted: Sat Jan 26, 2019 3:13 pm Post subject: |
|
|
| Quote: | | Is that --info from inside your aarch64 chroot? |
Yes.
| Quote: | | CBUILD="x86_64-pc-linux-gnu" is incorrect. |
I followed instructions and run emerge as
| Code: | | ROOT=/ CBUILD=aarch64-unknown-linux-gnu HOSTCC=aarch64-unknown-linux-gnu ebuild |
when running in chroot (actually made an alias "emerge-chroot" as in this guide. So I suppose CBUILD is overriden and config swapping is not really needed.
The same story for packages directory, it depends on ROOT.
Actually I have run "emerge --info" without these environment variables, so the compiler is displayed wrong. Here is the correct version:
| Code: | # emerge-chroot --info
Portage 2.3.58 (python 3.6.6-final-0, default/linux/arm64/17.0, gcc-8.2.0, glibc-2.28-r5, 4.14.83-gentoo aarch64)
=================================================================
System uname: Linux-4.14.83-gentoo-aarch64-with-gentoo-2.6
KiB Mem: 8158344 total, 502808 free
KiB Swap: 8388604 total, 8388596 free
Timestamp of repository gentoo: Fri, 25 Jan 2019 18:30:01 +0000
Head commit of repository gentoo: ac194697f14c2298a4b18ab43d811397a59e84d5
sh bash 5.0_p2
ld GNU ld (Gentoo 2.31.1 p5) 2.31.1
app-shells/bash: 5.0_p2::gentoo
dev-lang/python: 2.7.15::gentoo, 3.6.6::gentoo
dev-util/pkgconfig: 0.29.2::gentoo
sys-apps/baselayout: 2.6-r1::gentoo
sys-apps/openrc: 0.40.3::gentoo
sys-apps/sandbox: 2.15::gentoo
sys-devel/binutils: 2.31.1-r3::gentoo
sys-devel/gcc: 8.2.0-r6::gentoo
sys-devel/gcc-config: 2.0::gentoo
sys-devel/make: 4.2.1-r4::gentoo
sys-kernel/linux-headers: 4.20::gentoo (virtual/os-headers)
sys-libs/glibc: 2.28-r5::gentoo
Repositories:
gentoo
location: /usr/portage
sync-type: rsync
sync-uri: rsync://rsync.gentoo.org/gentoo-portage
priority: -1000
sync-rsync-verify-metamanifest: yes
sync-rsync-extra-opts:
sync-rsync-verify-max-age: 24
sync-rsync-verify-jobs: 1
ACCEPT_KEYWORDS="arm64 ~arm64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="aarch64-unknown-linux-gnu"
CFLAGS="-march=armv8-a+crc -mtune=cortex-a53 -O2 -pipe -fomit-frame-pointer -fno-stack-protector -U_FORTIFY_SOURCE"
CHOST="aarch64-unknown-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=armv8-a+crc -mtune=cortex-a53 -O2 -pipe -fomit-frame-pointer -fno-stack-protector -U_FORTIFY_SOURCE"
DISTDIR="/usr/portage/distfiles"
ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news noclean parallel-fetch pid-sandbox preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
PKGDIR="/usr/aarch64-unknown-linux-gnu/packages/"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/usr/aarch64-unknown-linux-gnu/tmp/"
USE="arm64 berkdb bzip2 cli crypt cxx dri fortran gdbm iconv ipv6 libtirpc multilib ncurses nls nptl openmp pcre readline seccomp ssl tcpd unicode xattr zlib" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_ARM="edsp neon thumb thumb2 v4 v5 v6 v7 v8 vfp vfp-d32 vfpv3 vfpv4" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6 php7-1" POSTGRES_TARGETS="postgres9_5 postgres10" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" RUBY_TARGETS="ruby21 ruby24" USERLAND="GNU" VIDEO_CARDS="fbdev dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset: CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LINGUAS, MAKEOPTS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS |
For some reason it still incorrectly shows packages and temporal path, looks like "--info" ignores part of environment, but when running merge, the temporal path is /tmp/portage as it should be according to my make.conf (based on what was generated by crossdev):
| Code: | # cat /etc/portage/make.conf
# Note: profile variables are set/overridden in profile/ files:
# etc/portage/profile/use.force (overrides kernel_* USE variables)
# etc/portage/profile/make.defaults (overrides ARCH, KERNEL, ELIBC variables)
CHOST=aarch64-unknown-linux-gnu
CBUILD=x86_64-pc-linux-gnu
HOSTCC=${CBUILD}-gcc
ROOT=/usr/${CHOST}/
ACCEPT_KEYWORDS="${ARCH} ~${ARCH}"
USE="${ARCH} -pam -acl"
CFLAGS="-march=armv8-a+crc -mtune=cortex-a53 -O2 -pipe -fomit-frame-pointer -fno-stack-protector -U_FORTIFY_SOURCE"
CXXFLAGS="${CFLAGS}"
FEATURES="noclean"
# Be sure we dont overwrite pkgs from another repo..
PKGDIR=${ROOT}packages/
PORTAGE_TMPDIR=${ROOT}tmp/
PKG_CONFIG_PATH="${ROOT}usr/lib/pkgconfig/"
#PORTDIR_OVERLAY="/usr/portage/local/"
|
Now I have tried to set "PKGDIR=/packages/" in this config, but it did not affect the problem. If I correctly understand, this only used if building binary packages which is not what I am doing in chroot. |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54232
Location: 56N 3W
|
| Posted: Sat Jan 26, 2019 3:20 pm Post subject: |
|
|
vagran,
I use this guide but I'm biased. :)
Once in the qemu chroot, emerge will work as normal, but as I say, you need a native make.conf when you use qemu and a cross make.conf when you cross compile.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
vagran
n00b
Joined: 26 Jan 2019
Posts: 13
|
| Posted: Sat Jan 26, 2019 4:31 pm Post subject: |
|
|
And what about question about emerge troubleshooting? Is it really not possible to get fail reason in easy way?
What I am currently doing is debugging it in PDB to get at least exact phase it failed. |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54232
Location: 56N 3W
|
| Posted: Sat Jan 26, 2019 6:10 pm Post subject: |
|
|
vagran,
| Code: | | qemu: qemu_thread_create: Invalid argument |
is the first error.
It looks like qemu isn't happy.
I'm using a statically built qemu-9999 because some syscalls are not yet implemented. The live code is as good as it gets.
A few syscall errors can be ignored but one or two break portage.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
vagran
n00b
Joined: 26 Jan 2019
Posts: 13
|
| Posted: Sat Jan 26, 2019 8:27 pm Post subject: |
|
|
I have installed qemu-9999, unfortunately it didn't help. I also tried to change make.conf to not depend on environment variables, did not help as well.
What I have so far:
- The problem is in "clean" phase called by emerge. Checked this in debugger. The same effect is also achieved by running it manually as "ROOT=/ CBUILD=aarch64-unknown-linux-gnu HOSTCC=aarch64-unknown-linux-gnu ebuild htop-2.2.0.ebuild clean".
- Message about missing PORTAGE_BUILDDIR is normal behaviour in this case. It occurs exactly in case if initial clean fails because the build directory is not yet created, but "die_hooks" phase expects it exists. It's some design flaw of the portage system. It even has corresponding comment in the code:
| Code: | 62 # This can happen if the pre-clean phase triggers
63 # die_hooks for some reason, and PORTAGE_BUILDDIR
64 # doesn't exist yet.
65 if need_builddir and \
66 not os.path.isdir(self.settings['PORTAGE_BUILDDIR']):
67 msg = _("The ebuild phase '%s' has been aborted "
68 "since PORTAGE_BUILDDIR does not exist: '%s'") % \
69 (self.phase, self.settings['PORTAGE_BUILDDIR']) |
- The phase implementation is call of "/bin/bash -c /usr/lib/portage/python3.6/ebuild.sh clean" command with proper environment set. And this command works well, the script returns zero exit code at the end. But for some reason python spawner thinks it is failed (I've seen some strange code -6 while debugged). |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54232
Location: 56N 3W
|
| Posted: Sat Jan 26, 2019 8:57 pm Post subject: |
|
|
vagran,
My arm64 build root is about a year out of date. I'll update it from my BINHOST and see if I can reproduce this.
Eww ... - dev-util/mdds-9999::gentoo (masked by: EAPI 7) its older than I thought.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
vagran
n00b
Joined: 26 Jan 2019
Posts: 13
|
| Posted: Sun Jan 27, 2019 11:27 am Post subject: |
|
|
So far I found that child process ("bash -c /usr/lib/portage/python3.6/ebuild.sh clean") returns this result to ebuild:
posix.waitid_result(si_pid=21843, si_uid=0, si_signo=17, si_status=6, si_code=2)
It basically means it was kill by SIGABRT. As found before, the script ebuild.sh itself completes successfully so it probably happens after the script is executed. I suspect qemu crash. Will proceed with obtaining core dump. |
|
| Back to top |
|
|
vagran
n00b
Joined: 26 Jan 2019
Posts: 13
|
| Posted: Sun Jan 27, 2019 11:46 am Post subject: |
|
|
And seems that's really qemu problem:
| Code: | # file tmp/core-python3.6m-6-1548588223
tmp/core-python3.6m-6-1548588223: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from '/usr/bin/qemu-aarch64 /usr/bin/python3.6m /usr/lib/portage/python3.6/pid-ns-ini', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/usr/bin/python3.6m', platform: 'x86_64' |
As I understand, executable is not bash because ebuild spawns it by fork()/exec() so it inherits parent command line. The trace corresponds to the error message seen before.
| Code: | (gdb) bt
#0 0x00000000601f1bcb in raise ()
#1 0x00000000601f2111 in abort ()
#2 0x000000006000796e in error_exit ()
#3 0x0000000060174da0 in qemu_thread_create ()
#4 0x000000006017b649 in rcu_init_complete ()
#5 0x00000000601e04f4 in __libc_csu_init ()
#6 0x00000000601dfd1d in __libc_start_main ()
#7 0x000000006000902a in _start () |
|
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54232
Location: 56N 3W
|
| Posted: Sun Jan 27, 2019 12:17 pm Post subject: |
|
|
vagran,
| Code: | qemu: qemu_thread_create: Invalid argument
* The ebuild phase 'die_hooks' has been aborted since PORTAGE_BUILDDIR
* does not exist: '/var/tmp/portage/._unmerge_/net-
* libs/wvstreams-4.6.1-r5' |
So far, I've only updated 4 packages. My qemu-9999 is over a year old too, so if the problem is in qemu, it being triggered by one of
| Code: | 1548541291: >>> emerge (1 of 4) app-crypt/openpgp-keys-gentoo-release-20190102 to /
1548541299: === (1 of 4) Merging Binary (app-crypt/openpgp-keys-gentoo-release-20190102::/packages/app-crypt/openpgp-keys-gentoo-release-20190102.tbz2)
1548541319: >>> AUTOCLEAN: app-crypt/openpgp-keys-gentoo-release:0
1548541331: === (1 of 4) Post-Build Cleaning (app-crypt/openpgp-keys-gentoo-release-20190102::/packages/app-crypt/openpgp-keys-gentoo-release-20190102.tbz2)
1548541331: ::: completed emerge (1 of 4) app-crypt/openpgp-keys-gentoo-release-20190102 to /
1548541331: >>> emerge (2 of 4) dev-python/bz2file-0.98 to /
1548541332: === (2 of 4) Merging Binary (dev-python/bz2file-0.98::/packages/dev-python/bz2file-0.98.tbz2)
1548541351: >>> AUTOCLEAN: dev-python/bz2file:0
1548541365: === (2 of 4) Post-Build Cleaning (dev-python/bz2file-0.98::/packages/dev-python/bz2file-0.98.tbz2)
1548541365: ::: completed emerge (2 of 4) dev-python/bz2file-0.98 to /
1548541365: >>> emerge (3 of 4) app-portage/gemato-14.0 to /
1548541365: === (3 of 4) Cleaning (app-portage/gemato-14.0::/usr/portage/app-portage/gemato/gemato-14.0.ebuild)
1548541366: === (3 of 4) Compiling/Packaging (app-portage/gemato-14.0::/usr/portage/app-portage/gemato/gemato-14.0.ebuild)
1548541439: === (3 of 4) Merging (app-portage/gemato-14.0::/usr/portage/app-portage/gemato/gemato-14.0.ebuild)
1548541452: >>> AUTOCLEAN: app-portage/gemato:0
1548541467: === (3 of 4) Post-Build Cleaning (app-portage/gemato-14.0::/usr/portage/app-portage/gemato/gemato-14.0.ebuild)
1548541467: ::: completed emerge (3 of 4) app-portage/gemato-14.0 to /
1548541467: >>> emerge (4 of 4) sys-apps/portage-2.3.58 to /
1548541467: === (4 of 4) Cleaning (sys-apps/portage-2.3.58::/usr/portage/sys-apps/portage/portage-2.3.58.ebuild)
1548541467: === (4 of 4) Compiling/Packaging (sys-apps/portage-2.3.58::/usr/portage/sys-apps/portage/portage-2.3.58.ebuild)
1548541649: === (4 of 4) Merging (sys-apps/portage-2.3.58::/usr/portage/sys-apps/portage/portage-2.3.58.ebuild)
1548541689: >>> AUTOCLEAN: sys-apps/portage:0
1548541689: === Unmerging... (sys-apps/portage-2.3.16)
1548541711: >>> unmerge success: sys-apps/portage-2.3.16
1548541727: === (4 of 4) Post-Build Cleaning (sys-apps/portage-2.3.58::/usr/portage/sys-apps/portage/portage-2.3.58.ebuild)
1548541727: ::: completed emerge (4 of 4) sys-apps/portage-2.3.58 to /
1548541727: *** Finished. Cleaning up...
1548541735: *** exiting successfully.
1548541747: *** terminating. |
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
vagran
n00b
Joined: 26 Jan 2019
Posts: 13
|
| Posted: Sun Jan 27, 2019 2:46 pm Post subject: |
|
|
This package list doesn't seem to be really related...
I debugged a bit the core dump.
| Code: | (gdb) bt
#0 0x00000000602951eb in raise ()
#1 0x0000000060295771 in abort ()
#2 0x000000006020b760 in error_exit (err=22, msg=0x603715b0 <__func__.18188> "qemu_thread_create")
at /var/tmp/portage/app-emulation/qemu-3.1.0/work/qemu-3.1.0/util/qemu-thread-posix.c:36
#3 0x000000006020c58c in qemu_thread_create (thread=0x7ffc85db0e20, name=0x603730c9 "call_rcu", start_routine=0x60215011 <call_rcu_thread>, arg=0x0, mode=1)
at /var/tmp/portage/app-emulation/qemu-3.1.0/work/qemu-3.1.0/util/qemu-thread-posix.c:534
#4 0x000000006021538c in rcu_init_complete () at /var/tmp/portage/app-emulation/qemu-3.1.0/work/qemu-3.1.0/util/rcu.c:327
#5 0x00000000602154cc in rcu_init () at /var/tmp/portage/app-emulation/qemu-3.1.0/work/qemu-3.1.0/util/rcu.c:383
#6 0x00000000602836a4 in __libc_csu_init ()
#7 0x0000000060282ecd in __libc_start_main ()
#8 0x0000000060001bfa in _start ()
(gdb) f 3
#3 0x000000006020c58c in qemu_thread_create (thread=0x7ffc85db0e20, name=0x603730c9 "call_rcu", start_routine=0x60215011 <call_rcu_thread>, arg=0x0, mode=1)
at /var/tmp/portage/app-emulation/qemu-3.1.0/work/qemu-3.1.0/util/qemu-thread-posix.c:534
534 error_exit(err, __func__);
(gdb) p mode
$7 = 1 |
The problem is with pthread_create in this line. It returns EINVAL which probably caused by invalid attributes. The attributes are properly initialised earlier and pthread_attr_setdetachstate() was called with PTHREAD_CREATE_DETACHED. I do not really have ideas what to do next. The only guess is that it is somehow related to static linkage and libpthread was somehow broken during this process. I will also try to research live debugging possibility of this place in the qemu code. |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54232
Location: 56N 3W
|
| Posted: Sun Jan 27, 2019 6:09 pm Post subject: |
|
|
vagran,
Well, my chroot used to work until I updated those packages.
With a broken portage, its a bit difficult to downgrade portage to test :)
-- Edit --
However, once you get | Code: | >>> sys-apps/portage-2.3.51-r1 merged.
Unsupported ioctl: cmd=0x5441 |
the qemu chroot works again.
The Unsupported ioctl: cmd=0x5441 is new and I got an Unsupported syscall 285?
Its not in the build logs.
Later portage versions may work too but portage-2.3.58 does not.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
vagran
n00b
Joined: 26 Jan 2019
Posts: 13
|
| Posted: Sun Jan 27, 2019 9:02 pm Post subject: |
|
|
That helped, thanks!
I also checked version 2.3.57 - broken as well. 2.3.51-r1 is the last one working.
I think its subprocess spawning manner somehow triggered the bug in qemu or libpthread. Probably this should be reported somewhere. |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54232
Location: 56N 3W
|
| Posted: Sun Jan 27, 2019 10:29 pm Post subject: |
|
|
vagran,
File a bug at bugs.gentoo.org.
You have a lot more detail than me. I'm not convinced its a portage bug bet if not, qemu will need a test case to be able to fix it.
The portage team will report it upstream if that's what it needs.
-- edit --
| #gentoo-arm wrote: | <NeddySeagoon> Team Portage 2.3.57 and later won't work in a qemu arm64 chroot. https://forums.gentoo.org/viewtopic-t-1092314-highlight-.html Anyone seen this on other qemu chroots?
<mrueg> NeddySeagoon: my suggestion is to play with enabling disabling the sandboxes
<mrueg> start with FEATURES="-pid-sandbox"
<NeddySeagoon> mrueg: Ill try it, Thanks |
-- edit 2 --
| Code: | | FEATURES="-pid-sandbox" |
works here.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
vagran
n00b
Joined: 26 Jan 2019
Posts: 13
|
| Posted: Mon Jan 28, 2019 12:36 pm Post subject: |
|
|
| Works for me as well. Thank you! |
|
| Back to top |
|
|
vagran
n00b
Joined: 26 Jan 2019
Posts: 13
|
| Posted: Mon Jan 28, 2019 12:54 pm Post subject: |
|
|
I also have another problem also for this chrooted configuration. If I attempt to install some python packages, the following error occurs:
(for emerge dev-python/pygments)
| Code: |
* --------------------------- ACCESS VIOLATION SUMMARY ---------------------------
* LOG FILE: "/var/log/sandbox/sandbox-23732.log"
*
VERSION 1.0
FORMAT: F - Function called
FORMAT: S - Access Status
FORMAT: P - Path as passed to function
FORMAT: A - Absolute Path (not canonical)
FORMAT: R - Canonical Path
FORMAT: C - Command Line
F: open_wr
S: deny
P: /usr/lib64/python3.6/lib2to3/Grammar3.6.6.final.0.pickle
A: /usr/lib64/python3.6/lib2to3/Grammar3.6.6.final.0.pickle
R: /usr/lib64/python3.6/lib2to3/Grammar3.6.6.final.0.pickle
C: /usr/bin/python3.6 setup.py build -j 4
F: open_wr
S: deny
P: /usr/lib64/python3.6/lib2to3/PatternGrammar3.6.6.final.0.pickle
A: /usr/lib64/python3.6/lib2to3/PatternGrammar3.6.6.final.0.pickle
R: /usr/lib64/python3.6/lib2to3/PatternGrammar3.6.6.final.0.pickle
C: /usr/bin/python3.6 setup.py build -j 4
* -------------------------------------------------------------------------------- |
Similar error e.g. for dev-util/meson which is systemd dependency. Do you have any idea how to fix this?
EDIT:
FEATURES="-usersandbox" helps. Not sure, may be it is related to the same problem with qemu. Native environment does not have this problem with python packages. |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54232
Location: 56N 3W
|
| Posted: Mon Jan 28, 2019 5:25 pm Post subject: |
|
|
vagran,
Check the version of sandbox that you have,
| Code: | $ eix sandbox
[I] sys-apps/sandbox
Available versions: 2.12 2.13{tbz2} (~)2.14{tbz2} (~)2.15{tbz2} |
upgrade if needed.
If that fails, turn bits of sandboxing off. Check FEATURES= in
Warning. Sandboxing is a safety feature. Access Violations usually mean that an ebuild is doing something its not supposed to.
It looks like it was trying to open /usr/lib64/python3.6/lib2to3/Grammar3.6.6.final.0.pickle for writing.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
diddly
n00b
Joined: 11 Mar 2004
Posts: 27
Location: Halifax, Nova Scotia, Canada
|
| Posted: Sun Mar 17, 2019 10:53 am Post subject: |
|
|
Sorry to resurrect, but did a bug in either bugs.gentoo.org or qemu's tracker ever get filed re: the original PTHREAD_CREATE_DETACHED throwing an invalid argument in qemu problem?
Now that portage-2.3.62 has stabled I hit this on all my chroots as well.
| vagran wrote: | This package list doesn't seem to be really related...
I debugged a bit the core dump.
| Code: | (gdb) bt
#0 0x00000000602951eb in raise ()
#1 0x0000000060295771 in abort ()
#2 0x000000006020b760 in error_exit (err=22, msg=0x603715b0 <__func__.18188> "qemu_thread_create")
at /var/tmp/portage/app-emulation/qemu-3.1.0/work/qemu-3.1.0/util/qemu-thread-posix.c:36
#3 0x000000006020c58c in qemu_thread_create (thread=0x7ffc85db0e20, name=0x603730c9 "call_rcu", start_routine=0x60215011 <call_rcu_thread>, arg=0x0, mode=1)
at /var/tmp/portage/app-emulation/qemu-3.1.0/work/qemu-3.1.0/util/qemu-thread-posix.c:534
#4 0x000000006021538c in rcu_init_complete () at /var/tmp/portage/app-emulation/qemu-3.1.0/work/qemu-3.1.0/util/rcu.c:327
#5 0x00000000602154cc in rcu_init () at /var/tmp/portage/app-emulation/qemu-3.1.0/work/qemu-3.1.0/util/rcu.c:383
#6 0x00000000602836a4 in __libc_csu_init ()
#7 0x0000000060282ecd in __libc_start_main ()
#8 0x0000000060001bfa in _start ()
(gdb) f 3
#3 0x000000006020c58c in qemu_thread_create (thread=0x7ffc85db0e20, name=0x603730c9 "call_rcu", start_routine=0x60215011 <call_rcu_thread>, arg=0x0, mode=1)
at /var/tmp/portage/app-emulation/qemu-3.1.0/work/qemu-3.1.0/util/qemu-thread-posix.c:534
534 error_exit(err, __func__);
(gdb) p mode
$7 = 1 |
The problem is with pthread_create in this line. It returns EINVAL which probably caused by invalid attributes. The attributes are properly initialised earlier and pthread_attr_setdetachstate() was called with PTHREAD_CREATE_DETACHED. I do not really have ideas what to do next. The only guess is that it is somehow related to static linkage and libpthread was somehow broken during this process. I will also try to research live debugging possibility of this place in the qemu code. |
_________________
Dave Flogeras |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
