Emerge / gcc hang with -O1 through -O3 flags

Message

atrix · Post by **atrix** » Thu Mar 31, 2022 8:22 pm

Hello,

I'm running into some strange behavior with gcc while emerging certain packages. If I have -O1, -O2, or -O3 in my COMMON_FLAGS, the build hangs indefinitely at different points for each problematic package. I've been testing mainly against opus, but have seen this behavior when emerging libtool and gcc itself. Omitting the '-O' flag or using '-O0' allows these packages to build, and the majority of packages build without issue with '-O2.'

This behavior is occurring on an armv8 / aarch64 machine.

When emerging opus, compilation hangs on:

Code: Select all

libtool: link: aarch64-unknown-linux-gnu-gcc -O2 -pipe -march=armv8-a+crc+fp+simd+crypto -mtune=cortex-a73.cortex-a53 -mcpu=cortex-a73.cortex-a53+crc+fp+simd+crypto -ftree-vectorize -fuse-linker-plugin -flto=6 -mfix-cortex-a53-835769 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=256 -fvisibility=hidden -W -Wall -Wextra -Wcast-align -Wnested-externs -Wshadow -Wstrict-prototypes -Wl,-O1 -Wl,--as-needed -o tests/.libs/test_opus_padding tests/test_opus_padding.o  ./.libs/libopus.so -lm

I've removed each of the above flags individually and have emerged opus and libtool without issue, am certain the -O parameter is causing these hangs.

Emerging gcc hangs on:

Code: Select all

>>> Configuring source in /var/tmp/portage/sys-devel/gcc-11.2.1_p20220115/work/gcc-11-20220115 ...

I let the above sit for hours with no luck.

One core is pegged at 100% during these hangs. I let opus go overnight for 9 hours last night with no progress. I've tried emerging with --debug but don't receive any additional clues. I don't see any errors logged, and nothing in kernel logs.

I'm not sure where to look for additional clues or how to move past this. Has anyone else experienced this behavior before?

Post by **NeddySeagoon** » Thu Mar 31, 2022 8:33 pm

atrix,

Post the output of

Code: Select all

emerge --info

please.

Code: Select all

-flto=6

That takes lots of RAM, or if you it into swapping its very slow.
Swapping does to mean using a swap partition. The kernel has other ways to swap too.

What CPU do you have?

Code: Select all

-mcpu=cortex-a73.cortex-a53+crc+fp+simd+crypto

suggests that is a big.LITTLE CPU.

Post by **NeddySeagoon** » Thu Mar 31, 2022 8:35 pm

Moved from Portage & Programming to Gentoo on ARM.

Its likely an arm64 problem, so fits better here.

atrix · Post by **atrix** » Thu Mar 31, 2022 8:42 pm

Hi NeddySeagoon,

Thanks for the reply. emerge --info output is below:

Code: Select all

Portage 3.0.30 (python 3.9.11-final-0, default/linux/arm64/17.0/systemd, gcc-11.2.1, glibc-2.33-r13, 5.15.26-gentoo aarch64)
=================================================================
System uname: Linux-5.15.26-gentoo-aarch64-with-glibc2.33
KiB Mem:     3757964 total,    538508 free
KiB Swap:   20971516 total,  20969188 free
Timestamp of repository gentoo: Thu, 31 Mar 2022 17:00:01 +0000
Head commit of repository gentoo: 63d6452df80de5c0ad593fefa11d25ac5520fba0
Timestamp of repository gamarouns: Mon, 28 Mar 2022 20:05:07 +0000
Head commit of repository gamarouns: 3ab74256f73ccef78978c7141ca4cf514a8a94b8

Timestamp of repository quarks: Mon, 28 Mar 2022 20:04:54 +0000
Head commit of repository quarks: 70d1e880bc483a51c25fdefc62312b41844f728f

sh bash 5.1_p16
ld GNU ld (Gentoo 2.37_p1 p2) 2.37
ccache version 4.5.1 [disabled]
app-misc/pax-utils:        1.3.3::gentoo
app-shells/bash:           5.1_p16::gentoo
dev-java/java-config:      2.3.1::gentoo
dev-lang/perl:             5.34.0-r6::gentoo
dev-lang/python:           3.9.11::gentoo, 3.10.3::gentoo
dev-lang/rust:             1.58.1::gentoo
dev-util/ccache:           4.5.1::gentoo
dev-util/cmake:            3.22.2::gentoo
dev-util/meson:            0.60.3::gentoo
sys-apps/baselayout:       2.7-r3::gentoo
sys-apps/sandbox:          2.25::gentoo
sys-apps/systemd:          249.9::gentoo
sys-devel/autoconf:        2.13-r1::gentoo, 2.71-r1::gentoo
sys-devel/automake:        1.16.4::gentoo
sys-devel/binutils:        2.37_p1-r2::gentoo
sys-devel/binutils-config: 5.4::gentoo
sys-devel/gcc:             11.2.1_p20220115::gentoo
sys-devel/gcc-config:      2.5-r1::gentoo
sys-devel/libtool:         2.4.6-r6::gentoo
sys-devel/llvm:            13.0.1::gentoo
sys-devel/make:            4.3::gentoo
sys-kernel/linux-headers:  5.15-r3::gentoo (virtual/os-headers)
sys-libs/glibc:            2.33-r13::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000
    sync-rsync-verify-jobs: 1
    sync-rsync-verify-metamanifest: yes
    sync-rsync-extra-opts: 
    sync-rsync-verify-max-age: 24

gamarouns
    location: /var/db/repos/gamarouns
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/gamarouns.git
    masters: gentoo

quarks
    location: /var/db/repos/quarks
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/quarks.git
    masters: gentoo

ACCEPT_KEYWORDS="arm64"
ACCEPT_LICENSE="@FREE"
CBUILD="aarch64-unknown-linux-gnu"
CFLAGS="-O2 -pipe -march=armv8-a+crc+fp+simd+crypto -mtune=cortex-a73.cortex-a53 -mcpu=cortex-a73.cortex-a53+crc+fp+simd+crypto -ftree-vectorize -fuse-linker-plugin -flto=6  -mfix-cortex-a53-835769 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=256"
CHOST="aarch64-unknown-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe -march=armv8-a+crc+fp+simd+crypto -mtune=cortex-a73.cortex-a53 -mcpu=cortex-a73.cortex-a53+crc+fp+simd+crypto -ftree-vectorize -fuse-linker-plugin -flto=6  -mfix-cortex-a53-835769 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=256"
DISTDIR="/var/cache/distfiles"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2 -pipe -march=armv8-a+crc+fp+simd+crypto -mtune=cortex-a73.cortex-a53 -mcpu=cortex-a73.cortex-a53+crc+fp+simd+crypto -ftree-vectorize -fuse-linker-plugin -flto=6  -mfix-cortex-a53-835769 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=256"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg-live compress-build-logs compress-index compressdebug config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe -march=armv8-a+crc+fp+simd+crypto -mtune=cortex-a73.cortex-a53 -mcpu=cortex-a73.cortex-a53+crc+fp+simd+crypto -ftree-vectorize -fuse-linker-plugin -flto=6  -mfix-cortex-a53-835769 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=256"
GENTOO_MIRRORS="rsync://mirrors.rit.edu/gentoo/ rsync://rsync.gtlib.gatech.edu/gentoo https://gentoo.osuosl.org/ https://mirrors.rit.edu/gentoo/"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j6"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
SHELL="/bin/bash"
USE="X aac acl alsa arm64 bash-completion bluetooth bzip2 cairo calendar cli crypt curl dbus directfb dri egl fbcon ffmpeg flac fortran ftp fuse gdbm gif gimp gles1 gles2 gnutls gstreamer gtk iconv imagemagick inotify introspection ipv6 javascript jingle jit jpeg lame ldap libglvnd libtirpc lirc lzo mad mmap mp3 mp4 mpeg mplayer ncurses nfs nls nptl nsplugin odbc offensive ogg oggvorbis openmp openssl pam pcre pdf png posix pulseaudio qt3support qt5 rar raw readline sasl sdl seccomp snmp sound speex spell split-usr ssl svg symlink syslog systemd theora threads tiff udev unicode upnp upnp-av usb video vorbis wayland xattr xml xmpp xorg xpm xrandr xv xvmc zlib" ADA_TARGET="gnat_2020" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_ARM="edsp thumb vfp vfpv3 vfpv4 vfp-d32 aes sha1 sha2 crc32 v4 v5 v6 v7 v8 thumb2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-4 php8-0" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_9" PYTHON_TARGETS="python3_9" RUBY_TARGETS="ruby26 ruby27" USERLAND="GNU" VIDEO_CARDS="fbdev" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LEX, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS

I do have a 20GB swap file but haven't seen much swapping during these hangs. The board itself is an Odroid N2+ with 4GB of RAM.

You are correct, it is big.LITTLE, a Cortex-A73 and a Cortex-A53. I agree it's probably related to the architecture; I've never encountered this on AMD64.

Post by **NeddySeagoon** » Thu Mar 31, 2022 9:23 pm

atrix,

I run -flto=6 on an X-Gene 1 based server.

I see several problems there,
1. Things fail to build because needed libraries are optimised away.
2. Things build but other packages that depend on them fail at run time.
3. Now and again, it appears to run out out of RAM and I have 128G

Swap space is only used for dynamically allocated RAM as it has no permanent home on disk
The kernel can also drop clean buffers and code.
Dirty buffers must be written to make them clean before they can be dropped.

Buffers and code will be reloaded from permanent storage as required.

Remove the -flto=6 for your test case.
Both

Code: Select all

MAKEOPTS="-j6"

and

Code: Select all

-flto=6

are too high for 4G RAM.
Try

Code: Select all

MAKEOPTS="-j4"

and

Code: Select all

-flto=4

for everyday use.
Even then, expect to use package.env to tune them on a per package basis for a few trouble makers.

atrix · Post by **atrix** » Thu Mar 31, 2022 10:20 pm

I see, thank you for those details. I didn't realize my -flto option was so aggressive.

I've lowered MAKEOPTS="-j6" to -j4 and -flto=6 to -flto=4 but am still hanging at the same place when emerging opus. I gave it a shot with -flto=0 as well, same result. I'll continue to play with these options and will start carving out package.env profiles.

The majority of the base packages were built in qemu chroot and I started noticing these problems when trying to emerge -e @world on this hardware with the correct optimizations and cpu flags, so may have some gremlins lying in wait.

Thanks for your help.

Post by **NeddySeagoon** » Fri Apr 01, 2022 10:42 am

atrix,

There are two constrains on MAKEOPTS.

The traditional one of cores/threads+1 and the more modern one of 2G RAM per make thread.
The first gives 5 for your CPU, the second gives 2 based on your RAM.

Its only big packages that would need MAKEOPTS="-j2" so on memory constrained systems, I would set MAKEOPTS="-j4" in make.conf and set a lower number on a per packages basis.
Portage can do that, you don't nee to do it at the command line.

-flto=0 may not do what you think it does. Its -fno-lto.
Sometimes passing =0 means go nuts. Unlimited. That's not what you want at all.

I'm in the middle of an arm64 server upgrade, It's suffering a faulty RAM stick, so I can't do a lot with it.

Code: Select all

[I] media-libs/opus
     Available versions:  1.3.1-r2{tbz2} {custom-modes doc static-libs ABI_MIPS="n32 n64 o32" ABI_S390="32 64" ABI_X86="32 64 x32" CPU_FLAGS_ARM="neon" CPU_FLAGS_X86="sse"}
     Installed versions:  1.3.1-r2{tbz2}(12:51:23 01/26/22)(-custom-modes -doc -static-libs ABI_MIPS="-n32 -n64 -o32" ABI_S390="-32 -64" ABI_X86="-32 -64 -x32" CPU_FLAGS_ARM="-neon" CPU_FLAGS_X86="-sse")
     Homepage:            https://opus-codec.org/
     Description:         Open codec for interactive speech and music transmission over the Internet

it is installed though.

My make.conf is aimed for a Pi4 but other than instruction ordering, its the same as a Pi3.
If you want to cheat for now, my Pi 4 BINHOST is open to the public.

atrix · Post by **atrix** » Tue Apr 05, 2022 2:13 am

Thanks NeddySeagoon. I hadn't heard of the 2GB of RAM per thread recommendation. I thought I was playing it slightly safer by just using one thread per core than the standard/older recommendation. I will default to MAKEOPTS="-j4" and handle each package I run into issues with on an individual basis.

Ah, that's good to know. -flto=0 made just about everything worse, so I'm glad to hear that; I was a bit confused by that behavior. -fno-lto it is (when needed).

Due to my tardiness in replying (apologies) I can no longer access your make.conf, but your BINHOST is extremely helpful. I'll be leaning heavily on this as I work my way through these issues. My goal is to emerge -e @world without any hangs. I'm concerned that I can't build gcc and am wondering if this is related to my trouble building other packages. Tweaking MAKEOPTS="-j#" and -flto=# have a direct effect on where I hang while building gcc, but I still get stuck when using -j1 and -fno-lto. I'm still using opus as my test case since I can recreate the hang quicker, but am curious and concerned about not being able to build gcc.

Post by **NeddySeagoon** » Tue Apr 05, 2022 8:07 pm

atrix,

My /etc/portage/* is there somewhere too.
Most of the Raspberry Pi entries have been commented out but they should still be there.
Try looking here
Oops, it should not be in /gcc-10.x/ any more.

I'm in the process of moving from one arm64 build box to another, so the binhost is not getting updated as frequently as it used to.

atrix · Post by **atrix** » Thu Apr 07, 2022 2:12 am

Ah, great, thank you Neddy; that has what I was looking for.

As a result of using different combinations of values for -flto, -O#, and occasionally using -fno-lto, I am able to make my way through most problematic packages by creating env profiles. However, I am at a complete impasse when it comes to rebuilding gcc. No combination of options seems to be doing the trick; trying to emerge different versions yield the same results. I either hang on "configuring source" or "STRIP FLAGS." I've removed all gcc flags individually until I had none left with no change in results.

I'll continue troubleshooting and will report back if I'm able to make any progress here. Your binhost has been extremely helpful as I work through these issues. You helped me on an ARM issue once long ago, and as always your help is greatly appreciated.

Post by **NeddySeagoon** » Thu Apr 07, 2022 5:30 pm

atrix,

gcc has a strange build system. Its done the way it is with good reason though.

1. enough of gcc is built with some random compiler on the system, so the gcc can build itself.
2. the gcc just built is used to build gcc again. This ensures that it is not contaminated with code from the host compiler, which is unknown.
3. the gcc built at stage 2 is used to build gcc again and the results between 2. and 3. are compared.
If the compare fails, its all over, the build terminates.

If all is well, the build system builds the optional language support, selected by your use flags and this gets installed.

A few things drop out of this. distcc can only help with stage 1.
There is only one copy of gcc as output from stage 1, so distcc cannot help.
If follows that the poor wee arm system is all on its own doing the building.

When the build fails, grep the build log for killed.

Code: Select all

grep -i  killed

If its there, you ran out of RAM and the OOM kicked in.
If its not that easy, put the entire build log onto a pastebin.

atrix · Post by **atrix** » Thu Apr 07, 2022 7:59 pm

Hi NeddySeagoon,

This is really interesting; I was trying to get to the bottom of how gcc was actually built and these details clear that up. That said, I seem to be failing/hanging at stage 1.

I've posted the two different types of logs I get based on my USE flags, the first is a build with no special flags enabled - I make it past "Configuring source" then hang on the "strip-flags" portion.

http://dpaste.com/2BLAKLW2C

The second log is when attempting to build with additional USE flags - specifically custom-cflags, go, graphite, lto, zstd. I've tried this with just custom-cflags, same result.

http://dpaste.com/3NYEAMJNZ

With both builds, they hang at the above stages indefinitely until I kill the process. During these hangs, one core is at 100% CPU while the rest are idle. I've since disabled all custom USE flags and below is the process is the one at 100% CPU while the build is stopped at "strip-flags":

Code: Select all

/usr/libexec/gcc/aarch64-unknown-linux-gnu/11.2.1/f951 /mnt/portage/sys-devel/gcc-11.2.1_p20220115/temp/test-flag.f -ffixed-form -quiet -dumpdir /mnt/portage/sys-devel/gcc-11.2.1_p20220115/temp/ -dumpbase test-flag.f -dumpbase-ext .f -mlittle-endian -mabi=lp64 -Werror -fintrinsic-modules-path /usr/lib/gcc/aarch64-unknown-linux-gnu/11.2.1/finclude -fpre-include=/usr/include/finclude/math-vector-fortran.h -o -

Based on your above comments, I'm guessing f951 is the system compiler which is helping build gcc in stage 1. I'll let my current attempt sit for a few hours before killing it in case these "hangs" aren't actually hangs, but strace produces no output when the build is stuck like this, and no files are being modified in the portage build directory.

Post by **NeddySeagoon** » Thu Apr 07, 2022 8:35 pm

atrix,

My gcc is built with USE="cxx fortran graphite jit lto nls nptl openmp pie sanitize ssp"

USE=cxx is required to build a gcc that can built itself next time.
USE=pie and ssp come from your profile starting with the /17.0/ profiles.

Is your real time clock correct. Some configure scripts check that their output files are newer than the source files and run again if the test fails.
With the RTC set to 01-Jan-1970, with no RTC, and source code timestamped 2020, the check will take over 50 years to pass.
I've been bitten by that a few times :)

If the RTC is OK, then after you stop it, try the following

Code: Select all

ebuild </full/path/to/>gcc-11.2.1_p20220115.ebuild configure

That will run the configure phase function without the aid of a safety net. Just you as root and the ebuild phase.
Its possible to run every ebuild phase like this.

Its possible that one of the sandboxes and your toolchain don't play nicely together.

Code: Select all

FEATURES:   network-sandbox sandbox userpriv usersandbox

any or several of those features can cause problems.

If you started your install with the arm64 stage3 then you would have gcc.

gcc can be cross compiled and the binary package installed.
It's one of the more difficult things to cross compile and often needs some USE flags turned off to make it work.

atrix · Post by **atrix** » Thu Apr 07, 2022 10:50 pm

Good to know about cxx.

Interesting thought on the RTC - it is accurate, but sadly I receive the same results whether it is enabled or disabled.

The explicit ebuild configure command does hang in exactly the same spot, but at least without unpacking around a gig of files first. I also tried disabling each of the listed FEATURES individually, and all of them, same hang.

Your stage3 comment got me thinking.. I did build from stage 3, but in a qemu chroot, where I emerged many packages I wanted to have ready for this system (it hadn't arrived in the mail yet, I didn't have another arm64 host online).

I grabbed my stage3 tarball and unpacked it, chrooted into it, tested re-emerging gcc with default options, got past the hang point and killed it. I then moved my make.conf from my live system to the chroot, gcc got past the hang point, I killed it. I then copied the below dirs to my live system and tried emerging gcc, same hang:

Code: Select all

/usr/share/gdb/auto-load/usr/lib/gcc/aarch64-unknown-linux-gnu/11.2.1
/usr/share/gcc-data/aarch64-unknown-linux-gnu/11.2.1
/usr/libexec/gcc/aarch64-unknown-linux-gnu/11.2.1
/usr/aarch64-unknown-linux-gnu/gcc-bin/11.2.1
/usr/lib/gcc/aarch64-unknown-linux-gnu/11.2.1

So I'm thinking something became screwy when built in qemu chroot, but I haven't been able to identify what. I'm hesitant to pull too much over from stage3 to my system without knowing exactly what I'm looking for. My fallback plan at this point is to rebuild my system in chroot from this device, but I'm not giving up on finding the culprit causing these hangs (yet), and rebuilding will be a last resort.

My next thought was to try (backing up and) replacing all 'aarch64-unknown-linux-gnu' directories I could find, but I don't want to keep shooting blindly.

Post by **NeddySeagoon** » Fri Apr 08, 2022 4:38 pm

atrix,

Run

Code: Select all

gcc -v

on both gccs and compare the output.

My Pi4 KVM says

Code: Select all

$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/aarch64-unknown-linux-gnu/11.2.1/lto-wrapper
Target: aarch64-unknown-linux-gnu
Configured with: /var/tmp/portage/sys-devel/gcc-11.2.1_p20220115/work/gcc-11-20220115/configure --host=aarch64-unknown-linux-gnu --build=aarch64-unknown-linux-gnu --prefix=/usr --bindir=/usr/aarch64-unknown-linux-gnu/gcc-bin/11.2.1 --includedir=/usr/lib/gcc/aarch64-unknown-linux-gnu/11.2.1/include --datadir=/usr/share/gcc-data/aarch64-unknown-linux-gnu/11.2.1 --mandir=/usr/share/gcc-data/aarch64-unknown-linux-gnu/11.2.1/man --infodir=/usr/share/gcc-data/aarch64-unknown-linux-gnu/11.2.1/info --with-gxx-include-dir=/usr/lib/gcc/aarch64-unknown-linux-gnu/11.2.1/include/g++-v11 --with-python-dir=/share/gcc-data/aarch64-unknown-linux-gnu/11.2.1/python --enable-languages=c,c++,jit,fortran --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --disable-libunwind-exceptions --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 11.2.1_p20220115 p4' --disable-esp --enable-libstdcxx-time --with-build-config=bootstrap-lto --disable-libstdcxx-pch --enable-host-shared --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --disable-multilib --disable-fixed-point --enable-libgomp --disable-libssp --disable-libada --disable-cet --disable-systemtap --disable-valgrind-annotations --disable-vtable-verify --disable-libvtv --without-zstd --enable-lto --with-isl --disable-isl-version-check --enable-default-pie --enable-default-ssp
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 11.2.1 20220115 (Gentoo 11.2.1_p20220115 p4)

Building in a qemu chroot should be harmless.

atrix · Post by **atrix** » Fri Apr 08, 2022 5:29 pm

The output of both is exactly the same:

Code: Select all

# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/aarch64-unknown-linux-gnu/11.2.1/lto-wrapper
Target: aarch64-unknown-linux-gnu
Configured with: /var/tmp/portage/sys-devel/gcc-11.2.1_p20220115/work/gcc-11-20220115/configure --host=aarch64-unknown-linux-gnu --build=aarch64-unknown-linux-gnu --prefix=/usr --bindir=/usr/aarch64-unknown-linux-gnu/gcc-bin/11.2.1 --includedir=/usr/lib/gcc/aarch64-unknown-linux-gnu/11.2.1/include --datadir=/usr/share/gcc-data/aarch64-unknown-linux-gnu/11.2.1 --mandir=/usr/share/gcc-data/aarch64-unknown-linux-gnu/11.2.1/man --infodir=/usr/share/gcc-data/aarch64-unknown-linux-gnu/11.2.1/info --with-gxx-include-dir=/usr/lib/gcc/aarch64-unknown-linux-gnu/11.2.1/include/g++-v11 --with-python-dir=/share/gcc-data/aarch64-unknown-linux-gnu/11.2.1/python --enable-languages=c,c++,fortran --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --disable-libunwind-exceptions --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 11.2.1_p20220115 p4' --disable-esp --enable-libstdcxx-time --disable-libstdcxx-pch --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --disable-multilib --disable-fixed-point --enable-libgomp --disable-libssp --disable-libada --disable-cet --disable-systemtap --disable-valgrind-annotations --disable-vtable-verify --disable-libvtv --without-zstd --enable-lto --without-isl --enable-default-pie --enable-default-ssp
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 11.2.1 20220115 (Gentoo 11.2.1_p20220115 p4)

Yeah.. I know I'm reaching with the qemu chroot theory, I'm just thoroughly confused at this point. I'm not sure what else would be different outside of gcc itself that would be affecting the build behavior and causing these hangs.

atrix · Post by **atrix** » Sun Apr 10, 2022 2:16 am

Unfortunately, I haven't gotten any further with this and am going to pursue rebuilding in chroot. I'm not sure what went wrong with the initial install, but I can build gcc in stage 3 chroot and also am not seeing hangs on opus, libtool, and others with optimization flags enabled. This really is puzzling.

Post by **NeddySeagoon** » Sun Apr 10, 2022 9:05 am

atrix,

Does it really hang, that is the load average drops to zero, or very close, or does it stick in a loop?
The load average will be about one, if its doing a busy wait.

I'm not sure its worth the effort to find out. Build the package in the chroot and install the binary.
Keep the chroot around in case that's not the problem, so yo can build another package.

atrix · Post by **atrix** » Tue Apr 12, 2022 5:23 pm

Hi NeddySeagoon,

"Hang" might not have been the best word to describe this behavior; the load average is slightly above one when this gets stuck. Strace output halts and I don't see any files being modified or IO activity when this happens.

Definitely, I've been leaning heavily on the chroot environment and have not had any issues building there, so will continue building problem packages there. If I can still recreate these hangs/stuck builds after some time I'll copy everything from the updated chroot to my system. Thanks very much for your assistance troubleshooting this strange issue.