halz
n00b
Joined: 04 Jul 2003
Posts: 14
Location: California
|
 Posted: Fri May 07, 2021 10:38 pm Post subject: [solved] libtool/dlopen woes affecting multiple packages |
 |
|
I have been struggling to pin down the source of this issue that is affecting multiple packages (graphviz/umockdev/lua/imagemagick/neovim/etc), seemingly starting around the time I upgraded gcc from 10.2.0-r5 to 10.3.0. I have since upgraded to 11.1.0 before noticing things were so broken. Unfortunately I have no firm record of when I last successfully used one of these now-broken programs. I typically run a world/deep update every other day or so. Within the last few days I have run an empty tree world rebuild and identified a handful of additional packages that failed during build, letting me know that this was a systemic issue in the toolchain. Out of 1673 installed packages, only a handful fail to build outright (but that of course does not suggest the package wont work when it is run!)
The issue shows clearly enough when running dot -c (with dev-libs/libltdl built with -DLT_DEBUG_LOADERS), which reports the less-than-helpful 'file not found' error, after failing through all methods to load the plugin shared objects. The files are indeed there, and do not appear to be missing any symbols. I have been using a stage3 docker container to build the graphviz package for comparison of shared objects/binaries, where everything functions correctly/as expected.
| Code: | loaders: lt_preopen
try_dlopen (/usr/lib64/graphviz/libgvplugin_core.so.6, (null))
tryall_dlopen (/usr/lib64/graphviz/libgvplugin_core.so.6, (ALL))
Calling lt_preopen->module_open (/usr/lib64/graphviz/libgvplugin_core.so.6)
Result: Failed
Error message << file not found >>
Warning: Could not load "/usr/lib64/graphviz/libgvplugin_core.so.6" - It was found, so perhaps one of its dependents was not. Try ldd. |
Something I do see happen in the stage3 docker, which my host does not manage, is the loading of `dlopen.a` before the plugin shared objects are loaded.
| Code: | try_dlopen (dlopen.a, (null))
tryall_dlopen (dlopen.a, lt_preopen)
Calling lt_preopen->module_open (dlopen.a)
Result: Success
loaders: lt_dlopen, lt_preopen
try_dlopen (/usr/lib64/graphviz/libgvplugin_pango.so.6, (null))
tryall_dlopen (/usr/lib64/graphviz/libgvplugin_pango.so.6, (ALL))
Calling lt_dlopen->module_open (/usr/lib64/graphviz/libgvplugin_pango.so.6)
Result: Success |
I have overridden my 'crazy' LTO-leaning (since 2019) CFLAGS for libtoo/libltdl/graphviz to the more conservative -march=native -O2 -pipe -Wl,-O1 -Wl,--as-needed with no avail.
I have tried to compile the plugins in graphviz, entirely outside of portage, with flags such as -Wl,--no-undefined -Wl,-z,defs, getting predictable results that the plugins encounter unresolved symbols during build time. But this seems to just be in the nature of the building "plugins" that are loaded dynamically.
I have stepped through graphviz/libltdl with gdb (and built with installsources/-ggdb), attempting to isolate the specific errors preventing the shared libraries from being loaded correctly, with no avail. I can see where lt_dlopen decides that the result of loading the library is 'null', but I have not caught the specific error message that causes the failure.
...attempting to preload plugins does not make any progress
LD_PRELOAD="/usr/lib64/graphviz/libgvplugin_core.so.6" LD_DEBUG=all /usr/bin/dot -c
... there is this warning, relating to a debug symbol in glibc
Warning: 2492163: symbol=__vfprintf_chk; lookup in file=/usr/bin/dot [0]
...which apparently does exist in libc, so is again perhaps just noise from the larger issue of dlopen not functioning correctly.
| Code: | $ readelf -a -W /lib/libc-2.33.so | grep __vfprintf_chk
2043: 00110f40 37 FUNC GLOBAL DEFAULT 13 __vfprintf_chk@@GLIBC_2.3.4 |
Looking into the existence of dlopen, I can see the following symbol output from the stage3 docker vs my host is missing one symbol from the Relocation section/.rela.dyn (dlopen_LTX_get_vtable) but does show up in the symbol table (.dynsm):
The container:
| Code: | 3f0799fe9928 ~ # readelf -a -W /usr/lib64/libltdl.so | grep dlopen
000000000000ada8 0000003700000001 R_X86_64_64 0000000000006510 dlopen_LTX_get_vtable + 0
000000000000b140 0000004a00000007 R_X86_64_JUMP_SLOT 0000000000005650 lt_dlopen + 0
000000000000b150 0000001700000007 R_X86_64_JUMP_SLOT 0000000000000000 dlopen@GLIBC_2.2.5 + 0
000000000000b200 0000005800000007 R_X86_64_JUMP_SLOT 00000000000054a0 lt_dlopenadvise + 0
23: 0000000000000000 0 FUNC GLOBAL DEFAULT UND dlopen@GLIBC_2.2.5 (5)
55: 0000000000006510 150 FUNC GLOBAL DEFAULT 11 dlopen_LTX_get_vtable
74: 0000000000005650 7 FUNC GLOBAL DEFAULT 11 lt_dlopen
75: 0000000000005660 123 FUNC GLOBAL DEFAULT 11 lt_dlopenext
88: 00000000000054a0 430 FUNC GLOBAL DEFAULT 11 lt_dlopenadvise
|
... my host is missing dlopen_LTX_get_vtable from the entries in the 'Relocation section'.
| Code: | $ readelf -a -W /usr/lib64/libltdl.so | grep dlopen
000000000000b148 0000004b00000007 R_X86_64_JUMP_SLOT 0000000000005630 lt_dlopen + 0
000000000000b158 0000001800000007 R_X86_64_JUMP_SLOT 0000000000000000 dlopen@GLIBC_2.2.5 + 0
000000000000b208 0000005900000007 R_X86_64_JUMP_SLOT 0000000000005480 lt_dlopenadvise + 0
24: 0000000000000000 0 FUNC GLOBAL DEFAULT UND dlopen@GLIBC_2.2.5 (5)
56: 00000000000064e0 150 FUNC GLOBAL DEFAULT 11 dlopen_LTX_get_vtable
75: 0000000000005630 7 FUNC GLOBAL DEFAULT 11 lt_dlopen
76: 0000000000005640 123 FUNC GLOBAL DEFAULT 11 lt_dlopenext
89: 0000000000005480 430 FUNC GLOBAL DEFAULT 11 lt_dlopenadvise
|
So, I am at a bit of a loss for how to proceed with getting this pinned down and resolved, save for a scratch rebuild from backups. The host has been running as a daily driver since 2015, so I wont easily give up hope  Are there any other suggestions on how to step through repairing something like this?
And for what its worth, running the ./libtool-2.4.6/tests/testsuite produces a very sorry set of results. (But the results at least match the output between the clean/working stage3 container and the host system, for better or worse, the tests don't seem happy with the gentoo toolchain)
| Code: |
## ------------- ##
## Test results. ##
## ------------- ##
ERROR: 123 tests were run,
117 failed (4 expected failures).
47 tests were skipped.
|
Edit: Narrowed this down to something in my build environment breaking the libltdl.so with the removal of that dlopen_LTX_get_vtable entry in the relocation section/.rela.dyn. I was able to just copy the libtldl.so in from a raw source build of libtool/libltdl, using the same configure flags as generated by the ebuild, to temporarily "replace" my host's built copy of the library.. and all of the packages that didn't work before now magically work and build just fine. I'll update and close out here when I narrow down how my build environment has decided it would be a good idea to remove this relocation symbol.
Ediit Edit: Resolved and unable to reproduce the issue's origin. With libltdl.so replaced, I tried a normal emerge build of libltdl without making any special build flag/environment/feature concessions-- and it built just fine this time, containing the previously missing 'dlopen_LTX_get_vtable' symbol. What a detour!
----------------
Almost forgot emerge --info!
| Code: | Portage 3.0.18 (python 3.9.2-final-0, default/linux/amd64/17.1/desktop, gcc-11.1.0, glibc-2.33, 5.12.1-gentoo x86_64)
=================================================================
System uname: Linux-5.12.1-gentoo-x86_64-Intel-R-_Xeon-R-_CPU_E3-1505M_v5_@_2.80GHz-with-glibc2.33
KiB Mem: 32714564 total, 9599868 free
KiB Swap: 8388604 total, 8388092 free
Timestamp of repository gentoo: Fri, 07 May 2021 19:35:16 +0000
Head commit of repository gentoo: 54e263c308ec1ef363ff393bb2e7de1d3eef658f
Head commit of repository mv: cc28e9cf8703da7a20313a19a097b1458e2cafd0
Head commit of repository lto-overlay: 46854380feca8e444c56e56cc085f8f3902b137b
Head commit of repository steam-overlay: 9a1249fdb3e3005041fb768af02e235c46dc4a18
sh bash 5.1_p8
ld GNU ld (Gentoo 2.36.1 p3) 2.36.1
ccache version 4.2.1 [disabled]
app-shells/bash: 5.1_p8::gentoo
dev-java/java-config: 2.3.1::gentoo
dev-lang/perl: 5.32.1::gentoo
dev-lang/python: 2.7.18_p9::gentoo, 3.8.8-r1::lto-overlay, 3.9.2-r1::lto-overlay
dev-lang/rust: 1.52.0::gentoo
dev-util/ccache: 4.2.1::gentoo
dev-util/cmake: 3.20.2::gentoo
dev-util/pkgconfig: 0.29.2::gentoo
sys-apps/baselayout: 2.7-r2::gentoo
sys-apps/openrc: 0.42.1-r1::gentoo
sys-apps/sandbox: 2.24::gentoo
sys-devel/autoconf: 2.13-r1::gentoo, 2.69-r5::gentoo
sys-devel/automake: 1.16.3-r1::gentoo
sys-devel/binutils: 2.36.1-r1::gentoo
sys-devel/gcc: 11.1.0::gentoo
sys-devel/gcc-config: 2.4::gentoo
sys-devel/libtool: 2.4.6-r6::gentoo
sys-devel/make: 4.3::gentoo
sys-kernel/linux-headers: 5.12::gentoo (virtual/os-headers)
sys-libs/glibc: 2.33::gentoo
Repositories:
gentoo
location: /usr/portage
sync-type: git
sync-uri: https://github.com/gentoo-mirror/gentoo
priority: -1000
sync-git-verify-commit-signature: true
mv
location: /usr/local/portage/mv-overlay
sync-type: git
sync-uri: https://anongit.gentoo.org/git/user/mv.git
masters: gentoo
priority: 40
lto-overlay
location: /usr/local/portage/lto-overlay
sync-type: git
sync-uri: https://github.com/InBetweenNames/gentooLTO.git
masters: gentoo mv
priority: 50
steam-overlay
location: /usr/local/portage/steam-overlay
sync-type: git
sync-uri: https://github.com/anyc/steam-overlay.git
masters: gentoo
priority: 50
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O3 -fgraphite-identity -floop-nest-optimize -fdevirtualize-at-ltrans -fipa-pta -fno-semantic-interposition -flto=7 -fuse-linker-plugin -falign-functions=32 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-march=native -O3 -fgraphite-identity -floop-nest-optimize -fdevirtualize-at-ltrans -fipa-pta -fno-semantic-interposition -flto=7 -fuse-linker-plugin -falign-functions=32 -pipe"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--with-bdeps=y"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j7"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
RUSTFLAGS="-Copt-level=3 -Ctarget-cpu=native -Cdebuginfo=0"
USE="X a52 aac acl acpi alsa amd64 audit berkdb bluetooth branding bzip2 cairo cdda cdr clang cli crypt cups dbus dri dts dvd dvdr elogind emboss encode exif fam flac fortran gdbm gif gles gles2 gnome-keyring gpm gstreamer gtk gui hardened iconv icu idn ipv6 jack jpeg kerberos ladspa lcms ldap libglvnd libinput libnotify libtirpc lz4 lzma mad mms mng mp3 mp4 mpeg multilib ncurses nls nptl ogg opengl openmp openrc opus pam pango pcre pdf png policykit ppds pulseaudio qt5 readline sasl sdl seccomp smp spell split-usr ssl startup-notification svg tcpd threads tiff truetype udev udisks unicode upower usb v4l vaapi vorbis vulkan wavpack wayland webp wxwidgets x264 xattr xcb xml xrandr xv xvid zlib zstd" ABI_X86="64" ADA_TARGET="gnat_2018" ALSA_CARDS="hda-intel usb-audio" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-64" INPUT_DEVICES="synaptics libinput wacom" KERNEL="linux" L10N="en en-US" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-3 php7-4" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_9" PYTHON_TARGETS="python3_9" RUBY_TARGETS="ruby26" USERLAND="GNU" VIDEO_CARDS="nvidia nouveau intel i965 iris fbdev" XFCE_PLUGINS="power" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset: CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS |
[Moderator edit: Moved poster's self-response into main post, then deleted that self-response to return the thread to the unanswered list.
OP: Generally, if post length permits, you should edit extra information into your first post instead of replying to it. Some users run a canned search for threads with zero replies as a way of finding posts to answer. When you replied to yourself, you removed your thread from that list, even though you were not replying to announce that the thread was solved. Thus, they would no longer see your thread on the list of unanswered threads.
-Hu]
Last edited by halz on Sat May 08, 2021 12:51 pm; edited 2 times in total |
|
Portage & Programming
