[Solved] openjdk-17.0.13_p11 extreme RAM usage

Message

shadowfly · Post by **shadowfly** » Sun Oct 20, 2024 12:08 pm

Hello there!
Background
I have dev-java/openjdk (17) installed since a few years back. A month (or so) ago i recompiled my system, because i had a KDE Plasma bug and wanted debug symbols and source code to be more helpful. Back then everything recompiled smoothly with emerge -e @world.
I update once a week, every weekend.

The problem
Today i tried to update again (emerge -aquDN @world), but my RAM usage shot up, and then the PC locked up.
lto1-wpa was using >50GB of RAM (32GB + swap).

lto sounds like Link Time Optimization. I have that enabled as far as i remember.

Workaround:

Code: Select all

emerge -aquDN @world --exclude dev-java/openjdk

The question
Can someone confirm this, or is it just for me?

Additional info
CPU: Ryzen7-7700x
RAM: 32GB
Swap: 32GB

emerge --info '=dev-java/openjdk-17.0.13_p11::gentoo'
Portage 3.0.65 (python 3.12.6-final-0, default/linux/amd64/23.0/desktop/plasma/systemd, gcc-13, glibc-2.39-r6, 6.6.51-gentoo-shadowfly x86_64)
=================================================================
System Settings
=================================================================
System uname: Linux-6.6.51-gentoo-shadowfly-x86_64-AMD_Ryzen_7_7700X_8-Core_Processor-with-glibc2.39
KiB Mem: 32565448 total, 23441972 free
KiB Swap: 33554428 total, 32952316 free
Timestamp of repository gentoo: Sun, 20 Oct 2024 10:30:00 +0000
Head commit of repository gentoo: f3ddae3deeeaab23f91f0c3202da8f72edcccbc2
Timestamp of repository farmboy0: Mon, 14 Oct 2024 12:50:23 +0000
Head commit of repository farmboy0: 6e660ad699863c00866ed5d941ea851f3c8ea1a9

Timestamp of repository guru: Sat, 19 Oct 2024 00:18:16 +0000
Head commit of repository guru: e344dda63b0598ac594840f6198e411290978b89

sh bash 5.2_p37
ld GNU ld (Gentoo 2.42 p6) 2.42.0
app-misc/pax-utils: 1.3.7::gentoo
app-shells/bash: 5.2_p37::gentoo
dev-build/autoconf: 2.71-r7::gentoo
dev-build/automake: 1.16.5-r2::gentoo
dev-build/cmake: 3.30.2::gentoo
dev-build/libtool: 2.4.7-r4::gentoo
dev-build/make: 4.4.1-r1::gentoo
dev-build/meson: 1.5.1::gentoo
dev-java/java-config: 2.3.4::gentoo
dev-lang/perl: 5.40.0::gentoo
dev-lang/python: 3.12.6_p4::gentoo
dev-lang/rust: 1.81.0::gentoo
sys-apps/baselayout: 2.15::gentoo
sys-apps/sandbox: 2.39::gentoo
sys-apps/systemd: 255.11::gentoo
sys-devel/binutils: 2.42-r2::gentoo
sys-devel/binutils-config: 5.5.2::gentoo
sys-devel/gcc: 13.3.1_p20240614::gentoo
sys-devel/gcc-config: 2.11::gentoo
sys-devel/llvm: 18.1.8-r1::gentoo
sys-kernel/linux-headers: 6.6-r1::gentoo (virtual/os-headers)
sys-libs/glibc: 2.39-r6::gentoo
Repositories:

gentoo
location: /var/db/repos/gentoo
sync-type: rsync
sync-uri: rsync://rsync.gentoo.org/gentoo-portage
priority: -1000
volatile: False
sync-rsync-verify-max-age: 24
sync-rsync-extra-opts:
sync-rsync-verify-jobs: 1
sync-rsync-verify-metamanifest: yes

farmboy0
location: /var/db/repos/farmboy0
sync-type: git
sync-uri: https://github.com/gentoo-mirror/farmboy0.git
masters: gentoo
volatile: False

guru
location: /var/db/repos/guru
sync-type: git
sync-uri: https://github.com/gentoo-mirror/guru.git
masters: gentoo
volatile: False

ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="@FREE"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -pipe -flto -Werror=odr -Werror=lto-type-mismatch -Werror=strict-aliasing -Og -ggdb"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d"
CXXFLAGS="-march=native -pipe -flto -Werror=odr -Werror=lto-type-mismatch -Werror=strict-aliasing -Og -ggdb"
DISTDIR="/var/cache/distfiles"
EMERGE_DEFAULT_OPTS="--jobs 2"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-march=native -pipe -flto -Werror=odr -Werror=lto-type-mismatch -Werror=strict-aliasing"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live clean-logs config-protect-if-modified distlocks ebuild-locks fixlafiles installsources ipc-sandbox merge-sync merge-wait multilib-strict network-sandbox news nostrip parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-march=native -pipe -flto -Werror=odr -Werror=lto-type-mismatch -Werror=strict-aliasing"
GENTOO_MIRRORS="https://ftp.halifax.rwth-aachen.de/gentoo/ https://mirror.eu.oneandone.net/linux/d ... oo/gentoo/ https://mirror.netcologne.de/gentoo/ https://mirror.init7.net/gentoo/"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs"
LEX="flex"
MAKEOPTS="-j12 --load-average=16"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
SHELL="/bin/zsh"
USE="X a52 aac acl acpi activities alsa amd64 bluetooth branding bzip2 cairo cdda cdr cet crypt cups dbus declarative dri dts dvd dvdr encode exif flac gdbm gif gpm grub gtk gui iconv icu ipv6 jpeg kde kf6compat kwallet lcms libnotify libtirpc lto mad mng mp3 mp4 mpeg multilib ncurses networkmanager nls ogg opengl openmp pam pango pcre pdf pipewire plasma png policykit ppds pulseaudio qml qt5 qt6 readline screencast sdl seccomp semantic-desktop sound spell ssl startup-notification svg systemd test-rust tiff truetype udev udisks unicode upower usb vaapi vdpau vorbis vulkan wayland widgets wxwidgets x264 xattr xcb xft xml xorg xv xvid xwayland zlib" ABI_X86="64" ADA_TARGET="gcc_12" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2 aes avx avx2 avx512_bf16 avx512_bitalg avx512_vbmi2 avx512_vnni avx512_vpopcntdq avx512bw avx512cd avx512dq avx512f avx512ifma avx512vbmi avx512vl f16c fma3 pclmul popcnt rdrand sha sse3 sse4_1 sse4_2 sse4a ssse3 vpclmulqdq" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" GRUB_PLATFORMS="efi-64" GUILE_SINGLE_TARGET="3-0" GUILE_TARGETS="3-0" INPUT_DEVICES="libinput" KERNEL="linux" L10N="en en-US" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-2" POSTGRES_TARGETS="postgres16" PYTHON_SINGLE_TARGET="python3_12" PYTHON_TARGETS="python3_12" RUBY_TARGETS="ruby32" VIDEO_CARDS="amdgpu radeonsi" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account"
Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS

logrusx · Post by **logrusx** » Sun Oct 20, 2024 3:15 pm

shadowfly wrote: The question
Can someone confirm this, or is it just for me?

Recompile the whole system with debug symbols disabled.

Best Regards,
Georgi

shadowfly · Post by **shadowfly** » Sun Oct 20, 2024 5:52 pm

Hello,

for now, i just deactivated LTO for openjdk like here: https://wiki.gentoo.org/wiki/LTO#Disabl ... er_Package
This does work, the compilation does not consume much RAM anymore.

Personally, i would like to avoid recompiling everything and rather like someone else (who does have LTO, but does not have debug symbols) confirm the bug. For them this would not take more than like 10 minutes.
Then i (or they) could report it as described here: https://wiki.gentoo.org/wiki/LTO#New_Bugs_Procedure

But if nobody is found by next weekend, i am going to do recompile myself.

logrusx · Post by **logrusx** » Sun Oct 20, 2024 6:13 pm

I misread your first post. Yes, LTO consumes a lot during compilation. So does debug symbols. I wonder how you even were able to recompile *the whole system* with debug symbols *and LTO*. Or at least the packages for which both applied to. That combination doesn't even make sense. Debug symbols also degrade performance. I guess you weren't around during the compilation to see what's going on. One does not usually enable debug symbols system wide either, especially when the problem is in higher level packages. I doubt debug symbols were even remotely useful to diagnose your issue with Plasma.

Best Regards,
Georgi

shadowfly · Post by **shadowfly** » Sun Oct 20, 2024 6:55 pm

I definitely don't know what LTO is, only that it is supposed to make programs run a tiny bit faster. I also don't know what debug symbols are, except that i hoped to get more detailed crash reports at the expense of making everything slower while it is enabled.

Initially i enabled debug symbols and sources for kde-plasma/plasma-meta, kde-plasma/plasma-workspace and sys-libs/glibc. However there were still a lot of ??? in the stack traces, so i thought maybe more would be better?

This is a crash report with LTO, but without debug symbols/source code:

Code: Select all

Thread 1 (Thread 0x7f05c3f659c0 (LWP 1311)):
[KCrash Handler]
#4  0x00007f05c891a8d2 in QV4::QObjectWrapper::wrap(QV4::ExecutionEngine*, QObject*) () at /usr/lib64/libQt6Qml.so.6
#5  0x00007f05c8abe2f0 in ??? () at /usr/lib64/libQt6Qml.so.6
#6  0x00007f05c8ac0e4a in QV4::QObjectWrapper::getQmlProperty(QV4::ExecutionEngine*, QQmlRefPointer<QQmlContextData> const&, QV4::Heap::Object*, QObject*, QV4::String*, QFlags<QV4::QObjectWrapper::Flag>, bool*, QQmlPropertyData const**) () at /usr/lib64/libQt6Qml.so.6
#7  0x00007f05c8c740be in QV4::QQmlTypeWrapper::virtualGet(QV4::Managed const*, QV4::PropertyKey, QV4::Value const*, bool*) () at /usr/lib64/libQt6Qml.so.6
#8  0x00007f05c8a91879 in QV4::Lookup::getterFallback(QV4::Lookup*, QV4::ExecutionEngine*, QV4::Value const&) () at /usr/lib64/libQt6Qml.so.6
#9  0x00007f05c8c7237e in QV4::QQmlTypeWrapper::virtualResolveLookupGetter(QV4::Object const*, QV4::ExecutionEngine*, QV4::Lookup*) () at /usr/lib64/libQt6Qml.so.6
#10 0x00007f05c8c44136 in ??? () at /usr/lib64/libQt6Qml.so.6
#11 0x00007f05c8c4833f in ??? () at /usr/lib64/libQt6Qml.so.6
#12 0x00007f05c8aed78e in ??? () at /usr/lib64/libQt6Qml.so.6
#13 0x00007f05c8a239fa in QQmlJavaScriptExpression::evaluate(QV4::CallData*, bool*) () at /usr/lib64/libQt6Qml.so.6
#14 0x00007f05c8a23cdd in QQmlBinding::evaluate(bool*) () at /usr/lib64/libQt6Qml.so.6
#15 0x00007f05c8a28938 in QQmlBinding::doUpdate(QQmlJavaScriptExpression::DeleteWatcher const&, QFlags<QQmlPropertyData::WriteFlag>, QV4::Scope&) () at /usr/lib64/libQt6Qml.so.6
#16 0x00007f05c8a4af73 in QQmlBinding::update(QFlags<QQmlPropertyData::WriteFlag>) () at /usr/lib64/libQt6Qml.so.6
#17 0x00007f05c898ec97 in QQmlObjectCreator::finalize(QQmlInstantiationInterrupt&) () at /usr/lib64/libQt6Qml.so.6
#18 0x00007f05c8a27c4d in QQmlComponentPrivate::complete(QQmlEnginePrivate*, QQmlComponentPrivate::ConstructionState*) () at /usr/lib64/libQt6Qml.so.6
#19 0x00007f05c8a54cab in QQmlComponentPrivate::completeCreate() () at /usr/lib64/libQt6Qml.so.6
#20 0x00007f05ca3b5b0a in PlasmaQuick::SharedQmlEngine::completeInitialization(QHash<QString, QVariant> const&) () at /usr/lib64/libPlasmaQuick.so.6
#21 0x00007f05ca39979f in PlasmaQuick::AppletQuickItem::itemForApplet(Plasma::Applet*) () at /usr/lib64/libPlasmaQuick.so.6
#22 0x00007f05ca3c8659 in ??? () at /usr/lib64/libPlasmaQuick.so.6
#23 0x00007f05c898f522 in QQmlObjectCreator::finalize(QQmlInstantiationInterrupt&) () at /usr/lib64/libQt6Qml.so.6
#24 0x00007f05c8a27c4d in QQmlComponentPrivate::complete(QQmlEnginePrivate*, QQmlComponentPrivate::ConstructionState*) () at /usr/lib64/libQt6Qml.so.6
#25 0x00007f05c8a54cab in QQmlComponentPrivate::completeCreate() () at /usr/lib64/libQt6Qml.so.6
#26 0x00007f05ca3b5b0a in PlasmaQuick::SharedQmlEngine::completeInitialization(QHash<QString, QVariant> const&) () at /usr/lib64/libPlasmaQuick.so.6
#27 0x00007f05ca39979f in PlasmaQuick::AppletQuickItem::itemForApplet(Plasma::Applet*) () at /usr/lib64/libPlasmaQuick.so.6
#28 0x00007f05ca3ae639 in ??? () at /usr/lib64/libPlasmaQuick.so.6
#29 0x000055dc0e5fd137 in ??? ()
#30 0x00007f05c7cdeb35 in ??? () at /usr/lib64/libQt6Core.so.6
#31 0x00007f05c7c9cfee in QTimer::timeout(QTimer::QPrivateSignal) () at /usr/lib64/libQt6Core.so.6
#32 0x00007f05c7c7d556 in QObject::event(QEvent*) () at /usr/lib64/libQt6Core.so.6
#33 0x00007f05c9d1d839 in QApplicationPrivate::notify_helper(QObject*, QEvent*) () at /usr/lib64/libQt6Widgets.so.6
#34 0x00007f05c7cf5298 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () at /usr/lib64/libQt6Core.so.6
#35 0x00007f05c7c01b6b in QTimerInfoList::activateTimers() () at /usr/lib64/libQt6Core.so.6
#36 0x00007f05c7ac18b1 in ??? () at /usr/lib64/libQt6Core.so.6
#37 0x00007f05c67207a2 in ??? () at /usr/lib64/libglib-2.0.so.0
#38 0x00007f05c6723a97 in ??? () at /usr/lib64/libglib-2.0.so.0
#39 0x00007f05c67240f0 in g_main_context_iteration () at /usr/lib64/libglib-2.0.so.0
#40 0x00007f05c7ac1ab1 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () at /usr/lib64/libQt6Core.so.6
#41 0x00007f05c7d209fa in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () at /usr/lib64/libQt6Core.so.6
#42 0x00007f05c7d20bb7 in QCoreApplication::exec() () at /usr/lib64/libQt6Core.so.6
#43 0x000055dc0e5be7b0 in ??? ()
#44 0x00007f05c746b2f0 in ??? () at /usr/lib64/libc.so.6
#45 0x00007f05c746b3a9 in __libc_start_main () at /usr/lib64/libc.so.6
#46 0x000055dc0e5bec55 in ??? ()

And this is with LTO and debug symbols/source code:

Code: Select all

Thread 1 (Thread 0x7f0b660da9c0 (LWP 1337)):
[KCrash Handler]
#6  0x00007f0b6cae4925 in QQmlData::wasDeleted (priv=0x200000001) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/qml/qqmldata_p.h:305
#7  0x00007f0b6cae554c in QQmlData::wasDeleted (object=0x562dc14157e0) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/qml/qqmldata_p.h:318
#8  QV4::QObjectWrapper::wrap (engine=0x562dc12d6040, object=0x562dc14157e0) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/jsruntime/qv4qobjectwrapper_p.h:233
#9  0x00007f0b6cc69d62 in QV4::loadProperty (v4=<optimized out>, v4@entry=0x562dc12d6040, wrapper=<optimized out>, wrapper@entry=0x7f0afbe95420, object=<optimized out>, property=...) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/jsruntime/qv4qobjectwrapper.cpp:134
#10 0x00007f0b6cc6a825 in QV4::QObjectWrapper::getProperty (engine=engine@entry=0x562dc12d6040, wrapper=wrapper@entry=0x7f0afbe95420, object=<optimized out>, object@entry=0x562dc141cb60, property=property@entry=0x562dc3286cc0, flags=flags@entry=...) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/jsruntime/qv4qobjectwrapper.cpp:391
#11 0x00007f0b6cc6b518 in QV4::QObjectWrapper::getQmlProperty (engine=engine@entry=0x562dc12d6040, qmlContext=..., wrapper=0x7f0afbe95420, object=object@entry=0x562dc141cb60, name=name@entry=0x7f0b383b85a8, flags=..., flags@entry=..., hasProperty=0x0, property=0x0) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/jsruntime/qv4qobjectwrapper.cpp:519
#12 0x00007f0b6cdba4a2 in QV4::QQmlTypeWrapper::virtualGet (m=m@entry=0x7f0b383b8598, id=..., receiver=receiver@entry=0x7f0b383b8598, hasProperty=hasProperty@entry=0x0) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/qml/qqmltypewrapper_p.h:65
#13 0x00007f0b6cc16505 in QV4::Object::get (this=this@entry=0x7f0b383b8598, name=name@entry=0x7f0b383b85a0, hasProperty=hasProperty@entry=0x0, receiver=0x7f0b383b8598, receiver@entry=0x0) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/jsruntime/qv4object_p.h:271
#14 0x00007f0b6cc407a9 in QV4::Lookup::getterFallback (l=<optimized out>, engine=0x562dc12d6040, object=<optimized out>) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/jsruntime/qv4lookup.cpp:203
#15 0x00007f0b6cd94d04 in QV4::Moth::VME::interpret (frame=frame@entry=0x7ffce5f56630, engine=engine@entry=0x562dc12d6040, code=0x7f0aaf0c84e0 "<-N\b\022K\030\t\264.\001\t\030\006\002") at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/jsruntime/qv4vme_moth.cpp:674
#16 0x00007f0b6cd994df in QV4::Moth::VME::exec (frame=frame@entry=0x7ffce5f56630, engine=engine@entry=0x562dc12d6040) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/jsruntime/qv4vme_moth.cpp:487
#17 0x00007f0b6cc927e3 in QV4::doCall (self=0x562dc32784c0, thisObject=0x7f0b383b8500, argv=argv@entry=0x7f0b383b8518, argc=0, context=<optimized out>) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/jsruntime/qv4function.cpp:54
#18 0x00007f0b6ccac324 in QV4::Function::call (this=this@entry=0x562dc32784c0, thisObject=thisObject@entry=0x7f0b383b8500, argv=argv@entry=0x7f0b383b8518, argc=0, context=<optimized out>) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/jsruntime/qv4function.cpp:79
#19 0x00007f0b6cbec208 in QQmlJavaScriptExpression::evaluate (this=this@entry=0x562dc3295d70, callData=0x7f0b383b84e8, isUndefined=isUndefined@entry=0x7ffce5f56880) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/qml/qqmljavascriptexpression.cpp:238
#20 0x00007f0b6cbec347 in QQmlBinding::evaluate (this=this@entry=0x562dc3295d70, isUndefined=isUndefined@entry=0x7ffce5f56880) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/qml/qqmlbinding.cpp:188
#21 0x00007f0b6cbeea15 in QQmlBinding::doUpdate (this=0x562dc3295d70, watcher=..., flags=..., scope=...) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/qml/qqmlbinding.cpp:699
#22 0x00007f0b6cbfd11f in QQmlBinding::update (this=this@entry=0x562dc3295d70, flags=flags@entry=...) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/qml/qqmlbinding.cpp:164
#23 0x00007f0b6cbfd2a9 in QQmlBinding::setEnabled (this=0x562dc3295d70, e=<optimized out>, flags=...) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/qml/qqmlbinding.cpp:619
#24 0x00007f0b6cb6564f in QQmlObjectCreator::finalize (this=0x562dc3293330, interrupt=...) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/qml/qqmlobjectcreator.cpp:1508
#25 0x00007f0b6cbedc22 in QQmlComponentPrivate::complete (enginePriv=0x562dc12adef0, state=state@entry=0x562dc3248548) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/qml/qqmlcomponent.cpp:1172
#26 0x00007f0b6cc06fca in QQmlComponentPrivate::completeCreate (this=0x562dc3248490) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/qml/qqmlcomponent.cpp:1277
#27 0x00007f0b6cc06ff1 in QQmlComponent::completeCreate (this=<optimized out>) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/qml/qqmlcomponent.cpp:1255
#28 0x00007f0b6e336d2a in PlasmaQuick::SharedQmlEngine::completeInitialization (this=0x562dc3292a20, initialProperties=<optimized out>) at /usr/src/debug/kde-plasma/libplasma-6.1.5/libplasma-6.1.5/src/plasmaquick/sharedqmlengine.cpp:239
#29 0x00007f0b6e319669 in PlasmaQuick::AppletQuickItem::itemForApplet (applet=<optimized out>) at /usr/include/qt6/QtCore/qhash.h:836
#30 0x00007f0b6e3456ca in ContainmentItem::init (this=0x562dc2529cf0) at /usr/src/debug/kde-plasma/libplasma-6.1.5/libplasma-6.1.5/src/plasmaquick/plasmoid/containmentitem.cpp:78
#31 0x00007f0b6e30ea29 in PlasmaQuick::AppletQuickItem::componentComplete (this=0x562dc2529cf0) at /usr/src/debug/kde-plasma/libplasma-6.1.5/libplasma-6.1.5/src/plasmaquick/appletquickitem.cpp:760
#32 0x00007f0b6cb65938 in QQmlObjectCreator::finalize (this=0x562dc24bfbc0, interrupt=...) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/qml/qqmlobjectcreator.cpp:1567
#33 0x00007f0b6cbedc22 in QQmlComponentPrivate::complete (enginePriv=0x562dc12adef0, state=state@entry=0x562dc24f1ae8) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/qml/qqmlcomponent.cpp:1172
#34 0x00007f0b6cc06fca in QQmlComponentPrivate::completeCreate (this=0x562dc24f1a30) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/qml/qqmlcomponent.cpp:1277
#35 0x00007f0b6cc06ff1 in QQmlComponent::completeCreate (this=<optimized out>) at /usr/src/debug/dev-qt/qtdeclarative-6.7.2/qtdeclarative-everywhere-src-6.7.2/src/qml/qml/qqmlcomponent.cpp:1255
#36 0x00007f0b6e336d2a in PlasmaQuick::SharedQmlEngine::completeInitialization (this=0x562dc25045e0, initialProperties=<optimized out>) at /usr/src/debug/kde-plasma/libplasma-6.1.5/libplasma-6.1.5/src/plasmaquick/sharedqmlengine.cpp:239
#37 0x00007f0b6e319669 in PlasmaQuick::AppletQuickItem::itemForApplet (applet=<optimized out>) at /usr/include/qt6/QtCore/qhash.h:836
#38 0x00007f0b6e32c899 in PlasmaQuick::ContainmentViewPrivate::setContainment (this=0x7f0b50003260, cont=cont@entry=0x562dc114ae20) at /usr/src/debug/kde-plasma/libplasma-6.1.5/libplasma-6.1.5/src/plasmaquick/containmentview.cpp:113
#39 0x00007f0b6e32cafd in PlasmaQuick::ContainmentView::setContainment (this=this@entry=0x562dc2489950, cont=cont@entry=0x562dc114ae20) at /usr/src/debug/kde-plasma/libplasma-6.1.5/libplasma-6.1.5/src/plasmaquick/containmentview.cpp:260
#40 0x0000562dbf3c3f07 in ShellCorona::createWaitingPanels (this=0x562dc0d10a70) at /usr/src/debug/kde-plasma/plasma-workspace-6.1.5-r2/plasma-workspace-6.1.5/shell/shellcorona.cpp:1544
#41 0x00007f0b6bc66e0d in QtPrivate::QSlotObjectBase::call (a=0x7ffce5f577c0, r=0x562dc0d10a70, this=0x562dc1226cc0) at /usr/src/debug/dev-qt/qtbase-6.7.2-r4/qtbase-everywhere-src-6.7.2/src/corelib/kernel/qobjectdefs_impl.h:469
#42 doActivate<false> (sender=0x562dc0d10b80, signal_index=<optimized out>, argv=<optimized out>) at /usr/src/debug/dev-qt/qtbase-6.7.2-r4/qtbase-everywhere-src-6.7.2/src/corelib/kernel/qobject.cpp:4086
#43 0x00007f0b6bc28684 in QMetaObject::activate (sender=sender@entry=0x562dc0d10b80, m=m@entry=0x7f0b6bf6e440 <QTimer::staticMetaObject>, local_signal_index=local_signal_index@entry=0, argv=argv@entry=0x7ffce5f577c0) at /usr/src/debug/dev-qt/qtbase-6.7.2-r4/qtbase-everywhere-src-6.7.2/src/corelib/kernel/qobject.cpp:4146
#44 0x00007f0b6bc28cb6 in QTimer::timeout (this=this@entry=0x562dc0d10b80, _t1=...) at /usr/src/debug/dev-qt/qtbase-6.7.2-r4/qtbase-everywhere-src-6.7.2_build/src/corelib/Core_autogen/include/moc_qtimer.cpp:224
#45 0x00007f0b6bc43a2f in QTimer::timerEvent (this=0x562dc0d10b80, e=<optimized out>) at /usr/src/debug/dev-qt/qtbase-6.7.2-r4/qtbase-everywhere-src-6.7.2/src/corelib/kernel/qtimer.cpp:258
#46 0x00007f0b6bc13458 in QObject::event (this=0x562dc0d10b80, e=0x7ffce5f57aa0) at /usr/src/debug/dev-qt/qtbase-6.7.2-r4/qtbase-everywhere-src-6.7.2/src/corelib/kernel/qobject.cpp:1427
#47 0x00007f0b6dc3453f in QApplicationPrivate::notify_helper (this=this@entry=0x562dc0bb4c80, receiver=receiver@entry=0x562dc0d10b80, e=e@entry=0x7ffce5f57aa0) at /usr/src/debug/dev-qt/qtbase-6.7.2-r4/qtbase-everywhere-src-6.7.2/src/widgets/kernel/qapplication.cpp:3287
#48 0x00007f0b6dc360ab in QApplication::notify (this=0x7ffce5f57df0, receiver=0x562dc0d10b80, e=0x7ffce5f57aa0) at /usr/src/debug/dev-qt/qtbase-6.7.2-r4/qtbase-everywhere-src-6.7.2/src/widgets/kernel/qapplication.cpp:3238
#49 0x00007f0b6bc90c5c in QCoreApplication::notifyInternal2 (receiver=0x562dc0d10b80, event=event@entry=0x7ffce5f57aa0) at /usr/src/debug/dev-qt/qtbase-6.7.2-r4/qtbase-everywhere-src-6.7.2/src/corelib/kernel/qcoreapplication.cpp:1142
#50 0x00007f0b6bc90ced in QCoreApplication::sendEvent (receiver=<optimized out>, event=event@entry=0x7ffce5f57aa0) at /usr/src/debug/dev-qt/qtbase-6.7.2-r4/qtbase-everywhere-src-6.7.2/src/corelib/kernel/qcoreapplication.cpp:1583
#51 0x00007f0b6bba2cf8 in QTimerInfoList::activateTimers (this=this@entry=0x562dc0c73b20) at /usr/src/debug/dev-qt/qtbase-6.7.2-r4/qtbase-everywhere-src-6.7.2/src/corelib/kernel/qtimerinfo_unix.cpp:434
#52 0x00007f0b6ba91dad in timerSourceDispatch (source=source@entry=0x562dc0c73ac0) at /usr/src/debug/dev-qt/qtbase-6.7.2-r4/qtbase-everywhere-src-6.7.2/src/corelib/kernel/qeventdispatcher_glib.cpp:150
#53 0x00007f0b6a88692d in g_main_dispatch (context=0x7f0b60000ef0) at ../glib-2.78.6/glib/gmain.c:3476
#54 0x00007f0b6a8869ea in g_main_context_dispatch_unlocked (context=context@entry=0x7f0b60000ef0) at ../glib-2.78.6/glib/gmain.c:4284
#55 0x00007f0b6a88d430 in g_main_context_iterate_unlocked (context=context@entry=0x7f0b60000ef0, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../glib-2.78.6/glib/gmain.c:4349
#56 0x00007f0b6a88d4bb in g_main_context_iteration (context=0x7f0b60000ef0, may_block=1) at ../glib-2.78.6/glib/gmain.c:4414
#57 0x00007f0b6ba920d9 in QEventDispatcherGlib::processEvents (this=0x562dc0bcb870, flags=...) at /usr/src/debug/dev-qt/qtbase-6.7.2-r4/qtbase-everywhere-src-6.7.2/src/corelib/kernel/qeventdispatcher_glib.cpp:394
#58 0x00007f0b6c6aa4ea in QPAEventDispatcherGlib::processEvents (this=<optimized out>, flags=...) at /usr/src/debug/dev-qt/qtbase-6.7.2-r4/qtbase-everywhere-src-6.7.2/src/gui/platform/unix/qeventdispatcher_glib.cpp:87
#59 0x00007f0b6bc7f2bd in QEventLoop::processEvents (this=this@entry=0x7ffce5f57cf0, flags=..., flags@entry=...) at /usr/src/debug/dev-qt/qtbase-6.7.2-r4/qtbase-everywhere-src-6.7.2/src/corelib/kernel/qeventloop.cpp:100
#60 0x00007f0b6bca06d0 in QEventLoop::exec (this=this@entry=0x7ffce5f57cf0, flags=flags@entry=...) at /usr/src/debug/dev-qt/qtbase-6.7.2-r4/qtbase-everywhere-src-6.7.2/src/corelib/global/qflags.h:34
#61 0x00007f0b6bca0806 in QCoreApplication::exec () at /usr/src/debug/dev-qt/qtbase-6.7.2-r4/qtbase-everywhere-src-6.7.2/src/corelib/global/qflags.h:74
#62 0x00007f0b6c344878 in QGuiApplication::exec () at /usr/src/debug/dev-qt/qtbase-6.7.2-r4/qtbase-everywhere-src-6.7.2/src/gui/kernel/qguiapplication.cpp:1926
#63 0x00007f0b6dbd552b in QApplication::exec () at /usr/src/debug/dev-qt/qtbase-6.7.2-r4/qtbase-everywhere-src-6.7.2/src/widgets/kernel/qapplication.cpp:2555
#64 0x0000562dbf3857b0 in main (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/kde-plasma/plasma-workspace-6.1.5-r2/plasma-workspace-6.1.5/shell/main.cpp:188

So it worked for plasma, because now there are no more ???, there are line numbers referencing the source code and there are a lot more method names. How helpful this is? Idk.

[Administrator edit: changed [quote] tags to [code] tags to preserve output layout. -Hu]

shadowfly · Post by **shadowfly** » Sun Oct 20, 2024 7:00 pm

I forgot, this is with debug symbols and sources for kde-plasma/plasma-meta, kde-plasma/plasma-workspace and sys-libs/glibc

Code: Select all

#0  0x00007f05c74d38ec in ??? () at /usr/lib64/libc.so.6
#1  0x00007f05c7481856 in raise () at /usr/lib64/libc.so.6
#2  0x00007f05ca2e064c in KCrash::defaultCrashHandler(int) () at /usr/lib64/libKF6Crash.so.6
#3  0x00007f05c7481900 in <signal handler called> () at /usr/lib64/libc.so.6
#4  0x00007f05c891a8d2 in QV4::QObjectWrapper::wrap(QV4::ExecutionEngine*, QObject*) () at /usr/lib64/libQt6Qml.so.6
#5  0x00007f05c8abe2f0 in ??? () at /usr/lib64/libQt6Qml.so.6
#6  0x00007f05c8ac0e4a in QV4::QObjectWrapper::getQmlProperty(QV4::ExecutionEngine*, QQmlRefPointer<QQmlContextData> const&, QV4::Heap::Object*, QObject*, QV4::String*, QFlags<QV4::QObjectWrapper::Flag>, bool*, QQmlPropertyData const**) () at /usr/lib64/libQt6Qml.so.6
#7  0x00007f05c8c740be in QV4::QQmlTypeWrapper::virtualGet(QV4::Managed const*, QV4::PropertyKey, QV4::Value const*, bool*) () at /usr/lib64/libQt6Qml.so.6
#8  0x00007f05c8a91879 in QV4::Lookup::getterFallback(QV4::Lookup*, QV4::ExecutionEngine*, QV4::Value const&) () at /usr/lib64/libQt6Qml.so.6
#9  0x00007f05c8c7237e in QV4::QQmlTypeWrapper::virtualResolveLookupGetter(QV4::Object const*, QV4::ExecutionEngine*, QV4::Lookup*) () at /usr/lib64/libQt6Qml.so.6
#10 0x00007f05c8c44136 in ??? () at /usr/lib64/libQt6Qml.so.6
#11 0x00007f05c8c4833f in ??? () at /usr/lib64/libQt6Qml.so.6
#12 0x00007f05c8aed78e in ??? () at /usr/lib64/libQt6Qml.so.6
#13 0x00007f05c8a239fa in QQmlJavaScriptExpression::evaluate(QV4::CallData*, bool*) () at /usr/lib64/libQt6Qml.so.6
#14 0x00007f05c8a23cdd in QQmlBinding::evaluate(bool*) () at /usr/lib64/libQt6Qml.so.6
#15 0x00007f05c8a28938 in QQmlBinding::doUpdate(QQmlJavaScriptExpression::DeleteWatcher const&, QFlags<QQmlPropertyData::WriteFlag>, QV4::Scope&) () at /usr/lib64/libQt6Qml.so.6
#16 0x00007f05c8a4af73 in QQmlBinding::update(QFlags<QQmlPropertyData::WriteFlag>) () at /usr/lib64/libQt6Qml.so.6
#17 0x00007f05c898ec97 in QQmlObjectCreator::finalize(QQmlInstantiationInterrupt&) () at /usr/lib64/libQt6Qml.so.6
#18 0x00007f05c8a27c4d in QQmlComponentPrivate::complete(QQmlEnginePrivate*, QQmlComponentPrivate::ConstructionState*) () at /usr/lib64/libQt6Qml.so.6
#19 0x00007f05c8a54cab in QQmlComponentPrivate::completeCreate() () at /usr/lib64/libQt6Qml.so.6
#20 0x00007f05ca3b5b0a in PlasmaQuick::SharedQmlEngine::completeInitialization(QHash<QString, QVariant> const&) () at /usr/lib64/libPlasmaQuick.so.6
#21 0x00007f05ca39979f in PlasmaQuick::AppletQuickItem::itemForApplet(Plasma::Applet*) () at /usr/lib64/libPlasmaQuick.so.6
#22 0x00007f05ca3c8659 in ??? () at /usr/lib64/libPlasmaQuick.so.6
#23 0x00007f05c898f522 in QQmlObjectCreator::finalize(QQmlInstantiationInterrupt&) () at /usr/lib64/libQt6Qml.so.6
#24 0x00007f05c8a27c4d in QQmlComponentPrivate::complete(QQmlEnginePrivate*, QQmlComponentPrivate::ConstructionState*) () at /usr/lib64/libQt6Qml.so.6
#25 0x00007f05c8a54cab in QQmlComponentPrivate::completeCreate() () at /usr/lib64/libQt6Qml.so.6
#26 0x00007f05ca3b5b0a in PlasmaQuick::SharedQmlEngine::completeInitialization(QHash<QString, QVariant> const&) () at /usr/lib64/libPlasmaQuick.so.6
#27 0x00007f05ca39979f in PlasmaQuick::AppletQuickItem::itemForApplet(Plasma::Applet*) () at /usr/lib64/libPlasmaQuick.so.6
#28 0x00007f05ca3ae639 in ??? () at /usr/lib64/libPlasmaQuick.so.6
#29 0x000055dc0e5fd137 in ShellCorona::createWaitingPanels (this=0x55dc0f87c450) at /usr/src/debug/kde-plasma/plasma-workspace-6.1.4-r1/plasma-workspace-6.1.4/shell/shellcorona.cpp:1544
#30 0x00007f05c7cdeb35 in ??? () at /usr/lib64/libQt6Core.so.6
#31 0x00007f05c7c9cfee in QTimer::timeout(QTimer::QPrivateSignal) () at /usr/lib64/libQt6Core.so.6
#32 0x00007f05c7c7d556 in QObject::event(QEvent*) () at /usr/lib64/libQt6Core.so.6
#33 0x00007f05c9d1d839 in QApplicationPrivate::notify_helper(QObject*, QEvent*) () at /usr/lib64/libQt6Widgets.so.6
#34 0x00007f05c7cf5298 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () at /usr/lib64/libQt6Core.so.6
#35 0x00007f05c7c01b6b in QTimerInfoList::activateTimers() () at /usr/lib64/libQt6Core.so.6
#36 0x00007f05c7ac18b1 in ??? () at /usr/lib64/libQt6Core.so.6
#37 0x00007f05c67207a2 in ??? () at /usr/lib64/libglib-2.0.so.0
#38 0x00007f05c6723a97 in ??? () at /usr/lib64/libglib-2.0.so.0
#39 0x00007f05c67240f0 in g_main_context_iteration () at /usr/lib64/libglib-2.0.so.0
#40 0x00007f05c7ac1ab1 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () at /usr/lib64/libQt6Core.so.6
#41 0x00007f05c7d209fa in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () at /usr/lib64/libQt6Core.so.6
#42 0x00007f05c7d20bb7 in QCoreApplication::exec() () at /usr/lib64/libQt6Core.so.6
#43 0x000055dc0e5be7b0 in main (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/kde-plasma/plasma-workspace-6.1.4-r1/plasma-workspace-6.1.4/shell/main.cpp:188

[Administrator edit: changed [quote] tags to [code] tags to preserve output layout. -Hu]

Post by Hu » Sun Oct 20, 2024 7:35 pm

logrusx wrote:Debug symbols also degrade performance.

I think this is incorrect. Do you have a source for this? Debug symbols will definitely slow down the compilation process because of the extra work that the compiler must do to create them, but the resulting program should have the same code whether or not debug symbols were generated.

logrusx wrote:One does not usually enable debug symbols system wide either, especially when the problem is in higher level packages.

True, though this is mainly because of the impracticality of generating and storing that much extra data.

logrusx wrote:I doubt debug symbols were even remotely useful to diagnose your issue with Plasma.

It depends on the failure. If the failure is a crash that generates a core file, and debug symbols are generated and stored properly, they can make the difference between a bug report that is handled easily versus one that is rejected out of hand.

shadowfly wrote:I definitely don't know what LTO is, only that it is supposed to make programs run a tiny bit faster.

LTO is Link Time Optimization. It lets the compiler perform optimizations that can only be identified when the whole program is visible at once. To maintain a consistent ABI, the compiler is normally confined to making optimizations within a single translation unit, and then only optimizations that do not change the observable ABI. LTO lets it treat the entire program as one massive translation unit, allowing inlining, constant propagation, and other optimizations across source files.

shadowfly wrote:I also don't know what debug symbols are, except that i hoped to get more detailed crash reports at the expense of making everything slower while it is enabled.

Debug symbols allow a debugger to map the generated code back to the source code used to generate it, and in some cases, show the values of variables as the original author would have understood them.

shadowfly wrote:Initially i enabled debug symbols and sources for kde-plasma/plasma-meta, kde-plasma/plasma-workspace and sys-libs/glibc. However there were still a lot of ??? in the stack traces, so i thought maybe more would be better?

Yes, but there is a difference between "more" and "all." You would have had an easier time if you had reported to us the partially complete trace, and sought our help identifying where you needed to add more symbols. That would have let you add symbols to a few extra packages, instead of rebuilding everything.

logrusx · Post by **logrusx** » Sun Oct 20, 2024 7:49 pm

First, something I forgot to mention

shadowfly wrote: someone else (who does have LTO, but does not have debug symbols) confirm the bug.

That's not a bug. That's a consequence of what you've done.

shadowfly wrote:I definitely don't know what LTO is, only that it is supposed to make programs run a tiny bit faster. I also don't know what debug symbols are, except that i hoped to get more detailed crash reports at the expense of making everything slower while it is enabled.

LTO is Link Time Optimization. The compiler emits additional output which is used during linking phase to optimize the binary. This causes larger memory usage during link time, but I've compiled Chromium with LTO on a quat core with 6GB RAM, later upgraded to the max 8 GB.

What are the optimizations? I don't know exactly. Parallelizing loops whenever possible, inlining stuff, other tricks. I don't think you need it on a Zen 4 Ryzen 7. I used to compile Chromium with LTO just for that tiny bit improvement in performance on the above mentioned CPU, but when I got my current Zen3 Ryzen 7 I realized it brought little to no benefit on the expense of longer and more CPU and memory intensive compilation. I dropped it. However I know LTO is now enabled by default for Chromium and I think Firefox. But those are well tested packages.

LTO can also uncover bugs. I'd not use it system-wide. Especially on things like compilers and stuff, including JVM. That's complicated code with many opportunities for LTO to uncover bugs. Certainly do not use it on packages which are not tested with LTO.

Debug symbols is extra information in the binary code so that you can map the source code onto traces. Not very useful for people like us but unnecessarily eats resources.

shadowfly wrote: How helpful this is? Idk.

Just what I expected. Those things are for people who know. I don't know this stuff either. I can learn it as I am a programmer but I prefer not to need it at all.

You don't need either of those. They only take up resources. And occasionally cause problems.

Hu wrote:
logrusx wrote:Debug symbols also degrade performance.
I think this is incorrect. Do you have a source for this? Debug symbols will definitely slow down the compilation process because of the extra work that the compiler must do to create them, but the resulting program should have the same code whether or not debug symbols were generated.

It's funny how you go and correct tags and what not but your whole post is on a single line :)

No, I'm not sure about that. Just a deduction which might be wrong or not entirely true. Maybe they take up more memory. That would indeed degrade performance as more stuff needs to be addressed. Is it significant or even noticeable? I don't know. Not enough knowledge to decide on that and I certainly have no experience with it.

Best Regards,
Georgi

shadowfly · Post by **shadowfly** » Sun Oct 20, 2024 9:56 pm

Thanks for the explanation of LTO. As far as i know now:

Regular optimization: Optimization within methods. Maybe within source files.
LTO: Optimization between source files.

I hope my brain can remember that.

Hu wrote:You would have had an easier time if you had reported to us the partially complete trace, and sought our help identifying where you needed to add more symbols.

I definitely try to use this forum as my main resource for asking for help and reporting bugs, but when the crash happened it was saved by DrKonqi, the KDE crash reporter.
DrKonqi asked me to set up a KDE account, which was then used to generate and upload the crash report to KDE automatically. I thought this would be the best option, as DrKonqi maybe saves more information than i could. Later a KDE developer asked me to provide a more detailed crash report with debug symbols. And by then i was already to deep in the KDE bug reporting system to even consider this forum.
Back then i thought that i would have to find all the packages containing the right .so files, and then activate debug symbols for them. So i thought switching on debug for everything would be easier.

As for the LTO and openjdk:

logrusx wrote:That's not a bug. That's a consequence of what you've done.

I enabled LTO system wide at least half a year ago. I believe openjdk updated multiple times since then, without problems.
A month or so ago i enabled system wide debug and sources, and while i did not read the terminal while it was running, i checked to make sure emerge said 1200/1200 packages upgraded, because it did fail once due to me having the verify use-flag enabled.
If you think that LTO+debug should use more than 64GB or RAM (or maybe should not work at all), then the only explanation could be that i never had LTO+debug, and emerge being buggy.

To find out if that's the case:

How can i check if Java was compiled with debug?
How can i check if Java was compiled with LTO?
How do i downgrade Java to openjdk-17.0.12 without breaking things?

The easiest approach would still be to find someone who does not have debug enabled, but has LTO. Because if they can compile openjdk-17.0.13 (using less than 32GB RAM), that means the problem is either LTO+debug or just my system. In either case it would not be worth it to investigate. If they can not, it would be interesting to know if they can compile openjdk-17.0.12 with LTO. If they can, there is something wrong between those two versions. If they can not, the LTO use flag should be removed from openjdk.

logrusx · Post by **logrusx** » Mon Oct 21, 2024 8:25 am

I actually remember a thread someone, just like you, had enabled debug symbols system-wide and it was giving them trouble compiling software, it was terribly slow.

shadowfly wrote: I enabled LTO system wide at least half a year ago.

As I said in another thread that reminded me the above, if you hit obscure bugs, you should know this is of your own doing. LTO can uncover such bugs. I wouldn't be surprised if your Plasma issue turned out to be LTO related.

Best Regards,
Georgi

shadowfly · Post by **shadowfly** » Mon Oct 21, 2024 10:17 am

I see. So LTO introduces bugs that have not been there before. And debug symbols do that too. And having both at the same time makes it even worse.

Well, i did not know that. I guess i will just remove LTO and debug symbols from my make.conf, and see if the Plasma "bug" remains. If not, it was not a Plasma bug, but a LTO bug.
I knew that one should not use -O3, as it creates problems, but i thought LTO was OK, as there is a big red warning on O3 https://wiki.gentoo.org/wiki/GCC_optimization#-O, but nothing on debug/sources https://wiki.gentoo.org/wiki/Project:Qu ... Backtraces or LTO https://wiki.gentoo.org/wiki/LTO#New_Bugs_Procedure, except for

https://wiki.gentoo.org/wiki/LTO wrote:- Always be prepared to try without it if something is acting odd.

which i probably just forgot about since it does not look that discouraging.

I think i will let it recompile tonight.

logrusx · Post by **logrusx** » Mon Oct 21, 2024 10:46 am

shadowfly wrote:I see. So LTO introduces bugs that have not been there before. And debug symbols do that too. And having both at the same time makes it even worse.

No, no, no. LTO can uncover bugs that were already there, just didn't reveal themselves. Debug symbols cannot introduce bugs, but slows down compilation dramatically as it needs to generate immense amount of data that needs to be then incorporated into the binary. Maybe I was a bit too harsh in my judgement, having LTO and debug symbols may be useful for finding those bugs. But I think the KDE developer used you to do stuff they realized they can get by without doing themselves.

What the issue might be with both enabled is what you've already hit - insane amount of memory needed to compile the stuff they are both enabled for. Also compilation time may and will usually increase.

shadowfly wrote:Well, i did not know that. I guess i will just remove LTO and debug symbols from my make.conf, and see if the Plasma "bug" remains. If not, it was not a Plasma bug, but a LTO bug.

For now do not rush into doing it. You may be able to get by without unnecessary compiling. The KDE devs already got you into doing too much of it.

What I suggest is to remove debug symbols altogether and enable LTO only on certain packages in package.use or package.env, depending if the package has a use flag for that. But avoid enabling LTO on compilers, especially gcc, JVM et.c. And then let the system gradually recompile packages as they get upgraded.

You are OK without recompiling the whole system and leaving whatever had debug symbols be until it gets upgraded. You may be OK leaving whatever has LTO be and only if you hit problems recompile it without LTO. The complicated part is how to keep track of what has and what does not have LTO. Portage saves CFLAGS in /var/db/pkg/category/package in a file with the same name, so you can check which packages have LTO and which do not.

Maybe recompile only the Plasma stuff. Should I understand your issue with it is still unresolved?

Best Regards,
Georgi

shadowfly · Post by **shadowfly** » Mon Oct 21, 2024 12:03 pm

logrusx wrote:For now do not rush into doing it.

Understood. No recompiling tonight.

The rest will have to wait for tonight.

shadowfly · Post by **shadowfly** » Mon Oct 21, 2024 8:37 pm

Now, as i had a bit of time, i did the following things:

Compile openjdk-17.0.13 without debug and with LTO: ~5GB RAM usage
Compile openjdk-17.0.13 with debug and with LTO: The RAM usage starts out like without debug, where it jumps up and down dynamically, then at some point, it just goes up in a straight diagonal line. There is no sign of slowing down or actually doing something with the RAM.

logrusx wrote:What I suggest is to remove debug symbols altogether and enable LTO only on certain packages [...] depending if the package has a use flag for that

I will definitely remove the debug symbols, however i don't understand what the benefit of switching on LTO only for selected packages is. As far as i understand it, "lto" is a use-flag, so enabling it in the make.conf will only affect those packages that have a "lto" use flag.
And for avoiding LTO with compilers: If LTO does not introduce bugs, i don't understand why i should avoid them.

Also: If i avoid compilers, there will be nothing left that profits from LTO:

Code: Select all

equery hasuse lto
 * Searching for USE flag lto ... 
[IP-] [  ] dev-java/openjdk-17.0.13_p11:17
[IP-] [  ] dev-lang/rust-1.81.0:stable/1.81
[IP-] [  ] net-libs/nodejs-22.4.1-r1:0/22
[IP-] [  ] sys-devel/gcc-13.3.1_p20240614:13

I also tried to compile openjdk-17.0.12 with debug and LTO, because i thought it had been compiled like that before, but the RAM usage starts going in a straight line immediately there.
I did it by copying the ebuild to a local repo, then doing

Code: Select all

ebuild /var/db/repos/downgrade/dev-java/openjdk/openjdk-17.0.12_p7.ebuild unpack prepare configure compile

logrusx wrote:Should I understand your issue with it is still unresolved?

I guess when i recompiled everything with debug a month back, not actually everything was recompiled. Therefore i (wrongly) assumed the upgrade from openjdk-17.0.12 to openjdk-17.0.13 made it impossible to compile Java with debug and LTO, when in reality it was already broken before. While of course it would be better if it were possible, i would count this as resolved.

logrusx · Post by **logrusx** » Tue Oct 22, 2024 4:27 am

shadowfly wrote: And for avoiding LTO with compilers: If LTO does not introduce bugs, i don't understand why i should avoid them.

Do you understand the difference between introduces and uncovers? I understand English may not be your native language, but those are two different words.

https://wiki.gentoo.org/wiki/GCC_optimization#Link_Time_Optimization_.28LTO.29 wrote:LTO is still experimental. LTO may need to be disabled before reporting bugs because it is a common source of problems. The -flto flag is used, with an optional auto argument (Detects how many jobs to use) or an integer argument (An integer number of jobs to execute parallel).

shadowfly wrote:Also: If i avoid compilers, there will be nothing left that profits from LTO:
Code: Select all
equery hasuse lto
 * Searching for USE flag lto ... 
[IP-] [  ] dev-java/openjdk-17.0.13_p11:17
[IP-] [  ] dev-lang/rust-1.81.0:stable/1.81
[IP-] [  ] net-libs/nodejs-22.4.1-r1:0/22
[IP-] [  ] sys-devel/gcc-13.3.1_p20240614:13

First of all this is not how LTO is enabled system-wide. You do it by adding compiler arguments in CFLAGS. If you only enabled lto USE flag in make.conf, that's nothing. Also the Plasma issue cannot be related to LTO in this case, because it's not compiled with LTO. At most it could be related to LTO being enabled on GCC if Plasma compiles with GCC and not Clang.

This is how you enable LTO:
https://wiki.gentoo.org/wiki/LTO#Enable_LTO

Did you do that? Did you put -flto in CFLAGS?

Second, I already explained complex code have more opportunities for bugs to hide and LTO to uncover them.

From the listed above you would benefit only from LTO on the JDK as it's a runtime environment. The rest are compilers and they won't produce faster binaries. They themselves may run a bit faster which is insignificant compared to the total time they run.

shadowfly wrote: I guess when i recompiled everything with debug a month back, not actually everything was recompiled.

Well, that begs the question how did you enable debug symbols? Did you do something like putting debug in make.conf? If you did that, this is also not how you enable debug symbols system-wide. Again, that requires compiler arguments in CFLAGS.

Again, this is how you enable debug symbols: https://wiki.gentoo.org/wiki/Project:Qu ... iler_flags

Did you put -ggdb in CFLAGS?

So if you put debug in make.conf, you can easily see what has that use flag enabled and recompile only those packages.

But you don't need that, just remove debug and/or ggdb from USE and CFLAGS and let it fall off gradually with updates.

shadowfly wrote:While of course it would be better if it were possible, i would count this as resolved.

It is possible. Edit the first post and change add "solved" to the title in some visible form.

Best Regards,
Georgi

shadowfly · Post by **shadowfly** » Tue Oct 22, 2024 11:06 am

logrusx wrote:Do you understand the difference between introduces and uncovers? I understand English may not be your native language, but those are two different words.

Yes, but my brain told me: The bugs are not introduced by LTO, so it's a real bug. And finding bugs improves the code (if i report them). Unfortunately it did not realize that extreme memory usage is not a bug.

https://wiki.gentoo.org/wiki/GCC_optimization#Link_Time_Optimization_.28LTO.29 wrote:LTO is still experimental. LTO may need to be disabled before reporting bugs because it is a common source of problems.

I should definitely turn it off before thinking about reporting a bug.

logrusx wrote:Did you do that? Did you put -flto in CFLAGS?

Yes, i put it in COMMON_FLAGS, which is then inserted into CFLAGS, CXXFLAGS, FCFLAGS and FFLAGS like in the wiki. Back then i thought -flto was just to tell the compiler to paralellize, but i had it on anyways, because i thought if it compiles faster, it's better. I read it up now. It does also enable LTO.

logrusx wrote:Did you put -ggdb in CFLAGS?

Yes, i put "-Og -ggdb" in CFLAGS, and in CXXFLAGS.

My make.conf back then:

Code: Select all

#LTO
WARNING_FLAGS="-Werror=odr -Werror=lto-type-mismatch -Werror=strict-aliasing"
COMMON_FLAGS="-march=native -pipe -flto ${WARNING_FLAGS}"

#Debug
CFLAGS="${COMMON_FLAGS} -Og -ggdb"
CXXFLAGS="${COMMON_FLAGS} -Og -ggdb"

FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}"

# Please keep this setting intact when reporting bugs.
LC_MESSAGES=C

L10N="en en-US"

USE="lto dbus policykit udisks X xorg xwayland wayland grub alsa acl pulseaudio pipewire bluetooth vaapi vdpau unicode cups -thunderbolt"
VIDEO_CARDS="amdgpu radeonsi"
MAKEOPTS="-j12 --load-average=16"
EMERGE_DEFAULT_OPTS="--jobs 2"
PORTAGE_NICENESS="19"
PORTAGE_IONICE_COMMAND="/usr/local/bin/io-priority \${PID}"
GENTOO_MIRRORS="https://ftp.halifax.rwth-aachen.de/gentoo/ https://mirror.eu.oneandone.net/linux/distributions/gentoo/gentoo/ https://mirror.netcologne.de/gentoo/ https://mirror.init7.net/gentoo/"
GRUB_PLATFORMS="efi-64"


PORTAGE_ELOG_CLASSES="log warn error"
PORTAGE_ELOG_SYSTEM="echo save"
PORTAGE_LOGDIR="/var/log/portage"
# Sources
FEATURES="clean-logs installsources nostrip"
PORTAGE_LOGDIR_CLEAN="find "\${PORTAGE_LOGDIR}" -type f ! -name "summary.log*" -mtime +7 -delete"

Big thanks for the patience and the great explanations.

shadowfly · Post by **shadowfly** » Tue Oct 22, 2024 11:08 am

Conclusion (for the search engines) (EDITED)

Problem
High memory/RAM usage (50GB+) when compiling with "-flto" and "-Og -ggdb".

Problem sources

LTO requires more memory than without LTO.
Debug symbols require more memory than no debug symbols.
LTO is still experimental.

Solution
Avoid using "-flto" and "-Og -ggdb" at the same time, if you have less than 128GB of RAM (or unless you masked out large programs).

If you want to report a bug while having LTO enabled, follow the following procedure:

User has LTO enabled and encounters a bug.
User reproduces bug with LTO enabled.
User disables LTO and enables "-Og -ggdb" for better bug reporting. If bug is still reproducible, user reports bug now.
User can no longer reproduce bug. User enables "-flto" and "-Og -ggdb" at the same time and is able to reproduce bug. User informs in bug report that LTO is required to find bug.

Post by **sam_** » Thu Oct 24, 2024 5:26 pm

I don't think that interpretation/conclusion is correct. It does make sense to combine LTO and debugging symbols if you wish to have them. I've updated that page a bit now.

However, there's two things to keep in mind:
a) OpenJDK is a large application so linking it with LTO, especially with -Og which means it has to stream more information (because it can't prune it out (early)), is going to consume a lot of resources;
b) If static libraries are involved, there's some issues which can lead to RAM use explosions.

shadowfly · Post by **shadowfly** » Thu Oct 24, 2024 6:15 pm

sam_ wrote:It does make sense to combine LTO and debugging symbols if you wish to have them.

Is this the use-case you are thinking about?

Bug that only shows up with LTO:

User has LTO enabled and encounters a bug.
User reproduces bug with LTO enabled.
User disables LTO and enables "-Og -ggdb" for better bug reporting.
User can no longer reproduce bug.
User enables "-flto" and "-Og -ggdb" at the same time and is able to reproduce bug. User informs in bug report that LTO is required to find bug.

Post by **sam_** » Thu Oct 24, 2024 6:19 pm

I think that's a valid scenario for it, yeah. But I build loads of things with e.g. -O3 -ggdb3 too as I find it convenient as I often am debugging applications.

The important thing to keep in mind here is that -Og is essentially "no optimisation, except for some optimisations which actually generate debugging information". That's not good to daily-drive

charles17 · Post by **charles17** » Fri Oct 25, 2024 7:20 am

shadowfly wrote:Hello there!
Background
I have dev-java/openjdk (17) installed since a few years back. A month (or so) ago i recompiled my system, because i had a KDE Plasma bug and wanted debug symbols and source code to be more helpful. Back then everything recompiled smoothly with emerge -e @world.
I update once a week, every weekend.

The problem
Today i tried to update again (emerge -aquDN @world), but my RAM usage shot up, and then the PC locked up.
lto1-wpa was using >50GB of RAM (32GB + swap).

lto sounds like Link Time Optimization. I have that enabled as far as i remember.

Workaround:
Code: Select all
emerge -aquDN @world --exclude dev-java/openjdk
The question
Can someone confirm this, or is it just for me?

https://bugs.gentoo.org/942067

[Solved] openjdk-17.0.13_p11 extreme RAM usage

[Solved] openjdk-17.0.13_p11 extreme RAM usage

Re: openjdk-17.0.13_p11 extreme RAM usage

Re: [Solved] openjdk-17.0.13_p11 extreme RAM usage