Debug Heaven: separating debug symbols (and debug sources)

ecatmur · Last edited by ecatmur on Wed Nov 23, 2005 11:40 am; edited 2 times in total

As a source-based distro, Gentoo should be ideal for debugging packages. But it isn't; portage strips debugging symbols from binaries, making them smaller and load faster but also making stack traces unreadable and making using a debugger all but impossible. Worse, although packages are built from source, once built the code is deleted; if you want to debug a binary alongside the source code you have to locate the source tarball in distfiles (if it's still there), unpack the source and apply all Portage patches; generated sources (lexers and bindings) are even worse.

RPM-based distros have separate -debuginfo packages, which contain just the debugging symbols and debugging source code to help debug the parent package. Now, Gentoo doesn't (yet) support building more than one package from a single ebuild, but we can certainly put debugging symbols and debugging sources into the built package.

I've written two bashrc hacks, debugsymbols and debugsources, that do just that. To use them:

Add the contents of phase_hooks.sh to your /etc/portage/bashrc
Save debugsymbols.hook.sh to /etc/portage/phase_hooks.d/before_install/debugsymbols.hook.sh
Save strip_keep_debug_symbols to /usr/local/bin/strip_keep_debug_symbols
Save debugsources.hook.sh to /etc/portage/phase_hooks.d/after_install/debugsources.hook.sh
Add "debugsymbols" and "debugsources" to FEATURES in /etc/make.conf to activate the hooks
(optional) Add -ggdb3 to CFLAGS and CXXFLAGS to compile with gdb specific debug symbols

Note:

You need a fairly recent version of binutils (for objcopy --add-gnu-debuglink); any version in Portage will do
For the debugsources hook, you need x11-misc/shared-mime-info or app-misc/mime-types (the hack uses mime info to choose which file types to save as debug sources); alternatively you can add patterns to EXTRA_DEBUG_SOURCE_PATTERNS in /etc/make.conf or /etc/portage/bashrc.
It uses a lot of space. As a rough guide, debug symbols add about 40% to a package size, and debug sources add about 60%, so enable both and you're looking at double the hard disk usage. Note though that everything is installed to /usr/lib/debug (symbols) and /usr/src/debug (sources) so you can make those links to another partition. You could also only enable "debugsymbols" and "debugsources" for specific packages.
Installing debug sources is disabled for package names ending in "-sources" or "-headers". Generally this is the right thing to do, but it's a nasty hack.
Unfortunately, for packages where lots are built from the same sources (e.g. the ruby-gnome family) the same set of debug sources will be saved many times over. I can't see a way around this; the problem is portage not being able to build more than one package from a single ebuild.

Hope someone finds this useful; post, pm or email me with any comments or suggestions.
_________________
No more cruft
dep: Revdeps that work
Using command-line ACCEPT_KEYWORDS?

TGL

ecatmur · Posted: Wed Nov 23, 2005 1:17 pm Post subject:

Oops, fixed that 404.

Right; I've added DEBUGSOURCES_EXCLUDE and DEBUGSYMBOLS_EXCLUDE; they match on $CATEGORY/$PF. DEBUGSOURCES_EXCLUDE defaults to */*-sources-* */*-headers-*. Great idea. Refresh to get the latest versions.

Interestingly, I just discovered solar also has a keep-debugging-symbols hack; his is a drop-in replacement to /usr/lib/portage/bin/prepstrip, so a slightly different approach. Me, I prefer knowing it'll still work (well, probably) after a portage upgrade, hence the bashrc hack approach.

w.r.t memory footprint: yes, the debug symbols aren't loaded during normal execution, only when using a debugger. If you delete the .debug file, the binary will run fine; it's exactly the same binary as you get using normal strip. If you then try to debug that binary, you get "(no debugging symbols found)", which I'm sure you're sick of seeing...

And yeah, this does use a lot of disk space... I've just gone and ordered a new 300GB drive (well, I only really need an extra 10GB, but seeing as 300GB gives the best bang-for-buck ratio...)
_________________
No more cruft
dep: Revdeps that work
Using command-line ACCEPT_KEYWORDS?