| View previous topic :: View next topic |
| Author |
Message |
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 2903
Location: Rasi, Finland
|
 Posted: Mon May 22, 2023 8:06 pm Post subject: [musl/llvm] busybox shell segfaults, let's debug? |
 |
|
So on my musl/llvm setup busybox segfaults when trying to run bb or sh.
It also segfaults when I try to boot Gentoo there since my custom initrd uses busybox. So currently I'm in a chrooted environment with this setup.
What debugging tools I can use with musl+llvm? I guess I'd need some kind of backtrace?
It's been a while since I last used any debugging tools. (Tells much how stable Gentoo stable profiles are.)
This is all I get when trying to run busybox: | Code: | | Segmentation fault (core dumped) |
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
 |
Anon-E-moose
Watchman
Joined: 23 May 2008
Posts: 5937
Location: Dallas area
|
| Posted: Mon May 22, 2023 8:30 pm Post subject: |
|
|
compile with debug flag.
_________________
PRIME x570-pro, 3700x, 5.17 zen kernel
gcc 11.2.0/12.2.0, profile 17.1 amd64-no-multilib, openrc, wayland
copy of my local repo |
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 2903
Location: Rasi, Finland
|
| Posted: Mon May 22, 2023 8:32 pm Post subject: |
|
|
Forgot to mention that. So I did it already. The error message is the same.
I also tried with USE="static" to make busybox shell static, but there was no effect.
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
Anon-E-moose
Watchman
Joined: 23 May 2008
Posts: 5937
Location: Dallas area
|
| Posted: Mon May 22, 2023 8:50 pm Post subject: |
|
|
dev-util/lldb?
_________________
PRIME x570-pro, 3700x, 5.17 zen kernel
gcc 11.2.0/12.2.0, profile 17.1 amd64-no-multilib, openrc, wayland
copy of my local repo |
|
| Back to top |
|
|
sam_
Developer
Joined: 14 Aug 2020
Posts: 1145
|
| Posted: Tue May 23, 2023 2:34 am Post subject: |
|
|
| Zucca wrote: | Forgot to mention that. So I did it already. The error message is the same.
I also tried with USE="static" to make busybox shell static, but there was no effect. |
The error message would be the same if you compiled with debugging symbols but it'll give you a far better backtrace under gdb or lldb.
After building busybox and musl with debugging symbols, please try: gdb --args busybox, then 'r', then 'bt'.
Note that this is separate from USE=debug and USE=debug should be avoided normally unless you've been given specific instructions to use it. |
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 2903
Location: Rasi, Finland
|
| Posted: Tue May 23, 2023 8:55 am Post subject: |
|
|
Thanks for the info guys!
I've prepared few things now for debugging. But next I need to do some work which isn't as interesting as this but for which I get paid for. 
Should I enable FEATURES="nostrip" globally in make.conf? I mean since this is highly experimental profile what I'm using here...
The debug env file I've now created disables it (per package) as instructed by wiki.
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 2903
Location: Rasi, Finland
|
| Posted: Tue May 23, 2023 8:51 pm Post subject: |
|
|
| sam_ wrote: | | After building busybox and musl with debugging symbols, please try: gdb --args busybox, then 'r', then 'bt'. |
| output: | # gdb --args busybox bb
GNU gdb (Gentoo 13.1.90_p20230325 vanilla) 13.1.90.20230325-git
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-gentoo-linux-musl".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from busybox...
(No debugging symbols found in busybox)
(gdb) r
Starting program: /bin/busybox bb
warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
Program received signal SIGSEGV, Segmentation faultwarning: could not convert 'si_code' from the host encoding (ISO-8859-1) to UTF-32.
This normally should not happen, please file a bug report.
.
0x000055555567740f in ?? ()
(gdb) bt
#0 0x000055555567740f in ?? ()
#1 0x00005555556b5240 in ?? ()
#2 0x00007ffff7ffe4a0 in lock () from /lib/ld-musl-x86_64.so.1
#3 0x0000000100000003 in ?? ()
#4 0x00005555555f35d9 in ?? ()
#5 0x00000000f7f290c0 in ?? ()
#6 0x6e5ad37d21d200d3 in ?? ()
#7 0x00007ffff7ffe810 in builtin_tls () from /lib/ld-musl-x86_64.so.1
#8 0x00007ffff7ffe4d8 in fini_head () from /lib/ld-musl-x86_64.so.1
#9 0x0000000000000006 in ?? ()
#10 0x00007ffff7ffe4a0 in lock () from /lib/ld-musl-x86_64.so.1
#11 0x0000000000000003 in ?? ()
#12 0x00007ffff7ff3d75 in do_init_fini (queue=0x5555556b6fb0) at ldso/dynlink.c:1546
#13 0x0000555555591482 in ?? ()
#14 0x0000000000000002 in ?? ()
#15 0x00007ffff7f97dd9 in libc_start_main_stage2 (main=0x0, argc=<optimized out>, argv=0x7fffffffe0b8)
at src/env/__libc_start_main.c:95
#16 0x00005555555909d6 in ?? ()
#17 0x0000000000000002 in ?? ()
#18 0x00007fffffffe373 in ?? ()
#19 0x00007fffffffe380 in ?? ()
#20 0x0000000000000000 in ?? ()
(gdb) |
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
GDH-gentoo
Veteran
Joined: 20 Jul 2019
Posts: 1215
Location: South America
|
| Posted: Tue May 23, 2023 9:52 pm Post subject: |
|
|
| Zucca wrote: | | output: | Reading symbols from busybox...
(No debugging symbols found in busybox) |
|
Looks like you missed building BusyBox with debugging symbols too; still too many "??"s in the backtrace.
_________________
| NeddySeagoon wrote: | I'm not a witch, I'm a retired electronics engineer  |
| Ionen wrote: | | As a packager I just don't want things to get messier with weird build systems and multiple toolchains requirements though |
|
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 2903
Location: Rasi, Finland
|
| Posted: Tue May 23, 2023 10:52 pm Post subject: |
|
|
Yet I did.
I'll start digging trough build log of busybox...
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 2903
Location: Rasi, Finland
|
| Posted: Wed May 24, 2023 9:46 am Post subject: |
|
|
| *snip*: | | clang -Wp,-MD,archival/libarchive/.data_extract_all.o.d -std=gnu99 -Iinclude -Ilibbb -include include/autoconf.h -D_GNU_SOURCE -DNDEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -DBB_VER='"1.34.1"' -O2 -march=native -ggdb3 -fno-strict-aliasing -Wall -Wshadow -Wwrite-strings -Wundef -Wstrict-prototypes -Wunused -Wunused-parameter -Wunused-function -Wunused-value -Wmissing-prototypes -Wmissing-declarations -Wno-format-security -Wdeclaration-after-statement -Wold-style-definition -fno-builtin-strlen -ffunction-sections -fdata-sections -funsigned-char -fno-unwind-tables -fno-asynchronous-unwind-tables -fno-builtin-printf -Wno-string-plus-int -Wno-constant-logical-operand -DKBUILD_BASENAME='"data_extract_all"' -DKBUILD_MODNAME='"data_extract_all"' -c -o archival/libarchive/data_extract_all.o archival/libarchive/data_extract_all.c |
The whole log is little over 500kb in size.
To me it seems that busybox is indeed compiled with debugging symbols.
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
Anon-E-moose
Watchman
Joined: 23 May 2008
Posts: 5937
Location: Dallas area
|
| Posted: Wed May 24, 2023 10:02 am Post subject: |
|
|
Are you stripping the binary before install though? If so that will get rid of much debugging stuff.
_________________
PRIME x570-pro, 3700x, 5.17 zen kernel
gcc 11.2.0/12.2.0, profile 17.1 amd64-no-multilib, openrc, wayland
copy of my local repo |
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 2903
Location: Rasi, Finland
|
| Posted: Wed May 24, 2023 6:33 pm Post subject: |
|
|
I followed the wiki at: https://wiki.gentoo.org/wiki/Debugging
The problem may be in that FEATURES="installsources" doesn't work. I get nothing under /usr/src.
I tried rtfm man make.conf, but I didn't saw anything I have done wrong.
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
GDH-gentoo
Veteran
Joined: 20 Jul 2019
Posts: 1215
Location: South America
|
| Posted: Wed May 24, 2023 9:24 pm Post subject: |
|
|
Could you upload BusyBox's build log to a pastebin site?
_________________
| NeddySeagoon wrote: | | I'm not a witch, I'm a retired electronics engineer |
| Ionen wrote: | | As a packager I just don't want things to get messier with weird build systems and multiple toolchains requirements though |
|
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 2903
Location: Rasi, Finland
|
| Posted: Thu May 25, 2023 7:52 am Post subject: |
|
|
I had to compress it. :|
So you'll need to run | Code: | | curl -s https://dpaste.com/CKCK3GKZP.txt | base64 -d | xzcat |
... to view it.
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
sam_
Developer
Joined: 14 Aug 2020
Posts: 1145
|
| Posted: Thu May 25, 2023 7:54 am Post subject: |
|
|
Can you show emerge --info busybox?
wrt the original bug: please try the latest ~arch version too (committed yesterday). |
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 2903
Location: Rasi, Finland
|
| Posted: Thu May 25, 2023 8:12 am Post subject: |
|
|
| sam_ wrote: | Can you show emerge --info busybox?
wrt the original bug: please try the latest ~arch version too (committed yesterday). |
I'll try the latest. I'll be AFK for several hours now. I'll get back as soon as possible.
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
GDH-gentoo
Veteran
Joined: 20 Jul 2019
Posts: 1215
Location: South America
|
| Posted: Thu May 25, 2023 4:47 pm Post subject: |
|
|
| Zucca wrote: | | Code: | | curl -s https://dpaste.com/CKCK3GKZP.txt | base64 -d | xzcat |
|
This particular build did produce debugging information. Now GDB should be more verbose.
| Code: | >>> Merging sys-apps/busybox-1.34.1-r1 to /
...
>>> /bin/busybox
...
>>> /usr/lib/debug/bin/busybox.debug |
_________________
| NeddySeagoon wrote: | | I'm not a witch, I'm a retired electronics engineer |
| Ionen wrote: | | As a packager I just don't want things to get messier with weird build systems and multiple toolchains requirements though |
|
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 2903
Location: Rasi, Finland
|
| Posted: Thu May 25, 2023 6:58 pm Post subject: |
|
|
| GDH-gentoo wrote: | | This particular build did produce debugging information. Now GDB should be more verbose. |
Should, but yet it doesn't.
EDIT: | Code: | | gdb --symbols=/usr/lib/debug/bin/busybox.debug --args busybox bb |
doesn't change the output.
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 2903
Location: Rasi, Finland
|
| Posted: Fri May 26, 2023 4:17 am Post subject: |
|
|
| sam_ wrote: | | please try the latest ~arch version too (committed yesterday). |
| Code: | Keywords for sys-apps/busybox:
| | u |
| a a p s l r a | n |
| m r h p p i o i s l m m | e u s | r
| d a m p p c a x a o s 3 p 6 i | a s l | e
| 6 r 6 p p 6 r 8 6 n c 9 h 8 p | p e o | p
| 4 m 4 a c 4 c 6 4 g v 0 a k s | i d t | o
-------------+-------------------------------+-------+-------
1.34.1-r1 | + + + + + + + + ~ ~ ~ ~ ~ ~ ~ | 7 o 0 | gentoo
1.34.1-r2 | ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ | 7 # | gentoo
1.35.0-r1 | o o o o o o o o o o o o o o o | 7 # | gentoo
1.35.0-r2 | o o o o o o o o o o o o o o o | 7 # | gentoo
[I]1.36.1 | ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ | 8 o | gentoo
9999 | o o o o o o o o o o o o o o o | 8 o | gentoo |
... and busybox no longer segfaults. Yay!
But I'm still pondering why the debug indo from gdb was still garbled... Or rather what I've been doing wrong? I bet it's an user error. I so rarely need to use gdb.
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
sam_
Developer
Joined: 14 Aug 2020
Posts: 1145
|
| Posted: Fri May 26, 2023 7:36 am Post subject: |
|
|
Depending on what the issue is, sometimes this is just how it is (because things got corrupted).
I am a little bit suspicious because no sources got installed though. You definitely had debugedit installed?
It might be interesting to bisect to see what the issue was? |
|
| Back to top |
|
|
GDH-gentoo
Veteran
Joined: 20 Jul 2019
Posts: 1215
Location: South America
|
| Posted: Fri May 26, 2023 3:23 pm Post subject: |
|
|
| sam_ wrote: | | You definitely had debugedit installed? |
It was installed, but choked with Clang 15's debugging output:
| Code: | | debugedit: /var/tmp/portage/sys-apps/busybox-1.34.1-r1/image/bin/busybox: Unknown DWARF DW_FORM_0x25: No error information |
_________________
| NeddySeagoon wrote: | | I'm not a witch, I'm a retired electronics engineer |
| Ionen wrote: | | As a packager I just don't want things to get messier with weird build systems and multiple toolchains requirements though |
|
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 2903
Location: Rasi, Finland
|
| Posted: Fri May 26, 2023 3:41 pm Post subject: |
|
|
| sam_ wrote: | | You definitely had debugedit installed? |
| Code: | # qlist -vI debug
dev-util/debugedit-5.0-r1 |
Also: emerge --info busybox
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 2903
Location: Rasi, Finland
|
| Posted: Fri May 26, 2023 3:46 pm Post subject: |
|
|
| GDH-gentoo wrote: | It was installed, but choked with Clang 15's debugging output:
| Code: | | debugedit: /var/tmp/portage/sys-apps/busybox-1.34.1-r1/image/bin/busybox: Unknown DWARF DW_FORM_0x25: No error information |
|
Good catch.
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
|
Other Things Gentoo
