[solved] no bash prompt with qemu for aarch64

[viacheslavg]

Hello,

I'm trying to setup crossdev for aarch64 with qemu support. I'm following this guide https://wiki.gentoo.org/wiki/Cross_build_environment to setup qemu for aarch64.

It is installed well on both host and target. Now when I do

Code: Select all

# /etc/init.d/qemu-binfmt start && cd /usr/aarch64-unknown-linux-gnu/ && chroot . /bin/bash --login
 * Registering qemu-user binaries (flags: OC) ...                                                                                                                                  [ ok ]

I have no bash prompt, history, etc.

Shell is working, I can run commands.

Any ideas what could be wrong? is it something related to qemu? Or maybe I'm missing some essential packages on target?

Thanks.

[RumpletonBongworth]

The plausible reasons for no bash prompt being visible are:

• the PS1 variable being unset or empty (declare -p PS1)
• STDERR not being the open terminal (ls -l /proc/self/fd/2)

You may benefit from tracing the commands executed by bash during initialisation (if any).

Code: Select all

BASH_XTRACEFD=9 chroot . /bin/bash -lxc '' 9>bash-startup-trace.log

[viacheslavg]

Here is the output of

Code: Select all

BASH_XTRACEFD=9 chroot /usr/aarch64-unknown-linux-gnu/ /bin/bash -lxc '' 9>bash-startup-trace.log

Code: Select all

+ '[' -e /etc/profile.env ']'
+ . /etc/profile.env
++ export CONFIG_PROTECT=/usr/share/gnupg/qualified.txt
++ CONFIG_PROTECT=/usr/share/gnupg/qualified.txt
++ export 'CONFIG_PROTECT_MASK=/etc/sandbox.d /etc/gentoo-release /etc/ca-certificates.conf'
++ CONFIG_PROTECT_MASK='/etc/sandbox.d /etc/gentoo-release /etc/ca-certificates.conf'
++ export GCC_SPECS=
++ GCC_SPECS=
++ export INFOPATH=/usr/share/gcc-data/aarch64-unknown-linux-gnu/15/info:/usr/share/binutils-data/aarch64-unknown-linux-gnu/2.45/info:/usr/share/info
++ INFOPATH=/usr/share/gcc-data/aarch64-unknown-linux-gnu/15/info:/usr/share/binutils-data/aarch64-unknown-linux-gnu/2.45/info:/usr/share/info
++ export 'LESS=-R -M --shift 5'
++ LESS='-R -M --shift 5'
++ export 'LESSOPEN=|lesspipe %s'
++ LESSOPEN='|lesspipe %s'
++ export LEX=flex
++ LEX=flex
++ export MANPAGER=manpager
++ MANPAGER=manpager
++ export MANPATH=/usr/share/gcc-data/aarch64-unknown-linux-gnu/15/man:/usr/share/binutils-data/aarch64-unknown-linux-gnu/2.45/man:/usr/local/share/man:/usr/share/man
++ MANPATH=/usr/share/gcc-data/aarch64-unknown-linux-gnu/15/man:/usr/share/binutils-data/aarch64-unknown-linux-gnu/2.45/man:/usr/local/share/man:/usr/share/man
++ export PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin
++ PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin
++ export ROOTPATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin
++ ROOTPATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin
+ export EDITOR=vim
+ EDITOR=vim
+ export PAGER=/usr/bin/less
+ PAGER=/usr/bin/less
+ unset ROOTPATH
+ for sh in /etc/profile.d/*.sh
+ '[' -r /etc/profile.d/gawk.sh ']'
+ . /etc/profile.d/gawk.sh
+ unset sh
+ '[' -n '5.2.37(1)-release' ']'
+ '[' -f /etc/bash/bashrc ']'
+ . /etc/bash/bashrc
++ [[ hxBc != *i* ]]
++ return

I have additional info on this topic:
I took my old gentoo box which was not updated for several months. I have aarch64 crossdev created previously. There I have no problems with qemu chroot

Code: Select all

desna ~ # chroot /usr/aarch64-unknown-linux-gnu/ /bin/bash --login
desna / # 

I took sd card from my RPi with Gentoo installed put it into that box, mount and chroot into it... and... it doesn't work

Code: Select all

desna ~ # chroot /mnt/p2 /bin/bash --login

so, I suspect that the problem is not with host environment (missing /dev/, /sys, PS1, etc) but with target env. I guess recent updates broke something.
I already tried to downgrade bash and readline to same versions as I have in old (working) aarch64 crossdev environment -- doesn't help.
What else could it be? I didn't install anything in it apart @system set.
Would be interesting to here if others (who have up to date aarch64 crossdev env) have this problem or not.

[pingtoo]

Suggest do the same test on the successful chroot env, you likely see what is going on.

I suspect the

Code: Select all

+ . /etc/bash/bashrc
++ [[ hxBc != *i* ]]
++ return

is a indicator or something wrong in your chroot:/etc

[viacheslavg]

This is the output of working chroot

Code: Select all

+ '[' -e /etc/profile.env ']'
+ . /etc/profile.env
++ export 'CONFIG_PROTECT=/usr/share/gnupg/qualified.txt /usr/share/config'
++ CONFIG_PROTECT='/usr/share/gnupg/qualified.txt /usr/share/config'
++ export 'CONFIG_PROTECT_MASK=/etc/sandbox.d /etc/fonts/fonts.conf /etc/gentoo-release /etc/terminfo /etc/dconf /etc/ca-certificates.conf'
++ CONFIG_PROTECT_MASK='/etc/sandbox.d /etc/fonts/fonts.conf /etc/gentoo-release /etc/terminfo /etc/dconf /etc/ca-certificates.conf'
++ export GCC_SPECS=
++ GCC_SPECS=
++ export GSETTINGS_BACKEND=dconf
++ GSETTINGS_BACKEND=dconf
++ export INFOPATH=/usr/share/gcc-data/aarch64-unknown-linux-gnu/14/info:/usr/share/binutils-data/aarch64-unknown-linux-gnu/2.44/info:/usr/share/autoconf-2.72/info:/usr/share/automake-1.17/info:/usr/share/info
++ INFOPATH=/usr/share/gcc-data/aarch64-unknown-linux-gnu/14/info:/usr/share/binutils-data/aarch64-unknown-linux-gnu/2.44/info:/usr/share/autoconf-2.72/info:/usr/share/automake-1.17/info:/usr/share/info
++ export LADSPA_PATH=/usr/lib64/ladspa
++ LADSPA_PATH=/usr/lib64/ladspa
++ export 'LESS=-R -M --shift 5'
++ LESS='-R -M --shift 5'
++ export 'LESSOPEN=|lesspipe %s'
++ LESSOPEN='|lesspipe %s'
++ export LEX=flex
++ LEX=flex
++ export LV2_PATH=/usr/lib64/lv2
++ LV2_PATH=/usr/lib64/lv2
++ export MANPAGER=manpager
++ MANPAGER=manpager
++ export MANPATH=/usr/share/gcc-data/aarch64-unknown-linux-gnu/14/man:/usr/share/binutils-data/aarch64-unknown-linux-gnu/2.44/man:/usr/local/share/man:/usr/share/man
++ MANPATH=/usr/share/gcc-data/aarch64-unknown-linux-gnu/14/man:/usr/share/binutils-data/aarch64-unknown-linux-gnu/2.44/man:/usr/local/share/man:/usr/share/man
++ export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin
++ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin
++ export ROOTPATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin
++ ROOTPATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin
++ export XDG_CONFIG_DIRS=/etc/xdg
++ XDG_CONFIG_DIRS=/etc/xdg
++ export XDG_DATA_DIRS=/usr/local/share:/usr/share
++ XDG_DATA_DIRS=/usr/local/share:/usr/share
+ export EDITOR=vim
+ EDITOR=vim
+ export PAGER=/usr/bin/less
+ PAGER=/usr/bin/less
+ unset ROOTPATH
+ for sh in /etc/profile.d/*.sh
+ '[' -r /etc/profile.d/gawk.sh ']'
+ . /etc/profile.d/gawk.sh
+ unset sh
+ '[' -n '5.2.37(1)-release' ']'
+ '[' -f /etc/bash/bashrc ']'
+ . /etc/bash/bashrc
++ [[ hxBc != *i* ]]
++ return

looks identical.. no?

(meanwhile downgraded ncurses -- doesn't help)

Note: on SD card I have fully functional Gentoo install. RPi works fine, bash is working there. But chroot-ing is broken. I don't think /etc is broken there somehow.

Maybe under qemu bash/readline can't load some libs? but i don't have any errors...

[pingtoo]

Will my guess is wrong

I think I had this before but I don't recall how I fix it.

So I can only think of step by step debug. By eliminate things that work as expected than what is left must be the problem.

I would first examine environment variables in the chroot to make sure there are nothing out of ordinary.

Or you can try with another shell, for example busybox sh (or python/perl, just to see if you got a prompt). this will help to narrow down if the issue is relate to shell or it is chroot environment issue.

bash start up usually will check /dev/ for things it need. so I suspect you will need to bind mount /dev to your chroot:/dev. However since you have a exist working crossdev you may want to check what is your existing crossdev:/dev content to see if that can give some hints on what is going on.

[NeddySeagoon]

viacheslavg,

You need /dev, /proc, /sys, /dev/pts and some /tmp with 1777 permissions in the chroot.

[viacheslavg]

Doing this:

Code: Select all

oster ~ # cd /usr/aarch64-unknown-linux-gnu/
oster /usr/aarch64-unknown-linux-gnu # mount -t proc none proc
oster /usr/aarch64-unknown-linux-gnu # mount -o bind /dev dev
oster /usr/aarch64-unknown-linux-gnu # mount -o bind /dev/pts dev/pts
oster /usr/aarch64-unknown-linux-gnu # mount -o bind /sys sys
oster /usr/aarch64-unknown-linux-gnu # mount -o bind,mode=1777 /tmp tmp
oster /usr/aarch64-unknown-linux-gnu # chroot . /bin/bash --login
mount
none on /proc type proc (rw,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,noexec,relatime,size=10240k,nr_inodes=4068444,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
/dev/nvme0n1p2 on /tmp type xfs (rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota)

still no prompt, no line editing capabilities.

Am I doing something wrong?

Thanks for support.

[pingtoo]

If you only want to test with bash may be you can try

Code: Select all

chroot . /bin/bash -i --login

The "-i" tell bash the session should be setup as interactive session.

Compare to the working crossdev both session executed /etc/bash/bashrc

Code: Select all

# /etc/bash/bashrc

# Proceed no further in the case of a non-interactive shell.
if [[ $- != *i* ]]; then
	return
fi

therefor indicate the bash session for both chroot/bash session believe it is running in a non-interactive fashion. I could not found explain why one work and the other don't.

[viacheslavg]

pingtoo,

Indeed, after adding "-i" I have prompt now, however still no line editing, history, etc.
We get closer, but still something is missing.

Thanks!

[pingtoo]

pingtoo,

Indeed, after adding "-i" I have prompt now, however still no line editing, history, etc.
We get closer, but still something is missing.

Thanks!

IMHO, use "-i" is at best is just a workaround, I suggest it just want to see if this will take the session pass the /etc/bash/bashrc check.

I am not sure where you are in your current workflow, the "-i" option may or may not help you continue your work but I suggest if it work you can continue and not to pursuit further.

I think something not quick right in your new crossdev environment but in order to understand what possible be wrong will take many iterations to get to bottom of it.

On the other hand you can examine the successful prompted bash session, check environment variables, check shopt. there are setting could enable "readline", "history". you just need to enable/disable them and you can be on your way.

If you really want to learn the "WHY", then you need to share more of how the crossdev setup. the more detail is better. you also should compare the old working crossdev with this new one. (Not just files, but also version of programs/libraries too)

you may need to setup strace in chroot so it can be use to trace when bash start what it is trying to access to see if there anything is obvious incorrect.

[viacheslavg]

pingtoo,

I'm following this guide https://wiki.gentoo.org/wiki/Crossdev to bootstrap crossdev. No, additional steps were made.

To summarize, as of now I have two "instances" of aarch64 environments:
* current one. At /usr/aarch64-unknown-linux-gnu created by crossdev recently (does NOT work)
* old one. I copied it from old machine and put it at /var/tmp/aarch64-unknown-linux-gnu. Also created by crossdev several month ago (works fine)

I will try to update "instance" step by step and watch at what stage (hopefully) it will stop working.

[viacheslavg]

OK, seems like glibc is a culprit. Downgrading it to 2.41 fixes the problem.

As a matter of fact embedded handbook mention about such problems https://wiki.gentoo.org/wiki/Embedded_H ... root#Glibc

Lesson learned: RTFM, carefully, several times

Thanks everyone for support!