Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
arm chroot on x86_64 host [solved]
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Gentoo on Alternative Architectures
View previous topic :: View next topic  
Author Message
dinominant
Tux's lil' helper
Tux's lil' helper


Joined: 27 Oct 2006
Posts: 102

PostPosted: Wed Jul 23, 2014 5:13 pm    Post subject: arm chroot on x86_64 host [solved] Reply with quote

I'm trying to build an arm stage4 template on my x86_64 box. I have been following Crossdev qemu-static-user-chroot and Compiling with qemu user chroot to get going, but I am running into a problem when I finally chroot into the stage3 environment.

My static qemu-arm binary appears to be installed and working on the x86_64 host as I can run it just fine on it's own. I can also use it to directly run the statically linked busybox inside the stage3 tarball:
Code:
x86_64-host ~ # file armv6j_hardfp-20140723/bin/busybox
armv6j_hardfp-20140723/bin/busybox: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, for GNU/Linux 2.6.16, stripped
x86_64-host ~ # qemu-arm armv6j_hardfp-20140723/bin/busybox cat --help
BusyBox v1.21.0 (2014-06-05 15:17:01) multi-call binary.

Usage: cat [FILE]...

Concatenate FILEs and print them to stdout


I also have the /etc/init.d/qemu-binfmt service running and the qemu-arm interpreter is working because I can run that same arm busybox binary directly:
Code:
x86_64-host ~ # armv6j_hardfp-20140723/bin/busybox cat --help
BusyBox v1.21.0 (2014-06-05 15:17:01) multi-call binary.

Usage: cat [FILE]...

Concatenate FILEs and print them to stdout


However, when I mount /proc, /dev, /sys, and attempt to chroot into the arm environment I get this error:
Code:
x86_64-host ~ # chroot armv6j_hardfp-20140723 /bin/bash
/bin/bash: /bin/bash: cannot execute binary file


I've extracted the statically linked qemu-arm binary from the qemu-user-static_2.0.0+dfsg-6_arm64.deb package and that is what is in /usr/bin/qemu-arm on both the host and chroot environment. Do you have any ideas on what I might be missing and why I can't chroot into the stage3 environment?


Last edited by dinominant on Fri Jul 25, 2014 9:16 pm; edited 1 time in total
Back to top
View user's profile Send private message
Telemin
l33t
l33t


Joined: 25 Aug 2005
Posts: 753
Location: Glasgow, UK

PostPosted: Thu Jul 24, 2014 9:39 am    Post subject: Reply with quote

Hi there, I've been messing about with this too, and ran into the same problem. I'm still working on it, but I *think* I may have found a fix. I need to get on the embedded mailing list and ask some questions to be sure I understand what is going on though.

So, to what I know:


  • The message
    Code:
     /bin/bash: /bin/bash: cannot execute binary file
    means that the system can't find an appropriate interpreter for the file, or the interpreter is failing, I can't be sure which.

  • I found a "raspbian" VMDK image (read ubuntu image seemingly used for raspbian development), in which an arm chroot does work: http://archive.raspbian.org/qemu/rpi-qemu-user-dev.tar.bz2, launch the vmdk in qemu etc. and take a look in the /chroots dir

  • The difference between the debian setup and gentoo comes in the binfmt_misc handlers:


Gentoo:
Code:

# cat /proc/sys/fs/binfmt_misc/arm
enabled
interpreter /usr/bin/qemu-arm
flags: P
offset 0
magic 7f454c4601010100000000000000000002002800
mask ffffffffffffff00ffffffffffff00fffeffffff


Raspbian-ubuntu-dev:
Code:

# cat /proc/sys/fs/binfmt_misc/qemu-arm
enabled
interpreter /usr/bin/qemu-arm
flags: OC
offset 0
magic 7f454c4601010100000000000000000002002800
mask ffffffffffffff00fffffffffffffffffeffffff


Notice in particular that the mode flag for debian is OC, and for gentoo is P. Now I will admit that this is the part that I don't full understand, however taking a look at the kernel.org documentation for binfmt_misc, it appears that the raspbian image is using the new mode, and gentoo is using the old legacy mode. Changing this mode on gentoo fixes the issue, but I am unsure at what cost, hence I say I need to check this out with people who know more than me, probably the embedded mailing list.

There is also a difference in one byte of the mask, which I matched in my setup also. Since I don't really have enough knowledge of the ELF format, I am not 100% sure what effect this has, apart from to say the debian version is somehow stricter in what it matches (since setting the magic to 00 and the mask to 00 effectively means "ignore this byte"), although looking at wikipedia it seems the debian form checks the last byte of the padding block is set to 0x00, the gentoo form ignores it.

In the meantime however, by patching the provided /etc/init.d/qemu-binfmt script to match the raspbian settings, I have been able to get my chroot minimally working. My next step is to provide a native binary arm compiler and a native python interpreter to the chroot, but it remains to be seen how easy that will be or when I get the time to sit down and sort it out. My intention is eventually to get a working lxc template written for this and posted to github+wiki, but at the moment I am in a very preliminary stage.

Hope this is of some help

-Telemin-

P.S watch out for the step
Code:
chroot . /bin/busybox mdev -s

it can (probably will) trash your host /dev requiring a reboot. I have made do with bind mounting dev and not running mdev
_________________
The Geek formerly known as -Freestyling-
When you feel your problem has been solved please add [Solved] to the topic title.
Please adopt an unanswered post
Back to top
View user's profile Send private message
dinominant
Tux's lil' helper
Tux's lil' helper


Joined: 27 Oct 2006
Posts: 102

PostPosted: Fri Jul 25, 2014 9:16 pm    Post subject: Reply with quote

Thanks for the help Telemin. That fixed it for me! Modifying the /etc/init.d/qemu-binfmt script to use OC instead of P fixed it for me.

In /etc/init.d/qemu-binfmt change this
Code:
   if [ $cpu != "arm" -a -x "/usr/bin/qemu-arm" ] ; then
      echo   ':arm:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\x00\xff\xfe\xff\xff\xff:/usr/bin/qemu-arm:P' > /proc/sys/fs/binfmt_misc/register
   fi


To this
Code:
   if [ $cpu != "arm" -a -x "/usr/bin/qemu-arm" ] ; then
      echo   ':arm:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\x00\xff\xfe\xff\xff\xff:/usr/bin/qemu-arm:OC' > /proc/sys/fs/binfmt_misc/register
   fi


I also updated https://bugs.gentoo.org/show_bug.cgi?id=512780 so it can be changed to confirmed and hopefully fixed in the actual ebuild.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo on Alternative Architectures All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum