[SOLVED] Developing a simple busybox initramfs

[MALDATA]

I'm trying to learn a little about building and using an initramfs, so I was following along with this page: https://wiki.gentoo.org/wiki/Custom_Initramfs

My thought was to try a very simple test, with just the statically-linked busybox executable. So, as in the wiki, I created an initramfs directory:

Code: Select all

# INITRDDIR=/root/initrdworking
# mkdir -p ${INITRDDIR}
# mkdir -p ${INITRDDIR}/{bin,dev,proc,sys}
# cp --archive /bin/busybox ${INITRDDIR}/bin/
# cp --archive /dev/{null,console,tty,sda} ${INITRDDIR}/dev/
# cp --archive ./init ${INITRDDIR}/

The init script is:

Code: Select all

#!/bin/busybox sh

mount -t proc none /proc
mount -t sysfs none /sys

exec sh

And then finally,

Code: Select all

# chmod +x ${INITRDDIR}/init
# cd ${INITRDDIR}
# find . -print0 | cpio --null --create --verbose --format=newc | gzip --best > /boot/test-initramfs.cpio.gz

My kernel already supports gzipped initramfs files, so to test booting it, I reboot my system and just edit the grub boot lines. I edit the initrd line to point at this test initramfs file, then boot. The way I see it, this should use my existing working kernel, but instead of booting my root filesystem as usual, it should run the init script in the context of the initramfs.

The kernel does seem to boot as usual, but I eventually get

Code: Select all

Kernel panic - not syncing: Attempted to kill init! exitcode=0x00007f00

If everything ran ok, I should get a busybox shell and the init process shouldn't end. The script does exist at the specified location, it's executable, and so on. That being said, it seems like this would've been TOO easy if it had worked, so I think I probably missed something (like doing something more than just pointing grub toward a different initramfs).

Let me know if you see where I've gone wrong here. Thanks!

[axl]

I'm trying to learn a little about building and using an initramfs, so I was following along with this page: https://wiki.gentoo.org/wiki/Custom_Initramfs

My thought was to try a very simple test, with just the statically-linked busybox executable. So, as in the wiki, I created an initramfs directory:

Code: Select all

# INITRDDIR=/root/initrdworking
# mkdir -p ${INITRDDIR}
# mkdir -p ${INITRDDIR}/{bin,dev,proc,sys}
# cp --archive /bin/busybox ${INITRDDIR}/bin/
# cp --archive /dev/{null,console,tty,sda} ${INITRDDIR}/dev/
# cp --archive ./init ${INITRDDIR}/

The init script is:

Code: Select all

#!/bin/busybox sh

mount -t proc none /proc
mount -t sysfs none /sys

exec sh

And then finally,

Code: Select all

# chmod +x ${INITRDDIR}/init
# cd ${INITRDDIR}
# find . -print0 | cpio --null --create --verbose --format=newc | gzip --best > /boot/test-initramfs.cpio.gz

My kernel already supports gzipped initramfs files, so to test booting it, I reboot my system and just edit the grub boot lines. I edit the initrd line to point at this test initramfs file, then boot. The way I see it, this should use my existing working kernel, but instead of booting my root filesystem as usual, it should run the init script in the context of the initramfs.

The kernel does seem to boot as usual, but I eventually get

Code: Select all

Kernel panic - not syncing: Attempted to kill init! exitcode=0x00007f00

If everything ran ok, I should get a busybox shell and the init process shouldn't end. The script does exist at the specified location, it's executable, and so on. That being said, it seems like this would've been TOO easy if it had worked, so I think I probably missed something (like doing something more than just pointing grub toward a different initramfs).

Let me know if you see where I've gone wrong here. Thanks!

I play with some of the same tools you play.


some comments.


first of all, it's not just executables. or executable files. like busybox. u most likely had busybox built as static.

but other things that you might want to include.

Code: Select all

[axl@magdalina:~]$ lddtree -la /bin/bash | sort -u
/bin/bash
/lib64/ld-linux-x86-64.so.2
/lib64/libc.so.6
/lib64/libdl.so.2
/lib64/libhistory.so.8
/lib64/libncursesw.so.6
/lib64/libreadline.so.8
/lib64/libtinfo.so.6
/lib64/libtinfow.so.6

so let's assume, you figure out the static shared library thing and you DO include all things in it...


devtmpfs. automatically built in kernel. have to do it manually... something to look at.

and then... cpio. it's cpio. u have to crc32. beware 64 bits systems.

that xz gzip bzip lzo whatever is completely irrelevant.



to make it work, the initrd.cpio image has to have a /init or a /linuxrc. literally. an executable file, named init or linuxrc. inside the cpio. could be a shell using busybox.

either you mount dev or it's self mount based on kernel.

even dirs. like dev, proc, sys, run, tmp. u have to create them in the cpio, which is a sort of temporary root. coz if you dont make them prior to cpio... they wont be there when you start mounting shit and starting daemons and where is tmp!?

i could not stop talking about this. I know I find this stuff incredibly exciting so I'll just stop myself talking and just look for something else to talk about.

[axl]

attempting to kill init... that error

usually means u are trying to kill pid 1.

or pid 1 has nothing to execute. your busybox setup has no inittab file or something. i dont know what u're doing, but what u're doing is


the system has nothing to execute, therefore "trying to kill pid 1" is like trying to kill self because self has nothing to do.

[389292]

idk, maybe you also need an actual separate shell executable? like /bin/sh and/or /bin/bash with all of the libraries.
I also mount things with -n -t in my initramfs, I don't remember if it's really needed, but since I put it there it must be for some reason..

[axl]

https://www.youtube.com/watch?v=hV2Q41o-rwE

|| error.

[GDH-gentoo]

The kernel does seem to boot as usual, but I eventually get

Code: Select all

Kernel panic - not syncing: Attempted to kill init! exitcode=0x00007f00

If everything ran ok, I should get a busybox shell and the init process shouldn't end.

If the script exited, it should have printed error messages before the kernel panic. Did it not?

[Hu]

My thought was to try a very simple test, with just the statically-linked busybox executable.

In the interest of thoroughness, are you sure your busybox is static? It can be built as non-static, depending on system configuration.

So, as in the wiki, I created an initramfs directory:

Although this may be convenient, particularly for development, you don't need to do this when you are ready to make an initramfs for embedding in the kernel.

Code: Select all

# cp --archive /dev/{null,console,tty,sda} ${INITRDDIR}/dev/

sda is probably not right here. It may refer to non-existent hardware if you use an NVMe drive. Even if you use a drive that sda does reference, the drive is probably partitioned, so having only the base block device will be insufficient. That is not relevant to your panic though.

Code: Select all

#!/bin/busybox sh

mount -t proc none /proc
mount -t sysfs none /sys

exec sh

No /dev?

and then... cpio. it's cpio. u have to crc32. beware 64 bits systems.

The shown command is very similar to the one in the kernel documentation:

Code: Select all

find . | cpio --quiet -H newc -o | gzip -9 -n > /boot/imagefile.img

The one OP shows is actually a bit better, since it avoids using whitespace delimited find output.

to make it work, the initrd.cpio image has to have a /init or a /linuxrc. literally. an executable file, named init or linuxrc. inside the cpio. could be a shell using busybox.

OP appears to have already done this. Do you think his shown commands are incorrect?

even dirs. like dev, proc, sys, run, tmp. u have to create them in the cpio, which is a sort of temporary root. coz if you dont make them prior to cpio... they wont be there when you start mounting shit and starting daemons and where is tmp!?

OP showed creation of dev, proc, and sys. I doubt run or tmp are required for the minimal level of functionality that OP hoped to reach.

[axl]

did we just?


ok. put the pin back. one step at a time. easy does it.

[MALDATA]

Thanks for all the thoughts, everyone.

let's assume, you figure out the static shared library thing and you DO include all things in it...

I can confirm that my busybox binary definitely is statically linked. That being the case, I shouldn't need any other libraries or executables, other than an init script (or a symlink to busybox as an init).

devtmpfs. automatically built in kernel. have to do it manually... something to look at.

I did not check this, so I'll make sure that this is included.

and then... cpio. it's cpio. u have to crc32. beware 64 bits systems.

This is a 64-bit system. Can you elaborate more on how the procedure would be different on this system? Does the procedure in the wiki skip a step related to the CRC32?

to make it work, the initrd.cpio image has to have a /init or a /linuxrc. literally. an executable file, named init or linuxrc. inside the cpio. could be a shell using busybox.

There is an init script that uses "/bin/busybox sh" as the shell, and it is set to executable.

dev, proc, sys, run, tmp. u have to create them in the cpio

I have bin, dev, proc, and sys, but not tmp.

your busybox setup has no inittab file or something.

I do not have an inittab file. I can try adding one.

maybe you also need an actual separate shell executable? like /bin/sh and/or /bin/bash with all of the libraries.
I also mount things with -n -t in my initramfs, I don't remember if it's really needed, but since I put it there it must be for some reason..

Statically-linked busybox should not require anything else. I am not familiar with the -n and -t options for mount, I'll have to read up on those.

If the script exited, it should have printed error messages before the kernel panic. Did it not?

Nope. I recorded the boot log on my phone, and didn't see any error messages. I'd transcribe it here, but that seems like a lot.

Although this may be convenient, particularly for development, you don't need to do this when you are ready to make an initramfs for embedding in the kernel.

Good point, I forgot to mention that the initramfs is not embedded in the kernel, it is separate. The intent was to use the exact same kernel I'm using now, but with an initramfs instead of my normal filesystem.

No /dev?

Nope... I'm just following along with the wiki page, which does copy a few device nodes into the initramfs, but does not mount them. Is that an error?

It's entirely possible that the wiki page has some errors. If I can get this working, I'll gladly make the corrections.

[Hu]

I understood it not to be embedded. The process for embedding it is notably different from what you described doing here.

Copying device nodes to /dev is fine, provided you copy all the nodes you will need. Mounting a devtmpfs on /dev will let the kernel handle providing nodes that it can service, which means less work for you. Could you check the phone's recording for the last 25-30 lines before the panic, to see if there are any other errors or warnings reported? If you don't see anything there, then try modifying your initramfs to print something before it switches to the interactive shell, and something else after it switches. This latter message should appear only if switching to the interactive shell fails. For example:

Code: Select all

#!/bin/busybox sh

set -x
mount -t proc none /proc
mount -t sysfs none /sys

exec sh
echo exec failed
sleep 10
echo panic

We need to determine (1) whether your initramfs is even started and (2) if it does start, why it doesn't produce an interactive shell.

[MALDATA]

Could you check the phone's recording for the last 25-30 lines before the panic, to see if there are any other errors or warnings reported?

I checked through it frame-by-frame, and didn't see anything interesting. There was one line that said

Code: Select all

Unpacking initramfs...

But that's about it. After that, it starts setting up a ton of devices and then panics.

After I posted originally, I added an echo to print something to the screen before hitting the 'exec sh', and I did NOT see it get printed. I will add an echo to the very beginning of the script, and one after the 'exec sh', and see what happens.

[axl]

well, if the cpio successfully uncompresses then it's just a matter of what the /init or /linuxrc file is.

or the inittab. busybox is a distro into itself.

I DO this:

Code: Select all

cat << EOF > init
#!/bin/busybox sh
/bin/busybox --install -s
mount -t devtmpfs devtmpfs /dev
if [ ! -x "/dev/pts" ]; then mkdir /dev/pts; fi
if [ ! -x "/dev/shm" ]; then mkdir /dev/shm; fi
mount -t proc proc /proc
mount -t sysfs sysfs /sys
mount -t devpts devpts /dev/pts -o gid=5,mode=620,ptmxmode=000
mount /dev/vda1 /srv
if [ "\$?" -ne 0 ]; then
	xfs_repair /dev/vda1

	mount /dev/vda1 /srv
	if [ "\$?" -ne 0 ]; then
		xfs_repair -L /dev/vda1

		mount /dev/vda1 /srv
		if [ "\$?" -ne 0 ]; then
			echo cannot repair xfs filesystem
			/bin/sh
		fi
	fi
fi
mount -a
exec /sbin/init
EOF
chmod a+x init

chroot . ldconfig
chroot . locale-gen
chroot . depmod -av
chroot . updatedb

ofc, this works for me, but its kinda what I do.

And if you need to mount stuff just like in the gentoo handbook. the dev. the dev/pts. dev/shm. proc, sys. run. tmp. and then start some sort of init.


i'm not sure what you're doing with it. if you a desktop user, for you the loop ends with switch_root /newroot /sbin/newinit. and that's it.

I work with machines that never get out of initramfs.

[MALDATA]

or the inittab. busybox is a distro into itself.

This might be where I've gone wrong. I don't have an inittab. I figured that if I had created /init as a symlink to busybox, then busybox's implementation of init would look for an inittab. However, since I'm providing my own init as a shell script, I assumed that inittab was not needed. So, even though I'm not using busybox's init, do I still need inittab?

Also, I don't fully understand the script you posted. It looks like your init process is a shell script that sets up a few things and then execs /sbin/init... Is your /sbin/init a symlink to the busybox binary? Then I guess when you exec /sbin/init, then it would need inittab.

Thank you for helping me figure this stuff out, I appreciate it!

[axl]

or the inittab. busybox is a distro into itself.

This might be where I've gone wrong. I don't have an inittab. I figured that if I had created /init as a symlink to busybox, then busybox's implementation of init would look for an inittab. However, since I'm providing my own init as a shell script, I assumed that inittab was not needed. So, even though I'm not using busybox's init, do I still need inittab?

Also, I don't fully understand the script you posted. It looks like your init process is a shell script that sets up a few things and then execs /sbin/init... Is your /sbin/init a symlink to the busybox binary? Then I guess when you exec /sbin/init, then it would need inittab.

Thank you for helping me figure this stuff out, I appreciate it!

No, that's ok. I'm as invested in this as you are.

So let's finish this riddle.

inittab is a remnant of a system with multiple users.

again. it matters a lot what you trying to accomplish. like start raid. start crypt. start wireless. start pretty much any weird thing... and then...

launch into the real system. switch_root command. we talked about that.

so let's put it this way. what are you trying to accomplish in initrd space, that you can't accomplish in root?


I mean... what is the hurdle to cross from init to switch root ?

[Hu]

If your /init is a busybox shell script, you do not need an inittab for that /init to work properly.

[NeddySeagoon]

With /dev not mounted, you will be missing /dev/null and /dev/console.

The kernel can mount devtmpfs on /dev itself if that option is selected.
You still need the /dev mount point but none of the content.

The permissions in /dev will be incorrect and there will not be any symlinks like

Code: Select all

$ ls -l /dev/std*
lrwxrwxrwx 1 root root 17 May 12  2013 /dev/stderr -> ../proc/self/fd/2
lrwxrwxrwx 1 root root 17 May 12  2013 /dev/stdin -> ../proc/self/fd/0
lrwxrwxrwx 1 root root 17 May 12  2013 /dev/stdout -> ../proc/self/fd/1

You don't care about the permissions as the init script runs as root.
The missing symlinks probably don't matter either.

My system with a static /dev and root on lvm on raid5 runs its initrd init script with only

Code: Select all

 # we have a static /dev so we need all dev entries too
# nod <name> <mode> <uid> <gid> <dev_type> <maj> <min>
# e.g. /dev/console below
nod /dev/tty            0666 0 0 c 5 0
nod /dev/console        0600 0 0 c 5 1
nod /dev/null           0666 0 0 c 1 5
nod /dev/urandom        0644 0 0 c 1 9

in dev. Well, it has the sd* and dm-* entries too.

[MALDATA]

it matters a lot what you trying to accomplish. like start raid. start crypt. start wireless. start pretty much any weird thing... and then...

launch into the real system. switch_root command. we talked about that.

so let's put it this way. what are you trying to accomplish in initrd space, that you can't accomplish in root?

In short, changes to the root filesystem. I want to be able to make changes to the partitions and filesystems, kind of like a rescue CD. So, I don't want to mount my actual rootfs and switch_root into it. I want to just have a busybox system running in RAM that can make administrative changes.

So, while I will not mount the partitions, I do still need access to the /dev/sdX device nodes.

With /dev not mounted, you will be missing /dev/null and /dev/console.

The kernel can mount devtmpfs on /dev itself if that option is selected.
You still need the /dev mount point but none of the content.

In the boot log, I do see "devtmpfs initialized," and when I create the initramfs, I do

Code: Select all

# INITRDDIR=/root/initrdworking
# mkdir -p ${INITRDDIR}
# mkdir -p ${INITRDDIR}/{bin,dev,proc,sys}
# cp --archive /bin/busybox ${INITRDDIR}/bin/
# cp --archive /dev/{null,console,tty,sda} ${INITRDDIR}/dev/
# cp --archive ./init ${INITRDDIR}/

So, /dev/null and /dev/console should exist in the initramfs. But maybe "devtmpfs initialized" doesn't necessarily mean that devtmpfs is mounted on /dev?

I will try adding the "mount -t devtmpfs devtmpfs /dev" line and see if that helps.

[MALDATA]

My init script is now as follows:

Code: Select all

#!/bin/busybox sh

echo 'Start of init...'

mount -t proc none /proc
mount -t sysfs none /sys
mount -t devtmpfs devtmpfs /dev

echo 'About to run exec...'

exec sh

echo 'This is after exec!!!'

I don't see any of the echoed output while booting. I assume it'd show up in the console, right? Still get a kernel panic.

[MALDATA]

I tried several more things today, and was able to get it working. My procedure is now:

Code: Select all

# INITRDDIR=/root/initrdworking
# mkdir -p ${INITRDDIR}
# mkdir -p ${INITRDDIR}/{bin,dev,proc,sys}
# mkdir -p ${INITRDDIR}/lib/modules/$(uname -r)/kernel/drivers/hid/usbhid
# cp --archive /bin/busybox ${INITRDDIR}/bin/
# cp --archive /dev/{null,console,tty,tty0,tty1,sda,zero} ${INITRDDIR}/dev/
# cp --archive /lib/modules/$(uname -r)/modules.* ${INITRDDIR}/lib/modules/$(uname -r)/
# cp --archive /lib/modules/$(uname -r)/kernel/drivers/hid/usbhid/usbhid.ko ${INITRDDIR}/lib/modules/$(uname -r)/kernel/drivers/hid/usbhid/
# cp --archive /lib/modules/$(uname -r)/kernel/drivers/hid/hid.ko ${INITRDDIR}/lib/modules/$(uname -r)/kernel/drivers/hid/
# cp --archive /lib/modules/$(uname -r)/kernel/drivers/hid/hid-generic.ko ${INITRDDIR}/lib/modules/$(uname -r)/kernel/drivers/hid/
# cp --archive ./init ${INITRDDIR}/

Comparing this to what I posted initially, this now includes:

• /dev/zero
• /dev/tty0
• /dev/tty1
• Kernel module for hid
• Kernel module for hid-generic
• Kernel module for usbhid

The init script is now:

Code: Select all

#!/bin/busybox sh

/bin/busybox --install -s

mount -t proc none /proc
mount -t sysfs none /sys
mount -t devtmpfs devtmpfs /dev

modprobe usbhid
modprobe hid-generic

exec setsid sh -c 'exec sh </dev/tty1 >/dev/tty1 2>&1'

Compared to my original init script, this runs /bin/busybox --install -s to set up the symlinks, mounts devtmpfs, inserts the kernel modules so the keyboard works, and then avoids the busybox tty problem.

I still create the cpio file the same way:

Code: Select all

find . -print0 | cpio --null --create --verbose --format=newc | gzip --best > /boot/custom-initramfs.cpio.gz

Now in GRUB, I can select my default kernel, hit e to edit the commands, change the initramfs from the regular one to my custom one, and add "console=tty0" to the kernel args. It then boots and gets me to a busybox shell, which is exactly what I was looking for. Thanks for the help, everyone!