[SOLVED] Help making custom initramfs work on EFI Stub

[towel1]

solved. solution in a reply below: https://forums.gentoo.org/viewtopic-p-8 ... ml#8876535

hey fellas. merry christmas.

I'm installing Gentoo for the first time

my laptop has one NVME M.2 slot.

/dev/nvme0n1p1 is /efi, 1 GB
/dev/nvme0n1p2 is swap, 40 GB
/dev/nvme0n1p3 is root /, remainder of 500 GB SSD, formatted as F2FS (no rationale for choosing F2FS)

I'm using custom initramfs, custom installkernel script to install initramfs, EFI stub, EFI entry manually set using efibootmgr, and distribution sys-kernel/gentoo-kernel patched with config snippets, OpenRC. more on it and rationale later.

when I boot, kernel panics. I'm unable to scroll up (or else, I don't know how to scroll up). I tried to record it, but my smartphone can't keep up.

https://www.newgrounds.com/dump/item/ed ... 3758c995ed

manual transcription of excerpts of photo:

Code: Select all

  (driver?)
   [...] nvme0n1p1
   [...] nvme0n1p2
   [...] nvme0n1p3

List of all bdev filesystems:
  ext3
  ext2
  ext4
  xfs
  f2fs
  btrfs

Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[...]
Call trace:
[...]
Kernel offset: [...]
--- [ end Kernel panic [...]

EFI entries:

Code: Select all

$ ssh root@192.168.100.106 'efibootmgr --unicode'
BootCurrent: 0006
Timeout: 1 seconds
BootOrder: 0006,0000,0002,0003,0004,0005
Boot0000* Gentoooo	HD(1,GPT,3938c59e-3163-42c1-84be-c6838440f01e,0x800,0x200000)/\EFI\Gentoo\vmlinuz.efi initrd=\EFI\Gentoo\cringe.cpio initrd=\EFI\Gentoo\amd-uc.img
Boot0002* UEFI: PXE IPv4 Realtek PCIe GBE Family Controller	PciRoot(0x0)/Pci(0x1,0x2)/Pci(0x0,0x0)/MAC(50a13270bcc4,0)/IPv4(0.0.0.0,0,DHCP,0.0.0.0,0.0.0.0,0.0.0.0)
Boot0003* UEFI: HTTP IPv4 Realtek PCIe GBE Family Controller	PciRoot(0x0)/Pci(0x1,0x2)/Pci(0x0,0x0)/MAC(50a13270bcc4,0)/IPv4(0.0.0.0,0,DHCP,0.0.0.0,0.0.0.0,0.0.0.0)/Uri()
Boot0004* UEFI: HTTP IPv6 Realtek PCIe GBE Family Controller	PciRoot(0x0)/Pci(0x1,0x2)/Pci(0x0,0x0)/MAC(50a13270bcc4,0)/IPv6([::],0,Static,[::],[::],64)/Uri()
Boot0005* UEFI: PXE IPv6 Realtek PCIe GBE Family Controller	PciRoot(0x0)/Pci(0x1,0x2)/Pci(0x0,0x0)/MAC(50a13270bcc4,0)/IPv6([::],0,Static,[::],[::],64)
Boot0006* UEFI: SanDisk, Partition 1	PciRoot(0x0)/Pci(0x8,0x1)/Pci(0x0,0x4)/USB(3,0)/USB(0,0)/HD(1,GPT,f2155e16-532c-4a0e-b30c-3165fdbd81d5,0x800,0x1dcf7d0)
Boot01FF* UMC 1 Gentoo Linux 6.12.58	HD(1,GPT,3938c59e-3163-42c1-84be-c6838440f01e,0x800,0x200000)/\EFI\Gentoo\vmlinuz-6.12.58-gentoo-dist.efi initrd=\EFI\Gentoo\amd-uc.img

Boot0000 is what I created and intend to use, Boot01FF was previously created by uefi-mkconfig (called by installkernel with USE efistub)

the following command was used to create Boot0000:

Code: Select all

# efibootmgr -C --disk /dev/nvme0n1 --part 1 --label Gentoooo --loader '\EFI\Gentoo\vmlinuz.efi' --unicode ' initrd=\EFI\Gentoo\cringe.cpio initrd=\EFI\Gentoo\amd-uc.img'

I created custom initramfs following this guide: https://wiki.gentoo.org/wiki/Custom_Initramfs

Code: Select all

$ ssh root@192.168.100.106 'tree /mnt/gentoo/usr/src/initramfs'
/mnt/gentoo/usr/src/initramfs
├── bin
│   ├── busybox
│   └── fsck.f2fs
├── dev
│   ├── console
│   ├── null
│   ├── nvme0n1p3
│   └── tty
├── etc
├── init
├── lib
├── lib64
│   └── ld-linux-x86-64.so.2
├── mnt
│   └── root
├── proc
├── root
├── run
├── sbin
├── sys
└── usr
    ├── bin
    │   └── fsck.f2fs
    └── lib64
        ├── libc.so.6
        ├── libf2fs.so.10
        └── liblz4.so.1

16 directories, 12 files

ESP folder:

Code: Select all

$ ssh root@192.168.100.106 'ls -l /mnt/gentoo/efi/EFI/Gentoo'
total 100768
-rwxr-xr-x 1 root root 11338561 Dec 25 02:42 System.map
-rwxr-xr-x 1 root root 11261638 Dec 24 00:51 System.map-6.12.58-gentoo-dist
-rwxr-xr-x 1 root root 11087030 Dec 21 22:19 System.map-6.12.58-gentoo-dist.old
-rwxr-xr-x 1 root root   177152 Dec 25 02:42 amd-uc.img
-rwxr-xr-x 1 root root   278829 Dec 25 02:42 config
-rwxr-xr-x 1 root root   278829 Dec 24 00:51 config-6.12.58-gentoo-dist
-rwxr-xr-x 1 root root   278829 Dec 21 22:19 config-6.12.58-gentoo-dist.old
-rwxr-xr-x 1 root root  5398016 Dec 25 02:42 cringe.cpio
-rwxr-xr-x 1 root root 20673024 Dec 21 22:19 vmlinuz-6.12.58-gentoo-dist-old.efi
-rwxr-xr-x 1 root root 21111296 Dec 24 00:51 vmlinuz-6.12.58-gentoo-dist.efi
-rwxr-xr-x 1 root root 21271040 Dec 25 02:42 vmlinuz.efi

I have automated creation and installation of initramfs with this script called by installkernel:

Code: Select all

$ ssh root@192.168.100.106 'cat /mnt/gentoo/etc/kernel/postinst.d/cringe.config'
#/bin/sh
set -xe

mkdir -p /efi/EFI/Gentoo
mkdir -p /usr/src/initramfs/{bin,dev,etc,lib,lib64,mnt/root,proc,sbin,sys,run}
cp --archive /dev/{null,console,tty,nvme0n1p3} /usr/src/initramfs/dev/

make V=$1 -f /etc/kernel/postinst.d/Makefile -j

Code: Select all

$ ssh root@192.168.100.106 'cat /mnt/gentoo/etc/kernel/postinst.d/Makefile'
R=/usr/src/initramfs
E=/efi/EFI/Gentoo

all:$E/amd-uc.img $E/vmlinuz.efi $E/config $E/System.map $E/cringe.cpio

$E/cringe.cpio:$R/init $R/usr/bin/fsck.f2fs $R/bin/busybox
	find $R/ -print0 | cpio --null --create --format=newc > $@

$R/init:/etc/fstab /etc/kernel/postinst.d/init.m4
	m4 /etc/kernel/postinst.d/init.m4 > $@
	chmod +x $@

$R/usr/bin/fsck.f2fs:/usr/bin/fsck.f2fs
	lddtree --copy-to-tree $R $?

$R/bin/busybox:/bin/busybox
	cp $? $@

$E/config:/boot/config-$V
	mv $? $@

$E/vmlinuz.efi:/boot/vmlinuz-$V
	mv $? $@

$E/amd-uc.img:/boot/amd-uc.img
	mv $? $@

$E/System.map:/boot/System.map-$V
	mv $? $@

I acknowledge that the approach using Makefile is totally unnecessary

Code: Select all

$ ssh root@192.168.100.106 'cat /mnt/gentoo/etc/kernel/postinst.d/init.m4'
#!/bin/busybox sh
rescue() {
 echo "bruh"
 exec sh
}
set -x
mount -t proc proc /proc || rescue
mount -t sysfs sysfs /sys || rescue
fsck.f2fs -f /dev/nvme0n1p3 || rescue
mount -o syscmd(awk '/nvme0n1p3/ { printf "%s"`,'$4 }' /etc/fstab) /dev/nvme0n1p3 /mnt/root || rescue
umount /proc || rescue
umount /sys || rescue
exec switch_root /mnt/root /sbin/init || rescue

dump of more information:

Code: Select all

$ ssh root@192.168.100.106 'for i in /mnt/gentoo/etc/portage/package.use/*; do echo IN $(basename $i); cat $i; done'
IN 00cpuflags
*/* CPU_FLAGS_X86: aes avx avx2 bmi1 bmi2 f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 vpclmulqdq
IN 00local
*/* LINGUAS: pt_BR en
*/* L10N: pt-BR en
IN 00video
*/* VIDEO_CARDS: -* amdgpu radeonsi
IN busybox
sys-apps/busybox -pam static
IN f2fs-tools
sys-fs/f2fs-tools lz4
IN gentoo-kernel
sys-kernel/gentoo-kernel -initramfs
IN installkernel
sys-kernel/installkernel
IN lddtree
app-misc/pax-utils python
IN libcrypt
sys-libs/libxcrypt static-libs
virtual/libcrypt static-libs
IN networkmanager
net-wireless/wpa_supplicant dbus

Code: Select all

$ ssh root@192.168.100.106 'cat /mnt/gentoo/etc/kernel/config.d/50cringe.config'
CONFIG_F2FS_FS=y
CONFIG_AMD_SFH_HID=y

CONFIG_NVME_KEYRING=y
CONFIG_NVME_AUTH=y
CONFIG_NVME_CORE=y
CONFIG_BLK_DEV_NVME=y

I used diff and make menuconfig to obtain the last four overrides =y, the first two I set manually because I didn't know about make menuconfig then

I hope you guys can help me, or otherwise point me to another direction. should I create an UKI?

I previously tried to use installkernel with USE efistub ugrd, but the following line was botching my boot:

https://github.com/desultory/ugrd/blame ... s.py#L1053

because it would try to combine the mount options ro and F2FS's flush_merge, causing exactly the error described in:

https://wiki.archlinux.org/title/F2FS#R ... me_options

I spent so much time debugging then before figuring it out, and I learned so much, that even if I could write a TOML or something to fix ugrd behavior, I decided that rolling out my own solution would be easier at this point, hence my current approach.

for the sake of printing, I'm ssh'ing into my laptop from another laptop, but everything I did was while inside arch-chroot as the installation guide instructs!!

[zen_desu]

I previously tried to use installkernel with USE efistub ugrd, but the following line was botching my boot:

https://github.com/desultory/ugrd/blame ... s.py#L1053

because it would try to combine the mount options ro and F2FS's flush_merge, causing exactly the error described in:

https://wiki.archlinux.org/title/F2FS#R ... me_options

I spent so much time debugging then before figuring it out, and I learned so much, that even if I could write a TOML or something to fix ugrd behavior, I decided that rolling out my own solution would be easier at this point, hence my current approach.

for the sake of printing, I'm ssh'ing into my laptop from another laptop, but everything I did was while inside arch-chroot as the installation guide instructs!!

thanks for pointing out this issue, do problems still occur if you don't explicitly pass the root= on the kernel command line? Does passing appropriate rootflags work as you'd expect? ugrd will only use passed rootflags if root= is passed too, and will use built in config (detected at build time and put in the initramfs /etc/profile) if the passed root= does not work. The built in config is used in the "mount_default_root" function which ends up using the vars set in the /etc/profile:
https://github.com/desultory/ugrd/blob/ ... ts.py#L961

The handling for f2fs mount options in ugrd differs from other filesystems because it tries to inherit the current mount options and removes the rw option. Maybe it should just preserve them entirely?

https://github.com/desultory/ugrd/blob/ ... #L781-L784


Have you been able to run your custom initramfs in qemu? That should help you see if it's packed wrong or missing parts and you'll be able to see what the console says since you can scroll back. You can see how ugrd runs a vm here:
https://github.com/desultory/ugrd/blob/ ... est.py#L40
https://github.com/desultory/ugrd/blob/ ... ml#L11-L12

You mostly just need to set -kernel and -initrd args for qemu if you simply want to see if the initramfs itself will give you a shell or something. Setting the -cpu type to "host" can also help if you have custom compiled/optimized binaries in the initramfs. setting ` -append "console=ttyS0,115200" ` should ensure the kernel log is sent to the console you run qemu in.

[towel1]

the error is exactly the following line of previously shown Makefile:

Code: Select all

	find $R/ -print0 | cpio --null --create --format=newc > $@

must rather be:

Code: Select all

	cd $R && find . -print0 | cpio --null --create --format=newc > $@

otherwise, my custom initramfs, when unpacked, will have all files and dirs incorrectly under /usr/src/initramfs/, rather than being located at the root /, hence the Linux kernel won't be able to locate my /init script, thus panicking.

outside of a Makefile, pushd and popd are more useful than cd.

the following lines appear in kernel "output" when running my kernel and custom initramfs:

Code: Select all

[    1.665636] Trying to unpack rootfs image as initramfs...
[    1.733566] Freeing initrd memory: 5272K

this evidences that my custom initramfs was indeed loaded, the size matches.

were my /init script executed, the following line would appear, but didn't:

Code: Select all

[    2.450710] Run /init as init process

I used the following command to run qemu, in order to better understand what happened:

Code: Select all

qemu-system-x86_64 -serial stdio -kernel /efi/EFI/Gentoo/vmlinuz.efi -initrd /efi/EFI/Gentoo/cringe.cpio -append 'console=ttyAM0 console=ttyS0' |& less

thx zen_desu and folks from #gentoo at Libera.Chat for helping me two days ago!!!! I owe you one