systemd-boot, UKI, and stub requirements

[mutiny]

Hello,
I am starting to transition to a setup incorporating secureboot with separate ESP and XBOOTLDR partitions, and I think the easiest path forward to support most of my goals is to get all my components as they were already into a UKI and use a simpler bootloader/manager like systemd-boot. However, I seem to be finding that either UKI or systemd-boot (I'm not sure exactly which) has a requirement that the kernel be configured with CONFIG_EFI_STUB and be an executable itself, otherwise I get a failed boot with "load error". Reading some of the UKI specification, my understanding coming out of it seems like UKI format would support a standard linux image that is not necessarily bootable/executable itself and systemd-stub or some other stub feature makes the entire UKI executable regardless of the kernel data internally. Am I misunderstanding how UKI functions or is this a bug with systemd-boot or ukify?

[pietinger]

A UEFI system can boot directly:
- Windows
- A boot manager/loader
- A Linux kernel if it has CONFIG_EFI_STUB enabled.

For Linux, however, there are (unfortunately) three possibilities:

1. UEFI starts the Linux kernel directly (https://wiki.gentoo.org/wiki/User:Pieti ... l_via_UEFI) , or
2. UEFI starts a boot manager such as grub, which is able to start ANY Linux kernel (even a Linux kernel that does NOT have CONFIG_EFI_STUB).
3. And now for the special case: UEFI starts a boot manager such as refind [1] which is only able to start a Linux kernel that has CONFIG_EFI_STUB enabled. The same applies to systemd-boot [2] as to refind.

So if you have a kernel that does NOT have CONFIG_EFI_STUB enabled, then you need grub!

1) https://wiki.gentoo.org/wiki/REFInd#Kernel
2) https://wiki.gentoo.org/wiki/Systemd/sy ... oot#Kernel

[mutiny]

That is understood. However I am trying to build a UKI, not boot a stub kernel directly. The UKI itself includes a boot stub (systemd-stub) separately from the kernel, so why does it seem like the kernel still needs EFI_STUB itself to work in a UKI?

[dmpogo]

That is understood. However I am trying to build a UKI, not boot a stub kernel directly. The UKI itself includes a boot stub (systemd-stub) separately from the kernel, so why does it seem like the kernel still needs EFI_STUB itself to work in a UKI?

Isn't it the case 3) in the answer above ?

[zen_desu]

That is understood. However I am trying to build a UKI, not boot a stub kernel directly. The UKI itself includes a boot stub (systemd-stub) separately from the kernel, so why does it seem like the kernel still needs EFI_STUB itself to work in a UKI?

Why use a UKI if you're not booting it directly?

In any case, I think systemd-boot may only be able to boot a kernel with EFI stub capability. UKIs (made by ukify or similar) actually include a system boot stub. I don't think there are any other implementations of a UKI, or I haven't heard of them.

You can kinda make a custom "UKI like" kernel where you simply build the initramfs and whatever else directly into the kernel as part of the build process, but this process is a bit cumbersome if you need any kernel modules in the initramfs.

[GDH-gentoo]

The UKI itself includes a boot stub (systemd-stub) separately from the kernel, so why does it seem like the kernel still needs EFI_STUB itself to work in a UKI?

Simply because the two tools that produce UKIs (dracut and systemd's ukify) expect an EFI stub as input. If I had to guess why, I'd say because a UKI is a PE32+ file, and an EFI stub looks like one too, so the algorithm for producing the former might be simpler that way.

rEFInd, on the other hand (and systemd-boot, I suppose, but didn't check) requires an EFI stub because it chainloads, i. e. uses LoadImage() and StartImage() from UEFI boot services to transfer control to the kernel.

GRUB on Intel / AMD architectures does either that or the Linux/x86 boot protocol, it depends. Because it can do the latter, GRUB also works with 'plain' bzImages.

[pingtoo]

That is understood. However I am trying to build a UKI, not boot a stub kernel directly. The UKI itself includes a boot stub (systemd-stub) separately from the kernel, so why does it seem like the kernel still needs EFI_STUB itself to work in a UKI?

I am interest to know this too. So I did some research and I learn,

1. systemd-stub is a "chain-loader". It is stored in the PE+ file (usually the linux kernel image file. however this is NOT always the case). At the end of systemd-stub function it will eventually call linux_exec_efi_handover() which execute UEFI Handover protocol. therefor require the payload be EFI ready hence linux kernel need to have EFI_STUB support.

2. The "UKI" have blurred recognition, it could be a simple impression of linux kernel with embedded initramfs and linux own bootload stub (EFI_STUB) or it is UAPI Group Specifications (Unified Kernel Image),

2.1 For the first simple impression because Linux kernel build system can embed a initramfs cpio archive directly in to kernel image file using CONFIG_INITRAMFS_SOURCE kernel configuration option. Now add CONFIG_EFI_STUB you can have one file that can boot by EFI firmware therefor a unified kernel image.

2.2 Whereas the UAPI Group Specifications (Unified Kernel Image) create a EFI program using PE+ format which contain many sections for different objects, for example there is .linux section to store linux kernel binary (with EFI_STUB) and .initd section for initrd object (initramfs cpio archives) and .cmdline section to store kernel command line options text and .ucode section to store CPU microcode for early loading. and there are some more section I will not bring up here.

2.2.1 BTW, one of interesting UKI specification is "profile" which allow that single PE+ file have multiple version kernel/initrd/etc... and the systemd-stub implemetation allow select which profile to boot. (control by a EFI variable "StubProfile")

3. systemd-boot which is a standalone EFI program (Note. it is not linux program). It's function to present selection from stored disk configuration and finally execute selected bootloader on the ESP. in this regard it is Second-stage bootloader under Wikipedia Bootloader definition. So systemd-boot in its own have nothing to do with UKI.

[mutiny]

Thank you all for comments and discussion, I find it interesting given the lack of documentation/clarity.

Isn't it the case 3) in the answer above ?

My initial impression was that this is slightly different. rEFInd and systemd-boot being chain loaders that can only themselves load EFI executables makes sense why a standalone kernel would need to have CONFIG_EFI_STUB. But a UKI itself is an EFI executable due to systemd-stub included in the container, so it still wasn't clear why the embedded kernel itself inside the UKI needed to be PE.

Why use a UKI if you're not booting it directly?

UEFI booting it directly requires it to be in the ESP, of which I am not particularly a fan of. So I am trying to take advantage of systemd-boot and XBOOTLDR definition to keep the boot images/UKI where I can define.

Simply because the two tools that produce UKIs (dracut and systemd's ukify) expect an EFI stub as input. If I had to guess why, I'd say because a UKI is a PE32+ file, and an EFI stub looks like one too, so the algorithm for producing the former might be simpler that way.

rEFInd, on the other hand (and systemd-boot, I suppose, but didn't check) requires an EFI stub because it chainloads, i. e. uses LoadImage() and StartImage() from UEFI boot services to transfer control to the kernel.

GRUB on Intel / AMD architectures does either that or the Linux/x86 boot protocol, it depends. Because it can do the latter, GRUB also works with 'plain' bzImages.

I am able to build a UKI including a standard kernel image that is not EFI executable and does not have CONFIG_EFI_STUB using ukify just fine. So it doesn't seem that it is a requirement to build a UKI in general of these tools. Now executiny/booting it obviously turned out to be a different story...

I am interest to know this too. So I did some research and I learn,

1. systemd-stub is a "chain-loader". It is stored in the PE+ file (usually the linux kernel image file. however this is NOT always the case). At the end of systemd-stub function it will eventually call linux_exec_efi_handover() which execute UEFI Handover protocol. therefor require the payload be EFI ready hence linux kernel need to have EFI_STUB support.

2. The "UKI" have blurred recognition, it could be a simple impression of linux kernel with embedded initramfs and linux own bootload stub (EFI_STUB) or it is UAPI Group Specifications (Unified Kernel Image),

2.1 For the first simple impression because Linux kernel build system can embed a initramfs cpio archive directly in to kernel image file using CONFIG_INITRAMFS_SOURCE kernel configuration option. Now add CONFIG_EFI_STUB you can have one file that can boot by EFI firmware therefor a unified kernel image.

2.2 Whereas the UAPI Group Specifications (Unified Kernel Image) create a EFI program using PE+ format which contain many sections for different objects, for example there is .linux section to store linux kernel binary (with EFI_STUB) and .initd section for initrd object (initramfs cpio archives) and .cmdline section to store kernel command line options text and .ucode section to store CPU microcode for early loading. and there are some more section I will not bring up here.

2.2.1 BTW, one of interesting UKI specification is "profile" which allow that single PE+ file have multiple version kernel/initrd/etc... and the systemd-stub implemetation allow select which profile to boot. (control by a EFI variable "StubProfile")

3. systemd-boot which is a standalone EFI program (Note. it is not linux program). It's function to present selection from stored disk configuration and finally execute selected bootloader on the ESP. in this regard it is Second-stage bootloader under Wikipedia Bootloader definition. So systemd-boot in its own have nothing to do with UKI.

Thank you for the additional insight. I also tried to understand UKI format and requirements from the UAPI group documentation, yet it still wasn't or isn't clear to me that the embedded kernel itself needs to be PE just from reading existing guides and references. As an experiment, I tried to boot a non-PE kernel embedded UKI directly from UEFI without systemd-boot or any other bootloader/manager involved - with no success. So clearly, it absolutely is a required that UKI format itself requires the kernel itself to be PE and configured with CONFIG_EFI_STUB unrelated to systemd-boot etc. What would be nice is that this is explained in documentation, the gentoo wiki, etc. Probably it also comes down to my lack of knowledge/understanding of the PE format on which UKI is based, and probably that embedded PE sections themselves need to be PE?

[pingtoo]

Thank you all for comments and discussion, I find it interesting given the lack of documentation/clarity.
...

I am interest to know this too. So I did some research and I learn,

1. systemd-stub is a "chain-loader". It is stored in the PE+ file (usually the linux kernel image file. however this is NOT always the case). At the end of systemd-stub function it will eventually call linux_exec_efi_handover() which execute UEFI Handover protocol. therefor require the payload be EFI ready hence linux kernel need to have EFI_STUB support.

2. The "UKI" have blurred recognition, it could be a simple impression of linux kernel with embedded initramfs and linux own bootload stub (EFI_STUB) or it is UAPI Group Specifications (Unified Kernel Image),

2.1 For the first simple impression because Linux kernel build system can embed a initramfs cpio archive directly in to kernel image file using CONFIG_INITRAMFS_SOURCE kernel configuration option. Now add CONFIG_EFI_STUB you can have one file that can boot by EFI firmware therefor a unified kernel image.

2.2 Whereas the UAPI Group Specifications (Unified Kernel Image) create a EFI program using PE+ format which contain many sections for different objects, for example there is .linux section to store linux kernel binary (with EFI_STUB) and .initd section for initrd object (initramfs cpio archives) and .cmdline section to store kernel command line options text and .ucode section to store CPU microcode for early loading. and there are some more section I will not bring up here.

2.2.1 BTW, one of interesting UKI specification is "profile" which allow that single PE+ file have multiple version kernel/initrd/etc... and the systemd-stub implemetation allow select which profile to boot. (control by a EFI variable "StubProfile")

3. systemd-boot which is a standalone EFI program (Note. it is not linux program). It's function to present selection from stored disk configuration and finally execute selected bootloader on the ESP. in this regard it is Second-stage bootloader under Wikipedia Bootloader definition. So systemd-boot in its own have nothing to do with UKI.

Thank you for the additional insight. I also tried to understand UKI format and requirements from the UAPI group documentation, yet it still wasn't or isn't clear to me that the embedded kernel itself needs to be PE just from reading existing guides and references. As an experiment, I tried to boot a non-PE kernel embedded UKI directly from UEFI without systemd-boot or any other bootloader/manager involved - with no success. So clearly, it absolutely is a required that UKI format itself requires the kernel itself to be PE and configured with CONFIG_EFI_STUB unrelated to systemd-boot etc. What would be nice is that this is explained in documentation, the gentoo wiki, etc. Probably it also comes down to my lack of knowledge/understanding of the PE format on which UKI is based, and probably that embedded PE sections themselves need to be PE?

I cannot say for sure but I believe all these second-stage bootloader that support UEFI use UEFI boot service LoadImage() to perform kernel image loading, therefor it require the section blob also have PE header (That is what CONFIG_EFI_STUB cause the arch/x86/boot/header.S include some code with PE information.)