Secure Boot custom keys

Message

NathanZachary · Post by **NathanZachary** » Thu Oct 26, 2023 8:14 pm

Hello,

I'm working on getting Secure Boot working with custom keys, but am running into a strange problem and would really appreciate any troubleshooting advice. It's on an Ubuntu system, but I haven't gotten anywhere with their documentation so this is the last bastion of hope.

I have created the PK, KEK, and db keys

I have enroled the keys in the UEFI via `sbkeysync` - no errors reported

I have signed the binaries - no errors reported

I have validated the signed binaries - no errors reported

Yet when I enable Secure Boot (in Custom mode with my keys), the host won't boot. The part that I can't figure out is how everything passes validation, but then results in a boot failure. For instance:

Code: Select all

sbverify --cert my_db.crt vmlinuz
sbverify --cert my_db.crt BOOTX64.EFI
sbverify --cert my_db.crt shimx64.efi

all report 'Signature verification OK'.

However, when I enable Secure Boot, I don't even get to GRUB. All I see is:

Code: Select all

Booting from ubuntu
Boot failed: ubuntu

If I then go into the 'One time UEFI boot menu', it shows 'UNAVAILABLE: ubuntu'. Further, if I try to add a boot option, it shows that there are no filesystems available. It almost seems that with Secure Boot enabled, UEFI doesn't see the filesystem itself.

I would greatly appreciate any insights or pointers.

Thank you.

Post by **pietinger** » Thu Oct 26, 2023 9:07 pm

If your UEFI shall start grub at first ... THEN grub must be signed.

If you sign your kernel, you will need a stub kernel and you must tell UEFI to boot this stub kernel directly (=without grub).

What is the output of "efibootmgr" ?

Maybe you want read: https://wiki.gentoo.org/wiki/EFI_stub

and my approach to do SecureBoot:

viewtopic-p-8492354.html#8492354

NathanZachary · Post by **NathanZachary** » Thu Oct 26, 2023 9:22 pm

Thank you for your reply. Ideally, I would like to keep GRUB instead of using a stub kernel directly with EFI. Interestingly, though, when I enable Secure Boot with only my custom keys (PK, KEK, and db), I don't have any options at all for booting. I'm starting to wonder if something needs to be done regarding the RAID controller itself?

Post by **pietinger** » Thu Oct 26, 2023 10:15 pm

NathanZachary wrote:[...] I'm starting to wonder if something needs to be done regarding the RAID controller itself?

I dont think so.

I have asked for your "efiboomgr" to be able to see what is your grub (because you said you have signed BOOTX64.EFI; maybe this is your grub). To get an overview of your system I would need:

- efibootmgr
- lsblk
- blkid
- parted -l
- cat /proc/cmdline

Do you mount your ESP to /boot, or /boot/efi, or to /efi ? If 1) or 2) please give us the output of ls -R /boot If 3) please give us the ouput of ls -R /efi AND ls -R /boot

szatox · Post by **szatox** » Thu Oct 26, 2023 11:19 pm

Ideally, I would like to keep GRUB instead of using a stub kernel directly with EFI

I wonder... Doesn't GRUB break secure boot by decoupling kernel image from EFI?
I mean, if you use GRUB with trusted boot, GRUB is the thing that will be verified by EFI, but the kernel image is still going to take over the controls. So, does your signed GRUB actually verify that kernel has not been tampered with before handing over?

NathanZachary · Post by **NathanZachary** » Fri Oct 27, 2023 3:00 am

Looking at these details that you requested, I still can't see anything that is misconfigured:

Code: Select all

# efibootmgr -v
BootCurrent: 0004
BootOrder: 0004
Boot0002* Hard drive C:	VenHw(d6c0639f-c705-4eb9-aa4f-5802d8823de6)..1.........................F.........................................................A.....................P.E.R.C. .H.7.4.5. . .F.r.o.n.t. . .(.b.u.s. .3.1. .d.e.v. .0.0.)...
Boot0003* BRCM MBA Slot 0400 v21.6.0	BBS(128,BRCM MBA Slot 0400 v21.6.0,0x0)................f...........K.........................................................A.....................B.R.C.M. .M.B.A. .S.l.o.t. .0.4.0.0. .v.2.1...6...0...
Boot0004* ubuntu	HD(2,GPT,b84a6824-bee2-4af8-8e48-e47de457b385,0x1000,0xf4000)/File(\EFI\ubuntu\shimx64.efi)
MirroredPercentageAbove4G: 0.00
MirrorMemoryBelow4GB: false

Code: Select all

# lsblk
NAME                   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda                      8:0    0  1.8T  0 disk 
|-sda1                   8:1    0    1M  0 part 
|-sda2                   8:2    0  488M  0 part /boot/efi
|-sda3                   8:3    0  977M  0 part /boot
|-sda4                   8:4    0  7.6G  0 part 
`-sda5                   8:5    0  1.8T  0 part 
  |-vg00-var+log       253:0    0 76.3G  0 lvm  /var/log
  |-vg00-var+log+audit 253:1    0 19.1G  0 lvm  /var/log/audit
  `-vg00-root          253:2    0  1.7T  0 lvm  /

Code: Select all

# blkid
/dev/sda1: UUID="77604c85-9cc3-4d05-acb7-aac1dcccd0e8" TYPE="ext4" PARTUUID="1d142e69-fef4-40d5-af12-f27269ebd729"
/dev/sda2: UUID="8DD8-BE54" TYPE="vfat" PARTLABEL="EFI System Partition" PARTUUID="b84a6824-bee2-4af8-8e48-e47de457b385"
/dev/sda3: UUID="e8ed5fac-71d2-43fd-b8c0-ed5c383fac17" TYPE="ext3" PARTUUID="42e8c629-42aa-469b-be9b-5de8e016d2e9"
/dev/sda4: UUID="c2aed846-d4a0-4421-aa1a-1074f5625eae" TYPE="swap" PARTUUID="3880e24e-9874-4ea3-b829-2d70171cf0de"
/dev/sda5: UUID="6xBVk5-OZ5p-u4DO-20le-iokk-t6Af-U0AwEL" TYPE="LVM2_member" PARTUUID="1753979f-bbc5-47cb-a062-d3d1566b1f08"
/dev/mapper/vg00-var+log: UUID="ae813b6b-1bd4-44f3-99f5-c2638c0069df" TYPE="ext4"
/dev/mapper/vg00-var+log+audit: UUID="b1204f4e-5dca-4a3d-8216-e05579c0265b" TYPE="ext4"
/dev/mapper/vg00-root: UUID="206e7186-5837-418e-b2fc-18ed5611dc10" TYPE="ext4"

Code: Select all

# cat /proc/cmdline 
BOOT_IMAGE=/vmlinuz-4.15.0-213-generic root=/dev/mapper/vg00-root ro console=tty0 console=ttyS1,115200n8 audit=1 systemd.gpt_auto=no nowatchdog nosoftlockup iommu=off intel_iommu=off nopti <snip>

Code: Select all

# ls -R /boot/
/boot/:
abi-4.15.0-20-generic     config-4.15.0-213-generic  grub                          initrd.img-4.15.0-213-generic           lost+found                   System.map-4.15.0-20-generic   vmlinuz-4.15.0-20-generic
config-4.15.0-20-generic  efi                        initrd.img-4.15.0-20-generic  initrd.img-4.15.0-213-generic.old-dkms  retpoline-4.15.0-20-generic  System.map-4.15.0-213-generic  vmlinuz-4.15.0-213-generic

/boot/efi:
EFI

/boot/efi/EFI:
BOOT  Dell  ubuntu

/boot/efi/EFI/BOOT:
BOOTX64.EFI  fbx64.efi

/boot/efi/EFI/Dell:
BootOptionCache

/boot/efi/EFI/Dell/BootOptionCache:
BootOptionCache.dat

/boot/efi/EFI/ubuntu:
BOOTX64.CSV  grub.cfg  grubx64.efi  mmx64.efi  shimx64.efi

/boot/grub:
fonts  grub.cfg  grubenv  locale  unicode.pf2  x86_64-efi

/boot/grub/fonts:
unicode.pf2

/boot/grub/locale:
en_AU.mo  en_CA.mo  en_GB.mo  en@quot.mo

/boot/grub/x86_64-efi:
acpi.mod          cbtable.mod           echo.mod             gcry_dsa.mod            hashsum.mod               lsacpi.mod       mmap.mod        partmap.lst          regexp.mod          terminfo.mod            video_cirrus.mod
adler32.mod       cbtime.mod            efifwsetup.mod       gcry_idea.mod           hdparm.mod                lsefimmap.mod    moddep.lst      part_msdos.mod       reiserfs.mod        test_blockarg.mod       video_colors.mod
affs.mod          chain.mod             efi_gop.mod          gcry_md4.mod            hello.mod                 lsefi.mod        modinfo.sh      part_plan.mod        relocator.mod       testload.mod            video_fb.mod
afs.mod           cmdline_cat_test.mod  efinet.mod           gcry_md5.mod            help.mod                  lsefisystab.mod  morse.mod       part_sun.mod         romfs.mod           test.mod                videoinfo.mod
afsplitter.mod    cmp.mod               efi_uga.mod          gcry_rfc2268.mod        hexdump.mod               lsmmap.mod       mpi.mod         part_sunpc.mod       scsi.mod            testspeed.mod           video.lst
ahci.mod          cmp_test.mod          ehci.mod             gcry_rijndael.mod       hfs.mod                   ls.mod           msdospart.mod   parttool.lst         search_fs_file.mod  tftp.mod                video.mod
all_video.mod     command.lst           elf.mod              gcry_rmd160.mod         hfspluscomp.mod           lspci.mod        mul_test.mod    parttool.mod         search_fs_uuid.mod  tga.mod                 videotest_checksum.mod
aout.mod          configfile.mod        eval.mod             gcry_rsa.mod            hfsplus.mod               lssal.mod        multiboot2.mod  password.mod         search_label.mod    time.mod                videotest.mod
appleldr.mod      core.efi              exfat.mod            gcry_seed.mod           http.mod                  luks2.mod        multiboot.mod   password_pbkdf2.mod  search.mod          tpm.mod                 wrmsr.mod
archelp.mod       cpio_be.mod           exfctest.mod         gcry_serpent.mod        iorw.mod                  luks.mod         nativedisk.mod  pata.mod             serial.mod          trig.mod                xfs.mod
ata.mod           cpio.mod              ext2.mod             gcry_sha1.mod           iso9660.mod               lvm.mod          net.mod         pbkdf2.mod           setjmp.mod          tr.mod                  xnu.mod
at_keyboard.mod   cpuid.mod             extcmd.mod           gcry_sha256.mod         jfs.mod                   lzopio.mod       newc.mod        pbkdf2_test.mod      setjmp_test.mod     true.mod                xnu_uuid.mod
backtrace.mod     crc64.mod             f2fs.mod             gcry_sha512.mod         jpeg.mod                  macbless.mod     nilfs2.mod      pcidump.mod          setpci.mod          udf.mod                 xnu_uuid_test.mod
bfs.mod           cryptodisk.mod        fat.mod              gcry_tiger.mod          json.mod                  macho.mod        normal.mod      pgp.mod              sfs.mod             ufs1_be.mod             xzio.mod
bitmap.mod        crypto.lst            file.mod             gcry_twofish.mod        keylayouts.mod            mdraid09_be.mod  ntfscomp.mod    play.mod             shift_test.mod      ufs1.mod                zfscrypt.mod
bitmap_scale.mod  crypto.mod            fixvideo.mod         gcry_whirlpool.mod      keystatus.mod             mdraid09.mod     ntfs.mod        png.mod              signature_test.mod  ufs2.mod                zfsinfo.mod
blocklist.mod     cs5536.mod            font.mod             geli.mod                ldm.mod                   mdraid1x.mod     odc.mod         priority_queue.mod   sleep.mod           uhci.mod                zfs.mod
boot.mod          ctz_test.mod          fshelp.mod           gettext.mod             legacycfg.mod             memdisk.mod      offsetio.mod    probe.mod            sleep_test.mod      usb_keyboard.mod        zstd.mod
bsd.mod           datehook.mod          fs.lst               gfxmenu.mod             legacy_password_test.mod  memrw.mod        ohci.mod        procfs.mod           smbios.mod          usb.mod
bswap_test.mod    date.mod              functional_test.mod  gfxterm_background.mod  linux16.mod               minicmd.mod      part_acorn.mod  progress.mod         spkmodem.mod        usbms.mod
btrfs.mod         datetime.mod          gcry_arcfour.mod     gfxterm_menu.mod        linuxefi.mod              minix2_be.mod    part_amiga.mod  raid5rec.mod         squash4.mod         usbserial_common.mod
bufio.mod         diskfilter.mod        gcry_blowfish.mod    gfxterm.mod             linux.mod                 minix2.mod       part_apple.mod  raid6rec.mod         strtoull_test.mod   usbserial_ftdi.mod
cat.mod           disk.mod              gcry_camellia.mod    gptsync.mod             loadbios.mod              minix3_be.mod    part_bsd.mod    random.mod           syslinuxcfg.mod     usbserial_pl2303.mod
cbfs.mod          div.mod               gcry_cast5.mod       grub.efi                load.cfg                  minix3.mod       part_dfly.mod   rdmsr.mod            tar.mod             usbserial_usbdebug.mod
cbls.mod          div_test.mod          gcry_crc.mod         gzio.mod                loadenv.mod               minix_be.mod     part_dvh.mod    read.mod             terminal.lst        usbtest.mod
cbmemc.mod        dm_nv.mod             gcry_des.mod         halt.mod                loopback.mod              minix.mod        part_gpt.mod    reboot.mod           terminal.mod        video_bochs.mod

What I gather is that it is using EFI boot option '0004', which is:
Boot0004* ubuntu HD(2,GPT,b84a6824-bee2-4af8-8e48-e47de457b385,0x1000,0xf4000)/File(\EFI\ubuntu\shimx64.efi)

That UUID points to /boot/efi, which means the full path is /boot/efi/EFI/ubuntu/shimx64.efi. When checking that file using:

Code: Select all

sbverify --cert $MY_DB /boot/efi/EFI/ubuntu/shimx64.efi

I see that the signature is valid. So it seems like the BOOTX64.EFI wasn't necessary to sign, but it still doesn't explain to me why I can't boot.

Thanks for you continued help.

Post by **pietinger** » Fri Oct 27, 2023 1:12 pm

If you want to boot your system with SecureBoot there are three main possibilities under Linux:

1. You use the SHIM, which already has a Microsoft signature. To my knowledge Ubuntu uses this way. The SHIM itself then loads the GRUB. Here I can't help you, because this was never an option for me (because then still "foreign" keys are used).

2. You work with MOK (machine owner keys). Here I can't help you, because this was never an option for me (because then still "foreign" keys are used).

3. You do EVERYTHING manually yourself. For this you need your own keys. You have to give these to your UEFI. Since there are motherboards that cause problems when using "efi-updatevar" (or "sbkeysync"), I recommend to take over the keys yourselv in the BIOS (beware you must reboot your machine twice).

As @szatox already wrote correctly, signing a bootloader is not very useful. That is why the UKI (unified kernel image) was "invented".

I don't know Ubuntu and can't help you there. I strongly suspect that re-signing the SHIM will not work (since it already has a Microsoft signature). Maybe this helps you:

https://wiki.ubuntu.com/UEFI/SecureBoot
https://wiki.ubuntu.com/UEFI/SecureBoot/Testing

My installation looks like this:

- Before the switch to SecureBoot -

1. I use "gentoo-sources" and configured my kernel myself.
2. I configured everything I need statically <*> in the kernel and then turned off the module support. With this I have a monolithic kernel. (But this is not a requirement for SecureBoot).
3a. I configured this kernel as a STUB kernel (so it will boot directly).
3b. This also requires configuring the kernel command line parameters INTO the kernel.
4a. One of my boxes does not have an initramfs.
4b. The other box has the initramfs EMBEDDED into the kernel.
I used: https://wiki.gentoo.org/wiki/User:Pieti ... _file-list
If you have already a CPIO and want embed it, then you can use:
https://wiki.gentoo.org/wiki/User:Pieti ... IO_archive
5. UEFI boots directly this (UKI) kernel. For this I created a UEFI entry with "efibootmgr"

Code: Select all

$ efibootmgr
BootCurrent: 0000
Timeout: 1 seconds
BootOrder: 0000,0002,0001
Boot0000* Secure        HD(1,GPT,0adcbfee-21aa-42ea-9a9a-2e53bd05e6a2,0x800,0x7f800)/File(\efi\secure\bzImage.efi)
Boot0001* gentoo        HD(1,GPT,0adcbfee-21aa-42ea-9a9a-2e53bd05e6a2,0x800,0x7f800)/File(\EFI\gentoo\grubx64.efi)
Boot0002* Unlocked      HD(1,GPT,0adcbfee-21aa-42ea-9a9a-2e53bd05e6a2,0x800,0x7f800)/File(\efi\unlocked\bzImage.efi)

- Switching to SecureBoot -

1. I have created my own keys.
2. I signed the existing kernel with it (overwriting the non-signed kernel).
3. I saved the existing keys of the UEFI for safety. (Not absolutely necessary).
4. I have directly in the BIOS, deleted the existing keys and taken my own (you must reboot twice).

I have described the whole thing here: viewtopic-p-8492354.html#8492354

If you want go this way, I think you must make an own UEFI entry (with "efibootmgr -c ...") for your GRUB ( https://wiki.gentoo.org/wiki/Efibootmgr ). Check if you can boot this grub WITHOUT SecureBoot. Then sign your grub. And maybe best: Add the Keys directly in BIOS (= not using "sbkeysync"). ... Again ... Signing a boot loader is not very useful ...

NathanZachary · Post by **NathanZachary** » Fri Oct 27, 2023 1:30 pm

Thank you very much for taking the time to look at it with me. I think that the problem somehow revolves around the shim, which I don't even want to use in the first place. Like you, I haven't done this with Ubuntu before, and have never had any problems with using my own Secure Boot keys in Gentoo. I'm going to see if I can find a way to bypass using the signed shim, and just go directly to GRUB. How I understand it is that Ubuntu looks to /boot/efi/EFI/ubuntu/ for all of the related options. It chooses shimx64.efi for Secure Boot, and grubx64.efi when Secure Boot isn't needed. I'm thinking that I may be able to bypass shim, and update the entry to point directly to grubx64.efi. Maybe it would even work with signing shimx64.efi, grubx64.efi, and the kernel. I haven't attempted signing grubx64.efi, but that might be my next step.

Thanks again for taking the time to reply.

Cheers,
Nathan Zachary

Post by **pietinger** » Fri Oct 27, 2023 1:37 pm

NathanZachary wrote:[...] I'm thinking that I may be able to bypass shim, and update the entry to point directly to grubx64.efi. Maybe it would even work with signing shimx64.efi, grubx64.efi, and the kernel. I haven't attempted signing grubx64.efi, but that might be my next step.

To my knowledge you must sign ONLY the efi-binary (*.efi) which UEFI wants to load/start. If you boot a grub THEN it is not necessary to sign the kernel - only grubx64.efi must be signed then.

NathanZachary wrote:Thanks again for taking the time to reply.

You are very Welcome !

Many Greetings,
Peter

GDH-gentoo · Post by **GDH-gentoo** » Fri Oct 27, 2023 3:48 pm

szatox wrote:I wonder... Doesn't GRUB break secure boot by decoupling kernel image from EFI?
I mean, if you use GRUB with trusted boot, GRUB is the thing that will be verified by EFI, but the kernel image is still going to take over the controls. So, does your signed GRUB actually verify that kernel has not been tampered with before handing over?

GRUB detects whether it is running on a UEFI system with Secure Boot enabled (by reading EFI variables), and refuses to process files of certain types in the filesystem that can't be verified, or fail verification —including kernel EFI stubs or bzImages—. This 'second stage' verification is GRUB-specific, and carried out by components called "verifiers".

Verifiers aren't very well documented, one has to read the code to find out how they work.

CaptainBlood · Post by **CaptainBlood** » Fri Oct 27, 2023 9:52 pm

GDH-gentoo wrote:This 'second stage' verification is GRUB-specific, and carried out by components called "verifiers".

Verifiers aren't very well documented, one has to read the code to find out how they work.

I always felt full secure-boot activation to be poorly documented, or hard to understand.
At least hard enough that it will require many trial & errors loops...
And a tedious path to clean things up in case of a fallback if giving up.

Above statement reinforces that feeling since I'd be willing to keep my M$ dual booting grub along the way.

I've been warned.

Thks 4 ur attention, interest & support.

sMueggli · Post by **sMueggli** » Wed Nov 01, 2023 4:28 pm

What is the goal of this exercise?

Secure Boot is based on a chain of trust. The root of trust is the PK in the firmware. The PK is usually a public key of the mainboard vendor. Then there are several KEKs, which all are signed by the the private key of the PK. Usually the public keys of Microsoft are part of the preinstalled/signed KEKs.

The db and dbx are not keys, but databases of signatures. In db are the authorised signatures and dbx contains the forbidden signatures.

Roughly the Ubuntu boot process is as follows:

The firmware is checking the EFI binary \EFI\ubuntu\shimx64.efi. The binary shimx64.efi was audited by Microsoft and then signed with the private key of the Microsoft certificate. During the boot process, the firmware is checking that the signature of shimx64.efi is not part of dbx, exists in db and the used KEK is signed by the PK.

If this was successful, shimx64.efi is loading grubx64.efi and checking the signature of grubx64.efi. The grubx64.efi EFI binary is signed with a distribution-specific certificate and the public key is part of the shimx64.efi binary. If also Grub is passing the check, it takes over the control. Also Grub is checking the binaries to load (https://www.gnu.org/software/grub/manua ... -shim.html) which are also signed by the distribution-specific certificate.

To be able to load kernel modules that are not signed by the distribution-specific certificate there is a "MOK" (machine owner key). Enrolling this key in the firmware allows you to load kernel modules (e.g. WLAN) signed with the private key of the MOK part.

If you want a "Do-It-Yourself-Secure-Boot" you need to

setup a working PKI (storing the private keys on the same machine is not a "working" PKI)
setup a signing and auditing process for every piece of software you want to sign
need to learn how to incorporate your certificate into the software (shim_lock and kernel lockdown)

A boot process that deserves the name "Secure Boot" involves a lot of work and knowledge. If the chain of trusts ends at Grub (you can disable the validation process in shim) or attackers can easily sign their binaries with your keys you end up with what I call a "Signed Boot" instead of a Secure Boot.

GDH-gentoo · Post by **GDH-gentoo** » Wed Nov 01, 2023 9:48 pm

sMueggli wrote:What is the goal of this exercise?

Secure Boot is based on a chain of trust.

Remove everyone who has supplied keys for verification other than the owner of the hardware him/herself, from the chain of trust, I suppose. E. g. Microsoft and the hardware vendor. That's what people who bother messing with UEFI firmware keys usually want, it seems.

nvaert1986 · Post by **nvaert1986** » Fri Nov 17, 2023 8:24 am

You can keep using grub without using a EFI stub kernel by generating a strandalone grub version and signing that compiled EFI binary and the kernel. I use this myself with sbctl. You do need to create an sbat file. This is a CSV file with information required as a standard.

To generate the standalone grub image use the following command:

Code: Select all

grub-mkstandalone --fonts=all -O x86_64-efi -o grubx64.efi "/boot/grub/grub.cfg" --sbat sbat.csv -v

For more information regarding sbat, please see: https://github.com/rhboot/shim/blob/main/SBAT.md

Notes:
- Make sure your grub.cfg is up-to-date (if you add a new kernel you still need to generate a new grub.cfg).
- Make sure /boot is mounted if your grub.cfg is located in boot

For details on sbctl: https://github.com/Foxboron/sbctl and https://packages.gentoo.org/packages/app-crypt/sbctl

EDIT: If you don't want to meet the sbat requirement, you can always remove this check from grub (though I would strongly recommend against it): https://wejn.org/2021/09/fixing-grub-ve ... ody-cares/ but it does work.