Kernel 5.15.11 not booting

[reimers]

Hi all,

yesterday I was offered the new linux-5.15.11-gentoo kernel by portage and went ahead to update from 5.10.76 (which is running perfectly fine). The system runs inside a VM on an vmware esxi 6.7 server.

I did the usual make oldconf using mostly the defaults for the new kernel options except for new device drivers. The new kernel now gets stuck during boot; I reset the VM after there was no progress for 5 minutes. The output I can see on the screen at that point matches the old kernel, except that it hangs at the spot marked below (output taken from dmesg using the old kernel; marked is the last line visible on screen):

Code: Select all

...
[    0.333023] pci_bus 0000:0a: resource 1 [mem 0xfb800000-0xfb8fffff]
[    0.333038] pci_bus 0000:0a: resource 2 [mem 0xe9e00000-0xe9efffff 64bit pref]
[    0.333054] pci_bus 0000:0b: resource 0 [io  0x5000-0x5fff]
[    0.333070] pci_bus 0000:0b: resource 1 [mem 0xfd300000-0xfd3fffff]
[    0.333085] pci_bus 0000:0b: resource 2 [mem 0xeb900000-0xeb9fffff 64bit pref]
[    0.333101] pci_bus 0000:0c: resource 0 [io  0x9000-0x9fff]
[    0.333116] pci_bus 0000:0c: resource 1 [mem 0xfcf00000-0xfcffffff]
[    0.333132] pci_bus 0000:0c: resource 2 [mem 0xeb500000-0xeb5fffff 64bit pref]
[    0.333148] pci_bus 0000:0d: resource 0 [io  0xd000-0xdfff]
[    0.333163] pci_bus 0000:0d: resource 1 [mem 0xfcb00000-0xfcbfffff]
[    0.333179] pci_bus 0000:0d: resource 2 [mem 0xeb100000-0xeb1fffff 64bit pref]
[    0.333195] pci_bus 0000:0e: resource 1 [mem 0xfc700000-0xfc7fffff]
[    0.333210] pci_bus 0000:0e: resource 2 [mem 0xead00000-0xeadfffff 64bit pref]
[    0.333226] pci_bus 0000:0f: resource 1 [mem 0xfc300000-0xfc3fffff]
[    0.333241] pci_bus 0000:0f: resource 2 [mem 0xea900000-0xea9fffff 64bit pref]
[    0.333301] pci_bus 0000:10: resource 1 [mem 0xfbf00000-0xfbffffff]
[    0.333317] pci_bus 0000:10: resource 2 [mem 0xea500000-0xea5fffff 64bit pref]
[    0.333332] pci_bus 0000:11: resource 1 [mem 0xfbb00000-0xfbbfffff]
---> new kernel is stuck here <---
[    0.333348] pci_bus 0000:11: resource 2 [mem 0xea100000-0xea1fffff 64bit pref]
[    0.333363] pci_bus 0000:12: resource 1 [mem 0xfb700000-0xfb7fffff]
[    0.333379] pci_bus 0000:12: resource 2 [mem 0xe9d00000-0xe9dfffff 64bit pref]
[    0.333395] pci_bus 0000:13: resource 0 [io  0x6000-0x6fff]
[    0.333410] pci_bus 0000:13: resource 1 [mem 0xfd200000-0xfd2fffff]
[    0.333426] pci_bus 0000:13: resource 2 [mem 0xeb800000-0xeb8fffff 64bit pref]
...

Based on the memory adress and the lspci output it looks like its at a "VMware PCI Express Root Port" - however the device at 0a is one as well with the exact same output in lspci except for the adress.

I've already tried playing around with some of the new kernel options such as the IOMMU default group, but neither passthrough nor strict made any difference. Does anyone have an idea what I could do? It feels like I'm just missing some trivial setting somewhere

[NeddySeagoon]

reimers,

I suspect that the new kernel has just switched form a working console driver to a broken one, so everything is fine but you can't see it.

Boot the old kernel and set up sshd, if its not set up already. Don't forget to test it.
Boot the new kernel, get a coffee when the console appears to be stuck.

After the coffee delay, can you ssh into the VM and pastebin its dmesg?

[reimers]

Hi NeddySeagoon,

indeed you were right, it is only a problem with the graphics driver and the system boots fine.

I've copied what I think are the relevant parts of dmesg - if you need the full log, I can upload it somewhere.

The old kernel does the following

Code: Select all

[    0.355596] Linux agpgart interface v0.103
[    0.355663] agpgart-intel 0000:00:00.0: Intel 440BX Chipset
[    0.355857] agpgart-intel 0000:00:00.0: AGP aperture is 256M @ 0x0
[    0.356136] [drm] DMA map mode: Caching DMA mappings.
[    0.356152] [drm] Capabilities:
[    0.356168] [drm]   Rect copy.
[    0.356183] [drm]   Cursor.
[    0.356199] [drm]   Cursor bypass.
[    0.356215] [drm]   Cursor bypass 2.
[    0.356230] [drm]   8bit emulation.
[    0.356246] [drm]   Alpha cursor.
[    0.356251] [drm]   Extended Fifo.
[    0.356321] [drm]   Multimon.
[    0.356337] [drm]   Pitchlock.
[    0.356352] [drm]   Irq mask.
[    0.356368] [drm]   Display Topology.
[    0.356384] [drm]   GMR.
[    0.356399] [drm]   Traces.
[    0.356415] [drm]   GMR2.
[    0.356430] [drm]   Screen Object 2.
[    0.356446] [drm] Max GMR ids is 64
[    0.356462] [drm] Max number of GMR pages is 65536
[    0.356477] [drm] Max dedicated hypervisor surface memory is 163840 kiB
[    0.356493] [drm] Maximum display memory size is 4096 kiB
[    0.356509] [drm] VRAM at 0xec000000 size is 4096 kiB
[    0.356524] [drm] MMIO at 0xfe000000 size is 256 kiB
[    0.356668] [TTM] Zone  kernel: Available graphics memory: 1013736 KiB
[    0.356684] [TTM] Initializing pool allocator
[    0.356702] [TTM] Initializing DMA pool allocator
[    0.357044] [drm] Screen Objects Display Unit initialized
[    0.357060] [drm] width 640
[    0.357076] [drm] height 480
[    0.357091] [drm] bpp 32
[    0.357827] [drm] Fifo max 0x00040000 min 0x00001000 cap 0x0000077f
[    0.357854] [drm] Atomic: yes.
[    0.359897] fbcon: svgadrmfb (fb0) is primary device
[    0.363326] Console: switching to colour frame buffer device 100x37
[    0.364164] [drm] Initialized vmwgfx 2.18.0 20200114 for 0000:00:0f.0 on minor 0
[    0.365738] loop: module loaded

whereas the new one seems to explicitly disable the vga console and uses a dummy device instead of the higher-resolution frame buffer:

Code: Select all

[    0.313468] Linux agpgart interface v0.103
[    0.313584] agpgart-intel 0000:00:00.0: Intel 440BX Chipset
[    0.313844] agpgart-intel 0000:00:00.0: AGP aperture is 256M @ 0x0
[    0.313961] vmwgfx 0000:00:0f.0: vgaarb: deactivate vga console
[    0.314027] Console: switching to colour dummy device 80x25
[    0.314187] [TTM] Zone  kernel: Available graphics memory: 1012440 KiB
[    0.314215] vmwgfx 0000:00:0f.0: [drm] FIFO at 0x00000000fe000000 size is 8192 kiB
[    0.314226] vmwgfx 0000:00:0f.0: [drm] VRAM at 0x00000000ec000000 size is 65536 kiB
[    0.314242] vmwgfx 0000:00:0f.0: [drm] Running on SVGA version 2.
[    0.314250] vmwgfx 0000:00:0f.0: [drm] DMA map mode: Caching DMA mappings.
[    0.314266] vmwgfx 0000:00:0f.0: [drm] Legacy memory limits: VRAM = 4096 kB, FIFO = 256 kB, surface = 163840 kB
[    0.314270] vmwgfx 0000:00:0f.0: [drm] MOB limits: max mob size = 0 kB, max mob pages = 0
[    0.314274] vmwgfx 0000:00:0f.0: [drm] Capabilities: rect copy, cursor, cursor bypass, cursor bypass 2, 8bit emulation, alpha cursor, extended fifo, multimon, pitchlock, irq mask, display topology, gmr, traces, gmr2, screen object 2,
[    0.314279] vmwgfx 0000:00:0f.0: [drm] Max GMR ids is 64
[    0.314281] vmwgfx 0000:00:0f.0: [drm] Max number of GMR pages is 65536
[    0.314283] vmwgfx 0000:00:0f.0: [drm] Maximum display memory size is 4096 kiB
[    0.314834] vmwgfx 0000:00:0f.0: [drm] Screen Object display unit initialized
[    0.315860] vmwgfx 0000:00:0f.0: [drm] Fifo max 0x00040000 min 0x00001000 cap 0x0000077f
[    0.315876] vmwgfx 0000:00:0f.0: [drm] Available shader model: Legacy.
[    0.316429] [drm] Initialized vmwgfx 2.19.0 20210722 for 0000:00:0f.0 on minor 0
[    0.317542] loop: module loaded

[NeddySeagoon]

reimers,

Make friends with wgetpaste.

Use wgetpaste to share both kernel .config files and both dmesg outputs,
That will give you four URLs

Please post the URLs.

The old kernel appears to use

Code: Select all

[    0.359897] fbcon: svgadrmfb (fb0) is primary device 

and the new one starts off like that

Code: Select all

[    0.313961] vmwgfx 0000:00:0f.0: vgaarb: deactivate vga console

then switches to vmwgfx, which isn't working.

There are no hints in that dmesg fragment as to why its not working.

If you want a wild guess, turning off the kernel option that provides vmwgfx may restore the old behaviour, as the kernel won't switch to a console driver that's not there.
It's probably not the right solution though, especially if you use a GUI in the VM.

[reimers]

I just looked again in the Kernel configuration and compared the menuconfig of the old with the new one.

The 5.10 menuconfig has the option "Enable framebuffer console under vmwgfx by default" underneath the "DRM driver for VMware Virtual GPU" setting the Symbol DRM_VMWGFX_FBCON

The 5.15 does not show the option in the menu, however searching with / for VMWGFX shows the symbol, but set to "n". Manually adding the symbol to the .config does nothing, it is removed during compilation. But I guess that might be the reason for the frame buffer not working with vmwgfx.

Here are the complete files:

Code: Select all

dmesg 5.10 https://pastebin.com/6X2t6DQ9
dmesg 5.15 https://pastebin.com/Yvbdt9BX

config 5.10 https://pastebin.com/m5ua1enT
config 5.15 https://pastebin.com/5sCnGTjc

[grknight]

I just looked again in the Kernel configuration and compared the menuconfig of the old with the new one.

The 5.10 menuconfig has the option "Enable framebuffer console under vmwgfx by default" underneath the "DRM driver for VMware Virtual GPU" setting the Symbol DRM_VMWGFX_FBCON

The 5.15 does not show the option in the menu, however searching with / for VMWGFX shows the symbol, but set to "n". Manually adding the symbol to the .config does nothing, it is removed during compilation. But I guess that might be the reason for the frame buffer not working with vmwgfx.

Here are the complete files:

Code: Select all

dmesg 5.10 https://pastebin.com/6X2t6DQ9
dmesg 5.15 https://pastebin.com/Yvbdt9BX

config 5.10 https://pastebin.com/m5ua1enT
config 5.15 https://pastebin.com/5sCnGTjc

The /usr/src/linux/scripts/diffconfig on the 2 configs shows the immediate issue of DRM_VMWGFX_FBCON being disabled in the new config by default. This can also be forced on as a test via the command line option vmwgfx.enable_fbdev=1 (like on linux line of grub)

Edit: You might need to enable the dependency of DRM_FBDEV_EMULATION to have DRM_VMWGFX_FBCON show itself again.

[NeddySeagoon]

reimers,

Horrible things happen if you use a text editor on the kernel .config file.
If you are lucky, you will get a kernel that won't build. If not, it may be broken in ways that nobody has ever seen before.

Use menuconfig and its search. Press /
If the symbol you want is not found, press the 'z' key to toggle the display of hidden symbols.
Search again.

The search will find it but you still can't select it. Read the help on the menu option.
Pay attention to the Depends on: That boolean expression must be true before the item can be selected.
Select other things so that your symbol can be selected.

Read the Selects: too. Now ask yourself if you would have got that right with your text editor?

The usual advice to someone who has used a text editor on the .config file is to throw it away and start it again.

[reimers]

Hi,

thank you very much for your help, NeddySeagoon and grknight

Enabling DRM_FBDEV_EMULATION and then DRM_VMWGFX_FBCON worked beautifully, now I have my console back.

Of course, I made a backup of the .config before messing around with it