amdgpu fatal error during init because of firmware blobs?

[L1NTHALO]

Hey,

I've been getting random artifacts (cursor showing twice and blinking, random parts of screen turning black/blue) and it's not my hardware because windows works without problems.

It just appeared out of nowhere, I had no problems yesterday and now I get it on every boot. It's only in a wayland compositor.

I looked into my dmesg and for some reason it tries to load binary blobs that are not listed on the AMDGPU wiki page. When I insert all of the ones that failed to load my kernel doesn't start. Is that normal? It's an RX 7900 XT (Navi 31).

Code: Select all

[    1.167009] Loading firmware: amdgpu/psp_13_0_5_toc.bin
[    1.167020] amdgpu 0000:0f:00.0: Direct firmware load for amdgpu/psp_13_0_5_toc.bin failed with error -2
[    1.167025] [drm:amdgpu_device_init] *ERROR* early_init of IP block <psp> failed -19
[    1.167033] Loading firmware: amdgpu/dcn_3_1_5_dmcub.bin
[    1.167038] amdgpu 0000:0f:00.0: Direct firmware load for amdgpu/dcn_3_1_5_dmcub.bin failed with error -2
[    1.167042] [drm:amdgpu_device_init] *ERROR* early_init of IP block <dm> failed -19
[    1.167048] Loading firmware: amdgpu/gc_10_3_6_pfp.bin
[    1.167053] amdgpu 0000:0f:00.0: Direct firmware load for amdgpu/gc_10_3_6_pfp.bin failed with error -2
[    1.167056] [drm:amdgpu_device_init] *ERROR* early_init of IP block <gfx_v10_0> failed -19
[    1.167062] Loading firmware: amdgpu/sdma_5_2_6.bin
[    1.167068] amdgpu 0000:0f:00.0: Direct firmware load for amdgpu/sdma_5_2_6.bin failed with error -2
[    1.167072] [drm:amdgpu_device_init] *ERROR* early_init of IP block <sdma_v5_2> failed -19
[    1.167077] Loading firmware: amdgpu/vcn_3_1_2.bin
[    1.167082] amdgpu 0000:0f:00.0: Direct firmware load for amdgpu/vcn_3_1_2.bin failed with error -2
[    1.167085] [drm:amdgpu_device_init] *ERROR* early_init of IP block <vcn_v3_0> failed -19
[    1.167090] amdgpu 0000:0f:00.0: amdgpu: Fatal error during GPU init
[    1.167094] amdgpu 0000:0f:00.0: amdgpu: amdgpu: finishing device.

[pietinger]

There are only two possible causes for your problem:
1. you have not installed the package “linux-firmware”, AND/OR
2. you configured your kernel yourself and included amdgpu statically (instead of as a module), but NOT the firmware files; see here:
https://wiki.gentoo.org/wiki/User:Pieti ... s_Firmware
(Yes, sometimes an update to a newer kernel version gives you also a newer driver/module which needs other/newer firmware files)

[L1NTHALO]

I have installed linux-firmware and I also supplied all the firmware blobs listed in the corresponding AMDGPU wki page. When I additionally add the ones in the error message, my kernel doesn't boot.

My gentoo system has been running for over a year without problems so I don't really get why this shows up now.

[pietinger]

Which version of sys-kernel/linux-firmware do you have installed?

What happens if you go back one version?

(I ask because there was a report in our forum that the newest have problems with AMD)

[L1NTHALO]

I downgraded and the problem still persists. I could try replacing corresponding binary blobs but the ones in the error message also seem to be older versions? No idea how it's versioned though

[pietinger]

[...] My gentoo system has been running for over a year without problems so I don't really get why this shows up now.

Maybe it would be helpful if you expain (detailed) what you have changed (update world; kernel; installed something) ...

You said if you include the missed firmware files, then your kernel does not start. What happens exactly? Black screen? Kernel panic? Maybe you have a totally different problem with the kernel?

Maybe check with
https://wiki.gentoo.org/wiki/User:Pieti ... rsion_6.12
-OR-
https://wiki.gentoo.org/wiki/User:Pieti ... ent_Kernel
your kernel.

If you need further help we would need the kernel .config file and a (complete) dmesg after booting your working kernel (please use wgetpaste for this).

[L1NTHALO]

Downgrading linux-firmware does seem to have fixed it. I only downgraded the package without recompiling the kernel with the new firmware blobs before...

It makes sense because all I did was upgrade to 6.15.3 yesterday and when I got the problem I tried downgrading to 6.15.2 without sucess. 6.15.3 probably was the first kernel I compiled with the new linux-firmware (which I updated a week ago so I didn't remember) and of course 6.15.2 was also compiled with them.

I still get the firmware blob errors though. That is really weird.

Not sure if that's worth looking into since the artifacts are gone but still here is my complete dmesg. When adding all of the listed firmware blobs in the error I do get a display but only about 10 lines about some gpu stuff. I'll add all of them and send a picture.

https://bpa.st/ZIWA

[L1NTHALO]

Ok so it want's me to add amdgpu/vcn_3_1_2.bin amdgpu/sdma_5_2_6.bin amdgpu/dcn_3_1_5_dmcub.bin amdgpu/psp_13_0_5_{toc,ta}.bin and amdgpu/gc_10_3_6_{pfp, me, ce}.bin

It all works fine until I add amdgpu/gc_10_3_6_ce.bin, then I get to the following output and nothing more happens:

https://imgur.com/a/Fi83gy6

[pietinger]

I see you have an AMD CPU with integrated gpu ... and an external AMD graphics card ... so, two "cards":

Code: Select all

[    0.315341] amdgpu: ATOM BIOS: 113-D70401XT-P11

[    1.159556] amdgpu: ATOM BIOS: 102-RAPHAEL-008

The first one can load all needed firmware files; the second has missing firmware files. Now it depends how your monitor is connected; and which UEFI BIOS settings you have ... (TBH: I know that AMD does many, many patches - and changes in their firmware - in every kernel release ... sometimes they break something and it is better to wait for a new (minor) kernel version or newer firmware files; sorry I am not an AMD expert ).

[pietinger]

[...] then I get to the following output and nothing more happens: [...]

Do you have a running sshd on this machine? If yes, can you try to ssh in this machine from another? If not:

If you wait a little bit, can you do a CTRL-ALT-DEL? Maybe your system starts but you dont see anything because missing Framebuffer? Shall I check your .config?

[L1NTHALO]

Ahh that makes sense, I completely forgot. I don't use the APU anyways so I'll probably just ignore the firmware files.

Thanks very much for your help!

[Ralphred]

Code: Select all

failed with error -2

This is "file not found"

When I additionally add the ones in the error message, my kernel doesn't boot.

This was so much easier when it was done with "<codename>_<part>.bin" files. Are you adding them to CONFIG_EXTRA_FIRMWARE or an initrd?

My SOP* for "my card is too new and/or not listed in a wiki yet" is to set amdgpu to a module**, then let the firmware load from disk (/lib/firmware/amdgpu/), and build CONFIG_EXTRA_FIRMWARE according to firmware loaded in dmesg (if you try that from a static amdgpu module sometimes some files get missed and you get a partially initialised card). By letting the kernel pick up blobs from the entire firmware repository you also get to see if something is missing due to a kernel/firmware package version mismatch (read: you'll get "error -2" amdgpu messages in dmesg). Once CONFIG_EXTRA_FIRMWARE matches the firmware the module chooses to load you can switch back to a static amdgpu.

*I run a static kernels with no module loading at all
**You will get a moment of "blank monitor" during the boot process, I assume that's while udev does it's work...

[pietinger]

Thanks very much for your help!

You are very Welcome!

[L1NTHALO]

CTRL-ALT-DEL doesn't work. Can't post my .config on wgetpaste for some reason. I think I read somewhere if you have 2 GPU's you shouldn't load the firmware files for both so it's presumably a problem relating to that. Most likely had that error for a year and just didn't notice because I didn't look into my dmesg.

[Ralphred]

I think I read somewhere if you have 2 GPU's you shouldn't load the firmware files for both.

No, it's fine to load firmware to both - I run a Raphael from a Ryzen9 with both Dimgrey_cavefish (RX6xxx series) and Navi33 (RX7xxx series). If you don't want to use the integrated gpu at all then disable it in bios (note: This is known to provoke a "very long pre-post pause" on some chipsets following saving the bios and it restarting, this is normal and OK, just be patient).

[pietinger]

L1NTHALO,

linux-firmware was updated tonight from 20250613 to 20250613-r1. It would be interesting to know if that solves your problem too.

[pietinger]

See also: https://bugs.gentoo.org/958322

Now we have version for sys-kernel/linux-firmware ~20250613_p20250624

[spm]

Ok so it want's me to add amdgpu/vcn_3_1_2.bin amdgpu/sdma_5_2_6.bin amdgpu/dcn_3_1_5_dmcub.bin amdgpu/psp_13_0_5_{toc,ta}.bin and amdgpu/gc_10_3_6_{pfp, me, ce}.bin

It all works fine until I add amdgpu/gc_10_3_6_ce.bin, then I get to the following output and nothing more happens:

https://imgur.com/a/Fi83gy6

I have exactly same issue. Radeon Tonga PRO, everything seems fine until I add amdgpu/gc_10_3_6_ce.bin to the kernel. Then it stops booting in early kernel loading, not responding on keyboard which seems that it never boot correctly. linux-firmware-20250627 installed, still same behaviour. (On the other hand without this firmware graphic seems running fine so I don't know if it's really issue).

[pietinger]

spm,

please give me:

Code: Select all

emerge -pv linux-firmware
grep EXTRA_FIRMWARE /usr/src/linux/.config
dmesg | grep Loading

[spm]

Code: Select all

[ebuild   R    ] sys-kernel/linux-firmware-20250627::gentoo  USE="initramfs redistributable -bindist -compress-xz -compress-zstd -deduplicate -dist-kernel -savedconfig (-unknown-license)" 0 KiB

Code: Select all

CONFIG_EXTRA_FIRMWARE="amd-ucode/microcode_amd_fam19h.bin amdgpu/tonga_sdma.bin amdgpu/tonga_sdma1.bin amdgpu/tonga_mc.bin amdgpu/tonga_pfp.bin amdgpu/tonga_me.bin amdgpu/tonga_ce.bin amdgpu/tonga_rlc.bin amdgpu/tonga_mec.bin amdgpu/tonga_mec2.bin amdgpu/tonga_uvd.bin amdgpu/tonga_vce.bin amdgpu/tonga_k_smc.bin amdgpu/psp_13_0_5_toc.bin amdgpu/dcn_3_1_5_dmcub.bin amdgpu/gc_10_3_6_pfp.bin amdgpu/sdma_5_2_6.bin amdgpu/vcn_3_1_2.bin amdgpu/psp_13_0_5_ta.bin amdgpu/gc_10_3_6_me.bin amdgpu/gc_10_3_6_ce.bin"
CONFIG_EXTRA_FIRMWARE_DIR="/lib/firmware"

Code: Select all

[    2.797455] Loading firmware: amdgpu/tonga_sdma.bin
[    2.797486] Loading firmware: amdgpu/tonga_sdma1.bin
[    2.802476] Loading firmware: amdgpu/tonga_mc.bin
[    2.803793] Loading firmware: amdgpu/tonga_pfp.bin
[    2.803822] Loading firmware: amdgpu/tonga_me.bin
[    2.803849] Loading firmware: amdgpu/tonga_ce.bin
[    2.803866] Loading firmware: amdgpu/tonga_rlc.bin
[    2.803890] Loading firmware: amdgpu/tonga_mec.bin
[    2.804289] Loading firmware: amdgpu/tonga_mec2.bin
[    2.804853] Loading firmware: amdgpu/tonga_uvd.bin
[    2.805819] Loading firmware: amdgpu/tonga_vce.bin
[    2.806215] Loading firmware: amdgpu/tonga_k_smc.bin
[    3.316645] Loading firmware: amdgpu/psp_13_0_5_toc.bin
[    3.316657] Loading firmware: amdgpu/psp_13_0_5_ta.bin
[    3.317017] Loading firmware: amdgpu/dcn_3_1_5_dmcub.bin
[    3.317387] Loading firmware: amdgpu/gc_10_3_6_pfp.bin
[    3.317791] Loading firmware: amdgpu/gc_10_3_6_me.bin
[    3.318193] Loading firmware: amdgpu/gc_10_3_6_ce.bin
[    3.318222] Loading firmware: amdgpu/sdma_5_2_6.bin
[    3.318277] Loading firmware: amdgpu/vcn_3_1_2.bin
[    3.331370] Loading compiled-in X.509 certificates
[    6.359004] Loading firmware: rtl_nic/rtl8125d-1.fw

[pietinger]

spm,

does it work if you (re-)emerge linux-firmware-20250410 (this is the current stable; and AFAIK it worked for @L1NTHALO) ?

Another option would be to try the newest kernel version 6.12.36 or 6.15.5 (I have seen a LOT of amdgpu patches in this version).

[spm]

Ok, I've tried kernel 6.15.4 with linux-firmware-20250410, still same issue. So I've tried new kernel 6.15.5 with new linux-firmware-20250627 and also still same. Should I try 6.15.5 with stable linux-firmware-20250410?

(In all cases still helps if I move amdgpu/gc_10_3_6_ce.bin out of kernel so it's not loaded).

[pietinger]

[...] Should I try 6.15.5 with stable linux-firmware-20250410? [...]

Yeah ... it would be very interesting to see if the amdgpu module or the firmware causes the problem.

[spm]

I've tried it but unfortunately still same issue - with amdgpu/gc_10_3_6_ce.bin it hangs immediately after loading firmware.

Maybe one interesting notice: I did this change recently. Previously I had amdgpu as module and no firmware compiled in kernel - it was loading from /lib/firmware dir. Then I changed motherboard with CPU (graphic card remains the same) and I was forced to compile amdgpu with firmware to kernel - new motherboard for some reason won't init graphic card in PCIe slot and output only to integraded VGA in CPU. And I also have crypted /, so I compiled amdgpu to kernel to get video output when I'm unlocking rootfs. Since that time I find out this issue. Point is that even on this same setup, same kernels and same linux-firmware it didn't happend when firmware was autoloaded from /lib/firmware dir.

[pietinger]

I've tried it but unfortunately still same issue - with amdgpu/gc_10_3_6_ce.bin it hangs immediately after loading firmware. [...]

Do you get exact the same list of loaded firmware files as in your previous post, if you boot our newest GentooLiveCD (not the admin or minimal CD) and do a "dmesg | grep Loading" ?

(I ask because it could be there is one firmware file missing in your CONFIG_EXTRA_FIRMWARE and this missing one leads to a problem with gc_10_3_6_ce.bin)