Broken video on Intel iGPU and Mesa 25.2.x and Mesa 25.3.x

[mortonP]

Since early Dec my video playing did no longer work and I finally found some time to somewhat debug this.

Reproducer:
Observed on Intel iGPU, Gen 10 and Gen 13 CPU tested.
Play with mpv a video track dumped from a DVD (this is MPEG2?).

Result: mpv window opens, it is black, sometimes seems to play as audio is hearable, sometimes just stuck, when killing mpv process remotely system usually unstable with weird errors afterwards -> best is to immediately reboot

Mesa:
25.1.9 ok
25.2.5 broken
25.2.8 broken
25.2.9 broken
25.3.1 broken
25.3.2 broken

I thought this is bug https://gitlab.freedesktop.org/mesa/mesa/-/issues/12910 (because DVD video is small res and always resized to due aspect?)
but this patch should be in 25.3.2 - and it still does not work.

Unfortunately, I need these 2 machines working, so today was finally some time to debug this.
Ideas to debug this welcome!
How to get older mesa&mesa_clc ebuilds 25.2.0-25.2.4 for testing?
Or how to efficiently bisect this over the next days?


Back to family and cookies...

[Asch]

Around that same time I started having problem with vulkan.

Maybe if you add the --gpu-api=opengl parameter to mpv it will work all right.

In the meanwhile, I will just wait for an update to magically solve the issue.

[mortonP]

>Around that same time I started having problem with vulkan.
>Maybe if you add the --gpu-api=opengl parameter to mpv it will work all right.

You are right - running mpv with --gpu-api=opengl allows to play all video formats on 25.3.2 without problem.
If this bug is not fixed/found, how to disable this globally for applications?


>In the meanwhile, I will just wait for an update to magically solve the issue.

Well, open source requires bug reports and minimal reproducible cases for developers to find and fix problems.
What is you kernel, Mesa version, CPU/GPU generation?



Meanwhile I found out:
It does not matter what kind of video, MPEG2/DVD, H264, VP9, ....

Everything works fine on Arrowlake+Mesa 25.3.2+Kernel 6.18.2
...the older machines still run on 6.12.x, so unclear whether it is the kernel - but probably not?


When not working I see:

in kernel log on player start:
kernel: i915 0000:00:02.0: [drm] Resetting rcs0 for preemption time out
kernel: i915 0000:00:02.0: [drm] vo[17558] context reset due to GPU hang
kernel: i915 0000:00:02.0: [drm] GPU HANG: ecode 9:1:e757fefe, in vo [17558]

in "mpv -v" output:
.....
[cplayer] audio ready
[cplayer] starting audio playback
[cplayer] playback restart complete @ 0.000000, audio=playing, video=playing
[statusline] AV: 00:00:00 / 00:01:56 (0%) A-V: 0.000
[vo/gpu/x11] Disabling screensaver.
[statusline] AV: 00:00:00 / 00:01:56 (0%) A-V: 0.000
[ao/alsa] starting AO
[statusline] AV: 00:00:00 / 00:01:56 (0%) A-V: 0.000
[vo/gpu/libplacebo] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../libplacebo-v7.351.0/src/vulkan/command.c:504)
[statusline] AV: 00:00:00 / 00:01:56 (0%) A-V: 0.000
[vo/gpu/libplacebo] Failed holding swapchain image for presentation
[statusline] AV: 00:00:00 / 00:01:56 (0%) A-V: 0.000
[vo/gpu] Failed presenting frame!
[statusline] AV: 00:00:00 / 00:01:56 (0%) A-V: 0.000
[vo/gpu/libplacebo] Retrieving query pool results: VK_ERROR_DEVICE_LOST (../libplacebo-v7.351.0/src/vulkan/gpu.c:105)
[statusline] AV: 00:00:00 / 00:01:56 (0%) A-V: 0.000
[vo/gpu/libplacebo] Retrieving query pool results: VK_ERROR_DEVICE_LOST (../libplacebo-v7.351.0/src/vulkan/gpu.c:105)
[statusline] AV: 00:00:00 / 00:01:56 (0%) A-V: 0.000
[vo/gpu/libplacebo] Retrieving query pool results: VK_ERROR_DEVICE_LOST (../libplacebo-v7.351.0/src/vulkan/gpu.c:105)
[statusline] AV: 00:00:00 / 00:01:56 (0%) A-V: 0.000
[vo/gpu/libplacebo] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../libplacebo-v7.351.0/src/vulkan/command.c:504)
[statusline] AV: 00:00:00 / 00:01:56 (0%) A-V: 0.000
[vo/gpu/libplacebo] Retrieving query pool results: VK_ERROR_DEVICE_LOST (../libplacebo-v7.351.0/src/vulkan/gpu.c:105)
[statusline] AV: 00:00:00 / 00:01:56 (0%) A-V: 0.000
[vo/gpu/libplacebo] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../libplacebo-v7.351.0/src/vulkan/command.c:504)
[statusline] AV: 00:00:00 / 00:01:56 (0%) A-V: 0.000
[vo/gpu/libplacebo] Retrieving query pool results: VK_ERROR_DEVICE_LOST (../libplacebo-v7.351.0/src/vulkan/gpu.c:105)
[statusline] AV: 00:00:00 / 00:01:56 (0%) A-V: 0.000


when problem, depending on Mesa 25.2, or Mesa 25.3, or which video codec was used I get either
- screen stuck, but hear audio playing correctly in background
- screen stuck, no audio playing

when screen stuck, remote ssh login still works, manual reboot leads to kernel crash during shutdown: https://i.imgur.com/iiUAf8x.jpeg
Hard reboot by power button required.

Overall, back to Mesa 25.1.9 meanwhile....

[Asch]

>Around that same time I started having problem with vulkan.
>Maybe if you add the --gpu-api=opengl parameter to mpv it will work all right.

You are right - running mpv with --gpu-api=opengl allows to play all video formats on 25.3.2 without problem.
If this bug is not fixed/found, how to disable this globally for applications?


>In the meanwhile, I will just wait for an update to magically solve the issue.

Well, open source requires bug reports and minimal reproducible cases for developers to find and fix problems.
What is you kernel, Mesa version, CPU/GPU generation?

I'm using a RTX 2060 with a Ryzen 3700X, 32 GB RAM. I'm using nouveau +NVK on a clang+musl build. I reported this on this forums, "Other things Gentoo" and on IRC. AFAIK, Alpine Linux uses a similar setup to mine and is also holding back on mesa updates. So it is highly probably not just me or my system.

It might be true that I enabled very aggressive transformation *FLAGS in a global level, but I can't state for sure that is the cause. I also tried multiple versions of mesa, compiling mesa and musl with debug flags and no hardening/optimization whatsoever, but segfaults still persist and traces are not very informative. I could file a bug on bugs.gentoo.org, but I think the forums are a better platform to debug and delve into traces, and teach me to do tracing properly.

Something I noticed is that xdg-desktop-portal* segfaults trying to access libc.so some time after I have any vulkan-related segfault when my system suspends to ram. This doesn't happen if I don't have a vulkan segfault before.

Meanwhile I found out:
It does not matter what kind of video, MPEG2/DVD, H264, VP9, ....

Everything works fine on Arrowlake+Mesa 25.3.2+Kernel 6.18.2
...the older machines still run on 6.12.x, so unclear whether it is the kernel - but probably not?

Could be that you arrowlake machine is on stable amd64 config, whereas you are getting the vulkan issues on ~amd64.


BTW, cheers and ty for the tips [:D]

[mortonP]

>using a RTX 2060 with a Ryzen 3700X, 32 GB RAM. I'm using nouveau +NVK on a clang+musl build.

That's sounds kinda different than mine.
Could this be a different bug?

>but I think the forums are a better platform to debug and delve into traces,

well, I also didn't want to file a bug yet before narrowing it more down...

>you arrowlake machine is on stable amd64 config, whereas you are getting the vulkan issues on ~amd64.

Actually the arrowlake as the fastest is usually the first to build and run new packages - and see whether they have some bugs - that's why brand-new kernel and mesa.
As I wrote above the older ones are production and I _really_ need them working, therefore they usually receive any updates rather late - I waited 2-3 weeks with the mesa 25.1 -> 25.2 jump - and then I had a panic...

[Chiitoo]

I use the mesa-9999 ebuild for bisecting mesa issues, and at least with a commit to blame, they haven't asked for a minimal example yet. :]

You can set the commit with 'EGIT_OVERRIDE_COMMIT_MESA_MESA'.

Since the ebuild is meant to track git master, it will sometimes break when going back in time far enough, but it has worked very well for me (and if it does break, it's fairly easy to get the ebuild (or older release ebuilds) via git in a state it has previously been in, though it can of course get funky with dependencies changing).

[Asch]

That's sounds kinda different than mine.
Could this be a different bug?

I don't know. Fact is, my workaround worked for you too. This could indicate that some recent update messes with Vulkan somehow, and this affects our systems in different ways because they are, well, different.

I downgraded mesa multiple times, to multiple versions. Tried compiling it with debug flags, no flags at all, etc. for no avail. This leads me to believe this is caused by some update to some other package. Seeing that libc.so belongs to musl, I tried compiling musl with only debug *FLAGS, but that made no difference as well .

I could try running

Code: Select all

genlop -s

to see what has been updated around that time, and then checking each package with

Code: Select all

genlop -t <packagename>

to see what version was installed before and try downgrading packages to see if this solves the issue. However, I am a bit lazy at this point. On the other hand, it is something you could do as well if you have the time and the will.

Actually the arrowlake as the fastest is usually the first to build and run new packages - and see whether they have some bugs - that's why brand-new kernel and mesa.
As I wrote above the older ones are production and I _really_ need them working, therefore they usually receive any updates rather late - I waited 2-3 weeks with the mesa 25.1 -> 25.2 jump - and then I had a panic...

When running production systems like this, it is important to have a rollback mechanism in place, just in case. It is usual to perform routine backups, as this isn't really system-specific.

If you are running on a BTRFS subvolume, you can try creating snapshots and then restoring them if something unexpected happens.

When running Gentoo, you could try using quickpkg to create binary packages of any package you are going to subsequently update. If anything fails, you could run emerge -G <packagename> to downgrade said package.

[mortonP]

There is also bug https://gitlab.freedesktop.org/mesa/mesa/-/issues/14445
a problem of Mesa and libplacebo interaction, which also seems to affect Nvidia,
but looking at the bisected problem patch https://gitlab.freedesktop.org/mesa/mes ... 7e8c0e53b2
the reported tags do not match with the Mesa version I'm observing - so its not my problem?

I use the mesa-9999 ebuild for bisecting mesa issues, and at least with a commit to blame, they haven't asked for a minimal example yet. :]
You can set the commit with 'EGIT_OVERRIDE_COMMIT_MESA_MESA'.

Ah, with the 9999 build I can build with a specific commit?

It seems mesa AND mesa_clc need to be up/downgraded at the same time (and mesa_clc is needed for Vulkan, right?) - so I need to find matching commits.
Unfortunately, now there is also the LLVM 21 upgrade waiting, too...
...this is turning into a mess :-/

[mortonP]

Found a Intel Gen 8 CPU machine for testing.
Reproduced problem.

[many hours and reboots later]

Everything works fine on Arrowlake+Mesa 25.3.2+Kernel 6.18.2
...the older machines still run on 6.12.x, so unclear whether it is the kernel - but probably not?

I have to correct myself - it appears to be the kernel.
If you are on 6.12 and Mesa 25.2/25.3 -> problem
6.18.2 and Mesa 25.2/25.3 -> no problem

That also aligns with why Arrowlake never showed the problem - ARL needs a brand-new kernel and was never tested with 6.12.

I have not tested all combinations yet, and I'm a bit nervous upgrading the production machines right now to a .2 kernel, so I'll still stay a bit on Mesa 25.1 until 6.18 receives a few more revisions.

Also, don't know what to do about the LLVM 21 upgrade yet - but maybe this resolves itself in the next days?:
llvm-core/libclc:0
(llvm-core/libclc-21.1.8:0/0::gentoo, ebuild scheduled for merge) USE="spirv -verify-sig" ABI_X86="(64)" LLVM_SLOT="21" VIDEO_CARDS="-nvidia -r600 -radeonsi" conflicts with
=llvm-core/libclc-20* required by (dev-util/mesa_clc-25.2.7:0/0::gentoo, installed) USE="-debug" ABI_X86="(64)" LLVM_SLOT="20 -18 -19" VIDEO_CARDS="-asahi -panfrost"


Now I can go celebrating and drinking... right?

[mortonP]

Also, don't know what to do about the LLVM 21 upgrade yet - but maybe this resolves itself in the next days?:

For reference, meanwhile someone filed a bug about that: https://bugs.gentoo.org/968274

[Tep04Ka]

Can confirm the issue and the solution with upgrading the kernel.

I have Thinkpad T480 with Intel i7-8550U with integrated graphics, and I use default/linux/amd64/23.0/hardened profile. I used gentoo-sources-6.12.58. My mesa useflags: {X llvm lm-sensors opengl proprietary-codecs vaapi vulkan zstd}, VIDEO_CARDS=intel.

Around the beginning of December my system would hard-freeze whenever GPU was used, and the solution I had found was to mask media-libs/mesa-25.2.7, thus reverting back to mesa-25.1.9. A couple of days back during the @world system update I've upgraded to mesa-25.2.8, and after the reboot system would become corrupted whenever Vulkan was involved: this includes the browser (qutebrowser, qtwebengine-6.10.1), Anki and mpv with `hwdec=vulkan`. The MRE I've found is `vkcube`: it spawns black window, `dmesg` writes

Code: Select all

[drm] Resetting rcs0 for preemption time out
[drm] vkcube[2996] context reset due to GPU hang
[drm] GPU HANG: ecode 9:1:e757fefe, in vkcube[2996]

and the whole system becomes corrupted: for example `gcc` starts to segfault, and I had similar kernel crashes during shutdown as mortonP have mentioned. I also have tried upgrading to ~amd64 and downgrading: both to no avail unless I installed mesa-25.1.9 back.

Right now I am using gentoo-sources-6.18.2 and mesa-25.3.3 and everything seems to be working fine, including the `vkcube`. Thank you for the solution, as this is my main and only work machine.

Should this bug be reported?

[mortonP]

I'll still stay a bit on Mesa 25.1 until 6.18 receives a few more revisions.

With the poppler and icu bumps also happening, this weekend was Update Day[tm].

Finally made the jump to kernel 6.18.10 + mesa 25.3.5

So far no problems observed, yay! :-)