sci-ml/ollama in GURU seems broken in latest ebuild

[robinelvin]

Just asking here before posting a bug report. I'm running sci-ml/ollama from the GURU overlay and a recent upgrade or 0.13.5-r1 broke CUDA/Nvidia support.

The log message:

Code: Select all

time=2026-01-06T16:33:52.735Z level=DEBUG source=runner.go:153 msg="filtering device which didn't fully initialize" id=GPU-81103a2a-829d-6143-3c44-364f5faae525 libdir=/usr/lib64/ollama/backends pci_id=0000:00:07.0 library=CUDA

This means Ollama won't use my GPU.

I downgraded to 0.13.4 and got the same problem, rebooted just in case, same problem. Downgrading to 0.13.0 allows the GPU to work again.

Does anyone have any insights?

[Navar]

ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon RX 6900 XT, gfx1030 (0x1030), VMM: no, Wave Size: 32,
ID: GPU-b1ec48fea8c82ef1
load_backend: loaded ROCm backend from /usr/lib64/ollama/backends/libggml-hip.so
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon RX 6900 XT (RADV NAVI21) (radv) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 1 | matrix cores: none
load_backend: loaded Vulkan backend from /usr/lib64/ollama/backends/libggml-vulkan.so
load_backend: loaded CPU backend from /usr/lib64/ollama/backends/libggml-cpu-haswell.so


[I-O] [ ] sci-ml/ollama-0.13.5-r1:0 working fine here with its wrapper around CUDA.

It should be easier on your end than here. Check your CUDA toolkit install/version, etc. Are you running in a container as well? Are you checking nvtop to monitor the GPU for other signs of life like vram allocation/activity? Should be a lot of noise from other people if the situation is commonplace. I didn't see anything interesting that was net new for a year+ on usual troubleshooting.

[robinelvin]

I changed the debug level to 2 and now I can see the issue:

Code: Select all

time=2026-01-09T10:14:26.542Z level=INFO source=ggml.go:136 msg="" architecture=llama file_type=unknown name="" description="" num_t
ensors=0 num_key_values=3
time=2026-01-09T10:14:26.542Z level=DEBUG source=ggml.go:94 msg="ggml backend load all from path" path=/usr/lib64/ollama
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
/var/tmp/portage/sci-ml/ollama-0.13.5-r1/work/ollama-0.13.5/ml/backend/ggml/ggml/src/ggml-cuda/ggml-cuda.cu:335: GGML_ASSERT(ggml_cu
da_has_arch(info.devices[id].cc) && "ggml was not compiled with support for this arch") failed
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
/var/tmp/portage/sci-ml/ollama-0.13.5-r1/work/ollama-0.13.5/ml/backend/ggml/ggml/src/ggml-cuda/ggml-cuda.cu:335: GGML_ASSERT(ggml_cu
da_has_arch(info.devices[id].cc) && "ggml was not compiled with support for this arch") failed
/usr/lib64/ollama/libggml-base.so.0(+0x17686) [0x7f9c55f8e686]
/usr/lib64/ollama/libggml-base.so.0(ggml_print_backtrace+0x20a) [0x7f9c55f8ebda]
/usr/lib64/ollama/libggml-base.so.0(ggml_abort+0x15b) [0x7f9c55f8edbb]
/usr/lib64/ollama/libggml-cuda.so(+0x10a93b) [0x7f9c2e10a93b]
/usr/lib64/ollama/libggml-cuda.so(_Z14ggml_cuda_infov+0x71) [0x7f9c2e10b5e1]
/usr/lib64/ollama/libggml-cuda.so(ggml_backend_cuda_reg+0x95) [0x7f9c2e10ca85]
/usr/bin/ollama(+0xe4f602) [0x559301a70602]
/usr/bin/ollama(+0xe4d206) [0x559301a6e206]
/usr/bin/ollama(+0xe4e374) [0x559301a6f374]
/usr/bin/ollama(+0xd2581) [0x559300cf3581]
/usr/lib64/ollama/libggml-base.so.0(+0x17686) [0x7f64905df686]
/usr/lib64/ollama/libggml-base.so.0(ggml_print_backtrace+0x20a) [0x7f64905dfbda]
/usr/lib64/ollama/libggml-base.so.0(ggml_abort+0x15b) [0x7f64905dfdbb]
/usr/lib64/ollama/libggml-cuda.so(+0x10a93b) [0x7f646e10a93b]
/usr/lib64/ollama/libggml-cuda.so(_Z14ggml_cuda_infov+0x71) [0x7f646e10b5e1]
/usr/lib64/ollama/libggml-cuda.so(ggml_backend_cuda_reg+0x95) [0x7f646e10ca85]
/usr/bin/ollama(+0xe4f602) [0x557fcaf8c602]
/usr/bin/ollama(+0xe4d206) [0x557fcaf8a206]
/usr/bin/ollama(+0xe4e374) [0x557fcaf8b374]
/usr/bin/ollama(+0xd2581) [0x557fca20f581]
SIGABRT: abort

[BJP]

I changed the debug level to 2 and now I can see the issue:

Did you get this fixed? I found in my case that I had to explicitly set CUDAARCHS="75" to support my RTX 2060 Super, as without setting CUDAARCHS the ebuild was building the library with support only for very recent cards.

[robinelvin]

I changed the debug level to 2 and now I can see the issue:

Did you get this fixed? I found in my case that I had to explicitly set CUDAARCHS="75" to support my RTX 2060 Super, as without setting CUDAARCHS the ebuild was building the library with support only for very recent cards.

I reported a bug here: https://bugs.gentoo.org/968549

It's been closed but it still isn't working for me. I have tried with your CUDAARCHS flag and that doesn't work either. Only 0.13.0 worked for me and now that's gone so my setup is broken now.

[BJP]

I changed the debug level to 2 and now I can see the issue:

Did you get this fixed? I found in my case that I had to explicitly set CUDAARCHS="75" to support my RTX 2060 Super, as without setting CUDAARCHS the ebuild was building the library with support only for very recent cards.

I reported a bug here: https://bugs.gentoo.org/968549

It's been closed but it still isn't working for me. I have tried with your CUDAARCHS flag and that doesn't work either. Only 0.13.0 worked for me and now that's gone so my setup is broken now.

Funny thing for me is I think the fix put in for your bug is what broke it for me. When built with CUDAARCHS="all-major" libggml-cuda.so only had support for sm100 and sm120, leading my card to be filtered out with the same "filtering device which didn't fully initialize" message.

You can check what which architectures are supported on the currently installed build by running:

Code: Select all

% strings /usr/lib64/ollama/libggml-cuda.so | grep -E "sm_[0-9]+" | sort -u
.target sm_75

Identify the architecture of your card by running:

Code: Select all

% nvidia-smi --query-gpu=compute_cap --format=csv,noheader
7.5

Assuming you don't have the same card as me, there's probably a different number you would need in CUDAARCHS make.conf setting.

[robinelvin]

Did you get this fixed? I found in my case that I had to explicitly set CUDAARCHS="75" to support my RTX 2060 Super, as without setting CUDAARCHS the ebuild was building the library with support only for very recent cards.

I reported a bug here: https://bugs.gentoo.org/968549

It's been closed but it still isn't working for me. I have tried with your CUDAARCHS flag and that doesn't work either. Only 0.13.0 worked for me and now that's gone so my setup is broken now.

Funny thing for me is I think the fix put in for your bug is what broke it for me. When built with CUDAARCHS="all-major" libggml-cuda.so only had support for sm100 and sm120, leading my card to be filtered out with the same "filtering device which didn't fully initialize" message.

You can check what which architectures are supported on the currently installed build by running:

Code: Select all

% strings /usr/lib64/ollama/libggml-cuda.so | grep -E "sm_[0-9]+" | sort -u
.target sm_75

Identify the architecture of your card by running:

Code: Select all

% nvidia-smi --query-gpu=compute_cap --format=csv,noheader
7.5

Assuming you don't have the same card as me, there's probably a different number you would need in CUDAARCHS make.conf setting.

Ah, that's what I was missing! nvidia-smi returned 8.6 for my card so I set CUDAARCHS="86" and now it is working.

Thank you @BJP!