Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Nvidia version mismatch between kernel and driver.
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Multimedia
View previous topic :: View next topic  
Author Message
dartleader
Tux's lil' helper
Tux's lil' helper


Joined: 21 Apr 2019
Posts: 119

PostPosted: Mon Apr 26, 2021 2:21 am    Post subject: Nvidia version mismatch between kernel and driver. Reply with quote

Hello everyone, I'm having some trouble reconciling versions of my nvidia-drivers, and I'm unable to use my video card currently. On the last kernel update to 5.10.27 I mistakenly made the new kernel using oldconfig and genkernel all, which wiped out most of my kernel customization. I have since rebuilt the kernel with my desired settings but cannot figure out how to point genkernel at my most recently built kernel. Despite several kernel rebuilds and running emerge @module-rebuild, my nvidia module continues to point to the outdated version.

Output of modprobe nvidia
Code:
 filename:       /lib/modules/5.10.27-gentoo-x86_64/video/nvidia.ko
alias:          char-major-195-*
version:        460.39
supported:      external
license:        NVIDIA
srcversion:     88778FD9C7BD9CA16708044
alias:          pci:v000010DEd*sv*sd*bc03sc02i00*
alias:          pci:v000010DEd*sv*sd*bc03sc00i00*
depends:        i2c-core
retpoline:      Y
name:           nvidia
vermagic:       5.10.27-gentoo-x86_64 SMP mod_unload modversions
parm:           NvSwitchRegDwords:NvSwitch regkey (charp)
parm:           NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp)
parm:           nv_cap_enable_devfs:Enable (1) or disable (0) nv-caps devfs support. Default: 1 (int)
parm:           NVreg_ResmanDebugLevel:int
parm:           NVreg_RmLogonRC:int
parm:           NVreg_ModifyDeviceFiles:int
parm:           NVreg_DeviceFileUID:int
parm:           NVreg_DeviceFileGID:int
parm:           NVreg_DeviceFileMode:int
parm:           NVreg_InitializeSystemMemoryAllocations:int
parm:           NVreg_UsePageAttributeTable:int
parm:           NVreg_RegisterForACPIEvents:int
parm:           NVreg_EnablePCIeGen3:int
parm:           NVreg_EnableMSI:int
parm:           NVreg_TCEBypassMode:int
parm:           NVreg_EnableStreamMemOPs:int
parm:           NVreg_EnableBacklightHandler:int
parm:           NVreg_RestrictProfilingToAdminUsers:int
parm:           NVreg_PreserveVideoMemoryAllocations:int
parm:           NVreg_EnableS0ixPowerManagement:int
parm:           NVreg_S0ixPowerManagementVideoMemoryThreshold:int
parm:           NVreg_DynamicPowerManagement:int
parm:           NVreg_DynamicPowerManagementVideoMemoryThreshold:int
parm:           NVreg_EnableUserNUMAManagement:int
parm:           NVreg_MemoryPoolSize:int
parm:           NVreg_KMallocHeapMaxSize:int
parm:           NVreg_VMallocHeapMaxSize:int
parm:           NVreg_IgnoreMMIOCheck:int
parm:           NVreg_NvLinkDisable:int
parm:           NVreg_EnablePCIERelaxedOrderingMode:int
parm:           NVreg_RegisterPCIDriver:int
parm:           NVreg_RegistryDwords:charp
parm:           NVreg_RegistryDwordsPerDevice:charp
parm:           NVreg_RmMsg:charp
parm:           NVreg_GpuBlacklist:charp
parm:           NVreg_TemporaryFilePath:charp


Output of emerge --info nvidia-drivers:

Code:
Portage 3.0.17 (python 3.8.8-final-0, default/linux/amd64/17.1/hardened, gcc-10.2.0, glibc-2.32-r7, 5.10.27-gentoo-x86_64 x86_64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-5.10.27-gentoo-x86_64-x86_64-AMD_Ryzen_7_4800H_with_Radeon_Graphics-with-glibc2.2.5
KiB Mem:    16323832 total,   1697116 free
KiB Swap:   16777212 total,  16777212 free
Timestamp of repository gentoo: Sun, 25 Apr 2021 00:45:01 +0000
Head commit of repository gentoo: 5999a72a0c55d81b7092560c91fb1eb95ee92ccc
Head commit of repository steam-overlay: 9a1249fdb3e3005041fb768af02e235c46dc4a18

sh bash 5.0_p18
ld GNU ld (Gentoo 2.35.2 p1) 2.35.2
distcc 3.3.3 x86_64-pc-linux-gnu [enabled]
app-shells/bash:          5.0_p18::gentoo
dev-lang/perl:            5.30.3::gentoo
dev-lang/python:          2.7.18_p8::gentoo, 3.8.8_p1::gentoo, 3.9.2_p1::gentoo
dev-lang/rust:            1.51.0-r2::gentoo
dev-util/cmake:           3.18.5::gentoo
sys-apps/baselayout:      2.7::gentoo
sys-apps/openrc:          0.42.1-r1::gentoo
sys-apps/sandbox:         2.20::gentoo
sys-devel/autoconf:       2.13-r1::gentoo, 2.69-r5::gentoo
sys-devel/automake:       1.16.2-r1::gentoo
sys-devel/binutils:       2.35.2::gentoo
sys-devel/gcc:            10.2.0-r5::gentoo
sys-devel/gcc-config:     2.4::gentoo
sys-devel/libtool:        2.4.6-r6::gentoo
sys-devel/make:           4.3::gentoo
sys-kernel/linux-headers: 5.10::gentoo (virtual/os-headers)
sys-libs/glibc:           2.32-r7::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000
    sync-rsync-verify-jobs: 1
    sync-rsync-verify-metamanifest: yes
    sync-rsync-verify-max-age: 24
    sync-rsync-extra-opts:

haskell
    location: /var/lib/layman/haskell
    masters: gentoo
    priority: 50

steam-overlay
    location: /usr/local/portage/steam-overlay
    sync-type: git
    sync-uri: https://github.com/anyc/steam-overlay.git
    masters: gentoo
    priority: 50

ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="@FREE"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=znver2 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=znver2 -O2 -pipe"
DISTDIR="/var/cache/distfiles"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-march=znver2 -O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distcc distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-march=znver2 -O2 -pipe"
GENTOO_MIRRORS="http://gentoo.mirrors.tera-byte.com/ https://mirror.csclub.uwaterloo.ca/gentoo-distfiles/"
LANG="en_CA.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="acl amd64 bzip2 crypt elogind hardened iconv ipv6 libglvnd libtirpc multilib ncurses nls nptl openmp pam pcre pie readline seccomp split-usr ssl ssp unicode xattr xtpax zlib" ABI_X86="64" ADA_TARGET="gnat_2018" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-3 php7-4" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_8" PYTHON_TARGETS="python3_8" RUBY_TARGETS="ruby26" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LINGUAS, MAKEOPTS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RUSTFLAGS

=================================================================
                        Package Settings
=================================================================

x11-drivers/nvidia-drivers-460.56::gentoo was built with the following:
USE="X driver kms multilib tools -compat (-dist-kernel) -static-libs -uvm -wayland" ABI_X86="32 (64) (-x32)"


Output of uname:
uname -a:
Code:
Linux regicide 5.10.27-gentoo-x86_64 #1 SMP Sat Apr 3 08:37:00 -00 2021 x86_64 AMD Ryzen 7 4800H with Radeon Graphics AuthenticAMD GNU/Linux

I have rebuild my kernel several times since April 3, I'm not sure why it is still pointing to something made on that date. Is there any way to update it?


I appreciate any help, thank you!
Back to top
View user's profile Send private message
fedeliallalinea
Administrator
Administrator


Joined: 08 Mar 2003
Posts: 30923
Location: here

PostPosted: Mon Apr 26, 2021 5:14 am    Post subject: Reply with quote

There are other messages in dmesg?
_________________
Questions are guaranteed in life; Answers aren't.
Back to top
View user's profile Send private message
Banana
Veteran
Veteran


Joined: 21 May 2004
Posts: 1393
Location: Germany

PostPosted: Mon Apr 26, 2021 7:20 am    Post subject: Reply with quote

as you rebuilded your kernel did you made sure /boot was mounted so the new kernel kan be used?
_________________
My personal space
My delta-labs.org snippets do expire

PFL - Portage file list - find which package a file or command belongs to.
Back to top
View user's profile Send private message
dartleader
Tux's lil' helper
Tux's lil' helper


Joined: 21 Apr 2019
Posts: 119

PostPosted: Tue Apr 27, 2021 4:36 am    Post subject: Reply with quote

fedeliallalinea wrote:
There are other messages in dmesg?

Code:
[  372.941345] NVRM: API mismatch: the client has the version 460.56, but
               NVRM: this kernel module has the version 460.39.  Please
               NVRM: make sure that this kernel module and all NVIDIA driver
               NVRM: components have the same version.


And yes banana, I have made sure my /boot is mounted when I am making the kernel and when I run make install.
Back to top
View user's profile Send private message
Banana
Veteran
Veteran


Joined: 21 May 2004
Posts: 1393
Location: Germany

PostPosted: Tue Apr 27, 2021 10:22 am    Post subject: Reply with quote

is this still valid ?
Quote:
I have rebuild my kernel several times since April 3, I'm not sure why it is still pointing to something made on that date. Is there any way to update it?


if so then you are not using the new builds. uname -a tells you what is currently used and if this does not change then you are not using the new kernel build
_________________
My personal space
My delta-labs.org snippets do expire

PFL - Portage file list - find which package a file or command belongs to.
Back to top
View user's profile Send private message
dartleader
Tux's lil' helper
Tux's lil' helper


Joined: 21 Apr 2019
Posts: 119

PostPosted: Thu Apr 29, 2021 3:41 am    Post subject: Reply with quote

So I've ended up solving this issue by copying over an old 5.4 kernel, running make oldconfig and just rebuilding the kernel from there. I'll have to manually shave down the kernel again but I would rather do that than deal with the hassle of the genkernel/manual make issue.

My current kernel is working and I generated it using plain old make. I'm only going to use genkernel initramfs moving forward and stay away from genkernel all; that's where I started running into all these problems.

My nvidia module and nvidia-drivers are currently both pointing to the same version, 460.67.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54266
Location: 56N 3W

PostPosted: Thu Apr 29, 2021 8:21 am    Post subject: Reply with quote

dartleader,

Your problem stems from the kernel you build and the kernel you run need not be the same kernel.
On the build side, external kernel modules like nvidia-drivers follow the /usr/src/linux symlink to know which kernel to build against.

That also installs a matching Xorg driver.

When you update your kernel, rebuild nvidia-driver and get an updated nvidia-driver, the Xorg part will no longer work with the old kernel.
If you reboot into the old kernel, you get the old kernel module and the updated Xorg module, which generates the symptoms you describe.

There is another, more subtle way to break it too. You only update nvidia-drivers against your running kernel. Then you start the GUI.
The new kernel module is on disk, the new Xorg driver gets loaded but it fails as the old nvdia kernel module is still in RAM.
You need to modprobe -r the old nvidia drivers, so the kernel will load the new kernel part.

Every time you update the kernel, be sure to run
Code:
uname -a
to chech that you are running the kernel you think you are.
The Date/Time is the build Date/Time of the running kernel.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
kj184050
Apprentice
Apprentice


Joined: 29 Sep 2021
Posts: 164
Location: Devon

PostPosted: Mon Oct 09, 2023 7:41 am    Post subject: Reply with quote

Hi guys.
I don't want to start a new topic.

I just updated nvidia drivers this morning. I did
Code:
emerge --ask @module-rebuild
and after rebooting I got a very laggy system. Still, 120 Hz and VRR works fine


Code:
$ nvidia-smi
Failed to initialize NVML: Driver/library version mismatch
NVML library version: 535.113



Code:
$ nvidia-settings --version | grep version
nvidia-settings:  version 535.113.01
$ modinfo nvidia | grep version | head -n 1
version:        535.104.05
aps@gentoolnx ~/Prqs/X/CRISPR_mosaic $



Code:
[  949.719147] NVRM: API mismatch: the client has the version 535.113.01, but
               NVRM: this kernel module has the version 535.104.05.  Please
               NVRM: make sure that this kernel module and all NVIDIA driver
               NVRM: components have the same version.



So I did as usual:

  • genkernel all
  • make modules_prepare
  • emerge --ask @module-rebuild

    nothing

    so I tried

    Code:
    $:  modprobe --remove-dependencies nvidia
    $:  modprobe -r nvidia
    modprobe: FATAL: Module nvidia_drm is in use.
    modprobe: FATAL: Error running remove command for nvidia
    $:  modprobe --remove-dependencies nvidia_drm
    $:  modprobe -r nvidia
    modprobe: FATAL: Module nvidia_drm is in use.
    modprobe: FATAL: Error running remove command for nvidia


I solved this part by not running it on X, still, it didn't help after updating the module

to be able to use the system I temporarily masked the newest Nvidia driver version

is is safe to force it?
I have no idea what to try next.
Can you please help?
Back to top
View user's profile Send private message
fedeliallalinea
Administrator
Administrator


Joined: 08 Mar 2003
Posts: 30923
Location: here

PostPosted: Mon Oct 09, 2023 1:38 pm    Post subject: Reply with quote

What output return eselect kernel list and uname -a commands?
_________________
Questions are guaranteed in life; Answers aren't.
Back to top
View user's profile Send private message
kj184050
Apprentice
Apprentice


Joined: 29 Sep 2021
Posts: 164
Location: Devon

PostPosted: Mon Oct 09, 2023 1:54 pm    Post subject: Reply with quote

Code:
$: eselect kernel list
Available kernel symlink targets:
  [1]   linux-6.1.53-gentoo-r1 *


Code:
$: uname -a
Linux gentoolnx 6.1.53-gentoo-x86_64 #1 SMP PREEMPT_DYNAMIC Mon Sep 18 10:21:59 BST 2023 x86_64 AMD Ryzen 9 5950X 16-Core Processor AuthenticAMD GNU/Linux


It looks like Genkernel didn't overwrite it today.
Kernels are the same, are they?
Back to top
View user's profile Send private message
fedeliallalinea
Administrator
Administrator


Joined: 08 Mar 2003
Posts: 30923
Location: here

PostPosted: Mon Oct 09, 2023 2:33 pm    Post subject: Reply with quote

The kernel linux-6.1.53-gentoo-r1 is not the same as 6.1.53-gentoo-x86_64.
The genkernel should name it 6.1.53-gentoo-r1-x86_64.
With command cat /proc/version you can see with gcc version you compiled the kernel.
_________________
Questions are guaranteed in life; Answers aren't.
Back to top
View user's profile Send private message
kj184050
Apprentice
Apprentice


Joined: 29 Sep 2021
Posts: 164
Location: Devon

PostPosted: Mon Oct 09, 2023 3:16 pm    Post subject: Reply with quote

Code:
$: ls -l /boot/ | grep vmlinuz
(...)
-rw-r--r-- 1 root root 10493728 Oct  9 08:30 vmlinuz-6.1.53-gentoo-r1-x86_64
-rw-r--r-- 1 root root 10493728 Oct  9 07:38 vmlinuz-6.1.53-gentoo-r1-x86_64.old
-rw-r--r-- 1 root root 10487744 Sep 18 10:22 vmlinuz-6.1.53-gentoo-x86_64


what a mess...
What would be the best way to approach it?
Back to top
View user's profile Send private message
fedeliallalinea
Administrator
Administrator


Joined: 08 Mar 2003
Posts: 30923
Location: here

PostPosted: Mon Oct 09, 2023 3:57 pm    Post subject: Reply with quote

Probably is missing the entry for boot new kernel in your boot loader.
_________________
Questions are guaranteed in life; Answers aren't.
Back to top
View user's profile Send private message
kj184050
Apprentice
Apprentice


Joined: 29 Sep 2021
Posts: 164
Location: Devon

PostPosted: Tue Oct 10, 2023 7:31 am    Post subject: Reply with quote

I'll just change it to the new kernel in grub.
But something confused Genkernel. The name is very similar.
Back to top
View user's profile Send private message
fedeliallalinea
Administrator
Administrator


Joined: 08 Mar 2003
Posts: 30923
Location: here

PostPosted: Tue Oct 10, 2023 7:36 am    Post subject: Reply with quote

kj184050 wrote:
The name is very similar.

Yes kernel version isn't changed but is applied a patch for a NFS issue, so the ebuild name simply added -r1.
Code:
commit 963140f26ae955299892109cf2ed54dc6de082c5
Author: Sam James <sam@gentoo.org>
Date:   Tue Sep 19 00:18:44 2023 +0100

    sys-kernel/gentoo-sources: stabilize 6.1.53-r1
   
    Pick up the NFS/SELinux fix.
   
    Bug: https://bugs.gentoo.org/914204
    Signed-off-by: Sam James <sam@gentoo.org>

_________________
Questions are guaranteed in life; Answers aren't.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Multimedia All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum