| View previous topic :: View next topic |
| Author |
Message |
Bloxlord
n00b
Joined: 06 Nov 2023
Posts: 7
|
 Posted: Mon Mar 18, 2024 10:34 pm Post subject: Help with Nvidia Prime and VFIO |
 |
|
Hello,
I have configured my system to run X on my integrated graphics and to load VFIO drivers to my Nvidia GPU on boot, for use in a virtual machine. I would like to be able to unload the VFIO drivers and load the Nvidia drivers to play games native on Linux, and have hit a wall trying to accomplish this.
I am able to run the following command to reattach the gpu and load the Nvidia drivers: | Code: | | sudo virsh nodedev-reattach pci_0000_01_00_0 && sudo rmmod vfio_pci vfio_pci_core vfio_iommu_type1 && sudo modprobe -i nvidia nvidia_drm |
After running that command nvidia-smi returns the following, confirming the gpu is recognized by the system: | Code: | +---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.154.05 Driver Version: 535.154.05 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 3090 Off | 00000000:01:00.0 Off | N/A |
| 0% 29C P0 92W / 420W | 0MiB / 24576MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| No running processes found |
+---------------------------------------------------------------------------------------+
|
However running xrandr --listproviders does not list the GPU as a provider so Nvidia Prime cannot offload to the GPU: | Code: | Providers: number : 1
Provider 0: id: 0x56 cap: 0xf, Source Output, Sink Output, Source Offload, Sink Offload crtcs: 4 outputs: 4 associated providers: 0 name:AMD Radeon Graphics @ pci:0000:16:00.0
|
Any help would be appreciated. Here is my xorg.conf in case it is needed: | Code: | Section "ServerFlags"
Option "AutoAddGPU" "false"
EndSection
Section "Device"
Identifier "iGPU"
Driver "amdgpu"
BusID "PCI:22:00:0"
Option "DRI" "3"
Option "PrimaryGPU" "yes"
EndSection
Section "Device"
Identifier "dGPU"
# Driver "vfio-pci"
BusID "PCI:1:00:0"
Option "DRI" "3"
# Option "PrimaryGPU" "no"
# Option "Ignore" "1"
EndSection
Section "Screen"
Identifier "Screen0"
Device "iGPU"
GPUDevice "dGPU"
EndSection
Section "ServerLayout"
Identifier "layout"
Screen 0 "Screen0"
Option "AllowNVIDIAGPUScreens"
EndSection
|
|
|
| Back to top |
|
 |
blu3bird
Retired Dev
Joined: 04 Oct 2003
Posts: 614
Location: Munich, Germany
|
| Posted: Tue Mar 19, 2024 1:16 pm Post subject: |
|
|
Xorg does not support hot plugging of gpus. You might be able to trick it by starting it with the nvidia driver loaded and then unloading it once it's running.
You can also try switching to Wayland as it has far better hot plugging support - not fully supported yet (at least Gnome/KDE), but good enough for DRI_PRIME. In this case you also want to start with the nvidia driver loaded and then switch the driver to vfio-pci using a libvirt hook.
_________________
Black Holes are created when God divides by zero! |
|
| Back to top |
|
|
Bloxlord
n00b
Joined: 06 Nov 2023
Posts: 7
|
| Posted: Sat Mar 23, 2024 7:53 pm Post subject: |
|
|
Thank you for your reply! I tried what you said starting with the nvidia driver loaded, but for some reason X was still using 4mb of the GPU's memory, despite running on the integrated graphics. I switched to Wayland with Wayfire and was able to achieve the desired result; although I did need to change the commands I was running.
Booting with the VFIO drivers loaded and running: | Code: | sudo virsh nodedev-reattach pci_0000_01_00_0 && sudo rmmod vfio_pci vfio_pci_core vfio_iommu_type1 && sudo modprobe -i nvidia
|
Followed by: | Code: | | sudo modprobe -i nvidia_modeset nvidia_uvm |
Brings the GPU back to my system and allows me to run games without restarting Wayland. Thanks again otherwise I would still be trying to get it to work on X. |
|
| Back to top |
|
|
|
Desktop Environments
