Page 1 of 1

[SOLVED] Crash Bandicoot N. Sane Trilogy via Docker/Flatpak

Posted: Sun Mar 03, 2019 12:08 am
by ShinyDoofy
Hi,

for quite some time, I've been successfully running Steam in its own docker container with Ubuntu 18.04 as I don't fully trust it or its games and would like to keep my main system straight 64bit without the 32bit multilib mess.

CSGO, DOOM 2016, Wolfenstein 2, and Rise of the Tomb Raider all run perfectly with FPS well above 100 (maxed out by my 120Hz displays).

Crash Bandicoot, however, has given me nothing but trouble. According to its protondb page, it's supposed to run near flawlessly for about every current system. Whatever settings I change or tweaks I perform, nothing gets me good performance. The framerate is really unstable and generally too low for comfortable play (~10fps), but it spikes up to 60fps for half a second every now and then. Audio, on the other hand, has no issues whatsoever.

Things I've tried so far:
  • Upgrade dockered Ubuntu 18.04 to 18.10
  • Turn shader caching in Steam on/off
  • Use Proton 3.16 instead of 3.16 Beta (3.16-7)
  • Disable DXVK (falling back to wined3d)
  • Play around with ESync
  • set PROTON_FORCE_LARGE_ADDRESS_AWARE via Proton's user_settings.py
  • disable DXVK's nvapi hack to not fake an AMD graphics card for the game
  • start the game with -4kb or -4kf
  • Set the CPU governor to performance instead of ondemand
  • Move the game to an SSD
  • Set the game to fullscreen, windowed, windows fullscreen, vsync on/off, motion blur on/off, low/mid/high/ultra presets, about every setting in the game, switch resolutions up and down (640x480 -> 1920x1080)
  • turn KDE's compositor on/off
  • turn GSync on/off, set my displays to anything between 60/100/120/144/180 Hz
  • turn forced composition pipeline in Nvidia driver on/off
And random combinations thereof, keeping mostly to the defaults as nothing but ESync is mentioned in the protondb entry. None of those changes seem to have a meaningful impact. Meanwhile, my GPU stays below the fan's starting temperature and is almost idling. The other games mentioned above at least make the fans rev up and cause substantial GPU load as per the Nvidia X server settings tool.

My system:
  • AMD Threadripper 2950X
  • Nvidia GeForce RTX 2080 Ti with proprietary drivers (around 413 when I bought the game, now 418.43; USE=X acpi driver gtk3 kms tools uvm -compat -multilib -static-libs -wayland)
  • 32GiB RAM @3200MHz XMP
  • Vanilla Kernel 4.20.12 (originally was 4.20.0) (config)
  • fully updated ~amd64 system with OpenRC
Fearing docker might be the culprit, I tried flatpak via an overlay, installed Steam via it and copied the game files over (file verification inside Steam succeeded). The same issue persists there.

Looking into the running processes, the game exe consumes ~150% CPU and wineserver.exe does ~60%. Via strace, the game exe does virtually nothing and wineserver is doing tons of readv/writev (for the shaders to be compiled, I guess).

I'm at a loss. What else could I be looking into?

/edit: Updated to Proton 4.2 and everything is running as it should now, with 120fps and without hiccups. Going back to Proton 3.16 shows all the same symptoms still, but 4.2 reproducibly fixes it for me with an untouched Proton installation (no config changes necessary whatsoever).

Posted: Sun Mar 03, 2019 9:40 am
by Juippisi
Well, inspired by this post I decided to launch my Crash Bandicoot straight from steam using proton. It indeed lags like h**l. It used to run smoothly.

I tried with different proton versions, but it didn't help. So I'm guessing some game update or system update has caused this, but no idea what it could specifically be.

Shame :\

EDIT: https://github.com/ValveSoftware/Proton/issues/731 advised to downgrade mesa, so I did downgrade to latest stable, but it didn't solve this for me either.

Posted: Sun Mar 03, 2019 12:05 pm
by ShinyDoofy
Thank you for also looking into this. Now I know I'm not alone on this.

I assume you're running steam natively? That would likely exclude the container being the culprit.

Would you mind posting your specs/setup? Currently, my guess would be on the kernel because I don't see any positive reports on Kernel 4.20.x and I'd be willing to try a few rounds of git bisect to see whether that would change things...

Posted: Sun Mar 03, 2019 1:39 pm
by Juippisi
I'm sorry to tell you, but I got it working :P

Yes I run steam natively from steam-overlay. I booted to an older kernel which didn't have muqss enabled, so my culprit seems to be CPU Scheduler that I switched some month ago. Before that I tried downgrading mesa and openrc (because I remembered they updated a while ago), but those didn't help.

i7-2700k, RX570, amdgpu driver.

Posted: Sun Mar 03, 2019 1:45 pm
by ShinyDoofy
Aha, CPU scheduler. I'll try fiddling with those, thanks once more! Just booted into 4.18.8 and that didn't change a thing. Will go back to 4.20.12 and report back for anybody else who might be affected by this...

Posted: Sun Mar 03, 2019 5:13 pm
by Hu
Generally, in-kernel schedulers should not behave that poorly. If you can isolate why MuQSS makes the game behave badly, it may be worth a bug report.

Posted: Sun Mar 03, 2019 7:54 pm
by ShinyDoofy
I've toyed around with some kernel settings: switched from NO_HZ_FULL to NO_HZ_IDLE, disabled X86_AMD_FREQ_SENSITIVITY and NUMA, set CPU affinity with tasksel for the game exe and wineserver - no dice, still just as inconsistent and bad as before.

To try another title that would need DXVK, I installed the Shadow of the Tomb Raider demo. Heavy loading stutters aside, it runs perfectly smooth at high(est) settings with good fps. So I doubt it's DXVK, but rather a game-specific issue. Ah, well.

Posted: Mon Mar 04, 2019 9:11 am
by Juippisi
Hu wrote:Generally, in-kernel schedulers should not behave that poorly. If you can isolate why MuQSS makes the game behave badly, it may be worth a bug report.
I've never had a good run with muqss, or BFS before that. Seems like I last had it enabled with kernel 4.8 for a short time, and with 4.2 before that. I have overclocked my CPU a long time ago and using BFS/muqss used to crash my computer in heavy loads, at least with 4.20 it works with daily usage, but then that Crash thing happened... What's weird is, it didn't show up in any other games, although I don't play much nowadays.

I compiled 4.20.12 without it, and Crash works beatifully, so it's not some kernel option but the CPU sched for me.

Posted: Tue Mar 05, 2019 4:00 am
by Ant P.
The combination of BFS and overclocking was probably overheating the CPU. CFS wouldn't exhibit the same issue because it fails to use the full CPU[1]

[1]: The Linux Scheduler: a Decade of Wasted Cores, Jean-Pierre Lozi, Baptiste Lepers, Justin Funston, Fabien Gaud, Vivien Quéma, and Alexandra Fedorova. To appear in Proceedings of the Eleventh European Conference on Computer Systems (EuroSys '16), London, United Kingdom, 2016.

Posted: Tue Mar 05, 2019 10:03 pm
by ShinyDoofy
Well, I "fixed" it and the game now runs beautifully.

I had to turn off SMT in the BIOS/UEFI. So I get to choose between half my CPU and this game or my entire CPU and anything but this game. This is bonkers.

Thank you for the CPU hint, Juippisi!

Posted: Wed Mar 06, 2019 12:52 am
by Ant P.
That's weird… have you tried pinning the game to one core per CCX? Might work without rebooting the entire system that way.

Posted: Wed Mar 06, 2019 6:00 am
by ShinyDoofy
I tried to dig into this a bit more, yeah. To be honest, these CPU details are a bit beyond me. Playing with taskset and the two PIDs for the game exe and wineserver didn't solve this before (up to 20fps now, "wow"). Anyway, I re-enabled SMT in UEFI, set the memory config(?) to "interleaved" and activated NUMA in the kernel config again, now I get this:

Code: Select all

# numactl -H
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5 6 7 16 17 18 19 20 21 22 23
node 0 size: 15966 MB
node 0 free: 11099 MB
node 1 cpus: 8 9 10 11 12 13 14 15 24 25 26 27 28 29 30 31
node 1 size: 16121 MB
node 1 free: 12208 MB
node distances:
node   0   1 
  0:  10  16 
  1:  16  10
Aha! Now I know the CPU ids. This didn't show before when I had that UEFI memory setting on "Auto". Anyway, I ran the following:

Code: Select all

for c in $(numactl -H | grep -oP '(?<=node 1 cpus: ).*'); do echo 0 > /sys/devices/system/cpu/cpu${c}/online; done
Wouldn't you know it, I now at least get consistent performance. Still not quite enjoyable (30-40fps), but much more consistent than before (anything between 7fps and 15fps).

My next idea is to switch this memory config thingy back to "Auto" and manually deactivating these exact cpus from node 1 for my next reboot when I get the time for it again.

This epiphany would have been a single click under Windows to activate "Game Mode" with AMD's Ryzen Master, it seems. Sadly, this tool is not supported/available for Linux.

Main sources for this:
https://community.amd.com/community/gam ... for-gaming
https://www.reddit.com/r/Amd/comments/6 ... r/dlg9i0i/

(Kernel 5.0 because why not? Yes, I know changing too many variables at once is a very stupid idea for trying to pinpoint an exact source of this phenomenon, but performance was consistently horrible across the board no matter what I did before, so...)

Posted: Wed Mar 06, 2019 6:32 am
by Juippisi
Wow, glad you solved it. Something like touching BIOS/UEFI would have never crossed my mind :)

Nice verbose info there. This thread might help someone someday :)

Posted: Wed Mar 06, 2019 7:15 pm
by Ant P.
I think having NUMA enabled is better with this CPU (because the inter-package latency is pretty bad, and the kernel won't know to avoid it otherwise).

Avoiding SMT does make some sense with games, because each pair of cores shares one FPU/vector unit. Not entirely sure what the topology of these things are under linux, you'd have to check with lscpu.

Posted: Wed Mar 06, 2019 9:27 pm
by ShinyDoofy
Memory interleaving on Auto or Channel, disabling CPUs, starting the docker container with cpusets 0-7 or 0-7,16-23... none of it makes a difference.

Disabling SMT still was the "best" option and I'm out of ideas again, bummer.

Posted: Wed Mar 27, 2019 5:59 am
by ShinyDoofy
Proton 4.2-1 fixes everything. Thank you everyone once more - I'll look into enabling NUMA again at some point for said memory latency improvement, but I don't suspect that setting to rebreak the game.