Steam for Linux-proton fsync testing announcement from valve

Message

Linubie · Post by **Linubie** » Tue Jul 30, 2019 11:35 pm

https://steamcommunity.com/app/221410/d ... 006906163/

It seems we have to patch the kernel for it but I have no idea how to get the full patchset for it.
From what understand there are only kernels for ubuntu and arch right now.

If someone has an idea how to patch the kernel, to let us participate in the testing please feel free to share your knowlegde.

Thank you!

netfab · Post by **netfab** » Wed Aug 07, 2019 10:22 am

Linubie wrote: If someone has an idea how to patch the kernel, to let us participate in the testing please feel free to share your knowlegde.

Not tested. Two patchs here :
https://lkml.org/lkml/2019/7/30/1398
https://lkml.org/lkml/2019/7/30/1399
According to here should apply to latest 5.3 branch.
If you test it, please update this thread with useful informations (kernel version, tested game, ...). Thanks.

Ant P. · Post by **Ant P.** » Wed Aug 07, 2019 2:30 pm

What does this do, exactly? It doesn't seem to be related to fsync(2) at all.

netfab · Post by **netfab** » Wed Aug 07, 2019 4:32 pm

Ant P. wrote:What does this do, exactly? It doesn't seem to be related to fsync(2) at all.

From here :

As such, we're proposing changes to the Linux kernel[lkml.org] to extend the futex() system call to expose
what we think is the needed extra bit of core functionality needed to support optimal thread pool synchronization.
Proton 4.11 includes the fsync patchset, which will leverage this new Linux kernel functionality to replace esync
when supported. For more technical details, refer to the link to the proposed kernel changes above.

donmartio · Post by **donmartio** » Sun Aug 11, 2019 10:07 am

I'm testing this already with witcher3. Posted here what i've done to patch the kernel.

The results for witcher are good. Runs smoother on my Ryzen 5 1600 with nvidia GTX 970.

The Main Man · Post by **The Main Man** » Sun Aug 11, 2019 11:45 pm

There are also patches for glibc

donmartio · Post by **donmartio** » Wed Aug 14, 2019 7:04 pm

The glibc patch is just a proof of concept proposal, as far as i understood.

adcdam · Post by **adcdam** » Sat Aug 17, 2019 9:33 pm

i followed your guide but the patches has to go to /etc/portage/patches/sys-kernel/gentoo-sources-what ever the kernel and you put the patch in /etc/portage/patches/gentoo-sources-5.2.5
if the patch is well applied it say user patches applied.

donmartio · Post by **donmartio** » Tue Aug 20, 2019 7:00 pm

Uhhh yea you are right. I did not copy the whole stuff but have written it down.
Fixed that. Thanks.

Linubie · Post by **Linubie** » Wed Sep 18, 2019 8:21 pm

There is also a compiled version of proton available.

https://github.com/GloriousEggroll/prot ... m/releases

You only need to unpack your downloaded version into

Code: Select all

~/.steam/steam/compatibilitytools.d/

and choose the version you want to test in steam setting.

hedmo · Post by **hedmo** » Sat Dec 28, 2019 8:40 am

hi all

i am on the fsync train to.you only need one patch now to get it to work. i did not have the kernel option in my kernel so i copied
the kernel config from the clone and now :

Code: Select all

fsync: up and running

can someone tell me what kernel option i need tog get this working on my custom config?

Linubie wrote:There is also a compiled version of proton available.

https://github.com/GloriousEggroll/prot ... m/releases

You only need to unpack your downloaded version into
Code: Select all
~/.steam/steam/compatibilitytools.d/
and choose the version you want to test in steam setting.

Linubie
i am running a git version of proton from my local overlay :

https://github.com/hedmo/hedmos-overlay

and the fsync is running there to so it is integrated in proton.the

https://github.com/GloriousEggroll/prot ... m/releases

does not work for me and i cannot edit that build

regards hedmo

Post by **Ionen** » Sat Dec 28, 2019 9:02 am

hedmo wrote:can someone tell me what kernel option i need tog get this working on my custom config?

There's no kernel option for fsync (not right now anyway), you just need the patch and you're done (it makes no kconfig changes). I just used the patch given by archlinux (futex-wait-multiple-5.2.1.patch) and it still works on 5.4.6 alongside my custom configs.

Personally using normal wine-staging (from gentoo) + rebased tkg staging fsync patch + a few other handpicked patches, and it's working great (don't want to use proton, but I am so done with esync). Using wine-tkg/eggroll/proton directly is a simpler solution though, I need to sort out the patches myself with new versions.

hedmo · Post by **hedmo** » Sat Dec 28, 2019 9:41 am

Ionen wrote:
hedmo wrote:can someone tell me what kernel option i need tog get this working on my custom config?
There's no kernel option for fsync (not right now anyway), you just need the patch and you're done (it makes no kconfig changes). I just used the patch given by archlinux (futex-wait-multiple-5.2.1.patch) and it still works on 5.4.6.

Personally using normal wine-staging (from gentoo) + rebased tkg staging fsync patch + a few other handpicked patches, and it's working great (don't want to bother with proton, and I am so done with esync). Using wine-tkg/eggroll/proton directly is a simpler solution though, I need to sort out the patches myself with new versions.

Ionen

i am going to try my custom config again.what is :

Code: Select all

wine-tkg/eggroll/proton

i like to know if there is other proton ebuilds i can use and patch.i need to patch the wine version on proton to get BOTW to run with vulkan.

edit: found it (wine-tkg/eggroll/proton) but how do you patch it?

Post by **Ionen** » Sat Dec 28, 2019 10:05 am

If you want a proton overlay, this might work: https://github.com/pchome/proton-gentoo-overlay, not that I've used it (but pchome typically knows what he's doing assuming it's not outdated). Edit: then again, looking at your overlay it looks like a fork of that so I guess that's not helping (are you not able add patches your own overlay? ...or just set normal user patches).

As I mentioned I only take patches from those bigger projects (like fsync support) and use gentoo's native wine ebuild, so I can't really help you with proton.

hedmo · Post by **hedmo** » Sat Dec 28, 2019 11:16 am

Ionen

https://github.com/pchome/proton-gentoo-overlay is dead and i have cloned and updated it.i have my patches on my overlay
and it is working quite good for now .

regards

Mappy · Post by **Mappy** » Sat Jan 25, 2020 2:59 am

Ionen wrote:
hedmo wrote:can someone tell me what kernel option i need tog get this working on my custom config?
There's no kernel option for fsync (not right now anyway), you just need the patch and you're done (it makes no kconfig changes). I just used the patch given by archlinux (futex-wait-multiple-5.2.1.patch) and it still works on 5.4.6 alongside my custom configs.

Personally using normal wine-staging (from gentoo) + rebased tkg staging fsync patch + a few other handpicked patches, and it's working great (don't want to use proton, but I am so done with esync). Using wine-tkg/eggroll/proton directly is a simpler solution though, I need to sort out the patches myself with new versions.

Do you have a rebased patch that would work for wine-staging 4.21 or 5.0? I tried the 4.13 one that's available here but it seems like it isn't compatible:
https://github.com/doitsujin/dxvk/issues/1163

Post by **Ionen** » Sat Jan 25, 2020 5:55 am

Mappy wrote:Do you have a rebased patch that would work for wine-staging 4.21 or 5.0?

I still have the one I used for wine-staging-4.21 that been working great (WINEFSYNC=1 to enable), probably not be a hard fix but it doesn't apply on 5.0 (was planning to look into that much later). Can always check tkg patches instead but the staging patch need to be rebased to work with gentoo's ebuild due to the way it applies staging patches, mine should apply 100% cleanly with wine-staging-4.21.ebuild as a user patch, unless want wine-vanilla.

On the kernel end, the archlinux patch still works on 5.4.13 (haven't built 14 yet but I'm sure it's fine).

Mappy · Post by **Mappy** » Sun Jan 26, 2020 1:05 am

Ionen wrote:
Mappy wrote:Do you have a rebased patch that would work for wine-staging 4.21 or 5.0?
I still have the one I used for wine-staging-4.21 that been working great (WINEFSYNC=1 to enable), probably not be a hard fix but it doesn't apply on 5.0 (was planning to look into that much later). Can always check tkg patches instead but the staging patch need to be rebased to work with gentoo's ebuild due to the way it applies staging patches, mine should apply 100% cleanly with wine-staging-4.21.ebuild as a user patch, unless want wine-vanilla.

Thank you, it works great. Do you have a guide I could read to learn how to rebase a patch myself? Or would you by any chance consider maintaining a repo for your patches? I would like to be able to use fsync with future updates, but if it's too much of a bother then don't worry about it.

Post by **Ionen** » Sun Feb 02, 2020 1:53 pm

Mappy wrote:Thank you, it works great. Do you have a guide I could read to learn how to rebase a patch myself? Or would you by any chance consider maintaining a repo for your patches? I would like to be able to use fsync with future updates, but if it's too much of a bother then don't worry about it.

It's nothing that need actual work. Just need a wine tree with staging patches applied with --no-autoconf then use fuzz (-F) for fsync patches (it could potentially produce wrong results but well, it works).

For example, assuming have in the same directory: wine-5.8.tar.xz extracted, wine-staging-5.8.tar.gz patchset extracted as well (both from /var/cache/distfiles), and fsync patches from Tk-Glitch, can run:

Code: Select all

WINEVER=5.8 # update as needed
./wine-staging-${WINEVER}/patches/patchinstall.sh DESTDIR=wine-${WINEVER} --no-autoconf --all &&
cp -a wine-${WINEVER} wine-fsync &&
patch -d wine-fsync -p1 -F 3 --no-backup-if-mismatch < fsync-staging.patch &&
patch -d wine-fsync -p1 -F 3 --no-backup-if-mismatch < fsync-staging-no_alloc_handle.patch &&
diff -Naur wine-${WINEVER} wine-fsync > wine-staging-fsync.patch

Resulting wine-staging-fsync.patch should be usable as a gentoo user patch with the main repos' ebuild.

If it doesn't work (or want older wine version), may need to use legacy patches from TKG, sometimes latest has been updated for the upcoming version and isn't for currently released wine. Alternatively, can also try eggroll's which is based on TKG's,

Update 2020-05-12: fixed up post a bit and added ready-to-use patch for wine-staging-5.8 due to a request
Update 2020-06-07: known broken with 5.10+ (both esync and fsync), so don't bother until someone makes working patches again
Update 2020-09-14: removed dead ready-to-use patch links, can still be remade using above instructions for 5.0/5.8, just use right versions from git.
Update 2020-10-19: not released yet but looking at commits wine-staging-5.20 is bringing back esync, and wine-tkg has new fsync-unix patches

TechwoIf · Post by **TechwoIf** » Mon Nov 02, 2020 6:53 am

Update 2020-10-19: not released yet but looking at commits wine-staging-5.20 is bringing back esync, and wine-tkg has new fsync-unix patches

Would love to get those patches. I use the zen-kernel and would like to do some testing. Every patch I found was either for an old version or custom wine-xxxx that incompatible with wine-staging.

Mappy · Post by **Mappy** » Fri Nov 06, 2020 4:26 am

TechwoIf wrote:
Update 2020-10-19: not released yet but looking at commits wine-staging-5.20 is bringing back esync, and wine-tkg has new fsync-unix patches
Would love to get those patches. I use the zen-kernel and would like to do some testing. Every patch I found was either for an old version or custom wine-xxxx that incompatible with wine-staging.

The fsync-unix-staging.patch from here works for me using Ionen's script. Just replace the fsync-staging.patch and fsync-staging-no_alloc_handle.patch lines with it.

TechwoIf · Post by **TechwoIf** » Sat Nov 14, 2020 7:54 pm

Mappy wrote:
The fsync-unix-staging.patch from here works for me using Ionen's script. Just replace the fsync-staging.patch and fsync-staging-no_alloc_handle.patch lines with it.

Thank you for the info. I downloaded fsync-unix-staging.patch to /etc/portage/patches/app-emulation/wine-staging-5.20 and emerge -1av wine-staging. Turns out fsync-staging-no_alloc_handle.patch is no longer needed.

Starting up wine with "export WINEFSYNC=1" gives no "fsync: up and running." I am running linux-zen and the output of "cat /proc/kallsyms | grep futex_wait_multiple" shows the kernel does have it.

Wine-staging 5.20(without the fsync patch) does have esync enable and shows it in the output. I get about 50% to 100% performance increase in Elder Scrolls Online. Was hoping to get fsync working and get a bit more on top of that.

EDIT/UPDATE: Turns out re-emergeing wine-staging set "eselect wine" to wine-vanilla. No wonder it was not working. The wrong wine was being used. Selected the right one and now its working. Will be doing some A/B testing with Elder Scrolls Online.

Post by **Ionen** » Sat Nov 14, 2020 8:35 pm

TechwoIf wrote:Wine-staging 5.20(without the fsync patch) does have esync enable and shows it in the output. I get about 50% to 100% performance increase in Elder Scrolls Online. Was hoping to get fsync working and get a bit more on top of that.

While using esync or fsync can give a noticeable boost on cpu-bound games, using fsync over esync is almost identical performance as far as I'm aware. But esync is more like a hack that use an absurd amount of file descriptors in unintended ways, so fsync would be a better long term solution (or at least if kernel gets support officially added).

If esync works, then can stick with that for less wine patching hassle (that's what I do now). I haven't tested fsync in a while myself, so I can't really help anymore but good luck to anyone wanting it.

lgael · Post by **lgael** » Fri Jan 08, 2021 6:21 pm

For information there are patches for wine / wine-staging made by Linux SC users :

wine-vanilla asof 7th Dec: https://pastebin.com/raw/xkkbg2JV
wine-staging 6.0-rc2 asof 10th Dec: https://pastebin.com/raw/b2ynUegW

I tried to applied the second on top of wine-staging by putting it in /etc/portage/patches/app-emulation/wine-staging/fsync.patch and emerging wine-staging-9999
However, it fails to apply on some files, because it seems that the ebuild/portage try to apply on top of wine files, not wine files fully patched with staging.
my patch is applied as the last one => ok but at this moment, for example include/wine/server_protocol.h is wine iso, not wine-staging iso (with esync code referred by fsync patch).

Code: Select all

* Applying wine-5.9-Revert-makedep-Install-also-generated-typelib-for-in.patch ...                                                                                                                                                                                                                                                                                                     [ ok ]
 * Applying fsync-wine-staging-6.0-rc2.patch ...
(...)
patching file include/wine/server_protocol.h
Hunk #1 succeeded at 5347 (offset -87 lines).
Hunk #2 FAILED at 5801.
Hunk #3 FAILED at 6087.
Hunk #4 FAILED at 6371.
(...)

If I build wine-staging-9999 without my fsync patch it does the following :

Code: Select all

* Applying wine-5.9-Revert-makedep-Install-also-generated-typelib-for-in.patch ...                                                                                                                                                                                                                                                                                                     [ ok ]
 * Running eautoreconf in '/var/tmp/portage/app-emulation/wine-staging-9999/work/wine-9999' ...
 * Running autoconf --force ...                                                                                                                                                                                                                                                                                                                                                         [ ok ]
 * Running autoheader ...                                                                                                                                                                                                                                                                                                                                                               [ ok ]
 * Running elibtoolize in: wine-9999/
 * server/protocol.def was patched; running tools/make_requests
include/wine/server_protocol.h updated
include/wine/server_protocol.h updated
server/trace.c updated
server/request.h updated
>>> Source prepared.

So it looks like some files are updated/patched after my user patch... How can I apply my patch just before >>> Source prepared ?
I modified the patch so it compiles but I found strange that I can't patch files on top of wine-staging...

For gentoo-sources-5.4x I applied an arch fsync patch 5.1. It works.

It is difficult to understand which patches to apply...

BobWya · Post by **BobWya** » Sat Jan 09, 2021 9:35 pm

@lgael

The patch is obviously not designed for Gentoo app-emulation/wine-staging - which runs some additional configuration steps after User patches have been applied (Wine auto-generates some files - using a Wine builtin tool utility).

I've rebased the patch for: app-emulation/wine-staging:6.0_rc5 - so at least it applies cleanly without any fuzzing:

wine-staging-6.0_rc5-fsync.patch

Disclaimer: I've not done much testing with this patched Wine Staging build!

Bob

Steam for Linux-proton fsync testing announcement from valve

Steam for Linux-proton fsync testing announcement from valve

Are you sure the patches were applied?