Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
OpenRC: set cpu affinity or disable hyperthreading? [SOLVED]
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
Zucca
Moderator
Moderator


Joined: 14 Jun 2007
Posts: 4046
Location: Rasi, Finland

PostPosted: Sat Apr 12, 2025 1:19 pm    Post subject: OpenRC: set cpu affinity or disable hyperthreading? [SOLVED] Reply with quote

I'll write this briefly.
I started playing around with LLMs by using ollama.

Shortly after I noticed that ollama used exact half the available CPU threads. But there's a reason: It tries to avoid hyperthreading, which should make processing LLMs faster.
It looks like the code just reduces the number of threads by the amount it finds thread siblings.
I run ollama as service using the provided OpenRC init script. I wonder if I can set cpu affinity (choose exact cpu threads which would execute on different core) via /etc/conf.d/ollama? Or is it simply possible to outright disable hyperthreading of a process via the config?
Because when looking at process monitor it looks like the loaded threads are now chosen randomly (by kernel).
_________________
..: Zucca :..

My gentoo installs:
init=/sbin/openrc-init
-systemd -logind -elogind seatd

Quote:
I am NaN! I am a man!


Last edited by Zucca on Sat Apr 12, 2025 4:12 pm; edited 1 time in total
Back to top
View user's profile Send private message
pietinger
Moderator
Moderator


Joined: 17 Oct 2006
Posts: 5642
Location: Bavaria

PostPosted: Sat Apr 12, 2025 1:37 pm    Post subject: Re: OpenRC - set cpu affinity or disable hyperthreading? Reply with quote

Zucca wrote:
Because when looking at process monitor it looks like the loaded threads are now chosen randomly (by kernel).

AFAIK the kernel prefers the physical cores of your cpu when distributing the threads. However, since it does not matter whether a thread runs on the logical 1st CPU or the 2nd CPU of the actual 1st physical CPU, it can look random. I would like to illustrate this with this example:
PHY 1 L1: run
PHY 1 L2: idle
PHY 2 L1: idle
PHY 2 L2: run
has the the same result as:
PHY 1 L1: idle
PHY 1 L2: run
PHY 2 L1: idle
PHY 2 L2: run
or:
PHY 1 L1: idle
PHY 1 L2: run
PHY 2 L1: run
PHY 2 L2: idle
or ...
_________________
https://wiki.gentoo.org/wiki/User:Pietinger
Back to top
View user's profile Send private message
szatox
Advocate
Advocate


Joined: 27 Aug 2013
Posts: 3624

PostPosted: Sat Apr 12, 2025 2:52 pm    Post subject: Reply with quote

You can disable HT in bios. I don't know of any way to do that in runtime, perhaps uefi vars could do that, but... Unless you're going to keep flipping that flag on an off, it will be faster to just reboot than research it.
_________________
Make Computing Fun Again
Back to top
View user's profile Send private message
Zucca
Moderator
Moderator


Joined: 14 Jun 2007
Posts: 4046
Location: Rasi, Finland

PostPosted: Sat Apr 12, 2025 4:11 pm    Post subject: [solved] Reply with quote

pietinger wrote:
AFAIK the kernel prefers the physical cores of your cpu when distributing the threads. However, since it does not matter whether a thread runs on the logical 1st CPU or the 2nd CPU of the actual 1st physical CPU, it can look random.
Thanks. I think you just confirmed what I was guessing before. :)

szatox wrote:
I don't know of any way to do that in runtime

Oh there is /sys/devices/system/cpu/smt/active. ;)
But that does it for the entire system. I was looking for per process.

Anyways. I think this was [SOLVED] even before I asked the question. ;)
_________________
..: Zucca :..

My gentoo installs:
init=/sbin/openrc-init
-systemd -logind -elogind seatd

Quote:
I am NaN! I am a man!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum