View previous topic :: View next topic |
Author |
Message |
Zucca Moderator


Joined: 14 Jun 2007 Posts: 4046 Location: Rasi, Finland
|
Posted: Sat Apr 12, 2025 1:19 pm Post subject: OpenRC: set cpu affinity or disable hyperthreading? [SOLVED] |
|
|
I'll write this briefly.
I started playing around with LLMs by using ollama.
Shortly after I noticed that ollama used exact half the available CPU threads. But there's a reason: It tries to avoid hyperthreading, which should make processing LLMs faster.
It looks like the code just reduces the number of threads by the amount it finds thread siblings.
I run ollama as service using the provided OpenRC init script. I wonder if I can set cpu affinity (choose exact cpu threads which would execute on different core) via /etc/conf.d/ollama? Or is it simply possible to outright disable hyperthreading of a process via the config?
Because when looking at process monitor it looks like the loaded threads are now chosen randomly (by kernel). _________________ ..: Zucca :..
My gentoo installs: | init=/sbin/openrc-init
-systemd -logind -elogind seatd |
Quote: | I am NaN! I am a man! |
Last edited by Zucca on Sat Apr 12, 2025 4:12 pm; edited 1 time in total |
|
Back to top |
|
 |
pietinger Moderator

Joined: 17 Oct 2006 Posts: 5642 Location: Bavaria
|
Posted: Sat Apr 12, 2025 1:37 pm Post subject: Re: OpenRC - set cpu affinity or disable hyperthreading? |
|
|
Zucca wrote: | Because when looking at process monitor it looks like the loaded threads are now chosen randomly (by kernel). |
AFAIK the kernel prefers the physical cores of your cpu when distributing the threads. However, since it does not matter whether a thread runs on the logical 1st CPU or the 2nd CPU of the actual 1st physical CPU, it can look random. I would like to illustrate this with this example:
PHY 1 L1: run
PHY 1 L2: idle
PHY 2 L1: idle
PHY 2 L2: run
has the the same result as:
PHY 1 L1: idle
PHY 1 L2: run
PHY 2 L1: idle
PHY 2 L2: run
or:
PHY 1 L1: idle
PHY 1 L2: run
PHY 2 L1: run
PHY 2 L2: idle
or ... _________________ https://wiki.gentoo.org/wiki/User:Pietinger |
|
Back to top |
|
 |
szatox Advocate

Joined: 27 Aug 2013 Posts: 3624
|
Posted: Sat Apr 12, 2025 2:52 pm Post subject: |
|
|
You can disable HT in bios. I don't know of any way to do that in runtime, perhaps uefi vars could do that, but... Unless you're going to keep flipping that flag on an off, it will be faster to just reboot than research it. _________________ Make Computing Fun Again |
|
Back to top |
|
 |
Zucca Moderator


Joined: 14 Jun 2007 Posts: 4046 Location: Rasi, Finland
|
Posted: Sat Apr 12, 2025 4:11 pm Post subject: [solved] |
|
|
pietinger wrote: | AFAIK the kernel prefers the physical cores of your cpu when distributing the threads. However, since it does not matter whether a thread runs on the logical 1st CPU or the 2nd CPU of the actual 1st physical CPU, it can look random. | Thanks. I think you just confirmed what I was guessing before. :)
szatox wrote: | I don't know of any way to do that in runtime |
Oh there is /sys/devices/system/cpu/smt/active. ;)
But that does it for the entire system. I was looking for per process.
Anyways. I think this was [SOLVED] even before I asked the question. ;) _________________ ..: Zucca :..
My gentoo installs: | init=/sbin/openrc-init
-systemd -logind -elogind seatd |
Quote: | I am NaN! I am a man! |
|
|
Back to top |
|
 |
|