Do you USE="threads"?

grant123 · Veteran Joined: 23 Mar 2005 Posts: 1080

Is there any reason not to have threads in your make.conf USE variable?

Logicien · Posted: Wed Dec 26, 2012 10:40 pm Post subject:

I got that flag per package that emerge ask for in /etc/portage/package.use. I do not set any global use flag even if it apper often in package.use. My idea is not to change the defaults use flags of the ebuilds as much as possible. I add or remove some only per package when I have no other choice. I think it can help to advoid some problems that emerge can encounter.
_________________
Paul

dol-sen · Posted: Thu Dec 27, 2012 12:30 am Post subject:

No, there are many pkgs which rely on threads being enabled, so it is default to prevent many bugs and problems.

Not to say that there may be need for a pkg now and then to be compiled without it.
_________________
Brian
Porthole, the Portage GUI frontend irc@freenode: #gentoo-guis, #porthole, Blog
layman, gentoolkit, CoreBuilder, esearch...

grant123 · Veteran Joined: 23 Mar 2005 Posts: 1080

Actually, threads is disabled by default on my profile (default/linux/amd64/10.0/desktop).

dol-sen · Posted: Fri Dec 28, 2012 12:09 am Post subject:

ah, I confused it with python having threads defaulted on. It is python without threads that can cause a lot of problems, so is enabled by default.
_________________
Brian
Porthole, the Portage GUI frontend irc@freenode: #gentoo-guis, #porthole, Blog
layman, gentoolkit, CoreBuilder, esearch...

grant123 · Veteran Joined: 23 Mar 2005 Posts: 1080

Maybe threads is disabled system-wide in case you don't have a multi-core CPU, and is meant to be enabled system-wide if you do?

jdhore · Retired Dev Joined: 13 Apr 2007 Posts: 106

I think part of why USE="threads" is not enabled by default is if done incorrectly, adding threads can hurt performance.

Let me explain. Threads can hurt performance in 2 ways:
1. Locks. If you're using threads, you need to take/release locks in a lot of places in your code and if you're not EXTREMELY careful and you spawn a lot of threads, you'll suffer lock wait that could significantly hurt performance.
2. Context Switching. Your CPU can only do so many things at once. If a program spawns 10 threads and you have a quad-core CPU with other stuff running...Performance isn't going to be great because the CPU is getting bogged down with context switching instead of actual processing.

This problem is particularly visible in, for example, Audacious 2.4 versus Audacious 3.1 . Audacious got a new main developer who thought threads are good, added a crapton of them and SIGNIFICANTLY hurt performance (for example, try adding 25,000 files to a playlist in audacious 2.4 and then try adding the same files to a playlist in Audacious 3.1...Audacious 3.1 will be SIGNIFICANTLY slower. (note: This problem has been mostly resolved in audacious 3.2/3.3 since people smacked some sense into the developer, but still.))

Generally, if you're thinking of using threads, there are 3 alternatives:

1. Coroutines. There's implementations for most languages. Basically, fairly simple multiprocessing by letting tasks do other things while doing stuff in the background (Like letting a connection go past OSI level 4 (i think) while checking it against DNS Blacklists).
2. OpenMP. It's basically threads that can't really be screwed up since you're using their API for all that stuff (which is well-tested and avoids many of these problems), not pthreads or whatnot.
3. Full Spawned processes. This one is not as lightweight as threads, but it serves a similar purpose, but with far more advantages. You only give the spawned process the data it needs (like a SSLd instead of handling SSL in your main process) so there's less locking to worry about and you pass off security issues and bugs SOMEWHAT. You should still make fixing them a priority, but if there's a SSL bug (in the previous example), it'll just take down your SSLd/SSL support, not your whole process.

grant123 · Veteran Joined: 23 Mar 2005 Posts: 1080

Is it generally better to leave threads disabled then?

jdhore · Retired Dev Joined: 13 Apr 2007 Posts: 106

grant123 · Veteran Joined: 23 Mar 2005 Posts: 1080

OK, I'll take USE=threads back out of make.conf. Is USE=threads a good choice for ffmpeg?

jdhore · Retired Dev Joined: 13 Apr 2007 Posts: 106

grant123 · Veteran Joined: 23 Mar 2005 Posts: 1080

My laptop mostly watches videos, but I have a desktop running motion so I'll leave threads enabled on ffmpeg there.

Massimo B. · Posted: Thu Jul 03, 2014 10:34 am Post subject:

As said here, privoxy is another candidate for definitely enabling USE="threads".
_________________
HP ZBook Power 15.6" G8 i7-11800H|HP EliteDesk 800G1 i7-4790|HP Compaq Pro 6300 i7-3770