What benefits there are to NPTL?

Zarhan · l33t Joined: 27 Feb 2004 Posts: 997

Not sure whether this would belong here or the portage forum, but here goes...

What is this NPTL USE flag that I occasionally see? Native Posix Thread Library, yes. A replacement for linuxthreads I gather, but the question is: What good will it do from the user point of view? Improved performance? More efficient memory usage? Anything else?

Is it yet stable to take into production use? If I want to start using NPTL, are there any other steps besides setting the USE flag and re-emerging packages that have the nptl option? Do I need to compile any options into kernel (I'm using 2.6 series).

However, doing a quick check, emerge -epv world | grep nptl yields only

[ebuild N ] sys-libs/glibc-2.3.3.20040420-r1 -build -debug -debug -erandom +nls -nptl -pic 18 kB

In effect, the only package that actually needs to be recompiled is Glibc should I want to take advantage of NPTL?

GatoVolador · Posted: Fri Aug 13, 2004 6:14 pm Post subject:

Zarhan -

If I understand correctly, the main advantage of NPTL is that it uses lightweight threads. With linux threads every thread gets its own process, which, if you have a lot of threads, needlessly eats up a lot of resources in both CPU time and memory. NPTL does away with this, now multiple threads can exist inside one process, decreasing resource and cpu usage.

I (and I believe a good number of others I've read about here) have been using NPTL with 2.6.5-gentoo-r1 for a few months and have seen no problems whatsoever, so it feels stable. Thread management is part of glibc, so this is the only package you have to rebuild, but then other apps that use threads will benefit (these apps won't have the nptl use flag, that's why you didn't find anything).

For me the big winner was Java. I do a lot of heavy multithread stuff, and before I'd have one Java program where the JVM would have 100 thread-processes, now I have one JVM process that handles the 100 lightweight threads. Much less resources are used and Java apps now seem much more stable.

Regards,
Jay

Zarhan · l33t Joined: 27 Feb 2004 Posts: 997

butters · Posted: Fri Aug 13, 2004 6:45 pm Post subject:

NPTL requires a 2.6.x kernel such as gentoo-dev-sources or development-sources or mm-sources, and also requires the 2.6.x kernel headers. Get those by emerging linux26-headers (this should be detected as a dependency for glibc if nptl is in your USE flags). If your emerge of glibc still fails with linux26-headers installed, edit the file

/etc/make.profile/virtuals

and change two instances of "linux-headers" to "linux26-headers" in that file. Whether or not you need to do this depends on how recent your portage version is (you might want to do emerge portage before emerge glibc to avoid this).

You should consider NPTL a Good Thing. UNIX variants have had real POSIX threads for years, while linux has emulated threads as processes (adding significant overhead as mentioned previously). You don't need to be running servers in order to reap the benefits. Most web browsers and email clients (as well as p2p clients and many other network apps) make extensive use of multithreading. Firefox may run with 10 or more threads concurrently. NPTL can make a big difference is such cases, especially if your internet connection is on the slow side.

Similarly to XFree86-->Xorg, I moved from linux-threads to NPTL primarily on the basis that it is the next step in GNU/Linux evolution, so why not give the future a shot?
_________________
If tugboats were bigger, they'd be the ones getting tugged.

Zarhan · l33t Joined: 27 Feb 2004 Posts: 997

Zarhan · l33t Joined: 27 Feb 2004 Posts: 997

codergeek42 · Posted: Fri Aug 13, 2004 7:29 pm Post subject:

John5788 · Posted: Fri Aug 13, 2004 7:43 pm Post subject:

so all i need to do to enable NPTL is to add nptl to my use flags and re-emerge glibc along with packages i want to have nptl?
_________________
John5788

Voltago · Posted: Fri Aug 13, 2004 7:51 pm Post subject:

Zarhan · l33t Joined: 27 Feb 2004 Posts: 997

codergeek42 · Posted: Fri Aug 13, 2004 8:36 pm Post subject:

GatoVolador · Posted: Fri Aug 13, 2004 9:20 pm Post subject:

Actually if you're running a server that fires a lot of threads it would be useful. While of course nothing can make each thread run faster (except optimizing the code, if possible), with NPTL resources usage goes way down since you don't have to spawn a new process (costly) for each new thread.

I am using linux 2.4 headers with NPTL and haven't seen any problems. In the next couple of weeks I'll take a look at moving to linux26-headers.

And no, you don't have to rebuild anything except glibc unless you have statically linked apps, like codergeek said. Glibc abstracts away the details of thread management, so the apps don't care how it gets done, they just want it done. By the way whether you're using NPTL or linuxthreads, you're still using POSIX threads, since they are both just implementations of the POSIX thread specification.

Regards,
Jay

codergeek42 · Posted: Fri Aug 13, 2004 9:21 pm Post subject:

Zarhan · l33t Joined: 27 Feb 2004 Posts: 997

Glibc finished compiling. Restarted firefox, it started running with NPTL threads immediately (not creating n+10 processes when browsing). No need to reboot or anything else for that matter. Great!

finr · Tux's lil' helper Joined: 24 Feb 2004 Posts: 126

I just updated my system to an NPTL glibc but to my surprise gkrellm2 is showing more proc's than before (i always had like 95 runnig before, but since a reboot this number is usually > 110 but also greatly varying).. Is this normal? As I understood NPTL should decrease the number of processes since more threads can be run in a single process?
BTW, for firefox it seems NPTL has had it's effect, it's down from 7 processes to 2 directly after being started.. But still, my over-all process count seems to hae increased?

WaVeX · Posted: Sun Aug 15, 2004 9:10 pm Post subject:

ok I got a question Whats all this statically linked and dynamically lnked stuff.
How do I check what my gentoo is?
Which is better?

thechris · Veteran Joined: 12 Oct 2003 Posts: 1203

dynamic linking uses less disk space and is more secure. most apps in linux will be dynamically linked.

butters · Posted: Mon Aug 16, 2004 1:35 am Post subject:

Glad to see you've been successful. Sorry about the whole 2.6-headers thing, when I tried to emerge glibc with nptl, the configure script failed saying I needed 2.6.5-headers or newer. It could be that newer glibc builds no longer need 2.6-headers for NPTL.

IMHO gkrellm is and always was a resource hog. It's a shame that a system monitor should have so much affect on system performance. Try re-emerging gkrellm now that NPTL is in glibc, since gkrellm might be statically linked.
_________________
If tugboats were bigger, they'd be the ones getting tugged.

jdgill0 · Posted: Mon Aug 16, 2004 1:56 am Post subject:

How about NPTL and OpenMosix ... the pthreads will not migrate due to reasons discussed at openmosix.org ... Now I know OpenMosix is currently stable only for 2.4 kernels, however I read a post earlier today that said OpenMosix for 2.6 would be released in 5 days ... When OpenMosix gets released for the 2.6 kernels (whether that is 5 days or not), will the NPTL be able to migrate? If they will migrate between cluster nodes, then I would see NPTL having a huge advantage over pthreads for clusters.

charlieg · Posted: Mon Aug 16, 2004 5:02 am Post subject:

asiobob · Posted: Mon Aug 16, 2004 5:12 am Post subject:

after enabling nptl xmms and mplayer wouldn't start (segfault), the only common thing here is perhaps the reliance of the nvidia drivers (glx) on both. A remerge of that fixed the problem (emerge nvidia-glx).

charlieg · Posted: Mon Aug 16, 2004 6:12 am Post subject:

asiobob · Posted: Mon Aug 16, 2004 6:16 am Post subject:

yes of course, however I was under the impression that by compiling glibc with NPTL was the only thing that had to be done "enable" NPTL (assuming one had a 2.6 kernel). Are you hinting there's a kernel config option for this?

John5788 · Posted: Mon Aug 16, 2004 7:42 am Post subject:

i added nptl to my use flags and recompiled glibc. it seems like im using NPTL and firefox seems to use 3 less processes than before, never needed to emerge linux26-headers