thoughts about resources while compiling [answered]

[DaggyStyle]

Greetings,

a few months ago I got a new machine, 16GB of ram, ryzen x2700, the mb is ga B450 aorus m.
I have 4 hdds within a sw raid. my /tmp/portage is tmpfs of 10 gb.
from the start, I got some peaks where the machine got sluggish or froze for a few seconds and continued while compiling heavy pkgs.
after upgrade to kernel 5.2, I got complete freeze of minutes even.
my previous machine was an i7-2600 with slower ram (16GB), the mb was GA-Z68P-DS3 with the same hdds (/tmp/portage is tmpfs of 10 gb. too) and I don't remember getting sluggish or complete freeze. moreover, I had boinc running in the background.

I read that there is a rule of 2GB per compiling thread but the 10 gb isn't enough for the previous setup.
so I wonder, what can be the issue that causes the system to freeze on heavy loads?
_________________
Only two things are infinite, the universe and human stupidity and I'm not sure about the former - Albert Einstein

[mike155]

1. Please show us the output of
   

   Code:

   emerge --info

   
   
2. Is multithreading enabled? In this case, you would need up to 42 GB Ram (16 x 2GB + 10 GB /var/tmp/portage) if you want to avoid swapping. Well, 32GB will probably be sufficient, but 16GB is not enough.
   

[NeddySeagoon]

DaggyStyle,

The 2G per gcc thread is good for large C++ packages.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[ff11]

I don't like it too, but is the way that the source codes are evolving.

THEN I will evolve too:
* I have created my /etc/portage/env/makeopts-j2:

[DaggyStyle]

[DaggyStyle]

[DaggyStyle]

[Anon-E-moose]

[Anon-E-moose]

[ff11]

[DaggyStyle]

I always think wcs, e.g. 2GB but even for firefox, it doesn't means that all cores are using 2GB per core, for example. 0.5GB per core means 8*0.5+10 = 14GB where 16*0.5+10 = 18GB.

I'll limit the core count for firefox.

Thanks.
_________________
Only two things are infinite, the universe and human stupidity and I'm not sure about the former - Albert Einstein

[mike155]

Try the following:

1. Set

   Code:

   MAKEOPTS="-j16"

   and run
   

   Code:

   time emerge --oneshot gcc

   
   
2. Disable hyperthreading in your BIOS and make sure that it's disabled after you booted Linux (look at the output of 'lscpu': it should print: 'CPU(s): 8' and 'Thread(s) per core: 1').
   
   Set

   Code:

   MAKEOPTS="-j8"

   and run
   

   Code:

   time emerge --oneshot gcc

   
   
3. Compare the two results. Most probably, you will see that the hyperthreading test is NOT faster than the non-hyperthreading test. That's the reason why I disable hyperthreading on all of my machines. Disabling hyperthreading also reduces the amount of RAM needed and makes administration and monitoring easier.
   

[firephoto]

On my week old 2700x system I've had my makeopts at -j13 and have experienced no noticeable slowdowns even during parallel builds of big things like qtwebengine while various other things finish. With this setup I've noticed the ram use does hit 16G (I have 32G). I purposely have kept it backed off because of the parallelism.

Relevant make.conf bits.

[DaggyStyle]

[DaggyStyle]

qtwebengine is a pkg I don't intend to install on my system.
_________________
Only two things are infinite, the universe and human stupidity and I'm not sure about the former - Albert Einstein

[firephoto]

[DaggyStyle]

I've ended up binding /var/tmp/portage to an old hdd I have, might loose performance but better than the alternative
_________________
Only two things are infinite, the universe and human stupidity and I'm not sure about the former - Albert Einstein

[firephoto]

[DaggyStyle]