Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
MAKEOPTS breaks portage
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
DavidF
n00b
n00b


Joined: 12 Jan 2004
Posts: 28
Location: Rocky Mountains

PostPosted: Wed Nov 23, 2022 5:00 pm    Post subject: MAKEOPTS breaks portage Reply with quote

This pc was built in April. On the initial build I had miss-spelled MAKEOPTs = "-J12". When I found the error and corrected it portage would crash. Since then my fix was to comment out MAKEOPTS and everything has worked without error. I have a couple of days and would like to see if this can be fixed.

My make.conf -useflags

COMMON_FLAGS="-march=alderlake -O2 -pipe"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}"

CPU_FLAGS_X86= "aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 ssse3"
# NOTE: This stage was built with the bindist Use flag enabled
PORTDIR="/var/db/repos/gentoo"
DISTDIR="/var/cache/distfiles"
PKGDIR="/var/cache/binpkgs"
#MAKEOPTS="--Jobs 12"
# This sets the language of build output to English.
# Please keep this setting intact when reporting bugs.
LC_MESSAGES=C

GENTOO_MIRRORS="http://www.gtlib.gatech.edu/pub/gentoo https://mirrors.rit.edu/gentoo/"
GRUB_PLATFORMS="efi-64 efi-32 pc emu"
ACCEPT_License="_* @FREE"
ACCEPT_LICENSE="*"
# ACCEPT_LICENSE="MPEG-4 freedist"
VIDEO_CARDS="nvidia intel"
PHP_TARGETS="php-8.0 php8-1"

Thanks David
Back to top
View user's profile Send private message
fedeliallalinea
Administrator
Administrator


Joined: 08 Mar 2003
Posts: 29090
Location: here

PostPosted: Wed Nov 23, 2022 5:03 pm    Post subject: Reply with quote

The option is -jXX or --jobs XX (with J in lower case).
_________________
Questions are guaranteed in life; Answers aren't.
Back to top
View user's profile Send private message
DavidF
n00b
n00b


Joined: 12 Jan 2004
Posts: 28
Location: Rocky Mountains

PostPosted: Wed Nov 23, 2022 5:12 pm    Post subject: Reply with quote

Now I really feel like a noob. I have been doing this way to long to miss that. Thank you! I have 24 threads with my i9 12900k chip and 64g ram. Should I be set at -j24? Have I been compiling as if I was -j24?
Back to top
View user's profile Send private message
fedeliallalinea
Administrator
Administrator


Joined: 08 Mar 2003
Posts: 29090
Location: here

PostPosted: Wed Nov 23, 2022 5:31 pm    Post subject: Reply with quote

DavidF wrote:
Now I really feel like a noob. I have been doing this way to long to miss that. Thank you! I have 24 threads with my i9 12900k chip and 64g ram. Should I be set at -j24? Have I been compiling as if I was -j24?

Empirical calculation is 2GB per job, so 24*2=48GB I would say that can work.
_________________
Questions are guaranteed in life; Answers aren't.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 19532

PostPosted: Wed Nov 23, 2022 5:36 pm    Post subject: Re: MAKEOPTS breaks portage Reply with quote

DavidF wrote:
When I found the error and corrected it portage would crash.
Portage should show you the output of Make when Make fails. In turn, Make appears to diagnose this error for you:
Code:
$ make -f /dev/null --Jobs 12
make: unrecognized option '--Jobs'
Usage: make [options] [target] ...
Options:
Was that output not shown for your faulty invocation?
Back to top
View user's profile Send private message
DavidF
n00b
n00b


Joined: 12 Jan 2004
Posts: 28
Location: Rocky Mountains

PostPosted: Wed Nov 23, 2022 5:43 pm    Post subject: Re: MAKEOPTS breaks portage Reply with quote

Quote:
Was that output not shown for your faulty invocation?


Its been so long that I don't remember. I am pretty sure if I would have seen that error I would have corrected it. It seems to me it had something to do with finding the kernel. I have made the change and re-emerged firefox without error.

I think it would fail at this line "Checking for suitable kernel configuration options ..." some things would compile
Back to top
View user's profile Send private message
DavidF
n00b
n00b


Joined: 12 Jan 2004
Posts: 28
Location: Rocky Mountains

PostPosted: Wed Nov 23, 2022 6:51 pm    Post subject: Re: MAKEOPTS breaks portage Reply with quote

Quote:
Empirical calculation is 2GB per job, so 24*2=48GB I would say that can work.


So does this mean that portage will use this memory to compile saving disk io?

If so, is tempfs for portage redundant?
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 19532

PostPosted: Wed Nov 23, 2022 7:08 pm    Post subject: Reply with quote

The build system will write files to the PORTAGE_TMPDIR filesystem. If that filesystem is tmpfs, then the kernel will not persist those, except to swap, and then only when forced to do so by memory pressure. If the filesystem is disk-backed, the kernel will write the files to disk, except possibly for short builds where the file is deleted before the kernel has time to persist it. tmpfs is redundant for small builds from the perspective that the kernel will likely have the file in the page cache anyway, and will not need to go to the disk even for a disk-backed build. tmpfs is non-redundant from the perspective that it more strongly encourages the kernel not to bother writing to the disk.

The quoted calculation is how much free RAM you want per job. It excludes any memory spent on a tmpfs.
Back to top
View user's profile Send private message
DavidF
n00b
n00b


Joined: 12 Jan 2004
Posts: 28
Location: Rocky Mountains

PostPosted: Wed Nov 23, 2022 7:34 pm    Post subject: Reply with quote

Hu wrote:
The quoted calculation is how much free RAM you want per job. It excludes any memory spent on a tmpfs.


Thank you for that explanation!
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 51543
Location: 56N 3W

PostPosted: Wed Nov 23, 2022 8:04 pm    Post subject: Reply with quote

DavidF,

If you have the RAM for PORTAGE_TMPDIR in tmpfs. The kernel will do it anyway as everything will be in the kernel cache.

Explicitly setting PORTAGE_TMPDIR to a tmpfs location saves two things.
1. The memory bandwidth consumed by disk writes. That's almost too small to measure. so you won't see any performance gain.
2. Disk writes that will never be read. If you are paranoid about SSD life, that's a very good thing.
It all depends on your SSD usage pattern.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
axl
Veteran
Veteran


Joined: 11 Oct 2002
Posts: 1124
Location: Romania

PostPosted: Wed Nov 23, 2022 8:16 pm    Post subject: Reply with quote

Can I just remind people to use zram instead of tmpfs/ramfs for portage build dir? Use zstd. It's decent.
Back to top
View user's profile Send private message
Genone
Retired Dev
Retired Dev


Joined: 14 Mar 2003
Posts: 9347
Location: beyond the rim

PostPosted: Thu Nov 24, 2022 9:14 am    Post subject: Reply with quote

DavidF wrote:
Now I really feel like a noob. I have been doing this way to long to miss that. Thank you! I have 24 threads with my i9 12900k chip and 64g ram. Should I be set at -j24? Have I been compiling as if I was -j24?

Mind that loading that CPU with so many threads will also cause it to draw a ton of power and therefore generate a ton of heat, so you should really check that your cooling is adequate for this. Even a 360mm AiO may struggle with that task. I have a 12900k at work, and initially it would easily reach 100°C (=thermal throttling) during large compile jobs within seconds. We had to upgrade the cooling and tweak BIOS settings to keep it in check.
Back to top
View user's profile Send private message
DavidF
n00b
n00b


Joined: 12 Jan 2004
Posts: 28
Location: Rocky Mountains

PostPosted: Thu Nov 24, 2022 2:52 pm    Post subject: Reply with quote

Thanks for the watch out. My CPU maxes out at 88c. I have never seen it over 90. I have that chip on an Asus maximus extreme board and using a Corsair iCUE H150i ELITE CAPELLIX 360mm RGB Water Cooling Kit
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum