Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
A portage story
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
patrix_neo
Guru
Guru


Joined: 08 Jan 2004
Posts: 520
Location: The Maldives

PostPosted: Tue Jan 15, 2019 5:23 pm    Post subject: A portage story Reply with quote

I have tried to update my system for a few days now. 30+ days since.
I did the emerge --sync before anything.

The emerge --with-bdeps=y -DNu @world then gave me some befuddled messages of multi instances of qt5 and perl 5.24.3 and 5.26.2
Some packages needed the lower version to the stable ones while others needed the higher stable version of qt/perl.

No worries. I just do a perl-clean --all on the perl thingy (took me a day before I finally got it).
And way it went to compiling 5.26.2 with all modules and dependant programs/libraries recompiling later on.

Some package installs later, I was still stuck on the qt5 ordeal of multislot packages of qt 5.x.x.
So I updated glibc. Which could break some all over the spectrum of packages, if not 2.27.x ( was at 2.26.x ).

I then waited a day, rebooted, made an emerge --sync again. Suddenly qt5 ordeal was gone. I now had 'only' 177 packages to install.
Great, I thought. Starting off, I thought, why didn't I have a webbrowser??? So I stopped at 50 packages in and tried to install chromium.

Went well 3 hrs in, and then broke of memory limits. Not so great!

It was some preserved libs waiting too. So, I have 8 GB of mem, sysctl.conf with a stopper for OOM, that is
Code:

vm_overcommit_ratio = 2


The break of processes with OOM started around the kernel 3.14 branch. I hate that.

This OOM thing has, as you can see, haunted me for a while. Then I stumbled upon a way to get 4GB of swap 'on-the-fly'. That I do use for all emerges, including the compile of chromium...to no avail in the end. (memory limit break).

So, I have MAKEOPTS=-j4 in the make.conf to use for compiling multiple packages at once, I think I have to lower that value to like -j2.
One package at the time. Still, at the time when the memory limit hit the roof, chromium was the only package left compiling. I did have a browserwindow up and running, taking some 0,6 GB of mem.

Anyways, in the start of all, some strange behaviour with perl, then qt5 circling around with diff versions. Not much to hinge to to find out why.

I will do an emerge -e @system after this. That older glibc breakage made me worried.

Anyways, gentoo makes an option to use a system without systemD and pulseaudio, so it's worth the hassle.

Have a crazy time solving stuff
Back to top
View user's profile Send private message
Muso
Veteran
Veteran


Joined: 22 Oct 2002
Posts: 1052
Location: The Holy city of Honolulu

PostPosted: Tue Jan 15, 2019 5:43 pm    Post subject: Re: A portage story Reply with quote

patrix_neo wrote:
Have a crazy time solving stuff


To be fair, that's fun. Annoying, but fun. There's something dull about a pre-made environment geared towards all. Specializing each machine you have with gentoo is just more satisfying.
_________________
"You can lead a horticulture but you can't make her think" ~ Dorothy Parker
2021 is the year of the Linux Desktop!
Back to top
View user's profile Send private message
patrix_neo
Guru
Guru


Joined: 08 Jan 2004
Posts: 520
Location: The Maldives

PostPosted: Tue Jan 15, 2019 5:50 pm    Post subject: Re: A portage story Reply with quote

Muso wrote:
patrix_neo wrote:
Have a crazy time solving stuff


To be fair, that's fun. Annoying, but fun. There's something dull about a pre-made environment geared towards all. Specializing each machine you have with gentoo is just more satisfying.


I do agree on that part. The thing that ilkes me, is that I have to re-read about commands, perfectly working since the days of yor.
Then I get pissed. Still, the alternative is scary. At least feels worse.
Back to top
View user's profile Send private message
mike155
Advocate
Advocate


Joined: 17 Sep 2010
Posts: 4438
Location: Frankfurt, Germany

PostPosted: Tue Jan 15, 2019 6:01 pm    Post subject: Reply with quote

If you set
Code:
vm_overcommit_memory = 2

make sure to set - and understand -
Code:
vm.overcommit_ratio

as well. The default value of vm.overcommit_ratio is 50 - that's not even close to the value I need on my machines.

/usr/src/linux/Documentation/sysctl/vm.txt wrote:
overcommit_ratio:

When overcommit_memory is set to 2, the committed address
space is not permitted to exceed swap plus this percentage
of physical RAM.


Last edited by mike155 on Tue Jan 15, 2019 6:55 pm; edited 1 time in total
Back to top
View user's profile Send private message
patrix_neo
Guru
Guru


Joined: 08 Jan 2004
Posts: 520
Location: The Maldives

PostPosted: Tue Jan 15, 2019 6:23 pm    Post subject: Reply with quote

I did never needed that 'over_commit_ratio' or 'over_commit_memory' before that kernel change.
What did Linus think of!?? :)

I mean, I want to use like TOTAL_MEMORY -1Kib and then have a warning message : 'Too low on memory'.
That would be ideal to me. Totally.

Now, I've heard the kernel is as such, OOM is inevitable, unless you use some kind of sysctl settings.
With, the little as I know, you can only use so much of a farm of, lets say 0,5 peta RAM of memory, or a single PC of 8GB of RAM.
I suffer with all the HPC having peta byte of RAM....or even cRAM....very tight. <joke>

I'll look up on that vm.overcommit_ratio.
Back to top
View user's profile Send private message
skellr
l33t
l33t


Joined: 18 Jun 2005
Posts: 975
Location: The Village, Portmeirion

PostPosted: Tue Jan 15, 2019 7:00 pm    Post subject: Re: A portage story Reply with quote

patrix_neo wrote:

The emerge --with-bdeps=y -DNu @world then gave me some befuddled messages of multi instances of qt5 and perl 5.24.3 and 5.26.2
Some packages needed the lower version to the stable ones while others needed the higher stable version of qt/perl.


Do you run a "stable" system and then selectively install "unstable" things of interest?

IMO, a pure "unstable" system is less work than mixing the two.
Back to top
View user's profile Send private message
Muso
Veteran
Veteran


Joined: 22 Oct 2002
Posts: 1052
Location: The Holy city of Honolulu

PostPosted: Tue Jan 15, 2019 7:11 pm    Post subject: Re: A portage story Reply with quote

skellr wrote:
[IMO, a pure "unstable" system is less work than mixing the two.


++

I started out stable on my current desktop, but was adding so many programs to /etc/portage/package.accept_keywords, it was just much easier to just go back to using ~amd64.
_________________
"You can lead a horticulture but you can't make her think" ~ Dorothy Parker
2021 is the year of the Linux Desktop!
Back to top
View user's profile Send private message
mike155
Advocate
Advocate


Joined: 17 Sep 2010
Posts: 4438
Location: Frankfurt, Germany

PostPosted: Tue Jan 15, 2019 7:14 pm    Post subject: Reply with quote

I believe that 'over_commit_memory = 0' (the default) is perfectly fine for most users and most use cases. I use that setting on all of my desktop and development machines.

I use 'over_commit_memory = 2' only on database servers, where I want to make sure that the database processes won't get killed. And I had to learn the hard way (2 failed installations) that it's essential to set vm_overcommit_ratio as well.
Back to top
View user's profile Send private message
patrix_neo
Guru
Guru


Joined: 08 Jan 2004
Posts: 520
Location: The Maldives

PostPosted: Tue Jan 15, 2019 7:41 pm    Post subject: Re: A portage story Reply with quote

skellr wrote:
patrix_neo wrote:

The emerge --with-bdeps=y -DNu @world then gave me some befuddled messages of multi instances of qt5 and perl 5.24.3 and 5.26.2
Some packages needed the lower version to the stable ones while others needed the higher stable version of qt/perl.


Do you run a "stable" system and then selectively install "unstable" things of interest?

IMO, a pure "unstable" system is less work than mixing the two.


Short answer, 'NO'. I did, before this. I got rid of all unstable packages, no matter what.
I shat at package.use, package.keywords, package.accept_keywords, package.mask.
Got rid of all eix --installed-unstable too.

I forgot mentioning that. Sorry.

...and Muso is just bat shit crazy...
Back to top
View user's profile Send private message
Muso
Veteran
Veteran


Joined: 22 Oct 2002
Posts: 1052
Location: The Holy city of Honolulu

PostPosted: Tue Jan 15, 2019 10:48 pm    Post subject: Re: A portage story Reply with quote

patrix_neo wrote:
...and Muso is just bat shit crazy...


The proper term is guano loco.
_________________
"You can lead a horticulture but you can't make her think" ~ Dorothy Parker
2021 is the year of the Linux Desktop!
Back to top
View user's profile Send private message
patrix_neo
Guru
Guru


Joined: 08 Jan 2004
Posts: 520
Location: The Maldives

PostPosted: Thu Feb 21, 2019 8:50 pm    Post subject: Re: A portage story Reply with quote

Muso wrote:
patrix_neo wrote:
...and Muso is just bat shit crazy...


The proper term is guano loco.


You just like to correct me, don't you?
_________________
Life is a fog where some thinks to know where to go
To make an error is human, letting it be is the error.
Deus Vult
Back to top
View user's profile Send private message
Muso
Veteran
Veteran


Joined: 22 Oct 2002
Posts: 1052
Location: The Holy city of Honolulu

PostPosted: Fri Feb 22, 2019 12:52 am    Post subject: Re: A portage story Reply with quote

patrix_neo wrote:
Muso wrote:
patrix_neo wrote:
...and Muso is just bat shit crazy...


The proper term is guano loco.


You just like to correct me, don't you?


It's called mansplaining ;)
_________________
"You can lead a horticulture but you can't make her think" ~ Dorothy Parker
2021 is the year of the Linux Desktop!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum