Why does my system freeze when 1/2 RAM is used?

Martux · Veteran Joined: 04 Feb 2005 Posts: 1917

Hi!
I´ve got a Intel Quad Q9450 with 4 gigs of Ram. I also added 2 gigs of swap.
/var/tmp/portage is mounted in tmpfs (3500MB).
Every now and then (when compiling boost, for example) my system totally freezes for 10-20 m.
I can see RAM usage increase till ~2000MB aqnf then this happens. But why?
I´ve never seen that gkrellm showed more than 2000MB RAM used.
_________________
"Coincidence is God's way of remaining anonymous."
Albert Einstein
"The road to success is always under construction"

Mike Hunt · Watchman Joined: 19 Jul 2009 Posts: 5287

Hmmm, weird because I have seen boost build just fine on 1Gig RAM boxes no problem.

Maybe something in your emerge --info might provide a clue.

pigeon768 · l33t Joined: 02 Jan 2006 Posts: 683

Does memtest86+ turn up any errors? Does the compile eventually complete?

eccerr0r · Posted: Thu Dec 24, 2009 11:58 pm Post subject:

I have an older Q6600 with 4GB RAM, 32-bit install. I'm sure I've compiled boost many times, but it's not my biggest consumer of RAM. I can run two copies of WoW on this machine and each copy uses at least 1-2GB RAM after a while, and sometimes runs out of RAM (wine memory leak?)...affecting just that instance of WoW.

When you say "system totally freezes" it's confusing when you then continue with "for 10-20 m" ... when a system freezes, usually it implies it's dead forever, do you mean it freezes after 10-20 minutes? In this case I'd look to see if your chipset is overheating or not?
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

beli.sk · n00b Joined: 25 Dec 2009 Posts: 11

The tmpfs can be eating quite a lot of memory, and the rest (500 MB?) may not be enough for the processes. Try decreasing the tmpfs max size to for example 2GB, or less, just for testing. The meters may, or may not, count the cache memory (used for tmpfs) to the memory usage shown. Or the problem can be somewhere else, not memory related.
_________________
Beli | independent IT consultant | public(a)beli.sk | www.beli.sk

Martux · Veteran Joined: 04 Feb 2005 Posts: 1917

Hi all!
Thanks for all the input.
Tonight I will emerge -e my system to hopefully fix another problem I have with korganizer.
After that, I am going to fiddle with the tmpfs size a bit.
If that does not fix it, I´ll gonna test with memtest.
To make clearer what I meant with "freezing":
The system hangs when 2gig of memory are used (as shown by gkrellm). I then can see how swap space is occupied. This brings the system to halt for some minutes, mouse stops moving and music stops.
If I do nothing, it will work again after a while.
So, I need some time for all of your suggestions.
Thanks,
Marcus
_________________
"Coincidence is God's way of remaining anonymous."
Albert Einstein
"The road to success is always under construction"

eccerr0r · Posted: Fri Dec 25, 2009 9:21 pm Post subject:

Okay, that clarification is better. If it's really slow (meaning, it eventually comes back) then it's more likely a software issue (or poorly behaved software due to hardware fault).

When I run out of swap and RAM it somewhat behaves much like how it's described - freezes for a while, as the computer freaks out trying to figure out what to do when there's no more free ram and it's requested.

Easier to trigger this OOM condition where it slows really badly as it tries to evict text pages for data/new text. Having gobs of swap helps this, I found. Having no swap *really* behaves poorly.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

Martux · Veteran Joined: 04 Feb 2005 Posts: 1917

Well, fiddling with tmpfs size did not help. I came to a point, when it was set to 1500MB, when boost simply stated it ran out of space. Neither 200MB or 2500MB solved the hanging.
Memtest86+ did not find any errors after 3 passes and several hours running.
Maybe the hanging is not exactly related to RAM usage, but I saw insanely high "load" values: 31 (!!!)
Shouldn´t the "load average" parameter take care of this? I set it to 5.0 (btw, setting "jobs" to 2 doesn´t help either).
So, running out of options right now, here´s the output of emerge --info:
I really hope you guys have more ideas!

eccerr0r · Posted: Mon Dec 28, 2009 2:05 am Post subject:

Question still remains - how deeply are you into swap when this happens? Was your swap partition full?
If all your ram was used to store compile temp files, did it run out of space to work with compiling?
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

Martux · Veteran Joined: 04 Feb 2005 Posts: 1917

Yes, indeed swap space is used. I´ve got 2GB.
Strange with that is, that I have set vm.swappiness to 0 and it begins swapping like hell when RAM is used ~1800MB. How could I take control of the load better?
_________________
"Coincidence is God's way of remaining anonymous."
Albert Einstein
"The road to success is always under construction"

eccerr0r · Posted: Mon Dec 28, 2009 6:44 pm Post subject:

You *want* it to swap versus text paging. Swap is much faster than text paging.

How does the machine behave if you don't use tmpfs for /var/tmp/portage?

For me in all honesty I don't believe in having portage use only RAM for building new packages. This is because disk RAM cache usually does a good job as it is, dealing with minimizing disk writes -- it will still write stuff to disk that's unnecessary but so what? It's clearly not perfect, after all it's a heuristic and balancing act, but it's much better than if you were building with absolutely no disk cache at all (or equivalently, with insufficient RAM).

If your load average is hitting 31 while emerge with -j5, likely means that the CPU can't get forward progress done because of RAM constraints.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

BitJam

Martux · Veteran Joined: 04 Feb 2005 Posts: 1917

Yeah it really seems to under-report RAM usage.
However, to test if this also is happening without tmpfs I threw the unholy triad of evil (tm) upon it:
emerge --oneshot boost wxGTK PyQt4 :twisted:

And well it really works well with -j5, even without ccache.
Top btw. showed a completely different RAM usage before system freezed than gkrellm, so it really seems I ran out of RAM. Thanks guys for helping me out.
It would still be lovely, if one could set per package tempdir for those special candidates

Thx, Marcus
_________________
"Coincidence is God's way of remaining anonymous."
Albert Einstein
"The road to success is always under construction"

devsk · Posted: Wed Dec 30, 2009 5:53 am Post subject:

Boost is heavy duty that way. It freezes my 6GB system temporarily because of swap activity. It doesn't matter what your swappiness is. I think you are pumping too many cc1plus processes at a time. Reduce your MAKEOPTS a bit or get rid of the tmpfs on /var/tmp/portage. You are killing your ram on /var/tmp/portage as well as on a large number of 250MB+ cc1plus processes. System doesn't know whether to give ram to the files on /var/tmp/portage or to the cc1plus processes. So, kswapd is busy finding free pages (and wasting CPU cycles, remember pages are 4KB and there are a large number of them in modern systems to sift through) and there aren't too many free, so X gets paged out, kwin gets paged out, all your other programs get paged out, and hence you get the feeling of a freeze.

Martux · Veteran Joined: 04 Feb 2005 Posts: 1917

I found out, there is a possibility to set MAKEOPTS on a per package basis.
I just had to create /etc/portage/env/category/package and put "export MAKEOPTS=-j1" in there.
That makes perfect sense for me, as even setting down MAKEOPTS to -j2 stalled the computer for me.
For all smaller packages except boost, wxGTK and PyQt4 -j5 works really great. This way I don´t have to give up tmpfs for compiling either, because what devsk stated is true: The stalling is not a matter of RAM.
Thanks y´all, I suppose this thread is fixed

_________________
"Coincidence is God's way of remaining anonymous."
Albert Einstein
"The road to success is always under construction"