Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Why does my system freeze when 1/2 RAM is used?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
Martux
Veteran
Veteran


Joined: 04 Feb 2005
Posts: 1747
Location: Mannheim, Germany

PostPosted: Thu Dec 24, 2009 9:22 am    Post subject: Why does my system freeze when 1/2 RAM is used? Reply with quote

Hi!
I´ve got a Intel Quad Q9450 with 4 gigs of Ram. I also added 2 gigs of swap.
/var/tmp/portage is mounted in tmpfs (3500MB).
Every now and then (when compiling boost, for example) my system totally freezes for 10-20 m.
I can see RAM usage increase till ~2000MB aqnf then this happens. But why?
I´ve never seen that gkrellm showed more than 2000MB RAM used.
_________________
"Coincidence is God's way of remaining anonymous."
Albert Einstein
"The road to success is always under construction"
Back to top
View user's profile Send private message
Mike Hunt
Watchman
Watchman


Joined: 19 Jul 2009
Posts: 5287

PostPosted: Thu Dec 24, 2009 2:20 pm    Post subject: Reply with quote

Hmmm, weird because I have seen boost build just fine on 1Gig RAM boxes no problem.

Maybe something in your emerge --info might provide a clue.
Back to top
View user's profile Send private message
pigeon768
l33t
l33t


Joined: 02 Jan 2006
Posts: 669

PostPosted: Thu Dec 24, 2009 10:45 pm    Post subject: Reply with quote

Does memtest86+ turn up any errors? Does the compile eventually complete?
_________________
My political bias.
Back to top
View user's profile Send private message
eccerr0r
Advocate
Advocate


Joined: 01 Jul 2004
Posts: 4102
Location: USA

PostPosted: Thu Dec 24, 2009 11:58 pm    Post subject: Reply with quote

I have an older Q6600 with 4GB RAM, 32-bit install. I'm sure I've compiled boost many times, but it's not my biggest consumer of RAM. I can run two copies of WoW on this machine and each copy uses at least 1-2GB RAM after a while, and sometimes runs out of RAM (wine memory leak?)...affecting just that instance of WoW.

When you say "system totally freezes" it's confusing when you then continue with "for 10-20 m" ... when a system freezes, usually it implies it's dead forever, do you mean it freezes after 10-20 minutes? In this case I'd look to see if your chipset is overheating or not?
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed to be advocating?
Back to top
View user's profile Send private message
beli.sk
n00b
n00b


Joined: 25 Dec 2009
Posts: 11

PostPosted: Fri Dec 25, 2009 6:54 pm    Post subject: Reply with quote

The tmpfs can be eating quite a lot of memory, and the rest (500 MB?) may not be enough for the processes. Try decreasing the tmpfs max size to for example 2GB, or less, just for testing. The meters may, or may not, count the cache memory (used for tmpfs) to the memory usage shown. Or the problem can be somewhere else, not memory related.
_________________
Beli | independent IT consultant | public(a)beli.sk | www.beli.sk
Back to top
View user's profile Send private message
Martux
Veteran
Veteran


Joined: 04 Feb 2005
Posts: 1747
Location: Mannheim, Germany

PostPosted: Fri Dec 25, 2009 7:32 pm    Post subject: Reply with quote

Hi all!
Thanks for all the input.
Tonight I will emerge -e my system to hopefully fix another problem I have with korganizer.
After that, I am going to fiddle with the tmpfs size a bit.
If that does not fix it, I´ll gonna test with memtest.
To make clearer what I meant with "freezing":
The system hangs when 2gig of memory are used (as shown by gkrellm). I then can see how swap space is occupied. This brings the system to halt for some minutes, mouse stops moving and music stops.
If I do nothing, it will work again after a while.
So, I need some time for all of your suggestions.
Thanks,
Marcus
_________________
"Coincidence is God's way of remaining anonymous."
Albert Einstein
"The road to success is always under construction"
Back to top
View user's profile Send private message
eccerr0r
Advocate
Advocate


Joined: 01 Jul 2004
Posts: 4102
Location: USA

PostPosted: Fri Dec 25, 2009 9:21 pm    Post subject: Reply with quote

Okay, that clarification is better. If it's really slow (meaning, it eventually comes back) then it's more likely a software issue (or poorly behaved software due to hardware fault).

When I run out of swap and RAM it somewhat behaves much like how it's described - freezes for a while, as the computer freaks out trying to figure out what to do when there's no more free ram and it's requested.

Easier to trigger this OOM condition where it slows really badly as it tries to evict text pages for data/new text. Having gobs of swap helps this, I found. Having no swap *really* behaves poorly.
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed to be advocating?
Back to top
View user's profile Send private message
Martux
Veteran
Veteran


Joined: 04 Feb 2005
Posts: 1747
Location: Mannheim, Germany

PostPosted: Sat Dec 26, 2009 2:10 pm    Post subject: Reply with quote

Well, fiddling with tmpfs size did not help. I came to a point, when it was set to 1500MB, when boost simply stated it ran out of space. Neither 200MB or 2500MB solved the hanging.
Memtest86+ did not find any errors after 3 passes and several hours running.
Maybe the hanging is not exactly related to RAM usage, but I saw insanely high "load" values: 31 (!!!)
Shouldn´t the "load average" parameter take care of this? I set it to 5.0 (btw, setting "jobs" to 2 doesn´t help either).
So, running out of options right now, here´s the output of emerge --info:
I really hope you guys have more ideas!
Code:

Portage 2.2_rc61 (default/linux/amd64/10.0/desktop, gcc-4.4.2, glibc-2.11-r1, 2.6.32-gentoo x86_64)
=================================================================                                 
System uname: Linux-2.6.32-gentoo-x86_64-Intel-R-_Core-TM-2_Quad_CPU_Q9450_@_2.66GHz-with-gentoo-2.0.1
Timestamp of tree: Sat, 26 Dec 2009 14:00:01 +0000                                                   
ccache version 2.4 [enabled]                                                                         
app-shells/bash:     4.0_p35                                                                         
dev-java/java-config: 2.1.10                                                                         
dev-lang/python:     2.6.4, 3.1.1-r1                                                                 
dev-util/ccache:     2.4-r8                                                                           
dev-util/cmake:      2.8.0                                                                           
sys-apps/baselayout: 2.0.1                                                                           
sys-apps/openrc:     0.6.0                                                                           
sys-apps/sandbox:    2.2                                                                             
sys-devel/autoconf:  2.64                                                                             
sys-devel/automake:  1.9.6-r3, 1.10.3, 1.11.1                                                         
sys-devel/binutils:  2.20                                                                             
sys-devel/gcc-config: 1.4.1                                                                           
sys-devel/libtool:   2.2.6b                                                                           
virtual/os-headers:  2.6.30-r1                                                                       
ACCEPT_KEYWORDS="amd64 ~amd64"                                                                       
ACCEPT_LICENSE="*"                                                                                   
CBUILD="x86_64-pc-linux-gnu"                                                                         
CFLAGS="-march=native -O2 -pipe -fomit-frame-pointer"                                                 
CHOST="x86_64-pc-linux-gnu"                                                                           
CONFIG_PROTECT="/etc /usr/share/X11/xkb /usr/share/config"                                           
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"                                                                 
CXXFLAGS="-march=native -O2 -pipe -fomit-frame-pointer"                                                                 
DISTDIR="/usr/portage/distfiles"                                                                                         
EMERGE_DEFAULT_OPTS="--jobs=5 --load-average=5.0 --with-bdeps=y --keep-going"                                           
FEATURES="assume-digests ccache distlocks fixpackages news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unmerge-logs unmerge-orphans userfetch"                                                                             
GENTOO_MIRRORS="ftp://ftp.gentoo.mesh-solutions.com/gentoo/distfiles/ http://mirrors.sec.informatik.tu-darmstadt.de/gentoo/  http://distfiles.gentoo.org http://www.ibiblio.org/pub/Linux/distributions/gentoo"                                   
LANG="de_DE@euro.UTF-8"                                                                                                 
LC_ALL="de_DE.UTF-8"                                                                                                     
LDFLAGS="-Wl,-O1"
LINGUAS="de"
MAKEOPTS="-j5"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="X a52 aac aiglx alsa amd64 asf audiofile bash-completion berkdb bitmap-fonts bzip2 cairo cdda cdparanoia cdr cli cracklib css cups custom-cflags custom-cxxflags cxx dbus de_DE digest divx dri dts dvd dvdr dvdread eds emboss encode evo fam ffmpeg firefox flac foomaticdb fortran fuse gdbm gif gimp glib glibc-omitfp gpm hal hddtemp iconv id3tag java jpeg jpeg2k kde kdehiddenvisibility kdexdeltas lame libnotify mad mikmod mmx mng modules mp3 mp4 mpeg mplayer mudflap multilib ncurses nls noamazon nptl nptlonly nsplugin nvidia ogg opengl openmp pch pcre pdf perl pic png ppds pppd python qt3support qt4 quicktime readline reflection sdl session smp spell spl srt sse sse2 sse3 ssl ssse3 startup-notification stream svg symlink sysfs tcpd theora threads thunar tiff transparency truetype truetype-fonts twolame type1-fonts unicode usb userlocales vcd vdpau vmware vorbis wavpack wmp x264 xcomposite xine xinerama xml xorg xulrunner xv xvid xvmc zlib" ALSA_CARDS="snd-ca0106" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="de" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="nv nvidia"
Unset:  CPPFLAGS, CTARGET, FFLAGS, INSTALL_MASK, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY

_________________
"Coincidence is God's way of remaining anonymous."
Albert Einstein
"The road to success is always under construction"
Back to top
View user's profile Send private message
eccerr0r
Advocate
Advocate


Joined: 01 Jul 2004
Posts: 4102
Location: USA

PostPosted: Mon Dec 28, 2009 2:05 am    Post subject: Reply with quote

Question still remains - how deeply are you into swap when this happens? Was your swap partition full?
If all your ram was used to store compile temp files, did it run out of space to work with compiling?
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed to be advocating?
Back to top
View user's profile Send private message
Martux
Veteran
Veteran


Joined: 04 Feb 2005
Posts: 1747
Location: Mannheim, Germany

PostPosted: Mon Dec 28, 2009 11:18 am    Post subject: Reply with quote

Yes, indeed swap space is used. I´ve got 2GB.
Strange with that is, that I have set vm.swappiness to 0 and it begins swapping like hell when RAM is used ~1800MB. How could I take control of the load better?
_________________
"Coincidence is God's way of remaining anonymous."
Albert Einstein
"The road to success is always under construction"
Back to top
View user's profile Send private message
eccerr0r
Advocate
Advocate


Joined: 01 Jul 2004
Posts: 4102
Location: USA

PostPosted: Mon Dec 28, 2009 6:44 pm    Post subject: Reply with quote

You *want* it to swap versus text paging. Swap is much faster than text paging.

How does the machine behave if you don't use tmpfs for /var/tmp/portage?

For me in all honesty I don't believe in having portage use only RAM for building new packages. This is because disk RAM cache usually does a good job as it is, dealing with minimizing disk writes -- it will still write stuff to disk that's unnecessary but so what? It's clearly not perfect, after all it's a heuristic and balancing act, but it's much better than if you were building with absolutely no disk cache at all (or equivalently, with insufficient RAM).

If your load average is hitting 31 while emerge with -j5, likely means that the CPU can't get forward progress done because of RAM constraints.
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed to be advocating?
Back to top
View user's profile Send private message
BitJam
Advocate
Advocate


Joined: 12 Aug 2003
Posts: 2455
Location: Silver City, NM

PostPosted: Tue Dec 29, 2009 6:48 pm    Post subject: Re: Why does my system freeze when 1/2 RAM is used? Reply with quote

Martux wrote:
I´ve never seen that gkrellm showed more than 2000MB RAM used.

I'm a big fan of gkrellm but it seems to consistently under-report ram usage when compared to conky or free or htop.
Back to top
View user's profile Send private message
Martux
Veteran
Veteran


Joined: 04 Feb 2005
Posts: 1747
Location: Mannheim, Germany

PostPosted: Tue Dec 29, 2009 9:16 pm    Post subject: Reply with quote

Yeah it really seems to under-report RAM usage.
However, to test if this also is happening without tmpfs I threw the unholy triad of evil (tm) upon it:
emerge --oneshot boost wxGTK PyQt4 :twisted:
And well it really works well with -j5, even without ccache.
Top btw. showed a completely different RAM usage before system freezed than gkrellm, so it really seems I ran out of RAM. Thanks guys for helping me out.
It would still be lovely, if one could set per package tempdir for those special candidates :)
Thx, Marcus
_________________
"Coincidence is God's way of remaining anonymous."
Albert Einstein
"The road to success is always under construction"
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2739
Location: Bay Area, CA

PostPosted: Wed Dec 30, 2009 5:53 am    Post subject: Reply with quote

Boost is heavy duty that way. It freezes my 6GB system temporarily because of swap activity. It doesn't matter what your swappiness is. I think you are pumping too many cc1plus processes at a time. Reduce your MAKEOPTS a bit or get rid of the tmpfs on /var/tmp/portage. You are killing your ram on /var/tmp/portage as well as on a large number of 250MB+ cc1plus processes. System doesn't know whether to give ram to the files on /var/tmp/portage or to the cc1plus processes. So, kswapd is busy finding free pages (and wasting CPU cycles, remember pages are 4KB and there are a large number of them in modern systems to sift through) and there aren't too many free, so X gets paged out, kwin gets paged out, all your other programs get paged out, and hence you get the feeling of a freeze.
Back to top
View user's profile Send private message
Martux
Veteran
Veteran


Joined: 04 Feb 2005
Posts: 1747
Location: Mannheim, Germany

PostPosted: Wed Dec 30, 2009 10:15 am    Post subject: Reply with quote

I found out, there is a possibility to set MAKEOPTS on a per package basis.
I just had to create /etc/portage/env/category/package and put "export MAKEOPTS=-j1" in there.
That makes perfect sense for me, as even setting down MAKEOPTS to -j2 stalled the computer for me.
For all smaller packages except boost, wxGTK and PyQt4 -j5 works really great. This way I don´t have to give up tmpfs for compiling either, because what devsk stated is true: The stalling is not a matter of RAM.
Thanks y´all, I suppose this thread is fixed :)
_________________
"Coincidence is God's way of remaining anonymous."
Albert Einstein
"The road to success is always under construction"
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum