Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Linux memory management
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Gentoo Chat
View previous topic :: View next topic  
Author Message
Catach
Apprentice
Apprentice


Joined: 06 Nov 2003
Posts: 268
Location: Australia

PostPosted: Thu May 13, 2004 8:40 am    Post subject: Linux memory management Reply with quote

One for the Gurus :wink:

I've noticed recently that my box seems to be consuming rediculously large amounts of memory (200Mb for mozilla, xmms, as KSystemGuard reports it). At first I thought it was a problem but a friend assures me it's mainly buffers and shared memory.

He went into an explanation that I didn't follow, so I was wondering if anyone could do a better job at how Linux manages memory: why so much RAM for tiny apps? the difference between what top lists as 'RES' 'SHR'? what is actually stored in the 'cache' section that /proc/meminfo talks about, etc.

Or even better, an online resource that has a good explanation of it all (from noob intro to detailed analysis).
_________________
Zak: Luke's making a 3D interface for Linux"
Jon G: "Yeah, Luke's making a 3D user interface with Logowriter"
Jon H: "He's only got 99 shapes to work with"
Jon G: "And they're all turtles."
Back to top
View user's profile Send private message
Joe Kinley
n00b
n00b


Joined: 31 May 2003
Posts: 74
Location: Germany

PostPosted: Thu May 13, 2004 10:05 am    Post subject: Reply with quote

Can't describe it now, but i read this book "Linux Kernelarchitecture" ... dunno if it is called same in US, but its big, heavy and costed me about 50€ (similar to 50 bucks) so quite expensive... but it described the whole memorysystem great.... gotta have a look on it
Back to top
View user's profile Send private message
Plain-old-Jeb
n00b
n00b


Joined: 19 Dec 2003
Posts: 51
Location: My House

PostPosted: Thu May 13, 2004 11:06 am    Post subject: Reply with quote

Don't take my word for it but linux will use all the physical memory possible :) for the disk cache however your problem sounds more like its just the fac that X and X-Apps use shared memory for rendering .etc. (Sorry I don't know much details, however it aint hurting ur box so what does it matter :))
_________________
emerge -C world
Back to top
View user's profile Send private message
spb
Retired Dev
Retired Dev


Joined: 02 Jan 2004
Posts: 2135
Location: Cambridge, UK

PostPosted: Thu May 13, 2004 11:56 am    Post subject: Reply with quote

The short answer is that Linux hates wasting RAM. If you have any physical RAM unused, it will try to use it for I/O buffers and disk cache. Basically, if you have a process that's reading sequentially from the hard disk, it'll read as far ahead as it can, to fill up the unused RAM, and then just read from that buffer when the application next asks for data. If you have insane amounts of RAM, try ripping a CD on the fly (ripping and encoding in one process). You might find the CD drive spinning down before the encoding gets half way through.

It's a bit more complex than that, but that's basically how it goes.
Back to top
View user's profile Send private message
placeholder
Advocate
Advocate


Joined: 07 Feb 2004
Posts: 2500

PostPosted: Thu May 13, 2004 12:03 pm    Post subject: Reply with quote

Basically, if it utilizes more RAM that's probably a good thing because what is the point of having a ton of RAM that just sits there? ;)
Back to top
View user's profile Send private message
John5788
Advocate
Advocate


Joined: 06 Apr 2004
Posts: 2140
Location: 127.0.0.1

PostPosted: Thu May 13, 2004 2:15 pm    Post subject: Reply with quote

exactly :) unused ram is wasted ram.
_________________
John5788
Back to top
View user's profile Send private message
sapphirecat
Guru
Guru


Joined: 15 Jan 2003
Posts: 376

PostPosted: Thu May 13, 2004 2:45 pm    Post subject: Re: Linux memory management Reply with quote

Catach wrote:
why so much RAM for tiny apps?

Graphics are huge :wink:

Anything that does its own skinning (xmms, mplayer, mozilla, openoffice, etc.) is likely to be bigger than other things simply because they implement this skinning themselves. Even if they're drawing widgets from the toolkit, the code for other skins is still there. Add in some caching like in Mozilla and things get large quick.

Quote:
the difference between what top lists as 'RES' 'SHR'?

VIRT is the virtual size, which is the amount of RAM that is actually mapped into the process address space; this is the sum of all shared libraries (actually loaded or not), AGP aperature and video RAM (for X and DRI clients), shared memory segments, and maybe other stuff. RES is the amount resident, which is how much memory the application is actually presently occupying. SHR, for shared, indicates how much of VIRT corresponds to shared libraries. Like VIRT, this can be bigger than the parts of it which are actually loaded. For instance, if you have a GLib program that isn't using gmodule, none of the gmodule library (beyond the symbol table) will be loaded.

Quote:
Or even better, an online resource that has a good explanation of it all (from noob intro to detailed analysis).

I ought to write one. As soon as I figure out the difference between buffers and cache.
_________________
Former Gentoo user; switched to Kubuntu 7.04 when I got sick of waiting on gcc. Chance of thread necro if you reply now approaching 100%...
Back to top
View user's profile Send private message
PaV
Apprentice
Apprentice


Joined: 28 Jan 2004
Posts: 176
Location: Poland

PostPosted: Fri May 14, 2004 6:35 pm    Post subject: Reply with quote

i love it.
but i have a lot of my 128MB graphic card's memory sitting there doing nothing. kernel devs: thats a feature request :D
Back to top
View user's profile Send private message
lbrtuk
l33t
l33t


Joined: 08 May 2003
Posts: 910

PostPosted: Fri May 14, 2004 8:02 pm    Post subject: Reply with quote

You can already use that ram. I saw documentation a while ago showing you how to use your video card's ram as a MTD (memory technology device), creating a ramdisk that you can add as swap.

http://kerneltrap.org/node/view/143
Back to top
View user's profile Send private message
PaV
Apprentice
Apprentice


Joined: 28 Jan 2004
Posts: 176
Location: Poland

PostPosted: Fri May 14, 2004 8:10 pm    Post subject: Reply with quote

wow... everything you want, linux has :D
Back to top
View user's profile Send private message
dalek
Veteran
Veteran


Joined: 19 Sep 2003
Posts: 1344
Location: Mississippi USA

PostPosted: Fri May 14, 2004 9:04 pm    Post subject: Reply with quote

I recently upgraded from 512Mb to 1Gb. It still will fill it up with cache data. It does it pretty quick too.

I have also noticed that Mozilla uses a lot of memory on mine to, one reason I upgraded. I have had it use 200 Mbs and sometimes over 300Mbs. I guess it is fairly normal since we both use a lot. I have noticed that it does it a lot when messing with images. You may want to check the settings for Mozilla cache. I have mine set to 200Mb because I mess with images.

You may also want to go to tools and download manager and clear out all that stuff to. You can do ctrl A to select all and then press delete. It may take a while and may cause your system to not respond for a while. <scared the jeepers out of me for a minute when I did mine. It must have been over a thousand of them in there. >

Guess what, as soon as memory drops again, I plan to add another stick of 512Mbs. That will do till I build another rig. It does make it faster though. More the merrer I guess. < can't spell >

Later

:D :D :D :D
_________________
My rig: Gigabyte GA-970A-UD3P mobo, AMD FX-8350 Eight-Core CPU, ZALMAN CNPS10X Performa CPU cooler,
G.SKILL 32GB DDR3 PC3 12800 Memory Nvidia GTX-650 video card LG W2253 Monitor
9TBs of hard drive space using LVM
Cooler Master HAF-932 Case
Back to top
View user's profile Send private message
Clansman
Tux's lil' helper
Tux's lil' helper


Joined: 10 Jan 2004
Posts: 140

PostPosted: Sun Jun 20, 2004 10:36 am    Post subject: Reply with quote

hi everyone.

excelent post!

yet, i think few people that posted here got the original post's point according to the post's replies... overall, about the memory size and occupancy one can conclude:

"Having low free memory reported from 'top' DOES NOT MEAN that your system is desperately and unrecoverably occupying all of your memory (or most of it), meaning that you must rush to the nearest and most expensive store to buy another Gb of ram.

The linux kernel has a memory management policy that cleverly tries to use up as much memory as it possibly can, to improve performance. This performance improvement can be very visible if that used memory is disk cache because as stated before, hard drives are about 1000 times slower than RAM, so getting disk's data (cache) from ram is very fast. The RAM is the fastest and second biggest memory available outside the cpu.

That big memory cache forcedly used by the linux kernel to avoid wasted memory can be VERY QUICKLY shrinked up to allow user applications to use memory/more memory.

So, having 70Mb of free ram when you have only 1Gb is not a bad thing and does not mean you have to buy new memory to boost performance. You would probably find out that with your new 2Gb of ram, you'd end up having... 100Mb free..."

[]
Back to top
View user's profile Send private message
Ari Rahikkala
Guru
Guru


Joined: 02 Oct 2002
Posts: 370
Location: Finland

PostPosted: Sun Jun 20, 2004 11:46 am    Post subject: Reply with quote

Linux memory management, or 'Why is there no free RAM?' by sapphirecat.
_________________
<laurentius> gentoo linux?
<ari> Yesh.
<laurentius> they look horny
Back to top
View user's profile Send private message
Clansman
Tux's lil' helper
Tux's lil' helper


Joined: 10 Jan 2004
Posts: 140

PostPosted: Sun Jun 20, 2004 12:16 pm    Post subject: Reply with quote

shame on me! i posted in the wrong thread!!

the correct post here is the redirect link that Ari Rahikkala posted above.

if any moderator sees this, please remove my above post in this thread. it is partially unrelated with the subject. the redirect is more useful.

thanks and sorry for the inconvenience.

[]
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo Chat All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum