Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Help interprete RAM usage ("free" and "ps" output)
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
1970
n00b
n00b


Joined: 07 May 2010
Posts: 46

PostPosted: Thu Oct 10, 2013 4:37 pm    Post subject: Help interprete RAM usage ("free" and "ps&quo Reply with quote

Hi,

this is a generic linux question, not gentoo specific and not hardware related- sorry for that, but since Gentoo is an expert distribution I hope that some geek can help me out with this strange thing.

It's about a Linux hardware box (no VM) with 32 GB RAM. Here the output of the free and ps command (ps output shortened).

How can it be, that free states only 846 MB used, but ps tells that informix (a database software) process uses 14 GB RAM RSS (42%)?

According to the ps man page, the VSZ and RSS values are in KiB.

Code:

# free -m
             total       used       free     shared    buffers     cached
Mem:         32051      26577       5474          0        551      25178
-/+ buffers/cache:        846      31204
Swap:         2047          0       2047

# ps auxww
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
informix 11619  7.0 43.0 14254836 14125908 ?   S<   Sep14 2543:42 oninit -D
informix 11979 12.6 42.2 14255260 13881416 ?   S<   Sep14 4579:22 oninit -D
informix 11981  7.2 42.2 14255528 13878504 ?   S<   Sep14 2616:48 oninit -D


OS: Suse Linux Enterprise Server 11 (x86_64)
Kernel: 2.6.27.19-5-default

Hints where to ask this question elsewhere welcome.

:?:
Back to top
View user's profile Send private message
nemectic
Apprentice
Apprentice


Joined: 20 Aug 2004
Posts: 181

PostPosted: Thu Oct 10, 2013 5:33 pm    Post subject: Reply with quote

I expect the database itself is taking up cache, which is represented by free but still available for other apps to use if necessary.
Back to top
View user's profile Send private message
1970
n00b
n00b


Joined: 07 May 2010
Posts: 46

PostPosted: Thu Oct 10, 2013 5:46 pm    Post subject: Reply with quote

nemectic wrote:
I expect the database itself is taking up cache, which is represented by free but still available for other apps to use if necessary.

What I have ever read, the caching thing is something that the Linux kernel does, not the processes itself. Otherwise that would mean, that the process ram usage would shrink, if other processes "really" eat the RAM?
Back to top
View user's profile Send private message
nemectic
Apprentice
Apprentice


Joined: 20 Aug 2004
Posts: 181

PostPosted: Thu Oct 10, 2013 6:22 pm    Post subject: Reply with quote

Yeah, the database can be read fine from disk, but it's obviously much faster from RAM. The process itself will take up an amount of memory, but much of that 42% will be the database which is just cached. If other programs need to make use of the memory some of the cache will be lost and the database will read from disk instead, slowing down access considerably - the process itself will still consume its own allotment of memory though, which will probably be a couple of percent at most I'd guess.
Back to top
View user's profile Send private message
Atom2
Tux's lil' helper
Tux's lil' helper


Joined: 01 Aug 2011
Posts: 124

PostPosted: Fri Oct 11, 2013 10:31 pm    Post subject: Reply with quote

I would think the answer to your question lies in the mmap(2) system-call which allows a process to memory-map a (regular or device) file to its address space thus showing up in the resident set size (RSS) column in the ps command.

Accessing data being stored in a memory-mapped file from within the mapping process would usually still go through the kernel's file caching algorithms: In case the requested data has been read from or written to recently, requests can easily be served directly from memory through the in-kernel page cache. Otherweise the first access will populate (i.e. fill-up) the page cache to then serve the request.

In case the kernel requires memory previously allocated to the page cache for other purposes (e.g. to start a new process) it will either simply re-use non-dirty cache pages (i.e. those that contain unchanged data only) or, lacking the availability of any such pages, flush dirty pages (those that contain changed data) to disk followed by marking those pages non-dirty thus making those available for re-use.

If you think this through, it would also mean that the RSS of the process will not shrink if the kernel's page cache is re-used elsewhere (albeit, the file is still memory-mapped to the process, though it's data may no longer be in-memory). It would however make access to data in the memory-mapped file slower due to the need to re-populate the requested data to the kernel's page-cache.

As my system is currently down, I can't test this at the moment, but
Code:
pmap -x
or accessing
Code:
cat /proc/<pid>/maps
might shed some more light on your situation.
Back to top
View user's profile Send private message
Naib
Advocate
Advocate


Joined: 21 May 2004
Posts: 4491
Location: Removed by Neddy

PostPosted: Sat Oct 12, 2013 1:47 pm    Post subject: Reply with quote

Quote:

# free -m
total used free shared buffers cached
Mem: 32051 26577 5474 0 551 25178


5gig free
26gig used
25gig in cache

basically you have available, IF the system has to, 30gig.

whats the problem?

Also you say it is 32gig RAM but are using free -m (return in meg's)
_________________
The best argument against democracy is a five-minute conversation with the average voter
Great Britain is a republic, with a hereditary president, while the United States is a monarchy with an elective king
Back to top
View user's profile Send private message
1970
n00b
n00b


Joined: 07 May 2010
Posts: 46

PostPosted: Mon Oct 14, 2013 8:17 am    Post subject: Reply with quote

Thanks Atom2 for Your answer.

Currently it looks like so:

Code:

# free -m
             total       used       free     shared    buffers     cached
Mem:         32051      26422       5628          0        571      24990
-/+ buffers/cache:        860      31191
Swap:         2047          0       2046

# ps auxww |grep '\(informix\|USER\)' | grep -v grep
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
informix 11619  7.1 43.0 14254836 14134856 ?   R<   Sep14 3094:55 oninit -D
informix 11979 12.9 42.3 14255260 13890472 ?   S<   Sep14 5558:26 oninit -D
informix 11981  7.3 42.3 14255528 13887508 ?   S<   Sep14 3183:31 oninit -D

# pmap 11619
11619: oninit
START               SIZE     RSS     PSS   DIRTY    SWAP PERM MAPPING
0000000044000000 13555776K 13555536K 4670091K 13537928K      0K rwxs /SYSV52564801

Total:           14254840K 14134856K 4864083K 14104184K      0K

4224K writable-private, 53728K readonly-private, 14196888K shared, and 13889092K referenced


The output of pmap is shortened. I only posted the line with the extreme big RSS value, the other lines are all much smaller.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum