View previous topic :: View next topic |
Author |
Message |
suineg Apprentice
Joined: 02 Mar 2004 Posts: 200 Location: Los Angeles
|
Posted: Mon Feb 28, 2005 2:53 am Post subject: memory leak in portage? |
|
|
I think I may have found a memory leak in portage, I noticed it for sure with an emerge sync, not sure about the rest of portage.
basically, I am running nothing but xfce4, it consumes about 14% of my memory (512) with gdm running etc. then today after an emerge sync, it jumped up to 19% and stayed there, a few hours later I decided to test my theory by running another emerge sync, this time it jumped up to 33% and still refuses to go down.
whats going on here!? is there a leak in portage? or do I have some other problem with my system.
I am just using torsmo for my readings, which is pretty accuarate afaik.
I can't stand wasted memory
any ideas? |
|
Back to top |
|
|
_never_ Apprentice
Joined: 10 Jun 2004 Posts: 285 Location: BW, Germany
|
Posted: Mon Feb 28, 2005 3:10 am Post subject: |
|
|
Don't worry, this isn't wasted memory. =)
Linux tries to "remember" as much as possible to speed up hardware access. It has disk caches and other stuff, which all count to used memory. It is freed as soon as other, more recent data arrives. If you want Linux to try to have a constant amount of free memory at all times, then you might want to echo some numeric value to /proc/sys/vm/min_free_kbytes:
Code: | echo 1024 > /proc/sys/vm/min_free_kbytes |
But be careful. You can easily drop Linux into an endless loop, if you set this value too high. If you want to read the current value, use this command:
Code: | cat /proc/sys/vm/min_free_kbytes |
You might also play around with /proc/sys/vm/swappiness. This value controls (in percent), how much Linux likes to swap data. The higher, the more gets into swap and the later it gets out again. The lower, the less Linux likes to swap and it gets back to real memory sooner. Be warned: never set this to 0, else your system will crash sooner or later - as soon as Linux has to swap something. _________________ Knowledge is Power. |
|
Back to top |
|
|
SerfurJ l33t
Joined: 10 Apr 2004 Posts: 824 Location: Texas
|
Posted: Mon Feb 28, 2005 3:13 am Post subject: |
|
|
if there's a memory leak, it would be freed up when the process is done. you're not looking at cached memory are you? look at the second row of output from "free". |
|
Back to top |
|
|
suineg Apprentice
Joined: 02 Mar 2004 Posts: 200 Location: Los Angeles
|
Posted: Mon Feb 28, 2005 3:41 am Post subject: |
|
|
It's not disk cache, because torsmo automatically subtracts your disk cache to get the actual used memory. All processes relating to portage that I can tell, are finished yet I still have this excess used memory. |
|
Back to top |
|
|
suineg Apprentice
Joined: 02 Mar 2004 Posts: 200 Location: Los Angeles
|
Posted: Mon Feb 28, 2005 3:43 am Post subject: |
|
|
I checked free just to be sure and sure enough, if I had been looking at cached memory, my memory usage would be something more like 60% |
|
Back to top |
|
|
_never_ Apprentice
Joined: 10 Jun 2004 Posts: 285 Location: BW, Germany
|
Posted: Mon Feb 28, 2005 3:47 am Post subject: |
|
|
suineg wrote: | It's not disk cache, because torsmo automatically subtracts your disk cache to get the actual used memory. All processes relating to portage that I can tell, are finished yet I still have this excess used memory. |
If this was true, my whole memory would be leaked. =) _________________ Knowledge is Power. |
|
Back to top |
|
|
suineg Apprentice
Joined: 02 Mar 2004 Posts: 200 Location: Los Angeles
|
Posted: Mon Feb 28, 2005 3:52 am Post subject: |
|
|
_never_ wrote: | suineg wrote: | It's not disk cache, because torsmo automatically subtracts your disk cache to get the actual used memory. All processes relating to portage that I can tell, are finished yet I still have this excess used memory. |
If this was true, my whole memory would be leaked. =) |
This actualy IS true, and can be enabled/disabled from your .torsmorc with this option:
Code: | # Subtract file system buffers from used memory?
no_buffers yes
|
so I am most positive I am seeing the correct numbers, plus using free to verify the torsmo numbers, subtracting disk cache myself, it would seem that torsmo is accurate. |
|
Back to top |
|
|
_never_ Apprentice
Joined: 10 Jun 2004 Posts: 285 Location: BW, Germany
|
Posted: Mon Feb 28, 2005 4:04 am Post subject: |
|
|
If you are sure, then look at top (as root, if you use grsecurity) and identify the process that is leaking so much memory. Maybe it's your terminal emulator or the X server - or torsmo itself. =D
Well, serious: If it's really not the cache, try to identify the process that is responsible for it. If it's your terminal emulator, then everything is fine - in that case it's the scrollback. Sort by RAM usage in top and watch it while a emerge sync is running. _________________ Knowledge is Power. |
|
Back to top |
|
|
suineg Apprentice
Joined: 02 Mar 2004 Posts: 200 Location: Los Angeles
|
Posted: Mon Feb 28, 2005 4:21 am Post subject: |
|
|
okay, I did as you suggested, and my terminal mem usage did not increase during the sync at all. emerge jumps way up to about 8% this time however all but 1% of the increase was freed.
at this point you might think i'm crazy, but i don't know where else this memory could be going to , I definately noticed the fact that it didn't drop back down the last two times I did an emerge sync (now three), I will try it again as soon as portage lets me |
|
Back to top |
|
|
SerfurJ l33t
Joined: 10 Apr 2004 Posts: 824 Location: Texas
|
Posted: Mon Feb 28, 2005 4:40 am Post subject: |
|
|
if you start "top", then press "M", you can see very easily what's using memory under the "RES" column (resident memory). |
|
Back to top |
|
|
_never_ Apprentice
Joined: 10 Jun 2004 Posts: 285 Location: BW, Germany
|
Posted: Mon Feb 28, 2005 4:48 am Post subject: |
|
|
Maybe it was the cache. And now everything is already in the cache, so it doesn't need to load it again. =)
There is something you could do to test it: try to fill the memory with something. For example with this command:
Code: | (dd if=/dev/zero bs=1M count=400; read) | sort |
This forcibly loads 400 MB into memory and keeps it there until the return key is pressed. If it is the cache, then at some point, when you increase the count= value, the memory usage will not increase. This means, that the kernel began forgetting the cache contents, as they are never swapped to disk. Once you press the return key, your memory then drops back to an even lower value than before, because the cache entries disappeared - if it's the cache.
Another way to test it: grab some large file, which wouldn't fit into your memory, let's say movie.avi and do the following:
Code: | cat movie.avi > /dev/null |
This will do nothing more than loading the file into the cache. Look at torsmo while this is done. Does the memory usage increase? It shouldn't, if the cache is not taken into account. If you don't have such a large file, create it:
Code: | dd if=/dev/urandom of=movie.avi bs=1M count=600 |
Hopefully this answers your questions. _________________ Knowledge is Power. |
|
Back to top |
|
|
suineg Apprentice
Joined: 02 Mar 2004 Posts: 200 Location: Los Angeles
|
Posted: Mon Feb 28, 2005 5:04 am Post subject: |
|
|
right, so according to top, I am using ~136mb, while torsmo/free claim that I am using 189mb, there seems to be a missing 11+ percent there.
if I close firefox, which I didn't have running when I read the 19% usage, I am using about 19% plus I have gaim conversations open which I didn't before so that could easily account for the extra ~4mb I am missing (I think). so this would seem about right.
everything that I am seeing points me to believe that I have lost at least 11% of my ram to nothingness. |
|
Back to top |
|
|
suineg Apprentice
Joined: 02 Mar 2004 Posts: 200 Location: Los Angeles
|
Posted: Mon Feb 28, 2005 5:10 am Post subject: |
|
|
_never_ wrote: | Maybe it was the cache. And now everything is already in the cache, so it doesn't need to load it again. =)
There is something you could do to test it: try to fill the memory with something. For example with this command:
Code: | (dd if=/dev/zero bs=1M count=400; read) | sort |
This forcibly loads 400 MB into memory and keeps it there until the return key is pressed. If it is the cache, then at some point, when you increase the count= value, the memory usage will not increase. This means, that the kernel began forgetting the cache contents, as they are never swapped to disk. Once you press the return key, your memory then drops back to an even lower value than before, because the cache entries disappeared - if it's the cache.
Another way to test it: grab some large file, which wouldn't fit into your memory, let's say movie.avi and do the following:
Code: | cat movie.avi > /dev/null |
This will do nothing more than loading the file into the cache. Look at torsmo while this is done. Does the memory usage increase? It shouldn't, if the cache is not taken into account. If you don't have such a large file, create it:
Code: | dd if=/dev/urandom of=movie.avi bs=1M count=600 |
Hopefully this answers your questions. |
okay I tried your first suggestion, my memory usage jumps up to 91%, then after I quit, jumps back down to 18% which seems about right for my running system. This would indicate that it IS the disk cache as you had guess.
The strange this is that I just very recently noticed this, and I know that torsmo, when configured to do so is supposed to subtract disk cache.
so I am going to assume that the problem here is perhaps the latest version of torsmo.
I will look around and see if anyone else has been having such a problem with torsmo, and post back if I don't find anything.
thanks for all the help |
|
Back to top |
|
|
_never_ Apprentice
Joined: 10 Jun 2004 Posts: 285 Location: BW, Germany
|
Posted: Mon Feb 28, 2005 5:18 am Post subject: |
|
|
You're welcome.
One source for this problem could be the kernel processes like pdflush and the various processes for your filesystem types. But they don't seem to consume any userspace memory - their memory consuption seems to be shown as cache. I don't know exactly.
Edit: as cache or as buffers. _________________ Knowledge is Power.
Last edited by _never_ on Mon Feb 28, 2005 5:20 am; edited 1 time in total |
|
Back to top |
|
|
suineg Apprentice
Joined: 02 Mar 2004 Posts: 200 Location: Los Angeles
|
Posted: Mon Feb 28, 2005 5:19 am Post subject: |
|
|
so maybe a problem with reiser4 not properly showing disk cache as cache? |
|
Back to top |
|
|
_never_ Apprentice
Joined: 10 Jun 2004 Posts: 285 Location: BW, Germany
|
Posted: Mon Feb 28, 2005 5:22 am Post subject: |
|
|
Well, I guess, it's not a problem at all. =)
I don't know how stable ReiserFS 4 is. Still on ReiserFS 3 here - I'm yet too paranoid for the next version. =) _________________ Knowledge is Power. |
|
Back to top |
|
|
suineg Apprentice
Joined: 02 Mar 2004 Posts: 200 Location: Los Angeles
|
Posted: Mon Feb 28, 2005 5:29 am Post subject: |
|
|
I can't easily check an older version of torsmo as version .17 won't compile on my machine anymore for some reason.
it seems unlikely that it would be torsmo anyway
if it were a reiser4 problem, would it be related to the kernel driver or something from the reiser4progs package?
I haven't updated reiser4progs in forever (if ever since I last installed).
but I have switched kernels recently.
one thing I do know for certain is that I didn't used to have this problem, so there MUST be a way to fix it |
|
Back to top |
|
|
suineg Apprentice
Joined: 02 Mar 2004 Posts: 200 Location: Los Angeles
|
Posted: Mon Feb 28, 2005 5:31 am Post subject: |
|
|
_never_ wrote: | Well, I guess, it's not a problem at all. =)
I don't know how stable ReiserFS 4 is. Still on ReiserFS 3 here - I'm yet too paranoid for the next version. =) |
heh, i have been using reiser4 since before it was officially released, and have had no problems whatsoever (aside from this sortof non-problem ).
but I know some people have, so I guess that makes me lucky.
and no, I guess it wouldn't REALLY be a problem, but more like an irritation |
|
Back to top |
|
|
_never_ Apprentice
Joined: 10 Jun 2004 Posts: 285 Location: BW, Germany
|
Posted: Mon Feb 28, 2005 5:34 am Post subject: |
|
|
suineg wrote: | if it were a reiser4 problem, would it be related to the kernel driver or something from the reiser4progs package?
I haven't updated reiser4progs in forever (if ever since I last installed). |
It may be the progs, if you ever used them. But mostly it's the driver. _________________ Knowledge is Power. |
|
Back to top |
|
|
suineg Apprentice
Joined: 02 Mar 2004 Posts: 200 Location: Los Angeles
|
Posted: Mon Feb 28, 2005 8:37 am Post subject: |
|
|
okay, still trying to remedy this, I ran another test, I rebooted, check my ram with free -m. and under the +- buffers/cache I get 93mb. then I run an emerge sync, and check it after it finishes, and my new value, under the same section is 194.
I believe I have the latest reiser4 patches, so I'm really stumped.
not trying to wear you out, but I really would like this "problem" fixed |
|
Back to top |
|
|
_never_ Apprentice
Joined: 10 Jun 2004 Posts: 285 Location: BW, Germany
|
Posted: Mon Feb 28, 2005 8:46 am Post subject: |
|
|
This is totally normal. The buffers get filled and filled, until they reach the memory limits. But as I said, they never get swapped out. Don't worry, this is not taking any memory. If memory is needed, then the oldest records in the buffers/cache are deleted. _________________ Knowledge is Power. |
|
Back to top |
|
|
suineg Apprentice
Joined: 02 Mar 2004 Posts: 200 Location: Los Angeles
|
Posted: Mon Feb 28, 2005 8:56 am Post subject: |
|
|
_never_ wrote: | This is totally normal. The buffers get filled and filled, until they reach the memory limits. But as I said, they never get swapped out. Don't worry, this is not taking any memory. If memory is needed, then the oldest records in the buffers/cache are deleted. |
I know that buffer swapping etc is normal and all that, just to make sure I wasn't mistaken I refreshed my memory using this
https://forums.gentoo.org/viewtopic.php?t=175419&highlight=memory
which is a pretty good post.
according to that, you can read the number minus the biffers and cache, which should be the value actually being used by running processes etc, using free -m and checking out the +/- buffers/cache part.
that is the number i'm reading, and it still appears as if there were nothing wrong.
If it were just a buffer/cache problem, wouldn't other programs have this "problem"?
i've tested almost every app that I have and none of them have this same problem.
I know this may not be important, but its nice to see how much ram is actually free, so that for example when I go to run a game, I need to know whether or not I should close some of the apps I have open or whatever. |
|
Back to top |
|
|
_never_ Apprentice
Joined: 10 Jun 2004 Posts: 285 Location: BW, Germany
|
Posted: Mon Feb 28, 2005 9:05 am Post subject: |
|
|
suineg wrote: | according to that, you can read the number minus the biffers and cache, which should be the value actually being used by running processes etc, using free -m and checking out the +/- buffers/cache part. |
Exactly. But remember that kernel processes are also shown.
suineg wrote: | that is the number i'm reading, and it still appears as if there were nothing wrong. |
There is nothing wrong. =)
suineg wrote: | If it were just a buffer/cache problem, wouldn't other programs have this "problem"? |
That's the point. It isn't a problem.
suineg wrote: | I know this may not be important, but its nice to see how much ram is actually free, so that for example when I go to run a game, I need to know whether or not I should close some of the apps I have open or whatever. |
Don't worry about this. Active programs always get the highest memory priority. So if there should ever swapping be needed, the game gets more memory than any other process, since the other processes are idle (if they are). And with idle I don't mean the CPU power used. For example the X server uses lots of RAM, but most of it is swapped out all the time, because the server doesn't actually use it. _________________ Knowledge is Power. |
|
Back to top |
|
|
suineg Apprentice
Joined: 02 Mar 2004 Posts: 200 Location: Los Angeles
|
Posted: Mon Feb 28, 2005 9:07 am Post subject: |
|
|
okay I guess I will have to be content with that.
thanks again. |
|
Back to top |
|
|
_never_ Apprentice
Joined: 10 Jun 2004 Posts: 285 Location: BW, Germany
|
Posted: Mon Feb 28, 2005 9:13 am Post subject: |
|
|
suineg wrote: | okay I guess I will have to be content with that.
thanks again. |
Finally! =D
No problem. _________________ Knowledge is Power. |
|
Back to top |
|
|
|