View previous topic :: View next topic |
Author |
Message |
o5gmmob8 Guru
Joined: 17 Oct 2003 Posts: 465
|
Posted: Tue Oct 18, 2016 2:05 am Post subject: OOM even though I have available swap space |
|
|
Hi all,
I am trying to sort out why my system is raising an OOM exception and ultimately kills the running task. I have 4GB of RAM, with a 4GB swap partition. When I run free, I see that about 3.3GB are used for buffers / cache. If my system is really low on "free" memory, it could discard the buffers or cache, but it doesn't.
Also, my root filesystem is squashfs with a TMPFS overlay for read-write functionality. I'm not sure if that may have some influence.
Thanks, |
|
Back to top |
|
|
Ant P. Watchman
Joined: 18 Apr 2009 Posts: 6920
|
Posted: Tue Oct 18, 2016 2:18 am Post subject: |
|
|
tmpfs occupies page cache space. Check the output of `findmnt --df`, dmesg should have plenty of details when the OOM happens too. |
|
Back to top |
|
|
o5gmmob8 Guru
Joined: 17 Oct 2003 Posts: 465
|
Posted: Tue Oct 18, 2016 10:09 am Post subject: |
|
|
Thanks, I did not know that.
Well, I checked that and so I only have allocated a tmpfs size of 256M for the /rw volume and of that, 144M is used. My root image is about 900M for this system as it is headless, maybe I should try loading the squashfs image into memory and seeing if that helps or hurts.
Walter |
|
Back to top |
|
|
o5gmmob8 Guru
Joined: 17 Oct 2003 Posts: 465
|
Posted: Wed Oct 19, 2016 6:00 pm Post subject: |
|
|
I think I may know what the issue is ... I had a sysctl setting vm.drop_caches = 0 and once I changed that to 1, that freed up a bunch of space.
I was under the general impression that buffers / caches would be freed if needed; however, after reading the kernel docs for that setting, that apparently is not the case.
So, I will try to load up my system now and see what happens. I suspect that it will work as I originally expected it to, it will clear the buffers / cache when it needs memory.
Thanks,
Walter |
|
Back to top |
|
|
o5gmmob8 Guru
Joined: 17 Oct 2003 Posts: 465
|
Posted: Wed Oct 19, 2016 6:40 pm Post subject: |
|
|
I spoke too soon, why initially setting it freed up memory, after I ran git annex sync, it soon ran OOM again. Even after git annex finished, the buffers / caches is still high. It doesn't seem to make any sense.
Walter |
|
Back to top |
|
|
o5gmmob8 Guru
Joined: 17 Oct 2003 Posts: 465
|
Posted: Wed Oct 19, 2016 8:28 pm Post subject: |
|
|
Okay, so now, I think I may have solved or at least render the problem less of a problem. I set this value
vm.zone_reclaim_mode=5
This seems to work as I have been git-annex now for a while and I haven't gotten an OOM yet. Additionally, the memory fluctuates, but the free # is at 150M .
Thanks,
Walter |
|
Back to top |
|
|
o5gmmob8 Guru
Joined: 17 Oct 2003 Posts: 465
|
Posted: Fri Nov 11, 2016 2:37 am Post subject: |
|
|
Hi,
Even with the zone reclaim mode set at 5, I still get OOM. Is there anything more I can do to get the system to use more swap rather than throw an OOM exception and kill tasks?
Swap is being used, but only < 100 M out of 4GB I allocated.
Walter |
|
Back to top |
|
|
szatox Advocate
Joined: 27 Aug 2013 Posts: 3103
|
Posted: Sun Nov 13, 2016 2:41 pm Post subject: |
|
|
TMPFS counts towards buffer/cache that can't be discarded when you run out of memory. They are not backed with persistent memory, so all pages used are dirty.
You can increase swappiness though. The allowed values range from 0 (don't use swap unless RAM is completely filled up) to 100 (use as much as possible).
Code: | sysctl vm.swappiness |
|
|
Back to top |
|
|
o5gmmob8 Guru
Joined: 17 Oct 2003 Posts: 465
|
Posted: Mon Jan 02, 2017 2:50 am Post subject: OOM even though I have available swap space [SOLVED] |
|
|
Hi,
I think the problem was from a bug in the kernel. I haven't had this specific issue since upgrading to 4.8.13 or newer.
Walter |
|
Back to top |
|
|
|