| View previous topic :: View next topic |
| Author |
Message |
Pistos
Apprentice
Joined: 29 Jul 2003
Posts: 175
Location: Canada
|
 Posted: Wed Jul 09, 2014 2:57 pm Post subject: |
 |
|
| eccerr0r wrote: | | the OOM killer will have a hard time selecting this application. This is the situation where the UI performace will suffer. Hence the curiousity: what is being run that's causing this issue? |
In a typical work day, I will have most or all of the following running:
- multiple Chromium windows, each with several tabs, maybe 3 or 4 of which with SPA ("Single-Page App") Javascript apps running in them for days (these tend to grow to 150 to 250 MB RAM usage each). Less beefy tabs still consume maybe 40 to 130 MB each, and I'll have probably 10 to 15 of these open.
- Mysql with production-level quantities of data (if left running, grows to at least 500-750 MB in a typical day, but can go over 2 GB under certain circumstances)
- Rails app along with multiple supporting processes, all amounting to about 5 to 8 processes, each 150-250 MB
- Firefox (multi-browser testing), which eats 200-350 MB
- Thunderbird - about 250 MB
- the main X windows process - about 700 MB
- KDE's processes, probably about 500 MB or so collectively
So, after a warmup of 48 hours or so after boot, at any given moment in a typical day, I am humming along at 70% RAM usage at the least, but usually hovering around more like 80 to 85% (of 8 GB). So, this generally leaves just a little breathing room for opening "just need it for a moment" apps like GIMP, libreoffice, running an emerge, watching something online via a Flash player instance, etc. I don't bother trying to do anything like video editing without gracefully ending other processes first.
So yeah, living at or near the edge of memory limits is the norm for me. The way you guys talk, it's as if you do not more than two or three things at once, and even then, with curses-based applications, or something.  I'm surely not so anomalous among power users? |
|
| Back to top |
|
 |
kernelOfTruth
Watchman
Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)
|
|
| Back to top |
|
|
Pistos
Apprentice
Joined: 29 Jul 2003
Posts: 175
Location: Canada
|
| Posted: Wed Jul 09, 2014 3:47 pm Post subject: |
|
|
@kernelOfTruth:
From what I can tell from years of experience, many different things in Linux slowly grow in memory. What leads you to conclude X is special here, that it is "the" one and only problem causing my system lockups? Based on what I can see, it is the low memory situation, period, regardless of what causes it, whether X, or MySQL, or kdenlive, or anything else, whether individually, or in aggregate.
And I've had these issues across multiple systems, regardless of whether the graphics are ATI, nvidia, or Intel on-board. |
|
| Back to top |
|
|
SlashBeast
Retired Dev
Joined: 23 May 2006
Posts: 2922
|
| Posted: Wed Jul 09, 2014 4:40 pm Post subject: |
|
|
Hai.
Tuning linux memory managment is a tricky, but here's what I think about your very case.
Linux, when flushing data to underlaying storage can lock the whole system and it does fit your description of how laggy system can be. By default linux start flushing when either in-memory data is old enough or you hit 10% of your memory, then, force flush when you hit 20% of memory.
My solutuon here is to alter the values to 50M and 100M. Use `sysctl` to set the values and save them later in /etc/sysctl.conf so the survive reboots.
| Code: | vm.dirty_bytes=104857600
vm.dirty_background_bytes=52428800 |
This will make your storage be used often, but not saturate it. blocking whole 'pipe'. Wanna test it? Put high capacity slowish usb2.0 pendrive and smash it with data, then after a few moments see that your system is instanly laggy, to make it even worst, try to umount it, which will force flushing all the dirty pages to it.
Next, if you play with IO schedulers, avoid NOOP at any cost, unless you have awesome underlaying stoage like hardware raid10 or netapp-provided FC storage or its VM with disk image on regular filesystem. If you have regular hardware raid, go with deadline, if you have single disk (whatever ssd or hdd), go with either CFQ or BFQ.
Swapping - by default linux will try to preserve 60% of ram memory for disk caches, thats may be not optimal for regular desktop with 8G of ram. I would tune it down to about 15, so via sysctl set
Next one, to avoid overcommit, simply, disable it and cover your system with swap. I have 16 GB swap partition on my 16 GB workstation.
To disable overcommit:
| Code: | vm.overcommit_memory=2
vm.overcommit_ratio=100 |
Also, how you check overall system usage? You should not count buffers and cached memory as 'used' one (excluding the cached that is in fact tmpfs). |
|
| Back to top |
|
|
Pistos
Apprentice
Joined: 29 Jul 2003
Posts: 175
Location: Canada
|
| Posted: Wed Jul 09, 2014 6:40 pm Post subject: |
|
|
@SlashBeast:
Thanks for the interesting and educational reply. Doing some websearching on /vm\..*dirty.*/, I came across this page, and based on that, and your reply, I have changed from
| Code: | vm.dirty_background_ratio = 5
vm.dirty_background_bytes = 0
vm.dirty_ratio = 10
vm.dirty_bytes = 0
|
to
| Code: | vm.dirty_background_ratio = 2
vm.dirty_background_bytes = 0
vm.dirty_ratio = 5
vm.dirty_bytes = 0
|
I will maybe stress test my system and see what happens.
And just a reminder that this system has no swap disk or partition. |
|
| Back to top |
|
|
kernelOfTruth
Watchman
Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)
|
| Posted: Wed Jul 09, 2014 7:06 pm Post subject: |
|
|
| Pistos wrote: | @kernelOfTruth:
From what I can tell from years of experience, many different things in Linux slowly grow in memory. What leads you to conclude X is special here, that it is "the" one and only problem causing my system lockups? Based on what I can see, it is the low memory situation, period, regardless of what causes it, whether X, or MySQL, or kdenlive, or anything else, whether individually, or in aggregate.
And I've had these issues across multiple systems, regardless of whether the graphics are ATI, nvidia, or Intel on-board. |
Pistos, I only view it as one of the issues leading to high memory pressure
talking about it:
I wonder why no one mentioned it
if you're NOT using any swap
try raising /proc/sys/vm/vfs_cache_pressure to
10000 or even 100000
I've seen much issues with low or even default values (100) with that setting
also try setting
cat /proc/sys/vm/min_free_kbytes
131072
or to 65536 (if that's not already the default value)
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa
Hardcore Gentoo Linux user since 2004  |
|
| Back to top |
|
|
Pistos
Apprentice
Joined: 29 Jul 2003
Posts: 175
Location: Canada
|
| Posted: Wed Jul 09, 2014 7:16 pm Post subject: |
|
|
@kernelOfTruth:
My current values are:
| Code: | # cat /proc/sys/vm/vfs_cache_pressure
100
# cat /proc/sys/vm/min_free_kbytes
11456 |
I will try tweaking those in a few days, after testing my recent changes to vm.dirty*. Thanks for pointing these two other ones out. |
|
| Back to top |
|
|
SlashBeast
Retired Dev
Joined: 23 May 2006
Posts: 2922
|
| Posted: Wed Jul 09, 2014 7:37 pm Post subject: |
|
|
| @Pistos If your storage cannot handle write at 160 MB/s I woudn't set dirty dirty background to 2, you can set exact amount of memory you want to preserve for said purpose with the _bytes one. |
|
| Back to top |
|
|
Pistos
Apprentice
Joined: 29 Jul 2003
Posts: 175
Location: Canada
|
| Posted: Fri Aug 15, 2014 6:56 pm Post subject: |
|
|
Update on this:
Since my last post, there have been maybe 3 to 5 times where memory started getting low, and I was able to just manually close programs or TERM processes, etc. the usual way, before things got out of hand.
However, today, I hit the problem again, and it struck quickly enough that I couldn't respond in time. However, for the first time I went to try the SysRq solution(s). Reading some documentation from another computer, I used Alt-SysRq-R to set the keyboard mode (whatever that means), and then Alt-SysRq-F to manually call the OOM Killer. About 10 seconds later, my system recovered, and /var/log/messages recorded that a Chromium process was dutifully killed, as requested. Awesome!
Now, I don't know if I need the SysRq-R, so next time, I will try to go straight for the SysRq-F. |
|
| Back to top |
|
|
Pistos
Apprentice
Joined: 29 Jul 2003
Posts: 175
Location: Canada
|
| Posted: Tue Aug 19, 2014 7:56 pm Post subject: |
|
|
Manually invoked the OOM killer again via SysRq-F . Worked like a charm.
So the question is: Why is the OOM killer not automatically activating under these conditions? |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Other Things Gentoo
