Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
too many "buffers" in kernel
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
HELPcho
n00b
n00b


Joined: 10 Jun 2006
Posts: 9
Location: Plovdiv

PostPosted: Tue Aug 25, 2009 8:36 am    Post subject: too many "buffers" in kernel Reply with quote

Hello,
I have a strange situation with on of my servers. I have two servers for image serving with let say 10 milion images and two servers with 32G ram each. First to say what is expected, buffers about 3G, and cached about 23-25G. The software was the same on the machines like glibc 2.6.1 and gcc 4.1.2, the hardware also 3ware controller and 2xcpus xeons. Serving round robin the content with nginx. So the first server has 3G buffers, and 25G cached and is cool, no problem with I/O, the second server is IO starving and the RAM is 14G "buffers" and 15G "cached". Apparently the second server needs more ram for "cached", but I cannot figure out why this is happening, why the first fills up to 3G "buffers" and everything else goes to cached, but the second fills above 4 times more "buffers". The only great difference is the kernel version - the first is with old 2.6.22.18 kernel, and the second is 2.6.29.1 (the same with 2.6.30).

First server:
total used free shared buffers cached
Mem: 32964644 32806120 158524 0 3001732 23770288

~ # cat /proc/sys/fs/inode-state
4971217 852167 0 0 0 0 0
~ # cat /proc/sys/fs/dentry-state
4215476 4193181 45 0 0 0


Second server:
total used free shared buffers cached
Mem: 32944792 21438124 11506668 0 8205736 10484772

~ # cat /proc/sys/fs/inode-state
1659561 42 0 0 0 0 0
~ # cat /proc/sys/fs/dentry-state
1736219 1718366 45 0 0 0

At the moment the socond server is still caching, because the I used echo 3 > drop_caches to clear all the cache, but when the RAM fills up it the ratio for the buffers/cached will be 14G/15G or 14/16G.
Back to top
View user's profile Send private message
Akkara
Bodhisattva
Bodhisattva


Joined: 28 Mar 2006
Posts: 6702
Location: &akkara

PostPosted: Tue Aug 25, 2009 11:41 am    Post subject: Reply with quote

The only time I've seen a lot of 'buffers' being used, is when doing raw I/O to a device, without going through a filesystem. Is there something on the second server doing this kind of raw I/O?
Back to top
View user's profile Send private message
HELPcho
n00b
n00b


Joined: 10 Jun 2006
Posts: 9
Location: Plovdiv

PostPosted: Tue Aug 25, 2009 6:12 pm    Post subject: Reply with quote

No, on the server is running just nginx and nfs, there is no special device. Just 3ware controller in RAID10 with stripe size 16K. Ammm, I checked and the stripe size to the raid10 on the first server is 64K, but I don't think this could cause the strange behavour.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum