harddrive usage eats up memory

kortec · n00b Joined: 20 Dec 2004 Posts: 68

I just finished installing Gentoo on a box, and it seems like everything is going just fine -- got X looking great, sound working, and all that junk -- but there's an odd lingering bug.

Whenever there's a relatively large disk read, the memory utilization goes through the roof, and everything slows to a crawl. This could be as simple as loading the firefox binary if it isn't in RAM already, downloading a file that's bigger than about 1 MB, starting acroread, editing id3 tags on mp3s -- whatever. As long as the hard drive is spinning, this happens. (When I say 'memory utilization goes through the roof', I mean that I have half a gig of ram in here, and after disk IO starts, only about 9MB is free according to 'free -m').

Any ideas what might be causing this? I did check to make sure that DMA support was enabled in the kernel, and it is.

tobr · Guru Joined: 29 May 2006 Posts: 330

That sounds normal.

a) Hard drive access is (relatively) slow. If there’s a large read things can slow down.
b) because of a) Linux caches hard disk in RAM (because empty RAM is useless). This will be freed if an application needs it. free shows the cache (surprisingly) under “cached”, which you need to subtract from the used RAM if you want to know what your apps use.
_________________
Please add [SOLVED] to your message title if your problem is solved.

Death to all blobs!

gentoo_ram · Posted: Wed Jan 23, 2008 11:24 pm Post subject:

Yes, repeat after me:

"Free memory is wasted memory."

The kernel will use as much RAM as possible to cache disk accesses. This is a good thing. If the memory is needed by real programs, the kernel will free the disk cache. Don't worry about it unless you're running out of swap space too.

kortec · n00b Joined: 20 Dec 2004 Posts: 68

So lots of RAM usage is normal, and I do not pretend to understand the kernel's vm decisions, but the severe slow down isn't normal. Moving a 80 MB tar file from one directory to another shouldn't take 5 mins and make it impossible to move the mouse. The hardware isn't that old (athlon xp, 7200 RPM 250 GB hd, etc.). This isn't just in X either -- if I'm doing similar things just in a terminal, I can't switch between ttys or, if I background the process with a lot of disk usage, the tty takes too long to update from keyboard input to make anything possible.

eccerr0r · Posted: Thu Jan 24, 2008 12:14 am Post subject:

The problem is likely not memory usage, but rather DMA issues. Is DMA turned on for your hard drive?
Did you compile your disk controller's native driver into the kernel?

hdparm -t /dev/harddrive?
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

fenrisulfr · n00b Joined: 23 Jan 2008 Posts: 1

Is the drive on an addon controller card? SATA or IDE? What is the result of hdparm -t /dev/device? That choking on the drive is definitly not right. My little 233 laptop does just fine with read writes to the drive.
_________________
"It is not advisable to venture unsolicited opinions. You should spare yourself the embarrassing discovery of their exact value to your listener." -Francisco D' Anconia

kortec · n00b Joined: 20 Dec 2004 Posts: 68

I get this from hdparm:

kortec · n00b Joined: 20 Dec 2004 Posts: 68

Found the bug, I think. I recompiled my kernel with more specific/correct IDE chipset drivers, and got this output.

eccerr0r · Posted: Thu Jan 24, 2008 3:15 pm Post subject:

glad you got it working.

The weirdest symptoms can mislead the proper solution eh?
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

kortec · n00b Joined: 20 Dec 2004 Posts: 68

Seriously. The weirder thing is that I had tried to set the DMA on the drive earlier and got that very error and I barely noticed it. And then I had to talk through the really silly things I was noticing happening to get to any sort of a solution at all. Ah well.

bunder · Bodhisattva Joined: 10 Apr 2004 Posts: 5934