AMD64 system slow/unresponsive during disk access...

Message

Captain Newbie · Post by **Captain Newbie** » Tue Jun 09, 2009 3:03 pm

lagalopex wrote:They perhaps found something: http://bugzilla.kernel.org/show_bug.cgi?id=12309#c360

Hrm, I'll have to try that in my Copious Free Time™.

nero37 · Post by **nero37** » Wed Jun 10, 2009 12:46 pm

Linus has released kernel 2.6.30;
http://thread.gmane.org/gmane.linux.kernel/849020

Readable change log;
http://kernelnewbies.org/Linux_2_6_30

darc · Post by **darc** » Fri Jun 12, 2009 8:53 am

For me 2.6.30 seems to be worst. When I try to copy someting on pendrive I've got like max 2mb/s which drops every second till it reaches 0b/s and then it freezes for a second and starting again at 2mb/s. And it's happening over and over till copying is finished.

LoSeR_5150 · Post by **LoSeR_5150** » Sat Jun 13, 2009 4:34 pm

I've only done limited testing thus far linux-2.6.30 is the best for me so far. The rc's of 2.6.30 were an improvement but this feels even better. I can compile, burn a cd, copy a 700mb file, play mp3, as well as ssh'ing into various machines with no mouse lag. I'll do some more testing when I can.

kernelOfTruth · Post by **kernelOfTruth** » Sat Jun 13, 2009 4:53 pm

I'm not impressed with 2.6.30 it still lags the same during heavy I/O with rsync compared to 2.6.29

Sujao · Post by **Sujao** » Sat Jun 13, 2009 5:37 pm

Just compiled 2.6.30-gentoo-r1. Tested only one thing that was really annoying me and it is still there. I start mmg (the muxer gui for the Matroska container that uses mkvmerge) and remux a 6.9GB file. I choose the movie and just let mkvmerge read it and write it again to hdd (from xfs to xfs) After about 15s the system locks up as usual.

So maybe other minor things are solved but the main problem is unchanged.

Captain Newbie · Post by **Captain Newbie** » Sun Jun 14, 2009 3:15 pm

Sujao wrote:Just compiled 2.6.30-gentoo-r1. Tested only one thing that was really annoying me and it is still there. I start mmg (the muxer gui for the Matroska container that uses mkvmerge) and remux a 6.9GB file. I choose the movie and just let mkvmerge read it and write it again to hdd (from xfs to xfs) After about 15s the system locks up as usual.

So maybe other minor things are solved but the main problem is unchanged.

Could it be that genpatches is not helping?

For me, vanilla 2.6.30 is at least an improvement - lower I/O wait than pre-.30, but I still see processes diskbound. System responsiveness is better.

Again, I think knowing what your hardware and .config are like would at least be interesting.

devilheart · Post by **devilheart** » Thu Jun 18, 2009 9:56 am

did anyone try app-benchmarks/piozone ? every time I use this tool everything else is frozern (iowait around 10%)

Lepaca Kliffoth · Post by **Lepaca Kliffoth** » Thu Jun 18, 2009 5:23 pm

All of the problems I had are back with gentoo-sources-2.6.30, I'm back on 2.6.30-rc7-git3 and I'm compiling the latest vanilla.

Sujao · Post by **Sujao** » Fri Jun 19, 2009 1:23 pm

So you are suggesting some gentoo patch is causing the problem?

Lepaca Kliffoth · Post by **Lepaca Kliffoth** » Sat Jun 20, 2009 8:36 am

I have no fucking idea, it takes too damn long to test this stuff

DigitalCorpus · Post by **DigitalCorpus** » Sun Jun 21, 2009 10:04 am

Lepaca Kliffoth wrote:I have no fucking idea, it takes too damn long to test this stuff

I'm right there with you

Jorgo · Post by **Jorgo** » Sun Jun 21, 2009 2:52 pm

Same difficulties here. I tried vanilla-sources 2.6.30 and its much better than before, but not solved.
I wondered if it could be a special gentoo patch. Therefore i tried the vanilla-source instead of the gentoo-sources.

Yamakuzure · Post by **Yamakuzure** » Tue Jun 23, 2009 11:29 am

My ideas with the latency and timer configs (300Hz instead of 1kHz and voluntary instead of low-latency) where a dud. The system worked alot better for some days, but nothing more.

And with the 2.6.30 kernel nothing works better or with lower latencies.

*but* awakening processes out of wait state via num lock in top now works!

(But I can't have a console with top open all day, can I?)

I'll try data=writeback for /dev/root next...

Yzzyx · Post by **Yzzyx** » Tue Jun 23, 2009 6:05 pm

I havent follow this thread so I don't know if anyone have suggested this yet.
I had massive problems with 64bit and bad IO. My solution was to turn off optimize for size. With it one on, jerky poor I/O. Without optimized for size... smooth and predictable. Dunno why the kernel default is with optimize for size on. It has give me nothing but problems.

Yamakuzure · Post by **Yamakuzure** » Wed Jun 24, 2009 10:40 am

No, I never use "optimize for size".

...but I tried two thinks yesterday which *really* seem to help!

Changed mount options for /dev/root (which is ext3 on my system) to "data=writeback" (That need an additional bootoption: "rootflags=data=writeback")
I installed the most recent (~amd64) version of laptop-mode-tools, Version 1.47-r1.
There, in /etc/laptop-mode/laptop-mode.conf I set up the following values for dirty-bytes-handling, and now all problems are greatly reduced (not gone, though...) whenever the AC is plugged in:
Code: Select all
```
# grep -i _dirty /etc/laptop-mode/laptop-mode.conf
LM_DIRTY_RATIO=20
NOLM_DIRTY_RATIO=10
LM_DIRTY_BACKGROUND_RATIO=1
NOLM_DIRTY_BACKGROUND_RATIO=2
```

this reminds me: Could it be, that most (or all) of us having these heavy disk access problems work with laptops?

I ask, because on both my other desktop machines, I can easily do everything I want while emerging something large, while, until yesterday, emerging anything caused the laptop system to nearly freeze.
The differences are, that both desktop systems have video cards with dedicated memory, while the laptop has an integrated chipset with shared video memory. Both desktops have more than one HD, while the laptop does everything on a single HD.

All three systems work with almost the same software configuration on 2.6.30 kernel...

stevegriffdtcom · Post by **stevegriffdtcom** » Wed Jun 24, 2009 10:43 am

Sorry for jumping into this discussion way to late, I can confirm here too that 2.6.30 has made a detrimental effect with desktop responsiveness regarding high i/o. Video and Audio suffers from more bad stutter, plus everything just goes to a sludge.

Oh well.

Yzzyx · Post by **Yzzyx** » Wed Jun 24, 2009 10:56 am

Yamakuzure wrote: Could it be, that most (or all) of us having these heavy disk access problems work with laptops?

I have only ran 64bit on workstations and servers. My laptops run OSX, Ubuntu and Gentoo 32bit. (BTW. One of my laptops is a Toshiba Protege 3010. and it is _thinner_ than a mac air! I have measured... to bad is just 300mhz and 900MB RAM.

Yamakuzure · Post by **Yamakuzure** » Wed Jun 24, 2009 12:02 pm

Yzzyx wrote:
Yamakuzure wrote: Could it be, that most (or all) of us having these heavy disk access problems work with laptops?
I have only ran 64bit on workstations and servers.

Rats. Then this is another useless trace, it has nothing to do with the hardware layout.

Well, it was an idea at least...

devsk · Post by **devsk** » Wed Jun 24, 2009 10:04 pm

Yamakuzure wrote: *but* awakening processes out of wait state via num lock in top now works!

Can you please elaborate on that? How is this supposed to work? Just numlock in a window with 'top' running? is it?

Yamakuzure · Post by **Yamakuzure** » Thu Jun 25, 2009 9:18 am

devsk wrote:
Yamakuzure wrote: *but* awakening processes out of wait state via num lock in top now works!
Can you please elaborate on that? How is this supposed to work? Just numlock in a window with 'top' running? is it?

It has been your own post: XD

devsk wrote:someone claimed that they hit numlock to get their processes out of 'D' (disk sleep, directly relates to the IO Wait % number you see in the top output) state. Its bizarre!!

http://lkml.org/lkml/2009/4/5/197

On my laptop it does not work when pressing numlock on an attached USB keyboard, I have to use the laptops own keys for that. (aka PS2)

btw.: While I am writing this, everything works normal while emerge is compiling mesa. A few days ago, it would have been impossible to move the mouse or write anything during emerging, so the two changes described above really help!

My current vm status:

Code: Select all

# for file in $(ls /proc/sys/vm/*dirty*) ; do echo -n "${file} : " ; cat ${file} ; done
/proc/sys/vm/dirty_background_bytes : 0
/proc/sys/vm/dirty_background_ratio : 2
/proc/sys/vm/dirty_bytes : 0
/proc/sys/vm/dirty_expire_centisecs : 3000
/proc/sys/vm/dirty_ratio : 10
/proc/sys/vm/dirty_writeback_centisecs : 500

Edit: And after finishing it emerge has moved over to xorg-server and there was not a single lag in mouse movement, firefox scrolling, window switching, nothing!

devsk · Post by **devsk** » Thu Jun 25, 2009 9:41 am

LOL. yeah, I know but it never really worked for me. I thought I wasn't doing it right. And once I heard you used it, I was curious.

Yamakuzure · Post by **Yamakuzure** » Thu Jun 25, 2009 1:15 pm

devsk wrote:LOL. yeah, I know but it never really worked for me. I thought I wasn't doing it right. And once I heard you used it, I was curious.

I guess the thing is, that top does not react if the keyboard isn't PS2. Using my USB keyboard did nothing, but someone hinted about PS2-kb and that eventually worked.

Lepaca Kliffoth · Post by **Lepaca Kliffoth** » Sat Jun 27, 2009 7:52 am

I'm narrowing it down quite quickly now. With 2.6.30-zen1 (zen-sources) the mouse almost stops moving if and only if I transfer a file over the wireless network. I have this problem when I transfer files using samba and sftp, so I'm going to assume it's the fault of the wireless card driver. I am using the driver provided by madwifi-ng (atheros chipset), so the next thing I'm going to do is to unmerge that driver and use the one included in the kernel (ath5k). I won't elaborate on why I was using madwifi as it's offtopic and irrelevant.

edit: recompiled, rebooted and switched to the new driver. In a few minutes I'm going to do the copy-shit-over-the-network test and see....

edit2: the problem is only a little better. Well, actually 200% better, but the mouse doesn't move smoothly at all.

kernelOfTruth · Post by **kernelOfTruth** » Sat Jun 27, 2009 9:10 am

Lepaca Kliffoth, there are more factors involved in this problem:

for me it's actually the other way around:

- copying large files from one hdd to another
-> results in 100% halt of network-card

-> often also the mouse-pointer stops reacting and even the sound (if streaming on the net, of course - if playing locally it also stops)

in several situations if you raise the priority via renice of the affected apps sound and the mouse-pointer continue running/working

it pretty much looks like a bottleneck in the I/O subsystem combined with cpu-scheduler issues ...