AMD64 system slow/unresponsive during disk access...

[cz0]

Oh! You gays fritend me!.. But good news. I'm not having any of explaned problems.
Here is my conf:
AMD 64 x2 4600+, ASUS A8N32-SLI, 512 + 512 MB Corsair, ASUS 7900GT Top, Seagate SATA 120G firs g. and 2 workin DVDs on PATA.
GCC 4.1.1, xorg 7.1, vanilla-sources 2.6.17, glibc 2.4-r3, KDE 3.5.4, nvidia-drivers 8774, SMP on.
So, having KDE started, I'm coping 2.4G iso image from vfat partition to ext3 and I have 7 - 10% usage for one core and 0 - 1% for enother and less then 12% tatal.

[energyman76b]

So definitly is the SATA driver! Or has anyone experienced this with PATA?
And no opentaka, it's not just with reiserfs. I have only vfat/ext3 partitions on the SATA drives.
Well ok, now to get the kernel devs aware of the problem.. or maybe they are already..?!
Edit: Arg, I cannot manage the lkml archive. It does have NUMEROUS occurences of SATA bugs/patches, but I cannot cope with the amount. Someone can?
http://www.google.com/search?q=site%3Al ... F2006+sata

I do. mldonkey can totally destroy performance with its constant disk accesses. It does not matter if the drives are pata oder sata. disc accesses make the system crawl. And that at up/download speeds of a few kb/sec

It is not amd64 related - I had the same prob with an athlon-xp and no sata disks at all.

Something is pretty f* up in linux land if it comes to disk access...

[bexamous]

Just wanted to chime in and say I've noticed the same issue. hdparm performance is good but system starts to lag with lots of disk access. I went out and bought another GB of RAM so I could get rid of my swap file... it was lag as in mouse would stop for 5 seconds sometimes. I've got individual disks, a raid1 and two raid5s... doesn't make a difference, also using both nvidia sata and sil3114 controllers and doesn't seem to matter.

Usually when the mouse starts to stutter a few times I'll know a download just finished and its being hashed.

2.6.18-ck1 #1 SMP PREEMPT, Opteron 180

Myabe I'll try turning preempt off again, although honestly every kernel / setting I've tried hasn't made much of a difference.

[defenderBG]

By any chance, is there someone NOT using the nvidia kernel module that is also experiencing this issue?

i do... i have a amd64... and use ext3!
i was thinking

[merlin]

I'm running an Athlon64 3000+ on an ECS nforce3 based motherboard. hdparm shows good throughput for my drives (all PATA). But when I do things like emerge --sync everything slows to a crawl, with the system at 99%+ wait.

I don't have preemption enabled in my kernel.

Linux naruto 2.6.17-gentoo-r8 #1 Mon Sep 11 04:20:54 PDT 2006 x86_64 AMD Athlon(tm) 64 Processor 3000+ AuthenticAMD GNU/Linux

[Bornio]

Very much the same issue here.

Any ideas?

[t0mcat]

yeah i'm having the same problem... insanely slow and unresponsive system since last backup-restore.

i did it with a plain "cp -a" from an ext3 fs to an ext3 partition usb drive. repartitioned the drive with xfs, then copied back from the usb ext3 to the new xfs partition. now the system (an acer 1511lmi laptop) is even slower than my k7-700 server-box, with a pata drive and xfs file-systems... i'm gonna try some defrag with the xfs tools, but sounds useless.

[trueshanti]

still waiting for 2.6.19

i really hope this kernel would erase this nasty lagginess

best wishes

[t0mcat]

downgrading to gentoo-sources 2.6.17-r8 seems to have improved the situation here, although the system is not responsive as it used to be some time ago. by the way i did a mass update so i can't really say wich the cause could be.

just for info:

i had no sata drives, it's an amd64 laptop compiled with march=ahtlon64 & mtune=k8 , using kernel built-in drivers for the nforce3 chipset, and gcc-4.1.1.

the issue was slowing down the system even with a 1% cpu occupation, during disk accesses like source-unpacking, portage cache upgrading, and generic emerge usage.

i'm gonna do more testing with different kernel sources and options.

[Bornio]

Yep, same problem here.
It seems like the slow down doesnt happen only when there is a high access to the HD, or completely overload the HD, but simple tasks like syncing or regen world by portage will cause the same thing.
I hope there is a solution soon. this is VERY frustrating.

[funkmankey]

same problem here as everyone, have had it for much too long without trying hard enough to find a solution!!! went away when I switched from a venice core to denmark, but quickly returned. hell I am pretty sure I even had some symptoms of it back when I was still using socket-a.

currently:

Code: Select all

$ uname -a
Linux ah1 2.6.18-gentoo-r3 #2 SMP Tue Nov 21 14:15:58 EST 2006 x86_64 Dual Core AMD Opteron(tm) Processor 165 AuthenticAMD GNU/Linux

based on this thread, I have now completely disabled all preempt config options, and now everything is fast and smooth even when running eix-sync or when ktorrent is starting up and running checksums on tens of gb of data... so, thanks!

[Torangan]

Disabling preemption helped for me as well. If there's high disk traffic I'll get a short stutter in video but then it display smoothly throughout the disk activity. Not good, but better then before.

[Bornio]

Same results. Disabling preempt helps a lot.
Though I expected better performance from my Core2Duo E6600 with 2GB RAM.
But oh well

Also, I notice that the kernel does not divide the interrupts, at all!

cat /proc/interrupts
CPU0 CPU1
0: 28997783 0 IO-APIC-edge timer
1: 41468 0 IO-APIC-edge i8042
5: 0 0 IO-APIC-edge parport0
8: 0 0 IO-APIC-edge rtc
9: 0 0 IO-APIC-level acpi
12: 76267 0 IO-APIC-edge i8042
14: 470231 0 IO-APIC-edge ide0
15: 81712 0 IO-APIC-edge ide1
50: 0 0 IO-APIC-level ehci_hcd:usb2, uhci_hcd:usb5
58: 0 0 IO-APIC-level uhci_hcd:usb4
74: 201206 0 PCI-MSI HDA Intel
169: 2562544 0 IO-APIC-level ide4, uhci_hcd:usb3, nvidia
177: 0 0 IO-APIC-level uhci_hcd:usb6
225: 2078668 0 PCI-MSI sky2
233: 0 0 IO-APIC-level ehci_hcd:usb1, uhci_hcd:usb7
NMI: 28997728 28997613
LOC: 28420843 28420772
ERR: 0
MIS: 0

though once I enabled irqbalance it was alright again. But it feels more like a hack rather then a fix.

Code: Select all

Linux localhost 2.6.18-ck1-r2 #1 SMP Thu Nov 23 09:24:47 IST 2006 x86_64 Intel(R) Core(TM)2 CPU          6600  @ 2.40GHz GenuineIntel GNU/Linux

[bexamous2]

bexamous@bexus ~ $ cat /proc/interrupts
CPU0 CPU1
0: 3743415 575638883 IO-APIC-edge timer
1: 1122 145350 IO-APIC-edge i8042
8: 1127010 181646262 IO-APIC-edge rtc
9: 0 0 IO-APIC-level acpi
50: 61263 1208400 IO-APIC-level libata
58: 66309 1225770 IO-APIC-level libata
66: 0 3 IO-APIC-level ehci_hcd:usb1
74: 42333 5175845 IO-APIC-level EMU10K1
217: 981836 156447437 IO-APIC-level ohci_hcd:usb2, eth0
225: 235323 40705714 IO-APIC-level libata, nvidia
233: 245213 49464098 IO-APIC-level libata, nvidia
NMI: 270 178
LOC: 579297399 579240030
ERR: 0
MIS: 0
bexamous@bexus ~ $

I don't have to use irqbalance and both cpus are handling interrupts... not perfectly 50/50 though. Anyone know why libata and nvidia have to share irqs though? I don't think there is any way to change IRQs really though is there? -- other than moving cards around which isn't doable.

Nov 23 11:25:52 bexus warning: many lost ticks.
Nov 23 11:25:52 bexus Your time source seems to be instable or some driver is hogging interupts

I've gotten that message 41 times since January (around when I installed gentoo).

Linux bexus 2.6.18-ck1-r1 #1 SMP Thu Nov 16 00:21:13 PST 2006 x86_64 Dual Core AMD Opteron(tm) Processor 170 AuthenticAMD GNU/Linux

[Bornio]

you probably have dynamic overclocking in the kernel causing the lost ticks.

[st0ne]

hi,

same problem here on two machines...
one is Core2Duo E6300 with sata hardware, the other is an old celeron 2.4Ghz with ide hardware...
both show same error... when i have much disk IO, then the cpu goes to wait and systemperformance is very poor...
on both machines is preemt off!!

greez st0ne

[titusthefox]

Hello I have the same problem when I make emerge --sync or when I load Eclipse with java (more small files).

My kernel is:

Linux localhost 2.6.18-gentoo-r2 #1 Fri Dec 1 19:13:23 CET 2006 x86_64 Mobile AMD Athlon(tm) 64 Processor 3000+ AuthenticAMD GNU/Linux

I have tried all suggestion that that you gift up (disable irq sharing .....) but the performances have the same problem. I havre tried to downgrade at kernel 2.6.17-r9 but I haven't seen improvements....

[juhah]

Have you tried to re-produce this problem by dual booting to some other OS?

If you can observe the same behavior under some other OS then the issue is most likely hardware related. However, if you cannot reproduce this then you should try to dual boot to some other well known distro and see how it behaves.

With these tests you should be able to narrow problem to either bad hardware, Linux kernel in general or to problem with your Gentoo installation.

If the issue is in your Gentoo installation then it might be a good idea to try sys-kernel/genkernel. Also, double check that you have correct and reasonable CFLAGS.

[st0ne]

hi,

i've tried it with windows... the speed there is normal...

i'm using genkernel, next time i will try without it...

greez st0ne

[Icer]

If you are using gamin check this out. I noticed that gamin was eating over 50% of available memory. I'm now trying out the tips mentioned in the thread.

Alternatively if you use fam you might want to search the forums if it can cause similar problems.

[kingmanowar]

Hi

I have exactly the same problem. I was upgrading to gnome 2.16 yesterday night and I could barely do something else while compiling. Even browsing the web is terrible, scrolling the page...becomes jumping in the page, flash videos become really choppy... I tried the 'tips' from this thread but none of them really worked.
Is it definitely related with the kernel ? SATA drivers ?
Anyone has tried the kernel 2.6.19 yet ? It is a really annoying problem. I have ubuntu for x86 on my machine as well and i do not have the problem.
My config is an amd athlon 64 3800+ dual core, SATA hard drive.

Thanks for anyhelp.

[Desintegr]

I have exactly the same problem. I was upgrading to gnome 2.16 yesterday night and I could barely do something else while compiling.

I use Kernel 2.6.19, I think it's better. But maybe, it's only the placebo effect.

If you have a right amount of memory (1G or+), you can mount « /var/tmp/portage » in RAM. This will avoid disk I/O during compilations.

http://gentoo-wiki.com/TIP_Speeding_up_ ... with_tmpfs

With tmpfs mounted on /var/tmp/portage, compiling is completely done in RAM, and not on the hard disk, which is probably the slowest part on a PC. This also saves your hard disk from fragmentation, leaving you with an all around faster system and longer drive life.

[Bornio]

Okay, so I am using 2.6.19-ck2 sources.
It seems like the freezing is still there. No preempt.

I did some testing, and I noticed freezing happens a lot when using VMware.
So I started logging with iostat, and this is what i found that happens DURING a freeze:

Code: Select all

Time: 23:04:02
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           3.61    0.00    2.06   50.52    0.00   43.81

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
hde               0.00         0.00         0.00          0          0
sda               7.22         8.25        74.23          8         72
sda1              0.00         0.00         0.00          0          0
sda2              0.00         0.00         0.00          0          0
sda3              0.00         0.00         0.00          0          0
sda4             10.31         8.25        74.23          8         72
sdb             583.51       131.96     11802.06        128      11448
sdb1              0.00         0.00         0.00          0          0
sdb2              0.00         0.00         0.00          0          0
sdb4              1.03       131.96         0.00        128          0
sdb5              0.00         0.00         0.00          0          0
sdb6           1689.69         0.00     13517.53          0      13112
sdb7              0.00         0.00         0.00          0          0


Time: 23:04:03
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           1.01    0.00    2.53   96.46    0.00    0.00

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
hde               0.00         0.00         0.00          0          0
sda               1.02         0.00         8.16          0          8
sda1              0.00         0.00         0.00          0          0
sda2              0.00         0.00         0.00          0          0
sda3              0.00         0.00         0.00          0          0
sda4              1.02         0.00         8.16          0          8
sdb             637.76         0.00     12187.76          0      11944
sdb1              0.00         0.00         0.00          0          0
sdb2              0.00         0.00         0.00          0          0
sdb4              0.00         0.00         0.00          0          0
sdb5              0.00         0.00         0.00          0          0
sdb6           1560.20         0.00     12481.63          0      12232
sdb7              0.00         0.00         0.00          0          0


Time: 23:04:04
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.52    0.00    2.07   97.41    0.00    0.00

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
hde               0.00         0.00         0.00          0          0
sda               1.03         0.00        16.49          0         16
sda1              0.00         0.00         0.00          0          0
sda2              0.00         0.00         0.00          0          0
sda3              0.00         0.00         0.00          0          0
sda4              2.06         0.00        16.49          0         16
sdb             795.88         0.00     16659.79          0      16160
sdb1              0.00         0.00         0.00          0          0
sdb2              0.00         0.00         0.00          0          0
sdb4              0.00         0.00         0.00          0          0
sdb5              0.00         0.00         0.00          0          0
sdb6           2210.31         0.00     17682.47          0      17152
sdb7              0.00         0.00         0.00          0          0

What I want to emphesize is that the %iowait under load, going to a peak of 97.41 !

Code: Select all

% iowait 	 Shows the percentage of time that the CPU or CPUs were idle during which the system had an outstanding disk I/O request. This value may be slightly inflated if several processors are idling at the same time, an unusual occurrence.

And perhaps also the tps.

Code: Select all

tps 	 Indicates the number of transfers per second that were issued to the physical disk. A transfer is an I/O request to the physical disk. Multiple logical requests can be combined into a single I/O request to the disk. A transfer is of indeterminate size.

I am not too sure what it all means yet, but this is a bit interesting.

Any ideas, anyone?

EDIT:
Here is some information:
According to this link
I`ll quote:

The first thing you should look at is iowait. If you have a high percentage of CPU time idle while it�s waiting on disk I/O, that�s a good indicator that you have an I/O bottleneck. Moving on to the device section, you should be able to easily see how I/O is being distributed between disks. Do you have a lot of activity on one disk while another one is sitting idle? If so, you should see if you can move some of the activity from the active disk to the idle disk. You may have a case where all of your available disks are being utilized or you can�t evenly distribute the load among the existing disks. In that case, you need to either add additional disks (if you have the capacity) or replace the current disks with ones that have a faster spindle speed, higher throughput, and lower seek times.

This makes sense on slow disks, but does it really on Western Digital 250GB Model: WD2500KS with 16MB cache that I have?
SIGH! [/url]

[Bornio]

Well, I thought changing /proc/sys/vm/dirty_writeback_centisecs to something low like "2" or even "0" would help, and it seemed like it did, i am not sure, i launched vmware and this is what happened at some point:

Code: Select all

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.00    0.00    0.00  100.00    0.00    0.00

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
hde               0.00         0.00         0.00          0          0
sda               1.00         0.00         8.00          0          8
sda1              0.00         0.00         0.00          0          0
sda2              0.00         0.00         0.00          0          0
sda3              0.00         0.00         0.00          0          0
sda4              1.00         0.00         8.00          0          8
sdb               0.00         0.00         0.00          0          0
sdb1              0.00         0.00         0.00          0          0
sdb2              0.00         0.00         0.00          0          0
sdb4              0.00         0.00         0.00          0          0
sdb5              0.00         0.00         0.00          0          0
sdb6              0.00         0.00         0.00          0          0
sdb7              0.00         0.00         0.00          0          0

as you can probably guess everything froze completely.. and this makes no sense to me at all.
SIGH

[Icer]

If you are using gamin check this out. I noticed that gamin was eating over 50% of available memory. I'm now trying out the tips mentioned in the thread.

Alternatively if you use fam you might want to search the forums if it can cause similar problems.

In my system the culprit was gamin. Other bugger is firefox which eats memory if I leave it open for a long time with lot of tabs opened, but that's nowhere as annoying as the gamin/famin issue. No more lockups here.

Since you have tested the kernel(s) already and there's 3 pages worth of talk about kernel and no kernel provided a solution I'd say there could be something else you should be investigating than the kernel.