| View previous topic :: View next topic |
| Author |
Message |
ArneBab
Apprentice
Joined: 24 Jan 2006
Posts: 299
Location: Heidelberg, Germany
|
 Posted: Fri May 01, 2009 11:09 am Post subject: |
 |
|
[quote="neuron"] | nero37 wrote: | | Installed RC4 to see, and it's actually better. It's the first change I've done that's felt like more than a small tweak to a bad situation. I went from 2.6.29, and I use ext4 not ext3 so the writeback change doesn't affect my results. I dont have any numbers but it sure feels a lot better. |
That sounds great!
As soon as 2.6.30 is in portage, I'll give it a try.
I currently have a test which shows the problem amost instantly: Firing off and exchanging commits in Mercurial automatically at a rate of 2 per second or so (stresstesting  ).
_________________
Being unpolitical
means being political
without realizing it.
- Arne Babenhauserheide ( http://draketo.de ) |
|
| Back to top |
|
 |
graysky
n00b
Joined: 25 Nov 2006
Posts: 54
Location: /dev/null
|
| Posted: Fri May 01, 2009 6:32 pm Post subject: |
|
|
| nero37 wrote: | | Installed RC4 to see, and it's actually better. It's the first change I've done that's felt like more than a small tweak to a bad situation. I went from 2.6.29, and I use ext4 not ext3 so the writeback change doesn't affect my results. I dont have any numbers but it sure feels a lot better. |
That's good to hear... can you try the copying a 2 or 3 gig file from one partition to another experiment while using firefox or whatever? Move the window while the copy is in progress... does it get choppy? |
|
| Back to top |
|
|
Lepaca Kliffoth
l33t
Joined: 28 Apr 2004
Posts: 737
Location: Florence, Italy
|
| Posted: Sun May 03, 2009 9:18 am Post subject: |
|
|
The issue is FIXED in 2.6.30-rc4. The IO scheduler you choose doesn't matter.
If the Nvidia drivers won't compile with this kernel version, you can modify the ebuild so that it applies this patch I made:
http://pastebin.com/m63cf9998
I'm not an expert in this stuff but it works.
Happy testing and good luck.
_________________
It isn't enough to win - everyone else must lose, and you also have to rub it in their face (maybe chop off an arm too for good measure).
Animebox! |
|
| Back to top |
|
|
DigitalCorpus
Apprentice
Joined: 30 Jul 2007
Posts: 245
|
| Posted: Mon May 04, 2009 8:57 pm Post subject: |
|
|
Can you provide more documentation as to what is the change in that RC of 2.6.30 that fixed this problem that was existent for so long?
_________________
Atlas (HDTV PVR, HTTP & Media server)
http://atlas.selfip.net/Info/ |
|
| Back to top |
|
|
neuron
Advocate
Joined: 28 May 2002
Posts: 2371
|
| Posted: Tue May 05, 2009 5:43 am Post subject: |
|
|
| I've seen reports that it's not fixed as well, but it's a huge improvement in my case atleast. And I dont know the exact patches sorry, I already read a information thing from linus complaining about it, and some problems in the io scheduler. |
|
| Back to top |
|
|
kernelOfTruth
Watchman
Joined: 20 Dec 2005
Posts: 5345
Location: Vienna, Austria; Germany; hello world :)
|
|
| Back to top |
|
|
H0bb3z
Tux's lil' helper
Joined: 28 May 2003
Posts: 121
Location: in the transmogrifier
|
| Posted: Fri May 08, 2009 3:06 am Post subject: vanilla or gentoo kernel sources |
|
|
Just trying to look into this myself now as well and found this thread.
For 2.6.30-rc4, are you using gentoo-source or vanilla? I'm at 2.6.28-r5 gentoo sources. I'll flip over to vanilla if it drastically improves this issue.
I'm running Intel QX9770 (Quad Extreme), Core 2 optimizations (Funtoo build), DDR3 1600MHz mem w/Intel P45/ICH10R chipsets.
This thing simply flies even with all the heavy compiz whiz-bang extras running -- except when I'm transferring large chunks of data. I'm running striped 1Tb drives w/LVM for larger volumes (ext3). I also notice that the MS wireless mouse gets really sluggish during heavy disk access and the click function "misses" occasionally - requiring me to click a couple times for proper response. That's annoying...
I don't have tangible statistics, but a system with this level of hardware should show much better drive performance...
_________________
-H0bb3z-
-----------
Intel QX9770 | 6Gb DDR3 | ATI 4870 HD 1Gb | 2xSeagate 1Tb SATAs
Running Funtoo Core2 build |
|
| Back to top |
|
|
Lepaca Kliffoth
l33t
Joined: 28 Apr 2004
Posts: 737
Location: Florence, Italy
|
| Posted: Sat May 09, 2009 9:33 am Post subject: |
|
|
Coming back to report that the issue is not fixed completely after all; however, the awful mouse stuttering I used to get all the time appeared only once when I had to copy ~30GB of data from my linux main partition to a windows one (mounted with ntfs-3g). Since I switched to vanilla 2.6.30-rc4 I haven't noticed the problem in any other situation.
_________________
It isn't enough to win - everyone else must lose, and you also have to rub it in their face (maybe chop off an arm too for good measure).
Animebox! |
|
| Back to top |
|
|
teapot
Tux's lil' helper
Joined: 09 Nov 2006
Posts: 85
Location: Stockholm , Sweden
|
| Posted: Wed May 13, 2009 11:28 pm Post subject: |
|
|
the problem is not completely fixed for me either. whenever a big torrent download is finished , the hashing procedure sends latencies through the roof , even if i ionice the rtorrent process down. it used to be a lot worse , but the problem is very much still there.
is it just me , or do you also find it strange that a serious issue like this can exist for such a long time? ( 3 years or something like that )
is it because this problem just exist on consumer level hardware and isn't considered to be a problem on servers?
the problem has been documented for a long time , but has more or less been ignored by the kernel developers until recently.
good that something is finally happening , but i get the feeling that the latencies are caused by a whole bunch of errors.
bringing down a few of them will sure make things better , but i am not convinced that the real reason behind all this is found yet... |
|
| Back to top |
|
|
darklegion
Guru
Joined: 14 Nov 2004
Posts: 423
|
| Posted: Thu May 14, 2009 4:46 am Post subject: |
|
|
| teapot wrote: | the problem is not completely fixed for me either. whenever a big torrent download is finished , the hashing procedure sends latencies through the roof , even if i ionice the rtorrent process down. it used to be a lot worse , but the problem is very much still there.
is it just me , or do you also find it strange that a serious issue like this can exist for such a long time? ( 3 years or something like that )
is it because this problem just exist on consumer level hardware and isn't considered to be a problem on servers?
the problem has been documented for a long time , but has more or less been ignored by the kernel developers until recently.
good that something is finally happening , but i get the feeling that the latencies are caused by a whole bunch of errors.
bringing down a few of them will sure make things better , but i am not convinced that the real reason behind all this is found yet... |
It's probably just one of those phantom bugs that's difficult to track down.And it doesn't occur on all configurations, I haven't had the problem on my system since about 2.6.24, and it only occured on one of my systems and not the other.Sure, I get a bit of latency when copying files around, but it's not any worse than a standard windows installation. |
|
| Back to top |
|
|
Elv13
Guru
Joined: 13 Nov 2005
Posts: 377
Location: Socialist land of North America
|
| Posted: Thu May 21, 2009 5:35 am Post subject: |
|
|
| Just updated to 2.6.30-rc5. I confirm that the problem is gone for me. I can unzip a kernel while watching a HD video, something that was impossible with earlier version. (with or without VDPAU or whatever the new nvidia thing is called) |
|
| Back to top |
|
|
albright
Veteran
Joined: 16 Nov 2003
Posts: 1763
Location: Near Toronto
|
| Posted: Thu May 21, 2009 12:00 pm Post subject: |
|
|
| Quote: | | it's not any worse than a standard windows installation. |
so that's the level of extreme excellence we're shooting for eh? Maybe
linux will someday attain such heights.
_________________
.... there is nothing - absolutely nothing - half so much worth
doing as simply messing about with Linux ...
(apologies to Kenneth Graeme) |
|
| Back to top |
|
|
E001754
Guru
Joined: 01 Aug 2004
Posts: 431
Location: Paris, France
|
| Posted: Thu May 28, 2009 1:38 pm Post subject: |
|
|
| Yamakuzure wrote: | Just a note on this issue with unresponsiveness:
I found, that setting up a preemptible kernel with 1000Hz clock is a very very bad idea!
Since I configured all my amd64 machines to build a kernel with "voluntary preemption" and 300Hz clock, performance went incredibly up everywhere.
Before I did this, whenever I emerged world updates, and locked the screen, I needed to move the mouse like mad and hammer on the keyboard for 20 - 30 seconds before the login window came up. Now I press "ctrl" and the login window is there in 1 - 2 seconds. Quite a difference, right?
This applies for a HP laptop with intel core2 cpu and 1GB RAM as well as my new AMD PhenomII 940/4GB RAM machine. Both benefit from the clock and preemption change, so maybe if you have issues with responsiveness, you might want to try that? | Code: | Processor type and features --->
Preemption Model --->
( ) No Forced Preemption (Server)
(X) Voluntary Kernel Preemption (Desktop)
( ) Preemptible Kernel (Low-Latency Desktop)
Timer frequency --->
( ) 100 HZ
( ) 250 HZ
(X) 300 HZ
( ) 1000 HZ |
|
1000 Hz was a setting needed by some ID Software games if I remember well.
Did you try playing ET:QW or Quake 4 with one of your computer since you changed this preemption model and that timer frequency ?? |
|
| Back to top |
|
|
Captain Newbie
Apprentice
Joined: 22 Dec 2006
Posts: 174
Location: Socal
|
| Posted: Sat May 30, 2009 4:32 am Post subject: |
|
|
| Lepaca Kliffoth wrote: | The issue is FIXED in 2.6.30-rc4. The IO scheduler you choose doesn't matter.
etc |
*** IF YOU ARE REPORTING THIS FIXED ***
Please tell us your kernel and hardware configuration.
I can report things as 'subjectively better' in 2.6.30-rc7-git3 (sys-kernel/git-sources). IOWait still loads up to high levels (84%) and processes report as diskbound in top (status D), but firefox keeps chugging happily away as does VLC.
Hardware:
Intel Core 2 Quad Q9300 on Intel P35 Express
8GB DDR2-800
Seagate 1TB Serial ATA hard drive
_________________
jmp resume_userspace
"As much as it pains me, we hope that developers know what they're doing." - wolf31o2 |
|
| Back to top |
|
|
DigitalCorpus
Apprentice
Joined: 30 Jul 2007
Posts: 245
|
| Posted: Sat May 30, 2009 8:18 am Post subject: |
|
|
That sounds like great news. I'll move over to 2.6.30 when it becomes stable.
Out of curiosity, where can we find details to exactly what the problem was?
_________________
Atlas (HDTV PVR, HTTP & Media server)
http://atlas.selfip.net/Info/ |
|
| Back to top |
|
|
Elv13
Guru
Joined: 13 Nov 2005
Posts: 377
Location: Socialist land of North America
|
| Posted: Mon Jun 01, 2009 12:28 am Post subject: |
|
|
I got it again after all, but it is much, much better.
Kernel 2.6.30-rc5, nvidia nforce 680i, Intel C2Q Q6600, EVGA motherboard |
|
| Back to top |
|
|
poly_poly-man
Advocate
Joined: 06 Dec 2006
Posts: 2477
Location: RIT, NY, US
|
| Posted: Mon Jun 01, 2009 12:32 am Post subject: |
|
|
last week, on 2.6.28, had a nearly unusable system with nothing more than a torrent in the background. Now, on .30_rc7, bittorrent seems to screw with my network card a little, but that, installing ubuntu in a vm, etc. are all fine.
_________________
iVBORw0KGgoAAAANSUhEUgAAA
avatar: new version of logo - see topic 838248. Potentially still a WiP. |
|
| Back to top |
|
|
forsaken1
n00b
Joined: 30 May 2008
Posts: 27
Location: Lovely Skåne, Sweden
|
| Posted: Mon Jun 01, 2009 12:02 pm Post subject: |
|
|
| DigitalCorpus wrote: | That sounds like great news. I'll move over to 2.6.30 when it becomes stable.
Out of curiosity, where can we find details to exactly what the problem was? |
http://lwn.net/Articles/328363/
It's focused on ext3, but alot of the fixes affects pretty much all fs. |
|
| Back to top |
|
|
Elv13
Guru
Joined: 13 Nov 2005
Posts: 377
Location: Socialist land of North America
|
| Posted: Fri Jun 05, 2009 4:44 am Post subject: |
|
|
| Interesting, anybody know how to change the size from 512k to 64k like expressed in the article, I might give it a try if it make things even better than they are now. |
|
| Back to top |
|
|
fangorn
Veteran
Joined: 31 Jul 2004
Posts: 1872
|
| Posted: Fri Jun 05, 2009 6:52 am Post subject: |
|
|
| poly_poly-man wrote: | | last week, on 2.6.28, had a nearly unusable system with nothing more than a torrent in the background. Now, on .30_rc7, bittorrent seems to screw with my network card a little, but that, installing ubuntu in a vm, etc. are all fine. |
I am doing video conversion a lot. So I am moving hundreds of GB from one disk to the another on a daily basis.
I am using 2.6.28 atm and had some minor advancements changing I/O schedulers, assuming that only one disk I/O stream is working. Whenever two or more disk I/O streams interfere the system is totally unusable. It takes _minutes_ to refresh the display of an already running firefox after a desktop switch. Also I/O slows down to a crawl (less than 10% of normal data rate) instead of approx. 50 % per stream that I would have expected. It is a little better if one of the disks involved is a share mounted over NFS, but not much.
I have two machines that I would like to upgrade to 2.6.30. How is the support for this version in NVIDIA and FGLRX drivers?
_________________
Video Encoding scripts collection | Project page |
|
| Back to top |
|
|
kernelOfTruth
Watchman
Joined: 20 Dec 2005
Posts: 5345
Location: Vienna, Austria; Germany; hello world :)
|
|
| Back to top |
|
|
fangorn
Veteran
Joined: 31 Jul 2004
Posts: 1872
|
| Posted: Sat Jun 06, 2009 4:47 am Post subject: |
|
|
OK, tried vanilla-sources-2.6.30-rc8 on my cruncher.
System is more responsive, but whenever I copy data the workload (two video encodings in parallel) on the cpu (Core i7 920) drops from 100 % on all 8 (virtual) cores to approx. 30 %.
Increased responsiveness could also result from the change from preemptile to voluntary preemptile and from 1000 Hz to 250 Hz.
So, I cannot confirm that 2.6.30 has changed too much.
Here is my .config http://nopaste.info/98d98c1e5d.html
Edit: for me anticipatory i/o scheduler made better results than cfq. That's why it is default in my config.
_________________
Video Encoding scripts collection | Project page |
|
| Back to top |
|
|
DigitalCorpus
Apprentice
Joined: 30 Jul 2007
Posts: 245
|
| Posted: Tue Jun 09, 2009 9:20 am Post subject: |
|
|
fangorn, which filesystems are you using and what mount options?
_________________
Atlas (HDTV PVR, HTTP & Media server)
http://atlas.selfip.net/Info/ |
|
| Back to top |
|
|
fangorn
Veteran
Joined: 31 Jul 2004
Posts: 1872
|
| Posted: Tue Jun 09, 2009 11:55 am Post subject: |
|
|
using xfs for video storage (internal, on USB or eSATA external harddiscs), ext3 or reiserfs (v3) for system storages, nfs for network storages (with xfs on the server machine).
Edit: Using mainly default(,noatime) options IIRC, but will check this evening.
_________________
Video Encoding scripts collection | Project page |
|
| Back to top |
|
|
lagalopex
Guru
Joined: 16 Oct 2004
Posts: 423
|
|
| Back to top |
|
|
|
Gentoo on AMD64
