Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
AMD64 system slow/unresponsive during disk access (Part 2)
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4, 5, 6, 7  Next  
Reply to topic    Gentoo Forums Forum Index Gentoo on AMD64
View previous topic :: View next topic  
Author Message
CarstenIQ
n00b
n00b


Joined: 04 Jan 2007
Posts: 27
Location: Mexico

PostPosted: Mon Sep 03, 2012 3:54 pm    Post subject: Reply with quote

I ended up in using the pf-sources (3.5.2) with the BFQ patch set and zram. My 4 core Phenom II works smooth in x86_64 environment.
I works so good that the Flash player doesn't even crash anymore in Opera. For me the gentoo-sources (3.4.9 ) was just to sluggish and had too many lock-ups. It might be a considered a stable kernel but for me it is not usable kernel in x86_64 environment.
_________________
Gentoo Linux Rocks...!!!
Back to top
View user's profile Send private message
yoshi314
l33t
l33t


Joined: 30 Dec 2004
Posts: 819
Location: PL

PostPosted: Wed Sep 05, 2012 11:16 am    Post subject: Reply with quote

btw, running vanilla kernel i noticed that unrar command is incredibly slow. (similariy when unpacking the file with 7z)

unpacking a 3GB archive took well over an hour. is anyone else catching the same issue when not doing any performance improving workarounds?

edit: it stopped happening recently, for some reason.
_________________
~amd64, ~x86
shrink your /usr/portage with squashfs+aufs


Last edited by yoshi314 on Mon Nov 26, 2012 8:51 am; edited 1 time in total
Back to top
View user's profile Send private message
smlbstcbr
n00b
n00b


Joined: 08 Apr 2006
Posts: 34

PostPosted: Sat Nov 24, 2012 1:23 am    Post subject: Reply with quote

Bump. Using kernel 3.5.7 and the deadline scheduler the problem is greatly reduced. Tough also modifying the PCI latency timer seems to have had a significant impact in the responsiveness of the whole system.
Back to top
View user's profile Send private message
_______0
Guru
Guru


Joined: 15 Oct 2012
Posts: 521

PostPosted: Tue Dec 18, 2012 12:54 pm    Post subject: how to use? Reply with quote

smlbstcbr wrote:
... PCI latency timer seems to have had a significant impact in the responsiveness of the whole system.



how to do this?? What's better higher values on all pci slots or something?

Also why on earth does setpci use hex values instead of more user-friendly NUMBERS???!!

Quote:
where xx:yy.z are the first numbers listed by lspci, and n is a hex value between 0 and ff (setpci will round down ff, which is actually 255, to the required 248).


dis crazy... Ideally it'd be nice to have a web interface for setpci controlled by pretty sliders for each pci slot and with css shadows.

And for maximum priority for sound card (to prevent xruns) which value would be? Something like this?

Code:

setpci -v -s 06:05.0 latency_timer=ff


thanks.

I forgot to add, how do intenger values between 0 and 248 translate into hex values? It be nice to have it as reference. This is confusing because according to a blog setpci hex values are ODD numbers instead of even oO

If anybody know this I need a table of the exact ODD setpci hex value with the corresponding BIOS even values, like this.

hex | hex odd intenger | bios even intenger
ff 255 248

here is the referenced site http://www.mythtv.org/wiki/PCI_Latency


And to expand on this topic. How can person determine which is the best combination of latency for all devices in order for a computer to run optimal?? Let's say I know now that I want to prevent the sound card to have xruns the set it to maximum priority, then for recording the hard drive would need an optimal value too. It's not accurate just to set something high because it 'feels' it'll run better.
Back to top
View user's profile Send private message
smlbstcbr
n00b
n00b


Joined: 08 Apr 2006
Posts: 34

PostPosted: Thu Dec 20, 2012 11:21 pm    Post subject: Reply with quote

The PCI timer is set in the BIOS setup options (each BIOS has a different setup, though should be very similar).
For INT to HEX: http://www.statman.info/conversions/hexadecimal.html
Back to top
View user's profile Send private message
yoshi314
l33t
l33t


Joined: 30 Dec 2004
Posts: 819
Location: PL

PostPosted: Mon Mar 04, 2013 6:23 pm    Post subject: Reply with quote

most of you probably know this, but it won't hurt to have links in the thread, for reference.

anyone brave enough to tackle latest 3.9 git?

this is said to help with original issue : http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=e0a79f529d5ba2507486d498b25da40911d95cf6

and more info is here : https://bugzilla.kernel.org/show_bug.cgi?id=12309#c621
_________________
~amd64, ~x86
shrink your /usr/portage with squashfs+aufs
Back to top
View user's profile Send private message
mack1
Guru
Guru


Joined: 18 Mar 2006
Posts: 315

PostPosted: Wed Mar 20, 2013 5:36 pm    Post subject: Reply with quote

I have installed git-sources-3.9-r3 and high latency is greatly reduced under heavy i/o operations :D !
Back to top
View user's profile Send private message
smlbstcbr
n00b
n00b


Joined: 08 Apr 2006
Posts: 34

PostPosted: Thu Mar 21, 2013 9:38 pm    Post subject: Reply with quote

Really?! I had noticed a significant improvement by using the RCU option in 3.7.10 with the deadline scheduler.
Back to top
View user's profile Send private message
depontius
Advocate
Advocate


Joined: 05 May 2004
Posts: 2585

PostPosted: Thu Mar 21, 2013 11:20 pm    Post subject: Reply with quote

smlbstcbr wrote:
Really?! I had noticed a significant improvement by using the RCU option in 3.7.10 with the deadline scheduler.


I've at least heard of the deadline scheduler, though I use the default CFQ. I've also read of RCU, but I thought that was just plain baked-in as a feature to be used by other subsystems, especially handy for multiprocessor.

What is "the RCU option"?
_________________
.sigs waste space and bandwidth
Back to top
View user's profile Send private message
smlbstcbr
n00b
n00b


Joined: 08 Apr 2006
Posts: 34

PostPosted: Fri Mar 22, 2013 3:42 pm    Post subject: Reply with quote

depontius wrote:
smlbstcbr wrote:
Really?! I had noticed a significant improvement by using the RCU option in 3.7.10 with the deadline scheduler.


I've at least heard of the deadline scheduler, though I use the default CFQ. I've also read of RCU, but I thought that was just plain baked-in as a feature to be used by other subsystems, especially handy for multiprocessor.

What is "the RCU option"?

https://www.kernel.org/doc/Documentation/RCU/rcu.txt
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5720
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Sat Mar 23, 2013 12:55 am    Post subject: Reply with quote

smlbstcbr wrote:
depontius wrote:
smlbstcbr wrote:
Really?! I had noticed a significant improvement by using the RCU option in 3.7.10 with the deadline scheduler.


I've at least heard of the deadline scheduler, though I use the default CFQ. I've also read of RCU, but I thought that was just plain baked-in as a feature to be used by other subsystems, especially handy for multiprocessor.

What is "the RCU option"?

https://www.kernel.org/doc/Documentation/RCU/rcu.txt


please post the CONFIG_...

option from your /usr/src/linux/.config file you're referring to

thanks :)
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.3.0-r2
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
smlbstcbr
n00b
n00b


Joined: 08 Apr 2006
Posts: 34

PostPosted: Sat Mar 23, 2013 9:41 pm    Post subject: Reply with quote

Here it is
Code:

# RCU Subsystem
#
CONFIG_TINY_PREEMPT_RCU=y
CONFIG_PREEMPT_RCU=y
# CONFIG_TREE_RCU_TRACE is not set
CONFIG_RCU_BOOST=y
CONFIG_RCU_BOOST_PRIO=1
CONFIG_RCU_BOOST_DELAY=500
Back to top
View user's profile Send private message
Mr.EVIL
n00b
n00b


Joined: 22 Mar 2013
Posts: 10

PostPosted: Sun Mar 24, 2013 7:04 pm    Post subject: Reply with quote

it was fixable long time ago with "realtime" patches...
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5720
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Mon Mar 25, 2013 12:34 pm    Post subject: Reply with quote

smlbstcbr wrote:
Here it is
Code:

# RCU Subsystem
#
CONFIG_TINY_PREEMPT_RCU=y
CONFIG_PREEMPT_RCU=y
# CONFIG_TREE_RCU_TRACE is not set
CONFIG_RCU_BOOST=y
CONFIG_RCU_BOOST_PRIO=1
CONFIG_RCU_BOOST_DELAY=500


thanks !


I believe though that most of us here already us a preemptible implementation of RCU




Mr.EVIL wrote:
it was fixable long time ago with "realtime" patches...


unfortunately not an option with ZFS (?)

do you use virtualbox ? does that work for you with the rt-kernel ?
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.3.0-r2
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
depontius
Advocate
Advocate


Joined: 05 May 2004
Posts: 2585

PostPosted: Mon Mar 25, 2013 12:53 pm    Post subject: Reply with quote

smlbstcbr wrote:
Here it is
Code:

# RCU Subsystem
#
CONFIG_TINY_PREEMPT_RCU=y
CONFIG_PREEMPT_RCU=y
# CONFIG_TREE_RCU_TRACE is not set
CONFIG_RCU_BOOST=y
CONFIG_RCU_BOOST_PRIO=1
CONFIG_RCU_BOOST_DELAY=500


Just a side-note... The CONFIG_PREEMPT_RCU option doesn't even show unless you're building a preemptible kernel. I normally build with "voluntary preemption," but this weekend when building 3.8.4 I went the pre-emptible route, and ticked CONFIG_PREEMPT_RCU=y. I don't think I ticked CONFIG_TINY_PREEMPT_RCU - I really need to go back and read if I should have used it for a mere dual-core CPU.

EDIT - a few typos, a little clarification.
_________________
.sigs waste space and bandwidth


Last edited by depontius on Mon Mar 25, 2013 3:36 pm; edited 1 time in total
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5720
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Mon Mar 25, 2013 2:55 pm    Post subject: Reply with quote

smlbstcbr wrote:
The PCI timer is set in the BIOS setup options (each BIOS has a different setup, though should be very similar).
For INT to HEX: http://www.statman.info/conversions/hexadecimal.html


now giving the pci latency timer approach a try:

http://web.archive.org/web/20080829162702/http://www-128.ibm.com/developerworks/library/l-hw2.html


will see if there's a difference with sound or not - thanks !
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.3.0-r2
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
ryao
Developer
Developer


Joined: 27 Feb 2012
Posts: 123

PostPosted: Mon Mar 25, 2013 5:43 pm    Post subject: Reply with quote

kernelOfTruth wrote:
Mr.EVIL wrote:
it was fixable long time ago with "realtime" patches...


unfortunately not an option with ZFS (?)

do you use virtualbox ? does that work for you with the rt-kernel ?


Why is that? If there is an incompatibility, I would like a bug report.
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5720
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Mon Mar 25, 2013 6:57 pm    Post subject: Reply with quote

ryao wrote:
kernelOfTruth wrote:
Mr.EVIL wrote:
it was fixable long time ago with "realtime" patches...


unfortunately not an option with ZFS (?)

do you use virtualbox ? does that work for you with the rt-kernel ?


Why is that? If there is an incompatibility, I would like a bug report.


from what I saw there are already at least 2 issues on github referring to kernels

one of them being: https://github.com/zfsonlinux/spl/issues/218

edit:

https://github.com/zfsonlinux/spl/issues/47

edit.


I get similar/same errors either building it patched into the kernel or as ebuilds as external modules (with 3.8.4 / 3.8-patched rt-kernel)


thanks for your continued support :)


edit:

that probably would be the only issue right now preventing me to use rt-kernels

edit.
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.3.0-r2
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
ryao
Developer
Developer


Joined: 27 Feb 2012
Posts: 123

PostPosted: Mon Mar 25, 2013 8:18 pm    Post subject: Reply with quote

kernelOfTruth wrote:
from what I saw there are already at least 2 issues on github referring to kernels

one of them being: https://github.com/zfsonlinux/spl/issues/218


I had missed this when looking through the bug tracker. Thanks for bringing it to my attention. I will see what I can do to fix the build failure.
Back to top
View user's profile Send private message
ryao
Developer
Developer


Joined: 27 Feb 2012
Posts: 123

PostPosted: Mon Mar 25, 2013 9:16 pm    Post subject: Reply with quote

I tried building a 3.8.4 realtime kernel and was promptly greeted by a build failure for the kernel source tree itself:

Code:
*/usr/src/linux/kernel/events/uprobes.c:1143: undefined reference to `percpu_down_read'
*kernel/built-in.o: In function `uprobe_end_dup_mmap':
*/usr/src/linux/kernel/events/uprobes.c:1148: undefined reference to `percpu_up_read'
*kernel/built-in.o: In function `init_uprobes':
*/usr/src/linux/kernel/events/uprobes.c:1617: undefined reference to `__percpu_init_rwsem'


I cannot produce a fix for ZFS unless I have a fix for this and I do not have time to fix issues in other people's kernel code right now. :/
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5720
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Mon Mar 25, 2013 9:39 pm    Post subject: Reply with quote

ryao wrote:
I tried building a 3.8.4 realtime kernel and was promptly greeted by a build failure for the kernel source tree itself:

Code:
*/usr/src/linux/kernel/events/uprobes.c:1143: undefined reference to `percpu_down_read'
*kernel/built-in.o: In function `uprobe_end_dup_mmap':
*/usr/src/linux/kernel/events/uprobes.c:1148: undefined reference to `percpu_up_read'
*kernel/built-in.o: In function `init_uprobes':
*/usr/src/linux/kernel/events/uprobes.c:1617: undefined reference to `__percpu_init_rwsem'


I cannot produce a fix for ZFS unless I have a fix for this and I do not have time to fix issues in other people's kernel code right now. :/


I'll have a look at it & see whether I get it to build reliably within the next days :)

thanks again



on-topic:

using the latency-timer tweaks + BFS and I couldn't observe any sound stuttering so far

even while compiling GCC which ALWAYS had resulted in at least some minor stuttering or lag on the GUI & sound



I also followed advice and use schedtool on X & pulseaudio

Code:
schedtool -p 0 -n -20 -N `pgrep X`

schedtool -p 0 -n -20 -N `pgrep pulseaudio`

_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.3.0-r2
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5720
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Mon Mar 25, 2013 9:54 pm    Post subject: Reply with quote

@ryao:

let's break out the discussion related to ZFSonLinux and continue in the following thread:

http://forums.gentoo.org/viewtopic-p-7276174.html
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.3.0-r2
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5720
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Sun Jun 09, 2013 2:08 pm    Post subject: Reply with quote

the following RCU settings seem to improve things:

Code:
zcat /proc/config.gz | grep RCU
# RCU Subsystem
CONFIG_TREE_PREEMPT_RCU=y
CONFIG_PREEMPT_RCU=y
CONFIG_RCU_STALL_COMMON=y
CONFIG_RCU_USER_QS=y
CONFIG_RCU_FANOUT=32
CONFIG_RCU_FANOUT_LEAF=2
# CONFIG_RCU_FANOUT_EXACT is not set
CONFIG_RCU_FAST_NO_HZ=y
# CONFIG_TREE_RCU_TRACE is not set
CONFIG_RCU_BOOST=y
CONFIG_RCU_BOOST_PRIO=99
CONFIG_RCU_BOOST_DELAY=331
CONFIG_RCU_NOCB_CPU=y
CONFIG_RCU_NOCB_CPU_ALL=y
# RCU Debugging
# CONFIG_PROVE_RCU_DELAY is not set
# CONFIG_SPARSE_RCU_POINTER is not set
# CONFIG_RCU_TORTURE_TEST is not set
CONFIG_RCU_CPU_STALL_TIMEOUT=32
CONFIG_RCU_CPU_STALL_VERBOSE=y
CONFIG_RCU_CPU_STALL_INFO=y
# CONFIG_RCU_TRACE is not set



also for those who often have heavy i/o going on on their system

the ROW i/o scheduler (read over write) might be interesting:

http://marc.info/?l=linux-mmc&m=137077433718450&w=2
http://marc.info/?l=linux-kernel&m=137077436518458&w=2
http://marc.info/?l=linux-kernel&m=137077469818542&w=2
http://marc.info/?l=linux-mmc&m=137077472018547&w=2

to cut down latencies

it currently seems to undergo some refinements but it's already widely used in the smartphone community XDA (e.g. I'm also using it on my Galaxy Note and it's working fine)



I've some more tweaks related to latency used in my system - when they've been extensively tested I'll post them in a sort of round-up
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.3.0-r2
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
yoshi314
l33t
l33t


Joined: 30 Dec 2004
Posts: 819
Location: PL

PostPosted: Mon Jun 10, 2013 9:02 am    Post subject: Reply with quote

i'm going to try this.

So far the only major issue i have is when copying big files (1.5-2GB) to usb. probably due to memory pressure or something, everything slows down to a crawl until the file is copied over, especially during final 10% of the copy process (when using mc or bar application).

That's on 64bit system with core2 cpu and 3gb of ram. It hits pretty hard with ck-3.9 kernel, with no further manual tweaks.
_________________
~amd64, ~x86
shrink your /usr/portage with squashfs+aufs
Back to top
View user's profile Send private message
ulenrich
Veteran
Veteran


Joined: 10 Oct 2010
Posts: 1159

PostPosted: Mon Jun 10, 2013 4:32 pm    Post subject: Reply with quote

yoshi314 wrote:
So far the only major issue i have is when copying big files (1.5-2GB) to usb. probably due to memory pressure or something, everything slows down

Yes, for me using mc too. I suspect caching. Why the hell we need normal caching?

We need a special purpose cache for copying:
read-cache-write-forget

No fighting over obsolete data wants to keep staying in cache ...
Or is it just the old midnight source code missing to use an existing special Linux kernel function?
_________________
fun2gen2
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo on AMD64 All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6, 7  Next
Page 6 of 7

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum