Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
CK is working on a new scheduler for Multimedia/Gaming PCs
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3  Next  
Reply to topic    Gentoo Forums Forum Index Off the Wall
View previous topic :: View next topic  
Author Message
energyman76b
Advocate
Advocate


Joined: 26 Mar 2003
Posts: 2040
Location: Germany

PostPosted: Tue Sep 08, 2009 4:17 pm    Post subject: Reply with quote

RealNC wrote:
energyman76b wrote:
I guess you are the person who got told to install a more recent latencytop?


Yes (and I did and also posted results with it.)


yeah, saw that. Reading that thread on marc ;)
_________________
Study finds stunning lack of racial, gender, and economic diversity among middle-class white males
Back to top
View user's profile Send private message
energyman76b
Advocate
Advocate


Joined: 26 Mar 2003
Posts: 2040
Location: Germany

PostPosted: Fri Sep 11, 2009 8:47 am    Post subject: Reply with quote

so, how is your impression exchanging with Ingo&co so far?
_________________
Study finds stunning lack of racial, gender, and economic diversity among middle-class white males
Back to top
View user's profile Send private message
RealNC
Tux's lil' helper
Tux's lil' helper


Joined: 13 Apr 2008
Posts: 148

PostPosted: Fri Sep 11, 2009 9:12 am    Post subject: Reply with quote

Well, as long as you keep the "smart-ass attitude" out of those discussions, you can keep their interest in resolving issues. If you don't, you just end up talking to the hand. I suppose most people in the past had their smart-ass mode enabled due to Ingo's and other devs inability to actually witness the issues (I suspect the devs work only with terminals or use some kind of 10000$ monster machines). But as long as I tried to answer their questions and run their test cases, they were all pretty friendly and co-operative.

In the end, the solution seems to be disabling the fair scheduler part of CFS for the time being. BFS is still a tad better, but not by much.
Back to top
View user's profile Send private message
energyman76b
Advocate
Advocate


Joined: 26 Mar 2003
Posts: 2040
Location: Germany

PostPosted: Fri Sep 11, 2009 9:24 am    Post subject: Reply with quote

the biggest obstacle is probably reproduction of the problem. If there is a problem that only shows with 2 and 4 core Intel CPUs or AMD machines doing heavy disk io, while watching videos with composite turned on, you have a good chance that the devs will miss that. Or one that only shows up with a special version of a certain nvidia chipset. Or some amd southbridge revision that was only sold for a couple of weeks/month....

in my experience it is always worthwile reporting problems on lkml. They might not be able to solve your problem, but more often than not they are.
_________________
Study finds stunning lack of racial, gender, and economic diversity among middle-class white males
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5868
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Fri Sep 11, 2009 9:49 am    Post subject: Reply with quote

indeed !

seems like it was a problem with the fair-scheduling part:

http://thread.gmane.org/gmane.linux.kernel/886319

Quote:
>
> what is /debug/sched_features - is NO_NEW_FAIR_SLEEPERS set? If not
> set yet then try it:
>
> echo NO_NEW_FAIR_SLEEPERS > /debug/sched_features
>
> that too might make things more fluid.

_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.5.2

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
Etal
Veteran
Veteran


Joined: 15 Jul 2005
Posts: 1641

PostPosted: Fri Sep 11, 2009 4:28 pm    Post subject: Reply with quote

kernelOfTruth wrote:
indeed !

seems like it was a problem with the fair-scheduling part:

http://thread.gmane.org/gmane.linux.kernel/886319

Quote:
>
> what is /debug/sched_features - is NO_NEW_FAIR_SLEEPERS set? If not
> set yet then try it:
>
> echo NO_NEW_FAIR_SLEEPERS > /debug/sched_features
>
> that too might make things more fluid.


Is there any way to do this without enabling debugfs?
_________________
“And even in authoritarian countries, information networks are helping people discover new facts and making governments more accountable.”– Hillary Clinton, Jan. 21, 2010
Back to top
View user's profile Send private message
energyman76b
Advocate
Advocate


Joined: 26 Mar 2003
Posts: 2040
Location: Germany

PostPosted: Fri Sep 11, 2009 4:33 pm    Post subject: Reply with quote

this is independent of debugfs - you need to activate scheduler debug.
_________________
Study finds stunning lack of racial, gender, and economic diversity among middle-class white males
Back to top
View user's profile Send private message
RealNC
Tux's lil' helper
Tux's lil' helper


Joined: 13 Apr 2008
Posts: 148

PostPosted: Fri Sep 11, 2009 4:36 pm    Post subject: Reply with quote

You don't even need to do that. Just change:

SCHED_FEAT(NEW_FAIR_SLEEPERS, 1)

to:

SCHED_FEAT(NEW_FAIR_SLEEPERS, 0)

in kernel/sched_features.h and rebuild.
Back to top
View user's profile Send private message
energyman76b
Advocate
Advocate


Joined: 26 Mar 2003
Posts: 2040
Location: Germany

PostPosted: Fri Sep 11, 2009 4:40 pm    Post subject: Reply with quote

RealNC wrote:
You don't even need to do that. Just change:

SCHED_FEAT(NEW_FAIR_SLEEPERS, 1)

to:

SCHED_FEAT(NEW_FAIR_SLEEPERS, 0)

in kernel/sched_features.h and rebuild.


has the disadvantage that you can not change the setting on the fly to compare directly.

I for example have seen some regressions with NO_NEW_FAIR_SLEEPERS.
_________________
Study finds stunning lack of racial, gender, and economic diversity among middle-class white males
Back to top
View user's profile Send private message
RealNC
Tux's lil' helper
Tux's lil' helper


Joined: 13 Apr 2008
Posts: 148

PostPosted: Fri Sep 11, 2009 4:53 pm    Post subject: Reply with quote

Can you elaborate so that others can try to reproduce?
Back to top
View user's profile Send private message
energyman76b
Advocate
Advocate


Joined: 26 Mar 2003
Posts: 2040
Location: Germany

PostPosted: Fri Sep 11, 2009 4:58 pm    Post subject: Reply with quote

RealNC wrote:
Can you elaborate so that others can try to reproduce?


sure. KDE 4.3.1 with composite temporary disabled. tvtime displaying tv.
konsole emerging something big (xulrunner for example) with -j5
cpu phenom II X4

without NO_FAIR tv is smooth (or mostly smooth)
with NO_FAIR tv is pretty jerky.

without emerge in the background both are fine

kernel is 2.6.31+reiser4

ondemand governor

c1e disabled.
_________________
Study finds stunning lack of racial, gender, and economic diversity among middle-class white males
Back to top
View user's profile Send private message
Etal
Veteran
Veteran


Joined: 15 Jul 2005
Posts: 1641

PostPosted: Fri Sep 11, 2009 5:37 pm    Post subject: Reply with quote

Thanks! I guess I'll add the scheduler debug since it would be hard to evaluate how much effect each has without being able to switch back and fourth.
_________________
“And even in authoritarian countries, information networks are helping people discover new facts and making governments more accountable.”– Hillary Clinton, Jan. 21, 2010
Back to top
View user's profile Send private message
RealNC
Tux's lil' helper
Tux's lil' helper


Joined: 13 Apr 2008
Posts: 148

PostPosted: Fri Sep 11, 2009 5:43 pm    Post subject: Reply with quote

energyman76b wrote:
RealNC wrote:
Can you elaborate so that others can try to reproduce?


sure. KDE 4.3.1 with composite temporary disabled. tvtime displaying tv.
konsole emerging something big (xulrunner for example) with -j5
cpu phenom II X4

without NO_FAIR tv is smooth (or mostly smooth)
with NO_FAIR tv is pretty jerky.

without emerge in the background both are fine

kernel is 2.6.31+reiser4

ondemand governor

c1e disabled.


Weird. Though I can't use tvtime (don't have a tv card), with mplayer it's pretty much the opposite. I can only get a smooth playing video with NO_FAIR... (In both cases with PORTAGE_NICENESS=19 set in make.conf.)
Back to top
View user's profile Send private message
energyman76b
Advocate
Advocate


Joined: 26 Mar 2003
Posts: 2040
Location: Germany

PostPosted: Fri Sep 11, 2009 6:18 pm    Post subject: Reply with quote

a friend of mine has a similar result. More jerkiness with NO_NEW...
_________________
Study finds stunning lack of racial, gender, and economic diversity among middle-class white males
Back to top
View user's profile Send private message
doubleagent
Guru
Guru


Joined: 15 Apr 2005
Posts: 444
Location: 127.0.0.1

PostPosted: Fri Sep 11, 2009 6:35 pm    Post subject: Reply with quote

Sometimes I think it's ridiculous to peg problems on the scheduler. Individual applications should endeavor to slice up workloads in such a way that the scheduler can optimize them.

It would be much more difficult for one scheduler to adapt to the million different quirky ways other applications do work than for each one of them to adapt to a single scheduler.

This does mean that scheduler performance tests probably aren't robust enough, which is a scary thought when considering how long it currently takes to run them (upwards of 8 hours).
_________________
shickapooka wrote:
i think they programmed [otw] based on a right-wing jewish-nigger-nazi, his gay, retarded, left-wing love slave with webbed feet, and their three headed cat that poops uncontrollably. the cat is also an apple fanboy
Back to top
View user's profile Send private message
energyman76b
Advocate
Advocate


Joined: 26 Mar 2003
Posts: 2040
Location: Germany

PostPosted: Fri Sep 11, 2009 6:47 pm    Post subject: Reply with quote

An app can never now when or if it is going to be pushed aside.
_________________
Study finds stunning lack of racial, gender, and economic diversity among middle-class white males
Back to top
View user's profile Send private message
sts
Tux's lil' helper
Tux's lil' helper


Joined: 02 Jul 2007
Posts: 97

PostPosted: Fri Sep 11, 2009 7:09 pm    Post subject: Reply with quote

doubleagent wrote:
Sometimes I think it's ridiculous to peg problems on the scheduler. Individual applications should endeavor to slice up workloads in such a way that the scheduler can optimize them.

WTF? Userland application developers shouldn't have to worry about scheduling, virtual memory, file system intricacies, etc. That's insane. I don't think applications should even have to worry about threading but the current situation is too crappy to avoid his much of the time. Though, Apple has had some good ideas.


Last edited by sts on Fri Sep 11, 2009 7:12 pm; edited 1 time in total
Back to top
View user's profile Send private message
doubleagent
Guru
Guru


Joined: 15 Apr 2005
Posts: 444
Location: 127.0.0.1

PostPosted: Fri Sep 11, 2009 7:09 pm    Post subject: Reply with quote

energyman76b wrote:
An app can never now when or if it is going to be pushed aside.
I'm not sure what you mean. All meaningful projects in the open source world are maintained because open source would not produce quality applications without this part of the development cycle (which is why games aren't very successful in open source - they have no maintenance phase). If you're suggesting that a change in the scheduler would cause an app to be 'pushed aside', by which you simply mean 'no longer run efficiently', the maintainers will fix that and deploy a new version, or speak with the kernel devs to have the changes modified if the code-base is too large.

But because everyone is viewing the scheduler as the devil, you don't see maintainers doing any conversing with the kernel devs (or at least _I_ don't)...only users.

If I were maintaining a widely used application that was suffering in efficiency, and were having to balance my time between getting it to run fast and implementing new features, I would probably let the users continue fussing at the devs and instead worry about getting new features out. Why would I need to make changes when changes can be made elsewhere by somebody else?

Now, if you're referring to package manager maintainers, distros like gentoo may find it desirable to sync certain stable components into the tree, which would make their job slightly more difficult but not impossible. That already happens anyway.
_________________
shickapooka wrote:
i think they programmed [otw] based on a right-wing jewish-nigger-nazi, his gay, retarded, left-wing love slave with webbed feet, and their three headed cat that poops uncontrollably. the cat is also an apple fanboy
Back to top
View user's profile Send private message
doubleagent
Guru
Guru


Joined: 15 Apr 2005
Posts: 444
Location: 127.0.0.1

PostPosted: Fri Sep 11, 2009 7:13 pm    Post subject: Reply with quote

sts wrote:
doubleagent wrote:
Sometimes I think it's ridiculous to peg problems on the scheduler. Individual applications should endeavor to slice up workloads in such a way that the scheduler can optimize them.

WTF? Userland application developers shouldn't have to worry about scheduling, virtual memory, file system intricacies, etc. That's insane.
Maybe not at the level you're implying. All I'm advocating is that developers should profile their applications or test them see how well they run.

One of the confusing parts to me is that application developers should be doing those things anyway. I wouldn't expect them to release software with piss-poor performance to anyone but other developers and yet that's what users seem to be experiencing: piss-poor performance.
_________________
shickapooka wrote:
i think they programmed [otw] based on a right-wing jewish-nigger-nazi, his gay, retarded, left-wing love slave with webbed feet, and their three headed cat that poops uncontrollably. the cat is also an apple fanboy
Back to top
View user's profile Send private message
doubleagent
Guru
Guru


Joined: 15 Apr 2005
Posts: 444
Location: 127.0.0.1

PostPosted: Fri Sep 11, 2009 7:18 pm    Post subject: Reply with quote

sts wrote:
I don't think applications should even have to worry about threading but the current situation is too crappy to avoid his much of the time. Though, Apple has had some good ideas.
I disagree with you and apple. Why take away such a rewarding part of the development experience? This is like that openmpi crap that's floating around now...at least that's not in the kernel. If this is done I'd prefer it to be in a library or in the language itself.
_________________
shickapooka wrote:
i think they programmed [otw] based on a right-wing jewish-nigger-nazi, his gay, retarded, left-wing love slave with webbed feet, and their three headed cat that poops uncontrollably. the cat is also an apple fanboy
Back to top
View user's profile Send private message
energyman76b
Advocate
Advocate


Joined: 26 Mar 2003
Posts: 2040
Location: Germany

PostPosted: Fri Sep 11, 2009 7:20 pm    Post subject: Reply with quote

and on how many systems are you going to test? FreeBSD 6? 5? 4? DragonflyBSD? Netbsd? Openbsd? Windows? Linux 2.2? 2.4? 2.6 pre cfs and post cfs? Solaris? AIX? MacOSX? Windows?

you can not profile them all. So the best way is to just write it with performance in mind and let the OS sort it out.
_________________
Study finds stunning lack of racial, gender, and economic diversity among middle-class white males
Back to top
View user's profile Send private message
doubleagent
Guru
Guru


Joined: 15 Apr 2005
Posts: 444
Location: 127.0.0.1

PostPosted: Fri Sep 11, 2009 7:22 pm    Post subject: Reply with quote

energyman76b wrote:
and on how many systems are you going to test?
The systems you're interested in your application running on, within your means. What kind of a question is this, anyway?
_________________
shickapooka wrote:
i think they programmed [otw] based on a right-wing jewish-nigger-nazi, his gay, retarded, left-wing love slave with webbed feet, and their three headed cat that poops uncontrollably. the cat is also an apple fanboy


Last edited by doubleagent on Fri Sep 11, 2009 7:23 pm; edited 1 time in total
Back to top
View user's profile Send private message
sts
Tux's lil' helper
Tux's lil' helper


Joined: 02 Jul 2007
Posts: 97

PostPosted: Fri Sep 11, 2009 7:23 pm    Post subject: Reply with quote

doubleagent wrote:
sts wrote:
doubleagent wrote:
Sometimes I think it's ridiculous to peg problems on the scheduler. Individual applications should endeavor to slice up workloads in such a way that the scheduler can optimize them.

WTF? Userland application developers shouldn't have to worry about scheduling, virtual memory, file system intricacies, etc. That's insane.
How so? How is that any different from profiling an application or testing it to see how well it runs? That's all I'm advocating anyway.

The inner workings of the kernel and hardwre are supposed to be abstracted away and transparent to application developers. If billy bob wants to write an email client he shouldn't have to dance around the CPU scheduler and count how many bytes of memory he is using. What happens when the scheduler is changed or he ports his app to a different OS? Profiling can be used to both fix inefficient program logic and fix problems related to OS performance but we'd all be better off if we didn't have to worry about the latter. In a perfect world to write an email app you'd just write code for handling email and presenting it to the user and not have to worry about OS architecture.

That isn't strictly how things work but that should be what we're shooting for.
Back to top
View user's profile Send private message
sts
Tux's lil' helper
Tux's lil' helper


Joined: 02 Jul 2007
Posts: 97

PostPosted: Fri Sep 11, 2009 7:25 pm    Post subject: Reply with quote

doubleagent wrote:
sts wrote:
I don't think applications should even have to worry about threading but the current situation is too crappy to avoid his much of the time. Though, Apple has had some good ideas.
I disagree with you and apple. Why take away such a rewarding part of the development experience?

Because it is tedious, time consuming, and error prone? You don't have to use their job queuing system, you can still use the manual thread management APIs...


Last edited by sts on Fri Sep 11, 2009 7:25 pm; edited 1 time in total
Back to top
View user's profile Send private message
doubleagent
Guru
Guru


Joined: 15 Apr 2005
Posts: 444
Location: 127.0.0.1

PostPosted: Fri Sep 11, 2009 7:25 pm    Post subject: Reply with quote

sts wrote:
...
ninja edit!

Besides, I'm not even vaguely talking about interoperability. That's a separate issue that you and energyman are trying to tie in. If you want that then do Java, but at least aim for decent performance.
_________________
shickapooka wrote:
i think they programmed [otw] based on a right-wing jewish-nigger-nazi, his gay, retarded, left-wing love slave with webbed feet, and their three headed cat that poops uncontrollably. the cat is also an apple fanboy
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Off the Wall All times are GMT
Goto page Previous  1, 2, 3  Next
Page 2 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum