2.6.12-rc2-JadeX1 [[EBUILD ADDED]]

[Tiger683]

JadeX Sources based on Realtime Preemption work by Ingo Molnar
Included patches put together and made to peacefully coexist by Tiger683
See the credits section for further contributors.
REISER4 is officially working with PREEMPT_RT enabled!!!!! Yehaw! *
_don't_forget_to_switch_off_4K_STACKS_when_using_it !!!!!!!!!
On request, i might create a patch with only RTP against a R4 patched kernel...
* See notes on architectures tested.
<=========================================================================>
Legal:
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License
as published by the Free Software Foundation; either version 2
of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

<==========================================================================>
Ok, now to the point:
this patchset is an attempt to bring the RTP to the Desktop environment
by providing it in a fairly stable form along with a few other patches enhancing
the features of the kernel and/or increasing the interactivity.
<==========================================================================>

realtime-preempt-2.6.12-rc2-V0.7.44-01 ------------> YEAH, THIS IS ___THE____ SH*T

RT_44-01_to_44-02.patch ------------> update for RTP

reiser4_28.03.05_for_2.6.12-rc1.patch ------------> reiser4, now with RTP-compatibility (not included in broken-out! -> request

2.6.12-rc2-kj.patch ------------> Kernel janitor's patchset, various small fixes from ongoing
kernel-hackers

fix-module_param_string-calls.patch ------------> umm, from mm, fixes two driver modules

320-irqbal_fast ------------> faster irq-balancing ( SMP ? )

390-slabtune ------------> slab tuning from mjb

bug-fix-in-usbdevfs.patch ------------> some usb bugfix

legacy-remap_page_range ------------> some legacy stuff still isnt converted to remap_pfn_range, lets make it happy...

stack-limit ------------> disable stack-limit on x86_64

tiocgdev ------------> my GF runs SuSE, she needs it during boot

tty-output-lossage-fix.patch ------------> from mm, prevents loosing data in *tty

x86_64-ia32-icecream-fix.diff ------------> x86_64 cleanup/fix (no idea actually)

oprofile-add-check_user_page_readable.patch ---------> needed to compile OProfile

quota-possible-bug-in-quota-format-v2-support.patch------> bugfix for quota v2

1g_lowmem1_i386.diff ------------> 1G Ram while staying in Lowmem

cddvd-cmdfilter-drop.patch ------------> burn cds as non-root

cfq-iosched-update-to-time-sliced-design ------------> cfq-ts from mm (akpm)

mapped_watermark3.diff ------------> reduce swappiness

nvidia_6111-6629_compat2.diff ------------> enables partial compatibility for NVidia proprietary module

fuse-2.6.12-rc1.patch ------------> fuse-fs stripped from mm

inotify-0.22-rml-2.6.12-rc2-2.patch ------------> Inotify V 0.22

squashfs2.1-r2-patch ------------> compact read-only fs

cflags-selection.patch ------------> mmmm, ricing......

daconfig-2.2.0.patch ------------> prettier Kconfig screen

lufs-0.9.7.patch ------------> well, LUFS

2.6.12-rc2-jadeX-r1_name.patch -------------> Say my name, b****

<==========================================================================>
Pending patches:
vesafb-tng + fbsplash -------> This is a no-go for now, it crashes the system at boot or hardlocks, working on it
but dont expect anything anytime soon
swsusp2 ---------------------------> This is coming up soon, although with no warranty, probably in next release together
with requested features
Win4Lin ----------------------------> Same as SWS2, although might drop if absolutely noone bugs me about it

Dynamic Tick ------------------- > I sure hope to get this working, havent even tried to patch it yet.

prio_list-mutex-for-RTP.patch ---> BIG SUCCESS, merged into X2

hrtimers ----------------------------> ooh, i will try probably instead of dyna_tick <==========================================================================>
Known issues:

1) VMWare modules seem to lock system after getting modprobed if
you try to run something afterwards as root. will look at it sometime soon.

_ONLY_ Enable Kernel Debugging in kernel options for bugreports!
Realtime Preempt introduses false warnings in dmesg under certain circumstances!!!!!!!!!!

Before posting a bug, be sure to disable any extra CFLAGS and recompile the kernel. If problem
remains, report it here. Do not post bugs already listed in _KNOWN ISSUES_ !
This was only tested on x86 UniProcessor! For other arch's and/or x86_64 and/or SMP i need your
active help!
<===========================================================================>

FILES:
AT SOURCEFORGE:
http://sourceforge.net/projects/jadex-sources/
-> broken out and the ebuild (gzipped) available at SF.net only until i find an alternative mirror.

UPDATE:
ok, this is my direct mirror, my geocities space, no idea how much BW i have there, but it worx for now.....
patch-2.6.12-rc2-jadeX1.gz
jadeX-sources-2.6.12_rc2-r1.ebuild.gz

CREDITS:
Ingo

Con

mjb

SuSE Labs

no-sources

nitro-sources

predatorfreak for support and will to host on his sf.net before i got
my project there

feld for the ebuild

<==========================================================================>
Hunt me down on irc.debian.org #jadex or Tiger683@jabber.org or this forum.

To be updated...

[slarti`]

Why?

[cokey]

explain what is PREEMPT_RT is please

[Tiger683]

slarti' : It has __at_least__ scientific value, and for me it has improved the desktop-interactivity A LOT.
no matter what ppl say, i experienced what is more than just audiophiles' need....about the release name... im a fan of ciaranm, as a dev, no matter what i think of his view on certain things..

cokehabit: Try it.... if i begin to explain a flamewar breaks out.... sorry____

[ewan.paton]

what is it you need to do to enable realtime preemption, i read up on it a while back but never bothered trying anything, does it just work or do progs have to have conditions applied to them.

also is this about realtime cpu or disk access, i rember hearing about a patch that alowed disk access renicing which sounded cool but if this alows disk access premption it would be very useful.

[predatorfreak]

what is it you need to do to enable realtime preemption, i read up on it a while back but never bothered trying anything, does it just work or do progs have to have conditions applied to them.

also is this about realtime cpu or disk access, i rember hearing about a patch that alowed disk access renicing which sounded cool but if this alows disk access premption it would be very useful.

Disk IONICE values where implamented in cfq-ts. This is about realtime CPU access, by reducing latency to almost nothing. Things under a Realtime Preempt kernel will be able to respond very fast because there's almost no latency.

[rhill]

oh dear god.

[predatorfreak]

oh dear god.

Penny for your thoughts? :\

[ewan.paton]

what is it you need to do to enable realtime preemption, i read up on it a while back but never bothered trying anything, does it just work or do progs have to have conditions applied to them.

also is this about realtime cpu or disk access, i rember hearing about a patch that alowed disk access renicing which sounded cool but if this alows disk access premption it would be very useful.

Disk IONICE values where implamented in cfq-ts. This is about realtime CPU access, by reducing latency to almost nothing. Things under a Realtime Preempt kernel will be able to respond very fast because there's almost no latency.

so how do you tell the os what is to be preempted or is it just based on noce values, for that matter how would i use cfq-ts to reduce certain programs disk priority

ive an x86_64 install i'll give this a bash when i reboot from my 32bit install though i woulnt be able to test any reiser4 stuff obviously

[predatorfreak]

what is it you need to do to enable realtime preemption, i read up on it a while back but never bothered trying anything, does it just work or do progs have to have conditions applied to them.

also is this about realtime cpu or disk access, i rember hearing about a patch that alowed disk access renicing which sounded cool but if this alows disk access premption it would be very useful.

Disk IONICE values where implamented in cfq-ts. This is about realtime CPU access, by reducing latency to almost nothing. Things under a Realtime Preempt kernel will be able to respond very fast because there's almost no latency.

so how do you tell the os what is to be preempted or is it just based on noce values, for that matter how would i use cfq-ts to reduce certain programs disk priority

ive an x86_64 install i'll give this a bash when i reboot from my 32bit install though i woulnt be able to test any reiser4 stuff obviously

I can't answer all those questions, but I can answer a few.

On preemptation, there are points in the kernel code where preempting is made possibly. I am no expert, but I believe preempt_enable and preempt_disable can create preemptation points. I might be wrong on that though. Preemptation has little todo with nice values.

For cfq-ts, I couldn't tell you exactly, I believe that it functions via normal nice values and you can define nice values in a C file by scheduling class (e.g SCHED_RR, SCHED_ISO, SCHED_OTHER).

~predatorfreak

[Jake]

ive an x86_64 install i'll give this a bash when i reboot from my 32bit install though i woulnt be able to test any reiser4 stuff obviously

Why not? Namesys hasn't been getting any AMD64 bug reports lately. Do you know something I don't? I'm running an all-reiser4 AMD64 system right now.

[predatorfreak]

ive an x86_64 install i'll give this a bash when i reboot from my 32bit install though i woulnt be able to test any reiser4 stuff obviously

Why not? Namesys hasn't been getting any AMD64 bug reports lately. Do you know something I don't? I'm running an all-reiser4 AMD64 system right now.

Maybe he doesn't have reiser4?

[Tiger683]

Heya,

this is more a conceptual release where i was trying to keep everything working,
and the work on getting r4 stable with this was enough for this one....

Realtime Preemption is about reducing the latency.... if a task times out, runs out of timeslice or
another task with higher prio and/or harder nice value comes up in a runqueue, the other one has
to get lost immediately..... It's reached through change of locking mechanisms in the Kernel,
and that fundamentally, more and more semaphores get replaced by mutex etc.

hmm, i wonder if i should tke a look at this dynamic tick patch now.....


cheers,

T

[Chaosite]

Heya,

this is more a conceptual release where i was trying to keep everything working,
and the work on getting r4 stable with this was enough for this one....

Realtime Preemption is about reducing the latency.... if a task times out, runs out of timeslice or
another task with higher prio and/or harder nice value comes up in a runqueue, the other one has
to get lost immediately..... It's reached through change of locking mechanisms in the Kernel,
and that fundamentally, more and more semaphores get replaced by mutex etc.

hmm, i wonder if i should tke a look at this dynamic tick patch now.....


cheers,

T

I don't think so. Task preemption (AKA, userspace) is handled just fine by the schedular.

Realtime preemption is about making the kernel preempt for tasks, so that latency in waiting for syscalls will be reduced.

This is very good for audio people, for instance.

[Tiger683]

Ummm, ok, thats what everybody say, it may be true, but i have major interactivity improvement overall....

so if my desktop reacts faster AND my audio profits, it looks perfectly fine for me...

cheers

T

PS: No, i dont want a discussion about the sense of RTP here....if anyone does, go to OTW and start a poll like "Is RTP worth trying"

[Tiger683]

Ok, im currently testing the

Code: Select all

plist-mutex-for-RTP.patch

patch from montavista.
i think it might be another thing to get merged first, a few more and a release bump is likely..

T

[darklegion]

Okay the performance seems otherwise good,especially as far as preventing audio skipping is concerned but I got some pretty serious errors when I tried recompiling the kernel modules for vmware,which caused any apps launched after it to not startup at all.All apps already loaded continued to work smoothly(I'm still running right now) .Here's the first part of the message,from dmesg:

Code: Select all

BUG: Unable to handle kernel NULL pointer dereference at virtual address 00000069
 printing eip:
c02e590b
*pde = 00000000
Oops: 0000 [#1]
PREEMPT 
Modules linked in: vmnet vmmon joydev xpad snd_pcm_oss snd_mixer_oss snd_intel8x0 snd_ac
97_codec snd_pcm snd_timer snd snd_page_alloc smbfs nvidia sch_sfq evdev usbhid 
nvidia_agp sr_mod cdrom agpgart ide_scsi sg ohci_hcd rtc
CPU:    0
EIP:    0060:[<c02e590b>]    Tainted: P      VLI
EIP is at sk_alloc+0x1b/0x170
eax: 00000001   ebx: 0000000e   ecx: c14fe000   edx: 00000020
esi: d2b99c00   edi: d2b99c0c   ebp: 00000001   esp: c6907de8
ds: 007b   es: 007b   ss: 0068   preempt: 00000001
Process vmnet-bridge (pid: 10527, threadinfo=c6906000 task=d4eb45d0)
Stack: c036b857 c0419700 c14fe000 d2b99c00 0000000e d2b99c00 d2b99c0c 00000000 
       e0b2d322 00000010 00000020 00000001 00000000 d2b99c00 c14fe005 d2b99c11 
       d2b99c0c e0b2d575 d2b99c00 d2b99c0c c0420d20 d2b99c00 c14fe000 d2b99c00 

EDIT: Minor correction,seems to be only things that are launched as root or require root privaleges won't work,but anything run as a normal user works fine.I am also seeing a lot of IRQ# processes which may be normal on this kernel.

EDIT 2 : Source for kernel modules is GPL and you can grab the latest source from here: http://ftp.cvut.cz/vmware/ (get the latest vmware-any-any-update)
IIRC "Petr" from the vmware forums maintains the drivers,so you can ask him there if you need more info : http://www.vmware.com/community/index.jspa?categoryID=1

[Tiger683]

darklegion:

Can you give me a link to the kernel modules?

I might take a look at it and if i get it to work, ill make a patch with it that applies to the kernel and makes it selectable in Kconfig....
Or are these not GPL? if not i might just make a patch for these.
Anyway, ill get back home in about 3-4 hours, so if you have a link give it, if not email them to
Tiger683 <AT> gmx <DOT> net

EDIT: Thx, ill look at it

[hyp0r]

slarti' : It has __at_least__ scientific value

Yes. At least...

[Tiger683]

slarti' : It has __at_least__ scientific value

Yes. At least...

Having english not as my native language: i meant in my opinion it has much more value,
but it's what stays if i __unlikely__ took all the negative arguments floating around for valid...

T

[hyp0r]

It's not the negative side of it. I can only ask the same as slarti did:

Why?

I don't understand, what this fuzz and the comic-style posting is about. I'm into realtime applications for quite _very long_ time now, and I can tell you that you _cannot_ patch a realtime-scheduler into the kernel with the expectation that everyting will run in realtime from now on.
Have you read RTAI's or RT-Linux' concept? Why do you think there is a separation between the conventional linux running in its own thread and the "realtime"-world running deadline-aware processes?

Answer me the following questions to point out the real use of this over patches, that Andrew Morton already included into his mm-sources, that also enable low-latency scheduling with sensitive preemption:

- What kind of scheduler is it (LL, EDF...)??
- How are deadlines for processes calculated in your solution?
- How can I control the sensitivity of critical processes just like memory-swappers?
- What happens to a non-deadline-aware process at all?
- What happens if a crucial process is being interrupted in favor of another process?
- Is the resource-lock concept still the same (RT-OSes usually use priority ceiling, priority inheritance etc.)? Which are used?
- Why do just "audio applications" benefit from these patches? The kernel does not make any difference between audio data and graphics...
- Why should software suspend use any realtime-aware environment if it's the only process running at this time, that shouldn't be interrupted anyway??
- What does fbsplash have to do with this at all?
- Do you have any real scientific evaluation on this? Latency of processes before/after, swappiness, behaviour, consistency etc.

Nowadays, I believe one can come by and say he's invented the ultimate quantum-register machine and I'd give him a chance to prove it. But he may never dare saying something without any proof as this will be fatal.

Now it's your move to convince the public. Papers please.

[Tiger683]

Ok, at first you might expect me to say something like "stop flaming my thread", but its not me and i respect your doubts
about the sense of it.
To be honest, the _primary_ reason for this was both not doing the same as (loosely said) many other patchsets already include AND
following what i experienced in my desktop environment, so better interactivity of applications (i don't play games, but non-skipping audio
and immediate response of applications are also a plus i think ).

- What kind of scheduler is it (LL, EDF...)??

scheduler is not changed, its ingosched ( O(1) in Big-O notation for "how expensive" ) ( You mean cpu-scheduler, right?

- How are deadlines for processes calculated in your solution?

__NOT__ my solution, ask ingo, need email address?

- How can I control the sensitivity of critical processes just like memory-swappers?

I think you know it better than i do....

- What happens to a non-deadline-aware process at all?

Ask the process... Its not about deadlines and timeslices, its about locking rescheduling fast...

- What happens if a crucial process is being interrupted in favor of another process?

If it has higher priority (crucial) i don't think it will be a problem....

- Is the resource-lock concept still the same (RT-OSes usually use priority ceiling, priority inheritance etc.)? Which are used?

From what i know prio-ceiling is not being used here... Fusyn might use this approach, RT mutex does not... (i did not contact an expert, so
dont beat me if Fusyn doesn't )

- Why do just "audio applications" benefit from these patches? The kernel does not make any difference between audio data and graphics...

Umm, not just audio, instead of theory (which is your best friend as i can see) i have the experience being made currently.

- Why should software suspend use any realtime-aware environment if it's the only process running at this time, that shouldn't be interrupted anyway??

umm, swsusp2 isa ugly hack, having to be fixed to even compile with full palette of kernel features enabled.
it changes the code for initializing and manipulating of process workqueues and restricts the interruption where itneeds it, i never said it has
any problems with RTP....but it certainly doesn't NEED it...

- What does fbsplash have to do with this at all?

well, it does not work
i think i know what it is...
it initializes a spinlock on the framebuffer to perform it's stuff undisturbed,
but the compat_semaphore that is left as semaphore for tasks not able to perform
under preempt_RT just doesn't solve the problem and the thing deadlocks... shortly before releasing the framebuffer device

- Do you have any real scientific evaluation on this? Latency of processes before/after, swappiness, behaviour, consistency etc.

Latency of processes: lower, at least as far as my perception goes.
swappiness: umm, with MW3 its not really noticeable anyways,
behaviour/consistency: well, as this approach (by Ingo, not me... ) is still being developed and far from perfect, it has rough edges,
so it is possible to overinteract your desktop ad get your audio to actually DO jump, but its out of the range of what one would do in allday..
And sorry for not being scientific like you expect it. About things im not sure about, i rather keep my mouth shut and leave what
i believe is "scietifically" right, than make a clown out of myself...
So, lets end this discussion, or otherwise
you may file a bugreport in this thread...

cheers

T

[ewan.paton]

i get a kernel panic when i compile in cpufreq suport, without it it boots fine this is an amd64 in 32bit mode, i didnt try modules as i had stuff i needed the pc to do but will try later

[hyp0r]

Yes, I have a bugreport. The bugreport is about your documentation, which should be sane.

Ok, at first you might expect me to say something like "stop flaming my thread", but its not me and i respect your doubts
about the sense of it.

I'm glad you didn't. Those who start behaving like this, because their "idea" is criticized qualify themselves as incompetent people. Morons, as some people call those.

scheduler is not changed, its ingosched ( O(1) in Big-O notation for "how expensive" ) ( You mean cpu-scheduler, right?

Cpu-scheduler? This one regulates itself and thanks to dynamic scheduling, your O(1) is more often referred to as CPI=1. And static scheduling is compiler-based, not kernelbased. No, I did not mean cpu-scheduler but the task-scheduler. I thought this was clear by mentioning LL and EDF.

- How are deadlines for processes calculated in your solution?

__NOT__ my solution, ask ingo, need email address?

Eh? Then why are you talking about realtime then? Realtime implies deadlines. And if you don't have deadlines, you don't have realtime.
RT-Scheduling is based on deadlines otherwise the scheduler does not know how to schedule.
So, what is the "realtime", that you talk about?

Ask the process... Its not about deadlines and timeslices, its about locking rescheduling fast...

Ok, you say "locking rescheduling". _What_ does it reschedule? My CPU-dispatcher already enables rescheduling by preempting tasks with important desires, just like I/O. But that's an invention, which is more than 30 Years old.
You are not going to tell me this is new, are you?

Umm, not just audio, instead of theory (which is your best friend as i can see) i have the experience being made currently.

Feeling is not really scientific, is it? I have killed my swap today and running everything from memory on the expense of killing processes, when my memory is full. There has been a nice tool for benchmarking the dma-usage and latency of audio transfers, but I don't remember its name.

About things im not sure about, i rather keep my mouth shut and leave what
i believe is "scietifically" right, than make a clown out of myself...

Ahm, yes, it's like that. But when you open your mouth and talk about "special features" you should be able to explain. That's all I expect. I don't intend to make a clown out of s.o., but I take topics like this one serious enough to ask questions about it. Some people are afraid of being asked and therefore should not participate in such a discussion.

So, lets end this discussion, or otherwise

I will stop, if the discussion gets annoying for the readers including me. I think this discussion is very interesting and may lead to real interesting insights. If you don't think so, then simply don't answer.

[Zinic]

Just like to thank Tiger for the nice work and all. Booted up fine and really does seem faster IMO.
Hope to see more of this patchset!