xx-sources

eldiablo · Posted: Wed Oct 20, 2004 2:01 pm Post subject:

Have anyone tested this kernel on a AMD64 platform?
Is it runnning as smooth as on x86?
_________________
Always running the most experimental

Evil Dark Archon · Posted: Fri Oct 22, 2004 11:03 pm Post subject:

2.6.9-mm1 is out, and the crowd goes wild!
_________________
This post has been over explained for newb-informing purposes.

Registered Linux user 347334
Abit AV8-3rd eye, AMD Athlon64 3500+ 90nm, ATI Radeon x850 pro

xiphux · Posted: Mon Oct 25, 2004 1:54 am Post subject:

Sorry for the lack of updates; I have a rough midterm coming up this week, so I haven't really had much free time...

Ok, I thought about things a little bit. I think I'm probably going to stop basing xx on the latest mm's, for a number of reasons:
1. I want to use gensplash again.
2. akpm is a machine.
3. Since it's a testbed, while the newest stuff makes it in, sometimes busted stuff makes it in. For example, 2.6.9-mm1 has something in it that completely breaks lvm, making my system unbootable. I'm still trying to hunt down the source of it.
4. While I wish I could sit here and kernel hack all day like the old days, I don't really have the time anymore.

However, this does not mean that I'm going to drop mm. What I intend to do is follow mm's as they release just like before, but just checking out the new stuff. I'll add in stuff that's worthwhile, including, but not limited to:
1. Reiser4
2. scheduling fixes
3. patches that have a known and noticeable positive effect

But stuff like typo corrections, fixes for arches that no one here uses, etc, don't really need to go in. I'll let them take care of merging that.

Hopefully this will make things a little easier for everyone. It will be a little less frantic work for me, hopefully less broken things (on the whole), and probably more chances for actual releases.

So I'll start by syncing up to 2.6.10-rc1, and then poking through the massive list of mm patches and picking out the good ones. Sound good?

Evil Dark Archon · Posted: Mon Oct 25, 2004 2:51 am Post subject:

sounds very good, i'll be looking forward to the next cvs commit.
_________________
This post has been over explained for newb-informing purposes.

Registered Linux user 347334
Abit AV8-3rd eye, AMD Athlon64 3500+ 90nm, ATI Radeon x850 pro

chrissicool · n00b Joined: 26 Oct 2004 Posts: 1

I used xx-sources from the very beginning. I always liked the newest stuff. But I think it is a good choice not to base on mm too heavy. It breaks too often lately (due to the new development process?).

The only thing I miss in xx is the DSDT patch for initrds (http://gaugusch.at/kernel.shtml)

Looking forward for the next commit.

cc

xiphux · Posted: Tue Oct 26, 2004 2:04 pm Post subject:

I can add the dsdt initrd patch if you want. But I thought it was redundant since acpi has an option to include one right into the kernel... ?

In other news, after meticulous testing (or rather, starting with nothing, adding a couple patches, test booting, adding a couple more, etc...) I found the patch that was breaking my lvm. It was dio-handle-eof.patch, a patch to change the direct io code to shorten the read and return the number of bytes read if someone tries to read past the end of a device, instead of returning -EIO like before. It didn't occur to me because I don't know enough about the lvm code to know why exactly this breaks stuff. (I was trying to back out device-mapper and ide patches)

So now that that's fixed, hopefully I'll be able to make a commit soon.

[edit]
I'm committing right now. I've updated to 2.6.10-rc1, with some stuff from 2.6.10-rc1-mm1. Unfortunately, I had to leave out gensplash since spock's page doesn't have the 2.6.10-rc1 patches up, even though it says it does. I'll add it as soon as they do show up, though. Also, it seems I've done something to break xsched; I'm looking into it. I'm going to check out the cpu load balancing and vm stuff in 2.6.10-rc1-mm2, and see if it's worth adding.
[/edit]

[edit2]
... Is it just me, or did Con Kolivas do exactly what I did several weeks prior?
http://lkml.org/lkml/2004/10/30/75
[/edit2]

Evil Dark Archon · Posted: Sun Oct 31, 2004 1:54 am Post subject:

spock has the 2.6.10-rc1 patches for fbsplash up, i would patch it in my own tree myself but there is one reject that i don't know how to fix.
_________________
This post has been over explained for newb-informing purposes.

Registered Linux user 347334
Abit AV8-3rd eye, AMD Athlon64 3500+ 90nm, ATI Radeon x850 pro

xiphux · Posted: Wed Nov 03, 2004 3:01 am Post subject:

Sorry for the delay, been busy.

I tried the fbsplash patches, and while I fixed the rejects, there are still a bunch of fb changes that break compilation. (it's in linus.patch in -mm, which means it's about to be merged into mainline) And since linus.patch is a big rolled up patch, I can't track down each specific change. What I'll probably do is add it when the next mainline release candidate comes out, and from there watch the -mm fbdev changes closely, which will be easier since they'll come in initially in individual patch form.

The other reason I haven't been around is I've been looking into plugsched, Con Kolivas' version of the boot-time selectable scheduler system. The way it was done is almost _exactly_ the same way I did it, which is interesting. But since they (Con and William Lee Irwin III) have more experience with that stuff, they were able make things a bit better in certain spots. (for example, they knew how to correctly re-initialize the scheduler without resorting to architecture-specific boot parameters, minimized the number of duplicate functions, and other minor optimizations)
So I've been taking the time to undo my version and add in plugsched. However, it's almost done. Pretty soon (tonight or tomorrow), after just a bit more testing and after I finish un-breaking xsched, I can commit a version of xx with plugsched and parts of 2.6.10-rc1-mm2.

[edit]
Committing now. Has the worthwhile parts of 2.6.10-rc1-mm2, and plugsched. It's almost exactly the same as before, it's just that the boot parameter has changed from 'scheduler=' to 'cpusched='. Con nicknamed the default scheduler 'ingosched', so that's the name used to load the stock scheduler. That and the option moved from 'Extra xx options' to 'Processor features', since it's not an xx-specific feature anymore. Con has it so that the default bootup scheduler is an option, but actually compiling each scheduler in can only be changed via the embedded menu, otherwise all are compiled in (in case you were wondering where that option went).
Oh yeah, and un-broke xsched. Actually, made a lot of changes, so the version is at v03a. (in case you haven't figured it out, the last letter increments with minor updates and the number increments with major updates)
[/edit]

[edit2]
I'm going to be doing another audit/cleanout of the patches, so let me know if something you use is missing.
[/edit2]

Evil Dark Archon · Posted: Fri Nov 05, 2004 12:26 am Post subject:

it would be great if realtime preempt were put back in, i'm not sure how possible that is since xx-sources is not using the full -mm.
_________________
This post has been over explained for newb-informing purposes.

Registered Linux user 347334
Abit AV8-3rd eye, AMD Athlon64 3500+ 90nm, ATI Radeon x850 pro

xiphux · Posted: Fri Nov 05, 2004 2:01 am Post subject:

I'll see what I can do. I don't think it should be _that_ bad, since the large amount of preempt stuff that rt-preempt builds on is still in xx.
But I originally stopped keeping up with it because I stopped using preempt entirely; for some reason, that and whatever bad sectors on my hard drive were triggering lockups every time I tried to access that area of the disk. (/usr and /var... not good)

gilkyboy · n00b Joined: 05 Sep 2003 Posts: 40

xiphux, would it be possible to put a link to your sources in your first thread, be that editing or whatnot. When I want to try your newest kernel, I always forget what page the link is on. Keep up the great work.
_________________
A newbie is new enough not to be aware of a groups history and culture. An obnoxious newbie believes that a group has no history or culture. An intolerably obnoxious newbie thinks he can reshape the culture to his liking, without regard to anyone else

xiphux · Posted: Sat Nov 06, 2004 11:25 pm Post subject:

I put a link to the sourceforge page in the first post.

I hope to commit a version synced up with 2.6.10-rc1-mm3 soon. I've just been struggling with some mount problems created by the 4level page table patches added.
I've also been fighting with two relatively intrusive patches, hi-res timers and the realtime-preempt patch.
The problem with the hi-res timers at the moment is the sched domain tunables. Since you can set all the timings and settings for the domains during runtime, the system is a little more dependent on the timer system being accurate. (as opposed to having all hardcoded values before) And since the hi-res timers change the way the timer system works, it causes oopses in the sched domains.
The problem with realtime preempt is that it's very instrusive. For example, there are a number of new scheduling functions that involve priorities for mutexes, which means these functions have to be ported over to the other schedulers. So the sheer amount of stuff that has to be merged by hand is quite a lot, especially working around plugsched. Plus, it's such a dangerous change that one little thing wrong can cause the whole setup to oops. But I think it's just a matter of time before I get it.
I mailed my patch for the creation of scheduler specific sysctls to Con Kolivas, but he said he was planning on using sysfs instead. So, taking the cue, I wrote up a sysfs interface for plugsched. I mailed that too but haven't gotten a response yet. So it's very much like the /proc/sys/sched controls, except it's in /sys/sched/<scheduler name>/. I decided to leave the /proc sysctls in too, which is redundant, but I don't want people to freak out if their sysctls disappear.

[edit]
I updated cvs to -mm3, but I forgot to mention it, sorry. So I've already ended up updating to -mm4. Nothing has really changed that much; just plugsched is updated and a couple other miscellaneous things. I'm still working on rt-preempt; that's proving to be one tough patch.
[/edit]

Evil Dark Archon · Posted: Tue Nov 16, 2004 11:00 am Post subject:

2.6.10-rc2-mm1 just came out and it looks like most of the fb changes in -mm have been merged upstream.
_________________
This post has been over explained for newb-informing purposes.

Registered Linux user 347334
Abit AV8-3rd eye, AMD Athlon64 3500+ 90nm, ATI Radeon x850 pro

xiphux · Posted: Thu Nov 18, 2004 5:32 am Post subject:

Apologies, I've been ungodly busy. I'm syncing up with 2.6.10-rc2-mm1 now. To get things up quicker, I'll do a minimum of messing around with mm, just sync up the tree as it is, and then commit. After I have a working updated copy in cvs, I'll work on changes (fbsplash, etc).

[edit]
I'm committing a version synced up with 2.6.10-rc2-mm2 right now. After that's done, I'll start catching up on other patches.
[/edit]

[edit 20041201]
Yeah, it's been a long time.

I hate to say this, but unfortunately, I will not be able to work on xx-sources for most of the month of December. (Half-Life 2 was addictive like crack, the heat from the system toasted my laptop's hard drives as well as other hardware components... not a pretty situation) I can use my basic system, but my data drives with all my free space and kernel work are out of commission. I have ordered another laptop, but it still needs to be assembled and sent out, so it probably won't be ready for a few weeks. (Probably around winter break time)

So I haven't forgotten about xx-sources; I intend to continue. I just can't for a few weeks. If you're bored and want to work on it for the month I can't, I can give cvs access if anyone's interested.

Although the upside is that the new machine will be an AMD64, so I'll have a bunch of new things to work on and fix, my first target being Reiser4 on x86-64.
[/edit]

eldiablo · Posted: Wed Dec 22, 2004 10:08 pm Post subject:

So, is this project dead, or is it just looking like it is?

_________________
Always running the most experimental

Evil Dark Archon · Posted: Thu Dec 23, 2004 2:17 am Post subject:

It's just looking like it is, xiphux's computer got fried and he's waiting on a new one and it won't come until late this month or sometime next month. So don't expect any updates 'till January.
_________________
This post has been over explained for newb-informing purposes.

Registered Linux user 347334
Abit AV8-3rd eye, AMD Athlon64 3500+ 90nm, ATI Radeon x850 pro

xiphux · Posted: Fri Jan 14, 2005 3:39 pm Post subject:

"... and I am as the phoenix, arisen from ashes, reborn of flame..."

*ahem* Sorry. Anyway, I'm back. Just got the new computer yesterday, and got gentoo compiled and everything moved over today. So now, unfortunately, I need to catch up on the stuff I've missed with regards to kernel development, so that's what I'll be doing.

Well, actually, the first thing I need to do is get xx to even compile and boot on this amd64 machine...

I don't know how long catching up will take, since I don't really know how much I've missed. But if there's something new and cool that you'd like to see included, then go ahead and ask, because I'm so far out of the loop that it'll almost certainly be news to me...

So nothing really earth-shaking for you right now, but at least development has started again.

MrApples · Guru Joined: 13 Dec 2002 Posts: 511

welcome back, and good luck with all the catching up
_________________
http://www.whatsinyourbox.org -- Technology discussion, news, and more.

Evil Dark Archon · Posted: Fri Jan 14, 2005 11:18 pm Post subject:

4 words: cfq time slices patch: http://www.kernel.org/pub/linux/kernel/people/axboe/patches/v2.6/2.6.11-rc1/
_________________
This post has been over explained for newb-informing purposes.

Registered Linux user 347334
Abit AV8-3rd eye, AMD Athlon64 3500+ 90nm, ATI Radeon x850 pro

eldiablo · Posted: Sat Jan 15, 2005 6:48 pm Post subject:

Welcome back

_________________
Always running the most experimental

xiphux · Posted: Sun Jan 16, 2005 5:50 am Post subject:

I've kind of caught up. I think. I've synced up against 2.6.11-rc1 with some of -mm1, with mostly the same patch list as before, plus the cfq time slices patch, fbsplash, and a couple other little things from -ck. I'll be able to commit to cvs a little later.

The kernel boots and runs ok on my amd64 machine as well as my old x86 laptop, although there are more warnings in the amd64 compile I haven't fixed yet. However, I'm unable to use the kernel on my amd64 laptop because of a curious problem: while I'm on the console doing anything on the commandline or through directfb, the keyboard works fine, but when I get into X, it completely stops working. I can't type anything, can't terminate with ctl-alt-bksp, can't even use the magic sysrq keys. And once I quit out of X, it's fine again. I've reverted and it's still a problem even with stock 2.6.11-rc1, and I highly doubt it's a configuration change since it's pretty much the same config as the one I'm using for gentoo-dev-sources-2.6.9-r14 with just the extra features filled in, but I'm still looking into it.

Weird. But there's a thread on the lkml right now where people are having similar keyboard issues, so I'm following that. So I haven't really been able to test this kernel a lot, but we'll see how it goes. I just hope I haven't gotten too rusty at this...

[edit]
I'm a big dumb idiot. The event device numbers switched around between the kernel versions, so the keyboard and touchpad event devices switched. And it was trying to read touchpad mouse movements from the keyboard events, and the actual keyboard events weren't getting through... oops. Oh well.
[/edit]

xiphux · Posted: Thu Jan 20, 2005 9:06 am Post subject:

I normally don't bump a thread when I'm already the last poster, to avoid drawing attention. But I'm making another release, 2.6.11-rc1-xx2. CVS has been changing frequently, but it's been a while since the last release, so there are some significant differences.

Based on and applies to 2.6.11-rc1. Has most of the worthwhile parts of -mm1. Peter Williams, who took over the plugsched project from Con Kolivas, just released Plugsched version 2.0 not too long ago, but I chose not to convert over. There are two main differences in version 2 of plugsched: 1 - proc tunables are in sysfs now. But this patch came from me in the first place (plus I'm mentioned in the announcement... I like getting credit for my work), and it's been in -xx for a while now, so there's nothing new. 2 - he did an abstraction of the scheduler runqueue, so all the schedulers would use the same common runqueue. While good for plugsched overall, it's set up so that the static_prio is fixed across all schedulers. This presents issues for nicksched (and, in turn, xsched), since nicksched has a 1.5x bigger static_prio range. In the end, it wasn't worth the effort.

Staircase is updated to 10.3. Fbsplash is back in and works fine, even on my 1680x1050 widescreen. Re-added mapped watermark and the nvidia fix from -ck. I took out the cfq timeslices patch that was in cvs for a while - ck was seeing some serious issues with it on his mailing list, and I occasionally noticed less-than-satisfactory performance on my machine. It's just a bit too early.

There are a couple patches from the LKML that redo the way memory is allocated. Theoretically, it should make better and more efficient use of your ram. I also added the genetic lib that everyone else jumped on while I was gone. It only tunes the AS io scheduler at the moment, since that was really simple to implement (aside from a little fiddling to make it work correctly with the modular schedulers). Adding it to each and every cpu scheduler would be a real pain, and genetically tuning the cpu scheduler is a terrible idea, anyway. Although I might extend it to cover CFQ too if I get bored sometime.

I've compiled and test booted all the cpu schedulers (ingosched, staircase, nicksched, xsched) on both i386 and amd64, it all works fine. I'm using it on my main amd64 machine right now, so amd64 users can be fairly certain this kernel works for them. (Well, it all works for me... hopefully it will for everyone else too)

As always, use cpusched=foo to choose a scheduler at boot. Should be up at http://sourceforge.net/projects/xx-sources/ within the hour, and then CVS after that.

Evil Dark Archon · Posted: Thu Jan 20, 2005 10:27 am Post subject:

xiphux · Posted: Thu Jan 20, 2005 10:49 am Post subject:

Yeah, Fortuna did have some issues. I remember that it would fire off hundreds of smp debugging messages, but I don't know enough about RNGs and cryptography to fix it, and I think the author only did it as a side project or something.
I guess it's in more for the coolness factor than anything....

Evil Dark Archon · Posted: Thu Jan 27, 2005 5:42 am Post subject:

new -mm out (2.6.11-rc2-mm1), should probably leave the fortuna rng out because of the problems its having.
_________________
This post has been over explained for newb-informing purposes.

Registered Linux user 347334
Abit AV8-3rd eye, AMD Athlon64 3500+ 90nm, ATI Radeon x850 pro