Upstart, InitNG and other SysVInit replacements

[leo.fontenelle]

[gwn=20050509]The May 9th, 2005 edition of Gentoo Weekly News[/gwn] told us about Jimmy Wennlund's InitNG, a replacement for the old and inefficient sysvinit. According to the site, "this asynchronous execution can dramatically improve boot time by better utilizing the system resources", and "this is the most advanced init system ever seen", in part because of "the possibility to add new features as plugins that can be loaded into initng in runtime."
Now Ubuntu is seeking for its own replacement for init. Upstart brings the concept of an event-driven init system, taking benefit from hotplug to decide when to start processes. Eventualy upstart should even replace crond and inetd (InitNG has similar plans to cron). Besides a dependency awareness (like the current gentoo initscripts), parallel startup (idem) and better hardware utilization (like InitNG and others), Upstart is supposed to bring response to ACPI events, avaliability of network connections and so on. One can learn more from Scott James Remnant's blog. According to the main page, InitNG is already seeking for a event-basedness plug-in.
Both replacements aim for becoming universaly used by Linux distributions (maybe other *NIX too?). I'd like to know some news or opinions on any of those becoming accepted in mainstream Gentoo and other distros.

[.t.]

At the moment, I think Upstart is better, because the change is gradual. I mean - it supports SysVInit scripts as well, while they get rewritten to work with the new format. InitNG was quite cool when I tested it, but the lack of support for every service was disappointing.

[mmmarcin]

I use initng for one or two weeks now, and must say it is great - even if i had to correct scripts for apache and postgres, but is it real problem? It seems initng in gentoo is already usable. And I don't think scripts format's change is anything that should block progresses - it is matter of minutes to write one.

I would test upstart and would do it right now (i.e. installing this ubuntu package which is available) but my ISP won't let me do it untill friday. If it is as fast as initng, then great. If not, then I will wait untill it is.

[EmmEff]

InitNG was quite cool when I tested it, but the lack of support for every service was disappointing.

Are these mechanical changes? Could a script be written to convert existing SysVinit scripts over to InitNG?

[.t.]

No. InitNG .i files are just a configuration files, containing the command needed to start the service, and the dependencies of the same, and whether it conflicts with any other service. A SysVInit script does all this checking itself, so only a very intelligent application could do the conversion - be able to tell what commands were starting the service rather than any other checks.

[EmmEff]

Ok, thanks for the info.

Given the nature of the problem, the number of services, and the size of the community, I don't foresee major issues getting all init scripts moved over to the new format. The biggest problem to me is supporting both formats concurrently. I am not sure which provisions exist to allow this. I've just started paying attention to this stuff over the weekend, actually.

[mikegpitt]

I said a long time ago on some thread that I wish gentoo would get behind one of the new init systems. Sysvinit works, but is old. The support of gentoo behind a next-gen init system would probably accelerate its development.

Fast boot time is a deserible feature as most people deem how fast soemthing boots with how well it works... sad but true.

[EmmEff]

It probably would be best to get behind Upstart... the more Linux dists behind it, the more third-party service support and adoption will exist.

I installed it on my Ubuntu Edgy installation today. I didn't delve too far into it, but it seems to work in compatibility mode. They're not planning on making it official for about another year or so. If Gentoo adopted it, the sooner the better.

[playfool]

I think Upstart has a better chance of getting adopted, it has wellknown developers and what is potentially one of the biggest distributions pushing it. Also we are seeing a lot of interest within Fedora for Upstart, it fits our needs well but initng will be supported as well via Extras.

One thing Upstart has going for it is that it's a gradual approach and that will certainly help with adoption just from a rollout POV as it would be easier to handle for vendors.

[.t.]

As I said, I agree. Compatibility mode is a big step towards developers producing new scripts; they can do it if and when they want, but when most have changed theirs will still work.

[leo.fontenelle]

I don't think scripts format's change is anything that should block progresses - it is matter of minutes to write one.

I don't package software for any distribution, but I think yes, it could block progress. The script translation would require a policy, which would take effect as long as the "deprecated" scripts are produces by software developers. Probably Gentoo suffers with that also, since it has its own kind of init scripts. There is a long way for an alternate init to become an standard: it must be used in most Linux distribution as well in *BSD and other UNIX-like systems; only then software developers would start writing scripts for the new system.

[mmmarcin]

You are right, it can block, but it should not, i.e. I don't won't it to do so If one init system is better than other or distro developers see it that way, it should be used. Maintaining work must be done anyway, and I would prefer maintaining initng i files to SuSE start scripts. But it is my personal opinion, and devs from distro like desktop, rpm ones won't share it.

And see - format's change blocks using new format (== new init system and it's features), so what can we do about it but use it more?

Gentoo is better this time, because it can have several init systems and user decides which one want, e.g. uses overlay. It is metadistribution, isn't it? Now I safely can use initng with gentoo, and main services worked for me as they ought to - udev, alsa (even better for me), checking fs, etc etc. If someone wants something less tested, then he should correct/write scripts (more like configuration than shell script, initng even uses /etc/conf.d) and maybe share them. And this is the main thing I thought when I wrote cited sentence.

Try it: install layman, add overlay with initng, install, you can change init systems between boots and there is no blocking with current init. I would like to hear what is not working for you using default scripts. There are some instructions on gentoo-wiki.com and initng site.

[leo.fontenelle]

Sorry, but I won't be able to test InitNG these days. I reboot my computer very seldom, and I need it running to do some translation and related stuff.

[mdeininger]

just dug out this thread and thought i'd throw my 2 cents in, since the thread title included "other SysVInit replacements".

i'm hacking on my own init system right now: einit. it's pretty fast and stable right now, and writing configuration entries for services is easy as pie (like initng: just add path to binary or commands and dependencies). it has some features that i haven't seen in initng and upstart yet, like for example that you can have more than one "module" provide a certain service, and if one fails to get up, it'll try to enable backup services.
then there's service groups, (can also be misused to alias service names) and i made some provisions to aid in accessibility (there's even a module that allows for tts feedback using festival to vocalise error messages, for those of us who can't/don't want to read logfiles and error messages).
it's a mix between the "traditional" way to parallelise things (i.e. lay out plan of what to do based on dependencies and then do it) and the event-based approach, and i'm looking into some ways to extend my current event-framework to allow for things to be done when certain system-events happen (nic cable plugged in, someone calling your number and your isdn-card/modem detects it, harddrive dying, system temperature rising, power failure, stuff like that).
i'm also working on connecting different instances of einit on different machines, so that important information can be passed from one machine to the others, for example to allow for automatic nfs mounting, or to allow for automatic load balancing, clustering and some interesting encryption-experiments.
the configuration file is written in xml, btw, so it's pretty easy to modify and should be fairly straightforward.

anyone interested should check out the two links in my signature; flames should go to the project's sf.net forums or one of my im addresses, while bug reports should only be sent to the bugtracker on sf.net

[Q-collective]

just dug out this thread and thought i'd throw my 2 cents in, since the thread title included "other SysVInit replacements".

i'm hacking on my own init system right now: einit. it's pretty fast and stable right now, and writing configuration entries for services is easy as pie (like initng: just add path to binary or commands and dependencies). it has some features that i haven't seen in initng and upstart yet, like for example that you can have more than one "module" provide a certain service, and if one fails to get up, it'll try to enable backup services.
then there's service groups, (can also be misused to alias service names) and i made some provisions to aid in accessibility (there's even a module that allows for tts feedback using festival to vocalise error messages, for those of us who can't/don't want to read logfiles and error messages).
it's a mix between the "traditional" way to parallelise things (i.e. lay out plan of what to do based on dependencies and then do it) and the event-based approach, and i'm looking into some ways to extend my current event-framework to allow for things to be done when certain system-events happen (nic cable plugged in, someone calling your number and your isdn-card/modem detects it, harddrive dying, system temperature rising, power failure, stuff like that).
i'm also working on connecting different instances of einit on different machines, so that important information can be passed from one machine to the others, for example to allow for automatic nfs mounting, or to allow for automatic load balancing, clustering and some interesting encryption-experiments.
the configuration file is written in xml, btw, so it's pretty easy to modify and should be fairly straightforward.

anyone interested should check out the two links in my signature; flames should go to the project's sf.net forums or one of my im addresses, while bug reports should only be sent to the bugtracker on sf.net

Sounds interesting, too bad development seems to be rather slow (2 point releases in almost 5 months?)

[mdeininger]

Sounds interesting, too bad development seems to be rather slow (2 point releases in almost 5 months?)

i don't like to pack up and release point releases of alpha software. i'm trying to only release (or commit to cvs) code that is solid and not hack-ish and has at least been tested on two distinct operating systems on two distinct architectures.

next release will be first or second week next month, and it'll be the first beta that should be stable. i also hope to get the documentation up to date till then.

[iamarug]

Since I reset my computer alot, boot time has become somewhat of an annoyance for me. So far I have bootcharted initng time down to 24 seconds but I know I could get it faster if I knew how to mess with the config files.

Even though it would be quite hackish, I would manully put mount commands for my filesystems in place of having a script read the fstab and do parsing/awk/etc...

I have a feeling I could get it to around 15 seconds if I were to rewrite alot fo the init scripts.

mdeininger, I think I may delve into the init system you have and see how it works. Unfortunately it seems initng isnt so actively developed anymore and I wasnt able to understand some parts of their init scripts myself in the 1-2 hours I have spent setting it up.

Maybe this weekend I will refresh my scripting skills and test it to see exactly how much time I would be able to save with manually writing init scripts.


Also, seeing as how disk seeks are relatively quite expensive, it may be wise to find a way to batch the scripts into 1 file and so on.

What I would give for a 10 second boot time on my desktop after grub

[mdeininger]

hehe, go for it! 10 seconds is well possible

you're right about disk seeks. it seems like a small thing to do, but it does take a few seconds... i noticed that when i started hacking on a binary-module for mounting filesystems ( before that module i had just used a mount -a command ). scanning all block devices on my computer for filesystem type, mount state, uuids and labels can take up to two seconds... glad i found a way to keep that time down (without even using things like mmap, i'll have to look into that for version 0.10.2 )

[smlgbl]

I had a look at some of the suggestions made here, and honestly so far upstart seems the most promising and reasonable choice to me.
Einit also seems good, but somehow i didn't manage to see the real concept behind it, especially when it comes to event-based stuff, which upstart is basically all about. Besides i am not too much into speeding up my boot process, as with sysvinit and parallel boot it is already relatively fast. But the change comes at a good time, taking into account the changes linux has gone through in the last years/months.

For the interested:

http://upstart.ubuntu.com/doc/getting-started.html
http://www.netsplit.com/blog/articles/category/upstart

I am currently into checking upstart and will report back soon.

[matrixhax0r]

Well, seeing that Edgy is going to release in under an month and they are still planning on using it, I'd say Upstart is going to get a lot of development in the future. Looks like a winner.

EDIT:
Sorry if I offended anyone. That's not something I meant at all. Opensource is all about choice and I respect that.
What I meant was, sysvinit has been around for a long time and that it appears that upstart is going to be a viable project for getting many distros to change their init process.

[mdeininger]

Well, seeing that Edgy is going to release in under an month and they are still planning on using it, I'd say Upstart is going to get a lot of development in the future. Looks like a winner.

well, not sure if "winner" is the right word, at least with einit the focus is different from that of upstart...

[smlgbl]

Hey, mdeininger, don't get discouraged. Please keep up the good work! Honestly i haven't gotten einit to work for me so far, but i'll keep an eye on it.
But to be honest, i guess there are more developers behind upstart right now, so maybe it'll get there faster? What do you think?
Maybe you could also explain what the main focus of einit is, maybe especially compared to upstart and how you will implement the event handling.

Thank you very much.

[mdeininger]

Hey, mdeininger, don't get discouraged. Please keep up the good work! Honestly i haven't gotten einit to work for me so far, but i'll keep an eye on it.
But to be honest, i guess there are more developers behind upstart right now, so maybe it'll get there faster? What do you think?
Maybe you could also explain what the main focus of einit is, maybe especially compared to upstart and how you will implement the event handling.

Thank you very much.

Hey sure thing I wasn't so much discouraged as a bit annyed. Not at this discussion, really, I was just cleaning up einit and writing more documentation just now to make it easier to try it out Next release (first proper beta) is scheduled for the end of this week/start of next week, so i'm hurrying a bit

The main focus of einit is to be fast and to keep a small memory footprint. Also, it's intended for embedded devices, cluster nodes and servers and it'll contain a "just-shuddup-and-get-it-somewhat-running"-mode once it's done. (this should come in handy on headless boxes... i hate it when i boot my server after an update, just to find out it doesn't work, and i have to attach a monitor and keyboard just to find out something wants me to press "y".)

Now, we all know that the ubuntu guys are good at hacking software that "just works", but if we look at the latest ubuntu releases, i'm sure we all agree that they're not exactly being "mild" on hardware resources, am i right?
Also, the ubuntu guys said that one of the primary goals of upstart would be to be compatible with the current sysv init: this is all nice and good for administrators and others that are used to this system, but it produces useless overhead on embedded devices, for example. Now, this could easily be implemented using einit's current module system (v0.01 actually had a module for that), but it's definitely not a high priority on my list.

To sum it up: i want this thing to be fast and small but still "intelligent" ( that's why, for example, things that are to be enabled are specified using abstract "service names", like "xdm": xdm could be anything from gdm over kdm to entrance; whatever works first wins and stays enabled. )

as for those events: einit currently has an internal event systems, but it's only used for user-feedback (and some other internal things): modules emit a "feedback"-event, and feedback modules display appropriate data (or vocalise it... yes, i already added a tts module that uses festival, it's rather handy too).
so, the next step will be to enhance the event-system with more events that can be emitted, so that modules can scan the hardware and network status and they can then emit appropriate events (link change, new hosts available, power failure). then i'll add an interface for this in the configuration file... in a way, that'll be able to replace (x)inetd among other services, just like upstart plans to.

ah, one more thing i should point out: while einit is intended to replace /sbin/init, it is not limited to that function alone. you could run einit like any other ordinary daemon. for example, if i got the xinetd-workalike module done, you could run einit as a standalone xinetd replacement while, for example, the computer is using upstart as its init system...

[smlgbl]

Has anybody actually gotten the recent versions of upstart to work under gentoo? I tried it, but so far without success. Somehow it doesn't adapt nicely to "our" dependeny driven system. Complains about / not writeable. I can put "mount -o remount rw /" into the first run rc-script, but then it reboots, because it thinks the filesystem is dirty...aso.
I thought i could just adapt the runlevel setup within /etc/event.d/ to behave like the normal gentoo init, but somehow i don't know how to get sysinit and boot to get running right. At least this is what seems to be the problem to me.
Any help is appreciated, as i am in the process of trying a couple of new init-systems...

[dhave]

I'm another one who'd like to see upstart adapted for Gentoo. I've played around with initng and have seen some of the benefits, but upstart looks to be more promising.