2006-09-09

A critical view on upstart

There has been lately much fuss about upstart which is supposed to be a single replacement for several daemons: SysV-init, cron, inetd, hotplug... This article is a commercial trying to sell upstart, but somehow it hasn't convinced me.

The first reason I'm not comfortable with the idea is that UNIX is built on the philosophy "one tool for one job". Every tool should do one job and do it well. Merging several different tasks into one program just feels "yucky". It feels "windows-way".

The second reason is security and stability. Take for example cron. Even though it has a seemingly simple task, a very popular implementation, vixie-cron, had some security bugs in the past. Now it's going to be reimplemented again. Not to mention that upstart then becomes a single point of failure. Imagine e.g. remotely induced reboot or kernel panic by triggering some bug in upstart's networking code and making it crash. (And since it's running instead of init, it'll bring the whole system down).

Rest of this post is a dissection of the article cited above.

The first part of the article is what I call "Problem setting." In trying to explain why SysV init doesn't work today, the autor says "The simple answer is that our computer has become far more flexible." and enumerates certain situations which do not really pose a problem. Most of them are related to hotplugged hardware which is already handled (I see it working nicely on RH and SuSE). He concludes with "We've been able to hack the existing system to make much of this possible, however the result is chock-full of race conditions and bugs." While I admit that there may be some problems, saying "chock-full" would be a blatant exaggeration.

Question 1: Why replace replace everything instead of sticking with the UNIX philosophy and making the current system better?

The second part is "Design". On the surface it seems sane and well-designed, but take a look at the example list of events; the most striking one for me is "the root filesystem is now writable". He doesn't say who is supposed to generate these events! This is a shift of responsibility from getting the startup script ordering right to generating the right
events
at the right time. Currently we have a small, well-controlled set of dedicated processes, and the upstart system seems to lead towards an explosion of possibilites along at least two dimensions: kinds of events and when they are generated.

Question 2: Who is generating events? Who is writing event handlers? If the event handling system is extendible, how is the system integrity guaranteed (so that the faulty handler doesn't bring the whole upstart process down)? What happens when an event isn't handled because a handler is missing? Is it an error, how is it reported and to whom, is it
simply ignored..?

The third part is "showing off" or FUD-ing. Showing existing tools in black light in order sell "upstart" better. This is the funniest part! Namely, the author doesn't seem to find good arguments against initng, a dependency-based system, so he resorts to ridiculous argumentation: "However this means that you need to have goals in mind when you boot the system, you need to have decided that you want gdm to be started in order for it, and its dependencies, to be started.", continuing with "[..initng] It can reorder a fixed set of jobs, but cannot dynamically determine the set of jobs needed for that particular boot." and finishing with "initng starts with a list of goals and works out how to get there, upstart starts with nothing and finds out where it gets to."

Question 3: How is the computer supposed to figure out, even before it is turned on, what the user has in mind and what should be the target configuration? How could it know that the user on a particular boot wants e.g. xdm to boot, without any user input (e.g. without being given a goal)?

upstart seems like a solution to an invented (or, to say the least, exaggerated) problem. I hope the author does better job of coding than argumenting its usefulness.

[From personal experience, dependency-based system is used on FreeBSD, NetBSD, and on Gentoo Linux. It's very easy to maintain, and I like it better than SysV-init style boot process.]

[Another note: One should distinguish between the init program and the SysV-init style boot scripts. It is possible to use the (SysV-)init program, with a dependency-based system. And that's exactly what Gentoo is does.]

Tags:

7 comments:

Senko Rašić said...

Personally, I think upstart is a good idea, although I'm going to stay conservative and wait till it proves (in Edgy) before trying to use it.
That said, I think you misinterpreted a few facts about it: First, upstart is not trying to be many things at once - it does just one job (implementing the event system for service startup/shutdown scripts, etc), which is exactly what sysvinit now does. So it fits nicely into the Unix paradigm.
Your second point, that this is a single point of failure, is by itself true, but so is the current sysvinit. I agree that sysvinit's daemon itself is now probably more stable that upstart's daemon, but that is a technical issue, not a design issue.
Also, you mention that everything "works nicely on RH and SuSE" (implying that sysvinit is really ok). It works, but it's not an elegant solution, it's inherently limited and unscalable (you have just a few preset runlevels, and even the script ordering is a file-naming kludge).
You recognize the paradigm shift (from ordered list to events), but fail to recognize that it is needed because the event tree model is much more expressive than a simple linear list (and in today's computer system, linear lists of tasks are inadequate).
On to answer your question: Both the event generation and handling are written in the service startup/shutdown scripts, as are done now. That part isn't changed, the package maintainers will just have to write different scripts. As for fault handlers, maybe you've missed the fact that upstart events are broadcasted. Faulty handler doesn't block other handlers which want to process the event.
The responsibility situation isn't worse than in sysvinit if a startup script reports success while failing to start the service.
To the last question, how does a computer know which set of "goals" it wants to achieve, at startup? Well, the question is the same as "how does the computer know which run level it wants to go to?"
Upstart is still to early in use to see the effects, and decide whether is better or worse in practice than, for example, initg. While I do think that we should accept (and test) it cautiosly, I think you're being to harsh to the idea.

ivoks said...

SysVinit is like buying 200HP sports car and drive it 40mph. 40mph was "oh, my god!" couple of decades ago, but today it's nothing. Instead of having tons of I/O wait and CPU idling while booting, event based booting takes advantages of your hardware.

Btw, both Novell and RedHat are interested in this concept. It's backward compatible, and offers solution sysvinit doesn't.

zvrba said...

But that's the thing - most of the boot process is spent in I/O wait. If, e.g. 3 different processes are started up in parallel (instead of 1), then all of them will compete for (e.g.) disk. The net result can hardly be much faster than sequentially assigning disk to each process..

ivoks said...

Booting speed up is just a side effect, not a mission or goal.

Real life examples of upstart:

My comp is running on battery:
- don't start mysql, apache and all other dev tools
- plug in power, without user interaction, apache, mysql, etc. are started (of course, this is configurable)

Oh my, 30 mounts, and it's fscking 200GB disk; this will take a while:
- luckily I use upstart, so I can at least log in to console and ssh to other machine or do some else non-rw jobs; like kill fsck :)

Speeding up booting (for a more than couple of seconds) can be done by using dash, instead of bash (implemented in development version of Ubuntu - this is why most of the users think that upstart brings speed-up).

Anonymous said...


The first reason I'm not comfortable with the idea is that UNIX is built on the philosophy "one tool for one job". Every tool should do one job and do it well. Merging several different tasks into one program just feels "yucky". It feels "windows-way".


The "unix philosopy" is just hand waving to cover the fact that the community failed to agree on better standards. A million poorly integrated tools. Also you seem to "feel" the windows-way is always wrong regardless any technical reasons. The headline should have been "Biased rant...", very poor.

Anonymous said...

Lol, Ubuntu newfags, thousands of them! Posting vague comments without either understanding of the subject or the blog entry in question. This is the dark side of popularity: a large pool of incompetent groupthinked opinions.
Merging inetd with init is sure a stupid thing to do. I see where this system is going >.>

Anonymous said...

Also, could you please allow anonymous comments? It is quite uncomfortable to sign up from a mobile phone just to answer a random discussion on the internets.