GNU HURD: Altered visions and lost promise (The H) [LWN.net]

The H has a lengthy historical look at the GNU HURD. "The dinner was prepared, but the meal has yet to appear. By 2006, there was on ongoing project to port the HURD from L4 to Coyotos, which was considered more secure, and more recently [~~Neil~~ Neal] Walfield has worked on moving the HURD to the viengoos microkernel. The HURD has never had the community and resources that amassed around Linux, and looking in from the outside, sometimes looks more like a focus for research into OS design than it does a working kernel."

to post comments

In Theory, Microkernels Are Good

Posted Jul 1, 2010 5:34 UTC (Thu) by ldo (guest, #40946) [Link] (15 responses)

Its only in practice that they fall short.

If Mach was the failure point, why?

Posted Jul 1, 2010 13:44 UTC (Thu) by markhb (guest, #1003) [Link] (9 responses)

One thing the article hints at but doesn't really describe: the failure point (if we can call it that... although I notice that this site isn't called HWN.Net) of the project was supposedly in the decision to base it around Mach. Is that due to particular failings of Mach relative to other microkernels, or are they just saying that the microkernel-based architecture in itself is a bad idea?

If Mach was the failure point, why?

Posted Jul 1, 2010 14:31 UTC (Thu) by neal (subscriber, #7439) [Link] (8 responses)

If you are really interested in understanding the Hurd and some of its shortcomings, I recommend you read a "A Critique of the GNU Hurd Multi-server Operating System," which you can find here:

http://walfield.org/papers/200707-walfield-critique-of-th...

A shorter answer is that there are a number of technical shortcomings with the design of the Hurd. Many of these can be categorized as either resource management issues or security & protection issues.

Regarding resource management, a problem which becomes particularly acute in highly-decomposed systems (such as multi-server systems), is the difficulty in coordinating resource use. The underlying issue is that coordination must be formalized and agents may be suspicious of one another.

To understand the issue, considering how memory is managed on Linux. When there is pressure, Linux can reach into the various subsystems and ask them to free memory. For instance, the file system code might shrink the inode or dentry cache. That's cheap. If you start moving such components out of the kernel (as the Hurd does), you also need a mechanism to recover this ability, for instance, an upcall asking applications to free memory. But this is hard: the kernel cannot trust applications to behave correctly. This is the thrust of the work on Viengoos.

If Mach was the failure point, why?

Posted Jul 1, 2010 16:03 UTC (Thu) by patrick_g (subscriber, #44470) [Link] (7 responses)

>>>Regarding resource management, a problem which becomes particularly acute in highly-decomposed systems (such as multi-server systems), is the difficulty in coordinating resource use.

Funny. It was exactly the point raised by Linus in this mail.

Extract :

Now, the real problem with split access spaces is not the performance issue (which does exist), but the much higher complexity issue. It's ludicrous how micro- kernel proponents claim that their system is "simpler" than a traditional kernel. It's not. It's much much more complicated, exactly because of the barriers that it has raised between data structures.

If Mach was the failure point, why?

Posted Jul 1, 2010 17:20 UTC (Thu) by pboddie (guest, #50784) [Link]

Here's an interesting attempt to overcome such problems:

Moreover, Nemesis has been designed such that these Quality of Service guarantees are meaningful: In a microkernel environment, an application is typically implemented by a number of processes, most of which are servers performing work on behalf of more than one client. This leads to enormous difficulty in accounting for resource usage. In a kernel-based system, multimedia applications spend most of their time in the kernel, leading to similar problems.

(From the Nemesis documentation.) That was ten years ago now, however, and I guess nobody picked it up when the funding ran out.

If Mach was the failure point, why?

Posted Jul 2, 2010 8:17 UTC (Fri) by mjthayer (guest, #39183) [Link] (5 responses)

> Now, the real problem with split access spaces is not the performance issue (which does exist), but the much higher complexity issue.

Compare, for interest, with gist of The Art of Unix Programming (http://www.faqs.org/docs/artu/).

It is also interesting that many parts of a Linux system which perform a kernel-like function (X11, PulseAudio) are in user space servers. Although if that proves anything it is that there is no one-size-fits-all here.

If Mach was the failure point, why?

Posted Jul 2, 2010 12:08 UTC (Fri) by nix (subscriber, #2304) [Link] (4 responses)

Yeah, but quite a lot of X11's kernel-like function has finally moved into the kernel where it belongs :) I'm not sure the protocol handling really belongs in kernel space, but the hardware access surely always did. And now it's there.

If Mach was the failure point, why?

Posted Jul 2, 2010 18:26 UTC (Fri) by mjthayer (guest, #39183) [Link] (3 responses)

> Yeah, but quite a lot of X11's kernel-like function has finally moved into the kernel where it belongs :) I'm not sure the protocol handling really belongs in kernel space, but the hardware access surely always did. And now it's there.

I must admit that I've always rather liked the micro-kernel idea, but yes, having minimal hardware drivers in the kernel, with enough extra logic for similar devices to present similar interfaces to user space (although I'm not sure I would consider e.g. a five-to-ten year old network card similar to a modern one in that respect) also always seemed like a clean separation. I do wonder how it would be if things like the TCP/IP stack, or at least the upper layers, were in user space. Not sure if I could get many kernel people to wonder about that sort of thing though...

If Mach was the failure point, why?

Posted Jul 4, 2010 19:18 UTC (Sun) by nix (subscriber, #2304) [Link] (2 responses)

Wasn't pushing the TCP/IP layer to userspace the point of the 'channels' stuff?

(not sure how you'd do advanced routing or firewalling that way, though.)

If Mach was the failure point, why?

Posted Jul 5, 2010 21:30 UTC (Mon) by mjthayer (guest, #39183) [Link] (1 responses)

>Wasn't pushing the TCP/IP layer to userspace the point of the 'channels' stuff?
>(not sure how you'd do advanced routing or firewalling that way, though.)

Your mentioning that gave me an impulse to read over the channels stuff again, and it is pretty neat stuff, even if it seems to have hit a few snags (any idea what happened to it? I presume that at least some of the ideas got taken up). I must admit that what I had in mind was just the idle thoughts of someone not particularly knowledgeable about networking, and I certainly hadn't given thought to the details of CPU core cache and things.

Just for the fun of it I will try to give those thoughts a shape though.

* an in-kernel network device driver (probably with a /dev entry) which can accept data to transmit and return data, as naked ethernet frames, or raw TCP data, or whatever the card can do in hardware.
* a userspace server to multiplex the driver which can accept socket connections or whatever from clients and can accept and return data as ethernet frames, ip packages or whatever, but can tell the clients what it can handle most efficiently. It will do whatever checks are needed on outbound data to be sure that the client is allowed to send what it is sending, and will do whatever checks are needed on the inbound data to know where to send it. That is likely to put an end to keeping things on a single core unless the checks can be offloaded to the hardware.
* hopefully some optimisation of the kernels socket primitives so that as much as possible of the data copying accross sockets can be collapsed into single operations, preferable DMA between card and the final client. Should be doable for e.g. outbound TCP data with clever cards, perhaps even for inbound data if the card can buffer it for a short time and give the server just enough information to know where to forward it to.

I'm sure that there are millions of holes in that logic, just for a start what happens when in-kernel clients need to send network data (if someone pushing a network stack into userspace really cares about in-kernel clients). Feel free to poke a few more if you like...

If Mach was the failure point, why?

Posted Jul 7, 2010 11:36 UTC (Wed) by nix (subscriber, #2304) [Link]

a userspace server to multiplex the driver which can accept socket connections or whatever from clients and can accept and return data as ethernet frames, ip packages or whatever

I thought the whole point of VJ tunnels was to avoid copies? Possibly we want a scheme where the kernel hands you raw IP packets which may be zero-copied. If you have NAT or some kinds of firewalling in the way obviously they would not be, but sometimes they would, and when they were you could gain the speed benefits. This doesn't stop the kernel inspecting the raw IP packets and sending them to different clients or anything like that, so I don't think you need a userspace multiplexor at all.

In Theory, Microkernels Are Good

Posted Jul 1, 2010 17:18 UTC (Thu) by HelloWorld (guest, #56129) [Link] (1 responses)

Yeah right, except that microkernel systems (Green Hills Integrity, QNX) have been in wide use for ages where reliability actually matters.

In Theory, Microkernels Are Good

Posted Jul 1, 2010 19:22 UTC (Thu) by emk (subscriber, #1128) [Link]

It should be noted, however, that the typical QNX system is (IIRC) a specialized, embedded system running known software. Now, that involves solving some very hard problems, but they're not necessarily the same problems encountered in a general-purpose OS or even a smart phone.

In Theory, Microkernels Are Good

Posted Jul 2, 2010 4:40 UTC (Fri) by rsidd (subscriber, #2582) [Link] (2 responses)

Apple did manage to make Mach work for them (even if it's a hybrid with BSD). So in practice, millions of users are happy with a reliable desktop OS, even though they have no idea what's inside.

In Theory, Microkernels Are Good

Posted Jul 2, 2010 16:31 UTC (Fri) by coriordan (guest, #7544) [Link] (1 responses)

Didn't they use a single-server architecture though?

Hurd's speciality is its multi-server architecture, so the Mac OSX example doesn't prove much. (If my first sentence is correct.)

In Theory, Microkernels Are Good

Posted Jul 2, 2010 17:22 UTC (Fri) by gnb (subscriber, #5132) [Link]

>Didn't they use a single-server architecture though?
I think so. And I'm pretty sure OSF/1, err... Digital Unix... Tru64... whatever did too. Basically using MACH to provide the low-level primitives in order to make getting a Unix kernel running, and subsequently porting it, easier.

GNU HURD: Altered visions and lost promise (The H)

Posted Jul 1, 2010 10:53 UTC (Thu) by mbanck (subscriber, #9035) [Link] (1 responses)

It's Neal Walfield, not Neil, if you have to edit in his first name.

Also, it's GNU Hurd, but that's the H's fault I guess.

GNU HURD: Altered visions and lost promise (The H)

Posted Jul 1, 2010 12:16 UTC (Thu) by jake (editor, #205) [Link]

> It's Neal Walfield, not Neil

Indeed, fixed now, thanks!

jake

Concerning HURD

Posted Jul 1, 2010 15:35 UTC (Thu) by gilboa (guest, #23856) [Link] (2 responses)

If you read the story, you'll come across this link: "LINUX is obsolete" [1] from Jan 29, 1992.

"While I could go into a long story here about the relative merits of the
two designs, suffice it to say that among the people who actually design
operating systems, the debate is essentially over. Microkernels have won.
...
This is a giant step back into the 1970s.
That is like taking an existing, working C program and rewriting it in
BASIC. To me, writing a monolithic system in 1991 is a truly poor idea."

It should be amusing to ask the OP what he thinks about his post, 19 years later. (Especially given the fact that all the major OS' are either monolithic [Linux, FBSD] or hybrid [NT, OSX]...)

- Gilboa

Concerning HURD

Posted Jul 1, 2010 17:28 UTC (Thu) by gilboa (guest, #23856) [Link]

Link: http://groups.google.com/group/comp.os.minix/browse_threa...

Concerning HURD

Posted Jul 1, 2010 21:20 UTC (Thu) by sthibaul (✭ supporter ✭, #54477) [Link]

Well, AST just hired new engineers to work on minix3...

GNU HURD: Altered visions and lost promise (The H)

Posted Jul 23, 2010 19:28 UTC (Fri) by phallus99 (guest, #69220) [Link]

Here is my opinion on HURD: it is entirely necessary for the propagation of the GNU foundation and the free software movement for HURD to be complete. For people to continue to call the GNU/Linux OS just "linux" erroneously, I doubt this will continue with a kernel called "HURD".