Redox: a Rust-based microkernel
Creating a new operating system from scratch is a daunting task—witness the slow progress of GNU Hurd, for example. But it would be hard to argue that the existing systems are the be-all and end-all of how we interact with computer hardware. At least some of the deficiencies in today's offerings can arguably be traced to the language used to implement them; C, for all its power, has some fairly serious flaws that lead to bugs and security holes of various sorts. So it is interesting to see a new entrant that uses the Rust language, which is focused on memory safety. The result is Redox, which is still far from ready for everyday use, but may provide an interesting counterpoint to Linux, the BSDs, OS X, Windows, and others.
As laid out in the Redox book (which is also a work in progress), Rust was chosen for its memory and type safety. Some of those features were described in an LWN article back in 2013. The Redox developers see these safety features as a critical part of a modern operating system, even one that is based on the much-older Unix model:
The design of Linux and BSD is secure. The implementation is not.
So the goal is to eliminate whole classes of problems that plague C programs, but the focus on security doesn't stop there. Unlike Linux, Redox has a microkernel design, where much of the code that resides in a monolithic kernel like Linux is moved to user space. A microkernel simply provides resource management, interprocess communication (IPC), process/thread management and scheduling, system calls, and little else. The result is a much smaller kernel (16,000 lines of code for Redox and plans to reduce that further) with a concomitant reduction in the attack surface, as well as a much smaller amount of critical kernel code to audit.
In Redox, as in MINIX that it is partly patterned after, drivers live in user space so they can do far less damage when they fail. In monolithic kernels, a bad driver can crash the whole system because it runs in the kernel's address space and with its privilege level. So, for Redox, filesystems, device drivers, networking, and so on are all implemented in user space.
Beyond that, Redox has taken the Unix "everything is a file" philosophy (which was further expanded by Plan 9) and taken it a step further. In Redox, everything is a URL (or, really, a URI), which means that the API is based on using URLs to access local and remote resources: files, networking protocols and sites, devices, and so forth.
The "scheme" portion of a URI (e.g. the "http:" in a web URL) is used to
identify the type of the resource being accessed. These are implemented as
Redox schemes,
which can be implemented in user space or the kernel, though user space is
preferred. Schemes are effectively virtual filesystems with a set of
operations made available that are appropriate to the type. As the book
notes: "'Everything is a scheme, identified by an URL' is more
accurate, but not very catchy.
"
The Redox wiki goes into a bit more detail of how URLs and schemes are used. For example, tcp://10.11.12.13:80 would be used for a remote TCP connection, while file:/apps/terminal/example.txt refers to a local file. In the networking wiki page, there is mention of using a URL like tcp:///80 for a web server or ethernet://12:34:56:78:9A:BC/800 to receive or send IPv4 (type 800) packets for the specified MAC address.
Underneath the schemes are resources, which come in two flavors: file-like (i.e. blocking and buffered) and socket-like (i.e. non-blocking and stream-like). Resources are closely related to schemes, as the book notes:
Resources are closely connected to schemes, and are sometimes intertwined with schemes. The difference between them is subtle, yet crucial.
While the kernel is the main focus of the project currently, there are some side projects of interest as well. To start with, ZFS is planned as the filesystem of choice for Redox, which means writing a compatible implementation in Rust or adapting the existing open-source version (OpenZFS). There is a minimal graphics server, orbital, that is based on the Wayland protocol. There is also a vi-like editor (Sodium), games, and more.
The code is available on GitHub and has been released under the MIT license. The explanation for choosing the MIT license has been somewhat controversial among some GPL advocates, but the developers clearly want Redox to spread far and wide—proprietary forks and extensions are explicitly condoned. ISO images for various targets are available, though they are yet another work in progress. The book also has some instructions (starting here) for those who want to build Redox for themselves.
A Discourse forum is provided to discuss Redox and its development. There are contribution guidelines and ideas, a best practices page, and the inevitable issues list as well. Those interested can keep up with the project by checking out the "This Week in Redox" newsletters, though they are rather more sporadic than the name would imply. The Code of Conduct is fairly extensive and clearly shows an intent to foster an inclusive community. In short, there is quite a bit of information available about Redox, though it is somewhat scattered.
One question that many will ask is "why a new operating system?" The Redox
developers have an answer
that boils down to dissatisfaction with Linux, the BSDs, and, even, MINIX
(which is "the most in line with Redox's philosophy
"). There
is an admitted "Not Invented Here" sentiment within the project. But one
thing that is explicitly not a goal for the project is replacing Linux. A
one-word
answer ("No
") to the "will Redox replace
Linux?
" question is contained in the book.
That's probably a good thing for a number of reasons. For one thing, hardware support is a huge task that is unlikely to be surmounted without an enormous effort from lots of companies—something that the MIT license might make difficult. Like it or not, the reciprocal nature of the GPL certainly seems to be a driving force for the corporate involvement in Linux. It is also an area where more permissively licensed alternatives (the BSDs and MINIX) have run aground to some extent.
But there are other hurdles as well. Redox has deliberately chosen to implement a minimal subset of the Linux (and POSIX) system calls. That will make it more difficult to port existing applications to Redox. That is partly by design, however, as a Rust-based user-space ecosystem is envisioned, but the huge number of existing freely licensed applications, which may be difficult to port, could be a big impediment to Redox adoption. Longstanding issues with the performance of microkernels (which require many more context switches than monolithic kernels do) may also make Redox a hard sell for the more lucrative server segment—though performance for desktops may turn out to be reasonable, especially if the security gains are substantial.
It is clearly an ambitious project—perhaps even overly ambitious; only time will tell. It does bear watching, though, as an experiment in several dimensions. The microkernel design and Rust-based implementation will provide answers (or at least evidence in one direction or another) to some questions that have been floating around for some time. We clearly don't have all the operating system answers for the future, so more data is called for. Redox might just provide some of that data.
Posted Apr 8, 2016 1:11 UTC (Fri)
by smoogen (subscriber, #97)
[Link]
To start with, ZFS is planned as the filesystem of choice for Redox, which means writing a compatible implementation in Rust or adapting the existing open-source version (OpenZFS)
Is that a real deep plan or someone just posted it to the wiki that anyone can write to? I say this because this is a HUGE undertaking and seems to be like a leftturn to try and implement a filesystem in a different language which requires experts to keep working in C.
It is also these sorts of big things without resources which took HURD from being something that might have been possible in 1994 to something that is sort of usable in limited settings in 2016.
Posted Apr 11, 2016 3:12 UTC (Mon)
by ksandstr (guest, #60862)
[Link] (1 responses)
"Limited attack surface", sure, that's noble in the abstract -- however, an architectural sleight-of-hand that moves responsibility out of the theoretically powerful ring 0 matters for zilch when privileged processes are just as subject to compromise outside the kernel proper as they are within it. This kind of design justification makes redundant the choice of language (or vice versa): Rust, a tool that promises absolute memory safety by straightjacketing the programmer, and which should therefore eliminate the desirability of moving stuff out of ring 0 for security reasons in the first place. So either Rust doesn't do what it said on the tin[0], or these people are fawning over microkernel designs for reasons[1] that've gone without proper analysis.
And look, it parses URIs and everything. Certainly there's never been security compromise due to arbitrary-length string processing, right?
It seems that the Redox team is chasing after 1st-generation microkernel goals (viz. increased security and reliability by applying the MMU to a subset of kernel functions) with 1st-generation methods (writing things into ring 0, then moving them out using an ad-hoc interface) and recent tools designed to solve security and reliability at the level of implementation language. Not unlike older experiments conducted in Java, and later C#[2]. It's like deciding that the baseball bat isn't doing enough by itself, that it needs a turn or twenty of rusted razorwire on it, and might as well substitute something with a six-digit Scoville rating for lubrication while they're at it. And you have to wear a dunce cap upside down, too.
All this being said, it's good that tools from the pathological corner are used to explore silly designs. This way they'll knock two things off the chart of things that the next generation of misguided-but-well-informed youths might otherwise be tempted to try again. Who knows, maybe there's even an unrelated but interesting advantage one day; holding one's breath doesn't kill, so why not do that while spinning around?
[0] it does, in programs where there's no unsafe sections, and the stdlib has those out of necessity. The road to NIH is paved with concerns for purity.
Posted Apr 11, 2016 4:59 UTC (Mon)
by eternaleye (guest, #67051)
[Link]
> [0] it does, in programs where there's no unsafe sections, and the stdlib has those out of necessity. The road to NIH is paved with concerns for purity.
The thing "total absence of unsafe" thing is from a proof about the transitively safe form of Rust, which proves that a total lack of unsafe is *sufficent*, not that it is *necessary*. There is active, ongoing work to improve on this result: http://plv.mpi-sws.org/rustbelt/
In addition, even with the use of unsafe, it still constrains the unsafe behavior significantly - though this is only valid in combination with privacy, the stdlib uses them in concert in a responsible manner. This enables the language to have a considerably smaller TCB that required auditing. See also, http://huonw.github.io/blog/2014/07/what-does-rusts-unsaf...
> [1] such as hype
Here I agree with you - in particular, that the hypothetical security advantages of microkernels are moot without a security _model_ in which to evaluate them. For that reason, I am considerably more excited by seL4 (and by extension, Robigalia) - seL4 has a strong security model, a formal proof its spec satisfies that, and a formal proof (on ARMv6 so far, but more in progress) that the implementation - all the way down to the machine code - implements that faithfully. Pairing that with a userspace written in Rust, as Robigalia[3] seeks to, further aids the goal of building reliable systems. In addition, seL4 is a modern microkernel, with performant IPC at its core and far more minimal design than the (IMO rather Mach-like, or more maximal yet) Redox design.
> [2] which, as I recall (for the Java incarnation anyway, the one Sun was going to try on thin clients), while they did do away with invalid pointer accesses, remained vulnerable to all other categories of exploit; so below the line it was uncomfortable and didn't really help.
I will note that Rust's guarantees are considerably stronger: In particular, while Java does away with actual null pointer accesses, it retains null objects with much the same semantics (which Rust avoids via strict reference lifetimes and the Option<T> type - essentially the Maybe monad), data races (which Rust avoids via strict ownership in its type system), and - if you're referring to JavaOS - it predated Java 1.3, much less 1.5 which introduced generics, and thus would have had to be downcasting from Object and so forth, introducing runtime type errors.
Beyond that, I do broadly agree with you that the state of microkernels has advanced a long way since the first generation, and repeating the same mistakes doesn't advance the art so much.
But just as history has shown that the first generation of microkernels could not live up to their promises, it's also shown that humans are still fundamentally fallible, and that tools which help reduce error have measurable, and often significant, effects on real-world reliability. I certainly don't think either microkernels or Rust are a write-off as they are today (the whole L4 family is succeeding in the world as a foundation for real-time systems, and seL4 has provable worst-case execution time guarantees, along with confinement and resource isolation.)
[0]: NMF
Posted Apr 11, 2016 8:22 UTC (Mon)
by robbe (guest, #16131)
[Link]
Am I the only one that thinks about PHP’s allow_url_fopen when they hear „everything is an URL“?
Posted Apr 12, 2016 0:04 UTC (Tue)
by nix (subscriber, #2304)
[Link] (1 responses)
But, hey, it's all people know now :(
Posted Apr 14, 2016 9:16 UTC (Thu)
by farnz (subscriber, #17727)
[Link]
The other problem with URIs is that they are a universal (and hence unambiguous) identifier that needs no additional context to interpret correctly. Great for computers, but horrible for humans; we seem to universally prefer shorter context-dependent identifiers[0] - so, for example, "Mark" is a perfectly clear identifier for a single unique individual in context, and yet, depending on context, you could be referring to Mark Shuttleworth (plausible in an Ubuntu setting) through Mark Twain (plausible in a literary setting) through Mark the Evangelist (Christian church). A URI can't deal with that - the context has to form part of the URI so that it's a universal identifier.
In that respect, filesystem paths are better - they're not universal, and thus it is reasonable that ls can refer to (depending on context) ./ls, or /bin/ls, or ~/bin/admin-help/ls, or even /usr/bin/ls on your machine instead of mine. Further, we have tools (like search paths, current working directories, aliases, chroots, bind mounts, symbolic links etc) that allow you to define your own context for a given filesystem path.
[0] Think pronouns and other pro-forms in natural language - I can't find evidence of a natural language that doesn't have any pro-forms at all, nor can I find one where it's completely unacceptable to use an ambiguous identifier and rely on context to resolve it.
Redox: a Rust-based microkernel
Oh boy, it's this season again
[1] such as hype
[2] which, as I recall (for the Java incarnation anyway, the one Sun was going to try on thin clients), while they did do away with invalid pointer accesses, remained vulnerable to all other categories of exploit; so below the line it was uncomfortable and didn't really help.
Oh boy, it's this season again
[1]: NMF
[2]: NMF
[3]: https://robigalia.org
Redox: a Rust-based microkernel
Nice understatement, there. „boil the ocean“ seems more apropriate.
Redox: a Rust-based microkernel
Redox: a Rust-based microkernel