Bedrock Linux

By Jonathan Corbet
September 12, 2012

The Linux ecosystem does not lack for distributions; there are hundreds of them catering to almost any need that one might think of. But where does that leave the poor user who is unable to decide between the virtues of two or more different distributions? Distribution choice tends to be all-or-nothing: there is no easy way to obtain the benefits from multiple distributions at the same time on a single machine. That is a situation that the folks at Bedrock Linux are trying to change — but the result is unlikely to be suitable for everybody.

The design goal of Bedrock Linux is to allow users to mix and match pieces from multiple distributions at will. If one wants something that looks mostly like openSUSE, but with the ability to source-install a bleeding-edge LibreOffice from Gentoo next to tried-and-stable PostgreSQL server from CentOS, Bedrock Linux will make that possible. If one wants to fall back to Emacs from Debian stable because the latest Rawhide update broke it again, one can. As the project web site says: "Packages feel disposable, like toothpicks. No need to fret over one breaking; just use another." The end result should be a world where users never have to commit to a specific distribution because all distributions of interest to them will be available simultaneously.

The Bedrock Linux core exists mostly to boot the system and facilitate the running of applications from multiple "client" distributions. As such, it is a bit of a strange and minimal beast. The syslinux bootloader is used to keep things as simple as possible. The kernel is Linux, of course; the Bedrock user space is based on Busybox, so there is not a lot to it. It is all intended to be robust and to function even if an important client distribution breaks; to that end, the core Bedrock binaries are all statically linked. Static linking is also needed to ensure that those utilities will work regardless of the client distribution context they might be run in.

The client distributions are installed under /var/chroot, so /var/chroot/fedora would host a Fedora installation. Users can run a command out of any distribution, but the work that must be done under the hood to make that happen is significant. As might be guessed from the directory names, Bedrock Linux uses the chroot() system call to run programs in the distribution context they expect. The brc command makes all this happen; typing:

    brc fedora firefox

would use chroot() to restrict the process's view to /var/chroot/fedora, then run the firefox binary from there.

The idea is conceptually simple, but it quickly runs into complications. What if that firefox binary wants to run a helper program from a different distribution? That requires the ability to break out of the restricted root, somehow. Bedrock makes this happen with a combination of setuid binaries and bind mounts. From the outside, it looks like it might have been better to make use of namespaces to reshape each process's view of the filesystem, but that is not the direction Bedrock took.

The result is that everything that must be visible to any distribution-specific program must be bind-mounted into the restricted root. That is, to begin with, how commands get access to the user's home directory. Regardless of which distribution context a process is running in, /home will be properly mounted to be visible there. In the current alpha release (1.0alpha2, known as "Momo"), every process has all of the directories for all distributions bind-mounted into it. This mounting is done on the fly as commands are run, leading to some complex topologies.

That complexity turns out to be a bit of a problem; as the Bedrock developers say in the notes for the upcoming 1.0alpha3 release:

Real-world use of Momo can easily result in hundreds of thousands of mount points. Since it is not possible to predict how deep the tree will go, Momo cannot set up all of these mounts at boot. Rather, it attempts to create these mounts just before they are necessary on-the-fly. This has a number of down sides.

Performance is at the top of the list of "down sides," but it's clear that a system configured like this can be unwieldy in a number of ways. So it will be replaced in 1.0alpha3 with a new version of the brc command that, when a distribution boundary is to be crossed, will simply break out of the restricted root and enter a new one for the new distribution. That eliminates the need to set up lots of new bind mounts at every command invocation, at the cost of needing to run a setuid breakout program instead.

Given that initialization systems have been a hot topic recently, one might well wonder what Bedrock does in this area. Can one easily switch from, say, OpenRC to systemd? The answer is that Bedrock literally does nothing:

The Bedrock Linux developer has been unable to think of any sane way of determining which init script to run when the clients conflict (which CUPS daemon should run, if multiple are available?). Additionally, an automated way to determine the launch order from all of the possible systems it will run into seems far too challenging of a project. Thus, Bedrock Linux requires manually setting which programs from which client's init is launched when.

Setting up Bedrock, thus, is not a task for users wanting a one-click installation experience. Especially since Bedrock does not really have an installer of its own at all. The FAQ entry on why there is no installer is amusing reading, to say the least. Suffice to say that Bedrock, for now, is for users who want to do a lot of their own setup and tweaking. It may be especially well suited for those users who have gotten frustrated with the way distributions like Gentoo do everything for them.

It is also not for users who want something in a hurry; 1.0alpha3 is planned for release by "end-of-summer 2013." And that will still be an alpha release.

Whether Bedrock Linux will ever get past the alpha stage is anybody's guess. It has the look of an ambitious project with a relatively small amount of developer time available to it. It also looks a little bit insane. But, then, what is free software for if somebody can't take an insane idea and run with it? Some interesting ideas may well come out of the Bedrock Linux project; it will be fun to watch.

Bedrock Linux

Posted Sep 14, 2012 22:40 UTC (Fri) by paradigm (guest, #86730) [Link] (5 responses)

Hello, I am the founder/lead dev of Bedrock Linux. I was initially extremely anxious when I learned that LWN had an article on my project, as I have a tremendous amount of respect for wonderful folks at LWN, and couldn't bear to hear a negative response from them. Suffice it to say my worry was for naught. The article is, like usual, spot on.

Well, baring one extremely minor issue with the article: I am utilizing Linux capabilities in place of full setuid binaries for the one Bedrock-specific area it is needed. I typically see Linux capabilities contrasted against setuid, and thus unless I am mistaken it would be technically incorrect to describe a Linux capability'd executable as setuid. For what it is worth, the only capability needed (both in the current and upcoming release) is CAP_SYS_CHROOT.

I experimented with utilizing namespaces to handle the unusual filesystem layout requirements but ran into some difficulties. I might revisit the possibility in the future, if not for filesystem then for process isolation, such that I could have multiple init programs each think they are PID 1. This could be a reasonable start for automating boot items, and as the article pointed out automating boot scripts from clients is one area that needs a lot of work.

I'm not sure I follow why the discussed FAQ entry is an amusing read. If I've done something silly, I'm read to admit fault and try to remedy it, if it could be stated more directly. The main issue is limited developer time - I haven't yet had time to write an installer, and I am not confident I will be able to stay on top of pushing out updates for upstream security issues (most notably the kernel). I'm hoping to eventually find a way to use a client's kernel for Bedrock, and just let the client update the kernel as it normally would.

I'd also like to note that, despite being in alpha, Bedrock Linux is quite functional. I have been using it as my primary system on all of my machines (baring the VPS that hosts the website) for quite a while before it ever went public. However, it is certainly very do-it-yourself right now, subject to big changes at this point, and has a few more known issues than I feel comfortable with in a stable release.

Thank you Mr. Corbert for the well written article.

Bedrock Linux

Posted Sep 15, 2012 1:22 UTC (Sat) by paradigm (guest, #86730) [Link]

s/Corbert/Corbet

I've read the name a few times a week for years; I should get it right >.<

My apologies.

Bedrock Linux

Posted Sep 15, 2012 20:41 UTC (Sat) by cmccabe (guest, #60281) [Link] (3 responses)

> I'm not sure I follow why the discussed FAQ entry is an amusing
> read. If I've done something silly, I'm read to admit fault and
> try to remedy it, if it could be stated more directly.

Doesn't it seem silly not to have an installer? Might limit the number of, um, installs.

Sounds like a fun project, though. I bet you'll learn a lot at the very least.

Bedrock Linux

Posted Sep 15, 2012 22:14 UTC (Sat) by paradigm (guest, #86730) [Link] (2 responses)

> Doesn't it seem silly not to have an installer? Might limit the number of, um, installs.

Well yes, of course. I most definitely want one and expect Bedrock Linux will have one eventually - almost certainly before the first stable release. However, I remain concerned about the items listed in the FAQ entry. The items listed are non-issues for most other distros as they either (1) have a large team including dedicated security update people, or (2) are based on and heavily pull from an existing distribution. Bedrock Linux isn't really in that position. From what I can tell, other small-team non-derived distros in Bedrock's position just don't worry about the things I am. Perhaps I am setting my standards to high.

> Sounds like a fun project, though. I bet you'll learn a lot at the very least.

Oh, it most certainly is, and I can happily say I've already learned quite a bit.

Bedrock Linux

Posted Sep 15, 2012 23:19 UTC (Sat) by rahulsundaram (subscriber, #21946) [Link] (1 responses)

Just curious. Is there a reason you are not using namespaces?

Bedrock Linux

Posted Sep 16, 2012 0:05 UTC (Sun) by paradigm (guest, #86730) [Link]

A combination of the following three things:

1. I had difficulties with early experimentation with namespaces, mostly due to gaps in my knowledge about them.

2. I wanted to get something functional done by certain self-imposed deadlines, even if it is built on work which would later be scrapped. I simply reached the point where I could do what is being done in the current release of Bedrock Linux before I learned enough to do something comparable with namespaces. Bedrock Linux is the OS I've wanted for years. Despite the issues pointed out in the article above (all of which I agree with), its very nice to finally have and use it.

3. Insufficient knowledge in how namespaces would be preferable to what I already know how to do for the upcoming release (namely, going into and breaking out of chroots with a capabilities/setuid'd executable). While there are issues with the current system, the new system should fix most if not all of them fairly cleanly, and will likely be faster for me to implement than anything with namespaces.

If there are good reasons for using namespaces over the plans for the next release, I'd be happy to switch tracks, or stay on track for the upcoming release and plan to move to namespaces for the next one. The issue could simply be a failure on my part to learn enough about namespaces - perhaps my google-fu is simply to weak. I would be greatly appreciative if someone could point me in the right direction to learn more.