The mkosi OS generation tool

[Posted June 28, 2017 by ris]

Last week Lennart Poettering introduced casync, a tool for distributing system images. This week he introduces mkosi, a tool for making OS images. "mkosi is definitely a tool with a focus on developer's needs for building OS images, for testing and debugging, but also for generating production images with cryptographic protection. A typical use-case would be to add a mkosi.default file to an existing project (for example, one written in C or Python), and thus making it easy to generate an OS image for it. mkosi will put together the image with development headers and tools, compile your code in it, run your test suite, then throw away the image again, and build a new one, this time without development headers and tools, and install your build artifacts in it. This final image is then "production-ready", and only contains your built program and the minimal set of packages you configured otherwise. Such an image could then be deployed with casync (or any other tool of course) to be delivered to your set of servers, or IoT devices or whatever you are building."

(Log in to post comments)

Posted Jun 28, 2017 18:57 UTC (Wed) by jgg (guest, #55211) [Link]

To me, the idea is good, but, there is a lot of mature stuff out there that does this, and arguably does the sub-steps better (eg docker). Lots of people have per-project wrapper scripts to make this work right, eg rdma-core has a docker based tool to build the source under various containerised distributions images and produce .rpm/.deb files.

https://github.com/linux-rdma/rdma-core/blob/master/build...

Which was inspired by other tools and customised to run smoothly with the source tree it is embedded in.

mkosi is basically the same idea but for systemd. It doesn't really do anything particularly special, expect that it exports the container into a unique disk image format. Other tools like rkt can already do this, but their images are for running in kvm, not bare metal.

The last time I tried to use mkosi (232) it just failed. The main trouble seems to be that it does a lot of work in the host, so it is quiet sensitive to the host configuration, which to my mind, largely defeats the entire purpose.. It was also bonkers slow, but maybe I didn't get far enough to have the caching turn on, or that version didn't have sane caching yet.. Plus it wanted to run as root on the host and do all sorts of strange things - again not the kind of safety I want from a containerised build tool.

The mkosi OS generation tool

Posted Jun 28, 2017 19:24 UTC (Wed) by mezcalero (subscriber, #45103) [Link]

hmm, Docker cannot generate images that you can boot on bare-metal, can it?

"unique disk image format"? Do you mean the raw GPT disk images mkosi can generate with that? I don't find that particularly "unique", it's probably one of the most universally understood disk format these days, as any Windows, any ChromeOS and pretty much everything recent uses GPT.

And to clarify, after the initial packages have been extracted (i.e. right after the initial debootstrap or dnf --rootinstall), everything relevant we run runs inside an nspawn container.

mkosi is currently at version number 3, hence I don't think you can have tested mkosi 232 ;-). "232" sounds more like a systemd release version.

Lennart

The mkosi OS generation tool

Posted Jun 28, 2017 21:05 UTC (Wed) by jgg (guest, #55211) [Link]

Sure docker can, I've done it a few times. You start with the default dockerhub image for the OS and the install a few packages (eg grub, a kernel, etc) and a few other small modifications. Trivially embodied in a dockerfile and the result is something that can boot containerised systemd as pid 1 or be poured into a disk image and booted on bare metal. The same kind of basic steps you need to do anyhow if you used debbootstrap.. Docker just caches it and makes the initial image available no matter what the host OS is.

In the mainstream container world the popular way to run a container under kvm is with 9pfs (eg in rkt), this is what I mean by 'unique'. This is actually where I ended up after trying mkosi for systemd. Instead, I built systemd using a building container (with a script similar to the one I linked), then 'make installed' it into a temporary running container and directly ran kvm and 9pfs exporting the containers / as rootfs to run the build. Much faster than building an image, and I get more trivial points to customise things I need, like using my own kernel build or modifying the FS before booting to test the stuff I needed..

For mkosi it was the initial package extracting/install that failed for me, and it didn't cache the download step before that, so the entire thing was a PITA to debug. It also seemed like it wanted to run various loop devices and other things on the host when it was building the image, which I'd honestly rather it do inside a privileged container.. I also gave up trying to build a Fedora boot image from my Ubuntu box (again, needs too many host dependencies).

IMHO, the thing about docker, is it has one really good idea - dockerfile, with the overlayfs based caching layers and downloadable seed - and a bunch of not so good, out dated execution around that good idea. systemd has good execution in systemd-nspawnd, rkt provides most of the orchestration piece on top of that, but that entire world lacks a refined equivalent to dockerfile to make the images in the first place..

You are right, I used something between v1-v2 (dec 16 github TOT) in an attempt to do some work with systemd 232.

The mkosi OS generation tool

Posted Jun 29, 2017 13:16 UTC (Thu) by gdamjan (subscriber, #33634) [Link]

the fs layers are a good idea in docker (if inefficient)

The docker file itself, is just awful. It's like a worse shell script

The mkosi OS generation tool

Posted Jun 29, 2017 17:05 UTC (Thu) by zlynx (subscriber, #2285) [Link]

If it gets complicated you pull in a layer that has an actual script in it. Then call that instead of trying to duplicate Puppet or whatever in Dockerfile.

The mkosi OS generation tool

Posted Jun 30, 2017 8:30 UTC (Fri) by gdamjan (subscriber, #33634) [Link]

that's only practical now with the support for multi-stage builds (perhaps, I haven't looked too deeply into it)

The mkosi OS generation tool

Posted Jul 2, 2017 1:33 UTC (Sun) by sbaugh (subscriber, #103291) [Link]

9P shouldn't be used to boot a container with QEMU/KVM in production, though. It's too slow and quite insecure. So building GPT disk images is still very useful.

The mkosi OS generation tool

Posted Jul 5, 2017 23:23 UTC (Wed) by nix (subscriber, #2304) [Link]

Yeah. I wish there was a faster way to share existing host filesystems than NFS over localhost, but there doesn't seem to be one... (and NFSv4 is pretty nippy, actually. It's only if you're stuck on v3 or God forbid v2 that it starts to bite hard.)

The mkosi OS generation tool

Posted Jun 29, 2017 1:29 UTC (Thu) by pabs (subscriber, #43278) [Link]

The proliferation of tools in this space is incredible:

https://wiki.debian.org/SystemBuildTools

The mkosi OS generation tool

Posted Jun 29, 2017 2:11 UTC (Thu) by rahulsundaram (subscriber, #21946) [Link]

A lot of tools listed in that page seem pretty Debian specific though.

The mkosi OS generation tool

Posted Jun 29, 2017 17:37 UTC (Thu) by iainn (subscriber, #64312) [Link]

It's a useful tool, which coincidentally I discovered this week.

I wanted an easy way of spinning up another distro. My first thought was to check if LXD were suitable, but to use LXD on Fedora you have to disable SELinux. My second thought was Docker, but it wasn't suitable because I wanted to persist state changes.

I found mkosi + systemd-nspawn slightly easier, and hopefully provide better isolation, than setting up a chroot.

The mkosi OS generation tool

Posted Jun 30, 2017 16:57 UTC (Fri) by stgraber (subscriber, #57367) [Link]

Hmm, I'm the LXD project leader and we do run automated tests on Fedora 25 using LXD installed through snapd on systems that have SELinux in enforced mode, so you shouldn't have to turn it off.

sudo dnf install snapd
sudo snap install lxd
sudo lxd init

lxd on Fedora

Posted Jul 3, 2017 9:41 UTC (Mon) by iainn (subscriber, #64312) [Link]

Ah. I had been considering a COPR repo (similar to a PPA) to install LXD on Fedora. The LXD package from the COPR doesn't support SELinux unfortunately.

But thanks, that's good to know. I hadn't considered a Snap package. Nor had I seen the recent news that snaps work on Fedora with SELinux.

The mkosi OS generation tool

Posted Jul 2, 2017 3:39 UTC (Sun) by mrdocs (subscriber, #21409) [Link]

I'm wondering why no one looked at https://github.com/SUSE/kiwi It also builds for RH and Debian, as well as many output formats, ISO, docker images, cloud images etc.

Seems to me another NIH here by RH.

The mkosi OS generation tool

Posted Jul 2, 2017 15:02 UTC (Sun) by rahulsundaram (subscriber, #21946) [Link]

> Seems to me another NIH here by RH.

What makes you think Red Hat is deciding this?

The mkosi OS generation tool

Posted Jul 2, 2017 21:15 UTC (Sun) by jospoortvliet (subscriber, #33164) [Link]

A full time paid employee that did it? Who is working in and around this stack? I know rh doesn't work that black and white and I know lennart well enough to know he was most likely not told to do this but it isn't a crazy assumption...

The mkosi OS generation tool

Posted Jul 2, 2017 21:46 UTC (Sun) by rahulsundaram (subscriber, #21946) [Link]

>A full time paid employee that did it?

That is a really weak rationale to think that this is done under some organizational direction. systemd itself was written by Lennart on his own time originally.