LWN: Comments on "IncludeOS: a unikernel for C++ applications"

IncludeOS: a unikernel for C++ applications

perbu — Fri, 04 Aug 2017 07:01:07 +0000

It is worth noting that port of IncludeOS to ukvm/solo5 that IBM has done boots, executes and exits in 4-5 ms. So it definitely possible for a unikernel to go way faster than Linux.

Do Unikernels even have real value?

perbu — Thu, 03 Aug 2017 14:59:38 +0000

The amount of legacy shit we currently have to deal with in order to boot virtual machines is incredible. If virtual machines didn't have to pretend it's 1982 development would be a lot quicker.

The reason unikernels need to run in ring 0 is only because of legacy. At some point I expect IncludeOS to start in ring 0, set up page tables and hardware and then chain-load a second includeos unikernel that is running in ring 3. As they are single process the need to restrict access to things like virtual network adaptors is not needed.

But even if we're running in ring 0 trying to compromise a virtual machine with an unknown memory layout and no system calls is .,. challenging. And contrary to what you're indicating there is nothing magical about running in ring 0. Code doesn't automatically get insecure by escalating its privileges. If your application is running on ring 3 on a Linux server that application has a lot more control over the vm than a unikernel has. If I compromise your linux application I can execute the shell, write files, execute processes, call home and do all sort of crazy stuff. If I compromise a Unikernel I can ... well, there isn't really much you can do as everything that isn't explicitly used by the application gets left out by the linker.

Even if there is functionality to say, connect home, how would you call that function? Trying to guess 64 bit addresses?

IncludeOS: a unikernel for C++ applications

perbu — Thu, 03 Aug 2017 14:43:12 +0000

The dev branch of IncludeOS does support SMP. For an upcoming delivery we need to do TLS at scale and we're using SMP to handle a large amount of incoming TLS connections.

So multicore is supported. Sort of, at least.

IncludeOS: a unikernel for C++ applications

smckay — Wed, 02 Aug 2017 02:14:24 +0000

That's my take as well. If what you want is to ignore OS administration as much as possible, there are multiple cloud vendors working very hard on that. You even get logging!

IncludeOS: a unikernel for C++ applications

viesti — Tue, 01 Aug 2017 19:26:04 +0000

I might be totally off, but to me services like AWS Lambda provide some of the unikernel idea (narrowly scoped services), but with a very convential infrastructure. I'd think that these services would be a tough competition for unikernels.

Do Unikernels even have real value?

robbe — Mon, 31 Jul 2017 14:29:29 +0000

For hypervisors in general, you are right. But unikernels eschew user space, so you are only left with rings -1 and 0…

Do Unikernels even have real value?

paulj — Sun, 30 Jul 2017 10:05:38 +0000

The one difference is hardware protection. You now have the hypervisor in ring -1, the guest kernel(s) in ring 0 and userspace(s) in respective ring 1s. Each with different hardware privilege levels.

IncludeOS: a unikernel for C++ applications

epa — Fri, 28 Jul 2017 20:31:12 +0000

Moreover, remember that a Linux process's execution environment can also be considered a virtual machine. It certainly isn't running on the physical hardware with physical addresses...

IncludeOS: a unikernel for C++ applications

flussence — Fri, 28 Jul 2017 16:22:21 +0000

It sounds like there's some logic to the design choice: not bothering to support multiple cores neatly sidesteps having to think about or write code for whole classes of problems - no need to handle scheduling, synchronisation or IPIs in every program, just pure computation and bit-banging virtio devices.

The downside is that ignoring the problems won't make most of them go away; to avoid side-channel attacks you'll need a whole air gap between instances, not a hypervisor.

Do Unikernels even have real value?

alonz — Fri, 28 Jul 2017 07:22:27 +0000

I'm in the camp that considers the entire idea of unikernels to be a net regression in almost all meaningful metrics.

When you ignore the hype, a hypervisor is no different than a run-of-the-mill kernel. It has precisely the same tools at its disposal for dividing resources among tasks (sorry, “domains” is the new buzzword for those) and protecting them from interfering with each other. The only true difference is that hypervisors expose a lower-level API (virtual hardware) vs. kernels' richer interfaces, which actually gives the hypervisor less power to regulate behavior.

When this is combined with unikernels, we have userspace code running in the CPU's protected mode—which means vulnerabilities in this code give attackers even more powerful tools to play with. And this code talks to hardware via minimally-protected interfaces. Nothing to worry about, right?

In short: I hope this remains as an academic exercise. And I doubt any serious cloud providers will allow such magical-thinking solutions near their hardware.

IncludeOS: a unikernel for C++ applications

clugstj — Thu, 27 Jul 2017 12:07:59 +0000

An OS that only supports a single CPU core?

IncludeOS: a unikernel for C++ applications

drag — Wed, 26 Jul 2017 15:36:47 +0000

I think that sometimes people have a wrong perspective on virtual machines vs containers vs applications.

The virtual machine itself really is just a special case application. The virtual machine is not 'emulated PC' anymore. There is no interpretation of cpu instructions going on. The application code running in a virtual machine is executing on the CPU just like any normal application. Just has a couple extra layers of memory address abstraction and whatnot.

Modern VMs depend heavily on paravirtualization to be efficient. They are aware of the cpu types and make all sorts of efforts to reduce software layers between applications and physical network and physical storage to as minimal as possible. As time goes on the VM becomes more and more aware that it is a VM.

So what we end up with is really much closer to Java virtual machines then anything else. Major difference is that instead of executing Java bytecode they use x86_64 machine code.

After all this really isn't that much different then the same sort of transition that happened when you went from 'single user' application environments like a DOS.. which was little more then a fancy program loader.. to multi-user multi-process environments. Throw in some memory address abstractions and there you go.

And if a attacker is able to seize control of a virtual machine, which shouldn't be very difficult, then
it's not much more difficult to attack the hosting system then it is if you took over any other userland process like Firefox. If you want substantial improvements in security you are still going to need to depend on available DAC and MAC controls in the host OS to aid in the natural sort 'sandboxing' that VMs offer.

All these things... running virtual machines, containers, or executing applications... really should be thought of as pretty much the same things. There isn't a really good reason, other then historical happenstance and tradition, why they are treated so differently.

IncludeOS: a unikernel for C++ applications

Tara_Li — Wed, 26 Jul 2017 15:06:16 +0000

The biggest advantage I see is the reduced attack surface. You can turn off services, you can block them from not connecting to the outside world - but if they're never compiled into existence in the first place, they *cannot* be turned back on, or unblocked. And if it were running on bare metal instead of in a VM, your responsiveness would likely be a lot more predictable.

IncludeOS: a unikernel for C++ applications

jhoblitt — Wed, 26 Jul 2017 14:45:18 +0000

There are some novel uses for ulta-fast boot times. Imagine a service that follows a pre-fork style pattern but instead of fork()ing, spawns a new unikernel to handle each incoming request. That probably earns the service some extra tin-foil points.

My continuing ignorant and naive concern about the basic concept is debugging. In particular, some sort of valgrind analog...

IncludeOS: a unikernel for C++ applications

josh — Wed, 26 Jul 2017 14:14:09 +0000

> Booting up in 100 ms instead of two seconds is surely nice, but is really VM startup time an important metric in a cloud deployment?

VM startup time is important, but these days, you can boot a Linux VM in around 150ms. See the Clear Containers work.

I do see value in these kinds of unikernels for other purposes, such as experiments in reduced overhead between the application and network. (On the other hand, you can also do that with Linux, and bypass its network stack. One of these days I hope we improve the Linux network stack to the point that nobody feels the need to.)

But for boot time, no, I don't think this helps.

IncludeOS: a unikernel for C++ applications

mato — Wed, 26 Jul 2017 12:59:08 +0000

> You also lose multiple major benefits of normal OSes. The unikernel is almost certainly not as scalable, tested, or featureful as the kernel it replaces (definitely true here!)

That's only partially true. Modern green-field unikernels can provide better implementations of existing features (e.g. MirageOS' type- and memory-safe TCP/IP and TLS stacks). And the implementations will mature with time and use, as does all software.

> and you are essentially taking yourself back to the 1960s, when all OSes were distinct monsters tied to specific preferred languages with unique APIs, and if you wanted to introduce something in a different language you were suddenly writing lots of bridging layers

I'd argue that you'd face the same problem (bridging language "worlds" and competing runtimes) in any conventional application where you use multiple languages in the same *process*.

IncludeOS: a unikernel for C++ applications

nix — Wed, 26 Jul 2017 10:47:19 +0000

You also lose multiple major benefits of normal OSes. The unikernel is almost certainly not as scalable, tested, or featureful as the kernel it replaces (definitely true here!), and you are essentially taking yourself back to the 1960s, when all OSes were distinct monsters tied to specific preferred languages with unique APIs, and if you wanted to introduce something in a different language you were suddenly writing lots of bridging layers: as for switching away from IncludeOS if you found you needed something a real kernel can provide, forget it.

IncludeOS: a unikernel for C++ applications

mnowak@suse.com — Wed, 26 Jul 2017 10:05:40 +0000

Agreed. Maybe if Linux (SELinux-less) containers actually contained, we would not have to bother with this concept at all. For every such article should mandatory to answer concerns from http://dtrace.org/blogs/bmc/2016/01/22/unikernels-are-unf..., I believe.

IncludeOS: a unikernel for C++ applications

Sesse — Wed, 26 Jul 2017 09:45:48 +0000

So, a question I never really see answered in this kind of context is: Is it really worth it? Sure, the OS is 1/100 the size of a full Linux installation, but so? Is really 64 MB extra of RAM per instance (and some disk space) an issue if your software uses gigabytes of it anyway, and is it worth giving up significant amounts of debuggability and compatibility for? Booting up in 100 ms instead of two seconds is surely nice, but is really VM startup time an important metric in a cloud deployment? How do the TCP stacks of these things stack up against modern OSes anyway, and what happens if you actually need to use multiple cores in a non-trivial way?

/* Steinar */