User: Password:
Subscribe / Log in / New account

ELCE11: Sandboxing for automotive Linux

Did you know...? is a subscriber-supported publication; we rely on subscribers to keep the entire operation going. Please help out by buying a subscription and keeping LWN on the net.

November 2, 2011

This article was contributed by Nathan Willis

On the second day of the Embedded Linux Conference Europe (ELCE), Iisko Lappalainen from MontaVista Software presented a method for running secondary Linux environments inside a "host" Linux OS with strict sandboxing and security requirements. The example use-case was running Android inside a GENIVI-based Linux in-vehicle infotainment (IVI) system, though other combinations are possible. The setup would permit a car-maker to ship a system with full access to an Android application ecosystem, while maintaining isolation from the underlying OS.

As GENIVI's Matt Jones explained in an earlier session, the GENIVI middleware stack is isolated from the vehicle's safety-critical systems like engine control and anti-lock braking; running on separate hardware on electrically isolated circuits. But there are still important functions in an IVI system that, if interrupted, would greatly inconvenience the user. Navigation on the head unit, or proximity sensors on the bumpers, for example — neither one should hang or crash just because a child in the back is playing a buggy video game on a rear-seat entertainment screen. Buggy games aside, there is also always the prospect of intentionally malicious code.

That provides one use-case for running applications inside some sort of sandboxed environment. Lappalainen listed several others. First, it would provide a way to run applications written for multiple UI frameworks, in particular, frameworks not natively supported by the base IVI system. The example he presented was an HTML5-based web runtime, which was not a component planned for the MeeGo IVI user experience (UX), which GENIVI designated in 2010 as its reference platform. Canonical has subsequently announced its own GENIVI-compliant IVI remix, which also does not address a web runtime; MeeGo's successor Tizen, however, does have a web runtime.

The Android environment in particular offers its own advantages as the sandboxed OS, Lappalainen said. The existing ecosystem is enormous, both in terms of applications and trained developers. Android's "app store" model also explicitly supports multiple, branded app stores, which would allow OEMs to provide their own software product channel directly in the IVI system. Finally, if done right, sandboxing should allow the OEM to enforce a tight security model on the applications inside — perhaps providing a more isolated environment for untrusted, user-installed applications, while factory-installed applications are allowed to run natively.

The containerized approach

The sandboxing approach taken by MontaVista utilizes Linux Containers (LXC) to isolate the sandboxed environment, and SELinux to supply a security layer. LXC containers provide a form of virtualization by isolating the sandboxed processes in a separate control group — thus allowing the host OS to limit resource usage and isolate file access — and by maintaining separate process ID and network namespaces. Separate namespaces not only hide the host OS from each container, but isolate each container from the others.

Lappalainen referred to this approach as "virtualization," but that term can mean different things to different people. Specifically, LXC containers provide OS-level virtualization akin to OpenVZ or Virtuozzo. A system running inside an LXC container can have its own view of the filesystem and a separate group of processes — with entirely different user-space code — but it still uses the same kernel as the "host" (for lack of a better term) OS. This is a distinct difference from hardware-level virtualization, which supports running any flavor of guest OS on top of the host OS. On the other hand, OS-level virtualization is generally faster because there is no overhead associated with running a virtual machine layer.

But OS-level virtualization also introduces a hurdle to running one Linux-based OS inside of another if the two OSes differ significantly in kernel features, not just userspace. That is certainly the case with Android, which replaces several stock kernel features and adds several other features. In MontaVista's Android-on-GENIVI project, the host kernel is patched with Android-specific features.

Lappalainen listed the Android kernel's IPC binder, low-memory killer, logger device, and asynchronous shared-memory system (ashmem) in particular on his slides; in the talk however he simply described the kernel as including the "Android patches." He also mentioned that these kernel functions needed to be adapted to work only within the context of the Android container. In particular, Android replaces the standard Linux out-of-memory (OOM) killer with its own variety. One would only want the low-memory killer to watch for low memory conditions within the Android container, and then to only kill one of the Android container's processes.

The guest-OS containers are configured so that their processes run at lower priority than the host OS's. There are also various mechanisms used to process IO, graphics, and other resources for the collection of containers. The "event dispatcher" tracks the window coordinates of each application, for example, so it can route input events to the proper container or to the host OS. Graphics output is handled by capturing the Android container's frame buffer, and sending it to a "layer manager" that overlays it on the display together with video output from the other applications. Audio is less tricky to coordinate, he said, because it can be down-mixed into one output by the audio server. This is already what ALSA and PulseAudio do when multiple applications play sound simultaneously.

Power management is handled entirely by the host OS, which Lappalainen said required changes to the Android wakelock code. On multi-core systems, he added, the container-management code can also be used to bind containers to specific processors, which provides another method of ensuring that they cannot bring down the host OS even in the event of a serious fault.

Lappalainen did not go into much detail on the role that SELinux plays in providing further isolation for the LXC containers. It is certainly possible that SELinux could simply be set up to duplicate the filesystem isolation and other sandboxing mechanisms provided by LXC, acting as a separate, back-up "wrapper" around the containers. But SELinux might also plug security holes in LXC. For example, LXC does not provide user namespaces, which means that a malicious root user could escape from its container and execute code as the root user on the host OS.

Code and product

Lappalainen outlined various use-cases for the LXC/SELinux containerization approach, noting that it could also be beneficial in other embedded Linux projects because it can isolate untrusted applications, but without the performance hit of running them in an emulator. MontaVista's implementation of this configuration is its Automotive Technology Platform (ATP), a commercial IVI product.

The company announced ATP's Android-and-HTML5 support feature in an October 10 press release, which positions ATP as a competitor to open projects like MeeGo/Tizen and Ubuntu IVI — in particular, one that has a leg up on the competition thanks to the vast array of already-written Android and HTML5 applications. IVI was not a major topic at ELCE; Jones' talk was the only other session dedicated to IVI, and it dealt as much with the plans and in-house experiments of his employer Jaguar Land Rover (JLR) as it did with GENIVI.

An illuminating snippet from that talk, however, was that it will be 2014 at the earliest before any Linux-based IVI systems are available in JLR vehicles. That is an exceptionally long time in kernel and distribution time-scales. A few other car-makers are reported to be closer, notably BMW, but have not announced a deployment schedule.

In fact, ever since the announcement of the MeeGo IVI platform, it seems that the IVI software industry has changed drastically faster than the car industry with which the rival platforms are vying to go into business. There were rumors that GM would adopt Android as the next-generation base for its OnStar system, only to have the company join GENIVI instead. MeeGo brought on several major car-makers as partners (including Toyota) in early 2011, then MeeGo morphed into Tizen without warning.

That much change can make it difficult to handicap the players. However, the big obstacle for ATP is likely to be asking car-makers to undertake supporting Android and a GENIVI Linux distribution. Even apart from the handset-and-tablet-centric stance that Google takes with the product, it sounds like a challenging customer support undertaking. In 2011, GENIVI quietly began shifting its language away from talk of a MeeGo "reference implementation" and towards "GENIVI compliance," which blesses multiple distributions. That could be because GENIVI had early warning of the migration from MeeGo to Tizen; regardless of whether or not GENIVI formally adopts Tizen as its reference platform, Tizen will match ATP's HTML5 support, which could make the web-runtime selling-point moot.

In short, though the work that has gone into virtualizing "guest" Linux OSes in MontaVista's ATP is interesting, it seems odd to position it as an IVI-specific technology. There are certainly plenty of users of other form factors that would love the chance to run thousands of Android applications inside a secure sandbox — starting with smartphone and desktop Linux distributions. Whether ATP, Android itself, or some other solution entirely is adopted by GENIVI or the car-makers remains to be seen, but it does seem likely that a hybrid like ATP will be fighting an uphill battle.

[The author would like to thank the Linux Foundation for assisting with his travel to the Embedded Linux Conference Europe 2011.]

(Log in to post comments)

ELCE11: Sandboxing for automotive Linux

Posted Nov 3, 2011 1:26 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]

Could they post the models of cars which are going to run in-car entertainment and navigation/lighting controls on the same CPU? I need to know which models I should avoid.

ELCE11: Sandboxing for automotive Linux

Posted Nov 3, 2011 7:23 UTC (Thu) by simlo (guest, #10866) [Link]

If you have real-time operating system - with no bugs - it should not be a problem. Of course, any OS have a bug somewhere - especially around ensuring timing.

I once saw Windriver talking about the same thing: Using time slices for virtual machines (running VxWorks) within a simplified VxWorks. This was for aerospace, not automotive.

ELCE11: Sandboxing for automotive Linux

Posted Nov 3, 2011 16:23 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]

I don't trust OS-level sandboxes to contain malicious processes. System-level sandboxes (KVM, Xen) just might be able to do it but even that is doubtful.

That's why I'd just separate critical functionality into a completely separate CPU, maybe even with a separate network.

ELCE11: Sandboxing for automotive Linux

Posted Nov 10, 2011 20:17 UTC (Thu) by lamadiHH (guest, #80884) [Link]

When you refer to "aerospace" I think you mean VxWorks 653 that is compliant to the ARINC 653 (Integrated Modular Avionics). This edition is qualified according to DAL A and thus meets the highest safety requirements in the industry. VxWorks is not Linux.

ELCE11: Sandboxing for automotive Linux

Posted Nov 3, 2011 14:16 UTC (Thu) by davecb (subscriber, #1574) [Link]

It's also interesting that GENIVI uses both a kernel virtual machine, LXC, and SELinux. Solaris based their kernel virtual machine *on* the security primitives used by Trusted Solaris, their B1-level military offering.

I speculate that there is likely to be quite an overlap between SEL and LXC, something that might trigger some interesting cross-pollination.

--dave (my uncle the farmer really liked cross-pollination) c-b

ELCE11: Sandboxing for automotive Linux

Posted Nov 3, 2011 18:15 UTC (Thu) by jimparis (subscriber, #38647) [Link]

That's a strange approach to virtualization. Trying to blend the Android and host kernels into one that has the features that Android needs, but still provides the isolation the host needs, sounds tricky, and prone to bugs and security problems. If done right, hardware-level virtualization (ie KVM) should have only minor performance implications, especially on a platform where things like network and file I/O are already quite slow and shouldn't be affected by a small CPU overhead.

Power management seems like a minor concern. Cars have orders of magnitude more energy available -- a typical car battery holds something like 500 W-h, whereas the Nexus One battery is about 5 W-h. Your overhead dome lamp draws more power than your phone.

Is the assumption that all car IVI systems will have some form of always-on network connection? Android certainly doesn't seem designed for an offline use case. I don't know how things like a store would even work in that case, and many applications and games are supported by ads.

ELCE11: Sandboxing for automotive Linux

Posted Nov 3, 2011 23:00 UTC (Thu) by martinfick (subscriber, #4455) [Link]

> If done right, hardware-level virtualization (ie KVM) should have only minor performance implications, especially on a platform where things like network and file I/O are already quite slow and shouldn't be affected by a small CPU overhead.

I think that is dreaming. OS level virtualization can handle 1000s of guests, do you think KVM "done right" could even handle 100?

ELCE11: Sandboxing for automotive Linux

Posted Nov 4, 2011 19:59 UTC (Fri) by jimparis (subscriber, #38647) [Link]

When you start talking about 100 or 1000 guests, the limiting factors to full virtualization quickly become I/O bandwidth, scheduler pressure, RAM, etc. Virtualizing exactly 1 guest is an entirely different problem, especially if the primary goals are security and trying to mix two dissimilar systems. So yeah, I do think that "KVM done right" is far better for isolating a single Android instance than trying to modify both the host and guest to coexist. That's not to say that OS level virtualization doesn't have its uses.

Copyright © 2011, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds