User: Password:
Subscribe / Log in / New account

LC-Asia: An Android upstreaming update

Benefits for LWN subscribers

The primary benefit from subscribing to LWN is helping to keep us publishing, but, beyond that, subscribers get immediate access to all site content and access to a number of extra site features. Please sign up today!

By Jonathan Corbet
March 12, 2013
Many people have talked about the Android kernel code and its relation to the mainline. One of the people who has done the most to help bring Android and the mainline closer together is John Stultz; at the 2013 Linaro Connect Asia event, he talked about the status of the Android code. The picture that emerged shows that a lot of progress has been made, but there is still a lot of work yet to be done.

What's out there

John started by reviewing the existing Android kernel patches by category, starting with the core code: the binder interprocess communication mechanism, the ashmem shared memory mechanism, the Android logger, and monotonic event timestamps. The timestamp patch is needed to get timestamps from the monotonic clock for input events; otherwise it is hard to be sure of the timing between events, which makes gesture recognition hard. The problem is that these events cannot be added without breaking the kernel's ABI, so they cannot be just merged without further consideration.

There is a set of changes that John categorized as performance and power-consumption improvements. At the top of the list is the infamous "wakelock" mechanism, used by Android to know when the system as a whole can be suspended to save power. There is a special alarm device that can generate alarms that will wake the system from a suspended state. The Android low-memory killer gets rid of tasks when memory gets [John Stultz] tight; it is designed to activate more quickly than the kernel's out-of-memory killer, which will not act until a memory shortage is seriously affecting system performance. Also in this category is the interactive CPU frequency governor, which immediately ramps the CPU up to its maximum speed in response to touch events; its purpose is to help the system provide the fastest response possible to user actions.

The "debugging features" category includes a USB gadget driver that supports communication with the adb debugging tools; it is also used to support file transfer using the media transfer protocol (MTP). The FIQ debugger is a low-level kernel debugger with some unique features — communication through the device's headphone jack being one of them. The RAM console will save kernel messages for later recovery in case of a crash. There is the "key-reset" driver, a kind of "control-alt-delete for phones." The patches to the ARM architecture's "embedded trace macrocell" and "embedded trace buffer" drivers offer improved logging of messages from peripheral processors. Then there is the "goldfish" emulator, derived from QEMU, which allows Android to be run in an emulated mode on a desktop system.

The list of networking features starts with the "paranoid networking framework," the mechanism that controls which applications have access to the network; it restricts that access to members of a specific group. There is a set of netfilter changes mostly aimed at providing better accounting for which applications are using data. There are some Bluetooth improvements and the Broadcom "bcmhd" WiFi driver.

In the graphics category is the ION memory allocator, which handles DMA buffer management. The "sync" driver provides a sort of mutex allowing applications to wait for a vertical refresh cycle. There is also a miscellaneous category that includes the battery meta-driver, which provides wakelock support and thermal management. That category contains various touch screen drivers, the "switch" class for dealing with physical switches, and the timed GPIO facility as well. Finally, the list of deprecated features includes the PMEM memory allocator, the early suspend mechanism, the "apanic" driver, and the yaffs2 filesystem, which has been replaced by ext4.

Upstreaming status

Having passed over the long list of Android patches, John moved on to discuss where each stands with regard to upstreaming. The good news is that some of these features are already upstream. Wakelocks are, arguably, the most important of those; Rafael Wysocki's opportunistic suspend work, combined with a user-space emulation library, has made it possible for Android to move over to a mainline-based solution. John's monotonic event timestamp patches are also in the mainline, controlled by a special ioctl() command to avoid breaking the ABI; Android is using this mechanism as of the 4.2 ("Jelly Bean") release. The RAM console functionality is available via the pstore mechanism. The switch class is now supported via the kernel's "extconn" driver, but Android is not yet using this functionality.

A number of the Android patches are currently in the staging tree. These include the binder, ashmem, the logger, the low-memory killer, the alarm device, the gadget device, and the timed GPIO feature. The sync driver was also just pulled into the staging tree for merging in the 3.10 development cycle. With all of the staging code, John said, Android "just works" on a mainline kernel.

That does not mean that the job is done, though; quite a few Android patches are still in need of more work to get upstream. One such patch is the FIQ debugger; work is being done to integrate it with the kdb debugger, but, among other problems, the developers are having a hard time getting review attention for their patches. The key-reset driver was partially merged for the 3.9 kernel, but there are a number of details to be dealt with still. The plan for the low-memory killer is to integrate it with the mempressure control group patch and use the low-memory notification interface that is part of that mechanism; the developers hope to merge that code sometime soon. Ashmem is to be reimplemented via one of the volatile ranges patch sets, but there is still no agreement on the right direction for this feature. Much of the goldfish code has been merged for the 3.9 release.

The ION memory allocator has not yet been submitted for consideration at all. Much of this code duplicates what has been done with the CMA allocator and the DMA buffer sharing mechanism; integrating everything could be a challenge. There should be pieces that can be carved out and submitted, John said, even if the whole thing requires more work.

The interactive CPU frequency driver has been rejected by the scheduler developers in its current form. Supporting this feature properly could require some significant reworking of the scheduler code.

The netfilter changes have been submitted for inclusion, but there is some cleanup required before they can be merged. The paranoid networking code, instead, is not appropriate for upstream and will not be submitted. The right solution here would appear to be for Android to use the network namespaces feature, but that would require some big changes on the Android side, so it is not clear when it might happen.

The alarm device code needs to be integrated with the kernel's timerfd subsystem. Much of that integration has been done, but it requires an Android interface change, which is slowing things down. The embedded trace driver changes have been submitted, but the developer who did that work has moved on, so the code is now unmaintained. It is also undocumented and nobody else fully understands it at this point. There is a desire to replace the Android gadget driver with the CCG ("configurable composite gadget") code that is currently in the staging tree, but CCG does not yet do everything that Android needs, and it appears to be unmaintained as well. There was talk in the session of Linaro possibly taking over the development of that driver in the future.

Finally, it would be good to get the binder and logger patches out of the staging tree. That, however, is "complicated stuff" and may take a while. There is hope that the upcoming patches to support D-Bus-like communication mechanisms in the kernel will be useful to provide binder-like functionality as well.

There are a few issues needing longer-term thought. The integration of the sync driver and the DMA buffer sharing mechanism is being thought through now; there are a lot of details to be worked out. The upstreaming of ION could bring its own challenges. Much of that code has superficial similarities to the GEM and TTM memory managers that already exist in the kernel. Figuring out how to merge the interactive CPU frequency driver is going to be hard, even before one gets into details like how it plays with the ongoing big.LITTLE initiative. Some fundamental scheduler changes will be needed, but it's not clear who is going to do this work. The fact that Google continues to evolve its CPU frequency driver is not helping in this regard. There will, in other words, be plenty to keep developers busy for some time.

Concluding remarks

In total, John said, there are 361 Android patches for the kernel, with the gadget driver being the largest single chunk. Some of these patches are quite old; one of the patches actually predates Android itself. Google is not standing still; there is new code joining that which has been around for a while. Current areas of intensive development include ION, the sync driver, the CPU frequency driver, the battery driver, and the netfilter code. While some of the code is going into the mainline, the new code adds to the pile of out-of-tree patches shipped by the Android project.

Why should we worry about this, John asked, when it really is just another one of many forks of the kernel? Forking is how development gets done; see, for example, the development of the realtime patches or how many filesystems are written. But, he said, forks of entire communities, where code does not get merged back, are more problematic. In this case, we are seeing a lot of ARM systems-on-chip being developed with Android in mind from the beginning, leading to an increase in the use of out-of-tree drivers and kernels. Getting the Android base into the mainline makes it easier for developers to work with, and makes it easier to integrate Android-related code developed by others. John would like Android developers to see the mainline kernel, rather than the Android world, as their community.

Things are getting better; Zach Pfeffer pointed out that the work being done to bring Android functionality into the mainline kernel is, indeed, being used by the Android team. The relationship between that team and the kernel development community is getting better in general. It is a good time for people who are interested to join the effort and help get things done.

[Your editor would like to thank Linaro for travel assistance to attend this event.]

(Log in to post comments)

LC-Asia: An Android upstreaming update

Posted Mar 12, 2013 17:44 UTC (Tue) by gnu (subscriber, #65) [Link]

That's a great update.

Apart from the main Android patches, there are many vendor specific patches to Android. For example, for ICS, TI added a "dsscomp" driver. They chose not to use the DRM driver (perhaps it was not ready when ICS was in development) or the V4L2 driver. The net result is that the V4L2 output driver for OMAP4, which was perfectly working in the linux kernel in the 2.6.3x series, became unmaintained in the 3.x mainline and is not working till this date (even in the linaro's kernel tree).

LC-Asia: An Android upstreaming update

Posted Mar 12, 2013 19:38 UTC (Tue) by jstultz (subscriber, #212) [Link]

This is very true. For my talk, I did not cover the device specific android git trees, and instead I kept it limited to the common.git tree (which contains the changes required for every device).

LC-Asia: An Android upstreaming update

Posted Mar 12, 2013 17:45 UTC (Tue) by epa (subscriber, #39769) [Link]

Rafael Wysocki's opportunistic suspend work, combined with a user-space emulation library, has made it possible for Android to move over to a mainline-based solution.
But are the Android developers going to do that? They might have different ideas about whether the solution blessed by the kernel developers is the right one for their needs.

LC-Asia: An Android upstreaming update

Posted Mar 12, 2013 21:48 UTC (Tue) by jstultz (subscriber, #212) [Link]

So for wakelocks, the Android developers have already moved over to the same code upstream is using (they even backported it to their 3.4 kernel branch, and I believe Nexus4 & Nexus10 devices are already using it).

But your larger point is right, collaborating and getting buy in from the Android developers at Google is really important. We don't want to push code upstream to "solve their problems" and then find they won't or can't actually make use of it.

That said, sometimes what gets merged upstream is more of a base foundational framework and does not provide 100% coverage for their needs. In those cases we can still rework portions of their drivers to make use of the properly merged code, simplifying the delta.

In my mind, the best approach is an iterative one, slowly chipping away at the portions we can find agreement on, which allows for better understanding and discussion of the exact differences.

Uptake by Android

Posted Mar 13, 2013 12:09 UTC (Wed) by corbet (editor, #1) [Link]

This was mentioned twice in the article, but I probably should have made an even bigger deal of it: yes, the Android developers are making use of the mainline features as they become useful. The whole wakelock/suspend blocker fight has quietly come to an end, and nobody even noticed. It's great news, and a credit to both the Android and mainline camps, both of which have worked hard to reach this point.

Uptake by Android

Posted Mar 14, 2013 10:09 UTC (Thu) by epa (subscriber, #39769) [Link]

That's great news. I felt the article was a bit ambiguous, saying the work has 'made it possible' for the Android developers to use the mainline feature - which can mean that this has happened, but can also mean just that it's 'possible'.

LC-Asia: An Android upstreaming update

Posted Mar 12, 2013 18:21 UTC (Tue) by mlankhorst (subscriber, #52260) [Link]

merging sync driver sounds like fun

LC-Asia: An Android upstreaming update

Posted Mar 12, 2013 19:11 UTC (Tue) by gregkh (subscriber, #8) [Link]

Already merged, is in linux-next, will show up in the 3.10 kernel release.

LC-Asia: An Android upstreaming update

Posted Mar 12, 2013 19:12 UTC (Tue) by jstultz (subscriber, #212) [Link]

Thanks for the great summary, Jon!

If folks want to also see the slides from the talk, they can be found here:


Posted Mar 13, 2013 11:42 UTC (Wed) by tialaramex (subscriber, #21167) [Link]

As of the last time I was able to check current Android still arbitrarily merges device driver keyboard events for no discernible reason.

The scenario is an external (USB, Bluetooth) keyboard plugged into an Android device, European keyboards tend to have one extra key KEY_102ND compared to a US keyboard. The purpose of this key varies by layout but (of course) it's not just an identical copy of an existing key so it needs to be treated separately and Linux does so. Android takes the Linux kernel's internal key identifiers and lays its own equivalent yet different identifiers on top (presumably for some reason that made sense to the early Android developers). In the process it takes the "extra" key KEY_102ND and maps it to Android's backslash key, even though there's already such a mapping for KEY_BACKSLASH.

And that's doom. Despite having an extensive layout and remapping capability Android has already irrevocably broken these keyboards, because software on top of Android can't tell the two different keys apart. I can scarcely begin to imagine how somebody thought this was a good idea, unless perhaps they had literally never seen a non-US keyboard or used a non-English language.


Posted Mar 13, 2013 21:31 UTC (Wed) by khim (subscriber, #9252) [Link]

I can scarcely begin to imagine how somebody thought this was a good idea, unless perhaps they had literally never seen a non-US keyboard or used a non-English language.

I'm pretty sure they have seen non-US keyboard and wondered just why this keyboard wastes valuable space and cripples Enter key just to introduce second backslash. Guy from Israel or Russia can do that just as easily as a guy from US, you know: it may surprise you but lots of languages don't need this 102ND key and for them it's just "#@^%%#@(*!@ second backslash".


Posted Mar 13, 2013 22:08 UTC (Wed) by mpr22 (subscriber, #60784) [Link]

*scratches head* Um, it's American keyboards that have the crippled one-row-high Enter key.


Posted Mar 14, 2013 7:27 UTC (Thu) by khim (subscriber, #9252) [Link]

You are correct, of course: most US keyboards sacrifice Enter key to make Backspace larger (as if I'm obsessed with deleting stuff). My bad.

European keyboards sacrifice Left Shift size which is even worse: I can still hit small US-style Enter key with my little finger, but small Shift is a disaster since attention is usually on other hand. Ideally I want classic layout similar to this one (with two large Shifts, and large Enter L-shaped Enter) which is rare enough with 101-keys-layout keybords (nowadays they have 104 keys because Microsoft added three keys) and I'm yet to see even a single 102-keys-layout keyboard (with 105 keys) with all three.

LC-Asia: An Android upstreaming update

Posted Mar 13, 2013 12:26 UTC (Wed) by dvdeug (subscriber, #10998) [Link]

So we have CCG code being compiled by GCC. Maybe we should starting going to FLAs (four letter acronyms) (err, ETLAs, extended TLAs?)

LC-Asia: An Android upstreaming update

Posted Mar 14, 2013 12:01 UTC (Thu) by njwhite (guest, #51848) [Link]

What proportion of this good upstreaming work is being done by Google's android team? I'm glad they're generally making use of upstream once things are merged acceptably, but I'm not clear on whether it's the Google people themselves who are pushing in this direction.

CCG on its way out?

Posted Mar 21, 2013 8:52 UTC (Thu) by pebolle (subscriber, #35204) [Link]

> There is a desire to replace the Android gadget driver with the CCG
> ("configurable composite gadget") code that is currently in the staging
> tree, but CCG does not yet do everything that Android needs, and it
> appears to be unmaintained as well. There was talk in the session of
> Linaro possibly taking over the development of that driver in the future.

Note that, only two days after this article was published, Greg Kroah-Hartman removed CCG from the staging-next tree. See, for the first of two patches involved:

This is not in mainline yet, but it looks like CCG is on its way out.

CCG on its way out?

Posted Mar 29, 2013 0:04 UTC (Fri) by jstultz (subscriber, #212) [Link]

Yea, so this was maybe not as clear/precise in the summary. But CCG being removed is in part due to some of the discussions at Linaro Connect.

CCG, basically a modified version of the android gadget driver, was merged into staging, but the Android developers decided against moving to use it, and the upstream maintainers want to see a configfs based gadget instead. Thus the CCG driver had been effectively abandoned and developer efforts have been focused on the configfs gadget.

Mix in the fact that the functionfs gadget driver already can also be used for adb, and you get a bit of a mess as to what the right approach is going forward. (How many configurable composite gadgets do we need?)

That said, the configfs gadget driver has been slow-going in development, so if necessary it may be something Linaro would try to help along (rather then "taking over development").

Copyright © 2013, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds