Many people have talked about the Android kernel code and its relation
to the mainline. One of the people who has done the most to help bring
Android and the mainline closer together is John Stultz; at the 2013 Linaro
Connect Asia event, he
talked about the status of the Android code. The picture that emerged
shows that a lot of progress has been made, but there is still a lot of
work yet to be done.
What's out there
John started by reviewing the existing Android kernel patches by
category, starting with the core code: the binder interprocess
communication mechanism, the ashmem shared
memory mechanism, the Android
logger, and monotonic event
timestamps. The timestamp patch is needed to
get timestamps from the monotonic clock for input events; otherwise it is
hard to be sure of the timing between events, which makes gesture
recognition hard. The problem is that these events cannot be added without
breaking the kernel's ABI,
so they cannot be just merged without further consideration.
There is a set of changes that John categorized as
performance and power-consumption improvements. At the top of the list is
the infamous "wakelock" mechanism, used by Android to know when the system
as a whole can be suspended to save power. There is a special alarm
device that can generate alarms that will wake the system from a suspended
state. The Android low-memory killer gets rid of tasks when memory gets
tight; it is designed to activate more quickly than the kernel's
out-of-memory killer, which will not act until a memory shortage is
seriously affecting system performance. Also in this category is the
interactive CPU frequency governor, which immediately ramps the CPU up to
its maximum speed in response to touch events; its purpose is to help the
system provide the fastest response possible to user actions.
The "debugging features" category includes a USB gadget driver that
supports communication with the adb debugging tools; it is also
used to support file transfer using the media transfer
protocol (MTP). The FIQ debugger is a low-level kernel debugger
with some unique features — communication through the device's headphone
jack being one of them. The RAM console will save kernel messages for
later recovery in case
of a crash. There is the "key-reset" driver, a kind of
"control-alt-delete for phones." The patches to the ARM architecture's
"embedded trace macrocell" and
"embedded trace buffer" drivers offer improved logging of
messages from peripheral processors. Then there is the "goldfish"
from QEMU, which allows Android to be run in an emulated mode on a desktop
The list of networking features starts with the "paranoid networking
framework," the mechanism that controls which applications have access to
it restricts that access to members of a specific group. There is a set of
netfilter changes mostly aimed at providing better accounting for which
applications are using data. There are some Bluetooth improvements and the
Broadcom "bcmhd" WiFi driver.
In the graphics category is the ION memory
allocator, which handles DMA buffer management. The
"sync" driver provides a sort of mutex allowing applications to wait for
a vertical refresh cycle. There is also a miscellaneous category that
includes the battery meta-driver, which provides wakelock support and thermal
management. That category contains various touch screen drivers, the
"switch" class for
dealing with physical switches, and the timed GPIO facility as well.
Finally, the list of deprecated features includes the PMEM memory
allocator, the early suspend mechanism, the "apanic" driver, and the
yaffs2 filesystem, which has been replaced by ext4.
Having passed over the long list of Android patches, John moved on to
discuss where each stands with regard to upstreaming. The good news is
that some of these features are already upstream. Wakelocks are, arguably,
the most important of those; Rafael Wysocki's opportunistic
suspend work, combined with a user-space emulation library, has made it
possible for Android to move over to a mainline-based solution. John's
monotonic event timestamp patches are also in the mainline, controlled by a
special ioctl() command to avoid breaking the ABI; Android is
using this mechanism as of the 4.2 ("Jelly Bean") release. The RAM console
is available via the pstore mechanism. The
switch class is now supported via the kernel's "extconn" driver, but
Android is not yet using this functionality.
A number of the Android patches are currently in the staging tree.
These include the binder, ashmem, the logger, the low-memory killer, the
alarm device, the gadget device, and the timed GPIO feature. The sync
driver was also just pulled into the staging tree for merging in the 3.10
development cycle. With all of
the staging code, John said, Android "just works" on a mainline kernel.
That does not mean that the job is done, though; quite a few Android
patches are still in need of more work to get upstream. One such patch is
the FIQ debugger; work is being done to integrate it with the kdb
debugger, but, among other problems, the developers are having a hard time
getting review attention for their patches. The key-reset driver was
partially merged for the 3.9 kernel, but there are a number of details to
be dealt with still. The plan for the low-memory killer is to integrate it
with the mempressure control group patch and
use the low-memory notification interface that is part of that mechanism; the
developers hope to merge that code sometime soon. Ashmem is to be
reimplemented via one of the volatile ranges patch sets, but
there is still no agreement on the right direction for this feature.
Much of the goldfish code has been merged for the 3.9 release.
The ION memory allocator has not yet been submitted for consideration
at all. Much of this code duplicates what has been done with the CMA allocator and the DMA buffer sharing mechanism; integrating
everything could be a challenge. There should be pieces that can be carved
out and submitted, John said, even if the whole thing requires more work.
The interactive CPU frequency driver has been rejected by the scheduler
developers in its current form. Supporting this feature properly could
require some significant reworking of the scheduler code.
The netfilter changes have been submitted for inclusion, but there is some
cleanup required before they can be merged. The paranoid networking
code, instead, is not appropriate for upstream and will not be submitted.
The right solution here would appear to be for Android to use the network
namespaces feature, but that would require some big changes on the Android
side, so it is not clear when it might happen.
The alarm device code needs to be integrated with the kernel's timerfd subsystem. Much of that integration
done, but it requires an Android interface change, which is slowing things
down. The embedded trace driver changes have been submitted, but the
developer who did that work has moved on, so the code is now unmaintained.
It is also undocumented and nobody else fully understands it at this
point. There is a desire to replace the Android gadget driver with the CCG
("configurable composite gadget") code that is currently in the staging
tree, but CCG does not yet do everything that Android needs, and it appears
to be unmaintained as well. There was talk in the session of Linaro
possibly taking over the development of that driver in the future.
Finally, it would be good to get the binder and logger patches out of the
staging tree. That, however, is "complicated stuff" and may take a while.
There is hope that the upcoming patches to support D-Bus-like communication
mechanisms in the kernel will be useful to provide binder-like
functionality as well.
There are a few issues needing longer-term thought. The integration of the
sync driver and the DMA buffer sharing mechanism is being thought through
now; there are a lot of details to be worked out. The upstreaming
of ION could bring its own challenges. Much of that code has superficial
similarities to the GEM and TTM memory managers that already exist in the
kernel. Figuring out how to merge the interactive CPU frequency driver is
going to be hard, even before one gets into details like how it plays with
the ongoing big.LITTLE initiative. Some fundamental
scheduler changes will be needed, but it's not clear who is going to do
this work. The
fact that Google continues to evolve its CPU frequency driver is not
helping in this regard. There will, in other words, be plenty to keep
developers busy for some time.
In total, John said, there are 361 Android patches for the kernel, with the
gadget driver being the largest single chunk. Some of these patches are quite
old; one of the patches actually predates Android itself. Google is not
standing still; there is new code joining that which has been around for a
while. Current areas of intensive development include ION, the sync
driver, the CPU frequency driver, the battery driver, and the netfilter
code. While some of the code is going into the mainline, the new
code adds to the pile of out-of-tree patches shipped by the Android project.
Why should we worry about this, John asked, when it really is just another
one of many forks of the kernel? Forking is how development gets done;
see, for example, the development of the realtime patches or how many
filesystems are written. But, he said, forks of entire communities, where
code does not get merged back, are more problematic. In this case, we
are seeing a lot of ARM systems-on-chip being developed with Android in
mind from the beginning, leading to an increase in the use of out-of-tree
drivers and kernels. Getting the Android base into the mainline makes it
easier for developers to work with, and makes it easier to integrate
Android-related code developed by others. John would like Android
developers to see the mainline kernel, rather than the Android world, as
Things are getting better; Zach Pfeffer pointed out that the work being
done to bring Android functionality into the mainline kernel is, indeed,
being used by the Android team. The relationship
between that team and the kernel development community is getting better in
general. It is a good time for people
who are interested to join the effort and help get things done.
[Your editor would like to thank Linaro for travel assistance to attend
to post comments)