Python on mobile systems

A session "led" by Russell Keith-Magee at the 2015 Python Language Summit was actually a video he created about mobile Python. He has been a Django core team member since 2006 and is the co-founder and CTO of TradesCloud, which has led him to look into Python's role in the mobile space. His video was meant to provide an overview of the mobile development landscape, present the current state of Python for mobile devices, and to explain the decisions he believes the Python project needs to make in order to fully support mobile development.

From his perspective, the only sensible meaning for "mobile Python" is for it to mean "embedded Python"—the language bundled into a mobile app. There may someday be a market for the full Unix Python experience on an iPhone or other mobile device, but there isn't today. So by his definition, developers will be able to embed Python into a mobile app so that all or parts of the app can be written in the language.

In terms of mobile operating systems, there are quite few out there. His talk will be focusing on "the big two" (iOS and Android), he said. Not only are those two the bulk of the market, but they are also the platforms where most of the mobile Python work has been done so far.

For iOS, the toolchain is based on Clang. The binaries and libraries for iOS are "fat", as they actually contain the executables and object files for multiple architectures. Those binaries are created via multiple passes of the compiler, then merged together into a fat file. CPython can be relatively easily built as an embedded library in that environment. Native services on the platform can be accessed by bridging through the iOS Objective C runtime.

Android is quite different, with a runtime that is Java-based. You can access native services from C through the Java Native Interface (JNI), but there are some pretty serious limitations to that approach, especially with respect to reference counting (for garbage collection). It can be done, Keith-Magee said, but it is difficult. There are two possible approaches to supporting Python on Android: either using Jython-compiled bytecode or building the standard interpreter (i.e. CPython) as an embedded library. The latter is what has seen the most attention.

There are patches available for Python 3.4, either as patch sets or as meta-build scripts that grab the latest Python, patch it, and then build it. Kivy has gotten Python apps using its toolkit into both the iOS and Google app stores that way. The current focus of the mobile Python community is to get as many of those patches upstream as possible, he said.

To that end, there are issues in the Python bug tracker for adding support for iOS and for Android. In both cases, patches for Python 3.4.2 are available in the bug entries. The Python test suite is only partially working; just some simple smoke tests as well as tests to access data from C extensions using ctypes can be run. Both of these efforts are currently awaiting fixes to the ARM support in libffi.

The biggest outstanding technical issue at this point is the test suite and Buildbot integration, he said. The next step is to be able to build the test suite as an app, push it to the device, run it there, and retrieve the test results. Scripts need to be written to do that, but there is also a problem of hardware availability. Either someone needs to buy a bunch of devices and hook them up to a system somewhere or the community could engage a company that provides a service with many different mobile platforms for building and testing.

One decision that needs to be made by the Python core developers is how the platform identifies itself via the sys.platform variable. There are two schools of thought. One is that the platform should be "ios" or "android" as appropriate, while the other is that it should be "linux" for android (and presumably something else for iOS, but he didn't say). At least notionally, programs for other Linux distributions could run on Android, he said, which is why some advocate for the latter.

Building fat binaries/libraries is currently done with a meta-Makefile that runs configure and the standard Makefile multiple times, then merges the resulting targets into the fat files. That work could be moved into the standard Makefile instead, though there are still some difficulties because configure must be run each time for different architectures.

The current approach is to build all of CPython into a single library because Apple does not allow dynamic linking for iOS apps. That makes for large app file sizes, but that size can be cut down by only putting the modules needed by the app into the library. The patches for iOS provide a mechanism to do that, but it may be a desirable feature for more than just that platform, he said.

Apps will need to access native system services to do things like draw widgets on the screen. There are a number of different projects that allow Python to bridge to the native runtimes, but Keith-Magee wondered if some kind of bridging was in-scope for the standard library. If other services like geolocation or access to the accelerometer are to be provided by a mobile framework in the standard library, some form of bridging would be needed. After the video, several attendees seemed pretty adamant that those kinds of features were not at all in-scope for the standard library—at least in their minds.

There is also a question of what gets shipped. Apps are self-contained, so the project can't just ship a "Python app" that other apps use. For Android, the project could build a library for each platform; for iOS, a single fat library could be created. The problem is that apps are likely to want customized builds that are stripped down to only what that app needs. There is also the question of which CPU versions and platform versions will be supported by the project. For Android, the platform problem is particularly acute because there are so many different versions of Android running on devices. In fact, Android 2.3 ("Gingerbread") is shipping on brand new devices being made today, he said.

Third-party modules also raise questions. Pure Python modules are easy—just put the code somewhere on the PYTHONPATH—but C extensions raise some other issues. They will need to be built into the library somehow; will there be a standard mechanism to do so? Ethan Furman has volunteered to shepherd the Android patches moving forward. As far as he is aware, there is no one currently doing so for iOS, Keith-Magee said.

Many of the issues he raised could be deferred for a while, but there are a few that really need to be resolved before any forward progress can be made. Changes to the build system to support the mobile operating systems, as well as the platform identification issue, are the most pressing, he said.