Python 3 in Fedora

At the 2016 Python Language Summit, Petr Viktorin, who is the team lead for the Python maintenance group at Red Hat, described the progress that Fedora has made in switching to Python 3 by default. He also presented some work that has been done to split up the standard library to try to reduce Python's footprint for cloud deployments.

Viktorin pointed to a site that is tracking Fedora's Python 3 porting efforts. In particular, he showed the history graph that displays the progress since October 2015. Some 1300 packages are now either able to run on both Python 2 and 3 or just on 3, though there are still 1700 or so to go.

There is a group of nearly 200 packages that are "mispackaged" (shown as blue in the graph). Those are packages that have been ported to Python 3 upstream, but are not packaged for Fedora. That is "the biggest problem we have right now", he said. The team is trying to address that using an automated system to grab the package from PyPI, turn it into an RPM, and then test it. Every other package that depends on it would then also be rebuilt and tested using the updated package.

The goal is to reach 50% of packages for Fedora 25, which is planned for release in November. There are various pieces that are still on Python 2, including some desktop toolkits. GTK+\ 2 support will not be ported to Python 3, which affects GIMP, Inkscape, and Sugar that all have Python-based plugins. For the enterprise, Samba is one of the biggest projects that remains based on Python 2. Viktorin said he would "probably end up porting Samba", which was met with applause.

Both the Mercurial and Bazaar revision-control systems remain on Python 2, as does a lot of Fedora infrastructure. Kushal Das spoke up to point out that the fedmsg message bus is Python 2, so "if you want on the bus, you have to be running Python 2". OpenStack is making progress toward Python 3 due to the efforts of Victor Stinner, while Twisted has likewise been making progress, thanks to the work done by Amber Brown.

The Red Hat team is small, Viktorin said, so the focus is on helping others to do the porting to Python 3. To that end, it has been working on some porting guides. There is the py3c library that will help porting C extensions to Python 3 if the C foreign function interface (CFFI) cannot be used, though he suggested that C extensions should be using CFFI. Py3c includes a porting guide. There is also an RPM porting guide available.

He then shifted gears to discuss the Python standard library. There is "lots of stuff" that isn't being used in the library and the Fedora cloud developers are saying that Python is too big to include in the cloud images. So Fedora is splitting up the library in support of its system-python effort.

The idea is that for essential system tools and cloud deployments there will be a Python available with a subset of the full standard library. There are some precedents, including Debian's python-minimal and Fedora's unbundling of some standard library packages, such as Tkinter, IDLE, and the test package (which is the largest package in the standard library by far).

Someone in the audience wondered about simply compressing the modules using ZIP compression. Viktorin said that was considered, as was simply shipping the bytecode cache files (i.e. .pyc). But doing so doesn't save much space.

The selection of standard library modules in system-python will likely change over time, Viktorin said. There may be value in finding a way for packages to declare what part of the standard library they need to use. It could be useful for MicroPython, he said; an attendee noted that it might be useful for Windows as well.