Debian plans for Python 3

Moving a distribution to Python 3 is a big job, but there is a clear deadline for Python 2: it will be unsupported by upstream in 2020 (or at least that is the plan). That may seem far away, but it is only roughly two Debian release cycles from now. By the time Debian 10.0 ("Buster") is released, all of Debian's tools, and most of its packages, should be using Python 3 if all goes according to plan. Paul Tagliamonte and a "shadowy Debian Python cabal" met at PyCon 2015 to discuss this issue with an eye toward creating a plan of attack.

The rudiments of that plan were outlined by Tagliamonte after the conference. The first step, he said, is to look at things that are completely under Debian's control, which is its infrastructure. He asked those who are working on infrastructure projects to send him a census of the tools, their dependencies, and their Python versions, so that he could start building a tree of dependencies.

He also suggested filing bugs for packages that support Python 3 upstream, but are not built that way for Debian, as well as for packages that do not have Python 3 support at all. Beyond that, he suggested porting any projects that only support Python 2 to support both with a hybrid 2/3 code base. Finally, he will be forming a "Python 3 porting team" to help move the code in Debian's infrastructure to Python 2/3.

Those are ambitious plans, of course, and, as evidenced by the first census response, there are still wrinkles to be worked out. Enrico Zini listed half a dozen services that he maintains (e.g. contributors.debian.org, debtags.debian.net) along with their dependencies, which were mostly things like the Django web framework, database access libraries, and other web-focused Python packages. He noted that he is in favor of switching to Python 3, in principle, but there are some barriers to doing so:

Zini is willing to work with the porting team, but he wants to be sure that the team will continue its effort past the initial port. If incompatible changes come in three or four years as part of Python 3.7, say, he wants someone to be responsible for whatever forward porting is needed. It is clear that Zini feels that he has been burned by Python's changes in the past and is happy to see the stability offered by the feature-frozen 2.7 branch.

Tagliamonte acknowledged Zini's concerns with regard to the porting team's help now and in the future. He suggested that the team could focus on making both the dependencies and Zini's services run on both 2 and 3:

But Zini had some concerns about accepting patches that use the six compatibility library or, at least, that use it indiscriminately. He wants maintainable code moving forward:

In addition, adding Python 3 support adds to the testing burden, Zini said. There were several suggestions of tools (e.g. tox, nose2) that would help, but the fact remains that someone has to run the tests and fix any fallout. He would like to see that be handled by the porting team (or someone else) at least until 2019 or so, when Python 3 is ascendant and Python 2 largely gone.

Beyond the porting question, Zini also noted that there might be some problems running the services in production atop Python 3. Debian infrastructure runs on Debian stable, so any packages required to run the services would need to be available for stable as well. As Scott Kitterman pointed out, the fact that there will be two releases of Debian in the interim makes the production transition easier. There is a goal to have all of the dependencies available in Debian 9.0 ("Stretch"), so that the final switch for 10.0 will go smoothly.

That discussion should provide a bit of the flavor of the work ahead for the Python 3 porting team. It is clear that the dependencies are the key piece of the puzzle at this point, and that is where the focus is. Tagliamonte has created a new py3porters-devel mailing list to discuss plans and coordinate action going forward. It is a big task, but one that Debian appears to be getting a good head start on.