Postponing some feature removals in Python 3.9

Python 2 was officially "retired" on the last day of 2019, so no bugs will be fixed or changes made in that version of the language, at least by the core developers—distributions and others will continue for some time to come. But there are lots of Python projects that still support Python 2.7 and may not be ready for an immediate clean break. Some changes that were made for the upcoming Python 3.9 release (which is currently scheduled for October) are causing headaches because support for long-deprecated 2.7-compatibility features is being dropped. That led to a discussion on the python-dev mailing list about postponing those changes to give a bit more time to projects that want to drop Python 2.7 support soon, but not immediately.

There will actually be one final release of Python 2, Python 2.7.18, in April. It is something of a celebratory release that will be made in conjunction with PyCon. There were some fixes that accumulated in the branch between the 2.7.17 release in October and the end of the year, so those fixes will be flushed and the branch retired. Other than the release itself, no other changes will be allowed for that branch in 2020.

Compatibility

Fedora has recently started testing the Python 3.9 alpha releases in preparation for shipping 3.9 with Fedora 33, which is due around the end of the year. Victor Stinner and Miro Hrončok reported that a lot of packages will not build with the new version due to dropping the compatibility with Python 2.7. When that was posted on January 23, there were more than 150 packages broken by these changes, so they suggested that a handful of the ones causing the most problems be reverted.

As described in the message, many existing Python packages have code to handle differences between Python 2.7 and Python 3.x, but the changes currently in 3.9 might require another layer of compatibility code—one that may be short-lived. Instead of requiring that all of these other developers make those changes immediately, they said that it makes more sense for the core language to maintain the compatibility for one more release:

One of the examples provided is for the Sequence abstract base class (ABC) that can be used to create types similar to lists and tuples. In Python 2.7, it lives in the top level of the collections module, but for Python 3.3, the ABCs moved into collections.abc, though the top-level aliases were maintained. Those aliases are slated to be removed for 3.9, which will mean adding code like the following for projects that are still supporting the older version:

    try:
	from collections.abc import Sequence
    except ImportError:
	# Python 2.7 doesn't have collections.abc
	from collections import Sequence

If the developers of the third-party packages end up dropping support for 2.7 over the next year or so, then they will have needlessly added code like that when they could simply have put in the .abc wherever it was needed. While fixes are in progress for many of the projects, it will take time to get them out there, so Python 3.9 support could lag, which is probably not what the core developers want. Toward the bottom of the message, they list five incompatible changes (from the list in the in-progress Python 3.9 release notes); delaying those would eliminate most of the problems.

Reaction

The reaction was somewhat mixed. Steering council member Barry Warsaw was in favor of postponing those changes. Since Python has recently moved to a one-year cadence (down from 18 months), that shortened the time frame for projects to make these kinds of changes, he said. "And if it helps with the migration off of Python 2.7, then +1 from me."

Others were concerned that doing so would simply add another year to a longstanding plan to get rid of the compatibility cruft once 2.7 reached end-of-life. Ivan Levkivskyi thought that it was important to stick to the schedule that was established, but he also was not sure that delaying a year is truly beneficial: "For example, importing ABCs directly from collections was deprecated 8 years ago, what would 1 extra year change?" Eric V. Smith was also unsure about adding another year:

But Hrončok saw things differently:

Given that, Smith said that he was not opposed to a postponement as one-time thing to help those projects in the interim. He was guessing that deprecation warnings had been ignored from Python 3 in order to continue supporting 2.7, so a transition period is needed. It should be noted that deprecation warnings were not as prominent prior to the Python 3.7 release in 2018. Council member Brett Cannon concurred with postponing the deprecations:

Paul Moore was concerned that it will just give projects more time to procrastinate: "I think that far more people will see this as yet another delay before 2.7 dies, and treat it as one more reason to do nothing". He was not opposed to the specific changes suggested, however. But Serhiy Storchaka thought that postponing would just lead to problems when it came time to put together 3.10, 3.11, and beyond. Also, the deprecations will help surface code that is no longer maintained:

Guido van Rossum cautioned against that approach, though he admitted to making similar statements in the past:

Overall, the sense is that the specific reversions are reasonable, even if there are some concerns about delaying them for another release—thus sending the wrong signal to some projects. Stinner is also a steering council member, so it would seem there is a majority on that body in favor of pushing them back to 3.10, which will presumably come out in late 2021. It's hard to see things pushing out further than that, though, so projects should definitely take the opportunity to work out their strategy with regard to 2.7 compatibility.

Though Python 2.7 is "dead" in some sense, there are differing ideas of what that means in practice. There will certainly still be an enormous amount of Python 2 code running at the end of 2020; in fact, there will undoubtedly still be a lot of that code running at the end of the 2020s, but presumably much less at that point. The core development team has long been ready to leave that legacy behind, but smoothing the path for those who are not quite ready to come to terms with the demise of the Norwegian Blue ("beautiful plumage") is seemingly for the best.