|
|
Subscribe / Log in / New account

Debating a "transitional" Python 2.8

By Jake Edge
January 1, 2014

Python 3 has been around for five years now, but adoption of the language still languishes—at least according to some. It is a common problem for projects that make non-backward-compatible changes, but it is even more difficult for programming languages. There is, typically, a huge body of installed code and, for Python, libraries that need to be upgraded as well. If the new features aren't compelling enough, and the previous version is already quite capable, it can lead to slow adoption of the new version—and frustration for some of its developers. Solving that problem for Python 3 is on the minds of some.

Recently, Alex Gaynor, who has worked on core Python and PyPy, expressed his concerns that Python 3 would never take off, partly because the benefits it brings are not significant enough to cause projects and users to switch over to Python 3. After five years, few are downloading packages for Python 3 and developers aren't moving to it either:

Looking at download statistics for the Python Package Index, we can see that Python 3 represents under 2% of package downloads. Worse still, almost no code is written for Python 3. As I said all of my new code supports Python 3, but I run it locally with Python 2, I test it locally with Python 2; Travis CI [continuous integration] runs it under Python 3 for me; certainly none of my code is Python 3 only. At companies with large Python code bases I talk to no one is writing Python 3 code, and basically none of them are thinking about migrating their codebases to Python 3.

That leads to language stagnation at some level, Gaynor said. Everyone is still using Python 2, so none of the features in 3.x are getting exercised. Since Python 2 is feature frozen, all of the new features go into versions that few are using. And it is mostly Python-language developers (or others closely related to the project) who are using the new features; the majority of "real" users are not, so the feedback on those features may be distorted:

The fact that Python 3 is being used exclusively by very early adopters means that what little feedback happens on new features comes from users who may not be totally representative of the broader community. And as we get farther and farther in the 3.X series it gets worse and worse. Now we're building features on top of other features and at no level have they been subjected to actual wide usage.

Gaynor concluded that the divergence of Python 2 and 3 has been bad for the community. He suggested releasing a Python 2.8 that backported all of the features from Python 3 and emitted warnings for constructs that would not work in Python 3. That, he said, would give users a path for moving to Python 3—and a way to start exercising its new features.

As might be guessed, not all were in favor of his plan, but the frustration over the Python 3 adoption rate seems fairly widespread—at least with the commenters on Gaynor's blog. So far, the conversation has not spilled over to python-ideas or some other mailing list.

Alan Franzoni agreed that some kind of transition path, rather than an abrupt jump, is needed.

We badly need smooth transitions, a version where features are deprecated and replacements exist at the same time, so we can progressively improve our libraries. Very few developers are willing to invest their time and resources in porting code to a not-so-used language version, and just one non-py3k-[compatible] library is enough for a project for not migrating to Python 3. It's a chicken-and-egg problem.

David A. Wheeler and others agreed with Franzoni, but "rwdim" felt that Gaynor did not go far enough:

The solution is clear: build a full compatibility layer, not a conversion tool, into 3.x that makes 2.x code run without change and ABRUPTLY stop development on 2.x. Keep the compats backward compatible for 2 releases and then deprecate items that need to die, forcing people to update.

Burn the ship and force people to move ashore or go away.

In addition, rwdim suggested that the Python package index (PyPI) stop serving packages that are not Python 3 compatible at some point. That last suggestion was not well received by PyPI maintainers and others, but it did attract other semi-belligerent comments. For example, "jgmitzen" likens users sticking with 2.x to terrorists (and to the Tea Party in another comment). While perhaps a bit overwrought, jgmitzen's point is that supporting 2.x in the Python ecosystem is taking time and energy away from 3.x—to the detriment of the language.

But "gsnedders" is not sure that a 2.8 really brings anything to the table. In the libraries that gsnedders maintains, things have gotten to the point where a single code base can support both >=2.6 and 3.x, and that should be true for most projects. The more recent feature additions for Python 3 are in the standard library, which means they are available for 2.x via PyPI.

Like rwdim, Sean Jensen-Grey would like to see an evolutionary approach so that a single interpreter can be used with both Python 2 and 3. In another comment, he referenced a March 2012 blog post from Aaron Swartz that outlines his vision of how the Python 3 transition should have worked. It followed the established pattern of adding new features to Python 2.x, which is clearly an evolutionary approach.

But Python 3 set out with a non-evolutionary approach. Python Enhancement Proposal (PEP) 3000 clearly specified a break with 2.x backward compatibility. The question seems to be: is it time to rethink that strategy in light of the slow adoption for Python 3?

It may simply be a matter of time, too. Linux distributions are starting to plan for shipping Python 3 as the default—some already have made the switch. Those kinds of changes can only help spur adoption, though it may still take a while.

In addition, Some don't seem convinced that Python 3 adoption is lagging, or at least that it is lagging as badly as is sometimes portrayed. To start to answer that question, Dan Stromberg has put together a survey on Python 2.x/3.x use. Whatever the outcome of that, though, it seems likely that many core developers are less than completely pleased with where Python 3 uptake is today—and will be looking for ways to improve it.



to post comments

Debating a "transitional" Python 2.8

Posted Jan 2, 2014 4:13 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

>While perhaps a bit overwrought, jgmitzen's point is that supporting 2.x in the Python ecosystem is taking time and energy away from 3.x—to the detriment of the language.
Well, so does maintaining strict backwards-compatible ABI in Linux.

Python developers should ask themselves what's more important to them: designing a new language or doing something that is useful for the developer community?

Stable ABI

Posted Jan 2, 2014 14:40 UTC (Thu) by vstinner (subscriber, #42675) [Link] (1 responses)

> Well, so does maintaining strict backwards-compatible ABI in Linux.

"Since Python 3.2, a subset of the API has been declared to guarantee a stable ABI."
http://docs.python.org/3/c-api/stable.html

Before Python 3.2, no effort was put to maintain the stability of the ABI.

With Python 3.3, there is no more narrow or wide build for Unicode strings, but I don't know the impact on the ABI.

Stable ABI

Posted Jan 2, 2014 14:46 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]

I'm talking about trying hard not to break end-users' programs. Python hadn't tried to do that, Python designers gleefully went on and broke compatibility. Is it a wonder that most developers simply decided to wait and see?

Debating a "transitional" Python 2.8

Posted Jan 2, 2014 6:40 UTC (Thu) by dlang (guest, #313) [Link]

trying to do a survey of python version usage will only get responses from the early adopter crowd that is paying attention to such things.

There will be no feedback from the people who have python code they are running that is 2.x only because they don't know or care about python 3

most people who use a language are not involved with discussions about changing it (or if they are, the language is either very new or breaking a lot)

Getting to where one binary can run both python 2 and python 3 code would be a very good starting point. After that the debate can begin about disabling features. That will be an interesting debate, as most of the time the people trying to turn them off are justifying doing so 'for their own good', which won't make people who have code that gets broken very happy.

Debating a "transitional" Python 2.8

Posted Jan 2, 2014 7:50 UTC (Thu) by osma (subscriber, #6912) [Link] (1 responses)

Having one binary support both 2.x and 3.x code sounds good, but given the deliberately backwards-incompatible features of 3.x (especially the new, saner string/unicode handling), is it really feasible?

I mean, of course you can have something like a wrapper that detects the code flavor and selects the proper interpreter internally, but I think the interesting scenario would be having support for mixed 2.x/3.x code bases where only part of the code is ported / written to 3.x.

Debating a "transitional" Python 2.8

Posted Jan 2, 2014 23:38 UTC (Thu) by david.a.wheeler (subscriber, #72896) [Link]

Of course it's feasible. Python has been doing it a long time, and the changes for Python3 aren't really that monumental.

Why should OpenStack move to Python 3 right now?

Posted Jan 2, 2014 14:10 UTC (Thu) by vstinner (subscriber, #42675) [Link]

I wrote recently an article titled "Why should OpenStack move to Python 3 right now?":
http://techs.enovance.com/6521/openstack_python3

You may be interested to read it because it is somehow related to the discussion:

Debating a "transitional" Python 2.8

Posted Jan 2, 2014 19:57 UTC (Thu) by iabervon (subscriber, #722) [Link] (2 responses)

I kind of like the idea of a "Python 2.8" which is produced from a Python 3 source tree, is compatible with 2.7, has 2.x semantics (unmarked string constants are bytes, not unicode), but insists on more portable usage.

For example, it could still recognize the print statement by default, but consider it deprecated or illegal. If you want to use "print" you need "from __future__ import print_function;" and to use it as a function, but it also doesn't just start handling things the new way. Likewise, it understands ambiguous "except" clauses the 2.x way, not the 3.x way, but it prohibits it.

Personally, I think the biggest impediment to Python 3 adoption is that it lacks support for "unicode" and the "u'foo'" syntax. Other than that, it would be possible to have a 2.8 that makes you write version-agnostic code, but there's no way to get Python-3 strings that also works in 2.7 for 2.8 to demand.

Debating a "transitional" Python 2.8

Posted Jan 4, 2014 23:15 UTC (Sat) by juliank (guest, #45896) [Link] (1 responses)

u"foo" is back in Python 3.3. And b"foo" is supported in Python 2.7, so you can just use b"foo" and u"foo" in code and it will work in both versions.

Debating a "transitional" Python 2.8

Posted Jan 4, 2014 23:47 UTC (Sat) by iabervon (subscriber, #722) [Link]

Excellent! I'd only found a 3.0-era comparison between 2.x and 3.x, and tested on 3.2.

Debating a "transitional" Python 2.8

Posted Jan 3, 2014 2:22 UTC (Fri) by therealmik (guest, #87720) [Link] (1 responses)

I've been writing mostly python 3 code for the past year or so, and generally like it. The problem is that sometimes I need to run my code on a RHEL system. Perhaps poking the last holdout distros and asking them to ship python3 alongside (as an optional package) might be enough.

Adding some more __future__ statements might help (eg. a proper bytes class and the new exception syntax) - once I've written python3 code it's really hard to backport.

Python 3 on RHEL

Posted Jan 7, 2014 1:59 UTC (Tue) by dave_malcolm (subscriber, #15013) [Link]

FWIW, Red Hat ships such an optional Python 3.3 for RHEL, as part of the "Software Collections" product. See http://developerblog.redhat.com/2013/09/12/rhscl1-ga/

Myopic view of Python ecosystem

Posted Jan 6, 2014 21:51 UTC (Mon) by michaelkjohnson (subscriber, #41438) [Link]

I posted about this on G+ as part of this discussion. Recapping and expanding here: The Python development community made it unnecessarily difficult to migrate from Python 2 to Python 3 by creating incompatibilities without an ecosystem-aware migration strategy. They did this intentionally; see PEP 3000. So by design, we have in fact two languages, separated by a common heritage. But the language developers then try to use persuasion to convince us to join them in a pretense that there is really only one language. When, years after a new language version release, the language development community continues to have to harangue people to try to convince them to move, they should reconsider the source of the problem.

This downplays the complexity of the Python ecosystem. Developers aren't always in complete control of the process of adapting syntax. For example, when Python is adopted as an internal "scripting" language, the developer (of the container) doesn't have control over the user-provided scripts, and thus moving to an incompatible version is a cost to the end user that doesn't bring benefits to that user. This is hard to coordinate!

The implicit assumption that Python is only an implementation language undervalues embedded python. I remember early Python documentation specifically calling out Python's strengths as an embedded interpreted language. Those burned by being ignored are likely to be looking toward other language families in the future. I have nothing against (for example) Lua, but I know Python better, and perhaps it is selfish for me to wish that Python were being managed with better care for this use case, and to regret that this language I enjoy so much will be used less as the result.

A transitional Python 2.8, especially if it is combined with reconsidering the things that were (from the point of view of a user) broken just for the sake of doing things differently and not for any actual advantage, could be a benefit for this ecosystem.


Copyright © 2014, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds