The PEP 572 endgame

Over the last few months, it became clear that the battle over PEP 572 would be consequential; its scale and vehemence was largely unprecedented in the history of Python. The announcement by Guido van Rossum that he was stepping down from his role as benevolent dictator for life (BDFL), due in part to that battle, underscored the importance of it. While the Python project charts its course in the wake of his resignation, it makes sense to catch up on where things stand with this contentious PEP that has now been accepted for Python 3.8.

We first looked at the discussion around PEP 572 back in March, when the second version of the PEP was posted to the python-ideas mailing list. The idea is to allow variable assignment inline, so that certain constructs can be written more easily. That way, an if or while, for example, could have a variable assignment in the statement and the value of the variable could be used elsewhere in the block (and, perhaps, beyond). The scope of those assignments is one of the areas that has evolved most since the PEP was first introduced.

Discussing PEPs

Even early on, some chunk of the discussion was really a "meta-discussion" about the medium for debating PEPs—whether a mailing list is truly better than, say, a forum of some kind. That debate somewhat foreshadowed later developments, as there are now various thoughts on changing the PEP-discussion process to make it so that people can follow the arguments and chime in with their own without having to commit to constant monitoring of the mailing list posts. As seen with this PEP, those posts are often redundant or repetitive, evince no familiarity with either the PEP or the preceding discussion, and generally are simply a waste of participants time—but, of course, not all of them are.

Discussions on python-ideas are meant to be more freewheeling but to eventually converge on something concrete and potentially agreeable before moving to python-dev. It has been hoped that the python-dev discussions will be more focused and less rancorous, but that clearly did not play out for PEP 572. Van Rossum brought up the "PEP 572 mess" in a session at the 2018 Python Language Summit; he wanted to explore other ways to discuss PEPs as part of that session. Shortly after that session, he posted an idea about better PEP discussions to the python-committers mailing list, where only core developers can post. He suggested moving PEP discussions to GitHub repositories (or those of another online Git provider):

There were some questions about the suitability of GitHub for PEP discussion, but most seemed inclined to try it and see how it worked out. Mark Shannon did try the idea out for PEP 576, but it did not really work, at least in that case, as it "didn't reduce the amount of email traffic on python-dev".

Meanwhile back at the PEP itself, much of the complexity was removed or filed down and the title changed from "Syntax for Statement-Local Name Bindings" to "Assignment Expressions". The underlying motivation was much the same, however. PEP 572 allows for assigning values to variables in places where only expressions are allowed. The ":=" operator was chosen (among a wide array of alternative spellings) to indicate that, which could be read as "becomes". One of the canonical examples of its use is to replace a so-called "loop and a half":

    line = f.readline()
    while line:
        ...  # process line
	line = f.readline()

    while True:
        line = f.readline()
	if not line:
	    break
	... # process line

    while line := f.readline():
        ... # process line

The main argument for the feature is that it is more readable and makes the programmer's intent clearer. But most of the traffic in the threads, from core developers and others, was against adding it. Even the original PEP author, Chris Angelico, was not wildly in favor ("hey, I'm no more than +0.5 on it myself"), at least in the early going.

While the PEP was still being discussed on python-ideas, Van Rossum got more involved and Tim Peters posted some of his own code that he thought could benefit from assignment expressions. As the "Rationale" section of the PEP notes, "toy" examples are not necessarily useful in trying to evaluate a language feature:

That section also describes some code that Van Rossum found in the Dropbox code base, where it was clear that programmers valued conciseness even to the point of repeating a potentially expensive operation to express it on a single line. In addition, an essay from Peters giving some real world examples is Appendix A of the PEP. While it was still not a foregone conclusion that Van Rossum would accept the PEP, his interest and participation seemed to point that way.

To python-dev

After "four rounds in the boxing ring at python-ideas", Angelico posted the PEP to the more widely read python-dev mailing list in mid-April. That set off a firestorm of replies, alternatives, a survey of how other languages handle this feature (if at all), appeals to PEP 20 ("The Zen of Python"), and so on. As Van Rossum said in his Language Summit talk, the opposition may have picked up once people realized he was seriously considering accepting the PEP.

Angelico seemed less confident of acceptance. In his second posting of the PEP to python-dev, he said: "So here's the PEP again, simplified. I'm fairly sure it's just going to be another on a growing list of rejected PEPs to my name". That posting predates much of the firestorm, however. As it turns out, he was a bit premature with his pessimism.

Many of the objections (those that were not bikeshedding over the syntax anyway) seem to revolve around the idea that PEP 572 would add a new way to assign to a variable that would be confusing to new programmers. Several core developers were concerned about how to teach the new operator and how to distinguish it from the "normal" assignments using =. Van Rossum admitted the benefits of the feature are "moderate", so it perhaps should not be a huge surprise that there was not wild acclaim for it. In fact, an informal poll of core developers right after the summit found few in favor of the change.

Another huge thread was spawned from Antoine Pitrou's thoughts on the LWN writeup of the summit session on PEP 572. It went on in much the same vein as many of the other discussions, though there are real efforts to improve the PEP in the thread instead of simply opposing it or rehashing syntax questions that had been long resolved. On July 2, however, Van Rossum posted his decision in the middle of the thread: "Thank you all. I will accept the PEP as is." As much as anything, it may have been his way of simply ending the interminable arguments to hopefully focus on tightening up the PEP's language and to fix any corner cases.

To a large extent, that's what most of the core developers still participating started doing. For example, Steve Dower started looking more closely at the "Syntax and Semantics" section of the PEP, with an eye toward clarifying the language (and his understanding). Victor Stinner started a thread to discuss possible places to use the feature in the standard library—with an eye toward which would actually be helpful to make the code clearer.

There were, naturally, a few last-gasp attempts to head off PEP 572, but Van Rossum was having none of that. He and Peters had joined with Angelico as authors of the PEP in mid-May and multiple changes were made, though they were not posted to the list until Van Rossum's more formal "intention to accept" was posted on July 9. In it, he asked that replies and changes stick to making the PEP better:

Acceptance ... and resignation

He then accepted the PEP on July 11; the next morning he posted his resignation as BDFL, saying: "Now that PEP 572 is done, I don't ever want to have to fight so hard for a PEP and find that so many people despise my decisions." Obviously the whole experience was painful and frustrating for him, which is truly sad. It is clear that many were loudly and sometimes annoyingly opposed to the feature, and some seriously unhappy that it was accepted, but "despise" seems rather strong—that sounds like frustration talking as much as anything else.

So Python will "soon" have assignment expressions, where soon means in Python 3.8, currently scheduled for October 2019. Whatever else can be said of the feature, it has been hashed out over many months, which likely helped eliminate any major issues with it. It is spelled a bit weirdly for Python, as the language has generally eschewed multi-character operators where it can, but that was a conscious choice. Van Rossum tried to suggest using = early on, which would be syntactically possible, but "the negative reaction to that version was way stronger". Even the biggest opponent of := can likely see uses for it even if they may avoid it on principle.

The consequences of the PEP and the battle are likely to be with the project for a long time to come. Obviously, losing Van Rossum as the final arbiter of all things Python is a big blow, but the PEP-discussion process is also going to change. It is truly the end of an era for the language, but it is also the start of a new era—Python seems likely to survive and probably thrive once it gets its feet back under it.