The return of None-aware operators for Python

The saga of the None-aware (or null-coalescing) operators for Python continues. We last looked in on the topic a little over a year ago and noted that either adoption or a clear rejection of the idea might help tamp down its regular recurrence. That has not happened, so, predictably, it was raised again—and does not look any closer to resolution this time around.

Back in October, Bora Savas posted a proposal for a "safe navigation operator" to the Python Ideas discussion forum. It would add an operator ("?.") that would only access an attribute if the object is not None:

    if obj is not None:
        print(obj.attr)

    # could (mostly) be replaced with

    print(obj?.attr)

Chris Markiewicz immediately pointed Savas at PEP 505 ("None-aware operators"), which proposes a more extensive set of operators. It has the "?." operator for attribute access, as Savas proposed, but also the "??" binary operator (along with an augmented-assignment version "??=") and an indexing operator ("?[]"). Their operation is fairly straightforward; our previous article, and another from 2018, have several examples, as does the PEP, of course.

Savas thought that the PEP was a good starting point for reviving the idea, but its status is "Deferred". Chris Angelico, who has long been an advocate for the feature, cautioned that further progress will require that the objections leading to the deferral be addressed. Normally, if a PEP is rejected, the reasons for that are described in it, but PEP 505 never reached that state, so it is lacking for concrete information about those objections. Sebastian Rittau said that made it difficult to proceed:

The PEP lists no objections, the commit changing the status to "Deferred" lists no reasons. The PEP doesn't even have a "Discussions-To" header. After five years it seems appropriate to start a new discussion, instead of digging through three years worth of mailing list archives. (The time between the creation of the PEP and the change to "Deferred".)

Jean Abou Samra pointed out that there have been several different discussions about the feature in the Python discussion forum, so one may not have to go back as far as the mailing-list archives. Savas suggested that the scope of the PEP might be part of the problem with it: having four separate operators may be making it too complex to adopt at once. One of the underlying problems, though, as Angelico (and others) pointed out, is that the PEP authors are not interested in pursuing the idea any longer; in fact, Steve Dower, one of those authors, is no longer even in favor of the idea.

From there, the thread took a fairly predictable path, with advocates extolling the virtues of the feature, while others complained about various aspects of it. Whether it is readable is obviously in the eye of the beholder and there are plenty of beholders of both types commenting. "Laurie O" had a somewhat different take, though, about the style of programming that the feature would enable based on her JavaScript experience:

I find code with more nulls flying around less safe, with types more fluid. After years of use of both languages, JS/TS feel less reliable to me due to this, whereas Python I have more confidence in the type (or shape, when I'm duck-typing).

Making nulls more ubiquitous means every value must be considered as possibly null unless explicitly guarded. I find Python's current idiom where shapes are stronger to be safer.

A construct like:

    a = b?.c?.d?.e

[...] though in my case it's the extra years of C# experience that have helped sway me (C# already had these operators for years before the Python or JS proposals, and the movement since then has been towards non-nullable types that can never be null).

Ensuring that functions always return correctly shaped objects (or raise if they cannot) is a better idiom. With ?., API designers will assume that their callers are happy to deal with the occasional None and will design lazy/slack APIs that return them instead of properly avoiding it.

There is a difference between Python and the languages that provide these operators, Beni Cherniavsky-Paskin said. Python raises exceptions, unless the programmer explicitly opts into using the APIs that can return a default (e.g. dict.get()):

It tends to exist in languages where common APIs/operations return such values a lot. JS obj.no_such_attr aka obj['no_such_attr'] fails by returning undefined; Ruby hash[no_such_key] fails by returning nil... Whereas Python raises exceptions — unless you specify explicit defaults to getattr() / dict.get(). For dictionaries, this pattern works great: (d or {}).get('e', {}).get('f', {})

Peter Lovett, who does a lot of Python teaching, pointed out that there are some real advantages to avoiding None in data structures. He agreed with Dower and said that "sometimes I've found the best solution is to stop the None's getting into the list (or whatever structure)", because it encourages better coding practices. He noted that Raymond Hettinger, who is a well-known Python instructor, offers similar advice: "It's easier to get the length of a list that doesn't have None's, than to find out how many non-None objects a list has."

But Angelico said that avoiding adding a feature because it can be misused is problematic; there are already lots of such features in the language. Instead, the feature should be judged on its benefits, as well. "I'm more interested in how something can be used well than in how it can be used badly."

There were, of course, ideas about alternate syntax or ways to accomplish the same goal within the existing language; along with fairly short discussions of them and their merits—or lack thereof. All of that is pretty standard fare for the Ideas forum. What did not appear, however, was any real interest in picking up the PEP and pushing it forward. That would require a core developer to step up as the sponsor of the PEP and either push it themselves or work with others to do so.

But these kinds of discussions are going to keep popping up periodically until either the feature gets added or it gets rejected. The limbo that PEP 505 (and more generally the None-aware operators feature) occupy simply leaves room to discuss it all again—and again. One gets the sense that the feature faces too much headwind to get added to Python, but having a steering council make that pronouncement and documenting the rejection in a PEP would be a kind of public service at this point. That could head off further discussions down the road—as could adding the feature, of course.