Toward a conclusion for Python dictionary "addition"
One of Guido van Rossum's last items of business as he finished his term on the inaugural steering council for Python was to review the Python Enhancement Proposal (PEP) that proposes a new update and union operators for dictionaries. He would still seem to be in favor of the idea, but it will be up to the newly elected steering council and whoever the council chooses as the PEP-deciding delegate (i.e. BDFL-Delegate). Van Rossum provided some feedback on the PEP and, inevitably, the question of how to spell the operator returned, but the path toward getting a decision on it is now pretty clear.
PEP 584 ("Add
+ and += operators to the built-in dict class
") has been in the
works since last March, but the idea has been around for a lot longer than
that. LWN covered a discussion back in
March 2015, though it had come up well before that as well. It is a
seemingly "obvious" language enhancement, at least for proponents, that
would simply create an operator for dictionaries to either update them
in-place or to easily create a combination of two dictionaries:
>>> d = {'spam': 1, 'eggs': 2, 'cheese': 3}
>>> e = {'cheese': 'cheddar', 'aardvark': 'Ethel'}
>>> d + e
{'spam': 1, 'eggs': 2, 'cheese': 'cheddar', 'aardvark': 'Ethel'}
>>> e + d
{'cheese': 3, 'aardvark': 'Ethel', 'spam': 1, 'eggs': 2}
>>> d += e
>>> d
{'spam': 1, 'eggs': 2, 'cheese': 'cheddar', 'aardvark': 'Ethel'}
As can be seen, the operation would not be commutative as the value for any shared keys will come from the second (i.e. right-hand) operand, which makes it order-dependent. There are some who do not see the operators as desirable features for the language, but the most vigorous discussion over the last year or so has been about its spelling, with a strong preference for using | and |= among participants in those threads—including Van Rossum.
At the beginning of December, Van Rossum posted his review of the PEP to the python-ideas mailing list. He encouraged the authors (Brandt Bucher and Steven D'Aprano) to request a BDFL-Delegate for the PEP from the steering council, noting that he would not be on the council after the end of the year. D'Aprano indicated that he would be doing so. Apparently that happened, because, tucked away in the notes from the November and December steering council meetings was a mention that a BDFL-Delegate had been assigned—none other than Van Rossum himself.
In his review, he comes down strongly in favor of | and
|= and had some other minor suggestions. He said: "All in
all I would recommend to the SC to go forward with this proposal, targeting
Python 3.9, assuming the operators are changed to | and |=, and the PEP is
brought more in line with the PEP editing guidelines from
PEP 1 and PEP 12.
" Given that, and that he is the
decision maker for the PEP, it would seem to be smooth sailing for its acceptance.
That did not stop some from voicing objections to the PEP as a whole or the spelling of
the operator in particular, of course, though the discussion was collegial
as is so often the case in the Python world. Van Rossum thought that
| might be harder for newcomers, but was not particularly
concerned about that: "I don't think beginners should be taught these
operators as a major tool in their toolbox
". But Ryan Gonzalez thought
that beginners might actually find that spelling easier because of its
congruence to the Python
set union operator.
Serhiy Storchaka is not a fan of the PEP in general, but believes that | is a better choice than +. He thinks there are already other ways to accomplish the same things that the operators would provide and that their use may be error-prone. He also had a performance concern, but Brett Cannon pointed out that it might only exist for CPython; PyPy and other Pythons might not have the same performance characteristics. Furthermore:
Marco Sulla made the argument that using | is illogical because sets also support a set difference operation using -, while the PEP does not propose that operator for dictionaries (though it should be noted that a previous incarnation of the PEP did have "subtraction", but it was not well-received and was dropped). Andrew Barnert felt that "illogical" was not the right reason to choose one spelling over the other:
Sulla continued
by saying that since list and string subtraction make no sense, that it is an unfair
comparison. But Chris Angelico pointed
out that's not necessarily the case either, since that operation does
make sense in some contexts. While he doesn't necessarily think Python
should add support for those use cases, "I do
ask people to be a little more respectful to the notion that these
operations are meaningful
". What followed was a bit of a digression
into mathematics and the meaning of various operations, much of which had
little to do with Python.
There were two offshoots of the discussion. "Random832" suggested
a generic way to add an operator specific to a module: all code in the
module could use the operator but it would not bubble out from there.
Cannon thought
it could be quite confusing to programmers who did not realize the operator
was redefined. "And debugging this wouldn't be fun either.
"
Storchaka brought up some
performance concerns, which could perhaps be worked around, but the general
reaction to Random832's idea was negative.
Jonathan Fine thought that since the proposed | operator gives preference to the right operand ("merge-right" in his terminology), there was a need for a merge-left operation. He called it gapfill(), which was a puzzling name choice to some; it would only add values for keys in the right-hand operand that were not present in the left-hand one. While the use case of, say, filling in defaults to a dictionary that held command-line options is reasonable, there are a number of other ways to do that (as is also true for |, however). Fine did not propose that an operator be added but did note that some other Python operations could be seen to give preference to the left-hand operand, which might make the merge-right | operator confusing. There was not a lot of reaction to the idea, but it doesn't look to be going anywhere for now.
D'Aprano plans to update the PEP based on the feedback from Van Rossum and others. It presumably also needs to run the gauntlet of the python-dev mailing list before Van Rossum can decide its fate. There is still plenty of time for all of that before the Python 3.9 release, even though the project adopted a 12-month release cycle a few months back. Python 3.9 is due in early October; it's a pretty good bet that | and |= for dictionaries will make the cut. Even if they do not, though, one of the goals was to put the subject to rest once and for all; a rejected PEP would serve as a place to point those who ask about dictionary "addition" in the future.
| Index entries for this article | |
|---|---|
| Python | Dictionaries |
| Python | Python Enhancement Proposals (PEP)/PEP 584 |