Python dictionary "addition" and "subtraction"

A proposal to add a new dictionary operator for Python has spawned a PEP and two large threads on the python-ideas mailing list. To a certain extent, it is starting to look a bit like the "PEP 572 mess"; there are plenty of opinions on whether the feature should be implemented and how it should be spelled, for example. As yet, there has been no formal decision made on how the new steering council will be handling PEP pronouncements, though a review of open PEPs is the council's "highest priority". This PEP will presumably be added into the process; it is likely too late to be included in Python 3.8 even if it were accepted soon, so there is plenty of time to figure it all out before 3.9 is released sometime in 2021.

João Matos raised the idea in a post at the end of February. (That email is largely an HTML attachment, which is not getting archived cleanly, its contents can be seen here or quoted by others in the thread.) In that message, he suggested adding two new operators for dictionaries: "+" and "+=". They would be used to merge two dictionaries:

    tmp = dict_a + dict_b      # results in all keys/values from both dict_a and dict_b
    dict_a += dict_b           # same but done "in place"

    tmp = { **dict_a, **dict_b }     # like tmp = dict_a + dict_b
    dict_a = { **dict_a, **dict_b }  # like dict_a += dict_b

    tmp = dict_a.copy(); tmp.update(dict_b)  # like tmp = dict_a + dict_b
    dict_a.update(dict_b)                    # like dict_a += dict_b     

Matos's idea drew the attention of Guido van Rossum, who liked it. It is analogous to the "+=" operator for mutable sequence types, he said.

There were some questions about non-commutative "+" operators (i.e. where a+b is not the same as b+a), but several pointed out that there are a number of uses of the "+" operator in Python that are not commutative, including string concatenation (e.g. 'a'+'b' is not the same as 'b'+'a'). Steven D'Aprano suggested that instead of "addition", the operation should be treated like a set union, which uses the "|" operator. He had some other thoughts on new dictionary operators as well:

All of this hearkens back to a 2015 python-ideas thread that was summarized here at LWN. As a comment from Raymond Hettinger on a Python bug tracker issue notes, however, Van Rossum rejected the idea back in 2015. Some things have changed since then: for one, Van Rossum is no longer the final decision-maker on changes to the language (though his opinion still carries a fair amount of weight, of course), but he has also changed his mind on the feature.

Hettinger made it clear that he opposes adding a new "+" operator, both in another comment on the bug and in a python-ideas post. He does not see this operation as being "additive" in some sense, so using "+" does not seem like the right choice. In addition, there are already readable alternatives, including using the collections.ChainMap() class:

Several thought "|" made more sense than "+", but that was not universal. For the most part, any opposition to the idea is similar to Hettinger's: dictionary addition is unneeded and potentially confusing. It also violates "some of the unifying ideas about Python", Hettinger said.

Van Rossum suggested that D'Aprano write a PEP for a new dictionary operator. Van Rossum also thought the PEP should include dictionary subtraction for consideration, though that idea seemed to have even less support among participants in the thread. D'Aprano did create PEP 584 ("Add + and - operators to the built-in dict class"); that post set off another long thread.

As part of that, Jimmy Girardet asked what problem is being solved by adding a new operator, given that there are several alternatives that do the same thing. Stefan Behnel noted that "+" for lists and tuples, along with "|" for sets, provide a kind of basic expression for combining those types, but that dictionaries lack that, which is "a gap in the language". Adding such an operator would "enable the obvious syntax for something that should be obvious". Van Rossum has similar thoughts:

There is an implementation of the PEP available, but it is still seemingly a long way from being accepted—if it ever is. The PEP itself may need some updating. It would seem that the arguments for switching to "|" may be gaining ground. Early on, Van Rossum favored "+", but there have been some strong arguments in favor of switching. More recently, Van Rossum said that he is "warming up to '|' as well". Other than his arguments early on, there have been few others who were strongly advocating "+" over "|".

There have also been discussions of corner cases and misunderstandings that might arise from the operators. But the main objections seem to mostly fall, as they did in the PEP 572 "discussion", on the question of each person's "vision" for the language. It could certainly be argued that the PEP 572 assignment expression (i.e. using ":=" for assignments within statements) is a more radical change to the overall language, but many of the arguments against dictionary "addition" sound eerily familiar.

It is, as yet, unclear how PEPs will be decided; that will be up to the steering council. It may well be somewhat telling that none of the other four steering council members have been seriously involved in the discussion, but it is also the case that many may have tuned out python-ideas as a forum that is difficult to keep up with. Only time will tell what happens with PEP 584 (and the rest of the PEPs that are pending).

Counter({'a': [1], 'b': [1, 2]}) + Counter({'a': [2, 3], 'b': [3]})

c = Counter({'a': [1], 'b': [1, 2]})
c.update(Counter({'a': [2, 3], 'b': [3]}))