Python dictionary "addition" and "subtraction"

Posted Mar 13, 2019 21:35 UTC (Wed) by NYKevin (subscriber, #129325)
In reply to: Python dictionary "addition" and "subtraction" by iabervon
Parent article: Python dictionary "addition" and "subtraction"

Indeed, the problem is that:

1. Single pipe is traditionally commutative (bitwise OR, set union, etc.). It has been used non-commutatively (shell pipelines), but that usage is so far afield that it provides no intuition here.
2. Double pipe (or the "or" keyword) traditionally short-circuits to the left. I've never seen it short-circuit to the right.
3. Plus is sometimes commutative, but its noncommutative uses traditionally preserve the order of items (concatenation) rather than allowing one item to "override" another. So the closest analogue here would be c = ChainMap(a, b) (so that you have c.maps == [a, b]) as Hettinger suggests... but that actually gives precedence to the left! ChainMap.new_child() does give precedence to the "right" in a sense, but it's type-asymmetric (self is a ChainMap, the argument usually isn't), and probably should not be an operator at all.

Python dictionary "addition" and "subtraction"

Posted Mar 13, 2019 22:11 UTC (Wed) by iabervon (subscriber, #722) [Link] (6 responses)

For "a+b", I was thinking that "a+=b" seems like it should do something for every element of b, and what "a.update(b)" does feels like what "a+=b" would most likely do. Last-wins also matches what would happen if you just made a new dict and set all the items from a and b in the order they appear, so it feels right for concatenation.

I'd sort of guess that, if you've got an "or"-like thing, even if it doesn't short-circuit (i.e., evaluates all of its arguments), it'll pick between distinguishable values with the same truth value as if it were short-circuiting.

Python dictionary "addition" and "subtraction"

Posted Mar 14, 2019 15:41 UTC (Thu) by nybble41 (subscriber, #55106) [Link] (5 responses)

My intuition for "a + b", where "a" and "b" are both dictionaries, would be that keys which exist in both "a" and "b" have their values combined recursively with the same "+" operator.

>>> { 'apple': 5, 'pear': 7 } + { 'strawberry': 2, 'pear': 10 }
{ 'apple': 5, 'pear': 17, 'strawberry': 2 }
>>> { 'x': 'abc', 'z': { 'a': 13 } } + { 'x': 'def', 'y': 'ijk', 'z': { 'a': 9, 'b': 10 } }
{ 'x': 'abcdef', 'y': 'ijk', 'z': { 'a': 22, 'b': 10 } }

This would make it similar to the Semigroup operator (<>) in Haskell.

Python dictionary "addition" and "subtraction"

Posted Mar 14, 2019 17:17 UTC (Thu) by NYKevin (subscriber, #129325) [Link] (4 responses)

That already exists, it's called collections.Counter.

Python dictionary "addition" and "subtraction"

Posted Mar 15, 2019 11:29 UTC (Fri) by jani (subscriber, #74547) [Link] (3 responses)

Doesn't that require the dict values to be integers? The proposal above only expects them to have + and - operators.

Python dictionary "addition" and "subtraction"

Posted Mar 15, 2019 23:56 UTC (Fri) by quietbritishjim (subscriber, #114117) [Link] (2 responses)

The values in Counter don't need to be integers, or even numbers. Both the constructor and the update method can take either iterables or mappings, and the iterable versions put integer counts in while the mapping versions just directly use the values (with the + operator, in the case of update). There's a note in the documentation [1] (scroll down to grey box) saying the implementation of each method makes minimal assumptions on the value type.

[1] https://docs.python.org/3/library/collections.html#collec...

Python dictionary "addition" and "subtraction"

Posted Mar 17, 2019 11:35 UTC (Sun) by jani (subscriber, #74547) [Link] (1 responses)

Given two dictionaries mapping string keys to list values, how do you actually use Counter to concatenate the lists (list + operation) for keys that exist in both dictionaries?

Python dictionary "addition" and "subtraction"

Posted Mar 20, 2019 20:48 UTC (Wed) by quietbritishjim (subscriber, #114117) [Link]

Hmm, it turns out it doesn't work. The obvious thing is to use +:

Counter({'a': [1], 'b': [1, 2]}) + Counter({'a': [2, 3], 'b': [3]})

This fails for two reasons:

The + operator is guaranteed to include only positive results even if summing the values is negative, so it includes the comparison <0 for each element, which fails for lists.
If the set of keys is different on the two sides then it does an addition with 0 (this only applies to keys missing in the left hand list, but that is presumably an implementation detail).

An alternative is to use the update() method, which doesn't have a restriction to positive values so doesn't compare against zero:

c = Counter({'a': [1], 'b': [1, 2]})
c.update(Counter({'a': [2, 3], 'b': [3]}))

But this doesn't work either:

As with the + operator, missing keys are interpreted as having value 0.
The values are passed to + in the opposite order than you would expect, so the above example results in c['a'] == [2,3,1]! This is true in Python 3.7 but not Python 2.7.

I think this is a bug in the documentation, which seems to say that these should be possible (at least for update()), or even a straight up bug in the code. But in fairness it is an unusual use of the class.