Extending Python's enums

Enumerated types or "enums" are a feature of many languages, including Python; enums provide a convenient way to collect up a bunch of related symbols that (typically) evaluate to integer values. The canonical example would seem to be for colors, at least for demonstration purposes, but there are others, especially for handling "magic" constants from source likes POSIX or the host operating system. A recent thread on the python-ideas mailing list discusses different ways to add a new feature to enums—seven years after they were added to the standard library as part of Python 3.4.

Background

We covered some of the discussion around PEP 435 ("Adding an Enum type to the Python standard library") back in 2013 when it was approved. For those who are unfamiliar with the enum feature, a mini-introduction may be in order; following the documentation for the enum module, the examples will use upper case for the names, though it is not required (and the PEP does not).

    >>> from enum import Enum
    >>> class Color(Enum):
    ...     RED = 1
    ...     GREEN = 2
    ...     BLUE = 3
    ... 
    >>> Color.GREEN
    <Color.GREEN: 2>
    >>> Color.GREEN == 2
    False
    >>> Color.GREEN.value == 2
    True

    >>> Color(3)
    <Color.BLUE: 3>
    >>> Color.RED != Color.BLUE
    True
    >>> print(Color['RED'])
    Color.RED

In the vast majority of use-cases, one doesn't care what the actual value of an enumeration is. But if the value is important, enumerations can have arbitrary values.

Beyond that, enums can be iterated over in the order the members were defined and enum members are hashable, so they can be used as keys in dictionaries. While each member name must be unique, values do not have to be, so aliases are allowed:

    >>> class Color(Enum):
    ...     RED = 1
    ...     GREEN = 2
    ...     BLUE = 3
    ...     ROJO = 1
    ...     VERDE = 2
    ...     AZUL = 3
    ... 
    >>> Color.AZUL
    <Color.BLUE: 3>
    >>> Color.VERDE is Color.GREEN
    True

    >>> for i in Color:
    ...     print(i)
    ... 
    Color.RED
    Color.GREEN
    Color.BLUE

    >>> Color.__members__
    mappingproxy({'RED': <Color.RED: 1>, 'GREEN': <Color.GREEN: 2>, 'BLUE': <Color.BLUE: 3>,
    'ROJO': <Color.RED: 1>, 'VERDE': <Color.GREEN: 2>, 'AZUL': <Color.BLUE: 3>})

Testing for an enum member

Ethan Furman, who is one of the authors of the PEP and maintainers of the enum module, posted a question about a common Stack Overflow query on March 12: is there a way, other than using try and except, to determine whether a given value is a member of an enum? There was a way to do so in an earlier version of Python, he said, but that was fixed as a bug along the way. He suggested adding a get() to Enum with a default to be returned if the value is not found, similar to dict.get(), adding a recipe to the documentation for enum, or to "do nothing".

Since it was a python-ideas thread, "do nothing" was probably not going to win out, though Guido van Rossum acknowledged it as a possibility. He also said that get() did not make sense and had an alternative to suggest:

But the way to convert a raw value to an enum value is Color(1), not Color[1], so Color.get(1) seems inconsistent.

Maybe you can just change the constructor so you can spell this as Color(1, default=None) (and then check whether that's None)?

Furman agreed that get() was not the right interface and liked the idea of changing the constructor, though he wanted a slightly different approach:

I think I like your constructor change idea, with a small twist:

    Color(value=<sentinel>, name=<sentinel>, default=<sentinal>)

This would make it possible to search for an enum by value or by name, and also specify a default return value (raising an exception if the default is not set and a member cannot be found).

In that message, Furman also answered Van Rossum's question about the earlier removal of a way that worked to determine if a value was a member of an enum:

    if 1 in Color:
       ...

But Van Rossum also wondered about a corner case: "I'm not sure what Color(value=1, name='RED') would do -- insist that both value and name match? Would that have a use case?" Furman said that he did not really see a good use case, but that requiring both to match seemed like the right approach. He also noted that in some strange cases, names and values could overlap, "although having the name of one member match the value of a different member seems odd". He seemed inclined to ignore that possibility and concluded:

Everything considered, I think I like allowing `__contains__` to verify both names and values, adding `default=<sentinel>` to the constructor for the value-based "gimme an Enum or None" case, and recommending `getattr` for the name-based "gimme an Enum or None" case.

With an explicit "+1" from Van Rossum, it seemed like this relatively minor detail for the enum module had been resolved. But perhaps both Furman and Van Rossum got caught up in the discussion and did not fully consider the nature of a class constructor returning None (or whatever other random object was given as the default), rather than an object of the type being constructed. That did not strike Matt Wozniski as particularly Pythonic:

I find the idea of having the constructor potentially return something other than an instance of the class to be very... off-putting. Maybe it's the best option, but my first impression of it isn't favorable, and I can't think of any similar case that exists in the stdlib today off the top of my head. It seems like we should be able to do better.

He suggested adding from_value() and from_name() class methods, each of which would take an optional default value as a better approach. Van Rossum agreed that the constructor idea was not the right one ("and as static typing proponent I should have thought of that"), but he thought there might not really be a need for an arbitrary default, so changing __contains__() might be best. "The ergonomics of that seem better for the dominant use case ('is this a valid value for that enum?')."

There was some discussion of whether aliases (i.e. multiple names with the same value) would cause a problem for "by value" lookups, but Furman made it clear that there actually is no ambiguity. Values are associated with the first name that uses them, which is why "Color.AZUL" displays as "Color.BLUE" in the example above.

Furman has not circled back with a final decision on his plans, but it seems clear that the constructor-based idea is a non-starter even though it seemed like the right way to go from early on. Overall, the short thread gave a nice little peek into the kinds of thinking that go on in language and library design. Sometimes, the obvious solution is really not quite right, even when it seems to tick all of the right boxes. And, once again, this kind of discussion shows the value of FOSS communities openly working to arrive at a better solution; while it is not perfect by any means, the results generally speak for themselves.

if x in Colour.names():
    ...

if x in Colour.values():
    ...

(x.value for x in Color.__members__.values())

x.value