Python attributes, __slots__, and API design

A discussion on the python-ideas mailing list touched on a number of interesting topics, from the problems with misspelled attribute names through the design of security-sensitive interfaces and to the use of the __slots__ attribute of objects. The latter may not be all that well-known (or well-documented), but could potentially fix the problem at hand, though not in a backward-compatible way. The conversation revolves around the ssl module in the standard library, which has been targeted for upgrades, more than once, over the years—with luck, the maintainers may find time for some upgrades relatively soon.

Thomas Grainger posted about a problem he encountered when setting the minimum TLS version to use for a particular SSLContext using the following code:

    context.miunimum_version = ssl.TLSVersion.TLSv1_3

It is, of course, no surprise that Python happily accepts the attribute name, even though it is "wrong". In a dynamic language, there is nothing inherently wrong with setting an attribute with an arbitrary name, but this case is a little different. For one thing, SSLContext is, obviously, a security-sensitive object, so an API that requires setting attributes—correctly spelled—may be less than ideal.

One way to potentially fix the problem is by using the __slots__ class variable for the SSLContext, as Jonathan Fine pointed out. A Python class that has a __slots__ entry is restricted to attribute names that are listed in the class variable:

    >>> class Foo:
    ...     __slots__ = ('bar', 'baz')
    ... 
    >>> x = Foo()
    >>> x.bar = 3
    >>> x.qux = 9
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: 'Foo' object has no attribute 'qux'

Fine also noted that the use of __slots__ is not well-documented; he laid down a challenge, in fact: "Find a page in docs.python.org that describes clearly, with helpful examples, when and how to use __slots__." One of the links he suggested, the "Descriptor HowTo Guide", does have a section with some concrete examples on how __slots__ could be used, though it seemingly falls short of what he was looking for.

Under the covers, __slots__ shorts out the normal per-instance dictionary for attributes and replaces it with a fixed-sized array, which saves some memory and speeds up attribute lookup; it will also catch the kind of pesky spelling error that Grainger ran into by raising an AttributeError at run time. Fine pointed to the warning in red in the ssl description, which strongly suggests reading the "Security Considerations" section of the module documentation. "Given that context is important for security, perhaps it's worthwhile closing the door to spelling errors creating security holes."

"Bluenix" also suggested __slots__, but as Guido van Rossum said, that is not a backward-compatible solution. There could be "code out there that for some reason adds private attributes to an SSLContext instance, and using __slots__ would break such usage." Even if it were determined that a compatibility break was in order here, __slots__ could not be used for other reasons, as Christian Heimes, one of the ssl maintainers, pointed out:

Also __slots__ won't work. The class has class attributes that can be modified in instances. You cannot have attributes that are both class and instance attributes with __slots__. We'd have to overwrite __setattr__() and block unknown attributes of exact instances of ssl.SSLContext.

There are two class attributes, sslsocket_class and sslobject_class, that are specifically mentioned as being settable in an instance, but __slots__ does not work with attributes that can switch between class and instance attributes. Eric V. Smith thought that making a specific fix for SSLContext was not the right approach, though:

Isn't this a general "problem" in python, that's always been present? Why are we trying to address the problem with this specific object? I suggest doing nothing, or else thinking big and solve the general problem, if in fact it needs solving.

But the problem at hand is "that assigning attributes is a bad API", Oscar Benjamin said. He suggested adding a new class with a better interface as a backward-compatible way forward. Others agreed that because of the security-sensitive nature of SSLContext, it deserves "special" treatment; the general "problem" of misspelled attributes is not seen as something that needs to be addressed in the language, however.

Grainger is advocating using a frozen dataclass for SSLContext, though that would also break backward compatibility. Python dataclasses were added for Python 3.7 (in 2018) as a way to represent a collection of data items as the attributes on a object, similar to a C struct. A frozen dataclass gets initialized with a set of values that cannot be changed by setting the attribute directly; Grainger suggested having explicit methods to change the attributes of an SSLContext.

Most who commented in the thread seemed to agree that there is a problem to be solved; Marc-Andre Lemburg put it this way:

IMO, a security relevant API should not use direct attribute access for adjusting important parameters. Those should always be done using functions or method calls which apply extra sanity checks and highlight issues in [the] form of exceptions.

Steven D'Aprano thought that a compatibility break might be in order to try to resolve a "mildly troubling security flaw/bug/vulnerability" Others were less sure of that, though. If there is to be a compatibility break, it should create "a cleaner, more Pythonic API", Brendan Barnwell said. He had some suggestions for how that might look:

Why not have the class accept only valid options at creation time and raise an error if any unexpected arguments are passed? Is there even any reason to allow changing the SSLContext parameters after creation, or could we just freeze them on instance creation and make people create a separate context if they want a different configuration? I think any of these would be better than the current setup that expects people to adjust the options by manually setting attributes one by one after instance creation.

Heimes said that there will not be any incompatible changes made to SSLContext in the near future, however. If time is found to work on this problem for Python 3.11, changes along the lines of the configuration object in PEP 543 ("A Unified TLS API for Python") would be made. We looked at the PEP in early 2017, but it was withdrawn in mid-2020, "due to changes in the APIs of the underlying operating systems". There are still pieces of the PEP that could be used to address the problem that Grainger encountered.

The ssl module has always been a thin layer atop OpenSSL, which has undergone a number of API (and other) changes over the years. Support for TLS in the Python standard library has changed as well; up until Python 3.4 in 2014, TLS certificates were not able to be checked for validity using it at all, for example. The ssl module has seemingly always had a lack of available developer time, which is rather worrisome for a critical piece of security infrastructure. Hopefully some time can be found to at least resolve problems like this that can be caused by a simple misspelling.