|
|
Subscribe / Log in / New account

Development

Opposition to Python type hints

By Jake Edge
May 6, 2015

Type hints for Python seem to be a feature that is on the fast track. These optional hints provide information that would allow static type-checking. But a recent discussion on the python-dev mailing list shows that not everyone is on-board with the idea of adding the annotations inline—they would rather see the hints be "hidden" in separate files. Others seem opposed to the idea of type hints at all, fearing that the feature will change the character of the language entirely.

We first looked at the type hints back in December, though the discussion started in August 2014. The feature has been championed by Python's benevolent dictator for life (BDFL), Guido van Rossum, who also described the feature in a Python Language Summit session (and PyCon keynote) in April. It is targeted for the Python 3.5 release that is slated for September. The idea of static typing for Python goes back much further, however, at least as far as a December 2004 blog post from Van Rossum.

Self-described "mediocre programmer" Harry Percival posted his reaction to the type hints. He argued that type hints are "ugly" and will make the language harder to understand, especially for beginners. He didn't buy the argument that the type annotations for functions will truly be optional, suggesting that over time there will be a strong uptake of the feature. He didn't suggest doing away with type hints entirely, just that there be a recommendation to put those annotations into stub files.

The idea of stub files is already part of the type hints proposal, largely to support C extensions where there may be no Python code to annotate. There are other situations where they will be needed, including for modules that must support both Python 2 and 3. The syntax used by the annotations is only available in Python 3. Stub files have the same syntax as regular Python files—just containing function signatures with "pass" for the function body—but use a .pyi extension

There are some problems with always using stub files, however. For one thing, the way things are currently implemented, the stub files for a module would only be consulted for code that uses that module. The module itself would not have the benefit of the annotations, so its code could not be type-checked. Also, the annotations from stubs are not accessible for introspection from within Python code as inline annotations are.

Percival's post garnered a fair amount of support. To many, type hints are not particularly "Pythonic" and they will make Python code harder to read. By far the strongest condemnation came from Jack Diederich (who is evidently still smarting from a suggestion Van Rossum made in 2003). Some of his complaints are based on misunderstandings of the plans moving forward, but those misconceptions are shared by a fair number of others in what turned into a monster thread. But the core of his complaint can be found at the end of the message:

If this wasn't in a PEP and it wasn't going to ship in the stdlib very few people would use it. If you told everyone they had to install a different python implementation they wouldn't. This is much worse than that - it is Python4 hidden away inside a PEP.

There are many fine languages that have sophisticated type systems. And many bondage & discipline languages that make you type things three times to make really really sure you meant to type that. If you find those other languages appealing I invite you to go use them instead.

Adding type declarations to a dynamically typed language seems to be the crux of the problem for Diederich and others. It fundamentally changes what Python is. But Gregory P. Smith noted that the PEP is really targeted at creating a common way for projects using types (and type-checking) for Python to annotate code, not at all for a wholesale change in existing code:

One thing I feel is often overlooked in the discussion on this PEP: It is about creating a unified type expression syntax for everyone working on Python typing to centralize around. Regardless of if the PEPs version falls short for some purposes. It allows for sharing work. There are multiple ongoing projects that are trying to make use of type information with Python, this allows them to all speak the same language. (MyPy, MicroPython, Cython, the static analyzer we are trying to create at Google, several others not on the top of my head I'm sure, etc.)

We will not be putting type annotations anywhere in the stdlib or expecting anyone else to maintain them there. That would never happen until tools that are convincing enough in their utility for developers to _want_ to use are available and accepted. That'll be a developer workflow thing we could address with a later PEP. IF it happens at all.

A more practical complaint about the annotations came from Requests developer Cory Benfield, who tried to annotate some parts of that library. For certain parameters, he got incredibly complicated annotations that still didn't fully restrict the parameter types. He summarized the problems, as follows:

I suppose the TLDR of this email is that I think that libraries with 'pythonic' APIs are the least likely to take up this typing system because it will provide the least value to them. Maintaining signatures will be a pain (stub files are necessary), writing clear signatures will be a pain (see above), and writing signatures that are both sufficiently open to be back-compatible and sufficiently restrictive to be able to catch bugs seems like it might be very nearly impossible.

After Chris Angelico suggested that Benfield might "want to just stop caring about the exact type", Benfield pointed to exactly that as part of the problem:

This change doesn't catch any of the bugs anyone was going to hit (no-one is passing a callable to this parameter), and it doesn't catch any of the bugs someone might actually hit (getting the tuple elements in the wrong order, for example). At this point the type signature is worse than useless.

Another complaint, voiced by Arnaud Delobelle, is that type hints are a threat to duck typing. The concern is that if the tools start complaining about interfaces that use duck typing, it will cause programmers to shy away from that longstanding Python type convention. That "could encourage a very significant cultural change in the way Python code is written towards a less flexible mindset". But Van Rossum has not forgotten about duck typing:

For me, PEP 484 is a stepping stone. Among the authors of PEP 484 there was much discussion about duck typing, and mypy even has some limited support for duck typing (I think you can still find it by searching the mypy code for "protocol"). But we ran out of time getting all the details written up and agreed upon, so we decided to punt -- for now. But duck typing still needs to have a way to talk about things like "seek method with this type signature" (something like `def seek(self, offset: int, whence: int=SEEK_SET) -> int`) so the current proposal gets us part of the way there.

The hope is that once 3.5 is out (with PEP 484's typing.py included *provisional* mode) we can start working on the duck typing specification. The alternative would have been to wait until 3.6, but we didn't think that there would be much of an advantage to postponing the more basic type hinting syntax (it would be like refusing to include "import" until you've sorted out packages). During the run of 3.5 we'll hopefully get feedback on where duck typing is most needed and how to specify it -- valuable input that would be much harder to obtain of *no* part of the type hints notation were standardized.

The roughly one and a half to two years that fall between each major Python release were clearly part of Van Rossum's thinking. He chose to implement something that could be done before 3.5 hit its feature freeze, but to keep the feature in a provisional state so that it could still be changed during the 3.6 development cycle. Several things may get addressed during that cycle, including support for duck typing and, possibly, adding new syntax for variable type declarations (instead of the comment-based annotations that are part of PEP 484). That all assumes that the PEP is accepted for 3.5, which seems likely unless Van Rossum has a change of heart.

The concerns about the standard library getting annotations are largely based on misunderstandings—there are no plans afoot to add them en masse either inline or as stubs. In fact, changes of that sort to "modernize" the standard library are frowned upon, Van Rossum said, because they often cause subtle regressions "in dark corners". Adding stub files for the standard library in 3.6 is a possibility, PEP co-author Łukasz Langa said. Additions to the standard library might include inline annotations, if the author is a fan of type hints, but that is going to be a ways out, according to Van Rossum.

The standard library is often seen as a place to look at "good" Python code, but that isn't really the case. It is a shared community code base, as Nick Coghlan described it but, as Van Rossum pointed out, it is not the epitome of Python coding style:

The stdlib contains a lot of Python code, and you can learn a lot from it, but good coding habits aren't generally something you learn there -- the code is crusty (some of the oldest Python code in existence lives in the stdlib!), often has to bend over backwards to support backward compatibility, and is riddled with performance hacks.

The debate got a bit testy at times but, as is generally the case in the Python world, it never got really out of hand. That community has found a way to moderate its communication, generally by way of fairly gentle reminders about respecting others' views. In the end, any change of this magnitude is going to cause some to be worried about the impacts—as they should. Contrary to some of the less temperate reactions (evidently Twitter was especially bad), this hardly seems like a change that "breaks" the language. It would not be hugely surprising to see it as quite a popular feature in, say, five years, once all the kinks have been worked out.

Comments (17 posted)

Brief items

Quotes of the week

If you have to ask "if my code calls GPL'd code in this complicated way, can I avoid having to be GPL-compatible?" then the answer is pretty much always "you're trying to reduce freedom, stop it".
— Lars Wirzenius

This is my favorite piece, 'The Blue Screen of Failure.' It's been shown all over the world.
— Digital artist Antonio Roberts at Libre Graphics Meeting, after his live demo of video "glitch" art crashed the computer running the presentation software.

Comments (1 posted)

Apache SpamAssassin 3.4.1 released

The Apache SpamAssassin 3.4.1 release is out. "Highlights include: Improved automation to help combat spammers that are abusing new top level domains; Tweaks to the SPF support to block more spoofed emails; Increased character set normalization to make rules easier to develop, block more international spam and stop spammers from using alternate character sets to bypass tests; Continued refinement to the native IPv6 support; and Improved Bayesian classification with better debugging and attachment hashing."

Full Story (comments: none)

Jython 2.7.0 released

Version 2.7 of Jython, the Python implementation for Java, has been released. As the version number suggests, this update provides language and runtime compatibility with CPython 2.7.

Full Story (comments: none)

GNU moe 1.7 released

Version 1.7 of the GNU moe console text editor has been released. This update includes a number of fixes for translating text data to and from UTF-8, adds a function for encoding as ASCII, and incorporates several fixes to search/replace functionality.

Full Story (comments: none)

Git v2.4.0 released

Git 2.4.0 has been released. LWN took a look at version 2.4.0 in April. Little seems to have changed since the release-candidate builds, but users are advised to give the release notes a close read nonetheless.

Full Story (comments: none)

PyTables 3.2.0 released

PyTables version 3.2.0 has been released. The update "is the result of more than a year of accumulated patches, but most specially it fixes a couple of nasty problem with indexed queries not returning the correct results in some scenarios." The release notes contain considerably more detail.

Full Story (comments: none)

Git code hosting beta (launchpadblog)

Early support for hosting Git repositories directly on Launchpad has been announced. "This has been by far the single most commonly requested feature from Launchpad code hosting for a long time; we’ve been working hard on it for several months now, and we’re very happy to be able to release it for general use. This is distinct from the facility to import code from Git (and some other systems) into Bazaar that Launchpad has included for many years."

Comments (59 posted)

Newsletters and articles

Development newsletters from the past week

Comments (none posted)

App Container spec gains new support as a community-led effort

CoreOS looks at community adoption of the App Container spec (appc). "In order to ensure the specification remains a community-led effort, the appc project has established a governance policy and elected several new community maintainers unaffiliated with CoreOS: initially, Vincent Batts of Red Hat, Tim Hockins of Google and Charles Aylward of Twitter. This new set of maintainers brings each of their own unique points of view and allows appc to be a true collaborative effort. Two of the initial developers of the spec from CoreOS, Brandon Philips and Jonathan Boulle, remain as maintainers, but now are proud to have the collective help of others to make the spec what it is intended to be: open, well-specified and developed by a community."

Comments (16 posted)

Page editor: Nathan Willis
Next page: Announcements>>


Copyright © 2015, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds