Status of mypy and type checking

In his 2017 Python Language Summit session, Jukka Lehtosalo updated attendees on the status of type checking for the language, in general, and for the mypy static type checker. There are new features in the typing module and in mypy, as well as work in progress and planned features for both. For a feature, type hints, that is really only around three years old, there has been a lot of progress made—but, of course, there is still more to come.

The most significant new thing for types in Python is the adoption of PEP 526, which adds a way to annotate variables with their types. As of Python 3.6, variable annotations can be used for regular variables, instance variables, and class variables. The latter is made possible with the ClassVar[] annotation that has been added to typing. Other additions include NewType() for creating distinct types and NoReturn for functions that do not return.

Some recent mypy features include function overloads in source files (and not just stub files) and basic metaclass support, but there is still work to be done on the latter. There is also a new "quick mode" that is up to ten times faster. Quick mode is an incremental check; it just looks at the file itself and assumes that what it imports does not need to be checked.

There are also some experimental mypy features that Lehtosalo mentioned. The mypy_extensions module contains various extensions to typing that are being tried out. Some of those may get promoted to typing if they work out. One of those is the more flexible Callable[] type, which has a syntax that is "not pretty" but works. More information about these and other features can be found in his mypy 0.510 release announcement.

There are also some features in progress for mypy, he said. The TypedDict type, which will allow dictionaries that specify the types of values for specific keys, is one. Another is support for structural subtyping using Protocols. There are some planned improvements for type variables, including adding support for variadic type variables and for variables that describe function argument specifications. Decorators sometimes change a function's signature, so support for declaring the decorated type of a function is planned as well.

Mypy is starting to be used in production. At Dropbox, where Lehtosalo works, 700,000 lines of code have been annotated and are being checked with mypy. The Zulip project has 95,000 lines of code annotated; Facebook, Quora, and others are using the tool as well. There has been quite a bit of positive user feedback, he said. Performance is still an issue, however; a full run at Dropbox takes around two minutes, which is "barely acceptable". But a large scale roll-out at Dropbox is under way.

There were some lessons learned along the way. To start with, changing type systems is "very expensive" and causes a fair amount of pain for users. That means Dropbox may become stuck with some early choices it made before some features had been added to typing and mypy. Having the typing module in the standard library has turned out to be annoying, because there are new features in the 3.6 release that can't be used in 3.5, which is the version used by Dropbox. typing is moving fast, so sometimes it makes sense to backport features into earlier versions, he said.

There are a lot of contributors to the projects (both typing and mypy), especially for typeshed, which collects annotations for Python built-ins and the standard library. The two other major type checkers, pytype and PyCharm, also contribute, so there is a real community building up around type annotations.

Mark Shannon asked when the project would decide to stop adding features for ever-more-obscure type constructs; "at what point do you say 'just use Any'?" Lehtosalo said that the project tends to consider constructs that have multiple users and uses throughout the ecosystem and is not interested in adding support for lots of one-off corner cases.

[I would like to thank the Linux Foundation for travel assistance to Portland for the summit.]