Growing pains for typing in Python

Posted Jan 23, 2024 1:12 UTC (Tue) by MrWim (subscriber, #47432)
Parent article: Growing pains for typing in Python

I feel like I've been getting tremendous value from Python typing. I recently[^1] converted a medium sized Python project from Python 2 to Python 3. The project exists at the interface between web technologies and unix I/O - so there's lots of places where you want to be using bytes, and others where unicode makes sense.

I wouldn't have been able to do it without the type checker. I basically went through and annotated where things should be bytes and where things should be unicode and then went through and fixed all the red squiggles in VSCode - adding additional type annotations when it couldn't work it out for itself.

Typing makes the IDE[^2] experience worthwhile. For the first 15 years of my Python experience I used gedit. I didn't get the point of IDEs, because they offered no useful information for a language as dynamic as Python, and were just big and heavy. They're still heavy, but now it's worth it. Features like autocomplete[^3], go to definition, find references, and the other features those basic tools enable we're functionally useless in my Python codebases before typing - now I use them as a matter of course.

Typing affects the way I design APIs for libraries I write for others. I'm not thinking about type system purity, but rather IDE developer experience for my customers. For example:

I avoid using **kwargs in public facing functions, as the IDEs can't help the user - instead I list out all the arguments explicitly. It's more code, but it makes my APIs easier to consume
I avoid having functions where the return type depends on the arguments - separate functions can be clearer in this case, even if it involves a bit of duplication. Failing that APIs can be designed with @typing.overload in mind - so the IDE can still understand your code even when types can vary.
APIs involving sum types, previously too difficult to keep track of with Python, can now be effectively used. The IDE will now tell you if you've forgotten to handle a particular case, or have missed a guard. This is particularly powerful with the new match statement.

I've also had a bit of fun with some fancier typing shenanigans. I've used it to add type checking to our use of SQL by generating a .pyi file based on strings that look like they contain SQL in our codebase. I've also used it in combination with decorators for parsing and validating HTTP requests meaning we produce better 400 HTTP error responses and have machine-checked confidence that the requests have been validated. I've also been building tools for our customers that allows them to do basic coding with dropdown boxes for options - driven by type annotations.

[^1] in the last year or so

[^2] my experience is with VSCode and the pylance LSP (based on PyRight)

[^3]: I tend to agree with the grug on type systems here:

grug very like type systems make programming easier. for grug, type systems most value when grug hit dot on keyboard and list of things grug can do pop up magic. this 90% of value of type system or more to grug