The 2020 Python Language Summit

[Posted May 1, 2020 by jake]

The 2020 Python Language Summit was held virtually this year, over two days, via videoconference, with discussions via voice and chat. The summit is a yearly gathering for developers of CPython, other Python implementations, and related projects. As with last year, A. Jesse Jiryu Davis covered the summit; his writeups are being posted to the Python Software Foundation (PSF) blog. So far, all of the first day's session writeups are up, as well as two (of six) from the second day. Topics include "All strings become f-strings", "The path forward for typing", "A formal specification for the (C)Python virtual machine", and more.

The 2020 Python Language Summit

Posted May 2, 2020 2:16 UTC (Sat) by NYKevin (subscriber, #129325) [Link] (2 responses)

> There's better news about mypyc, an experimental project to translate type-annotated Python into C. The translator's main use for now is converting mypy to C for speed. There is work in progress to allow a mix of Python and Python-translated-to-C in the same program, and to write documentation. The mypyc project expects a Google Summer of Code student this summer.

Cython is (sorta) prior art, but this is nevertheless fascinating. Cython currently requires an incompatible type hinting syntax, and type-hinted variables are given C semantics rather than Python semantics. A middle ground, faster than CPython but still fully Python-compatible, would be very Nice To Have.

The 2020 Python Language Summit

Posted May 2, 2020 7:33 UTC (Sat) by HelloWorld (guest, #56129) [Link] (1 responses)

There is also Nuitka that does the same thing.

The 2020 Python Language Summit

Posted May 4, 2020 23:42 UTC (Mon) by jafd (subscriber, #129642) [Link]

Nuitka tries hard to be as semantically compatible to vanilla Python as possible (it's its stated goal after all), so it will do to type annotations what Python will do, namely: ignore them.

Although I think it could probably be beneficial to have a mode in Nuitka where it would actually enforce type annotations.

The 2020 Python Language Summit

Posted May 2, 2020 11:18 UTC (Sat) by bluss (guest, #47454) [Link] (2 responses)

I think the Should All Strings Become f-strings proposal shows more what happens when there is no linter in the default "compiler" (Python compiles the input before it runs). Gcc, Clang, Rust etc are all adding smarter and smarter lints to help the user, catch common mistakes, and relevant for those languages, flag allowable but dangerous code.

Without a default linting step, Python doesn't have this way of dealing with common mistakes. It needs to choose between some harsh alternatives, like either rejecting the code based on syntax, raising a runtime exception or a noisy runtime warning, that makes no difference between "compilation in development" and running code in production.

The 2020 Python Language Summit

Posted May 2, 2020 16:22 UTC (Sat) by iabervon (subscriber, #722) [Link] (1 responses)

There is support for turning on warnings in your development/test environments and leaving them off in production where nobody who could see them can fix them. I think the issue in this case is more that what people tend to write by mistake is the correct way of doing something they don't intend, and it's hard to tell if the programmer meant to write a format string to be used in a different context or an f-string to be used immediately.

The 2020 Python Language Summit

Posted May 2, 2020 23:54 UTC (Sat) by NYKevin (subscriber, #129325) [Link]

The problem with Python's warnings, in my opinion, is a set of mutually reinforcing difficulties:

The standard set of warnings is rather minimal, and thus only good for catching a narrow set of bugs.
- For example, there's a function called compiler_warn() which emits a SyntaxWarning at compile-time (or converts it to a SyntaxError if the user has requested such conversion). This function is called exactly five times in the whole of compile.c (and nowhere else). All five of these are cases where the code is semantically wrong (e.g. trying to subscript a lambda expression or call a string literal), rather than merely error-prone or ugly (as with -Wimplicit-fallthrough or -Wmisleading-indentation in C).
- There are other warnings that happen at runtime, but these are mostly about the APIs of standard library modules rather than the core language.
People who care about warnings put their development efforts into third-party tools like pyflakes, pylint, and mypy, rather than CPython. pylint in particular has an extraordinarily elaborate set of linting rules which catches all sorts of questionable or probably-wrong lines of code.
Developers have not been trained to pass -W[something] to python in the same way they pass -Wall to gcc etc. Python's warning control is highly customizable, but most people seem to ignore it entirely, probably because of the above points.

Should All Strings Become f-strings?

Posted May 2, 2020 16:56 UTC (Sat) by mb (subscriber, #50428) [Link] (9 responses)

Nah. Please don't do that.
It forces every Python developer on earth to go through all their strings and check them for possible f-string formatting. We already went through all of our strings at the 2to3 conversion. Yes this time it's not as hard, because only immediates would be affected and a conversion tool could probably be written.

And all that just to prevent some easy to catch errors caused by missing f prefixes.
How's a missing f prefix different from a typo'ed f notation inside of the string anyway?

Should All Strings Become f-strings?

Posted May 3, 2020 16:41 UTC (Sun) by Conan_Kudo (subscriber, #103240) [Link]

I would also argue that defaulting to f-strings would potentially be a violation of the "explicit is better than implicit" aspect of the Zen of Python. In general, I don't expect magic in my string declarations unless I explicitly say so. And I have a fair amount of code that leverages this to only activate the templating at a later phase rather than immediately, as defaulting to f-strings would do. There are still problems with f-strings and internationalization, too...

Should All Strings Become f-strings?

Posted May 3, 2020 17:47 UTC (Sun) by mirabilos (subscriber, #84359) [Link] (5 responses)

That’s going to break SO MUCH if they make all strings f-strings.

Should All Strings Become f-strings?

Posted May 3, 2020 18:29 UTC (Sun) by kooky (subscriber, #92468) [Link] (4 responses)

me too. I have thousands of

'The numbers are {} and {}'.format(x,y)

Should All Strings Become f-strings?

Posted May 4, 2020 20:43 UTC (Mon) by kevincox (subscriber, #93938) [Link] (3 responses)

I imagine that it would have to be opt-in. `from __future__ import fstrings` but then you need that boilerplate at the top of every file.

And you could trivially write a conversion tool that fixed your files and added the "import". It could even convert most `.format()` calls to native fstrings fairly easily.

Should All Strings Become f-strings?

Posted May 7, 2020 11:12 UTC (Thu) by mgedmin (subscriber, #34497) [Link]

__future__ imports imply that this feature will be turned on by default in some future version of Python, so this is just kicking the problem down the road for a bit.

Should All Strings Become f-strings?

Posted May 7, 2020 13:52 UTC (Thu) by mathstuf (subscriber, #69389) [Link] (1 responses)

My main issue with this is that the migration ends up making 3.8 -> 3.9 a Python2 -> Python3 barrier again. The "don't format this" syntax is invalid in old releases, so any migration now means you're no longer compatible with older 3.x releases.

Should All Strings Become f-strings?

Posted May 10, 2020 21:57 UTC (Sun) by NYKevin (subscriber, #129325) [Link]

> The "don't format this" syntax is invalid in old releases, so any migration now means you're no longer compatible with older 3.x releases.

My working assumption is that this syntax would have to be valid and available for a lengthy period before f-strings would actually become the default. Ideally, this would be long enough that even RHEL is no longer using the old versions, but maybe that's a bit too optimistic.

Fortunately, enabling the p prefix and the future import would not actually break any existing code, so it could be backported if desired. This actually happened with the Python 3 u and b prefixes, and I tend to assume the Python devs learned from that experience.

Should All Strings Become f-strings?

Posted May 4, 2020 7:50 UTC (Mon) by rschroev (subscriber, #4164) [Link] (1 responses)

What? I couldn't believe I read that correctly the first time I saw it. All strings f-strings by default? That's just insane.

There seems to be consensus that f-strings are the best thing ever ("the killer feature of Python 3.6"). I don't agree, but that's not even the point.

There is a fundamental difference between a string itself and formatting a string. That difference is there for a reason; don't mess with it! If you want to format a string, then format it. Explicitly.

> Smith conceded that p-strings make the language more complicated, but, he said, "There's going to be very few p's in the wild, and I think their explanation will be fairly obvious."

Smith wants to make things more simple; acknowledges his proposal would make the language more complicated instead; insists it doesn't matter.

What happened to "Explicit is better than implicit"? That's one of the main reasons I like Python. Recent changes are complicating the language but they don't bother me too much. This proposal is a game changer: it it gets accepted, Python will no longer be my language of choice for all kinds of scripts and tools.

Should All Strings Become f-strings?

Posted May 6, 2020 22:08 UTC (Wed) by NYKevin (subscriber, #129325) [Link]

> There is a fundamental difference between a string itself and formatting a string. That difference is there for a reason; don't mess with it! If you want to format a string, then format it. Explicitly.

While I generally agree, "f-strings by default" is much less insane than some of the other programming languages I've worked with. Inform 7, among its many other oddities, treats all strings as *lazily-evaluated* f-strings, and thus the contents of a string may be different every time you print it! In effect, every string literal implicitly becomes a thunk, closure, or lambda (whichever terminology you prefer), and that lambda is implicitly evaluated by the I/O system whenever a string needs to be printed (there's also a function to force immediate evaluation, because Inform 7 is not Haskell). Fortunately, in the Python world, everyone has been very clear from the beginning not to go down that bizarre and implausible road.

(To be fair, Inform 7 does have reasons for doing this. It is predominantly used for writing text adventures, which naturally contain a ton of text substitutions, and so it dedicates an enormous amount of effort to making text substitutions look grammatical and reasonable in a wide variety of circumstances. For example, it has full support for plurals, verb conjugation, comma-delimited lists, and a variety of other "little things" that are annoying to do by hand. Making it as easy as possible for printed text to contain "natural-looking" variations is obviously concordant with that goal. Also, it was originally designed to compile to Z-machine, which has severe memory constraints, so deferring text substitution to the last possible moment is obviously preferable to building a large string in memory and having it sit there indefinitely.)

The 2020 Python Language Summit

Posted May 4, 2020 20:01 UTC (Mon) by ofranja (guest, #11084) [Link]

"The path forward for typing" has a curious paragraph:

> Van Rossum offered an update on mypy. He admitted he hadn't been active on mypy recently, and "my former colleagues at Dropbox have not been able to make as much progress as we did in the past."

The missing piece of information here is that Dropbox decided to rewrite the Sync engine in Rust, see [1]. There's an interesting insight from this article from Dropbox:

> Rust has been a force multiplier for our team, and betting on Rust was one of the best decisions we made. More than performance, its ergonomics and focus on correctness has helped us tame sync’s complexity. We can encode complex invariants about our system in the type system and have the compiler check them for us.

I recommend both [1] and [2] as very interesting articles describing the rationale, design and trade-offs chosen on the development of this critical piece of software.

[1] https://dropbox.tech/infrastructure/rewriting-the-heart-o...
[2] https://dropbox.tech/infrastructure/-testing-our-new-sync...