CPython without a global interpreter lock

Posted Aug 10, 2023 16:59 UTC (Thu) by NYKevin (subscriber, #129325)
In reply to: CPython without a global interpreter lock by tialaramex
Parent article: CPython without a global interpreter lock

> Reading the PEP did clarify that the plan really is as cavalier as I feared, that environment variable is a YOLO feature and it's going to be left in plain sight like a tempting red button at toddler height. I don't expect that to go well.

I know I previously said that I hoped they would provide stronger guarantees than this, but honestly, I'm having a hard time getting excited about it. If you go fiddling around with the interpreter's settings and cause your codebase to break, well, you get to keep both pieces. That's maybe not the safest way of doing things, but it's a valid perspective IMHO. This is more or less the same reasoning that leads Python to have no real encapsulation, for example.

CPython without a global interpreter lock

Posted Aug 13, 2023 11:01 UTC (Sun) by tialaramex (subscriber, #21167) [Link] (2 responses)

This sentiment makes lots of sense for Bob's home grown Python program, sole maintainer Bob, sole user Bob, chief executive and bottle washer also Bob. If it breaks it's Bob's fault. Duh.

But this sort of work is happening because people are trying to write Python at scale and having problems. Jim writes tricky code that's fine assuming the GIL, Sarah uses Jim's code in a sub-routine for her team's new project. Mark uses that sub-routine with Sarah's permission from the new Project Foo, and the Project Foo lead Andy sets the environment variable because it's a "known workaround" for a problem they have, but now about one run in twenty has corrupt results and nobody knows why.

Whose fault is the corruption? Jim? Sarah? Mark? Andy? It is likely that there's no intersection between the people who would have known this won't work and the people who did it anyway.

Maybe it's fine, but I wouldn't want to be the Python community finding that out the hard way.

CPython without a global interpreter lock

Posted Aug 15, 2023 5:41 UTC (Tue) by NYKevin (subscriber, #129325) [Link] (1 responses)

If Jim's code is maintained, he can just add a startup check for the offending environment variable, and cleanly fail the import at that time with an explanatory message. If Jim refuses to do that, then it's his fault. Checking one environment variable is much cheaper than importing an entire module, so you just do it at import time and it's effectively free.

If Jim's code is unmaintained, then it's... well, not necessarily Andy's *fault*, but it is Andy's *problem* because it's his project. Such is an occupational hazard of relying on unmaintained code, in any language, but especially in Python. Python has had a very well-established practice of slowly, carefully deprecating and removing old APIs, both before and after the 2-to-3 transition. 2-to-3 was not an anomaly because it broke backcompat, it was an anomaly because it did it abruptly in a single release, and on a much larger scale than has otherwise been typical. This problem sounds much more like a classic Python deprecation than a "break the world" flag day.

CPython without a global interpreter lock

Posted Aug 16, 2023 16:57 UTC (Wed) by tialaramex (subscriber, #21167) [Link]

I would not characterise Python's approach as "careful", on the whole. "Slapdash" is the word which comes to mind. There are some good ideas in their compatibility approach, but they're inconsistently applied, and so I wouldn't rely on them.

My guess is that if it became normal to write environment variable checks and block execution, the response from CPython would be to change the name of the environment variable. This is an intentional foot gun, not that they'd accept that description, de-fanging it would be contrary to their purpose in offering it.