LWN: Comments on "Python finally offloads some batteries"

Python finally offloads some batteries

LtWorf — Mon, 25 Apr 2022 07:33:09 +0000

On the long run these sort of things bring people to using other languages.

Python finally offloads some batteries

kleptog — Tue, 29 Mar 2022 10:11:09 +0000

Maybe it's a cultural thing? A leech (in my experience) doesn't tend to harm the host, and they go away by themselves. They are at most temporarily irritating.

However, I just looked it up in the dictionary and it has definitions involving the words "exploit" and "extort" which are much more negative. If that's the meaning you're using then I can see the statement being interpreted very differently.

Python finally offloads some batteries

pebolle — Mon, 28 Mar 2022 21:21:29 +0000

> it implies that the speaker *utterly despises* the person they're talking about.

Exactly. And the infuriating part, for me, is that this metaphor is used for people and organizations doing what I thought was the right thing: using Free Software.

Python finally offloads some batteries

nye — Mon, 28 Mar 2022 13:23:48 +0000

> If the name describes the situation adequately ("leech", in this case a metaphor), I don't have issues with it

A leech is a parasite that latches on to an unwilling host and drains the life out of it to sustain itself. Some metaphor. In fact, the word is specifically used as a particularly emotive term of derision and hatred; it implies that the speaker *utterly despises* the person they're talking about.

Basically, it's a more specific way of describing somebody as a "worthless fucking cunt", or similar. On the offensiveness scale, it's high enough that I can only really think of one word that's higher but that I might occasionally use in close company when senselessly enraged; everything higher than *that* I wouldn't even *think*, let alone say.

Python finally offloads some batteries

edgewood — Sun, 27 Mar 2022 13:53:33 +0000

I'm so sorry that I missed that. I had skimmed the PEP, and had seen that section, but my eyes got pulled to the code sample at the bottom of the section.

I just had a chance to convert a script that used cgi.FieldStorage to use urllib.parse.parse_qs instead, and it only took me about a half an hour. It helped that I had a helper method to smooth over some weirdness caused by the interaction of FieldStorage and the structure of my existing HTML, and a lot of the accesses of the FieldStorage values already went through that method. I just changed it to access the parse_qs dictionary instead, and changed a handful of sites that directly accessed FieldStorage values directly to call the helper, and it all worked.

My previous plan was to vendor cgi.py, so thank you for responding to my question and pointing me to how I could use supported stdlib code!

Python finally offloads some batteries

oldtomas — Fri, 25 Mar 2022 07:13:09 +0000

If the name describes the situation adequately ("leech", in this case a metaphor), I don't have issues with it.

That's what names have been made for, after all.

Python finally offloads some batteries

dm93 — Thu, 24 Mar 2022 20:49:01 +0000

This was my experience as well.

I was told not to fix things because "it would clutter up the git history."

PCM

davidgerard — Thu, 24 Mar 2022 12:19:27 +0000

> BUT I have to agree that if someone _has_ decided to introduce children to computers using a high level language like Python (which clearly suits some children, even it it didn't suit me),

It's taught in UK schools now. Scratch in primary school, Python in high school.

Python finally offloads some batteries

tao — Tue, 22 Mar 2022 10:31:50 +0000

Obviously if I wanted a car to *stay put* I would not pump any gasoline into it at all. Refuelling it would just make it easier for it to drive off. :P

Python finally offloads some batteries

nilsmeyer — Mon, 21 Mar 2022 08:09:04 +0000

> They don't care at all to break a library with only tens of projects using it.

Should they?

Python finally offloads some batteries

flussence — Fri, 18 Mar 2022 22:07:35 +0000

If *Perl* can successfully get rid of its bundled CGI module, Python has no excuse. Surely 20 years is long enough to figure out what alternatives are worth using and have staying power.

Python finally offloads some batteries

cjwatson — Fri, 18 Mar 2022 18:49:11 +0000

The PEP itself has several suggestions (https://peps.python.org/pep-0594/#cgi).

Python finally offloads some batteries

cjwatson — Fri, 18 Mar 2022 18:48:15 +0000

https://bugs.python.org/issue27777 was an absolute blocker in my application causing extremely confusing failures (and that only due to quite pedantic tests - we might easily have missed it until it hit production), and caused me weeks of work trying to work around it before I eventually concluded that cgi.FieldStorage was engaged in playing core-wars with other bits of itself and there was no realistic prospect of it ever being fixed, so switched to something different instead. See https://github.com/zopefoundation/zope.publisher/issues/39.

Python finally offloads some batteries

edgewood — Fri, 18 Mar 2022 18:35:40 +0000

Did you find an alternative? I have a few scripts that are only called occasionally, and aren't performance sensitive, so are fine as CGI scripts, which saves me deployment hassle.

But I do need to parse URL and form parameters, and use FieldStorage for that. I was planning to just copy cgi.py when it went EOL, but if there's a better replacement I'll use it.

Python finally offloads some batteries

NYKevin — Fri, 18 Mar 2022 17:40:45 +0000

> Most python packages only support of subset of python reasons for lots of good reasons.

In my experience, this "subset" is usually of the form "version 3.x or later" for some value of x (or, for a handful of very old libraries, "version 2.7.x only"). I don't believe I have seen a whole lot of libraries that set a maximum version, other than the ones which were never ported to 3.

Python finally offloads some batteries

NYKevin — Fri, 18 Mar 2022 17:37:46 +0000

> I think the original idea was to save time on doing js files downloads… but apparently the person saying this didn't know about all the headers and roundtrips that need to happen before a file can be downloaded.

I believe this may have been a contributing factor to the invention and adoption of HTTP/2, because now the server can just say upfront "here are all of the libraries you are going to need, when you eventually get around to running the JS."

Is this one of those 80/20 problems?

canatella — Fri, 18 Mar 2022 17:21:31 +0000

This:

> I'm looking to see python's package management improve.

As of now there are multiple ways of managing dependencies and installing packages: setuptools, pip, pipx, pipenv, poetry, conda, pdm, twine, you name it (I'm probably mixing stuff here but that's part of the problem). There is no common file format for a lock file so end user installing using pip or pipx means that you never know which version of a dependency will be installed (I know I do generate requirements.txt from the lock files but I still got it to somehow break)

I'm okay with moving things out of stdlib, but first fix the package dependency management mess. Bless one solution and document it and support it. The packaging.python.org advises to use pipenv but then warns: "By contrast, Pipenv explicitly avoids making the assumption that the application being worked on will support distribution as a pip-installable Python package." Why is that ? The officially documented way of building a package doesn't support the recommended way of installing packages ? Also recommending pip to install packages ? Guaranteed version conflicts ahead...

Sorry for ranting here but I can't count how many days I lost fixing dependency issues in Python, maybe I'm doing it wrong but there is so many way to manage dependencies and delivery in Python and nothing blessed and cleary documented on packaging.python.org. I'm completely lost.

Never had any problem with Ruby with gem and bundler. And honestly, these days, even C++ dep management is better here with Conan then Python, which says a lot.

Python finally offloads some batteries

notriddle — Fri, 18 Mar 2022 16:33:20 +0000

If you want Python to stay put, don't upgrade. If you don't want your cake to go away, don't eat it.

Python finally offloads some batteries

logang — Fri, 18 Mar 2022 15:46:12 +0000

That seems like wishful thinking at best. It is not nearly as stable as you think it is. Most python packages only support of subset of python reasons for lots of good reasons. If large swaths of the python library are now in PyPi they also now gain complicated dependencies between them as well. Maybe the kernel's driver API experiences more churn, but the point remains.

As NYKevin pointed out the requests module depends on urllib (yikes) so if that module gets removed from the standard library then you've broken requests for the latest release of python.

If tons of important modules are ejected then the core teams haS to stop removing or deprecating things to avoid the same dependancy hell the kernel would have with out of tree drivers.

Python finally offloads some batteries

mpr22 — Fri, 18 Mar 2022 09:26:54 +0000

> is actually that it's less work and more desirable for them to just port to Python 3.

Or to not-Python.

Python finally offloads some batteries

milesrout — Fri, 18 Mar 2022 08:31:06 +0000

That pretty much confirms my point, though. There were supposedly tens of thousands of developers out there all dependent on Python 2. They couldn't *possibly* move to Python 3. They insisted Python 2 had to be maintained. A few of them, like you, may have inquired about contributing to maintenance of Python 2.

But where is the big effort to actually do it? I can't see one. There are a few basically-dead projects out there to maintain forks of Python 2.7. I don't think (?) any of them are still going. So despite all the handwringing and complaining, it turns out that the *revealed preference* of those people is actually that it's less work and more desirable for them to just port to Python 3.

It's a lot easier to complain or to inquire about something than to actually *do* it. If it were really important to you to maintain Python 2, you would have done it with a bunch of other people for whom it was important.

Python finally offloads some batteries

milesrout — Fri, 18 Mar 2022 08:25:47 +0000

It's a bit different with Python, because the language itself is - pretty much - stable. They don't go out of their way to break core language interfaces in every version.

Linux developers, on the other hand, have no qualms about changing core interfaces in any old version. They don't exactly *go out of their way to*, especially where it would complicate backporting fixes to older versions. But look at the discussions happening around list iterators. They clearly are willing to change fundamental interfaces quite readily.

This means that out-of-tree modules for the kernel are in a very different level of support (none at all) than third-party modules for Python. The whole *point* of Python is a stable interface against which to write third-party modules! That's what a language *is*!

Python finally offloads some batteries

LtWorf — Fri, 18 Mar 2022 07:48:21 +0000

Well js lacks even kinda basic functions, but I agree with you.

The other problem with npm is that somehow they've decided (clearly without ever taking the time to do any measurement) that a library of 1 function is faster than putting a bunch of related functions all into the same library.

I think the original idea was to save time on doing js files downloads… but apparently the person saying this didn't know about all the headers and roundtrips that need to happen before a file can be downloaded.

Python finally offloads some batteries

mb — Fri, 18 Mar 2022 07:03:04 +0000

>cgi.FieldStorage is so broken as to be a snare and a delusion, so I'm glad they're removing it.

It works just fine for me.
What's broken with it?

I'm not against removing unused or rarely used modules.
But removing those widely used modules, like cgi, is going to cause major waste of developer time in the order of hundreds of thousands of hours. That's not Ok and it will hurt Python's reputation. Again.

Is this one of those 80/20 problems?

smitty_one_each — Fri, 18 Mar 2022 01:46:39 +0000

We want stability, but not stagnation.

I'm looking to see python's package management improve.

Maybe doing some battery maintenance will establish precedent to go for further modernization.

Python finally offloads some batteries

cjwatson — Thu, 17 Mar 2022 23:06:03 +0000

I don't know about the rest of cgi, but cgi.FieldStorage is so broken as to be a snare and a delusion, so I'm glad they're removing it. I got involved with maintaining the multipart package when I found that FieldStorage was unusably broken for my purposes and that its design was enough of a ball of wax that it couldn't realistically be fixed without breaking something else.

Python finally offloads some batteries

Paf — Thu, 17 Mar 2022 22:01:52 +0000

I think it’s passive because they’re saying they’re not going to do it. It would be *allowed*, but they’re not doing it, which seems reasonable enough.

Python finally offloads some batteries

NYKevin — Thu, 17 Mar 2022 21:17:27 +0000

> but dropping useful modules that people depend upon is not.

If nobody is willing to maintain the code, it is de facto unmaintained. You can slap a "maintained" label on it all you like, but that does not cause maintenance work to get done.

> but they'd get help dealing with underlying core changes that affect their module.

I believe we've all learned the hard way that core changes should be backwards-compatible, and so this kind of help should not be required in the vast majority of reasonable cases.

> But after the pain, the benefit is a quality url module in the standard library and urllib could be re-implemented as a thin wrapper over it, and/or dropped after a very long deprecation period.

They certainly can't do that until requests stops depending on urllib.

PCM

mbunkus — Thu, 17 Mar 2022 19:40:20 +0000

It actually isn't that easy as not all PCM data is equal. There are several parameters that a decoder must know in order to play that sound back properly, including:

* sampling frequency (also called sample rate)
* number of channels
* order of channels
* number of bits per sample (also called bit depth)
* signed or unsigned
* Endianess if bits per sample > 8
* PCM format (e.g. a-law, µ-law, linear, etc.)

PCM used on CDs isn't the same as PCM used on Blu-rays, for example.

Not one of those parameters is really optional. Sure, you can specify all of them manually when decoding (or when using tools such as sox to convert from raw PCM data to one of the containerized variants), but the container's whole purpose is to relief us humans of the need to keep that extra information around somehow. That's what the WAV container does. As do other container formats, but WAV one of the simplest one to use (within certain limits), making it especially suitable for people new to programming or new to audio processing.

Python finally offloads some batteries

logang — Thu, 17 Mar 2022 18:29:42 +0000

I strongly disagree.

This would be like the Linux Kernel developers deciding they don't have time to maintain large swaths of drivers and just dropping them. Then expecting people who need them to get the oot drivers from github using dkms and shift the weight on distributions to package all these drivers and deal with the resulting dependency hell. It would be a _giant_ mess. What would really happen is the drivers would be maintained even worse then they are now and become even more broken, and people who need them would be out of luck.

I think the emphasis should be on growing the project and the number of developers, not splintering off poorly maintained code into situations of even worse maintenance. Dropping modules that are obsolete and which nobody really uses is fine (allowing for the option of a maintainer who cares to step up); but dropping useful modules that people depend upon is not.

IMO, the best solution to the urllib issue would be to absorb the requests module into the standard library and bring all their developers with it. A development model similar to the kernel where a subsystem maintainer collects and sends patches upstream to the core python maintainers. There may be issues with this, but none that couldn't be worked out in the long run. Yes, the requests developers may need to do more work to ensure backwards compatibility, but they'd get help dealing with underlying core changes that affect their module. The python core team may have to make process changes to allow for a higher volume of security changes to stable releases. But after the pain, the benefit is a quality url module in the standard library and urllib could be re-implemented as a thin wrapper over it, and/or dropped after a very long deprecation period.

I've written many simple scripts that run on urllib because I have no interest in dealing with the additional dependency. If requests was in the standard library I'd certainly use it universally, but it is not and in many circumstances that disqualifies it. I suspect this is a common practice. The core team have a responsibility not to break such widespread usage with no real alternative for developers.

Python finally offloads some batteries

NYKevin — Thu, 17 Mar 2022 17:23:18 +0000

Why? I just described a solution in which all of the same modules would still be installed by default, meaning your individual scripts wouldn't need to track anything.

Python finally offloads some batteries

NYKevin — Thu, 17 Mar 2022 17:19:52 +0000

You have the right to fork (which someone has already exercised, see the Tauthon project). If you don't want to do that, then you were (presumably) expecting to get something out of the PSF that you can't get out of a fork, and the PSF is not obligated to give that something to you if they do not wish to do so.

Python finally offloads some batteries

mb — Thu, 17 Mar 2022 17:19:13 +0000

cgi will be removed? Really?

I bet that's being used in thousands of in-production scripts. Including some of mine.
Why does this have to be removed?
What are the alternatives (that aren't deprecated tomorrow)?

Python finally offloads some batteries

NYKevin — Thu, 17 Mar 2022 17:18:00 +0000

I never said it was. My point is that, even if it *was* 10 lines on a pastebin, people would still expect support, because people, in general, are unreasonable, and developers get tired of dealing with them.

Python finally offloads some batteries

anselm — Thu, 17 Mar 2022 14:25:16 +0000

In Python, my impression is that migrating a module is quite a big job, so there's a temptation to just drop it and leave it nowhere and its few remaining users out in the cold.

The code is there, and making Python packages for PyPI isn't exactly rocket science. The seven people who actually still use ossaudiodev or nntplib can do their own legwork, or else the modules must not have been all that essential after all.

Python finally offloads some batteries

Wol — Thu, 17 Mar 2022 13:20:19 +0000

Yup. The Python people need to create some fjords for old batteries to pine for.

Cheers,
Wol

Python finally offloads some batteries

nix — Thu, 17 Mar 2022 12:28:56 +0000

I don't know. Doing that has the advantage that moving things in and out of core is uncontroversial and done routinely, and that things that leave core are *still easily accessible* (to the majority of users who can install stuff as needed), because they're still on CPAN. This in turn is easy because the majority of the standard library is maintained using the same build system as CPAN, so moving things from core to CPAN or dual-lifing them in both is literally a matter of running a couple of commands: almost no source changes are usually needed. In Python, my impression is that migrating a module is quite a big job, so there's a temptation to just drop it and leave it nowhere and its few remaining users out in the cold.

PCM

nix — Thu, 17 Mar 2022 12:26:13 +0000

> The actual Microsoft WAV format technically does lots more than this, but scarcely anybody bothers implementing any of that stuff, and I think that includes Python's wave. So we should admit that we don't want WAV we just wanted to store PCM data.

Yes, but... huge amounts of software accepts and produces WAV files. Much less accepts or produces PCM. It's an interchange format in that respect, like it or not...

Python finally offloads some batteries

azumanga — Thu, 17 Mar 2022 12:08:41 +0000

I genuinely looked at how I could join the Python 2 team, to keep it going (with minimal future changes). Turns out the people in charge of Python didn't want that -- they explicitly wanted it to stop, even if people and companies were willing to provide future support.

PCM

tialaramex — Thu, 17 Mar 2022 11:49:19 +0000

All WAV is doing in these scenarios is providing a few header bytes on some raw PCM data. If you have software to just treat some data as 16-bit unsigned PCM data and play it as a sound, you'll find your WAV file is exactly that sound, except there's a brief fraction of a second glitch at the start because the first few bytes are a header.

The actual Microsoft WAV format technically does lots more than this, but scarcely anybody bothers implementing any of that stuff, and I think that includes Python's wave. So we should admit that we don't want WAV we just wanted to store PCM data.

If you've ever run into files (or downloads) named .xls but they were actually CSV files, it's a little similar to that. Do you want an XLS reader? Well, that's pretty complicated, I worked on one (for Gnumeric) and it's quite a ride. But if all you need is to read the CSV file you didn't actually want a XLS reader and it's probably misleading to name your CSV parser "xls".