Python finally offloads some batteries [LWN.net]

Python finally offloads some batteries

Posted Mar 17, 2022 18:29 UTC (Thu) by logang (subscriber, #127618)
In reply to: Python finally offloads some batteries by NYKevin
Parent article: Python finally offloads some batteries

I strongly disagree.

This would be like the Linux Kernel developers deciding they don't have time to maintain large swaths of drivers and just dropping them. Then expecting people who need them to get the oot drivers from github using dkms and shift the weight on distributions to package all these drivers and deal with the resulting dependency hell. It would be a _giant_ mess. What would really happen is the drivers would be maintained even worse then they are now and become even more broken, and people who need them would be out of luck.

I think the emphasis should be on growing the project and the number of developers, not splintering off poorly maintained code into situations of even worse maintenance. Dropping modules that are obsolete and which nobody really uses is fine (allowing for the option of a maintainer who cares to step up); but dropping useful modules that people depend upon is not.

IMO, the best solution to the urllib issue would be to absorb the requests module into the standard library and bring all their developers with it. A development model similar to the kernel where a subsystem maintainer collects and sends patches upstream to the core python maintainers. There may be issues with this, but none that couldn't be worked out in the long run. Yes, the requests developers may need to do more work to ensure backwards compatibility, but they'd get help dealing with underlying core changes that affect their module. The python core team may have to make process changes to allow for a higher volume of security changes to stable releases. But after the pain, the benefit is a quality url module in the standard library and urllib could be re-implemented as a thin wrapper over it, and/or dropped after a very long deprecation period.

I've written many simple scripts that run on urllib because I have no interest in dealing with the additional dependency. If requests was in the standard library I'd certainly use it universally, but it is not and in many circumstances that disqualifies it. I suspect this is a common practice. The core team have a responsibility not to break such widespread usage with no real alternative for developers.

to post comments

Python finally offloads some batteries

Posted Mar 17, 2022 21:17 UTC (Thu) by NYKevin (subscriber, #129325) [Link]

> but dropping useful modules that people depend upon is not.

If nobody is willing to maintain the code, it is de facto unmaintained. You can slap a "maintained" label on it all you like, but that does not cause maintenance work to get done.

> but they'd get help dealing with underlying core changes that affect their module.

I believe we've all learned the hard way that core changes should be backwards-compatible, and so this kind of help should not be required in the vast majority of reasonable cases.

> But after the pain, the benefit is a quality url module in the standard library and urllib could be re-implemented as a thin wrapper over it, and/or dropped after a very long deprecation period.

They certainly can't do that until requests stops depending on urllib.

Python finally offloads some batteries

Posted Mar 18, 2022 8:25 UTC (Fri) by milesrout (subscriber, #126894) [Link] (2 responses)

It's a bit different with Python, because the language itself is - pretty much - stable. They don't go out of their way to break core language interfaces in every version.

Linux developers, on the other hand, have no qualms about changing core interfaces in any old version. They don't exactly *go out of their way to*, especially where it would complicate backporting fixes to older versions. But look at the discussions happening around list iterators. They clearly are willing to change fundamental interfaces quite readily.

This means that out-of-tree modules for the kernel are in a very different level of support (none at all) than third-party modules for Python. The whole *point* of Python is a stable interface against which to write third-party modules! That's what a language *is*!

Python finally offloads some batteries

Posted Mar 18, 2022 15:46 UTC (Fri) by logang (subscriber, #127618) [Link] (1 responses)

That seems like wishful thinking at best. It is not nearly as stable as you think it is. Most python packages only support of subset of python reasons for lots of good reasons. If large swaths of the python library are now in PyPi they also now gain complicated dependencies between them as well. Maybe the kernel's driver API experiences more churn, but the point remains.

As NYKevin pointed out the requests module depends on urllib (yikes) so if that module gets removed from the standard library then you've broken requests for the latest release of python.

If tons of important modules are ejected then the core teams haS to stop removing or deprecating things to avoid the same dependancy hell the kernel would have with out of tree drivers.

Python finally offloads some batteries

Posted Mar 18, 2022 17:40 UTC (Fri) by NYKevin (subscriber, #129325) [Link]

> Most python packages only support of subset of python reasons for lots of good reasons.

In my experience, this "subset" is usually of the form "version 3.x or later" for some value of x (or, for a handful of very old libraries, "version 2.7.x only"). I don't believe I have seen a whole lot of libraries that set a maximum version, other than the ones which were never ported to 3.