Python packaging targets [LWN.net]

Python packaging targets

Posted Mar 2, 2023 8:27 UTC (Thu) by callegar (guest, #16148)
In reply to: Python packaging targets by rra
Parent article: Python packaging targets

This would be easier if you could

apt install some-python-pkg-1.0

/and/ also

apt install some-python-pkg-2.0

without having the latter replace the former. It would allow the distro to provide the new package, while temporarily maintaining stuff that might get broken with the old package.

In fact, given the python approach of not allowing this kind of thing, I now tend to think that a few things would be simpler if there was a policy so that at least the "major" version of the package was formally required to become a part of the package name. Even without a formal requirements, for some packages this seems to be already done.

to post comments

Python packaging targets

Posted Mar 2, 2023 17:58 UTC (Thu) by mathstuf (subscriber, #69389) [Link] (4 responses)

How do you propose these two package co-exist on the disk? If both are installed, what does `import some_python_pkg` do?

Python packaging targets

Posted Mar 2, 2023 21:19 UTC (Thu) by nix (subscriber, #2304) [Link] (2 responses)

Sorts and picks the highest version number (using standard version sorting), unless a suitable piece of metadata in the installed package directory (or import line, since obviously we'd want to augment that and allow version ranges!) specifies otherwise.

This is not rocket science. It's just a tricky compatibility and syntax thing.

Python packaging targets

Posted Mar 3, 2023 0:43 UTC (Fri) by mathstuf (subscriber, #69389) [Link] (1 responses)

> This is not rocket science. It's just a tricky compatibility and syntax thing.

Isn't this kind of the use case that this whole article series has been about though? It might not be rocket science, but it certainly doesn't seem easy. We (as a species) actually seem to be better at rocket science if you were to ask me…

Corner cases that come to mind:

- Two libraries provide metadata to describe their dependencies and they disagree about what version to use.
-- What is preferred?
-- Does every library/application need to specify their full transitive requirements?
- How do I prod random script developers to provide this information? Can it be specified in-source or is it sidecar metadata?
- If they are allowed to co-exist, what does `sys.modules` look like now?

Python packaging targets

Posted Mar 3, 2023 13:30 UTC (Fri) by nix (subscriber, #2304) [Link]

> - Two libraries provide metadata to describe their dependencies and they disagree about what version to use.

I don't see how (in the absence of hermetic namespacing) this can be anything but a runtime error. Of course, again, other languages can do this easily...

> - How do I prod random script developers to provide this information? Can it be specified in-source or is it sidecar metadata?

I was assuming both: a file in the package directory versioning lots of things at once, or extra syntax for import. Sidecar seems totally wrong: this is something *the package* is expecting, not something the admin is supposed to be nailing down (how could the admin possibly know the right answer?)

Obviously if a given import doesn't have a version nailed in, the system works exactly as it does now: anything else would be a compatibility disaster. (Needless to say I have written none of this code so I'm totally bikeshedding and waving away all the no-doubt-horrible problems!)

Python packaging targets

Posted Mar 2, 2023 21:19 UTC (Thu) by atnot (guest, #124910) [Link]

Like in many other language ecosystems (and even sort of python with virtualenv), there would be some shim that tells the interpreter/compiler what packages the program should have available in what version. This would be read from some sort of file provided along with the program or source code.