|
|
Log in / Subscribe / Register

PEP517

PEP517

Posted May 22, 2025 12:56 UTC (Thu) by zahlman (guest, #175387)
In reply to: PEP517 by geofft
Parent article: Recent disruptive changes from Setuptools

> My first reading of the comment linked from the comment thread you linked (https://github.com/pypa/packaging-problems/issues/576#iss...) is that while `setup.py install` is happy to put files in an OS-owned directory, pip is not, and for reasons like this, "distro maintainers hate pip because it does NOT do what distro maintainer want." If I'm reading this right, this is exactly backwards: pip's refusal to put files in OS-owned directories started out as a patch from distro maintainers overriding the behavior of upstream pip, because they specifically wanted pip to stop overwriting OS-owned files. That change slowly got upstreamed in a form that hopefully let the OS maintainers drop their patches.

It's... a lot more complicated than this, unfortunately. First, to clarify a few things about your description:

Pip originally was happy to "put files" in *certain* OS-owned directories. Specifically, it will put them within any *Python environment*. If you tell it to install for the system environment, and give it sudo rights, it would obediently write files to (typically) `/usr/lib/pythonX.Y/site-packages`. Without sudo, it could still write to a user-specific directory; if you run the system Python as a regular user, Python can still `import` from there.

Eventually, distros realized that not only do those system-level installs cause obvious problems for the system package manager, but even a user-level install can cause problems for scripts that come with the system. This is especially problematic when you have a package manager (or interface thereto) *written in Python* that still does useful things when run without sudo. Hence https://peps.python.org/pep-0668/ (I think that's the "upstreamed patch" you have in mind).

Notably, though, this change was *entirely orthogonal* to the transition to the new PEP 517 system.

Separately, the wheel format can *only* put files *into the current Python environment* - as defined by the `sysconfig` standard library module. (By "current" I mean the one that pip is running under.) It can't ordinarily put them in /etc - or ~/.config , even though that isn't system-owned.

However, pip *can* effectively put files in those places (assuming filesystem permissions) if it installs from an sdist and builds from source - because that will request the build backend to do whatever is necessary to make a wheel, including running arbitrary Python code from the package. In the case of Setuptools, it's effectively equivalent to `setup.py install`. (At least the last I checked, pip run with sudo *will not drop permissions* when it invokes the build backend!) But this is not playing nice with either the Python ecosystem *or* with the Linux distro's package management.

----

But the comment is talking about something completely different. It's not about end users causing problems for the distro maintainer by using pip to install a third-party package. It's about the distro maintainer *trying to use* pip to *create a system* package.

They might try to do this by installing the code into a staging folder with pip, then using another tool to pack that tree into .deb or .rpm or whatever. As described above, theoretically the pip/Setuptools combination could be wrangled to write files to /path/to/staging/etc or whatever is needed, even while the Python environment is nominally rooted at /path/to/staging/usr. But it appears that this doesn't work with explicitly provided Setuptools features - you'd have to leverage the fact that you can write arbitrary code in `setup.py`, and do your own file I/O etc. And that code might not play nice with *other* packaging flows....

(Also, pip doesn't really know how to install cross-environment. It figures out installation paths based on the Python executable that's being used to run it. Newer versions have a `--python` argument to try to fix this - but it works by tracking down the other Python executable and *spawning it to re-run the pip code*.)


to post comments

PEP517

Posted May 23, 2025 1:37 UTC (Fri) by sam_c (subscriber, #139836) [Link]

I'm not talking about using pip here (just invoking `setup.py install` or whatever will have the same issue) but rather any software where upstream expect data files to be installed to standard locations using their own setup.py, and all of their documentation refers to it and so on, which will stop working. The Python community hasn't done a great job of communicating this change.

But I'll reply more to the other comment rather than duplicating it.


Copyright © 2026, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds