PEP517 [LWN.net]

PEP517

Posted May 23, 2025 1:48 UTC (Fri) by sam_c (subscriber, #139836)
In reply to: PEP517 by geofft
Parent article: Recent disruptive changes from Setuptools

> Can you explain what you mean by "Pythonic Makefiles"?

There's a lot of software that used `setup.py` as an entrypoint and relied on it being able to install to arbitrary locations on disk, often using it only because the software was written in Python and they felt it was natural that the build system also be in Python, despite it not needing to be.

It would also often (as the article covers) handle the testsuite but other administrative tasks, as well as "more elaborate" installation.

> The reason setup.py was able to do this is that it was just an ordinary Python script ...

Precisely.

> Is there a path forward where your setup.py script does not use setuptools (which is a Python-packaging-specific tool) but uses generic Python functionality like open("/lib/...", "w")?

Sure, that can be done - but if the software is installed via a wheel, that's not going to work, because there's no way to map extra data file locations in the wheel to real locations on the system. And none of the authors of these pieces of software relying on this functionality working ever seem to be aware of the problem.

i.e. All of it is solvable, it's just that it's a lot of work across a lot of different projects, and I don't think many of them are aware of the need to even do it. The software relying on this generally doesn't want to be installed as a wheel (it's usually "written in Python, not a library that should be imported"), they've just been piggybacking on setuptools and the setup.py entrypoint for years.

to post comments

PEP517

Posted May 29, 2025 18:33 UTC (Thu) by zahlman (guest, #175387) [Link]

> And none of the authors of these pieces of software relying on this functionality working ever seem to be aware of the problem.

> i.e. All of it is solvable, it's just that it's a lot of work across a lot of different projects, and I don't think many of them are aware of the need to even do it.

My experience of discussion in Python packaging circles has been: people are very aware that Python code gets used and distributed in a wide variety of ways beyond the standard packaging system. But the people who try to "solve packaging" have very little idea of how to track down major projects that work this way. It often seems like the best they can do is to trawl through PyPI looking for sdists that will break in the future and send emails to whatever maintainer email addresses are listed in the metadata. If they don't publish an sdist then there's no clear way to figure out what their needs are to service them better.

A bit of speculation as well as *personal* investigation from Paul Moore (of the pip team) led to his submission of PEP 722, subsequently rejected in favour of PEP 723 by Ofek Lev (author of Hatch). That makes life easier for people who write single-file scripts with dependencies. But there are still huge unanswered questions about several other kinds of projects, e.g. large monorepos.

I had a bunch of post IDs saved on the Python Discourse forums about this general topic, but I lost those when I got kicked out (https://zahlman.github.io/posts/2024/07/31/an-open-letter... ; you may also remember me from https://lwn.net/Articles/988894/) and my local bookmark collection is quite large and hard to organize.

If you personally know people (or projects) that use deprecated workflows around setup.py, I'd heavily encourage you to warn them - and link to https://blog.ganssle.io/articles/2021/10/setup-py-depreca... . If you know projects that use only Python but publish only a "legacy" sdist instead of wheels, https://pradyunsg.me/blog/2022/12/31/wheels-are-faster-pu... is a solid reference.

If they otherwise make inappropriate use of setup.py (e.g. to convey metadata that could be in a pyproject.toml), probably the best way to warn them is to point out what happened to Requests with Setuptools version 78 (and 72): twice in less than a year, one of the most popular projects out there, written in pure Python, had its packaging break twice in easily avoidable and completely unnecessary ways. Packaging is part of maintenance, too.

But for those who don't publish to PyPI and whose projects aren't a dependencies for anything else, there might not actually *be* a need to do anything - except to stop upgrading local installations of Setuptools.