LWN: Comments on "New terms of service for PyPI"

Some context

zahlman — Thu, 03 Apr 2025 19:43:48 +0000

Since I'm catching up on LWN a bit today, I thought I'd post a few random things I know about the situation that others might find relevant:

1. Kiraly is no stranger to the politics of the open-source world. He leads the "sktime" project (something to do with analyzing time-series data with scikit-learn in Python), which apparently had to deal with a hostile fork; and in the fallout, he ended up facing Code of Conduct charges filed with NumFOCUS (a nonprofit that supports a lot of scientific-Python OSS development, including Numpy, Pandas, Matplotlib, Jupyter etc. as well as the Julia language and many other things). It comes across to me that his mode of communication in that thread is typical. Personally I think the community is better off for having people willing to levy such criticism, even when it turns out to be misguided or irrelevant.

2. Paul Moore, in my experience, has a gift for understatement and humility. His "close involvement with the packaging community", for the record, is mainly that he is a major contributor to / maintainer of Pip.

3. PyPI, per the PSF's reporting, serves on the order of 600 petabytes of data per year, graciously handled by an in-kind donation from Fastly. If assessed at 2 cents per gigabyte (the going bulk retail rate from CDNs like AWS, the last time I checked) this would amount to a few times the PSF's entire operating budget. It's not at all a small concern, and I would agree it's a good thing that the volunteers maintaining it can have this much sovereignty. But I think we would be much better off if someone could find the resources to staff it properly, and if we could reduce that download burden.

(A lot of things could help: offering slimmed-down distributions for major packages like NumPy, e.g. by allowing tests and documentation to be omitted or making the functionality more modular; enabling better compression methods and writing the standards language to get installers and build systems to work with that; making it easier to set up local private indexes; teaching workflows that don't redundantly download many copies of Pip and Setuptools; figuring out if/why Pip's cachings is being defeated in that regard....)

Difficult balance and alternatives

mathstuf — Fri, 21 Mar 2025 09:12:07 +0000

> So are there people that managed to self-host compatible repositories and somehow modify or configure pip to point to them?

One can host wheels with a static HTTP server, so yes. Additionally, forges may provide PyPI-compatible registries for your packages as well (e.g., at least GitHub and GitLab do so for Python).

Difficult balance and alternatives

GNUtoo — Thu, 20 Mar 2025 16:26:22 +0000

One of the issue with rules is that the more precise they are, the more they increase false positive (people that found ways to respect the rule by the letter while managing to do things that the rules were meant to forbid) and negative (people doing things that are meant to be OK but forbidden by the rules), and the more lax they are, the more they are subject to interpretations by the people who are charged to enforce the rules, and so their application can depend on the people doing the enforcement at a given time.

Here it seems to be lax enough to allow to take a lot of context into account, and there are examples as well, which helps a lot to clarify things. So we seem to have some good balance here.

Note that I didn't register yet to pypi so I'm not sure exactly what service it provide, but I read the new terms of services to understand if it was worth applying on behalf of a project I ended up co-maintaining.

In (from https://policies.python.org/pypi.org/Acceptable-Use-Policy/) we have:

> Posting text, imagery, or audio content glorifying or containing a graphic depiction of violence toward oneself, another individual, group, or animal

Does that means that many free software games are out of the scope of pypi? Are games that have the issue mentioned above typically referenced somewhere else, or not care about pypi?

In my case the package I co-maintain is not a game, so it doesn't fall into that (it's an application that interacts with an online service). I also don't know if this part is a good or bad thing, so I've no objections to it.

Another question is if it is possible to avoid pypi completely and/or how hard it is to setup another compatible repository. The use case would be to have only 100% free packages hosted/referenced.

pip install can at least refer to specific URL like with 'pip install git+https://some-forge/project-group/project', and PEP 508 allows some URLs, but I guess that at some point in the dependency chain, it will depend on packages that take their dependencies from pypi. And making sure to always have the latest revision of a dependency probably increase the amount of work.

So are there people that managed to self-host compatible repositories and somehow modify or configure pip to point to them? Or are there ways to somehow filter packages/dependencies on the license?

This reminds me ...

paulj — Wed, 19 Mar 2025 12:59:49 +0000

Would I be correct in thinking you could be banned from Pip simply cause of things outside of your control - like where you were born and live?

This reminds me ...

ballombe — Wed, 19 Mar 2025 10:42:17 +0000

Yes.

On the other hand, I would argue that the stake are much higher. Being banned from Pip can have real life consequence. I would not like to invest in a language where I can be banned from the de facto canonical archive, and where there are strong expectation of software being available there.
That is the contradiction : one set up an archive with the stated purpose that everything will be available there,
and then suddenly there is an "except".

One feature of free software is that we avoid having power over other people, because they can just fork or ignore us.
Centralization effort like canonical package archive create power structures that are dangerous.
Centralization is a costly convenience.

They weren't already doing that?

chris_se — Fri, 14 Mar 2025 09:59:26 +0000

I agree with you, and I totally don't get the drama about this at all.

A clause in the ToS that they can remove access to who they want is completely reasonable, because it's a purely legal statement. I view that as similar as all those disclaimers of warranty in software licenses - they state that users of free software are completely on their own legally. But there still is the social expectation of a well-run free software project that it's reasonable to at least report bugs to the project - and most projects are interested in fixing bugs at least in general, even though they may not do so for specific bugs due to technical, "man"power, or other constraints. (Or at least not immediately.) And I don't see people arguing for "write in your free software license that the author must fix bugs", even though projects fixing reported bugs is generally a social expectation. Worst case, because it's free software, if a project does stagnate, it will always be possible to fork it and do it better.

There is a social consensus that PyPI is open to reasonable contributions, and should the PSF ever do something that violates that consensus, and aren't willing to reconsider, then people will create alternatives to PyPI.

But being outraged that people in the PSF don't want to be responsible for monetary damages, and don't want to have to spend resources to have to fight bad actors with a lot of time on their hands in court, isn't very productive in my opinion. I view the clause more of a "the legal system is not fit for the purpose of handling these specific kinds of disputes, and since we are putting in the money for hosting all this stuff for free, we want to be legally safe".

They weren't already doing that?

raven667 — Fri, 14 Mar 2025 04:02:29 +0000

I agree that understanding this up front as a platform owner and making it clear to everyone else saves misunderstanding, angst and embarrassment in the long run, but I think well meaning people are conditioned to be a little more accommodating than they should be and sometimes shy away from the responsibility of making their own judgements (which could be wrong!) by hiding behind a thick rulebook, leaving them open to bad actors willing to game the system. Make the judgement call and as a consequence be judged by the community and it will all find balance.

This reminds me ...

JoeBuck — Fri, 14 Mar 2025 00:49:02 +0000

... of our discussion when we formulating the rules for egcs. When could we ban someone from the community? Should we be able to? Should we have a specific set of rules, or just require a supermajority of the steering committee? We would up deciding that anyone can be banned by a 3/4 vote from the steering committee. When a few people complained, I replied that we didn't want to be stuck if someone found a new and creative way to cause damage.

It turns out that we only ever banned one person (in the time that egcs was independent, before it was re-merged with FSF gcc). The reason was that after his patches were repeatedly declined, he started threatening the release manager, in private mail, up to and including "I know where you live". Those who only saw his behavior on-list might find him annoying and argumentative but not much worse than some others. Fortunately after he was banned, he appeared to back off, so as far as I know law enforcement was never involved.

So yes, I can see why PSF may feel that they need to reserve the right to do what's needed for self-preservation.

They weren't already doing that?

NYKevin — Thu, 13 Mar 2025 21:46:44 +0000

> PSF has the right to suspend or terminate your access to all or any part of the Website at any time, with or without cause, with or without notice, effective immediately. PSF reserves the right to refuse service to anyone for any reason at any time.

I'm honestly pretty shocked to learn they didn't already have a sentence like this in their ToS to begin with. It is impractical to promise otherwise, especially for a free service. You need the ability to cut off spammers and other folks who are misusing the system, without having to worry about what a bunch of lawyers are going to think of it.

It is *helpful* to give some examples of the general kinds of things that are likely to get someone's account terminated (e.g. "no distributing malware or [list of various other problematic things]," which I'm sure is also in there somewhere), but it's not wise to promise that such a list is exhaustive. People are always coming up with ingenious new ways of abusing things, and you don't want to be stuck trying to figure out whether your existing terms cover some new attack vector. Yes, you can write a catch-all "if you cause problems for the service" clause, but that's so broad that it's hardly any better than writing "for any reason at any time," and the latter is more bulletproof if you do get sued.

The value of PyPI is more than just the packages

kpfleming — Thu, 13 Mar 2025 10:43:30 +0000

> Nearly all of the work that has gone into PyPI has been done by unpaid volunteers; the vast majority of the packages stored there come from unpaid community members as well

While this is certainly true, it glosses over the fact that *operating* PyPI is incredibly expensive and is not done by volunteers or community members. There is tremendous value in the collection of Python packages present on PyPI, but there is also tremendous value in the existence of a central, well-known, repository where Python packages can be found.

This sort of discussion has happened before, when other major package repositories underwent ownership changes or policy changes; in every case it's been said that package publishers and consumers are free to use any repository they wish, but that the community itself finds value in having a single place to publish and consume packages - that comes at a cost.