Debian debates AI models and the DFSG

Anyway, there doesn't seem to be a lot of coverage in the proposal about what a _good_ large model would do in terms of licensing. The seconded proposal says, in the proposal text, that AI models whose weight/operational code is FOSS-licensed and "is trained on data or simulator that is private, proprietary, or inaccessible to the public" and/or "does not provide the original training program" is not DFSG-free. In the proposal introduction it is noted that if the weights are not FOSS-licensed in the first place then it can't possibly be DFSG-free, and that if the weights, training data, and training code are all under DFSG-compliant licenses then the system is DFSG-free. But that leaves a lot of room in between for what should be considered DFSG-free and for what ought to be encouraged.

Ian Jackson says in a reply (https://lwn.net/ml/all/26631.60259.508655.293013@chiark.g...) that they think the proposal obviously means that the training code and data have to be in Debian main themselves, and that Debian Members and ftpmasters will read it this way. This is surprising to me (though, to be fair, I have never gotten around to becoming a Debian Member), especially in the context of extremely large data sets, and I am curious if this is in fact how others read it. (I'm also curious whether Ian intends that the training code and data need to make it into a binary package or if just being in the source package is enough.)

In an email replying to the earlier draft (https://lists.debian.org/debian-project/2025/02/msg00041....) Stefano Zacchiroli does not assume that Mo's earlier draft means that the training code and data must be distributed alongside the runtime AI model, and in fact points out that in practice, the upstream tarballs / source repositories usually won't have them and it will be up to the packager to figure out how to address that. He asks, "Do we repack source packages to include the training datasets? Do we create *separate* source packages for the training datasets? Do we create a separate (ftp? git-annex? git-lfs?) hosting place where to host large training datasets to avoid exploding mirror sizes? Do we simply refer to external hosting places that are not under Debian control?"

A more interesting question to me is the acceptable licenses of the training code and data. If they must go into Debian main, then that question has a straightforward answer; they must be DFSG-free. But you can imagine plenty of ways to make data available in a way that is not "private, proprietary, or inaccessible to the public" yet is not DFSG-free, especially for models trained on creative content (prose in natural languages, code in programming languages, visual art, speech samples, music, medical data, etc.). Taking code as an example, one obvious issue is LLMs that are trained on code that is available to the public but not under a DFSG-compatible license, whether that's unlicensed public code, things under a non-commercial license, or one of the new please-let-us-IPO licenses. This code can't be shipped in Debian main. Some, but not all, of it could go into non-free. Is that permissible for a model in main? Or for natural-language prose, a lot of prominent LLMs have relied heavily on "fair use" arguments in their training, meaning that Debian can't be in the business of redistributing the original texts on which they were trained, but the act of training is (claimed to be) non-infringing (in a certain jurisdiction), and the resulting model is (claimed to be) not a derivative work and so no restrictions on distributing or licensing the model are implied.

The question arises with training code as well as data. What if rerunning training requires CUDA (which is quite common), and the training is implemented in a way that there's alternative (CPU, OpenCL, etc.) backend? CUDA is proprietary but redistributable without cost, and is available in Debian non-free. For this reason, an actual program with a dependency (Depends:, Build-Depends:, or even Recommends:) on CUDA couldn't go into Debian main, but it could go into contrib. If the training program requires CUDA but the runtime AI model does not, can the model go into Debian main?

For this reason I actually don't think there is a ton of distinction between Mo Zhou's proposal and Thorsten Glaser's, despite the latter being _ideologically_ anti-AI. If Ian Jackson's reading is correct, both proposals require that training code and data be suitable for Debian main. Arguably, Ian Jackson's reading is more stringent in practice than Thorsten Glaser's proposal, in that (if I'm reading right) Thorsten's proposal allows a DFSG-free AI with redistributable but DFSG-incompatible training data to go into contrib with its training data packaged up in non-free, but Mo's proposal with Ian's interpretation says that AI models that cannot be packaged up with their own training data "are not seen as DFSG-compliant," which would keep them out of contrib.

If Mo's proposal goes to a vote I think it would be very worthwhile to clarify the intention here, because I expect this to be one of the first practical issues for a serious "more free" competitor to the current crop of proprietary or open-weights LLMs.

Finally, I think there is actually a much stronger anti-AI part of Thorsten's proposal that was underdiscussed, because it affects things that are _not_ AI models. The proposal has Debian take the view that "'generative AI' output are derivative works of their inputs (including training data and the prompt)" and so "Any work resulting from generative use of a model can at most be as free as the model itself; e.g. programming with a model from contrib/non-free assisting prevents the result from entering main." In other words, if someone uses GitHub Copilot or Cursor to help them write some software, then adopting Thorsten's proposal means that Debian cannot consider their software DFSG-free, regardless of what the stated license is on that software, because the underlying models are not DFSG-free.

This strikes me as hugely impractical (which is not necessarily wrong, if your goal is "a hard anti-AI stance"!). These tools are, in practice incredibly popular. While probably not much of the "old guard" writing things like glibc is likely using these tools, I would expect a decent fraction of casual FOSS contributors to be using them, to the point where I assume that by now, at least one package in the current Debian archive has at least some code that was written with the help of such a tool. You would need upstream maintainers to not only commit to refusing to use these tools themselves but also to telling contributors not to use them either, and you would need Debian packagers to tell people who send in patches or Salsa merge requests the same thing.

To Thorsten's credit, he does take exactly this approach (https://mbsd.evolvis.org/permalinks/wlog2021_e20240726.htm). But adopting this proposal would essentially make Debian require every project it packages to adopt this approach, too, and perhaps to engage in a significant audit or scrub of code from between the availability of tools like Copilot and the point of adopting such a policy.

My understanding of the current legal situation is that even for things like the NYT lawsuit against OpenAI, the claim is that the act of training was not fair use, and OpenAI's commercial gain from that act of infringement must be stopped. That is, I think, a different claim from that both the act of training produces a model that is a copyrightable derivative work _and_ that the act of doing inference with that model also produces derivative works of the model, such that all model outputs, and not just the model itself, are subject to the training data's license (or are infringing). Again, to be clear, I see the moral argument for such a position. But it seems likely to me that this is a much more copyright-maximalist position than the actual legal reality, and so it shouldn't be adopted lightly.

Among other things, I think the FOSS community needs to constantly remember that the whole moral deal about copyleft is that it subverts copyright; it doesn't embrace it. The point is that, to the extent that copyright exists, we might as well turn it around to good ends; that's not the same thing as wanting to stretch its scope as far as possible. (As an analogy, consider that despite many FOSS licenses having explicit text about patent licenses, the common position in the FOSS community is that software patents should not exist, not that we should patent everything novel and cool in the FOSS world and assign the patents to Conservancy to better arm their lawyers.) Distros like Debian benefit, for instance, from the idea that facts and data are not themselves copyrightable, in that this is what enables you to reverse-engineer a proprietary system and produce a compatible FOSS system that is not a derivative work. I worry that if Debian argues that the outputs of AI models are derivative works of their training data, it becomes harder to argue that a clean-room reimplementation of a protocol from a reverse-engineered spec is not a derivative work of the proprietary system that was reverse-engineered. Even if you want to incur the logistical trouble of refusing upstreams that have accepted AI-assisted pull requests, it might be a better rationale to say that Debian believes that AI-generated code does not live up to the spirit of the Social Contract and the goals of its developers, rather than to say that Debian believes the licenses inherit in this way.

https://salsa.debian.org/deeplearning-team/ml-policy