Training data, weights and model [LWN.net]

Training data, weights and model

Posted Oct 29, 2024 15:27 UTC (Tue) by paulj (subscriber, #341)
Parent article: OSI readies controversial Open AI definition

To start off with, I think data-sets for training are obviously incredibly important. I think having open data sets available for training fully-reproducible and open "AI" models will be incredibly important to society, sooner rather than later. I think it will be so important that ultimately societies will do 1 or both of:

1. Provide public funding for projects to gather, collate, store and make available data for public use (in various arenas);
2. Regulate the use of large-scale corporate data-sets for training, potentially making provisions for fair-use access under reasonable terms

That said, I have to take with a common theme in some of the objections that suggest that a piece of software can not be considered open if some significant data inputs to that software are not open, e.g. as per Tom Calloway's suggestion:

> "If we let the Open Source AI definition contain a loophole that makes data optional, we devalue the meaning of "open source" in all other contexts."

Or the suggestion that ANN software is not open, even if different sets of weights are available, if input data is not open, e.g.:

> For example, julia ferraioli wrote that without including data, the only things that the OSAID guarantees are the ability to use and distribute an AI system.

The former suggestion, that at least all significant possible input data to some software must be open, for the software to be considered open, is not something the Free Software world has stood fast on before. We did not require that every possible C programme must be open for GCC to be considered open. There are a number of Free Software game engines, for which game-data packs exist that are proprietary - e.g., often the original game that spawned the game engine, along with other proprietary games using the same engine. We don't consider a GPL Quake or Descent game engine to not be Free Software because there are game packs that are non-free, do we?

The latter suggestion, that an ANN is not trainable without the original input data is technically wrong. The ANN is eminently trainable. At worst, you start with random weights. If you've been given a set of weights from a previous training run, you've actually got a _lossy compressed form_ of the original data-set, that will be a much much better starting point than otherwise (given you appear to be interested in the original data set).

We need to be clear there are a number of different components here:

1. The ANN software
2. The model specification (if not hard-coded in 1)
3a. The model state (the parameters and activation states)
3b. The input training data that created 3a

3a is a lossily compressed form of 3b.

1 is useful on its own.
2 is useful on its own (either described in some DSL to be used by some instance of 1, or described in literature)
1 + 2 are very useful.

1 + 2 + 3a are very very useful, and you can add your own input data set and train further from there. I.e., you are in a much better position starting your training from here, than from 3b, given you are demanding 3b to be open to you. 3a saves you a *lot* of compute!

3b will ultimately be required in many context for social reasons, as I mention at the start, but I see that as a different issue and one that will be solved by public means.

I think it will be extremely hard to /require/ that 3b is made available, just for logistical reasons, given the vast volume of data - even if the input data is already public. And if the data is already public (e.g. web crawls), you can assemble it yourself. Obviously 1 and 2 can and should be publishable under Free Software licences, without demanding that all cases of 3b must be made available under a Free Software licence.

So it comes down to whether someone should be able to publish a 3a, while not publishing 3b, and still be able to claim some "Open Source" label. If 3b is merely collated from public data, and the methodology to replicate that collation is obvious (web scraping, say) or otherwise described, perhaps there should be some accommodation that allows such 3a data-sets to be distributed along with Free Software by Free Software entities?

For other cases, where 3a is a distillation of some other data-set that not be replicated from public data, should this be treated as code or data? Data is what it is really. What if 3a is a distillation of personal data that could not be distributed otherwise under other laws (e.g. GDPR)? Should we say that no Free Software entity should be able to distribute a useful data-set, that could be used to run some Free Software LLM and do useful things for users?

What if the 3a lossy-compressed data is the /only/ way that the 3b data-set could be legally distributed anywhere in the world? E.g., 3b is personal medical data, and 3a is the model state of an ANN that uses homomorphic-encryption precisely to ensure that the very sensitive data of 3b can still be distilled to a compressed data set of 3a that can then be used by ANNs around the world? Is Free Software to be cut-off from such advances forever?

Training data, weights and model

Posted Oct 30, 2024 12:45 UTC (Wed) by kleptog (subscriber, #1183) [Link] (1 responses)

I think that for most people, 3a (the model state) is the "preferred form of modification" anyway. Having all the training data is a nice idea, but useless in practice because you can't meaningfully review all the data anyway. And just retraining with the same data doesn't give you anything new. Just about anything you might want to do you can do with just the weights: fine tuning for the results you want.

The case for training on private datasets (like medical data) is a different issue. Obviously you can only train an AI to detect lung cancer if you have access to datasets with information about people who may have lung cancer. Ideally you'd like such datasets to be available on a Reasonable and Non-Discriminatory basis. This sort of access is already available for other purposes, I don't see how AI requires anything new here.

(Sometimes I wonder if in the future we'll have AI psychologists that specialise in diagnosing issues in AIs and recommend treatments i.e. new fine-tuning to reduce issues. Maybe the AI psychologist will also have an AI to help them.)

Training data, weights and model

Posted Oct 30, 2024 12:54 UTC (Wed) by mathstuf (subscriber, #69389) [Link]

> (Sometimes I wonder if in the future we'll have AI psychologists that specialise in diagnosing issues in AIs and recommend treatments i.e. new fine-tuning to reduce issues. Maybe the AI psychologist will also have an AI to help them.)

Asimov had "robopsychologists" doing things just like that.