Back to basics: Are weights software at all?

You raise excellent questions, and I don't have a good answer, but I suspect at some point this will come down to functionality. The functionality of an email application that includes a PNG icon file is to read email; the PNG icon file, whether modifiable or not, has no real impact on the behavior of the email application. There's a philosophical argument to be made, especially for Debian, but functionally that PNG file has no impact on your ability to read emails.

For an ML model, however, whether it's a GPT or a Bayesian classifier, changing the weights can substantially affect the function of the overall software package. Whether the weights constitute a Turing-complete system is (in my opinion) irrelevant - there are plenty of useful non-Turing-complete systems. What matters is: without those weights, the application using them is unable to function properly. Thus, I think you _have_ to consider them a critical component of the software.