Deep learning and free software

There's an interesting analogy here, in that this is not the first time that the free-software community has grappled with the idea that having the source code is not sufficient to edit the operation of a computing system without a lot of resources. A similar problem arises with computer hardware: Even if I had all of the source code to the processor in my television (never mind the one in my workstation!), I couldn't rebuild that processor with a bugfix without a few billion dollars in resources.

I've been at a conference where Richard Stallman addressed that issue -- which is to say, he basically punted and said that Free Software was irrelevant to computer hardware because nobody would be able to fab a replacement chip. Personally, I think that's a shortsighted conclusion (and it was particularly ironic that he said this at a conference of people largely funded by DoD contracts, many of whom likely would be in a position to fab their own chips), but I do find it an interesting datapoint. Is the sort of neural-net training data that can only be generated by running something on the equivalent of Google's CloudTPU clusters for zillions of hours a thing that is limited in its ability to be "free" in much the same way that the Xeon in my desktop is limited by the technological inability of anyone but Intel to make one? And, if so, is the right answer actually to say "that's not a problem we can solve with Software Freedom right now," as Stallman did with hardware?

If so, then the question becomes: What is practical can we do instead? And, of those options, what do we want to do?