What about trust?

Posted Jun 27, 2025 10:46 UTC (Fri) by paulj (subscriber, #341)
In reply to: What about trust? by bluca
Parent article: Supporting kernel development with large language models

An LLM is literally a database of fine-grained features found in a data-set. A quite sophisticated one that captures the relationships between features and their likelyhood, over varying and overlapping spans of features.

The LLM weights are a function of and a transformation of the data that was fed in. No more, no less.

What about trust?

Posted Jun 27, 2025 10:49 UTC (Fri) by paulj (subscriber, #341) [Link] (2 responses)

Oh, and at inference time you are combining that "trained" data-base with your own context window (a fraction the size of the data-set), to have the LLM extend that context window to produce something that 'follows' from the 2.

What about trust?

Posted Jun 27, 2025 11:34 UTC (Fri) by bluca (subscriber, #118303) [Link] (1 responses)

As I said, it's not copying from a database, which was implied by the post I answered to

What about trust?

Posted Jun 30, 2025 22:40 UTC (Mon) by koh (subscriber, #101482) [Link]

There was no mentioning of "database" in the post you replied to. You invented that notion. As with nearly all your comments, they are in a subtle but quite fundamental way: wrong. And yes, LLMs do copy, too, see e.g. Q_rsqrt. It's called overfitting in the corresponding terminology.