Thoughts from a younger generation.. [LWN.net]

Thoughts from a younger generation..

Posted May 20, 2026 6:13 UTC (Wed) by anselm (subscriber, #2796)
In reply to: Thoughts from a younger generation.. by malmedal
Parent article: RIP Peter G. Neumann

Switching from one LLM to another on the other hand is trivially easy.

Perhaps, but that won't really help you because the LLM operators will all need to raise their prices dramatically if they want to have any chance at keeping their heads above water.

to post comments

Thoughts from a younger generation..

Posted May 20, 2026 8:47 UTC (Wed) by farnz (subscriber, #17727) [Link] (3 responses)

However, that only applies if you require a frontier model that's only available via a remote API. There are free LLMs like Qwen 3.6 that get you a significant fraction of the abilities of the latest and greatest LLM operator models, and that can be run extremely well on local hardware costing under $10,000, plus whatever your energy cost is for a sub 1 kW tower PC (not a laptop - laptops tend not to have fast enough hardware for this sort of thing).

That puts a hard bound on how much the LLM operators can charge - their maximum price is bound by how much better than models like Qwen, DeepSeek and Gemma they are, and the cost of running those models locally. If you charge enough, people will just switch to local hardware, and local models over time - and you can't make up for that by charging more.

So the question ends up being "do the API-linked models provide enough value that it's worth paying whatever the LLM operators charge for them, or do you get better value from a local model?". And that's assuming that you get yourself tied into needing an LLM to begin with, of course.

Thoughts from a younger generation..

Posted May 20, 2026 11:34 UTC (Wed) by malmedal (subscriber, #56172) [Link] (2 responses)

Rumours are that Anthropic's margins on serving the LLMs are like 70% including what they lose by the free quota. I think this is plausible when you compare prices with the open weights models.

You can check openrouter.ai/models or hugginface.co/inference/ for pricing details.

Cost of LLMs in the cloud

Posted May 20, 2026 14:18 UTC (Wed) by farnz (subscriber, #17727) [Link] (1 responses)

That sounds like a plausible gross margin - the cost of running inference for you given that you have the datacenter, the hardware and the model already is relatively low. The expensive bit is developing new models, and building new datacenters.

And that's why the LLM providers can't raise their prices that far - if they do, then instead of paying extra or doing without LLMs completely, people will move to open models. The only way to avoid that is to charge fairly, or to have a model that's sufficiently better than any open model that it's worth paying your prices.

Cost of LLMs in the cloud

Posted May 20, 2026 14:34 UTC (Wed) by malmedal (subscriber, #56172) [Link]

Yes, I agree. I am arguing against what was said earlier in the thread about the models going away permanently because they are too expensive too run.