Thoughts from a younger generation.. [LWN.net]

Thoughts from a younger generation..

Posted May 18, 2026 21:11 UTC (Mon) by kleptog (subscriber, #1183)
In reply to: Thoughts from a younger generation.. by jpeisach
Parent article: RIP Peter G. Neumann

Well, there's no particular reason why training has to be so expensive. It's just we haven't yet figured out how to do it better. But we will, I'm sure of it.

Reminds me of the story of a telephone exchange being built for a new suburb, three storeys high. By the time they had actually completed construction the hardware required fit in the basement.

Currently there are more GPUs planned to be installed than we have power to switch on. So there is a huge amount of pressure to make the process much more efficient.

to post comments

Thoughts from a younger generation..

Posted May 18, 2026 21:19 UTC (Mon) by dskoll (subscriber, #1630) [Link] (19 responses)

Pretty much all the big AI companies are bleeding cash, and bleeding it really, really fast.

You can bet that once people are absolutely dependent on AI, the price hikes will be breathtaking. Possibly to the point where human software engineers might start looking cost-effective again, just like in The Feeling of Power.

Thoughts from a younger generation..

Posted May 18, 2026 22:14 UTC (Mon) by malmedal (subscriber, #56172) [Link] (18 responses)

This "bleeding cash to serve the LLMs" is pure fiction. They are spending a lot of money to build datacenters, probably too much, but even if they can't handle their loans the serving is profitable enough that the top providers will just go to chapter 11 and continue to operate.

I do expect the prices to go up soon, though. Simply because the demand is so much higher than the supply. Arguably this has already happened. Anthropic's latest model Opus 4.7 uses 40% more tokens on the same input as 4.6. The headline token price is the same, but you'll be paying more.

Thoughts from a younger generation..

Posted May 19, 2026 8:40 UTC (Tue) by paulj (subscriber, #341) [Link] (17 responses)

The top software-tech companies are venturing into modes of business operation they've never been in before. They have _never_ had such vast CapEx programmes before. And they do _not_ have the revenue in hand today to make that CapEx profitable. They have an /expectation/ of future increased revenue.

Some of the big tech companies are funding this huge CapEx increase with profits from their existing, hugely profitable business - i.e., advertising for the likes of Facebook. So, should that /expectation/ of future significantly increased AI revenue not quite arrive, they should be able to cope. And they have that other revenue to allow them to fund R&D/engineering to find other uses for that CapEx if needs be. The ones who funded that CapEx with debt, who did /not/ have another vast revenue source to fund this CapEx with, well they... or else their lenders, may take quite a hit.

Course, AI is the bright future, and it's incredible ability to pour out unending reams of highly accurate information (documents, code, etc.) *of course* is going to result in *huge revenues*. It's obvious! Look at all the white-collar jobs companies will be able to pull and replace with AI. It's going to save *loads* of money! And as everyone knows, cutting swathes of jobs in an economy is *guaranteed* to result in huge economic growth - the AI companies are going to reap the bounties of those savings, oh yes!

Thoughts from a younger generation..

Posted May 19, 2026 10:27 UTC (Tue) by malmedal (subscriber, #56172) [Link] (16 responses)

Almost every new technology has gone through these overinvestment boom and bust cycles (only exception I can think of is that cars don't exactly fit the standard pattern).

It will be very weird if the current AI boom does not get a corresponding bust in the near future. But like computers, the telegraph, railroads etc. it's clearly useful and is going to persist after the bust.

Thoughts from a younger generation..

Posted May 19, 2026 11:13 UTC (Tue) by paulj (subscriber, #341) [Link] (15 responses)

I agree with you generally. With the caveat that the current "AI" over-investment cycle is one of the more extreme cases. E.g., the "AI" investment hype-train may be the only thing that has kept the US economy (and hence rest of the western economies) out of recession the last year or two.

"AI" is not really new technology. What is new is the level of brute-force compute we can throw at it today. It is certainly useful for some stuff - ability to query a vast data-set and extract answers from that data-set tailored to the query context (i.e., prompt), with a number of features from the data-set stitched together in somewhat meaningful ways according to the prompt, is indeed very useful. It also has problems - in particular, the technology doesn't understand what it is doing, and is generally trained to be authoritative and massively over-confident (i.e. "helpful"). It is frustrating the level of faith people will put into the output, in part cause of that trained-in over-confident tone in every AI (that I can find).

I think the hype will subside. Utility will certainly remain, but it's nowhere as high as the industry thinks - particularly in the ultimate metric of sustainable revenue, I greatly suspect. That of itself will lead to further issues for the AI industry - as revenue fails to meet the required levels to pay off the investment and OpEx (energy isn't getting cheaper), they will need to squeeze more from each user, which will deter some users, etc., and so the sustainable revenue may be a lot lower than people think.

We shall see...

But, the gigantic profits simply can not be there. A technology whose success - i.e. massive, gigantic profits - is predicated on making large chunks of the white-collar classes of western economies redundant - is a technology that has to somehow thrive in the recession it must create in order to thrive. A paradox.

Thoughts from a younger generation..

Posted May 19, 2026 13:31 UTC (Tue) by Wol (subscriber, #4433) [Link] (12 responses)

> That of itself will lead to further issues for the AI industry - as revenue fails to meet the required levels to pay off the investment and OpEx (energy isn't getting cheaper), they will need to squeeze more from each user, which will deter some users, etc., and so the sustainable revenue may be a lot lower than people think.

The other big problem that could bite, is that as they try and squeeze more revenue from the customers that bought into the hype, those customers could find themselves under attack from companies that didn't (or startups), and then the AI companies could just drag their customers down with them if the customers can't disengage fast enough.

Cheers,
Wol

Thoughts from a younger generation..

Posted May 19, 2026 14:52 UTC (Tue) by dskoll (subscriber, #1630) [Link] (11 responses)

All we need to do is look at what happened after Broadcom acquired VMWare to understand the pricing strategy likely to be used by AI companies once customers are hooked.

Thoughts from a younger generation..

Posted May 19, 2026 15:25 UTC (Tue) by malmedal (subscriber, #56172) [Link] (10 responses)

Switching from one virtualization platform to another is one of those things that sounds like it should be easy, but isn't. Especially VMWare.

Switching from one LLM to another on the other hand is trivially easy. The providers are working very hard to find some kind of lock-in, but so far without success. Also some of the attempts that are closest to working are problematic, e.g. Claude for Excel depends on Microsoft cooperating.

Thoughts from a younger generation..

Posted May 19, 2026 16:11 UTC (Tue) by dskoll (subscriber, #1630) [Link] (4 responses)

Switching from one LLM to another on the other hand is trivially easy

Perhaps today (though even that, I'm not sure of), but I expect there will be consolidation in the LLM space and we'll be left with 3-4 giants who will all adopt the Broadcom strategy. It wouldn't make sense not to.

Thoughts from a younger generation..

Posted May 19, 2026 16:55 UTC (Tue) by malmedal (subscriber, #56172) [Link]

As I said, they are trying hard. But so far without success.

There's still an escape valve that can't be taken away, We currently have free models that works on modest hardware while being comparable in quality to top commercial models from about 12 months ago. (Qwen 3.6 27B and Gemma 4 31B). And less than six months if you can afford immodest hardware.

Thoughts from a younger generation..

Posted May 19, 2026 19:12 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

As an example of local models, I have two AMD graphics cards with 64Gb total RAM and Qwen 3.6 was able to autonomously write an implementation of LTFS (a filesystem for tape drives) in Go, given the spec from SNIA and the reference implementation in C. In about 7 days.

I just wrote a detailed plan and left the agent to its own devices inside a sandbox. It first used a flat file backend to validate itself against the C implementation and then fuzz its own code. It found several security bugs in the C implementation, huge surprise. It's now working on adding real device support via iSCSI to a real tape drive.

All of this on hardware that is available for casual hobby users, no unobtanium-level hardware. Even _this_ level of competency from models is enough to seriously affect the way software development works.

And free models are still getting better. Qwen 3.6 is just 35B parameters, this is the level where you can feasibly _train_ a model from scratch independently without being a hyperscaler. It'll still cost you hundreds of thousands, but it's not out of reach for a non-profit or a crowd-funded startup.

Which cards?

Posted May 19, 2026 22:10 UTC (Tue) by edgewood (subscriber, #1123) [Link] (1 responses)

Would you mind sharing which AMD cards?

Which cards?

Posted May 19, 2026 22:44 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]

Sure, I have a pair of AMD Radeon AI PRO R9700. They were retailing for $1100 when I bought them last year but went up a bit after the RAMPocalypse. I like them because they don't require any special support and "just work" in the most recent Fedora.

I can see that the agents were using 300W on average for the whole machine (a mid-level Xeon-based server with 128Gb RAM, that I was lucky to get 2 years ago). I probably can do the same at a fraction of the time with OpenAI/Anthropic models, but that's just boring.

Thoughts from a younger generation..

Posted May 20, 2026 6:13 UTC (Wed) by anselm (subscriber, #2796) [Link] (4 responses)

Switching from one LLM to another on the other hand is trivially easy.

Perhaps, but that won't really help you because the LLM operators will all need to raise their prices dramatically if they want to have any chance at keeping their heads above water.

Thoughts from a younger generation..

Posted May 20, 2026 8:47 UTC (Wed) by farnz (subscriber, #17727) [Link] (3 responses)

However, that only applies if you require a frontier model that's only available via a remote API. There are free LLMs like Qwen 3.6 that get you a significant fraction of the abilities of the latest and greatest LLM operator models, and that can be run extremely well on local hardware costing under $10,000, plus whatever your energy cost is for a sub 1 kW tower PC (not a laptop - laptops tend not to have fast enough hardware for this sort of thing).

That puts a hard bound on how much the LLM operators can charge - their maximum price is bound by how much better than models like Qwen, DeepSeek and Gemma they are, and the cost of running those models locally. If you charge enough, people will just switch to local hardware, and local models over time - and you can't make up for that by charging more.

So the question ends up being "do the API-linked models provide enough value that it's worth paying whatever the LLM operators charge for them, or do you get better value from a local model?". And that's assuming that you get yourself tied into needing an LLM to begin with, of course.

Thoughts from a younger generation..

Posted May 20, 2026 11:34 UTC (Wed) by malmedal (subscriber, #56172) [Link] (2 responses)

Rumours are that Anthropic's margins on serving the LLMs are like 70% including what they lose by the free quota. I think this is plausible when you compare prices with the open weights models.

You can check openrouter.ai/models or hugginface.co/inference/ for pricing details.

Cost of LLMs in the cloud

Posted May 20, 2026 14:18 UTC (Wed) by farnz (subscriber, #17727) [Link] (1 responses)

That sounds like a plausible gross margin - the cost of running inference for you given that you have the datacenter, the hardware and the model already is relatively low. The expensive bit is developing new models, and building new datacenters.

And that's why the LLM providers can't raise their prices that far - if they do, then instead of paying extra or doing without LLMs completely, people will move to open models. The only way to avoid that is to charge fairly, or to have a model that's sufficiently better than any open model that it's worth paying your prices.

Cost of LLMs in the cloud

Posted May 20, 2026 14:34 UTC (Wed) by malmedal (subscriber, #56172) [Link]

Yes, I agree. I am arguing against what was said earlier in the thread about the models going away permanently because they are too expensive too run.

Thoughts from a younger generation..

Posted May 19, 2026 14:59 UTC (Tue) by malmedal (subscriber, #56172) [Link]

> The gigantic profits simply can not be there.

They *can*, I am sure the eventual business spend will be gigantic. However I don't know if profits *will* be gigantic. I can see it going multiple ways:

1) One actor gets so much better than the others that it captures almost all the money.

2) Multiple actors which roughly equivalent capabilities compete on price and are forced to sell at cost plus a few percent.
(most likely IMO)

3) Businesses buy hardware to run LLMs by themselves. Least likely, I think.

Hype isn't going anywhere at this rate

Posted May 19, 2026 19:37 UTC (Tue) by jpeisach (subscriber, #181966) [Link]

> I think the hype will subside. Utility will certainly remain, but it's nowhere as high as the industry thinks - particularly in the ultimate metric of sustainable revenue, I greatly suspect.

When employers are pushing developers to "use AI or else we will fire you" and awarding people for more AI use, then this isn't going to happen. I think the best bet for the hype to die down is:

a) data centers are built and start screwing people over
b) the environmental impacts become clear
c) these AI companies start charging people appropriately for its use

As for local LLMs: They tend to be slow, my guess is because of a lack of training? I don't know, I stopped trying after a while because I didn't want to waste my time and spend my laptop battery figuring it out.