Well this is interesting

Posted Mar 2, 2026 18:31 UTC (Mon) by valderman (subscriber, #56479)
In reply to: Well this is interesting by roc
Parent article: Gram 1.0 released

Unlikely, since the models running in data centres are an order of magnitude larger (and thus more energy consuming) than the ones you can run on most consumer GPUs.

to post comments

Well this is interesting

Posted Mar 3, 2026 6:32 UTC (Tue) by roc (subscriber, #30627) [Link] (7 responses)

In thinking mode, weaker models generally require more tokens to get to the same result, sometimes a lot more, so can end up being more expensive. Also, best to not make assumptions about the size or (in)efficiency of models running in big-tech datacenters; you might be surprised. As Cyberax said below, there are huge efficiency gains running things at large scale and sharing hardware across users.

It is at least *not guaranteed* that running a model locally helps the environment. Unless, as I said, you're powering it with your own renewable electricity, in which case good for you.

Well this is interesting

Posted Mar 3, 2026 12:47 UTC (Tue) by jpeisach (subscriber, #181966) [Link]

Huh. I see.

Well, I do feel like Ollama is the best option anyway because otherwise you have to start dealing with subscriptions, limits, and third parties.

Well this is interesting

Posted Mar 5, 2026 10:51 UTC (Thu) by lproven (guest, #110432) [Link] (5 responses)

> It is at least *not guaranteed* that running a model locally helps the environment.

I feel someone needs to point out that the most ecologically-friendly model is to run it in your head, and not use any LLMs at all ever.

This is not some way-out position. The pro-LLM lobby is loud and pervasive, but what I hear from readers of the Register and other techies is widespread profound LLM scepticism. I have no use for the things at all. There is nothing they can do for me that I can't do better myself.

So, "no LLLs" is my own personal policy. The *only* one I permit to run on any of my machines is the Firefox local translation feature, and I am increasingly considering replacing Firefox with Waterfox on macOS. I have already done so on Linux.

Well this is interesting

Posted Mar 5, 2026 15:01 UTC (Thu) by dskoll (subscriber, #1630) [Link]

I agree with this position. I do use AI tools locally, but they are not LLMs. Specifically, I use Whisper to transcribe audio to text.

Well this is interesting

Posted Mar 5, 2026 18:42 UTC (Thu) by roc (subscriber, #30627) [Link]

That's fine, but there are some amazing things going on.
https://normalcomputing.com/blog/building-an-open-source-...
is a recent example.

Well this is interesting

Posted Mar 5, 2026 21:49 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

> I feel someone needs to point out that the most ecologically-friendly model is to run it in your head, and not use any LLMs at all ever.

A person in the US has about a 50 ton a year CO2 footprint. Are you sure that the wetware LLM is going to be more efficient?

Well this is interesting

Posted Mar 6, 2026 8:59 UTC (Fri) by anton (subscriber, #25547) [Link] (1 responses)

Pretty sure. That person has that CO2 footprint whether he/she uses its wetware or lets it atrophy by using artificial LLMs. But using the articificial LLM is going to have an additional CO2 footprint.

A potential environmental benefit of using your wetware may be that thinking about the problem may take longer, and you may have less time for activities that harms the environment more than programming does.

Well this is interesting

Posted Mar 6, 2026 15:07 UTC (Fri) by Wol (subscriber, #4433) [Link]

The other massive advantage is you may have an answer that is provably correct and reproducible, rather than having to repeatedly run an expensive project that says "this answer is probably correct". Yes, even if you're trying to solve the Travelling Salesman on a daily basis (and I am).

Cheers,
Wol