Consider the web in 1994

Posted Jul 24, 2024 13:18 UTC (Wed) by geofft (subscriber, #59789)
In reply to: Consider the web in 1994 by b7j0c
Parent article: Imitation, not artificial, intelligence

Not that I disagree with you, but, I don't think this particularly responds to the article. The talk was overall quite positive on AI and specifically encouraged the audience to build good things with it, and the speaker made the point multiple times that he (who actually is an expert in many things!) was able to build software quickly that would have been impractical for him previously because it would take too long to learn. See the parts about Tkinter and speech recognition.

It was just realistic about its limitations and downsides. (It was, if anything, rather soft on downsides, in that it didn't address the energy consumption of AI training and didn't talk about generating images/video at all, and it didn't talk very much about how training a high-quality model from scratch is only within the reach of a few rich corporations.)

Do not consider the web in 1994

Posted Jul 24, 2024 14:45 UTC (Wed) by atnot (subscriber, #124910) [Link]

Yes, agreed. Even with those glaring omissions, I found it the most compelling argument for LLMs I've read yet. Exactly *because* it was missing all of the usual "it's early" apologism, was mostly honest about the problems and didn't engage in any of the "imagine what it will be worth in ten years *wink wink*" book-talking boosterism.

Especially the idea of being able to just have a 4-20GB large file on my computer that has a large portion of the written internet in a somewhat queriable form offline seems pretty compelling. Although I do have reservations about how many useful queries you could get out of one battery charge. I also suspect a compressed text-only crawl of code documentation sites may end up more useful per byte and watt, if someone made one of those. But you never know what you're going to need and it seems like a good idea to at least have it laying around if you have the space and travel regularly.

The code generation stuff seems much less compelling to me. Yeah, I had my fun playing around with it when it launched too and I used it to write stuff in a language I was unfamiliar with at the time which was pretty cool. But then the novelty wore off and I really haven't found it that useful since. When you can usually just search "[thing] examples" or so to get a more reliable answer with just a few more clicks[1]. But I can see the utility, especially offline.

The portrayal of so-called "hallucination", prompt injection and safety as a temporary wrinkle that will soon be ironed out is indeed pretty soft though. Since they are an inherent part of the technology that can not be solved.

Which would be fine if it was just a thing you used to generate example code sometimes. I think this article convinced me of the utility as such a thing better than any other. But it can't just be a minor productivity tool anymore, because it's cost too many millions to make. It has to be the future and upend everything.

[1] Especially after a project I maintain started getting a steady stream of people with very confidently wrong understandings of how things worked wasting our time asking us why methods that did not exist were throwing errors.