Consider the web in 1994

Posted Jul 24, 2024 14:32 UTC (Wed) by Paf (subscriber, #91811)
In reply to: Consider the web in 1994 by paulj
Parent article: Imitation, not artificial, intelligence

They’re incredibly, absurdly useful for this stuff. They do not need to be 100% accurate to be incredibly useful.

Most tutorials online are also a bit wrong or out of date. The LLM synthesizes from many and is generally, in my experience, stronger and more accurate than most individual articles.

It’s easy to say things like Bayesian parrot, but whatever label you attach they are in practice *really good* at this stuff. That’s from substantial personal experience.

Consider the web in 1994

Posted Jul 24, 2024 17:10 UTC (Wed) by legoktm (subscriber, #111994) [Link] (2 responses)

I'll echo this. LLMs are, to my surprise, quite good at generating code, and crucially, code is something we have extensive tooling and practices to verify the correctness of.

I feel people are falling into the AI effect trap: "The AI effect occurs when onlookers discount the behavior of an artificial intelligence program as not 'real' intelligence." No, LLMs are not anywhere close to human intelligence, that doesn't stop them from being quite useful regardless.

Consider the web in 1994

Posted Jul 25, 2024 4:08 UTC (Thu) by raven667 (subscriber, #5198) [Link] (1 responses)

There may be some kinds of repetitive boilerplate code which is well represented in a training data set which can then be reproduced by an LLM more or less correctly, but for anything else the LLM isn't going to be able to understand the requirements or the goals and may struggle to make a response that is syntactically valid, let alone solving your problem. Even when the response superficially seems correct, you will end up maintaining and refactoring the code produced, that you didn't write and may not really understand, when you have to find the bugs when the output is incorrect. This may _feel_ very productive in the beginning and generate great gobs of code but once the tech debt bill comes due it may not be that productive after all.

Consider the web in 1994

Posted Jul 29, 2024 0:27 UTC (Mon) by Paf (subscriber, #91811) [Link]

I strongly encourage you to actually *try* these tools. It is very apparent from your comments about boilerplate code that you have not. Seriously - I’m a programmer with 12 years of pretty successful work experience, largely on an out of tree distributed file system. I wondered if they might be limited to boilerplate. While they are *extremely* good at boilerplate, they absolutely are not limited to it. Sure, they can’t really do much kernel work of length, but they are wildly good at even moderately complex scripting and helping you use unfamiliar APIs. Yes, the API part is sort of boilerplate, but it doesn’t have to be common stuff - it can be obscure ones, including kernel APIs.

Consider the web in 1994

Posted Jul 25, 2024 10:02 UTC (Thu) by paulj (subscriber, #341) [Link]

Yes, I'm sure they're useful for stuff like code, and other fields where it's often required to produce volumes of material that fit to already established patterns in the field, and tailor parts of them to the current task. IF you get the AI to produce the material for an expert to review, correct, and tweak; then you can save the time of that expert having to do the initial trawl and production. Sure, that's a time save.

It's a big IF though. Cause some people who are not experts will use it to produce reams of material that /look plausible/. Worse, people who are experts but are a bit lazy (or are doing their "homework" a bit too late) may /fail/ to do the review, correct and tweak step and instead present the AI generated bullshit as the work of their own expertise! (The article I linked to being a case in point - how much money did the defendant have to expend on legal fees to prove it was bullshit! Had they not had those resources, they may have had to fold and settle or lost the case - and then been on the hook for the costs!).

So yes, useful. With caveats. And the problem is some will be ignorant of or just ignore the caveats - like in the article!