Consider the web in 1994

Careful with the "No True Scotsman" iterations on the Turing test there... :)

Unlike Turing's mathematical work, the imitation game was merely a thought experiment we all sort of rolled with over the years. The idea is to build a Turing machine (i/o + finite state machine) that would be able to *fake* it when judged by some "average person" interacting with it, and we can say LLMs have mostly succeeded in that regard.

It is concievable that in the future, a more complex (but still finite) state machine might be able to fake understanding how to program based on some rules (if you think about it, Searle's Chinese Room is just another Turing machine: i/o + state machine, that is, in this case, able to fake understanding Chinese to a Chinese speaker standing outside the room).

However, true intelligence isn't when the Turing machine can make *you* believe it's intelligent. Rather, it happens when the machine thinks to *itself* that it is intelligent -- the whole Descartes "cogito ergo sum" thing). As to how *we* get to test for that, I don't think science has that figured out yet... :)

Try asking it to write a program in DataBASIC :-) The only reports I've heard about people trying that is that it's been a complete disaster competing for the "how many pages of printout can you generate from one syntax error" prize (something my fellow students did at school - they were programming in FORTRAN).

Cheers,
Wol

I think a reductionist view has value because an LLM is not an intelligence, it is a thoroughly lobotomized parody of speech and writing, that only works because people are _very_ willing to extend the theory of mind to anything making vaguely human-sounding noises, even eliza was taken to be intelligent to some people, LLMs are only a little more sophisticated, but with just great gobs of training data the LLM can spit out something plausible sounding given a wide variety of inputs.

An LLM can't train someone in programming because it doesn't know how to program and it doesn't know how to teach and it has no way to model the state of the learner and adjust a pedagogy based on a feedback loop consulting hundreds or thousands of different neural networks in your brain. An LLM hardly has any feedback loops of any kind and barely one network.

An LLM can spit out text real good but it doesn't have enough varied systems to have the intelligence of a fruit fly. All it has are the strings of symbols that people have used to communicate with each other and with the computer and all it can do is combine those symbols in abstract and pleasing ways. People will invent a theory of mind when talking to it, but people are capable of anthropomorphism far less animate objects than computers.

It may well be that some day an artificial mind is created, it's not like brains are made out of magic unobtainium, they obey and rely on the physics of the same universe that allows computers to function, but there is just nowhere near the complexity and interlocking learning and feedback systems present in modern "AI" that are present in _any_ vertebrate, let alone human intelligence.

It's a big IF though. Cause some people who are not experts will use it to produce reams of material that /look plausible/. Worse, people who are experts but are a bit lazy (or are doing their "homework" a bit too late) may /fail/ to do the review, correct and tweak step and instead present the AI generated bullshit as the work of their own expertise! (The article I linked to being a case in point - how much money did the defendant have to expend on legal fees to prove it was bullshit! Had they not had those resources, they may have had to fold and settle or lost the case - and then been on the hook for the costs!).

So yes, useful. With caveats. And the problem is some will be ignorant of or just ignore the caveats - like in the article!

That seems like wishful thinking to me for a few reasons:

The whole idea of LLMs is that bigger models and more data is supposed to get you some kind of intelligence, but the current crop of LLMs has already been trained on pretty much the sum total of human text output available today and it's still crap. In fact one reason why they're not getting better is that improvements require exponentially more data and we're already out of data to train on. Getting superior results with a fraction of that with the same technology seems unlikely.

Second, effective teaching requires both understanding the subject matter and good knowledge and experience with how humans tend to think an learn. LLMs can't remotely do either of things at all, let alone for a complex subject like programming. Summarization, sure. That just requires pattern matching the phrases humans tend to use to indicate asides and writing conventions like the three part essay structure. But that's very different.

That's why the tsunami caused the nuclear disaster in Japan - their defences were designed to stop a 10m wave. But the defenses were on the plate that dropped, which is why a 9m wave went straight over the top of it ...

I've seen the Victoria Embankment with the Thames almost to the top. That was a long time ago, before the Thames Barrier. The Barrier's estimated lifetime has been about halved. If we have a rise of a meter, I think the Thames will simply flow *round* the barrier, and if the Embankment goes - well - the Strand is so called because it was the strand - the beach on the banks of The Thames. The water will probably go a LOT further than that ...

Cheers,
Wol