Emacs NSA bait [LWN.net]

Emacs NSA bait

Posted Feb 16, 2026 10:58 UTC (Mon) by anselm (subscriber, #2796)
In reply to: Emacs NSA bait by rsidd
Parent article: Poisoning scraperbots with iocaine

This is essentially Weizenbaum's ELIZA. It looks clunky now, but when it was new in the 1960s it was enough to convince many people that they were talking to an actual psychotherapist – which probably tells us more about humans than about the program itself.

to post comments

Emacs NSA bait

Posted Feb 16, 2026 16:21 UTC (Mon) by karkhaz (subscriber, #99844) [Link] (2 responses)

I do nowadays wonder whether the stories of people being convinced by ELIZA are entirely fabricated or at least exaggerated.

Today's LLM boosters created an entire genre of press releases disguised as science publications, claiming that LLMs can take the initiative to blackmail people, that they demonstrate self-awareness, that their inevitable power-seeking presents a risk to human survival. It's clearly part of a campaign to diminish the idea that companies and ultimately individuals would be responsible for instigating any such scenario in the first place. The money being spent to prevent government regulation is much more effective when coupled with widespread credulity of LLMs' agency and anthropomorphisation that gets parroted by the compliant press. There are plenty of people who would benefit from an environment of increased impunity where the key takeaway of the "AI agent published a hit piece on me" blog post is that...an AI agent published a hit piece on somebody. And it was nobody's fault. Nor with the deepfakes, war crimes, AI-induced psychosis and so on.

It's quite easy to see how the idea that "people were convinced by ELIZA" could become embedded in the wider culture if that idea benefited somebody powerful.

Emacs NSA bait

Posted Feb 16, 2026 19:20 UTC (Mon) by excors (subscriber, #95769) [Link] (1 responses)

> I do nowadays wonder whether the stories of people being convinced by ELIZA are entirely fabricated or at least exaggerated.

I can believe it, because people seem to be really bad at judging intelligence. There's a recent paper "Large Language Models Pass the Turing Test" (https://arxiv.org/abs/2503.23674) where participants had five-minute conversations with a pair of one human and one AI, and had to guess which was human. A suitably-prompted ChatGPT (imitating a young nerd with poor writing skills) was significantly more convincing than the actual humans, winning 73% of the time. But even ELIZA won 23% of the time.

Like, a quarter of people were convinced by a dumb algorithm that responded to every question with either a verbatim echo of the question or a vague unrelated question of its own, even when they were specifically tasked with identifying the non-human. They liked its "question handling" and "personality traits" and apparently didn't mind it was all empty words. (Outside of the test, I guess that helps explain why so many people are tricked by LLMs that are far more sophisticated at echoing and validating their delusional thoughts while they fall down a rabbit hole.)

Emacs NSA bait

Posted Feb 17, 2026 9:13 UTC (Tue) by farnz (subscriber, #17727) [Link]

Note, too, that the claim is not "everyone was convinced by ELIZA some of the time", but rather "some people were convinced by ELIZA some of the time".

To misquote an apocryphal quote, "there's a huge overlap between ELIZA's best output, and what people think someone really dumb would say"; the Turing Test was still in vogue when ELIZA was new, and under the rules of the test, you are "convinced" by ELIZA if you cannot determine whether you're conversing with a human, or with a bot to an extent better than random guessing - but that means that you count as "convinced" if you erroneously identify humans as the bot repeatedly.