There's AI slop and anti-AI slop [LWN.net]

There's AI slop and anti-AI slop

Posted May 30, 2026 0:00 UTC (Sat) by dilinger (subscriber, #2867)
In reply to: There's AI slop and anti-AI slop by rbranco
Parent article: Nesbitt: Protestware for coding agents

"There's a lot to dislike about LLM bots DDOSing the Internet but"

I'm going to stop you right there. This is literally destroying the world wide web. I don't mean that figuratively. More and more, I'm being locked out of sites because my IP is on a residential phone network that other people are launching LLM scrapers from. I had to take down my gitweb instance because it was being DDoS'd by the scrapers. Sites I rely regularly rely on for development or general information are slowing immensely and are locked in an arms race with LLM bots.

This is a huge, huge problem, and to wave it away like that doesn't help any case you're trying to make.

to post comments

There's AI slop and anti-AI slop

Posted May 30, 2026 5:09 UTC (Sat) by mb (subscriber, #50428) [Link]

>I had to take down my gitweb instance because it was being DDoS'd by the scrapers.

I have put it behind a cookie check. The cookie is set (always) on the main non-gitweb/cgit entry site that humans would typically use for entry.
If that cookie is not set, Apache will redirect to a small html-only page explaining what is going on.

This works really well, but it took many months for the bots to drop their caches and finally give up fetching the html redirect page over and over again.
They do not seem to be smart enough (yet) to retrieve and store the cookie from the main site.
It immediately helped to reduce the server load to an acceptable level again, because the static page is much less resource consuming than gitweb/cgit.

Of course this also disables traditional scraper bots.

It is a medium annoyance since direct links now no longer work and require two extra clicks instead.
But that's Ok for my use case and it's trivial enough to implement.

There's AI slop and anti-AI slop

Posted May 30, 2026 13:33 UTC (Sat) by rbranco (subscriber, #129813) [Link]

> I'm going to stop you right there. This is literally destroying the world wide web. I don't mean that figuratively. More and more, I'm being locked out of sites because my IP is on a residential phone network that other people are launching LLM scrapers from. I had to take down my gitweb instance because it was being DDoS'd by the scrapers. Sites I rely regularly rely on for development or general information are slowing immensely and are locked in an arms race with LLM bots.

We all know that but do I really need to list all the problems, real and imagined?

Browsing the Internet has been a PITA long before LLMs anyway if you're in Europe.

There's AI slop and anti-AI slop

Posted May 31, 2026 7:54 UTC (Sun) by Lennie (subscriber, #49641) [Link] (1 responses)

I would say copyright is basically dead.

Also an other example: a bunch of sites are funded by ads (and this had already been going down for years, which is why the number of ads keep going up), sites that perform a useful function (in aggregate at the very least). But now... if a LLM does the search and collect of the content for you... nobody sees the ads and the sites will go away.

There's AI slop and anti-AI slop

Posted Jun 1, 2026 8:34 UTC (Mon) by taladar (subscriber, #68407) [Link]

There is an argument to be made that the ad-funding model has done a lot more damage to the internet since long before LLMs.