I do not see this traffic [LWN.net]

I do not see this traffic

Posted Aug 29, 2026 16:25 UTC (Sat) by zdzichu (subscriber, #17118)
Parent article: Ryabitsev: Creepy crawlies

I've implemented Anubis and iocaine for my blog, webpage, Gitea repositories. The graphs in Grafana show a handful of requests weekly. Where's the traffic? I see no sign of scrappocalipse :(

My domain is 18 years old. Blog notes and migrated Google+ post date back to 2003. I have an email in the domain which is in a million places of mailing list archives. Yet crawlers ignore my site. My 16-core home server on 100 Mbps connection is bored.

to post comments

I do not see this traffic

Posted Aug 29, 2026 17:23 UTC (Sat) by pizza (subscriber, #46) [Link] (1 responses)

> I've implemented Anubis and iocaine for my blog, webpage, Gitea repositories. The graphs in Grafana show a handful of requests weekly. Where's the traffic? I see no sign of scrappocalipse :(

Lucky you.

Over the past month, 89% of the traffic on one site I administer is blocked before it ever gets to anubis. Of the stuff that gets through, about 50% is purely attributable to AI crawlers using residential proxies for one query apiece. Most of the rest is from random other, better-behaved crawlers. That one hosts a gerrit instance that only a handful of folks actively interact with; I'd be surprised if even 1% of the non-AI traffic load (ie of that 11%) is from humans.

Another site blocks about 75% of traffic before it gets to anubis. 80% of what's left is again purely attributable to residential-proxy scrapers. There's probably a higher proportion of what's left attributable to humans, but it's still in the low single-digit percentages (again, of the non-blocked 25%).

Meanwhile, my hosting costs are going to roughly double in a couple of months. If not for these countermeasures, I'd have to throw even more money at this just to stay on the air.

They are constantly re-scraping my words and my software, leaving me holding the bill.

A pox on all their houses.

I do not see this traffic

Posted Aug 29, 2026 23:12 UTC (Sat) by JanC_ (subscriber, #34940) [Link]

And don’t forget: hosting costs are getting more expensive because of the more expensive hardware (memory, etc.) too. And more expensive energy too. And extra cooling costs because of global warming.

This is all hurting us in several different ways.