another possible source of scraper traffic
another possible source of scraper traffic
Posted Jul 14, 2026 20:54 UTC (Tue) by rgmoore (✭ supporter ✭, #75)In reply to: another possible source of scraper traffic by corbet
Parent article: An update on the scraper situation
That's what I'd assume, too. An AI agent looking something up is going to follow a pattern at least somewhat similar to a human being looking something up. They'll start by looking at a few pages, spend some time digesting the contents, and then maybe come back for some related topics. Maybe the AI agent will be more aggressive about downloading a bunch of articles at once, but it's still going to be focused on whatever task it's been given. An AI agent that's trying to write a device driver is going to look at articles at least somewhat related to device drivers. It isn't going to dig up articles on a long-ago DPL election or arguments about which init system to use they way a deliberate attempt to scan the whole archive would.
