|
|
Log in / Subscribe / Register

Collateral damage

Collateral damage

Posted Aug 29, 2026 12:35 UTC (Sat) by snajpa (subscriber, #73467)
Parent article: Ryabitsev: Creepy crawlies

Treating all AI traffic as bulk crawling throws real users overboard. I have already run into this on other sites: I ask an LLM to trace a problem through commits, mailing-list posts, documentation, and bug reports, and the requests get blocked as though they were harvesting a corpus.

“Random old commits” are weak evidence of crawling. Systematic traversal or broad coverage would support that conclusion. A scattered handful of old commits is exactly what research into a specific issue often produces.

Calling these users collateral damage does not show much concern for them. The policy effectively says that manual browsing counts, while delegating the same research to an LLM disqualifies you.

This way of working will become far more common because it saves an enormous amount of time. Sites that lump it together with mass scraping are choosing to exclude a growing class of users.


The LWN site is currently under high scraper load, so comment display has been suppressed for anonymous users. If you are a human, you may read the comments by clicking the button below:

Note: you can avoid this step in the future by logging into your LWN account.


Copyright © 2026, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds