Volume difference with older crawlers?
Volume difference with older crawlers?
Posted Feb 16, 2026 9:21 UTC (Mon) by taladar (subscriber, #68407)In reply to: Volume difference with older crawlers? by marcH
Parent article: Poisoning scraperbots with iocaine
One aspect is probably users prompting the models to look at specific sites which turns the requests from one per service (e.g. OpenAI) to one per user of the service.