Volume difference with older crawlers?
Volume difference with older crawlers?
Posted Feb 16, 2026 0:55 UTC (Mon) by corbet (editor, #1)In reply to: Volume difference with older crawlers? by Wol
Parent article: Poisoning scraperbots with iocaine
I don't see how that would help. Humans get grumpy about a five-second delay... The bot just waits, then has the page it was after. Meanwhile your server, which is now trying to keep open five seconds worth of full-on scraper bot traffic, melts down into slag.
The LWN site is currently under high scraper load, so comment display has been suppressed for anonymous users. If you are a human, you may read the comments by clicking the button below:
Note: you can avoid this step in the future by logging into your LWN account.