|
|
Log in / Subscribe / Register

Volume difference with older crawlers?

Volume difference with older crawlers?

Posted Feb 16, 2026 7:24 UTC (Mon) by mb (subscriber, #50428)
In reply to: Volume difference with older crawlers? by Wol
Parent article: Poisoning scraperbots with iocaine

That just results in the resource consumption on the server to immediately skyrocket.
The port is used. Lots of memory is used already. Some CPU has been consumed already.
Other addresses won't stop hammering, while I hold the connections alive due to the delay.

And real users will be affected the most.

The only thing that really works for me is the exact opposite: Try to handle the request as quickly as possible.


to post comments


Copyright © 2026, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds