Volume difference with older crawlers?

Oh, wow. Ok.

If things really are as dire as painted by a number of people here, in the article and in the comments, then this may ultimately the end of the current Internet architecture, at the L3 -> L4 interface, at least so far as smaller content providers are concerned. Everything will have to go behind something that requires the requester to pay in some way - either by providing some proof of work done (as Anubis does), some other form of proof of extra effort (perhaps registering an account, but that's weak and the abusers will just do that if they need to) or some other micro-payment.

Wow. We live in interesting times.