Not logged in
Log in now
Create an account
Subscribe to LWN
LWN.net Weekly Edition for June 20, 2013
Pencil, Pencil, and Pencil
Dividing the Linux desktop
LWN.net Weekly Edition for June 13, 2013
A report from pgCon 2013
in my opinion, the fix here is for apache to change how it does timeouts.
it currently has one timeout variable that is used for three things.
1. time from initial connection to when it gets the header
2. idle time while waiting for new data to arrive
3. total time that a CGI is allowed to run.
there are _many_ sites where this timeout needs to be set fairly large to accommodate #3, but that the timeouts for #1 and #2 could be _much_ shorter.
Apache attacked by a "slow loris"
Posted Jun 29, 2009 1:31 UTC (Mon) by njs (guest, #40338)
For instance, suppose that you have, somewhere on your web-site, a 1 megabyte file. Suppose that you set up your timeouts so that a people have to download at at least 5 KB/s or get cut off (locking out some modem users, but oh well). And suppose that you allow a maximum of 100 Apache worker processes.
If I download that file right at the minimum speed, then I can tie up one of those 100 worker processes for 204.8 seconds. To tie up all 100 of them indefinitely, I have to issue ~1 request every 2 seconds, and use 500 KB/s of bandwidth. That's trivially achievable from a residential connection.
I won't lock other users *quite* as badly as the full slow loris attack -- other connections will get a chance to be serviced once every 2 seconds! -- but it's near as makes no difference in practice.
So fiddling around with header timeouts or whatever might feel good, but isn't going to actually solve the problem, and thus is a waste of time.
Posted Jun 29, 2009 7:24 UTC (Mon) by dlang (✭ supporter ✭, #313)
if you have large files to download (or heavy CGI's to run), the attacker can overwelm you by sending small requests that generate huge responses (or in the case of the CGIs, small requests that take significant amounts of CPU time)
timeouts don't help in those cases, but allowing for a connection to tie up a slot for 300 seconds before it sends _any_ request is far worse. it doesn't take a broadband line to take down a server, dialup will do the job.
you use the example of 500K/sec of bandwidth to eat up 100 connections, with the default 300 second timeout you need to send 2 64 byte packets every 300 seconds to tie it up, so 100 connections is 6400 bytes * 8 bits/byte / 300 seconds or 170 bps, a thousand connections pushes 2k bps
now in reality the issue isn't when you have one attacker IP (that shows up and is easy to block), it's when you have thousands of attackers, and in that case even a small amount of bandwidth and CPU from each of them can overwhelm a server, and at that point it can become very hard to tell the attackers from legitimate users unless the attacker is dumb enough to do something that stands out
Posted Jun 29, 2009 9:04 UTC (Mon) by njs (guest, #40338)
Yeah, so... is your argument that the Apache folks should do a bunch of work to patch a million of these holes to protect us against evil modem users? A fix that only protects us against evil modem users doesn't seem worth the effort.
> now in reality the issue isn't when you have one attacker IP (that shows up and is easy to block), it's when you have thousands of attackers, and in that case even a small amount of bandwidth and CPU from each of them can overwhelm a server, and at that point it can become very hard to tell the attackers from legitimate users unless the attacker is dumb enough to do something that stands out
Right -- this seems like a more realistic threat model. And how will fiddling with Apache's timeout handling, like you advocate, protect anyone from it?
Posted Jun 29, 2009 9:36 UTC (Mon) by dlang (✭ supporter ✭, #313)
I argue that using the same variable for
1. how long after a connection do I wait for a request
2. how long can the flow of data pause
3. what is the max length of time that a CGI can run, even if it is passing data continuously
is just bad design period, even if the attackers aren't taking advantage of it, these are three very different cases, and the appropriate values are very different between them. this affects the activity of a site even when it's not under attack.
if someone is out to get you and is willing to burn/use a large botnet to do it, they are going to get you. I don't care who you are, thousands of bots collectivly have more bandwidth than you do, so they can take you down by just doing legitimate things on your site.
the current situation is that trivial attacks can take apache down, at almost no cost to the attackers in terms of load. this means that they can do it on a botnet without doing anything that would make the owners of the machine notice that something is wrong.
fixing the timeouts so that each of the three cases above are handled seperatly would force the attacker to use significantly more bandwidth for their attack, this would either raise the possibility that the owners of the machines would notice something was wrong (because the letgitmate use is slow due to the load), or it would force the attackers to use much larger botnets. either of these would make it more expensive for the attacker
Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds