KHB: Failure-oblivious computing [LWN.net]

KHB: Failure-oblivious computing

Posted Jun 29, 2006 11:03 UTC (Thu) by walterh (guest, #19113)
Parent article: KHB: Failure-oblivious computing

This seems to me to be a very dangerous idea. After all, we are talking about security critical applications here. The poster before me has already mentioned that papering over bugs will remove the incentive for the developers to fix them. But the problem is actually much more severe: Who says that, say, clipping a buffer that is being overrun by an attacker is a safe choice? This could just as well open new and hard to discover security holes.

Only to make an example, consider ACLs: Take a buffer that is supposed to contain a list of users who are to be denied access to a resource. If the buffer overflows and the program is terminated, then not much harm is done. If the buffer overflows and is clipped, then some people can access the resource that shouldn't.

Or, take the example mentioned in the article, pine: If I sent you a virus infected attachment with filename
loooooooooooooooooo...ooongname.txt.exe
which will overflow the (hypothetical) built in virus scanner in pine, so that it only sees the name
loooooooooooooooooo...ooongname.txt
then the file may go through unscanned. If the save-to-disk routine in pine uses a different buffer length, then the recipient could still save this as an executable.

Walter

(Log in to post comments)

KHB: Failure-oblivious computing

Posted Jun 29, 2006 17:48 UTC (Thu) by cventers (subscriber, #31465) [Link]

Are you responding to someone who is talking about the risk of
unauthorized access and suggesting that the current situation is worse
because it would be a denial of service attack?

This article is very interesting, but I can't say that the technique
described would be something I'd be at all comfortable using. One of the
biggest sources for bugs in software is unpredictable code paths and
input. This method takes the results of a broken assumption and breaks
more assumptions. While this may work OK in certain scenarios, I propose
something fundamentally better (and less expensive?)

What if an invalid memory access simply resulted in an /exception/ versus
a signal? Then I wrap my "parse this RFC2833 junk" function around a try
{} / catch block. If I got an invalid memory access trying to parse the
RFC2833, I write a detailed log entry (hell, if I were uber-clever, I
could use a special 'snapshot my memory' syscall to tell the kernel to
immediately mark my whole state as COW, so it can be dumped to disk. Now
you can do run-time core dumps and continue.). Then I simply deny service
to that operation. The bug is contained. If someone was trying to exploit
it, I've got their IP, and if I'm a programmer I've got a core. If I'm an
administrator, no users complained at all, or perhaps only one if it was
triggered accidentally.

The details, of course, lie in the implementation....

KHB: Failure-oblivious computing

Posted Jun 29, 2006 17:51 UTC (Thu) by cventers (subscriber, #31465) [Link]

To expand on this a bit, the author alluded to the better performance of
the failure-oblivious Apache due to its workers not dying. The fact that
Apache breaks itself down into workers in this way is an important
reliability feature of Apache -- one that is capable of somewhat avoiding
denial of service attacks.

KHB: Failure-oblivious computing

Posted Jun 29, 2006 17:53 UTC (Thu) by cventers (subscriber, #31465) [Link]

Silly me, this special syscall even already exists! How about stuffing
this in a SIGSEGV signal handler.

/* Dump some core. */
if (fork() == 0) raise(SIGABRT);

/* Magic to initiate an exception from whatever context we were running
in before our signal handler got called */

KHB: Failure-oblivious computing

Posted Jun 29, 2006 18:09 UTC (Thu) by cventers (subscriber, #31465) [Link]

I apologize to the readers and editors about the volume of my comments replying to myself, but I think we need a longjmp() that is safe to use as an exit from a signal handler. Then my above code could be:

int handle (int sig)
{
  struct jmp_buf buf;

  if (sig == SIGSEGV) {
     if (fork() == 0) raise(SIGABRT);
     pop_jmp_buf_from_tls_stack(&buf);
     longjmp_after_sig(&buf, sig);
  }
}

...then strategically place:

// Prepare an exception context
if (save_context_in_tls_stack() == SIGSEGV) {
   // operation failed
}

do_something_possibly_unsafe();

// Forget about that context
pop_jmp_buf_from_tls_stack(NULL);

Comments?

KHB: Failure-oblivious computing

Posted Jun 29, 2006 21:34 UTC (Thu) by nix (subscriber, #2304) [Link]

Throwing from certain signal handlers is safe in the presence of -fasynchronous-unwind-tables, but I sort of doubt that SIGSEGV is necessarily one of them (what if, e.g., the unwinder segfaults, something which is not unknown?)

Joe Buck would know for sure, I'd guess. :)

KHB: Failure-oblivious computing

Posted Jul 4, 2006 8:42 UTC (Tue) by walterh (guest, #19113) [Link]

>> Who says that, say, clipping a buffer that
>> is being overrun by an attacker is a safe choice?
>
> If it is not, the program is severely broken in other way(s) as well,
> and this could be exploited without the buffer overrun in the first place;
> so what's your point?

My point is that clipping buffers is worse than just terminating the program -- and I gave examples why this is so. Your assertation that you can safely clip buffers is clearly wrong, as is it not hard to think about otherwise perfectly safe programs that get exploitable if you just clip a buffer somewhere.

KHB: Failure-oblivious computing

Posted Jul 4, 2006 9:02 UTC (Tue) by evgeny (guest, #774) [Link]

> it not hard to think about otherwise perfectly safe programs that get exploitable if you just clip a buffer somewhere.

Might be; then do think a bit harder to come with a reasonable example proving it; the ones you suggested are absolutely irrelevant. Let's take the first one: say, a buffer is defined as char buf[5] and the attacker manages to pass to it "Hello, world!" causing an overrun. Now, the failure-oblivious runtime notices this and clips the string to just "Hell" (plus the terminating zero). You say when "Hell" propogates further, it might cause a compromise. It could be of course, but my point is that the attacker could send this "Hell" in the first place, and get a successful exploit anyway, whether the buffer overruns are clipped or not. Such things are called a failure to sanitize user input. Do you follow?

KHB: Failure-oblivious computing

Posted Jul 4, 2006 9:29 UTC (Tue) by walterh (guest, #19113) [Link]

> Might be; then do think a bit harder to come with a reasonable example
> proving it; the ones you suggested are absolutely irrelevant.

Just to show you how relevant these examples are, think about the ACL example again: Say a server has a block list which is consists of a dynamic part supplied by the IDS and a static part supplied by the administrator. Joe Hacker has been put on the static part by the administrator. To get around the block, he injects forged packets into the IDS to grow the dynamic block list to the point where the buffer for the combined block list overruns. Now, as the buffer contains just IP addresses this would result in a program crash enabling a denial of service attack. But, when the buffer just gets clipped so that the static part of the block list is gone, Joe Hacker can get access.

Or, another example: A PHP app checks user+password combinations with the following SQL command:

SELECT * FROM users WHERE user='%s' AND password='%s';

Of course, it will escape and special characters like ' in input. Now Joe Hacker supplies
looooo....ongname'
as username. The ' at the end gets escaped to \'. Assume that due to some buffer magic the last character is clipped. Now the string is
looooo....ongname\
Choosing
or 1=1 limit 1;
for the password the SQL command now becomes

SELECT * FROM users WHERE user='looooo....ongname\' AND password=' or 1=1 limit 1;';

which will let Joe hacker in.

So you can see that is is easy to come up with examples where the technique advertised in the article creates gaping security holes.

To be honest, I think it is very worrying how some people here defend a broken idea. If those are the ones writing critical applications, then there is plenty or reason to be afraid.

KHB: Failure-oblivious computing

Posted Jul 4, 2006 11:11 UTC (Tue) by evgeny (guest, #774) [Link]

> Say a server has a block list which is consists of a dynamic part supplied by the IDS and a static part supplied by the administrator.

Well, any seasoned C programmer will notice that this is a very strange coding practice:
1) Why to concatenate the lists instead of checking first the static and then the dynamic part?
2) If the concatenation is for a reason indeed necessary, one'd do it the other way around, thus simplifying the compile-time optimization and run-time memory access patterns.

These alone would suggest the prorgam is badly written and perhaps shouldn't be trusted anyway.

However, a more seriously wrong assumption you made is that in the absence of the buffer clipping _any_ buffer overrun results in SEGFAULT. Alas, this is NOT the case. Otherwise, the issue of the buffer overruns would be considered only in the context of DoS. Instead, we're talking about executing practically anything, including the shell access (which is arguably much worse than failure to throwing a bad guy away right at the entrance). And even if that doesn't happen, an equivalent of the buffer clipping could take place. E.g.,

int main(void)
{
char buf[5];
sprintf(buf, "Hello, world!\n");
printf(buf);
exit(0);
}

works nicely here (i.e., outputting the entire "Hello, world!\n" string), instead of segfaulting. However, were there enough of code between the sprintf() and printf() lines, the result could be very different, including in theory the clipping right after the "Hell" substring, or any other type of garbling.

> Or, another example: A PHP app

All strings in PHP are dynamic... You should use a different language.

> checks user+password combinations with the following SQL command:
>
> SELECT * FROM users WHERE user='%s' AND password='%s';

Oh, just let's not return to this deadly beaten horse. You can read e.g. this nice article (http://lwn.net/Articles/185813/) and the very informative comments to it to understand that constructing an SQL statement directly from the user input is 99.999% garanteed to be vulnerable, even in languages that don't have the string buffer overrun problem.

Your examples above show signs of programming ignorance. On the contrary, the buffer overruns are periodically spotted in very mature pieces, written by very respectable programmers. They're often typos (or off-by-one miscalculations) than real mistakes. I wish libc had functions to deal with strings in a safe manner. sigh...

> To be honest, I think it is very worrying how some people here defend a broken idea.

Nobody said this is a silver bullet. As practically any other technique, it has a restricted area of applicability. (E.g., as mentioned in the original article, it certainly shouldn't be used in the scientific computing).