Fuzzing for Security (The Chromium Blog) [LWN.net]

A posting on the Chromium blog describes the project's efforts to do fuzz testing of the browser. "Chrome’s fuzzing infrastructure (affectionately named "ClusterFuzz") is built on top of a cluster of several hundred virtual machines running approximately six-thousand simultaneous Chrome instances. ClusterFuzz automatically grabs the most current Chrome LKGR (Last Known Good Revision), and hammers away at it to the tune of around fifty-million test cases a day. That capacity has roughly quadrupled since the system’s inception, and we plan to quadruple it again over the next few weeks. [...] To appreciate just what that means, consider that ClusterFuzz has detected 95 unique vulnerabilities since we brought it fully online at the end of last year. In that time, 44 of those vulnerabilities were identified and fixed before they ever had a chance to make it out to a stable release." There is mention of pushing the fixes upstream to WebKit and FFmpeg, but there is no mention of whether the ClusterFuzz code will be made available, unfortunately.

Fuzzing for Security (The Chromium Blog)

Posted Apr 27, 2012 13:28 UTC (Fri) by welinder (guest, #4699) [Link]

I wrote something for Gnumeric that can do fuzzing of xml files
and fuzzing of zip-file members. Search for "zz" here:

http://git.gnome.org/browse/gnumeric/tree/test

A generic fuzzer would normally just damage the zip or xml layers and
thus be a test of the parser. That's fine, but the more interesting
parts to test are those behind the packaging.

Fuzzing for Security (The Chromium Blog)

Posted Apr 27, 2012 17:56 UTC (Fri) by rwst (guest, #84121) [Link] (1 responses)

Now why o why don't Adobe do such a thing with Flash?

Fuzzing for Security (The Chromium Blog)

Posted Apr 27, 2012 21:46 UTC (Fri) by nix (subscriber, #2304) [Link]

Well, actually they do it by letting Google do it for them (starting some years ago).

Is nothing enough?

Posted Apr 27, 2012 23:49 UTC (Fri) by kripkenstein (guest, #43281) [Link] (9 responses)

What is shocking is that despite this measure that Chrome does (and other browsers as well), it was still quite possible to hack Chrome (and all other browsers) at recent hacking events.

What else can be done, if current measures are not enough?

Is nothing enough?

Posted Apr 28, 2012 0:37 UTC (Sat) by robert_s (subscriber, #42402) [Link]

Well it's not that shocking. I mean, fuzz testing is useful, but the problem is a hacker is cleverer than a random number generator.

Is nothing enough?

Posted Apr 28, 2012 0:57 UTC (Sat) by Company (guest, #57006) [Link] (7 responses)

You know those math proofs you have to do? There's 2 types: "There exists one element that..." or "There exists no element that...". The first one is hard to do usually - you need to find the one element that qualifies, but the second one is a lot harder, because you need to look at every possible element and prove that things hold.

Security is like that: The crackers need to find the one element that breaks things. And the developers need to prove that for all possible inputs, the output is valid to be sure they don't have security issues.
As a side note, that proof would be equal to a formal proof of the correctness of the software. And I remember my university courses about how possible that is even for simple systems (read: Touring machines).

So in short: It's de-facto impossible to make a browser (security-) bug-free. It's just a question of time until people find the next bug and exploit it. Most likely, they are already doing it.

Is nothing enough?

Posted Apr 28, 2012 22:07 UTC (Sat) by MisterIO (guest, #36192) [Link] (6 responses)

That's incorrect. It's de-facto impossible to _prove_ that a piece of software doesn't have bugs, but that doesn't mean that you can't actually write bug-free code. Just talking about theory.

Is nothing enough?

Posted Apr 29, 2012 0:05 UTC (Sun) by Company (guest, #57006) [Link]

That's incorrect. It's de-facto impossible to _write_ a piece of software that doesn't have bugs, but that doesn't mean that you can't actually prove code being bug-free. Just talking about theory.

Is nothing enough?

Posted Apr 29, 2012 1:37 UTC (Sun) by mrons (subscriber, #1751) [Link]

I'm reminded of the following quote by Knuth:

"Beware of bugs in the above code; I have only proved it correct, not tried it."

Is nothing enough?

Posted Apr 30, 2012 20:51 UTC (Mon) by quentin.casasnovas (guest, #58238) [Link]

Saying it is
> de-facto impossible to _prove_ that a piece of software doesn't have bugs
does sound a bit strong... Do you have a proof to back-up that somewhat no piece of software can be proven bug-free? I think it is sometimes possible: in synchronous languages for example, using formal analysis.
Think also about pure functions, which could certainly be proven bug-free mathematically. Lambda calculus and functional languages are an extent to that.

Is nothing enough?

Posted May 3, 2012 5:46 UTC (Thu) by Comet (guest, #11646) [Link] (2 responses)

So formal methods do not exist?

Is nothing enough?

Posted May 3, 2012 14:29 UTC (Thu) by nybble41 (subscriber, #55106) [Link] (1 responses)

> So formal methods do not exist?

No, they're just impractical for software of any reasonable complexity. The largest piece of software I've ever heard of which had a formal proof of correctness was a version of the L4 microkernel--essentially not much more than a low-level message-passing interface, with less than 9k lines of C code--and even then the proof is only as good as the requirements defining "correctness".

[1] http://ertos.nicta.com.au/research/l4.verified/

Is nothing enough?

Posted May 4, 2012 5:00 UTC (Fri) by AndreE (guest, #60148) [Link]

Probably of greater interest is this page.

http://ertos.nicta.com.au/research/l4.verified/proof.pml

Fuzzing for Security (The Chromium Blog)

Posted Apr 28, 2012 2:00 UTC (Sat) by SLi (subscriber, #53131) [Link]

[Posted also as a comment to the Chromium blog entry]

Interesting timing. I just released couple of minutes ago a toolkit for (semi-)automated fuzzing of Linux kernel filesystem implementations.

Obviously it's much smaller scale, but being able to run an entirely automated system that checks out git HEADs of kernel trees and to automatically bisect crashes before firing away report emails has been a goal in my mind.

Currently it must still be invoked manually; the scripts in the toolkit take care of running repeated fuzz tests in a KVM virtual machine and minimizing bit-differences of crashing test cases to pristine filesystem images.

The announcement is here: http://www.spinics.net/lists/linux-ext4/msg31856.html