How to kill a web browser [LWN.net]

Michal Zalewski recently decided to look for exploitable vulnerabilities in web browsers. So he write a little CGI script which generates random HTML and feeds it to the browser; a refresh tag is used so that the browser will repeatedly request new pages - until things come to a crashing halt. Mr. Zalewski reported his results on Bugtraq as "a mini-farce." It seems that most of the browsers he tested fared rather poorly.

The key word here is "most." One browser was able to absorb noisy input indefinitely without crashing; that browser was Internet Explorer.

There has been quite a bit of talk recently about Internet Explorer's security problems, and how the alternatives - both free and proprietary - are more secure. So this kind of result is somewhat embarrassing. As Mr. Zalewski put it:

It appears that the overall quality of code, and more importantly, the amount of QA, on various browsers touted as "secure", is not up to par with MSIE; the type of a test I performed requires no human interaction and involves nearly no effort. Only MSIE appears to be able to consistently handle malformed input well, suggesting this is the only program that underwent rudimentary security QA testing with a similar fuzz utility.

So what sort of HTML turned out to be problematic? A few examples have been posted - but all you smug, free-software-using folks might want to think twice before clicking on them. Use of a tool like wget is probably more appropriate. One of the examples, which, as your smug, free-software-using editor can attest, kills Firefox is, in its entirety:

    <HTML><INPUT

The post notes that this bug is probably exploitable, and that many others certainly exist. The tester also does nothing involving either cascading style sheets or JavaScript - one suspect that those areas might, just maybe, be the source of a bug or two themselves.

The Mozilla project has been quick to capitalize on the recent bout of Internet Explorer security problems. This incident demonstrates, however, that the free software community can, at times, be a little too quick to claim better security. Testing against malformed input has been a standard quality assurance technique for decades; the fact that Mozilla, seemingly, has not done this testing is a little discouraging. Security can be a winning point for free software, but it doesn't happen automatically. If we're going to claim to have a more secure product, we should be sure we've done the homework first. Meanwhile, expect a new set of Mozilla patches sometime soon.

to post comments

Pretty sad that an approach this simple can kill so many apps

Posted Oct 21, 2004 1:47 UTC (Thu) by lakeland (guest, #1157) [Link] (5 responses)

I seem to recall a very similar test being done with unix command-line
utilities, at which gnu fared extremely well.

FWIW: Konqueror from CVS HEAD passes all the 'example' tests but dies when
you click on the 'big honking' warning' :-(

crashme, fuzz, etc.. are good tools!

Posted Oct 21, 2004 4:48 UTC (Thu) by dank (guest, #1865) [Link] (2 responses)

You're thinking of 'fuzz', I think:
http://www.cs.wisc.edu/~bart/fuzz/

Another, older, test was 'crashme', which did similar
things for kernels (and found an exploitable bug
in sparc's math instructions early on). There
were probably other implementations, but one is at:
http://people.delphiforums.com/gjc/crashme.html

These are both examples of software fault injection
systems. I think
http://citeseer.ist.psu.edu/sieh02umlinux.html
has a list of others...

crashme, fuzz, etc.. are good tools!

Posted Oct 21, 2004 9:18 UTC (Thu) by hildeb (guest, #6532) [Link] (1 responses)

I took this even further in my diploma thesis.
http://www.infosun.fmi.uni-passau.de/st/papers/issta2000/

crashme, fuzz, etc.. are good tools!

Posted Oct 21, 2004 10:09 UTC (Thu) by nix (subscriber, #2304) [Link]

I must say, the automated cause-effect chain analysis stuff linked from there looks quite thoroughly fascinating.

Pretty sad that an approach this simple can kill so many apps

Posted Oct 21, 2004 6:14 UTC (Thu) by ncm (guest, #165) [Link] (1 responses)

As I recall, the GNU utilities didn't do "extremely well"; they just did better than any of the commercial equivalents extant at the time. That's not the same thing.

GNU vs proprietary utilities and garbage input

Posted Oct 23, 2004 19:28 UTC (Sat) by giraffedata (guest, #1954) [Link]

If it's the study I'm thinking of, it was the first of its kind and was groundbreaking.

ca. 1989, a university research project fed garbage input to a bunch of classic Unix programs and watched them crash. They found that the proprietary programs crashed more than GNU. It wasn't a LOT more, but the fact that it wasn't a lot less flew in the face of conventional wisdom at that time, which was that proprietary software had to have fewer bugs than free software because of all the money behind it for testing and other quality assurance activity.

Then the same team shed some light on the reason for the difference by repeating the study on current versions of the same programs two years later. At the conclusion of the original, they had reported the bugs, and fixes where possible, to the maintainers. In the followup, most of the free software had been fixed, almost none of the proprietary had.

This made sense to me as a software developer for IBM. I certainly didn't have the power to fix a bug in my program just because I found out about it. But free software developers can quite easily fix a bug.

That was the beginning of my conviction that shipping debuggable, fixable code early is better than shipping presumably bug-free code late.

How to kill a web browser

Posted Oct 21, 2004 2:19 UTC (Thu) by iabervon (subscriber, #722) [Link]

It seems to me that the issue is really a failure to parse the document as generic SGML before getting into the tag-specific stuff. With a single front line, fewer pieces of code are exposed to really broken possibilities, and few fewer operations have to be written correctly by someone thinking about tags.

How to kill a web browser

Posted Oct 21, 2004 5:56 UTC (Thu) by mmarkov (guest, #4978) [Link] (10 responses)

I don't know the free browsers' internals, but don't
they do syntax analysis of the page in accordance
to some context free grammar of HTML?

I mean, come on, you cannot crash a C compiler
with an invalid C source. Why should a browser
choke on "<HTML><INPUT" ??

Maybe tolerance to legacy pages is part of the problem?

Posted Oct 21, 2004 6:53 UTC (Thu) by eru (subscriber, #2753) [Link] (1 responses)

It would certainly be trivial to weed out these really malformed pages if you simply parsed them strictly according to the HTML spec and rejected all pages that do not pass, but in practice users expect browsers to also render many syntactically incorrect pages, otherwise a large chunk of the web would be unviewable.

I suspect attempts to implement fuzzy parsing that tolerates slightly broken pages is partly to blame here (but of course that is no excuse for buffer overflow errors).

Maybe tolerance to legacy pages is part of the problem?

Posted Oct 21, 2004 10:44 UTC (Thu) by NAR (subscriber, #1313) [Link]

in practice users expect browsers to also render many syntactically incorrect pages

Not to mention that some of us like the feature that a browser starts to display the page even when it hasn't fully download it yet.

Bye,NAR

How to kill a web browser

Posted Oct 21, 2004 7:08 UTC (Thu) by hodoscek (subscriber, #5290) [Link] (1 responses)

It doesn't kill it, at least in my version of firefox nothing happens

How to kill a web browser

Posted Oct 21, 2004 7:36 UTC (Thu) by dale77 (guest, #1490) [Link]

Likewise, Firefox 1.0PR on WinXP (sorry) handles <HTML><INPUT fine

How to kill a web browser

Posted Oct 21, 2004 8:30 UTC (Thu) by nathan (subscriber, #3559) [Link] (5 responses)

> I mean, come on, you cannot crash a C compiler
> with an invalid C source.

I beg to differ :)

How to kill a web browser

Posted Oct 21, 2004 9:38 UTC (Thu) by nix (subscriber, #2304) [Link] (4 responses)

Definitely you can. The old `confused by earlier errors, bailing out' message in GCC, and the new `internal compiler error' messages are definitely crashes... Have a look at:
<http://gcc.gnu.org/bugzilla/buglist.cgi?keywords=ice-on-i...;

At the time of writing, that's 689 cases of crashing a C (and C++, Ada, Java, Fortran) compiler with invalid code. Add an &component=c to the end of that URI, and you find 56 examples of crashing the C compiler.

This is not a crash!

Posted Oct 21, 2004 15:21 UTC (Thu) by spudbeach (guest, #5837) [Link] (1 responses)

A "crash" is when a program completes incorrectly, often with a segfault. A compiler saying "giving up" isn't a crash -- it is a normal termination for bad input. No signal, no possibility of exploitation, and the user got some output that is useful.

This is not a crash!

Posted Oct 22, 2004 13:46 UTC (Fri) by nix (subscriber, #2304) [Link]

The old `bailing out' message was triggered, IIRC, by a SIGSEGV being caught. So there was a signal.

I doubt it could be regarded as exploitable, though. :)

How to kill a web browser

Posted Oct 22, 2004 0:10 UTC (Fri) by JoeBuck (subscriber, #2330) [Link] (1 responses)

Almost all of the GCC crashes occur when GCC itself detects an inconsistency and quits with an internal error. There are a few, though, where the compiler actually crashes because of internal corruption, but those are far rarer.

Cases where the application itself decides to abort aren't security risks (though they could possibly be used to create a denial of service attack if the app can be made to repeatedly shut itself down).

How to kill a web browser

Posted Oct 22, 2004 13:45 UTC (Fri) by nix (subscriber, #2304) [Link]

The internal-corruption crashes certainly are rare, now. I was really thinking of the days of egcs and before, when the `bailing out' message often *was* the result of a SIGSEGV being caught, if memory serves, and was really rather easy to trigger...

... and no, this isn't a security risk. (Who would execute a compiler in a security-critical environment anyway? The security risks are generally in the compiler's output when it succeeds :) )

How to kill a web browser

Posted Oct 21, 2004 6:41 UTC (Thu) by yknaidu (subscriber, #6615) [Link]

It is really very appalling that Mozilla based browsers fail a two letter faulty html.

How to kill a web browser

Posted Oct 21, 2004 10:35 UTC (Thu) by smurf (subscriber, #17840) [Link] (1 responses)

Ouch. While these certainly are embarrasing, they're presumably rather easily fixable, in the sense that those fixes are unlikely to break anything.

Most of IE's holes, on the other hand, seem to be due to insecure integration with other OS components, semi-intentionally-permeable security domain borders, et al. These are a wholly different class of problems.

How to kill a web browser

Posted Oct 21, 2004 16:49 UTC (Thu) by error27 (subscriber, #8346) [Link]

No.

The moral of the story is that we need to be humbler and fix our browser of choice.

How to kill a web browser

Posted Oct 21, 2004 11:23 UTC (Thu) by melevittfl (guest, #5409) [Link] (6 responses)

I don't think it's fair to call this a security issue.

The worst that heppens is you restart the browser and avoid loading the same page.

I also don't think this means IE is "better" or "higher quality" or whatever.

All this means is that someone at MS wrote a test case that fed garbage HTML to the browser and then they fixed it whenever it crashed. It's not as if they are just magically smarter or better, they just had the resources to test a wider coverage.

How to kill a web browser

Posted Oct 21, 2004 11:56 UTC (Thu) by zezaz (guest, #5465) [Link] (2 responses)

This is most probably a security issue. If firefox dies like this, it should not be too hard for a competent cracker to write a page that relies on a buffer overflow to execute code on the client PC. This is not just about your browser dying.

I don't think that this article said there was magic in MSIEs more robust behaviour on this tests either. One of the biggest strengths of MS is the ability to hire the resources to make their products adhere to some quality standards. They relied on this to overcome all their concurrents.

This ability certainly has been much underestimated by many, claiming that IE was horribly bug-ridden, whereas mozilla/firefox/links/emacs-w3-mode... was much better because of code quality. This is the point of this article, and i completely agree with its author. MS should not be underestimated, and opensource programs should not be overestimated.

How to kill a web browser

Posted Oct 21, 2004 23:00 UTC (Thu) by jonabbey (guest, #2736) [Link] (1 responses)

En anglais, on dit 'competitor', et non 'concurrent'. ;-)

How to kill a web browser

Posted Oct 28, 2004 10:30 UTC (Thu) by zezaz (guest, #5465) [Link]

Drat! Je sentais bien que ça ne collait pas! Concurrent est un superbe faux ami, merci de la correction :)

How to kill a web browser

Posted Oct 21, 2004 17:19 UTC (Thu) by smoogen (subscriber, #97) [Link] (2 responses)

They might even have used a bunch of tests that Spyglass QA (the original writers of MSIE1.0) gave them as part of the sale. Netscape also had a large testing amount of pages that contained lots of bad text for people to try at. The main issue is that it is man-power intensive in looking at pages and seeing a) if a crash occurred and b) where it occurred.

How to kill a web browser

Posted Oct 21, 2004 20:08 UTC (Thu) by Baylink (guest, #755) [Link] (1 responses)

Wow, a large collection of HTML pages broken in various ways for testing browsers. That sounds like a project suited to the Open Source Community...

How to kill a web browser

Posted Oct 22, 2004 2:53 UTC (Fri) by bronson (subscriber, #4806) [Link]

Heck, that sounds like the World Wide Web!

MSIE passes the test because Microsoft used this test

Posted Oct 21, 2004 12:26 UTC (Thu) by pdc (guest, #1353) [Link] (3 responses)

The reason IE passed this test is that this sort of stress testing is done by the IE developers. They obviously take pride in having a parser that can cope with arbitrarily lousy HTML. (The flip-side of this parse-at-all-costs strategy is that there is no reason for people to learn to write non-lousy HTML, which in turn creates headaches for all future browser creators. See the endless discussions of error-handling in XML for details.)

MSIE passes the test because Microsoft used this test

Posted Oct 23, 2004 17:59 UTC (Sat) by jhouchin (guest, #2966) [Link] (1 responses)

I think that IE passes the test because it has to be able to handle all the malformed HTML produced by all the MS Frontpage users. :)
Probably a part MS's embrace and extend. :)

MSIE passes the test because Microsoft used this test

Posted Oct 23, 2004 21:03 UTC (Sat) by mly (guest, #2171) [Link]

Well, if the competing browsers would have rejected the bad HTML in a controlled way, you could have viewed it like that...

Having worked in projects where C++ apps were coded in Visual Studio with Solaris as target environment, I've certainly seen Sun's compiler complain quite correctly over obvious bugs which Microsoft's "friendly" compiler happily swallowed. A sort of "he must have meant..." attitude.

As a programmer, I certainly prefer a compiler that rejects my bugs over one which tries to guess what I intended...but it wasn't difficult to understand why MS did it like that, and I certainly felt that it was a bad thing for the programmer, and a reason not to use Microsofts development tools. (As I returned to that site four years later, it seems everybody stopped compiling anything on Windows before moving to the target platform. :)

In the browser case, whether you see it as "embrace and extend" or not, it certainly speaks in favour of Microsofts browser. (But that still doesn't have tabs, does it?)

MSIE passes the test because Microsoft used this test

Posted Nov 3, 2004 23:45 UTC (Wed) by turpie (guest, #5219) [Link]

Or,
They obviously take pride in having a parser that DOESN'T CRASH with arbitrarily lousy HTML.
There is no point excusing the open source developers for these bugs, because that is what they are bugs. In this case Microsoft has done a better job. If you dont wont your browser to attempt to display invalid HTML thats fine, but it shouldn't crash. It's quite possible that these bugs could be security holes.

The correct thing to do here is admit our software has these problems, work out why it has happened, and implement changes so it can be prevented from happening in the future.

Shooting the messenger

Posted Oct 21, 2004 15:12 UTC (Thu) by bjn (guest, #2179) [Link]

I was thoroughly disgusted with the feedback Michal got (apparently a lot of email flames). Talk
about shooting the messenger.

This isn't just about the specific tests; he came up with a very nice framework and methodology.
The great news about this is that those bugs will get fixed, and the browsers will get stronger.

How to kill a web browser

Posted Oct 21, 2004 18:53 UTC (Thu) by amarjan (guest, #25108) [Link]

Judging by all the comments here, I see part of the reason why these defects have persisted in open source browsers for so long: everyone ignores what's happening in IE-land.

That MSIE passed these tests because of a priori superior coding is nonsense.

It passed these tests because someone found the *exact same* class of bugs in MSIE some years ago, much noise was made of the fact, and the Microsoft folks clearly did a great job of (eventually) fixing these defects.

Did none of you see the 1-valid-HTML-tag-kills-MSIE-even-in-well-formed-pages alerts?

How to kill a web browser

Posted Oct 22, 2004 12:25 UTC (Fri) by KaiRo (subscriber, #1987) [Link]

If someone's interested in seeing how the Mozilla folks are dealing and will deal with that issue, look into bugzilla.mozilla.org bug 264944. This is where the Mozilla developers will work on the issues. Further test cases and/or even help with fixes are welcome!

Work on Mozilla solutions tracked in bug 264944

Posted Oct 22, 2004 12:28 UTC (Fri) by KaiRo (subscriber, #1987) [Link]

How to kill a web browser

Posted Oct 29, 2004 13:55 UTC (Fri) by guest01 (guest, #25274) [Link]

Doesn't this issue simply highlight the fact that given the choice between writing new, exiting code and features into a product is much more likely what someone is going to want to do with their time when volunteering on a software project rather than fixing bugs?

Fixing bugs (especially in code we are unfamiliar with) is mind-numbingly boring and tedious. But being paid well for it certainly helps to justify the activity.