Advisories and relative security [LWN.net]

A recent CNN article asks why the Linux community hasn't used the Blaster and SoBig worms for marketing purposes. The author concludes:

Etiquette and naiveté aside, however, perhaps the biggest reason Linux companies haven't touted their products' security advantages is that it's unclear right now how much of an advantage they really possess. Consider this: The Computer Emergency Response Team (CERT) released data showing that 16 of the 29 security advisories it released last year involved Linux or open-source products.

This seems like a good time to go and look at what these advisories really covered. CERT's 2002 advisories were:

Linux-related	Microsoft	Something else
2002-03 (SNMP) 2002-05 (PHP) 2002-06 (Radius) 2002-07 (zlib) 2002-12 (dhcpd) 2002-15 (BIND9) 2002-17 (Apache) 2002-18 (OpenSSH) 2002-19 (libresolv) 2002-21 (PHP) 2002-23 (OpenSSL) 2002-24 (OpenSSH trojan) 2002-25 (XDR) 2002-27 (mod_ssl worm) 2002-28 (sendmail trojan) 2002-29 (kerbd) 2002-30 (tcpdump trojan) 2002-31 (BIND8)	2002-02 (AOL ICQ) 2002-04 (IE) 2002-09 (IIS) 2002-13 (MSN Chat) 2002-22 (SQL Server) 2002-33 (MDAC) 2002-37 (Windows shell)	2002-01 (CDE) 2002-03 (SNMP) 2002-08 (Oracle) 2002-10 (rpc.walld) 2002-11 (cachefs) 2002-14 (JRun) 2002-16 (Yahoo Messenger) 2002-20 (CDE) 2002-26 (CDE) 2002-32 (OmniSwitch) 2002-34 (Solaris XFS) 2002-35 (RaQ Servers) 2002-36 (proprietary SSH)

Interestingly, we count 37 advisories for last year, not 29. There is no contesting the fact that the Linux-related column is significantly longer than the others. One could quibble a bit: the mod_ssl worm advisory covers the same vulnerability as the OpenSSL advisory, and the three trojan advisories are individual site compromises rather than widespread vulnerabilities. But that sort of quibbling wouldn't really change the situation.

On the other hand, it is a legimate question to ask why the mod_ssl worm (which affected very few systems) merits a CERT advisory, when worms like Klez, Bugbear, Badtrans, Nimda, and Sircam do not. The costs imposed by any one of those worms is likely to exceed that of all the Linux vulnerabilities put together.

The real point is that anybody who tries to make a security point by counting advisories is building a weak argument. A more honest look at the situation would ask how many vulnerabilities have been actively exploited, and how quickly they have been fixed.

That said, we couldn't resist putting together a 2003 table while we were at it:

Linux-related	Microsoft	Something else
2003-01 (dhcpd) 2003-02 (cvs) 2003-07 (sendmail) 2003-10 (XDR) 2003-12 (sendmail) 2003-13 (snort) 2003-21 (GNU FTP crack)	2003-03 (Locator) 2003-04 (SQL server worm) 2003-08 (Windows shares) 2003-09 (ntdll) 2003-14 (html32) 2003-16 (RPC) 2003-18 (DirextX) 2003-19 (RPC exploits) 2003-20 (Blaster) 2003-22 (IE)	2003-05 (Oracle) 2003-06 (SIP) 2003-11 (Lotus) 2003-15 (IOS) 2003-17 (IOS)

This table suggests that the record for Linux-related software is nothing to be all that proud of, but certain other operating systems are currently in the lead in the "advisory count" race. On the other hand, in the fast-changing free software world, it is somehow comforting to see that sendmail continues to give advisory writers something to do - as long as you're running a different MTA...

to post comments

Advisories and relative security

Posted Sep 4, 2003 7:45 UTC (Thu) by addw (guest, #1771) [Link]

The trouble is that any attempt to reduce a complex real life situation to a single metric is doomed to failure. We all, however, like simple numbers as it permits easy comparisons and an unambiguous declaration of a *winner*.

The counting_CERT_advisories approach is too far removed from real life to be anything other than a starting point for where there might have been problems. A much better approach would include:

* cost per user of all security problems (time lost, data lost)
* cost of maintaining security patches as per software vendor's/distrubutor's recommendations. Cost includes: upgrade/... fees; downtime; administrator's time

Even then what do I mean by a 'user' ? Think: web/file servers vs desktops.

BTW: I am a smug exim user :-)

Advisories and relative security

Posted Sep 4, 2003 8:40 UTC (Thu) by AnswerGuy (guest, #1256) [Link] (1 responses)

The metrics here aren't reasonable. When I did software quality assurance
we eschewed the usual categorization of bugs (cat. 1 through cat. 4) for
a three dimensional rating system: virulence, severity, embarassment.

I won't go into details on that; but consider this: In the last 5 years
there have been 3 Linux worms that I can remember: Lion, Ramen, and Adore.
All three of them combined affected perhaps a few 10's of thousands of
systems (and almost all of them a few years ago).

In the last 2 years I can think of Nimda, Code Red, MS-Sapphire (SQL
Slammer), and MS-Blaster the smallest of which affected hundreds of
thousands of systems. The SoBig and other viruses are almost one order
of magnitude more virulent than the worms.

So, when we evaluate the costs and risks based the number of affected
systems and the severity of the compromises --- we see that UNIX and
Linux are a better bet.

Advisories and relative security

Posted Sep 4, 2003 19:34 UTC (Thu) by vonbrand (guest, #4458) [Link]

Yes, but if you factor in that there are a lot less Unix/Linux systems around, the picture isn't all that rosy.

If Linux became dominant, I´d bet it will get a lot more attention from the miscreants who write malware than it gets today. That probably would change the picture quite a bit.

Advisories and relative security

Posted Sep 4, 2003 9:08 UTC (Thu) by ressu (subscriber, #14615) [Link]

Ofcourse it is reasonable to compare software suites when it comes to vulnerabilities, it is not that clear if you wish to compare vulnerabilities in the kernel of linux and windows.

But still, the article doesn't put enough stress on the fact that that is infact Applications running on linux vs. applications made by microsoft. And when you take that in to account, the linux column should actually have even more vulnerabilities (which kind of makes me wonder). I suppose it has something to do with the fact that there are so many small apps out there that most of them are so uncommon that it is unreasonable to post an vulnerability alert on them, even though i would expect that those small holes cover even more ground than most of the alerts posted.

Advisories and relative security

Posted Sep 4, 2003 12:10 UTC (Thu) by zmi (guest, #4829) [Link]

Just some short comments:

- Why is Outlook (and -Express) not listed? It's THE virus spreader software and
you could even see it as "part" of a virus - where would that SOBIG be without it?

- One should not compare a browser, SQL server, webserver and some small tools
with the Linux world. There are bugs listed of software that does SNMP, there's
even a programming language (PHP), DNS software and so on.

- Why is DNS (BIND), PHP and Apache only under Linux? It exists also for
Windows, and the description of the bugs don't state that it's a Linux-only problem.
Not very fair to count the same bug only for one system.

- Service Pack #3 for W2K contained about 1000 bugfixes, Service Pack #4
contains about 1700 bugfixes.

have fun. zmi.

Advisories and relative security

Posted Sep 4, 2003 13:41 UTC (Thu) by iabervon (subscriber, #722) [Link]

Another major factor you need to consider with CERT advisories is that Linux software doesn't originate at a single point, while most important Windows software comes from MicroSoft. This means that CERT is less significant for Windows, because there's a single point of contact. In the Linux world, it is useful to have an entity to coordinate getting information from various projects to direct users and from projects to distributions; since CERT is more useful, it is likely to be more verbose.

Another note is that practically nobody is actually affected by all of the Linux advisories listed, since there's a BIND9 advisory before a BIND8 one. People are somewhat unlikely to have downgraded after the first one. People could easily have been affected by every single one of the Windows advisories.

Advisories and relative security

Posted Sep 4, 2003 13:41 UTC (Thu) by beejaybee (guest, #1581) [Link]

See e.g. MS03-037 issued Sep 03 (Flaw in Visual Basic for Applications Could Allow Arbitrary Code Execution). One "critical" issue affecting (at least) 29 applications.

If this was a linux advisory, there'd be 29 to count.

Lies, damn lies & statistics...

Advisories and relative security

Posted Sep 4, 2003 14:41 UTC (Thu) by jamesm (guest, #2273) [Link]

One potentially useful metric would be to look at the cost of security insurance premiums for installations of different operating systems.

Same FUD different day

Posted Sep 4, 2003 15:40 UTC (Thu) by cdmiller (guest, #2813) [Link] (1 responses)

I do find it interesting that the great CERT does not report on Linux kernel vulnerabilities.

Anyone know the reasoning for this?

- cameron

Same FUD different day

Posted Sep 29, 2003 13:59 UTC (Mon) by kreutzm (guest, #4700) [Link]

CERN is reporting Kernel vulnerabilities:

CAN-2002-499 CAN-2002-510 CAN-2002-1319 CAN-2003-0018 CAN-2003-0127 CAN-2003-0187 CAN-2003-0244 CAN-2003-0246 CAN-2003-0247 CAN-2003-0248
CAN-2003-0364 CAN-2003-0461 CAN-2003-0462 CAN-2003-0464 CAN-2003-0465
CAN-2003-0467 CAN-2003-0476 CAN-2003-0501 CAN-2003-0550 CAN-2003-0551
CAN-2003-0552 CAN-2003-0619

First table

Posted Sep 4, 2003 16:55 UTC (Thu) by Ross (guest, #4065) [Link] (1 responses)

The first table seems messed up to me. The MS advisories are in the Linux
column and the "other" advisories are in the MS column. The second table
is ok.

First table

Posted Sep 4, 2003 17:02 UTC (Thu) by corbet (editor, #1) [Link]

How about now? There was a </td> missing. I'm not sure how that came to be; our publishing system checks for problems like that... Also, which browser? Galeon was perfectly happy with it as it was...

Advisories and relative security

Posted Sep 5, 2003 6:40 UTC (Fri) by ekj (guest, #1524) [Link] (2 responses)

People should really stop writing all code in languages prone to buffer overflows. We've been trying the "just don't make mistakes" approach for 3 decades, and it is not working.

This is a solved problem people. And there are very few situations where a nice ruby or python or whatever (even perl if that's your fancy!) won't do.

Look at the list of vulnerabilities in this weeks LWN:

gkrellm: buffer overflow
node: buffer overflow, format string
atari800: buffer overflows
libpam-smb: exploitable buffer overflow

It's the same every other week. What is the excuse for this stupidity ? Yes, there are other bugs (this week a bit over half of the problems listed where "other" problems), but this is no excuse not to take the easy and painless way of squashing almost half our security-bugs.

Advisories and relative security

Posted Sep 5, 2003 8:05 UTC (Fri) by beejaybee (guest, #1581) [Link] (1 responses)

" People should really stop writing all code in languages prone to buffer overflows. We've been trying the "just don't make mistakes" approach for 3 decades, and it is not working.

This is a solved problem people. And there are very few situations where a nice ruby or python or whatever (even perl if that's your fancy!) won't do."

O'Really?

Does it not occur to you that there may be buffer overflows in libraries called by applications written in ruby/python/perl?

Does it not occur to you that there are at least some applications where the overhead of weakly typed interpreted languages is simply unacceptable? (Else I'd write only bash shell script!)

Does it not occur to you that the hidden complexity of languages like python makes possible - maybe even inevitable - a whole class of vulnerabilities which haven't been discovered yet?

Security problems are caused primarily by lack of foresight. Buffer overflows are only one aspect of this. Using a weakly typed interpreted language instead of C(++) may prevent _you_ from creating buffer overflow holes, but doesn't prevent them from affecting your software, nor does it prevent you from making any number of other coding mistakes which might result in an exploitable weakness in your application. And the run time cost penalty may sometimes be unacceptable.

Sure, write in python, if it's suitable for the application and you're comfortable with it. But don't be stupid enough to think that this action alone will solve the problem of insecure code.

Languages and security

Posted Sep 12, 2003 10:13 UTC (Fri) by dvdeug (guest, #10998) [Link]

Does it not occur to you that there may be buffer overflows in libraries called by applications written in ruby/python/perl?

Sure, but at least the problems in your application will be gone. And if the overflow is in the library, all the programs that use that library can be fixed with one fix.

Does it not occur to you that there are at least some applications where the overhead of weakly typed interpreted languages is simply unacceptable?

Since when have all buffer checked languags been weakly typed interpreted languages? Ada is strongly typed and compiled, as is Modula-3 and ML.

Does it not occur to you that the hidden complexity of languages like python makes possible - maybe even inevitable - a whole class of vulnerabilities which haven't been discovered yet?

The vulnerability that hasn't been discovered yet isn't a problem, because no one's exploiting it. In any case, again Ada and Modula-3 don't have the hidden complexity of Python or even C++. (Pound for pound, Ada is simpler then C++.)

Security problems are caused primarily by lack of foresight. Buffer overflows are only one aspect of this.

But they seem to be a huge part of it.

And the run time cost penalty may sometimes be unacceptable.

Ada, for one, can be compiled with buffer checking in only in debug mode. In any case, worst case scenario seems to be about 20% for bounds checking, and while there's code that's written in assembly to eak out those last few percents, there's a whole lot of servers that are only CPU bound when running CGI scripts (i.e. Perl and friends.) Movie players and a/v encoders seem to be the only things I'm worried about speed wise on my P3/450 that serves as my desktop.

But don't be stupid enough to think that this action alone will solve the problem of insecure code.

We put airbags in cars, despite the fact that side collisions happen. Solving one part of the problem, a big part of the problem, is certainly an improvement over doing nothing because there's no silver bullet that will solve the whole thing.