Weekly Edition
Return to the Front page
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
RHEL, kernel vulnerabilities, and days of risk
Security Innovation has joined the elite group of Microsoft-funded
researchers who somehow manage to reach pro-Microsoft conclusions. This
company's latest output is a report
on the relative security of Linux and Windows web servers [PDF] which
states that Windows is more secure, in this role, than Red Hat Enterprise
Linux. The group did its work
by looking at all of the vulnerabilities fixed by each vendor in 2004 (as
designated by CVE numbers), and determining how much time passed between
the initial disclosure of the problem and the resulting fix. Windows
showed fewer vulnerabilities, and significantly fewer "days of risk" when
disclosed problems lacked a patch.
Those who want to poke holes in this study should be able to find ample opportunity. Microsoft vulnerabilities are less likely to be disclosed prior to patching, to the point that the median "days of risk" for Windows was zero. The report cautions against writing off "low risk" vulnerabilities, but, somehow, Microsoft simply does not have any "low risk" problems. Either that, or Microsoft doesn't bother to fix them, resulting in many undisclosed "days of risk." Red Hat will also have gotten burned by this libpng vulnerability, which, by mistake, remained unfixed for two years. That's a lot of days of risk, even though no known exploits of this vulnerability took place. Let's focus on one specific claim, however:
There were thirty one [RHEL] vulnerabilities fixed in 2004 that had
more than 90 days of risk, and of these, seven were designated by
ICAT as high severity... Eleven of these vulnerabilities were in
the operating system kernel.
The report does not list the actual vulnerabilities it looked at, so we'll have to try to reproduce that work ourselves. Here's the kernel vulnerabilities fixed by Red Hat in 2004:
The attentive reader may have noticed that this is a rather long list of vulnerabilities. Summed up, it amounts to a total of 2943 days of risk - a substantial portion of the 12,415 days of risk cited in the report. One immediate conclusion is that, in many cases, we are talking about "days of very low risk." The strncpy() information leak was worth fixing, but few people were likely to be overly worried during the 305 days it took for Red Hat to issue updates with that fix. The same is true of the TTY character count leak (737 days of risk). Both ia-64 users could probably live with the floating point leak on that platform (209 days of risk). In other words, many of the vulnerabilities which had a big contribution to the total number of days of risk were of little concern. On the other hand, Red Hat was slow in fixing some important problems. The kmod denial of service and ELF vulnerabilities took months to fix - and they were clearly (locally) exploitable problems. Red Hat is, at times, leaving its paying customers with known security problems for longer than it should. Interestingly, many of these problems were fixed more quickly in other distributions - including Fedora Core. Red Hat's stability goals for its Enterprise Linux line could be an issue here. The need for more stress and regression testing of kernel updates, combined with a clear wish to minimize the number of disruptive kernel updates (many updates fixed several vulnerabilities), is causing those updates to be delayed. Thus, one might draw the ironic conclusion that, if you want the fastest security updates, you're better off not paying for them. There are some more predictable conclusions as well. One is that reports like the one from Security Innovation still do not mean a whole lot. There are too many variables; it is hard to get a handle on which system is truly more secure, and it is too easy to tilt the data in one direction or the other. Of course, one could look at the number and cost of actual security incidents, but these Microsoft-funded surveys tend not to do that. The final, predictable conclusion is this: regardless of how Linux performs relative to other systems, we are not doing nearly well enough. As long as we are producing such long lists of bugs (for a single system component), our claims to security will only hold so much water. (Log in to post comments)
RHEL, kernel vulnerabilities, and days of risk Posted Mar 23, 2005 20:30 UTC (Wed) by szoth (guest, #14825) [Link] "one might draw the ironic confusion that"
confusion -> conclusion?
RHEL, kernel vulnerabilities, and days of risk Posted Mar 23, 2005 20:44 UTC (Wed) by corbet (editor, #1) [Link] It seems "confusion" was closer to the mark. Fixed now.For future reference, typo reports mailed to lwn@lwn.net are likely to be acted on more quickly, and do not clutter up the article with comments.
RHEL, kernel vulnerabilities, and days of risk Posted Mar 23, 2005 20:42 UTC (Wed) by iabervon (subscriber, #722) [Link] Both ia-64 users could probably live with the floating point leak on that platform (209 days of risk).The study only covered x86, so this was (presumably) not included.
RHEL, kernel vulnerabilities, and days of risk Posted Mar 24, 2005 17:31 UTC (Thu) by j0el (guest, #1764) [Link] Both? According to IDC 30,473 ia64 servers have shipped with Linux since Q3 of 2001. 11,515 RISC servers (Power, MIPS, ALpha, SPARC, PA-RISC....) have shipped with Linux since Q1 of 99. Obviously not in the same class as the x86-32 and x86-64, but certainly substantial.
RHEL, kernel vulnerabilities, and days of risk Posted Mar 23, 2005 21:53 UTC (Wed) by dang (guest, #310) [Link] I am sure that the stability issue is a consideration, and it may be that any vendor is slightly screwed here.
Because this RHEL is targeted in part to backend DB tasks, there will be pressure to test long and hard before an rpm is revved. This is partly because there is zero room for problems on your platform db and partly because your friendly neighborhood DBA doesn't care so much about these types of bugs: If your db box is on a routable network or if you have users on the system you are already and always pooched. So if you want to rev an rpm on a DB box, you had better be damn sure that you mess nothing up because the cost/benefit for these types of changes aren't worth the cost of even a DB hiccup.
On the other hand, RHEL is also targeted to boxes with at least 80 and 443 public facing ( typically because you need or want certification for the client that connects to your backend db ), and here the pressure is different. Sysadmins for public facing boxes differ in their paranoia than do friendly neighborhood DBAs; they want these things fixed *now*. Then again, they usually have a lot of wiggle room to grab patches or patched SRPMS that are made available through other channels, so they don't necessarily have to wait.
I think any Enterprise distro is going to struggle a bit with these divergent pressures and will have to choose whether to offically release early ( and run the risk of DBAs bailing on updates ) or release later ( and have DBAs more comfortable with updates but have sysadmins a bit more skittery ). Ultimately I'm not sure that it matters as long as the notices go out and the fixes are available somewhere. The marketing weasels will always be there to spin things, though. That is the tricky part.
Re: public web service vs backend DBMS Posted Mar 25, 2005 4:54 UTC (Fri) by sweikart (guest, #4276) [Link] You make an excellent point.
If an enterprise distribution is going to support both markets, it seems like it needs to have two kernel tracks: for the public service, a quick release (one kernel release per security vulnerability); for the backend service, a slow release (which will typically bundle multiple security fixes, and will have been thoroughly tested by third-party software providers and hardware manufacturers).
At the moment, it seems like the enterprise vendors only provide the latter.
-scott
RHEL, kernel vulnerabilities, and days of risk Posted Mar 23, 2005 22:01 UTC (Wed) by tomsi (subscriber, #2306) [Link] Red Hat's stability goals for its Enterprise Linux line could be an issue here.These cases are reported as vulnerabilites. Were any expoits to these vulnerabilites created before the RedHat fixed the problems ? If there weren't any known exploits, maybe that influenced the long time it took to make a fix? Not that I would recommend that...
RHEL, kernel vulnerabilities, and days of risk Posted Mar 23, 2005 23:24 UTC (Wed) by mjc@redhat.com (guest, #2303) [Link] The headline metrics in the Security Innovation don't seem to match the stats we keep ourselves, and it would be interesting to see their raw data to see if there is some error in their methods, or perhaps if it's an effect of sampling on the package or vulnerability set they used.
The Red Hat Security Response team publish all our raw data so you can run your own metrics of interest against any set of vulnerabilities. For every CVE name we also give the impact we assigned to the issue (and running the metrics you'll see we do fix things that are most important quicker).
A link to the data and a small perl script to run metrics is available
But "days of risk" only tells you a portion of the story: it doesn't tell you how long a vendor knew about an issue before it was known to the public, or how long it was being exploited before being reported. We're trying to be a bit more transparant on that too, if you follow the bugzilla links in RHSA or Fedora Core advisories we've started this year adding "reported" date fields to the status_whiteboard section.
RHEL, kernel vulnerabilities, and days of risk Posted Mar 24, 2005 9:11 UTC (Thu) by freddyh (subscriber, #21133) [Link] But "days of risk" only tells you a portion of the story: it doesn't tell you how long a vendor knew about an issue before it was known to the public, or how long it was being exploited before being reported. Indeed. More than often, Microsoft replies to bug-reports with: "no, that's not a problem". 3 Months later they come out with a patch saying: "we found this 3 weeks ago during our own tests". What also sometimes happens is that Microsoft brings out a patch for a problem, that doesn't really solve anything. When people complain, the response usually is: "no, this is a new problem", causing the counter to start all over again. This *is* a problem, as most people higher up in organisations tend not to look to deep into how things were compared and simply draw their conclusions. "If it's written down, it must be true..." On the other hand, I tend to agree with Jonathan: we are doing very well, but could still do better... FreddyH
RHEL, kernel vulnerabilities, and days of risk Posted Mar 24, 2005 1:26 UTC (Thu) by brianomahoney (subscriber, #6206) [Link] There are three, orthogonal, conclusions to be drawn from all this nonsense:
1. Security is now a REAL purchase descriminator, and M$ is scared,
2. The PHB effect is alive and well, M$ exploits FUD and flawed statistics.
3. Re-learned lessons eg a non-executable stack, enforced by hardware;
The reality though is practical experience; I have been on the broadband,
RHEL, kernel vulnerabilities, and days of risk Posted Mar 24, 2005 16:03 UTC (Thu) by vonbrand (subscriber, #4458) [Link] Lucky you. Some security upgrade to Solaris here around 2000 (re)installed a remote administration package (which we had removed). Said package had (known) holes you could drive trucks through... and some kid promptly remotely administered the machine. Had a nice 24 hour day restoring everything and making sure no further holes were present. That incident, BTW, gave the last argument for migrating the servers to Linux (many desktops had been Linux for quite some time). First on the (ageing) Suns (even got better performance with Linux!), later on custom-build PCs. We see lots of (mostly very pathetic) intrusion attempts, no success to date (AFAIK...).
RHEL, kernel vulnerabilities, and days of risk Posted Mar 28, 2005 16:43 UTC (Mon) by Ross (guest, #4065) [Link] To be fair, the same problem (patches reenabling services) happens on Linuxas well. Debian is especially bad about this due to the "if it is installed we assume you want to run it" policy.
RHEL, kernel vulnerabilities, and days of risk Posted Mar 24, 2005 23:29 UTC (Thu) by cdmiller (subscriber, #2813) [Link] Much of the security of any Internet server is in the hands of the administrator. Vulnerability stats <> the security of a system. A good admin with enough time to do the job generally results in a secure installation.
RHEL, kernel vulnerabilities, and days of risk Posted Mar 25, 2005 0:38 UTC (Fri) by giraffedata (subscriber, #1954) [Link] Those who want to poke holes in this study should be able to find ample opportunity. Microsoft vulnerabilities are less likely to be disclosed prior to patching, to the point that the median "days of risk" for Windows was zero.That's not a hole. It shores up the conclusion by offering an explanation. If Microsoft can keep vulnerabilities secret until there's a fix for them, that's a reason to run your web server on Microsoft products instead of open source.
RHEL, kernel vulnerabilities, and days of risk Posted Mar 25, 2005 3:26 UTC (Fri) by bronson (subscriber, #4806) [Link] Starting the clock from the moment of disclosure isn't very informative. Risk starts well before then. Microsoft has proven better at keeping secrets from its own customers than it is at keeping secrets from black hat attackers...
RHEL, kernel vulnerabilities, and days of risk Posted Mar 25, 2005 3:41 UTC (Fri) by giraffedata (subscriber, #1954) [Link] Don't you think the risk is many times greater after it has become public knowledge?If only a few people know about a vulnerability, the chance is far less that one of them will attack my particular system than if the whole world knows. If I could have a system that is vulnerable only to bugs that are a closely held secret, I think I've eliminated a substantial portion of my risk.
RHEL, kernel vulnerabilities, and days of risk Posted Mar 25, 2005 8:08 UTC (Fri) by khim (subscriber, #9252) [Link] Ahh, good old "Security Through Obscurity" argument. It does work this way for small-scale systems (something like 10 or 100 installations). It does not work this way for widely deployed systems like Linux and Windows. "Black hats" do have incentive to find and use holes in Windows, "white hats" do not (what good will it do if you can not even ask for patch and doing it yourself is out of question?). Since most security problems are found in Linux with code auditing it's usually not exactly clear how to exploit hole at all for a long time but Windows holes are only ever acknowleadged when exploit is shown: there are still unpatched holes with obvious "buffer overflow crash" symptoms going back to NT 4.0 - but since exploits and not flying around Microsoft can claim "it's not a security problem - it's a feature". So in effect you are not having a system with a "bugs that are a closely held secret" but rather you are having a system with a "bugs that are not widely exploited yet" - quite a difference.
RHEL, kernel vulnerabilities, and days of risk Posted Mar 28, 2005 17:35 UTC (Mon) by giraffedata (subscriber, #1954) [Link] Those are all good points about security, but don't address the topic of this thread: whether the days of risk measurement in this study is a valid measurement for comparing security risk among programs. The article suggests that some bugs have an unfairly low days of risk measurement. The bugs you describe that are not a closely held secret, but just not widely exploited, would have a high days of risk measurement. So they're not the ones we're talking about. Also, I don't think whether Microsoft acknowledges a bug is part of the measurement. But one could imagine that the study biased the measurement by looking only to the easy sources -- Microsoft announcements -- to find out when bugs became known. That would make the issue moot.
RHEL, kernel vulnerabilities, and days of risk Posted Mar 28, 2005 16:45 UTC (Mon) by Ross (guest, #4065) [Link] What you say would only be true if the only way to find out about thevulnerability were from Microsoft. Obviously that is not true given that Microsoft is usually made aware of problems by third parties.
Obscurity is a poor substitute for security.
RHEL, kernel vulnerabilities, and days of risk Posted Mar 28, 2005 17:16 UTC (Mon) by giraffedata (subscriber, #1954) [Link] What you say would only be true if the only way to find out about the vulnerability were from Microsoft.You misread my comment. When I said "if Microsoft can keep vulnerabilities secret," I didn't mean if Microsoft can avoid telling people about them (though I understand that's sometimes what "keep secret" means). I meant if Microsoft can keep the vulnerabilities from becoming general knowledge. To the extent that Microsoft can't do that, because other people find and expose the vulnerabilities, my comment doesn't apply. But the article suggests that Microsoft can to some extent keep the vulnerabilities secret, because it says Microsoft lessens its "days of risk" measurement by not disclosing bugs.
RHEL, kernel vulnerabilities, and days of risk Posted Mar 28, 2005 17:23 UTC (Mon) by giraffedata (subscriber, #1954) [Link] Obscurity is a poor substitute for security.On the other hand, it's a great subsitute for no security and public knowledge of that fact. So I think the obscurity is worth measuring and using to gauge one's risk of using particular software.
RHEL, kernel vulnerabilities, and days of risk Posted Mar 25, 2005 3:30 UTC (Fri) by quickening (guest, #14807) [Link] Has anyone seen a study of days of risk from known exploits? How about number of successful intrusions? A simple up-time metric for web servers would indirectly indicate Linux is far more secure than Microsoft.
I am personally familiar with a samba DOS exploit against Win2K3 (it remotely crashes the box!) which took Microsoft 3 months to patch after I told them about it, and that patch is still not part of a service pack - and probably not installed on most servers. Something really stinks when you know there's lots of these patches which Microsoft never bothers to publish, and which never make it into these "official" studies.
RHEL, kernel vulnerabilities, and days of risk Posted Mar 25, 2005 8:10 UTC (Fri) by khim (subscriber, #9252) [Link] Exactly! If you can remotely crash the box then chances are high it's buffer overflow somewhere and that it can be used as exploit - but since exploit is not shown Microsoft can claim "it's not a security issue". When and if exploit will be shown they'll claim "it's new, just discovered problem" and counter will be reset...
detection differences Posted Mar 31, 2005 17:28 UTC (Thu) by guest01 (guest, #25274) [Link] There's one point I don't think anyone else has brought up. Aren't most Linux bugs discovered by people looking at the code while most Microsoft bugs are discovered by someone poking holes at a machine running Microsoft (i.e. through exploit program attempts)?
If this is true, I'd rather run an OS that has dozens of people per week/month saying "I was looking at this code and I think I've found a potential problem" rather than an OS that has a handful of people saying "I can root your machine, here's the program that does it".
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||