LWN.net Logo

Stability v. security fixes

May 9, 2007

This article was contributed by Jake Edge.

A whole pile of security fixes for Red Hat Enterprise Linux 4 (RHEL4) was released at the beginning of May; this event might not be noteworthy except that some of the vulnerabilities were nearly two years old. This stands in contrast to a recent Red Hat article describing the security track record of RHEL4, which was covered on this page, and made no mention of delays of this sort. Digging in a bit deeper to try and understand why seems logical.

Of the thirteen fixes listed by LWN for that day, eleven are categorized as having low severity by Red Hat, one is moderate and one is important. The latter is a recently reported vulnerability in xscreensaver that was given a CVE number less than a month ago. Of the dozen others, eight had CVE numbers from 2006 and four from 2005.

Red Hat classifies security issues based on their analysis of their impact; both "low" and "moderate" vulnerabilities are unlikely to be exploitable, with "moderate" vulnerabilities having worse consequences if it does happen. Under those definitions, it would certainly seem less important to get those fixes out, but it would also seem like a headache to keep track of them. Fedora Core released fixes for these issues ages ago and those seem to have worked out, why did Red Hat sit on them for RHEL4 for so long? Mark Cox, from the Red Hat Security Response Team explains:

So for example CVE-2005-4268 relies on an attacker giving a victim a carefully crafted rather large cpio file, and getting the victim to open it using the cpio command on a 64-bit platform. Even if the attacker manages that, the ability to lead to code execution is unlikely. So we defer these issues; customers don't want to go through an update and test cycle just to fix such an issue. Then, when other issues of a higher severity come up in the same package, or if we are to release an update to that package for any other reason, we also pick up any fixes we previously deferred.

Red Hat Enterprise Linux errata are batched into periodic 'updates'; what was released this week was Update 5 for Red Hat Enterprise Linux 4.

So, for low and some moderate impact bugs, RHEL4 users must wait for patches until some other issue with that package requires attention and then await the next batch of fixes as an update release. An intervening update cycle is not necessarily enough to push these fixes out as there have been several update releases to RHEL4 since they were reported. RHEL customers prize stability, and delayed security updates is part of Red Hat's process for delivering that stability.

Security issues (and bugs in general) are funny beasts and sometimes their implications do not become clear for a long time. Something that seems to have a low impact is suddenly used in an unexpected way by a worm or some other exploit and the impact needs to be recalculated. By holding back these fixes for seemingly trivial security issues, Red Hat could be setting itself up for an unpleasant security surprise someday.

Some customers may also feel that they are more at risk for a particular issue than Red Hat thinks they are. Perhaps they use cpio frequently on possibly untrusted data on their 64-bit machines. As things currently stand, they had no fix available to them (at least via the normal Red Hat update means) for more than a year; there was no easy way for them to even know there is a problem. Red Hat tracks these bugs via bugzilla which is open for anyone to use, but they only put out announcements when they release a fix. It is hard to argue that customers should be trolling security lists and/or bugzilla looking for security issues that might affect them; this is, after all, what they pay Red Hat for.

As with seemingly everything in the world of computers, there is a trade-off here; very few customers would want to be upgrading their production systems frequently for low impact bugs. On the other hand, they may not want to be exposed forever to those same low impact bugs. Batching these kinds of fixes up into security updates once or twice a year seems like a reasonable plan, but holding on to updates for over a year may be just a bit more stability than some customers were looking for.


(Log in to post comments)

Separate update repositories?

Posted May 10, 2007 4:49 UTC (Thu) by sweikart (guest, #4276) [Link]

This makes me wonder whether Red Hat should have separate repositories/channels for customers who value security over stability. Updates with security fixes could be released to the security-prioritized repositories ASAP. This has the added benefit of turning the security-sensitive customers into beta testers for the stability-sensitive customers.

Separate update repositories?

Posted May 10, 2007 7:31 UTC (Thu) by janfrode (subscriber, #244) [Link]

They already offer this as FasTrack http://www.redhat.com/rhn/rhndetails/fastrack/

Stability v. security fixes

Posted May 10, 2007 7:42 UTC (Thu) by mjcox@redhat.com (subscriber, #31775) [Link]

The cpio issue isn't running cpio on untrusted data, the overflow is in the filename handling, so you'd need to run cpio on a rather suspicious looking filename (where the filename contains the trigger for this issue, shellcode etc)

You can find out what issues we've deferred at any point in time by doing a bugzilla query against the product of interest where Keyword = "Security". We now also publish statements directly into the National Vulnerability Database (nvd.nist.gov) for these issues.

Where a package has been selected for the Update for some other reason (in
the cpio case to fix a couple of bugs), we'll also take that opportunity to
include fixes for any security issues we previously deferred. Whenever we include a fix for a security issue, even a low severity one, in
an Update release, we will promote it to a security update and include the CVE; even if it does mean we'll end up as being counted as fixing more vulnerabilities or having longer days of risk metrics for these low severity issues.

Stability v. security fixes

Posted May 10, 2007 8:12 UTC (Thu) by addw (guest, #1771) [Link]

The only reason why you may want to delay outputting a fix is if it might break something else; ie the fixed version is in some way incompatible with the previous version.

I would really doubt that the fixed cpio would break any backup/... script, so what is the harm in releasing it?

One of the reasons for paying for RedHat is the nice warm feeling that you are being looked after. If fixes are delayed like this you are allowing a cold draft into the blanket.

Stability v. security fixes

Posted May 10, 2007 14:50 UTC (Thu) by uravanbob (subscriber, #4050) [Link]

Actually, my industrial customers see ANY change as a requirement to recertify the software - this is of course very expensive. It is not always a completely rational view, but then it is their systems that they are making the decisions for. In this case we are talking about security fixes for problems that rate very low on the risk scale - security is very much a risk management game, so as long as RH makes these fixes available to those who feel they need it, I see no cause for complaints other than that RH is penalized in the counting game.

As a developer, it is very frustrating when a user wont apply a patch, however it is even more annoying when a 'minor - should not affect anything' change has major consequences because well, we're human and screwed up somewhere.

Stability v. security fixes

Posted May 10, 2007 17:06 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link]

the complaint people are makeing is that RedHat didn't make these patches available for a long time (over a year in several cases)

Stability v. security fixes

Posted May 10, 2007 8:27 UTC (Thu) by mjcox@redhat.com (subscriber, #31775) [Link]

(My team corrected me that the cpio issue is caused by a file with a carefully crafted rather large filesize, not filename -- see http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=172669 )

Stability v. security fixes

Posted May 14, 2007 20:23 UTC (Mon) by Brenner (subscriber, #28232) [Link]

Once again, LWN proves its value...

Not only do we get a good article, but we also get _very_ informed comments about the issue. (By the way, this extra information justifies an edit to the original article IMHO.)

I have been a RHEL customer for over two years, and I did not know about neither FasTrack, nor the use of Keyword = "Security" in redhat's bugzilla.

Thanks, LWN !

Copyright © 2007, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds