Cascading security updates

When following the distributions' security updates on a daily basis, as we do at LWN, certain days are more work than others. Two weeks ago we had a rather full update with no less than 28 packages updated for Fedora (most of those for both F7 and F8), along with a handful of updates from other distributions. It turns out that the majority of the Fedora updates had a single cause: a set of serious vulnerabilities in Mozilla Firefox.

How does a single update to an application ripple so far that more than a dozen packages have to be rebuilt? One would think there would be shared libraries that would get updated, with applications picking up those changes the next time they are run. That is, in theory, how things are supposed to work, but in this case, the underlying libraries have no fixed application binary interface (ABI). So, changes to those libraries require any applications that use them to be rebuilt and retested.

Gecko is the rendering engine used by Mozilla in their products to display HTML. Various other packages have started using it as well because of its speed and standards compliance. Because Mozilla sometimes breaks the ABI between releases, even minor releases, distributions may be stuck rebuilding those applications when a new version of the library is released. Normally, that only happens when packaging a new version of the distribution—or when serious security flaws are found.

Mozilla's solution for this problem is XULRunner which will provide a stable ABI for applications. As XULRunner and its companion libxul become more widely available, the applications that currently link to the Gecko libraries will presumably switch to avoid these kinds of problems in the future. It is highly unlikely that we have seen the last security problem in the Gecko engine, so reducing the cascade that results from finding one would be welcome.

Because of problems with the ABI changing in the past, Fedora chooses to make the applications' library version number exactly track the Mozilla release number. Some other distributions do not do that, so unless the ABI does change, they do not need to update each package that uses the libraries. This has some advantages, but could lead to broken applications if an ABI change goes unnoticed.

We have also seen similar cascades of updates, most notably from the xpdf PDF viewer. Unlike Gecko, there is no library for xpdf, leading multiple applications to include its source into their own. When a flaw is found, several different applications (cups, gpdf, etc.) across all distributions need to be updated immediately, leading to a similar effect as was seen with the Gecko vulnerabilities. Hopefully, over time, the development of the poppler library will mitigate this problem somewhat.

There are lots of good reasons to separate code into components where possible, but security is an important one. Creating and maintaining an ABI is sometimes difficult, but generally worth the trouble. Imagine the chaos that could result from a security vulnerability requiring an ABI change in glibc.

A good example of ABI backwards-compatibility is GTK+, which is still compatible back to
version 2.0, released 6-7 years ago. 

I wish more effort was made on ABI forward-compatibility (symbol versioning), which very few
libraries seem to care about. Without it, you have to build your program against the oldest
version of the library that you depend on to get a portable binary. This would also make
things like distribution upgrades a lot safer and more flexible (like being able to upgrade
one package at a time).

Symbol versioning doesn't help there unless your applications explicitly state which versions
of symbols they use (which none do because it's so annoying).

Instead they implicitly rely on a GNU extension whereby a symbol with a @@ between name and
version (rather than a single @) becomes the default version and is used if none is specified.
This is invariably the latest symbol version available for that library. So you don't gain
backcompat: what you gain is forward-compat (until you relink, anyway).

There's a classic discussion of ABI maintainability
by Paul Stachour (which I typed in) at Multicians.ORG:

http://www.multicians.org/stachour.html

--dave

Times really have changed when people are using Gecko because of its *speed*. :)

(of course, this is largely because it's sped up, although computers getting faster has played
some part.)

(btw, KHTML *does* have a published API and stable ABI. ;} )

Hmm, is there really anything that uses xpdf and not libpoppler today? (Other than xpdf
itself...)

Way too many things: I think I still have more copy-of-xpdfers on my local system than poppler
users. One which springs to mind (because I upgraded it yesterday) is CUPS.