Leading items

Some notes from the Coverity survey

Back in January, LWN reported on a grant awarded to Coverity by the U.S. Department of Homeland Security. Coverity (working with Stanford) would apply its static analysis tools to the code bases of a large set of free software projects and report on the results. The effort was designed to help provide a sense of the quality of free software while simultaneously helping to improve that quality.

Coverity has now announced its first set of results in the form of a press release, a table of defect counts, and a glossy report. The main point made in the report - and picked up on by most of the media coverage - is that the software which makes up the "LAMP stack" (kernel, Apache, MySQL, PostgreSQL, PHP, Perl, Python) has a significantly lower rate of defects than the larger set of projects reviewed. From this result, one might well conclude that the most heavily-used and carefully-reviewed projects tend to have better code. Perhaps not a breathtaking result, but it's still nice to know.

The projects with the lowest defect density include Ethereal, OpenVPN, Perl, and xmms; the all-time winner is xmms, with a total of six detected errors. At the other end of the scale, one finds Amanda, Firebird, NetSNMP, OpenLDAP, Samba, X, and Xine. The MySQL code base turned up 136 defects (a density of 0.224 per thousand lines of code), while PostgreSQL has 295 (density of 0.362). Those results are interesting in the context of this quote from the report:

We asked Coverity CTO Ben Chelf about the discrepancy between this claim and the published results, and heard back:

Unfortunately, that response does not really answer the question. The possibilities would seem to be: (1) whoever paid for the "certified versions" has not fed the resulting fixes back into the mainline; (2) all of the detected defects have been introduced into the code base since the certification run was done, or (3) the tests run on the "certified versions" were less comprehensive. None of those ideas is particularly reassuring.

That notwithstanding, the work being done at Coverity is clearly helping to clean up the code of the projects being surveyed. Patches for some bugs found in the kernel are already circulating, and various other projects are looking at the results as well. With regard to Samba, the Coverity folks provided us with a quote from Jeremy Allison:

Running static analysis tools on the code is a clear win for software quality and Coverity, by chasing down the resources to pay for this kind of work, is helping the free software community. Even so, we could not resist asking Mr. Chelf this question: wouldn't it help the community even more to release the checker under a free license, so that the community could do its own analysis and improve the tool as well? He responded:

In other words, we'll have to content ourselves with the reports from Coverity - when Coverity sees fit to provide them - for the foreseeable future. It is vastly preferable to not having those reports.

Still, there would be a great advantage to having static analysis tools which did not depend on any one corporation's generosity to run. The community seems to be a bit slow in the development of these tools, however. The "sparse" utility, written by Linus Torvalds, is regularly used to find certain types of bugs in the kernel. It has seen little use beyond the kernel, however, and has not developed anything close to the capabilities of Coverity's tools. The once-promising smatch project seems to have stalled for the last two years. Various other projects exist (Wikipedia has a list), but none seem to have reached any sort of critical mass.

The free software community prides itself on the quality of its code. Static analysis techniques will clearly be an important part of maintaining that quality in the future. Many eyeballs do indeed shake out bugs; adding some automated eyeballs to the mix will help find even more of them. We have been lucky that a company which has developed some interesting static analysis techniques has - for a few years, now - shared the results of its analysis with parts of the free software community. We should hope that this generosity continues for a long time, but we may also want to think about creating some tools of our own for the day when that generosity runs out.

Comments (43 posted)

The next generation office suite

OpenOffice.org is a great package. It provides powerful capabilities in a number of areas - document editing, spreadsheets, presentations, etc. - and makes it possible for Linux users to interoperate with the large part of the world which is dependent on proprietary office applications. Much of the time, OpenOffice is the tool needed to enable Linux to replace a proprietary desktop system. It would be a hard tool to live without.

That said, there is some truth in a comment recently posted by Jeff Waugh:

This statement is, perhaps, not entirely true; OpenOffice has, for example, been a big part of the push toward the Open Document Format. The open format push has most certainly shifted the battle, to the point that even Microsoft has had to respond. Beyond that, however, it is hard to point to a long list of new things which OpenOffice has brought to the office productivity arena. It is mostly a good copy of that other office application.

Critics of free software are fond of claims that the community is restricted to imitating developments done in the proprietary world. Free software, it is said, is not where innovation is done. To a great extent, OpenOffice could be said to validate that claim. It is not clear that this situation can change; OpenOffice is a large and intimidating code base which can be hard to contribute to, and the project's mission would seem to argue against the creation of surprising new features.

The community is not limited to OpenOffice, however. Jeff's posting points to a weblog entry by Marc Maurer, wherein he (by way of a large Flash file) demonstrates the long-anticipated collaborative editing addition to AbiWord. Authors, connected by the net, can simultaneously work on the same document and see each others' changes as they happen. Now every document can be written by committee, a process known to produce superior results.

Seriously, however, there are clear advantages to being able to work in this mode. Perhaps the tiresome process of sending document files around as attachments and trying to integrate changes from others could eventually fade away. And the world has shown, many times, that if people are given new ways to communicate and work together, they will do surprising things with that capability. So this addition to AbiWord (hopefully due to show up in the 2.6 release) is a welcome step forward.

Meanwhile, the KDE project recently held a "GUI and functionality design competition" for KOffice 2. the results of this competition have now been posted; they show that a number of smart people are thinking about where KOffice could go from here. The winning entry [PDF] from Martin Pfeiffer takes a long look at how people work with documents. His ideas, if realized, could take much of the tiresome clicking out of the editing process and make the task of putting together documents (especially large ones) much more straightforward and fun.

The fact that much effort in the free software community has gone into the replication of features available elsewhere is not particularly surprising. If one wants to build a user community for a software package, one is well advised to provide the capabilities that the target users have come to expect. In many areas, however, that goal has been met, and the time has come to move into new capabilities that users do not - yet - expect to find. By many accounts, office suites are one of those areas. We have the capabilities that most users need; it will be fun to watch as developers create features that users do not yet know that they need.

Comments (12 posted)

Some lessons from MythTV

Your editor's eighth-grade son was looking around for an end-of-year school project. Fearing the alternatives (most of which seemed to involve explosives), your editor made the logical suggestion: let's build a MythTV box together. That project looked like a good Linux learning project which might just yield a device which would be useful around the house. Plus, with what he thought was expert Linux guidance (kids are so gullible sometimes), the project couldn't fail.

Well, it didn't fail, but it was not always clear that a successful outcome was in the works. For the benefit of others who may be considering the creation of such a box, here's a few things your editor learned on the way.

Do not expect it to be easy. Contemporary Linux users tend to be a spoiled bunch. For the most part, any of thousands of programs can be installed by way of a single package manager operation. Often these programs come pre-configured in some sort of minimally working way; finishing the job is just a matter of making a few tweaks. So what could be so hard about installing MythTV? After all, there are packages for many distributions just waiting to be used.

Even with pre-built packages, installing MythTV reminds your editor of installing Linux back in 1993. Remember trying to come up with an XFree86 configuration file for a previously unknown monitor? MythTV is somewhat like that. There's a great deal of configuration to perform, and a lot of parameters to tweak. Get one wrong, and the whole thing fails in mysterious ways. Anybody who is not up for a long setup experience would be well advised to stick with simpler tasks - like writing new sendmail rulesets.

Choose your hardware with care. MythTV requires a fairly strong system in general; it's not a suitable application for that Pentium 100 system gathering dust in the basement. A capable (but supported!) video card is required. Then, there is the issue of choosing a TV card.

Your editor, after some digging, stumbled across the pcHDTV HD3000 tuner card. It had a number of seemingly nice features, such as the ability to tune in high-definition TV broadcasts while avoiding obvious obnoxious misfeatures - broadcast flag compliance, for example. What won your editor's heart, however, was the statement that, while Linux was supported, Windows drivers were not available. How could a card which supported only Linux fail to work?

And it does work, once one gets it configured correctly. That involves tracking down the firmware and putting it in the right place, ensuring that the correct modules get loaded (something that doesn't seem to happen by default), and going through a lengthy process of figuring out which stations can actually be tuned and carefully instructing MythTV to avoid all the others. That last step, incidentally, requires a development version of the dvb-apps package obtained from CVS. Then one finds out that, in order to cope with a high-definition signal, one needs a seriously fast processor; that 1.8GHz Athlon you have gathering dust in the basement just won't cut it. Meanwhile, getting plain old, low-resolution TV out of the card, while said to be possible, has proved to be a challenge in real life.

Expect pitfalls. One of the many MythTV configuration screens is for setting up the TV card(s). One of the options given there is the pcHDTV HD3000. Every day, some well-meaning MythTV user probably tells the system that his or her pcHDTV HD3000 is a pcHDTV HD3000, while a hundred experienced users, if they only knew, would be shouting "NO, YOU FOOL! It's a trap!" at the top of their lungs. This poor user is heading for some significant pain; MythTV will never work in that configuration.

As the battle-hardened veterans know, an HD3000 card should be configured as a DVB device (described in the documentation as "a video standard primarily found in Europe"). Then it will work. One can only imagine a legion of sadistic MythTV hackers leaving the pcHDTV-HD3000 option on the menu as a way of ensuring that beginning users spend more time staring at Google than watching TV.

The allegedly easy path isn't necessarily so. Part of the work plan involved researching the best distribution for the creation of a MythTV box. What better way for an eighth grader to learn about how Linux systems are created? He quickly settled on KnoppMyth, which comes with claims like:

Why bother with anything else when you can get all of the pieces off a single disk?

KnoppMyth does not appear to be a project which receives a great deal of development time; the 5.0 release has been in the works for quite a while. A number of the download links on the main page are dead. It still uses version 0.18 of MythTV. More to the point, however: while one may not have to edit configuration files, nothing gets one out of the need to go through a couple dozen MythTV setup and configuration screens. There are dozens of operating parameters to tweak. TV cards must be set up. A separate step is required to set up video sources. Yet another step exists just to connect the configured TV cards with the configured video sources. Then there's the set of channel configuration screens. One has to figure out where the programming information will come from and set that up. Then one has to actually make the resulting combination work - something your editor never succeeded in doing.

Among other things, KnoppMyth did not set up the video card (a Radeon 9250-based card) correctly in its XFree86-based graphics system, with the result that the XVideo extension was not available. Suffice to say that MythTV (along with lower-level tools like mplayer) is not happy without XVideo.

So your editor dumped the whole mess and installed Fedora Core 4, which had no trouble figuring out the video configuration. The excellent Fedora Myth(TV)ology document made most of the rest of the setup relatively easy - modulo the level-60 secret incantations required to make the HD3000 work properly.

Don't expect it to tell you anything.. The MythTV setup program will not work properly if the MythTV backend daemon is running. But it won't check for said daemon, and it won't say why it is failing. MythTV has a built-in logging system with eight log levels, but your editor has yet to find anything of interest there. Other things just fail silently, with no indication of why, for example, an attempt to watch TV in real time yields a black screen for ten seconds before returning to the menu.

In summary: MythTV may have a lot of things to recommend it, but there is some work to be done to make it installable by normal people. Today's MythTV reminds your editor of installing early Slackware releases: a long and fiddly process with the occasional trap to avoid. The Linux installation problem has been nicely solved; if the target hardware is supported, putting together a Linux system to use that hardware is usually a straightforward task. What has been done for Linux as a whole can certainly be done for MythTV. Until it has been done, MythTV is likely to be inaccessible to many who would like to use it.

Having written the summary, your editor would like to briefly touch on two other lessons.

It's seriously cool. Once the system works, it does just what it is claimed to do. It can watch and record television, skip over advertisements, move around quickly in the program, handle multiple tuners, juggle conflicting recording schedules, work with a wide variety of remote controls, browse the web, play games, etc. Packaged in a suitably powerful and quiet box, MythTV could be a welcome part of one's larger entertainment complex.

We may not be able to build MythTV boxes for much longer. The capabilities provided by MythTV go against everything the Powers That Be in the entertainment industry want us to have. As they continue to push for hostile legislation and DRM-encumbered hardware, they will eventually make the creation of a MythTV box impossible. Hardware which can tune in tomorrow's signals, and which makes the result available to software that doesn't know the secret handshakes, will be unavailable. MythTV is a powerful - if rough-edged - tool; it's how access to video programming should be. It would be a shame if MythTV were to smooth out the setup experience, only to be obliterated by legal systems worldwide.

Comments (27 posted)

Page editor: Jonathan Corbet
Next page: Security>>