Rate of bugs and rate of security holes are mostly uncorrelated

Posted Mar 6, 2006 15:33 UTC (Mon) by nix (subscriber, #2304)
Parent article: Coverity releases first defect survey results

Part of this is a question of the severity of any given bug. Ethereal runs with high privileges and listens to the network, so that virtually any crash or hang can be parlayed into a remote attack: X doesn't listen so promiscuously (on most distros), so its high rate is comparatively safe; and GCC's low rate doesn't help much, as it's so widely used and so large that even a low rate of bugs-per-line translates into a large number of actual bugs (although few are security holes because not many things other than the cursed libtool --mode=install ever run GCC as root).

(In any case, the more complex the code, the less likely it is that a static checker like Coverity's will isolate real faults without special understanding of that project. It has special understanding of the Linux kernel's idioms, IIRC, but it certainly doesn't have special understanding of the idioms used in GCC , say, so it can't find most of the classes of flaws that are really found in GCC. Of course, if Coverity was free software, it could be extended by the maintainers of specific projects so that it could find flaws specific to those projects. But, oh, look, it isn't, despite early promises that it would be. So it'll never be as good as it could be in those areas.)

Rate of bugs and rate of security holes are mostly uncorrelated

Posted Mar 6, 2006 16:11 UTC (Mon) by shahms (subscriber, #8877) [Link]

From reading the articles about Coverity's static checker (at least, the articles on the early versions of the Stanford checker), there is no reason to suspect they haven't written domain-specific extensions for at least some of the idioms and API rules of the projects they checked. One of the main points of their tool is making "system-specifc programmer written compiler extensions" relatively easy. The architecture described by those papers is powerful and extensible, able to detect and act on almost-arbitrary pattern rules. It's really a pretty nice architecture as described and I imagine the Coverity tool addresses the most urgent shortcomings of the early implementation.

Rate of bugs and rate of security holes are mostly uncorrelated

Posted Mar 6, 2006 19:49 UTC (Mon) by Ross (guest, #4065) [Link] (7 responses)

I haven't seen a distribution yet which doesn't run the X server as root and have it listening on TCP port 6000 + display number. Now it is certainly possible to disable the TCP X transport (using the -notcp option when starting the server), but I don't think it is "normal".

I think the difference may be that X is much bigger than the server. The bug may simply have been in other places than the part exposed to the network. Also, the X server normally uses an authentication mechanism like MIT Magic Cookies which will limit the exposure of bugs to a very small portion of the code if the shared secret is not known to the attacker. (Assuming the cookies aren't weak and that the server throttles brute force attacks.)

Rate of bugs and rate of security holes are mostly uncorrelated

Posted Mar 6, 2006 20:17 UTC (Mon) by nijhof (subscriber, #4034) [Link] (1 responses)

Now it is certainly possible to disable the TCP X transport (using the -notcp option when starting the server), but I don't think it is "normal".

It is certainly "normal" for Debian -- you have to take out the --nolisten tcp manually if you want it to listen to tcp traffic

Rate of bugs and rate of security holes are mostly uncorrelated

Posted Mar 6, 2006 21:49 UTC (Mon) by Ross (guest, #4065) [Link]

That's strange because I remember doing specifically on my Debian system. But obviously my memory is not infallible; I didn't even remember the option correctly. It is in the form you posted, though I just checked the man page and it is only prefixed with one hyphen.

Rate of bugs and rate of security holes are mostly uncorrelated

Posted Mar 7, 2006 5:24 UTC (Tue) by mattdm (subscriber, #18) [Link]

I haven't seen a distribution yet which doesn't run the X server as root and have it listening on TCP port 6000 + display number.

Check out Fedora Core, then. Still runs as root (although possibly limited by SELinux), but doesn't listen on TCP by default.

Now it is certainly possible to disable the TCP X transport (using the -notcp option when starting the server), but I don't think it is "normal".

-nolisten tcp with X.org. Actually, I think this *is* the default in the upstream Gnome GDM, so....

Rate of bugs and rate of security holes are mostly uncorrelated

Posted Mar 7, 2006 13:51 UTC (Tue) by daniels (subscriber, #16193) [Link]

> I haven't seen a distribution yet which doesn't run the X server as root and have it listening on TCP port 6000 + display number.

I can't think of one off the top of my head that does listen for TCP connections by default.

Rate of bugs and rate of security holes are mostly uncorrelated

Posted Mar 7, 2006 15:22 UTC (Tue) by grouch (guest, #27289) [Link]

Do you mean '-nolisten tcp'?

From my default /etc/X11/xinit/xserverrc file on Debian:

exec /usr/bin/X11/X -dpi 100 -nolisten tcp

Rate of bugs and rate of security holes are mostly uncorrelated

Posted Mar 7, 2006 18:33 UTC (Tue) by oak (guest, #2786) [Link]

SUSE X servers don't listen on TCP sockets by default either.

(I don't think this has been the case always, I'd guess it to have
changed within last couple of years.)

Rate of bugs and rate of security holes are mostly uncorrelated

Posted Mar 8, 2006 5:52 UTC (Wed) by djm (subscriber, #11651) [Link]

OpenBSD isn't a "distribution", but it has privilege separated Xorg so the vast majority no longer runs as root. Disabling the TCP listener isn't so much of a win as it breaks ssh forwarding of X sessions, and because X11's authentication is pretty good (barring chumps who run "xhost +").

Rate of bugs and rate of security holes are mostly uncorrelated

Posted Mar 7, 2006 8:34 UTC (Tue) by rwild (guest, #16004) [Link] (2 responses)

> not many things other than the cursed libtool --mode=install ever run GCC as root

Bugs notwithstanding, you should be able to prevent that mostly by doing a DESTDIR install as user, and copying over and libtool --mode=finish as root.

Rate of bugs and rate of security holes are mostly uncorrelated

Posted Mar 8, 2006 0:49 UTC (Wed) by nix (subscriber, #2304) [Link] (1 responses)

Indeed, although I'm not sure what effect this has on things that want to install with unusual userids (and there are some).

Rate of bugs and rate of security holes are mostly uncorrelated

Posted Mar 9, 2006 10:25 UTC (Thu) by rwild (guest, #16004) [Link]

Yes (therefore the "mostly"). I usually "make install -k DESTDIR=.." then and fix the setuid manually. Not so great. Better to get fast-install mode fixed in Libtool on more systems (which avoids the relinking in the first place).

Rate of bugs and rate of security holes are mostly uncorrelated

Posted Mar 7, 2006 19:10 UTC (Tue) by Junior_Samples (guest, #26737) [Link] (1 responses)

"In any case, the more complex the code, the less likely it is that a static checker like Coverity's will isolate real faults without special understanding of that project."

Au contraire. Complexity in and of itself is a defect. The measurement of complexity is considered the one of best metrics of software quality there is. It has been shown time and again that defects grow exponentially as complexity is increased. The fact that an automated parser should struggle with code which is "more complex" and full of obscure idioms should be proof enough that the code is defective.

Sadly, use of complexity analysis seems to be completely absent in the OSS domain despite the fact that basic measurement of complexity using McCabe's or Halstead's methods are very straightforward, and freely available automated parsers exist. The Linux Journal has an article about one such tool for Python.

Rate of bugs and rate of security holes are mostly uncorrelated

Posted Mar 8, 2006 0:54 UTC (Wed) by nix (subscriber, #2304) [Link]

Code which is complex is only defective if it's doing a simple job despite that complexity.

Things like GCC, say, are doing very complex jobs: now perhaps GCC is more complex than it needs to be if all were ideal, but there's no way you'll ever get it simple enough to make static analysis easy. It does a very complex job.

(And so do the knottier parts of the kernel.)