LWN.net Logo

Open source under the microscope (News.com)

News.com covers an academic study of the open source model. "Scacchi and fellow researchers have found a significant failure rate among open-source projects. But among those that get off the ground, research has shown not only that the open-source approach can yield better software more quickly and for less money than traditional methods but also that volunteering for an open-source project can be an effective way to get a job."
(Log in to post comments)

Open source under the microscope (News.com)

Posted Jan 6, 2004 4:08 UTC (Tue) by mark (guest, #1921) [Link]

"Scacchi and fellow researchers have found a significant failure rate among open-source projects".

We often see "comparisons" between free and proprietary software where the metrics for the former are easily obtained, but for the latter are unknowable.

In this example, what is the point of drawing attention to the failure rate of free software projects when no accurate comment could ever be made about the equivalent metric in proprietary software?

There are many books and publications dedicated to reducing the risk of software development. Most of the books that I own implicitly presume a proprietary development model. Presumably we can infer from this that failure rate of proprietary software development is also "significant".

Highlighting the failure rate of free software is a bit like highlighting the fact that it has to be designed, written, tested, released, deployed, installed, supported, etc. All of these things are common to both proprietary and free software. The free software movement as I understand it doesn't directly address /any/ of these problems. It simply says that people should have full access to the tools that they need, and the only way that can happen is if the tools are developed in the open. Apart from openness, the actual development methodology is, as far as I know, left open.

What's a failure?

Posted Jan 6, 2004 7:07 UTC (Tue) by error27 (subscriber, #8346) [Link]

I have a project on sf.net for "easy graphics" that never got off the ground.

Basically, it was a school project and I was planning to put it online when I got done. I worked my tail off for a week and the result was crap. I was too embarrassed to post it.

People sometimes look at sf.net and point out how many projects there are like this, but that's silly. I didn't invest any worthwhile time and it's not sad that it died.

Games are the worst because it's so fun to sit around talking about writing them but it's difficult to actually sit down and do it.

The point of the article isn't us vs. them

Posted Jan 6, 2004 10:13 UTC (Tue) by Duncan (guest, #6647) [Link]

> In this example, what is the point of drawing attention
> to the failure rate of free software projects when no
> accurate comment could ever be made about the
> equivalent metric in proprietary software?

It would seem that, by asking that question, you are demonstrating that
you've missed the point. Why must it always be us vs. them? In fact, the
article never mentioned proprietary software at all, that I saw, tho it DID
mention how open source differs from traditional software engineering
concepts, which by implication would be proprietary in nature.

Rather, this, like ESR's The Cathedral and the Bazaar, this is a study of how
Open source techniques and society function on their own, with an eye
toward recognizing what works and what may not, within that cultural and
design concept reference. I don't know about you, but I at least find such
studies useful on their own, and the fact that they are being done, that free
software is considered to be worth study on its own, quite apart from
comparison with closed source, rather refreshing.

Seen as such, and it would appear the study authors take this view as well,
open source project failure isn't specifically a negative, but simply recognized
as part of the process, an accepted part of the culture. The article talks about
how traditional software engineering techniques predict a rapid initial growth
phase, followed by meeting the initial goals and a leveling off of
development, Open source, by contrast, tends to show a hockey-stick curve --
a period of fairly slow initial development with the efforts of only one or a
small group of people, the blade of the hockey stick, followed if the project is
successful by a point at which public interest begins to grow and public
contributions kick in. leading to a higher than linear and far faster than
expected growth rate, the "handle" of the hockey stick. The article doesn't
mention it explicitly, but implicit in that model is that many projects will
"fail", that is, never make it out of the "blade" phase of primarily individual
interest, into the handle or public contribution phase. This is only natural,
since all that extra interest and contribution comprising the handle phase
must come from SOMEWHERE, and that a good portion of it may come
from others giving up on their similar projects, to join the common more
successful public one, as they discover it meets their needs with a lower level
of investment than they were putting into their own "blade" phase project,
shouldn't be surprising.

There were a number of other interesting tidbits in the article as well. One of
particular interest to me was what the article and study authors call
"informalisms", which substitute for the more rigid traditional design spec, and
can be anything from a threaded mailing list discussion of how the software
design should proceed, to FAQs, READMEs, and TODOs, to comments in
the source code, to comments on the web site, possibly in a roadmap
document.

The article and study is pointing out how the open source approach tends to
use these more informal specifications in an evolutional approach -- these
documents tend to reflect the present or historic view of the project, a view
which changes or "evolves" over time -- as compared to the traditional more
rigidly designed spec used in again traditional pay-for-product scenarios,
where the rigidity is needed as a method for measuring when the contracted
work has been done, thus, measuring performance. Without that
requirement, or with it more loosely defined, open source is free to evolve in
new and interesting ways not possible to originally foresee in a rigid design
spec.

All in all, a VERY interesting read. Thanks to LWN for calling our attention
to it, and providing a forum here for its discussion, as well. =:^)

Duncan

Free software versus software engineering

Posted Jan 7, 2004 4:45 UTC (Wed) by mark (guest, #1921) [Link]

I dislike the article because it seems to imply that free software is somehow better because it is a particular form of software development that's different from "taught" software engineering. I don't think this is true; free software doesn't specify the development model to use.

The "us versus them" dichotomy is set up by the news.com.com article. They are the ones who draw comparisons between "open source software" and "software engineering". ("Are the processes the same that are taught in engineering classes?").

Some free software groups will have formal development methodologies based on established software engineering principals, and some will not - this is exactly the same as in proprietary software, and is not therefore a driver of software quality that is specific to free software. In software engineering, aspects of extreme programming and the trends in prototyping, refactoring and late design mirror the free software universe illustrated in the article. Ideas such as the "hockey stick" and the "informalisms" can and do apply equally to free or proprietary software. If we are going to analyse what makes free software better, should we not stick to things that are unique to free software?

I think that the problem of writing good software is the same regardless of if you're 2000 people on the internet writing Linux, or 2000 people on the Microsoft intranet writing Windows XP. The real difference is that XP has a ship date and a resource budget, and Linux does not. In my opinion this is the true advantage of free software.

Copyright © 2004, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds