LWN.net Logo

Choosing between portability and innovation

Choosing between portability and innovation

Posted Mar 3, 2011 3:01 UTC (Thu) by josephrjustice (subscriber, #26250)
Parent article: Choosing between portability and innovation

I can think of one reason why portability can be desirable, and why focusing development efforts solely on the Linux platform might be undesirable, that I haven't seen mentioned yet. Namely, on the grounds of avoiding a operating system software monoculture.

My understanding is that software monocultures are often thought of as undesirable, because when a sufficiently severe problem occurs within the members of the monoculture it can wipe them all out (similarly to how a disease or pest can destroy an ecological monoculture vulnerable to that disease/pest) and, if there are no other alternatives available for use, then the functionality provided by that software will be lost until (and if!) the software can be altered to resist or become immune to the problem. When that software is the operating system, which provides the foundation required by all other uses of the computer (such as application software), it seems as if it would be even more important than usual to avoid the risks caused by a software monoculture.

We can easily see a real-life example of exactly this sort of situation in the problems regularly encountered by users of the Microsoft Windows family of operating systems. Historically, every so often we see a great furor erupt in the mainstream media, as another vulnerability is discovered and exploited within Windows which results in many people world-wide losing the use of their computer for a time, or even losing their data, until the vulnerability is patched or otherwise remedied. Of course, Windows is known to be especially vulnerable to this sort of thing due to its historical underpinnings and ancestry, as well as due to the fact that many Windows users are non-technical and unwilling (or unable!) to be proactive in keeping their systems (relatively) secure.

However, the fact that Windows is an especially vulnerable target with relatively unskilled users, while Unix-like operating systems (including Linux) are relatively resistant targets with (presumably) relatively knowledgeable users, does not excuse the latter group from having to beware the risks inherent in being a software monoculture! We know that Unix-like and Linux-based operating systems are known to have vulnerabilities too, and if we don't know this we can simply look at the ongoing series of vulnerabilities announced by CERT and listed in LWN every week. And, we've had our moments of furor and panic in the mainstream media spotlight too -- perhaps the most well known of these was Robert Morris's Internet Worm. (See, for instance, http://en.wikipedia.org/wiki/Morris_worm .)

A second reason for considering software monoculture to be undesirable is that it reduces the opportunity for instances of hybrid vigor (see http://en.wikipedia.org/wiki/Hybrid_vigor ) to occur. In the context of software, this would be the porting or reimplementation of desirable features or capabilities originally provided by some other independent implementation of that type of software (such as operating systems). Of course, with software, there are unique impediments to the act of hybridization that can occur, such as incompatible software licenses. Even so, we see instances of software hybridization occur all the time at all levels of software.

I agree that it is desirable and usually a good thing to try to fully use the capabilities provided by an operating system, including those capabilities which might not be portable to or currently unavailable on by other sibling operating systems. (If nothing else, we want to try to use those capabilities to see if they're worth the effort to reimplement elsewhere!) I agree that worrying about portability issues can make software harder to implement and slow down the pace of development. I agree that, in at least some instances, it may not be worth worrying about portability issues, or at least considering them only at very most a distant second (if not even lower) in terms of importance. In other instances, perhaps worrying about portability should be left to "somebody else", while the original developer concentrates on advancing the state of the art on the primary development platform.

However, I disagree that we should blithely agree to establish or limit ourselves to a software monoculture, especially an operating system software monoculture, without duly recognizing and considering the risks of this decision and the costs accompanying those risks (which costs might reduce or even eliminate the benefits of living within a software monoculture), and also recognizing the benefits and value potentially provided if we do not limit ourselves to a monoculture environment.

Joseph


(Log in to post comments)

Choosing between portability and innovation

Posted Mar 3, 2011 5:43 UTC (Thu) by drag (subscriber, #31333) [Link]

> I can think of one reason why portability can be desirable, and why focusing development efforts solely on the Linux platform might be undesirable, that I haven't seen mentioned yet. Namely, on the grounds of avoiding a operating system software monoculture.

The thing is that a OS and desktop environment exist for the sole purpose of running applications. It's a abstraction layer of sorts designed to facilitate the use and development of software.

Therefore anything to make the system more stable, make developer's lives easier, make user's lives easier, make things run smoother, faster, etc etc. Anything that the OS does to improve the attributes of the programs running on the system is a fantastic thing.

Avoiding monoculture for the sake of avoiding monoculture is extremely dubious approach and just ends up making the systems worse, not better.

> We know that Unix-like and Linux-based operating systems are known to have vulnerabilities too, and if we don't know this we can simply look at the ongoing series of vulnerabilities announced by CERT and listed in LWN every week.

The biggest problem for Linux on the desktop is that it's entirely unnecessary for somebody to actually root your system to do very significant damage to you. The /home/username/ and user account is a soft target. It is were all your passwords are stored, were you carry out online commerce, were you communicate, etc etc. Your only line of defense is not your OS, but your applications. Firefox or Chrome is the software that keeps you safe, not the Linux kernel.

At least not yet. I'm hoping that Ubuntu's use of AppArmor (or Smack or SELinux) will eventually provide some layered security.

Some divergence and trying different approaches is valuable, but security does not happen by accident. It's not like 'Oh we are different therefore we are secure'. It has limited utility against 'dumb' automated attacks from very basic viruses and worms... but really it provides almost no real benefit from real attacks other then by complete accident. Against a intelligent attacker with a directed focus it's just a paper tiger.

Windows security sucked not because everybody was using Windows 2000, but it was because Windows security was a pile of shit. Microsoft has made huge strides and security is no longer a significant selling point for Linux desktop usage, if it ever was..

> http://en.wikipedia.org/wiki/Hybrid_vigor

Software is not a biological thing. Be careful of drawing conclusions from false analogies. If there is a problem with the software it's fixable. Not so much with DNA. At least not yet.

Especially if we used layered designed with (a minimal amount of) formal APIs/ABIs. That way you can fix problems in a layer without perturbing the software above it or below it, unless it's very necessary.

Remember 'layered design' concept of 'Unix' and TCP/IP?

That's exactly what the kernel does. Formal API/ABI layer between it and 'userspace' creates a significant amount of freedom for the kernel to change and develop. Just don't break userspace and developers can do most anything they want if they are smart enough to pull it off. It's not perfect (sysfs), but it may not be that big of a deal as long as breakage is kept to a minimum.

Compare and contrast this with a non-layered approach like your X Server from a couple years ago. The same application that had access to your PCI bus to configure hardware, provided network services, provided your terminal services, and touched every almost every single application you used. Even the stuff that ran in your xterms had to get it's input from you filtered through X. Which also, of course, runs a setuid root. It's not only extremely questionable design security-wise... it makes for a very fragile system.

Thank goodness for DRI2/KMS/GEM/TTM/etc...

Choosing between portability and innovation

Posted Mar 3, 2011 5:57 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link]

Linux was able to grow and prosper due to the fact that the code was portable and therefor Linux was able to run it.

If Sun had managed to get people programming just for it (the way that people are advocating programming only for Linux) back in the days when it was the premier OS, Linux would have been much harder to get started.

Linux developers today owe it to everyone (including themselves) to not raise the bar for for the eventual linux replacement higher unnecessarily.

that being said, having software take advantage of the latest features is a good thing, but the software should degrade gracefully in the absence of those latest features. This may mean falling back to something not as good, or it may mean disabling some features where there is no fallback.

Choosing between portability and innovation

Posted Mar 3, 2011 7:38 UTC (Thu) by airlied (subscriber, #9104) [Link]

So you should have a whole lot of fallbacks that nobody is testing? and will most likely bitrot into all hell since nobody runs them except maybe some hero once every 2-3 years.

Choosing between portability and innovation

Posted Mar 3, 2011 10:13 UTC (Thu) by roblucid (subscriber, #48964) [Link]

It's called error handling, a major pain yes but robust programs have it.

If applications are no longer designed that way, then when there's a call for Linux 3.0 for some currently unforeseable reason, there'll be a terrible chicken & egg problem, which will make the KDE4.0 saga look minor.

The X redevelopments are actually a good example for need for this, as they have NOT provided uninterrupted functionality to the end users, very many complain about much breakage, and missing features over last couple of years.

Choosing between portability and innovation

Posted Mar 3, 2011 15:03 UTC (Thu) by mezcalero (subscriber, #45103) [Link]

It's really not so simple. You cannot claim: "by having N competing but mostly identical projects a bug can only be used to take down 1/Nth of the computers". It is more like: "by having N competing but mostly identical projects you increase the number of bugs in your system N times".

And if we had 10 competing implementations than we'd have 10x more bugs. That sounds pretty bad to me.

I think nourishing this kind of competition is a bad tool to combat computer insecurity. If you have a single well reviewed implemention I think you are much better off than having 10 badly reviewed ones.

Choosing between portability and innovation

Posted Mar 3, 2011 17:05 UTC (Thu) by nix (subscriber, #2304) [Link]

If you have ten implementations you have ten different sets of bugs: nobody will be hit by all of them at once.

Your implicit claim that a single well-reviewed implementation can somehow be free of security holes, or indeed any kind of bug, is laughable on its face. I don't know of any software product of any kind that this has ever been true of (even TeX).

Choosing between portability and innovation

Posted Mar 3, 2011 17:45 UTC (Thu) by martinfick (subscriber, #4455) [Link]

10 implementation may have 10 sets of bugs. But nothing prevents a bug from being in all 10 sets. Remember ping of death?

Choosing between portability and innovation

Posted Mar 3, 2011 18:06 UTC (Thu) by nix (subscriber, #2304) [Link]

Well, yes, but that was in all descendants of a single implementation, wasn't it? (More relevant perhaps is cases where buggy algorithms have been implemented out of books into lots of unrelated programs.)

Choosing between portability and innovation

Posted Mar 3, 2011 18:45 UTC (Thu) by martinfick (subscriber, #4455) [Link]

If windows inherited this bug from unix, I would say that there is just as good a chance that free unix implementations will inherit bugs from each other, if not a much greater one.

Choosing between portability and innovation

Posted Mar 3, 2011 22:58 UTC (Thu) by nix (subscriber, #2304) [Link]

Linux, almost uniquely, didn't use the BSD TCP stack. Windows did (for a long time, if not anymore).

So, no, unless it was an algorithmic error Linux would not have inherited the ping of death (at least not *that* ping of death).

Choosing between portability and innovation

Posted Mar 3, 2011 20:11 UTC (Thu) by jg (subscriber, #17537) [Link]

I think N being a small integer is useful. Stifling of innovation generally occurs when N=1.

N going to infinity (e.g. 10 and greater) is insanity...

We can argue about things in the middle...

Choosing between portability and innovation

Posted Mar 6, 2011 12:53 UTC (Sun) by pjm (subscriber, #2080) [Link]

>If you have a single well reviewed implemention I think you are much better off than having 10 badly reviewed ones.

That sounds good in itself. However, your advocated position is to standardize on a kernel that's featureful and consequently full of bugs (despite having so many developers contributing to it), to the complete exclusion of any kernel with fewer bugs. That's not to say that your advocated position is a bad one; but I do believe it runs counter to a goal of combatting computer insecurity.

(Incidentally, if you really did think that 10× the number of bugs is such a bad thing, then I think you'd probably use a different kernel. But most people do use Linux even if they know it has lots of bugs, and would even if they knew it had 10× the number of bugs of some other unixy kernel, even for just one or two extra features important to them. Similarly, each of the N competing implementations presumably has one or two features or attributes not present in (and not feasible to add to) the others.)

and other featureful-but-buggy software .)

Choosing between portability and innovation

Posted Mar 5, 2011 2:00 UTC (Sat) by skissane (subscriber, #38675) [Link]

Personally -- forget about the Linux monoculture -- what about the Unix monoculture, or the LUW (Linux/Unix/Windows) monoculture? People forget that Unix and Windows are relatives -- Windows shows significant influence of Unix. Some examples:
  • Base OS only supports unstructured bytestream files, rather than base OS support for more complex file structures (such as record structure, indexed files, forked files, etc.) -- this is Unix influence on Windows via CP/M then DOS
  • Filesystem does not have explicit recording of file types, only convention of file extensions -- again, Unix influence on Windows via CP/M then DOS
  • Hierarchical directories -- DOS 2.0 explicitly modelled these on Unix
  • C programming language -- direct Unix influence on DOS/Windows
  • Berkeley sockets influence on WinSock
If you want a non-monocultural OS, don't look for yet another Un*x or Windows. They are parts of the monoculture. Examples of non-monocultural OSes would be any IBM mainframe/midrange OSes (like z/OS, z/VM, z/VSE, z/TPF, OS/400), HP MPE, Siemens BS2000/OSD, Unisys ClearPath, language-specific OSes like Lisp Machines or PICK (back when PICK was an whole OS not just a database), etc. Of course, many of this stuff is rather ancient and decaying, but it just goes to show how the IT industry was far less monocultural in previous decades than it is today

I don't like hierarchical filesystems. We end up with these massive random directory trees and cannot find anything. Better would be a filesystem where we give files tags, some unique and some non-unique, and we can find files by their tags -- in library science terms, hierarchical filesystems are like Dewey Decimal or Library of Congress classification, I think we should adopt faceted classification instead

I really like the idea of getting rid of operating systems and making applications run directly on bare hardware. Especially with virtualization, who needs a general purpose OS to run a web server or database server? Why not just run the application directly on the hypervisor, with as thin a level possible in between. This is like MIT's exokernel research.

I think Oracle's WebLogic VE is a good implementation of this idea -- run the JVM directly on the hypervisor, with just a very thin custom OS that exists solely to meet the JVM's needs, no general purpose OS needed in between. I'd like to see the same idea extended to other areas (e.g. databases). (Full disclosure: I work for Oracle but this is just my personal opinions not those of my employer)

Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds