Brian Kernighan on the origins of Unix [LWN.net]

Brian Kernighan on the origins of Unix

Posted Jan 18, 2022 15:10 UTC (Tue) by karim (subscriber, #114)
Parent article: Brian Kernighan on the origins of Unix

"In your editor's opinion, Kernighan missed an opportunity to evaluate the free-software community in these terms. Companies may not have long time horizons, but many free-software projects do. It is, still, a place where people with good ideas can come together and see where those ideas lead."

I actually disagree with this. And I hesitate to even write this as a comment because I will likely not want to defend this point of view in response do to anyone -- 1) just not interested in a long-winded debate, 2) this is a very quick post that I haven't taken the time to think through completely, and the rough argument I make likely has many problems. So I'm going to drop this here, and come what may.

For having been involved in OSS since the mid/late-90s it's been my perception that the FOSS community is very good at _commoditizing_ existing technology and very bad at creating new tech. Think of the Linux phones before Android and the "Linux" phones since. Linux now runs on the vast majority of smartphones on the planet, but it's not because of the FOSS community, at least not directly. Here's Andy Rubin's (Android creator) take on OSS (https://techcrunch.com/2011/05/10/andy-rubin-on-androids-...): “Open source is different from a community-driven project. We’re light on community, but everything we do ends up in an open source repository." For him, open source wasn't an enabler, just a by-product.

Similar story with ChromeOS.

Similar story with gcc vs. llvm. In fact the entire GNU project was a commoditization of Unix tools/userspace. And, yet, when evolution came knocking to the compiler world, the GNU project couldn't handle it and llvm has since taken a lot more space.

etc.

Another angle to look at is how many FOSS projects/contributors have struggled with funding. This is something that LWN has covered multiple times. And it's been aptly immortalized by XKCD: https://xkcd.com/2347/ How can you innovate if you can't even pay the bills?

So, apologies, I'm skeptical that FOSS communities are inherently capable of sustaining long-term disruptive innovation. They can however certainly help sustain it by commoditizing the required building blocks that have already been shown to work elsewhere.

to post comments

Brian Kernighan on the origins of Unix

Posted Jan 18, 2022 18:36 UTC (Tue) by mfuzzey (subscriber, #57966) [Link] (1 responses)

I see what you're getting at and don't really disagree, major open source projects do seem to often be better at improving an existing idea rather than implementing completely new things.

However I think there are other factors at play in your examples.

Android: hardware was a huge factor too. The Android OS would be useless with appropriate hardware and that is still difficult / impossible to fully open source and even harder to develop in a community manner.
Now that suitable hardware exists there are projects building open source, non AOSP based, operating systems for them. They will probably never become mainstream, just as Linux isn't and will probably never be mainstream on the desktop but they do exist.
I also think Android wouldn't have succeeded as well as it has if it hadn't been open source, even though it definitely isn't a community driven project.
Furthermore Android had specific time to market pressures (to have a chance of avoiding Apple dominating the market and Microsoft taking the rest).

ChromeOS: hardware too (though you can run it on a PC so not entirely). But also the chromeOS web centric model relies on huge Google data centres that aren't really reproducible outside of huge companies.

GCC / LLVM: I think this was more a question of the incumbrent being less nimble. GCC, as the dominant OSS compiler saw less need to innovate and, because they had a large existing code base doing so was harder. This happens in the proprietary world too where innovation is often more likely to occur from a new entrant than a large existing player.

But I think where the open source community has really suceeded is in the development model rather than particular technologies.
Back in the 90's many people thought the future of computing would be building systems from sets of commercial , closed source, components. That was the model Microsoft technologies like ActiveX, OLE, COM/DCOM etc were supposed to enable.
That failed to materialize and most new languages / frameworks today are non starters unless they are open source.
For many if you can't get it with "git clone" and no jumping through hoops it doesn't exist.

Sure source availibility isn't everything, and isn't the same thing as community (the point Andy Rubin makes) but, when the source is available with no community it is still possible for a community to form later (by forking if needed). To some extent this happened, albeit temporarilly, with Android with early "community" Android forks like Cyanogen actually bringing significant new features that ended up becoming available in official Android only later (stuff like tethering and permissions modifications come to mind).

Brian Kernighan on the origins of Unix

Posted Jan 19, 2022 0:37 UTC (Wed) by bartoc (guest, #124262) [Link]

> Sure source availibility isn't everything, and isn't the same thing as community (the point Andy Rubin makes) but, when the source is available with no community it is still possible for a community to form later (by forking if needed). To some extent this happened, albeit temporarilly, with Android with early "community" Android forks like Cyanogen actually bringing significant new features that ended up becoming available in official Android only later (stuff like tethering and permissions modifications come to mind).

Interestingly this happened with iPhones too! Despite most of their software not being open source. All those quick control popouts and a lot of the additional interaction options for stuff like notifications were added by third party mods for "jailbroken" iphones long before apple implemented them. I'm not sure if this is Android/Apple taking ideas from the "community" or just obviously good ideas being implemented by the community first, because a hobbyist doesn't need to polish things up that much before shipping (or go through endless UI redesigns, UX research, etc, etc).

Brian Kernighan on the origins of Unix

Posted Jan 20, 2022 9:37 UTC (Thu) by gasche (subscriber, #74946) [Link] (1 responses)

Interesting discussion! I can think of several open source projects that were disruptive to the rest of the ecosystem (even if they are often built on pre-existing ideas that hadn't seen wide diffusion yet). They are limited to my own expertise and I'm sure there are many more examples:

- Git made distributed version-control widely available and changed our development practices in a radical way.
- Haskell is a radical programming language that gave a lot of ideas now adopted by other languages
- Nix and Guix are proposing a fresh take on software package management and generally the OS
- QubesOS is also a fairly radical take on OS (although it is a layer on top of existing systems for usability reasons)
- the Coq proof assistant ( https://en.wikipedia.org/wiki/Coq ), and in general Interactive Theorem Provers, are radical projects and all the active ones are open source, with most run as free software projects (with a strong academic bias among contributors).
- I believe that some peer-to-peer projects also had a radical impact (Bitorrent these days), but I don't know enough about their history to tell if the open source / free software community participated actively. (I think there was a fair amount of Windows-based freeware at the time.)
- I would guess that Bitcoin was also operated by a crew of enthusiast hackers from the open source community at the start -- but again I'm not sure.

Many of those projects were born in academia (a natural place to look for radical ideas), but they were also open source projects from the start, and in many cases the free software community contributed greatly to the fact that they became successful enough to spread their ideas to the rest of the software world.

For Git one could argue that this is a commoditization of the proprietary system BitKeeper; but there were also open source ancestors (GNU Arch) and competitors (darcs, Mercurial) that were also influential.
(I think it would be worth articulating whether we are discussing inventions that were born in the open source community and became successful-enough within it, or inventions that were born anywhere but remained fairly confidential, and became successful thanks to the open source community. I rather have the later in mind, and I think it is different from "commoditization".)

One important area of innovation that is missed here is machine learning, that saw relatively little involving from well-identified open source communities, possibly as the requirements for entering the space (having a *lot* of training data at hand) made distributed development difficult. There are now striving open source projects within the machine learning software ecosystem, but I wouldn't say that the field would be substantially different without the open source community (by which I mean: without large-scale cooperation of hobbyists / benevolent contributors).

Brian Kernighan on the origins of Unix

Posted Jan 20, 2022 16:55 UTC (Thu) by karim (subscriber, #114) [Link]

Thanks for this, interesting point of view.

I somewhat feel git might actually a good example of commoditization. I followed this very much in "real-time" as the BitKeeper drama unfolded and was at OLS in 2005 when Matt Mackall first presented Mercurial -- I loved mercurial btw, and I was sad to see git prevail, but I digress. It was my understanding that BitKeeper was tailor-built by Larry McVoy for Linux development based on direct conversations between Linus and Larry. I don't have a reference for this, so I could be misremembering, but that was my understanding. As such, when the history section of git on Wikipedia (https://en.wikipedia.org/wiki/Git#History) states: "Support a distributed, BitKeeper-like workflow." as one of the goals set out by Linus for git, one can probably easily overlook the work that Larry presumably did to actually and actively listen to Linus and come up with something that fit "the customer's needs". [side bar: I've butted heads with Larry several times in the early 2000s on unrelated topics, but credit should go where it's deserved.]

And maybe this is what it boils down to. "Listening to the customer's needs" and "creating a product the customer will actually use" have nothing to do with the ability of replicating in an open source manor the end result of what those customer-facing steps entail. In fact, I'd venture to say that the open source community generally has had a bad history of being able to listen to customer needs. It's been, on other hand, very effective at replicating what those that have have produced, albeit sometimes in a more sustainable fashion ... because direct monetization wasn't the end goal or even possibly a need.

Again: 1) very rough ideas/arguments off the top of my head, 2) I care not of being right nor making a point. i.e. destroy/demolish at will.