The sustainability of open source for the long term

The problem of "sustainability" for open-source software is a common topic of conversation in our community these days. We covered a talk by Bradley Kuhn on sustainability a month ago. Another longtime community member, Luis Villa, gave his take on the problem of making open-source projects sustainable at the 2019 Legal and Licensing Workshop (LLW) in Barcelona. Villa is one of the co-founders of Tidelift, which is a company dedicated to helping close the gap so that the maintainers of open-source projects get paid in order to continue their work.

Long term

He started out by noting that he is looking at the sustainability problem from a long-term perspective. There is an enormous amount of open-source code that we all rely on, but much of it is maintained on a volunteer basis, which means that it may not be getting the attention that it needs. In order to ensure these projects can thrive, that needs to change.

There are some people who don't really see a big sustainability problem; developers have no trouble getting jobs and there are organizations that are supporting projects. But that is just the healthy tip of the iceberg, he said. Much of that is the core infrastructure, which is generally well maintained.

For companies that are not specializing in developing open source and are, instead, using it to create a traditional application for their business (e.g. a mobile app for insurance claims), there is a great deal of code that is less well maintained. The core infrastructure makes up just 10% of the code in a typical application; maintenance for that core can be bought from AWS or Red Hat, say. Another 10% is the business-specific logic for the application. The other 80% is made up of various free and open-source software (FOSS) libraries, frameworks, and such. Those numbers are rough, Villa said, but there is some data behind the figures.

In that 80%, there is lots of good code, but it is no one's job to maintain it. There is plenty of free code out there, which enables "a ton of business innovation"; FOSS is not terrible by any means, he said, but "we can improve it". However unmaintained FOSS that is part of a business's application does lead to more maintenance costs for that business.

Beyond that, there are some real "garbage fires" that have happened over the last few years. These include the left-pad fiasco, where the deletion of a single, simple module from the npm repository of JavaScript code led to widespread problems because the module was a dependency for many other modules. More recently, the event-stream situation, where the maintainer unknowingly turned over a project to someone with malicious intent, actually led to the loss of Bitcoin. We know that there are more of these kinds of things coming, he said.

Why care?

He cares about this problem because he has been working on this "for better or worse my entire adult life"; he thinks open source is important and he wants to see it be healthy. Tidelift has a business model that is based on the observation that maintainers would like to get directly paid for the work that they do on a project. They know that they have projects that are being used widely, so they would like to receive some money to help offset the time that they spend on them. All of that "makes sense".

But if these maintainers could, say, get $1 per month from each of their users, that would likely cover the shortfall—but not if they have to sell that idea and negotiate a license with each user. Tidelift is putting itself in the middle, so it does the sales, it does the legal negotiation, and it defines what the product is. That allows users to pay small amounts that add up because there are many users; it is a "network effects business". In order for that to work, Tidelift needs a lot of users (customers) on one side and a lot of developers on the other side.

Over the last two years or so, he has been talking to many FOSS developers "about what makes them tick; why do they care? Why are they doing open source? What makes it economically viable for them?" Tidelift did a survey of several hundred developers; he would love to be GitHub or Stack Overflow, he said, and be able to survey several hundred thousand developers. So the survey numbers should be taken "with a grain of salt".

The survey asked participants how they were being supported on the open-source projects that they work on; the vast majority (60%) answered "self-funded/none". Participants could choose more than one response, so the slightly less than 50% choosing "employer" presumably overlapped with other options. "Foundations" was down around 4% and dual-licensing, which is a topic often discussed at LLW, had roughly 1% of the responses. This provides a good reminder to him that the things that people at LLW are concerned about (e.g. licenses) are not the same as what today's FOSS maintainers are focused on.

Tidelift recently looked at "hello world" programs in languages like JavaScript, Python, PHP, and Java to try to determine the amount of unmaintained code those programs are using. Depending on the repository, 10-20% of the code used by these supposed "best practices" applications are unmaintained based on the following criteria: they have not had any commits in over a year and more than half of the new issues or pull requests were not closed over that time span.

Looking at a fraction of the packages out there and using a generous definition of what it means to be maintained still shows tons of unmaintained code in what should be the core modules for applications in these languages. "People aren't paying attention to it at all", he said.

Depending on how you survey, 5-6% of FOSS contributors are women, which is roughly one-fourth of their participation in the rest of the software industry. Conventional wisdom suggests that the percentage of women maintainers is well below that. In part, that may be because maintaining open source is seen as something that is done for free in one's spare time; maintaining a FOSS project is how people start their careers in open source. Women often have more responsibilities at home, so their participation in unpaid, spare-time activities may be limited.

Coping strategies for the problem of unpaid FOSS maintenance are proliferating, Villa said. There are efforts like Patreon, for example, and even the large charitable foundations are starting to get involved. The Ford Foundation did a "Roads and Bridges" study in 2016 that looked at the importance of FOSS in the software world, for example; its author, Nadia Eghbal, gave a related talk at linux.conf.au in 2017. Tidelift is not the only startup in this space either.

Beyond that, there are organizations like the Software Freedom Conservancy and Open Collective that are helping projects get their funding situation figured out. Even more startups are coming and Villa said that he recently talked with one of the world's largest charitable foundations that is starting a FOSS sustainability program soon. "So there are a lot of people who think this is a real problem."

Takeaways

Villa then presented some quotes from maintainers of key libraries to illustrate and highlight the sustainability problems. He coupled those with some key takeaways that he is hoping LLW attendees will take back to their organizations. First he quoted the maintainer of a Java library that is used by nearly every Java application out there. "I had a donation link for ten years, got two donations." For many years, that wasn't a big deal because the maintainer was "single and young", but now they have children.

That illustrates that contributors are changing. This particular developer got involved because they were scratching their own itch, so they wrote code to solve their problem. That problem is well and truly solved at this point, but the maintainer still gets issues filed against the code. Others got involved in FOSS because of ideology, but the new open-source developers do not know or care about our ideology, he said. They created an open-source project because it was easy to do on GitHub.

That's good news, he said; it is easy to create open-source projects on GitHub, so a lot of open source is being created. But the assumptions that many in the room make about the motivations for doing so are no longer valid. The attitude toward money and open-source projects has changed as well; it is no longer a big emotional and political mess to try to inject money into a project. The new generation of open-source developers are "very reasonable, smart, and sophisticated about how money interacts with their projects".

The second takeaway featured quotes from two maintainers, one for a PHP library and another for a JavaScript library—both of which are likely used by applications in those languages. The first quote noted that people simply expect a library to be maintained, while the second pointed out that most of the maintainer's time was "spent listening to people complain about my software". If that second quote is not "a recipe for burnout", he does not know what one would be, Villa said.

The overarching message here is that "demands continue to increase". GitHub has made things easier, so issues are filed against projects. When you have 1000 users who needed to sign up with a Bugzilla instance somewhere to file a bug, the average response was to file no bugs. But when you have 100,000 users who are all logged into their GitHub accounts essentially all the time, the default action is to file many issues. Something that is a problem for Tidelift, but might an opportunity for some other organization, is that many developers would rather have help with bug triage than receive money for their work; they just want to get their time back, he said.

"Charity is not enough" was his third takeaway. There has been some talk about a few projects that are making a few thousand dollars per month on Patreon, but the maintainer of a key JavaScript library had a different perspective. They have spent half their adult life making things for free on the internet, "I'm not excited to be giving away t-shirts on Patreon". Charity simply isn't scalable to solving the 80% problem, Villa said.

Fourth, "the problem is not hypothetical". He pointed to a statement by the maintainer of event-stream who noted that there are plenty of dependencies "that are 'maintained' by someone who's lost interest, or is even starting to burnout, and that they no longer use themselves". The software was compromised because of Tarr's burnout and he is only the tip of that "burnout iceberg", Villa said. It would be easy to think that these are isolated incidents, but we will be seeing more of this problem; Tidelift may not be the solution, he said, but we as a community need to think about what that solution is.

All of the previous four takeaways apply widely, while the next three are targeted at the lawyers in the room. First, in-house code is where lawyers spend all of their time, but it is likely the smallest part of the application stack. The developers of the in-house code are not representative of the developers of the rest of the stack, he said. They are presumably paid well and may get lunch buffets, but that is not true for developers outside of these companies.

Second, the scale of the problem is enormous. Any solution needs to scale well beyond the number of developers at any one company; thinking about things that "solve for your company" will not be big enough. Estimates on the number of critically dependent libraries vary, from single-digit thousands to tens of thousands, so there are that many maintainers that are having a sustainability problem right now. Finally, the legal department is probably not the solution to the problem; he does think there needs to be more innovation in licenses, but that is not going to solve the problem either.

[I would like to thank the FSFE and the LLW Diamond sponsors, Intel, the Linux Foundation, and Red Hat, for their travel assistance to Barcelona for the conference.]