A brief history of RubyGems.org

Welcome to LWN.net The following subscription-only content has been made available to you by an LWN subscriber. Thousands of subscribers depend on LWN for the best news from the Linux and free software communities. If you enjoy this article, please consider subscribing to LWN. Thank you for visiting LWN.net!

Ruby libraries and applications are distributed via a packaging format called a gem. RubyGems.org has been the central hosting service for gems since about 2010. This article is part one of a two-part series on the RubyGems.org takeover by Ruby Central. Understanding the history of RubyGems.org, and the contributor community behind it, is vital to making sense of the current power struggle between Ruby Central and members of the Ruby community who have maintained those services and tools for many years.

Finding gems

Like many open-source projects, the RubyGems.org service and tooling developed organically over the years to meet the needs of the growing Ruby community. Yukihiro Matsumoto, better known as "Matz" in the community, released the first version of Ruby in December 1995. Early Ruby release announcements seem to be lost to time, but Matsumoto provided a timeline in his "Ruby After 25 Years" talk, which is available on YouTube for those interested in Ruby history.

The Ruby community needed a home for open-source Ruby projects, which led to the creation of RubyForge in 2003. The hosting costs for the service were funded by Ruby Central, the nonprofit founded in 2001 by Black and Fowler after they'd organized the first RubyConf. For most of its existence, Ruby Central's primary focus has been putting on Ruby events, but it has continued to help fund Ruby hosting services since RubyForge was launched.

Ruby did not have a package manager when it was introduced; According to a 2006 article, work on the RubyGems package-management framework began in November 2003. It was created by Paul Brannan, David Alan Black, Chad Fowler, Richard Kilmer, and Jim Weirich. The first release was announced in March 2004, nearly a decade after Ruby appeared. The gem command-line utility is a frontend for RubyGems; it can be used to do everything from creating, installing, and managing gem packages to publishing them to a gem server. Development of the framework is separate from the language, but RubyGems has been included with Ruby since the Ruby 1.9.0 release in 2007.

In June 2006, Fowler announced the start of integration of RubyGems with RubyForge, which led to the forge becoming the central repository for gems. GitHub, for a short time, also served as a source of Ruby gems, which meant that there were two popular sources for gems. However, GitHub dropped its automatic gem-building service when it moved from Heroku to Rackspace in 2009, which meant that it was up to the community to maintain RubyForge as a reliable service to serve and find Ruby gems.

Growing pains

"Reliable", however, is not necessarily the first word that would come to mind about the early days of gem hosting. There were frequent complaints about RubyForge's slowness and occasional downtime. In 2009, Nick Quaranto proposed "a fresh take" on gem hosting based on an MIT-licensed Ruby on Rails project he called Gemcutter. Gem hosting would move to RubyGems.org, with support from Ruby Central for hosting costs. Development for RubyGems moved to GitHub (archived repository here) in December 2010, and RubyForge was shut down entirely in 2014. The operation of RubyGems.org, as well as the development and maintenance of the software that powered it, were handled by volunteers.

Aside from gem hosting, the Ruby community had another package-management problem: dependency management. It was simple to install individual gems, but applications written in Ruby would have dependencies on specific versions of many gems. As with other package-management tools, users suffered from "dependency hell" trying to assemble the right combination of gems to run complex applications. This was exacerbated by the popularity of Ruby frameworks like Merb and Ruby on Rails that helped developers create web applications with many dependencies.

To solve the dependency-management problem, Carl Lerche and Yehuda Katz started the Bundler project in 2008. Bundler was designed to track and install the exact gems and versions needed by simply typing "bundle install". André Arko joined in on its development in 2010, around the time of the 0.9.5 release. Arko became Bundler's most active developer that year; Lerche's last contribution was in 2011, and Katz's was in 2012.

Bundler adoption took off, which had a side effect; it put a lot of stress on RubyGems.org because it made it much easier for users to grab many gems automatically. But, Arko noted in his blog post about the history of Bundler, it was slow at first. Early iterations of bundle would receive a list of all gems on RubyGems.org every time a user ran bundle install. Quaranto, Arko said, "pragmatically wrote a new API for RubyGems.org, shipped it, and let us know that we could use it" to list individual gems instead of "every gem in existence".

Bundler adoption took off, and Arko said that the tool was being downloaded about 30,000 times per day by 2012. That meant a lot of users hammering RubyGems.org:

The growing number of Bundler users slowly built up until October 2012, when we discovered that Bundler was effectively running a DDoS attack against RubyGems.org when the servers went down, hard. There was no way for the existing architecture to handle the huge number of requests coming in at all times. We had to completely disable the dependency API, and Bundler went back to being slow.

At this point, a team including myself, Terence Lee, Larry Marburger, and others, took the time to design, implement, deploy, and scale a separate Bundler API web application to serve the dependency API for Bundler users. With the cooperation of the RubyGems.org team, including Evan Phoenix and David Radcliffe, we were even able to make the original API urls continue to work.

Bundler was eventually included as a default gem in 2018 with the Ruby 2.6.0 release.

Ruby Together

The work that enabled RubyGems.org to meet increased usage was, of course, taken on by volunteers from the Ruby community. In 2015, Arko announced the formation of a nonprofit called Ruby Together to try to fund some of that work:

All of the infrastructure used by Ruby developers today, including Bundler, RubyGems, and RubyGems.org is maintained and developed by volunteers. While it's good that no one company controls resources shared by the community, it's terrible that the only people who work on our shared infrastructure are doing so for free and in their spare time.

Beyond funding development for package-management tools, Ruby Together was meant to fund incident response for RubyGems.org. Arko and David Radcliffe, who maintained the RubyGems.org servers, were the initial team funded by Ruby Together.

Moving from RubyForge to RubyGems.org had not solved the reliability problems for gem hosting. Arko wrote a retrospective on the first 18 months of Ruby Together in September 2016; he mentioned a security issue that led to a RubyGems.org outage, "about 3 years ago", that lasted more than a week:

There was a security issue, and although the team knew about the security issue and planned to fix it, everyone on the team had a day job. The best we could do was plan to fix the problem that weekend. Unfortunately, a motivated hacker figured out how to break to during the week.

We had to take the servers down and create completely new servers from scratch. We also had to download and verify every single one of the hundreds of thousands of .gem files, to make sure the hacker hadn't replaced any of them while he had access.

Arko also recounted the work that Ruby Together had helped fund, and lamented that membership had remained flat since it started:

We haven't seen any new companies join for almost six months. If that keeps up, we won't even be able to keep up paying for a few hours of work per week. If companies keep taking community benefits without giving back, it takes us straight back to unsustainable volunteers burning themselves out.

At the end of 2016, the nonprofit had revenue of $268,237, and expenses of $305,183. A look at all of the organization's tax filings shows that its revenue peaked in 2017 at $324,331. While Ruby Together paid for some development work and operations, Ruby Central covered the hosting costs for RubyGems.org.

This is, by now, a familiar story—an open-source community creates a service that is widely used but suffers the tragedy of the commons. Everyone wants to use the fruits of the project, but no one wants to pay for it.

Centralized

After years of trying to persuade companies to fund the organization, Ruby Together board member Jonan Scheffler suggested that the two nonprofits merge. Ruby Central announced the merger in 2021. Ruby Together was dissolved and Ruby Central was to provide support for RubyGems, Bundler, and operations of RubyGems.org in addition to paying for hosting. Development of the open-source projects and management of RubyGems.org continued to be led by the community with informal governance. Ruby Central formed an Open Source Software (OSS) committee in August 2023 to formalize processes and lay "the groundwork for the continued success of these essential Ruby Central tools".

All of that is what led up to the recent power struggle between Ruby Central and Ruby volunteers working on the tools and services. In the next article, we will dig into that struggle and its aftermath so far.

---

to post comments