On the maintainability of Ruby

Lucas Nussbaum, longtime maintainer of Debian's Ruby packages, announced that he was stepping back from the role on January 2, making the future of Ruby packaging on the distribution uncertain. He cited a list of reasons leading to the decision on his blog — an assortment of project management, release process, and packaging concerns with Ruby itself that compounded his frustration to the point where he felt quitting was the best option. The public reaction to the announcement has been sympathetic to the hardships of Debian package maintainer-ship, but it has also sparked an interesting debate about the competing needs of developers, end users, and Linux distributors.

Management

Nussbaum had several criticisms of the overall management of the Ruby project. The most obvious is the confusing array of development branches. He observes that there are currently six branches of the project in active development: 1.8, 1.8.6, 1.8.7, 1.9.1, 1.9.2, and trunk. While the present status of two of them (1.8.6 and 1.9.1) is clearly marked as important bug-fixes only, the rest are unclear. If 1.9.x is the current development branch, he asks, why are there any active branches in 1.8.x? He also asks about the relationship between 1.9.2 and trunk, and about whether or not there is any API or ABI compatibility between the two.

He also observed that there appears to be no "common understanding" of the release expectations of any of the branches. That phrase could be understood in several different ways, but he adds that it "would be fantastic to have something similar to Python Enhancement Proposals" for Ruby, which suggests the lack of a clear roadmap and/or sunset period for older releases is the underlying issue.

Ruby's direction is set primarily through mailing list discussions, which constitutes another issue. Nussbaum laments that the primary discussion list, ruby-dev, is written in Japanese, while a second list for English-speakers, ruby-core, misses out on the discussion and gets out-of-step. Ruby was invented in Japan, of course, and most of its core developers are Japanese, but maintaining two lists divides the community, keeps out most of the world, and even risks the possibility of a project fork somewhere down the line.

Releases

Releases themselves constitute another problem for Nussbaum, who observes that 1.8.7 and 1.9.2 releases were both made on December 25, 2010, apparently without any beta testing period or Release Candidates preceding them.

On a note similar to the language barrier between the lists, he observes that the Ruby project is in the habit of regularly making its major releases on December 25th, which is the traditional Western date for Christmas, and is a day when a large percentage of offices are closed. The result is that users in Europe, North and South America, and Australia are unavailable to test and provide feedback for several days or weeks.

But most importantly, Nussbaum argues that the Ruby platform consists of more than just the standard implementation Ruby interpreter, known as Matz's Ruby Interpreter (MRI). There are an increasing number of alternative interpreters popular with developers, including the Java Virtual Machine-based JRuby, Rubinius, and MacRuby, but there is no clear process for integrating them with the rest of the Ruby ecosystem.

Furthermore, the ecosystem includes large-scale projects and libraries such as Ruby on Rails, on which a great deal of real-world Ruby code depends. The development community, Nussbaum says, should create a process to ensure that new releases of MRI work with those "big players" before making a release.

Ruby's release process has been criticized by others in the past. In 2008, security holes based on integer overflows were uncovered in the released code, which might have been caught by a public beta testing period, and the fixes pushed out by the project broke the Rails framework.

Users, developers, and systems administrators, oh my....

The most contentious issue cited by Nussbaum in his blog post is how Ruby and Ruby libraries are packaged for distribution. The official, preferred method for installing Ruby is Ruby Version Manager (RVM), a command-line tool that allows individual users to install multiple self-contained Ruby environments, including interpreters, libraries, and applications. Ruby libraries are generally distributed through RubyGems, an online distribution network that functions much like Perl's CPAN. There is a command-line client, which fetches and installs individual packages in the Ruby-specific gem format.

This system works quite well for developers, Nussbaum says, but is unnecessarily hard on both system administrators and end users. End users, he says, expect to be able to install a Ruby-based application using the normal system tools (such as apt-get). Requiring them to instead install RVM and compile Ruby from source, then use the RubyGems package manager to install dependencies that live outside the normal package management system is unreasonable. The problems of the end user are compounded for the system administrator, who may be asked to maintain multiple Ruby interpreters over long periods of time, as well as cope with users each installing their own collections of gems.

Nussbaum also cites several examples of negative, offensive behavior in public — some of it directed at him (accusing him of intentionally creating "crippled" Ruby packages for Debian), but more directed at others.

As for the future, he points out that there are two other Ruby packagers for Debian, who may together decide to continue maintaining the package without him, and he wishes them luck. What is less clear is the future of the Ruby-extras package, which includes most of the Ruby libraries. There appears to be no successor for the maintainer's role for that package. On the plus side, he does say that he is open to returning as a Ruby packager in the future, if the landscape improves.

Reaction

Nussbaum's post spawned quite a bit of discussion, both in comments to his blog and on YCombinator. A few commenters on both sites accused him of "imperialism" because of the list language issue, before it was pointed out that Nussbaum himself is a native French speaker. But by and large, the debate swirled around the clash between the Ruby project's goals and the Debian project's goals.

The essential argument seems to be that Ruby, as a language project, should and does seek to make life as easy as possible for Ruby developers. Debian, on the other hand, seeks to package upstream software in a way to make life as easy as possible for end users.

Developers will almost always want to run bleeding-edge code, to make use of the latest and greatest language and interpreter features. Consequently, they want to compile and install the Ruby environment from source, and want to have access to specific versions of Ruby libraries — sometimes multiple versions, as concurrent projects may demand — and they want to manage those dependencies in gem packages, not Debian packages.

Users, in stark contrast, want installation of Ruby applications to be simple, which demands integration with the main distribution package management system, not use of a specialized Ruby tool. Stuck right in the middle are systems administrators, who get asked to support scores of locally-compiled and installed Ruby environments, and distributions, who get asked to package scores of minor releases for Ruby interpreters and libraries.

Commenter Meneer R pointed out that this a common tension when dealing with interpreted languages, because the execution environment and the development environment are the same. Compiled languages always have the relatively easy out of statically linking-in an old library. Meneer R and several others argue that the "Debian philosophy" needs to change; namely that the distribution should accept the Ruby community's packaging and otherwise get out-of-the-way. Developers, he said, target specific versions of the Ruby platform, not specific releases of Debian, and trying to shoehorn the two into one release system only complicates things for Ruby developers using Debian. Several others pointed out that Ruby and Ruby libraries have to work across all operating systems, and that Debian's packaging techniques simply don't apply to Windows, OS X, or other platforms.

Others from the Debian community take the opposite stance. Commenter Np237 argued that allowing multiple versions of one library to co-exist in the system is itself the problem, because:

Debian developer Gunnar Wolf posted a response on his own blog, generally showing support for Nussbaum. Regarding the conflict between Ruby packages and Debian packages, he observed that maintaining multiple versions of the same library on a system in order to support multiple incompatible applications has inherent risks, such as bug fixes not getting back-ported, and security and stability problems developing over time. The need to keep multiple versions of an interpreter or library installed generally means there are big changes between different releases, he said, and that means that the library is immature and not ready for long-term API commitment — which in turn makes it a bad bet for the developer.

Wolf also weighed in on one of the repeated claims in support of Ruby's packaging system: that because the language is primarily used by web developers, "end users" never need to install it anyway. Web frameworks like Rails unduly influence the discussion, he said, because the core packaging issues exist for non-web developers, too. In addition, saying that web applications live by different rules ignores the needs of systems administrators, he said.

Nussbaum is not the only packager to grapple with the difficulty of fitting Ruby and RubyGems into a distribution's package management system. Fedora has a Ruby special interest group (Ruby SIG) which tackled many of the same issues one year ago. Fedora manages multiple concurrent Ruby releases (1.85, 1.8.6, 1.9.1, and 1.9.2) using the alternatives system, and has packaging guidelines that attempt to wrap gems inside RPM files. Nussbaum commented on his blog that alternatives was not the solution for Debian, because by definition it is designed for tools that can be used interchangeably, which different releases of Ruby interpreters cannot.

Despite all of the arguments on both sides of the packaging issue, the discussion did remain completely civil. Nussbaum updated his post to point to two positive steps taken by the Ruby community: a JRuby maintainer outlining his project's vision and goals as relates to the larger Ruby development ecosystem, and a message from a core Ruby developer inviting more discussion on the English-speaking mailing list, and clarifying that there is no intention of keeping the two subscriber bases separate.

Paul Brannan, one of the original authors of RubyGems, even joined in the discussion, explaining that gem packages were designed with the goal of reducing the strain on administrators and users (including the ability to package a gem inside of an RPM or Debian package). He also offered mea culpas for those design decisions that, with hindsight, did not work out so well.

Language maturity

Wolf made an interesting aside in blog post, noting along the way that Ruby is no longer a "new language." That sentiment is clearly present in Nussbaum's retirement announcement, even if he did not explicitly say it. With a young language, people (developers and systems administrators) expect things to change rapidly and sometimes without warning. But Ruby is fifteen years old now. That may be young compared to C and FORTRAN, but it is certainly not a flash-in-the-plan.

Issues like those Nussbaum raised are difficult for projects to deal with. It is awkward to change the existing culture and tell developers that old language features are deprecated. It is particularly awkward to suggest that a project has grown so much that the main mailing list needs to shift to the lingua franca of distributed development, rather than the original authors' language. But like it or not, even if the two mailing lists do not separate developers into first- and second-class community members, they do split the community into two circles that don't work together.

Yet not all of the Nussbaum's issues are that difficult; there are things that Ruby can do to make life easier downstream. There is still no formal language specification, for example. Standardizing that would help unify the competing interpreters. A more formal process for language extension would ease a lot of minds, and a stricter release schedule with targeted betas would make the community feel more involved. Those are goals any open source project would be proud of, all the more so for the language designated as "A programmer's best friend".