Development
On the maintainability of Ruby
Lucas Nussbaum, longtime maintainer of Debian's Ruby packages, announced that he was stepping back from the role on January 2, making the future of Ruby packaging on the distribution uncertain. He cited a list of reasons leading to the decision on his blog — an assortment of project management, release process, and packaging concerns with Ruby itself that compounded his frustration to the point where he felt quitting was the best option. The public reaction to the announcement has been sympathetic to the hardships of Debian package maintainer-ship, but it has also sparked an interesting debate about the competing needs of developers, end users, and Linux distributors.
Management
Nussbaum had several criticisms of the overall management of the Ruby project. The most obvious is the confusing array of development branches. He observes that there are currently six branches of the project in active development: 1.8, 1.8.6, 1.8.7, 1.9.1, 1.9.2, and trunk. While the present status of two of them (1.8.6 and 1.9.1) is clearly marked as important bug-fixes only, the rest are unclear. If 1.9.x is the current development branch, he asks, why are there any active branches in 1.8.x? He also asks about the relationship between 1.9.2 and trunk, and about whether or not there is any API or ABI compatibility between the two.
He also observed that there appears to be no "common understanding
" of the release expectations of any of the branches. That phrase could be understood in several different ways, but he adds that it "would be fantastic to have something similar to Python Enhancement Proposals
" for Ruby, which suggests the lack of a clear roadmap and/or sunset period for older releases is the underlying issue.
Ruby's direction is set primarily through mailing list discussions, which constitutes another issue. Nussbaum laments that the primary discussion list, ruby-dev, is written in Japanese, while a second list for English-speakers, ruby-core, misses out on the discussion and gets out-of-step. Ruby was invented in Japan, of course, and most of its core developers are Japanese, but maintaining two lists divides the community, keeps out most of the world, and even risks the possibility of a project fork somewhere down the line.
Releases
Releases themselves constitute another problem for Nussbaum, who observes that 1.8.7 and 1.9.2 releases were both made on December 25, 2010, apparently without any beta testing period or Release Candidates preceding them.
On a note similar to the language barrier between the lists, he observes that the Ruby project is in the habit of regularly making its major releases on December 25th, which is the traditional Western date for Christmas, and is a day when a large percentage of offices are closed. The result is that users in Europe, North and South America, and Australia are unavailable to test and provide feedback for several days or weeks.
But most importantly, Nussbaum argues that the Ruby platform consists of more than just the standard implementation Ruby interpreter, known as Matz's Ruby Interpreter (MRI). There are an increasing number of alternative interpreters popular with developers, including the Java Virtual Machine-based JRuby, Rubinius, and MacRuby, but there is no clear process for integrating them with the rest of the Ruby ecosystem.
Furthermore, the ecosystem includes large-scale projects and libraries such as Ruby on Rails, on which a great deal of real-world Ruby code depends. The development community, Nussbaum says, should create a process to ensure that new releases of MRI work with those "big players
" before making a release.
Ruby's release process has been criticized by others in the past. In 2008, security holes based on integer overflows were uncovered in the released code, which might have been caught by a public beta testing period, and the fixes pushed out by the project broke the Rails framework.
Users, developers, and systems administrators, oh my....
The most contentious issue cited by Nussbaum in his blog post is how Ruby and Ruby libraries are packaged for distribution. The official, preferred method for installing Ruby is Ruby Version Manager (RVM), a command-line tool that allows individual users to install multiple self-contained Ruby environments, including interpreters, libraries, and applications. Ruby libraries are generally distributed through RubyGems, an online distribution network that functions much like Perl's CPAN. There is a command-line client, which fetches and installs individual packages in the Ruby-specific gem format.
This system works quite well for developers, Nussbaum says, but is
unnecessarily hard on both system administrators and end users. End users,
he says, expect to be able to install a Ruby-based application using the
normal system tools (such as
apt-get). Requiring them to instead
install RVM and compile Ruby from source, then use the RubyGems package
manager to install dependencies that live outside the normal package
management system is unreasonable. The problems of the end user are
compounded for the system administrator, who may be asked to maintain
multiple Ruby interpreters over long periods of time, as well as cope with
users each installing their own collections of gems.
Nussbaum also cites several examples of negative, offensive behavior in public — some of it directed at him (accusing him of intentionally creating "crippled" Ruby packages for Debian), but more directed at others.
As for the future, he points out that there are two other Ruby packagers for Debian, who may together decide to continue maintaining the package without him, and he wishes them luck. What is less clear is the future of the Ruby-extras package, which includes most of the Ruby libraries. There appears to be no successor for the maintainer's role for that package. On the plus side, he does say that he is open to returning as a Ruby packager in the future, if the landscape improves.
Reaction
Nussbaum's post spawned quite a bit of discussion, both in comments to his blog and on YCombinator. A few commenters on both sites accused him of "imperialism" because of the list language issue, before it was pointed out that Nussbaum himself is a native French speaker. But by and large, the debate swirled around the clash between the Ruby project's goals and the Debian project's goals.
The essential argument seems to be that Ruby, as a language project, should and does seek to make life as easy as possible for Ruby developers. Debian, on the other hand, seeks to package upstream software in a way to make life as easy as possible for end users.
Developers will almost always want to run bleeding-edge code, to make use of the latest and greatest language and interpreter features. Consequently, they want to compile and install the Ruby environment from source, and want to have access to specific versions of Ruby libraries — sometimes multiple versions, as concurrent projects may demand — and they want to manage those dependencies in gem packages, not Debian packages.
Users, in stark contrast, want installation of Ruby applications to be simple, which demands integration with the main distribution package management system, not use of a specialized Ruby tool. Stuck right in the middle are systems administrators, who get asked to support scores of locally-compiled and installed Ruby environments, and distributions, who get asked to package scores of minor releases for Ruby interpreters and libraries.
Commenter Meneer R pointed out that this a common tension when dealing with interpreted languages, because the execution environment and the development environment are the same. Compiled languages always have the relatively easy out of statically linking-in an old library. Meneer R and several others argue that the "Debian philosophy" needs to change; namely that the distribution should accept the Ruby community's packaging and otherwise get out-of-the-way. Developers, he said, target specific versions of the Ruby platform, not specific releases of Debian, and trying to shoehorn the two into one release system only complicates things for Ruby developers using Debian. Several others pointed out that Ruby and Ruby libraries have to work across all operating systems, and that Debian's packaging techniques simply don't apply to Windows, OS X, or other platforms.
Others from the Debian community take the opposite stance. Commenter Np237 argued that allowing multiple versions of one library to co-exist in the system is itself the problem, because:
Debian developer Gunnar Wolf posted a response on his own blog, generally showing support for Nussbaum. Regarding the conflict between Ruby packages and Debian packages, he observed that maintaining multiple versions of the same library on a system in order to support multiple incompatible applications has inherent risks, such as bug fixes not getting back-ported, and security and stability problems developing over time. The need to keep multiple versions of an interpreter or library installed generally means there are big changes between different releases, he said, and that means that the library is immature and not ready for long-term API commitment — which in turn makes it a bad bet for the developer.
Wolf also weighed in on one of the repeated claims in support of Ruby's packaging system: that because the language is primarily used by web developers, "end users" never need to install it anyway. Web frameworks like Rails unduly influence the discussion, he said, because the core packaging issues exist for non-web developers, too. In addition, saying that web applications live by different rules ignores the needs of systems administrators, he said.
Nussbaum is not the only packager to grapple with the difficulty of fitting Ruby and RubyGems into a distribution's package management system. Fedora has a Ruby special interest group (Ruby SIG) which tackled many of the same issues one year ago. Fedora manages multiple concurrent Ruby releases (1.85, 1.8.6, 1.9.1, and 1.9.2) using the alternatives system, and has packaging guidelines that attempt to wrap gems inside RPM files. Nussbaum commented on his blog that alternatives was not the solution for Debian, because by definition it is designed for tools that can be used interchangeably, which different releases of Ruby interpreters cannot.
Despite all of the arguments on both sides of the packaging issue, the discussion did remain completely civil. Nussbaum updated his post to point to two positive steps taken by the Ruby community: a JRuby maintainer outlining his project's vision and goals as relates to the larger Ruby development ecosystem, and a message from a core Ruby developer inviting more discussion on the English-speaking mailing list, and clarifying that there is no intention of keeping the two subscriber bases separate.
Paul Brannan, one of the original authors of RubyGems, even joined in the discussion, explaining that gem packages were designed with the goal of reducing the strain on administrators and users (including the ability to package a gem inside of an RPM or Debian package). He also offered mea culpas for those design decisions that, with hindsight, did not work out so well.
Language maturity
Wolf made an interesting aside in blog post, noting along the way that Ruby is no longer a "new language
". That sentiment is clearly present in Nussbaum's retirement announcement, even if he did not explicitly say it. With a young language, people (developers and systems administrators) expect things to change rapidly and sometimes without warning. But Ruby is fifteen years old now. That may be young compared to C and FORTRAN, but it is certainly not a flash-in-the-plan.
Issues like those Nussbaum raised are difficult for projects to deal with. It is awkward to change the existing culture and tell developers that old language features are deprecated. It is particularly awkward to suggest that a project has grown so much that the main mailing list needs to shift to the lingua franca of distributed development, rather than the original authors' language. But like it or not, even if the two mailing lists do not separate developers into first- and second-class community members, they do split the community into two circles that don't work together.
Yet not all of the Nussbaum's issues are that difficult; there are
things that Ruby can do to make life easier downstream. There is still no
formal language specification, for example. Standardizing that would help
unify the competing interpreters. A more formal process for language
extension would ease a lot of minds, and a stricter release schedule with
targeted betas would make the community feel more involved. Those are
goals any open source project would be proud of, all the more so for the language designated as "A programmer's best friend
".
Brief items
Quotes of the week
Amarok 2.4 released
Version 2.4 of the Amarok media player is available. "This release brings significant performance, usability, and stability improvements: Most of the bugs involving the context pane have been squished, a completely rewritten collection scanner is presented (hint: it's the thing that reads your music files' tags) -- Amarok now has the ability to write your statistics and covers directly to the audio files to keep the information across different devices and should detect compilations a lot better. At long last, we also have an option to hide the menu bar again." Also in this release are new device support, transcoding of tracks, and more.
A mailing list for Android audio developers
A new mailing list has been created for developers interested in audio issues on the Android platform. "This list is meant to be a place to discuss about audio development, in the context of music and sound applications, but also games and other apps which use audio on Android."
FFmpeg turmoil
A group of FFmpeg developers has announced that the project has a new set of maintainers - news which came as a surprise to existing maintainer Michael Niedermayer. Developer Diego Biurrun has posted an explanation of how the coup came to be, but it's clear that not everybody is satisfied. "The discontent reached the point where a fork was being contemplated and then planned, but it turned out that the momentum had soared way past critical mass and turned into a tidal wave of revolution. The focus moved from forking to avoiding a fork if possible. Since git was being set up on videolan.org, setting up an alternative git tree on mphq was the natural choice. With development moving to videolan.org and such a large group of developers already part of the revolution keeping the infrastructure was the logical consequence." (Thanks to Mattias Mattsson).
MDP release 3.0
Version 3.0 of the Modular Toolkit for Data Processing is out. "MDP is a Python library of widely used data processing algorithms that can be combined according to a pipeline analogy to build more complex data processing software." Changes include Python 3 support, some new extensions, new algorithms, an improved tutorial, and a switch to the BSD license.
Parrot 3.0.0 Released
Version 3.0.0 of the Parrot virtual machine is available. "On behalf of the Parrot team and an enthusiastic but undiscriminating dachshund that followed me home last week, I'm proud to announce Parrot 3.0.0, also known as "Beef Stew", or at the insistence of a shadowy government organization, "Snowflake". Parrot (http://parrot.org) is a virtual machine that dreams about running all dynamic languages everywhere, even the one you're think about right now. Parrot has big plans, even if needs a haircut and sometimes goes outside with its shoes untied." This version includes improvements in the core code and the documentation, new language support, and better test coverage.
Xfce 4.8 released
Version 4.8 of the Xfce desktop has been announced. "With Xfce 4.8 our users will be able to browse remote shares using a variety of protocols (SFTP, SMB, FTP and many more). The window clutter has been reduced by merging all file progress dialogs into a single one. Our panel application has been rewritten, thereby improving positioning, transparency, item and launcher management. It also introduces a new menu plugin to view directories. Its plugin framework remains compatible with 4.6 plugins." LWN previewed this release back in December.
Newsletters and articles
Development newsletters from the last week
- Caml Weekly News (January 18)
- LibreOffice development summary (January 18)
- PostgreSQL Weekly News (January 16)
- Tcl-URL! (January 14)
More about the Chrome HTML Video Codec Change (The Chromium Blog)
Google's Chromium blog has an update on the decision to remove H.264 support from the Chrome/Chromium browser. "We acknowledge that H.264 has broader support in the publisher, developer, and hardware community today (though support across the ecosystem for WebM is growing rapidly). However, as stated above, there will not be agreement to make it the baseline in the HTML video standard due to its licensing requirements. To use and distribute H.264, browser and OS vendors, hardware manufacturers, and publishers who charge for content must pay significant royalties—with no guarantee the fees won't increase in the future. To companies like Google, the license fees may not be material, but to the next great video startup and those in emerging markets these fees stifle innovation."
Page editor: Jonathan Corbet
Next page:
Announcements>>