Oregon State University's Open Source Lab (OSU-OSL) has gotten a hand up
from Facebook to create a on-demand testing infrastructure for open source
projects called Supercell.
The idea behind Supercell is to provide limited duration hosting for
open source projects that need to test on specific operating systems, as
well as providing facilities for projects to test software in a large
cluster with several VMs running concurrently.
finished, Supercell will provide test infrastructure for open source
projects that don't have their own server farm and testing
The project was announced
on Thursday, January 20 by Facebook's Scott MacVicar. MacVicar wrote that
there's a disparity in development resources between many open source
projects and companies doing in-house development of software: namely that
many open source projects lack the kind of hardware and testing
infrastructure that companies have at their disposal.
To help solve the problem, Facebook has decided to donate hardware and
funding to OSU-OSL to develop Supercell, a service for projects to test on
multiple operating systems and architectures. The Open Source Lab provides
hosting to quite a
few open source projects and communities, so it's not surprising that
Facebook would look to OSL for assistance with Supercell. Why is Facebook
interested? OSL's Leslie Hawthorn said that, while she didn't want to speak
for Facebook, her conversations with the company indicated that Facebook's
goal with open source is to "let people make useful stuff,"
and that OSL was a natural partner because "they know we're neutral,
and we're here for the benefit of open source."
Currently the hardware is x86 and AMD64, with a number of guest OSes
available. At present, Supercell supports Debian Lenny (5.0), CentOS 5.5, Gentoo,
Gentoo Hardened, and Ubuntu Lucid (10.04), Karmic (9.10), and Maverick (10.10). According to
MacVicar, some Mac OS X servers (two Apple Xserves) are also available
"for those projects that explicitly need to test on Mac OS."
The current hardware for Supercell is two Dell servers with 4 Opteron
2.1GHz 12-core CPUs and 128GB of RAM each, another with 2 Intel
E5620 2.4GHz 4-core CPUs and 12GB of RAM, and 12TB of disk for NFS
Plans are also on the table to support Fedora, FreeBSD, and OpenBSD in the
near future, and OSU-OSL is evaluating feedback from the community in
deciding on additional OSes and architectures. According to the FAQ, Supercell may support
Alpha, ARM, ARMel, PowerPC, SPARC, and others as a longer term goal.
The hardware cluster is being managed by Ganeti and Ganeti Web Manager
on top of Linux and KVM. The entire stack under Supercell is, of course,
Ganeti, which was
originally developed by Google, is a tool for virtualization management
that handles deploying and managing virtual machine instances on top of KVM
or Xen. According to its documentation,
Ganeti can deploy a new virtual machine running Ubuntu in under 30 seconds,
including hostname, networking, and serial console set up. Images
are a gzipped tarball or filesystem dump of an operating system, usually
running between 200 and 400MB in size.
Ganeti has been in development for some time, with its initial public
release in August
2007 under the GPLv2. Google started the project in its Zurich office
for cluster management of virtual servers on commodity hardware —
pretty much the same thing that OSU-OSL and Facebook were looking for.
The Ganeti Web Manager is a bit newer, a result of collaboration between
OSL and Google Code-In students from the 2010 Google
Code-In. Ganeti Web Manager is a Django-based application that provides
Web-based management for Ganeti clusters. It's still maturing, but the 0.4
release from December 22, 2010 is considered "enough to get people to start
using it in production" according
to OSL's Lance Albertson. The 0.4 release implements basic VM
management, VNC console, a permissions
system for managing clusters and virtual machines, and SSH
Since the service is "on-demand," what happens when a project comes back
for a second round of testing? According to OSL's operations manager Jeff
Sheltren, OSL plans to tie into a configuration management framework such
as Puppet so projects can save and reuse configurations. "This will
allow OSL to provide a base set of standard configurations people can use
(think: 'I need a LAMP stack') as well as giving projects the ability to
fine tune their environment and re-use that configuration for future
Projects eager to get hands on with Supercell will have to wait a few
more months, at least if they're hoping to use OSU's hardware and
services. The service is considered early alpha at this point, with a
projection that it will be ready by the third quarter of 2011. In the
interim, OSU is looking to find additional sponsors for Supecell. Sheltren
says that Facebook's donation amounts to about $50,000 in hardware and
funds. This will support a fair amount of development, but there's plenty
of work ahead.
Currently there is no discussion list for the Supercell service, but OSL
is looking for feedback
on what other operating systems that Supercell could support and other
requests from the community. By submitting the feedback form, interested
parties can sign up for the Supercell announcement list as well. Hawthorn
did say that OSL may set up a discussion list, and will be providing
regular updates about Supercell via the blog and Twitter.
But projects looking to implement their own "Supercell" can start today,
just add hardware. Developers interested in helping with Ganeti and Ganeti
Web Manager can find more info on the wiki including the
mailing list and
Though Ganeti Web Manager is considered "production ready" by Albertson,
it still has a lengthy
roadmap of features that OSL plans to integrate. For instance, templates for virtual
machines, ability to modify or reinstall VMs, implement
support for noVNC instead of
the Java VNC client, and serial console support.
With any luck, Facebook won't be the sole supporter of Supercell outside
of OSU-OSL. The project has a lot of potential to provide a much-needed
facility for short-term testing resources that many projects simply
to post comments)