Interview: Fedora's new cloud manager

Clearly, "Cloud" is a both marketing term and a hot business buzzword, neither of which lend themselves to clarity. However, there are some actual significant changes in computing represented by the word.

On the business side, there's a trend towards centralization of resources — sometimes described as a big, constant pendulum swing, with "cloud data center" simply standing in for "mainframe" this time around — but there are actually interesting new developments which make cloud computing compelling. It may be that I've been in a university setting too long, but I like the NIST definition [PDF] which describes the essential cloud characteristics. Or, there's the "OSSM" definition [video], which goes like this:

• On-demand: the resources are already set up and ready to be deployed.

• Self-service: you choose what you want, when you want it.

• Scalable: you also choose how much you want, and ramp up if necessary.

• Measurable: metering and reporting are integrated so you know you are getting what you pay for.

With this option, if you've got a new startup, putting together your own data center is suddenly crazy. If you're an agile developer, on-demand self-service is very appealing. And for larger enterprises, it's amazing to have built-in scalability and measurability.

Small, nimble companies are already benefiting from cloud computing; bigger companies are mostly dipping their toes in. There's a lot of interest in on-premises private cloud, and especially in transparent hybrid cloud (where local and public cloud infrastructure are mixed together). All of the technology is still in flux and there are plenty of unresolved questions, but it's certainly a matter of how-much-how-quickly, not of if-at-all.

On the user and client-platform side, the important trend is mobile and tablet computing, and the movement away from general-purpose computing devices to a locked-down "app store" model — even for desktop systems. Previous attempts at making restricted computing devices always failed in the market, because while most people only do a few things with their computers, there's a long tail of different things each person wants, and any narrow selection of most-common tools was never good enough.

Developer-friendly app marketplace and distribution channels which run right on the device get around that — even when the platform is locked down, consumer convenience is served by "there's an app for that". Consumers get access to tools, developers get a market, and vendors get lock-in and control. In proprietary operating systems, we're going to see more and more of that.

[PULL QUOTE: So, a key role for free and open source Linux distributions is to provide a viable alternative to the locked-in "consumption device" dystopia. This includes providing the convenience and functionality of cloud computing that proprietary platforms offer. END QUOTE] There's one key thing the NIST cloud definition covers that isn't in the OSSM one (to be pronounced "awesome", by the way). That's "broad network access", and it means you can get to your data from anywhere, from any client platform. It's the lure of cloud computing from an end-user point of view — don't worry if you lose your phone, because it's all safe in the cloud. In fact, don't worry if you're house burns down, because your family photos are all safely floating out there as well.

At least, don't worry as long as there are sufficient protections in place! A number of people are saying that open source on the desktop (or any client device) doesn't even matter — that open web is the new front, and that the battle is about making sure people have control over and access to their own remotely-stored data.

I think that's important, and I'm glad people are fighting for it, but I disagree that it's sufficient. It's not hard to imagine a future where the normal tech platform most people buy isn't able to run arbitrary code — look at the boot restrictions for Windows 8 ARM systems. Historically, the mass-market platform has also been the development platform, and that's done wonderful, magical things for the democratization of invention. But in a restricted future, the flexible development platform is a special niche product — more expensive and maybe even not available to everyone.

So, a key role for free and open source Linux distributions is to provide a viable alternative to the locked-in "consumption device" dystopia. This includes providing the convenience and functionality of cloud computing that proprietary platforms offer, whether it's through open web and open cloud initiatives, or through building in local cloud and peer-to-peer cloudlike services.

• free culture is welcoming and widespread,
• collaboration is commonplace, and
• people control their content and devices.

We want these things in cloud computing as well, and over the next few months I'll be working with other people in Fedora to identify more specific outcomes to work toward in cloud computing, and from there we'll develop programs and activities focused on those outcomes.

We have a strong base in the Fedora distribution, a great worldwide user and developer community, and excellent infrastructure. And, we have a lot of cloud-related work in progress.

We're also a huge center for cloud infrastructure software packaging and development. OpenShift Origin and OpenStack Folsom are two of the big features for the upcoming Fedora 18 release, along with Eucalyptus, OwnCloud, and Heat (a cloud orchestration API for OpenStack). We've also got work in progress on OpenNebula and CloudStack.

Now, I don't think anyone has delusions that a great number of people will run production infrastructure on top of Fedora with these packages. The main two use cases are: a) developers of cloud software (including, but definitely not limited to, Red Hat) working to make sure the software is ready for future use in enterprise Linux products, and b) cloud early-adopters who are following that development. Those constituent groups are going to remain crucial to Fedora in general and Fedora cloud in specific, and I think there are some gentle adjustments we can make that will help encourage those relationships even more.

Over the years we've lost a lot of our sysadmin and server-side Linux users, as they've felt somewhat left behind by all the energy and development around desktop. Sometimes it's seemed like the only voice available to that group has been a negative one — "Hey, slow down!" — which is a frustrating side of the conversation to be on, when really we all want to work together to make better software and a better world (back to that vision statement). When you're cast in the stop-energy role, it's hard to feel listened to, let alone constructive.

So, the rise in cloud and all the exciting new development work gives that group a positive voice. That's good for Red Hat Enterprise Linux as a downstream project. It's good for developers of cloud software because we'll make sure their code works in the distribution. And it's good for Fedora, because this is a large group of technology professionals with years of experience and wisdom.

I especially want to expand what we can do for the small, agile organizations working at the leading edge of cloud technology. For this, the Fedora focus isn't so much on the cloud infrastructure software as on the cloud guest images.

As I mentioned, we've got an up-to-date EC2 image, and we're working on making that even more lightweight and on offering more variants for different use cases. We also support appliance creation with BoxGrinder (a Fedora 15 feature), and JEOS ["just enough operating system"] builds with Oz. Or if you're looking for something more complete — for example, to make a virtualized developer's desktop — it's easy to make an image from our Live CD ISO (although we need to make that several steps easier).

Producing images isn't enough by itself, though. We also need to increase our engagement with the DevOps community. At this point, that means mostly listening and being present, more than direct evangelism. Fedora has always aimed at users who might become contributors, and the DevOps world is a natural fit, with good alignment with our values of "freedom, friends, features, first".

On a technical level, we see the continuous struggle between having the most up-to-date versions of specific software (Ruby gems, for example), while having a base that you just don't have to worry about. That's a hard problem, but in order to be relevant to actual users, we need to address it, whether it's through Software Collections, with something related to OpenShift cartridges, or in some other way.

We're also looking into how we can make better use of cloud technology for Fedora developers. The Fedora Infrastructure Team has deployed and is evaluating Eucalyptus and OpenStack for package mass rebuilds, for automated (and non-automated) testing, and eventually for more.

The Fedora features process has been great for the distribution as a whole and is successful overall in helping us look at development in a goal-oriented way. I want to make it easier for Fedora cloud contributors – both new and already involved – to participate in this. As with any process in software engineering, the "meta-work" involved is often very painful to the actual implementers, sometimes simply because of the context switches required, not because the actual paperwork is particularly onerous. That wastes valuable developer time and produces less than ideal process results. We need to have people – like, for example, me – involved in the various features who can both keep up with the technical work and keep meta-work from being a burden to developers, while still reaping the benefits of good process.

For end-user and desktop cloud services, we haven't yet explored all of the possibilities. We can make it easier for moderately-savvy users to set up their own private infrastructure using SparkleShare or OwnCloud as alternatives to proprietary hosted filesharing. Our OpenStack packages will include Puppet modules making it trivial to get a fully-functional private cloud out of the box, and we can extend that to other parts of the distribution as well. We'll also continue to look at what more we can do to enable users to connect themselves to open cloud offerings in ways that align with the project mission. It'd be nice to provide a push-button process where open source web applications can be deployed either locally or pushed to a cloud provider — a tie in to Red Hat's OpenShift, say, or to any open cloud provider.

But, those cloud providers should use Fedora for their testbeds, precisely because the field itself is on the bleeding edge and we can follow it more quickly. For example, QEMU 1.2 was pulled into Fedora right after it was released, and we track the upstream kernel closely, which means we get the latest hardware and low-level virtualization support. Also, Fedora is upstream for RHEL and is home to a lot of the exciting development activity done by Red Hat. If you have an open source / free software cloud project that you want to work in that ecosystem, Fedora is the natural path.

Meanwhile, those cloud providers should be interested in supporting Fedora as a guest OS, because it provides value to their users. It's important to stress that while Fedora is fast-moving, every actual release is production-ready and stable. I won't claim perfection, but we have an excellent quality engineering team that takes this very seriously. While we develop on a six-month development cycle, we make sure it works before we release.

There's always some background talk about a rolling release, but I like the cadence of actual releases, with planning and a features process, and the quality assurance work and release engineering would be hard to replicate in the rolling model. We have ongoing work on in-place upgrades using yum, and further polish to that will reduce the pain of upgrades. The model I mentioned earlier where OpenStack configuration is shipped as a Puppet module is also a good direction, since that helps abstract away the specifics of the underlying system.

Right now, we support upgrades going two releases back, which fits the Fedora lifecycle, but we also want to make sure people who miss that mark aren't entirely left behind. One specific thing I'm working on is putting together tools and resources for the many people still running on Fedora 8 in Amazon EC2.

We've also had a continuous discussion in Fedora about the fire hose of updates during a release's lifetime. Bringing that under control is important for real-world use, but we don't want to keep users from getting new features quickly or prevent developers and packagers from getting their code out. I'm very much in support of the idea of bundling non-security updates into monthly sets which can be tested and installed in a more controlled way. I also strongly prefer to see development targeted at Rawhide and future releases, with a focus on stability for the thirteen months of a release's maintenance cycle. Or, beyond that, a focus on stability for the first twelve months and a focus on smooth upgrades for the final one.

But, beyond software and beyond technology, we bring an important set of values and a vision for a free, open, and collaborative future which is just as vital in a cloud computing future as it is in the older local-computing model.

What needs to be done to make Fedora more interesting in the cloud setting? In addition to what I've already talked about, I think most crucially we need to increase community involvement and build up our user and contributor base. That's part of my job, too, and I want to help Fedora respond to what the community needs. I'm working hard to listen, discuss, read, and absorb as much as possible. We need input from everyone who wants to make Fedora better, and who is interested in extending the benefits of free and open source software to the cloud.

But, at the user level, when we're talking about software as a service and about protections for privacy with remote data, it's a legitimate worry. Fedora needs to offer better alternatives: connections to services which put privacy and freedom first, the ability to easily stand up cloud services under your own control, and, finally, a way to work which doesn't have to be cloud-dependent.

Seriously, though, the primary tool is connections with people – in the Fedora Project, at Red Hat, and in the open cloud community in general. I'm very accessible by email, including in the Fedora Cloud SIG and development mailing lists, and I'm trying to get my 1990s-era IRC habits back up to snuff (user mattdm on freenode). I'll also be visiting a lot of conferences, and I hope to see anyone who has read this far, and those of you I can't meet in person I hope to talk with in other ways.

If you're interested in making Fedora better through cloud computing, or in making the cloud better through Fedora, please join us in the Cloud SIG. There's no experience necessary and no formal join process — just jump into the mailing list and we'll get started.