Leading items

Running Firefox OS apps on Android

On June 12, the "Mozilla Hacks" blog posted a story explaining how to install and run HTML5 "web apps" on Android. Packaged, locally installed web apps are the cornerstone of Firefox OS's user experience, of course, and are an option on several other free mobile operating system platforms like Tizen and Ubuntu Touch. But they have not played a significant role in the Android story, so the opportunity to bring them to the most popular Linux-based mobile ecosystem is potentially big news for developers as well as users.

Web apps require a compatible web runtime, of course—one that implements the APIs for accessing the necessary device features. Firefox OS uses Mozilla's Gecko as its web runtime, and it reuses Android's kernel and hardware abstraction layer (HAL), so it should perhaps come as no surprise that relatively little work is required to get Firefox to function as a web runtime within Android itself. Indeed, that is what Mozilla has now done: the recently-released version 29 of Firefox for Android can serve as the web runtime for any web app packaged for Firefox OS and submitted to Mozilla's Firefox Marketplace.

Normally, web apps built for Firefox OS are packaged as .zip files. In conjunction with the new release of Firefox for Android, the project has also built a web service called APK Factory that converts the packages to Android's .apk package format. APK Factory can be installed and used locally, although the developer will need to sign the resulting .apk packages with his or her own key in order for it to be published through Google's Play Store. The web apps submitted to Firefox Marketplace have been converted to .apk form by Mozilla, and signed with Mozilla's key, so they, too, can be installed just like any native Android app.

To install a Firefox OS web app on an Android phone or tablet, one needs only to visit the Firefox Marketplace site with the device and click on the desired app's "install" link. The standard Android confirmation dialog pops up, although users may need to enable app installation from third-party sources if they have not done so already.

As an earlier Mozilla Hacks post explained it, the key benefits of this new plan to users is that "installed" web apps are fully integrated with the Android platform's services. While one could already launch a mobile-friendly web site from a browser shortcut in Android, the packaged apps appear in the recent-app list and the app drawer, and the user has access to the usual tools to monitor the apps' permissions, memory, storage, and CPU usage. Installed apps can also be updated and uninstalled using the standard Android mechanisms.

In practice, the only noticeable shortcoming of this plan may be that the Firefox Marketplace offers substantially fewer choices than Google Play and the side-loaded Android app ecosystem (including individual .apk releases plus alternative app stores like F-Droid). Of course, one might reasonably argue that a high percentage of the world's available Android apps are not worth installing (and the sheer numbers can make finding a good choice more difficult), but the reality is that, at this point, Firefox Marketplace includes a fairly basic, no-frills selection of apps.

But there are some major names among the available options, including Box, SoundCloud, and Twitter, plus a commendable variety of utility and productivity apps. Out of curiosity, I installed a handful on a Nexus tablet. A few of them exhibited some strange quirks—for instance, typing the same number twice in rapid succession on the scientific calculator app triggered a "zoom" event; not the kind of behavior a non–web-app would fall victim to. But none that I tried failed to install or to run.

In fact, in some cases the web-app version even offers an arguably nicer experience than the native Android version. Take the official Twitter apps, for example. Notably, the Android Twitter app takes up 16.52MB of space on the device, and it wants an extensive set of permissions—including phone status and identity, SMS access, GPS and network-based location, access to contacts, access to read and delete USB storage, access to add and remove accounts, full network access, vibration control, the ability to prevent the device from sleeping, installing "shortcuts", and the ability to read and change sync settings. The web-app version of Twitter from the Firefox Marketplace takes up 60KB and uses only the location privilege. The user experience is more-or-less the same.

For app developers who wish to explore the option, Mozilla has added a section to its developer documentation dealing with the process of building and publishing web apps for Android. There is a Node.js-based command-line interface to the APK Factory service with which developers can test their web apps on Android before submitting them to the Firefox Marketplace or self-publishing them.

The other remaining issue is API availability, which may differ between Firefox OS and Android. Mozilla maintains a list on its wiki, with all Android-supported APIs marked with their level of support. That includes not only whether or not the API itself is supported, but whether it requires the user to change any preferences on their device. At the moment, several key APIs are still partially incomplete, while several others are missing but in the planning stages, including the Alarm, SimplePush, and Web Activities APIs. Several others are marked as "not currently planned," including the WebTelephony and WebBluetooth APIs.

After all that the development community has heard about HTML5's suitability as a mobile app platform, it is always nice to find an opportunity to put web apps to the test on a real device. But, while there are already options for running Tizen, Ubuntu Touch, and Firefox OS on separate devices, this new ability to install and use Firefox OS web apps side by side with their native Android competition provides a new perspective. It is usable today, which is a boon to the curious. Perhaps more interesting will be to see what impact it could have on Android app development further down the line. A large potential user base like Android could motivate a lot of developers to seriously investigate HTML5-based apps who have not done so before.

Comments (3 posted)

A report from the first DockerCon

Docker Inc. and the Docker community celebrated a 1.0 release at the first DockerCon, which was held in San Francisco on June 9 and 10. The conference slogan was "Containers Are the New Virtualization", which was not only a vision for Docker, but also a challenge to virtualization software projects. DockerCon was packed with new product and project announcements as well as demos by Docker Inc., Google, Amazon, IBM, Red Hat, and other Docker-adopting companies. The conference certainly demonstrated the excitement around Docker that has built over the last year. While almost every presenter and keynote speaker had something new to show off, the most interesting announcements and demos came from Docker Inc. itself, and from Google.

First, however, a recap on the Docker project and technology for those unfamiliar is in order. If you already know about Docker and containers on Linux, you can skip to the next section.

Some background

According to its GitHub page: "Docker is an open source project to pack, ship and run any application as a lightweight container". More technically, Docker is a management tool that enables users to easily package and deploy single-service "containers" as an alternative to virtual machines or traditional installation scripts and packages. The project's goal is to deliver the advantages of virtual machines, including isolation, portability, and easy deployment, without the overhead.

The first thing to understand is: containers are not virtual machines. A virtual machine (VM) creates an isolated runtime environment for software based on "hardware virtualization", where the VM emulates a complete hardware environment for a full guest operating system and kernel. Containers operate at a different layer: they provide an isolated operating system (OS) and filesystem environment that supports an isolated guest user space that all runs on the same kernel as the host OS.

Containers have a long history, and Docker is just the latest implementation. Within open source, FreeBSD pioneered containers with jails in version 4.0. This was followed by Solaris Zones and Solaris Containers in OpenSolaris 10. Container support came to Linux in several competing libraries starting with OpenVZ in 2005 and LXC in 2006. Docker was originally based on LXC, but switched to using its own libcontainer in release 0.9.

Docker was created by the company DotCloud as part of its platform-as-a-service (PaaS) infrastructure. The company open-sourced Docker in March 2013 under the Apache License, and it quickly came to eclipse the company's cloud services in popularity. In October, the company was renamed to Docker Inc., and began planning for the first DockerCon.

The advantages of containers over VMs are that they require fewer system resources, start up much faster, and are smaller and easier to deploy. For example, on my laptop a VirtualBox VM running PostgreSQL on Ubuntu uses about 2GB of disk space and takes a couple minutes to start up, whereas a Docker container uses around 200MB of space and starts up in less than five seconds. This enables the Docker approach of "one application, one container", as it is reasonable to run dozens of containers on a single commodity server.

Compared to VMs, the main limitation of containers is that they run on the same kernel as the host operating system. This means that you cannot run a completely different operating system, such as Windows, on top of Linux using a container, and even the ability to run different Linux distributions is limited by kernel compatibility. Right now, this is especially restrictive since Docker recommends using Linux kernel 3.8 and higher for technical reasons. It will seem less restrictive as Red Hat Enterprise Linux 7.0, which is based on the 3.10 kernel, becomes more widely deployed.

What Docker adds to containers is a suite of integrated management tools. First there's the concept of "images", which are stripped-down sets of operating system files that supply the foundation of a container. There are a number of "base images" for various Linux distributions, including Ubuntu, CentOS, and Amazon Linux. Users then make their own changes to the OS environment and save new images. These sets of changes are known as "layers" and are implemented either via a union file system, such as aufs or, more commonly these days, using Btrfs snapshots.

The second major thing Docker does for you is allow you to customize these containers through the "Dockerfile", which is a configuration file that runs commands on the container and launches the service that will be the "main service" for the container. For example, if you have a container that is your Apache HTTPD container, that's the main service, and when HTTPD shuts down, so does the container. This makes it easier to use containers as part of automated testing and deployment, and is a great deal like the virtual machine management offered by Vagrant. Docker also helps create and manage virtual networking, file sharing, and system resource allocations for the containers.

Docker 1.0 and DockerHub

Docker CEO Ben Golub kicked off DockerCon with a keynote announcing two things: the release of Docker 1.0, and the launch of Docker Hub. Golub, who previously was CEO of Gluster, joined Docker in July 2013, after the first open-source release. He spent some time talking about the momentum and accomplishments of Docker Inc. and the Docker community, including the success of its open source community-building effort. Thanks to the Apache license and an open contribution policy, he said, it has received contributions from over 450 people, including nearly 400 outside contributors.

Then he unveiled Docker 1.0, which was released the day before DockerCon. For those who have been following Docker development, there were no big surprises; 1.0 was identical to the release candidate 0.12.0. For those who haven't touched Docker in a while, though, there are a bunch of changes, some of them fairly fundamental, which have come about in the last three months since Docker 0.8.

First, Docker is no longer based on LXC, and as of version 1.0 works with multiple container libraries. This means it's possible to run Docker instances using libcontainer, LXC, OpenVZ, and various virtualization tools. It is also theoretically compatible with Solaris Zones and FreeBSD jails, although there were no demonstrations of that. Not all Docker features will work with all container types, and it seems likely that only libcontainer will support everything. The developers have also made the filesystems pluggable, supporting Btrfs, aufs, and device-mapper for storage, with plans to support other filesystems, such as XFS, in the near future.

Docker 1.0 adds the ability to pause and resume containers to save CPU cycles. It has improved security and made Docker compatible with SELinux and AppArmor for high-security environments. There were also a bunch of minor improvements to Dockerfiles and Docker commands to fix longstanding issues. Boot2docker, a lightweight VM that allows Mac and Windows users to use Docker, has also been brought up to 1.0 status. Finally, libcontainer has become its own, standalone project.

More importantly, with the 1.0 release, the Docker project is declaring this a stable version of the software, and Docker Inc. will be offering long-term support for it. The project is promising a stable API with a commitment to backward compatibility for the future. Accordingly, Docker has requested and received its official port numbers from the Internet Assigned Numbers Authority (IANA) for HTTP and HTTPS API traffic: 2375 and 2376, respectively.

The other big new thing is Docker Hub, which is a centralized repository for container images. Users can upload and download images of OS and application containers from it, and downloading from Docker Hub is the default option for new container deployments in the Docker API. The images stored at Docker Hub include users' personal application images and "Official Repositories", which are vetted and curated images managed by Docker staff and trusted outsiders. These include both base-level OS images, like "Ubuntu", and application images, such as "WordPress".

Docker Hub and the namespace for images are organized like GitHub, except for the official images. For example, the official PostgreSQL image is at "postgres", and if I create and release my own version, it will be at "jberkus/postgres". Also like GitHub, public image repositories will be free, but private ones will require a paid account.

Google's Docker tools

During DockerCon, multiple companies, including Amazon, Red Hat, Rackspace, and IBM, announced and demonstrated various products and tools designed to work with Docker. To me, the most interesting of these talks was the keynote given by Google's Eric Brewer on the second morning of DockerCon. It was engaging because not only is Google using Docker, it is releasing a whole bunch of internal tools for Docker as open source.

According to Brewer, Google has been using containers for a while, which are based on an internal management tool set that was never open sourced. Containers are "application-centric", which is how Google does things, and are suitable for large-scale application framework load-balancing. "Google deploys over two billion containers per week," said Brewer. "We run containers inside VMs on top of containers."

So when Docker came along, Google decided to embrace it and portions of the company's infrastructure have been converted to using Docker. And, starting recently, Google has been contributing to it.

In October 2013, Google released its own container system that was originally in competition with Docker. It bears the cute name of "LMCTFY", which stands for "Let Me Contain That For You", after a well-known web site. LMCTFY offers resource-managed containers, which use control groups to limit CPU, memory, and I/O usage by each container so that more containers can share a single machine. LMCTFY also supports nested containers, which enables grouping containers.

Google plans to take this resource management code and move it into Docker to give it the same capabilities. Its first effort, released for the hackathon on the day before DockerCon, is cadvisor, which is a tool that reports container resource usage on the host system.

Like the Docker project, Google also endorses the mantra of "one service, one container". However, this means that you have a lot of closely related containers which need to be deployed, then start and stop together. For example, you might have a container which has a web application server, a second container which has mapped file storage, and a third which has a logger service. Google uses nested containers to group these containers into "pods", which are deployed as a unit and are intended to share a single IP address.

To support this architecture, Google has released the Kubernetes project, which is an "orchestration" system for groups of containers organized into pods. These pod configurations are controlled through a configuration file in JSON that also supports the idea of load-balanced groups of pods. Pods communicate through ports assigned to each service at declaration time. All of this is designed to enable the rapid provisioning of large groups of servers based on a declarative configuration.

Brewer said that Google plans to open source more internal container-management tools in the future to make Docker the "open standard" for containerization on the web. All of these projects will be under the Google Cloud Platform group of tools.

More DockerCon

Of course, there were many other presentations and demos during the conference. Amazon announced that it was changing its application deployment service, Elastic Beanstalk, to be based on Docker in the future. Speakers from Chef, SaltStack, and Puppet each showed off using their management tools to deploy containers. Red Hat talked about its new Project Atomic, a lightweight version of RHEL designed to both run, and be run on, containers.

One of the major areas of technical competition among the various companies at DockerCon was "orchestration", which means tools to manage large numbers of containers on many physical hosts. In addition to Google's Kubernetes project, there were talks and demos of Red Hat's GearD and Apache Mesos. Docker Inc. is also working on a new orchestration tool called libswarm.

Overall, DockerCon was impressive in the amount of enthusiasm, adoption, and technology arrayed around a project which is only fifteen months old. In a little over a year, it has acquired a full ecosystem of dependent projects and competing corporate contributors of which any open source project would be proud. I could not help but come away from the conference convinced that I'll see a lot more of Docker in the future. In fact, I'm already working on improvements to the official PostgreSQL image.

Comments (10 posted)

Karen Sandler on what we mean by "we"

Identity can be a nebulous issue for the free and open-source software (FOSS) community, perhaps in part because of how different FOSS is from other communities. Karen Sandler of the Software Freedom Conservancy (SFC) explored that topic from several angles during her keynote talk, "Identity crisis: are we who we say we are?" at the fifth annual Texas Linux Fest in Austin. In particular, the FOSS community often speaks of itself as a monolithic "we," but defining who "we" means is a tricky task in many FOSS contexts, she said. There are blurry boundaries, multiple roles, and overlapping objectives that permeate many FOSS projects, and the language used can exacerbate real-world problems.

Sandler started her talk by noting that she currently has many roles herself. In addition to her role as Executive Director of SFC, she was recently elected to the Board of the GNOME Foundation (where she previously served as Executive Director), is a practicing attorney in New York, and is associated with several other organizations, such as the Software Freedom Law Center (SFLC) and QuestionCopyright.Org. But in spite of her extensive experience with FOSS, she said, it is really only in the past year or so that she has really felt like she has gotten a handle on the complicated issues of identity and representation.

In March, she taught a seminar on legal ethics in free software on behalf of the Free Software Foundation. The seminar was a professional "continuing education" course (which practicing attorneys are required to complete a certain number of in order to maintain their certification), and although she initially worried that the time she spent preparing the material would prove boring, it turned out to be fascinating, since it highlighted just how differently FOSS does things from the rest of the world. To begin with, there are blurry lines everywhere in FOSS: between what is personal and what is professional, between volunteers and paid contributors, between non-profit organizations and for-profit companies, and even between the ideological and commercial goals that motivate the work.

Still, she said, "we say 'we' a lot," and figuring out who "we" is in any one instance can be difficult. As a lawyer, she continued, she has to think about the question in strict terms, since attorneys have definite legal obligations to their clients and rules they must abide by. For instance, she noted that strangers often come up to her at FOSS events and start to share "juicy gossip" about projects and companies. She stops them and asks whether they should be telling their story, and often hears a reply like "it's okay: you're my lawyer." In fact, Sandler said, she does not represent everyone in the FOSS world, and for those clients she does represent, she has obligations to protect their interests, which may include giving the client information that she learned from someone else.

But figuring out who you represent can be confusing in FOSS, she said. Other fields are all about keeping things secret, but FOSS wants to work in the open. Even within a project, the lines between client and outsider can be fuzzy; Sandler said she once had a conversation with a Red Hat employee about a legal question relating to the GNOME project, and had to tell the person she was required to take the question to Red Hat's legal team instead. Ethics rules dictate that an attorney speak only to another party's attorney (and not to the person involved) once they know the other party has representation.

Organizations, communities, and friends

Of course, many organizations exist in the FOSS universe, which can help to draw clearer lines about who is and is not "we" in a given context. But the various legal forms these organizations take affects matters deeply. Some organizations, for instance, are 501(c)(3) charities acting "for the public good," while others are 501(c)(6) trade associations, which act in the interests of the members to promote a business goal. Each type of organization is appropriate in some circumstances, yet in FOSS their goals can seem to align (such as promoting free software adoption in businesses) while they remain quite different from a legal standpoint.

The differences between the various types of non-profit organization are most certainly important to the US Internal Revenue Service (IRS), she explained. FOSS projects' rhetoric of "changing the world" may be genuine idealism, she said, but it sounds virtually identical to every for-profit tech company's advertising, too. A few years back, the IRS started taking a hard look at the various FOSS non-profit filings, apparently out of concern over whether they were genuinely doing their work for "the public good." Naturally, the agency found the question confusing; it tried to find clear lines—such as saying that a project using a copyleft license was a "public good" project, while one using a permissive license was interested in proprietarization and, thus, a trade association. But such simple rules do not encompass the wide range of ideas about licensing, Sandler said; one cannot blame the IRS for being confused, and many of the FOSS non-profit applications take a long time to process as a result.

In addition to an organization's purpose, what constitutes "we" also concerns how and why individuals choose to participate in FOSS. When Canonical rolled out its Unity desktop interface in 2011, Sandler was Executive Director of the GNOME Foundation. Since Unity was an alternative to the recently-released GNOME Shell, it was already a move of considerable interest to her in her role at GNOME, but Sandler said she also found it surprising that so many in the Ubuntu community did not object to contributing their effort to a for-profit company's project. So she went to the next Ubuntu Developer Summit (UDS) hoping to get a feel for the community's stance. Over the course of UDS, she said, she had many conversations with her assigned roommate (a motivated Ubuntu volunteer), in particular asking the roommate about her motivations for contributing. Fundamentally, Sandler said, the roommate's answer was "because my friends are part of this community."

Being with one's friends is a major motivator for participating, but it is also something that makes FOSS distinct. Sandler noted the well-publicized incident at a recent PyCon where two people joking with each other were overheard by someone else, and a contentious conference-harassment incident resulted. FOSS makes the line between personal and professional blurry, Sandler observed. Many of us like to go see our friends at FOSS events, she said; we even invite FOSS friends to personal events like birthday parties, and we work from "home offices." In addition, we also "play musical chairs" a lot, moving to different employers even while continuing to work on the same project.

We are even conflicted at times about who we are as individuals. She quoted ownCloud's Frank Karlischek, who said that in his startup sometimes he is an "evil capitalist" and sometimes an "ideological free software guy." Within the FOSS community, it can even be hard to tell which role someone is in from minute to minute. There are also developers who get paid by a company to work on code then work on the same code at night. The confusion this causes was pointed out in the Debian project's recent init system debate, when some people said that it was not clear when other people were expressing their personal views and when they were expressing their company's views. On the other hand, asking what "hat" another party was wearing for a particular comment was also seen as a type of attack.

Governance

Coping with these sorts of uncertainties is one of the reasons that FOSS projects have governance structures, Sandler said. Providing projects with assistance is why SFC was founded; it handles logistical duties (including fiscal oversight, conference travel, and even paid development contracts) for its 30-plus member projects, she said, but it also lets those projects make a statement about identity issues. Joining SFC allows a project to be clear that it is a charitable effort not controlled by a company, to commit its assets (financial and intellectual) to the social good, and to have a clearly defined "we" by establishing project governance and membership policies.

In addition to SFC's other operations—which Sandler described—the organization can help FOSS projects deal with some of the trickier identity questions. An example is trademarks, which she described as "a lot more important than you think they are." A common problem facing FOSS projects is that they are often started by a small, excited group of volunteers—who trust each other. Somewhere down the road as work progresses, the group decides it needs a trademark, which one person then registers as an individual. But sometimes those individuals subsequently form companies to do paid consulting work related to the project, and at that point the same individual owns the trademark and a company that conducts business related to it, which complicates matters for the community.

SFC is working to improve the services it offers, Sandler said, by publishing transparent annual reports to "show where the money goes," and by developing policies in the open, including keeping them in a public Git repository. She listed several other problems that still need to be addressed, such as policies about the default positions people speak from during public discussions (like Debian's init system debate), and better ways to handle the use of email addresses and aliases (which an audience member asked about in the Q&A period after the talk, concerned about when he could use his company email address for FOSS project work). She told attendees to "watch this space" for upcoming announcements about new efforts the SFC will be launching in the near term (while apologizing for making "one of those annoying 'pre-announcements'").

In the end, Sandler told the audience that she wants the FOSS community to "worry and not worry" about the complicated identity questions that surround it. People should worry, in the sense that they should think about the questions and try to get things right. But they should not worry, in the sense that they should know that the community is passionate about what it does and everyone else is trying to get things right, too.

Comments (2 posted)