LWN.net Weekly Edition for December 5, 2013

Another daemon for managing control groups

Control groups (cgroups) in the kernel are changing, in both their implementation and their interface. One of those changes is that systems that use the new cgroup interface will require a single management process to coordinate access. For many, that management daemon may well be the systemd init replacement, but there are distributions (notably Ubuntu) and users who will want a different choice. To that end, Serge E. Hallyn is working on an alternative cgroup management daemon that he calls "cgmanager".

Cgroups are a way to group processes and to apply various resource limits to the group. The current cgroup API in the kernel is a filesystem-based interface called cgroupfs. Groups are represented as directories and processes are placed into them by writing their process ID to a special file. Additional special files are used to set various limits and to control other aspects of the group, which depend on the specific controllers that are associated with the group (i.e. CPU, memory, block I/O, etc.).

Depending on permissions, any process may be able use that interface to set up its own hierarchy of groups, which is one of the problems with the existing implementation, according to cgroup co-maintainer Tejun Heo and others. So, in the future, there will be a single process that is responsible for managing the single hierarchy of groups that will be allowed under the new cgroup interface. While systemd already has code to perform that job, systemd has never been a requirement to use cgroups—but if it isn't used, something has to take its place.

So Hallyn put out a preliminary design for cgmanager on the lxc-devel mailing list at the end of November. It envisions a single daemon process with a D-Bus interface that will manage the hierarchy, which will mount cgroupfs inside a private namespace that is accessible only by cgmanager. Processes will make requests of the daemon to create cgroups, configure them, and to move processes into them. Obviously, some of those operations are privileged, and Hallyn has worked out a set of privileges required for each. Users own the cgroups they create and can place their own processes into those cgroups. More complicated arrangements are possible, of course, but generally either require real root privileges or at least root inside a user namespace owned by the user.

There is also the concept of handing the ownership of a cgroup (or hierarchy) to another user. Doing that using the standard filesystem permissions on the cgroupfs, as is done today, is a big security hole, which can lead to various kinds of denial of service. Mediating that access through cgmanager should alleviate that problem, however.

So far, Hallyn's design has been well-received; there are questions and suggestions, of course, but overall it would seem that Hallyn has taken several constituencies into account.

The first of those is the LXC containers project, but there are several other interested parties as well. Upstart-using distributions (mostly limited to Ubuntu and derivatives—though possibly Debian too, depending) will also need some kind of cgroup manager if they intend to use cgroups. Hallyn has clearly been talking to the Upstart folks, so it would seem that cgmanager should eventually fit the bill there.

Another constituent is Google, which uses cgroups extensively in its fairly sizable datacenter. Tim Hockin from the search giant had been rather critical of the single hierarchy cgroup plan. He has since largely made his peace with the plan, but an article from July outlines Google's use of cgroups as well as some of the problems with the "new cgroups" that Hockin had foreseen. Clearly Hallyn was also paying attention to Google's needs as Hockin seems relatively pleased with the direction of cgmanager's design. In particular, Google uses the "change ownership" feature today, but it trusts all of the users on its cluster. For others, who may not be able to have that trust level, Hallyn has provided a means to get that functionality, while reducing its security impact.

Where does systemd fit into this picture? The answer to that seems to essentially be: not at all. Systemd has its own control group interface that it intends to proceed with. Two messages from June make it pretty clear that Lennart Poettering, at least, is not particularly interested in having systemd work with another cgroup manager or to provide cgroup management in a library. Poettering's contention that init should not be dependent on some other daemon seems sensible, but forcing anyone who wants to use cgroups to use systemd clearly isn't.

It would be nice if some kind of common API could be worked out so that applications and users don't have to support two different ways of accessing cgroups. It would seem that Hockin approached the systemd folks about working together but was evidently rebuffed. That may make sense for the systemd project, but will unfortunately leave applications and users who need cgroup functionality in something of a bind. Supporting both systemd and cgmanager will likely be required.

While it will undoubtedly be annoying to support two (or more if another competitor shows up) interfaces to cgroups, having two separate implementations may actually be a good thing—at least in the short term. That wouldn't preclude a common interface, of course, but that doesn't seem to be in the cards. Cgroups have been a problematic feature in the kernel since they were merged for 2.6.24 back in 2007. One hopes we are on our way toward a better implementation and user-space interface for the feature—so the more code that exercises all of that, the better.

Comments (237 posted)

Searching for a pump.io client

When Identi.ca switched over from the old StatusNet software to the newly minted pump.io platform earlier this year, one ripple effect was that many of the existing desktop client applications suddenly were no longer supported. Thus, users of the service lost convenient access to the updates of their friends, and were limited to the less-convenient default web interface for posting updates of their own. In the intervening months, though, clients have started to emerge, with the potential for freeing users from the web-only interface.

The root issue is that pump.io uses its own API, based on the Activity Streams data format and Atom publishing. StatusNet made a point of copying the Twitter API, in order to make things easier for third-party developers who were already interested in supporting Twitter. Of course, the degree to which any particular client made an effort to support StatusNet's enhancements (!groups, for example: topic-based discussions that users could subscribe to) varied; no doubt many developers who were primarily interested in Twitter regarded supporting other services as an afterthought.

The pump.io wiki maintains a page of known client applications. The list includes web services (some of which are utilities, like pump2rss.com), mobile device platforms, and other applications aimed at desktop users. At the moment, there are two GUI client applications with Linux support: Dianara and Pumpa. Both are written for Qt.

Dianara

Dianara has been around longer; the first public releases were in early 2012. As such, several major Linux distributions make it available through their package managers.

The early versions of Dianara supported Diaspora, the alternative social networking platform famous for its colossal crowdfunding campaign on Kickstarter. Diaspora has not yet taken off as a microblogging or messaging platform, though, so it is perhaps unsurprising that Dianara author Jan Kusanagi eventually turned his attention elsewhere. In May, Kusanagi announced that the application would target pump.io support instead.

Version 1.0 was released on October 29, with support for a significant number of pump.io features. Users can create posts with rich text formatting, attach images, and address posts to specific individuals or lists. The interface shows separate tabs for the user's "timeline" (i.e., accounts the user follows), direct messages, the user's own account stream, favorited notes, and the list of contacts the user is following. As far as interactivity is concerned, Dianara supports replying to others' notes, re-sharing notes, and marking notes as favorites. Users can also edit their profile on the pump.io service they connect to, and configure what sorts of notes trigger a desktop notification.

Dianara is pretty basic in its feature set; it limits the user to one pump.io account, and so far there are few advanced features (for example, there is no searching). It does support some pump.io operations that are less common in Twitter-centric client applications, though, such as assigning post titles, creating lists of users, and updating old posts. Post titles are interesting, because the titles appear in the activity timeline. A titled post might appear as "Nathan Willis favorited LWN Weekly Edition for December 5," while an untitled post would be seen as the enigmatic "Nathan Willis favorited a note." A small functional difference, but far more useful for users.

Pumpa

Pumpa is a much newer project—at least "on paper"; it was started in June 2013. Of course, since Dianara undertook a major rewrite in May to switch over to pump.io support, the time difference is less significant in reality. But so far, no Linux distributions package Pumpa. There are third-party repositories run by volunteers for Fedora and Debian, however, which are linked to from the project site. Everyone else will need to grab the source from author Mats Sjöberg's Gitorious repository.

The latest release is version 0.8, from November 26. Pumpa includes most of the same functionality as Dianara, including writing, addressing, replying to, and favoriting notes. But the implementation is noticeably different. The biggest change is that Dianara presents a rich-text editor for new notes; one clicks on the "F" drop-down button (which presumably stands for "Formatting") to make text bold/italic/underlined/etc or to add a hyperlink. Pumpa, in contrast, presents only a plain text editor, and interprets all of the text as Markdown syntax. Whether that is an improvement or an impediment depends on who you ask; I for one do not have Markdown memorized.

On the other hand, Pumpa's rendering of the various pump.io timelines (followed users, direct messages, etc.) is nicer—it looks like most of the StatusNet and Twitter clients, with user avatars and note text neatly rendered on a vertically-scrolling window. Dianara shows every note (including other people's notes) as a text frame, which most Qt themes render in a different color to indicate editability. Other people's notes are not editable, of course, so the result is a bunch of nested boxes that look confusing. Dianara's nested text frames also have minimum-width requirements, which spawn horizontal scrollbars if the window is too narrow, and there are a number of places where there is excessive white space between elements. In short, Pumpa's visual layout is compact and simple, while Dianara's gets in the way and makes message content harder to read.

Pumping it up

Nevertheless, both clients are simple to set up and use, which is a boon to old StatusNet users stranded by older applications like Gwibber. There is clearly room for improvement on multiple fronts—I do not know why, for example, neither program has a "new note" button on the window itself, but anyone in a desktop environment that puts application menus at the top of the screen will soon notice its absence.

It is hard to gauge whether or not pump.io's simpler design is making it easier for application developers in comparison to StatusNet. Certainly Dianara and Pumpa have come a long way quickly. There are also multiple command-line and console clients available, as well as an Emacs mode.

The more important question will be whether or not the various clients evolve to support message types beyond simple text and image posts. Sure, these two categories capture the bulk of social networking content, but Activity Streams is open to all sorts of activity messages, including user-defined ones (for example, the ih8.it site sends "hate this" messages).

Both of these desktop applications are limited right now to microblog-style usage. There is a glimmer of hope in some of the unusual Web clients emerging from pump.io, although the initial ones (including ih8.it) were developed by pump.io creator Evan Prodromou. In the long run, clients will no doubt evolve to support the message types that users want to see; now that reliable clients are appearing, perhaps those users will start to explore the possibilities.

Comments (3 posted)

2013 Linux and free software timeline - Q2

Here is LWN's sixteenth annual timeline of significant events in the Linux and free software world for the year. As per tradition, we will divide up the timeline up into quarters; this is our account of April–June 2013. January through March was covered last week; timelines for the remaining quarters of the year will appear in the coming weeks.

There are almost certainly some errors or omissions; if you find any, please send them to timeline@lwn.net.

LWN subscribers have paid for the development of this timeline, along with previous timelines and the weekly editions. If you like what you see here, or elsewhere on the site, please consider subscribing to LWN.

For those readers in a truly nostalgic mood, our timeline index page includes links to the previous timelines and other retrospective articles that date all the way back to 1998.

Version 1.6 of the MATE desktop environment is released (announcement).

PostgreSQL announces the discovery of a severe hole, which prompts the project to re-examine its security-handling process (LWN article).

Fedora throws a subtle wrench into the works when it selects "Schrödinger's Cat" as the codename for Fedora 19, featuring an accented character and a punctuation mark (LWN article).

Yorba launches a crowdfunding campaign to underwrite work on the Geary email client (LWN blurb). The campaign falls short of the goal in May, prompting analysis by Yorba director Jim Nelson (LWN blurb).

Twisted 13.0.0 is released (announcement).

Google halts its participation in the WebKit web rendering engine project to develop its own fork, Blink (LWN blurb).

Google announces its Open Patent Non-Assertion Pledge (announcement).

The Subsurface project mourns the loss of Jan Schubert (LWN blurb).

Mozilla turns 15 (LWN blurb).

The Free Software Legal and Licensing Workshop is held in Amsterdam, April 4 to 5 (LWN coverage).

The OpenDaylight project is announced, starting a community-driven approach to developing Software-Defined Networking (announcement).

Libre Graphics Meeting 2013 is held in Madrid, April 10 to 13 (LWN coverage).

Lucas Nussbaum is elected Debian Project Leader (announcement).

OpenMandriva selects a logo (announcement), but does not decide to officially adopt the name "OpenMandriva" until May (announcement).

Wayland/Weston 1.1 is released (announcement).

Xen becomes a Linux Foundation project (announcement).

The 2013 Collaboration Summit is held in San Francisco, April 15 to 17 (LWN coverage).

The Storage, Filesystem, and Memory Management Summit is held in San Francisco, April 18 to 19 (LWN coverage).

The Google Test Automation Conference is held in New York City, April 23 and 24 (LWN article).

Kernel 3.9 is released (LWN announcement; development statistics; merge window summaries 1, 2, 3; KernelNewbies summary).

The Linux Foundation joins GNOME's Outreach Program for Women by supporting kernel internships (LWN blurb).

OpenBSD 5.3 is released (announcement).

Ubuntu 13.04 is released (announcement).

GDB 7.6 is released (announcement).

Open Build Service (OBS) 2.4 is released (announcement).

Debian 7.0 "Wheezy" is released (announcement).

Working with Google, Adobe donates its OpenType CFF renderer code to FreeType. (announcement).

The Open Source Initiative announces a major shift in its governance process and board, part of a planned transition toward being a member-driven organization. (announcement).

The World Wide Web Consortium (W3C) publishes its draft proposal for Encrypted Media Extensions, a DRM-support framework whose creation is driven by large media companies (LWN article). Among other voices, Richard Stallman criticizes the proposal as being antithetical to the stated goals of the W3C (LWN blurb).

PyPy 2.0 is released (announcement; article)

PacketFence 4.0 is released (announcement).

Version 1.1 of the Go language is released (announcement).

Google releases a draft of the patent license for VP8 that originated from the company's agreement with MPEG-LA (announcement). The Software Freedom Law Center (SFLC) says the agreement is compatible with FOSS licensing (LWN blurb).

Moodle 2.5 is released (LWN article).

The New Yorker unveils Strongbox, a secure and anonymous service for facilitating communication between whistleblowers and journalists. Strongbox and its underlying platform DeadDrop was the last major project developed by Aaron Swartz (LWN announcement; article).

The first version of kernel tracing tool ktap is released (announcement; LWN article).

Debian GNU/Hurd 2013 is released (LWN announcement; article).

Mageia 3 is released, dedicated to the memory of developer Eugeni Dodonov ( announcement).

NetBSD 6.1 is released (announcement).

QEMU 1.5.0 is released (announcement).

Groklaw celebrates 10 years (blurb).

Google Code shuts down the ability of projects to offer file downloads (LWN blurb).

pgCon 2013 is held in Ottawa, May 21 to 24 (LWN article).

The Tizen Developer Conference is held in San Francisco, May 22 to 24 (LWN coverage).

The spring Automotive Linux Summit is held in Tokyo, May 27 to 28 (LWN coverage).

LinuxCon Japan is held in Tokyo, May 29 to 31 (LWN coverage).

A whole bunch of longstanding X Window security vulnerabilities is revealed (LWN blurb; vulnerability database entry; article).

Ubuntu closes bug #1, with Mark Shuttleworth noting that the impact of Android has shifted the focus of the computing industry away from Windows (LWN blurb).

Debian mourns the loss of Ray Dassen (LWN blurb).

Texas Linux Fest 2013 is held in Austin, May 31 to June 1. It's hot (LWN coverage).

PulseAudio 4.0 is released (announcement).

Open Source leader Atul Chitnis passes away (LWN blurb).

FreeBSD 8.4 is released (LWN announcement).

Subversion 1.8.0 is released (LWN announcement).

LLVM 3.3 is released, introducing full C++'11 support (LWN announcement).

Groklaw reports that the SCO v IBM case has been reopened (LWN blurb).

The Xen4CentOS project is announced, helping CentOS users run Xen after upstream distribution Red Hat drops its own Xen support (LWN announcement).

Elections are held for the Fedora Advisory Board, Engineering Steering Committee, and Ambassadors Steering Committee (LWN announcement).

Xiph.org announces that it has begun work on the Daala video codec (announcement).

PHP 5.5.0 is released (announcement).

Firefox 22 is released, enabling WebRTC support by default (announcement).

Comments (none posted)