There are a number of release management styles to be observed in the free
software community, but many of them fall within the two broad categories
of "time-based" or "feature-based". A time-based project fixes a date for
its next release, then adjusts the set of features included in that release
to make the intended date at least somewhat plausible. Feature-based
projects, instead, pick a set of desired features and hold the actual
release "until it's ready." Arguably, the trend over the last decade has
been in the direction of time-based releases. Perl 5 is a relatively
recent convert to the time-based model; the project is now faced with a
decision between making a release with known problems - including possible
security issues - or delaying its 5.16 release.
Contemporary Perl's release schedule is approximately one major release per
year. The 5.14 release came out on May 14, 2011, so it is not too
soon to be thinking about 5.16. That release is indeed stabilizing with a
number of new features. A key aspect of this release is a familiar story:
as with most other languages, Perl developers are trying to improve their
support for Unicode in all situations. Many developers have participated
in this work, but Tom Christiansen has arguably been the most visible. His
recent work includes the preparation of an extensive Perl
Unicode cookbook demonstrating Perl's Unicode-related features which,
he has suggested, are now second to none.
As this cookbook was being discussed, Karl Williamson pointed out that some of the examples where
Perl is put into full-time UTF8 mode (a very useful mode for contemporary
programs) are unsafe because the language does not properly handle
malformed strings. At best, such problems could lead to the corrupted
strings seen in so many settings where character encodings are not properly
handled. But, as Christian Hansen suggested, things can be worse than that:
I would love for this to happen, I have advocated this on #p5p
several times, but there is always the battle of "backwards
compatibility disease". About 10 months ago I reported a security
issue reading the relaxed UTF-8 implementation (still undisclosed
and still exploitable) on the perl security mailing list.
What followed was a classic missive from
Tom trying to get to the bottom of the problem. He listed nine
different ways to tell Perl to operate with UTF8 and asked how many of them
were truly vulnerable to undetected encoding problems. If Perl's UTF8
handling is unsafe by default, he said, it needs to be fixed:
If there's something so important that it must be done everytime to
ensure correct behavior, then that is too important to be left up
to the programmer to forget to do. It needs to be done for him.
And, he said, this situation really needs to be treated as a blocker for
the 5.16 release; to do otherwise would be to delay the fix excessively and
cause the world to be filled with bad workaround code.
In many projects, the prospect of open security-related problems would at
least cause people to think about delaying a release. On the Perl list,
though, Tom found little support. Aristotle Pagaltzis responded:
5.16 *must* be released whatever the state of this issue. To not do
so is to fall into the thinking and behavioural pattern that
stifled the release of 5.10 by several years. Perl 5 switched to a
timeboxed release cycle because “this one more thing has to be
polished before we can ship it” meant it never shipped at all.
Ricardo Signes added:
It's definitely true, though, that 5.even.0 releases *are no longer
milestones.* Or, rather, they are milestones in a much more literal
sense than is often meant. 5.16.0 means that we've come about one
year since 5.15.0. It does *not* mean that we have met a series of
goals named at 5.15.0, for example.
And, with those words, the public email discussion faded away. At this
point, there is little clarity on which Unicode features are safe to use,
what might be required to fix the rest, how many of those fixes might be
ready in time for the 5.16 release, or whether programs using UTF8 in Perl
5.16 (and earlier releases) suffer from known-exploitable security
problems. Tom, who probably understands these issues better than just
about anybody else, said:
Right now I'm very hazy on the real status of all this stuff, and I
am very uncomfortable with the idea of relentlessly charging ahead
toward a release like a freight train with no brakes.
The response he got claimed another train
would be coming along in a year and the fixes could catch a ride on that
Releasing software with known bugs is a common practice; even a project
like Debian cannot make the claim that there are no known problems with its
releases. To do otherwise would make software releases into rare events
indeed. It is also true that time-based releases have value; users know
that they will get useful new code in a bounded period of time. Anybody
who has watched release dates slip indefinitely knows how frustrating that
can be for both users and developers; the 2.4.0 or 2.6.0 kernel releases
are a classic example - as is Perl 5.10.
So the Perl developers who say that the show must go on are doing so in
accordance with many years of free software development experience.
That said, releasing software with known, security-related issues in
something as fundamental as UTF8 support risks tarnishing the image of the
project for a long time. There is not enough information publicly
available to say whether Perl's UTF8 problems are severe enough to incur
this risk. But it is probably safe to assume that, as a result of this
conversation, crackers are looking at Perl's UTF8 handling rather more
closely than they were before. Perl may have had some of these problems
for years without massive ill effect, but they may not remain undiscovered
and undisclosed for much longer. If the problem is real and
exploitable, people are going to figure it out and take advantage of it.
One would assume that, behind the public positioning, the relevant
developers understand what's at stake and are taking the time to understand
the scope of the problem.
They may not want to stop the release train, but seeing it derailed by a
known security problem after release would not be much fun either. A
release delay now may prove less painful than a security-update fire drill
Comments (24 posted)
a deal with
Latin American mobile carrier Telefónica Digital on February 27
to start shipping HTML5-driven smartphones by the end of 2012. Although
the press release dubs the new platform "Open Web Device," many Mozilla
watchers know it better as Boot To
Gecko (B2G), a lightweight Linux-based system that uses Gecko as its
primary (if not sole) application framework. B2G has been in development
since July 2011, but it has advanced significantly since we last examined it in mid-August.
Initially, the project debated several options for the underlying operating system on which to perch the Gecko runtime — including Android (which served as the basis for the earliest experimental builds), MeeGo, and webOS. The team has since settled on an architecture of its own, which takes some components from Android, but constructs its own stack, called Gonk.
Gonk is a stripped-down Linux distribution, using the kernel and some standard userspace libraries for hardware access (e.g., libusb and Bluez). It also borrows hardware abstraction layer code from Android — the architecture document on the Mozilla wiki lists camera and GPS devices in particular. The architecture document also makes references to both the Android kernel and to modified kernels supplied by hardware vendors; evidently B2G is designed to tolerate minor variations.
Gonk also includes an Android-derived init program, D-Bus,
wpa_supplicant, and Network Manager. The media playback functionality also
comes from Android: there is a mediaserver player for the audio
and video formats that Gecko can decode natively, and
libstagefright to access binary-only codecs and hardware decoding
Mozilla's B2G team could not be reached for comment,
but several of the Android components in the Gonk stack appear to have been
chosen for the practical task of bootstrapping the OS on available
smartphone hardware (such as the Samsung Galaxy S II, which is the testbed
device at Mozilla) rather than on technical grounds. For instance, the
setup currently uses Android's Bionic library, but there has been some discussion of
replacing it with a more general libc implementation.
Apart from those components and the cellular modem layer, however, Gecko
is the only runtime process started. In B2G, there is a master "Gecko
server" process named b2g that spawns lower-privileged child
processes for each web application run. Only the b2g process can
directly access hardware devices; it mediates access requested by the child
processes. b2g and the application processes can pass messages to
each other using what Mozilla calls "IPC (Interprocess communication)
Protocol Definition Language" or IPDL, which originates from the Electrolysis project. Electrolysis is Mozilla's effort to refactor Firefox as a multi-process application; it is still in development, but it provides both synchronous and asynchronous communication, and isolates child processes from each other.
Ultimately, Gonk is different enough from both Android and typical desktop Linux systems to warrant its own build target for Gecko. Among other distinctions, it implements a lightweight input event system, and draws directly to OpenGL ES rather than the X Window system. But in order to provide the full smartphone application framework, the Gecko engine must also have direct access to low-level functionality that is normally handled by other processes — such as the telephony stack.
B2G's telephony stack is called the Radio Interface Layer (RIL), and is derived from the corresponding code in Android. RIL caters to device vendors by providing a rilproxy mechanism to manage connections between the b2g process and the cellular modem driver. The driver, called rild, can be proprietary (in fact, the RIL design assumes it is proprietary) and thus link to binary-only firmware blobs without triggering the license inheritance conditions from other parts of the stack.
RIL implements voice and 3G data calls, network status, text and MMS
messaging, and SIM card commands (such as storing and retrieving contacts,
In fact, much of the B2G-specific functionality requires the definition
of new APIs that "traditional" Gecko applications do not expect. Mozilla
has already invested in the drafting of OS-independent web APIs for other
reasons, including its Web
App Store project. Mozilla is also participating in the
standardization effort being managed by the W3C, primarily the Device APIs Working Group. At the
moment, not every API created by Mozilla has a corresponding W3C
specification, although Mozilla says standardizing all of them is the
ultimate goal. Mozilla's list includes a mixture of
high-level interfaces (such as the Contacts or Camera APIs) and low-level
interfaces (such as the Bluetooth and near field communication (NFC) stack
The most far-reaching APIs in the library are Open Web Apps, which specifies a manifest file format and packaging guidelines for "installable" web applications, and WebRTC, which provides a real-time communication framework used for streaming media as well as network-centric features like peer-to-peer networking. Some of the less familiar specifications under development are APIs for detecting screen orientation, processing notifications when the device is idle, and locking hardware resources (such as preventing screen-dimming).
Applications and interfaces
The demo application suite is code-named Gaia. The wiki page has
placeholders for a dozen or so basic applications, including more detailed
mock-ups for the web
gallery, and camera
application. There are also mock-up images on the Demo page for a home screen,
lock screen, and dialer.
More interesting than static mock-ups are the in-development demos found at B2G developer
Andreas Gal's GitHub site (screen shots of some are shown here). A recent version of Firefox will open the HTML
files from the repository; the homescreen.html file shows the home
screen and lock screen, and several other applications are accessible
through home screen launchers (the Mozilla wiki's Demo page also links to a
"live" demo, but that appears to be out-of-order at the moment). The UI
responds to mouse clicks and "slide" gestures, although the applications
vary widely in completion, of course — some are just static images.
Still, the home screen and generic applications are in place along with basic navigation and UI elements.
The road ahead
The B2G roadmap is ambitious; it predicts a product-ready milestone by the end of the second quarter of 2012. Considering that not all of the fourth quarter 2011 goals have been met yet, that may not be attainable without an infusion of developer time from a hardware partner, but the project did bring a B2G demonstration to Mobile World Congress alongside the February 27 announcement.
Perhaps notably, the Bluetooth, NFC, and USB tracking bugs appear to be behind schedule at the moment, while most of the higher-level work for sensor support and user applications appear to be on track. Arguably the biggest piece of the puzzle yet to get underway is WebRTC support. WebRTC itself is still undergoing rapid development, of course, but without it, the goal of supporting real-time audio and video on a web-runtime smartphone is difficult to imagine.
Resources for potential Open Web Device application developers are also
in short supply at the moment. Although Mozilla curates a collection of
HTML5 resources at its Mozilla Developer Network (MDN) site, so far only
the Open WebApps APIs are covered. Mozilla formally
opened its Open Web App "Marketplace" on February 28, in an
announcement that referenced the in-progress Web API effort in addition to
the traditional desktop platform. That is probably a sign that the Web
APIs will make their way to MDN in short order as well.
Comments (82 posted)
"/usr unification" (or simply "usrmove") is the idea of moving the contents
, and related root-level directories into
their equivalents under /usr
. That this scheme is controversial
can be seen from that fact that, when LWN pointed
to a document describing the
motivations behind the move, a thread of well over 250 comments resulted.
One would think, from the volume of comments on LWN and elsewhere, that
this change is a threat to the very nature of Linux. Either that, or it
has inspired one of the biggest bikeshedding exercises in some time.
One of the "advantages" of running a Rawhide system is that one gets to try
out the new toys ahead of the crowd. Said toys also have a certain
tendency to arrive at random and inopportune times. In the case of the
/usr move, most Rawhide users got little or no warning before updates
simply failed to work. The Rawhide repository, it seems, had suddenly been
populated with packages requiring a system where the /usr move had
already been done. Actually causing this move to happen was left as an
exercise for the system administrator.
As it happens, this is one of the more controversial aspects of the
/usr move in the Fedora community. Fedora policy has always said
that using the yum tool to update an installation to a newer
release of the distribution is not supported; one is supposed to use a CD
or the preupgrade tool for that purpose. But, in reality, just
using yum tends to work; the prospect of it not working to get to
Fedora 17 has caused some grumpiness
in the Fedora user community. If this limitation remains in the
Fedora 17 release, expect to hear some complaints; users don't like to
have their ponies taken away, even if nobody had ever said they could have
a pony in the first place.
For Rawhide users, it is possible to move past the usrmove flag day by
set of instructions. The process involves editing the grub
configuration files, installing a special initramfs image, then rebooting
with a special option that causes the actual thrashing of the system to be
done from the initramfs before the real root is mounted. Your editor,
setting out along this path with a well backed-up system, was reminded of
doing the manual transition from the a.out to the ELF executable format on
a Slackware system many years ago. In both cases, one had the sense of
toying with fundamental forces with no guarantee of a non-catastrophic
In both cases, as it happens, things worked out just fine. Directories
like /bin, /lib, /lib64, and /sbin are
now symbolic links into /usr, and the system works just like it
always did. No programs or scripts needed to be changed, custom kernels
work as they always did, and so on.
That will almost certainly be most users' experience of this
transition - they will not even notice that it has happened. This is not a
particularly surprising result; the practice of moving files and leaving a
symbolic link in their place has a long history. Of course, somebody has
probably patented it anyway.
Of course, an equally successful result for all systems depends on the
Fedora developers creating a bombproof
automatic mechanism for carrying out this transition prior to the
Fedora 17 release. Hopefully something will have been learned from
the GRUB 2 mess, where the Fedora 16 upgrade left a lot of bricked
systems in its wake. If Fedora 17 creates similar messes on systems
that, perhaps, are not set up quite as the Fedora developers expect, a lot
of unhappiness will ensue.
As an aside, it is interesting to compare how Fedora is managing this
transition with Debian's multiarch transition. Fedora has forced
a flag day requiring the entire thing to happen at once; Debian, instead,
has carefully avoided flag days in favor of a carefully-planned
step-by-step change. One could argue that the Debian approach is better:
it lets the transition "just happen" through normal package updates without
the need for any special actions on the part of administrators or users.
On the other hand, the multiarch transition has been a multi-year process
and is still not complete; Fedora, instead, has pushed this change through
in a small number of months.
Now that Fedora has made this jump and seems to be past the turning-back
point, will other distributions follow suit? Some investigation suggests
that Fedora will not be alone in this change:
- There have been no official statements from Red Hat, naturally, but
it seems likely that, given how much effort has been put into this
change by Red Hat employees, RHEL7 will feature the unified directory
The RHEL clones, including Oracle's distribution and CentOS, will have
little choice but to incorporate this change.
- OpenSUSE has been discussing the idea for a while and seems to be
receptive to it. There does not appear to be an official statement
that openSUSE will make the /usr move, but there is a web page set up
to track progress in that direction.
- Debian has had a number of long discussion threads about usrmove; it
will not surprise longtime Debian watchers that there is a variety of
opinions on the issue. There is some concern about possibly breaking
systems with strange setups, though the most often cited case -
systems with /usr on a separate partition - has been thought
about and is relatively easy to support. In the end, it may come down
to making the change to keep life easier; as Marco d'Itri put it: "I am not really looking
forward to keep reverting these changes in my package, and since Red
Hat controls most Linux infrastructure now other packages will face
the same problem."
Of course, Debian has some time to think about the problem. Nobody
has suggested that usrmove could be added to the list of goals for the
upcoming "wheezy" release, so any transition, even in the unstable
tree, is likely to be at least a year away.
- Ubuntu has been relatively quiet about the idea; what has been posted
by Ubuntu developers suggests a lack of enthusiasm for the idea. But
if Debian makes the change, it could be hard for Ubuntu to retain the
current filesystem layout.
In short, it looks like, over time, the move toward keeping all binaries
and libraries in /usr will spread through most Linux
Given that most users will likely not even notice the change, it is natural
to wonder why the change is so controversial. Undoubtedly, some people
dislike significant changes to how traditional Linux systems have been laid
out. Possibly, some creatively-organized systems will not handle the
transition well, leading to problems that must be fixed. Conceivably, the
change strikes some people as useless churn with no real benefits - at
least for them. And, almost certainly, it is a bikeshed-type issue that is
easy to have an opinion on and complain about.
The truth of the matter is that it appears to be a reasonably
well-thought-out change driven by some plausible rationales. Filesystem
hierarchies are not set in stone; there are times when it makes sense to
clean things up. The usrmove operation yields a filesystem that is easier
to understand, easier to manage, and more consistent in its organization.
Having been through the transition, your editor thinks it is probably worth
Comments (131 posted)
Page editor: Jonathan Corbet
Inside this week's LWN.net Weekly Edition
- Security: Fedora's Network Zones; New vulnerabilities in csound, glibc, kernel, samba, ...
- Kernel: Fixing control groups; The Android mainlining interest group meeting; A long-term support initiative update.
- Distributions: Chakra: An Arch Linux fork that is rough but promising; Fedora, MINIX, Stevland, Tizen, ...
- Development: Speeding up D-Bus; Netmap, nntpgit, PostgreSQL, ...
- Announcements: GStreamer partnership, TDF, HTTPS Everywhere, Free Android, Boot to Gecko, Mifos project, ...