LWN.net Weekly Edition for January 5, 2012
ownCloud moves ahead
There are lots of cloud services out there these days—many of them free in the "beer" sense—but privacy-sensitive and free-software-oriented users have long been looking for alternatives. One of the few "free as in freedom" choices is ownCloud, which is the brainchild of KDE developer Frank Karlitschek. The project has been making steady progress since last we looked in on it back in June 2010, releasing ownCloud 2 in October, while preparing for ownCloud 3 in January. In addition, Karlitschek and former SUSE/Novell executive Markus Rex have formed a venture-capital-backed ownCloud company to foster the project while providing support and services.
The idea behind ownCloud is pretty straightforward: provide a way for users to replace closed source cloud services like Dropbox (or Ubuntu One) for file backup and sharing, Google Calendar and Contacts, picture sharing services like Picasa or Flickr, and so on, as well as providing storage for other web applications via the remoteStorage protocol from the Unhosted project. All of that functionality would be free software and user-installable so that the provider lock-in problems (as well as privacy concerns about what service providers do with the data they collect) can be avoided. Add in access from a variety of different kinds of devices (desktops, laptops, tablets, cell phones, ...) and it makes for a sweeping vision. There are many parts of that vision that still need to be filled out, but much progress has been made in ownCloud 2, with more to come in version 3.
ownCloud 2
For version 2, there are four different "applications" that come in the ownCloud web interface: Files, Music, Contacts, and Calendar. Files can be uploaded to the server from the browser, shared with other users of the ownCloud instance, or published for anyone to access via a URL. In addition, using the WebDAV protocol and the user-space davfs2 filesystem allows one to mount the ownCloud files directory locally, which allows local applications (and file managers) to access the files directly.
The Music application allows playing music files within the web application or, using the Ampache server, to stream the music to desktop players. Calendar and Contacts provide the basics of managing that kind of data in the web application, while also providing access to that data for other applications via CalDAV and CardDAV.
The server side installation of ownCloud 2 is fairly simple for anyone with root access to a Linux server. It is mostly a matter of getting the pre-requisites installed, which includes PHP, a database (MySQL, PostgreSQL, and SQLite are supported), and the appropriate PHP "connector" for the database choice. After a bit of Apache configuration, you point your browser at the site, which lands you on the installation wizard; after answering a few questions, ownCloud is up and running. WebDAV installation on the client side is also fairly straightforward (or was for Fedora 16 at least). That said, neither task is so easy that non-technical folks can be expected to handle it.
Overall, ownCloud 2 seems to work reasonably well. There are glitches and some oddities in the interface, at least in the released version. There are two other versions of the code available, including a Git repository for the latest development version. Neither of those options worked for me when I tried them in late December, but those problems are presumably solvable. The key functionality, sharing data between disparate devices, works well enough that I will be spending the time to get newer versions running in the near future—or wait for ownCloud 3, which is due at the end of January.
ownCloud 3
One of the areas lacking in ownCloud is data encryption. Connections are made to the server via HTTPS, so the network communication is encrypted, but files are stored in plaintext on the server. Encryption is listed as one of the features for version 3, but details are a little hard to come by. For users that set up their own server, encrypting and decrypting the data on the server side may be sufficient (depending on the key management) to protect against a system compromise that gives access to the stored data. That seems to be the feature planned for ownCloud 3.
But, for users that are storing their data on someone else's server (e.g. a friend or service provider), client-side encryption will be needed. That seems like a harder nut to crack because it requires code on each and every client. Without that, though, there can never be any real assurance that the data is not accessible to whoever provides the service—and possibly to attackers that compromise the server.
There are some other features listed for ownCloud 3, including a photo sharing application and a better interface for the calendar application with support for recurring events. There is also a mention of allowing external applications to integrate into the ownCloud web interface so that things like the Roundcube mail application could be added to ownCloud installations.
One area that the project could do better with is documentation. Information about ownCloud, its development, bug tracking, and so on are scattered across a number of different sites including Gitorius, Shapado, the mailing list, the ownCloud web site, and so on. It makes it a bit difficult for new users to track down the information they need—some sort of consolidation or master index would likely go a long way toward fixing that problem. The project does make it pretty easy to get a feel for what can be done with ownCloud on its live demo page.
ownCloud.com
It's also a bit hard to tell what, exactly, the plans are for the ownCloud company. It appears to have a commitment to open source and is being led by two longtime open source community members, so there is good reason to believe that the project will continue—and prosper—with some corporate backing. From the information on its web site, part of what the company is selling is the open-source nature of ownCloud, and its lack of vendor lock-in. All of that bodes well.
There are certainly some areas that need work in ownCloud, particularly in the ease of installation and configuration areas. One would guess that ownCloud.com will be putting a fair amount of effort into that, but it's not alone. The openSUSE project has also been working on ownCloud integration. From the looks of the ownCloud mailing list, there is an active community springing up around the project.
Hosting ownCloud instances for those who don't want the hassle might make a reasonable business opportunity for ownCloud.com and others. A reasonably priced per-Gigabyte subscription service could be attractive for privacy conscious users if the encryption issues can be resolved. Whether there are enough of those users to truly support that particular business model remains to be seen.
The ownCloud project shows a lot of promise. It is certainly usable today and will only get better in the near (and hopefully, long) term. It may only be interesting to a subset of internet users, since many are willing to trade their privacy and security for "free" services, but for the other folks, it will be real boon. Companies may also find that hosting their own instances of ownCloud (possibly with a support contract from ownCloud.com or someone else) makes a lot more sense than relying on Dropbox, Facebook, or even Ubuntu One. In any case, ownCloud seems to have a bright future and is well worth checking out.
Google's disappearing Android GPL compliance opportunity
All versions of the GNU General Public License require that anybody shipping GPL-licensed software in binary form make the associated source code available, either as an accompaniment to the binary distribution or, failing that, to anybody who asks for it later. Getting companies to actually comply with the GPL's requirements has always been a challenge, especially in the embedded systems world. The recent proliferation of Android-based devices has brought with it a whole new set of GPL compliance failures. Android is an interesting case, though, in that there is one company - Google - that is well positioned to require better behavior from those who ship Android.
Recently, Matthew Garrett, who has long tracked GPL compliance (or the lack
thereof) in the Android world, has taken Google to task for
failing to require better GPL behavior from Android distributors. His
article (along with the
followup on why GPL violations can make economic sense) is well worth
reading in its entirety. In short, Matthew says that Google could force
GPL compliance among Android distributors by either suing them or by
requiring GPL compliance as a condition for use of the Android trademark
(and, presumably, Google's proprietary Android applications). But, Matthew
says, "
There is probably some truth to this argument. GPL compliance is not hard,
but it is not free; skipping it makes Android-based devices a little
cheaper to make and, thus, more competitive in the market. But there is
almost certainly more to it than that - a fact that does not make this
behavior any wiser in the long run.
One could start with the understanding that Google, and the Android part of
Google in particular, suffers from a fairly strong case of GPL aversion.
Considerable effort went into the creation of an almost GPL-free Android
system; this included creating a new C library and a new Java virtual
machine. Basing the whole thing on a GPL-licensed kernel may have been a
necessary evil from the Android group's point of view, but accepting that
evil does not require taking on an enthusiasm for enforcing the GPL's
requirements.
Add to that the fact that the patent wars have put Android vendors into an
unpleasant and uncertain legal situation. It seems clear that any
commercial success in the mobile world is going to end up with patent
leeches hanging all over it, but that doesn't change the fact that there is
an especially visible legal cloud over Android at the moment. Adding GPL
compliance problems to those already faced by Android vendors, especially
if those problems come in the form of a copyright infringement lawsuit,
could well be the straw that breaks the camel's back and causes those
vendors to look more favorably on other systems. Anybody within Google who
is considering a stronger stance on GPL compliance must certainly have that
concern at the front of their mind.
Then, there is the little issue that few people want to talk about: what
about those binary-only kernel modules? Google would not be able to take a
position on GPL compliance in Android distributions without taking a
position on binary-only modules. If use of the Android trademark is
conditioned on GPL compliance, somebody will have to decide if
distributions with proprietary modules (most of them, unfortunately) are
compliant. A "no" answer would strip large numbers of devices of their
trademark usage rights, while a "yes" answer would be a public statement
that binary-only modules are OK. Given how few companies are willing to
take a stand on this issue, it is unsurprising that Google does not want to
find itself in that position.
Finally, it is worth observing that, for whatever reasons, companies almost
never engage in GPL-enforcement actions despite owning the copyrights on
large amounts of code. Almost all of the successful
enforcement out there has been done by (or at least in the name of)
individuals. The few exceptions,
such as when IBM asserted GPL-infringement against SCO, show how serious
the situation has to be before a corporation is willing to make such a
move. So it could be said that Google is just behaving like all of its
peers in this regard.
Having just made all of these excuses for Google's lack of action in this
area, your editor would like to make one thing clear: a failure to insist
on better GPL behavior may be entirely understandable, but, in the long (or
not so long) run this approach is unwise and may prove dangerous for the
Android market. If Google does not get Android vendors to clean up their
act, others will lose their patience and do it for them. Indeed, as made
clear by Harald Welte with regard to HTC's policy of delaying source
releases, patience is already pretty well exhausted:
There are developers in our community who are prone to a certain amount of
empty noise and bluster, but Harald is not one of them; this kind of
threat, coming from him, is 100% credible.
One could say that this still works in Google's favor: individuals can
force Android vendors to improve their behavior without associating
Google's name with their actions. But if Google is worried about legal
uncertainty surrounding use of Android, it must certainly worry about
actions brought by individuals that are completely out of its control.
That is especially true given that some individuals have been
making ominous noises about GPL termination conditions and what might be
required to regain GPL distribution rights after a violation. By looking
the other way when Android vendors fail to live up to the GPL, Google may
be helping those vendors set themselves up for attack by people or groups
whose primary concern is not availability of the source code. Some of
those people may be after serious financial gain or an opportunity to make
the use of GPL-licensed code look like an inherently dangerous and risky
thing.
If nothing else, contempt for the GPL will breed more of the same; as
Matthew said, "
On the political front, it is a fairly safe bet that the mobile patent
wars will get worse. The participants in this battle appear to be
tooling up for an extensive and protracted fight, and there seems to be a
steady supply of new companies (British Telecom, for example) piling on in
the hope of gaining a piece of the pie for themselves. Things may reach a
point where it becomes impossible to market handsets or tablets in some
parts of the world. One could hope that a paralyzed mobile market would
suggest to lawmakers that the patent system is broken, and that those
lawmakers would respond by reforming said system. Alas, a more likely
outcome (though not in 2012) is the formation of a patent pool under strong
"encouragement" by governments that ends the fights and establishes the
positions of a small number of large players at the expense of new
entrants. How free software will fare in such a world is far from clear.
The fight for a free Internet will also continue in full force in
2012. Your editor predicts that the current round of net-censorship bills
in the US ("SOPA" and its variants) will be defeated. But the forces
behind such laws never quit and eventually something distasteful will
likely make it through the legislative process.
Laws like SOPA and governmental attempts to control the net in general seem
to be causing concern beyond the small group of people who habitually worry
about such things. As people wake up to threats to their freedom, they
will see that free software is on the front line in the fight
against censorship and repression. Hopefully that will result in a better
awareness of the value of freedom. But it carries a risk that free
software could find itself associated with piracy, crackers, and
criminals. We could have an interesting public relations problem to deal
with.
On the distributions front, it seems evident that Red Hat will have
another great year. Various articles on the net are suggesting that
2012 will be its first $1 billion year, and that the company is
looking to hire 1,000 more people. Red Hat has been helped by the growth
of Linux in general, by its dominant position in the market, and, arguably,
by the difficulties experienced by distributions like CentOS. It has
become reasonably clear that, if one really needs the committed support
that comes with RHEL, one really needs to just pay for RHEL. That said,
the RHEL-rebuild distributions will certainly not be going away in the
coming year; neither will the other commercial distributors.
But we will see more focused competition between distributors, with
more attention paid to the addition of unique features to differentiate
their offerings.
There was a long period where most Linux distributions were about the
same; the biggest differences were to be found in areas like package
management, and they were easy to adjust to. Now we see distributions like
Ubuntu focusing on their own desktop experience. Oracle seems to be
trying to differentiate with more current kernels and an early move to
technologies like Btrfs. Projects like GNOME are increasingly showing a
desire to become distributions in their own right.
Android and WebOS show the extent to which an
ostensibly "Linux" system can differ from its peers.
In a sense, the long-feared fragmentation of the
Linux system is happening. It is good in that we have a diverse set of
projects exploring ways to make a better operating system. But that
diversity also threatens to scatter our efforts to the point that nothing
ever becomes truly good enough to find widespread success.
Along those lines, Linux Mint will have a reckoning with reality in
2012. This distribution has gained a lot of attention with its approach to
desktop design. But it remains a minor distribution, and, at some point,
it can only become clear that the resources to maintain multiple
distributions (Linux Mint 12, Mint Debian, Mint 11 LXDE, etc.),
the MATE GNOME2 fork and the "Cinnamon" GNOME 3 fork simply are not
there. Users will also eventually begin to wonder about things like
security updates. Your editor fully expects Linux Mint to be a strong and
popular project at the end of the year, but it will need to focus its
efforts somewhat.
On a related topic, the GNOME3 wars will be long forgotten by the
end of the year. Users will have either made their peace with GNOME3 or
they will have moved on to something else. The growing availability of
GNOME Shell extensions should help considerably. The various attempts to
fork the GNOME environment or maintain GNOME2 will mostly fade away.
The fight for the attention of mobile device manufacturers will,
instead, continue through
2012. Android is well established, of course, and it is hard to see that
situation changing much. But there should be room for another free
platform. Quite a few projects - GNOME, KDE, Tizen, WebOS, Boot to Gecko,
etc. - would be delighted to occupy that space and be shipped on real
products. They cannot all succeed, though. It would be a shame if they
were to all fail and leave that space to Windows (which will make a big
push on Nokia's handsets) in 2012.
Speaking of Android, mainline kernels will be able to run Android's user
space by the end of the year - more or less. That is not the same as
saying that all of the Android kernel code will make it into the mainline;
some of it may be replaced by code with equivalent functionality. But a
lot of hardware vendors are increasingly concerned with Android kernels,
not mainline kernels, so the interest in closing the gap between the two
will only grow.
The gap between Apache OpenOffice and LibreOffice will also grow.
Over the year it will become increasingly clear that LibreOffice has
captured the initiative and the developer energy in this space.
LibreOffice will continue to improve while Apache OpenOffice struggles with
the replacement of GPL-licensed code, adapting to "the Apache Way," and
trying to produce a release based on a year-old beta. There are some good
people working on Apache OpenOffice, but they are swimming against the
current.
The kernel's "ARM mess" will be a memory by the end of the year as
the effort to consolidate code and clean up subarchitecture implementations
continues. The ARM tree was a victim of its own success; vendors actually
listened to the pleas to work upstream and contributed vast amounts of
code. Now that the problem is being addressed, the "victim" part will go
away, leaving only the success; ARM will take its place as one of the
primary Linux architectures, even if it does not become the primary
architecture in 2012.
The security mess will not go away, unfortunately. Our widespread
code repositories and distribution sites present an attractive target to
anybody wishing to compromise large numbers of machines. Targeted attacks
against these sites can only increase, and some of them will be
successful. Our community as a whole is going to have to learn to take
security much more seriously. That is starting to happen in some projects,
but not in others. With luck, we will not wake up one morning to learn
that we have distributed trojaned code to vast numbers of users - but there
is no guarantee that we will be so lucky.
On a happier note, a longstanding request from LWN users will be satisfied
in 2012: the site will finally use the UTF-8 encoding and will not be
limited to the Latin-1 character set. So, soon, it may be possible to find
text like "أي شخص يعتقد التوقعات في هذا الموقع هو أحمق" or "ನಿಮ್ಮ ಸಂಪಾದಕ ಒಂದು
ಕಾಗದದ ಚೀಲ ಹೊರಗೆ ದಾರಿಯಲ್ಲಿ ಪ್ರೋಗ್ರಾಂ ಸಾಧ್ಯವಾಗಲಿಲ್ಲ" or "明けましておめでとうご
ざいます" embedded directly within an
LWN article without the need for obnoxious HTML escapes. Your editor
predicts that change will come sometime quite soon.
Finally, your editor predicts that the world will not end in 2012. The
Mayans, he claims, were worse at this prediction game than he is - and those
who would draw conclusions from the Mayan calendar are even worse yet.
Despite all the coming challenges, the free software world will not end
either; instead, we will finish the year with more strength and momentum
than we had at the beginning. It will be great fun to watch.
Google makes money off other people's violation of the
GPL
", so it does nothing.
In the absence of enforcement, GPL compliance only
works if it's the norm
". By ignoring its partners' GPL violations,
Google is, at best, undermining that norm; at worst, it could be setting up
Android (and possibly Linux) for a serious fall. Google still has the
power to drive considerable improvements in this area if it takes the
initiative now. Continued inaction, though, risks letting that initiative,
and that power, slip away as others do the job that Google is unwilling to
do. That does not seem like an outcome that would make anybody happy.
Linux at the end of the world (our 2012 predictions)
Welcome to 2012. This is the first LWN Weekly Edition of the year, and
that can only mean one thing: it is time for your editor to go out on a
limb and make a number of predictions for the coming year that, by the end
of the year, will look thoroughly clueless and misguided. Even your editor
can foresee, though, that it is going to be an interesting and highly
political year.
Security
A hole in telnetd
An unexpected FreeBSD security team "holiday message" had an underlying cause that was, perhaps, even more surprising: a long-standing remote code execution flaw in telnetd. As the message notes, FreeBSD tries hard to avoid releasing security updates at inconvenient times, and the end of the year is pretty inconvenient for most. But, attacks were being seen in the wild and, since telnetd is typically run as root, the consequences were severe.
Some, perhaps many, readers can be forgiven for wondering what all the fuss is about, as it has been well over a decade since telnet was being actively discouraged for use on Linux (and other operating systems). It is still present in many distribution repositories, however, which led to a pile of Linux updates in addition to the FreeBSD update. It turns out that telnetd is actually being used much more than one might think, especially on embedded devices.
The reason that telnet has long been deprecated—at least over unencrypted networks—is that it is a cleartext protocol. Logging into a remote system using telnet means that your username and password were being sent in the clear, such that any man-in-the-middle could sniff your credentials. But, it turns out that the telnet protocol does have an encrypted mode (though it's not considered cryptographically strong), and it is that mode that is being exploited by the recent attacks.
The bug itself is a fairly mundane buffer overflow that is triggered when an overlong encryption key is sent to the server. That key is sent before any authentication has occurred, which allows random attackers to target any vulnerable telnetd server. Even readers without much knowledge of C (or buffer overflows) will likely understand the basics of the patch that fixes the problem. There is also a fairly comprehensive exploit available for study. It can target multiple Linux and BSD installations, and contains shell code (i.e. code that will create a root shell when executed by telnetd because of the exploit) for i386 and SPARC architectures.
While the bug itself has been around for quite some time, it is interesting—nearly amusing—to see that sites still have telnetd servers running. The updates imply that these hosts are both accessible by untrusted users (or no update would be needed) and are regularly accessed via telnet. There are few, if any, Linux distributions that enable telnetd by default, so administrators or device makers are knowingly enabling it. While sshd most certainly has had its share of bugs, and will likely have more down the road, one would guess that security researchers pay a lot more attention to sshd than they do to telnetd. Not so for attackers evidently.
Part of the reason that telnetd may still be hanging around is that Windows doesn't ship an SSH client, but does ship telnet. That may encourage device makers to enable telnet so that Windows users can access the device without installing any software. In addition, there were several reports that Cisco and Juniper routers are often accessed via telnet in the comment thread on our posting of the FreeBSD message. Given that those devices often sit in strategic internet locations, and may well be running a telnetd descended from the vulnerable code, it could lead to some fairly serious consequences. One hopes that Cisco, Juniper, and others are paying attention.
Brief items
Security quote of the week
But this problem is peanuts compared to what has just appeared. In the debate around the American Stop Online Piracy Act, American legislators have demonstrated a clear capability and willingness to interfere with the technical operations of American products, when doing so furthers American political interests regardless of the policy situation in the customer's country. Actually, it's even worse: American legislators have demonstrated a willingness to do this just because of the different laws in the customer's country, outside of the United States.
28C3: New attacks on GSM mobiles and security measures shown (The H)
The H reports from the Chaos Communication Congress; the open-source Osmocom package appears to be serving its intended purpose and finding vulnerabilities in the cellphone network. "The researchers explained and then demonstrated how, using the above technique and easily procurable tools, attackers are able to emulate a mobile phone to make phone calls and send text messages. They noted that some users have already received bills totalling thousands of euros for calls and texts to Caribbean premium rate services. In many cases, an attacker can, by simulating a GSM mobile, also query that subscriber's mailbox providing they know the subscriber's location and the key has not been changed."
Merry Christmas from FreeBSD
The FreeBSD security team has sent out a holiday card of sorts to its users. "No, the Grinch didn't steal the FreeBSD security officer GPG key, and your eyes aren't deceiving you: We really did just send out 5 security advisories." The motivating factor appears to be this vulnerability in telnetd; anybody who exposes a telnet port to the net on FreeBSD is currently open to a remote root exploit. The number of LWN readers doing so must be tiny (if, indeed, it's nonzero), but it seems worthwhile to get the word out there anyway.
The Linux kernel's memory allocators from an exploitation perspective
"Argp" has posted a lengthy look at the kernel's memory allocators and how they can be exploited to attack the system. "The attack vector of corrupting adjacent objects on the same slab is fully applicable to SLUB and largely works like in the case of the SLAB allocator. However, in the case of SLUB there is an added attack vector: exploiting the allocator’s metadata (the ones responsible for finding the next free object on the slab). As twiz and sgrakkyu have demonstrated in their book on kernel exploitation, the slab can be misaligned by corrupting the least significant byte of the metadata of a free object that hold the pointer to the next free object. This misalignment of the slab allows us to create an in-slab fake object and by doing so to a) satisfy safeguard checks as the one I explained in the previous paragraph when they are used, and b) to hijack the kernel’s execution flow to our own code."
New vulnerabilities
cacti: multiple vulnerabilities
| Package(s): | cacti | CVE #(s): | |||||||||
| Created: | December 23, 2011 | Updated: | January 4, 2012 | ||||||||
| Description: | Cacti to 0.8.7i fixes multiple security vulnerabilities. See the release notes for details. | ||||||||||
| Alerts: |
| ||||||||||
ffmpeg: multiple code-execution vulnerabilities
| Package(s): | ffmpeg | CVE #(s): | CVE-2011-4351 CVE-2011-4353 CVE-2011-4364 CVE-2011-4579 | ||||||||||||||||||||||||||||||||||||
| Created: | January 4, 2012 | Updated: | August 30, 2012 | ||||||||||||||||||||||||||||||||||||
| Description: | Multiple vulnerabilities have been found in the ffmpeg audio application.
| ||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||
ghostscript: code execution
| Package(s): | ghostscript | CVE #(s): | CVE-2009-3743 | ||||||||||||||||||||||||||||||||
| Created: | January 4, 2012 | Updated: | February 6, 2012 | ||||||||||||||||||||||||||||||||
| Description: | From the CVE entry: Off-by-one error in the Ins_MINDEX function in the TrueType bytecode interpreter in Ghostscript before 8.71 allows remote attackers to execute arbitrary code or cause a denial of service (heap memory corruption) via a malformed TrueType font in a document that trigger an integer overflow and a heap-based buffer overflow. | ||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||
ghostscript: denial of service
| Package(s): | ghostscript | CVE #(s): | CVE-2010-4054 | ||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | January 4, 2012 | Updated: | February 6, 2012 | ||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | Specially crafted font data in a compressed data stream can force the ghostscript interpreter to crash; see this patch for details. | ||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||
kernel: restriction bypass
| Package(s): | kernel | CVE #(s): | CVE-2011-4127 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | December 23, 2011 | Updated: | March 6, 2012 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the Red Hat advisory:
* Using the SG_IO IOCTL to issue SCSI requests to partitions or LVM volumes resulted in the requests being passed to the underlying block device. If a privileged user only had access to a single partition or LVM volume, they could use this flaw to bypass those restrictions and gain read and write access (and be able to issue other SCSI commands) to the entire block device. In KVM (Kernel-based Virtual Machine) environments using raw format virtio disks backed by a partition or LVM volume, a privileged guest user could bypass intended restrictions and issue read and write requests (and other SCSI commands) on the host, and possibly access the data of other guests that reside on the same underlying block device. Partition-based and LVM-based storage pools are not used by default. Refer to Red Hat Bugzilla bug 752375 for further details and a mitigation script for users who cannot apply this update immediately. (CVE-2011-4127, Important) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
kernel: denial of service
| Package(s): | kernel | CVE #(s): | |||||
| Created: | December 26, 2011 | Updated: | January 4, 2012 | ||||
| Description: | Linux kernel 3.1.6 restores the route cache garbage collector. Recent kernels could fill and exhaust their neighbor cache. | ||||||
| Alerts: |
| ||||||
moodle: lots of vulnerabilities
| Package(s): | moodle | CVE #(s): | CVE-2011-4581 CVE-2011-4582 CVE-2011-4583 CVE-2011-4584 CVE-2011-4585 CVE-2011-4586 CVE-2011-4587 CVE-2011-4588 CVE-2011-4589 CVE-2011-4590 CVE-2011-4591 CVE-2011-4592 CVE-2011-4593 | ||||||||||||
| Created: | December 22, 2011 | Updated: | January 4, 2012 | ||||||||||||
| Description: | The moodle 2.1.3, 2.0.6, and 1.9.15 releases fix a large number of information leak, code injection, and other vulnerabilities; see the 2.1.3 release notes for details. | ||||||||||||||
| Alerts: |
| ||||||||||||||
mozilla: multiple vulnerabilities
| Package(s): | mozilla, firefox, thunderbird, seamonkey | CVE #(s): | CVE-2011-3658 CVE-2011-3660 CVE-2011-3661 CVE-2011-3663 CVE-2011-3665 | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | December 26, 2011 | Updated: | March 23, 2012 | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the Mandriva advisory:
The SVG implementation in Mozilla Firefox 8.0, Thunderbird 8.0, and SeaMonkey 2.5 does not properly interact with DOMAttrModified event handlers, which allows remote attackers to cause a denial of service (out-of-bounds memory access) or possibly have unspecified other impact via vectors involving removal of SVG elements (CVE-2011-3658). Multiple unspecified vulnerabilities in the browser engine in Mozilla Firefox 4.x through 8.0, Thunderbird 5.0 through 8.0, and SeaMonkey before 2.6 allow remote attackers to cause a denial of service (memory corruption and application crash) or possibly execute arbitrary code via vectors that trigger a compartment mismatch associated with the nsDOMMessageEvent::GetData function, and unknown other vectors (CVE-2011-3660). YARR, as used in Mozilla Firefox 4.x through 8.0, Thunderbird 5.0 through 8.0, and SeaMonkey before 2.6, allows remote attackers to cause a denial of service (application crash) or possibly execute arbitrary code via crafted JavaScript (CVE-2011-3661). Mozilla Firefox 4.x through 8.0, Thunderbird 5.0 through 8.0, and SeaMonkey before 2.6 allow remote attackers to capture keystrokes entered on a web page by using SVG animation accessKey events within that web page (CVE-2011-3663). Mozilla Firefox 4.x through 8.0, Thunderbird 5.0 through 8.0, and SeaMonkey before 2.6 allow remote attackers to cause a denial of service (application crash) or possibly have unspecified other impact via an Ogg VIDEO element that is not properly handled after scaling (CVE-2011-3665). | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||
openstack-nova: directory traversal
| Package(s): | openstack-nova | CVE #(s): | CVE-2011-4596 | ||||||||
| Created: | December 23, 2011 | Updated: | January 20, 2012 | ||||||||
| Description: | From the Red Hat bugzilla:
Prevent potential directory traversal with malicious EC2 image tarballs, by making sure the tarfile is safe before unpacking it. Prevent potential directory traversal with malicious file names in EC2 image manifests. | ||||||||||
| Alerts: |
| ||||||||||
php: denial of service
| Package(s): | php | CVE #(s): | CVE-2011-4885 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | December 30, 2011 | Updated: | April 13, 2012 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the Mandriva advisory:
PHP before 5.3.9 computes hash values for form parameters without restricting the ability to trigger hash collisions predictably, which allows remote attackers to cause a denial of service (CPU consumption) by sending many crafted parameters. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
phpmyadmin: cross-site scripting
| Package(s): | phpMyAdmin | CVE #(s): | CVE-2011-4780 CVE-2011-4782 | ||||||||||||||||
| Created: | January 2, 2012 | Updated: | January 4, 2012 | ||||||||||||||||
| Description: | From the Red Hat bugzilla:
Multiple cross-site scripting (XSS) vulnerabilities in libraries/display_export.lib.php in phpMyAdmin 3.4.x before 3.4.9 allow remote attackers to inject arbitrary web script or HTML via crafted URL parameters, related to the export panels in the (1) server, (2) database, and (3) table sections. (CVE-2011-4780) From the Red Hat bugzilla: Cross-site scripting (XSS) vulnerability in libraries/config/ConfigFile.class.php in the setup interface in phpMyAdmin 3.4.x before 3.4.9 allows remote attackers to inject arbitrary web script or HTML via the host parameter. (CVE-2011-4782) | ||||||||||||||||||
| Alerts: |
| ||||||||||||||||||
t1lib: code execution
| Package(s): | t1lib | CVE #(s): | CVE-2011-0764 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | December 22, 2011 | Updated: | January 30, 2012 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | The t1lib package has a code execution vulnerability exploitable via a malicious font file. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
telnetd: code execution with root privileges
| Package(s): | telnetd krb5 krb5-appl heimdal | CVE #(s): | CVE-2011-4862 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | December 26, 2011 | Updated: | February 23, 2012 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the Debian advisory:
It was discovered that the Kerberos support for telnetd contains a pre-authentication buffer overflow, which may enable remote attackers who can connect to the Telnet to execute arbitrary code with root privileges. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
unbound: denial of service
| Package(s): | unbound | CVE #(s): | CVE-2011-4528 CVE-2011-4869 | ||||||||||||||||
| Created: | December 23, 2011 | Updated: | December 1, 2013 | ||||||||||||||||
| Description: | From the Debian advisory:
It was discovered that Unbound, a recursive DNS resolver, would crash when processing certain malformed DNS responses from authoritative DNS servers, leading to denial of service. CVE-2011-4528: Unbound attempts to free unallocated memory during processing of duplicate CNAME records in a signed zone. CVE-2011-4869: Unbound does not properly process malformed responses which lack expected NSEC3 records. | ||||||||||||||||||
| Alerts: |
| ||||||||||||||||||
Page editor: Jake Edge
Kernel development
Brief items
Kernel release status
The current development kernel is 3.2-rc7, released on December 23. As of this writing, the final 3.2 release is imminent; one assumes that Linus is waiting for the LWN Weekly Edition to be published first.Update: the 3.2 release did, indeed, happen at the predicted time.
Stable updates: the 2.6.32.52, 3.0.15 and 3.1.7 updates were released on January 3; they contain a single fix for a resume problem experienced by some users. The 2.6.32.53, 3.0.16, and 3.1.8 updates are in the review process as of this writing. They contain a longer list of fixes, and can be expected on or after January 6.
Quotes of the week
An IOPS-based I/O scheduler
I/O schedulers are charged with ordering block I/O operations in a way that maximizes throughput to the device and, perhaps, implementing the system's policy with regard to how the available bandwidth should be divided. The schedulers currently in use in Linux were designed with rotating storage in mind, with the result that they are concerned with avoiding disk seeks and tracking the number of bytes transferred. With solid-state devices, though, I/O locality is (nearly) irrelevant and the number of I/O operations performed is considered to be a better measurement of the amount of device capacity used. The kernel's CFQ scheduler has been evolving to deal better with solid-state devices, but everybody agrees there is more to be done.Shaohua Li has taken a new approach with the posting of a new I/O scheduler that is optimized for solid-state devices. The patch set factors out and generalizes the CFQ code that tracks device usage, but then uses that code to implement a different scheduling algorithm. Avoiding seeks is no longer a concern; neither is the number of bytes transferred. Instead, this scheduler simply tracks the number of I/O operations submitted by each user, trying to equalize the number from each.
The result should be a simpler scheduler that is better suited to solid-state devices. At this point, though, it is hard to say for sure. One of the key rules of kernel patch submission is that performance-oriented changes should be accompanied by benchmark results showing that they achieve the intended goal. This patch had no such results, so nobody knows if it is worth their while to look at the code further or not. Presumably the next submission will provide that information, at which point the real discussion of the new scheduler's merits can begin.
Kernel development news
A privilege escalation via SCSI pass-through
One of the important attributes for virtualization is to provide complete isolation between the virtual machines, so that attackers (or bugs) in one VM cannot interfere with the other VMs. But, as a recent bug report shows, the kernel is vulnerable, in some configurations, to VMs that can read and write the disks of other VMs. That's clearly a serious security problem, but the discussion about patches to fix the bug makes it clear that it may take some time before the fix can be applied.
The problem occurs when programs issue the SCSI pass-through SG_IO ioctl() to a particular disk partition (e.g. /dev/sdb2) or LVM volume, which causes the SCSI command to be sent to underlying block device (/dev/sdb). The actual commands that can be sent to the device via SG_IO are filtered for processes that don't have the CAP_SYS_RAWIO capability, but there are still dangerous things that can be done. In particular, if a process can write to the partition, it can write to the underlying device without being restricted to the boundaries of that partition.
For virtualization configurations that mingle partitions or volumes used by different VMs on the same block device, that means that a VM can access—and change—the data on another VM's disk. Worse still, if the host OS stores its own data on that block device, a rogue VM could potentially compromise the host. Exploiting the vulnerability does not require a virtualization (or containerization) scenario, but those are the most likely ways that it could come about. Any process that can open the partition device node will be able to issue the ioctl(), but, on "standard" Linux systems, that ability is typically restricted to root.
Based on the bug report, Paolo Bonzini found the problem back in November 2011, but security problems with SG_IO were known as far back as August 2004. Bonzini posted patches to fix the problem at the end of December (though it would appear that the issue was under discussion on the closed kernel security mailing list in the interim). The proposed fix would disallow most SCSI commands on partition-like devices. So, doing any of the "dangerous" SCSI commands would fail unless the ioctl() is being called on the underlying block device.
The patches sparked a few comments from Linus Torvalds, mostly regarding error return codes (partly because ENOTTY is badly named for its use as an indication of "no such ioctl"). But, beyond that, he started to wonder whether there might be situations where users do issue SCSI commands to partitions and expect them to be passed down to the block device. It turns out that there is at least one place where it may be a common event: "ejecting" USB sticks and other removable media. Torvalds notes:
And that's the *natural* way to eject a mounted device. Look at the USB memory sticks you have. They are almost all partitioned to have one partition, and that one partition doesn't cover the whole device. And it's that one partition you use to interact with it - it's what you mount, and what you eject.
According to Bonzini, the fact that the
CDROMEJECT fails on a kernel with his proposed fix doesn't cause
any problems in practice. But Torvalds's concern goes beyond that one
particular example. The fix has been suggested for merging late in the 3.2
development cycle and his concern was the level of testing that it has been
subjected to: "I absolutely do not get the feeling that this has been tested so much
and is so obvious that there is no risk of breakage.
" Based on the
discussion, the testing seems to have been focused on ensuring that the
security hole was closed, without considering the other impacts that a—fairly sweeping—change might have.
Torvalds would certainly like to see the vulnerability fixed, but not at
the expense of a regression in what users have come to depend on. As he pointed out: "Suddenly
totally changing things and saying 'you can't do that on a partition'
when clearly people *have* been doing that on partitions isn't
something we can do without serious testing.
" His plan is to wait
for the 3.3 merge window to bring in the fix, which should allow some
testing time for distributions and others to ensure that the code doesn't
have any unintended consequences.
While it is important to fix security holes, it is equally important to keep everything else working, which is the bulk of Torvalds's concern. While the 3.3 development cycle may still not be long enough to shake out all of the places where the SCSI pass-through is used on partial disks (partitions or logical volumes), it certainly will provide more of a chance to do so than would a merge in the final stages of 3.2 development. In the meantime, now that the bug and fix are out in the open, concerned administrators can apply the patch or take other steps to remedy the problem.
Safe device assignment with VFIO
As a general rule, most developers feel that device drivers belong in the kernel. Kernel-space drivers are (hopefully) widely reviewed, implement standard device interfaces, perform better, and are more secure than the user-space variety. There are exceptions, though. Some high-performance applications want to talk to devices directly. Virtualized guests can also be thought of as a sort of user-space process; it is often desirable to allow guests to work with hardware directly rather than funneling their I/O through the host. So the kernel really should support this mode of access for the times when it is needed.The kernel's UIO interface has been available for the implementation of user-space drivers for some years. UIO has some shortcomings, though, including a lack of support for direct memory access (DMA) operations. DMA under user-space control is challenging to support for a number of reasons, not the least of which is security. A DMA-capable device is normally capable of writing any page in memory; as a result, empowering a user-space process to set up DMA operations is equivalent to giving it full root access. Sometimes a user-space driver can be trusted with that access, but that is often not the case, especially when virtualization is involved.
More recent CPUs have added support for safe (or safer) access to devices from virtualized guests. Devices can be restricted, via an I/O memory management unit (IOMMU) so that only specific regions of memory are accessible to them. Technologies like KVM support a "device assignment" mechanism that uses the hardware capabilities to hand a device to a guest, but device assignment is not without its shortcomings. Among other things, device assignment alone cannot guarantee the isolation of a specific device, and it involves a fair amount of complexity in the kernel.
Alex Williamson's VFIO patch set is an attempt to come up with a better solution that allows the development of safe, high-performance user-space drivers. It provides interfaces allowing those drivers to work with DMA and interrupts while keeping overall control over how devices access the system's resources.
One problem with KVM's device assignment is that it assumes that all devices are fully independent of each other. In particular, groups of devices may be connected through the same IOMMU; that means that any device can access any memory regions made available to any other devices in the same group. That, in turn, implies that the group of devices must be assigned as a unit; if any of those devices are assigned separately, the isolation of the group as a whole can be broken.
So the first thing a VFIO driver writer will encounter is the group mechanism. The VFIO code creates the groups to match the hardware topology. It then ensures that every device in a group is controlled by a VFIO driver; if any device is unavailable, then the group as a whole cannot be used. Most devices on a typical system are unlikely to be bound to VFIO drivers at boot, so the system administrator must explicitly unbind them and tell VFIO to claim them. This is probably a good thing; exposing groups of devices to user space is best not done by default.
For each group, a virtual device is created under /dev/vfio; prior to working with any individual device, a driver must open the group, claiming ownership of it. The access permissions on the group file control access to the underlying devices. Once the group has been opened, the driver should do an ioctl(VFIO_GROUP_GET_INFO) call to determine whether the group is "viable" (meaning all of the relevant devices are assigned to it) and available for use. If the group is not viable, the driver will not be able to proceed.
To work with specific devices, the driver will "open" them with the VFIO_GROUP_GET_DEVICE_FD ioctl() call, which returns a file descriptor for access to the device. The VFIO_DEVICE_GET_REGION_INFO command can be used to learn about the device's memory-mapped I/O regions, which can then be accessed via an mmap() call. VFIO_DEVICE_GET_IRQ_INFO returns information about the device's interrupt assignment(s); the driver can use the eventfd() mechanism to receive notification of interrupts via a file descriptor. For most hardware, access to MMIO and interrupts is enough to communicate with the device.
That still leaves the DMA problem, though. To that end, the VFIO_GROUP_GET_IOMMU_FD command returns a file descriptor representing the IOMMU. DMA mappings can be set up by filling in a vfio_dma_map structure:
struct vfio_dma_map {
__u32 argsz;
__u32 flags;
__u64 vaddr; /* Process virtual address */
__u64 iova; /* IO virtual address */
__u64 size; /* Size of mapping (bytes) */
};
This structure is used to request a mapping of the user-space memory found at vaddr (of size bytes) into the device's I/O memory range starting at iova; the VFIO_IOMMU_MAP_DMA command actually gets the work done. For most user-space drivers, that should be about all that is needed, modulo a few details.
Not all VFIO drivers will be in user space, though. Inside the kernel, VFIO looks like a special bus type to which devices can be bound. A VFIO driver needs to provide a set of operations to the core:
struct vfio_device_ops {
bool (*match)(struct device *dev, const char *buf);
int (*claim)(struct device *dev);
int (*open)(void *device_data);
void (*release)(void *device_data);
ssize_t (*read)(void *device_data, char __user *buf,
size_t count, loff_t *ppos);
ssize_t (*write)(void *device_data, const char __user *buf,
size_t count, loff_t *size);
long (*ioctl)(void *device_data, unsigned int cmd,
unsigned long arg);
int (*mmap)(void *device_data, struct vm_area_struct *vma);
};
Most of these operations are analogous to those found in struct file_operations or the bus-specific device structures. A device registered in this way can be opened and used like any other device with one difference: the interlock with group ownership is always enforced. If a device has been opened individually, the group is not "viable" and cannot be used by a user-space driver. If, instead, the group has been opened, the individual devices are busy and cannot be opened.
VFIO is not the only patch set aimed at this problem; David Gibson's device isolation infrastructure is also intended to enable safe assignment of devices. The scope of this patch set is smaller, though, focusing mostly on the grouping aspect; there is no mechanism for controlling the IOMMU or working with individual devices. There is a certain amount of disagreement between the two on how grouping should be managed which suggests, in turn, that a certain amount of discussion will have to take place before either can be merged.
The logger meets linux-kernel
Toward the end of December, LWN looked at the new push to move various subsystems specific to Android kernels into the mainline. There seems to be broad agreement that merging this code makes sense, but that agreement becomes rather less clear once the discussion moves to the merging of specific subsystems. Tim Bird's request for comments on the Android "logger" mechanism shows that, even with a relatively simple piece of code, there is still a lot of room for disagreement and problems can turn out to be larger than expected.The logger is a simple device designed to collect log data from applications and make that data available to developers and the central logging system. It was designed around a number of goals, most of which are driven by the untrusted nature of Android applications: the system wants to allow those applications to send messages to the log in a reliable and trustworthy manner. So applications should not be able to fake who they are, consume too much memory with log data, or spam the log at the expense of data from the kernel or other applications. Writing to the log is also meant to be a fast operation; lots of log data is generated, but little of it is read, so the write operation is the one that should be the fastest.
The driver implements a handful of logging "devices" for different streams; they are known as "main," "events," "radio," and "minor." These devices are hardwired into the code; there is no way to change the set of devices at run time. Each device is implemented as a simple ring buffer in kernel space; writing to the device adds a message to the buffer (annotated with timestamp and process ID information) while any number of readers can pull data out of the buffer. Each log has 256KB of memory dedicated to it; that size, too, is wired into the code. There is a small set of ioctl() operations provided to get the size of the log or the amount of unread logging data stored there, or to flush all data from the log.
The first question to be raised was: why is this facility needed at all? The kernel already has a mechanism for letting user space add entries to the log stream. In mainline kernels, that is done by writing to /dev/kmsg; that device takes the data given to it and passes it straight to printk(). There are a few reasons why Android would not want to use this interface: it does not allow for the separation of logging streams (and, thus, is limited to the root account), it does not add process ID information, and printk() is quite a bit more expensive than simply copying the log data into kernel memory. While adding a duplicate logging infrastructure is obviously problematic, a case could be made that something like logger should be merged and used as a replacement for /dev/kmsg.
That said, "something like logger" need not be the logger itself. If nothing else, it is hard to imagine the current code being merged without corrections for the most obvious problems: the hardwired log streams, for example. Logger is entirely unaware of namespaces, making it unsuited to environments where different process groups may want their own logging streams. It also does not really succeed in keeping processes from spamming the logs and overwriting data from other processes; each of the four streams is isolated from the others, but applications still end up logging to the same streams.
There is also the question of whether the user-space API is the right one. Adding more ioctl() calls to the kernel is never a popular thing to do. Some participants have suggested that an entirely user-space-based logging daemon would work just as well. But a user-space solution has some disadvantages: it would not be able to provide trusted process ID information, and it would likely be slower. Communicating log data to a user-space daemon and adding information like time stamps would involve more system calls than simply logging the data through the kernel. It would also be hard to merge kernel and user-space log data (and, in particular, to ensure that the ordering between events is correct) with a user-space scheme.
Neil Brown suggested that a filesystem interface could be used instead of logger's char device ABI. A logging stream could be established simply by creating a file within that filesystem; regular file permissions could then be used to control access to the streams. The ioctl() calls could be replaced with regular filesystem calls; flushing a log would be done with truncate(), for example. A filesystem-based logger would, clearly, involve moving to a different user-space interface, but it seems like it should be able to satisfy the use case that led to the creation of logger with a more Unix-like ABI.
Android developer Brian Swetland has indicated that there might be interest in trying out an alternative logging implementation, but it would have to meet their needs and it's not something that they would be interested in creating themselves:
If somebody wants to go write a complete compatible replacement that just drops in, we certainly could take a look at it and see how it works, but given that the benefits are not clear to us, we're not interested in going off and doing that work ourselves.
There is another point to consider as well: the longstanding interest in adding a more structured logging interface to the kernel seems to be getting stronger. Adding a new logging mechanism that doesn't address the concerns of translators, documentation writers, and people maintaining automated network management systems is going to be a hard sell. Kay Sievers, one of the developers behind "The Journal," has a plan of his own for logging; it involves adding more structured information to the existing message stream. He thinks there is no need to add a new, incompatible logging mechanism; he is also strongly opposed to the idea of separating message streams as logger does. That separation, he says, just creates problems when messages from multiple sources need to be merged back together in the correct order.
Brian actually seems receptive to some of these ideas, but has expressed concerns around rate-limiting and controlling access to log data. Lennart Poettering believes these needs can be met by the Journal with a little help from control groups. It seems possible that something could be done to make both groups happy - though the code to actually do so has not been posted.
As of this writing, that is where the discussion stands. On the surface, it looks like another piece of Android technology has gotten tangled up in unrelated requirements from various kernel developers, with the result that it will sink like its predecessors. But there is the potential for a number of possible solutions here. One, of course, would be to simply merge a moderately cleaned-up logger for Android's use and be done with it. Another would be to, as Neil suggested, focus on Android's user-space ABI for logging. If that ABI were formalized in a way that would allow multiple underlying implementations, the mainline could merge something other than logger and still be that much closer to being able to run Android's user space.
Perhaps the most intriguing outcome, though, would come if developers from Android, the Journal, and others with an interest in structured logging could get together and agree on what the next generation of logging should look like. Their solution would be likely to start with a critical mass of users from the outset, improving its chances of being merged and adopted by others. Given the number of structured logging efforts that have floundered in the past, getting a widely-accepted solution in place would be a major accomplishment.
Patches and updates
Kernel trees
Architecture-specific
Core kernel code
Development tools
Device drivers
Filesystems and block I/O
Memory management
Networking
Miscellaneous
Page editor: Jonathan Corbet
Distributions
RHEL clones: a surfeit of riches
For better or for worse, Red Hat's Enterprise Linux (RHEL) is the standard form of Linux in many settings. Red Hat invests considerable resources into stabilizing RHEL and supporting it for many years after its initial release. That gives comfort to companies fearing the choice between a forced migration to a newer version of the distribution or having to take on the burden of maintaining it themselves. RHEL is the default choice in many situations where all software must come accompanied by a long-term support contract; it is telling that Red Hat's two closest competitors (SUSE and Oracle) both offer support for Red Hat's distribution.RHEL is built with free software, and Red Hat fully understands both the conditions attached to free software licenses and the expectations of the development community. So the source for each RHEL release is promptly made available to the community. It is natural to expect that plenty of people would like to take advantage of the stability of and support behind RHEL without actually paying for a support contract; the availability of the source makes it possible to do exactly that. All that is needed is to rebuild those source packages, minus any Red Hat branding, and make the result available to the community.
Except that, in reality, the problem seems to be harder than that. Creating a proper build and distribution environment requires work and equipment. Quality control can be a lot of work, especially if strict binary compatibility with RHEL is desired. The security update stream must be followed constantly. And Red Hat does not always make the job of keeping up with their releases easy. So anybody wanting to make and support a proper RHEL clone must be prepared to invest a lot of time and infrastructure into the task.
Given that the desired end result of any RHEL rebuild effort - something that looks as much like RHEL as possible - is clear and unvarying, one would think that there would not be room for a large number of RHEL rebuild projects. There is not much space for ego or interesting new development, so it would make sense for everybody with an interest in this area to work together for the best end result. That is not how things have worked out, though. One need not look to far to find plenty of rebuilds out there:
- Oracle
Linux is the most notorious of the commercial rebuilds. For the
most part, Oracle has been working from the RHEL base with an eye
toward binary compatibility. Over time, the company has started to
add in some features of its own, including a newer kernel.
- CentOS is arguably the largest and
best established of the free rebuild projects. Many commercial
hosting providers
offer CentOS installs, even if they have no other RHEL clones
available. CentOS is clearly successful, but it is not the most
community-oriented of projects. The project's leadership is jealous
of its user base and seemingly unwilling to open things up to
outsiders. At times the CentOS developers have failed spectacularly
to keep up with Red Hat's releases, with the result that CentOS users
have seen prolonged periods without security updates. That said, the
rapid release of CentOS 6.2 suggests that the situation is improving.
- Scientific Linux has a
significant and growing user base; it also has a small core of paid
developers. This distribution has arguably done a better job of
staying on top of security updates in recent times, but, as of this
writing, it has not caught up to the Red Hat 6.2 release.
- Ascendos claims to be "
is a free, community-supported, and professional-grade Linux platform
". It is also vaporware, with no actual releases to date. Its development was initially led by Troy Dawson, who has also worked with Scientific Linux, but a job at Red Hat evidently put an end to that work. Troy's replacement, Douglas McClendon, stepped down in mid-December. Despite its claimed community orientation, Ascendos does not appear to have attracted many other developers, so the project now appears stalled. - GoOSe, on its surface, looks a
lot like Ascendos. It, too, claims to be "
all about the community
", but has not yet created much of an actual development community. The project had hoped to get a 6.0 alpha release out by the end of the year, but that did not quite happen.
There have been discussions recently about cooperation between Ascendos and GoOSe, but nothing has been announced yet.
- ClearOS is a rebuild "
built on source code from a prominent North American Linux vendor
" offered by a group called "Clear Foundation." There is a lot of talk of community on Clear Foundation's web site, but "community" seems to consist mainly of access to a set of forums. By all appearances, ClearOS is meant to be a vehicle for a series of commercial support operations. The current ClearOS offering is based on RHEL 6.0, with a 6.2 release available as a beta.- Fermi Linux is a rebuild of Scientific Linux. Its current 6.1 release adds a number of packages and tweaks some settings.
- PUIAS Linux is a rebuild published by Princeton University. It is currently based on RHEL 6.2 with a number of additional packages.
- ClearOS is a rebuild "
There are undoubtedly others, but, at this point, the picture should be clear: a lot of independent groups are putting a lot of effort into their own rebuilds of Red Hat Enterprise Linux. Some of them are more successful than others, but none are as good as they could be with a bit more focused effort.
Given the long list of existing distributions and the effort required to create and maintain a new one, anybody thinking of adding to the list should think long and hard about why that seems like a good idea. If the intent is to provide a Linux experience that is not possible with any of the existing distributions, perhaps there is an excuse for making a new one. But, in the case of RHEL rebuilds, there is simply no latitude for the creation of that new experience. The desired result is something that looks and acts like RHEL. Perhaps the various groups making their own RHEL clones would get better results if they were to build a single base distribution to work from. They could then compete fiercely to provide the slickest desktop theme, which is where all the interesting action is anyway.
Brief items
Distribution quote of the week
Admittedly, Microsoft are now trying to use the patent system as fungicide, but I think the wider population are waking up to just how toxic that stuff is for everyone, especially when misused.
Extremadura dropping Linex
Back in 2006, LWN carried the announcement that the government of the Spanish state of Extremadura was moving to a locally-developed distribution called Linex. Now comes the sad news that the project has been ended, along with the contracts of the developers who were working on it. One of those developers, José Luis Redrejo Rodríguez, has posted his view on what has happened. "The new people in charge of the Extremadura government don't like the good press and name that Linex, and the free software, gave to the previous party in the government. And they want to change things. I don't say they're going to remove all the free software we have in education (I don't think that's technically possible, and also we can not afford it ), but they maybe will move from Debian to Ubuntu or to OpenSUSE or Fedora. They are firing all the people who made the previous situation possible." (Thanks to Paul Wise).
Linux Mint introduces Cinnamon
The Linux Mint Blog is carrying an introduction to "Cinnamon," that distribution's latest desktop initiative. "With Gnome 2 no longer an option we lost one of the most important upstream components our Linux Mint desktop was based on. Our entire focus shifted from innovating on the desktop, to patching existing alternatives such as Gnome Shell. We used MATE and MGSE to provide an easier transition away from Gnome 2, but without being able to truly offer an alternative that was better than Gnome 2. Both MATE and Gnome Shell are promising projects but MATE’s ultimate goal is to replicate Gnome 2 using GTK+ and Gnome Shell doesn’t provide what we need in a desktop and is going in a direction we do not want to follow. So for these reasons we’re designing a new desktop called Cinnamon, which leverages new technology and implements our vision."
Gentoo Linux releases 12.0 LiveDVD
Gentoo Linux has announced the availability of a new LiveDVD. As a source based distribution a Gentoo install can be easily kept up to date. If you don't already have Gentoo installed and would like to try it out, the LiveDVD is a good way to get started.
Newsletters and articles of interest
Distribution newsletters
- Debian Misc Developer News (#27) (December 23)
- DistroWatch Weekly, Issue 437 (January 2)
- Fedora Weekly News Issue 288 (December 21)
- Maemo Weekly News (January 2)
Page editor: Rebecca Sobol
Development
Kdenlive 0.8.2 adds effects and monitoring tools
Video editing in open source is a tumultuous space; projects come and go, and when they stick around they have a tendency to embark on sudden ground-up rewrites and media-framework transplants. But the dirty little secret is that that situation is just as true in the "professional" proprietary market as it is for those of us who stick with free software. Fortunately, while ambitious efforts like Lumiera remain experimental, the formerly low-end nonlinear video editors (NLEs) are closing the feature gap by adding steady, small improvements. Case in point is the KDE-based NLE Kdenlive, which released its most recent update in October 2011. Version 0.8.2 adds new audio/video inspection and monitoring tools, as well as effects that offer real editing functionality — not mere eye-candy. The high end of the market is not as safe as it used to be.
The 0.8.x series introduced a slew of new features — mostly in the "special effects" vein, but some that contribute to better project management and usability, too. The project wiki maintains a near-exhaustive list of build and installation options for users on every platform supported by KDE, including multiple Linux distributions. I opted for the Launchpad personal package archive (PPA) maintained for Ubuntu by Olivier Banus — it was "strongly recommended" over the official repositories, although exactly why is not explained. The primary package dependency to worry about is the low-level video editing framework MLT, which is also used by the GTK+-based OpenShot NLE (and is also provided by most of the project's distribution-specific package repositories).
On the whole, Kdenlive conforms to the same basic user-interface conventions shared by every major NLE. Each editing project you start consists of a set of video, audio, and other media "clips" that you select, and a timeline on which you drop, rearrange, and align those clips. Your experience in the editor is defined by manipulating items on the timeline, and any changes or effects you make to individual clips on the "clip monitor." When you are satisfied with your results, you render the final output into the format you want.
OpenShot, PiTiVi, and Kdenlive all offer largely comparable approaches
to this workflow, and have roughly the same level of support for the video
input and output formats most people need. Where they differ is in the
details: the ability to create special effects, the smoothness of
manipulating clips and settings, and support for esoteric non-video content
that you might want to generate for use in your production.
The canonical example of this is title cards; most NLEs allow you to
assemble text and images into title elements directly within the editor
(albeit with varying degrees of sophistication), rather than forcing you to
create them in an external application. Unlike some of the others,
Kdenlive can also generate
countdown sequences, clips of solid-color backgrounds, still-image
slideshows, and static "noise."
Via MLT, Kdenlive uses a plug-in API called Frei0r for its audio and video effects. The 0.8.2 release includes several hundred effects, ranging from simple audio/video corrections, to image transformations, to decorative effects. Over the years, Kdenlive has also added a series of inspection and monitoring tools that help you get a better handle on the contents of your media files: you can monitor audio waveforms and color scopes as you skip through your video, which makes editing far more precise than relying on pointing your own eyes and ears at the preview window.
What's new
I am not a video professional, but, based on discussions with friends and relatives who are, it is these sorts of effects and monitoring options that make up some of the biggest differences between "consumer" and "professional" NLEs. For example, a lightweight editor will allow you to cut and rearrange clips to your heart's content, but without a way to adjust the colors so that two clips shot in different locations have the same skin tones, the lightweight editor will never be useful for any kind of professional-looking output.
On that front, I am pretty pleased with what Kdenlive currently offers. For video, you have control over hue, contrast, and saturation, complete with adjustment curves (which, as with photographs, allow you to vary the amount of adjustment in different parts of the image) and automatic balancing. The various "scopes" for monitoring a clip include histograms, waveforms, and a vectorscope. For audio, there are standard VU meters, spectrum and spectrogram monitors, plus a dizzying assortment of audio filters.
Among the new special effects are some novelty techniques such as a stop-motion animator and a light-painting tool, but there are others with more practical aims. The first is a perspective image placement tool, which allows you to put a rectangle onto a clip and distort its sides and corners until it appears to match the perspective of other in-frame surfaces. You can then composite a still image (or another video clip) into the properly-aligned rectangle and have it render as if it were physically present in the scene. The second is named rotoscoping, but it essentially just allows you to mask off areas of the screen by drawing Bezier curves right onto the monitor. You can draw such masks on several keyframes, and the effect plug-in will interpolate on the intermediary frames. With a little feathering (i.e. an alpha-transparent border) added to the edges of your mask, you can replace objects or backgrounds. Both of these are effects that proprietary NLEs have offered (on other operating systems) for some time, but their addition to Kdenlive is another sign that the program is growing up.
Also in the "growing up" category are some changes to the user interface and project-management features. The biggest is that the individual components of the UI are all detachable and can be rearranged at will. For starters, this allows you to work around older annoyances like the inability to see the "clip monitor" and "project monitor" at the same time (as noted by editor Jonathan Corbet in 2008). But it also enables you to assemble separate layouts for specific tasks and save them for later reuse, so you can activate all of the audio monitoring options for an "audio editing layout" or set up video monitors for a "color correction layout." There are a lot of monitors in 0.8.2, and fitting them all on-screen at once is not feasible.
The project-management features I found welcome include a "notes" pane which allows you to keep text notes on your work directly within the project (a method that definitely beats maintaining them in a separate file that you periodically misplace), and something called "proxy clips" in the main editors. Proxy clips are CPU-friendly transcoded versions of your original clip which are shown in the effects and preview windows instead of the original, thus saving CPU time and speeding up the responsiveness of the entire UI. When a proxy clip is rendered, Kdenlive uses the original clip, but working on the proxy clip makes life easier at every step up to the final one. In a take-a-moment-to-think-about-it twist, the proxy clips are usually larger in size than the original — it is the highly-compressed HD video formats that consume lots of CPU, after all. Proxy clips decoded to something simpler like vanilla MPEG-2 use more bytes, but don't require overhead for every frame.
The final cut
Kdenlive has evolved considerably since I last took it for a drive (which if I recall correctly, must have been sometime during the 0.6.x series). There are, however, several areas where I still find the editing experience painful. The first is that many of the mission-critical UI widgets are tiny. Although the toolbar buttons for generic items like Open, Save, and Undo get a full 32 pixels each, the timeline's playback point (which you must grab and slide in order to change positions in your movie) is barely 8 pixels high. That is better than the playback point in the clip monitor, however, which is 5 pixels high. Most of the tool buttons on the monitors and timeline are larger than that, but they are still microscopic by GUI standards, which makes them harder to hit and harder to tell apart.
I also found the effects interface inconsistent. It uses bottom-tabs (which is atypical to begin with), but activating an effect is the real issue. A checkbox next to each available filter would be simple and unambiguous, but to activate an effect, you must select it from the "Effect List" tab, then drag it onto the clip on the timeline that you wish to apply it to — after which, it will appear in the "Effect Stack" tab back in the effects window. There is a separate interface for activating and editing transition effects, by means of pulsating icons that appear in the timeline only when you pause the cursor over the right spot. The transient pulsating buttons appear in a few other places when working with the timeline, such as when resizing a clip. That does not make sense to me either; in all of these cases there is nothing to be gained by hiding the button in some circumstances, nor does the animation make the button easier to see or hit.
Ultimately, as is the case with the wildly-varying button sizes, the transient pulsating buttons just contribute to an inconsistent look across the application as a whole. Some of the icons are flat in style, some are 3D and rounded, some have boxes drawn around them and some have circles. All of the new monitors use a black canvas as a background, while the rest of the interface uses the widget library's default color. It may be unfair, but little details like that can turn off potential new users, particularly artsy ones like video editors.
The good news is that Kdenlive is making steady progress in adding the mid-to-high-end features that it will need to eventually make a play for the "pro-sumer" NLE user. That pro-sumer user is not just the Linux fanatic patiently waiting on Lumiera, either — there is a great opportunity in the NLE market right now on every platform. Apple alienated quite a few of its own professional users when it released Final Cut Pro X in June 2011, which many customers considered a downgrade. Certainly lots of them simply jumped ship to Adobe Premiere, but the competition got a thorough examination. From the other direction, EditShare announced in 2010 that it would release a cross-platform version of its flagship Lightworks NLE as open source, but it has subsequently pushed back the release date to "to be announced." Meanwhile, the existing open source projects — including Kdenlive — keep on plugging away, release by release, feature by feature.
Brief items
Quotes of the week
Calypso - CalDAV/CardDAV/WebDAV for Android and Evolution
Keith Packard announces the launch of a new calendar server project. "I started hacking at Radicale to see how far I could get. I changed the storage code to store one event per file, then added hooks to use git for change management. Then, I found a full vcalendar/vcard parsing library in python, vobject, which I used to replace the ad-hoc parsing code. Finally, I added support for VCARD entries as well, allowing the system to store both calendar and contact information."
GNU ed 1.6 released
For all of you who like to complain that *Office is too big and bloated: version 1.6 of the venerable GNU "ed" editor has been released. There's not much in the way of new features, but a number of fixes have been made and it is now possible to use regular expressions containing NUL characters.Apache Hadoop 1.0 released
The Apache Software Foundation has announced the release of Hadoop 1.0. "A foundation of Cloud computing and at the epicenter of "big data" solutions, Apache Hadoop enables data-intensive distributed applications to work with thousands of nodes and exabytes of data. Hadoop enables organizations to more efficiently and cost-effectively store, process, manage and analyze the growing volumes of data being created and collected every day. Apache Hadoop connects thousands of servers to process and analyze data at supercomputing speed."
Jato 0.3 released
Version 0.3 of the Jato Java virtual machine is out. This release includes a number of performance and portability improvements; it is now claimed to be able to run JRuby, Clojure, and Eclipse on 32-bit x86 systems; 64-bit x86, PowerPC, and ARM support are in the works but not yet ready in this release.Scribus 1.4 released
Version 1.4 of the Scribus desktop publish system - the first major stable release in four years - has been announced. 1.4 is based on Qt4 and offers much more extensive undo/redo operations, lots of new import filters, better color management, and a lot more. "Now that Scribus 1.4.0 has been released, the Scribus Team will focus on stabilizing the 1.5 development branch, which will comprise amazing new features like support for PDF/X-1a, PDF/X-4 and PDF/E, Mesh Gradients, native PDF import, XAR import, a completely rewritten table implementation, a rewritten text system, and much more."
The "White Label Office" release candidate
Team OpenOffice.org has muddied the office suite waters further with its announcement (with a longer version [PDF]) of "White Label Office," a release candidate consisting of OpenOffice.org 3.3.0 with a number of fixes applied. "By publishing White Label Office 3.3.1, Team OpenOffice.org is taking the first step towards a maintenance release for OpenOffice.org 3.3.0. The release candidate is intended to serve as a starting point for creating the best possible version in collaboration with the users." Needless to say, folks in the Apache OpenOffice project are not amused.
Newsletters and articles
Development newsletters from the past two weeks
- Caml Weekly News (December 27)
- Caml Weekly News (January 3)
- Perl Weekly (December 26)
- Perl Weekly (January 2)
- PostgreSQL Weekly News (December 25)
- PostgreSQL Weekly News (January 1)
Queru: Another year of AOSP
Jean-Baptiste Queru reflects on the ups and downs of the Android Open Source Project in 2011. "Not releasing the Honeycomb source code was catastrophic for the AOSP community. I had never before received so many angry emails, so many threats, to the point where I had to take several weeks off at some point to get away from it. Even today, there's a lot of bitterness left on all sides. From start to finish, Honeycomb probably cost AOSP anywhere from 6 to 12 months."
Version 1.0 of the Clementine music player arrives (The H)
The H covers the release of version 1.0 of the Clementine music player. "The major update adds support for the Spotify and Grooveshark music streaming services. A Global Search feature has been added that allows users to find music on their local system or on the internet. Other changes include audio CD support, and improvements to the settings dialog and album cover searches, as well as the addition of more transcoder options. A number of bugs found in the previous versions have also been fixed." A brief announcement can be found on the project's website. See the changelog for more detailed information.
Page editor: Jonathan Corbet
Announcements
Brief items
Mozilla Public License 2.0 released
The Mozilla Project has announced the final release of version 2.0 of its Mozilla Public License. "Version 2.0 is similar in spirit to the previous versions, but shorter, better, and more compatible with other Free Software and Open Source Licenses."
Apache OpenOffice Announcement List
The Apache OpenOffice Project Management Committee has announced the availability of a new Announcement mailing list. "This is a low-volume list which will be used to announce the availability of OpenOffice betas and releases, to issue security bulletins, and to report similar relevant information. This is not a discussion list. It will be used only for authoritative announcements regarding Apache OpenOffice."
Articles of interest
Doctorow: The coming war on general-purpose computation
Cory Doctorow's 28C3 talk was called "the coming war on general-purpose computation." It's available as a video; there is also a transcript on Github. "And it doesn't take a science fiction writer to understand why regulators might be nervous about the user-modifiable firmware on self-driving cars, or limiting interoperability for aviation controllers, or the kind of thing you could do with bio-scale assemblers and sequencers. Imagine what will happen the day that Monsanto determines that it's really... really... important to make sure that computers can't execute programs that cause specialized peripherals to output organisms that eat their lunch... literally. Regardless of whether you think these are real problems or merely hysterical fears, they are nevertheless the province of lobbies and interest groups that are far more influential than Hollywood and big content are on their best days, and every one of them will arrive at the same place -- 'can't you just make us a general purpose computer that runs all the programs, except the ones that scare and anger us? Can't you just make us an Internet that transmits any message over any protocol between any two points, unless it upsets us?'"
FSFE: Fellowship Interview with Paul Boddie
Chris Woolfrey interviews Paul Boddie on behalf of the Fellowship of the Free Software Foundation Europe. "Paul Boddie: I’m writing applications that access and combine databases in the bioinformatics domain, although I also have an ongoing project dealing with the text-mining of biological and medical literature to extract gene and protein-related information. I also have a fair amount to do in terms of maintaining and hopefully developing further some software written in the group in which I work. We have to make the work available, so there’s a need to publish information on the web. It ends up as a mix of software development, some maintenance tasks, and a degree of web publishing and content management. I’m working on a range of personal projects, too, which includes things like the EventAggregator that is used on the FSFE Fellowship Wiki, a selection of MoinMoin-related extensions (which is why I was able to get involved a bit more with the Fellowship Wiki) and some longer term interests in things like source code analysis and compilation for dynamic programming languages like Python."
Calls for Presentations
CeBIT '12: call for papers and projects
CeBIT 2012 will be held March 6-10 in Hannover, Germany. Linux New Media is hosting the Open Source Forum at CeBIT; the Forum's Call for Projects deadline has passed, but the deadline for the Call for Papers was extended until January 8.REMINDER: OSCON Call for Proposals (deadline 1/12)
OSCON (O'Reilly Open Source Convention) will be held in Portland, Oregon July 16-20, 2012. This a reminder that the call for proposals ends January 12.
Upcoming Events
SCALE 10X Resolutions for 2012: Let SCALE help
SCALE 10X (Southern California Linux Expo) will be held January 20-22, 2012 in Los Angeles, California. Registration is still open and special room rates at the Hilton Los Angeles Airport are still available. Click below for a look at some of the events and presentations scheduled for SCALE 10X.Events: January 5, 2012 to March 5, 2012
The following event listing is taken from the LWN.net Calendar.
| Date(s) | Event | Location |
|---|---|---|
| January 12 January 13 |
Open Source World Conference 2012 | Granada, Spain |
| January 13 January 15 |
Fedora User and Developer Conference, North America | Blacksburg, VA, USA |
| January 16 January 20 |
linux.conf.au 2012 | Ballarat, Australia |
| January 20 January 22 |
Wikipedia & MediaWiki hackathon & workshops | San Francisco, CA, USA |
| January 20 January 22 |
SCALE 10x - Southern California Linux Expo | Los Angeles, CA, USA |
| January 27 January 29 |
DebianMed Meeting Southport2012 | Southport, UK |
| January 31 February 2 |
Ubuntu Developer Week | #ubuntu-classroom, irc.freenode.net |
| February 4 February 5 |
Free and Open Source Developers Meeting | Brussels, Belgium |
| February 6 February 10 |
Linux on ARM: Linaro Connect Q1.12 | San Francisco, CA, USA |
| February 7 February 8 |
Open Source Now 2012 | Geneva, Switzerland |
| February 10 February 12 |
Linux Vacation / Eastern Europe Winter session 2012 | Minsk, Belarus |
| February 10 February 12 |
Skolelinux/Debian Edu developer gathering | Oslo, Norway |
| February 13 February 14 |
Android Builder's Summit | Redwood Shores, CA, USA |
| February 15 February 17 |
2012 Embedded Linux Conference | Redwood Shores, CA, USA |
| February 16 February 17 |
Embedded Technology Conference 2012 | San José, Costa Rica |
| February 17 February 18 |
Red Hat, Fedora, JBoss Developer Conference | Brno, Czech Republic |
| February 24 February 25 |
PHP UK Conference 2012 | London, UK |
| February 27 March 2 |
ConFoo Web Techno Conference 2012 | Montreal, Canada |
| February 28 | Israeli Perl Workshop 2012 | Ramat Gan, Israel |
| March 2 March 4 |
Debian BSP in Cambridge | Cambridge, UK |
| March 2 March 4 |
BSP2012 - Moenchengladbach | Mönchengladbach, Germany |
If your event does not appear here, please tell us about it.
Page editor: Rebecca Sobol
