User: Password:
Subscribe / Log in / New account Weekly Edition for January 23, 2014

Wikipedia reconsiders H.264 support

By Nathan Willis
January 22, 2014

As a matter of policy, Wikipedia has historically opposed using the H.264 video codec, because the organization's mission dictates the use of open data formats. But the Wikipedia Foundation multimedia team has now opened a request for comments (RfC) asking the community's input as to whether or not this stance should be reconsidered—specifically, whether to make H.264 video a fallback option for users who cannot view open formats. The rationale for the proposal incorporates the difficulty of creating (in addition to viewing) open video formats, but there are still quite a few community members who oppose the idea of any H.264 support.

Pros and cons

The RfC was opened on January 15 and will remain open until February 14. At that time, the multimedia committee will consider the results of the public comments before it makes its decision, although the wording of the RfC indicates that the public comments will be merely advisory. The document leads off with a summary of Wikipedia's current video situation, which led to the consideration of a policy change. It then summarizes the basic arguments for and against adding H.264 support.

The impetus for the discussion is, essentially, that video has yet to make a significant impact on the overall Wikipedia experience. Despite the benefits that video resources could bring to a variety of topics (as well as to users who cannot read), so far only 40,500 video files have been uploaded to the project. That constitutes about 0.2% of the content at Wikimedia Commons and, the RfC points out, pales in comparison to the number of educational videos contributed to YouTube (at about 6.5 million).

One possible explanation is that Wikipedia's insistence on supporting only open video formats (Ogg Theora and WebM) hampers video contribution. Few readily-available devices can record Theora or WebM, and few applications can edit or export to either format. The situation for viewing open video formats is just as bad; one third of Wikipedia page hits come from mobile browsers (which by and large do not support Theora or WebM playback), and a significant portion of desktop browser users (namely those running Safari or Internet Explorer) cannot play back the open formats either.

As most people who follow the multimedia codec space know, the situation is almost the reverse for H.264: mobile phones and video cameras can record directly to H.264, video editing software support is widespread, and the majority of users can view it on the desktop and on mobile devices.

The problem, of course, is that H.264 is proprietary and patent-encumbered. Wikipedia's legal department has decided that the H.264 licenses required would be acceptable; essentially the project would need to implement H.264 encoding and decoding for videos uploaded to Wikimedia Commons, but Wikipedia would not have to pay license fees to publish H.264 videos, nor would readers have to pay fees to view them.

The basic plan being offered for comment would automatically transcode all video uploads, so that every video is available in both H.264 and in an open format. The open formats would be the default, but would fall back to H.264 for browsers and devices without support for Theora and WebM.

This plan, the RfC emphasizes, would also transcode H.264 uploads into an open format—not just transcode open formats to H.264. This is to ensure that users opposed to supporting proprietary video would not be locked out even if the original uploader used H.264. Nevertheless, four options are put forth for community comment: fully supporting both H.264 and open formats, supporting H.264 only for viewing, supporting H.264 only for uploads (i.e., automatically transcoding uploads to an open format), and not supporting H.264 at all.


The various arguments for and against H.264 support are discussed in more detail in the RfC (as is true of Wikipedia "talk" pages, most of the comments are signed, but unfortunately there are not direct links to them). Essentially, though, no variation of the proposed change involves dropping support for open video formats or even for dropping them as the default. Thus, the major arguments against adding H.264 support are not about whether or not H.264 is preferable to the open formats or whether open formats should be abandoned; rather, they are arguments about whether the compromise to support H.264 is a net gain or loss. Since the answer to that question requires peering into the future, it is naturally a bit speculative.

For example, one argument against adding H.264 support is that the open formats are catching up to it. First, Google has secured support for VP9 (the successor to WebM's current codec, VP8) from a long list of hardware vendors, so the next generation of mobile devices will be able to record and edit VP9 natively. Second, the development of's new Daala video codec is being staged out to compete head to head with future proprietary video codecs that are still several years down the road. Many attribute H.264's dominance over VP8 to the fact that the proprietary codec had a head start. Theora was never really competitive in that fight, as it was an even older codec.

But with Daala's opportunity to leapfrog the proprietary alternatives, the argument goes, the future actually looks quite good for open formats, so Wikipedia backing down on its stance against them would be detrimental. The flip side was argued by Michael Dale, who said that's interoperability with proprietary formats was what allowed it to gain a significant foothold. The usage of free formats cannot grow on principles alone, he said, "Interoperability is the only way to grow our video platform that has free format support."

Of course, there are other arguments against compromise. One mentioned in the RfC is that Wikipedia has a commitment to open source software in addition to open content; deploying H.264 transcoders on Wikipedia's servers could mean adding a component that Wikipedia would have trouble freely distributing to downstream users—since those users would likely still be required to purchase an H.264 license. The foundation prefers not to distribute components that might place users in such a bind, the RfC notes, although, in fact, Wikipedia's Director of Platform Engineering Rob Lanphier said in an IRC discussion: "I don't think we can say for sure we'll be using free software for this."

The soap box

Anyone with a viewpoint to share on the proposed plan is invited to add a comment; the RfC is not limited to registered Wikipedia users or contributors. As of this writing (about one week after the RfC was posted), 421 signed statements have been added by the public; more than half (245) oppose any H.264 support, while 123 are in favor full upload and playback support, 44 favor supporting H.264 uploads only, four favor enabling H.264 playback only, and five describe themselves as completely neutral.

There have been a few alternate proposals as well, such as supporting playback only for the H.264 "baseline" profile (the patents for which will be the first to expire) or partnering with YouTube (so that YouTube delivers video content for Wikipedia pages). Few of these suggestions have garnered any significant support.

Of course, it is also difficult to adjust the numbers for how well the voluntary public comments represent the overall views of the Wikipedia community. Since there is a major ideological component to opposing H.264 support, one would certainly expect a more vocal crowd to weigh in—but that does not mean that the opposition viewpoint is the real majority.

But the RfC is not a ballot measure being put up for a vote; the Wikipedia Foundation will make its own decision—it is simply asking for a public discussion of the issues. One of the benefits of the public discussion should be that additional viewpoints and concerns come out that were not raised by the multimedia team itself, and that appears to be the case. Several of the comments raise important side issues. For example, user Alterego asks:

The real RFC should be: do we want to change the entire culture of the Wikimedia ecosystem to be video centric and not text centric? The success of the Foundation has arguably been due to its focus on text, and making it easy to upload video is going to represent a HUGE cultural and content shift.

Other commenters have asked whether or not supporting H.264 could result in a tidal wave of low-quality video uploads, which would consume far more volunteer time and server resources to vet than do text or image submissions. Of course, the quality level of Wikipedia's text articles is the subject of frequent debates too, but the original point still stands: video itself may be desirable, but a large influx of it would pose practical challenges regardless of which codecs are used.

The other wrinkle to the discussion is that there is not a viable alternative plan that would substantially alleviate the initial concerns: the dearth of video content uploads, the difficulty of editing and converting Theora and VP9, and the inability of many Wikipedia users to view the open video formats. In other words, if Wikipedia rejects the idea of H.264 fallback support, it will still have to look for another solution—and so far no one has proposed one that has any traction.

In a sense, the community comments fall into camps that disagree on the most fundamental aspect of the question: those favoring H.264 support do so on practical grounds, while those opposing H.264 support do so largely on philosophical grounds. That distinction about the nature of the debate makes it difficult to imagine the two sides reaching a consensus in the middle. Thus, while the multimedia committee no doubt values the open discussion, that discussion is unlikely to resolve itself into an answer. The team will make its own decision, using whatever balance of practical and philosophical merits it chooses.

The greater free software world is no stranger to this debate; in 2012, Mozilla enabled fallback support of OS-level H.264 decoders for many of the same reasons articulated in the Wikipedia debate. If that is a valid predictor of the general temperature, the safe bet would be for Wikipedia to adopt some form of H.264 support, at least for video playback. Those with strong ideological opposition to that possibility could perhaps take comfort in the fact that Mozilla, true to its word to "fight another day," has indeed devoted considerable effort to making the next generation of free video codecs a best-in-class option. No doubt, though, that the next-generation codec battle will be just as contentious as this one.

Comments (20 posted)

US government opens access to scientific research

By Nathan Willis
January 22, 2014

On January 16, the US Congress passed its annual "omnibus" spending bill for the year, which included a provision that mandates the open access publishing of a large swath of scientific research that is funded by taxpayers. Estimates say that the proportion of research projects that will be released under the new provisions account for just over half of the government's total research funding. Although the importance of open access is easy to lose among all of the various "open x" movements that have arisen since the rise of open source software, the new policy is one with ramifications for the open source community—given how much scientific research these days is conducted with software.

Officially, the omnibus bill is known as the Consolidated Appropriations Act for 2014, which is available in the form of a 1,582-page PDF for those brave enough to download it. The important section, however, is on page 1,020, which the Electronic Frontier Foundation (EFF) kindly extracted into a standalone download [PDF] for the organization's own blog entry about the new policy.

The rule applies to those agencies under the Departments of Labor, Health and Human Services, and Education which have annual research budgets greater than $100 million, and stipulates that they must release to the public the final manuscripts of all research that is fully or partially funded by the government—and that they must do so no later than twelve months after the research is published in a peer-reviewed journal. The rule also specifies that the publication must be online, free, and published in a machine-readable format.

The three departments listed may not sound like the nexus of scientific research (the Department of Energy, for example, is home to a lot of research though its National Laboratories), but those departments do include some key agencies, such as the Food and Drug Administration (FDA), Centers for Disease Control and Prevention (CDC), National Institutes of Health (NIH), and Occupational Safety and Health Administration (OSHA). Certainly a lot of scientific research is funded through other agencies (including NASA and the National Science Foundation), but the section of the omnibus bill including the new research rules applies only to this set of departments. In its analysis of the bill, though, the Scholarly Publishing and Academic Resources Coalition (SPARC) says that the covered agencies (that is, excluding those agencies that fall below the $100 million limit) total around $31 billion out of the government's $60 billion annual research investment.

The new rule builds on two similar moves from recent years. In February 2013, the White House published an "open access memo" directing federal agencies to develop a plan for publishing publicly-funded research; the law essentially makes that plan mandatory for the specific agencies it covers. But even that 2013 memo is seen as the successor to the successful public access policy that the NIH voluntarily started in 2008. That policy is regarded as one of the cornerstones of the Human Genome Project and several subsequent NIH projects.


Naturally, there is an easy case to be made for publishing this research simply because it is funded by the public's tax dollars. That seems "fair" by most definitions, and indeed the open source community has repeatedly shown that it can take taxpayer-funded public releases and transform them into something with more value than the original—the Department of Veterans Affairs' electronic health platform VistA, for example, or OpenStreetMap's incorporation of the Census Bureau's TIGER map database.

But scientific research is special in other ways that make the new policy particularly important. Restricting access to published research hampers attempts to build on previous results and makes it difficult or impossible for independent researchers to verify a study's findings. Those principles are key to the "open access" movement as it is defined by the Public Library of Science (PLOS) and similar initiatives.

In most cases, however, the main voice opposing open access publication is not the researchers themselves, but rather the publishers of the scientific journals where the research is found. The journal publishers typically claim copyright over the author's articles, and only allow access to them for a fee. Since publication in a well-known journal is typically a prerequisite for considering the research validated (not to mention ensuring job security for the researcher), the publishers can get away with it, and will take extreme measures to protect their access-fee revenue.

In December, for example, The Washington Post reported that journal publisher Elsevier sent out thousands of copyright-infringement takedown notices to researchers and universities for making their own research available on their own web sites. Most complied, although there was considerable blowback against the company.

More data

The practical effect of the new regulation, therefore, is to free a great many researchers from draconian publishing contracts that prohibit publicly sharing the researchers' work; US government funding underwrites so much research that few if any journals could risk running afoul of its rules. That is undoubtedly a good thing for the research community, but for many it is not sufficient. First, the new rule only applies to the final paper that is published; it does not mandate that the raw data, for example, also be published. Admittedly, in most cases, if researchers are amenable to freely publishing a study, they are not likely to oppose publishing the data with it, but it is still optional.

Second, the rule only requires that the published research be online, machine-readable, and free; it does not mandate any document formats. Open access advocates emphasize that publishing in open formats is important to making a study genuinely accessible. That concern is obviously more relevant for publishing a data set than a manuscript, but PLOS and other open access proponents want to see both changes.

Finally, the EFF and others advocate the passage of a separate law, the Fair Access to Science and Technology Research (FASTR) Act, which would not only broaden the scope of the policy, but would be more resistant against change than the White House memo. The memo, they note, covers agencies not addressed by the new omnibus bill, but it is only a memo—the next administration can undo it with another memo.

The new open access policy has the potential to release a considerable volume of research studies to the public in the coming months and years. Modern scientific research is a field that relies heavily on open source: data mining, statistical analysis, and simulation tools, for example. As more and more research reaches the public that previously had no access to it, one can expect the software community to continue playing a key role in the resulting conversations—validating and reproducing results, correlating studies, and so forth. Consequently, an increase in open access research may lead to more software contributions from researchers, although they are likely to be limited to specialized tools.

Just as interesting, though, will be seeing how the academic publishers react to the changes in their business model—and what effect they will have on the continuing debate about data publication and open formats. It took the open source community years to sort out community norms for critical questions like license interoperability, attribution and sign-off, trademarks, business models, and distributed workflow—in fact, many would probably argue that most of these issues have yet to be "sorted out" entirely. So the scientific research community may contribute code, but the open source community will likely be called upon to contribute back in other ways, too, as the open access community grapples with its own set of analogous problems.

The open source community has certainly had its share of false starts, blind alleys, and fiascoes over the years; perhaps open access research will fare better by learning from the software community's example. But either way, open access is poised to become a major public issue, and its overlap with open source will no doubt have long-term consequences.

Comments (9 posted)

A Debian init system update

By Jonathan Corbet
January 21, 2014
When LWN last looked at the Debian Project's init system debate, the issue had just been referred to the project's Technical Committee for resolution. The hope was that a ruling from the committee would bring to an end a debate which has been long and acrimonious, even by Debian's standards. The committee has yet to call a vote as of this writing, but the early signs suggest that the project may not get the clear answer it was hoping for.

The discussion on the debian-ctte list has been long, detailed, and only occasionally heated. Over time, the committee members have revealed their positions on the matter. Russ Allbery came out in favor of systemd just before the end of the year; since then, he has been joined in that position by committee chair Bdale Garbee, brand-new member Keith Packard, and (with weak support) Don Armstrong. Members who have identified themselves as being in the upstart camp are Ian Jackson, Colin Watson, and Andreas Barth. The only member of the committee who has not posted a summary of his position is Steve Langasek, but, given that he is the Debian upstart maintainer, there are few doubts as to where he will land.

The posted positions have led to a complaint that the OpenRC option has not been fairly considered by the committee. OpenRC, it is claimed, is the only init system that brings new features while working on all of Debian's target architectures. This appeal does not appear to have changed many minds, though; the prevailing opinion appears to be that OpenRC, while having a number of nice attributes, is too far behind and lacks a development community large enough to catch up in the near future.

A divided committee?

Perhaps the more compelling question, though, is this: what happens if the committee ends up hung on a four-to-four split? Section 6.3.2 of the Debian Constitution states that the chair of the committee has a casting (tie-breaking) vote. So, if the committee is evenly split on the issue the final decision could be made by Bdale, who has identified himself as being in the systemd camp.

That assumes, of course, that section 6.1.4 of the constitution does not apply. That section states that, in cases where the committee is voting to overrule a Debian developer in the management of their packages, a three-to-one supermajority is required. It has been suggested in the discussion that a vote to pick anything other than sysvinit amounts to overriding the sysvinit maintainers, but it is unlikely that this viewpoint would carry the day. The committee would be forcing no changes to sysvinit; instead, it would be changing which init system is deemed to be the default for Debian. So a simple majority should suffice in this case.

That said, a simple majority in the form of a 4:4 decision pushed one way or the other by the committee chair will certainly not be seen by all developers as being truly decisive or representative of the project. In such a situation, the wider debate is unlikely to come to a conclusion once the results of the vote are announced; indeed, it may become stronger and more polarized than ever. It seems almost certain that the level of discontent would be high enough that a general resolution — a referendum voted on by all Debian developers — would be certain to follow.

Indeed, Guillem Jover has already proposed such a resolution without waiting for the committee to act. He claims that referring the problem to the committee is inappropriate for a number of reasons, including a lack of sufficient effort to reach consensus, a perceived attempt by the committee to do "design work" (not allowed by the constitution), and the above-mentioned need for the 3:1 supermajority. Others disagree with his reasoning, needless to say; it is hard to imagine, for example, that the numerous lengthy discussions spread out over years do not amount to a sufficient attempt to reach consensus.

But even people who disagree with Guillem's reasons might find themselves backing a general resolution; Ian, for example, described himself as "not entirely opposed to the idea." He made the point that much of the argument around init systems is not actually technical. Members of the committee on both sides of the debate have made it clear that, in the end, they could live with a decision for either systemd or upstart, even if it wasn't the one that they wanted. One way or another, either system could be made to meet Debian's needs. But that is not all that is being debated here.

Systemd opponents, for example, go to great pains to cite the allegedly poor attitude of that system's primary developers; there have also been worries about adopting a system that is seen to be controlled by Red Hat. Upstart detractors see attitude problems of their own at Canonical — another commercial distributor, albeit a Debian-based one — and nobody likes the contributor agreement that Canonical demands of those who would submit patches. The importance of the non-Linux ports to the Debian project is also an important issue; upstart is seen as being more friendly to the non-Linux ports (even if it supports neither of them currently) than systemd is. These issues are all hard to weigh in a committee that is supposed to restrict itself to technical decisions.

In a project like Debian, the more political issues might just be better suited to a general resolution. There is an interesting constitutional quirk that comes into play, though: section 4.1.4 allows the technical committee to be overridden by a general resolution, but only with a 2:1 supermajority. So the worst-case scenario could be ugly: a hung committee releases a decision with the chair's casting vote; the community then votes for a different alternative via general resolution, but without enough votes to actually override the committee. That seems unlikely to be a happy day in the Debian community.

In response to this concern, Ian suggested that the committee could explicitly include a clause in its decision allowing that decision to be overridden by a simple majority vote. As an alternative, the committee could simply step back and send the issue to a general resolution from the outset. That outcome seems unlikely, though; the committee members seem to feel a responsibility to reach a decision on this issue if at all possible. So there will likely be an attempt to come to some sort of conclusion within the committee. And, regardless of which system is chosen, there are a number of associated policy and transition details to be dealt with. To that end, Ian has been drafting a resolution in the committee's git repository; it talks about upstart, but that would be changed if the committee were to go with systemd.

When a decision will be made is unclear, though one might expect that the committee members would, by now, be united in their desire to put this particular issue behind them. What that conclusion will be is also still hard to guess, even after the members have posted their thoughts; some of them seem to feel more strongly about their positions than others, so it is possible that some changing of minds could occur. With luck, the committee will come to a reasonably well-supported decision that will allow the project to move on with getting the "jessie" release ready. Other distribution communities that have had messy init-system discussions have mostly found peace once the decision was made; hopefully Debian will be able to follow a similar path.

Comments (280 posted)

Page editor: Jonathan Corbet


FreeOTP multi-factor authentication

By Nathan Willis
January 22, 2014

Back in 2010, Google rolled out multi-factor authentication for its services using an open source project named Google Authenticator. The system generated one-time pad (OTP) passcodes on a mobile phone; users could log in to their favorite Google service with their usual password plus the passcode. In addition to its availability on Google servers, the open source project included a PAM module that Linux users could run on their own machines. Sadly, Google Authenticator fell victim to the gradual move toward proprietary code of recent "Google Play Services" releases and the open source version was discontinued, without fanfare, in 2013. Fortunately, the community has picked up the challenge and is developing a workalike called FreeOTP.

[Google Authenticator]

As we saw in our look at Google Authenticator, the system consisted of two halves: a mobile application and a server-side authentication plug-in. When provisioned with a key, the mobile app generated either HMAC-based OTP (HOTP) or Time-based OTP (TOTP) passcodes. Since HOTP and TOTP are both open IETF standards, the app could be used with any compliant server and supported storing multiple sets of credentials—the only prerequisite being that the phone would needed to be provisioned with the key for each service. The server-side tools included a key-generation program, and the project was popular enough that multiple third-party projects and web services deployed it.

The turn from free software to proprietary software came after the 2.21 release. The notice for the change was posted only as an update to the Google Authenticator web page at Google Code, which was highlighted in this Reddit discussion from May 2013. The wording change noted simply that the subsequent, closed-source versions of the app "contain Google-specific workflows." FreeOTP was started in August 2013; although it includes bits of code from Google Authenticator, it is predominantly a fresh rewrite—albeit one that does its best to serve as a free-software clone of the original.

The new project is hosted at, and is maintained by Red Hat's Nathaniel McCallum. On the mobile side of the equation, it replicates the Android and iOS client apps, but drops the Blackberry variant. As of right now, the iOS app has not yet been published on Apple's app store, but the Android version is available both through Google Play and through the free-software repository F-Droid.

On the server side, so far FreeOTP is not producing any components. Since any standards-compliant HOTP or TOTP server should work with FreeOTP, there is no absolute need for the project to produce its own server-side code. One of the nice things about Google Authenticator and FreeOTP, of course, is that the server and the client do not talk directly to each other: the client app generates the passcode, then the user types it in. Still, this does mean that the HOTP/TOTP Linux PAM module is now unmaintained. Although it is likely to work, that likelihood certainly comes with no guarantees—particularly since system security is at stake.

The FreeOTP project site points users to FreeIPA as a recommended solution. FreeIPA is an identity-management suite offered by Red Hat that glues together several existing open source components to provide LDAP, Kerberos, RADIUS, and other authentication or authorization services. McCallum, unsurprisingly, works on FreeIPA, and has added HOTP/TOTP support to MIT Kerberos. Work is also underway for a standalone HOTP/TOTP implementation in FreeIPA, though it is still under development.



FreeOTP replicates the same general workflow used in Google Authenticator. The first step is to provision the device with the key necessary to generate the HOTP or TOTP passcodes. How this step is done varies depending on the service one is trying to connect to, but the simplest option (if available) is to photograph a QR code that is generated by the server; these QR codes encode the key as well as the metadata one needs to keep track of the account (e.g., the server and account username).

FreeOTP is not as visually polished, but it does score some points over Google Authenticator by including the ZXing QR code–reading library within the app; Google requires downloading and installing a ZXing app separately. For services that do not generate QR codes for key provisioning, FreeOTP allows the user to manually enter the key, the key's options (e.g., the hash algorithm used) and information for the issuing service.

Once provisioned, opening the app shows a list of the saved accounts, and opening each account reveals the current one-time pad passcode. TOTP, the time-based passcode variant, is designed so that the codes are only valid for a brief interval of time (say, 30 seconds). The FreeOTP interface displays a little clock animation next to each code showing how much time remains, and automatically updates to the next passcode after every elapsed interval.

[Manual key entry in FreeOTP]

In practice, TOTP seems to be considerably more widespread than HOTP, presumably because HOTP seeds each passcode-generation run with a counter, rather than the current time, which in turn makes the resulting passcode valid for an indefinite length of time. This is of concern for Google Authenticator and FreeOTP because passcodes generated on phones can be compromised—a passcode with an indefinite lifetime allows more attack vectors than one that expires (for example, by jotting down the code and leaving, instead of stealing the phone).

But that is not the whole story; using one's Android phone as if it is a hardware crypto-token is a bit of a fudge regardless of the passcode-generation technique. In reality, since a phone can easily be lost or stolen, the only protection against the attacker using it to take over one's HOTP/TOTP online accounts is securing the phone's contents itself: password-protection, encrypted storage, and so on. What percentage of phone users actually take advantage of these options is anyone's guess.

For those who are concerned about the security of the device itself, it is worth noting that neither the Google Authenticator nor FreeOTP apps are password-protected. In an email, McCallum did mention that encrypted key storage is on the to-do list, but that other items (such as getting the iOS app released) have taken precedence for now.

On the other hand, many (or perhaps most) security schemes can lead into a "the perfect is the enemy of the good" cycle of disagreement that has no simple answer. Usage of two-factor authentication, even if the second "factor" is a software-generated passcode on a phone that could get stolen, is an improvement over user passwords alone. Similarly, while FreeOTP does not offer an expanded feature set over Google Authenticator, the fact that it is open source and thus auditable makes it a better option in absolute terms.

The shift of Google Authenticator to a proprietary app was certainly a step backward for the security-conscious, so it is good to see an open-source alternative arrive with feature parity. Of course, the mobile apps only cover one half of the problem; there is more work that remains to be done implementing multi-factor authentication on applications and servers.

Comments (21 posted)

Brief items

Security quotes of the week

To make matters worse, ownership of a Chrome extension can be transferred to another party, and users are never informed when an ownership change happens. Malware and adware vendors have caught wind of this and have started showing up at the doors of extension authors, looking to buy their extensions. Once the deal is done and the ownership of the extension is transferred, the new owners can issue an ad-filled update over Chrome's update service, which sends the adware out to every user of that extension.
Ron Amadeo in ars technica

And it is my considered opinion that the implementation of this process qualifies as idiotic and borderline criminal in terms of gross incompetency.

But then again, we're talking about Network Solutions.

Lauren Weinstein on phishy looking email from Network Solutions

But according to sources, the attackers broke in to Target after compromising a company Web server. Somehow, the attackers were able to upload the malicious POS [point-of-sale] software to store point-of-sale machines, and then set up a control server within Target’s internal network that served as a central repository for data hoovered by all of the infected point-of-sale devices.
Brian Krebs looks into the Target data breach

In conclusion, while EncFS is a useful tool, it ignores many standard best-practices in cryptography. This is most likely due to it's old age (originally developed before 2005), however, it is still being used today, and needs to be updated.

The EncFS author says that a 2.0 version is being developed. This would be a good time to fix the old problems.

As it is now, EncFS is not suitable for protecting mission-critical data.

Taylor Hornby audits EncFS

Comments (1 posted)

New vulnerabilities

almanah: does not encrypt its database

Package(s):almanah CVE #(s):CVE-2013-1853
Created:January 22, 2014 Updated:January 22, 2014
Description: From the bug report:

It was reported that Almanah does not encrypt its database when it closes, due to GApplication no longer using the quit_main_loop() event since GIO 2.32. This will keep the database unencrypted when it should be encrypted. The upstream bug report has a patch attached which corrects the issue.

openSUSE openSUSE-SU-2014:0118-1 almanah 2014-01-22

Comments (none posted)

augeas: world writable config files

Package(s):augeas CVE #(s):CVE-2013-6412
Created:January 21, 2014 Updated:January 22, 2014
Description: From the Red Hat advisory:

A flaw was found in the way Augeas handled certain umask settings when creating new configuration files. This flaw could result in configuration files being created as world writable, allowing unprivileged local users to modify their content.

Mandriva MDVSA-2014:022 augeas 2014-01-24
Scientific Linux SLSA-2014:0044-1 augeas 2014-01-21
Oracle ELSA-2014-0044 augeas 2014-01-20
CentOS CESA-2014:0044 augeas 2014-01-20
Red Hat RHSA-2014:0044-01 augeas 2014-01-20
Mageia MGASA-2014-0058 augeas 2014-02-12

Comments (none posted)

cups: information disclosure

Package(s):cups CVE #(s):CVE-2013-6891
Created:January 16, 2014 Updated:January 22, 2014
Description: From the Ubuntu advisory:

Jann Horn discovered that the CUPS lppasswd tool incorrectly read a user configuration file in certain configurations. A local attacker could use this to read sensitive information from certain files, bypassing access restrictions.

Mandriva MDVSA-2014:092 cups 2014-05-16
Mandriva MDVSA-2014:015 cups 2014-01-22
Mageia MGASA-2014-0021 cups 2014-01-21
Ubuntu USN-2082-1 cups 2014-01-15

Comments (none posted)

cups: cups.socket is listening on

Package(s):cups CVE #(s):
Created:January 22, 2014 Updated:January 22, 2014
Description: From the bug report:

default suse cups setup was always set that cupsd was listening on localhost:631 .. in suse 13.1 systemd creates for cups sockets (and whatver else) and cupsd is started just when something accesses these sockets or listening "init" service as it is recognized by netstat .. now systemd listens on both tcp/udp which seems wrong to me ..

openSUSE openSUSE-SU-2014:0119-1 cups 2014-01-22

Comments (none posted)

drupal7: multiple vulnerabilities

Package(s):drupal7 CVE #(s):CVE-2014-1475 CVE-2014-1476
Created:January 21, 2014 Updated:February 17, 2014
Description: From the Debian advisory:

CVE-2014-1475: Christian Mainka and Vladislav Mladenov reported a vulnerability in the OpenID module that allows a malicious user to log in as other users on the site, including administrators, and hijack their accounts.

CVE-2014-1476: Matt Vance and Damien Tournoud reported an access bypass vulnerability in the taxonomy module. Under certain circumstances, unpublished content can appear on listing pages provided by the taxonomy module and will be visible to users who should not have permission to see it.

Mageia MGASA-2014-0031 drupal 2014-01-31
Fedora FEDORA-2014-0980 drupal6 2014-01-25
Fedora FEDORA-2014-0999 drupal6 2014-01-25
Fedora FEDORA-2014-0983 drupal7 2014-01-25
Fedora FEDORA-2014-1015 drupal7 2014-01-25
Debian DSA-2847-1 drupal7 2014-01-20
Mandriva MDVSA-2014:031 drupal 2014-02-14
Debian DSA-2851-1 drupal6 2014-02-02

Comments (none posted)

drupal7-entity: multiple vulnerabilities

Package(s):drupal7-entity CVE #(s):CVE-2014-1398 CVE-2014-1399 CVE-2014-1400
Created:January 20, 2014 Updated:January 22, 2014
Description: From the Red Hat bugzilla:

The entity module for Drupal was recently reported to be affected by multiple access bypass vulnerabilities, which could be exploited by an attacker to gain unauthorized access to the data.

See the Drupal advisory for more information.

Fedora FEDORA-2014-0508 drupal7-entity 2014-01-19
Fedora FEDORA-2014-0509 drupal7-entity 2014-01-19

Comments (none posted)

ejabberd: information disclosure

Package(s):ejabberd CVE #(s):CVE-2013-6169
Created:January 17, 2014 Updated:February 13, 2014

From the CVE entry:

The TLS driver in ejabberd before 2.1.12 supports (1) SSLv2 and (2) weak SSL ciphers, which makes it easier for remote attackers to obtain sensitive information via a brute-force attack.

Debian DSA-2775-1 ejabberd 2013-10-10
Mandriva MDVSA-2014:005 ejabberd 2014-01-16
Mageia MGASA-2014-0057 ejabberd 2014-02-12

Comments (none posted)

hplip: file overwrites

Package(s):hplip CVE #(s):CVE-2013-6402
Created:January 21, 2014 Updated:January 28, 2014
Description: From the Ubuntu advisory:

It was discovered that the HPLIP Polkit daemon incorrectly handled temporary files. A local attacker could possibly use this issue to overwrite arbitrary files. In the default installation of Ubuntu 12.04 LTS and higher, this should be prevented by the Yama link restrictions.

openSUSE openSUSE-SU-2014:0146-1 hplip 2014-01-28
openSUSE openSUSE-SU-2014:0127-1 hplip 2014-01-24
Mandriva MDVSA-2014:023 hplip 2014-01-24
Ubuntu USN-2085-1 hplip 2014-01-21
Mageia MGASA-2014-0033 hplip 2014-02-05

Comments (none posted)

java-1.7.0: multiple unspecified vulnerabilities

Package(s):java-1.7.0-oracle CVE #(s):CVE-2013-5870 CVE-2013-5887 CVE-2013-5888 CVE-2013-5889 CVE-2013-5895 CVE-2013-5898 CVE-2013-5899 CVE-2013-5902 CVE-2013-5904 CVE-2013-5905 CVE-2013-5906 CVE-2014-0375 CVE-2014-0382 CVE-2014-0387 CVE-2014-0403 CVE-2014-0410 CVE-2014-0415 CVE-2014-0417 CVE-2014-0418 CVE-2014-0424
Created:January 16, 2014 Updated:April 17, 2014

From the Red Hat advisory:

Bug - CVE-2014-0410 Oracle JDK: unspecified vulnerability fixed in 6u71 and 7u51 (Deployment)

Bug - CVE-2014-0415 Oracle JDK: unspecified vulnerability fixed in 6u71 and 7u51 (Deployment)

Bug - CVE-2013-5889 Oracle JDK: unspecified vulnerability fixed in 6u71 and 7u51 (Deployment)

Bug - CVE-2014-0417 Oracle JDK: unspecified vulnerability fixed in 5.0u71, 6u71 and 7u51 (2D)

Bug - CVE-2014-0387 Oracle JDK: unspecified vulnerability fixed in 6u71 and 7u51 (Deployment)

Bug - CVE-2014-0424 Oracle JDK: unspecified vulnerability fixed in 6u71 and 7u51 (Deployment)

Bug - CVE-2013-5904 Oracle JDK: unspecified vulnerability fixed in 7u51 (Deployment)

Bug - CVE-2014-0403 Oracle JDK: unspecified vulnerability fixed in 6u71 and 7u51 (Deployment)

Bug - CVE-2014-0375 Oracle JDK: unspecified vulnerability fixed in 6u71 and 7u51 (Deployment)

Bug - CVE-2013-5905 Oracle JDK: unspecified vulnerability fixed in 5.0u71, 6u71 and 7u51 (Install)

Bug - CVE-2013-5906 Oracle JDK: unspecified vulnerability fixed in 5.0u71, 6u71 and 7u51 (Install)

Bug - CVE-2013-5902 Oracle JDK: unspecified vulnerability fixed in 6u71 and 7u51 (Deployment)

Bug - CVE-2014-0418 Oracle JDK: unspecified vulnerability fixed in 6u71 and 7u51 (Deployment)

Bug - CVE-2013-5887 Oracle JDK: unspecified vulnerability fixed in 6u71 and 7u51 (Deployment)

Bug - CVE-2013-5899 Oracle JDK: unspecified vulnerability fixed in 6u71 and 7u51 (Deployment)

Bug - CVE-2013-5888 Oracle JDK: unspecified vulnerability fixed in 6u71 and 7u51 (Deployment)

Bug - CVE-2013-5898 Oracle JDK: unspecified vulnerability fixed in 6u71 and 7u51 (Deployment)

Bug - CVE-2013-5870 CVE-2013-5895 CVE-2014-0382 Oracle JDK: multiple unspecified vulnerabilities fixed in 7u51 (JavaFX)

Red Hat RHSA-2014:0414-01 java-1.6.0-sun 2014-04-17
SUSE SUSE-SU-2014:0451-1 IBM Java 6 2014-03-26
SUSE SUSE-SU-2014:0266-3 IBM Java 6 2014-02-24
SUSE SUSE-SU-2014:0266-1 IBM Java 6 2014-02-20
SUSE SUSE-SU-2014:0266-2 IBM Java 6 2014-02-21
SUSE SUSE-SU-2014:0246-1 IBM Java 2014-02-18
Red Hat RHSA-2014:0135-01 java-1.6.0-ibm 2014-02-04
Gentoo 201401-30 oracle-jdk-bin 2014-01-26
Fedora FEDORA-2014-0945 java-1.7.0-openjdk 2014-01-16
Fedora FEDORA-2014-0885 java-1.7.0-openjdk 2014-01-16
Red Hat RHSA-2014:0030-01 java-1.7.0-oracle 2014-01-15
Red Hat RHSA-2014:0136-01 java-1.5.0-ibm 2014-02-04
Red Hat RHSA-2014:0134-01 java-1.7.0-ibm 2014-02-04

Comments (none posted)

libpng16: denial of service

Package(s):libpng16 CVE #(s):CVE-2013-6954
Created:January 20, 2014 Updated:June 10, 2014
Description: From the CVE entry:

The png_do_expand_palette function in libpng before 1.6.8 allows remote attackers to cause a denial of service (NULL pointer dereference and application crash) via (1) a PLTE chunk of zero bytes or (2) a NULL palette, related to pngrtran.c and pngset.c.

Mandriva MDVSA-2015:071 libpng12 2015-03-27
openSUSE openSUSE-SU-2014:1645-1 java-1_7_0-openjdk 2014-12-15
openSUSE openSUSE-SU-2014:1638-1 java-1_7_0-openjdk 2014-12-15
Gentoo 201406-32 icedtea-bin 2014-06-29
Fedora FEDORA-2014-6717 libpng 2014-06-10
SUSE SUSE-SU-2014:0733-2 IBM Java 7 2014-06-02
SUSE SUSE-SU-2014:0728-3 IBM Java 6 2014-06-03
SUSE SUSE-SU-2014:0733-1 IBM Java 7 2014-05-30
SUSE SUSE-SU-2014:0728-2 IBM Java 6 2014-05-30
SUSE SUSE-SU-2014:0728-1 IBM Java 6 2014-05-29
Fedora FEDORA-2014-6631 libpng 2014-05-28
Red Hat RHSA-2014:0508-01 java-1.6.0-ibm 2014-05-15
SUSE SUSE-SU-2014:0639-1 OpenJDK 2014-05-14
Red Hat RHSA-2014:0486-01 java-1.7.0-ibm 2014-05-13
Debian DSA-2923-1 openjdk-7 2014-05-05
Red Hat RHSA-2014:0412-01 java-1.7.0-oracle 2014-04-17
Red Hat RHSA-2014:0413-02 java-1.7.0-oracle 2014-04-17
Red Hat RHSA-2014:0414-01 java-1.6.0-sun 2014-04-17
Fedora FEDORA-2014-4564 mingw-libpng 2014-04-02
openSUSE openSUSE-SU-2014:0100-1 libpng16 2014-01-20
Mageia MGASA-2014-0076 libpng12 2014-02-16
Fedora FEDORA-2014-1803 libpng15 2014-02-11
Fedora FEDORA-2014-1754 libpng10 2014-02-07
Mandriva MDVSA-2014:035 libpng 2014-02-17
Fedora FEDORA-2014-1766 libpng12 2014-02-11
Fedora FEDORA-2014-1770 libpng12 2014-02-11
Mageia MGASA-2014-0075 libpng 2014-02-16
Fedora FEDORA-2014-1778 libpng10 2014-02-07

Comments (none posted)

libvirt: denial of service

Package(s):libvirt CVE #(s):CVE-2013-6458 CVE-2014-1447
Created:January 20, 2014 Updated:February 24, 2014
Description: From the Debian advisory:

CVE-2013-6458: It was discovered that insecure job usage could lead to denial of service against libvirtd.

CVE-2014-1447: It was discovered that a race condition in keepalive handling could lead to denial of service against libvirtd.

Gentoo 201412-04 libvirt 2014-12-09
openSUSE openSUSE-SU-2014:0270-1 libvirt 2014-02-21
openSUSE openSUSE-SU-2014:0268-1 libvirt 2014-02-21
Ubuntu USN-2093-1 libvirt 2014-01-30
Scientific Linux SLSA-2014:0103-1 libvirt 2014-01-28
Oracle ELSA-2014-0103 libvirt 2014-01-28
CentOS CESA-2014:0103 libvirt 2014-01-29
Red Hat RHSA-2014:0103-01 libvirt 2014-01-28
Fedora FEDORA-2014-1090 libvirt 2014-01-26
CentOS CESA-2014:X001 libvirt 2014-01-25
Fedora FEDORA-2014-1042 libvirt 2014-01-21
Debian DSA-2846-1 libvirt 2014-01-17
Mageia MGASA-2014-0051 libvirt 2014-02-11

Comments (none posted)

libxslt: denial of service

Package(s):libxslt CVE #(s):CVE-2013-4520
Created:January 17, 2014 Updated:January 22, 2014

From the CVE entry:

xslt.c in libxslt before 1.1.25 allows context-dependent attackers to cause a denial of service (crash) via a stylesheet that embeds a DTD, which causes a structure to be accessed as a different type. NOTE: this issue is due to an incomplete fix for CVE-2012-2825.

Mandriva MDVSA-2014:006 libxslt 2014-01-16

Comments (none posted)

memcached: multiple vulnerabilities

Package(s):memcached CVE #(s):CVE-2013-7290 CVE-2013-7291
Created:January 17, 2014 Updated:February 3, 2014

From the Mandriva advisory:

The do_item_get function in items.c in memcached 1.4.4 and other versions before 1.4.17, when running in verbose mode, allows remote attackers to cause a denial of service (segmentation fault) via a request to delete a key, which does not account for the lack of a null terminator in the key and triggers a buffer over-read when printing to stderr, a different vulnerability than CVE-2013-0179 (CVE-2013-7290).

memcached before 1.4.17, when running in verbose mode, allows remote attackers to cause a denial of service (crash) via a request that triggers an unbounded key print during logging, related to an issue that was quickly grepped out of the source tree, a different vulnerability than CVE-2013-0179 and CVE-2013-7290 (CVE-2013-7291).

openSUSE openSUSE-SU-2014:0951-1 memcached 2014-07-30
openSUSE openSUSE-SU-2014:0867-1 memcached 2014-07-03
Gentoo 201406-13 memcached 2014-06-14
Mageia MGASA-2014-0018 memcached 2014-01-21
Mandriva MDVSA-2014:010 memcached 2014-01-17
Fedora FEDORA-2014-0934 memcached 2014-02-03
Fedora FEDORA-2014-0926 memcached 2014-02-03
Debian-LTS DLA-701-1 memcached 2016-11-05

Comments (none posted)

mysql: two unspecified vulnerabilities

Package(s):mysql-5.5, mysql-dfsg-5.1 CVE #(s):CVE-2013-5891 CVE-2014-0420
Created:January 21, 2014 Updated:June 9, 2014
Description: From the CVE entries:

Unspecified vulnerability in the MySQL Server component in Oracle MySQL 5.5.33 and earlier and 5.6.13 and earlier allows remote authenticated users to affect availability via unknown vectors related to Partition. (CVE-2013-5891)

Unspecified vulnerability in the MySQL Server component in Oracle MySQL 5.5.34 and earlier, and 5.6.14 and earlier, allows remote authenticated users to affect availability via unknown vectors related to Replication. (CVE-2014-0420)

Mandriva MDVSA-2015:091 mariadb 2015-03-28
Gentoo 201409-04 mysql 2014-09-04
SUSE SUSE-SU-2014:0769-1 MySQL 2014-06-07
CentOS CESA-2014:0189 mariadb55-mariadb 2014-02-26
CentOS CESA-2014:0173 mysql55-mysql 2014-02-19
Red Hat RHSA-2014:0189-01 mariadb55-mariadb 2014-02-19
Scientific Linux SLSA-2014:0186-1 mysql55-mysql 2014-02-18
Oracle ELSA-2014-0186 mysql55-mysql 2014-02-18
CentOS CESA-2014:0186 mysql55-mysql 2014-02-19
Red Hat RHSA-2014:0186-01 mysql55-mysql 2014-02-18
Debian DSA-2848-1 mysql-5.5 2014-01-23
Ubuntu USN-2086-1 mysql-5.5, mysql-dfsg-5.1 2014-01-21
Mandriva MDVSA-2014:028 mariadb 2014-02-13
Red Hat RHSA-2014:0173-01 mysql55-mysql 2014-02-13

Comments (none posted)

mysql: multiple unspecified vulnerabilities

Package(s):mysql-5.1 CVE #(s):CVE-2013-5908 CVE-2014-0386 CVE-2014-0393 CVE-2014-0401 CVE-2014-0402 CVE-2014-0412 CVE-2014-0437
Created:January 20, 2014 Updated:June 9, 2014
Description: From the CVE entries:

Unspecified vulnerability in the MySQL Server component in Oracle MySQL 5.1.72 and earlier, 5.5.34 and earlier, and 5.6.14 and earlier allows remote attackers to affect availability via unknown vectors related to Error Handling. (CVE-2013-5908)

Unspecified vulnerability in the MySQL Server component in Oracle MySQL 5.1.71 and earlier, 5.5.33 and earlier, and 5.6.13 and earlier allows remote authenticated users to affect availability via unknown vectors related to Optimizer. (CVE-2014-0386)

Unspecified vulnerability in the MySQL Server component in Oracle MySQL 5.1.71 and earlier, 5.5.33 and earlier, and 5.6.13 and earlier allows remote authenticated users to affect integrity via unknown vectors related to InnoDB. (CVE-2014-0393)

Unspecified vulnerability in the MySQL Server component in Oracle MySQL 5.1.72 and earlier, 5.5.34 and earlier, and 5.6.14 and earlier allows remote authenticated users to affect availability via unknown vectors. (CVE-2014-0401)

Unspecified vulnerability in the MySQL Server component in Oracle MySQL 5.1.71 and earlier, 5.5.33 and earlier, and 5.6.13 and earlier allows remote authenticated users to affect availability via unknown vectors related to Locking. (CVE-2014-0402)

Unspecified vulnerability in the MySQL Server component in Oracle MySQL 5.1.72 and earlier, 5.5.34 and earlier, and 5.6.14 and earlier allows remote authenticated users to affect availability via unknown vectors related to InnoDB. (CVE-2014-0412)

Unspecified vulnerability in the MySQL Server component in Oracle MySQL 5.1.72 and earlier, 5.5.34 and earlier, and 5.6.14 and earlier allows remote authenticated users to affect availability via unknown vectors related to Optimizer. (CVE-2014-0437)

Mandriva MDVSA-2015:091 mariadb 2015-03-28
Gentoo 201409-04 mysql 2014-09-04
SUSE SUSE-SU-2014:0769-1 MySQL 2014-06-07
CentOS CESA-2014:0189 mariadb55-mariadb 2014-02-26
CentOS CESA-2014:0173 mysql55-mysql 2014-02-19
Red Hat RHSA-2014:0189-01 mariadb55-mariadb 2014-02-19
Scientific Linux SLSA-2014:0186-1 mysql55-mysql 2014-02-18
Oracle ELSA-2014-0186 mysql55-mysql 2014-02-18
CentOS CESA-2014:0186 mysql55-mysql 2014-02-19
Red Hat RHSA-2014:0186-01 mysql55-mysql 2014-02-18
Debian DSA-2848-1 mysql-5.5 2014-01-23
Ubuntu USN-2086-1 mysql-5.5, mysql-dfsg-5.1 2014-01-21
Debian DSA-2845-1 mysql-5.1 2014-01-17
Mandriva MDVSA-2014:028 mariadb 2014-02-13
CentOS CESA-2014:0164 mysql 2014-02-12
Red Hat RHSA-2014:0173-01 mysql55-mysql 2014-02-13
Scientific Linux SLSA-2014:0164-1 mysql 2014-02-12
Oracle ELSA-2014-0164 mysql 2014-02-12
Red Hat RHSA-2014:0164-01 mysql 2014-02-12

Comments (none posted)

nagios: denial of service

Package(s):nagios CVE #(s):CVE-2013-7205
Created:January 17, 2014 Updated:January 22, 2014

From the CVE entry:

Off-by-one error in the process_cgivars function in contrib/daemonchk.c in Nagios Core 3.5.1, 4.0.2, and earlier allows remote authenticated users to obtain sensitive information from process memory or cause a denial of service (crash) via a long string in the last key value in the variable list, which triggers a heap-based buffer over-read.

Gentoo 201412-23 nagios-core 2014-12-13
Mandriva MDVSA-2014:004 nagios 2014-01-16
Mageia MGASA-2014-0010 nagios 2014-01-17

Comments (none posted)

nss: information disclosure

Package(s):nss CVE #(s):CVE-2013-1740
Created:January 21, 2014 Updated:February 4, 2014
Description: From the Red Hat bugzilla:

A security issue has been reported in NSS, which can be exploited by a malicious user to disclose certain information.

The issue arises due to an error within the "ssl_Do1stHandshake()" function in lib/ssl/sslsecur.c, which can be exploited to potentially return unencrypted and unauthenticated data from PR_Recv. Successful exploitation requires false start to be enabled.

The issue is said to be fixed in NSS 3.15.4.

Oracle ELSA-2014-1948 nss 2014-12-02
CentOS CESA-2014:1246 nss, nspr 2014-09-30
Scientific Linux SLSA-2014:1246-1 nss and nspr 2014-09-26
Oracle ELSA-2014-1246 nss, nspr 2014-09-17
Red Hat RHSA-2014:1246-01 nss, nspr 2014-09-16
Scientific Linux SLSA-2014:0917-1 nss and nspr 2014-07-22
Oracle ELSA-2014-0917 nss, nspr 2014-07-22
Red Hat RHSA-2014:0917-01 nss, nspr 2014-07-22
Fedora FEDORA-2014-1100 nss-util 2014-02-04
Slackware SSA:2014-028-02 mozilla-nss 2014-01-28
Fedora FEDORA-2014-1100 nss-softokn 2014-02-04
Ubuntu USN-2088-1 nss 2014-01-23
Mandriva MDVSA-2014:012 nss 2014-01-20
Mageia MGASA-2014-0024 nss 2014-01-21
Fedora FEDORA-2014-1120 nss-util 2014-01-21
Fedora FEDORA-2014-1120 nss-softokn 2014-01-21
Fedora FEDORA-2014-1120 nss 2014-01-21
Fedora FEDORA-2014-1100 nss 2014-02-04
openSUSE openSUSE-SU-2014:0212-1 firefox 2014-02-08
openSUSE openSUSE-SU-2014:0213-1 Mozilla 2014-02-08

Comments (none posted)

ntp: denial of service

Package(s):ntp CVE #(s):CVE-2013-5211
Created:January 17, 2014 Updated:September 13, 2016

From the CVE entry:

The monlist feature in ntp_request.c in ntpd in NTP before 4.2.7p26 allows remote attackers to cause a denial of service (traffic amplification) via forged (1) REQ_MON_GETLIST or (2) REQ_MON_GETLIST_1 requests, as exploited in the wild in December 2013.

openSUSE openSUSE-SU-2014:1149-1 ntp 2014-09-20
openSUSE openSUSE-SU-2014:0949-1 ntp 2014-07-30
Mageia MGASA-2014-0032 ntp 2014-01-31
SUSE SUSE-SA:2014:001 ntp 2014-01-20
Gentoo 201401-08 ntp 2014-01-17
Slackware SSA:2014-044-02 ntp 2014-02-13
Oracle ELSA-2016-3612 ntp 2016-09-12
Oracle ELSA-2016-3613 ntp 2016-09-12
Oracle ELSA-2017-0252 ntp 2017-02-06

Comments (none posted)

rubygem-will_paginate: cross-site scripting

Package(s):rubygem-will_paginate CVE #(s):CVE-2013-6459
Created:January 21, 2014 Updated:February 12, 2014
Description: From the CVE entry:

Cross-site scripting (XSS) vulnerability in the will_paginate gem before 3.0.5 for Ruby allows remote attackers to inject arbitrary web script or HTML via vectors involving generated pagination links.

Fedora FEDORA-2014-0066 rubygem-will_paginate 2014-01-21
Fedora FEDORA-2014-0094 rubygem-will_paginate 2014-01-21
Mageia MGASA-2014-0054 ruby-will_paginate 2014-02-11

Comments (none posted)

virtualbox: multiple unspecified vulnerabilities

Package(s):virtualbox CVE #(s):CVE-2013-5892 CVE-2014-0404 CVE-2014-0405 CVE-2014-0406 CVE-2014-0407
Created:January 20, 2014 Updated:January 22, 2014
Description: From the CVE entries:

Unspecified vulnerability in the Oracle VM VirtualBox component in Oracle Virtualization VirtualBox prior to 3.2.20, 4.0.22, 4.1.30, 4.2.22, and 4.3.6 allows local users to affect confidentiality, integrity, and availability via unknown vectors related to Core. (CVE-2013-5892)

Unspecified vulnerability in the Oracle VM VirtualBox component in Oracle Virtualization VirtualBox prior to 3.2.20, 4.0.22, 4.1.30, 4.2.20, and 4.3.4 allows local users to affect integrity and availability via unknown vectors related to Core, a different vulnerability than CVE-2014-0406. (CVE-2014-0404)

Unspecified vulnerability in the Oracle VM VirtualBox component in Oracle Virtualization VirtualBox prior to 3.2.20, 4.0.22, 4.1.30, 4.2.20, and 4.3.4 allows local users to affect confidentiality, integrity, and availability via unknown vectors related to Core. (CVE-2014-0405)

Unspecified vulnerability in the Oracle VM VirtualBox component in Oracle Virtualization VirtualBox prior to 3.2.20, 4.0.22, 4.1.30, 4.2.20, and 4.3.4 allows local users to affect integrity and availability via unknown vectors related to Core, a different vulnerability than CVE-2014-0404. (CVE-2014-0406)

Unspecified vulnerability in the Oracle VM VirtualBox component in Oracle Virtualization VirtualBox prior to 3.2.20, 4.0.22, 4.1.30, 4.2.20, and 4.3.4 allows local users to affect confidentiality, integrity, and availability via unknown vectors related to Core. (CVE-2014-0407)

Mageia MGASA-2014-0184 virtualbox 2014-04-20
Debian DSA-2878-1 virtualbox 2014-03-13
Gentoo 201401-13 virtualbox 2014-01-20

Comments (none posted)

zabbix: SQL injection

Package(s):zabbix CVE #(s):CVE-2013-5743
Created:January 21, 2014 Updated:January 22, 2014
Description: From the Mageia advisory:

Fix SQL injection vulnerability (ZBX-7091)

Fedora FEDORA-2014-5551 zabbix 2014-05-01
Mageia MGASA-2014-0015 zabbix 2014-01-21

Comments (none posted)

Page editor: Jake Edge

Kernel development

Brief items

Kernel release status

The 3.13 kernel is out, released on January 19. "The release got delayed by a week due to travels, but I suspect that's just as well. We had a few fixes come in, and while it wasn't a lot, I think we're better off for it." Some of the headline features in 3.13 include the nftables packet filtering engine, a number of NUMA scheduling improvements, the multiqueue block layer work, ARM big.LITTLE switcher support, and more; see the KernelNewbies 3.13 page for lots of details.

Stable updates: there have been no stable updates in the last week, and none are in the review process as of this writing.

Comments (none posted)

Quotes of the week

I'm not sure whether the RNG is better characterized as the "bite shed" or the "coffee for refreshments" as described in Parkinson's 3rd chapter, "High Finance, or the Point of Vanishing Interest", but I know one thing for sure. It's not the 10 million pound nuclear reactor.
Ted Ts'o (worth reading in its entirety)

Anyway, I'd suggest you try to limit octopus merges to ~15 parents or less to make the visualization tools not go crazy. Maybe aim for just 10 or so in most cases.

It's pulled, and it's fine, but there's clearly a balance between "octopus merges are fine" and "Christ, that's not an octopus, that's a Cthulhu merge".

Linus Torvalds

Comments (none posted)

Kroah-Hartman: Kdbus Details

In something of a follow-up to our coverage of the kdbus talk given at by Lennart Poettering, Greg Kroah-Hartman has written a blog post giving more kdbus background. In particular, he looks at why kdbus will not be replacing Android's binder IPC mechanism anytime soon—if at all. "The model of binder is very limited, inflexible in its use-cases, but very powerful and extremely low-overhead and fast. Binder ensures that the same CPU timeslice will go from the calling process into the called process’s thread, and then come back into the caller when finished. There is almost no scheduling involved, and is much like a syscall into the kernel that does work for the calling process. This interface is very well suited for cheap devices with almost no RAM and very low CPU resources."

Comments (37 posted)

Kernel development news

3.14 Merge window part 1

By Jonathan Corbet
January 22, 2014
The merge window for the 3.14 kernel opened on January 20. As of this writing, just over 3300 non-merge changesets have been pulled into the mainline for this release, including some significant new functionality. The most significant user-visible changes pulled so far include:

  • The user-space lockdep patch set has been merged. This feature makes the kernel's locking debugging capabilities available to user-space applications.

  • After years of development, the deadline scheduling class has finally been merged. This class allows processes to declare an amount of work needing to be done and a deadline by which it must happen; with care, it can guarantee that all processes will meet their deadlines. See this in-progress document for more information about the current status of the deadline scheduler.

  • The m68k architecture has gained support for the kexec() system call.

  • The kexec() system call now works on EFI BIOS systems.

  • Xen is no longer supported on the ia64 architecture.

  • The arm64 architecture has gained support for jump labels and the CMA memory allocator.

  • The perf events subsystem has gained support for Intel "RAPL" energy consumption counters. The user-space perf tool has also gained a long list of new features and enhancements; see Ingo Molnar's pull request for a detailed list.

  • The kernel address space layout randomization patches have been merged. Depending on who you believe, this feature may make the kernel more resistant to certain types of attacks. Note that, as of this writing, enabling this feature breaks hibernation and perf events.

  • New hardware support includes:

    • Processors and systems: Intel Clovertrail and Merrifield MID platforms.

    • Audio: Tegra boards with MAX98090 codecs, Broadcom BCM2835 SoC I2S/PCM modules, Freescale SoC digital audio interfaces, Freescale enhanced serial audio interface controllers, and Analog Devices AXI-I2S and AXI-SPDIF softcore peripherals.

    • GPIO / pinmux: MOXA ART GPIO controllers, Xtensa GPIO32 GPIO controllers, SMSC SCH311x SuperI/O GPIO controllers, Freescale i.MX25 pin controllers, Qualcomm TLMM pin controllers, Qualcomm 8x74 pin controllers, NVIDIA Tegra124 pin controllers, Broadcom Capri pin controllers, and TI/National Semiconductor LP3943 GPIO expanders.

    • Miscellaneous: IBM Generic Work Queue Engine accelerators, Realtek RTS5208/5288 PCI-E card readers, Freescale MPL3115A2 pressure sensors, Freescale i.MX6 HDMI transmitters, DHT11 and DHT22 temperature/humidity sensors, HID 3D inclinometers, Capella Microsystem CM32181 ambient light sensors, Humusoft MF634 and MF624 data acquisition cards, Renesas RIIC I2C interfaces, TI/National Semiconductor LP3943 PWM controllers, ams AS3722 power-off units, and Maxim MAX14577 MUIC battery chargers.

    • USB: Maxim MAX14577 micro USB interface controllers, OMAP USB OTG controllers, Tahvo USB transceivers, Ingenic JZ4740 USB device controllers, Broadcom Kona USB2 PHYs, Aeroflex Gaisler GRUSBDC USB peripheral controllers, MOXA UPort serial hubs, and RobotFuzz Open Source InterFace USB to I2C controllers.

Changes visible to kernel developers include:

  • The new smp_load_acquire() and smp_store_release() memory barrier operations have been added; see this article for information on when they are needed and how they can be used.

  • The kernel can be built with the -fstack-protector-strong compiler option, available in GCC starting with version 4.9. This option allows more functions within the kernel to have stack overrun protection applied while still keeping the overhead to (hopefully) reasonable levels.

  • preempt_enable_no_resched() is no longer available to modules which, according to the scheduler developers, should not "be creative with preemption."

  • The internals of the sysfs virtual filesystem are being massively reworked to create a new "kernfs" that can serve as the base for a number of such filesystems. The first target is the cgroup control filesystem, but others may follow. This work is incomplete in 3.14, but has still resulted in a lot of internal code changes.

  • The new documentation file Documentation/driver-model/design-patterns.txt tries to formalize some of the design patterns seen in driver code. It is just getting started; contributions would undoubtedly be welcome.

  • There is a new "componentized subsystems" infrastructure for management of complex devices made up of a number of smaller, interacting devices; see this commit for details.

  • The Android ION memory allocator has been merged into the staging tree. A long list of improvements, including changes to make ION use the dma-buf interface and the CMA allocator, has been merged as well.

If the normal rules apply, the 3.14 merge window can be expected to remain open until around February 2. At this point, a number of large trees — networking, for example — have not yet been pulled, so one can expect quite a bit more in the way of changes between now and when the window closes. As always, next week's Kernel Page will continue to follow what gets into the mainline for this development cycle.

Comments (1 posted)

Btrfs: Send/receive and ioctl()

By Jonathan Corbet
January 22, 2014
LWN's guide to Btrfs
At this point, LWN's series on the Btrfs filesystem has covered most of the aspects of working with this next-generation filesystem. This article, the final installment in the series, will deal with a few loose ends that did not fit into the previous articles; in particular, we'll look at the send/receive functionality and a subset of the available ioctl() commands that are specific to Btrfs. In both cases, functionality is exposed that is not available in most other Linux filesystems.

Send and receive

The subvolumes and snapshots article described a rudimentary scheme for making incremental backups to a Btrfs filesystem. It was simple enough: use rsync to make the backup filesystem look like the original, then take a snapshot of the backup to preserve the state of affairs at that particular time. This approach is relatively efficient; rsync will only copy changes to the filesystem (passing over unchanged files), and each snapshot will preserve those changes without copying unchanged data on the backup volume. In this way, quite a bit of filesystem history can be kept in a readily accessible form.

There is another way to do incremental backups, though, if both the original and backup filesystems are Btrfs filesystems. In that case, the Btrfs send/receive mechanism can be used to optimize the process. One starts by taking a snapshot of the original filesystem:

    btrfs subvolume snapshot -r source-subvolume snapshot-name

The snapshot-name should probably include a timestamp, since the whole mechanism depends on the existence of regular snapshots at each incremental backup time. The initial snapshot is then copied to the backup drive with a command like:

    cd backup-filesystem
    btrfs send path-to-snapshot | btrfs receive .

This operation, which will copy the entire snapshot, can take quite a while, especially if the source filesystem is large. It is, indeed, significantly slower than just populating the destination filesystem with rsync or a pair of tar commands. It might work to use one of the latter methods to populate the backup filesystem initially, but using the send/receive chain ensures that things are set up the way those commands expect them to be.

Note that if the source snapshot is not read-only, btrfs send will refuse to work with it. There appears to be no btrfs command to set the read-only flag on an existing snapshot that was created as writable, but it is, of course, a simple matter to create a new read-only snapshot of an existing read/write snapshot, should the need arise.

Once the initial copy of the filesystem is in place, incremental backups can be done by taking a new snapshot on the source filesystem, then running a command like:

    cd backup-filesystem
    btrfs send -p path-to-previous-snapshot path-to-new-snapshot | btrfs receive .

With the -p flag, btrfs send will only send files (or portions thereof) that have changed since the previous-snapshot was taken; note that the previous snapshot needs to exist on the backup filesystem as well. Unlike the initial copy, an incremental send operation is quite fast — much faster than using a command like rsync to find and send changed files. It can thus be used as a low-impact incremental backup mechanism, possibly running many times each day.

Full use of this feature, naturally, is likely to require some scripting work. For example, it may not be desirable to keep every snapshot on the original filesystem, especially if space is tight there. But it is necessary to keep each snapshot long enough to use it for the next incremental send operation; using the starting snapshot would result in the unnecessary copying of a lot of data. Over time, one assumes, reasonably user-friendly tools will be developed to automate these tasks.

Btrfs ioctl() commands

Like most of the relatively complex Linux filesystems, Btrfs supports a number of filesystem-specific ioctl() commands. These commands are, as a rule, entirely undocumented; one must go to the (nearly comment-free) source to discover them and understand what they do. This article will not take the place of a proper document, but it will try to point out a few of the more interesting commands.

Most of the Btrfs-specific commands carry out operations that are available via the btrfs command-line tool. Thus, there are commands for the management of subvolumes and snapshots, devices, etc. For the most part, the btrfs tool is the best way to access this type of functionality, so those commands will not be covered here. It is amusing to note that several of these commands already come in multiple versions; the first version lacked a field (usually flags to modify the operation) that was added in the second version.

The structures and constants for all Btrfs ioctl() commands should be found in <linux/btrfs.h>; some distributions may require the installation of a development package to get that header.

  • Cloning files. The Btrfs copy-on-write (COW) mechanism can be used to make copies of files that share the underlying storage, but which still behave like separate files. A file that has been "cloned" in this way behaves like a hard link as long as neither the original file nor the copy is modified; once a change is made, the COW mechanism copies the modified blocks, causing the two files to diverge. Cloning an entire file is a simple matter of calling:

        status = ioctl(dest, BTRFS_IOC_CLONE, src);

    Where dest and src are open file descriptors indicating the two files to operate on; dest must be opened for write access. Both files must be in the same Btrfs filesystem.

    To clone a portion of a file's contents, one starts with one of these structures:

        struct btrfs_ioctl_clone_range_args {
      	__s64 src_fd;
      	__u64 src_offset, src_length;
      	__u64 dest_offset;

    The structure is then passed as the argument to the BTRFS_IOC_CLONE_RANGE ioctl() command:

        status = ioctl(dest, BTRFS_IOC_CLONE_RANGE, &args);

    As with BTRFS_IOC_CLONE, the destination file descriptor is passed as the first parameter to ioctl().

    Note that the clone functionality is also available in reasonably modern Linux systems using the --reflink option to the cp command.

  • Explicit flushing. As with any other filesystem, Btrfs will flush dirty data to permanent storage in response to the fsync() or fdatasync() system calls. It is also possible to start a synchronization operation explicitly with:

        u64 transid;
        status = ioctl(fd, BTRFS_IOC_START_SYNC, &transid);

    This call will start flushing data on the filesystem containing fd, but will not wait for that operation to complete. The optional transid argument will be set to an internal transaction ID corresponding to the requested flush operation. Should the need arise to wait until the flush is complete, that can be done with:

        status = ioctl(fd, BTRFS_IOC_WAIT_SYNC, &transid);

    The transid should be the value returned from the BTRFS_IOC_START_SYNC call. If transid is a null pointer, the call will block until the current transaction, whatever it is, completes.

  • Transaction control. The flush operations can be used by an application that wants to ensure that one transaction completes before starting something new. Programmers who want to live dangerously, though, can use the BTRFS_IOC_TRANS_START and BTRFS_IOC_TRANS_END commands (which take no arguments) to explicitly begin and end transactions within the filesystem. All filesystem operations made between the two calls will become visible to other processes in an atomic manner; partially completed transactions will not be seen.

    The transaction feature seems useful, but one should heed well this comment from fs/btrfs/ioctl.c:

         * there are many ways the trans_start and trans_end ioctls can lead
         * to deadlocks.  They should only be used by applications that
         * basically own the machine, and have a very in depth understanding
         * of all the possible deadlocks and enospc problems.

    Most application developers, one might imagine, lack this "very in depth understanding" of how things can go wrong within Btrfs. Additionally, there seems to be no way to abort a transaction; so, for example, if an application crashes in the middle of a transaction, the transaction will be ended by the kernel and the work done up to the crash will become visible in the filesystem. So, for most developers considering using this functionality, the right answer at this point is almost certainly "don't do that." Anybody who wants to try anyway will need the CAP_SYS_ADMIN capability to do so.

There are quite a few more ioctl() commands supported by Btrfs, but, as mentioned above, most of them are probably more conveniently accessed by way of the btrfs tool. For the curious, the available commands can be found at the bottom of fs/btrfs/ioctl.c in the kernel source tree.

Series conclusion

At this point, LWN's series on the Btrfs filesystem concludes. The major functionality offered by this filesystem, including device management, subvolumes, snapshots, send/receive and more, has been covered in the five articles that make up this set. While several developers have ideas for other interesting features to add to the filesystem, chances are that most of that feature work will not go into the mainline kernel anytime soon; the focus, at this point, is on the creation of a stable and high-performance filesystem.

There are few knowledgeable developers who would claim that Btrfs is fully ready for production work at this time, so that stabilization and performance work is likely to go on for a while. That said, increasing numbers of users are putting Btrfs to work on at least a trial basis, and things are getting more solid. Predictions of this type are always hard to make successfully, but it seems that, within a year or two, Btrfs will be accepted as a production-quality filesystem for an increasingly wide range of use cases.

Comments (4 posted)

Namespaces in operation, part 7: Network namespaces

By Jake Edge
January 22, 2014
Namespaces in operation

It's been a while since last we looked at Linux namespaces. Our series has been missing a piece that we are finally filling in: network namespaces. As the name would imply, network namespaces partition the use of the network—devices, addresses, ports, routes, firewall rules, etc.—into separate boxes, essentially virtualizing the network within a single running kernel instance. Network namespaces entered the kernel in 2.6.24, almost exactly five years ago; it took something approaching a year before they were ready for prime time. Since then, they seem to have been largely ignored by many developers.

Basic network namespace management

As with the others, network namespaces are created by passing a flag to the clone() system call: CLONE_NEWNET. From the command line, though, it is convenient to use the ip networking configuration tool to set up and work with network namespaces. For example:

    # ip netns add netns1

This command creates a new network namespace called netns1. When the ip tool creates a network namespace, it will create a bind mount for it under /var/run/netns; that allows the namespace to persist even when no processes are running within it and facilitates the manipulation of the namespace itself. Since network namespaces typically require a fair amount of configuration before they are ready for use, this feature will be appreciated by system administrators.

The "ip netns exec" command can be used to run network management commands within the namespace:

    # ip netns exec netns1 ip link list
    1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN mode DEFAULT 
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

This command lists the interfaces visible inside the namespace. A network namespace can be removed with:

    # ip netns delete netns1

This command removes the bind mount referring to the given network namespace. The namespace itself, however, will persist for as long as any processes are running within it.

Network namespace configuration

New network namespaces will have a loopback device but no other network devices. Aside from the loopback device, each network device (physical or virtual interfaces, bridges, etc.) can only be present in a single network namespace. In addition, physical devices (those connected to real hardware) cannot be assigned to namespaces other than the root. Instead, virtual network devices (e.g. virtual ethernet or veth) can be created and assigned to a namespace. These virtual devices allow processes inside the namespace to communicate over the network; it is the configuration, routing, and so on that determine who they can communicate with.

When first created, the lo loopback device in the new namespace is down, so even a loopback ping will fail:

    # ip netns exec netns1 ping
    connect: Network is unreachable
Bringing that interface up will allow pinging the loopback address:
    # ip netns exec netns1 ip link set dev lo up
    # ip netns exec netns1 ping
    PING ( 56(84) bytes of data.
    64 bytes from icmp_seq=1 ttl=64 time=0.051 ms
But that still doesn't allow communication between netns1 and the root namespace. To do that, virtual ethernet devices need to be created and configured:
    # ip link add veth0 type veth peer name veth1
    # ip link set veth1 netns netns1
The first command sets up a pair of virtual ethernet devices that are connected. Packets sent to veth0 will be received by veth1 and vice versa. The second command assigns veth1 to the netns1 namespace.
    # ip netns exec netns1 ifconfig veth1 up
    # ifconfig veth0 up
Then, these two commands set IP addresses for the two devices.
    # ping
    PING ( 56(84) bytes of data.
    64 bytes from icmp_seq=1 ttl=64 time=0.087 ms
    # ip netns exec netns1 ping
    PING ( 56(84) bytes of data.
    64 bytes from icmp_seq=1 ttl=64 time=0.054 ms
Communication in both directions is now possible as the ping commands above show.

As mentioned, though, namespaces do not share routing tables or firewall rules, as running route and iptables -L in netns1 will attest.

    # ip netns exec netns1 route
    # ip netns exec netns1 iptables -L
The first will simply show a route for packets to the 10.1.1 subnet (using veth1), while the second shows no iptables configured. All of that means that packets sent from netns1 to the internet at large will get the dreaded "Network is unreachable" message. There are several ways to connect the namespace to the internet if that is desired. A bridge can be created in the root namespace and the veth device from netns1. Alternatively, IP forwarding coupled with network address translation (NAT) could be configured in the root namespace. Either of those (and there are other configuration possibilities) will allow packets from netns1 to reach the internet and for replies to be received in netns1.

Non-root processes that are assigned to a namespace (via clone(), unshare(), or setns()) only have access to the networking devices and configuration that have been set up in that namespace—root can add new devices and configure them, of course. Using the ip netns sub-command, there are two ways to address a network namespace: by its name, like netns1, or by the process ID of a process in that namespace. Since init generally lives in the root namespace, one could use a command like:

    # ip link set vethX netns 1
That would put a (presumably newly created) veth device into the root namespace and it would work for a root user from any other namespace. In situations where it is not desirable to allow root to perform such operations from within a network namespace, the PID and mount namespace features can be used to make the other network namespaces unreachable.

Uses for network namespaces

As we have seen, a namespace's networking can range from none at all (or just loopback) to full access to the system's networking capabilities. That leads to a number of different use cases for network namespaces.

By essentially turning off the network inside a namespace, administrators can ensure that processes running there will be unable to make connections outside of the namespace. Even if a process is compromised through some kind of security vulnerability, it will be unable to perform actions like joining a botnet or sending spam.

Even processes that handle network traffic (a web server worker process or web browser rendering process for example) can be placed into a restricted namespace. Once a connection is established by or to the remote endpoint, the file descriptor for that connection could be handled by a child process that is placed in a new network namespace created by a clone() call. The child would inherit its parent's file descriptors, thus have access to the connected descriptor. Another possibility would be for the parent to send the connected file descriptor to a process in a restricted network namespace via a Unix socket. In either case, the lack of suitable network devices in the namespace would make it impossible for the child or worker process to make additional network connections.

Namespaces could also be used to test complicated or intricate networking configurations all on a single box. Running sensitive services in more locked-down, firewall-restricted namespace is another. Obviously, container implementations also use network namespaces to give each container its own view of the network, untrammeled by processes outside of the container. And so on.

Namespaces in general provide a way to partition system resources and to isolate groups of processes from each other's resources. Network namespaces are more of the same, but since networking is a sensitive area for security flaws, providing network isolation of various sorts is particularly valuable. Of course, using multiple namespace types together can provide even more isolation for both security and other needs.

Comments (29 posted)

Patches and updates

Kernel trees


Core kernel code

Development tools

Device drivers


Filesystems and block I/O

Memory management


Virtualization and containers


Page editor: Jonathan Corbet


OSTree for Fedora

By Jake Edge
January 23, 2014

Linux distributions are typically delivered either in piecemeal fashion via packages or as a single bundle like Android or Chrome OS. Colin Walters's OSTree project is taking something of a middle course. It is delivering a kind of bundle, but with a twist: the tool makes it easy to switch between multiple independent operating systems that are all simultaneously installed on the system.

We first looked at OSTree as part of our coverage of GUADEC 2012. Since that talk, though, Walters has not been idle. His gnome-continuous project has been building OSTree versions of GNOME using continuous integration for some time. Beyond that, he recently announced an OSTree repository for Fedora: fedostree:

I've often struggled with explaining OSTree to people - but for the audience here, I want to emphasize that OSTree is designed to be *complementary* to package systems like rpm/dpkg. While OSTree does take over some roles from RPM such as handling /etc, if you study it carefully, I think you'll come to agree.

The overall vision is to change Fedora (and really its prominent downstream) into a less flexible, but more reliable set of products, tested and delivered *much* *much* faster.

Rather than break things up into packages, OSTree builds fully assembled, versioned filesystem trees that hold what would have gone into a set of package (e.g. RPM) files. All of the binaries, libraries, documentation files, and so on are gathered together into the OSTree repository and can be retrieved using the ostree command. The fedostree installation instructions make it clear that the model Walters used was Git; many of the commands are clearly patterned after Git's. In fact, the project's home page calls it "git for operating system binaries".

For Fedora, one can install multiple different kinds of trees, based on either Fedora 20 or Rawhide, and switch between them at boot time. Each of the trees will have a focus on a particular project's output, such as GNOME, Docker, FreeIPA, or others. Those can be combined with a tree that provides the minimal F20 or Rawhide base to produce a system that can boot and run the project's output, and can do so repeatedly as each new version gets built from newer upstream code. Developers on those projects will then have a tool to quickly and easily switch between multiple versions of the project. That can make bisecting problems much easier, for example. Beyond the project documentation and Walters's announcement, he also describes some of the details of the tree switching in a blog post.

Each of the OSTree-installed systems will share the /etc directory; changes will be merged with those made by the other OSTrees. Also, /var will be shared between the instances. Beyond that, OSTree does quite a bit of filesystem rearranging to enable it to switch between the different trees. In the blog post, Walters gives some details:

Let’s try it; say you’ve installed the “fedostree/20/x86_64/base/minimal” ref. Now you want to change to the tree containing @standard along with docker-io. With the latest OSTree, this is just
# ostree admin switch fedostree/20/x86_64/server/docker-io

Semantically, this is somewhat like doing yum install @standard docker-io – except the transition is fully atomic. Furthermore, we get an exact replication of the tree from the build server; if for example we were transitioning from say the fedostree/20/workstation/gnome/core tree, then we wouldn’t get a mix of GNOME packages and docker-io. Our running GNOME tree would be untouched, still on disk, still running – when we reboot, then we’d be in the new server/docker-io tree. If we then did from inside the booted docker-io tree:

# ostree admin switch fedostree/20/workstation/gnome/core

Nothing would be downloaded (unless the ref was updated), and we’d be back to GNOME (again, atomically). All along this process, our changes to /etc will have a 3 way merge applied, and our data in /var will be carried along.

In addition to bisecting problems in broken umbrella projects (like GNOME or Mesa) that may not be easy to find otherwise, there are a number of other possible uses that Walters lists on the fedostree page. Installing RHEL and Fedora in parallel, or Fedora and Rawhide, would be one possibility. That would make switching between them far easier, while still keeping each up to date. It would also make testing Rawhide much safer since the only real risk is a filesystem-corrupting kernel bug. Making an Android-like recovery partition with a minimal install is another idea.

As he notes in the announcement (and elsewhere), fedostree (and OSTree itself) are rather "bleeding edge" at this point. He is looking for brave souls to evaluate the ideas, tools, and repositories to help him find problems. There is also a list of "development areas" that need to be addressed in the near future.

OSTree/fedostree is clearly a project that is just getting off the ground. OSTree has proven itself useful for GNOME continuous integration, so Walters plans to bring that to fedostree; trees would get rebuilt (and booted as a smoketest) immediately after a change is made that affects any package relevant to that tree. Before much of that can happen, though, some testing and shaking things out is required. It should be interesting to see where things go from here.

Comments (9 posted)

Brief items

Distribution quotes of the week

QA needs to recognize that people problems aren't always best solved with the use of command-line utilities.
-- Rich Freeman

I would like it on the record that I consider Fedora's current values - specifically, our *commitment to* (not just *preference for*) libre software - an immense social and technical good, I am strongly committed to those values, I am strongly opposed to this change (defined as 'any change which leads to the seamless availability of non-libre software in Fedora products'), and I might have to re-consider my work on Fedora if this change were to be approved.
-- Adam Williamson

Comments (none posted)

FreeBSD 10.0

The FreeBSD Release Engineering Team has announced the release of FreeBSD 10.0. Some highlights include; clang has replaced GCC as the default compiler on some architectures, Unbound has been imported as the local caching DNS resolver, BIND has been removed, make has been replaced with bmake, ZFS improvements, major enhancements in virtualization, and more. See the release notes for more information.

Comments (16 posted)

GNU Linux-libre 3.13-gnu

GNU Linux-libre 3.13-gnu deblobbing scripts, tarballs and patches are available. "Aside from the usual updates to deblobbing and false positive rules, to cover new drivers and changes to preexisting drivers, no major changes were made to the deblobbing infrastructure in this release."

Full Story (comments: none)

Distribution News

Debian GNU/Linux

bits from the DPL -- December 2013

Debian Project Leader Lucas Nussbaum presents a few bits about his December activities. Topics include delegations, public discussions, the events team, Debian copyrights, assets, and more.

Full Story (comments: none)


Fedora Workstation proposal: ease installation of non-free software

The Fedora Workstation working group has come out with a proposal to ease Fedora's traditional "see no evil" approach to non-free software in the hope of making the distribution appealing to a wider group of users. "In order to keep with the Fedora policy of only shipping free software we will only make available 3rd party software that offers their own repository for their software. Examples here include Google Chrome and Adobe Acrobat."

Full Story (comments: 51)

Newsletters and articles of interest

Distribution newsletters

Comments (none posted)

CentOS Project Leader Karanbir Singh Opens Up on Red Hat Deal (

Over at, Libby Clark has an interview with CentOS project leader Karanbir Singh about the recent deal between Red Hat and CentOS. In it, he discusses how the deal came about, what it means for both CentOS and Red Hat, and why it is a good thing for all concerned. "The CentOS project as it was is going to stay intact, but we'll be working on infrastructure other projects need to be successful with us. We won't be delivering the features. We'll make it as trivial as possible to come in and do the build you need, without needing to learn everything about building CentOS images. People will come together to work on the same problems."

Comments (100 posted)

Page editor: Rebecca Sobol


Of bytes and encoded strings

By Jake Edge
January 22, 2014

String handling is quite different between Python 2 and Python 3, largely because the latter is designed to handle Unicode natively. But those differences make it more difficult for Python-2-based projects to make the move to Python 3. A proposal to add a feature to the language that would make that transition easier has led to a series of massive threads on the python-dev mailing list. A seemingly simple idea has interesting ramifications, both technical and philosophical.

A bit of an introduction to Python string types is probably in order. Both Python 2 and Python 3 have two types that hold string-like data, but they are quite different. Python 2 has str and unicode. Its str contains 8-bit data that could be binary data or encoded text data in some unknown encoding. Typically, it is hoped that the unknown encoding will be ASCII compatible (i.e. shares the same mappings as 7-bit ASCII), but that is not guaranteed. On the other hand, unicode strings (e.g. u'foo') contain a series of Unicode code points.

For Python 3, though, the str type contains a sequence of Unicode code points, while the bytes type (e.g. b'foo') contains a sequence of integer values in the range 0–255, without any further interpretation of those values. The language removes the ambiguous string type in favor of clear boundaries between Unicode strings and binary data (which might, in fact, be encoded text). Also, unlike Python 2, the two string types are not interoperable. unicode + bytes is not defined and will raise an exception. So, in order to get external data (from a file or socket, say) into a Python 3 str, it must be decoded from a known encoding (e.g. "ascii", "utf-8", etc.), which is quite different from the Python 2 behavior. This is all covered in much more detail in various places including Nick Coghlan's Python 3 Q&A document, this section in particular.

Various projects that are looking into moving to Python 3 have stumbled over a particular problem: formatting (or interpolating) values into bytes arrays. The following is illegal in Python 3 today:

    b'%d' % 42
One might expect that to create the bytes string b'42', but instead it gives a TypeError exception. Similarly, both of the following give TypeError exceptions:
    b'%s' % 'foo'
    b'%s' % b'foo'
The former would require an implicit encoding of the Unicode str into bytes, which is exactly the kind of "magic" that Python 3 is trying to avoid. The failure of the latter is perhaps the most surprising, but there is no formatting operator ("%") defined for bytes.

PEP 460

Both the Mercurial and Twisted projects have requested support for certain kinds of bytes formatting to help in their Python 3 porting efforts. That is what led to an RFC post from Victor Stinner that contained a draft of Python Enhancement Proposal (PEP) 460. The proposal would add string-like interpolation to the Python 3 bytes type. Both the "%" operator and the format() method would be supported for a wide range of formatting characters. In Stinner's original proposal, "%s" would only be supported for numeric or bytes arguments. It was targeted as a change for Python 3.5, which is due sometime late in 2015.

There were multiple folks commenting on the idea in the thread, both for and against the idea. For example, Mark Shannon thinks that adding formatting for bytes strings, particularly when converting things like numbers to their ASCII representation, violates "the separation of str and bytes in the first place". On the other hand, Marc-Andre Lemburg agrees with Stinner's proposal as it makes things much easier in "situations where you have to work on data which is encoded text using multiple (sometimes even unknown) encodings in a single data chunk. Think MIME messages, mbox files, diffs, etc."

With Stinner's approval, Antoine Pitrou reworked PEP 460 to restrict the feature to a small subset of the formatting codes ("%s" for bytes and "%c" for single integers in the 0–255 range). The rework also added support for the bytearray type (mutable bytes). It got an immediate +1 from Coghlan, who was "initially dubious about the idea". Others were not happy to see the removal of numeric formatting. As Ethan Furman pointed out, allowing ASCII (in the form of numeric formatting) to be interpolated into the bytes type does not mean that the type is ASCII compatible:

No, it implies that portion of the byte stream is ASCII compatible. And we have several examples: PDF, HTML, DBF, just about every network protocol (not counting M$), and, I'm sure, plenty I haven't heard of.

At some level, it comes down to a question of "language purity" and how far to push the separation between str and bytes. In a fairly long message, Stephen Hansen described what currently needs to be done for HTTP's Content-Length header:

It seems those who are declaring a purity priority in bytes/string separation think it reasonable to do things like:
    headers.append((b"Content-Length": ("%d" % (len(content))).encode("ascii")))
Or something. In the middle of processing a stream, you need to convert this number into a string then encode it into bytes to just represent the number as the extremely common, widely-accessible 7-bit ascii subset of its numerical value. This isn't some rare, grandiose or fiendish undertaking, or trying to merge Strings and Bytes back together: this is the simple practical recognition that representing a number as its ascii-numerical value is actually not at all uncommon.

This position seems utterly astonishing in its ridiculousness to me. The recognition that the number "123" may be represented as b"123" surprises me as a controversial thing, considering how often I see it in real life.

But Coghlan is tired of seeing repeated arguments that ultimately lead to what he sees as a dangerous blurring of the line between str and bytes. One possible solution that he sees is in the nascent asciicompat project, which would add a new type (first as an extension, then perhaps to the language core) that had some of the properties being advocated in the thread.

PEP 461

The discussion in that thread (and others) got more heated than most in python-dev. Eventually, Furman forked the PEP into PEP 461, adding back the numeric formatting. Brett Cannon characterized the split as one that is between people intent on "explicit is better than implicit" (PEP 460) versus those who are focused on "practicality beats purity" (PEP 461). Both of those statements come from The Zen of Python, so it comes down to a question of the priority each person puts on them.

As it turns out, though, Python benevolent dictator for life (BDFL) Guido van Rossum is clearly in the "practicality beats purity" camp. He noted that there already is some inherent ASCII bias in the bytes type. Others had earlier pointed out some similar examples, but Van Rossum succinctly summarized the case:

But this does not mean the bytes type isn't allowed to have a noticeable bias in favor of encodings that are ASCII supersets, even if not all bytes objects contain such data (e.g. image data, compressed data, binary network packets, and so on).

IMO it's totally fine and consistent if b'%d' % 42 returns b'42' and also for b'{}'.format(42) to return b'42'. There are numerous places where bytes are already assumed to use an ASCII superset:

  • byte literals: b'abc' (it's a syntax error to have a non-ASCII character here)
  • the upper() and lower() methods modify the ASCII letter positions
  • int(b'42') == 42, float(b'3.14') == 3.14

That summary seemed to cut through a lot of the argument (Van Rossum's BDFL role plays a part in that, of course). The only real sticking point seems to be what should be returned for constructs like:

    b'%s' % 'x'

Van Rossum thinks it should work as follows:

b'%s' % 'x' == b"'x'" (i.e. the three-byte string containing an 'x' enclosed in single quotes)

Not everyone is happy with that, but it is in keeping with an existing behavior, he said:

It's symmetric with how '%s' % b'x' returns "b'x'". Think of it as payback time. :-)

While that symmetry had some appeal, it was seen as unnecessary and potentially a source of hard-to-track-down bugs. So, after another lengthy discussion, most seemed to settle for a restricted version of the "%s" specifier: it will only accept arguments that provide the buffer protocol or has a __bytes__() method, otherwise a TypeError will be raised. In practice, that will restrict %s to some internal CPython array types and to bytes; str and numeric types do not have __bytes__() methods.

In several followup threads, Furman revised PEP 461 into something that is getting close to a final version. Along the way, it lost support for the format() method, so only support for the __mod__() method ("%" operator) would be added. The current version of PEP 461 is still missing a "Rationale" section, and there have been some minor quibbles with the wording and such, but overall it is nearly complete. That is not to say that everyone agrees with the PEP, but it would appear that Van Rossum does, which makes it seem likely that we will see it in Python 3 before too long.

Comments (78 posted)

Brief items

Quotes of the week

Why, in this day and age, do smart tech people think Chatham House Rules will promote community collaboration?
Tom Marble

My theory is people like the name.
Richard Fontana, in response to Tom Marble.

Comments (3 posted)

Rivendell 2.7.0 released

Version 2.7.0 of the Rivendell radio automation system has been released. This version adds support for AIFF audio import, optimized CD ripping, and the ability to customize log message formatting through templates, among other changes.

Full Story (comments: none)

Libraw 0.16 available

Version 0.16 of the Libraw library for raw photo file manipulation has been released. This version adds support for many more camera models, as well as two new demosaicing algorithms and improved Exif tag parsing.

Comments (none posted)

LabPlot 2.0.0 released

Version 2.0.0 of the LabPlot data analysis and visualization application for KDE has been released. Denoted the "first stable version," the new release is a complete rewrite of the LabPlot 1.x codebase, and supports generating 2D plots in a wide array of output formats, including LaTeX.

Comments (none posted)

Newsletters and articles

Development newsletters from the past week

Comments (none posted)

Pocock: enabled for SIP federation, who will be next?

At his blog, Daniel Pocock notes Debian's new SIP federation service, which provides voice-over-IP (VoIP) service to approximately 1000 Debian Developers. "This means packages providing SIP (and soon XMPP/Jabber) will hopefully be under much more scrutiny from developers in the lead up to the next major Debian release," he observes, although the broader benefit might be that other free software community members will start to take federated SIP support more seriously.

Comments (none posted)

Happy 25th Tcl! (TkDocs)

The TkDocs weblog notes that the Tcl language is 25 years old. "For anyone under 40 years old, you can think of Tcl like Ruby or Python, if those languages didn't have objects, if absolutely everything in them was treated as a string, and if every syntax rule added to the language resulted in its creator being tortured for one year." The humor goes on, but the article also expresses serious appreciation for some of the ideas introduced by Tcl.

Comments (1 posted)

Page editor: Nathan Willis


Articles of interest

LinuxGizmos: Ultra-secure “Blackphone” to run on Android-based “PrivatOS”

LinuxGizmos is reporting on the Blackphone, a just-announced "ultra-secure" mobile phone to be jointly produced by Geeksphone (who released the Firefox OS developer phones) and security-conscious email vendor Silent Circle. The OS is advertised as "PrivatOS," a security-tuned derivative of Android. The phone is expected to be available in late February, in time to debut at Barcelona's Mobile World Congress. The Blackphone site calls it "the world's first smartphone which prioritizes the user's privacy and control, without any hooks to carriers or vendors," although as LinuxGizmos points out, few other details—including hardware details—are available yet.

Comments (24 posted)

Calls for Presentations

CFP Deadlines: January 23, 2014 to March 24, 2014

The following listing of CFP deadlines is taken from the CFP Calendar.

DeadlineEvent Dates EventLocation
January 28 June 19
June 20
USENIX Annual Technical Conference Philadelphia, PA, USA
January 30 July 20
July 24
OSCON 2014 Portland, OR, USA
January 31 March 29 Hong Kong Open Source Conference 2014 Hong Kong, Hong Kong
January 31 March 24
March 25
Linux Storage Filesystem & MM Summit Napa Valley, CA, USA
January 31 March 15
March 16
Women MiniDebConf Barcelona 2014 Barcelona, Spain
January 31 May 15
May 16
ScilabTEC 2014 Paris, France
February 1 April 29
May 1
Android Builders Summit San Jose, CA, USA
February 1 April 7
April 9
ApacheCon 2014 Denver, CO, USA
February 1 March 26
March 28
Collaboration Summit Napa Valley, CA, USA
February 3 May 1
May 4
Linux Audio Conference 2014 Karlsruhe, Germany
February 5 March 20 Nordic PostgreSQL Day 2014 Stockholm, Sweden
February 8 February 14
February 16
Linux Vacation / Eastern Europe Winter 2014 Minsk, Belarus
February 9 July 21
July 27
EuroPython 2014 Berlin, Germany
February 14 May 12
May 16
OpenStack Summit Atlanta, GA, USA
February 27 August 20
August 22
USENIX Security '14 San Diego, CA, USA
March 10 June 9
June 10
Erlang User Conference 2014 Stockholm, Sweden
March 14 May 20
May 22
LinuxCon Japan Tokyo, Japan
March 14 July 1
July 2
Automotive Linux Summit Tokyo, Japan
March 14 May 23
May 25
FUDCon APAC 2014 Beijing, China
March 16 May 20
May 21
PyCon Sweden Stockholm, Sweden
March 17 June 13
June 15
State of the Map EU 2014 Karlsruhe, Germany
March 21 April 26
April 27
LinuxFest Northwest 2014 Bellingham, WA, USA

If the CFP deadline for your event does not appear here, please tell us about it.

Upcoming Events

SCALE 12X: Event update

The Southern California Linux Expo (SCALE) will take place February 21-23, 2014 in Los Angeles, CA. Speakers slots may still be available for the Beginning Sysadmin Track. Other items covered in this update include a wide range of events including SCALE TNG, Chef training, DevOps Day, UbuCon, and much more.

Full Story (comments: none)

The Linux Foundation's 2014 event schedule

The Linux Foundation has announced its schedule of events for 2014. "LinuxCon and CloudOpen North America will take place this year in Chicago and will be co-located with the Linux Kernel Summit. LinuxCon and CloudOpen Europe will be in Duesseldorf, Germany, along with Embedded Linux Conference, KVM Forum and Linux Plumbers Conference." Several other events appear on the schedule as well; the call for papers for all events is open now.

Comments (none posted)

Events: January 23, 2014 to March 24, 2014

The following event listing is taken from the Calendar.

January 31 CentOS Dojo Brussels, Belgium
February 1
February 2
FOSDEM 2014 Brussels, Belgium
February 3
February 4
Config Management Camp Gent, Belgium
February 4
February 5
Open Daylight Summit Santa Clara, CA, USA
February 7
February 9
Django Weekend Cardiff Cardiff, Wales, UK
February 7
February 9 Brno, Czech Republic
February 14
February 16
Linux Vacation / Eastern Europe Winter 2014 Minsk, Belarus
February 21
February 23 2014 Gandhinagar, India
February 21
February 23
Southern California Linux Expo Los Angeles, CA, USA
February 25 Open Source Software and Govenrment McLean, VA, USA
February 28
March 2
FOSSASIA 2014 Phnom Penh, Cambodia
March 3
March 7
Linaro Connect Asia Macao, China
March 6
March 7
Erlang SF Factory Bay Area 2014 San Francisco, CA, USA
March 15
March 16
Chemnitz Linux Days 2014 Chemnitz, Germany
March 15
March 16
Women MiniDebConf Barcelona 2014 Barcelona, Spain
March 18
March 20
FLOSS UK 'DEVOPS' Brighton, England, UK
March 20 Nordic PostgreSQL Day 2014 Stockholm, Sweden
March 21 Bacula Users & Partners Conference Berlin, Germany
March 22 Linux Info Tag Augsburg, Germany
March 22
March 23
LibrePlanet 2014 Cambridge, MA, USA

If your event does not appear here, please tell us about it.

Page editor: Rebecca Sobol

Copyright © 2014, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds