By Jonathan Corbet
December 15, 2010
Richard Stallman recently
went
on record against cloud-based computing, and against Google's ChromeOS
in particular. Putting one's personal data on remote servers, he says,
necessarily entails loss of control over that data. It is far better to
keep one's data on a system which is under one's physical control. As with
most things, Richard has been most consistent with this message; he has
been saying similar things for a long time. But the increase in
cloud-based services - and systems designed to direct users toward them -
is adding urgency to this message.
Your editor does not always agree with Richard, but Richard has a point here.
We have worked for many years to build systems which we have some degree of
control over, with quite a bit of success. Even systems which have
traditionally been severely closed - phone handsets, for example - are
becoming more hackable over time. A suitably motivated and skilled user
can avoid proprietary software, and the long list of antifeatures such
software tends to include, much of the time. The situation is not perfect,
but things could certainly have been a lot worse.
When our systems become little more than a window into somebody else's
server, though, that control disappears. The results are predictable:
- People can come to depend on cloud-based services, but the providers
of those services assert their right to pull the plug at any time.
The eviction of Wikileaks from Amazon's cloud is a recent,
high-profile example, but almost every well-known network-based
service is followed by stories of users who have been locked out for
seemingly trivial (or nonsensical) reasons.
- Stories of data misuse abound. Facebook puts profile pictures into
advertisements served to others. Gmail reads
messages and tailors advertisements to match. Email addresses
find their way onto spam lists. Many sites track their users'
activity across the web as a whole and do their best to monetize that
information. And so on.
- Resources in the cloud are cloudy at best; reports
that Amazon has resumed deleting books that Kindle owners believed
they owned are just the latest example of when can happen when "our"
stuff lives at somebody else's will.
- Security breaches and data loss are a common occurrences.
- Many cloud-based services seem to maintain an open-door policy for
governmental agencies looking for information. There is no way to
know what information has been disclosed to whom.
With regard to the last item above, it is encouraging that a US appeals
court has just ruled
that email cannot be seized from a third-party provider without a search
warrant. But it is highly discouraging that such a ruling was necessary in
the first place. Seemingly obvious concepts - like the privacy of email -
seem to fall by the wayside when network-based providers are involved.
Given all this, one might well wonder why such services are seeing any use
at all. The simple fact of the matter is that they are awfully
convenient. A web-based email account is far easier to set up and maintain
than an independent mail server - even for those who have the skills to
maintain such a system. Anybody who has been through the tiresome
experience of moving into a new phone can only be thrilled when that new
Android handset automatically downloads the contact list - and all
previously-installed applications. Establishing contacts and sharing
information is easy on social networking sites - and essentially impossible
otherwise. These services have brought a wide range of capabilities and
features to a wide community of users; there is clear value in what these
companies are providing.
It is well to warn users of what they are giving up when they place their
personal information on such a site. Making sure people know when a cloud
provider misbehaves is clearly the right thing to do. Many LWN readers
heed those warnings and
take a great deal of care to limit the information given to cloud providers
and to maintain their own infrastructure. But it is futile to tell the
rest of the world to avoid cloud-based services when we cannot point them
to any alternatives that are useful to them. Such advice will be ignored,
and the message as a whole may be lost.
The right response to the cloud problem is to create alternatives which
give a higher degree of control - and which are usable by people who have
no interest in putting their time into making things actually work. That
means solving problems at a number of levels. We need applications which
provide a rich experience to users which are not tied to any specific
machine; the web is the obvious way to provide that experience, but it
might not be the only way. Needless to say, these applications must be
free software if we are to trust them at all.
We need freedom-friendly policies that raise
the bar for what users expect and demand. We need a mechanism for
deploying these applications on the net which allows users to easily create
and maintain their own instances while interoperating with others.
It would be good to contemplate what could be done when terabyte storage on
mobile platforms is commonplace - we can always have all of our data in our
pockets. With pieces like these in place, we might begin to have a story
which can compete with the existing providers.
Something else is needed, though: a means for financing these services must
be developed. "Free" is awfully nice, but, as people far wiser than your
editor have observed, if you are not paying for a service, you are not the
provider's customer - you are their product. That is a relationship which
will inevitably lead to conflicts of interest. Establishing a more
straightforward relationship between users and providers of online services
seems like an important step toward improving both control and privacy.
That does not mean getting companies out of the services business - indeed,
it could mean the opposite - but it does mean renegotiating the
relationship. (And, naturally, companies have all the same freedom and
privacy interests that individuals do when it comes to obtaining services
on the net).
Recent events have convinced many people that, as we have become
increasingly dependent on the net, we have also lost control over it.
We may see a more focused effort in the coming years to take back control
and freedom at the network level. As with all of these battles, it will be
difficult; there is no shortage of powerful interests pushing toward
central control. But it's one that we should be able to win.
Comments (45 posted)
December 13, 2010
This article was contributed by Grant Likely
On December 1st and 2nd, a group of about 40 leaders and
engineers in the embedded Linux community met in San Francisco for a
for two-day summit. On the table was the Linux Foundation's
Yocto project, which was
announced in October at the Embedded Linux Conference Europe (ELCE),
and aims to make the development of custom embedded Linux
systems simpler and easier. The Linux Foundation wants to see broad
industry adoption of and involvement in Yocto, and this meeting was
called to provide an overview of the project and to set up a community
governance process. As such, the engineers and management from a wide
cross-section of the industry were invited including representatives
from Intel, Wind River, MontaVista, Mentor Graphics, Linaro, Dell,
Texas Instruments, Qualcomm, Red Hat, and HP.
Jim Zemlin, executive director of the Linux Foundation opened the
summit and welcomed all the attendees with the results of a survey
that attendees were asked to fill out before the meeting. In typical
Zemlin style, he pronounced that in the embedded space, "Linux is kicking ass".
The bad news is that there is a shortage of
well-trained embedded Linux developer and, in particular, system
engineers. There is a desperate need to get more people "in the
tent". The merger of the Consumer Electronics Linux Forum
(CELF) with the Linux Foundation,
announced in October, along with
the Yocto project creation
announced at the same time, means that the Linux
Foundation is now directing significant resources toward improving the
state of embedded Linux development.
As a first step, the Zemlin announced that Richard Purdie, the
leader of the Poky project, has
become a Linux Foundation fellow
in the role of Yocto project architect.
Before the summit, attendees were surveyed about their view of the Yocto project and what they
would need to make it a useful technology. Rather than attempting to
analyze the results, Zemlin provided the raw answers to the room (with
names removed) so that attendees could make of it what they would.
Not surprisingly, it appears that few people are familiar with the Yocto
project, and
many were hoping to come out of the meeting with a better
understanding of what it was and whether or not it would be useful for
their work.
An overview of the project
The rest of the morning was devoted to an overview of the Yocto
project by Purdie and Dirk Hohndel, Intel's Chief Linux and Open Source
Technologist. According to Hohndel and Purdie, Yocto is an umbrella
project covering several initiatives with the goal of reducing the
duplication of effort required to build custom embedded Linux
systems. While there are numerous tools for building embedded
systems, such as
OpenEmbedded and
BuildRoot, all of them still
require a lot of pain and effort to stabilize into something usable.
Hohndel wants to "raise the bar and produce a great shared base
technology which is easy to build products on top of". Yocto aims to
achieve this goal by directing effort to improving the build tools,
providing tested releases, and improving documentation.
The most prominent component under the umbrella is the Poky build
system which is a derivative of OpenEmbedded. Additionally, Yocto
sponsors related projects including Pseudo: a fakeroot replacement,
Swabber: a tool that detects cross-build contamination from the host
filesystems, Eclipse and Anjuta plugins, and an SDK generator.
Documentation, testing, and a build/test infrastructure is also
covered. An complete list of projects can be found on the Yocto
website.
Many questions were asked about the relationship between Poky, Yocto,
and OpenEmbedded. Rather than reiterate the discussion here, an
openembedded-devel
mailing list posting from Purdie provides an excellent overview, and
covers all of the points touched on in his presentation.
Poky is closely related to OpenEmbedded, and both projects share the
BitBake build
tool and much of the build package metadata. Poky is
sometimes characterized as a fork of OpenEmbedded, yet it does not
appear to be a competing fork since a great deal of care has been taken
to maintain compatibility between the two projects, and there is
a lot of cross-pollination of metadata and features.
Poky differs from OpenEmbedded in its focus and policies.
According to Purdie, while OpenEmbedded is a wonderful
tool, it is actually quite difficult to base a product on top of it.
Most companies using it need to make a large number of changes.
Poky, on the other hand, aims to be a well-tested, stabilized set of
build metadata in a form that requires far fewer changes for use in a
commercial product.
Questions were also asked about the relationship between Yocto
and MeeGo, and if the two projects were
duplicating effort. The answer is simply that the two projects
address completely different segments of the embedded Linux ecosystem.
Where MeeGo is about a common application framework and environment
with a stable ABI and a consistent interface, Yocto is instead focused on
highly customized device-specific embedded systems where all the
choices about what is included is up to the system builder.
Much concern was expressed about whether or not Yocto will be
effectively perceived as a single vendor project since it is being
championed by Intel and its subsidiary, Wind River.
Ironically, Intel and Wind River employees were within the
group expressing this concern. It would appear that the developers of
the project deeply believe in the Yocto approach, and don't want
to see it written off as an Intel-only tool. Both Hohndel and Zemlin
expressed several times that they really want to see Yocto become
a vendor-neutral project that improves the state of embedded Linux for
everyone. For this reason, ownership of the Yocto and Poky trademarks
have been transferred to the Linux Foundation, and the steering group
is open to all eligible member companies.
Project governance
After lunch the meeting split into a technical track and a governance
track. The governance track was focused on choosing the structure
for the Yocto steering group. Prior to the summit, a
governance structure was drafted to be discussed at the meeting. As
much as possible, the governance structure is to take a hands-off
approach to the day-to-day and technical decisions of the project,
while still providing the advantages of a legal entity and to have a
mechanism for managing engineering and financial resources.
A steering group will be formed with one seat held by each of the
initial participating companies. In the spirit of hands-off governance, the
membership agreement is low-key with primary requirements
being to maintain at least one full time Yocto engineer and, in
Zemlin's words, "don't be a jerk". He wants to protect against the
possibility of a company joining the steering group in order to sabotage
the project, and so the membership agreement provides a mechanism for
removing unfriendly members. It would appear that the history lessons
of SCO and UnitedLinux have been learned, and Zemlin wants to protect against
the possibility of the entire project getting sabotaged by a single
hostile member.
From the companies represented in the governance track, the reception
was generally positive. A few people voiced concern that Yocto would
be yet another embedded Linux solution to support over and above
Android and OpenEmbedded. However, the close relationship between
Poky and OpenEmbedded somewhat mitigates those fears and some hope
was expressed that the two projects would continue to work together,
and possibly even merge.
Concern was also expressed that the Linux Foundation membership requirement
effectively excludes independent and community developers from holding
a seat on the steering group. According to Zemlin, the intent of
the membership requirements are to raise the bar so that only
committed companies become involved. The steering group is
completely free to waive requirements as needed to make sure that key
people are not excluded from participating.
Zemlin proposed polishing up the membership agreement and sending it out
to prospective members in the next week or so with the goal of
forming the initial steering group by mid-December. Once the initial
group is formed, the group itself can vote to change and
adapt the steering requirements as needed by the project. In fact,
one of the first tasks that will be put to the steering group after it is
formed is to review the membership policies and make decisions about
how the group should be composed. Questions like how large it should be, how
many seats will be held by community developers, and how members are
selected, will be addressed.
Preparing for 1.0
Meanwhile the technical track set out to spend the afternoon discussing
the current state of the project and what needs to be fixed or
completed before cutting a 1.0 release. A lot of time was devoted to
discussing the differences between OpenEmbedded and Poky, and by the
end of the day the technical track probably spent about as much time
on governance issues as the governance track did.
The development model is one of the major differences between
OpenEmbedded and Poky, and has been characterized as a push vs. pull
model. "Push" refers to the fact that many OpenEmbedded developers
have direct commit access to the source repositories. Poky has
instead adopted a "pull" model where one developer maintains the canonical
source tree and other developers send pull request to the maintainer
of said tree (similar to how Linus Torvalds maintains his Linux tree).
OpenEmbedded and Poky also differ in policy about what recipes are
accepted into the core repository. OpenEmbedded has historically
taken a laissez-faire approach where anything is acceptable, as long
as it doesn't break other things, and so it includes a huge number
of recipes. Unfortunately it's openness is double-edged sword since
many of those recipes are broken, unmaintained, or for ancient
versions of software. It is easy to get new recipes into
OpenEmbedded, but it is also very difficult to know which recipes
actually work.
Poky on the other hand has strict policies about what is acceptable in
the core and only maintains recipes that are actually used and
tested. Typically Poky will not contain more than two recipes for a
given package; one known working version, and an unstable testing
version. It also keeps the scope limited to the core recipes
required to get a working build. It is not surprising that the 700
recipes currently in Poky are an order of magnitude fewer than the 8000
in OpenEmbedded.
Instead of adding recipes to the core, Poky is extended by using
"layers". Layers are a mechanism for adding
additional recipes for things like board support,
special toolchains, new architectures, and applications on top of the
Poky core.
Since anything in Poky can be overridden or extended with a layer,
Poky users have lots of flexibility to adapt it to their needs.
Mark Hatle commented that a typical Wind River build will consist of
about 7 layers; the core, a toolchain layer, the kernel layer, a board
support package layer, and one or more user-space layers.
Poky and OpenEmbedded
The push vs. pull debate came to the forefront when the idea of merging
Poky with OpenEmbedded was raised.
Some developers in the Yocto project would like to see
Poky work more closely with OpenEmbedded, and effort has already been
expended in that direction. Koen Kooi, the founder of the
OpenEmbedded-based Ångström project and a Texas Instruments engineer,
stated that he's prototyped using the OpenEmbedded recipes required by
Ångström as a layer on top of Poky. It isn't a full OpenEmbedded
port, but it does demonstrate feasibility.
Specifically, the proposed idea is to use Poky as a "common core" layer
for both the Yocto and OpenEmbedded projects, and OpenEmbedded would
be maintained as layers on top of Poky. The advantage would be a
larger block of common functionality being used and tested by both
projects, which presumably would result in a better quality core.
Several OpenEmbedded developers were in the room,
and while all seemed
favorable to the idea, concern was
expressed that the OpenEmbedded community at large would not take
kindly to the change for a few reasons.
Requiring the developers to switch to a pull model for the core code
is the first concern, but it ended up being easy to address. Since
each layer gets its own source repository, layer maintainers
can make their own decisions about development process and
commit access. Plus, since anything can be overridden by a layer, an
OpenEmbedded layer would be able to change anything in the core code
that doesn't fit OpenEmbedded's needs.
Second was the concern that it would be viewed as a "hostile
takeover" of OpenEmbedded by Poky. Much hand wringing and worry
accompanied this possibility and a fair bit of time was consumed by
this discussion. Politically, any such move would require the agreement
of the OpenEmbedded board of directors, and have general assent from
the development community at large.
There was also some discussion about the name, and whether or not
using the "Poky" name would be palatable to the OpenEmbedded
community, or if it would be viewed as diluting the "OpenEmbedded
Brand". However, this seems to be the least of these concerns as
both Poky and OpenEmbedded developers expressed flexibility on
naming issues if it actually becomes an issue.
While many of the concerns raised during this discussion were
important and needed to be considered, there isn't much evidence that
there is actually a problem yet. The OpenEmbedded developers took an
action item away from the meeting to continue discussing using Poky as the
base layer for OpenEmbedded with the rest of the
developer community, with the intent of having some resolution
by the end of December. Since a proposal to include OpenEmbedded
representation in the Yocto
steering group is also on the table, the initial formation of the
steering group will probably be delayed until early January to give
the OpenEmbedded community time to resolve their concerns and to
nominate a representative.
After that, the summit seemed to wind down and focus on technical
details or the finer points of what Yocto would provide. Tim Bird's
statement that he's always like the design of OpenEmbedded, but has never been
able to get it to work became somewhat a mantra for the remainder of
the event, and "solve Tim's problem" was often heard in relation to
making Yocto a reliable and easy to use set of tools. The
Yocto
1.0 roadmap
was touched upon, as well as work needed on the BitBake user interface
and the IDE integration features. More questions were asked to
clarify details about policy and governance issues, but for the most
part no more big issues came up.
A number of action items were identified as the summit wound up.
The business folks from represented companies are going to get
together and come back with some form of joint declaration of support
for the project in the next month. The OpenEmbedded representatives
will discuss with the rest of their community about closer cooperation with
Poky and Yocto. Jon Masters asked if there was
a regular technical conference call for the project. Hohndel replied
that there is currently an Intel/Wind River call, but that the project
needs to decide if continuing with a conference call is the best way
to communicate.
Clearly the Yocto project is still in its infancy and there is a lot of work
to be done before it can truly be considered a vendor-neutral project.
However, judging from this meeting and from an informal poll of attendees
afterward, the project seems to be off to a good start and it is
likely to be adopted by the major embedded Linux and silicon vendors.
Overall the summit ended with a positive and optimistic tone any by
all indications Yocto is a project that embedded Linux engineers
should be keeping their eyes on.
Comments (8 posted)
Here is LWN's thirteenth annual timeline of significant events in the Linux
and free software world for the year.
In what is becoming a fairly standard pattern, 2010 brought various patent
lawsuits, company acquisitions, new initiatives, and new projects. It also
brought new releases of the software that we use on a daily basis. There
were licensing squabbles and development direction
disagreements—all things that we have come to expect from the Linux
and free software world over a year's time. Also as expected, though, were
the improvements in the kernel, applications, distributions, and so on that
make up that world. Linux and free software just keep chugging along, and
we are very happy to be able to keep on reporting about it.
Like last year, we will be breaking this up into quarters, and this is our
report on July-September 2010. Sometime in the next week or two, we'll put
out the timeline for the last quarter of 2010.
This is version 0.8 of the 2010 timeline. There are almost certainly some
errors or omissions; if you find any, please send them to timeline@lwn.net.
LWN subscribers have paid for the development of this timeline, along with
previous timelines and the weekly editions. If you like what you see here,
or elsewhere on the site, please consider subscribing to LWN.
For those with a nostalgic bent, our timeline index page has links
to the previous twelve timelines and some other retrospective articles
going all the way back to 1998.
Python 2.7 is released as the last major version in the 2.x series
(announcement).
When a respected information source covers something where you have
on-the-ground experience, the result is often to make you wonder how much
fecal matter you've swallowed in areas outside your own expertise.
-- Rusty
Russell
3D graphics drivers that require proprietary user-space drivers (or
"blobs") are blocked from inclusion in the kernel by graphics
maintainer Dave Airlie (LWN coverage).
OpenSolaris governing board issues an ultimatum to Oracle
threatening dissolution if no contact person is put forward by Oracle (news
article).
The first draft of an Open Source Hardware Definition is released
(draft, version 1.1).
Akademy, KDE's yearly conference, is held in Tampere, Finland (wrap-up
press release).
ISO changes its standardization processes to avoid some of the
abuses seen in Microsoft's OOXML push (LWN coverage).
The Battle for Wesnoth struggles with possible GPL violations because of
its appearance in the Apple App Store, which to some extent parallels
the Gnu Go problems in May (LWN article).
I appreciate the fact that [Motorola's Lori] Fraleigh and Motorola are
honest in their disdain for software developers. Unlike Apple - who tries
to hide how developer-unfriendly its mobile platform is - Motorola readily
admits that they seek to leave developers as helpless as possible, refusing
to share the necessary tools that developers need to upgrade devices and to
improve themselves, their community, and their software.
-- Bradley
M. Kuhn
Ubuntu experiments with open font development, though the closed
beta of the Ubuntu font does raise some eyebrows (LWN coverage).
The Women's Caucus publishes recommendations for increasing the
participation of women in open source (recommendations).
The GNOME Users and Developers European Conference (GUADEC) is held in
The Hague, Netherlands (LWN coverage: Luis Villa keynote, GNOME 3 release plans, Privacy, encryption, and the
desktop, GNOME Shell, Banshee, and GUADEC notes).
OSCON is held in Portland, Oregon (LWN coverage: "Open phones" and Building communities).
FWIW, security by obscurity has a bad rep in some circles, but it is an
essential component of any serious security policy. It just should never be
the *only* component.
-- Guido van
Rossum
Jos Poortvliet becomes the new openSUSE community manager, succeeding
Joe "Zonker" Brockmeier (announcement).
The EFF wins three DMCA exemptions for cellphone unlocking,
cellphone "jailbreaking", and fair use of DVD content (press release).
GNOME and KDE announce a second Desktop Summit to be held in Berlin,
Germany in August 2011—it will combine GUADEC and Akademy much as
was done at the Gran Canaria Desktop Summit in 2009 (announcement).
Linux 2.6.35 is released (announcement, KernelNewbies summary).
If my corporate overlords told me I had to use my Exchange "messaging"
account for external email communication, they would get a quite clear 'no'
in response. My response may also contain suggestions that they use certain
other objects for purposes for which they were not designed.
-- David
Woodhouse
AppArmor, the long out-of-tree kernel security module, is merged for the
2.6.36 kernel largely due to the efforts of Canonical (LWN blurb).
BusyBox once again prevails in a GPL enforcement suit (Groklaw
coverage).
The fourth Linux Storage and Filesystems Summit is held just prior
to LinuxCon (LWN
reports: day 1 and day 2).
GNOME announces a new copyright assignment policy such that new
modules that require copyright assignment need to get explicit approval (announcement,
guidelines).
The Indian government has refused to let [researchers] review the machine,
and insists that it's tamper-proof. Even after the initial report came out
proving this not to be the case, the government has continued to insist the
machines are fine and have no problems. Here in the US, it's quite
troubling how much the government has relied on e-voting machines without
allowing security researchers to really test them, but at least they don't
arrest those who have been able to access and test the machines.
-- Mike
Masnick
LinuxCon is held in Boston (LWN coverage: MeeGo, Media panel, One billion files, Two bootcharts, and LinuxCon moments).
Illumos, an fork of OpenSolaris, is announced in the wake of
much uncertainty in the OpenSolaris community (LWN coverage).
The Linux Foundation announces the Open Compliance Program to help
companies ensure they are complying with the free software licenses of the
code they use (Linux.com
article).
Oracle sues Google over the Dalvik Java reimplementation that's used in
Android; the suit is for both patent and copyright violations (LWN's thoughts).
Combining both of their work together, they have been able to make a 20
minute long voice call from a baseband processor running a Free Software
GSM stack. For all we know, it is the first time anything remotely like
this has been done using community-developed Free Software. Five years ago
I would have thought it's impossible to pull this off with a small team of
volunteers. I'm very happy to see that I was wrong, and we actually could
do it. With less than half a dozen of developers, in less than nine months
of unpaid, spare-time work.
-- Harald
Welte
Allison Randal is named as Ubuntu Technical Architect in addition to
her role as Chief Architect for the Parrot virtual machine (announcement).
Vim 7.3 is released after two years of development on "Vi IMproved"
(announcement,
LWN review).
The OpenSolaris governing board resigns en masse as expected
due to Oracle's unwillingness to appoint a liaison to the board (Simon
Phipps's blog posting).
LinuxCon Brazil is held in São Paulo (LWN coverage: Linus & Andrew and Consumers, experts, or admins?).
CyanogenMod 6.0—replacement firmware for Android phones—is
released (LWN blurb and
review).
Linux 2.4.37.10 is released for those still hanging onto the 2.4
series (announcement and 2.4 EOL
plans).
Lessons? Well, as many have noted, reporters do need to ask more questions
about too-good-to-be-true technology stories. Coders and architects need
to realize (as most do) that you simply can't build a safe, secure,
reliable system without consulting with other people in the field,
especially when your real adversary is a powerful and resourceful
state-sized actor, and this is your first major project.
-- Danny
O'Brien on Haystack
The Mozilla Labs Gaming project is announced to foster web gaming
(LWN blurb).
Microsoft's CodePlex.com code hosting site donates $25,000 for Mercurial
development, assisting with project lead Matt Mackall's efforts to
fund his work on the distributed version control system (DVCS) (announcement).
Linus Torvalds becomes a US citizen, which delays some patch testing
while he registers to vote (lkml
mention).
Broadcom releases an open source driver for its current wireless
chipsets, but notably does not free the firmware for earlier chipsets
(announcement, LWN coverage).
Nevertheless, everyone I know that has reviewed the newly released
[Broadcom] driver code is being treated for eye cancer. I wouldn't expect
to see it in F-14.
-- John
Linville
Fedora decides not to ship systemd in Fedora 14 in a rather
late-breaking change; it should reappear in Fedora 15 (FESCO meeting minutes, related
LWN coverage).
The Mageia community fork of Mandriva is announced (announcement, LWN article).
PostgreSQL 9.0 is released (announcement, LWN article).
Diaspora makes its first code release of the alternative free social
networking platform, though there
are some major security concerns with the early release (announcement,
security
issues).
Qt 4.7 is released (announcement,
new features).
Oracle updates the kernel used in its RHEL-based "Unbreakable Linux",
moving from 2.6.18 to 2.6.32 which may show other enterprise Linux vendors
that they don't have to drag old kernels forward forever (LWN blurb).
My package made it into Debian-main because it looked innocuous enough; no
one noticed "locusts" in the dependency list.
-- xkcd
The LibreOffice fork of OpenOffice.org is announced by a large
portion of the OOo development community (announcement, LWN interview with Michael Meeks).
Linux-Kongress is held in Nürnberg, Germany (LWN coverage: GSM security).
LinuxCon Japan is held in Tokyo (LWN coverage: Stable kernels, Kernel messages, and SSDs and the block layer).
GNOME 2.32 is released as the last in the 2.x series (announcement, new features).
Comments (none posted)
Page editor: Jonathan Corbet
Security
December 15, 2010
This article was contributed by Nathan Willis
The Initiative For Open Authentication (OATH) is a security-vendor-based collaboration bent on developing a standardized "strong authentication" infrastructure using open standards. Not to be confused with the web-sharing cross-site authorization scheme OAuth, OATH has a broad set of security models it hopes to cover with a unified suite of protocols and APIs — collecting hardware-based, public-key infrastructure (PKI)-based, and one-time password (OTP)-based authentication into one framework.
The end goal is a noble one: a common framework that can use any of the three authentication systems on the client-side, so that it can be used just as easily to connect a user to a cell phone network (which use hardware-based authentication keyed off of the phones' SIM cards), a corporate VPN (which uses PKI authentication via X.509 certificates), and a web application (which uses an OTP protocol to authenticate the user without transmitting a traditional password). Password-based logins are inherently vulnerable, OATH argues, and the hardware-token systems sold by vendors have no established standard. Thus, why not replace both, and in a way that allows vendors to reuse some high-level APIs and software developers to build authentication-agnostic middleware.
OATH's rallying cry throughout its documentation is strong authentication but, interestingly enough, it does not offer a definition of that term. The group does not seem to mean "strong" in the sense of true multi-factor authentication (such as requiring both a hardware token and a password); rather it seems to encompass password-less authentication schemes built around either trusted hardware tokens or challenge-response protocols. Existing PKI systems appear to pass OATH's standards both for security and standardization.
The consortium has a white paper
[PDF] on its web site that elaborates on how an organization might
deploy different OATH-based systems. Overall, the architecture starts with
a client user having "strong authentication" credentials of one form or
another (smart card, SIM module, or software-based PKI certificate). The service that the user wishes to connect to could be a VPN, a corporate WiFi or GSM network, or a web application. In any case, the company setting up the service would use one of OATH's strong authentication algorithms for sign-in. The type of service determines the connection over which the authentication step is performed: VPNs would use IPSec, for example, while WiFi networks would use Extensible Authentication Protocol (EAP), and web applications would use SSL/TLS.
In addition to the already-existing network layers like TLS and IPSec, the examples in the white paper tend to rely on existing open source infrastructure to validate user accounts on the server side, such as RADIUS and LDAP. The puzzle pieces that do not yet exist are the standardized credentials, standardized OTP protocols, and application connectors required to hook the OATH authentication interface into network services — bits like Apache modules, PHP and Perl libraries, and VPN code.
From theory to practice
Since 2004, OATH has focused its energies on developing the missing pieces in this roadmap [PDF], and has attempted to do so in the open, building on open and royalty-free specifications. The first result of this work was HOTP, the Hash-based Message Authentication Code (HMAC) OTP algorithm, published in 2005. In 2008, the algorithm was superseded by TOTP, the Time-based OTP. Both were published as IETF drafts.
TOTP extended HOTP by replacing the latter's moving event counter with a time-based value. Essentially, HOTP was a cryptographic function of a shared (symmetric) key and an integer event counter, the count on which the connecting client must keep in sync with the remote server in order to successfully authenticate. TOTP removes the need for client and server to stay in sync on the event counter by using a Unix timestamp instead; the algorithm allows the server to choose how far off an incoming timestamp it deems acceptable, in order to correct for clock drift.
In September of 2010, the most recent release was unveiled. Named OATH
Challenge-Response Algorithms (OCRA), the new algorithm extends TOTP
still further. First, it allows for the replacement of the counter or
timestamp value with any arbitrary input parameter. The IETF draft
describes the input parameter as "a structure that contains the
concatenation of the various input data values" that the parties
agree upon, and enumerates several acceptable values: event counters as
used in HOTP, time signatures as used in TOTP, hashed PIN or password
values, session identifiers, and general challenge/response questions (and
their answers). The input parameter also incorporates a header indicating which data values are employed, as well as the cryptographic hash function to be used.
The second change is the addition of more verification modes. HOTP and TOTP have exactly one: a client attempts to connect, and the server authenticates the client by sending it a challenge and checking that the response is valid. But OCRA also allows the client to authenticate the identity of the server, so that both parties can be sure they know who they are talking to. This "mutual challenge-response" variation of the algorithm doesn't add anything new, it just allows the client to issue a challenge of its own. Thus the mutual authentication boils down to doing two separate challenge-response computations: one client-to-server, one server-to-client; in other words, the challenge issued by the client is not connected to the challenge issued by the server.
Finally, OCRA also features a "plain signature mode," in which the server sends a "challenge" to the client which requires only that the client sign the challenge's data payload and return it. This mode does not depend on a shared secret key; any client can sign and return the response, so it is only useful for tracking purposes. But it can be combined with a regular challenge-response authentication, creating a "signature with server authentication mode."
Thus far, the HOTP/TOTP/OCRA work has been OATH's most visible development, but it is not the only product released. In 2009, OATH published the OATH Token Identifier Specification, which specifies a formatted alphanumeric identifier that can be used as a unique global identifier by all OATH-compliant products. The format breaks down hardware tokens, software tokens, and "embedded tokens" into separate classes. Thus far, the posted specification only covers hardware tokens, and consists of a 2-character manufacturer prefix, 2-character token type, and 8-character "manufacturer unique identifier."
In 2010, OATH contributed to the IETF's Portable Symmetric Key Container (PSKC) draft specification, which defines an XML-based format for transferring symmetric encryption keys, and Dynamic Symmetric Key Provisioning Protocol (DSKPP), which described a client-server method of initializing and installing symmetric keys. OATH also launched a certification program for vendors wishing to have their products certified for HOTP, TOTP, and OCRA compliance.
Criticism
Shortly after HOTP's launch, the SHA-1 hash function was discovered
to be considerably less collision-resistant than previously thought. The
bar was lowered yet again in
2009. HOTP's HMAC codes used SHA-1, which led to some concern of HOTP's
security itself. However, HOTP does not rely on SHA-1's
collision-resistance, but rather on its strength as a "trapdoor function"
— finding a collision would entail finding two HOTP responses that
have the same hash value, but doing so would not enable an attacker to hijack or replay a valid session; to do that the attacker would need to invert the hashed response and recover the event counter. Still, TOTP and OCRA allow for other hash functions in addition to SHA-1.
A more fundamental challenge to OATH's relevance is the notion that "industry wide" standards for OTP are pointless. This was the charge leveled in 2005 by RSA's Burt Kaliski, who argued that while PKI systems depend on "one-to-many" security, OTP is always "one-to-one;" i.e., although many vendors may need to verify a digital signature, the challenge-response algorithm between a server and client does not get stronger or more reliable because other vendors implement the same algorithm.
Kalinski further criticized HOTP's inflexibility, both in hash algorithm and use of event counter, which may have contributed to the improvements in those areas in TOTP and OCRA. Still, his initial complaint stands — the OATH OTP algorithms "standardize" something that does not benefit much (if at all) from standardization. One could make the same argument about the Token Identifier Specification: the specification does not make a strong case that a standardized ID string format makes life substantially easier than a randomly-generated string; the strength of the hardware token authentication system comes from the secure installation of the secret key, not the ID number printed on the back. Kalinski's criticism is at least true from the consumer's point-of-view; the user is not made more secure through standardization. Cryptographic token manufacturers, on the other hand, might stand to benefit from an industry-wide standard
HOTP, TOTP, and OCRA are all very simple, but (according to mining project data at Freshmeat and Ohloh) there are less than a dozen open source products that implement any of them — which puts them about on par with S/KEY and other, older OTP standards. That is hardly widespread adoption for a consortium with more than fifty paying members. OATH does not even provide a reference implementations; it only publishes specifications. Then again, the OTP component is just one piece of the overall puzzle OATH has set out to define and standardize — still to come are protocol handlers, validation handlers, credential storage and auditing, and more. Perhaps the full architecture will look more enticing to open source developers.
Comments (2 posted)
Brief items
When one hears allegations that the US FBI has paid people to put backdoors
into the OpenBSD IPSEC stack, one might normally be inclined to take them
with a large grain of salt. When such allegations are passed on by Theo de
Raadt, though, they merit a look. "
I have received a mail regarding the early development of the OpenBSD
IPSEC stack. It is alleged that some ex-developers (and the company
they worked for) accepted US government money to put backdoors into
our network stack, in particular the IPSEC stack. Around
2000-2001." It will be interesting to see if the forthcoming audit
turns up anything, or whether it is simply a strange FUD campaign.
Full Story (comments: 43)
Tim Brown digs into Linux/POSIX linkers in a
paper [PDF] called "Breaking the links: Exploiting the linker". He was
drawn to the subject because of the fuss made over Microsoft's recently discovered insecure library loading vulnerability along with claims that "
Linux could never be affected by such a problem". The paper looks into the linking process and how it can be subverted in various ways.
Full Story (comments: 1)
Mimi Zohar has put out a request for comments on an
Overview of the Linux Integrity Subsystem [PDF]. It covers the Integrity Measurement Architecture (IMA) and Extended Verification Module (EVM) that are part of the Linux kernel, as well as Trusted/Encrypted keys that are making their way into the mainline. Also: "
For anyone interested in the proposed integrity subsystem,
linux-ima.sourceforge.net has been updated with new, hopefully,
simplified installation directions, patches to use the new
Trusted/Encrypted keys, which is now in the security-testing/#next tree,
a few bug fixes, and a sample dracut patch to enable EVM in the
initramfs. (The patches are against the 2.6.36 stable tree.)"
Full Story (comments: none)
New vulnerabilities
bind: remote denial of service
| Package(s): | bind9 |
CVE #(s): | CVE-2010-3762
|
| Created: | December 13, 2010 |
Updated: | May 31, 2011 |
| Description: |
From the CVE entry:
ISC BIND before 9.7.2-P2, when DNSSEC validation is enabled, does not properly handle certain bad signatures if multiple trust anchors exist for a single zone, which allows remote attackers to cause a denial of service (daemon crash) via a DNS query. |
| Alerts: |
|
Comments (none posted)
bind: information disclosure
| Package(s): | bind |
CVE #(s): | CVE-2010-3615
|
| Created: | December 9, 2010 |
Updated: | December 17, 2010 |
| Description: |
From the ISC advisory:
When named is running as an authoritative server for a zone and receives a query for that zone data, it first checks for allow-query acls in the zone statement, then in that view, then in global options. If none of these exist, it defaults to allowing any query (allow-query {"any"};).
With this bug, if the allow-query is not set in the zone statement, it failed to check in view or global options and fell back to the default of allowing any query. This means that queries that the zone owner did not wish to allow were incorrectly allowed.
This bug doesn't affect allow-recursion or allow-query-cache acls, since they are not relevant to a zone for which the server is authoritative. |
| Alerts: |
|
Comments (none posted)
collectd: denial of service
| Package(s): | collectd |
CVE #(s): | CVE-2010-4336
|
| Created: | December 14, 2010 |
Updated: | May 17, 2011 |
| Description: |
From the Debian advisory:
It was discovered that collectd, a statistics collection and monitoring
daemon, is prone to a denial of service attach via a crafted network
packet.
|
| Alerts: |
|
Comments (none posted)
exim4: code execution
| Package(s): | exim4 |
CVE #(s): | CVE-2010-4344
|
| Created: | December 10, 2010 |
Updated: | January 27, 2011 |
| Description: |
From the Red Hat advisory:
A buffer overflow flaw was discovered in Exim's internal
string_vformat() function. A remote attacker could use this flaw to
execute arbitrary code on the mail server running Exim. (CVE-2010-4344) |
| Alerts: |
|
Comments (none posted)
exim: privilege escalation
| Package(s): | exim |
CVE #(s): | CVE-2010-4345
|
| Created: | December 13, 2010 |
Updated: | April 15, 2011 |
| Description: |
From the SUSE advisory:
The unprivileged user exim is running as could tell the exim daemon
to read a different config file and leverage that to escalate
privileges to root |
| Alerts: |
|
Comments (none posted)
firefox: multiple vulnerabilities
| Package(s): | firefox |
CVE #(s): | CVE-2010-3770
CVE-2010-3774
CVE-2010-3773
CVE-2010-3767
CVE-2010-3766
CVE-2010-3775
CVE-2010-3768
CVE-2010-3772
CVE-2010-3771
CVE-2010-3769
CVE-2010-3776
CVE-2010-3777
|
| Created: | December 10, 2010 |
Updated: | May 2, 2011 |
| Description: |
From the Mandriva advisory:
Security researchers Yosuke Hasegawa and Masatoshi Kimura reported that
the x-mac-arabic, x-mac-farsi and x-mac-hebrew character encodings are
vulnerable to XSS attacks due to some characters being converted to
angle brackets when displayed by the rendering engine. Sites using
these character encodings would thus be potentially vulnerable to
script injection attacks if their script filtering code fails to
strip out these specific characters (CVE-2010-3770).
Google security researcher Michal Zalewski reported that when a
window was opened to a site resulting in a network or certificate
error page, the opening site could access the document inside the
opened window and inject arbitrary content. An attacker could use
this bug to spoof the location bar and trick a user into thinking
they were on a different site than they actually were (CVE-2010-3774).
Mozilla security researcher moz_bug_r_a4 reported that the fix for
CVE-2010-0179 could be circumvented permitting the execution of
arbitrary JavaScript with chrome privileges (CVE-2010-3773).
Security researcher regenrecht reported via TippingPoint's Zero
Day Initiative that JavaScript arrays were vulnerable to an integer
overflow vulnerability. The report demonstrated that an array could
be constructed containing a very large number of items such that when
memory was allocated to store the array items, the integer value used
to calculate the buffer size would overflow resulting in too small a
buffer being allocated. Subsequent use of the array object could then
result in data being written past the end of the buffer and causing
memory corruption (CVE-2010-3767).
Security researcher regenrecht reported via TippingPoint's Zero Day
Initiative that a nsDOMAttribute node can be modified without informing
the iterator object responsible for various DOM traversals. This
flaw could lead to a inconsistent state where the iterator points
to an object it believes is part of the DOM but actually points to
some other object. If such an object had been deleted and its memory
reclaimed by the system, then the iterator could be used to call into
attacker-controlled memory (CVE-2010-3766).
Security researcher Gregory Fleischer reported that when a Java
LiveConnect script was loaded via a data: URL which redirects via a
meta refresh, then the resulting plugin object was created with the
wrong security principal and thus received elevated privileges such
as the abilities to read local files, launch processes, and create
network connections (CVE-2010-3775).
Mozilla added the OTS font sanitizing library to prevent downloadable
fonts from exposing vulnerabilities in the underlying OS font
code. This library mitigates against several issues independently
reported by Red Hat Security Response Team member Marc Schoenefeld
and Mozilla security researcher Christoph Diehl (CVE-2010-3768).
Security researcher wushi of team509 reported that when a XUL tree
had an HTML <div> element nested inside a <treechildren> element then
code attempting to display content in the XUL tree would incorrectly
treat the <div> element as a parent node to tree content underneath
it resulting in incorrect indexes being calculated for the child
content. These incorrect indexes were used in subsequent array
operations which resulted in writing data past the end of an allocated
buffer. An attacker could use this issue to crash a victim's browser
and run arbitrary code on their machine (CVE-2010-3772).
Security researcher echo reported that a web page could open a window
with an about:blank location and then inject an <isindex> element
into that page which upon submission would redirect to a chrome:
document. The effect of this defect was that the original page would
wind up with a reference to a chrome-privileged object, the opened
window, which could be leveraged for privilege escalation attacks
(CVE-2010-3771).
Dirk Heinrich reported that on Windows platforms when document.write()
was called with a very long string a buffer overflow was caused in line
breaking routines attempting to process the string for display. Such
cases triggered an invalid read past the end of an array causing a
crash which an attacker could potentially use to run arbitrary code
on a victim's computer (CVE-2010-3769).
Mozilla developers identified and fixed several memory safety
bugs in the browser engine used in Firefox and other Mozilla-based
products. Some of these bugs showed evidence of memory corruption
under certain circumstances, and we presume that with enough effort
at least some of these could be exploited to run arbitrary code
(CVE-2010-3776, CVE-2010-3777). |
| Alerts: |
|
Comments (1 posted)
fontforge: code execution
| Package(s): | fontforge |
CVE #(s): | CVE-2010-4259
|
| Created: | December 14, 2010 |
Updated: | January 23, 2012 |
| Description: |
From the CVE entry:
Stack-based buffer overflow in FontForge 20100501 allows remote attackers to cause a denial of service (application crash) or possibly execute arbitrary code via a long CHARSET_REGISTRY header in a BDF font file. |
| Alerts: |
|
Comments (none posted)
HelixPlayer - many potential vulnerabilities
Comments (none posted)
kernel: denial of service
| Package(s): | kernel |
CVE #(s): | CVE-2010-3086
CVE-2010-4162
|
| Created: | December 14, 2010 |
Updated: | July 14, 2011 |
| Description: |
From the SUSE advisory:
CVE-2010-3086: A missing lock prefix in the x86 futex code could be
used by local attackers to cause a denial of service.
CVE-2010-4162: A local denial of service in the blockdevice layer
was fixed.
|
| Alerts: |
|
Comments (none posted)
kernel: multiple vulnerabilities
| Package(s): | kernel-rt |
CVE #(s): | CVE-2010-3861
CVE-2010-4082
CVE-2010-4157
CVE-2010-4158
CVE-2010-4169
|
| Created: | December 9, 2010 |
Updated: | July 14, 2011 |
| Description: |
From the Red Hat advisory:
A flaw in ethtool_get_rxnfc() in the Linux kernel's ethtool IOCTL
handler. When it is called with a large info.rule_cnt, it could allow a
local user to cause an information leak. (CVE-2010-3861, Moderate)
Missing initialization flaws in the Linux kernel could lead to
information leaks. (CVE-2010-4082, CVE-2010-4158, Low)
Missing sanity checks in gdth_ioctl_alloc() in the gdth driver in the
Linux kernel, could allow a local user with access to "/dev/gdth" on a
64-bit system to cause a denial of service or escalate their privileges.
(CVE-2010-4157, Moderate)
A use-after-free flaw in the mprotect() system call could allow a local,
unprivileged user to cause a local denial of service. (CVE-2010-4169,
Moderate)
|
| Alerts: |
|
Comments (none posted)
moonlight: code execution
| Package(s): | moonlight |
CVE #(s): | CVE-2010-4254
|
| Created: | December 15, 2010 |
Updated: | March 31, 2011 |
| Description: |
Untrusted moonlight applications can break out of the sandbox, allowing an attacker to execute arbitrary code. |
| Alerts: |
|
Comments (none posted)
openttd: denial of service
| Package(s): | openttd |
CVE #(s): | CVE-2010-4168
|
| Created: | December 14, 2010 |
Updated: | November 14, 2011 |
| Description: |
From the CVE entry:
Multiple use-after-free vulnerabilities in OpenTTD 1.0.x before 1.0.5 allow (1) remote attackers to cause a denial of service (invalid write and daemon crash) by abruptly disconnecting during transmission of the map from the server, related to network/network_server.cpp; (2) remote attackers to cause a denial of service (invalid read and daemon crash) by abruptly disconnecting, related to network/network_server.cpp; and (3) remote servers to cause a denial of service (invalid read and application crash) by forcing a disconnection during the join process, related to network/network.cpp. |
| Alerts: |
|
Comments (none posted)
otrs: multiple vulnerabilities
| Package(s): | otrs |
CVE #(s): | CVE-2010-2080
CVE-2010-3476
CVE-2010-4071
|
| Created: | December 15, 2010 |
Updated: | December 24, 2010 |
| Description: |
The otrs business-process management system suffers from two cross-site scripting bugs and one denial of service vulnerability. |
| Alerts: |
|
Comments (none posted)
perl-CGI-Simple: HTTP response splitting
| Package(s): | perl-CGI-Simple |
CVE #(s): | CVE-2010-2761
|
| Created: | December 9, 2010 |
Updated: | December 9, 2011 |
| Description: |
From the Mandriva advisory:
The multipart_init function in (1) CGI.pm before 3.50 and (2) Simple.pm
in CGI::Simple 1.112 and earlier uses a hardcoded value of the MIME
boundary string in multipart/x-mixed-replace content, which allows
remote attackers to inject arbitrary HTTP headers and conduct HTTP
response splitting attacks via crafted input that contains this value,
a different vulnerability than CVE-2010-3172 (CVE-2010-2761).
|
| Alerts: |
|
Comments (none posted)
php: denial of service
| Package(s): | php |
CVE #(s): | CVE-2010-4409
|
| Created: | December 15, 2010 |
Updated: | March 23, 2011 |
| Description: |
An integer overflow in NumberFormatter::getSymbol in php 5.3.3 and prior can allow a remote attacker to crash the application. |
| Alerts: |
|
Comments (none posted)
xfig: code execution
| Package(s): | xfig |
CVE #(s): | CVE-2010-4262
|
| Created: | December 15, 2010 |
Updated: | January 17, 2011 |
| Description: |
The xfig drawing editor suffers from a buffer overflow which can be exploited via a malicious fig file. |
| Alerts: |
|
Comments (none posted)
Page editor: Jake Edge
Kernel development
Brief items
The current development kernel remains 2.6.37-rc5; there have been
no 2.6.37 prepatches in the last week. Linus has returned from his travels
and has resumed pulling patches, so a new release can be expected in the
near future.
Stable updates: the 2.6.27.57, 2.6.32.27, and 2.6.36.2 updates were released on
December 9.
Greg Kroah-Hartman notes that this
is the last 2.6.27 kernel he will be releasing as he is turning that kernel
over to Willy Tarreau for maintenance as a long-term kernel (in keeping with
the stable tree changes he
announced on December 3). "It was a good run, [encompassing] 57 releases, over 791 days, and included
1596 patches (not the largest of any stable release series, .32 has that
beat already).
[...]
Speaking of .32, I'd _strongly_ encourage all existing .27 users to
seriously look into migrating to the .32 kernel tree at this point in
time as well."
The 2.6.35.10 update was released - with
over 200 patches - on December 15 by new maintainer Andi Kleen. Says
Andi: "This release contains security
fixes and all 2.6.35 users are encouraged to update."
Comments (none posted)
Right now I'm traveling without access to my laptop to actually do
pulls. That was unintentional, but my laptop got co-opted for more
important stuff (child needed it for school an my other laptop has
a broken harddisk), so I've just been reading email with my
cellphone.
--
Linus Torvalds needs an Android git port
Abstracting something tends to make it permanent. When you have an
ugly, special-case temporary hack, there is merit to having it
sitting there in the middle of the code staring you in the face.
It's very explicit and we won't forget about it.
--
Andrew Morton
Comments (3 posted)
Jim Gettys continues his discussion of bufferbloat with
a
posting on how to improve the situation on home systems. "
Once
tuned, Linux's latency (and the router's latency) can be really nice even
under high load (even if I've not tried hard to get to the theoretical
minimums). But un-tuned, I can get many second latency out of both Linux
home routers and my laptop, just by heading to some part of my house where
my wireless signal strength is low (I have several chimneys that makes this
trivial). By walking around or obstructing your wireless router, you
should be easily able to reproduce bufferbloat in either your router or in
your laptop, depending on which direction you saturate."
Comments (23 posted)
By Jonathan Corbet
December 15, 2010
Contemporary CPUs have an interesting feature: they can detect when a
virtualized guest is spinning on a lock and trap back into the host
kernel. The idea is that the host can probably find better things to do
with a processor than dedicate it to spinning in a single guest. KVM will
currently respond to such a trap by sleeping briefly, allowing some other
process to run outside of the virtualized system in question. But, as Rik
van Riel
pointed out, that is not
necessarily the right thing to do.
If one thread in a virtualized system is spinning on a lock, another thread
within that system must be holding the lock. Rather than pause the entire
guest, it is better to run the lock-holding thread so that the lock can be
released. Pausing the guest just delays the release of the lock, causing
the virtual machine as a whole to be penalized; that, says Rik,
"results in eg. a 64 VCPU Windows guest taking forever and a bit to
boot up." Tempting as it may be to just blame Windows, it's
probably better to fix the problem.
Rik's fix is to change how the trap handler behaves; rather than yield the
CPU entirely, it will take a spinning thread's remaining CPU time slice and
give it to a process on another CPU. The hope is that the recipient of
this gift (which is essentially a priority boost) will be the one holding
the lock, but there is no real guarantee of that currently. This
functionality is implemented with a new yield_to() function which,
Rik says, could maybe be turned into a system call if it turns out to be
useful in that context.
The patch has been through a couple of rounds of review and may find its
way into 2.6.38.
Comments (none posted)
By Jake Edge
December 15, 2010
The kernel has two macros to assist the compiler and CPU in doing branch
prediction: likely() and unlikely(). As their names
imply, they are meant to annotate tests in the code based on the likelihood
that they will evaluate to true. But, getting it wrong such that something
marked likely is actually unlikely—or vice versa—can impact
performance as the CPU may prefetch the wrong instructions.
Steven Rostedt has been looking
at the problem using the annotated branch profiler and found ten places "that really do not need to have
an annotated branch, or the annotation is simply the opposite of
what it should be". So, he created a series of patches that either
switched the sense of the annotation or removed the
likely()/unlikely() entirely.
As an example, page_mapping() had an unlikely()
annotation that Rostedt reported as being "correct a total of 1909540379 times and
incorrect 1270533123 times, with a 39% being incorrect". Those
numbers come from his main workstation which runs a variety of standard
programs (Firefox, XChat, etc.) as well as participating in his build farm,
so it should represent a reasonably "normal" workload. Being wrong 39% of
the time was pretty obviously too much and led
to the removal of the annotation for that test.
The changes are various subsystems including the scheduler, memory
management, and VFS. So far, there have been no complaints, though there
have been several requests to completely remove annotations that had just
been changed in order to allow
the compiler's and CPU's branch prediction logic make the decision. That,
and breaking
the patches up into separate sets for each subsystem, caused Rostedt to
respin them. It would seem likely() that we'll see them make
their way into 2.6.38.
Comments (14 posted)
Kernel development news
By Jonathan Corbet
December 14, 2010
The Linux directory entry ("dentry") cache is a key part of the kernel's
filename lookup mechanism. The dentry cache speeds the process of looking
up files
considerably. On systems with large numbers of cores, though, the dentry
cache has become a scalability problem for workloads which perform a lot of
lookups. Nick Piggin's
dcache scalability
work looks like it may be set to be merged for 2.6.38, but, according
to Nick, this work "
has had nowhere near enough review." The
code is complicated and tricky, which, certainly makes review harder. Your
editor, never afraid to make a total fool of himself, will now attempt to
describe how this patch set works just in case it helps.
A dentry's core job is to represent a directory in the filesystem and to cache
the mapping between a name found within that directory and its associated
inode. To that end, dentries are organized into a hierarchy which mirrors
the on-disk hierarchy found in the filesystem. Each dentry looks vaguely
like this:
Every dentry keeps a list of children (names contained within the directory)
which can be looked up by name; each dentry also points to the inode it
represents, its parent dentry, and a number of other things. Note that the
real situation is a bit more complicated than shown here; children are kept
in hashed lists for faster lookup, an inode with more than one link may
have more than one dentry pointing to it, and so on. But, in a conceptual
sense, the above diagram shows what is going on.
When the VFS layer needs to turn a path provided by a program into a
pointer to the associated inode, it will perform this lookup through the
dentry cache. The first step is to get to the starting point - which could
be the root of the filesystem (for absolute paths), the current working
directory (for relative paths), or a program-specified directory. In the
first two cases, the associated dentries can be found by way of the
process's task structure. The first component of the path is looked up in
the starting dentry, yielding a pointer to the next dentry in the path;
that process is repeated until the end of the path is reached.
It may be that some of the dentries in the path are not present in the
cache; there is not enough room in memory to cache the entire filesystem
hierarchy. When that happens, the lookup code must ask the filesystem (by
way of the parent's inode operations) to perform the lookup; a dentry
structure can then be created with the result and inserted into the cache.
Of course, a lookup will fail if the file (or a component in the path) does
not exist; in that case, the VFS may insert a "negative dentry" into the
cache to speed up future failed lookups. That optimization is important -
just run a simple command under strace to see how common failed
lookups really are.
The dentry cache is a dynamic data structure; dentries will come and go
frequently in response to filesystem changes and the simple need to clean
out old dentries on occasion. Clearly, some sort of protocol is needed to
prevent changes from colliding with each other; if one processor removes a
dentry while another is using it to look up a name, good things will almost
certainly not result. Once upon a time, the global dcache_lock
was used to protect the cache during lookup operations. The global lock
scaled poorly, so it has not been used for lookups for some time - though
it still does protect many other things.
Current kernels use read-copy-update to
manage the removal of dentries from the cache, so a lookup operation need
not worry about a specific dentry going away as long as that operation does
not block. If a lookup has to call into the filesystem code, though,
things might indeed block; to ensure that a dentry will stay around in this
situation, a reference count is kept. So, as a lookup works its way down
the hierarchy, it will increment the reference count of every dentry it
passes through. Most of those references are eventually dropped, though
the reference on the final dentry must be kept as long as the file is held
open.
[PULL QUOTE:
Making path lookup truly scalable in a highly
parallel situation requires making no changes to the dentry structures
themselves.
END QUOTE]
The core of Nick's patch set is a new lookup algorithm called "RCU-walk."
The key to RCU-walk is the idea that, on workloads where
pathname lookup is likely to present scalability problems, the chances are
good that most dentries will already be present in the cache. One might
think that the current algorithm would perform well in such situations, but
there is a catch: the constant incrementing and decrementing of dentry
reference counts creates a great deal of cacheline bouncing - enough to
slow things considerably. Making path lookup truly scalable in a highly
parallel situation requires making no changes to the dentry structures
themselves - turning the cache into a read-only data structure during the
lookup process, essentially.
So, when the new code needs to perform a path lookup, it starts with an
rcu_read_lock() call. The dentry cache should then remain stable
enough for the lookup to get to the end of the path and increment the
reference count for the final dentry (only). So lookups can be done
without locks - and, crucially, without changing any information in the
dentries passed through on the way. That makes the lookup scalability
problems go away.
Of course, there are a few catches. The most obvious of these is the
situation where one of the dentries in the path is not resident in the
cache. At that point, the RCU-walk process must stop; the code will
attempt to obtain a reference on the final dentry it found, then return to
the older, reference-count-based method ("ref-walk") for the rest of the
lookup. Sometimes, getting that mid-point reference will not be possible;
that situation will force the lookup to restart from the beginning using
ref-walk for the entire path.
More challenging, perhaps, is what happens if one of the dentries changes
while the lookup is passing through. By normal RCU standards, that should
never happen; changing a structure requires making a copy and making the
changes there. The dentry cache bends those rules, though; RCU is mostly
used to protect against dentry deletion there. So, in particular, a rename
can cause a dentry to change attributes - and hashed lookup lists - while
another process is using it for a lookup.
Naturally, using a lock to protect against this possibility would wipe out
any scalability gains that would otherwise come from all of this work. So
the RCU-walk code uses a seqlock instead.
Seqlocks do not prevent concurrent changes, but they do allow code to
detect when such a change has occurred. Nick has added a seqlock to every
dentry which must be incremented whenever the associated name, parent, or inode
changes. If the sequence number changes while a lookup is using a dentry, the
lookup will be restarted (with the seqlock write-locked, to prevent heavy
renaming from starving other operations indefinitely).
For more information on the rename problem and
how it's handled, see path-lookup.txt,
which is included in the patch set.
The use of RCU has one other implication which forces a significant
change. Directory permissions must be checked as part of every step in a
lookup operation to ensure that users don't access parts of the filesystem
which should not be available to them. Permission checking is done by the
filesystem, by way of the permission() inode operation. If this
checking is to happen safely during the RCU-walk process, the inode
structure must not go away while the lookup is in progress. So, in other
words, inodes must now be freed with RCU rather than being released
directly. The change is relatively straightforward, but it requires
tweaking every filesystem implementation in the tree, so the patch is
large.
A number of other filesystem callbacks (d_compare() and
d_hash(), for example) have also had to be changed to support
RCU-walk. Anybody maintaining an out-of-tree filesystem will have some
work to do if and when this patch set is merged.
While RCU-walk is perhaps the most significant change in this patch series,
it's far from the only one. Many of the other patches are aimed at the
elimination of the global dcache_lock altogether. There is a new
dcache_hash_lock to protect hashing operations,
dcache_lru_lock for modifications to the dentry LRU list, and
dcache_inode_lock to protect inode dentry lists. The scope of the
dentry d_lock spinlock has been expanded to cover changes to much
of the structure; the reference count (formerly an atomic_t) is
also covered by the lock now. All told, it's a large set of patches making
fundamental changes to some of the core VFS code.
Interestingly, Nick also changed the dentry d_validate() callback,
which, he says, "has been broken for a long time." That
function depended on a truly scary routine called
kmem_ptr_validate(), described this way:
This verifies that the untrusted pointer looks sane; it is _not_ a
guarantee that the pointer is actually part of the slab cache in
question, but it at least validates that the pointer can be
dereferenced and looks half-way sane.
It is hard to imagine that such a function could be put to any sort of safe
use. The only user in current kernels is d_validate(); Nick's
patch fixes that usage and gets rid of the function. It seems unlikely to
be missed.
Given the complexity of this patch set, it is not surprising that reviews
have been relatively scarce. Review time for VFS-related patches has
always been hard to come by, and these are trickier than most. The ongoing
name-calling match between Nick and Dave Chinner, who has also been working
in this area, has not helped; neither has the fact that Al Viro appears to
have gone into one of his quiet periods. Given that Linus seems fairly
well determined to merge this work, it would be good if at least some of
the inevitable glitches could be found before the 2.6.38 merge window.
Hopefully enough developers will find the time to review and/or test these
patches to make some progress in that direction.
Comments (8 posted)
By Jonathan Corbet
December 13, 2010
Virtualization places some interesting demands on the host system, many of
which are related to memory management. When two agents within the system
both believe that they are in charge of memory, interesting conflicts are
bound to arise. A recent patch from Balbir Singh shows the kind of effort
which is being made to address these conflicts, but it also gives a hint at
how a more ambitious effort might really solve the problem.
The Linux page cache keeps copies of pages from on-disk files in main
memory in the hopes of avoiding I/O operations when those pages are
accessed. At any given time, the page cache can easily account for more
than half of the system's total memory usage. The actual size of the page
cache varies over time; as other types of memory use (kernel memory and
anonymous pages) grow, the page cache shrinks to make room. Balancing the
demands of the page cache with other memory users can be a challenge, but
Linux seems to get it close to right most of the time.
Balbir's patch is intended to give the
system administrator a bit more control over page cache usage; to that end,
it provides a new boot-time parameter (unmapped_page_control)
which sets an upper bound on the number of unmapped pages in the cache.
"Unmapped" pages are those which are not mapped into any process's address
space - they do not appear in any page tables. Unmapped pages arguably
have a lower chance of being needed in the near future; they are also a bit
easier for the system to get rid of. This patch thus gives the system
administrator a relatively easy way to minimize page cache memory usage.
The obvious question is: why? Page cache pages will be reclaimed anyway if
the system has other needs for the memory, so there would seem to be little
point in shrinking it prematurely. The problem, it seems, is
virtualization. When a process on a virtualized system reads a page from a
file, the guest operating system will dutifully store a copy of that page
in its page cache. The actual read operation, though, will be passed
through to (and executed by) the host, which will also store a copy in its
page cache. So the page gets cached twice - perhaps even more times if it
is used by multiple virtual machines. Caching a page can be a good thing,
but caching multiple copies is likely to be too much of a good thing.
So what Balbir's patch is doing can be explained this way: it is forcibly
cleaning copies of pages out of guest page caches to minimize duplicate
copies. The memory freed in this way can be captured by a balloon driver
and returned to the host, making it available for more productive use
elsewhere in the system.
This technique should clearly improve the situation. Less duplication is
good, and, if the guest ends up needing some of the freed pages, those
pages stand a good chance of being found in the host's page cache. But one
can't help but wonder if it might not be an overly indirect approach.
Rather than forcibly reclaim pages from the guests' caches, might it be
better to have all of the systems share the same page cache? A single,
unified page cache could be managed to maximize the performance of the
system as a whole; that should yield better results than managing a number
of seemingly independent page caches.
Virtualization based on containers has exactly this type of unified page
cache since all of the containers are running on the same kernel. That may
be one of the reasons why containers are seen to perform better than fully
virtualized systems. Bringing a shared page cache to the virtualized world
would be a bit of a challenge, though, which probably has a lot to do with
why it has not already been done.
To begin with, there would be some clear security issues. A virtualized
system should be unable to access any resources which have not been
explicitly provided to it. Any sort of shared page cache would have to be
designed in a way which would leave the host in control of which pages are
visible to each guest. In practice, that would probably mean using the
virtualized block drivers which make filesystems available to virtualized
guests now. Rather than "read" a page into a page controlled by the guest,
the driver could somehow just map the host's copy of the page into the
guest's address space.
Making that work correctly would require the addition of a significant new,
Linux-only API between the host and the guest. It would be hard to do it
in a way which maintained any sort of illusion that the guest is running on
hardware of its own. Such a scheme would complicate memory management in
the guest - hardware is increasingly dynamic, but individual pages of
memory still do not come and go spontaneously. A shared page cache would
also frustrate attempts to use huge pages for guest memory.
In other words, the difficulties of sharing the page cache between hosts
and guests look to be decidedly nontrivial. It is not surprising that we
are still living in a world where scarce memory pages can be soaked up with
duplicate copies of data. As long as that situation holds, there will be a
place for patches which cause guests to behave in ways which are more
friendly to the system as a whole.
Comments (12 posted)
By Jonathan Corbet
December 15, 2010
It has been well over five years since LWN
reviewed the second edition of Robert Love's
Linux Kernel Development. Needless to say, things have changed a
little since the 2.6.10 release covered by that edition. As it happens,
the third edition has been out for a few months now; your editor has
finally had a chance to read through his copy and put together a review.
In summary, the third edition is a much-needed update, and
Linux Kernel
Development remains a valuable resource, but there are some
disappointments to be found as well.
One has to dig a little bit to figure out which kernel version is covered
by the third edition; according to the preface, the target is 2.6.34.
Robert, ever the optimist, suggests that it will be good for a long time:
"As the Linux kernel matures, there is a greater chance of a snapshot
of the kernel remaining representative long into the future." Time
will tell.
The third edition has been extensively updated, but it retains the same
structure as its predecessors. The preface talks of "all-new" chapters,
but the number of chapters remains the same. The scheduler discussion has
been updated to reflect the merging of the completely fair scheduler. Other
relatively recent kernel changes (mutexes, for example) have been added,
and there are changes throughout to reflect what has happened over the last
24 kernel releases.
There is a new chapter on kernel data structures; it contains
the linked list discussion previously found in Appendix A, along with
coverage of FIFOs, red-black trees, and the idr subsystem. The low-level
device model chapter has been combined with the chapter on loadable
modules, for some reason, but the discussion is not much changed. The
appendix on the random number generator is gone.
All told, the coverage of the core kernel is well written and clear, in an
approachable style. Linux Kernel Development is, at this time,
probably the best reference available for developers wanting to learn how
the kernel works and how the major pieces fit together. Your editor is
glad to have a copy on hand.
With that understanding in place, this, too, must be said: the update to
the third edition appears to have been done in a hurry. As a result, the
book contains a number of errors and inconsistencies, and it fails to cover
no end of interesting things which have happened in the kernel over the
last five years while retaining text which was obsolete even in previous
editions. Robert has not been hugely present in the kernel development
community in the last few years (he got a job at a company with a
reputation for removing developers from the community) and, unfortunately,
it shows.
For example, this book (which covers 2.6.34) includes a section on the
anticipatory I/O scheduler - which was removed in 2.6.33. There is still
talk of the "Linus" scheduler - which (as is noted in the book) was a 2.4
feature. The mutual exclusion chapter discusses semaphores - which have
been deprecated for mutual exclusion purposes - with the section on mutexes
seemingly bolted on afterward.
(The book also says elsewhere that we cannot kill a process in an
uninterruptible sleep because it "may hold a semaphore")
There is a lengthy discussion of the old
"bottom half" mechanism which is long gone; the removed-in-2.5 task queue
mechanism also merits a section of its own. The unlamented
ksymoops tool, we are told, "probably came with your
distribution."
Some things are simply wrong. We're told that do_exit() calls
del_timer_sync(), but that last happened in 2.6.15. The workqueue
discussion appears to be stuck in a 2.6.10 time warp; changes which were
merged for 2.6.20 are not reflected here. Kernel stack size is said to be
16K on 64-bit architectures because they usually have 8K pages. The
version of struct file shown on page 279 never existed; it
includes f_ep_lock which was renamed (by your editor) to
f_lock in 2.6.30. The "process address space" chapter says,
several times, that all Linux systems have three-level page tables, despite
the fact that the fourth level was added for 2.6.11. The device model
chapter no longer mentions struct subsystem, but it still appears
in the associated diagram.
And many things which should be discussed in a contemporary book aren't.
Developers working on the kernel now should probably be familiar with
control groups, contemporary debugging tools (kmemleak, lockdep, the fault
injection framework, ...), debugfs, ftrace, git, high-resolution timers, huge
pages, linux-next, multiple slab allocators, namespaces, perf, power
management and quality of service, read-copy-update, readahead, reverse
mapping in the VM subsystem, scheduler domains, splice(),
TASK_KILLABLE, threaded interrupt handlers, virtualization, and so on.
No book on the kernel can hope to be complete and still be something that a
person of ordinary strength can physically lift, but these topics are all
important enough that, one would assume, at least some of them would merit
coverage, but none are mentioned in the third edition.
Keeping up with all that is happening in the kernel is hard - your editor
hopes that readers will trust him to understand this. So it is not
surprising that some mistakes would slip through when a book is
fast-forwarded from 2.6.10 to 2.6.34. But the amount of old stuff that
leaked through, combined with the things which should have been mentioned
but weren't, seems a bit high; some of them should, at least, have been
caught in technical review. As a result, the third edition of Linux
Kernel Development is not as good as it could have been.
These grumbles notwithstanding, your editor will restate what was said
above: this book remains
the best overview of contemporary kernel development available today.
Robert is a talented and engaging writer who is able to cover complex
topics in a readily understandable manner. The third edition merits a
place on the "L1 bookshelf" (the one which can be reached without getting
out of the chair); developers who are working with the kernel will
probably want a copy.
Comments (24 posted)
Patches and updates
Kernel trees
Core kernel code
Development tools
Device drivers
Filesystems and block I/O
Memory management
Networking
Architecture-specific
Security-related
Virtualization and containers
Miscellaneous
Page editor: Jonathan Corbet
Distributions
By Jake Edge
December 15, 2010
There are any number of organizations that have a need for a
security-oriented OS that can be freely used on computers at coffee shops,
hotels, and the like. The US Department of Defense (DoD) is one such
organization
and it has put together Lightweight
Portable Security (LPS), a live CD (or USB stick) Linux distribution,
for use by its employees to access the web—or their desktop
remotely. While the technology behind LPS is not particularly noteworthy,
though it does have some interesting features, it is noteworthy that DoD
chose Linux to deliver this kind of solution. Perhaps that shouldn't be
surprising either, though, as the proprietary OS vendors don't really offer
any way to customize their systems to anywhere near the extent that Linux does.
LPS was developed as part of the DoD's Software Protection Initiative (SPI),
which is run by the Air Force Research Laboratory (AFRL). SPI's mission is to
"marginalize a nation-state class threat's ability to steal and
exploit critical DoD intellectual property found in application software
(executables, source, and associated data)." While LPS will
certainly help with that mission, it doesn't seem anywhere near hardened
enough to fend off nation-state class threats.
The distribution is available as ISO files in either of two "public"
editions: standard or deluxe. The deluxe edition simply adds OpenOffice.org
and roughly doubles the size of the release. The existence of a public
version would seem to imply that there are less-public versions of
LPS—one of those may be the LPS Remote Access
Edition, which doesn't come with download links and instead has a way
to request custom versions.
Version 1.1.1 of LPS was released on November 15 and can be burned onto a
CD directly. In addition, bootable USB sticks can be created, but only (easily)
under Windows.
When booting into LPS, one is greeted by a screen with badges for the three
organizations responsible and a progress bar. After that, a window pops up
that gives three choices: read the user agreement, agree to it and
continue, or reject it and reboot. The agreement itself notes that the
software is governed by the GPL and disclaims any
warranty. While it is not unheard of for Linux distributions to have a
click-through license, it is a bit strange.
Once the agreement has been accepted, LPS loads an IceWM desktop,
which prominently features those three badges again, along with icons for a
number of applications (e.g. Firefox, OpenOffice.org, Documentation,
Xterm). The layout is fairly Windows-like, presumably so that it doesn't
scare off the target users. There are also menu entries for things like
SSH, Citrix, and Microsoft remote desktop clients.
Once you start poking around in LPS, though, some questionable things jump
out. Starting the Xterm gives a root BusyBox shell for example, and a
simple ps shows that everything runs as root. That includes
Firefox, IceWM, the wicd network manager, and so on. One of the features
of LPS is that it doesn't mount the local disks of the system, but that is
trivial to work around with mount.
If LPS is started from CD, making persistent changes to it is not possible,
but part of the idea is to isolate the data on the local disks from
internet-based
attacks. For public computers in hotels or elsewhere, there may not be
anything of interest on the local disks, but if users are booting LPS on
their home systems or laptops, that assumption may not have much merit.
Given that everything runs as root, and the local disks are accessible,
whatever OS is installed locally could be subverted.
For USB-based LPS systems, the situation is even worse. Though the USB
stick isn't mounted by default after LPS boots, it certainly can be. The
LPS user's guide [PDF]
notes that removing and re-inserting the USB stick will mount it, though
malware could also mount it directly. That would allow LPS itself
to be persistently modified.
There are some warnings that might alleviate some of these problems. It is
recommended that a separate USB stick be used for data, for example. In
addition, there are suggestions that LPS be rebooted before making any
"sensitive" transactions—and after after visiting dodgy web
sites. It seems a little unlikely that users will actually follow those
instructions, either because they forget or due to the annoyance of a
fairly lengthy boot time.
It is a fairly old kernel that LPS uses (2.6.27), but it has been updated
to one of the more recent—but not the most recent as of November
15—stable versions (2.6.27.53)
based on the
uname string. Whether there have been any patches applied on top
of that kernel is difficult to determine as there is no source code
provided—at least in any obvious location.
A query about the source location was answered by Rich Kutter of the AFRL
who said that LPS is based on Thinstation 2.2.2 with only minimal
modifications. A change to the OpenSC smart card
libraries/utilities to better support the DoD
Common Access Card (CAC) is the only substantive change. He said
that the code for that change will be placed in the ISO image for the
next release due later in December. But that doesn't satisfy the GPL
requirements, as the full source needs to be made available, which is
something they are planning to do, he said.
It would seem that SELinux has not been enabled for LPS, which may not be a
huge surprise for a, supposedly, read-only system. It is, however, another
US government security solution for Linux, and could have been used
to sandbox Firefox and its Flash plugin for example (though just running
them as non-root would be a good start). Overall, one gets the
feeling that the folks behind LPS may be working in something of a vacuum,
and not fully considering all of the threats that LPS might face. Perhaps
part of
the reason there is a public version is to get that kind of feedback.
There are some specific additions to LPS for DoD users, including support
for CAC and Personal
Identity Verification smart cards. Evidently, there are some web sites
that are only available to folks that have those cards and an available USB
smart card reader, so Firefox has been configured to do that kind of
verification.
There is also an Encryption
Wizard that allows for Advanced Encryption Standard (AES) encryption
and decryption of files. The Java-based wizard has also been turned into a
Firefox plugin so that web-based email (e.g. Yahoo, Gmail, Outlook Web
Access) can be encrypted.
Overall, LPS is perfectly usable—if painfully slow for unknown
reasons on a not underpowered laptop—for web surfing and document
creation. It has a very limited set of applications, presumably by design,
and no way to add any new ones. If you need GIMP or Thunderbird, it would
seem that you are simply out of luck. Once the source code for building
the distribution is available, one could presumably build their own
derivative with additional applications, but that is difficult to do at the
moment.
While it seems dubious that LPS would thwart a targeted attack from a
nation-state-sized attacker, that is probably also true of most or all Linux
distributions. But there is clearly more that could be done to harden LPS
against less targeted, or less deep-pocketed, attackers. LPS may give the
impression of being more secure than it actually is because of where it
comes from, and that is a bit worrisome. Given that there are entities
actively trying to access classified information—either for espionage
or posting on Wikileaks—LPS only provides a partial solution to
those problems.
Comments (12 posted)
Brief items
How to name MeeGo packages for Debian:
1: wontgo came to mind immediately, ...
--
Don Armstrong
The obvious choice would be 'yugo', to honor fine eastern European
solutions for mobility.
--
Teemu Ikonen
Comments (none posted)
DebXO 0.6 has been released. "
DebXO is a version of Debian (testing)
that is customized for the XO-1 hardware. The 0.6 release adds initial
support for the XO-1.5 hardware; however, XO-1.5 is not officially
supported. I'll update the wiki with instructions for XO-1.5, for the
early adopters."
Full Story (comments: none)
New Owl-current ISOs, OpenVZ container templates, and freshly rebuilt
package sets have been
released.
"
This might be the last Owl-current snapshot before we make our 3.0
release, so please test extensively and report both successes and failures
(in some detail). ;-)"
Comments (none posted)
Distribution News
Debian GNU/Linux
Neil McGovern has a report from the Debian release team. Topics include
squeeze in deep freeze, remaining bugs, schedule, themes and Wheezy.
Full Story (comments: none)
A long time goal of the Debian project has been to remove non-free firmware
from the Linux kernel shipped with Debian. "
We are proud to announce
that, to the best of our knowledge, all issues are solved and that we will
be able to deliver a Linux kernel which is completely Free, according to
the Debian Free Software Guidelines (DFSG), with Debian Squeeze."
Full Story (comments: none)
Fedora
Click below for the minutes from the December 13 meeting of the Fedora
Board. Topics include schedule, longer term goals and vision statement.
Full Story (comments: none)
SUSE Linux and openSUSE
The openSUSE Board has
announced
that Alan Clark has been appointed by Novell to be the chairperson for the
board. "
As you may be aware, we have focused a significant amount of our time on the creation of an openSUSE Foundation to be independent and to be able to collect and spread funds. And Novell has been very supportive with our desire to see this come to fruition. Markus told us that one of the reasons he selected Alan is that he has a lot of expertise in setting up foundations. Alan helped to form the Open Source Development Labs (OSDL) and the Linux Foundation, as well as several other open source projects and organisations. We, along with Markus, believe that Alan's experience and expertise will be an asset of immense value to the Board, and we welcome that as we, and Novell, partner together in forging ahead on an exciting and promising future for the openSUSE Project."
Comments (none posted)
Ubuntu family
The daily PlayStation 3 CD builds are no longer available for Ubuntu Natty
Narwhal (11.04). "
Changes made by Sony to the "Other OS" facilities
of the PS3 have made it increasingly difficult for people with new systems
to run Ubuntu on them, and there seem to be hardly any developers with the
interest and ability to keep the CD images working. Building separate CD
images for PS3 takes a substantial amount of resources (disk space, time
required to run the powerpc and powerpc+ps3 live filesystem builds
sequentially, developer time to fix problems, and so on), and right now it
doesn't seem to be worth it."
Full Story (comments: none)
Newsletters and articles of interest
Comments (none posted)
Joe "Zonker" Brockmeier
reviews Linux Mint 10 for Linux Magazine. "
So Linux Mint may continue using the traditional GNOME desktop with [Linux Mint] 11. I'm curious how that decision will sit with Mint users today, whether they've been hoping for a Mintized GNOME Shell or are excited by Unity. My experiences with Unity so far — as a desktop interface — have been less than satisfying. I'm very happy with Mint's GNOME implementation, and their decision to stick with a traditional GNOME interface would make Mint a great refuge for Ubuntu users who do't care for the newfangled Unity interface. Yes, Ubuntu will probably ship with a vanilla(ish) GNOME and GNOME Shell, but it's not going to have the same level of polish it has now."
Comments (5 posted)
The Ubuntu technical board has been reviewing the most popular topics in
Ubuntu Brainstorm—which allows people to vote on bugs and new features in the distribution—as Ubuntu CTO Matt Zimmerman
reports on his blog. The ten most popular topics have received "
expert answers", that Zimmerman describes. "
To tell us about it, Amit Kucheria, Ubuntu kernel developer and leader of the Linaro working group on Power Management, contributed a great writeup on this topic, with technical analysis, tips and recommendations, and a look at whats coming next. [Kucheria says:]
I am going to attempt to summarize the various use profiles and what Ubuntu does (or can do) to prolong battery life in those profiles. Power management, when done right, should not require the user to make several (difficult) choices. It should just work — providing a good balance of performance and battery life."
Comments (7 posted)
Susan Linton
takes
a quick look at the recent release of ZevenOS-Neptune. "
ZevenOS isn't just another Debian or Ubuntu clone. Its developers attempt to provide a different, easy, and high performing alternative to more well known distributions. ZevenOS is the main offering and Neptune is technically a community driven branch. Another branch with GNOME 2.3.0 and Epiphany is also available. The ZevenOS project is an interesting participant in the Linux landscape and Neptune is definitely one to test. If you commonly travel off the beaten path, put this one on your list."
Comments (none posted)
Page editor: Rebecca Sobol
Development
On December 6th the KDE Project announced
the Calligra Project, which was called a "continuation of the KOffice project." Others, though, might call it a fork or a split between the bulk of the KOffice developers and KWord developer Thomas Zander.
What's been widely publicized already is the plan for the Calligra side of the split. The new suite will contain the existing KOffice applications, mostly rebranded from the originals. KWord now becomes Words, KSpread becomes Tables, KPresenter becomes Stage, and so on. Calligra will roll up eight applications into one suite, with more promised as the project gains more contributors.
The development will also move to Git from Subversion, and will continue
to be hosted on KDE Project infrastructure. Though the new name and
detailed plan was announced on December 6th, the split has actually been a
long time in coming. Cyrille Berger Skott announced
the intent to split the KOffice community on October 23rd. As the post
indicates, this was after "months of discussion." Some of the
discussion has happened on the public mailing lists, and other discussions have taken place off the devel list through KDE's Community Working Group (CWG).
Causes for the split
One thing lacking in the official KDE post about the split is a rationale for the move. There's a terse section in the announcement about "Calligra and the past," but it provides very little insight into the split:
Nearly everyone in the KOffice community has joined together to make this move. Leaving the past behind us, we are excited at this opportunity to make our software more innovative and widely-used. At the same time that the Calligra project is created, we will move from Subversion to Git, making it an even better platform for innovation in the free office space.
Aaron Seigo's post on the matter was a bit more substantive, explaining that KOffice experienced an internal fork:
The fork itself came about through unresolved differences between a member of the KOffice team and the rest of the members over how to manage both long term targets and day-to-day development. This eventually resulted in people coming to the conclusion that those differences were not only unresolved but also unresolvable. To call a one person schism a fork may seem a bit overly dramatic, but that's certainly how it felt to those involved and was not a triviality. Coming to a fork, the rest of the KOffice team took the opportunity of change to rethink various aspects, including the name.
What's not been said here is who isn't joining together for the
move. That would be Zander, a former Nokia employee (Zander says he
resigned a while ago), who is the KWord maintainer and co-maintainer for the KOffice libraries.
Reading through the koffice-devel list, a lot of fingers are pointed at
Zander, while he points to Nokia's involvement in KOffice. A discussion in
July over a post
made by Zander to the KOffice.org front page demonstrates a fair amount of
tension between Zander and other KOffice developers. The source of
contention, in this case, stems from a posting
by Zander about some ideas for the direction of KWord development. This
typically would not raise eyebrows in a FOSS project, but Zander originally
chose to use the front page of the KOffice.org site, apparently without much agreement on the ideas or the placement of the post.
The placement of the posting, however, was not the only bone of contention. Asked via email about the split, Zander says that he was running into differences of opinion with Nokia-sponsored developers who have been trying to add features that mirror Microsoft Word and PowerPoint for Maemo/MeeGo use:
As you probably know, Nokia has been working with some KOffice members for about 18 months to create an MSOffice documents reader to run on Maemo/Meego. For this goal Nokia demands a lot of features to be added that Word/Powerpoint have. There are some 10 people working full time to add those features.
As the maintainer of KWord and a co-maintainer of the koffice-libraries this work was becoming a problem for the goals of releasing an end-user-ready version. More regressions than I could fix in my spare time and requests to those full-time authors to write new unit tests and to take care not to break unit tests etc. were ignored.
I saw 4 years of work that went into KWord2.x going in a direction that was not going to let me release a version an actual user could edit with for a long long time.
In the end, Zander says that other KOffice developers said "MeeGo had a deadline and they had to deliver or else," with a request to help them out. Zander says he declined, and that prompted the request to split KOffice which he agreed to. "I am not opposed to that; the goals of the two teams [are] just too different."
Zander claims that Nokia is responsible for the split, saying "lets call this whole thing what it is, a fork off of KOffice focusing on getting it ready to render MSOffice documents properly and getting it working for Meego using the ways of working that Nokia uses for such projects."
Berger Skott disagreed that the blame lies with Nokia, and said in email that KOffice work is relatively balanced between paid and unpaid developers:
Between Beta 4 and RC1, there have 229 commits, by 25 authors, 14 are paid developers, to the exception of one who is paid by the NLNet, the other are paid directly or indirectly by Nokia, six of them started as koffice volunteers developers, one comes from KDE, they made a total of 105 commits. The other 11 developers are volunteers, they did 120 commits (the four remaining commits were made by the internationalization script). As you can see this is rather balanced.
In a post on his blog, Berger Skott agrees that the goals and vision for the two projects are different. In IRC, he said that "we believe that the person who does the work is the one who [gets] to decide, all developers are treated equally and contribution is accepted, whether from community or paid developers, while Thomas is favorable to a stricter control on the code." He added that "community issues between him [Zander] and other members started a long time ago." Indeed, Zander had been asked to leave the KOffice project once prior, and came back in February 2008 with a promise to avoid non-technical discussions. Strictly speaking, that didn't happen.
It's unclear how many developers will continue to work on the non-Calligra suite, but Zander did ask for volunteers to continue KOffice and says "I have received various people's blessings and know I am not alone in this endeavor, and that does make me feel pretty good about this."
Zander says that Krita and Kexi will be removed from the current KOffice tree, as they are "too big to be without a maintainer," but that development will continue on the rest:
The other apps share the majority of their logic in the libraries and they will continue to be developed on; now with an even stronger focus on getting end-user-ready. So if users were annoyed that it took [so] long to get KWord in a decent shape; the biggest hurdle has not been taken away and we should move towards a better version much faster.
Moving forward with Calligra
Regardless of who and how, the Calligra team is energized now and might
be in a position to put the suite on the map. Seigo noted in his post that
the Calligra team is charged up and now has the challenge of tapping the momentum and "building a healthy, dynamic community with real leadership around it and a coherent vision under it."
The community building is taking place on the calligra-devel list. There's currently discussion about whether to elect maintainers and how to handle problems with individuals or conflicts if or when they arise in the future. Nothing has been decided yet, however.
On the technical side, much has been made of the Nokia/MeeGo influence on Calligra already and possible use for mobile devices. Berger Skott writes that Calligra is not mobile oriented:
Lets be clear, it is not true. Calligra is focused on developing technologies related to office and creativity applications, on top of those technologies, the Calligra project is delivering a set of desktop applications and mobile applications (and maybe tablet, in the future). All of it is [built] over the KDE technologies, using the Qt toolkit, which makes it potentially available to an incredible range of devices and operating system: Linux, Windows (desktop and mobile), Mac OSX, Symbian, Meego, Haiku (and maybe Android, iPhone, WebOS...), using a desktop computer, a laptop, a mobile phone, a tablet, your TV... And all with an user interface that is most suitable for your form factor.
But he adds that Qt and KDE availability is not enough to make Calligra work on those platforms, and that the majority of the volunteers are targeting the Linux desktop, with Nokia supporting MeeGo and mobile phones.
Despite the fuss over the launch of Calligra, there is still one KOffice release to come. The KOffice 2.3 release is being prepared now, with the first release candidate out as of December 9th. The final release does not have a set date, but will be released when all release blockers are fixed. The first Calligra release will be 2.4, and the schedule and feature plan for that haven't been set yet.
Now that the Calligra team is finding its footing, one hopes that it (and whatever rises from the other half of the KOffice fork) will be successful in providing a competitive productivity suite. The harsh truth is that KOffice as a whole has not been terribly successful as of yet. The FOSS community could use another office suite that can hold its own against Microsoft Office, maybe this restructuring will provide the foundation for Calligra to meet that need.
Comments (21 posted)
Brief items
Many programmers have created and promoted the computer programming
language known as "open source code" to be shared on public sites
at no cost, but licensing issues are murky.
--
Reuters
systemd documentation is actually pretty good and mostly
comprehensive. Humble as I am I would even say that it is vastly
superior to the majority of all open source projects. If you want
to criticise us on something, pick something else, please.
--
Lennart Poettering
Comments (5 posted)
Version 0.8.22 of the Denemo music notation editor is out. Changes include
microtonal music playback, support for historic tunings, better voice
handling, and a human-readable file format.
Full Story (comments: none)
The 1.0 release of the GNU Modula-2 compiler is out. At this point, it
represents a full implementation of the ISO standard, with a full set of
libraries as well; see
the GNU
Modula-2 page for more information.
Full Story (comments: none)
MySQL community server 5.5.8 - the first "production quality" release in
the 5.5 series - is now available. This release includes improved
scalability, a number of InnoDB improvements, a semisynchronous replication
interface, SIGNAL/RESIGNAL support, and more; see
the
what's new page for more information.
Full Story (comments: none)
PacketFence is a system
for controlling access to the network; the 2.0 release has been announced.
Improvements include new hardware support, simplified authentication
configuration, proxy interception, unauthenticated passthrough to specific
sites, and more. Much more information, including screenshots and video
tours, can be found on the project web site.
Full Story (comments: 2)
Newsletters and articles
Comments (none posted)
On his blog, Dave Neary has some
suggestions on helping developers get more comfortable with interacting in public forums like those typically used by free software projects. He points out a number of barriers that some developers face when considering participating in a mailing list along with some ideas on how to surmount them. "
In community projects, peer review is expected — in fact, it is a best practice, one of the things that separates successful community projects from the crowd. Developers expect to hear about features before they are developed, and have an opportunity to suggest better ways the feature can be implemented. They expect new contributors to submit patches that they can review — it is the way a new contributor builds trust before getting the keys to the house. It is such a recommended practice that the only treatment I can suggest is that you should help your developers to get over their nervousness & embrace peer review, by making it the norm in your team." (Thanks to Paul Wise.)
Comments (5 posted)
Page editor: Jonathan Corbet
Announcements
Brief items
The Apache Software Foundation (ASF) has resigned its seat on the Java Community Process (JCP) executive committee (EC) as
reported on the ASF blog. This comes after the EC voted to approve Java SE 7 (on a
vote of 12-3). The ASF had
threatened resignation over the issue back in November. "
The Apache Software Foundation concludes that that JCP is not an open specification process - that Java specifications are proprietary technology that must be licensed directly from the spec lead under whatever terms the spec lead chooses; that the commercial concerns of a single entity, Oracle, will continue to seriously interfere with and bias the transparent governance of the ecosystem; that it is impossible to distribute independent implementations of JSRs [Java Specification Requests] under open source licenses such that users are protected from IP litigation by expert group members or the spec lead; and finally, the EC is unwilling or unable to assert the basic power of their role in the JCP governance process."
Comments (93 posted)
Richard Purdie, founder of the
Poky build system, has been
appointed as a fellow of the Linux Foundation, where he will work full time on the
Yocto project, OpenEmbedded, and Poky.
"
Purdie was most recently a Core Developer at OpenEmbedded, where he was also lead maintainer of bitbake. He has also been an embedded Linux architect in Intels Open Source Technology Center. From 2005 to 2008, he was a Software Engineer at OpenedHand, where he worked with a variety of other open source projects such as Clutter, X server, Zaurus and Oprofile. He has also made numerous contributions to the Linux kernel, including as maintainer of the backlight and LED subsystems. Purdie received his MSci in Physics from University of Durham in 2003.
[...]
Current Linux Foundation Fellows include John Hawley, Till Kamppeter, Janina Sajka and Linus Torvalds. Previous Fellows include Steve Hemminger, Andrew Morton, Andrew Tridgell and Ted Ts'o."
Comments (none posted)
The CE Linux Forum is
looking to fund
proposals for embedded Linux projects. "
Each year, CELF spends
money on contract work to improve Linux for use in embedded systems. Some
of the projects we have sponsored in the past include Linux-tiny, DirectFB
enhancements, smem, U-boot and kexecboot improvements, and Squashfs and
YAFFS mainlining." If you have an idea, now is the time to submit
it and, with luck, be paid to implement it.
Comments (none posted)
Articles of interest
Over at opensource.com, Daniel Doubrovkine
recounts his experiences in trying to open source some of his company's code. "
Armed with a healthy dose of idealism, I went to executive management and proposed we open source the tool. I was hoping for a no-brainer and a quick decision at the division level. To my surprise, it took two years, a vast amount of bureaucracy, and far more effort than I ever anticipated. In this process I learned many valuable lessons that I wanted to share with engineers attempting to open source their first projects."
Comments (none posted)
At Linux.com, Philip Koltun
reports on the feedback to the release of the Linux Foundation's license compliance self-assessment checklist. "
Other downloaders lamented the fact that following the checklist doesn't guarantee freedom from compliance issues. True enough. There are limits to the utility of any checklist. But following a solid compliance process improves your chances of recognizing the FOSS that's present, determining its provenance, and complying with license obligations. Others noted that the checklist doesn't help directly with a key compliance question some companies face: whether a particular software architecture can incorporate GPL'ed software without exposing company-proprietary software to copyleft effect. True again. No community authority exists that can render such decisions; there are just a lot of educated opinions."
Comments (none posted)
The Guardian
reports on Richard Stallman's warning about cloud computing and ChromeOS in particular. "
I suppose many people will continue moving towards careless computing, because there's a sucker born every minute. The US government may try to encourage people to place their data where the US government can seize it without showing them a search warrant, rather than in their own property. However, as long as enough of us continue keeping our data under our own control, we can still do so. And we had better do so, or the option may disappear."
Comments (31 posted)
Glyn Moody
muses
on the importance of Linaro. "
This lends Linaro's focus a particular value: it is about spreading open source in one of the fast-growing sectors. Moreover, it's one where free software's low/zero cost, robustness, small size and customisability are crucial advantages over traditional proprietary solutions. Indeed, I think it's pretty clear that the embedded world will be one that Linux is likely to dominate, at least in the medium term, until something completely new and better comes along (assuming that ever happens)."
Comments (4 posted)
Ars technica
reports
that Wolfire Games has released the second Humble Bundle of DRM-free
games. "
The first Humble Bundle was a monster success, with over 100,000 people donating over $1 million in total to support the Electronic Frontier Foundation, Child's Play, and of course the developers behind the games themselves. The second bundle is now live, containing five great games: Braid, Cortex Command, Machinarium, Osmos, and Revenge of the Titans. You pay what you want, decide where your money goes, each game is DRM-free, and the games work on Windows, Mac OS X, and Linux."
Comments (48 posted)
New Books
O'Reilly Media has released "Canvas Pocket Reference - Scripted Graphics
for HTML5", by David Flanagan.
Full Story (comments: none)
Pragmatic Bookshelf has released "The RSpec Book: Behaviour-Driven
Development with RSpec, Cucumber, and Friends", by David Chelimsky.
Full Story (comments: none)
Resources
There are announcements from both
the
Apache Software Foundation and
Google
on the launch of
apache-extras.org,
which appears to be a hosting site for Apache-related projects with
(possibly) proprietary a wider range of licenses. "
Among the ASF's strengths are its well-established requirements relating to intellectual property management, license use, and community management. Apache-extras.org provides a home for projects that are unable to, or do not wish to, conform to those rules yet still want to signal their relationship to official Apache projects.
As projects on the new Google-hosted service will not be managed by The Apache Software Foundation, participants are allowed to use whatever license and project management process they desire."
Comments (10 posted)
Seemingly a bit tardy, the
2009 GNOME foundation annual report [PDF] has been
released. The 50+ page report contains articles about various conferences, hackfests, and other GNOME activities in 2009, along with a look at the finances of the foundation. Future foundation reports will be aligned with the foundation's fiscal year (October to September).
Comments (1 posted)
The Linux Foundation newsletter for December covers the annual 'Who Writes
Linux' study, individual membership drive, Linux distributions certified
with LSB 4.0, LSB 4.1 beta available, and several other topics.
Full Story (comments: none)
Education and Certification
The Linux Professional Institute (LPI) has reports that their training
partner program has grown nearly 10% over the past year, with 303
participating organizations currently. "
In addition, LPI added
partners in 11 additional countries in 2010: LPI now has training partners
in 55 countries around the world."
Full Story (comments: none)
Upcoming Events
The
PyCon 2011 is open for
registration. The conference takes place March 11-13, 2011 in Atlanta,
Georgia. The deadline for financial aid applications is January 2, 2011.
Full Story (comments: none)
There will be a PyPy sprint in Leysin, Switzerland, January 16-22, 2011.
"
This is a fully public sprint: newcomers and topics other than those
proposed below are welcome."
Full Story (comments: none)
Events: December 23, 2010 to February 21, 2011
The following event listing is taken from the
LWN.net Calendar.
| Date(s) | Event | Location |
January 16 January 22 |
PyPy Leysin Winter Sprint |
Leysin, Switzerland |
| January 22 |
OrgCamp 2011 |
Paris, France |
January 24 January 29 |
linux.conf.au 2011 |
Brisbane, Australia |
| January 27 |
Ubuntu Developer Day |
Bangalore, India |
January 27 January 28 |
Southwest Drupal Summit 2011 |
Houston, Texas, USA |
January 29 January 31 |
FUDCon Tempe 2011 |
Tempe, Arizona, USA |
February 2 February 3 |
Cloud Expo Europe |
London, UK |
| February 5 |
Open Source Conference Kagawa 2011 |
Takamatsu, Japan |
February 5 February 6 |
FOSDEM 2011 |
Brussels, Belgium |
February 7 February 11 |
Global Ignite Week 2011 |
several, worldwide |
February 11 February 12 |
Red Hat Developer Conference 2011 |
Brno, Czech Republic |
| February 15 |
2012 Embedded Linux Conference |
Redwood Shores, CA, USA |
If your event does not appear here, please
tell us about it.
Page editor: Rebecca Sobol