By Jake Edge
June 20, 2012
The final two sessions at LinuxCon Japan were targeted at helping Asian
developers become more involved in kernel development. James Bottomley
talked about how the social aspects of kernel engineering were as important
as its technical aspects. He then moderated a panel of kernel developers
that focused on the problems faced by international
developers, adding advice to assist new or aspiring kernel hackers
in clearing those hurdles.
Some history
Bottomley began with a bit of history, noting that the kernel started off
as a tiny "few thousand line" program. In the early days, features were
needed so there were scant reviews and very little feature justification
required. It was "almost the wild west of patches" and he is somewhat
embarrassed to think back on some of the patches he got into the kernel.
The only requirement was that a developer could write the code to implement
a needed feature.
As examples, he cited the error handling in both the SCSI and IDE layers,
which were done at interrupt level and could sometimes lock up the
machine. The block layer had a single lock for all devices, while the TTY
layer had a static array of TTYs.
The era that Bottomley and other kernel developers grew up in was an era of
feature development, but "that era is over", which is a good thing. That
"anything goes" style produced a full-featured kernel very quickly, but
left behind lots of problems. In particular, robustness and scalability
suffered under that scheme.
Ten years later, around 2002 or so, things started changing. Kernel
development was moving from "hot coders" trying to make a functional Unix
system to a community that cared more about robustness. Various subsystems
were rewritten, including SCSI, the block layer, and USB (several times).
There was also a "five year plan that took ten years" to move the kernel to more
fine-grained locking (and eliminate the big kernel lock).
The attitude today is all about code quality, Bottomley said, and that is
something that new kernel developers run into immediately. The kernel
development process is more about ensuring that new code doesn't affect
older code and that there aren't regressions in kernel functionality.
Patches get much more review before being merged, and can often get held up
waiting for reviewers.
Today's kernel is "incredibly feature rich", but that also means that is
complex and thus more fragile. Any new feature that adds complexity gets a
lot of scrutiny to see if it really is needed.
There is an elaborate process as well as tools to help enforce the
process. It is "exactly like ISO 9001, but worse", Bottomley said to a
round of laughter. He pointed out that the Linux development process is
compliant with the much-maligned quality management "standardization"
effort from the early
2000s. The "Signed-off-by" line in patches, as well as the coding style
that is enforced kernel-wide, are elements of a well-established process
that seeks to ensure the quality of kernel submissions.
Longtime kernel
developers have grown up with these processes, so they make sense to those
people,
but may be less clear to new contributors. For the most part, they came
about because of clear problems that were seen and then addressed. For
example, the
"Signed-off-by" requirement came about because of the SCO case, but it is
useful in its own right; allowing people to track back on the origin of
various parts of the kernel.
Getting patches into the kernel
The easiest way to get your patches into the kernel is by making a bug
fix, he said. The processes are much easier for a bug fix than they are
for a new
feature. But, don't label a new feature as a bug fix; it's been done before
and it won't be successful. For a bug fix, the patch just needs to
describe the bug and how the changes fix that bug.
Trying to get a new feature in is much harder; there is a need to
"socialize" it first. That means getting a group of users and others
interested in the feature, so that they can help advocate its inclusion.
Conferences are a great way to socialize a new feature by talking to other
kernel developers about it. Meeting face to face to discuss the feature,
perhaps over "wine or whisky", can be beneficial as kernel developers are
more inclined to listen in those settings, Bottomley said.
When posting patches for review, there is a natural human desire to show
your best work, but that is actually somewhat counter-productive. The
patch does not need to be perfect, and the discussion about the patch may
help build a community of people interested in the feature. In addition,
an imperfect patch makes people think they can still give input.
One thing to remember is that a patch posting will get input of various
sorts that will require the poster to be the advocate for their patch.
That means that one needs to be prepared to argue on the mailing list,
which can be difficult, especially for Asian developers. The key is to
only respond to the technical complaints and to ignore any personal attacks
that might get posted.
It is also important to identify which of the comments to focus on. There
are some people that "lurk" on the mailing lists, trying to make life
harder for contributors. Ignoring those comments, while responding to and
working with the
more influential developers is the right strategy. Developers who get lots
of patches accepted (of a technical, rather than janitorial, nature) are
likely to be influential, as are those whose feedback is listened to and
quoted by others in the discussion. Typically, those people will come up
with sensible and constructive suggestions, but they won't always be right,
Bottomley said.
For that reason, argument is an essential part of getting any patch
accepted. For those who are uncomfortable doing so he suggested practicing
defending the patch with friends and colleagues, ideally in English.
Starting out by posting to the proper subject mailing list, rather than
linux-kernel, will also help as the signal to noise ratio tends to be much
better on those lists. In addition, becoming better known on the mailing
lists will help with subsequent postings as people will recognize your
name, which will make them more favorably disposed toward new postings. It
can also help to meet some of the more influential developers at
conferences, he said.
If you are getting arguments from influential developers, it may be because
they don't understand what the feature is—or why it is needed. It is
important to be able to explain the feature to those folks, but don't waste
time arguing with the wrong people. One should definitely accept feedback
from the right people and make the appropriate changes; there are often
multiple iterations to a patch set before it gets accepted.
Bottomley rounded out his talk with some quick hit suggestions that will
help smooth the path for a patch. A good changelog message is essential,
one that doesn't say what the code does ("almost all maintainers can read
C") but why it is needed. Splitting up patches into manageable pieces is
important to simplify the review process. One way to do that is to
describe what the patch does to colleagues and split up the patch in a way
that corresponds to that description. It is also important to follow the
established rules, as they have evolved over a long time and there is a
reason for each of them.
Kernel panel
After that, several other kernel developers joined him on the stage: Arnd
Bergmann of IBM/Linaro, Chris Mason of Oracle (at that time), Herbert Xu of Red
Hat, and Masami Hiramatsu of Hitachi,. After they introduced themselves,
Bottomley
asked each to describe what obstacles they had faced and overcome in
becoming a kernel developer. Hiramatsu mentioned "English, of course" as
one of the hurdles, but also noted that the time difference between Japan
and the rest of kernel developers can be a barrier. Sometimes he has
stayed up late to be able to be a part of the discussion that would
otherwise start without him.
Xu talked about some of the problems he faced as a Debian kernel
maintainer. There were numerous patches carried by distributions that he
and others tried to push upstream, but they were rejected. The
distribution developers had a very different view of what was needed in the
kernel than the subsystem maintainers and other mainline developers had.
That problem has been largely overcome now, he said, by knowing what the
kernel as a whole needs and adapting the distribution patches appropriately.
Mason pointed out that the time zone problem can
actually be an advantage. One way to tamp the flames in a mailing list
discussion is
to reply slowly, which gives all of the participants some time to
reconsider their words and thoughts. He
"did everything wrong" in his first kernel project by choosing an enormous
feature that he had no idea
how to implement, then worked on it by himself for months without asking
any questions. His suggestion was that new contributors not follow his
lead and instead start with smaller bug fixes and get to know the community
before moving on to bigger projects.
Bergmann said that his first patch was to support some new hardware that he
sent to a maintainer who disappeared shortly thereafter. His next effort
displayed his lack of C knowledge and had three bugs in a single line
patch. Those problems were pointed out by a good maintainer, which led him
to go back and study C and programming some more before his next
submission. Bottomley pointed out that Bergmann's experience is an example
that normally the criticism is aimed at the code, and not at the person.
Problems specific to developers in Asia (beyond the time zone and language
issues) was next up. Hiramatsu noted that his teachers always wanted him
to make things perfect in his code, which is kind of the opposite of the
approach that works best for the Linux kernel. While we are ultimately aiming for the
perfect kernel, we get there by having an open discussion on
patches that are imperfect, he said.
Don't think, just do it
Bottomley asked specifically what he recommended for Japanese
developers so that they were willing to submit imperfect patches. "Don't
think, just do it", Hiramatsu—who may have a future in sports gear
marketing—said. The key is to not worry about it too much before
sending the patch, he said.
An area that Asian developers could improve, Xu said, is in participating
in the kernel as more than just their work duty. In the west, many
developers see their efforts in the kernel community as independent of
their work, so that even if they started working in an unrelated field,
"they would
still be part of the community". Engaging the community takes time, and
people are very busy with work and family, so they don't have time to make
a long-term commitment to one area of the kernel. But it is very useful to
do so, he said, and something that developers in Asia could do better.
Both IBM and Linaro have done private reviews for new developers, Bergmann
said. The idea is to help them describe and defend the patch by discussing
it on an internal mailing list. Essentially it helps train developers in
how to argue on mailing lists, so that it will be easier when it needs to
be done on linux-kernel.
Mason said that you can only create the perfect patch indirectly. The most
important thing to do is to "prove that you understand the problem". That
is done with a description of the feature or bug that shows that
understanding. What
code actually ends up in the mainline is dependent on the maintainers, so
the patch should be seen as a "sample solution" that may or may not end up
in the mainline. Bottomley pointed to the "Andrew Morton bug fix
solution", which is to post a bad patch as a challenge to the rest of the
kernel developers, as a similar idea.
Attracting and keeping new kernel developers was another topic covered.
Mason said that it is important to encourage people that offer to help, by
suggesting that they study the problem they are having. If they pick
a feature or bug fix that they need or really care about, they will be more
successful. Bergmann suggested that people be encouraged to become
maintainers of smaller pieces of the kernel, rather than take on a bigger
piece. He has done that and said it has "worked surprisingly well". It
makes for "better maintainers than I am", he said.
As LinuxCon Japan wound to a close, Bottomley asked for "inspirational
statements" from the panel. Mason said "Linux is yours, treat it like it's
yours", while Xu extended that thought: "Treat it like a hobby", he said.
Hiramatsu had the final word by reprising his earlier "Don't think, just do
it" message. Suitably inspired, the assembled masses then made their way
to the post-conference party.
Comments (none posted)
By Nathan Willis
June 20, 2012
Two professors from the University of California, Berkeley School of
Law have launched the Defensive Patent License
(DPL), a legal tool that is designed to do for patents what the GPL
did for software licenses. It creates a copyleft-style method for
patent holders to automatically share their patents with others who
agree to share theirs in return. The goal is to
"de-weaponize" patents and thus reduce the gridlock that
slows down technology sector innovation, but the DPL is likely to have
an uphill battle.
The DPL's creators are Jason Schultz and Jennifer Urban, both of whom
have a background in online legal activism — Schultz with the
Electronic Frontier Foundation's (EFF) Patent Busting project,
and Urban with ChillingEffects. We first
covered the effort in 2010, and the duo
have been developing the specifics of the license since (a thorough
examination can be found in the May 2011 video lecture
they link to from the project site). But the DPL itself is now in a
"public beta" phase, with feedback solicited from the Internet at
large. The current text is available as a PDF,
a Google Docs document,
and as Markdown-formatted text
on Github. There is also a paper
available describing the rationale for the DPL's specific terms.
The best offense is a good defense
The idea at the heart of the DPL is defensive patents —
those that a company files or purchases solely to deter its competitors
from bringing lawsuits against it. Defensive patents are not offered
for licensing under revenue-generating commercial terms, nor are they
used to initiate litigation against others. The result is that large
companies amass giant patent portfolios and enjoy the same relative
stability of the Cold War's mutually-assured destruction. An
unfortunate side effect of this popular strategy is patent proliferation.
That tends to make open source projects and small companies live in fear
of
being shut down by astronomically expensive infringement
lawsuits because they cannot stockpile their own defensive patents.
The DPL is a tool the companies could use to disarm the defensive
patent standoff. Under its terms, a participating company offers a
non-exclusive, royalty-free, perpetual, world-wide license for all of
the patents in its patent portfolio to every other patent holder that
also participates in the DPL. The DPL's license can be revoked for a
particular licensee only under two circumstances: if the
licensee sues another DPL licensor for patent infringement offensively, or if the licensee
withdraws its patent portfolio from the DPL. However, the revocation
is not automatic; each DPL licensor has the option to revoke a
licensee. The result is that the DPL creates a mutually
cross-licensing network, whose members have full access to each
others' patents. Consequently, they should have no reason to pursue infringement
litigation against each other, and defensive patents (both current and
future) are devalued.
Outside of the DPL family, however, licensors are permitted to license
any patents in their portfolios to any party, and to litigate to their
heart's content. In theory that allows them to continue making money
from their patents, and to respond to patent threats from outsiders.
Two additional terms are important. First, a licensor may withdraw
its portfolio from the DPL, but it must give advance notice before
doing so (six months in the current wording), and all existing DPL
licenses will remain intact. Second, the DPL stipulates that a
licensor must ensure that its patents continue to be DPLed even if
they are sold or acquired (by making that condition a term of the sale
or transfer).
The latter condition is an attempt to prevent players from "gaming"
the system by gaining access of the DPL patent pool then selling
themselves, and it is believed to ensure the DPL's persistence after a
bankruptcy declaration (although the authors solicit feedback on these
points, since preventing such gaming is vital to making the DPL work).
Under idyllic circumstances, then, all DPL participants have free and
perpetual access to each others' patents, but can still do whatever
they want against players outside of the DPL community. That provides
incentive for new parties to join, and no member has the power to
refuse membership in the community to another licensor. The
requirement that a licensor must place its entire patent portfolio
under the DPL is there to keep unscrupulous companies from donating
junk patents while keeping valuable ones private, which would prevent
the pool from becoming valuable in the first place.
Is it that simple?
In their talk, Schultz and Urban enumerated several concerns about the
DPL raised in their conversations with outsiders. One is that access to the
pool of DPL patents is not sufficient incentive to join. Another is
that the full-portfolio requirement is too off-putting and that a
smaller commitment ought to be required. There are also potential
anti-trust issues in some jurisdictions, the possibility of loopholes
not yet discovered, and the general criticism that the DPL simply adds
another entanglement to the already hard-to-navigate thicket of patent
problems.
They also admit that many of the technology sector's problematic
patents are not defensive, so the DPL will not end all patent
litigation. In particular, patent trolls would be essentially unaffected
by the existence of even a large DPL. Trolls litigate with offensive
patents, and they do so without fear of retaliation because they make
no products or services of their own (i.e., you cannot counter-sue a patent
troll for infringing on your own portfolio, because the troll has no
products; the mutually-assured-destruction strategy does not work
against them, DPL or not).
Since the DPL's public launch, there have been several responses that
offered additional concerns. David Hayes and Eric Schulman argue
that joining the DPL disproportionately favors small players with
fewer patents (who thus get access to more patents than they
contribute back). Stephan Kinsella notes
that small players do not get much for free because you can only join
the DPL community if you have patents, and patents remain expensive to
get and to retain. Kinsella also observes that it may be difficult to
get the DPL pool started given the unpredictability of the US federal
government.
After all, it’s unfair to let companies have too
big of a defense against the patent threat. That would thwart the very
purpose of the patent system, heavens to betsy! Or the FTC could jump
in and claim that this pooling is anticompetitive, even though the
purpose is obviously to permit competition to thrive, to block the
anticompetitive effect of aggressive patent lawsuits. Who knows what
the schizo feds would do.
Given the state of mutually-assured destruction, it is inherently
risky to be the first one to lay down one's weapon, but that concern
may be overstated by DPL critics. After all, for the first
patent-holder to join the pool, nothing changes: the company looks
benevolent, but still has free reign to litigate non-members (i.e.,
everyone) at will. Still, the "all-in" portfolio requirement has
another problem: it is only appealing to companies whose entire
portfolio is comprised of defensive patents, and leaves no room for
other kinds of patents. It does not take too
much speculation to see that there are companies working in both
software and hardware (e.g., Intel or IBM) with hardware-related
patents that even the staunchest software-patent critic might concede
are valid original inventions.
For open source software projects, though, the primary concern is
likely to be Kinsella's point about requiring patents to buy a seat at
the table. Schultz and Urban concede that open source projects
typically do not file for patents — for many reasons, including
cultural opposition and mistrust of the patent system. But the high
cost of acquiring a patent is not something the DPL can change.
There have been other approaches to fixing the patent problem from
open source projects' perspective, including the Open
Invention Network and Twitter's recent Innovator's
Patent Agreement. On June 19, the EFF launched
its own patent reform campaign with a seven-fold list of fixes.
Compared to the other efforts, the DPL is not so much an attempt to
fix the patent system as it is a way for interested patent holders to
remove themselves from the defensive-patent game.
That option certainly won't appeal to everyone — and certainly
won't to patent trolls or others who profit directly from gaming the
system — but then again, the GPL permits developers to escape
from the typical software licensing hijinks, and it has proven
remarkably successful, as has the Creative Commons license suite
for authors and artists. Not every such attempt to craft a
standardized license is a success; Canonical's Project Harmony
attempted to draft a standardized set of contributor agreements, but
so far does not seem to have caught on in widespread fashion. The DPL
project says it is open to public feedback, however, so if there
is a consensus to be reached on anything resembling a "GPL
for patents," this is probably how we will find it.
Comments (5 posted)
By Nathan Willis
June 19, 2012
Fedora is debating dropping the storied Liberation
font family from its distribution in favor of a fork. Liberation was one of the highest-profile open fonts, but it
has languished since its initial release. Licensing issues were part of the problem, but so was the subtler disconnect of Liberation's origin as a commissioned work from a proprietary company, without an interest in working with the community. But the pressures of internationalization
means the community has long sought a replacement; one that it can continue to develop.
Liberation through the ages
Liberation was released in 2007 by Red Hat, who had commissioned the
designs from the commercial foundry Ascender Corporation. The initial
set consisted of three fonts — Liberation Sans, Liberation
Serif, and Liberation Mono — specifically designed to have the
same metrics as the proprietary Monotype fonts Arial, Times New Roman,
and Courier New, respectively. That meant that every character in
Liberation would be the exact same height and width as its counterpart
in the proprietary font, and Liberation could serve as a "drop in"
replacement without disturbing line breaks or pagination. In 2010,
Oracle donated a fourth typeface to Liberation: Liberation Sans
Narrow, which was designed to be metric-compatible with Arial Narrow.
The Liberation family was regarded as high-quality, but it covered
only the Latin, Greek, and Cyrillic alphabets, which left a lot of
writing systems unaddressed. That alone is not a problem; fonts can
— and are — extended to new writing systems frequently.
But Liberation was licensed under unique terms, which inadvertently
prevented such expansion.
Originally, the license was the GPLv2 with the Free Software
Foundation's standard font
embedding exception (which specifies that embedding the font in a
PDF or similar document does not make the document itself a "combined
work" triggering the GPL). However, Red Hat subsequently appended
additional clauses to the license
covering trademark and intellectual property concerns, and
included a custom anti-Tivoization provision. After an examination of the
extra clauses, Debian decided
that they constituted additional restrictions on the GPLv2, which made
the license self-contradictory and the fonts impossible to
redistribute. The FSF reportedly found the Liberation license
not to be a self-contradicting paradox, but said
it was incompatible with the GPL. Furthermore, in recent times the GPL-with-font-embedding-exemption
approach has fallen out of favor as an open font licensing choice,
largely in favor of the SIL
Open Font License (OFL). Fedora is aware of this shift, and now recommends
the OFL for font projects.
Regardless of the exact details, however, the general consensus was
that Liberation's peculiar license was, at best, problematic. More
importantly, the practical upshot that few people were interested in
contributing new character sets. The fonts have essentially remained
unchanged since 2007. Minor fixes and isolated characters have been
added, but no entirely new scripts.
Replacement plan
But the metric-compatibility feature of
Liberation was its main selling point: it enabled Linux users to
share documents with colleagues that had the popular Monotype fonts
installed (e.g., all Windows systems), while ensuring
compatibility.
On May 17, Parag Nemade emailed
the Fedora-devel list to request packaging the Croscore family as a default, to serve as an alternative to Liberation. The Croscore family covers
all of the same language blocks as Liberation, plus several new ones
(such as Hebrew, Pinyin, and many
African alphabets). It consists of three fonts:
Arimo, Tinos, and Cousine, which also offer metric compatibility with
Monotype's Arial, Times New Roman, and Courier New.
They were commissioned by Google for use in ChromeOS, and not only are
they also the work of Steve Matteson, the same designer who created
Liberation, but they are in fact a more recent version of the exact
same designs. In an amusing bit of irony, however, Ascender Corporation
(Matteson's company) was acquired by Monotype in 2010, so the new
font family is copyrighted by Monotype, but designed to replace other
Monotype fonts.
More to the point, however, Google made Croscore available under the
OFL, which makes it simpler for outside contributors to extend the
fonts to new character sets. Following the discussion in Nemade's thread, Fedora font packager Pravin Satpute proposed
importing the Croscore sources into the Liberation package, replacing
the problematically-licensed content rather than starting a separate
package.
The Fontconfig package handles automatic font substitution on Linux,
so once a change is pushed through with rules that replace (for example) Courier
New with Croscore's Cousine instead of Liberation Mono, the only
remaining hurdle will be for users to get used to the new names in the
"Font" menu. On the other hand, growing the fonts to extend coverage
to new writing systems is not trivial. The OFL license makes it
easier; it enables developers to import and reshape glyphs from the large
assortment of other OFL-licensed fonts.
What makes all this so difficult, anyway?
The Fedora plan calls for the community to continue development on the
"Liberation 2.0" series, in the open, where the original
Liberation was not. It would probably be a minor story if it were not
for the fact that the same stalemate situation has developed for other
open font commissions.
Much the same sequence of events befell the Bitstream Vera
font family, which was designed by Bitstream (another commercial
foundry which has since been acquired by Monotype) for the GNOME
Foundation, and released in 2003. It, too, was under a license
unique to the project, and has not seen any significant updates since
its original release. Google has commissioned two fonts for
distribution with Android: the familiar Droid family and the newer Roboto;
both licensed under the Apache License (as is most of Android itself).
Both offer wide language coverage in at least one of the faces (the
sans serif), but have not otherwise seen significant expansion.
About the only open font commissioned from a commercial foundry that
has grown to include more languages and alphabets is the
Ubuntu font family designed by
the Dalton Maag foundry. Although the details are of course private,
Dalton Maag has an ongoing arrangement with Canonical to add more
character sets over time. But the project does use a public issue
tracker and accepts input and feedback from the user community,
which none of the other commercial font commissions do.
Those differences are revealing. Commissioned open font projects such
as Liberation and Bitstream Vera invariably attract significant attention
— as do large "donations" of other types to open source. But
when they are delivered in a self-contained bundle and not developed
further, they have far less impact. It is easy for those of us who
natively read European languages to forget just how many writing
systems are not covered by basic Latin, Greek, and Cyrillic.
Meanwhile, there are purely community-driven font projects that do
cover far more of the globe's writing systems, such as Linux Libertine or DejaVu (the latter of which
extends Bitstream Vera, side-stepping the peculiar Bitstream license
by releasing its changes into the public domain).
The perception among the public is that the commercial fonts are of
higher quality than the community-built fonts; a charge likely to
rankle anyone who works on free software professionally. But by
choosing non-standard licenses and not establishing the font as a
software package that can be studied and patched, the early commercial
commissions made that charge difficult to disprove. The problem is
exacerbated when the foundry is uninterested in continuing to
participate (as Bitstream was, when it said
it was only interested in extending Vera if it were paid to do
so).
But stagnation is detrimental to a font package just as it is to any
software. Not only can every font be extended to cover more
languages, but there are updated technologies like OpenType features
and the Web Open Font
Format (WOFF) to consider. Adding new character sets to a font is
clearly a challenging task, demanding familiarity with multiple
alphabets, and often requiring patches to be integrated by hand in a
tool like FontForge. Hopefully the re-started Liberation 2.0 effort
can draw on the lessons learned by Dalton Maag and DejaVu, and grow a
sustainable project around the family. The original Liberation fonts
filled a vital gap on Linux desktops and watching them languish has been
disconcerting. Liberation now has the opportunity to re-import an entire codebase under a better license than the one that has hampered it for five years; few projects get a chance to start over at that same level — this is one that at least deserves to take a second shot.
Comments (56 posted)
Page editor: Jonathan Corbet
Security
By Nathan Willis
June 20, 2012
The Internet Corporation for Assigned Names and Numbers (ICANN) is
committed to launching a slew of new generic top level domains (gTLDs;
i.e., those that are not country-code TLDs), and the first assortment of
proposals has been published. ICANN's process has attracted no shortage
of criticism, but there are also concerns over how the availability of
hundreds of unrestricted TLDs will impact security.
In 2000, ICANN approved the first new gTLDs since the dawn of the
DNS system in the 1980's. That set of seven domains (.aero, .biz,
.coop, .info, .museum, .name, and .pro) was selected by ICANN's board
out of roughly 40 applications, in one of its first official acts.
ICANN policy dictates that the "sponsored" gTLDs — .aero, .coop,
and .museum — be used only by particular industries or groups,
and that the "restricted" gTLDs — .biz, .name, and .pro —
be used only for specific purposes. Those requirements sound similar,
with the main difference being that sponsored gTLDs are proposed by
and subsequently managed by private entities. Eight more gTLDs were
approved in 2003: .asia, .cat, .jobs, .mobi, .tel, .travel., .post,
and .xxx, all of which are sponsored.
The current round of new gTLD selection is supposed to usher in
unlimited numbers of new domains. The application period started on
January 12 and ended on April 12, during which time ICANN took in
1,930 applications from 1,268 separate applicants. ICANN charged a
$185,000 application fee for each domain, with the understanding that
a $25,000 annual fee would accompany any domain eventually approved.
ICANN published a one-page overview
[PDF]
of the applications, noting that there were 230 domains that had more
than one applicant vying for control.
The list
reads much like you would expect; there are plenty of companies
seeking control of the .app, .secure, and .web gTLDs, many more out to
create a brand-specific gTLD (such as .google and .bmw), and a few
community- or geographically-oriented applications (such as .africa,
.catholic, or .ieee). Up next comes the objection
and dispute resolution process, which is tentatively slated to
last seven months. Each objection to a gTLD application must meet one
of ICANN's four grounds for objection (which are listed on the page),
be brought by someone who meets the "standing" criteria, and include
the appropriate fee (which varies depending on the objection). Those
without deep pockets can also leave
a comment at no charge, although comments that do not meet the
formal objection grounds will not be forwarded to the evaluation
panels.
Disputes between multiple organizations after the same domain will be
handled by an ICANN review committee. If a consensus cannot be
reached, the disputed domain will be auctioned off. The review
process divides the entire set of applications into batches, with the
first batch scheduled to land on reviewers' desks in July. ICANN has
devised a mechanism
for sorting applications into batches that is, shall we say, novel.
Each applicant logs in to the ICANN site and competes to click on a
timestamp-generating button; the applicants that come closest to
hitting the target time are in batch one. Applicants (although
perhaps "players" is more descriptive) get to select their own target
time, and are allowed to practice before generating their timestamp
for real.
Divide and conquer
The timestamp-generating process (which ICANN itself refers
to as "digital archery") has attracted plenty of
criticism and even mockery. But there are more substantial objections to
the batching process, too. Rohan Pearce at Computerworld quotes
one domain registrar as saying that applicants in later batches could
find themselves waiting a number of years before their applications
reach the examination stage.
The size of the fees associated with the process has also generated
criticism. There is not much data with which to impartially
compare ICANN's fee structure — apart from the fact that
$185,000 is a substantial hike from 2000 and 2003's $50,000 sticker
price. ICANN contends that running a gTLD is an expensive process not
to be undertaken casually, so the fees are meant in part to discourage
throngs of cybersquatters or mischief-makers from bogging down the
process. NPR says
that many see the high stakes as a "land grab" unfairly blocking
out non-profit and community groups in favor of well-heeled
businesses. It also notes that domain speculators shelled out a lot
of capital for gTLDs of common words, including one company that filed
307 separate applications. The National
Association of Advertisers even started a public petition to
protest the policy, arguing that it forces business to spend money
defensively acquiring domain names just to protect their brands.
Finally, there have long been critics who contend that ICANN and its
processes are too US-centric. SiliconValley.com reports
that China, Russia, and Brazil have lobbied to have ICANN's functions
transferred to the United Nations or another international body. 911
of the 1,930 gTLD applications came from North America, which is not a
majority, but may be enough to bolster such complaints.
Security implications
A radically-expanded set of valid gTLDs may also impact security. For
starters, with 2,000 TLDs in the wild, it will be more difficult for
legitimate businesses to police all of the possible variations on
their name and product brand — or expensive to register them
all. That will make it easier for domain phishing attackers to slip a
phony site past users' eyes. E.g., in the heat of the moment, are you
sure that your bank's actual URL was MyBank.finance
and not MyBank.financial, or that you typed
zork.games instead of zork.game? ICANN received
applications for all four of those gTLDs.
It is also possible that the massive influx of new top-level
registrars will make it more likely for a nefarious player of some
sort to get into the gTLD game. A phisher running a domain registrar
is a little far-fetched, but there are other possibilities. Some have
suggested that the expansion plan will overload the root DNS zone, and
that it would be better to partition the root. China has proposed
a plan to the IETF that implements multiple autonomous roots. Under
the plan, China would control its own country code TLD (.cn) and other
national domain names, but still call out to peer DNS networks to
resolve other domain names. Computerworld quotes
Patrik Wallström of OpenDNSSEC as saying that the
proposal instead amounts to "a way to severely segment the
Internet," and notes China's reputation for blocking access to
Internet content.
Then again, ICANN has had its own in-house security problems plague
the gTLD process. It accidentally posted
the mailing addresses and other personal information of applicants on
the public web site (information which was supposed to remain
confidential). That leak followed May's incident, in which the
organization had to shut down the gTLD application system because it
inadvertently exposed
personal information to other applicants.
Whatever the long-term impact is on security, one can rest assured
that increasing the number of TLDs by a factor of 100 will cause
considerable extra work for administrators and developers, on every
task from email address verification to traffic analysis. The fifteen
new gTLDs ICANN has already introduced still account for only a
fraction of the registrations in the original TLDs, and while none of
the newly-proposed TLDs are likely to unseat .com either,
rewriting the rules of what constitutes a valid domain will have
far-reaching impacts.
Comments (21 posted)
Brief items
Far-fetched tales of West African riches
strike most as comical. Our analysis suggests that is an
advantage to the attacker, not a disadvantage. Since
his attack has a low density of victims the Nigerian
scammer has an over-riding need to reduce false positives. By sending an email that repels all but the most
gullible the scammer gets the most promising marks to
self-select, and tilts the true to false positive ratio in his
favor.
--
Cormac
Herley [PDF] in
Why do Nigerian Scammers Say They are from Nigeria?
From the tone of the hearing, and the language of the House resolution, we are being asked to believe that "the position of the United States Government has been and is to advocate for the flow of information free from government control."
If only it were true. The reality is that Congress increasingly has its paws all over the Internet. Lawmakers and regulators are busier than ever trying to expand the horizons of cyber-control across the board: copyright mandates, cybersecurity rules, privacy regulations, speech controls, and much more.
--
Jerry Brito and Adam Thierer
This seems to be a result of a fundamental misunderstanding of the economic incentives involved here, combined with a magical thinking that a market solution solves all. In airport screening, the passenger isn't the customer. (Technically he is, but only indirectly.) The airline isn't even the customer. The customer is the U.S. government, who is in the grip of an irrational fear of terrorism.
It doesn't matter if an airport screener receives a paycheck signed by the Department of the Treasury or Private Airport Screening Services, Inc. As long as a terrorized government -- one that needs to be seen by voters as "tough on terror," wants to stop every terrorist attack regardless of the cost, and is willing to sacrifice all for the illusion of security -- gets to set the security standards, we're going to get TSA-style security.
--
Bruce
Schneier on a TSA privatization bill
Comments (none posted)
New vulnerabilities
389-ds-base: denial of service
| Package(s): | 389-ds-base |
CVE #(s): | CVE-2012-0833
|
| Created: | June 20, 2012 |
Updated: | July 10, 2012 |
| Description: |
From the Red Hat advisory:
A flaw was found in the way the 389 Directory Server daemon (ns-slapd)
handled access control instructions (ACIs) using certificate groups. If an
LDAP user that had a certificate group defined attempted to bind to the
directory server, it would cause ns-slapd to enter an infinite loop and
consume an excessive amount of CPU time. |
| Alerts: |
|
Comments (none posted)
389-ds-base: plain text password disclosure
| Package(s): | 389-ds-base |
CVE #(s): | CVE-2012-2678
CVE-2012-2746
|
| Created: | June 20, 2012 |
Updated: | July 11, 2012 |
| Description: |
From the Red Hat advisory:
A flaw was found in the way 389 Directory Server handled password changes.
If an LDAP user has changed their password, and the directory server has
not been restarted since that change, an attacker able to bind to the
directory server could obtain the plain text version of that user's
password via the "unhashed#user#password" attribute. (CVE-2012-2678)
It was found that when the password for an LDAP user was changed, and audit
logging was enabled (it is disabled by default), the new password was
written to the audit log in plain text form. This update introduces a new
configuration parameter, "nsslapd-auditlog-logging-hide-unhashed-pw", which
when set to "on" (the default option), prevents 389 Directory Server from
writing plain text passwords to the audit log. This option can be
configured in "/etc/dirsrv/slapd-[ID]/dse.ldif". (CVE-2012-2746) |
| Alerts: |
|
Comments (none posted)
abrt: information leak
| Package(s): | abrt, libreport, btparser, python-meh |
CVE #(s): | CVE-2012-1106
|
| Created: | June 20, 2012 |
Updated: | December 12, 2012 |
| Description: |
From the Red Hat advisory:
If the C handler plug-in in ABRT was enabled (the abrt-addon-ccpp package
installed and the abrt-ccpp service running), and the sysctl
fs.suid_dumpable option was set to "2" (it is "0" by default), core dumps
of set user ID (setuid) programs were created with insecure group ID
permissions. This could allow local, unprivileged users to obtain sensitive
information from the core dump files of setuid processes they would
otherwise not be able to access. |
| Alerts: |
|
Comments (none posted)
apt: man-in-the-middle attack
| Package(s): | apt |
CVE #(s): | CVE-2012-0954
|
| Created: | June 18, 2012 |
Updated: | June 20, 2012 |
| Description: |
From the Ubuntu advisory:
Georgi Guninski discovered that APT did not properly validate imported
keyrings via apt-key net-update. USN-1475-1 added additional verification
for imported keyrings, but it was insufficient. If a remote attacker were
able to perform a man-in-the-middle attack, this flaw could potentially be
used to install altered packages. This update corrects the issue by
disabling the net-update option completely. A future update will re-enable
the option with corrected verification. |
| Alerts: |
|
Comments (none posted)
clamav: multiple vulnerabilities
| Package(s): | clamav |
CVE #(s): | CVE-2012-1457
CVE-2012-1458
CVE-2012-1459
|
| Created: | June 18, 2012 |
Updated: | August 17, 2012 |
| Description: |
From the Mandriva advisory:
The TAR file parser in ClamAV 0.96.4 allows remote attackers to bypass
malware detection via a TAR archive entry with a length field that
exceeds the total TAR file size. NOTE: this may later be SPLIT into
multiple CVEs if additional information is published showing that the
error occurred independently in different TAR parser implementations
(CVE-2012-1457).
The Microsoft CHM file parser in ClamAV 0.96.4 allows remote attackers
to bypass malware detection via a crafted reset interval in the LZXC
header of a CHM file. NOTE: this may later be SPLIT into multiple CVEs
if additional information is published showing that the error occurred
independently in different CHM parser implementations (CVE-2012-1458).
The TAR file parser in ClamAV 0.96.4 allows remote attackers to
bypass malware detection via a TAR archive entry with a length field
corresponding to that entire entry, plus part of the header of the
next entry. NOTE: this may later be SPLIT into multiple CVEs if
additional information is published showing that the error occurred
independently in different TAR parser implementations (CVE-2012-1459). |
| Alerts: |
|
Comments (none posted)
ffmpeg: multiple vulnerabilities
| Package(s): | ffmpeg |
CVE #(s): | CVE-2011-3951
CVE-2011-3952
CVE-2012-0851
CVE-2012-0852
|
| Created: | June 15, 2012 |
Updated: | August 7, 2012 |
| Description: |
From the Debian advisory:
It was discovered that ffmpeg, Debian's version of the libav media
codec suite, contains vulnerabilities in the DPCM codecs
(CVE-2011-3951), H.264 (CVE-2012-0851), ADPCM (CVE-2012-0852), and the
KMVC decoder (CVE-2011-3952). |
| Alerts: |
|
Comments (none posted)
java: multiple unspecified vulnerabilities
| Package(s): | java-1.7.0-oracle |
CVE #(s): | CVE-2012-0551
CVE-2012-1721
CVE-2012-1722
CVE-2012-1726
|
| Created: | June 20, 2012 |
Updated: | September 28, 2012 |
| Description: |
From the CVE entries:
Unspecified vulnerability in the Java Runtime Environment (JRE) in Oracle Java SE 7 update 4 and earlier and 6 update 32 and earlier, and the GlassFish Enterprise Server component in Oracle Sun Products Suite GlassFish Enterprise Server 3.1.1, allows remote attackers to affect confidentiality and integrity via unknown vectors related to Web Container or Deployment. (CVE-2012-0551)
Unspecified vulnerability in the Java Runtime Environment (JRE) component in Oracle Java SE 7 update 4 and earlier, and 6 update 32 and earlier, allows remote attackers to affect confidentiality, integrity, and availability via unknown vectors related to Deployment, a different vulnerability than CVE-2012-1722. (CVE-2012-1721)
Unspecified vulnerability in the Java Runtime Environment (JRE) component in Oracle Java SE 7 update 4 and earlier, and 6 update 32 and earlier, allows remote attackers to affect confidentiality, integrity, and availability via unknown vectors related to Deployment, a different vulnerability than CVE-2012-1721. (CVE-2012-1722)
Unspecified vulnerability in the Java Runtime Environment (JRE) component in Oracle Java SE 7 update 4 and earlier allows remote attackers to affect confidentiality and integrity via unknown vectors related to Libraries. (CVE-2012-1726) |
| Alerts: |
|
Comments (2 posted)
kernel: multiple vulnerabilities
| Package(s): | kernel |
CVE #(s): | CVE-2012-2137
CVE-2012-2373
|
| Created: | June 19, 2012 |
Updated: | October 12, 2012 |
| Description: |
From the Red Hat advisory:
A buffer overflow flaw was found in the setup_routing_entry() function in
the KVM subsystem of the Linux kernel in the way the Message Signaled
Interrupts (MSI) routing entry was handled. A local, unprivileged user
could use this flaw to cause a denial of service or, possibly, escalate
their privileges. (CVE-2012-2137)
A race condition was found in the Linux kernel's memory management
subsystem in the way pmd_populate() and pte_offset_map_lock() interacted on
32-bit x86 systems with more than 4GB of RAM. A local, unprivileged user
could use this flaw to cause a denial of service. (CVE-2012-2373) |
| Alerts: |
|
Comments (none posted)
krb5: denial of service
| Package(s): | krb5 |
CVE #(s): | CVE-2012-1013
|
| Created: | June 14, 2012 |
Updated: | August 1, 2012 |
| Description: |
From the Kerberos release notes:
Fix a kadmind denial of service issue (null pointer dereference),
which could only be triggered by an administrator with the "create"
privilege. [CVE-2012-1013] |
| Alerts: |
|
Comments (none posted)
libav: multiple vulnerabilities
| Package(s): | libav |
CVE #(s): | CVE-2011-3945
CVE-2011-4031
CVE-2012-0848
CVE-2012-0850
CVE-2012-0858
CVE-2012-0859
|
| Created: | June 18, 2012 |
Updated: | February 18, 2013 |
| Description: |
From the Ubuntu advisory:
Mateusz Jurczyk and Gynvael Coldwind discovered that Libav incorrectly
handled certain malformed Kega Game Video (KGV1) files. If a user were
tricked into opening a crafted Kega Game Video (KGV1) file, an attacker
could cause a denial of service via application crash, or possibly execute
arbitrary code with the privileges of the user invoking the program. This
issue only affected Ubuntu 11.04 and Ubuntu 11.10. (CVE-2011-3945)
Jeong Wook Oh discovered that Libav incorrectly handled certain malformed
ASF files. If a user were tricked into opening a crafted ASF file, an
attacker could cause a denial of service via application crash, or possibly
execute arbitrary code with the privileges of the user invoking the
program. This issue only affected Ubuntu 11.10. (CVE-2011-4031)
It was discovered that Libav incorrectly handled certain malformed
Westwood SNDx files. If a user were tricked into opening a crafted Westwood
SNDx file, an attacker could cause a denial of service via application
crash, or possibly execute arbitrary code with the privileges of the user
invoking the program. This issue only affected Ubuntu 11.10.
(CVE-2012-0848)
Diana Elena Muscalu discovered that Libav incorrectly handled certain
malformed AAC files. If a user were tricked into opening a crafted AAC
file, an attacker could cause a denial of service via application crash, or
possibly execute arbitrary code with the privileges of the user invoking
the program. This issue only affected Ubuntu 11.04 and Ubuntu 11.10.
(CVE-2012-0850)
It was discovered that Libav incorrectly handled certain malformed Shorten
files. If a user were tricked into opening a crafted Shorten file, an
attacker could cause a denial of service via application crash, or possibly
execute arbitrary code with the privileges of the user invoking the
program. This issue only affected Ubuntu 11.04 and Ubuntu 11.10.
(CVE-2012-0858)
It was discovered that Libav incorrectly handled certain malformed Vorbis
files. If a user were tricked into opening a crafted Vorbis file, an
attacker could cause a denial of service via application crash, or possibly
execute arbitrary code with the privileges of the user invoking the
program. This issue only affected Ubuntu 11.04 and Ubuntu 11.10.
(CVE-2012-0859) |
| Alerts: |
|
Comments (none posted)
libguestfs: unintended file access
| Package(s): | libguestfs |
CVE #(s): | CVE-2012-2690
|
| Created: | June 20, 2012 |
Updated: | July 10, 2012 |
| Description: |
From the Red Hat advisory:
It was found that editing files with virt-edit left said files in a
world-readable state (and did not preserve the file owner or
Security-Enhanced Linux context). If an administrator on the host used
virt-edit to edit a file inside a guest, the file would be left with
world-readable permissions. This could lead to unprivileged guest users
accessing files they would otherwise be unable to. |
| Alerts: |
|
Comments (none posted)
libvirt: unintended access to USB devices
| Package(s): | libvirt |
CVE #(s): | CVE-2012-2693
|
| Created: | June 20, 2012 |
Updated: | January 17, 2013 |
| Description: |
From the Red Hat advisory:
Bus and device IDs were ignored when attempting to attach multiple USB
devices with identical vendor or product IDs to a guest. This could result
in the wrong device being attached to a guest, giving that guest root
access to the device. |
| Alerts: |
|
Comments (none posted)
mysql: temporary denial of service
| Package(s): | mysql |
CVE #(s): | CVE-2012-2102
|
| Created: | June 20, 2012 |
Updated: | July 10, 2012 |
| Description: |
From the Red Hat advisory:
A flaw was found in the way MySQL processed HANDLER READ NEXT statements
after deleting a record. A remote, authenticated attacker could use this
flaw to provide such requests, causing mysqld to crash. This issue only
caused a temporary denial of service, as mysqld was automatically restarted
after the crash. |
| Alerts: |
|
Comments (none posted)
nss: multiple vulnerabilities
| Package(s): | nss, nss-util, nspr |
CVE #(s): | |
| Created: | June 20, 2012 |
Updated: | July 11, 2012 |
| Description: |
Red Hat has updated nss, nss-util, and nspr packages that fix one security issue, several bugs, and add various enhancements. |
| Alerts: |
|
Comments (none posted)
openconnect: denial of service
| Package(s): | openconnect |
CVE #(s): | CVE-2012-3291
|
| Created: | June 18, 2012 |
Updated: | July 10, 2012 |
| Description: |
From the Debian advisory:
A buffer overflow was discovered in OpenConnect, a client for the Cisco
AnyConnect VPN, which could result in denial of service. |
| Alerts: |
|
Comments (none posted)
openldap: denial of service
| Package(s): | openldap |
CVE #(s): | CVE-2012-1164
|
| Created: | June 20, 2012 |
Updated: | August 13, 2012 |
| Description: |
From the Red Hat advisory:
A denial of service flaw was found in the way the OpenLDAP server daemon
(slapd) processed certain search queries requesting only attributes and no
values. In certain configurations, a remote attacker could issue a
specially-crafted LDAP search query that, when processed by slapd, would
cause slapd to crash due to an assertion failure. |
| Alerts: |
|
Comments (none posted)
openssh: denial of service
| Package(s): | openssh |
CVE #(s): | CVE-2011-5000
|
| Created: | June 20, 2012 |
Updated: | July 11, 2012 |
| Description: |
From the Red Hat advisory:
A denial of service flaw was found in the OpenSSH GSSAPI authentication
implementation. A remote, authenticated user could use this flaw to make
the OpenSSH server daemon (sshd) use an excessive amount of memory, leading
to a denial of service. GSSAPI authentication is enabled by default
("GSSAPIAuthentication yes" in "/etc/ssh/sshd_config"). |
| Alerts: |
|
Comments (none posted)
opera: multiple vulnerabilities
Comments (none posted)
oracle-update: man-in-the-middle attack
| Package(s): | oracle-update |
CVE #(s): | CVE-2012-1675
|
| Created: | June 20, 2012 |
Updated: | June 20, 2012 |
| Description: |
From the CVE entry:
The TNS Listener, as used in Oracle Database 11g 11.1.0.7, 11.2.0.2, and 11.2.0.3, and 10g 10.2.0.3, 10.2.0.4, and 10.2.0.5, as used in Oracle Fusion Middleware, Enterprise Manager, E-Business Suite, and possibly other products, allows remote attackers to execute arbitrary database commands by performing a remote registration of a database (1) instance or (2) service name that already exists, then conducting a man-in-the-middle (MITM) attack to hijack database connections, aka "TNS Poison." |
| Alerts: |
|
Comments (none posted)
php-symfony-symfony: session fixation flaw
| Package(s): | php-symfony-symfony |
CVE #(s): | CVE-2012-2667
|
| Created: | June 15, 2012 |
Updated: | June 20, 2012 |
| Description: |
From the Fedora advisory:
Bug #828079 - CVE-2012-2667 php-symfony-symfony: Session fixation flaw corrected in upstream 1.4.18 version [fedora-all]
https://bugzilla.redhat.com/show_bug.cgi?id=828079
|
| Alerts: |
|
Comments (none posted)
python: cross-site scripting
| Package(s): | python |
CVE #(s): | CVE-2011-4940
|
| Created: | June 18, 2012 |
Updated: | October 18, 2012 |
| Description: |
From the Red Hat advisory:
A flaw was found in the way the Python SimpleHTTPServer module generated
directory listings. An attacker able to upload a file with a
specially-crafted name to a server could possibly perform a cross-site
scripting (XSS) attack against victims visiting a listing page generated by
SimpleHTTPServer, for a directory containing the crafted file (if the
victims were using certain web browsers). |
| Alerts: |
|
Comments (none posted)
qt: multiple vulnerabilities
| Package(s): | qt |
CVE #(s): | CVE-2010-5076
CVE-2011-3922
|
| Created: | June 20, 2012 |
Updated: | July 10, 2012 |
| Description: |
From the Red Hat advisory:
A buffer overflow flaw was found in the harfbuzz module in Qt. If a user
loaded a specially-crafted font file with an application linked against Qt,
it could cause the application to crash or, possibly, execute arbitrary
code with the privileges of the user running the application.
(CVE-2011-3922)
A flaw was found in the way Qt handled X.509 certificates with IP address
wildcards. An attacker able to obtain a certificate with a Common Name
containing an IP wildcard could possibly use this flaw to impersonate an
SSL server to client applications that are using Qt. This update also
introduces more strict handling for hostname wildcard certificates by
disallowing the wildcard character to match more than one hostname
component. (CVE-2010-5076) |
| Alerts: |
|
Comments (none posted)
quagga: denial of service
| Package(s): | quagga |
CVE #(s): | CVE-2012-1820
|
| Created: | June 19, 2012 |
Updated: | April 10, 2013 |
| Description: |
From the Red Hat bugzilla:
A denial of service flaw was found in the way Quagga's bgpd daemon processed certain OPEN messages. A configured Border Gateway Protocol (BGP) peer could send a BGP OPEN message with specially-crafted value of the Outbound Route Filtering (ORF) capability Type/Length/Value (TLV) triplet, which would cause the master BGP daemon (bgpd) to abort with an assertion failure by processing of such a message. Also, all BGP sessions established by the attacked router would be closed and its BGP routing disrupted. |
| Alerts: |
|
Comments (none posted)
rubygem-actionpack: Unsafe query generation
Comments (none posted)
rubygem-activerecord: SQL injection
Comments (none posted)
sblim-cim-client2: predictable hash collisions
| Package(s): | sblim-cim-client2 |
CVE #(s): | CVE-2012-2328
|
| Created: | June 20, 2012 |
Updated: | January 23, 2013 |
| Description: |
From the Red Hat advisory:
It was found that the Java HashMap implementation was susceptible to
predictable hash collisions. SBLIM uses HashMap when parsing XML inputs. A
specially-crafted CIM-XML message from a WBEM (Web-Based Enterprise
Management) server could cause a SBLIM client to use an excessive amount of
CPU. Randomization has been added to help avoid collisions. |
| Alerts: |
|
Comments (none posted)
sos: privilege escalation
| Package(s): | sos |
CVE #(s): | CVE-2012-2664
|
| Created: | June 20, 2012 |
Updated: | July 11, 2012 |
| Description: |
From the Red Hat advisory:
The sosreport utility collected the Kickstart configuration file
("/root/anaconda-ks.cfg"), but did not remove the root user's password from
it before adding the file to the resulting archive of debugging
information. An attacker able to access the archive could possibly use this
flaw to obtain the root user's password. "/root/anaconda-ks.cfg" usually
only contains a hash of the password, not the plain text password. |
| Alerts: |
|
Comments (none posted)
Page editor: Jake Edge
Kernel development
Brief items
The current development kernel is 3.5-rc3,
released on June 16. "
The week
started calm with just a few small pulls, with people apparently really
trying to make my life easier during travels - thank you. But it kind of
devolved at some point, and I think more than half the pull requests came
in the last two days and they were bigger too. Oh well.." It's
mostly fixes, but there is also a new network driver for Tile-Gx systems.
Stable updates: the 3.0.35 and 3.4.3 stable kernel updates were released on
June 17; 3.2.21 was released on
June 19. These updates all contain the usual set of important fixes.
The 3.0.36 and 3.4.4 updates are in the review process as of
this writing; they can be expected on or after June 22.
Comments (none posted)
Sorry, it can't always be constructive, but I'll try my best. I'll
also try to not cast aversions about your cat, but if you taunt me,
all bets are off.
—
Greg Kroah-Hartman
Hooks and notifiers are a form of "COME FROM" programming, and they
make it very hard to reason about the code. The only way that that
can be reasonably mitigated is by having the exact semantics of a
hook or notifier -- the preconditions, postconditions, and other
invariants -- carefully documented. Experience has shown that in
practice that happens somewhere between rarely and never.
—
H. Peter Anvin
Comments (22 posted)
The planning process for the 2012 Kernel Summit (August 27-29, San Diego)
has begun. "
This year, in order to make the selection process more transparent,
we're trying a new mechanism where we'll be selecting this year's
attendees from amongst those who submit proposals to attend as
described below." There is no formal deadline for proposals, but
sooner is better.
Full Story (comments: none)
Neil Brown has written a
blog post about a nasty
RAID bug in some versions of the Linux kernel. "
The bug only
fires when you shutdown/poweroff/reboot the machine. While the machine
remains up the bug is completely inactive. So you will only notice the bug
when you boot up again. The effect of the bug is to erase important information from the metadata that is stored on the disk drives. In particular the level, chunksize, number of devices, data_offset and role of each device in the array are erased ... and probably some other information too. This means that if you know those details you can recover your data, but if you don't, it will be harder. Hence the "mdadm -E" command suggested earlier."
Comments (23 posted)
Kernel development news
By Jonathan Corbet
June 18, 2012
Memory storage devices, including flash, are essentially just random-access
devices with some peculiar restrictions. Given direct access to the
device, Linux kernel developers could certainly come up with drivers that
would provide optimal performance and device lifetime. In the real world,
though, these devices are hidden behind their own proprietary operating systems and
software stacks; much of the real (commercial) value seems to be in the
software bundled inside. As a result, the kernel must try to
coax the device's firmware into doing an optimal job. Over time, the
storage industry has added various mechanisms by which an operating system
can pass hints down to the device; the "trim" or "discard" mechanism is one
of those. Newer eMMC and unified flash storage (UFS) devices add a
new hint in the form of
"contexts"; patches exist to support this feature, but they seem to have
raised more questions than they have answered.
The standards documents describing contexts do not appear to be widely
available—or at least findable. From what your editor has been able to
divine, "contexts" are a small number added to I/O requests that are
intended to help the device optimize the execution of those requests. They
are meant to differentiate different types of I/O, keeping large,
sequential operations separate from small, random requests. I/O can be
placed into a "large unit" context, where the operating system promises to
send large requests and, possibly, not attempt to read the data back until
the context has been closed.
Saugata Das recently posted a small patch
set adding context support to the ext4 filesystem and the MMC block
driver. At the lower level, context numbers are associated with block I/O
requests by storing the number in the newly-added bi_context (in
struct bio) and context (in struct request)
fields. The virtual filesystem layer takes responsibility for setting
those fields, but, in the end, it defers to the actual filesystems to come
up with the proper context numbers. There is a new address space operation
(called get_context()) by which the VFS can call into the
filesystem code to obtain a context number for a specific request. The
block layer has been modified to avoid merging block I/O requests if those
requests have been assigned to different contexts.
There was little discussion of the lower-level changes, which apparently
make sense to the developers who have examined them. The filesystem-level
changes have seen rather more discussion, though. Saugata's patch set only
touches the ext4 filesystem; those changes cause ext4 to use the inode
number of the file
under I/O as the context number. Thus, all I/O requests to a single file
will be assigned to the same context, while requests to different files
would go into different contexts (within limits—eMMC hardware, for example, only supports 15 contexts, so many inode numbers will be mapped onto a single context number at the lower levels).
The question that came up was: is using the inode number the right policy?
Coming up with an answer involves addressing two independent questions:
(1) what does the "context" mechanism actually do?, and (2) how
can Linux filesystems provide the best possible context information to the
storage devices?
Arnd Bergmann (who has spent a lot of time
understanding the details of how flash storage works) has noted that the standard is deliberately vague
on what the context mechanism does; the authors wanted to create something
that would outlive any specific technology. He went on to say:
That said, I think it is rather clear what the authors of the spec
had in mind, and there is only one reasonable implementation given
current flash technology: You get something like a log structured
file system with 15 contexts, where each context writes to exactly
one erase block at a given time.
The effect of such an implementation would be to concentrate data written
under any one context into the same erase block(s). Given that, there are at
least a couple of ways to use contexts to optimize I/O performance.
For example, one could try to concentrate data with the same expected
lifetime, so that, when part of an erase block is deleted, all of the data
in that erase block will be deleted. Using the inode number as the context
number could have that effect; deleting the file associated with that inode
will delete all of its blocks at the same time. So, as long as the file is
not subject to random writes (as, say, a database file might be), using
contexts in this manner should reduce the amount of garbage collection and
read-modify-write cycles needed when a file is deleted.
Another helpful approach might be to use contexts to separate large,
long-lived files from those that are shorter and more ephemeral. The
larger files would be well-placed on the medium, and the more volatile data
would be concentrated into a smaller number of erase blocks. In this case,
using the inode number to identify contexts may or may not work well.
Large files would be nicely separated, but the smaller files could be
separated from each other as well, which may not be desirable: if
several small files would fit into a single erase block, performance might
be improved if all of those files were written in the same context.
In this case, some other policy might be more advisable.
But what should that policy be? Arnd suggested that using the inode number
of the directory containing the file might work better. Various commenters
thought that using the ID of the process writing to the file could work,
though there are some potential difficulties when multiple processes write
the same file. Ted Ts'o suggested that
grouping files written by the same process in a short period of time could
give good results. Also useful, he thought, might be to look at the size
of the file relative to the device's erase block size; files much smaller
than an erase block would be placed into the same context, while larger
files would get a context of their own.
A related idea, also from Ted, was to look
at the expected I/O patterns. If an existing file is opened for write
access, chances are good that a random I/O pattern will result. Files
opened with O_CREAT, instead, are more likely to be sequential;
separating those two types of files into different contexts would likely
yield better results. Some flags used with posix_fadvise() could
also be used in this way. There are undoubtedly other possibilities as
well. Choosing a policy will have to be done with care; poor use of
contexts could just as easily reduce performance and longevity instead of
increasing them.
Figuring all of this out will certainly take some time, especially since
devices with actual support for this feature are still relatively rare.
Interestingly, according to Arnd, there may
be an opportunity in getting ext4 to supply context information early:
Having code in ext4 that uses the contexts will at least make it
more likely that the firmware optimizations are based on ext4
measurements rather than some other file system or operating
system. From talking with the emmc device vendors, I can tell you
that ext4 is very high on the list of file systems to optimize for,
because they all target Android products.
Ext4 is, of course, the filesystem of choice for current Android systems.
So, conceivably, an ext4 implementation could drive hardware behavior in
the same way that much desktop hardware is currently designed around what
Windows does.
Given that the patches are relatively small and that policies can be
changed in the future without user-space compatibility issues, chances are
good that something will be merged into the mainline as soon as the 3.6
development cycle. Then it will just be a matter of seeing what the
hardware manufacturers actually do and adjusting accordingly. With luck,
the eventual result will be longer-lasting, better-performing memory
storage devices.
Comments (7 posted)
By Jonathan Corbet
June 19, 2012
Some kernel behavior is determined by standards like POSIX; others are
simply a function of what the kernel developers implemented. The latter
type of behavior can, in theory, be changed if there is a good reason to do
so, but there is always a risk of breaking an application that depended on
the previous behavior. Even worse, this kind of problem can be impossible
to find during development and, indeed, may lurk until long after the new
code has been deployed. A system call patch currently under consideration
shows how hard it can be to know when a change is truly safe.
The msync() system call exists to request that a file-backed
memory region created with mmap() be written back to persistent
storage. Once upon a time, msync() was the only way to guarantee
that modified pages would be saved to disk in any reasonable period of time;
the kernel could not always detect
on its own that they had been changed by the application. That problem has
long since been dealt with, but msync() is still a good way to
inform the kernel that now would be a good time to flush modified pages to
disk.
Paolo Bonzini recently posted a small patch
set making a couple of changes to msync(). The actual API
does not change at all, but how the system call implements the API changes
in subtle and interesting ways.
There are a few options to msync(), one of which
(MS_ASYNC) asks that the writeback of modified pages be
"scheduled," but not necessarily completed immediately. It is meant to be
a non-blocking system call that sets the necessary actions in motion, but
does not wait for them to complete. Current kernels will write back dirty
pages as part of the normal writeback process; the system behaves, in other
words, as if msync(MS_ASYNC) were being called on a regular basis
on every mapping. Writeback of dirty pages is already scheduled as soon as
the page is dirtied.
Given that, there's not much work for an explicit
MS_ASYNC call from user space to do, and, indeed, the kernel
essentially ignores such calls.
Paolo's patch causes the kernel to immediately start I/O on modified pages
in response to MS_ASYNC calls. The result is to get those pages
to persistent storage a bit more quickly than would otherwise happen, but
still avoid blocking the calling process. The change seems reasonable,
but Andrew Morton worried that this behavioral change might be a
problem for some users:
Means that people will find that their msync(MS_ASYNC) call will
newly start IO. This may well be undesirable for some. Also, it
hardwires into the kernel behaviour which userspace itself could
have initiated, with sync_file_range(). ie: reduced flexibility.
Most users are unlikely to notice the change at all. But it's entirely
possible that somebody out there has a precisely-tuned system that will
choke if the underlying I/O behavior changes. Users complain about exactly
this kind of change at times, but usually when the change shows up in a new
enterprise kernel, years too late.
That said, many patches make behavioral changes that can affect users in
surprising ways. The only thing that is different about this one is that
the nature of the change is understood from the beginning. Andrew's
concerns were not echoed by others and may not be enough to keep this
change from being merged.
The other change is potentially more troubling. msync() takes two
parameters indicating the offset and length of the memory area to be
written back. But the kernel has always ignored those parameters, choosing
instead to just write back all modified pages in the file, and the related
metadata as well. Paolo's patch changes the implementation to only
synchronize the specific pages requested by the user.
It would be hard to argue that the new behavior breaks the documented API;
the offset and length parameters are there for a reason, after all. Still,
once again, Andrew worried that
applications could break in especially unpleasant ways:
Would be nice, but if applications were previously assuming that an
msync() was syncing the whole file, this patch will secretly and
subtly break them.
No developer should have written a program with the assumption that
msync() would write pages outside of the range it was given. Any
such program would clearly be buggy. But, programs written that way will
work under current kernels. Changing msync() to not write some
pages that it currently writes could cause such programs to fail in strange
and difficult-to-reproduce ways.
In general, the kernel tries not to break existing applications, even if
those applications can be said to have been written in a buggy manner. If
something works now, it should continue to work with future kernels. If
the msync() changes described here break those programs, the
changes should probably not be merged into the kernel.
The problem, of course, is that it can be very difficult to know if
a specific change will break somebody's application. Any problems caused
by subtle changes are relatively unlikely to turn
up before the changes are included in a released kernel. So it is
necessary to proceed with care. That said, it is not practical to hold
back every change that might break a badly-written program
somewhere; kernel development would likely be slowed considerably by such a
constraint. So, probably, these changes will probably go in unless an
affected user happens to notice a problem in the near future.
Comments (18 posted)
By Jake Edge
June 20, 2012
As preparation for this year's Kernel Summit gets underway, a new "more
transparent" process is being used to
select the 80-100 participants. The Summit will take place August 27-29,
just prior to
LinuxCon North America in San Diego. Those interested in attending are
being asked to describe the technical expertise they will bring to the
meeting, as well as to suggest topics for discussion. All of that is
taking place on the ksummit-2012-discuss
mailing list since the announcement on June
14, so it seems worth a look to see what kinds of topics may find their way
onto the agenda.
Development process issues are a fairly common topic at the summit and they
figure in a number of the suggestions for this year. One of the hot topics
is the role of maintainers with multiple, at least partly related, ideas
about discussions in that area. Thomas Gleixner noted
a few concerns that he had in a mini-rant:
So the main questions I want to raise on Kernel Summit are:
- How do we cope with the need to review the increasing amount of
(insane) patches and their potential integration?
- How do we prevent further insanity to known problem spaces (like
cpu hotplug) without stopping progress?
A side question, but definitely related is:
- How do we handle "established maintainers" who are mainly
interested in their own personal agenda and ignoring justified
criticism just because they can?
As one might guess, that kicked off a bit of a conversation about those
problems on the list, but also led several developers to concur about the
need to discuss the problems at the summit. Somewhat more diplomatically, Trond
Myklebust suggested
a related discussion on a possible restructuring of the maintainer's role:
Currently, the Linux maintainer appears to be responsible for filling
all of the traditional roles of software architect, software developer,
patch reviewer, patch committer, and software maintainer.
My question is whether or not there might be some value in splitting out
some of these roles, so that we can assign them to different people, and
thus help to address the scalability issues that Thomas raised?
Greg Kroah-Hartman also wants
to talk about maintainership and offered to "referee" a discussion. He has
some ideas that he described at LinuxCon
Japan and in a recent linux-kernel
posting that he thinks "will go a long ways in helping smooth this
out". John Linville also expressed
interest in that kind of discussion.
Another area that is generating a lot of interest is the stable tree.
Kroah-Hartman is interested
in finding out how the process is working for the other kernel
developers:
[...] is it going well for
everyone? Are there things we can do differently? How can I kick
maintainers who don't mark patches for stable backports in ways that
do not harm them too much? How can I convey decisions about the
longterm kernel selection process in a better way so that it isn't
surprising to people?
Based on the number of other submissions that mentioned the stable tree,
there seems to be a fair amount to discuss. The relationship between the
stable tree and the distributions is one fertile area. Kroah-Hartman said
that he often has to go "digging through distro kernel trees" to find
patches to apply, to which Andrew Morton suggested
that the "distro people
need a vigorous wedgie" for not making that easier. Various distribution
kernel maintainers (e.g. Josh Boyer and Jiri Kosina) agreed that the
distributions could do better, but that some discussion of the process
would be worthwhile.
In addition, some discussion of how distributions could better work with
the upstream kernel for regression tracking and bug reporting was proposed
by Boyer. Kosina wants
to discuss the stable review process with an eye toward helping
distributions decide which patches to merge into their kernels.
Mark Brown is also interested
but from the perspective of embedded rather than enterprise distributions.
Others also expressed interest in having stable/longterm tree discussions.
How to track bugs and regressions was a topic proposed
by Rafael Wysocki, who has been reporting to the summit on that topic for
many years. He was joined by Dave Jones, who would like to report
on bugs and regressions, both those found by his "trinity"
stress-testing tool and ones that have been found in the Fedora kernel over
the last year. Like Wysocki, Kosina is also interested in discussing whether
the kernel bugzilla is the right tool for tracking bugs and regressions.
Kernel testing is another area that seems ripe for a discussion. Fengguang
Wu would like to report
on his efforts to test kernels as each new commit is added:
And I would like a chance to talk about doing kernel tests in a timely fashion:
whenever one pushes new commits to git.kernel.org, build/boot/stress tests will
be kicked off and possible errors be notified back to the author within hours.
This fast develop-test feedback cycle is enabled by running a test
backend that is able to build 25000 kernels and runtime test 3000
kernels (assuming 10m boot+testing time for each kernel) each day.
Just capable enough to outrace our patch creation rate ;-)
On an average day, 1-2 build errors are caught in the 160 monitored kernel trees.
Wu's posting spawned a long thread where various developers described their
test setups and what could be done better. Jones mentioned
the Coverity scanner in that thread, which led Jason Wessel to highlight
Jones's comment as well as give more information on the tool and the kinds
of information it can provide. More and better automated kernel testing is
definitely on the minds of a lot of potential summit attendees.
James Bottomley would like
to eliminate "kernel work creation schemes", in particular he targeted
the amount of code that is needed to support CONFIG_HOTPLUG:
[...] the
massive proliferation of __dev... _mem... __cpu... and their ilk are
getting out of control. Plus, the amount of memory they save is tiny (a
few pages at best) and finally virtually no-one compiles without
CONFIG_HOTPLUG, so they're mostly nops anyway. However, for that very
case, we've evolved a massive set of tools to beat ourselves up whenever
we violate the rules of using these tags. What I'd like to explore is
firstly, can we just eliminate CONFIG_HOTPLUG and make it always y (this
will clear up the problem nicely) or, failing that, can we just dump the
tags and the tools and stop causing work for something no-one cares
about.
There were few defenders of CONFIG_HOTPLUG=n in the thread, but he
was also interested in finding ways to avoid constructs that lead to a lot
of code churn to no good end. In a somewhat similar vein, H. Peter Anvin
would like to discuss the baseline
requirements for the kernel. Supporting some of the niche
uses of Linux (on exotic hardware or with seriously outdated toolchains)
creates an ongoing cost for kernel hackers that Anvin would like to see
reduced or eliminated.
Several PCI topics were proposed, including PCI
root bus hotplug issues by Yinghai Lu and a PCI
breakout session that Benjamin Herrenschmidt suggested. In the latter,
Lu's work, some PCI-related KVM issues, cleaning up some PowerPC special
cases, and the rework of the PCI hotplug
core could all be discussed. As Herrenschmidt put it: "I think there's enough material to keep us busy and a face to face round
table with a white board might end up being just the right thing to do".
Memory management topics also seem popular. Glauber Costa proposed
several topics, including kmem tracking and per-memory-control-group kmem
memory shrinking, while Hiroyuki Kamezawa suggested
memory control group topics. Johannes Weiner is also interested in talking
about a separate memory management tree that would supplement the work
that Morton does with the -mm tree. The ever-popular memory control group
writeback topic was also suggested by Wu and Weiner.
Srivatsa S. Bhat would like to present
a newcomer's perspective on kernel development with an eye toward
reducing some of the challenges new developers face. Josef Bacik has a similar
idea, and would like to discuss how to make it easier for new
contributors. In addition to a report on work in the USB subsystem
(and USB 3.0 in particular), Sarah Sharp would like
to "do a brief
readout" about what she learns at AdaCamp in July:
AdaCamp is a conference
focused on gathering tech women together to work on solutions for
getting women into open technology fields, and retaining them. I think
this would be of interest to the Linux kernel community, since we have
very few women kernel developers. I hope to keep this read out focused
on positive changes we can make.
As one can see, these proposals (and many more that were not mentioned)
range all over the kernel map. There tends to be a focus on more process
and social aspects of the kernel at the summit, mostly because the hardcore
technical topics are generally better handled by a more focused group. The
summit tries to address global concerns, and there seem to be plenty
to choose from.
Comments (2 posted)
Patches and updates
Kernel trees
Core kernel code
Development tools
Device drivers
Documentation
Filesystems and block I/O
Memory management
Networking
Architecture-specific
Security-related
Virtualization and containers
Miscellaneous
- Lucas De Marchi: kmod 9 .
(June 19, 2012)
Page editor: Jonathan Corbet
Distributions
June 20, 2012
This article was contributed by Bruce Byfield
After a release cycle dogged by problems and delays, the openSUSE
distribution announced the inevitable: its upcoming 12.2 release would be
delayed. However, what might have been less expected was that release
manager Stephan "Coolo" Kulow used the occasion to call
for a rethinking of openSUSE's release process. The result was an
ongoing discussion that, although short on definite conclusions, is
something of a case study of the challenges faced by large and growing projects.
Kulow made his statement against a background of steady growth for openSUSE project. According to Kulow, in the last year, the number of unique developers has increased from 182 to 222, or 22%, growth that is welcome, but strains long-established procedures and practices. As community manager Jos Poortvliet noted, in the last five months of the release schedule, "Pretty much every milestone of openSUSE 12.2 has been delayed or even canceled." The first two milestones were over a week late, and the first beta two weeks late. Even worse, the fourth milestone and first release candidate were canceled altogether. A revised release date has yet to be set, but Kulow is now aiming for mid-September, almost two months later than the original date.
Both Kulow and Poortvliet point to package integration, especially of GCC 4.7 and Automake, as the major problem. In other words, the packages work well enough themselves, but not enough work is being done to ensure that they interact with other packages properly. Talking about Factory, openSUSE's testing repository, Kulow stated that:
I don't remember a time during 12.2 development when we had less than 100 'red' [problem] packages. And we have packages that fail for almost five months without anyone picking up a fix. Or packages that have unsubmitted changes in their devel project for six months without anyone caring to submit it.
The result, Poortvliet said, is a "broken-window problem" in which existing problems create an atmosphere that encourages further neglect, creating a "downward spiral."
In announcing the delay in the release, Kulow suggested three improvements
were needed: more attention to integration work, more work in staging
projects (repositories for early development work on specialized
subsystems) rather than Factory, and a change in release schedules —
either abandoning them, or releasing only once a year. Kulow also declared
his intention to have "a no-tolerance strategy about packages
breaking other packages" and suggested: "It's time we realize delaying milestones is not a solution. Instead, let's use the delay of 12.2 as a reason to challenge our current development model and look at new ways."
Technologies, policies, and people
Kulow's announcement generated a long thread on the Factory mailing list,
with many potential problems raised, along with suggestions for possible solutions. Much of the discussion followed his lead and approached the state of openSUSE as a technical or organizational problem.
For example, Brian K. White asked
the basic question: "How does anyone find out that a change in their
package somehow breaks some other package?" The answer that emerged
was that Factory needed to be rebuilt after uploading packages to it.
However, since constant rebuilds would strain the project's hardware
resources, Adrian Schröter suggested
only having staging projects for packages that were a key part of the
distribution's structure. Another way to ease the demands on the servers
was proposed by Guido Berhoerster:
How about a multi-tiered Factory to address this and have all changes go into an openSUSE:Factory:Staging project and only migrate packages to openSUSE:Factory after a certain delay when they do not break others or introduce serious regressions. Factory would then ideally always build and thus make it easier to cut releases at fixed dates.
Alternatively, todd rme suggested
that Tumbleweed, the rolling release repository for openSUSE, could become
the source of official releases rather than Factory — moving openSUSE to
rolling releases entirely.
Or perhaps, as M. Edward Borasky speculated,
a combination of methods might be developed, with core packages using
cascading repositories and less vital packages using a repository set up for rolling releases.
As the discussion expanded to include these possibilities, Greg Kroah-Hartman pointed out that openSUSE's build system is so slow as to be impractical, and that packages were sometimes requiring unnecessary rebuilds. "We need a way to be able to mark a checkin as needing to cause this rebuild . . . or for a way for the build system to determine this on its own," Kroah-Hartman said. In response, Borasky mentioned Gentoo's USE flags, which can be toggled to include or exclude packages from a build, as a model for a solution.
Other sub-threads focused on policy and personnel. "Maybe we need
more public reminders of out of date and out of sync packages and people
from [the] community would step up," Alin M. Elena suggested,
adding that this system had worked in the preparation of GCC
4.7. Similarly, Marcus
Meissner pointed out that the technical suggestions would still require
people to do the integration work.
More generally, Sascha Peillicke pinpointed the problem as the lack of a "welcoming and open-minded spirit of doing community work" as well as a lack of direct contact among contributors. Robert Schweikert went even further, critiquing openSUSE's package maintenance model. According to Schweikert, so many people are listed as maintainers for some development projects that "in the end no one really feels responsible."
This discussion is still continuing, with more sub-threads than mentioned
here and with any conclusions still to be made. But, if still short of the
"soul searching" that openSUSE community manager Jos
Poortvliet mentioned in a media alert about Kulow's announcement, the
discussion provides a detailed illustration of some of the difficulties
faced by a growing project. It also highlights a number of plausible partial solutions to openSUSE's specific problems.
What's next
Asked to summarize the situation, Kulow replied:
It's not as dramatic as it may appear, but we do have problems - and we do need to realize that we do have core contributors we can't easily replace in case they have no time. So possibly we need to come up with a more flexible development process or we stick to the development process, but relax the release schedule. This is something to be discussed in the project - and so far the discussion is constructive.
On his blog, Jos Poortvliet was more specific. In trying to summarize the discussion to date, he suggested that either a
longer release cycle, or simply a determination to release only when ready, was
probable. He also foresees an increased role for the Tumbleweed repository and
the creation of staging projects in one form or the other. He also urged fewer
maintainers, each with greater responsibility for their packages as well as
changes in procedure and a need for new hardware to implement any improved
development model.
Whatever reforms are finally undertaken, openSUSE may receive criticism for
disorganization, or for not seeing the problems before the release schedule
collapsed. Still, it takes some courage and honesty to admit that the
present system is no longer working and to abandon the release
schedule. But, in the long run, the implementation of reforms in the
development model seems likely to be of more benefit to openSUSE than
keeping to its original schedule.
Comments (5 posted)
Brief items
So you come back from two weeks of vacation, and you find that there are
about 2000 non-spam mails in your inbox. And you hate yourself because you
have to work through them. But then you notice that 20% of those are a
single fedora-devel flamewar you don't care for, and you happily mark them
all and delete them, and you are happy and relieved, because you worked
through 400 mails within a second. So, should you be sad that fedora-devel
exists and spams you? Or be happy about the fact that it provides you with
a reliable source for such a feeling of quick accomplishment?
--
Lennart Poettering
We apologize for any inconvenience while we’re fixing the community :-) .
--
Charles-H. Schulz
on the formation of the Mandriva Foundation
Comments (none posted)
The Fedora ARM team has announced the availability of the Fedora 17 GA
release for ARM. "
The GA release includes prebuilt images for
Versatile Express (QEMU), Trimslice, Beagleboard xM, Pandaboard, Kirkwood
Plugs, Highbank and iMX based hardware platforms."
Full Story (comments: 7)
openSUSE release manager Stephan "Coolo" Kulow has put out a call for discussion of a new development model for the distribution at least partly in reaction to the delay in various milestones for 12.2.
"
openSUSE has grown. We have many interdependent packages in Factory. The
problems are usually not in the packages touched, so the package updates
work. What's often missing though is the work to fix the other packages
that rely on the updated package. We need to do a better job making sure
bugs caused by updates of "random packages" generate a working system.
Very fortunately we have an increasing number of contributors that
update versions or fix bugs in packages, but lately, the end result has
been getting worse, not better. And IMO it's because we can’t keep up in
the current model." Jos Poortvliet has
added his thoughts as well.
Full Story (comments: 63)
Distribution News
Fedora
The runoff election for the remaining Fedora Board seat has ended.
Nick Bebout is the lucky winner.
Full Story (comments: none)
Newsletters and articles of interest
Comments (none posted)
The H
examines the "Rascal Micro," a tiny ARM board running OpenEmbedded Linux that is designed to be hardware compatible with Arduino shields, while offering a more complete computing environment. "
The Rascal Micro offers on-board programming in Python via a built-in, web-based editor. The Pytronics Python library serves for implementing the actual sensor queries and for communicating with the shields. Data is visualised on the web interface via jQuery and jQplot." The list of diminutive, hobbyist-style Linux PCs continues to grow.
Comments (7 posted)
Page editor: Rebecca Sobol
Development
June 15, 2012
This article was contributed by Dave Phillips
Whether for a word processor, an image editor, or a Web browser,
plugins have become a common characteristic of modern application software
design. An audio plugin functions generally in the same manner as any other
kind of plugin: it extends the capabilities of the host application in
some way. For example, a DAW (digital audio workstation) may have its
own reverberation effect, but if the workstation supports plugins I can
substitute a 3rd-party reverb plugin with preferable features and sound
quality.
Plugins for audio purposes face performance constraints not commonly
encountered by plugins for non-audio applications. An audio plugin is
typically expected to perform some process on an incoming data stream,
possibly in realtime, and the processing should take place with minimal
disruption to the timing and audio quality of the affected stream. Latency
becomes a critical issue if the process takes more time than the data
stream permits, at which point mechanisms for latency compensation may be
invoked. Many control processes can take place in realtime with no
undesired effects while others require considerable care in their
deployment. Some parameters of a plugin may be adjusted in realtime without
concern, while others may involve considerably larger data and/or longer
processing, thus raising the likelihood of dropouts and other forms of
audio corruption. Programmers must resolve these and other problems if they
expect their plugins to be acceptable in performance-critical cases.
Linux audio enthusiasts can choose native plugins from a variety of
acronyms, of which DSSI, LADSPA, LV2, and VST are the best known and
developed formats. In addition to these Linux formats, limited support is
available for non-native VST plugins. In this article I'll present some of
the technical details of these formats along with some of their outstanding
examples, beginning with LADSPA.
LADSPA - Keeping It Simple
LADSPA is an acronym for the Linux
Audio Developers Simple Plugin API. As its name implies, the format was
designed with simplicity as a first consideration. The LADSPA API is easy
to understand and implement (see ladspa.h for the
details), and its practical design led to rapid adoption throughout the
Linux audio development community. The plugin format is best employed for
adding special effects such as distortion, echo, or reverb; for dynamics
processing (compression, limiting, normalization); and for spectral
processing with filters or equalizers. Thanks to its uncomplicated API more
than two hundred LADSPA plugins are currently available, and compatible
host applications now include DAWs, soundfile editors, dedicated processor
racks, and video editing suites.
The format's self-imposed austerity does have its drawbacks. Instrument
plugins (those which, like ordinary instruments, create sounds rather than
modifying them) are not well-supported, though some plugins provide
"building-blocks" for audio synthesis. The API's most conspicuous
limitation is its non-support for any GUI toolkit. The host application
provides the GUI, so if your DAW is built with Qt your LADSPA plugins will
appear in a Qt-based GUI. Depending on your point of view, this particular
limitation is a blessing or a curse, but it is difficult to argue against
the logical decision of the original developers. By keeping the GUI out of
the framework, the developer's focus remains on the plugin's processing
capabilities. Users may not get pretty pictures with LADSPA but they do get
some pretty good plugins.
I use only a few plugins with my own recordings, and I tend to use
plugins I know well enough to predict their response to the material, most
of which is either straightforward song composition or
not-so-straightforward electroacoustic experimentation. The songs get the
plugins, while experimental music typically includes its own processing. For
the sound quality of my voice and guitar I like the Versatile Plate Reverb
from Tim Goetze's C* Audio
Processing Suite. I keep a basic setting for it, but since the effect
has only a few parameters I tune it a little differently every time I use
it. My guitar typically gets treated further by Steve Harris's DJ EQ, a
3-band equalizer I find handy for quick fixes to my cheap guitar's less
than lovely tone. Last but far from least, Steve's SC compressor/limiter
plugins are also commonly employed as final dynamics processors in my
master track.
The rest of this article deals with plugin formats that have evolved
since LADSPA, but I assure readers that LADSPA itself is alive and
well. The Guitarix project
has formatted some of its processors as LADSPA plugins, and the developers
at Harrison Consoles thought that the format's capabilities were sufficient
for some of the internals of Mixbus,
their enhanced version of Ardour2. Harrison programmers have also developed
excellent GUIs for some LADSPA plugins (as seen on the right), prompting me to hope
that Harrison might consider developing more such plugins for general
distribution.
LV2 - The Heir Apparent
After LADSPA's initial success developers started to consider the next
stage in the evolution of high-performance audio plugins for
Linux. LADSPA's limitations were too severe to allow the required expansion
of capabilities, so work began on new formats that would permit such
amenities as embedded graphic envelope editors and synchronized clock
control for delay effects. The LV2 format
has been designed to supersede LADSPA, to address the earlier format's
limitations and to provide new capabilities to plugin developers. The full
specification can be read at the LV2
Trac page.
LV2's chief developer Dave Robillard recently announced that the format
has attained 1.0.0 release status. Along with this news comes Dave's LV2 ports of the MDA
plugins by Paul Keller, a set of thirty-six plugins that includes a
variety of instruments and audio signal processors. Developer Nick Bailey
has released test versions of his Minaton and Triceratops
synthesizers, and developer falkTX recently announced the availability of
his DISTRHO project, a
collection of LV2 versions of instruments and processors from other
projects, most of which were originally developed for Windows audio plugin
hosts. These additions expand a collection that already includes fine work
from Steve Harris, Tom Szilagyi, the CALF developers, and the Invada group. It's worth noting
that many of these programmers are also known for their involvement with
LADSPA-based projects.
Tom Szilagyi's IR
is a fine example of an advanced LV2 plugin with outstanding
audio capability and rich display features. IR processes an input signal
with data from an impulse response file, i.e. a kind of soundfile snapshot
of the reverberant characteristics of an acoustically interesting
environment. This processing method is called convolution, and it is an
effective way to achieve high-quality reverb effects without purchasing
expensive hardware and airline tickets to the great cathedrals of
Europe. Many free and non-free collections of response files are available
on-line, or you can create your own with only an interesting acoustic space
and some software
from Fons Andriaensen, a true DSP wizard of the Linux audio community.
Special mention must be made of the plugins from linuxDSP. These plugins are among the
best available for Linux - they aren't free, and they're not open-source,
but they are reasonably priced, multiformat, professional-grade audio
plugins. At this time all linuxDSP plugins are signal processors - EQ,
compressor, reverb, etc. - but the developer has hinted at the production
of a linuxDSP synthesizer plugin. Given the quality of his DSP work I'm
eager to hear what he can do with an LV2 synth plugin.
As I put the final touches to this article I learned of new sets of LV2
plugins from developers Joe Button and Rui Capela. Interest in the format
is clearly growing since the announcement of its milestone release, and I
look forward to seeing more work from more Linux plugin developers. Indeed,
LV2 appears to be on track to become the standard for Linux audio
plugins.
DSSI - A LADSPA For Instruments
DSSI - the Disposable
SoftSynth Interface - is a project originally headed by developers Chris
Cannam (one of the original and current developers of the Rosegarden DAW), Sean Bolton,
and Steve Harris. Like LV2, DSSI addresses a number of shortfalls in the
LADSPA format, particularly regarding instrument plugins. In fact, DSSI is
essentially a wrapper for a LADSPA plugin with the addition of an OSC (Open Sound Control) interface for toolkit-agnostic GUI and
MIDI control, features notably missing in LADSPA. This approach
reflects the author's desire to create a "LADSPA for instruments" that
retains the simplicity of the earlier format while providing new mechanisms
for features not included in the LADSPA API. The OSC protocol is also an
easily implemented method of establishing communications between programs
or parts of programs (as it's used in DSSI). Interested developers can
peruse the complete DSSI
specification for more information and full details on implementing the
format in their own applications.
Notable DSSI plugins include Sean Bolton's Hexter and xsynth
synthesizers for FM and analog sounds, the Fluidsynth-DSSI soundfont
synthesizer, and Nick Dowell's amSynth. The neko* synths,
WhySynth, and the HOLAP suite are
also worthy entries to the list.
Jeff Hubbard's LibModSynth project
demonstrates that DSSI lives and continues to evolve. The project has
already given us a development library (libmodsynth), a synthesizer (Ray-V,
seen to the right), a variety of effects processors, and a sampler under
construction. Jeff is dedicating his efforts to expand awareness of DSSI's
potential, and his contributions are eloquent proofs. I hope he continues
to expand the LibModSynth suite and that other Linux plugin developers will
consider adding their efforts to the cause.
VST/VSTi - The Steinberg Standard
In 1996 the programmers at Steinberg GmbH developed the VST plugin
format for use with the Cubase DAW for Windows. The company released a
development kit with agreeable licensing conditions that fostered the rapid
growth of an industry of 3rd-party VST plugin designers. Other host
manufacturers quickly supported the format - the host/plugin architecture
was a win for everyone - and a wave of VST plugins hit the market. The VSTi
specification extended the format's capabilities to include virtual
instruments, another win for everyone. VST/VSTi plugins now number in the
thousands, ranging in cost from free to expensive, and varying likewise in
quality and utility.
Musicians who want to migrate from Windows often inquire about the
feasibility of running their VST plugins under Linux. Limited support does
exist for native Windows plugins, but the extent of that support varies
dramatically. At least three methods currently exist for running Windows
VST/VSTi plugins under Linux - the fst software, the dssi-vst-wrapper
bridge utility, and the use of a Windows DAW such as Reaper running under
Wine. However, in my opinion support for native Windows VST plugins should
be considered experimental. Some plugins will work with any of these
methods, others with only one, some will run very well, others not at
all. The version of Wine can have significant impact on performance, and
all Linux VST support mechanisms depend on Wine.
Steinberg's terms are non-problematic for Windows programmers, but they
are not amenable to FLOSS developers. The license forbids the free
redistribution of the API, and there is no indication from Steinberg that
they will ever alter their policies. Despite the format's problematic legal
aspects the demand remains and designers must respond. Fortunately, thanks
to some clever reverse-engineering from Javier Serrano Polo's
VeSTige project, Ardour3 and QTractor can
be compiled with support for native Windows VSTs, and LMMS includes a module that
provides a VeSTige-based VST-to-Linux bridge. FeSTige is a convenient VST/VSTi
plugin launcher that will eventually be replaced by its evolving offspring
Carla, a multiformat launcher from developer falkTX. Clearly, users
want to be able to deploy their VST plugins under Linux. Some developers
will avoid the issue by refusing to support a non-native format, while
others see and pursue a unique opportunity.
Some VST/VSTi plugins can be compiled as native Linux shared object
files (e.g. fooplug.so) that require no dependency on Wine. Commercial
plugin developers have been quick to adopt the format - the current catalog
of Linux VST plugins includes excellent instruments and processors from Loomer Software (pictured, right), discoDSP, and linuxDSP. Some
excellent MIDI plugins can be found in the pizmidi and Mucoder collections, both of which
are freeware, not open-source free software. The FLOSS audio development
community has been less productive with the format, though notable work has
been done by the ccern and
the Jucetice projects. It
remains to be seen whether the catalog continues to expand, and that
depends most of all upon the quality of the processors and instruments
created for it.
Plug In The Future
The stabilization of LV2, the renewed interest in DSSI, and the
continuing utility of LADSPA point to a positive outlook for audio plugins
for Linux. Commercial interest is likely to maintain a focus on the VST
format - it does own the largest market share, and native Linux support
might open a door for enterprising Windows VST developers. New
cross-platform development tools promise easy output to multiple targets
and new hosts are appearing on the Linux applications stack. From where I
stand the future of Linux audio is sounding better all the time.
Comments (16 posted)
Brief items
Documentation is the sort of thing that will
never be great unless someone from outside
contributes it (since the developers can never
remember which parts are hard to understand).
—
Avery Pennarun
I have an interesting problem: How do I shoehorn "hired by The
Internet for a full year to work on Free Software" into my resume?
—
Joey Hess
Comments (12 posted)
Version 2.6.0 of the photo editing and organizing suite digiKam has been
released. The release includes a new maintenance tool for automating tasks, and support for new Kipi plugins.
Comments (none posted)
The PacketFence team has released version 3.4.0. "
This is a major release with new features and
important bug fixes. This release is considered ready for production use
and contains a security fix so upgrading to 3.4.0 is advised."
Full Story (comments: none)
The SystemTap project has announced the release of version 1.8. "
There is updated support for user-space probing against kernels >=
3.5, which have no utrace but do have the newer inode-uprobes work
by Srikar Dronamraju and colleagues." The release also includes improved IPv6 support and multiple concurrent connections.
Full Story (comments: none)
Version 2.0.0 of the ulogd userspace logging daemon for netfilter has been released. This release "
requires a Linux kernel >= 2.6.14, but Linux kernel >= 2.6.18 is strongly recommended. Note that if you need SQL database output [support], you will need the header files of the respective libraries."
Full Story (comments: none)
Newsletters and articles
Comments (none posted)
Libre Graphics World has an extensive
examination of Linux support for Firewire audio interfaces often used in professional sound work. The story includes interviews with developers from the Free FireWire Audio Drivers (FFADO) and ALSA projects. "
The ease at which FFADO can shift its streaming subsystem into the kernel comes about because of the way that device streaming and control has been carefully separated from the very beginning. This means that we can remove the streaming parts without affecting device control in any way. In fact, even now the device control part of FFADO would happily run on a system with a hypothetical kernel streaming driver."
Comments (none posted)
Andy Oram has written a thorough write-up of the 2012 Health Privacy Summit at O'Reilly Radar. "We can fix our health care systems if we educate doctors and patients to work together; create teams that have incentives to deliver the best care; open up data about the health care industry; incorporate low-cost devices into patient-centered medical homes, and incorporate the best research into clinical decision support. I'm sure readers could suggest other related elements of a solution. A crucial background role will be played by technological improvements and standards." Readers interested in healthcare IT will also want to follow the links to Oram's recaps of the co-located Patient Access Summit and Health Data Initiative forum.
Comments (none posted)
Page editor: Nathan Willis
Announcements
Brief items
The Electronic Frontier Foundation has announced a new patent reform
project called "Defend Innovation". "
EFF has posted seven proposals for software patent reform
at Defendinnovation.org, including shortening the term for
software patents from 20 years to no more than five years,
allowing winning parties in litigation to recover fees and
costs, and ensuring that infringers who arrive at a
patented idea independently aren't held liable, for
example. EFF is asking the public to sign on to the
proposals and to make additional comments of their own.
Additionally, we're calling on individual inventors,
lawyers, and academics to give feedback, and we're asking
technology companies to continue the conversation with
in-person meetings with EFF staff. EFF will take the
results from these comments and meetings and create a
whitepaper to help educate lawmakers and others about the
full reach of the problem and next steps forward to fix it."
Full Story (comments: 29)
The Free Software Foundation Europe (FSFE) has started an initiative to
advance fair public procurements in Finland. "
The initiative concentrates
on IT related procurement notices that require brand instead of defining
functionalities required by the procurer. To date FSFE has skimmed over
300 procurement notices, and of those taken into closer analysis, 14
have been found to clearly violate the Finnish procurement law. These
violating notices explicitly asked for tenders of specific brands of
software manufacturers or products and thus discriminate all other
brands and manufacturers, effectively stopping free competition."
Full Story (comments: none)
Linux.com has
the videos
from LinuxCon Japan, which was held earlier this month. (Thanks to Shawn Landden)
Comments (7 posted)
The Linux Foundation has announced 5 new Silver members: Adeneo, Feuerlabs, Omnibond, STEC and Synopsys.
Full Story (comments: none)
Articles of interest
On her blog, Máirín Duffy has written about four separate talks from the
USENIX Women in Advanced Computing 2012 Summit. Included are talks from
Leslie Lambert, Vice President & Chief Information Security Officer, Juniper Networks;
Emily Gladstone Cole, Operations Architect, Cisco Systems, Inc.;
Clea Zolotow, Senior Technical Staff Member, IBM; and
Sabrina Farmer, Site Reliabity Engineer, Google Inc. The write-ups are quite thorough and interesting. From Farmer: "
In Gmail, if something goes wrong, it’s in the news. You’re responsible for hundreds of millions of people’s email on that team. Her first press interview for Gmail was terrifying, but amazing. The next reporter asked for her by name. Having an outage in the NY Times would be really bad – it’s a very high risk position. You can’t, however, have success without failure – you won’t grow if you don’t fail along the way. It’s how you handle that failure that matters. The worst thing that can happen when you fail is probably not as bad as you think."
Comments (5 posted)
New Books
Lars Wirzenius has released his ebook
Getting Things Done For Hackers under the CC-BY-SA
3.0 license. "
David Allen's "Getting Things Done", or GTD for short, is a popular,
powerful system for managing one's life. If you have trouble dealing
with your e-mail inbox, or feel you're drowning under a flood of inputs
and information, or just don't seem to have time to do everything you
think you should be doing, or others want you to do, then GTD may be a
good thing for you to consider.
This book explains how I, a computer geek, have implemented it in my own
life. It is aimed at everyone whose lives include a lot of computer use,
and who know how to use their computers."
Full Story (comments: none)
O'Reilly Media has released "YUI 3 Cookbook" by Evan Goer.
Full Story (comments: none)
Education and Certification
The folks at Free Electrons have
announced the release of their Android training materials under the Creative Commons Attribution – Share Alike 3.0 license. "
Our course targets engineers who need to develop embedded systems with Google Android, customizing Android to support specific hardware and product requirements." See
the course agenda [PDF] for details on the covered material.
Comments (none posted)
The Linux Professional Institute (LPI) has announced the completion of the
beta test for its new Linux Essentials exam. "
The Linux Essentials exam will be available at the end of June 2012 in
Europe, the Middle East and Africa and in other nations around the world
in 2013."
Full Story (comments: none)
Calls for Presentations
FOSS.IN is back by popular demand. FOSS.IN is one of India's oldest and
best-known annual international technology events. Although there was no
2011 conference, it will be back for 2012 in Bangalore, India, November
29-December 1. The call for participation is currently open till July 8th.
Full Story (comments: 1)
The openSUSE Summit takes place September 21-23, 2012 in Orlando, Florida.
The call for papers
deadline has been extended until June 30.
Comments (none posted)
Upcoming Events
PyCon Australia will take place August 18-19, 2012 in Hobart, Tasmania.
The program has been announced. "
Our keynote presenters, Mark Ramm,
Engineering Manager on Juju at Canonical, and Kenneth Reitz, Python lead at
Heroku will be joined by a wide array of presenters covering a broad range
of backgrounds, including industry, research, government and academia."
Full Story (comments: none)
Events: June 21, 2012 to August 20, 2012
The following event listing is taken from the
LWN.net Calendar.
| Date(s) | Event | Location |
June 19 June 21 |
Solutions Linux Open Source |
Paris, France |
June 20 June 21 |
Open Source Summit (NASA, State Dept, VA) |
College Park, MD, USA |
June 26 June 29 |
Open Source Bridge: The conference for open source citizens |
Portland, Oregon, USA |
June 26 July 2 |
GNOME & Mono Festival of Love 2012 |
Boston, MA, USA |
June 30 July 1 |
Quack And Hack 2012 |
Paoli, PA, USA |
June 30 July 6 |
Akademy (KDE conference) 2012 |
Tallinn, Estonia |
July 1 July 7 |
DebConf 2012 |
Managua, Nicaragua |
July 2 July 8 |
EuroPython 2012 |
Florence, Italy |
| July 5 |
London Lua user group |
London, UK |
July 6 July 8 |
3. Braunschweiger Atari & Amiga Meeting |
Braunschweig, Germany |
July 7 July 8 |
10th European Tcl/Tk User Meeting |
Munich, Germany |
July 7 July 12 |
Libre Software Meeting / Rencontres Mondiales du Logiciel Libre |
Geneva, Switzerland |
July 8 July 14 |
DebConf12 |
Managua, Nicaragua |
July 9 July 11 |
GNU Tools Cauldron 2012 |
Prague, Czech Republic |
July 10 July 11 |
AdaCamp Washington, DC |
Washington, DC, USA |
July 10 July 15 |
Wikimania |
Washington, DC, USA |
| July 11 |
PuppetCamp Geneva @RMLL/LSM |
Geneva, Switzerland |
July 11 July 13 |
Linux Symposium |
Ottawa, Canada |
July 14 July 15 |
Community Leadership Summit 2012 |
Portland, OR, USA |
July 16 July 20 |
OSCON |
Portland, OR, USA |
July 26 July 29 |
GNOME Users And Developers European Conference |
A Coruña, Spain |
August 3 August 4 |
Texas Linux Fest |
San Antonio, TX, USA |
August 8 August 10 |
21st USENIX Security Symposium |
Bellevue, WA, USA |
August 18 August 19 |
PyCon Australia 2012 |
Hobart, Tasmania |
If your event does not appear here, please
tell us about it.
Page editor: Rebecca Sobol