LWN Weekly Edition
Leading itemsSecurity
Kernel development
Distributions
Development
Announcements
->Multipage format
This page
Previous weekFollowing week
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
LWN.net Weekly Edition for September 17, 2009A new open source foundationA foundation to promote increased participation by commercial software companies in free software projects would normally be seen as a good thing. That the foundation begins with a budget of $1 million would also seem to bode well for its success, at least in the near term. But when that foundation is sponsored—and funded—by Microsoft, it is met with a healthy dose of skepticism, which is exactly what greeted the recent announcement of the formation of the CodePlex Foundation. While the new foundation is not strictly related to CodePlex.com—Microsoft's SourceForge-like repository for open source code—they share the name from the "CodePlex brand", as well as supporting the "same mission", according the foundation's FAQ. Like CodePlex.com, the CodePlex Foundation is geared towards commercial software developers who are, evidently, not being served by existing organizations. Though it is far from clear what is missing from the existing sites and organizations, the FAQ makes it clear that CodePlex intends to fill that hole:
CodePlex.com launched in June of 2006 out of a need for a project hosting
site that operated in a way that other forges didn't – with features and
structures that appealed to commercial software developers. The next
chapter in solving for this challenge is the CodePlex Foundation
(Codeplex.org). The Foundation is solving similar challenges; ultimately
aiming to bring open source and commercial software developers together in
a place where they can collaborate. This is absolutely independent from the
project hosting site, but it is essentially trying to support the same
mission. It is just solving a different part of the challenge, a part that
Codeplex.com isn't designed to solve.
One would think that existing foundations—for GNOME, Linux, Mozilla, Apache, et al.—might also be interested in bringing commercial and free software developers together. In fact, those foundations have been fairly successful in doing so. There may be value in having a more all-encompassing foundation, which doesn't serve a particular sub-genre of free software, but it is a bit hard to see the CodePlex Foundation filling that role. To start with, the foundation's board will be Microsoft-dominated, with three current Microsoft employees on the six-member board. In addition, Sam Ramji, up until recently the leader of Microsoft's open source efforts, will serve as interim president of the foundation and chairman of the board. Ramji recently announced that he is leaving Microsoft for a cloud computing startup in Silicon Valley, but, as a former employee, his detachment from the Microsoft view is unclear. The two outside members are Mono/Moonlight developer Miguel de Icaza of Novell and DotNetNuke developer Shaun Bruce Walker, both of whom have fairly close ties to Microsoft. At some level, the make up of the board of directors is unsurprising, but it is rather puzzling that the board of advisors would be similarly stacked with Microsoft employees. Six of the twelve members of the advisory board (listed to the right of the board on its web page linked above) are employed by Microsoft. There are some seemingly independent voices on the advisory board, including Larry Augustin of SugarCRM and MySQL developer Monty Widenius of Monty Program AB. Since the advisory board is just meant to advise the board of directors, who will make any decisions, also packing the advisory board certainly gives the appearance that the foundation will just be a Microsoft mouthpiece. In a lengthy blog posting, Andy Updegrove analyzed the by-laws [PDF] and make up of the foundation. Updegrove is a lawyer known for his reporting on the ODF and OOXML standards process—and its apparent subversion by Microsoft—and is the director of standards strategy for the Linux Foundation, so he is hardly a disinterested observer. He notes, though, that he has helped set up many "consensus based consortia and foundations" over the years, which gives him a good basis to evaluate the CodePlex Foundation. There are several areas that Updegrove notes as problematic in the foundation as it stands. In order for an organization to be seen as neutral, it's best to have multiple partners or members, but the foundation is not set up as a membership organization. The by-laws mandate a new five-member board be elected in 100 days, but, since there are no members who can vote on board seats, the interim board will be doing the electing. Those five new board members will have complete control of the foundation, and will have been elected for as many as four years. In addition, because a 2/3 majority is needed to change many parts of the by-laws (size of the board, text of the contributor agreements, etc.), any two members of the board can effectively block those changes. So, even if the permanent board only has one Microsoft employee—which seems unlikely—and one "friendly" member, the company effectively has a veto over any fundamental changes the board might want to make. The draft Contribution Agreement [PDF] and Copyright Assignment Agreement [PDF] may give a look into what the foundation is trying to achieve. They both grant wide-ranging rights—including licenses to any patents held by the contributor that cover the contributed code—to the foundation and those who get code from it. The introductory slides specifically mention passing those rights to "downstream developers", but are mute when it comes to upstream. That may be an oversight, but it might also be an attempt to isolate the GPL from the CodePlex ecosystem. Exactly why anyone would want to contribute code to the foundation is unspecified. There are sometimes good reasons to do copyright assignment, but the foundation hasn't articulated what benefit developers would get by doing that. As Updegrove puts it:
They're not bad, if what you want to do is convey the right to create code
that can be distributed under any flavor of open source license. But why
would any developer or contributor want to sign such an all purpose
license? Lines of code are contributed to defined projects, not to some
code bank where they can be archived for posterity.
Like many newcomers to free software—along with some established players—Microsoft clearly does not feel comfortable working with existing organizations and communities. It wants to create its own playing field, where it can dominate, and control if necessary. By using contributor agreements and by-laws that are favorable to its interests, potentially at the expense of those who participate, it is creating something that it feels comfortable with. In some ways, it is similar to the control that Sun has, so far, exercised in the OpenSolaris and Java communities. For both Sun and Microsoft, there is a corporate aversion to allowing the community to drive the direction of projects. But, that is one of the hallmarks of successful free software development. Various companies have learned this, over time, so it is possible that Microsoft will as well. Five, or even three, years ago, it would have been unimaginable that it would create a foundation for open source; clearly some progress has been made.
Microsoft's penchant for co-opting others' ideas, then twisting them to their own ends (e.g. "Embrace, Extend, Extinguish") leads many to scrutinize the CodePlex Foundation more carefully than they might one created by a different company. There is a level of distrust within the free software community that can only be erased through fundamental changes to Microsoft's behavior over a prolonged period. It's certainly possible that the foundation is one step on that path, but the conflicting signals it continues to send about free software gives one pause. Updegrove voices just that concern: Unless CodePlex is set up in a
truly neutral fashion, that will lead many people to worry that Microsoft
wants to create and legitimize "their" kind of development environment,
where Microsoft can feel safe launching projects (all of the initial
projects under consideration are Microsoft projects) under IPR
[intellectual property rights] rules, and under licenses, that fit their
view of what open source should be all about.
While Updegrove is rather critical of the current foundation structure, he also has specific suggestions on changes that could be made. Things like increasing the size of the board, and decreasing the number of Microsoft employees on it, as well as taking back 3/4 of the initial funding and replacing it with contributions from other corporate members, are among his suggestions. The idea behind those suggestions is for the foundation to truly be a neutral party, rather than just an arm of Microsoft:
But if there really is a need for individual developers and commercial
vendors to get together in a new organization, then community members will
need to feel like CodePlex is a safe place to be. Right now, I can't see
that happening without some serious rethinking of the entire governance
structure as currently proposed.
Overall, the foundation idea seems half-baked. Its announcement may have been rushed to offset some recent negative publicity—in the form of patents that Microsoft tried to sell to patent trolls—or to blunt the impact of Ramji leaving the fold, but it certainly doesn't have a very polished look. Over time, it may evolve into something useful to the free software community, but only time will tell. That, really, is the crux of the matter. Speculating about Microsoft's motives and plans is certainly reasonable, but we will have to wait and see what the foundation actually does. The make up of the new board, along with any changes to the advisory board, will likely signal where the foundation is headed. How many commercial software developers "join" the foundation and start contributing code, will also be interesting to watch. Based on what we see today, though, it would not be terribly surprising to see the foundation die quietly on the vine a few years down the road.
Tornado and Grand Central Dispatch: a quick lookTwo traditionally proprietary companies made open source releases recently: Facebook released a Python-based web server and application framework called Tornado, and Apple released a thread-pool management system called Grand Central Dispatch. It is not the first open source code release for either company, but both projects are worth examining. Tornado is designed to suit specific types of web applications and is reportedly very fast, while Grand Central Dispatch may cause some developers to re-think task-parallelism. This Tornado serves youTornado is actually a product of FriendFeed, the social-networking-aggregator acquired by Facebook in August. It consists of a web server and surrounding framework (all written in Python), tailored to handle a very large number of established, open connections. The web server component (tornado.web) is "non blocking" — meaning that it is event-driven, designed around the Linux kernel's epoll facility, and can thus maintain large numbers of open TCP sockets without tying up excessive memory and without large numbers of threads. Event-driven Web servers like Tornado are single-threaded; each thread can manage potentially thousands of open connections as long as the application does not block while it waits for data from the socket — the thread simply polls them each in turn. Additional connections can be handled by running multiple server processes on SMP systems. In contrast, traditional web servers are blocked from handling additional connections while they wait for I/O, or must spawn additional threads to handle additional connections at the cost of context-switching and increased memory use. In addition to the web server itself, the Tornado release includes a suite of modules used to build web applications, including XHTML, JSON, and URL decoding, a MySQL database wrapper, a localization and translation module, a Python templating engine, an HTTP client, and an authentication engine. The latter supports third-party schemes such as OAuth and OpenID, plus site-specific schemes used by Facebook, Yahoo, and Twitter. The Tornado code is hosted on GitHub and is available under the Apache 2.0 license. Tornado works with Python 2.5 and 2.6, and requires PycURL and a working JSON library. Documentation is available on tornadoweb.org, and a live demo "chat" application is running on http://chan.friendfeed.com:8888/. FriendFeed's Bret Taylor announced the release on his blog, comparing Tornado to web.py and Google webapp. He claims that in Apache Benchmark tests, Tornado was able to handle four times the number of requests per second (or more) of competing frameworks, including web.py, Django, and CherryPy. Taylor's post, and the subsequent discussion, sparked some controversy among users and developers of the Twisted framework, who objected to disparaging comments about Twisted's code maturity and suitability. Twisted founder Glyph Lefkowitz posted a lengthy response responding to the claims made about Twisted, but, overall, approving of the Tornado release itself. Matt Heitzenroder posted his own head-to-head performance tests that show Tornado beating Twisted.web, but not dramatically. Aside from performance numbers, many in the open source community seemed impressed by what Tornado offers — a simple framework for building "long polling" web applications, including support for everything from templating to cookie management to localization in a single package. Since Tornado has proven itself viable as the framework underlying FriendFeed, it is likely to pick up a significant following as an open source project. Invisible threadsApple's Grand Central Dispatch (GCD) is an operating system-level feature that debuted in the recent release of OS X 10.6 ("Snow Leopard"). GCD is essentially a mechanism to allow application developers to parallelize their code, but let the OS worry about intelligently managing the threads. GCD determines the maximum number of concurrent threads for the system and manages the queues for all running applications. Thus the application developer only needs to write GCD-capable code, and trusts the OS to take optimal advantage of multiple cores and multiple processors. Apple's source code release consists of the Apache-licensed user space API library libdispatch and changes to the XNU kernel, Apple's open source Mach-based kernel common to OS X and Darwin. The XNU changes reportedly improve performance of the event notification interface Kqueue. GCD also relies on a non-standard extension to C, C++, and Objective-C known as "blocks," however, so blocks support in the compiler is a prerequisite for application developers wishing to take advantage of GCD. Blocks are supported for the LLVM compiler through the compiler-rt project. Because GCD abstracts thread creation from the application developer, it is most similar to OpenMP or Intel's Threading Building Blocks (TBB). All three allow the developer to designate portions of code as "tasks" to be parallelized in some fashion. GCD is different in that it leverages a language feature (blocks) rather than the preprocessor directives of OpenMP or templates of TBB. In addition, TBB is limited to C++, though OpenMP is available for C, C++, and Fortran. Blocks are essentially inline-defined, anonymous functions. They are
designated by a caret (^) in place of a function name, take arguments like
any function, and can optionally return a value. Blocks are different in
that they have read-only access to variables from their parent scope (a
feature similar to "closures" in languages such as Ruby). Consequently, in
replacing a From Apple's Concurrency Programming Guide, the following example loop iterates count times:
for (i = 0; i < count; i++) {
printf("%u\n",i);
}
which could be expressed as a block ready for GCD as follows:
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_apply(count, queue, ^(size_t i) {
printf("%u\n",i);
});
When executed, GCD creates count tasks, one for each iteration
of the block, placing them on a task queue. GCD makes a default queue
available through Apple provides a basic introduction to GCD and programming with blocks on its developers' site. In addition, the OS X scientific research community at MacResearch.org has a detailed tutorial complete with GCD examples and the equivalent code written for OpenMP. MacResearch.org has basic performance numbers posted for its tutorial code, and Apple has posted a benchmarking sample that compares GCD against serialized code and native POSIX threads. So far, GCD is only implemented for Mac OS X, but reaction from the developer community has been positive. Having the operating system worry about the details of thread pool management seems like a winning idea; most of the discussion on Mac forums has revolved around the wisdom of relying on a language extension such as blocks. Ars Technica commented on places where Linux could benefit from a native GCD implementation, such as in higher-level frameworks like QtConcurrent, but notes that use of the Apache license limits integration to projects using GPL version 3 and later. ImpactApple and Facebook have a history of making periodic releases of code projects under open source terms, even though both enjoy a reputation for maintaining "walled gardens" around their core products. As is predictable when large proprietary companies release open source code, considerable energy has been expended on the web speculating as to what each company hoped to "gain" from the release. A leading theory for GCD is that Apple hopes to further the adoption of blocks into standard C and C++, but no consensus has yet emerged for why Tornado was released. In fact, neither Tornado nor GCD has made major waves in the open source community, but if the initial reaction is a good indicator, both are solid and valuable products. GCD is the likelier of the two to stir up passionate debate going forward, as fully assimilating it into mainstream Linux would require touching not one but two of the fundamental pillars of the community: the kernel and the compiler. Although LLVM has its fans, the Linux community is still predominantly a GCC ecosystem. Pushing Apple code into the Linux kernel and into GCC won't happen lightly.
FOSS license compliance in the consumer electronics market[ This article is an opinion piece and does not contain legal advice. The authors are not lawyers. ] [ Editor's note: This is part 1 in a series of three. Part 2 looks at compliance engineering and part 3 looks at license compliance for companies. ] Free and Open Source Software (FOSS) license compliance is a contentious topic. There are different perspectives about when and how license terms apply, about which licenses can be used together, and about how potential issues should be resolved. The consumer electronics market is an area where FOSS license compliance is particularly problematic. This is primarily attributable to economic reasons rather than dishonesty, but in a market worth more than $335 Billion in 2008, it is an issue worth exploring. Due to the relative youth of the FOSS ecosystem, there is a lack of case law and best practice information available. In the past, one of the few resources available to the community was Debian Legal, and businesses had little beyond Open Bar (USA) and ifrOSS (EU) to support them. That situation is improving. Organizations like FSF's Free Software Licensing and Compliance Lab, gpl-violations.org, FSFE's Freedom Task Force and Software Freedom Law Center (SFLC) have helped push professional legal and business approaches to the forefront of FOSS discourse. The recent launch of the International Free and Open Source Software Law Review has provided a neutral platform for future discussions. As FOSS has matured so too has the level of information accessible to support businesses and projects. The consumer electronics businessConsumer electronics are sold in high volumes for low margins, and competition in the market is fierce. The majority of sales take place during the first three months after launch and consumers focus on price and functionality when selecting new technology. Products are developed in Asia by original device manufacturers (ODMs) and original equipment manufacturers (OEMs) and shipped in completed form. There are few western companies doing their own development, and even those with in-house skills are unlikely to build a finished product themselves. ODM/OEMs may develop products for competing western companies using a single board to save money. A board design and Software Development Kit (SDK) is provided by an upstream supplier like the chip vendor, the ODM/OEM will add hardware or software functions, and the finished system is placed into customized casings. Purchasing companies can label these variants as their own by adapting control panels, contact information, and documentation. During this process issues can arise regarding license compliance. FOSS originating from a chip vendor may be supplied with incomplete source code to the ODM/OEM. If the source is supplied in complete form it may later be customized by the ODM/OEM and only partially re-integrated into source tree. The marketing team may forget to place licenses or written offers for source code in the product manuals. The list of potential points of failure is lengthy. The fundamental issue is simple. If FOSS code and changes to that code are not integrated into source releases, or if other terms of popular licenses are not met, then compliance issues can occur. This problem is compounded when one board with a problem appears in devices supplied to a number of western companies. A host of violation reports spanning a dozen European and American businesses may eventually point towards a single mistake during development at an Asian supplier. Why violations occurThere are many types of FOSS compliance issues. The specific issues depend on the license being used, but may include people forgetting to add a copy of the license text to products, forgetting to include the source code with shipped binaries, or having no policy to handle source code requests after providing a written offer promising this service. There is often a disconnect between support, website maintenance, and legal departments, so even correctly prepared material gets lost in the shuffle. At first glance it can appear daunting to perform due diligence. However, FOSS compliance is not inherently more complex than proprietary compliance, and compliance in general is not so difficult as to be excusably ignored. There is even a field called compliance engineering where external specialists or in-house staff check that code shipped in products meets the required license terms. The problem for the consumer electronics market is that compliance engineering is perceived to endanger profit. There are two reasons for this. The first reason is that market timing is extremely important, and a delay reaching consumers could mean being beaten by the competition. Compliance engineering with any reasonable fidelity will take a few days, and when companies will only have one or two test samples of the product available for checking functionality, it's hard to find a way to schedule time for compliance checking. Furthermore, any questions raised will have to be answered by the supplier and potentially other parties in the supply chain. Any missing source code will have to be located and integrated in the SDK. If there is missing code or a supplier in the chain who simply won't release required code (and this happens more than you might imagine), then it is possible that a device will face months of delays. The second reason is that the cost of compliance engineering may drive a product out of profitability. A transaction cost of €1,200 for checking one device is reasonable given the current market rates, and this sum is a lot of money in the consumer electronics market. The initial release of a product is often a test run to check demand, and may consist of as few as 200 devices being made available to the public. A compliance check at this stage would raise the price of the product by €6, and while justified by law - license compliance is not based on quantity shipped - it may be difficult from an economic perspective. Even after the test run is complete and additional orders are made, if the company plans to ship 10,000 or fewer devices the cost per unit is still at least 12 cents. [PULL QUOTE: Because of these two pressures the companies involved often don't spend too much time trying to understand FOSS licensing or putting the infrastructure in place to ensure compliance. They may see themselves as facing a choice of shipping non-compliant software and risking a court case or facing a market loss from missed sales. END QUOTE]Because of these two pressures the companies involved often don't spend too much time trying to understand FOSS licensing or putting the infrastructure in place to ensure compliance. They may see themselves as facing a choice of shipping non-compliant software and risking a court case or facing a market loss from missed sales. With court cases relatively rare in FOSS today, risking a legal challenge may appear to be a less painful option than the alternative. Whether this perception will continue is debatable. Gpl-violations.org has made what appear to be permanent changes to how businesses approach FOSS in Europe since 2004, and SFLC have started to become pro-active in seeking compliance for projects in the USA. Community tolerance for negligent behavior by commercial entities is waning. This market adjustment is predictable given the status of FOSS technology. The European Commission estimated that the ecosystem of FOSS applications with reasonable quality control and distribution in 2007 was worth around €12 billion. The cost of obtaining this code is adherence to the license terms, and with rising value creators are naturally less tolerant of misuse then they may have been when FOSS was still in its infancy. What developers can do to protect their rightsDevelopers who own the copyright on code have various ways to ensure people obey the licenses. If you are not a copyright holder on code but have found clear evidence of a violation it is a good idea to tell the copyright holders. Ensuring fair play with using the licenses helps maintain confidence in the FOSS ecosystem. It means people can make a decision about how their code will be used and be reasonably sure everyone will stick to the terms. Perhaps the most important thing when assessing violations is to get the facts right. SFLC's Legal issues primer for Open Source and Free Software projects can assist with this, as can its Practical guide to GPL compliance. The second most important thing is to be fair and professional. Emotion or lack of understanding won't help correct a problem and it certainly won't help foster a potentially useful working relationship. If you are pretty sure a violation has taken place you can decide what route to take regarding enforcement (if you are a copyright holder) or informing the code owners (if you are not a copyright holder). The first step for everyone is probably to document everything carefully. FSFE and gpl-violations.org published a brief document on reporting and fixing license violations that explains some of the key points that you need to cover. The suggestion is that you should make a report with:
This information can then be used by you, the affected project, your lawyers, the infringing company, or a third party like gpl-violations.org, FSFE, FSF or SFLC, to examine the situation as applicable. Avoid doing things like forwarding email threads or inserting commentary as this makes it difficult to assess the situation. For copyright holders there is an established formal mechanism to enforce copyright through legal action. This can be done by taking an infringing party to court or by seeking an out-of-court settlement. There is no doubt this approach is effective, as members of the gpl-violations.org project can attest, though it can also be costly in time and initial fees. Other avenues for correcting misuse of licenses also exist and may be quicker in some circumstances. For example, informal discussions can work with accidental infringement, and mediation by FSFE's Freedom Task Force or FSF's Free Software Licensing and Compliance Lab has also proven to be effective in the past. When it comes to legal advice, independent professionals like Carlo Piana provide excellent advice for both developers and companies with concerns. Gaining compliance is most often an educational exercise. FOSS has a lot of a new adopters and many of the commercial entities using code in the consumer electronic market come from a proprietary background. That's no excuse for not understanding the licenses, but it is a strong case for considering how these companies can be turned into good community citizens. Productive compliance efforts should use carrots and/or sticks to encourage people to communicate and cooperate with the code creators, projects, and other businesses in this area. Punishment is not the name of the game. Working together in good faith is. About the authorsArmijn Hemel is a technology consultant with Loohuis Consulting in The Netherlands and the primary engineer for the gpl-violations.org project. Shane Coughlan is a business and technology consultant with Opendawn in Japan. He is an expert in Free/Open Source Software licensing, standardization, communication methods and business development
Security All the malware that's fit to printSome readers of the New York Times (NYT) web site were recently surprised to "learn" that their computers were infected with viruses. As it turns out, a rogue ad was responsible for the warning, and, as one would guess, anyone who downloaded the suggested fix for the virus problems was, instead, infected with malware. While the problem was fairly short-lived—and targeted Windows, not Linux or Mac OS X—it does point to a general problem for those who run web sites: how can one ensure that the ads running on the site don't contain anything objectionable, either because of the actual ad content, or because it contains malware? Ad content is typically served by ad networks, and a web site operator includes a little blob of Javascript into the proper place in a web page. That Javascript is responsible for retrieving the ad content and adding it into the page. But there is nothing stopping it from doing other things, such as downloading Javascript from other sites. Because the script code was served with the page, it has all the rights that any other Javascript has in the context of that page. Essentially, the site owner has given their ad network a "free pass" to do whatever is needed to put up the ad. In general, ad networks are careful to screen the ads they send to their partners—at least for malicious content—otherwise, those partners would switch to a different network. But, it is certainly possible, and has probably happened in the past, that a dodgy ad gets put into an ad network's rotation. That was the first guess for where the NYT problem was. But, as the paper itself reported, the ad actually came from elsewhere. In addition to running ads from ad networks, web sites often directly sell ads to customers. In this case, the NYT believed it was selling an ad to VoIP provider Vonage. When the ads were placed, they at first displayed normal Vonage ads. At some point, though, whoever placed the ads (and provided the Javascript to the NYT) switched to serving virus warnings. Obviously, in retrospect, the NYT should have been more careful to ensure that whoever they were dealing with was, in fact, representing Vonage. The ad content was not being served by vonage.com, but that's hardly surprising as many advertisers use other sites to serve their ads. Vetting advertisers can be rather difficult, though. There are multiple levels of both technical and administrative verification that need to be done, some of which is likely beyond the abilities of ad salespeople. It is, in some ways, like the kind of vetting that needs to be—and often isn't—done for SSL certificates. There needs to be a real organization behind the ad, though what constitutes "real" is an open question. The code to be inserted needs to be inspected as well. An excellent dissection of the NYT malware gives a good view of just how the attack worked. Without somehow figuring out that tradenton.com was not a legitimate ad serving network, there is nothing particularly suspicious about the top-level code. This is a problem we are likely to see more of over time. Because the ad networks want to be able to run code on the client, for geotargeting and other information gathering, sites must generally be willing to insert fairly opaque Javascript into their site. As the dissection shows, that can lead to bouncing around to multiple sites, grabbing code from each—even legitimate ad serving networks often have their own partners to whom the redirect requests. There is a sort of implicit web of trust that exists, but one that has the potential to be subverted. Another aspect of the problem is that site owners often cannot see all of the ads that are currently being displayed on their site. If some small percentage of the ads—or those targeted at a different region—contain objectionable content of any sort, the site owner may very well be completely unaware of it until users complain. It's not just malware ads that are a problem, here, but any kind of ad that the owner might prefer not to run. The NYT article mentions other similar incidents that have occurred in the past, but this attack, on a high-profile site, has, at least, served to raise the profile of the problem. Other than eliminating ad networks and customer-supplied Javascript from a site, there is very little defense against this type of subversion. By running other people's code in a site, one has, for all intents and purposes, turned over control of the site's content to third parties. It shouldn't be too surprising that attackers are taking advantage of that.
New vulnerabilities firefox: web content processing vulnerabilities
firefox: use-after-free flaw
firefox: URL concealment
firefox: JavaScript execution
firefox: multiple vulnerabilities
firefox: certificate vulnerability
freeradius: denial of service
horde: cross-site scripting
htmldoc: buffer overflow
kde: man-in-the-middle attack
kernel: missing capability check
libsamplerate: denial of service
nginx: arbitrary code execution
planet: missing input sanitizing
puppet: multiple vulnerabilities
rails: missing input sanitizing
silc-toolkit: format string vulnerabilities
wireshark: multiple vulnerabilities
xapian-omega: missing input sanitising
znc: arbitrary file overwrite
Page editor: Jake Edge Kernel development Brief items Kernel release statusThe 2.6.32 merge window is open, so there is no current development kernel release. The usual vast pile of patches has been merged; see the article below for a summary.The current stable kernel is 2.6.31; no stable updates have yet been released for this kernel. For older kernels, 2.6.27.34 and 2.6.30.7 were released on September 15. Both contain a handful of important fixes.
Quotes of the week
No need anymore to write some printk to debug, worrying,
sweating, feeling guilty because we know we'll need yet another
printk() after the reboot, and we even already know where while
it is compiling.
-- Frederic Weisbecker
We would build less kernels, then drink less coffee, becoming less nervous, more friendly. Everyone will offer flowers in the street, the icebergs will grow back and white bears will... And eventually we'll be inspired enough to write perf love, the more than expected tool to post process ftrace "love" events.
How can waiting for child1 to run a bit before forking off child2
_not_ hurt? The parent is the worker bee creator, the queen bee if
you will. Seems to me that making the queen wait until one egg
hatches and ages a bit before laying another egg is a very bad plan
if the goal is to have a hive full of short lived worker bees.
-- Mike Galbraith (thanks to Ingo Molnar)
And yes, but the engineering model of the kernel development cycle
is that engineer hours are wasted and thrown away all the time.
They are surplus, sorry. That's how life works here.
-- Greg Kroah-Hartman
One of my functions is pointlessly sending patches at maintainers
so you don't have to.
-- Andrew Morton
Writing kernel modules in HaskellThere must be a crowd of people out there thinking that they would get into kernel development, but only if they could do it in Haskell. Here is a web site with instructions on how to do just that. "By making GHC and the Linux build system meet in the middle we can have modules that are type safe and garbage collected. Using the copy of GHC modified for the House operating system as a base, it turns out to be relatively simple to make the modifications necessary to generate object files for the Kernel environment." This leads to code which looks like:
hello = newCString "hello" >>= printk >> return 0
Just don't try to merge it upstream.
Van de Ven: Introducing 'timechart'Arjan van de Ven introduces a new tool, called "timechart" on his weblog. Timechart is meant to help visualize and diagnose latency problems in a running Linux system. "To solve this, I have been working on a new tool, called Timechart, based on 'perf', that has the objective to show on a system level what is going on, at various levels of detail. In fact, one of the design ideas behind timechart is that the output should be 'infinitely zoomable'; that is, if you want to know more details about something, you should be able to zoom in to get these details."
Video buffer poolsThe Video4Linux2 API has a well-developed interface for sharing video buffers between user space and the kernel. It is not without its problems, though. Simple video acquisition devices transfer large amounts of data (video frames) but cannot do scatter/gather I/O, forcing the allocation of large, physically-contiguous buffers. Queueing buffers for frame transfers can be a significant source of latency, especially when user-space buffers need to be locked into memory or when the architecture requires significant cache invalidation operations. It would also be nice to be able to pass buffers directly between video devices and related devices, such as hardware codecs, but the current API does not support that well.In response to these problems, Laurent Pinchart has proposed a new subsystem implementing a global video buffer pool. These buffers would be allocated early in the system's lifetime, working around the unreliability of large contiguous allocations. Cache invalidation operations could be done ahead of time, eliminating a significant source of capture-time latency. Passing buffers between devices would be explicitly supported. The proposal is in an early stage, and Laurent would like comments from interested developers.
Bouncing off the merge windowAt this stage of the development cycle, attention naturally turns to what has been merged into the mainline kernel. It can also be interesting, though, to look at what is not getting in. This time around, a couple of things have run into opposition at merge time and may, as a result, not find their way into the 2.6.32 kernel.One of those is the reflink() system call (covered last week), which got an "I'm not pulling this" response from Linus. His objections included the way the system call was seemingly hidden in the ocfs2 tree, concern over how much VFS and security review it has received, and a dislike of the name. He would rather see a name like copyfile(), and he would like it to be more flexible; enabling server-side copying of files on remote filesystems was one idea which was raised. In response, Joel Becker has proposed a new system call, called copyfile(), which would offer more options regarding just how the copy is done. There has not been much input from developers other than Linus, but Linus, at least, seems to like the new approach. So reflink() is likely to evolve into copyfile(), but there is clearly not time for that to happen in the 2.6.32 merge window. The other development encountering trouble is fanotify (covered in July). The problem here is that there still is no real consensus on what the API should look like. The current implementation is based on a special socket and a bunch of setsockopt() calls, but there has been pressure (from some) to switch to netlink or (from others) to a set of dedicated system calls. Linus made a late entry into the discussion with a post in favor of the system call alternative; he also asked:
I still want to know what's so wonderful about fanotify that we
would actually want
yet-another-filesystem-notification-interface. So I'm not saying
that I'll take a system call interface. I just don't think that
hiding interfaces behind some random packet interface is at all any
better.
That led to an ongoing discussion about what fanotify is for, whether a new notification API is necessary, and whether fanotify can handle all of the things that people would like to do with it. See Jamie Lokier's post for a significant set of concerns. Linux developers have added two inadequate file notification interfaces so far; there is a certain amount of interest in ensuring that a third one would be a little better. So chances are good that fanotify will sit out this development cycle.
Kernel development news 2.6.32 merge window, part 1Linus started taking patches for the 2.6.32 merge window on September 10. Thus begins the process which should lead to a final kernel release around the beginning of December. As of this writing, some 4400 non-merge changes have been merged. The most significant user-visible changes include:
Changes visible to kernel developers include:
The merge window should stay open for at least another week; it is not clear how LinuxCon and the Linux Plumbers Conference might affect the schedule. Next week's edition will contain an update on changes merged after the publication of this page.
Various scheduler-related topicsScheduler-related development seems to come in bursts. Things will be relatively quiet for a few development cycles, then activity will suddenly increase. We would appear to be in one of those periods where developers start to show a higher level of interest in what the scheduler is doing. The posting of the BFS scheduler has certainly motivated some of this activity, but there is more than that going on.
InteractivityOn the BFS front, the (mildly) inflammatory part of the discussion would appear to have run its course. Anybody who has watched the linux-kernel list knows that serious attempts to fix problems often follow the storm; that appears to be the case this time around. Benchmarks are being posted by a number of people; as a general rule, the results of these benchmark runs tend to be mixed. There are also developers and users posting about problems that they are observing; see, for example, Jens Axboe's report of a ten-second pause while trying to run the xmodmap command. As part of the process of tracking down problems, the conversation turned to tuning the scheduler. Ingo Molnar pointed out that there is a whole set of flags governing scheduler behavior, all of which can be tweaked by the system administrator:
Note, these flags are all runtime, the new settings take effect
almost immediately (and at the latest it takes effect when a task
has started up) and safe to do runtime. It basically gives us
32768 pluggable schedulers each with a slightly separate algorithm
- each setting in essence creates a new scheduler.
The idea here is not that each user should be required to pick out the correct scheduler from a set of 32768 - a number which presumably seems high even to the "Linux is about choice" crowd. But these flags can be useful for anybody who is trying to track down why the behavior of the scheduler is not as good as it should be. When a tuning change improves things, it gives developers a hint about where they should be looking to find the source of the problem. A particular test suggested by Ingo was this:
echo NO_NEW_FAIR_SLEEPERS > /debug/sched_features
(Politically-correct developers will, of course, have debugfs mounted under /sys/kernel/debug. Your editor takes no position on the proper debugfs mount point.) One tester reported immediately that setting this flag made the problems go away. Jens also noted that his ten-second xmodmap problem was solved. The evidence of problems with the NEW_FAIR_SLEEPERS feature was compelling enough that Ingo posted a patch to disable it by default; that patch has been merged for 2.6.32. For the curious, the NEW_FAIR_SLEEPERS feature is a simple tweak which gives a process a small runtime credit when it returns to the run queue after a sleep. It is meant to help interactive processes, but, clearly, something is not working as expected. Once the real problem has been tracked down, it's possible that the NEW_FAIR_SLEEPERS feature could, once again, be enabled by default. In the mean time, users experiencing interactivity problems may want to try disabling it and seeing if things get better.
Child-runs-firstAnother default parameter is changing for 2.6.32; it controls which process runs first after a fork(). For much of the recent past, fork() has arranged things such that the child process gets to run before fork() returns to the parent; this behavior was based on the general observation that the child's work is often more important. There is a good reason to run the parent first, though: the parent's state is active in the processor, the translation lookaside buffer (TLB) contains the right information, etc. So parent-runs-first should perform better. It appears that recent tests showed that parent-runs-first does, indeed, outperform child-runs-first on that most important benchmark: kernel builds. That was enough to get the default changed. There are some concerns that this change could expose application bugs. Jesper Juhl expresses those concerns this way:
I'm just worried that userspace programs have come to rely on a
certain behaviour and changing that behaviour may result in
undesired results for some apps. In a perfect world people would
just fix those apps that (incorrectly) relied on a certain
child-/parent-runs-first behaviour, but the world is not perfect,
and many apps may not even have source available.
Child-runs-first has never been a part of the fork() API, though; it's not something that applications should rely on. Even before the change, behavior could differ as a result of preemption, SMP systems, and more. So it's really true that child-runs-first was never guaranteed. But that will not make users feel any better if applications break. To help those users, there is a new kernel.sched_child_runs_first sysctl knob; setting it to one will restore the previous behavior.
Better cpuidle governanceActive CPU scheduling is interesting, but there is also work happening in another area: what happens when nobody wants the CPU? Contemporary processors include a number of power management features which can be used to reduce power consumption when nothing is going on. Clearly, anybody who is concerned about power consumption will want the processor to be in a low-power state whenever possible. There are, however, some problems with a naive "go into a low power state when idle" policy:
It turns out that the CPU "governor" code in the mainline kernel often gets this decision wrong, especially for the newer Intel "Nehalem" processors; the result is wasted energy and poor performance, where "poor performance" means a nearly 50% hit on some tests that Arjan van de Ven ran. His response was to put together a patch aimed at fixing the problems. The approach taken is interesting. Clearly, it makes no sense to put the processor into a low-power state if it will be brought back to full power in the very near future. So all the governor code really has to do is to come up with a convincing prediction of the future so it knows when the CPU will be needed again. Unfortunately, the chip vendors have delayed the availability of the long-promised crystal-ball peripherals yet again, forcing the governor code to rely on heuristics; once again, software must make up for deficiencies in the hardware. When trying to make a guess about when a CPU might wake up, there are two things to consider. One is entirely well known: the time of the next scheduled timer event. The timer will put an upper bound on the time that the CPU might sleep, but it is not a definitive number; interrupts may wake up the CPU before the timer goes off. Arjan's governor tries to guess when that interrupt might happen by looking at the previous behavior of the system. Every time that the processor wakes up, the governor code calculates the difference between the estimated and actual idle times. A running average of that difference is maintained and used to make a (hopefully) more accurate guess as to what the next idle time will really be. Actually, several running averages are kept. The probability of a very long idle stretch being interrupted by an interrupt is rather higher than the probability when expected idle period is quite short. So there is a separate correction factor maintained for each order of magnitude of idle time - a 1ms estimate will have a different correction factor than a 100µs or a 10ms guess will. Beyond that, a completely different set of correction factors is used (and maintained) if there is I/O outstanding on the current CPU. If there are processes waiting on short-term (block) I/O, the chances of an early wakeup are higher. The performance concern, meanwhile, is addressed by trying to come up with some sort of estimate of how badly power-management latency would hurt the system. A CPU which is doing very little work will probably cause little pain if it goes to sleep for a while. If, instead, the CPU is quite busy, it's probably better to stay powered up and ready to work. In an attempt to quantify "busy," the governor code calculates a "multiplier":
multiplier = 1 + 20*load_average + 10*iowait_count
All of the numbers are specific to the current CPU. So the multiplier is heavily influenced by the system load average, and a bit less so by the number of processes waiting for I/O. Or so it seems - but remember that processes in uninterruptible waits (as are used for block I/O) are counted in the load average, so their influence is higher than it might seem. In summary, this multiplier grows quickly as the number of active processes increases. The final step is to examine all of the possible sleep states that the processor provides, starting with the deepest sleep. Each sleep state has an associated "exit latency" value, describing how long it takes to get out of that state; deeper sleeps have higher exit latencies. The new governor code multiplies the exit latency by the multiplier calculated above, then compares the result to its best guess for the idle time. If that idle time exceeds the adjusted latency value, that sleep state is chosen. Given the large multipliers involved, one can see that expected idle times must get fairly long fairly quickly as the system load goes up. According to Arjan, this change restores performance to something very close to that of a system which is not using sleep states at all. The improvement is significant enough that Arjan would like to see the code merged for 2.6.32, even though it just appeared during the merge window. That might happen, though it is possible that it will turned into a separate CPU governor for one development cycle just in case regressions turn up.
Hw-breakpoint: shared debugging registersModern processors support hardware breakpoint or watchpoint debugging functionality, but the Linux kernel does not provided a way for debuggers, such as kgdb or gdb, to access these breakpoint registers in a shared manner. Thus, debuggers running concurrently can easily collide in their use of these registers, causing the debuggers to act in a strange and confusing manner. For example, continuing execution through a breakpoint, rather than breaking, would certainly confuse a programmer. This issue is being addressed by a proposed kernel API called hw-breakpoint (alternatively hw_breakpoint). The hw-breakpoint functionality, developed in a series of patches by K. Prasad, Frederic Weisbecker, and Alan Stern, aims to provide a consistent, portable, and robust method for multiple programs to access special hardware debug registers. These registers are useful for any application that requires the ability to observe memory data accesses, or trigger the collection of program information based on data accesses. Such applications include debugging, tracing, and performance monitoring. While these patches initially target the x86, they attempt to provide a generic API that can be supported in an architecture independent manner on various processors. Although the details are still being ironed out, with hw-breakpoint hardware debug resources can be concurrently available to various users in a more portable manner. The most common debugging scenarios that would use the hw-breakpoint patches are memory corruption bugs. Programming mistakes such as bad pointers, buffer overruns, and improper memory allocation/deallocation can lead to memory corruption where valid data is accidentally overwritten. These bugs can be hard to find; the corruption can occur anywhere in the program. The error resulting from the corruption often occurs long after the corruption. These bugs cannot typically be found by focusing on the local sections of code that explicitly access the corrupted data. Instead, debugger watchpoints, which are a special type of breakpoint, are the first choice for debugging memory corruption problems. Debugger breakpoints halt program execution at a given address and transfer control to the debugger. This allows the program state (variables, memory, and registers) to be examined. When programmers talk of breakpoints they usually are referring to software breakpoints. For example, in gdb the break command sets a software breakpoint at the specified instruction address. The break command replaces the specified instruction with a trap instruction that, when executed, passes control to gdb. In contrast, watchpoints are best implemented using hardware breakpoints; software implementations of watchpoints are extremely slow. But, hardware breakpoints require special debug registers in the processor. These debug registers continuously monitor memory addresses generated by the processor, and a trap handler is invoked if the address in the register matches the address generated by the processor. Memory accesses can be for data read, data write, or instruction execute (fetch), so hardware breakpoints usually support trapping on not only the address, but also the type of access: read, write, read/write, or execute. Hardware debug registers may also support trapping on IO port accesses in addition to memory accesses. In either case, a watchpoint is a trap on any type of data access rather than just an instruction execute access. Since memory corruption can happen anywhere in the program, a watchpoint set to trap on writes to the corrupted variable/location can be a good way to catch these bugs in the act. These hardware debug registers are limited resources: Intel x86 processors support up to four hardware breakpoints/watchpoints using the special purpose DR0 to DR7 registers. Registers DR0 to DR3 can be programmed with the virtual memory address of the desired hardware breakpoint or watchpoint. DR4 and DR5 are reserved for processor use. DR6 is a status register that gives information about the last breakpoint hit, such as the register number of the breakpoint, and DR7 is the breakpoint control register. DR7 includes controls such as, local and global enables, memory access type, and memory access length. However, as with any limited hardware resource, multiple software users must contend for access of these registers. Since existing released kernels do not control or arbitrate access to these registers, software users can unknowingly clash in their usage, which usually will result in a software error or crash. Hw-breakpoint solves this problem by arbitrating the access to these limited hardware registers from both user-space and kernel-space software. User-space access, such as from gdb, is done via the ptrace() system call. Kernel-space access includes kgdb and KVM (only during context switches between host and guests). Hw-breakpoint arbitration keeps kernel and/or user space debuggers from stepping on each others' toes . Additional kernel patches have been developed to take advantage of the hw-breakpoint API. A plug-in for ftrace (ftrace has previously been discussed in LWN articles here and here) has been developed to dynamically trace any kernel global symbol. This functionality, called ksym_tracer, allows all read and write accesses on a kernel variable to be displayed in debugfs. Since it uses the hw-breakpoint API, it relies on underlying hardware breakpoint support. This new feature of ftrace could be very useful for memory corruption bugs that are difficult to catch with watchpoints. These difficulties include such things as: 1) an erroneous write that is lurking beneath a large quantity of valid writes, 2) the necessity to setup a remote machine to run Kgdb, and 3) kernel bugs which no longer manifest themselves when the machine is halted via breakpoints. Hw-breakpoint allows the concurrent use of both ksym_tracer and debugger watchpoints without the risk of hardware debug register corruption. In addition to ftrace, perfcounters (see LWN articles here and here) can be enhanced through the generic hw-breakpoint functionality. Specifically, counters can be updated based on data accesses rather than instruction execution. A patch to perfcounters has been developed to use kernel-space hardware breakpoints to monitor performance events associated with data accesses. For example, spinlock accesses can be counted by monitoring the spinlock flag itself. Currently this patch is rather limited in supporting the definition and use of breakpoint counters. However, additional features are planned. Since the additions to ftrace and perfcounter patches, the hw-breakpoint API can now be potentially used by several pieces of code: kgdb, KVM, ptrace, ftrace, and perfcounters. This increased potential usage has resulted in increased scrutiny of the API by various developers: hw-breakpoint is no longer solely of concern to debugger developers. This increased scrutiny has resulted in major changes to the hw-breakpoint code that are still ongoing. In particular, the coupling of perfcounters to hw-breakpoint has caused the rethinking of a significant chunk of the original hw-breakpoint functionality and structure. The original (pre-perfcounter support) hw-breakpoint functionality was primarily developed by K. Prasad. It supported global, system-wide kernel-space breakpoints and per-thread user-space breakpoints. Whereas user-space breakpoints were only enabled during thread execution, kernel breakpoints were always present on all CPUs in the system. Additionally, no reservation policy was implemented. Requests for hardware debug registers were granted on a first-come, first-serve basis. Once all physical debug registers were used, hw-breakpoint returned an error for further breakpoint requests. This original hw-breakpoint implementation is "an utter mis-match" to support perfcounter functionality for three reasons, as pointed out by Peter Zijlstra. First, counters (either user or kernel-space) can be defined per-cpu or per-task; this conflicts with hw-breakpoint's system-wide kernel breakpoints. Second, per-task counters are scheduled by perfcounter to save unnecessary context swaps of the underlying hardware resources when it is not necessary. Third, counters can be multiplexed, in a time-sliced fashion, beyond the underlying hardware PMUs (performance monitoring unit) resource limit, which for x86 hardware breakpoints is four. These incongruities between perfcounter and hw-breakpoint led to a debate about any coupling between hw-breakpoint and perfcounter. However, a consensus formed that integrating hw-breakpoint into perfcounter's PMU reservation and scheduling infastructure would be beneficial given perfcounters richer support for scheduling, reservation, and management of hardware resources. About these benefits Frederic Weisbecker writes:
And in the end we have a pmu (which unifies the control of
this profiling unit through a well established and known object for
perfcounter) controlled by a high level API that could also benefit to
other debugging subsystems.
Newly posted in the last week is Weisbecker's patch to integrate hw-breakpoint and perfcounter code. Conceptually, this splits the hw-breakpoint functionality into two halves: 1) the top level API, and 2) the low level debug register control. In between these halves lies the perfcounter functionality. With this patch each breakpoint is a specific perfcounter instance called a breakpoint counter. Perfcounter handles register scheduling, and thread/CPU attachment of these breakpoint counter instances. The modified hw-breakpoint API still handles requests from ptrace(), ftrace, and kgdb for breakpoints by creating a breakpoint counter. Breakpoint counters can also be created directly from the existing perfcounter system call (perf_counter_open()). The breakpoint counter layer interacts with the low-level, architecture specific hw-breakpoint code that handles reading and writing the processor's debug registers. Unfortunately, because of the very recent integration into perfcounters, the hw-breakpoint API has changed and additional changes to the API are planned. Rather than cover in detail the existing API, since it appears likely to change, I will give a summary of it. Two Function calls are provided to set a new hardware breakpoint.
int register_user_hw_breakpoint(struct task_struct *tsk, struct hw_breakpoint *bp);
int register_kernel_hw_breakpoint(struct hw_breakpoint *bp, int cpu);
where:
cpu is the cpu number to set the breakpoint on;
*tsk is a pointer to 'task_struct' of the process to which the address belongs;
*bp is a pointer to the breakpoint property information which includes:
1) a pointer to function handler to be invoke upon hitting the breakpoint;
2) a pointer to architecture dependent data (struct arch_hw_breakpoint).
The struct arch_hw_breakpoint provides breakpoint properties such
as the memory address of the breakpoint, type of memory access
(read/write, read, or write), and the length of memory access (byte,
short, word, ...). These parameters are highly dependent upon the
specific support provided by the hardware. For example, while x86
supports virtual memory addresses, other processors support physical
memory addresses. Since the API aims for architecture independence, this
structure is architecture dependent.
To avoid having to register and unregister a breakpoint if it just needs modification, the following function is provided:
int modify_user_hw_breakpoint(struct task_struct *tsk, struct hw_breakpoint *bp)
Hardware breakpoints are removed by an unregister function:
void unregister_hw_breakpoint(struct hw_breakpoint *bp)
Hw-breakpoint has made its way into the -tip tree, the kernel source development tree maintained by Ingo Molnar. In June it was tentatively targeted for merging from -tip into the 2.6.32 kernel. However, the delayed integration with perfcounters has pushed any merge out past 2.6.32. Whenever it is released, hw-breakpoint promises to provide a portable and robust method for debuggers to access hardware breakpoints without conflict. While the hw-breakpoint functionality started out as a relatively isolated feature to support debuggers, its existence has spawned new tracing and performance monitoring features. These new features should prove useful for various situations where data memory access, rather than instruction access provides the appropriate trigger to collect dynamic information. By leveraging the perfcounter resource scheduling and reservation functionality, hw-breakpoint has a very generalized method for managing limited hardware breakpoint registers. The release of hw-breakpoint promises to enable new ways for Linux users to track down difficult bugs such as memory corruption, and to enable diverse dynamic data access techniques (such as gdb watchpoints and ftrace ksym_tracer) to play well together.
Patches and updates Kernel trees
Core kernel code
Development tools
Device drivers
Documentation
Filesystems and block I/O
Kernel building
Memory management
Architecture-specific
Security-related
Virtualization and containers
Miscellaneous
Page editor: Jonathan Corbet Distributions News and Editorials Free Software Foundation approved distributionsThe Free Software Foundation (FSF) recently announced an update to their list of free Linux distributions, as defined by the GNU Guidelines for Free System Distributions. According to the guidelines all "software, documentation, fonts, and other useful works" must be licensed under one of these free licenses. Trademarks must be packaged separately so that they are easy to remove by anyone wishing to create a derivative distribution. Red Hat and the Fedora project are good examples of conforming to the trademark requirements. Potentially patented software is allowed, however.
It is effectively impossible for free software developers and distributors
to know whether or not a given piece of software infringes any patents:
there are too many of them, they vary from country to country, they're
often worded so as to make it hard to tell what they do or don't cover, and
it isn't easy to tell which ones are valid. Therefore, we don't generally
ask free system distributions to exclude software because of possible
threats from patents. On the other hand, we also don't object if a
distributor chooses to omit some software in order to avoid patent risk.
Everything in a distribution's repositories must conform to the guidelines. Separating some works into a non-free repository, as Debian does, is not sufficient. So far, Fedora conforms to these guidelines, but it fails to make the list by its inclusion of non-free firmware.
Some applications and drivers require firmware to function, and sometimes
that firmware is distributed only in object code form, under a nonfree
license. We call these firmware programs "blobs." On most GNU/Linux
systems, you'll typically find these accompanying some drivers in the
kernel Linux. Such firmware should be removed from a free system
distribution.
Unfortunately this last requirement makes these free distributions unusable by many users. The standard Linux kernel contains many of these blobs in order to function on some common hardware. Users that install one of these totally free distributions may find that their graphics card no longer works, or their wireless internet connection fails, or some other part of their system becomes totally unusable. Building a usable and completely free system is still a pipe dream for many users. Some users will find that their hardware and software needs are met by these criteria. For those interested in building blob free kernels, Brian Brazil, Jeff Moe, and Alexandre Oliva have developed a series of scripts to remove blobs from a vanilla kernel. For those users we will take a look at the current list of free distributions. These distributions are all actively maintained, another requirement stated in the guidelines. You'll find them all on the LWN Distribution list, along with many other (less free) distributions. We will begin with the FSF's own proof of concept project, gNewSense. gNewSense is based on Debian and Ubuntu. As required all trademarks have been stripped away, blobs have been removed from the kernel and the project does not host any non-free repositories. gNewSense 2.3 is the latest, recently released, version. No less than three distributions hail from Argentina. Ututo is a Gentoo-based distribution, one of the first to be recognized by the FSF. The website is in Spanish. This project aims to reduce the digital divide in developing countries. UTUTO XS 2009 was released in March of 2009. Dragora is another Argentinian project, developed by Matías A. Fonzo. It is designed to be a simple, yet stable and powerful, multipurpose system. Version 1.0 was also released in March of 2009. Musix GNU/Linux is the third Argentinian distribution. It is based on Debian/Knoppix and supports several languages, including Spanish, English, Italian, Russian and many more. The live DVD contains support for about 65 languages. Musix is aimed at multimedia production, with tools for music production, graphic design, audio and video editing, and more. dynebolic is developed by media activists, artists and other creative people. It is aimed at multimedia production. With dynebolic "you can manipulate and broadcast both sound and video with tools to record, edit, encode and stream, having automatically recognized most device and peripherals: audio, video, TV, network cards, firewire, usb and more; all using only free software!" Next is BLAG Linux and GNU. BLAG stands for brixton linux action group. It is Fedora-based and the first to use the Linux-libre blob free kernel. BLAG includes software for graphics, internet, audio, video, office, and peer-to-peer file-sharing applications. There are different flavors of BLAG, including a GNOME desktop, a server and "miniblag", which installs in about 350M. Trisquel hails from the Galicia region of Spain, originally sponsored by the University of Vigo. It is Ubuntu based and supports several languages including Spanish, Galego, and English. Trisquel GNU/Linux 3.0 release, codenamed Dwyn, was recently released. VENENUX targets people in the Spanish-speaking Americas. VENENUX is not an acronym and it is not from any particular country, but is developed and supported by volunteers from many Hispanic countries. VENENUX features the KDE desktop. Last on the FSF list is Kongoni, a system from Africa. Kongoni is the Shona word for a Gnu (also known as a Wildebeest). It is Slackware based, with inspiration from BSD-Unix. Kongoni features some unique tools, such as P.I.G (Ports Installation GUI) and K.I.S.S. (the Kongoni Instant Setup System). Kongoni version 1.12.2 (based on Slackware 12.2) was released in July 2009. That concludes the FSF list of free distributions. They are all GNU/Linux distributions, that is they combine the Linux kernel with GNU software. They should all run on lower end or older hardware, particularly those with Intel chipsets. There is potentially patent encumbered software included in some, particularly the multimedia distributions such as dynebolic and Musix. There is, of course, no guarantee that any of these distributions will work for you, but if you are serious about running FSF-approved free software, these distributions are for you.
New Releases ClearOS (formerly ClarkConnect) 5.1 announcedThe upcoming ClearOS 5.1 release has been announced, marking a number of changes in the distribution once known as ClarkConnect. "ClearOS has been made open and free by ClearFoundation, including software such as MultiWAN which allows multiple, balanced Internet connections. Other limitations and commercial restrictions on the ClearOS platform have also been removed for this version, and all subsequent versions."
gNewSense 2.3 released.The gNewSense project has announced version 2.3 of its FSF Free GNU/Linux distribution. "This release contains all security updates made up to 2009-09-11. As with other 2.x releases, this gNewSense release is available as a live cd for x86 (i386) systems, using GNOME as the default desktop environment (others such as XFCE or KDE are available post installation)."
openSUSE 11.2 Milestone 7 ReleasedThe openSUSE Project has announced that openSUSE 11.2 Milestone 7 is available for download and testing. "This release includes GNOME 2.28 beta 2, KDE 4.3.1, RPM 4.7.1, and much more."
Announcing Fedora 12 Snapshot 2Fedora 12 Snapshot 2 is available for testing. These snapshots consist of live images only, composed September 11, 2009 and made available September 14, 2009. Click below for download information.
Distribution News Debian GNU/Linux debmirror - new featuresdebmirror is a native package that is intended to be used for creating (partial) local mirrors. Version 2.2 is available.
Fedora Help with Fedora ResearchDuke University's MBA Program is conducting research on the Fedora community. They are looking for people willing to participate in an email interview or a phone interview with Duke professors. "We're looking for folks who don't work for Red Hat, and folks who do. We're looking for folks who are highly technical, and folks who aren't. We're looking for folks who contribute lots, or only a little bit."
FUDCon Toronto 2009 updateClick below for an update on FUDCon Toronto 2009. "We are holding FUDCon from Saturday through Monday, December 5-7. The first day, Saturday, will be the day of technical sessions. The second and third days will be hackfest days where people can gather to work on some of the projects they've seen on Saturday, or anything near and dear to their heart."
SUSE Linux and openSUSE openSUSE Core Test Team EstablishedThe openSUSE Core Test Team has been established to better organize the testing of upcoming openSUSE releases. "With the release of Milestone 7 of openSUSE 11.2 - the team is formed and already started his work to ensure that openSUSE will also be a rock-solid Linux distro in the future. First discussion about prioritizing tasks is already started and members of the team are planing their first real time meeting on IRC." Click below to see how non-team-members can participate.
Upcoming openSUSE Board ElectionsBryen Yunashko takes a look at the upcoming openSUSE board elections. "As agreed upon, prior to last year's elections, the highest vote-getters in both Novell and non-Novell positions would assume a two-year term, and the second vote-getters would get a one-year term. This was a one-time arrangement, with the intention of ensuring there would be an election every year for half the electable board positions. As such, we will be holding elections again this year. All winners this year and subsequent elections will have a two-year term."
Ubuntu family Minutes from Technical Board meetingClick below for the minutes of the September 8, 2009 meeting of the Ubuntu Technical Board. Topics include Debian technical committee participation in techboard, Java SRU policy, Removal of sun-java6 from Karmic, Developer Membership Board, and Archive reorganization.
Other distributions FSF updates its "fully free" distribution listThe Free Software Foundation has announced an update to its free distribution list. The latest addition is Kongoni. "Kongoni, named after the Shona word for 'gnu,' is based in Africa. For optimal performance with minimal bandwidth requirements, it uses a packaging system called 'ports' that downloads programs as source and builds them automatically."[Editor's note] This original LWN article is included in the weekly edition because of the many comments attached to it.
Foresight FOCUS log and draft minutesClick below for the minutes of the September 4, 2009 meeting of the Foresight Council. Topics include 'Boots, a Fedora Remix', the 2.1.2 ISO release, and more.
New Distributions moonOSmoonOS is a complete and fully functional operating system based Ubuntu, with LXDE and Enlightenment DR17. moonOS was started and designed by the Cambodian artist Chanrithy Thim (12rithy), and aims to work for any Desktop, Laptop PC or even for a Virtual Machine. Ubuntu 9.10 based moonOS 3 "Makara" has been released.
Trisquel GNU/LinuxTrisquel GNU/Linux is a Ubuntu based, free as in freedom GNU/Linux operating system. The main goal of the project is the production of a fully free as in speech system that is easy to use, complete, and with good language support. The project was started in 2004 with the sponsorship of the University of Vigo in Galicia, Spain. The first release was in April 2005. The project hosts its own repositories which are derivatives of Ubuntu's main and universe components, but with all proprietary software removed. The differences include the removal of all non-free packages, the substitution of the original Linux kernel with the blob-free linux-libre version, and the addition of several packages. Trisquel 3.0 has been released.
Distribution Newsletters DistroWatch Weekly, Issue 320The DistroWatch Weekly for September 14, 2009 is out. "With so many great distributions available today, how do you go about choosing one for your brand-new, state-of-the-art desktop computer? In the second part of his "distro odyssey", Michael Raugh delves into Arch Linux, a rolling-release distribution that is always up-to-date, even though it takes some work to install and set it up. But how did it fare in the test? Read on to find out. In the news section, Oracle hints at new investment into the recently acquired Solaris operating system, Phoronix takes an early look at OpenSolaris 2010.2, openSUSE releases a new set of 11.1 installation images with the latest KDE desktop, and Softpedia presents information about one Ubuntu issue that everybody seems to have an opinion about - the distribution's default artwork. Also in the news, Free Software Foundation expands its list of free distribution, while Debian developer Meike Reichle urges girls and women to join the development teams of free software projects. All this and more in this issue of DistroWatch Weekly - happy reading!"
Fedora Weekly News 193The Fedora Weekly News for September 13, 2009 is out. "In announcements, an invitation to the Fedora community to participate in research at Duke University on open source communities and an update on this December's FUDCon Toronto. A variety of recent postings from Fedora contributors follows in the Planet Fedora section. In marketing news, a call for Fedora 12 slogan ideas, update on weekly meetings and Zikula/Fedora Insight developments, and coverage of recent discussion aimed at getting Fedora into news channels to a greater extent. This week's QA news includes detail of the most recent Test Day activities around graphics card support, and coverage of this week's meetings. In translation news, more updates on Fedora 12 translation activities, and new Localization Project members from Spain. In Art news, a refresh of Fedora 12 wallpapers. This week's issue rounds out with Fedora Virtualization updates, including details on Fedora virt status and F12 virtualization release notes. Please enjoy FWN 193!"
Ubuntu Weekly Newsletter #159The Ubuntu Weekly Newsletter for September 12, 2009 is out. "In this issue we cover: Karmic Alpha 6 freeze ahead, UI freeze in place for Karmic, Ubuntu One has moved, New Ubuntu Developers, New Ubuntu Members, Ubuntu Stats, US LoCo Wiki Doc Day 2009, Ubuntu Pennsylvania, Launchpad meet-up: Sept. 28th in London, Meet Deryck Hodge, Ubuntu Forums Unanswered Post Team Meeting, An interview with Nathan Handler, Screencasting using Ubuntu: Part 1, Jono turning ?? - Wishlist predictable, Jolicloud innovates Atop Ubuntu Netbook Remix, and much, much more!"
Page editor: Rebecca Sobol Development Interview: Leslie Hawthorn on the 2009 Google Summer of CodeLWN last talked to Leslie Hawthorn, Google's Open Source Program Coordinator, in September, 2007 about the Google Summer of Code (GSoC). GSoC is a project where Google pays students to work with a mentor to write open-source code. The 2009 Google Summer of Code recently concluded, marking the end of the project's fifth year. The official end of project summary, Wrapping Our Fifth Google Summer of Code, covers this year's effort:
"The sun has set on our fifth year of introducing college and university
students to Free and Open Source software development, and what a year
it's been! Just under 2000 mentors and 1000 students began working together to improve the code
bases of 150 projects, and we're pleased to let folks know that 85
percent of our student participants have received passing final
evaluations, up a full two percent over 2008 and our best success rate
to date."
The List of all accepted organizations shows the many participating and planned projects, source code from the GSoC projects is available there. LWN: Greetings, Could you tell us about yourself and your role with the Google Summer of Code? L. Hawthorn:
I'm the Program Manager for Google Summer of Code and have been running the
program since 2006. In addition to setting the schedule and giving guidance
to all of the FLOSS projects involved on how to most effectively mentor
their students, etc., I'm also responsible for putting on our annual Mentor
Summit. At these annual conferences, which have been happening since 2007,
folks from all of our diverse projects get together and spend a weekend
determining how to collaboratively make Open Source work better.
There are some statistics from 2005-2007 of the at the end of the previously mentioned LWN article, could you fill in the statistics for 2008 and 2009? Google Summer of Code 2009
Google Summer of Code 2008
Have there been any changes to the program this year?
This year, we have run the program using Melange as the infrastructure to
run the GSoC website. The software is Open Source, Apache licensed and runs
on Google App Engine. Anyone can make use of the code base to run their own
GSoC like program and we're very excited to offer this software to the
community since we had many requests in the past for people to be able to
use our code to run their own mentoring programs. People can take a look at
the code base and contribute feedback and patches by visiting
code.google.com/p/soc/.
Has the economic downturn had any effect on the GSoC? We've certainly heard from our mentors that they had less time to spend on the program than they had hoped and more than a few cited the need to work longer hours or spend more time searching for contract work as a reason their time was constrained. We scaled back the size program a bit this year, taking on about 100 fewer students, but that was about making the program the right size - not stretching mentor resources too thinly - rather than economic constraints. We were happy to have the same budget once again in 2009 as we did for 2008. We also sent out a slightly less expensive start of program gift, offering students an ACM membership rather than a coding related tome like Producing Open Source Software or Beautiful Code. The beautiful part of this gift was that it not only allowed us to save funds for the program - which were reallocated to student travel scholarships - but to reduce our environmental impact by not shipping 1000 packages out to 70 countries. Our students were really excited by the ACM memberships and we plan to keep offering these to our student participants in the future. Are there plans to run the GSoC program again in 2010?
We certainly plan to do so, but won't have more certain announcements until
early next year.
Could you tell us where we could find more information on the accomplishments made during this year's GSoC? We published this report from the Grameen Foundation yesterday. And there should be a post forthcoming today on all the universities that student participants attended over the last five years today on the Open Source Blog. Links to actual source code should now be available from each organization home page on the GSoC 2009 site by clicking on that project's name on the full project list. Your readers can expect reports from at least MoinMoin, The Perl Foundation, SIP Communicator and Etherboot in the next two weeks to be published there as well. Are there any outstanding efforts by students and/or mentors that you would like to mention? Well, I tend to think all of our mentors and students are pretty spectacular. One story that sticks out in my mind is that of Anna Granudd from the Systers project. Anna returned to engineering after a hiatus. While Anna hasn't shared all details I get the impression that she, like many women in the technical fields, was initially discouraged by those close to her from pursuing engineering as a profession. She's now studying Mechanical Engineering and some Python but CS was not her main focus for either academics or personally. She dived right into coding for the Systers project to make things happen and the overall community experiences better for all the women involved in this global network for women in technology. The best part of all is Anna's stories of not being able to go to sleep because she's having too much fun coding. Needless to say, that's the hallmark of a good programmer. Is there anything else you would like to share with our readers about the Google Summer of Code program?
As mentioned in
this blog post,
the most important thing that students who
would like to participate in GSoC can do is to begin exploring Open Source
now. Our most successful students are those who make early contact with
their mentors and begin creating relationships within their project
communities that later help support their coding efforts.
Thank you for your time.
System Applications Audio Projects PulseAudio 0.9.17 releasedVersion 0.9.17 of PulseAudio, a cross-platform sound server, has been announced. This is a bug fix release, see the Milestone 0.9.17 report for details.
Database Software Firebird 2.1.3 releasedVersion 2.1.3 of the Firebird DBMS has been announced. "The team is pleased to announce the release of Firebird 2.1.3, with kits available for Linux and Windows 32-bit and 64-bit platforms." See the release notes for more information.
PostgreSQL 2009-09-09 Security UpdateA multi-version security release of the PostgreSQL DBMS has been announced. "The PostgreSQL Project today released minor versions updating all active branches of the PostgreSQL object-relational database system, including versions 8.4.1, 8.3.8, 8.2.14, 8.1.18, 8.0.22, and 7.4.26. This release fixes one moderate-risk and two low-risk security issues: an authentication issue, a denial of service issue, and a privilege-escalation exploit. All users should upgrade their database installations as soon as reasonably possible."
PostgreSQL Weekly NewsThe September 13, 2009 edition of the PostgreSQL Weekly News is online with the latest PostgreSQL DBMS articles and resources.
SQLite 3.6.18 releasedVersion 3.6.18 of the SQLite DBMS has been announced. "Changes associated with this release include the following: * Versioning of the SQLite source code has transitioned from CVS to Fossil. * Query planner enhancements..."
sqlkit 0.8.6 releasedVersion 0.8.6 of sqlkit has been announced. "This is the first stable release. It features a new interface for the standalone command (sqledit), many improvements and functions added. We have used pyinstaller to create standalone executable for Linux and Mac, you can download them to use the application and to run the demo. It's now registered in pypi so you can 'easy_install' it. I'm currently looking for a debian sponsor to upload the package in sqeeze."
Device Drivers libshcodecs 0.9.6 releasedVersion 0.9.6 of libshcodecs has been announced, it includes several new capabilities. "libshcodecs is a library for controlling SH-Mobile hardware codecs. The [SH-Mobile][0] processor series includes a hardware video processing unit that supports MPEG-4 and H.264 encoding and decoding. libshcodecs is available under the terms of the GNU LGPL."
LIRC 0.8.6 releasedVersion 0.8.6 of LIRC has been announced, it includes support for a number of new devices and protocols and some code structure changes. "LIRC is a package that allows you to decode and send infra-red signals of many (but not all) commonly used remote controls."
Interoperability Samba 3.4.1 releasedVersion 3.4.1 of Samba has been announced. "This is the latest stable release of the Samba 3.4 series."
Printing CUPS 1.4.1 releasedVersion 1.4.1 of CUPS has been announced. "CUPS 1.4.1 fixes several reported printing, web interface, PPD compiler, and CUPS API bugs."
Web Site Development nginx 0.6.39 announcedVersion 0.6.39 of the nginx web server has been announced. The CHANGES document says: "*) Security: a segmentation fault might occur in worker process while specially crafted request handling. Thanks to Chris Ries. *) Bugfix: a segmentation fault might occur in worker process, if error_log was set to info or debug level. Thanks to Sergey Bochenkov."
Facebook releases the "Tornado" web serverFacebook has announced the release of its Tornado web server under the Apache license. "Tornado is a relatively simple, non-blocking Web server framework written in Python, designed to handle thousands of simultaneous connections, making it ideal for real-time Web services. Tornado is a core piece of infrastructure that powers FriendFeed's real-time functionality, which we plan to actively maintain. While Tornado is similar to existing Web-frameworks in Python (Django, Google's webapp, web.py), it focuses on speed and handling large amounts of simultaneous traffic." The code can be had from tornadoweb.org.
Miscellaneous Poettering: Measuring Lock ContentionOn his weblog, Lennart Poettering describes a new tool, called "mutrace", for tracking down performance problems caused by mutex contention in applications. "For each mutex a line is printed. The 'Locked' column tells how often the mutex was locked during the entire runtime of about 10s. The 'Changed' column tells us how often the owning thread of the mutex changed. The 'Cont.' column tells us how often the lock was already taken when we tried to take it and we had to wait. The fifth column tell us for how long during the entire runtime the lock was locked, the sixth tells us the average lock time, and the seventh column tells us the longest time the lock was held. Finally, the last column tells us what kind of mutex this is (recursive, normal or otherwise)."
OLPC: criticisms and a defenseOver at OLPC News, Jon Camfield posts a defense of OLPC. He is reacting to two articles critical of the project: Alanna Shaikh's "One Laptop Per Child - The Dream is Over" and Timothy Ogden's "Computer Error?", both of which are unequivocal in their criticism ("Its time to call a spade a spade. OLPC was a failure." from the former, and "To even its most ardent supporters, the project seems nearly dead in the water. [...] And that may be great news for children in the developing world." from the latter.) Camfield is more hopeful: "Alanna says that 'The dream is over' - I think the nightmares are over; the real long-term and more sustainable dream may be just beginning."Update: OLPC founder Nicholas Negroponte weighs in with a defense of the project as well: "As a small non-profit, humanitarian organization, it is hard to battle giants who view children as a market, not a mission, and have other agendas. In spite of all that, the change is huge."
Desktop Applications Audio Applications Audacity GSoC 2009 projectsThe Audacity audio editor project has announced the successful completion of its 2009 Google Summer of Code projects. "The Audacity Team is very pleased to announce that both our GSoC students passed with flying colors, and on September 1, we released a new beta version (1.3.9) with many bugs fixed, thanks to their efforts and those of other team members. We are much closer to our target of a new stable 2.0 release later this year."
Surfing The Forge: Sound & MIDI Projects On SourceForge (Linux Journal)Dave Phillips surfs SourceForge for new and updated Linux audio software. "The following article represents only a small fraction of the software I discovered. However, it also represents the greater part of the viable software that I found. SF lists projects that are at various stages of development, including those at the "idea" stage. Fortunately there's no need to waste time looking at file listings - SF nicely indicates activity in a project's files base, and a quick glance at the activity metrics will tell the tale of the project's liveliness."
Desktop Environments GNOME 2.28.0 Release Candidate (2.27.92) releasedVersion 2.27.92 of GNOME has been announced. "We're a few days before the hard code freeze for 2.28.0, and having tried 2.27.92, I think we have something good there. Actually, better than just good. But well, we still have a few days to fix this pet bug that annoys so many people -- I even heard that, in case you'd be a bit late, some release team people can give +1 to freeze break requests if you have the right arguments. And food is always a good argument. But I can't tell who those people are. Or maybe I can, if you have the right arguments..."
GNOME Software AnnouncementsThe following new GNOME software has been announced this week:
KDE Software AnnouncementsThe following new KDE software has been announced this week:
Xorg Software AnnouncementsThe following new Xorg software has been announced this week:
Educational Software HTML5 + JS: The Future of Open Education (OLPC News)Over at OLPC News, Bryan Berry has announced a new framework for interactive educational content called "Karma". "Release 0.1 is very much an alpha release and we have long way to go to reach this project's goals. In essence, the Karma project aims to create a set of templates for creating interactive lessons using standard web development technologies. It bears special emphasis that these lessons can run online or offline. We are developing Karma expressly for the Sugar environment but we are making it flexible enough so that it can be used as broadly as possible."
Geographical Software MapOSMatic: generate city maps from OpenStreetMap dataThe MapOSMatic project has been launched. "We are pleased to announce the release of MapOSMatic, a set of tools to automatically generate cities' map from OpenStreetMap data. MapOSMatic takes care of generating a labelled grid over the map, a list of street with references matching the grid as well as a nice layout of the city if its administrative boundaries are known. For now, it only supports rendering French metropolitan cities' maps, but it will soon be extended to other parts of the world."
Graphics Inkscape: GSoC resultsThe Inkscape vector graphics editor project has announced the completion of its 2009 Google Summer of Code projects. Krzysztof Kosiński has completely rewritten the Node Tool, Arcadie Cracan has expanded the functionality of the Connector Tool, Felipe Sanches worked on better support for color management and Soren Berg has added a scripting API via D-Bus. All of the changes will show up in version 0.48.
Math Applications OpenOpt 0.25 releasedVersion 0.25 of OpenOpt, a Python-based numerical optimization package, has been announced. "OpenOpt is cross-platform (Windows, Linux, Mac OS etc) Python-written framework. If you have a model written in FuncDesigner, you can get 1st derivatives via automatic differentiation".
Medical Applications Open-source software may unify the medical-records realm (ComputerWorld)Steven J. Vaughan-Nichols takes a look at open source medical software. "One EHR [electronic health records] system, however, does have a proven record, since its introduction in 1982: VistA (Veterans Health Information Systems and Technology Architecture), the U.S. Veterans Administration's public-domain EHR. VistA has become the foundation for over a dozen proprietary and open-source medical record software suites."
Multimedia Moovida Media Center 1.0.7 releasedVersion 1.0.7 of Moovida Media Center has been announced. "This new release adds experimental support for DVD playback (including from DVD images). It also includes many bug fixes; most notably: Moovida now obtains album covers again (now using Last.fm) and the start-up looks nicer than ever."
Blender+Ardour, an Amazing CombinationJuan Pablo Bouza has written a short tutorial on synchronizing Blender and Ardour. "Everyone of you who ever tried to use Blender for video editing may know that the Audio workflow is very limited, in opposite to the limitless possibilities that are delivered for Image and video editing. Well, some time ago I found out that you can connect Blender to Ardour through a little script created by our friend ILdar AKHmetgaleev, also known as AkhIL. For those of you who don´t know what Ardour is, it is the best multitrack audio editing program for Linux. It is the Open Source alternative to Protools and Nuendo. So, the thing is that you can connect these two amazing applications through Jack, and huge possibilities arise!"
Office Suites KOffice 2.1 Beta 2 released (KDEDot)KDE.News has announced the release of KOffice 2.1 Beta 2. "Something that is not obvious from the changelog is that there has been much activity in the MS office import filters, especially for MS Word and Powerpoint. Many new formatting features have been implemented in both these filters. We expect KOffice 2.1 to be better at reading MS file formats than any previous KOffice version."
Streaming Media Oggz 1.0.0 releasedVersion 1.0.0 of Oggz has been announced, it includes security and bug fixes. "Oggz comprises liboggz and the tool oggz, which provides commands to inspect, edit and validate Ogg files. The oggz-chop tool can also be used to serve time ranges of Ogg media over HTTP by any web server that supports CGI. "
Video Applications PiTiVi 0.13.3 releasedVersion 0.13.3 of PiTiVi, an open source video editor, has been announced. "Features of this release: * Fix rendering failures * UI beautifications * Switch to themeable ruler * Speed optimisations * Show the project name in the window title".
Web Browsers Firefox 3.5.3 and 3.0.14 are availableVersions 3.5.3 and 3.0.14 of Firefox have been announced. "As part of Mozilla's ongoing stability and security update process, Firefox 3.5.3 and Firefox 3.0.14 are now available for Windows, Mac, and Linux as free downloads".
Languages and Tools Caml Caml Weekly NewsThe September 15, 2009 edition of the Caml Weekly News is out with new articles about the Caml language.
Python argparse 1.0.1 releasedVersion 1.0.1 of argparse has been announced. "The argparse module provides an easy, declarative interface for creating command line tools, which knows how to: * parse the arguments and flags from sys.argv * convert arg strings into objects for your program * format and print informative help messages * and much more.."
Jython 2.5.1 Release Candidate 2 is outVersion 2.5.1 Release Candidate 2 of Jython, an implementation of Python in Java, has been announced. "Jython 2.5.1rc2 fixes bugs that we found when testing rc1, including some db, codec, and locking issues."
pyftpdlib 0.5.2 releasedVersion 0.5.2 of pyftpdlib has been announced. "Python FTP server library provides a high-level portable interface to easily write asynchronous FTP servers with Python. pyftpdlib is currently the most complete RFC-959 FTP server implementation available for Python programming language. It is used in projects like Google Chromium and Bazaar and included in Linux Fedora and FreeBSD package repositories. This new version is mainly a bugfix release, including some important security-related patches."
Tcl/Tk Tcl-URL! - weekly Tcl news and linksThe September 10, 2009 edition of the Tcl-URL! is online with new Tcl/Tk articles and resources.
Libraries libjio 1.00 releasedVersion 1.00 of libjio has been announced. "The latest version of libjio, 1.00, has been released. It features minor fixes and documentation updates since the last release, but marks the beginning of the first stable series. libjio is a userspace library to do journaled, transaction-oriented I/O. It provides a very simple API to commit and rollback transactions, and on top of that a UNIX-alike set of functions to perform most common operations (open(), read(), write(), etc.) in a non-intrusive threadsafe and atomic way, with safe and fast crash recovery."
MPC 0.7 releasedVersion 0.7 of MPC, a C library for arithmetic of complex numbers, has been announced. "Of particular interest in this release are bugfixes, especially for complex division, and the introduction of mpc_pow used for folding cpow{,f,l} inside GCC. Note the complex "arc" functions are still missing and are now projected to be available in a future release, probably 0.8."
Version Control CGIT 0.8.3 releasedVersion 0.8.3 of CGIT has been announced. "A new feature-release of cgit, a fast webinterface for git, is now available".
GIT 1.6.4.3 releasedVersion 1.6.4.3 of the GIT distributed version control system has been announced, it includes bug fixes and documentation updates.
monotone 0.45 releasedVersion 0.45 of monotone has been announced. "The monotone project is proud to announce the release of version 0.45 of its version control software. The most important change in this release is that keys are no longer identified by their name, but by their unique hash, which makes the handling of lost private keys in bigger projects much easier. We all owe Timothy Brownawell a big time for his tremendous work here - thank you!"
Miscellaneous Pygments 1.1 releasedVersion 1.1 of Pygments, a generic syntax highlighter, has been announced. It includes Python 3 support, new lexers and bug fixes.
Page editor: Forrest Cook Announcements Non-Commercial announcements The 2009 Q2 GNOME Quarterly ReportThe GNOME Foundation has released its report for the second quarter of 2009 [PDF] - the first of what will apparently be a regular series of such reports. "In these quarterly reports we want to focus on what the GNOME Foundation and its members are doing, so most of the reporting is done by the teams doing the work. You'll hear directly from the release team about the incredibly busy quarter they've had, you'll see first hand how much the travel committee is improving our ability to sponsor travel for GNOME Foundation members, and you'll read updates from all our teams focused on our mission of providing a free desktop accessible for everyone."
Alan Turing gets a belated apologyA determined petition campaign has finally achieved its goal: the British government has apologized for its treatment of Alan Turing. "Thousands of people have come together to demand justice for Alan Turing and recognition of the appalling way he was treated. While Turing was dealt with under the law of the time and we cant put the clock back, his treatment was of course utterly unfair and I am pleased to have the chance to say how deeply sorry I and we all are for what happened to him." Congratulations to all the people who worked to bring a small bit of late justice to an important figure in our field.
Commercial announcements Apple Open Sources Snow Leopard's Grand Central Dispatch (Apple Insider)Apple has decided to open the code behind Snow Leopard's Grand Central Dispatch feature. "The user-space implementation of the Grand Central Dispatch services API, called libdispatch, has been delivered as its own open source project, joining with other components that are part of projects Apple has already designated as open, including the kernel components in the Darwin OS XNU kernel and the blocks runtime that is part of the LLVM project."
EmbedOne Linux shows the way for new level of OS integrationEmbedOne has announced the launch of its embedded Linux OS, tools and services. "Meshcom Technologies, Inc. today announced its new brand name - EmbedOne - for its new embedded Linux line of business. At the same time, new publicly available suite of software and services for the embedded Linux community was announced."
Microsoft launches open source foundation (Linux-Watch)Microsoft has launched the open source CodePlex Foundation. "The goals of the CodePlex Foundation are spelled out on the website as enabling the "exchange of code and understanding among software companies and open source communities," as well as "increasing participation in open source community projects." Further goals include complementing "existing open source foundations and organizations, providing a forum in which best practices and shared understanding can be established by a broad group of participants, both software companies and open source communities.""
The Nokia PUSH N900 programThe Nokia PUSH N900 program has announced its existence; essentially, Nokia is trying to jump-start a development community for this device. "The brief is simple: tell us how you would hack and mod the N900 & Maemo to connect the N900 to something you love. An expert judging panel will be selecting winning submissions and the groups behind them will receive N900 devices, funding and support to develop their PUSH idea."
New Books Hacking: The Next Generation--New from O'ReillyO'Reilly has published the book Hacking: The Next Generation by Nitesh Dhanjani, Billy Rios and Brett Hardin.
Manage Your Project Portfolio--New from Pragmatic BookshelfPragmatic Bookshelf has published the book Manage Your Project Portfolio by Johanna Rothman.
Resources FSFE NewsletterThe August, 2009 edition of the FSFE Newsletter is online with the latest Free Software Foundation Europe news. Topics include: "Fellowship meeting in Zurich, Switzerland, 26 August, 4th FrOSCon in St. Augustin, Germany, 22-23 August, Software patent infringed: Microsoft barred from selling Word, New overview on our community contacts, Fellowship meeting and KDE 4.3 release party in Stuttgart, Germany, 01 August".
Linux Foundation NewsletterThe September, 2009 edition of the Linux Foundation Newsletter has been published. Topics include: "* Updated Study Reveals 10% Increase in Linux Kernel Developers * Vote For Your Favorite Fake Linus Torvalds * Last Week to Register for LinuxCon Training - Special 20% Discount * Watch LinuxCon Keynotes Online for Free * LinuxCon Kernel Roundtable Moderator Interviewed * Members' Discounts for LF Events * VIA Technologies, Inc. Joins Linux Foundation * Zemlin to Keynote at Open World Forum * Linux Foundation in the News * From the Director".
Trademarks - The dinosaur in the room (H Online)The H has a lengthy article exploring trademarks and free software. "The issue is trust, and trust goes both ways. Users and developers, who are often involved on a purely voluntary basis, are resistant to the paternalism that is implicit in a Trademark License Agreement, and some view it as a surreptitious method for suppressing criticism. The most contentious clause in such agreements has been the claim to ownership of all domains that include the trademark, illustrated by the clause in the Drupal trademark and logo policy which seeks to deny a trademark license to domains which do not qualify as 'fostering the Drupal software', such as "creating a Drupal fork 'ImprovedDrupal', or 'publishing a website 'drupalhallofshame.com' with pictures of infamous Drupal contributors.'"
Contests and Awards Nominations open for 2009 Linux Medical News Freedom Award (LinuxMedNews)LinuxMedNews has announced the Linux Medical News Freedom Award nominations. "Nominations are officially open for the 9th annual Linux Medical News Freedom Award to be presented at the November 14th-18th AMIA Fall conference in San Francisco, CA. Deadline for entries is September 30th, 2009."
Calls for Presentations Distro Summit 2010: Call for PapersDistro Summit 2010 is a one-day technical conference with a strong focus on collaboration between Free Software distributions. The call for proposals is open until September 30, 2009. The event is co-hosted with linux.conf.au, which will be held in Wellington (New Zealand) on the 18-23 of January, 2010.
LCA2010 Miniconf Call for Papers is openThe Miniconf Call for Papers has been announced for LCA2010 "Miniconfs are an important part of linux.conf.au, allowing 1-day long dedicated streams for specific communities of interest over the course of two days. The Call for Miniconf Papers marks another opportunity for delegates to submit session proposals to a Miniconf of their choosing for selection. Among the Miniconfs being held in Wellington next year are - "The Business of Open Source", "Education", "Free the Cloud", and "Open and the Public Sector"."
Open Source Hardware: Call for PapersA call for papers has gone out requesting articles on Open Source Hardware for the Journal of Information Law and Technology/ European Journal of Law and Technology. "The question we are setting in this call for papers is: what is the best method to enforce the open source philosophy for hardware, to ensure that contributions based upon the work of others remain open for others to develop."
SecurityTubeCon CFP, Venue: CyberspaceA call for papers has gone out for the SecurityTubeCon, it takes place in cyberspace on November 6-8, 2009. "SecurityTube.net is pleased to announce the CFP for SecurityTubeCon, the first hacker conference, to be held completely online! SecurityTubeCon is aimed at democratizing hacker conferences by allowing any researcher, regardless of his physical location, to share his work with the community. Unlike other Cons we will not *accept / reject* speakers. If you have something interesting to share, you WILL be heard."
Upcoming Events FSF announces Boston Software Freedom DayThe Boston Software Freedom Day has been announced. "On Saturday, September 19th, Boston's casual free software users and the technologically curious will gather together for a Software Freedom Day event hosted by the Free Software Foundation (FSF). Software Freedom Day is the international holiday dedicated to recognizing and promoting software that respects computer users' freedom. While the FSF is hosting its day-long public gathering in Boston, other teams all over the globe will be celebrating in their own cities, towns and villages as well."
Government Open Source Conference announcedThe Government Open Source Conference (GOSCON) will take place on November 5 in Washington D.C. "The Government Open Source Conference (GOSCON) has moved to Washington D.C. this year to demonstrate how open source is fulfilling implementation of the Federal CIOs 5 Technology Pillars and allowing agencies to accomplish their mission goals in more secure, cost effective, and innovative ways."
LCA2010 Schedule is availableThe LCA2010 Organizers have announced the schedule of talks for linux.conf.au 2010. "A full schedule of talks is now available for LCA2010. The conference brings together speakers from around the world presenting a variety of topics; from the strongly technical such as Linux kernel development, to social interaction within communities and issues relating to diverse and minority groups. linux.conf.au 2010 runs for a full week starting Monday 18th January, with more than 65 talk sessions, together with a number of Keynote presentations and 14 Miniconfs."
LCA2010 announces three funding programsThree funding programs have been announced for LCA2010. "The three funding programmes - InternetNZ Oceania Programme, InternetNZ Kiwi Fellowship and Google Diversity Programme - have been created to assist those delegates contributing to the Open Source community who, without financial assistance, would not be able to attend LCA2010."
Linux Plumbers Conference schedule postedThe 2009 Linux Plumbers Conference has announced the the schedule of talks for the upcoming event. Christopher Marklund of Laika (the animation studio behind Coraline) has been added as a keynote. "If you're interested in attending LPC, please register right away, as we are very close to reaching our maximum capacity and will need to start turning away people soon."
Events: September 24, 2009 to November 23, 2009The following event listing is taken from the LWN.net Calendar.
If your event does not appear here, please tell us about it. Miscellaneous Linux Foundation to Microsoft: stop secretly attacking Linux (ars Technica)ars Technica discusses comments from the Linux Foundation's Jim Zemlin to Microsoft. "Zemlin's comments about Microsoft are a response to a patent auction carried out by Microsoft to sell a number of patents that the company allegedly said were related to Linux. The patents, which were originally obtained by Microsoft from SGI, were sold to Allied Security Trust (AST), a patent-holding group that grants its members perpetual licenses before reselling the patents. Zemlin suggests that Microsoft's intention was to surreptitiously slip the intellectual property to a patent troll that would then go after Linux companies."
Page editor: Forrest Cook
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||