User: Password:
|
|
Subscribe / Log in / New account

LWN.net Weekly Edition for May 2, 2013

LFCS: The Outreach Program for Women

By Nathan Willis
May 1, 2013

At the 2013 Linux Foundation Collaboration Summit, GNOME Foundation Executive Director Karen Sandler spoke about bringing more women into open source projects. GNOME has produced measurable results in recent years with its Outreach Program for Women (OPW), which has now expanded to include numerous other free software projects.

Sandler started off by reminding the audience why it is important to talk about the subject of increasing participation from women in open source. If you have not experienced incidents of sexism directly, she said, you are not necessarily aware of what it can be like. She noted that when she herself started out in software development, she did not want to talk about the issue. Instead, she said, she just wanted to concentrate on doing her own work. When she went to law school, however, the percentage of female participants was considerably greater, and once she returned to the free software community, she could not help but notice the distinctly different corporate environment.

At present, 25% of all software developers are women, Sandler said, and 18% of currently graduating computer science students are women, but only 3% of free software developers are women. However, the Google Summer of Code (GSoC) program experiences a distinctly higher percentage, 8.3% in 2012, which Sandler said reflects hard work being done on the GSoC team's part. Rather than diving into the motivators and reasons for open source software's low percentages, she said, she wanted to take a practical look at how GNOME has improved its own participation statistics through OPW.

In 2006, GNOME had 181 applicants to its GSoC projects, out of which zero were women. The project realized that it had a real problem, so it launched its Outreach Program For Women, which offers paid internships working with GNOME software projects. The program was "moderately successful" that year, Sandler said, but none of the women who participated as interns stuck around after their internships ended. In 2010, the program was relaunched, with the addition of Marina Zhurakhinskaya, who currently co-leads the program with Sandler.

This incarnation of the program started with an effort to look systematically at all of the reasons why women are not participating in the project. Sandler noted that in "every discussion about women in free software we're always saying 'well, I think the reason why they're not here is because ...'" followed by any one of a handful of reasons. Rather than so assuming the answer (or answers), however, the relaunched program systematically looked at all of the possible barriers to participation, and it tries to address each of them in turn.

The program runs two rounds every year, one concurrent with GSoC, which Sandler said inspired the overall mentoring and internship process (the other round begins in January and runs through April). OPW produces advertising that specifically targets women, which Sandler said produces better results than did more generic marketing materials. In fact, OPW periodically hears from an applicant that she did not know she could apply for GSoC (or thinks that she is underqualified for GSoC), and OPW sometimes helps applicants apply to GSoC instead, especially if the code project in question makes more sense as a GSoC internship.

Successes and moving beyond GNOME

The effort has produced measurable results. Prior to the relaunched OPW, GNOME's GSoC participation garnered zero or one female student each year (or "binary numbers," Sandler quipped), out of the 20 to 30 total GNOME students. But in 2011, seven of GNOME's 27 GSoC participants were women, followed by five of its 29 participants in GSoC 2012. Similarly, 4% of GUADEC attendees in 2009 were women; at GUADEC 2012, 17% of attendees were women.

The program proved successful enough that GNOME decided it should not keep the program a GNOME-only affair. Sandler first reached out to the Software Freedom Conservancy (SFC), and SFC member project Twisted agreed to run its own internship program in conjunction with OPW's on a test basis. The test went well, so GNOME concluded it that should expand OPW to include as many projects as possible.

The marketing and branding materials were revised to highlight free software in general, and a broader OPW was launched for the January 2013 round, with ten other software projects participating—such as Fedora, Tor, JBoss, Perl, and Mozilla.

How (and why) it works

Sandler attributes the success of OPW to the systematic approach it takes. The outreach addresses women directly, which has been shown to be more effective. It also accepts non-students and non-developers, which has attracted a more diverse set of applicants, and it connects the interns with mentors, which helps participants jump in and ask for help when they otherwise might not.

One of the differences between OPW and other internship programs is that OPW requires applicants to make a contribution (code or otherwise) with their application. Zhurakhinskaya came up with the idea, Sandler said, and it has two beneficial effects. First, not everyone who applies for an internship can be accepted, but helping applicants get connected with a project during the application process allows them to get involved and make contacts. Second, the contribution requirement gives the OPW program managers a far better idea of what the applicant is like than would filling out a form alone.

During the internships, the focus is on encouraging participants to take on manageable tasks that can land during the internship period; it is sometimes tempting to want to take on a large, ambitious contribution, Sandler said, but the downside is that not being able to deliver on such a project is not a positive experience. However, mentors are encouraged to stay in close contact with the interns, and can ramp up the project if the intern is getting things done. Interns are also required to post bi-weekly blog updates on their work, and the interns' posts are aggregated on Planet GNOME, which helps raise the profile of the program.

There is a specific IRC channel for OPW participants, plus regular meetings for interns. GNOME has historically tried to provide travel assistance for GUADEC to the interns, and is now building a specific travel stipend into program. All of these mechanisms help the interns build ongoing relationships with community members, Sandler said, which is important since (as the organizers recognize) building mentoring relationships is the most important aspect of the program. OPW makes a point of encouraging mentors to continue to keep in touch with their interns after the program ends, and of encouraging former interns to stay involved by (among other things) presenting at conferences.

A large number of interns have returned as mentors, Sandler observed, which is one of the best indicators of success. But even those interns who never return to open source software development leave the program with a better understanding of software freedom, the community that produces it, and of its potential to change the world, and they become advocates for the movement.

Similarly, implementing OPW has helped GNOME in ways that extend to participation by men as well as women. The process of finding mentors identifies people who will be good mentors for anyone, the process of identifying smaller tasks for OPW applicants' initial contributions helps the project find good starting points for all newcomers, and so on.

Up next

The next round of OPW internships is slated to begin in June (concurrent with GSoC); most applications are due on May 1, although applications for the kernel internships are due May 8. But there are many other ways to help get more women involved in open source software, Sandler said, such as the Ada Initiative, which runs the AdaCamp unconference for women in open source and open culture projects. The next AdaCamp is planned for June 8 and 9, in San Francisco. AdaCamp is invitation-only, she noted, so it is important to apply as soon as possible.

Some people feel like they must choose to support one initiative or another, Sandler added, but in reality all of the different efforts fit together, and they address different aspects of participation. She encouraged everyone to help out in some way, whether by spreading the word, inviting potential participants to apply, or by persuading your company or free software project to participate as a mentor. Participating in OPW as a mentoring organization requires just two things, she said: funding to pay the stipend for at least one intern, and mentors willing to work with newcomers. Zhurakhinskaya has set a tentative goal of 12% participation from women in GSoC 2013. That may be a little ambitious, Sandler said, but only by setting lofty goals can we get there.

Comments (5 posted)

A report from the Google Test Automation Conference

May 1, 2013

This article was contributed by David Röthlisberger

The focus of 2013's Google Test Automation Conference, held April 23 and 24 in New York City, was "Testing Media and Mobile". A major theme was WebDriver, which is an API for automating web browsers. Mozilla presented its work on WebDriver support in Gecko and extensions to WebDriver to allow automated testing of FirefoxOS beyond just the Gecko-powered content layer. Google talked about WebDriver support in Chrome/Chromium, including Chrome on Android. Others demonstrated FOSS software that re-purposes the WebDriver API for testing native mobile applications on Android and iOS. WebDriver seems to be gathering a lot of momentum in the world of automated user interface (UI) testing.

WebDriver

Selenium is a commonly used tool for web browser automation that is primarily used to automate UI-level tests for web pages and rich web applications. Selenium controls the browser by sending mouse and keyboard events, and gathers results by querying the state of particular document object model (DOM) elements. So, for example, a test script for an online contact-management application might click an "add contact" button (that loads a dialog via Ajax), then check that certain DOM elements have appeared, and that they contain a particular text. You target DOM elements using XPath locators. Selenium also comes with an IDE (implemented as a Firefox plugin) that "records" test scripts by watching the mouse and keyboard.

[Ken Kania]

This type of testing is not a replacement for unit testing or isolated component testing; but as Ken Kania pointed out in his talk [video], you still want some guarantees that what the user sees in the UI is correct.

Selenium initially relied on JavaScript run in the browser to simulate user interactions; but since version 2.0 (released in 2011) Selenium incorporates WebDriver. WebDriver fires events natively at the OS level, thus overcoming many limitations in the JavaScript-based approach.

WebDriver has three parts: A protocol (using JSON over HTTP and now a W3C working draft); a "local" implementation, which is a library used by the automated test scripts (implementations exist for a variety of programming languages); and a "remote" implementation, called the "browser driver" that is implemented as a browser plugin (Firefox, Safari) or as a separate executable that communicates with the browser's own debugging and inspection interfaces (Internet Explorer, Chrome, Opera). The Chrome and Opera implementations are officially supported by the respective browser vendor. There are also WebDriver implementations for mobile browsers. These are typically implemented as an app that uses a WebView object — i.e. the WebDriver app contains a browser in-process. These implementations extend the WebDriver protocol with mobile-specific APIs (finger gestures, screen orientation, etc.).

A WebDriver test script in Python might look like this (taken from a slide from Kania's talk):

    driver = webdriver.Chrome('chromedriver.exe')
    driver.get('http://www.google.com/xhtml')
    search_box = driver.find_element_by_name('q')
    search_box.send_keys('ChromeDriver')
    search_box.submit()
    self.assertTrue('ChromeDriver' in driver.title)
    driver.quit()

This script starts up a WebDriver "browser driver" targeting the Chrome browser (apparently on Windows). It then has the browser make an HTTP GET request, waits for the response (which contains an HTML form), and fills in and submits the form. Finally, the script does the actual test, which is to assert that the submitted text is present in the title of the HTML page sent as a response to the form submission. Real-world test frameworks would decouple the details of which particular browser is being tested (in this case, Chrome) from the test script itself, but this example gives the idea.

ChromeDriver

Kania, who is from the Google Chromium team, talked about Google's work on ChromeDriver (WebDriver for Chrome). ChromeDriver has been available for the desktop browser on Linux, OS X, and Windows for over a year; it is available in the Chromium source repository.

Recently ChromeDriver was re-architected into "ChromeDriver2" to support more platforms. Currently, the only addition is Android, and still at alpha-level quality; if you want support for Chrome on Chrome OS or iOS, "you have to be patient... or contribute", Kania said.

ChromeDriver is a stand-alone executable that speaks with WebDriver on one side, and communicates with the browser on the other side. Since ChromeDriver2, it uses the Chrome DevTools protocol to control the browser (see Kania's ChromeDriver architecture slide). Kania noted that Chrome is a multi-process browser, and the main browser process already uses this DevTools protocol to communicate with the renderer processes.

Michael Klepikov, from Google's "make the web faster" team, talked [video] about Google's work to make the profiling information from Chrome Developer Tools programmatically available to WebDriver scripts. This slide shows the specific API to enable this profiling. Klepikov demonstrated a test script that launches Chrome, goes to news.google.com, searches for "GTAC 2013", and switches to web (as opposed to news) results, all via the UI; it (presumably) also asserts that certain results are displayed. He demonstrated this test both on the desktop Chrome browser (video), and on Chrome running on a real Android phone (video). When the test completes, it prints the URL of a web server on localhost (powered by WebPagetest) serving an interactive display of the timing measurements, which can also be viewed within the Chrome Developer Tools interface.

Marionette: FirefoxOS meets WebDriver

Malini Das and David Burns from Mozilla presented [video] the work done on automated testing for FirefoxOS. The FirefoxOS runtime engine is based on Gecko (Firefox-the-browser's rendering engine), so the phone's user interface, as well as the bundled apps and third-party apps, are written using HTML5, JavaScript, and CSS.

[David Burns and Malini Das]

The existing Firefox WebDriver implementation (a browser plugin) works for the desktop browser, but not for FirefoxOS. So Mozilla has built WebDriver support directly into Gecko, under the name Marionette. Why WebDriver? The Firefox test suite contains thousands of Selenium WebDriver-based tests, and there are thousands of web developers already familiar with WebDriver.

Marionette allows full control of the phone (including telephony functions), and not just within an individual application. Marionette scripts can even control multiple devices at once (to test a call from one device to another, say). Mozilla has written client-side libraries (what the WebDriver spec calls the "local" side: the automated test script) for JavaScript and Python.

WebDriver isn't fully specified for gesture-based input, so Mozilla has added its own extensions. You can define gestures by chaining together actions (a "pinch" could be implemented as press-move-release with one finger, plus press-hold-release on another finger). Another Mozilla extension to WebDriver is the setContext function, to switch between the "content" and the privileged "chrome" contexts. Third party application developers only need to test at the "content" level, but testing FirefoxOS itself often requires access to privileged functions.

Marionette supports (non-UI) unit tests too, and can run the hundreds of thousands of tests that already exist for Firefox. Marionette can run these tests against Firefox on the desktop, the Boot to Gecko simulator, FirefoxOS device emulators, and real FirefoxOS devices. David demonstrated some geolocation unit tests running on an emulator, though there wasn't much to see other than the emulator booting up, plus some text output on a console. Unfortunately the demonstration of an actual UI test running on a real phone encountered technical difficulties. "Let me demo how quickly the phone resets", David said to much laughter.

Of course there are security concerns with allowing a phone to be controlled remotely. For now, Marionette is only available in engineering builds; Mozilla is working on a way to deploy it for third-party developers.

Appium: automation for native apps

Jonathan Lipps from Sauce Labs started his talk [video] with a complaint: while "mobile is taking over the world", test automation for mobile apps is extremely underdeveloped.

Lipps is the lead developer of Appium, a test automation framework for native (and hybrid) mobile apps; it is released under the Apache 2.0 license. Appium consists of a server that listens for WebDriver commands, and translates them into implementation-specific commands for the particular device under test (Android or iPhone, real device or emulator). Re-using the WebDriver protocol (originally intended for in-browser web applications) means that users can write tests in any existing WebDriver-based framework.

Appium extends WebDriver with mobile-specific features by hijacking WebDriver's executeScript command (you pass in a string starting with "mobile:" instead of actual JavaScript). Presumably a more official mechanism will be worked out eventually.

Behind the scenes, Appium uses the operating system's official automation frameworks: UI Automation and Instruments on iOS; uiautomator on Android. Appium also has an optional backend for the community project Selendroid which, unlike uiautomator, works on versions of Android prior to Jelly Bean.

Lipps demonstrated [video] Appium's brand-new support for FirefoxOS by running a test script against the FirefoxOS simulator. The test launched the Contacts app, asserted that Lipps's name wasn't present in the list of contacts, clicked the "add contact" button, added him as a contact, and finally asserted that the new contact was present in the list view. Lipps had added FirefoxOS support during the conference, since Burns and Das's talk the previous day, which really shows the advantages of open standards like WebDriver.

Android uiautomator

Guang Zhu (朱光) and Adam Momtaz from Google covered [video] in some detail the Android uiautomator framework. Much like Selenium WebDriver tests, a test for a mobile application needs a way to control the mobile device, and a way to receive feedback on the state of the device; Zhu provided a survey of the approaches taken by existing Android automation frameworks (slide 1, slide 2, slide 3). uiautomator uses the InputManager system service for control, and the Accessibility service for feedback. This means it will only work if the application's widgets meet accessibility guidelines — which will hopefully encourage more developers to make their apps accessible.

Set-top box testing with GStreamer and OpenCV

[David Röthlisberger] I (David Röthlisberger) demonstrated [video] how YouView uses GStreamer and OpenCV for video capture and image processing to run black-box automated tests against YouView's set-top box product. Using GStreamer's gst-launch command-line utility, I showed how easy it is to capture video and insert image-processing elements into the GStreamer media pipeline. (Incidentally, the ability to create a prototype so quickly using GStreamer's command-line tools was crucial to obtaining management buy-in for this project at YouView.)

Using these technologies, YouView built stb-tester, released publicly under the LGPL v2.1+. stb-tester scripts are written in Python, and have two primitive operations available: press to send an infrared signal to the system under test (pretending to be a human user pressing buttons on a remote control), and wait_for_match to search for a given image in the system-under-test's video output.

I also showed some examples of "model-based testing" built on top of stb-tester: First [video], a script that knows how to navigate within a menu arranged as two rows of thumbnail images (aka a double carousel), so instead of saying "press UP, then LEFT, etc.", the script says "navigate to entry X". Second [video], a state machine of YouView's "setup wizard", that allows test scripts to generate random walks through the possible user choices during the setup wizard.

In spite of the name, stb-tester can be used to test any consumer electronic device, not just a set-top box, as long as it outputs video and can be controlled via infrared. See stb-tester.com for documentation and introductory material.

Who's responsible for testing?

"Your developers — or worst case, test organization — produce tests."
Michael Klepikov [video]

Another major theme of the conference, which was more on the people and process side, was that the responsibility for testing is shifting from a separate test team to the developers themselves. In any organization following modern development practices, developers will write unit tests for their code; but end-to-end integration testing, performance testing, and security testing have traditionally been the domains of separate teams. Even Google has separate positions for "Software Engineer" and "Software Engineer in Test", and there seems to be a difference in status between those two positions — though judging from conversations with various Googlers, that difference is decreasing.

Claudio Criscione, who works on web security at Google, gave a talk [video] on an internal tool his team developed to find cross-site scripting vulnerabilities; he said they put a lot of effort into making the tool easy to use, so that development teams could run the security tests against their own applications, taking the load off the security team. Similarly, James Waldrop pointed out in his talk [video] that his performance testing team at Twitter can't possibly scale up to test all the systems being produced by the various development teams, so instead his team provides tools to make it easy for the developers to write and run performance tests themselves. Specifically, he presented Iago, which is a load generator developed at Twitter and released under the Apache 2.0 license.

Simon Stewart, who created WebDriver, leads the Selenium project, and now works at Facebook, said in his talk [video] that Facebook has no such thing as a "Software Engineer in Test", only "Software Engineers". Furthermore, Facebook has no test or QA departments. The developers are responsible for testing.

About the conference

The conference is in its 7th year (though it skipped a year in 2012). Attendance is free of charge, but by invitation and limited to around 200 technical participants; applications are announced each year on the Google Testing Blog. Live streaming was available to the public, along with a Google Moderator system for the remote audience to submit questions to the speakers.

It was a very polished conference — you can tell Google has done this before. It was held in Google's New York City office; meals were catered by Google's famous cafeteria. A sign-language interpreter was on site, and stenographers provided live transcripts of the talks. Videos and slides are available here.

Comments (5 posted)

LFCS: The value of FOSS fiscal sponsorship

April 30, 2013

This article was contributed by Martin Michlmayr

As open source becomes more popular and mature, questions of formalizing the governance and corporate structures of projects are becoming of increasing importance, as can been seen by the rising visibility of various FOSS foundations. At the Linux Foundation Collaboration Summit in San Francisco, Tony Sebro shared his insights about the value that fiscal sponsors bring as umbrella organizations for FOSS projects. Sebro is the General Counsel of Software Freedom Conservancy, which is the home of about 30 free and open source projects, including Samba, Git, and BusyBox.

[Tony Sebro]

Sebro kicked off his talk (slides [PDF]) by explaining some of the problems that unincorporated open source projects face. The Internet makes it easy for developers to get together and collaborate on a project. Unfortunately, there are several potential downsides in such a loose organization, especially when a project grows larger. While individuals and corporations can contribute time and code, it's difficult for unincorporated projects to raise and manage funds. Of course, many projects use PayPal, Flattr, and other means to obtain funding, but an unincorporated project has no central place to hold and manage the assets. What happens in practice is that the project founder or a similarly trusted developer will hold these assets.

Fortunately, this arrangement works out in most cases, but it can lead to severe problems in the case of a dispute—the person who registered the domain name owns it, rather than the project, and it's unclear who holds trademark rights. Sebro noted that it's possible to hold common-law trademark rights even if a trademark has not formally been registered. This is something few developers are aware of and it can lead to uncertainties as to the ownership if there is a dispute. More generally, unincorporated projects suffer from a lack of administrative support and expertise. Most developers want to write code and tend not to worry about administrative tasks.

Sebro further explained that developers working on an unincorporated project face potential risk and legal exposure. For example, if a project intends to organize a conference they may have to sign a contract with a venue. Since an unincorporated project cannot sign contracts, one of the developers would have to do so. The person who signed the contract would be on the hook if there were any serious problems, for example if the project had to cancel the conference due to lack of registrations or unforeseen events. Finally, Sebro argued that there's little insulation from corporate influence. An unincorporated project could be taken over by a company quite easily by employing the core developers or by spending significant engineering resources on the project.

Advantages of forming a legal entity

The solution for these problems, according to Sebro, is to provide a corporate home for the project. For most projects, a 501(c)(3) non-profit is the most appropriate structure, he said. 501(c)(3) refers to the US tax code that defines a charity. A 501(c)(3) charity can receive charitable donations on behalf of the project, manage and spend funds according to its mission, and enter into contracts. A non-profit can also provide administrative support, such as bookkeeping, legal services, and technical infrastructure for the project. The non-profit can act as the central place to hold project assets, so there are no uncertainties as to the ownership of trademarks, domain names, and physical assets (such as servers or t-shirts and other merchandise).

Sebro suggested that a non-profit may potentially limit the exposure to risk for individual developers. He cited the US Volunteer Protection Act, which limits the liability of volunteers acting on behalf of a public charity. Sebro noted that developers would have to act under the auspices of the charity in order to benefit from the protection and they cannot be grossly negligent. While Sebro seemed fairly confident that the Volunteer Protection Act would limit the risk for developers, "this has not been tested", he said. However, a non-profit home can certainly protect individual developers from risk associated with contracts, conferences, and similar activities.

Another advantage of a formal structure is that it insulates a project from corporate influence. A 501(c)(3) charity must work toward meeting its charitable mission and must act in the interests of the public. A charity may not tailor their activities toward one or more individuals or corporations. Furthermore, while corporations can make donations, they cannot dictate how their donations are to be used. Answering a question from the audience, Sebro explained that this is a fine line to draw. He said that a company cannot donate to a non-profit in order to get a specific feature implemented that is of primary interest to the company, as that would provide a direct benefit to the company. However, it can certainly make donations so the non-profit can add specific features on its roadmap—even if the company benefits from these features. At this point, Sebro briefly explained the difference between 501(c)(3) organizations, like Software in the Public Interest or Software Freedom Conservancy, and 501(c)(6) organizations, like the Linux Foundation and the Eclipse Foundation. While the former is a charity which has to act in the interest of the public, the latter is a trade association—companies working together to further a common goal. According to Sebro, a 501(c)(6) insulates the group from imbalanced corporate influence, as the organization has to balance the interests of competing members.

(Don't) do it yourself

Having explained the advantages of incorporating as a 501(c)(3) organization, Sebro explained different ways that projects can go about obtaining the benefits of such a structure. The one that may seem most obvious is to create your own. The process in the US would be to create a corporate and governance structure, such as officers and a board of directors, and to incorporate at the state level. The tricky part, however, is to attain 501(c)(3) status, which is required so that donors get tax deductions for their contributions. Unfortunately, new applications for 501(c)(3) undergo close scrutiny where the mission involves free and open source software, he said. Any newly formed FOSS organization will be handicapped, according to Sebro, as it will be difficult to obtain 501(c)(3) status.

[Cost/benefits]

A new organization would also have to answer many questions regarding its service plan: would it handle bookkeeping and legal services in-house or via a third party, and most importantly, does it have enough funding to cover that overhead? Summarizing pros and cons, Sebro said that the organization would benefit from the single brand and complete autonomy that comes from having its own non-profit. However, it comes with significant drawbacks—high overhead, a lack of clarity about who is actually going to do the work and provide the required business and legal skills, and the major uncertainty regarding attaining 501(c)(3) status.

The alternative solution is to create a relationship with an existing FOSS foundation that can act as a fiscal sponsor. It's important to note that a fiscal sponsor does not sponsor a project through monetary contributions. Its function is to provide a legal structure and environment in which projects can operate and raise funds. Many fiscal sponsors also offer a range of services to projects, typically for a fee or by taking a certain percentage from a project's donations. Sebro's talk explained two types of fiscal sponsors: comprehensive fiscal sponsors and grantor/grantee fiscal sponsors.

Comprehensive fiscal sponsors

When a project joins a comprehensive fiscal sponsor, they sign a fiscal sponsorship agreement (FSA) and become fully integrated into the sponsor—in a sense, it's like a merger and acquisition in which the fiscal sponsor takes over the open source project. The fiscal sponsor supervises the project to ensure that it follows rules of the IRS (Internal Revenue Service, the US tax agency) and is in compliance with the mission of the fiscal sponsor (which is broadly defined to encompass many projects). In the case of Software Freedom Conservancy, technical decisions and questions of infrastructure are decided by the project. The Apache Software Foundation (ASF), on the other hand, dictates parts of the development process (the Apache Way). In return for joining a comprehensive fiscal sponsor, projects will benefit from the service portfolio offered by the organization. This can include legal services, bookkeeping, conference organization, and more. Software Freedom Conservancy, for example, offers a service plan from which members can choose à la carte. Furthermore, developers are shielded from risk because they are acting as volunteers for the non-profit.

There are a number of advantages of this type of structure: overhead is shared, the fiscal sponsor has 501(c)(3) status already, and it is likely to have a lot of experience running a non-profit so it can offer reliable services and solid advice. A drawback is a certain loss of autonomy: since a fiscal sponsor is responsible for several projects, they are likely to be more conservative than a single project may be on its own. Another risk is that pooled resources imply pooled risk: if member A owes funds, would the assets of members B and C be at risk? Sebro cited the Charitable Trust Doctrine which says that donations have to be spent for the declared charitable purposes under which the donations were solicited. Spending member B's resources to pay for debts caused by member A may be in conflict with this doctrine, which could therefore provide some protection. Sebro mentioned a case that held "New York's long-standing policy honoring donors' restrictions on the use of the property they donate has greater weight than the claims of creditors." While there is some uncertainty in this area, Sebro argued that the best way to manage this problem is through good governance.

Grantor/grantee fiscal sponsors

An alternative arrangement is for projects to enter into a grantor/grantee relationship with a fiscal sponsor. This model is more lightweight: projects don't join the fiscal sponsor but remain autonomous. Sponsors can receive and manage funds on behalf of the project and provide these funds to the project in form of a grant. The sponsoring organization has a duty to ensure that the funds are spent in an appropriate way. In a sense, the fiscal sponsor acts as a bank for the project, but it can also offer a range of services to projects. Software in the Public Interest (SPI) follows this model.

This structure also allows the sharing of overhead burden and provides administrative functions to the project. While the project can stay autonomous, it won't benefit from potential risk protection at it would have if the work was done under the umbrella of a non-profit. The project and organization also have to be careful that grants are spent according to the purpose of the grant. If a fiscal sponsor takes money and gives it to a project without supervision on how it is being spent, it may run into problems with the IRS.

Advantages for developers and corporations

Sebro summarized the advantages of joining an existing non-profit foundation: it's a shortcut to benefit from 501(c)(3) status, there are economies of scale and lower administrative costs, and the administration can be outsourced to people who have experience running a non-profit organization.

In addition to these direct benefits, projects may also find it easier to obtain funding and other contributions from companies. Compared to an unincorporated entity, a 501(c)(3) fiscal sponsor provides better governance, better structure, and mentorship. In other words, it provides stability to an open source project, which is something corporations are seeking before they invest in a project. Non-profits also provide a neutral territory: a company is not able to control how the project is operating but neither can its competitors. However, corporations can still "vote" by allocating employee time: if a project has different sub-projects or tasks, it can decide which areas to focus on.

Role and sustainability of fiscal sponsors

Sebro explained that there are different roles that fiscal sponsors can play for projects. He asked whether a fiscal sponsor should be seen as a playpen, an apartment, or a nursing home. In fact, these roles are not mutually exclusive and a fiscal sponsor can play all of them. In the playpen role, the fiscal sponsor essentially provides an environment for incubation until a project reaches a certain maturity and decides to go its own way. Sebro cited former Conservancy member jQuery which decided to form its own trade association. Many projects view a fiscal sponsor as an apartment—a long term living solution. Sebro mentioned that a comprehensive fiscal sponsor could hire staff for a specific project, although Conservancy is far away from being able to do that. Finally, a nursing home, or mausoleum, can provide a home for projects that are essentially dying off. There is some value in that context as the copyright and other assets can be assigned to the fiscal sponsor. This makes it easier to the revive the project or relicense its code in the future.

Finally, Sebro raised the question of sustainability. Fiscal sponsors usually take a cut from donations to member projects in order to fund their operations (Conservancy: 10%; SPI: 5%). In the case of Conservancy, this is not sufficient to cover its operations and it relies on donations to the general fund from corporations and individuals. Sebro mentioned examples from other areas (e.g. the arts) where fiscal sponsors have achieved a sustainable model, but many FOSS fiscal sponsors still have a long way to go in that respect.

Comments (10 posted)

A goodbye note from Michael Kerrisk

May 1, 2013

This article was contributed by Michael Kerrisk.

I'm stepping back from full-time writing at LWN to focus on a number of other projects. Writing for LWN has been an exhilarating ride where I've learned a lot. I'd like to thank all readers for their interest in my articles and their often valuable comments. Even more, I'd like to thank my colleagues at LWN. Their thorough and insightful input made each and every one of my articles better—in some cases much better—than it would it would have otherwise been. I've thus gained a first hand understanding of why I have appreciated LWN so much as a reader. Although I step back from working on LWN with some sadness, I won't be disappearing from these pages altogether: I still plan to contribute occasional articles in the future as time and interests permit.

Comments (11 posted)

Page editor: Jonathan Corbet

Security

Code authenticity checking

By Nathan Willis
May 1, 2013

Cryptographically signed binary packages are mainstay of modern Linux distributions, used for verifying packages against both tampering and accidental corruption. But as ever fewer users need to compile their software from source, it is possible to forget that verifying the authenticity and integrity of binary packages depends first on the packagers' ability to verify the upstream source code. In April, Allan McRae posted an entry on his blog that looked at the verifiability of the source code releases made by a number of well-known projects, with less-than-perfect results.

McRae is a core Arch Linux developer, and he undertook his verifiability survey because Arch's packaging tool makepkg recently gained support for checking the PGP signatures of source files. Naturally, such verification is only possible when the source bundle includes a PGP signature, but McRae also looked at four other interrelated factors: whether or not the signing key can be easily verified as authentic, whether checksums for the source files are available, whether the checksums are signed, and whether the checksums are available on a different server than the one hosting the source files. This last criterion is a guard against a server compromise; if an attacker can replace the source files, he or she can also replace the checksum with one that matches the replacement source.

To provide a meaningful sample population for his survey, McRae looked at 63 "core" packages common to most Linux distributions. "For me, that basically means the packages required to build a fairly minimal booting system. This is essentially the package list from Linux From Scratch with a few additions that I see as needed …" His results are presented in a color-coded table; green cells indicate that a package meets all of the verification criteria, red indicates that no verification criteria pass, and yellow indicate some but not all criteria are met. Of the 63 packages examined, ten offered no means of verification and 14 others offered only partial verification.

The packages offering no verification mechanism are file, flex, iana-etc, isl, kbd, libarchive, pkg-config, procps, psmisc, and which. Those packages that partially meet the measured criteria fall into two basic categories, those that provide PGP signatures but have a difficult-to-verify key (gawk, groff, patch, sed, sudo, sysvinit, texinfo, and xz), and those that provide their checksum data on the same server as their files (bzip2, perl, tzdata, and zlib). In addition, gmp and less fell somewhere in between, providing a PGP signature, but making the public key or key ID available only on the same server as the source release.

To be sure, McRae's specific criteria and red/yellow/green assessments draw some rather arbitrary lines—as he observes, several of the projects have other means of verification available, and he admits that the definition of a "readily verifiable" key includes keys signed by keys that he trusts. But the aggregate picture is the important one: most of the packages are green (which is good news), but roughly 15% of them offer no source verification whatsoever (which is far from good). He also notes that the picture seems to rapidly deteriorate "as you move further away from this core subset of packages needed for a fairly standard Linux system."

Best practices

McRae's post was picked up on the oss-security mailing list, where the talk turned to how to establish a set of common guidelines for releasing source code with verifiable authenticity. Alan Coopersmith commented that X.org has received complaints asking it to do more, but without concrete suggestions. "If there was a common standard, with instructions, we'd be far more likely to spend the time to adopt it, than just a 'make signatures appear somewhere, in an unspecified format'." Eric H. Christensen concurred, saying he was interested in establishing a "best practices" recommendation for Red Hat—but, he asked, what really constitutes the best way to disseminate releases? A recommendation would advise against using MD5 for checksumming, he said, and although he favors PGP for signatures, perhaps it has its drawbacks as well.

Indeed, Alistair Crooks replied with a lengthy list of questions one might ask about a PGP-signed release, addressing everything from the key management practices employed by the signing entity to the options specified when generating the key itself (such as whether it is an RSA or DSA key and whether or not it requires a passphrase). A PGP signature proves only that a person with access to the signing key attests that the signed message had a particular value at a particular time, he argued, which does not provide much authentication:

So, all in all, what you have is a digest, signed by someone who knows the key, or who has access to the creds (if any) for the key, or who has found out the key creds, albeit with timestamp info for when the signature took place.

I'm not sure what using PGP gains us?

But the majority seemed to feel that PGP in fact provides quite a few gains. Nicolas Vigier and Florian Weimer both commented that key-continuity over multiple releases safeguards against a man-in-the-middle attacker replacing a release. Weimer noted that "hosting sites have been compromised, or serve their content exclusively over a mirror network which literally anyone can join." Kurt Seifried agreed, but acknowledged that the "real problem" with PGP is the cost of implementing it:

Key creation/storage/management/backup/etc is all non trivial and not free. Is the cost of this worth it?

I think if we are going to push this we need to come up with a pretty good set of guidelines that are easy to follow and implement.

Daniel Kahn Gillmor responded that even a simple "the release manager has the OpenPGP key for the project on the keyring of her personal development machine" workflow raises the bar for would-be attackers and would constitute an improvement for the numerous projects that do not currently sign their releases at all. But he still advocated producing guidelines:

I don't want us to spec out a byzantine ruleset that would put people off from *starting* to sign their releases. Maybe such a policy could break out the sophisticated stuff into the form of "baseline", "level 1", "level 2", etc.

That way we could encourage all projects to get to the "baseline" (which should be short and simple) without requiring them to "level up" right away (to offline key storage, key transition statements, etc).

He pointed to the existing OpenPGP best practices page at the Debian Grimoire wiki as an example.

For his part, McRae had comparatively simple goals in mind:

Despite PGPs limitations, what I really like to see when a release is made is a PGP signed release announcement email to the relevant mailing list with decent checksums and the file size in it. Bonus points if that email gets mirrored or at least archived on a different server than the source code. I figure for most open source software, a false release email would be spotted fairly quickly...

Identity cleft

Although a general consensus developed around the idea of crafting a "best practices" recommendation, Crooks's questions about the limitations of PGP signatures raised some valuable points—such as the importance of distinguishing between identity and trust. Some mistook his original email for a call to ditch PGP signatures on source releases, since they do not offer absolute security, but Crooks said that was a misinterpretation. "It's a bit disappointing that my advice (in pointing out ways that PGP can be worked around in order to diminish integrity and security) was categorised as an attack on PGP itself - I shall take that as a reminder that I should be more clear in what I write." Rather, he said, he hoped to "warn against the magic 'it's signed, so it's gospel' myth by pointing out the problems of key management."

The crux of Crooks's argument is that a PGP signature should not be trusted just because it is associated with a known identity (person or team of developers). As he elaborated in one message in the ensuing thread:

I don't know if you've ever done one of the key signing parties, where you get handed government id, and that is supposed to define someone's identity. It tells age, name, and ability to keep a dead pan face in front of a camera. It says nothing about how trust-worthy someone is, in the sense that I would compile/run software written by them.

Elsewhere in the same message, he noted:

I know lots of people who write software. Some of their personal lives are train wrecks. Some I wouldn't trust to sit the right way on a toilet seat. But, for various reasons, such as mentoring, peer-programming, peer review, stringent regression tests, personal audits of their work, or because of random audits, etc, I would trust the software they write.

In essence, this is an argument that the signer only earns trust by demonstrating their reliability over time. For individuals, the issue might be the quality of the software (as Crooks discussed above), but to trust the signature on a project's releases, more stringent requirements may be necessary. As he added later: "You actually know very little about the key before and after the signing took place; so you have no way of ascertaining whether the key has been used to sign other things fraudulently."

Add in Crooks's earlier questions about the key options and security on the machines used to sign releases, and proper key management, it would seem, remains an area where there is still work to be done. Crooks did reiterate his support for encouraging PGP signatures on all source releases; he simply cautioned that blind trust in the presence of a signature can lead to a false sense of security.

Then again, a campaign to persuade more upstream software projects to integrate easily-verified PGP signatures and out-of-band checksums into their release process has to start somewhere. There are practical challenges to consider, such as the role that popular hosting providers play in the build and release processes (Stuart Henderson noted that Github and Bitbucket dynamically generate tar archives, which complicates signing releases).

But convincing individual developers and release managers might be the best way to convince hosting services to adapt. Presumably few people enjoy seeing their project marked with the red "No verification available" label. It would certainly be informative to conduct a large-scale examination of McRae's criteria on other popular open source projects. Just over 38% of the core packages McRae examined could not be "readily validated"—a sizable minority, but a minority nonetheless; that gives one hope that the community in general takes source verifiability as a matter worth addressing.

Comments (11 posted)

Brief items

Security quotes of the week

Chase is committed to making your banking experience enjoyable, trouble-free, and, above all, safe. Which is why you should strike your computer with 20 to 25 forceful blows from a pipe wrench as soon as you reach international waters, toss the plastic and metal shards into the sea, and then immediately sink the ship you’re on. And then, once you dive to the sea floor, grab the scattered computer pieces, and shove them all inside living clams, you’ll be able to rest easy knowing you’re banking smarter and safer.
The Onion (hat tip to Don Marti)

Patent trolls know this and as a result, they sue companies in droves and make settlement demands designed to maximize their financial take while making it cheaper and less painful to settle than to devote the resources necessary to defeat their claims. The current system lets them do so even with claims that are unlikely to prevail on the merits. That is because, whether win lose or draw, the rules effectively insulate trolls from negative consequences except perhaps a lower return than expected from any given company in any given case. They can sue on tenuous claims and still come out ahead. And so the broken system with its attendant leverage allows trolls to extract billions in blackmail from U.S. companies and, in the final analysis, consumers.
Barnes & Noble pulls no punches (by way of Groklaw)

In the aftermath of the Boston bombings -- cameras were everywhere there -- which while horrendous and tragic, killed and injured fewer people than just a few days of "routine" gun violence here in the USA, we're hearing the predictable calls for vastly expanded government-operated video surveillance networks, even though virtually every study shows that while these systems may be useful in solving crimes after the fact, they are of little to no use in preventing crime or terrorism in the first place. This has proven true even in cities like London, where there's a camera focused on pretty much every individual pimple on each Londoner's face.

In some cities, like New York, the surveillance-industrial complex has its fangs deeply into government for the big bucks. It's there we heard the Police Commissioner -- just hours ago, really -- claim that "privacy is off the table."

Lauren Weinstein

Comments (15 posted)

Mozilla: Protecting our brand from a global spyware provider

The Mozilla blog reports that Mozilla is using its trademarks to back up a cease-and-desist letter to Gamma International, the maker of the infamous FinFisher surveillance system. "We cannot abide a software company using our name to disguise online surveillance tools that can be – and in several cases actually have been – used by Gamma’s customers to violate citizens’ human rights and online privacy."

Comments (1 posted)

New vulnerabilities

clamav: multiple vulnerabilities

Package(s):clamav CVE #(s):CVE-2013-2020 CVE-2013-2021
Created:April 30, 2013 Updated:June 21, 2013
Description: From the clamav bug tracker entries [1, 2]:

Bug was in handling of UPX-packed executables. When checking for a skewed offset section, it would not rule out a possible skew value that was larger than the size of the PE section. Because of unsigned math, it would cause the resulting expected section size to rollover to exceedingly large values and try inappropriate heap reads.

Problem was in handling of encrypted PDF's. A specially crafted PDF could set a user-controlled length which is then used to allocate heap memory and then read length bytes off of the stack, potentially beyond the size of a static array.

Alerts:
SUSE SUSE-SU-2014:1571-1 clamav 2014-12-05
Gentoo 201405-08 clamav 2014-05-16
Fedora FEDORA-2013-10953 clamav 2013-06-21
Fedora FEDORA-2013-10980 clamav 2013-06-21
openSUSE openSUSE-SU-2013:0813-2 clamav 2013-05-21
openSUSE openSUSE-SU-2013:0813-1 clamav 2013-05-21
Fedora FEDORA-2013-8047 clamav 2013-05-16
Ubuntu USN-1816-1 clamav 2013-05-03
Mageia MGASA-2013-0132 clamav 2013-05-02
Mandriva MDVSA-2013:159 clamav 2013-04-30
openSUSE openSUSE-SU-2013:0883-1 clamav 2013-06-10
openSUSE openSUSE-SU-2013:0881-1 clamav 2013-06-10

Comments (none posted)

glibc: denial of service

Package(s):glibc CVE #(s):CVE-2013-1914
Created:April 25, 2013 Updated:August 22, 2013
Description:

From the Red Hat advisory:

It was found that getaddrinfo() did not limit the amount of stack memory used during name resolution. An attacker able to make an application resolve an attacker-controlled hostname or IP address could possibly cause the application to exhaust all stack memory and crash. (CVE-2013-1914)

Alerts:
Debian-LTS DLA-165-1 eglibc 2015-03-06
Gentoo 201503-04 glibc 2015-03-08
Scientific Linux SLSA-2013:1605-2 glibc 2013-12-03
Mageia MGASA-2013-0340 glibc 2013-11-22
Oracle ELSA-2013-1605 glibc 2013-11-26
Mandriva MDVSA-2013:284 glibc 2013-11-25
Mandriva MDVSA-2013:283 glibc 2013-11-25
Red Hat RHSA-2013:1605-02 glibc 2013-11-21
Ubuntu USN-1991-1 eglibc 2013-10-21
openSUSE openSUSE-SU-2013:1510-1 glibc 2013-09-30
Fedora FEDORA-2013-15053 glibc 2013-08-22
Mageia MGASA-2013-0141 glibc 2013-05-09
Mandriva MDVSA-2013:163 glibc 2013-05-07
Mandriva MDVSA-2013:162 glibc 2013-05-07
Scientific Linux SL-glib-20130425 glibc 2013-04-25
Oracle ELSA-2013-0769 glibc 2013-04-25
CentOS CESA-2013:0769 glibc 2013-04-24
Red Hat RHSA-2013:0769-01 glibc 2013-04-24

Comments (none posted)

kernel: multiple vulnerabilities

Package(s):kernel CVE #(s):CVE-2013-3228 CVE-2013-3230 CVE-2013-3231 CVE-2013-3232 CVE-2013-3233 CVE-2013-3234 CVE-2013-3076 CVE-2013-3223 CVE-2013-3225 CVE-2013-1979 CVE-2013-3224 CVE-2013-3222
Created:April 29, 2013 Updated:July 19, 2013
Description: From the Red Hat bugzilla entries:

[CVE-2013-3228]: Linux kernel built with the Asynchronous Transfer Mode(ATM) support is vulnerable to an information leakage flaw. It occurs when doing recvmsg(2) calls on the ATM PVC & SVC sockets.

A user/program could use this flaw to leak kernel memory bytes.

[CVE-2013-3224]: Linux kernel built with Bluetooth networking support is vulnerable to an information leakage flaw. This occurs while receiving messages via recvmsg call.

A user/program could use this flaw to leak kernel memory bytes.

[CVE-2013-1979]: Commit 257b5358b32f ("scm: Capture the full credentials of the scm sender") changed the credentials passing code to pass in the effective uid/gid instead of the real uid/gid.

Obviously this doesn't matter most of the time (since normally they are the same), but it results in differences for suid binaries when the wrong uid/gid ends up being used.

An unprivileged local user could use this flaw to elevate their privileges.

[CVE-2013-3225]: Linux kernel built with Bluetooth networking with RFCOMM protocol support is vulnerable to an information leakage flaw. This occurs while receiving messages via recvmsg call.

A user/program could use this flaw to leak kernel memory bytes.

[CVE-2013-3223]: Linux kernel built with the Amateur Radio support(CONFIG_HAMRADIO) along with the Amateur Radio AX.25 Level 2 protocol support enabled is vulnerable to an information leakage flaw. It occurs while receiving messages via recvmsg(2) call.

A user/program could use this flaw to leak kernel memory bytes.

[CVE-2013-3076]: Linux kernel built with the User-space interface for hash algorithms (CONFIG_CRYPTO_USER_API_HASH) and User-space interface for symmetric key cipher algorithms (CONFIG_CRYPTO_USER_API_SKCIPHER) support is vulnerable to an information leakage flaw. It occurs while receiving messages via recvmsg(2) call.

A user/program could use this flaw to leak kernel memory bytes.

[CVE-2013-3234]: Linux kernel built with the ROSE virtual network devices (CONFIG_ROSE) support is vulnerable to an information leakage flaw. It occurs while receiving messages via recvmsg(2) call.

A user/program could use this flaw to leak kernel memory bytes.

[CVE-2013-3233]: Linux kernel built with the Near Field Communication(CONFIG_NFC) subsystem along with the Logical Link Control Protocol(CONFIG_NFC_LLCP) support is vulnerable to an information leakage flaw. It occurs while receiving messages via recvmsg(2) call.

A user/program could use this flaw to leak kernel memory bytes.

[CVE-2013-3232]: Linux kernel built with the NETROM virtual network devices (CONFIG_NETROM) support are vulnerable to an information leakage flaw. It occurs while receiving messages via recvmsg(2) socket call.

A user/program could use this flaw to leak kernel memory bytes.

[CVE-2013-3231]: Linux kernel built with Logical Link layer Control(CONFIG_LLC) protocol support is vulnerable to an information leakage flaw. It occurs while receiving messages via recvmsg(2) socket call.

A user/program could use this flaw to leak kernel memory bytes.

[CVE-2013-3230]: Linux kernel built with the Layer Two Tunneling Protocol (CONFIG_L2TP) support is vulnerable to an information leakage flaw. It occurs while receiving messages via recvmsg(2) call.

A user/program could use this flaw to leak kernel memory bytes.

[CVE-2013-3228]: Linux kernel built with the The Infrared Data Associations (CONFIG_IRDA) protocol is vulnerable to an information leakage flaw. It occurs while receiving messages via recvmsg(2) socket call.

A user/program could use this flaw to leak kernel memory bytes.

Alerts:
SUSE SUSE-SU-2014:1316-1 Linux kernel 2014-10-22
SUSE SUSE-SU-2014:1319-1 Linux kernel 2014-10-23
SUSE SUSE-SU-2014:0536-1 Linux kernel 2014-04-16
openSUSE openSUSE-SU-2013:1971-1 kernel 2013-12-30
Scientific Linux SLSA-2013:1645-2 kernel 2013-12-16
Oracle ELSA-2013-2584 kernel 2013-11-28
Oracle ELSA-2013-2584 kernel 2013-11-28
Oracle ELSA-2013-2585 kernel 2013-11-28
Oracle ELSA-2013-2585 kernel 2013-11-28
Red Hat RHSA-2013:1645-02 kernel 2013-11-21
Oracle ELSA-2013-1645 kernel 2013-11-26
Oracle ELSA-2013-2538 kernel 2013-07-18
Oracle ELSA-2013-2538 kernel 2013-07-18
Oracle ELSA-2013-2537 kernel 2013-07-18
Oracle ELSA-2013-2537 kernel 2013-07-18
Scientific Linux SL-kern-20130717 kernel 2013-07-17
Oracle ELSA-2013-1051 kernel 2013-07-16
CentOS CESA-2013:1051 kernel 2013-07-17
Red Hat RHSA-2013:1080-01 kernel 2013-07-16
Red Hat RHSA-2013:1051-01 kernel 2013-07-16
Oracle ELSA-2013-1034 kernel 2013-07-10
CentOS CESA-2013:1034 kernel 2013-07-10
SUSE SUSE-SU-2013:1182-2 Linux kernel 2013-07-12
Scientific Linux SL-kern-20130710 kernel 2013-07-10
Red Hat RHSA-2013:1034-01 kernel 2013-07-10
Oracle ELSA-2013-2546 enterprise kernel 2013-09-17
Mandriva MDVSA-2013:176 kernel 2013-06-24
Oracle ELSA-2013-2546 enterprise kernel 2013-09-17
SUSE SUSE-SU-2013:1022-3 Linux kernel 2013-06-18
SUSE SUSE-SU-2013:1022-2 Linux kernel 2013-06-17
SUSE SUSE-SU-2013:1022-1 kernel 2013-06-17
Ubuntu USN-1883-1 linux-ti-omap4 2013-06-14
Ubuntu USN-1882-1 linux-ti-omap4 2013-06-14
Ubuntu USN-1881-1 linux 2013-06-14
Ubuntu USN-1880-1 linux-lts-quantal 2013-06-14
Ubuntu USN-1879-1 linux-ti-omap4 2013-06-14
Ubuntu USN-1878-1 linux 2013-06-14
Ubuntu USN-1877-1 linux-ec2 2013-06-14
Ubuntu USN-1876-1 linux 2013-06-14
Oracle ELSA-2013-2525 kernel 2013-06-13
Oracle ELSA-2013-2525 kernel 2013-06-13
Ubuntu USN-1839-1 linux-ti-omap4 2013-05-28
Ubuntu USN-1833-1 linux 2013-05-24
Ubuntu USN-1837-1 linux 2013-05-24
Red Hat RHSA-2013:0829-01 kernel-rt 2013-05-20
Mageia MGASA-2013-01451 kernel-vserver 2013-05-17
Mageia MGASA-2013-0150 kernel-rt 2013-05-17
Mageia MGASA-2013-0149 kernel-tmb 2013-05-17
Mageia MGASA-2013-0148 kernel-linus 2013-05-17
Mageia MGASA-2013-0147 kernel 2013-05-17
Debian DSA-2669-1 linux 2013-05-15
Debian DSA-2668-1 linux-2.6 2013-05-14
Fedora FEDORA-2013-6999 kernel 2013-05-04
Ubuntu USN-1815-1 kernel 2013-05-02
Fedora FEDORA-2013-6537 kernel 2013-04-27

Comments (none posted)

libxml2: multiple unspecified vulnerabilities

Package(s):libxml2 CVE #(s):CVE-2013-1969
Created:April 25, 2013 Updated:June 11, 2013
Description: The libxml2 2.9.1 release apparently fixes multiple vulnerabilities, but nobody is saying exactly what they are.

Thanks to David Walser, we now know that one of the vulnerabilities mentioned by Fedora is a "use after free" that has been assigned CVE-2013-1969.

Alerts:
Gentoo 201412-11 emul-linux-x86-baselibs 2014-12-11
Gentoo 201311-06 libxml2 2013-11-10
Ubuntu USN-1817-1 libxml2 2013-05-07
openSUSE openSUSE-SU-2013:0729-1 libxml2 2013-04-30
Fedora FEDORA-2013-6147 libxml2 2013-04-25
Fedora FEDORA-2013-6110 libxml2 2013-04-25
openSUSE openSUSE-SU-2013:0945-1 libxml2 2013-06-10

Comments (none posted)

mantis: two vulnerabilities

Package(s):mantis CVE #(s):CVE-2013-1930 CVE-2013-1931
Created:April 25, 2013 Updated:August 5, 2013
Description:

From the Red Hat bugzilla entries [1, 2]:

CVE-2013-1930: A security flaw was found in the way MantisBT, a web-based issue tracking system, performed user-privilege check when displaying issue close button (close button was previously shown at a web page even when 'close' was not a valid status by / according to workflow definition). An unprivileged MantisBT user could use this flaw to close particular issue even though the particular workflow settings did not permit it.

CVE-2013-1931: A cross-site scripting (XSS) flaw was found in the way MantisBT, a web-based issue tracking system, sanitized content of the version name, when deleting a project version. A remote attacker could provide a specially-crafted URL that, when visited would lead to arbitrary HTML or web script execution in the context of the MantisBT user's session.

Alerts:
Fedora FEDORA-2014-15079 mantis 2014-12-12
Fedora FEDORA-2013-5801 mantis 2013-08-04
Fedora FEDORA-2013-5829 mantis 2013-04-25
Fedora FEDORA-2013-5833 mantis 2013-04-25

Comments (none posted)

mediawiki: multiple vulnerabilities

Package(s):mediawiki CVE #(s):CVE-2013-1951
Created:April 30, 2013 Updated:May 1, 2013
Description: From the Red Hat bugzilla:

Three flaws were corrected in the recently-released MediaWiki 1.20.4 and 1.19.5 releases:

* An internal review discovered that specially crafted Lua function names could lead to cross-site scripting. MediaWiki bug 46084

* Daniel Franke reported that during SVG parsing, MediaWiki failed to prevent XML external entity (XXE) processing. This could lead to local file disclosure, or potentially remote command execution in environments that have enabled expect:// handling. MediaWiki bug 46859

* Internal review also discovered that Special:Import, and Extension:RSS failed to prevent XML external entity (XXE) processing. MediaWiki bug 47251

CVE-2013-1951 was assigned to the first issue (the XSS), the other two do not have CVEs assigned as per a discussion on oss-sec.

Alerts:
Gentoo 201310-21 mediawiki 2013-10-28
Fedora FEDORA-2013-5874 mediawiki 2013-04-25
Fedora FEDORA-2013-6170 mediawiki 2013-04-30
Fedora FEDORA-2013-6171 mediawiki 2013-04-30

Comments (none posted)

mysql: multiple vulnerabilities

Package(s):mysql-5.5 CVE #(s):CVE-2012-0553 CVE-2013-1492 CVE-2013-1623
Created:April 26, 2013 Updated:May 1, 2013
Description:

From the Ubuntu issue tracker:

Buffer overflow in yaSSL, as used in MySQL 5.1.x before 5.1.68 and 5.5.x before 5.5.28, has unspecified impact and attack vectors, a different vulnerability than CVE-2013-1492. (CVE-2012-0553)

Buffer overflow in yaSSL, as used in MySQL 5.1.x before 5.1.68 and 5.5.x before 5.5.30, has unspecified impact and attack vectors, a different vulnerability than CVE-2012-0553. (CVE-2013-1492)

The TLS and DTLS implementations in wolfSSL CyaSSL before 2.5.0 do not properly consider timing side-channel attacks on a noncompliant MAC check operation during the processing of malformed CBC padding, which allows remote attackers to conduct distinguishing attacks and plaintext-recovery attacks via statistical analysis of timing data for crafted packets, a related issue to CVE-2013-0169. (CVE-2013-1623)

Alerts:
Gentoo 201308-06 mysql 2013-08-29
Gentoo GLSA 201308-06:02 mysql 2013-08-30
Ubuntu USN-1807-2 mysql-5.5 2013-04-25

Comments (none posted)

pdns-recursor: ghost domain name resolving flaw

Package(s):pdns-recursor CVE #(s):CVE-2012-1193
Created:May 1, 2013 Updated:May 1, 2013
Description: From the CVE entry:

The resolver in PowerDNS Recursor (aka pdns_recursor) 3.3 overwrites cached server names and TTL values in NS records during the processing of a response to an A record query, which allows remote attackers to trigger continued resolvability of revoked domain names via a "ghost domain names" attack.

Alerts:
Gentoo 201412-33 pdns-recursor 2014-12-22
Fedora FEDORA-2013-6316 pdns-recursor 2013-05-01
Fedora FEDORA-2013-6279 pdns-recursor 2013-05-01

Comments (none posted)

php-twig-Twig: file disclosure

Package(s):php-twig-Twig CVE #(s):
Created:April 29, 2013 Updated:May 1, 2013
Description: From the Twig advisory:

Your application is affected if you are using Twig_Loader_Filesystem for loading Twig templates but only if you are using non-trusted template names (names provided by a end-user for instance).

When affected, it is possible to go up one directory for the paths configured in your loader.

For instance, if the filesystem loader is configured with /path/to/templates as a path to look for templates, you can force Twig to include a file stored in /path/to by prepending the path with /../ like in {% include "/../somefile_in_path_to" %}

Note that using anything else (like ../somefile, /../../somefile, or ../../somefile) won’t work and you will get a proper exception.

Alerts:
Fedora FEDORA-2013-6107 php-twig-Twig 2013-04-27
Fedora FEDORA-2013-6114 php-twig-Twig 2013-04-27

Comments (none posted)

qemu: host file disclosure

Package(s):qemu CVE #(s):CVE-2013-1922
Created:April 25, 2013 Updated:May 3, 2013
Description:

From the Red Hat bugzilla entry:

A security flaw was found in the way qemu-nbd, the QEMU Disk Network Block Device server tool of QEMU, performed detection of image formats (the image format has been previously autodetected). A guest operating system administrator could write a header to particular raw disk image format, describing another format than original one for that disk image, leading to scenario in which after restart of that guest, QEMU would detect new format of the image, and could allow the guest to read any file on the host if QEMU was sufficiently privileged.

Alerts:
Gentoo 201309-24 xen 2013-09-27
openSUSE openSUSE-SU-2013:1404-1 xen 2013-09-04
Mageia MGASA-2013-0134 qemu 2013-05-02
Fedora FEDORA-2013-6211 qemu 2013-04-30
Fedora FEDORA-2013-6221 qemu 2013-04-26
Fedora FEDORA-2013-6185 qemu 2013-04-25

Comments (none posted)

strongswan: authentication bypass

Package(s):strongswan CVE #(s):CVE-2013-2944
Created:April 30, 2013 Updated:June 11, 2013
Description: From the Debian advisory:

Kevin Wojtysiak discovered a vulnerability in strongSwan, an IPsec based VPN solution.

When using the openssl plugin for ECDSA based authentication, an empty, zeroed or otherwise invalid signature is handled as a legitimate one. An attacker could use a forged signature to authenticate like a legitimate user and gain access to the VPN (and everything protected by this).

While the issue looks like CVE-2012-2388 (RSA signature based authentication bypass), it is unrelated.

Alerts:
Gentoo 201309-02 strongswan 2013-09-01
openSUSE openSUSE-SU-2013:0873-1 strongswan 2013-06-10
openSUSE openSUSE-SU-2013:0774-1 strongswan 2013-05-10
openSUSE openSUSE-SU-2013:0775-1 strongswan 2013-05-10
Debian DSA-2665-1 strongswan 2013-04-30
openSUSE openSUSE-SU-2013:0985-1 strongswan 2013-06-10

Comments (none posted)

Page editor: Jake Edge

Kernel development

Brief items

Kernel release status

The 3.9 kernel is out, released by Linus on April 28. Headline features in 3.9 include KVM virtualization on the ARM architecture, the near-completion of user namespace support, PowerClamp support, the dm-cache device mapper target, RAID5/6 support in the Btrfs filesystem, The ARC and MetaG architectures, and more. See the KernelNewbies 3.9 page for lots of details.

The 3.10 merge window is open; see the separate summary below for an overview of what has been merged so far.

Stable updates: 3.8.9, 3.4.42, 3.2.44 and 3.0.75 all came out on April 25; 3.8.10 followed one day later with a fix for a build problem. 3.8.11, 3.4.43, and 3.0.76 were released on May 1.

Comments (none posted)

Quotes of the week

If you're using custom kernels with [CyanogenMod] (particularly with nightly builds), be aware of this. Things will break. We aren't conspiring against you, nor are we "violating the spirit of the GPL". At the same time, I will freely admit that we aren't looking out for you either. If you want to run a fork of a single subsystem of a fast-moving highly interdepedent codebase, you will find dragons waiting for you.
Steve Kondik

I am trying to avoid perpetrating^Winventing new RCU flavors, at least for the next week or two.

Speaking as an IBMer, I can only hang my head in shame at my inability thus far to come up with an acronym that is at least five letters long, but consisting only of vowels.

Paul McKenney

Comments (8 posted)

Deloget: The SoC GPU driver interview

In a lengthy blog post, Emmanuel Deloget interviews nine developers of GPU drivers and tools for ARM system-on-chip (SoC) devices. Questions range from the status of various projects and how the projects got started to intra-project collaboration and the future of the ARM platform. The developers and projects are: Connor Abbot - Open GPU Tools, Eric Faye-Lund - grate (for Tegra GPUs), Herman H. Hermitage - Videocore (for Broadcom GPUs), Luc Verhaegen - lima (for MALI GPUs), Matthias Gottschlag - Videocore (for Broadcom GPUs), Rob Clark - freedreno (for Adreno GPUs), Thierry Reding - grate (for Tegra GPUs), Scott Mansell - Videocore (for Broadcom GPUs), and Wladimir J. van der Laan - etna_viv (for Vivante GPUs).

Comments (3 posted)

Output redirection vulnerabilities in recent kernels

Andy Lutomirski has posted a description of a set of security vulnerabilities fixed in recent stable updates. One is a fairly severe user namespace vulnerability that appeared in the 3.8 kernel; another dates back to 2.6.36. Exploit code is included.

Full Story (comments: 12)

Three Outreach Program for Women kernel internships available

The Linux Foundation has announced that it will be supporting three kernel internships for the upcoming Outreach Program for Women cycle. "The official deadline for applying to OPW is May 1st. However, the kernel project joined late, so that deadline is flexible. Please fill out your initial application, and then update by May 17th with your initial patch." Acceptance in the program brings a $5000 stipend plus $500 in travel funding.

Comments (1 posted)

Kernel development news

What's coming in 3.10, part 1

By Jonathan Corbet
May 1, 2013
As of this writing, nearly 5,100 non-merge changesets have been pulled into the mainline repository for the 3.10 development cycle. That is a pretty good pace, given that this merge window only opened on April 29. A number of interesting new features have been added, with more to come.

User-visible changes merged at the beginning of the 3.10 cycle include:

  • The "ftrace" tracing facility has seen a number of improvements. At the top of the list, arguably, is the ability to establish multiple buffers for tracing information and to direct specific events to different buffers. Additionally, it is now possible set up a trigger that will enable or disable specific events when a given kernel function is called, and it is possible to get a stack trace when a given function is called. See the improved ftrace.txt document for lots of details.

  • Control groups have seen a fair amount of work in this release, some of which is implementing the improvements planned over one year ago. The device and perf_event groups now offer full hierarchical support. There is a new mount option (with the unwieldy name "__DEVEL__sane_behavior") that tries to ensure more consistent hierarchy behavior across all groups; it is obviously meant for development rather than production, but it gives some clues about what is coming.

  • Applications within control groups can now request memory pressure notifications when the system is running low on available memory. See the new section 11 at the end of Documentation/cgroups/memory.txt for details on how to use this feature.

  • "Return probes" are now supported in user-space probing; they allow the activation of a breakpoint when a target function returns to its caller.

  • The generation of POSIX timer IDs has changed; IDs are no longer guaranteed to be unique across the system. A process's timers can now be queried by reading /proc/PID/timers. These changes make it possible for the checkpoint/restart feature to restore active timers without changing their IDs.

  • POSIX and high-resolution timers support a new clock (CLOCK_TAI) which operates in international atomic time.

  • The perf command (along with the perf_events subsystem) can now do memory access profiling.

  • The iSCSI subsystem has gained support for the iSCSI extensions for RDMA (iSER) protocol.

  • CPU frequency governors can now exist in multiple instances with different tuning parameters. This feature is needed in heterogeneous multiprocessing systems (ARM big.LITTLE, for example) to allow different CPUs to run with different parameters.

  • The ability to run scripts as executables (with the interpreter specified using the "#!" sequence) can now be built as a module — or left out altogether for systems that run no scripts.

  • New hardware support includes:

    • Display and graphics: Microsoft Hyper-V synthetic video devices and ILI Technology ILI9221/ILI9222 backlight controllers.

    • Hardware monitoring: Analog Devices ADT7310/ADT7320 temperature monitors, Nuvoton NCT6779D hardware monitoring chips, National Semiconductor LM95234 temperature sensors, and ST-Ericsson AB8500 thermal monitors.

    • Input: Apple infrared receivers.

    • Miscellaneous: Advantech PCI-1724U analog output cards, Analog Devices AD7923 analog to digital interfaces, Qualcomm single-wire serial bus interfaces, Broadcom BCM2835 SPI controllers, Aeroflex Gaisler GRLIB SPI controllers, NVIDIA Tegra114 SPI controllers, Silicon Labs 5351A/B/C programmable clock generators, on-chip static RAM regions, TI TPS65090 battery chargers, Samsung EXYNOS5440 CPU frequency controllers, and ARM big.LITTLE CPU frequency controllers.

    • Networking: Netlogic XLR/XLS network interfaces.

    • USB: DesignWare USB2 USB controllers.

    • Video4Linux: Rafael Micro R820T silicon tuners, ITE Tech IT913x silicon tuners, OmniVision OV7640 sensors, Philips UDA1342 audio codecs, Techwell TW9903, TW9906 and TW2804 video decoders, Silicon Laboratories 4761/64/68 AM/FM radios, Silicon Laboratories Si476x I2C FM radios, and Samsung EXYNOS4x12 FIMC-IS imaging subsystems.

    Note also that the Android "configurable composite gadget" driver has been removed from the staging tree. It is apparently difficult to maintain and no current hardware uses it.

Changes visible to kernel developers include:

  • The devtmpfs filesystem now provides drivers with the ability to specify which user and group ID should own a given device. This capability is somewhat controversial — there is resistance to encoding user and group ID policy in the kernel — but it will be useful for systems like Android.

  • The staging tree has gained a new "sync" driver (from Android) that can be used for synchronization between other drivers.

  • There is a new "dummy-irq" driver that does nothing other than register an interrupt handler. It exists to debug IRQ sharing problems by forcing the enabling of a specific interrupt line.

  • A lot of the low-level USB PHY access functions have been changed to GPL-only exports.

  • The new idr_alloc_cyclic() function allocates ID numbers in a cyclic fashion: when the given range is exhausted, allocations will start again at the beginning of that range.

  • The workqueue subsystem has seen some substantial reworking which, among other things, should make it perform better on NUMA systems. There is also a new sysfs interface that can be used to tweak some workqueue parameters.

If the usual pattern holds, this merge window should remain open until around May 12 and the 3.10 kernel can be expected in early July. As usual, LWN will follow the mainline as the merge window progresses.

Comments (none posted)

Wait/wound mutexes

By Jonathan Corbet
May 1, 2013
Developers wanting to add new locking primitives to the kernel tend to be received with a certain amount of skepticism. The kernel is already well equipped with locking mechanisms, and experience shows that new mechanisms tend to be both unnecessary and hard to get right. The "wait/wound mutex mechanism" proposed by Maarten Lankhorst may well get that kind of response. But it is an interesting approach to a specific locking problem that merits a closer look.

A conceptual overview

Situations where multiple locks must be held simultaneously pose a risk of deadlocks: if the order in which those locks are acquired is not always the same, there will eventually come a time when two threads find themselves blocked, each waiting for the other to release a lock. Kernel code tends to be careful about lock ordering, and the "lockdep" checking tool has gotten quite good about finding code that violates the rules. So deadlocks are quite rare, despite the huge number of locks used by the kernel.

But what about situations where the ordering of lock acquisition cannot be specified in advance, or, even worse, is controlled by user space? Maarten's patch describes one such scenario: a chain of buffers used with the system's graphical processing unit (GPU). These buffers must, at various times, be "owned" by the GPU itself, the GPU driver, user space, and, possibly, another driver completely, such as for a video frame grabber. User space can submit the buffers for processing in an arbitrary order, and the GPU may complete them in a different order. If locking is used to control the ownership of the buffers, and if multiple buffers must be manipulated at once, avoiding deadlocks could become difficult.

Imagine a simple situation where there are two buffers of interest:

[buffers]

Imagine further that we have two threads (we'll call them T1 and T2) that attempt to lock both buffers in the opposite order: T1 starts with Buffer A, while T2 starts with Buffer B. As long as they do not both try to grab the buffers at the same time, things will work. But, someday, each will succeed in locking one buffer and a situation like this will develop:

[locked buffers]

The kernel's existing locking primitives have no answer to a situation like this other than "don't do that." The wait/wound mutex, instead, is designed for just this case. In general terms, what will happen in this situation is:

  • The thread that "got there first" will simply sleep until the remaining buffer becomes available. If T1 started the process of locking the buffers first, it will be the thread that waits.

  • The other thread will be "wounded," meaning that it will be told it must release any locks it holds and start over from scratch.

So if T2 is wounded, the deadlock will be resolved by telling T2 to release Buffer B; it must then wait until that buffer becomes available again and start over. So the situation will look something like this:

[locked buffers]

Once T1 has released the buffers, T2 will be able to retry and, presumably, make forward progress on its task.

The details

The first step toward using a set of locks within the wait/wound mechanism is to define a "class"; this class is essentially a context within which the locks are to be acquired. When multiple threads contend for the same locks, they must do so using the same class. A wait/wound class is defined with:

    #include <linux/mutex.h>

    static DEFINE_WW_CLASS(my_class);

As far as users of the system are concerned, the class needs to exist, but it is otherwise opaque; there is no explicit initialization required. Internally, the main purpose for the class's existence is to hold a sequence number (an atomic counter) used to answer the "who got there first" question; it also contains some information used by lockdep to verify correct use of wait/wound locks.

The acquisition of a specific set of locks must be done within a "context" that tracks the specific locks held. Before acquiring the first lock, a call should be made to:

    void ww_acquire_init(struct ww_acquire_ctx *ctx, struct ww_class *ww_class);

This call will assign a sequence number to the context and do a bit of record keeping. Once that has been done, it is possible to start acquiring locks:

    int ww_mutex_lock(struct ww_mutex *lock, struct ww_acquire_ctx *ctx);

If the lock has been successfully acquired, the return value will be zero. When all goes well, the thread will manage to acquire all of the locks it needs. Once that process is complete, that fact should be signaled with:

    void ww_acquire_done(struct ww_acquire_ctx *ctx);

This function is actually a no-op in the current implementation, but that could change in the future. After this call, the processing of the locked data can proceed normally. Once the job is done, it is time to release the locks and clean up:

    void ww_mutex_unlock(struct ww_mutex *lock);
    void ww_acquire_fini(struct ww_acquire_ctx *ctx);

Each held lock should be released with ww_mutex_unlock(); once all locks have been released, the context should be cleaned up with ww_acquire_fini().

The above description describes what happens when all goes well, but it has left out an important case that all wait/wound mutex users must handle: the detection of a potential deadlock. That case comes about whenever an attempt is made to lock a ww_mutex that is already locked; in this case, there are three possible outcomes.

The first of these comes about if the locking thread already holds that ww_mutex and is attempting to lock it for a second time. With ordinary mutexes, this would be an error, but the wait/wound mechanism is designed for this case. Evidently, sometimes, the ordering of the locking is so poorly defined that multiple locking attempts can happen. In such cases, ww_mutex_lock() will return -EALREADY. The locking thread, assuming it knows how to respond to -EALREADY, can continue about its business.

The second possibility is that the sequence number in the context for the locking process is higher than the number associated with thread already holding the lock. In this case, the new caller gets "wounded"; ww_mutex_lock() will return -EDEADLK to signal that fact. The wounded thread is expected to clean up and get out of the way. "Cleaning up" means releasing all locks held under the relevant context with calls to ww_mutex_unlock(). Once all of the locks are free, the wounded thread can try again, but only when the contended lock is released by the victorious thread; waiting for that to happen is done with:

    void ww_mutex_lock_slow(struct ww_mutex *lock, struct ww_acquire_ctx *ctx);

This function will block the calling thread until lock becomes free; once it returns, the thread can try again to acquire all of the other locks it needs. It is entirely possible that this thread could, once again, fail to acquire all of the needed locks. But, since the sequence number increases monotonically, a once-wounded thread must eventually reach a point where it has the highest priority and will win out.

The final case comes about when the new thread's sequence number is lower than that of the thread currently holding the lock. In this case, the new thread will simply block in ww_mutex_lock() until the lock is freed. If the thread holding the contended lock attempts to acquire another lock that is already held by the new thread, it will get the -EDEADLK status at that point; it will then release the contended lock and let the new thread proceed. Going back to the example above:

[locked buffers]

Thread T1, holding the lower sequence number, will wait for Buffer B to be unlocked, while thread T2 will see -EDEADLK when it attempts to lock Buffer A.

The documentation in the patch does not describe what happens if the holding process never calls ww_mutex_lock() again. In this case, it will never know that it is supposed to back off. But, in this case, the holder must necessarily already have acquired all of the locks it needs, so there should be no reason why it cannot simply finish its work and release the locks normally. So the end result will be the same.

Conclusion

Needless to say, there are plenty of details that have not been covered here; see the ww-mutex-design.txt document included with the patch set for more information.

In that document, there are code examples for three different ways of working with wait/wound mutexes. One need not read for long to conclude that the API looks a bit complex and tricky to use; it will be far harder to write correct locking code using this facility than it would be with normal mutexes. Perhaps that complexity is necessary, and it seems certain that this mechanism will not be needed in many places in the kernel, so the complexity should not spread too far. But an API like this can be expected to raise some eyebrows.

What is missing at this point is any real code that uses wait/wound mutexes. Kernel developers will certainly want to see some examples of where this kind of locking mechanism is needed. After all, the kernel has made it through its first two decades without this kind of complex locking; convincing the community that this feature is now necessary is going to take a strong sales effort. That is best done by showing how wait/wound mutexes solve a problem that cannot be easily addressed otherwise. Until that is done, wait/wound mutexes are likely to remain an interesting bit of code on the sidelines.

Comments (15 posted)

LSFMM Summit coverage complete

By Jonathan Corbet
May 1, 2013
Since the first notes from the 2013 Linux Storage, Filesystem, and Memory Management Summit were posted, we have been busy filling in notes from the remaining sessions. We now have notes from every scheduled session at the summit.

Since the initial posting, the following sessions have been added:

Combined Filesystem/Storage sessions

  • dm-cache and bcache: the future of two storage-caching technologies for Linux.

  • Error returns: filesystems could use better error information from the storage layer.

  • Storage management: how do we ease the task of creating and managing filesystems on Linux systems?

  • O_DIRECT: the kernel's direct I/O code is complicated, fragile, and hard to change. Is it time to start over?

  • Reducing io_submit() latency: submitting asynchronous I/O operations can potentially block for long periods of time, which is not what callers want. Various ways of addressing this problem were discussed, but easy solutions are not readily at hand.

Filesystem-only sessions

  • NFS status: what is going on in the NFS subsystem.

  • Btrfs status: what has happened with the next-generation Linux filesystem, and when will it be ready for production use?

  • User-space filesystem servers: what can the kernel do to support user-space servers like Samba and NFS-GANESHA?

  • Range locking: a proposal to lock portions of files within the kernel.

  • FedFS: where things stand with the creation of a Federated Filesystem implementation for Linux.

Storage-only sessions

  • Reducing SCSI latency. The SCSI stack is having a hard time keeping up with the fastest drives; what can be done to speed things up?

  • SCSI testing. It would be nice to have a test suite for SCSI devices; after this session, one may well be in the works.

  • Error handling and firmware updates: some current problems with handling failing drives, and how can we do online firmware updates on SATA devices?

Thanks are due to those who helped with the creation of these writeups. We would like to thank Elena Zannoni in particular for providing comprehensive notes from the Storage track.

Comments (none posted)

Patches and updates

Kernel trees

  • Sebastian Andrzej Siewior: 3.8.10-rt6 . (April 30, 2013)
  • Sebastian Andrzej Siewior: 3.8.9-rt4 . (April 28, 2013)

Architecture-specific

Build system

Core kernel code

Development tools

Device drivers

Filesystems and block I/O

Memory management

Security-related

Miscellaneous

Page editor: Jonathan Corbet

Distributions

x32 ABI support by distributions

May 1, 2013

This article was contributed by Andrew Shewmaker

In 2011, LWN reported on the x32 system call ABI for the Linux kernel. Now x32 is officially supported by a number of upstream projects and had some time to mature. As x32 begins finding its way into distributions, more people will be able to try it out on real world applications. But what kind of applications have the potential for the greatest benefit? And how well do the distributions support x32? This article will address those questions and detail some ways to experiment with the x32 ABI.

The x32 ABI uses the same registers and floating point hardware as the x86-64 ABI, but with 32-bit pointers and long variables as on ia32 (32-bit x86). On the down side, fully supporting x32 means having a third set of libraries. However, its assembly code is more efficient than ia32. Additionally, if an application uses many pointers and requires less than 4GB of RAM, then x32 can be a big win over x86-64. It can theoretically double the number of pointers held in cache, reducing misses, and halve the amount of system memory used to store pointers, reducing page faults.

Benchmark results

The x32 project site lists two SPEC benchmarks that show particularly good benefits. MCF, a network simplex algorithm making heavy use of pointers, shows x32 with a 40% advantage over x86-64. Crafty, a chess program made up predominantly of integer code, shows x32 with a 40% advantage over ia32. In H. Peter Anvin's 2011 Linux Plumbers Conference talk x32 — a native 32-bit ABI for x86-64 [PPT], he lists embedded devices as one of the primary use cases. He reports 5-10% improvement for the SPEC CPU 2K/2006 INT geomean over ia32 and x86-64, and 5-11% advantage for SPEC CPU 2K/2006 FP geomean over ia32, but none against x86-64. The MCF results have been called out as the most impressive, but the majority of users may care more about video codec performance. The news is good there too. H.264 performed 35% better on x32 than on ia32 and 15-20% better than on x86-64.

Intel gives additional SPEC and Embedded Microprocessor Benchmark Consortium (EEMBC) performance numbers in a paper describing the work done to support x32 in GCC and their own compiler. Parser, a syntactic parser of English, shows an approximately 17% improvement for x32 over both ia32 and x86-64. And Eon, a probabilistic ray tracer, performs almost 30% better than ia32. In general, GCC's floating-point code generation isn't as good on x32 as on x86-64. However, Intel's compiler does at least as well for x32 as x86-64. Take Facerec, a facial recognition algorithm which compares graph structures with vertices containing information computed from fast Fourier transforms, where x32 performs almost 40% better than x86-64.

Scientific simulations are often limited by memory latency and capacity. Nathalie Rauschmayr has presented results showing how x32 benefits some of CERN's applications [PDF]. These applications use millions of pointers and, when compiled as x32, see a reduction in physical memory use of 10-30%. While some ran no faster, a couple sped up by 10-15%. In those cases, the underlying cause for the improvement is a 10-38% reduction in page faults. Loop unrolling is an important optimization for scientific code and x32 could use some work in that area, but H.J. Lu, the driving force behind x32, doesn't plan to do that work himself.

Distribution support

As Nathalie notes, a working x32 environment doesn't come out of the box in a Red Hat– or Debian–based Linux distribution just yet. Currently, a person must build glibc with an intermediate GCC build before building a full GCC. Fortunately, distributions are beginning to provide an easier way to experiment with x32. Two of the first projects to feature a release with official x32 support are Gentoo and the Yocto meta-distribution, and a couple of traditional distributions are following to one degree or another.

Gentoo's first x32 release candidate came out last June; support is now integrated into the main tree. You can download and install it like any other port. When I installed it, I ran into a problem creating an initramfs due to a BusyBox linking error. Configuring the kernel with everything built-in (eliminating the need for an initramfs altogether) worked around this problem, resulting in a working x32 environment. The remaining x32 issues are largely in multimedia packages' hand-optimized assembly routines. As soon as Gentoo's infrastructure team upgrades the kernel on a development box, Mike Frysinger plans on announcing the availability of an x32 environment for upstream multimedia projects to port their optimized assembly to the x32 ABI.

I have not tried the Linux Foundation's Yocto project. It is used for creating custom embedded distributions and has integrated x32 support into its core, as well as fixing most of its recipes.

x32 won't be an official ABI for Debian until after the next release, Wheezy. Daniel Schepler, who oversees the new port, told me that it is in reasonable shape. Grub2 doesn't currently build, so an x32-only install is not possible yet. But Daniel provides directions to install into a chroot environment from his private archive on the Debian x32 Port wiki. Newer packages are available from Debian Ports. Simpler desktop environments like XFCE are installable, and KDE is almost ready. However, more work needs to be done before GNOME is installable. Its biggest blocker is that Iceweasel currently doesn't build. Also, LibreOffice and PHP are not ready yet.

My experience with Debian went well. After a minimal installation from the stable repository and upgrading to Wheezy, I installed one of Daniel's kernels and used the debootstrap tool to quickly create an x32 chroot from Daniel's personal archive. After that, I was able to point to the most recent packages on the Debian Ports archive. Installing a chroot directly from those involves learning how to use the multistrap tool, and I stopped short of that.

Ubuntu shipped with limited x32 support in 13.04. However, it isn't planning on providing a full port of x32 packages like Debian. Ubuntu is simply providing a kernel with the x32 syscalls and a functional tri-arch toolchain with GCC and eglibc.

x32 hasn't found as much acceptance in other open-source projects. Fedora has no plans for x32, which means it's unlikely for RHEL too. Arch Linux has some unofficial packages. LLVM users can apply patches from June 2012. But merging of x32 support looks unlikely at this point. A team from Google inquired about it earlier this year, and they were told that the patch submitter had not addressed the maintainers' concerns yet.

Some experiments

After reading Diego Elio "Flameeyes" Pettenò's series of posts on debunking x32 myths, I decided someone ought to compare x32 and x86-64 performance of the B-heap simulator mentioned in Poul-Henning Kamp's ACM article "You're Doing It Wrong." This example is interesting because Kamp created a data structure called a "B-heap" to take advantage of the large address space provided by 64-bit pointers. That would seem to argue against using x32's smaller address space, but in his article he also says, "The order of magnitude of difference obviously originates with the number of levels of heap inside each VM page, so the ultimate speedup will be on machines with small pointer sizes and big page sizes."

Kamp's B-heap simulator compares the virtual memory performance of a traditional binary heap to his B-heap. A port from BSD to Linux went smoothly, and the results are identical. No changes were needed to compile the simulator in my Debian x32 chroot. Its virtual memory footprint was reduced by 19% and its resident set size was 88% smaller. Performance of x32 always exceeded x86-64, especially for larger problem sizes. In particular, 500K ran 15% faster. It's important to note that the simulator itself doesn't use much memory, so it would be interesting to see how a real-world use of a B-heap, as in the Varnish Cache, would compare.

It will be interesting to see which other applications will benefit from x32. If it shows as much promise for the embedded space as its creators suggest, high performance computing as CERN shows, and web servers as my experiment indicates, then desktop users in the middle of the computing spectrum might also benefit. The key to that segment will likely be whether multimedia codecs (other than the H.264 reference implementation tested by SPEC CPU 2006) see a substantial enough improvement to justify developing and maintaining another assembly code implementation of their core routines. I predict they will. Perhaps x32 will prove its value in such a broad number of cases that it becomes the default for many software packages. Perhaps we'll even see developers for other 64-bit architectures such as IBM's Power or upcoming ARM server chips propose similar ABIs.

Comments (32 posted)

Brief items

Distribution quote of the week

Honestly, this seems a bit like hurting our users so that developers feel sorry for them and jump through hoops. We might as well threaten to kill kittens anytime the bug wrangler list goes over a certain threshold, or a GLSA stays open too long.
-- Rich Freeman

Comments (5 posted)

DragonFly BSD 3.4 released

The DragonFly project has announced the release of version 3.4.1 of FreeBSD-based DragonFly BSD. See the release notes for details. LWN covered DragonFly in 2010.

Comments (none posted)

GNU Linux-libre 3.9-gnu

GNU Linux-libre 3.9-gnu is the latest version of the stripped down Libre kernel. "The greatest news about this release is that it no longer rejects ath9k firmware, after the former blobs were re-released as Free Software! Thanks to Jason Self for the patches to the deblobbing scripts to this end; over the next few days, I'm going to deblob stable releases using newer scripts containing Jason's patches, so that ath9k will be usable on them as well."

Full Story (comments: none)

OpenBSD 5.3 released

OpenBSD 5.3 has been released. The release announcement (click below) contains a lengthy list of new features and improvements.

Full Story (comments: 8)

Ubuntu 13.04 (Raring Ringtail) released

Ubuntu has announced the release of 13.04, "Raring Ringtail", with versions for both servers and desktops. Many improvements and updates have come with 13.04 and there are also corresponding Kubuntu, Xubuntu, Edubuntu, Lubuntu, and Ubuntu Studio releases. "Along with performance improvements to Unity, updates to common desktop packages, and updated core and toolchain components, Ubuntu 13.04 also includes the new Friends service, to consolidate all social networking accounts via Ubuntu Online Accounts. Also included is a tech preview of Upstart's new user session feature."

Full Story (comments: 22)

Distribution News

Debian GNU/Linux

Debian Installer 7.0 RC2 release

The second release candidate of the Debian 7.0 "Wheezy" installer is available for testing. This is one of the last items that must be ready for the final release of Debian 7.0.

Full Story (comments: none)

Bits from the 6th Debian Groupware Meeting

Guido Günther has a report from the recent Debian Groupware Meeting. "There's been progress on getting Kolab3 into Debian starting with libkolabxml 0.8.4 and libkolab 0.4.2 (also needed by kde-pim)."

Full Story (comments: none)

Ubuntu family

Next Ubuntu now open for development

Development has opened for Ubuntu's newest release and syncs from Debian unstable are in progress. "GCC 4.8 is now the default compiler, introducing among other things improved C++11 support, AddressSanitizer, a fast memory error detector, and ThreadSanitizer, a tool to detect data races."

Full Story (comments: none)

Newsletters and articles of interest

Distribution newsletters

Comments (none posted)

PCLinuxOS "So Cool Ice Cubes are Jealous" (Linux.com)

Carla Schroder reviews PCLinuxOS on Linux.com. "PCLinuxOS aims to please, and to provide a smooth out-of-the-box experience with everything working and easy to maintain. Every release has been more polished with a better user interface, and the latest release, 2013.04, is impressively sleek and well-organized. I'm partial to the Control Center, which contains a useful set of system configuration tools that help you through the complete process of setting up new services. For example, when you go to the Sharing tab and click the Configure Web Server button, it gives you the option to launch the installer and install Apache. Then it runs a wizard to set up a basic Web server. PCLinuxOS sets up a clean separation between system administration and desktop configuration. I prefer a standalone control panel, rather than dealing with the various configuration tools that come with the different desktop environments, because it's consistent and I can actually find things."

Comments (none posted)

Shuttleworth: The Supercalifragilisticexpialidocious Scorpionfish. Not.

On his blog, Mark Shuttleworth dashes the hopes of those looking for a Mary Poppins-inspired release name for Ubuntu 13.10. "Slipping the phrase 'ring ring' into the codename of 13.04 was, frankly, a triumph of linguistic engineering. And I thought I might quit on a high ... For a while, there was the distinct possibility that Rick's Rolling Release Rodeo would absolve me of the twice-annual rite of composition that goes into the naming of a new release. That, together with the extent of my travels these past few months, have left me a little short in the research department." No spoilers here, other than: the name is two words, an adjective and an animal, both of which start with "S".

Comments (35 posted)

Ward: So long Fuduntu, hello FuSE

The Fuduntu distribution has seen its last update, however several members of the team have decided to create a new distribution. Tentatively named FuSE, the new distribution will be based on openSUSE. "One of the first things discussed was the default desktop environment that would be used on the new distro. After discussing many different DE’s, including KDE, xfce, razor-qt, klyde, and Consort, it was put up to a vote. The majority ruled in favor of Consort which is currently being developed by Ikey Doherty, the founder of SolusOS. It is based off of GTK3 but designed to look, and act, like GNOME2. The team, however, still has plans to evaluate it to ensure integration and stability before officially declaring it as the default DE. KDE, however, will also be available as well as other DE’s by way of the openSUSE repos."

Comments (none posted)

Page editor: Rebecca Sobol

Development

Go and Rust — objects without class

May 1, 2013

This article was contributed by Neil Brown

Since the advent of object-oriented programming languages around the time of Smalltalk in the 1970s, inheritance has been a mainstay of the object-oriented vision. It is therefore a little surprising that both "Go" and "Rust" — two relatively new languages which support object-oriented programming — manage to avoid mentioning it. Both the Rust Reference Manual and The Go Programming Language Specification contain the word "inherit" precisely once and the word "inheritance" not at all. Methods are quite heavily discussed, but inheritance is barely more than a "by the way".

This may be just an economy of expression, or it may be an indication of a sea change in attitudes towards object orientation within the programming language community. It is this second possibility which this article will consider while exploring and contrasting the type systems of these two languages.

The many faces of inheritance

While inheritance is a core concept in object-oriented programming, it is not necessarily a well-defined concept. It always involves one thing getting some features by association with some previously defined things, but beyond that languages differ. The thing is typically a "class", but sometimes an "interface" or even (in prototype inheritance) an "object" that borrows some behavior and state from some other "prototypical" object.

The features gained are usually fields (for storing values) and methods (for acting on those values), but the extent to which the inheriting thing can modify, replace, or extend these features is quite variable.

Inheriting from a single ancestor is common. Inheriting from multiple ancestors is sometimes possible, but is an even less well-defined concept than single inheritance. Whether multiple inheritance really means anything useful, how it should be implemented, and how to approach the so-called diamond problem all lead to substantial divergence among approaches to inheritance.

If we clear away these various peripheral details (important though they are), inheritance boils down to two, or possibly three, core concepts. It is the blurring of these concepts that is created by using one word ("inheritance"), which, it would seem, results in the wide variance among languages. And it is this blurring that is completely absent from Go and Rust.

Data embedding

The possible third core concept provided by inheritance is data embedding. This mechanism allows a data structure to be defined that includes a previously defined data structure in the same memory allocation. This is trivially achieved in C as seen in:

    struct kobject {
	char		*name;
	struct list_head entry;
	...
    };

where a struct list_head is embedded in a struct kobject. It can sometimes be a little more convenient if the members of the embedded structure (next and prev in this case) can be accessed in the embedding object directly rather than being qualified as, in this case entry.next and entry.prev. This is possible in C11 and later using "anonymous structures".

While this is trivial in C, it is not possible in this form in a number of object-oriented languages, particularly languages that style themselves as "pure" object oriented. In such languages, another structure (or object) can only be included by reference, not directly (i.e. a pointer can be included in the new structure, but the old structure itself cannot).

Where structure embedding is not possible directly, it can often be achieved by inheritance, as the fields in the parent class (or classes) are directly available in objects of the child class. While structure embedding may not be strong motivation to use inheritance, it is certainly an outcome that can be achieved through using it, so it does qualify (for some languages at least) as one of the faces of inheritance.

Subtype polymorphism

Subtype polymorphism is a core concept that is almost synonymous with object inheritance. Polymorphic code is code that will work equally well with values from a range of different types. For subtype polymorphism, the values' types must be subtypes of some specified super-type. One of the best examples of this, which should be familiar to many, is the hierarchy of widgets provided by various graphical user interface libraries such as GTK+ or Qt.

At the top of this hierarchy for GTK+ is the GtkWidget which has several subtypes including GtkContainer and GtkEditable. The leaves of the hierarchy are the widgets that can be displayed, such as GtkEntry and GtkRadioButton.

GtkContainer is an ancestor of all widgets that can serve to group other widgets together in some way, so GtkHBox and GtkVBox — which present a list of widgets in a horizontal or vertical arrangement — are two subtypes of GtkContainer. Subtype polymorphism allows code that is written to handle a GtkContainer to work equally well with the subtypes GtkHBox and GtkVBox.

Subtype polymorphism can be very powerful and expressive, but is not without its problems. One of the classic examples that appears in the literature involves "Point and ColorPoint" and exactly how the latter can be made a subtype of the former — which intuitively seems obvious, but practically raises various issues.

A real-world example of a problem with polymorphism can be seen with the GtkMenuShell widget in the GTK+ widget set. This widget is used to create drop-down and pop-up menus. It does this in concert with GtkMenuItem which is a separate widget that displays a single item in a menu. GtkMenuShell is declared as a subtype of GtkContainer so that it can contain a collection of different GtkMenuItems, and can make use of the methods provided by GtkContainer to manage this collection.

The difficulty arises because GtkMenuShell is only allowed to contain GtkMenuItem widgets, no other sort of child widget is permitted. So, while it is permitted to add a GtkButton widget to a GtkContainer, it is not permitted to add that same widget to a GtkMenuShell.

If this restriction were to be encoded in the type system, GtkMenuShell would not be a true subtype of GtkContainer as it cannot be used in every place that a GtkContainer could be used — specifically it cannot be the target of gtk_container_add(myButton).

The simple solution to this is to not encode the restriction into the type system. If the programmer tries to add a GtkButton to a GtkMenuShell, that is caught as a run-time error rather than a compile-time error. To the pragmatist, this is a simple and effective solution. To the purist, it seems to defeat the whole reason we have static typing in the first place.

This example seems to give the flavor of subtype polymorphism quite nicely. It can be express a lot of type relationships well, but there are plenty of relationships it cannot express properly; cases where you need to fall back on run-time type checking. As such, it can be a reason to praise inheritance, and a reason to despise it.

Code reuse

The remaining core concept in inheritance is code reuse. When one class inherits from another, it not only gets to include fields from that class and to appear to be a subtype of that class, but also gets access to the implementation of that class and can usually modify it in interesting ways.

Code reuse is, of course, quite possible without inheritance, as we had libraries long before we had objects. Doing it with inheritance seems to add an extra dimension. This comes from the fact that when some code in the parent class calls a particular method on the object, that method might have been replaced in the child object. This provides more control over the behavior of the code being reused, and so can make code reuse more powerful. A similar thing can be achieved in a C-like language by explicitly passing function pointers to library functions as is done with qsort(). That might feel a bit clumsy, though, which would discourage frequent use.

This code reuse may seem as though it is just the flip-side of subtype inheritance, which was, after all, motivated by the value of using code from an ancestor to help implement a new class. In many cases, there is a real synergy between the two, but it is not universal. The classic examination of this issue is a paper by William R. Cook that examines the actual uses of inheritance in the Smalltalk-80 class library. He found that the actual subtype hierarchy (referred to in the paper as protocol conformance) is quite different from the inheritance hierarchy. For this code base at least, subtypes and code reuse are quite different things.

As different languages have experimented with different perspectives on object-oriented programming, different attitudes to these two or three different faces have resulted in widely different implementations of inheritance. Possibly the place that shows this most clearly is multiple inheritance. When considering subtypes, multiple inheritance makes perfect sense as it is easy to understand how one object can have two orthogonal sets of behaviors which make it suitable to be a member of two super-types. When considering implementation inheritance for code reuse, multiple inheritance doesn't make as much sense because the different ancestral implementations have more room to trip over each other. It is probably for this reason that languages like Java only allow a single ancestor for regular inheritance, but allow inheritance of multiple "interfaces" which provide subtyping without code reuse.

In general, having some confusion over the purpose of inheritance can easily result in confusion over the use of inheritance in the mind of the programmer. This confusion can appear in different ways, but perhaps the most obvious is in the choice between "is-a" relationships and "has-a" relationships that is easy to find being discussed on the Internet. "is-a" reflects subtyping, "has-a" can provide code reuse. Which is really appropriate is not always obvious, particularly if the language uses the same syntax for both.

Is inheritance spent?

Having these three very different concepts all built into the one concept of "inheritance" can hardly fail to result in people developing very different understandings. It can equally be expected to result in people trying to find a way out of the mess. That is just what we see in Go and Rust.

While there are important differences, there are substantial similarities between the type systems of the two languages. Both have the expected scalars (integers, floating point numbers, characters, booleans) in various sizes where appropriate. Both have structures and arrays and pointers and slices (which are controlled pointers into arrays). Both have functions, closures, and methods.

But, importantly, neither have classes. With inheritance largely gone, the primary tool for inheritance — the class — had to go as well. The namespace control provided by classes is left up to "package" (in Go) or "module" (in Rust). The data declarations are left up to structures. The use of classes to store a collection of methods has partly been handed over to "interfaces" (Go) or "traits" (Rust), and partly been discarded.

In Go, a method can be defined anywhere that a function can be defined — there is simply an extra bit of syntax to indicate what type the method belongs to — the "receiver" of the method. So:

    func (p *Point) Length() float64 {
	return math.Sqrt(p.x * p.x + p.y * p.y)
    }

is a method that applies to a Point, while:

    func Length(p *Point) float64 {
	return math.Sqrt(p.x * p.x + p.y * p.y)
    }

would be a function that has the same result. These compile to identical code and when called as "p.Length()" and "Length(&p)" respectively, identical code is generated at the call sites.

Rust has a somewhat different syntax with much the same effect:

    impl Point {
	fn Length(&self) -> float {
	    sqrt(self.x * self.x + self.y * self.y)
	}
    }

A single impl section can define multiple methods, but it is perfectly legal for a single type to have multiple impl sections. So while an impl may look a bit like a class, it isn't really.

The "receiver" type on which the method operates does not need to be a structure — it can be any type though it does need to have a name. You could even define methods for int were it not for rules about method definitions being in the same package (or crate) as the definition of the receiver type.

So in both languages, methods have managed to escape from existing only in classes and can exist on their own. Every type can simply have some arbitrary collection of methods associated with it. There are times though when it is useful to collect methods together into groups. For this, Go provides "interfaces" and Rust provides "traits".

    type file interface {
	Read(b Buffer) bool
	Write(b Buffer) bool
	Close()
    }
    trait file {
	fn Read(&self, b: &Buffer) -> bool;
	fn Write(&self, b: &Buffer) -> bool;
	fn Close(&self);
    }

These two constructs are extremely similar and are the closest either language gets to "classes". They are however completely "virtual". They (mostly) don't contain any implementation or any fields for storing data. They are just sets of method signatures. Other concrete types can conform to an interface or a trait, and functions or methods can declare parameters in terms of the interface or traits they must conform to.

Traits and interfaces can be defined with reference to other traits or interfaces, but it is a simple union of the various sets of methods.

    type seekable interface {
	file
	Seek(offset u64) u64
    }

trait seekable : file { fn Seek(&self, offset: u64) -> u64; }

No overriding of parameter or return types is permitted.

Both languages allow pointers to be declared with interface or trait types. These can point to any value of any type that conforms to the given interface or trait. This is where the real practical difference between the Length() function and the Length() method defined earlier becomes apparent. Having the method allows a Point to be assigned to a pointer with the interface type:

    type measurable interface {
        Length() float64
    }
The function does not allow that assignment.

Exploring the new inheritance

Here we see the brave new world of inheritance. It is nothing more or less than simply sharing a collection of method signatures. It provides simple subtyping and doesn't even provide suggestions of code reuse or structure embedding. Multiple inheritance is perfectly possible and has a simple well-defined meaning. The diamond problem has disappeared because implementations are not inherited. Each method needs to be explicitly implemented for each concrete type so the question of conflicts between multiple inheritance paths simply does not arise.

This requirement to explicitly implement every method for every concrete type may seem a little burdensome. Whether it is in practice is hard to determine without writing a substantial amount of code — an activity that current time constraints don't allow. It certainly appears that the developers of both languages don't find it too burdensome, though each has introduced little shortcuts to reduce the burden somewhat.

The "mostly" caveat above refers to the shortcut that Rust provides. Rust traits can contain a "default" implementation for each method. As there are no data fields to work with, such a default cannot really do anything useful and can only return a constant, or call other methods in the trait. It is largely a syntactic shortcut, without providing any really inheritance-like functionality. An example from the Numeric Traits bikeshed is

    trait Eq {
        fn eq(&self, other: &Self) -> bool { return !self.ne(other) };
        fn ne(&self, other: &Self) -> bool { return !self.eq(other) };
    }

In this example it is clear that the defaults by themselves do not provide a useful implementation. The real implementation is expected to define at least one of these methods to something meaningful for the final type. The other could then usefully remain as a default. This is very different from traditional method inheritance, and is really just a convenience to save some typing.

In Go, structures can have anonymous members much like those in C11 described earlier. The methods attached to those embedded members are available on the embedding structure as delegates: if a method is not defined on a structure it will be delegated to an anonymous member value which does define the method, providing such a value can be chosen uniquely.

While this looks a bit more like implementation inheritance, it is still quite different and much simpler. The delegated method can only access the value it is defined for and can only call the methods of that value. If it calls methods which have been redefined for the embedding object, it still gets the method in the embedded value. Thus the "extra dimension" of code reuse mentioned earlier is not present.

Once again, this is little more than a syntactic convenience — undoubtedly useful but not one that adds new functionality.

Besides these little differences in interface declarations, there are a couple of significant differences in the two type systems. One is that Rust supports parameterized types while Go does not. This is probably the larger of the differences and would have a pervasive effect on the sort of code that programmers write. However, it is only tangentially related to the idea of inheritance and so does not fit well in the present discussion.

The other difference may seem trivial by comparison — Rust provides a discriminated union type while Go does not. When understood fully, this shows an important difference in attitudes towards inheritance exposed by the different languages.

A discriminated union is much like a C "union" combined with an enum variable — the discriminant. The particular value of the enum determines which of the fields in the union is in effect at a particular time. In Rust this type is called an enum:

    enum Shape {
        Circle(Point, float),
        Rectangle(Point, Point)
    }

So a "Shape" is either a Circle with a point and a length (center and radius) or a Rectangle with two points (top left and bottom right). Rust provides a match statement to access whichever value is currently in effect:

    match myshape {
	Circle(center, radius) => io::println("Nice circle!");
	Rectangle(tl, br) => io::println("What a boring rectangle");
    }

Go relies on interfaces to provide similar functionality. A variable of interface type can point to any value with an appropriate set of methods. If the types to go in the union have no methods in common, the empty interface is suitable:

    type void interface {
    }

A void variable can now point to a circle or a rectangle.

    type Circle struct {
	center Point
	radius float
    }
    type Rectangle struct {
	top_left, bottom_right Point
    }

Of course it can equally well point to any other value too.

The value stored in a void pointer can only be accessed following a "type assertion". This can take several forms. A nicely illustrative one for comparison with Rust is the type switch.

    switch s := myshape.(type) {
    case Circle:
	printString("Nice circle!")
    case Rectangle:
	printString("What a boring rectangle")
    }

While Rust can equally create variables of empty traits and can assign a wide variety of pointers to such variables, it cannot copy Go's approach to extracting the actual value. There is no Rust equivalent of the "type assertion" used in Go. This means that the approaches to discriminated union in Rust and Go are disjoint — Go has nothing like "enum" and Rust has nothing like a "type assertion".

While a lot could be said about the comparative wisdom and utility of these different choices (and, in fact, much has been said) there is one particular aspect which relates to the topic of this article. It is that Go uses inheritance to provide discriminated unions, while Rust provides explicit support.

Are we moving forward?

The history of programming languages in recent years seems to suggest that blurring multiple concepts into "inheritance" is confusing and probably a mistake. The approach to objects and methods taken by both Rust and Go seem to suggest an acknowledgment of this and a preference for separate, simple, well-defined concepts. It is then a little surprising that Go chooses to still blend two separate concepts — unions and subtyping — into one mechanism: interfaces.

This analysis only provides a philosophical objection to that blend and as such it won't and shouldn't carry much weight. The important test is whether any practical complications or confusions arise. For that we'll just have to wait and see.

One thing that is clear though is that the story of the development of the object-oriented programming paradigm is a story that has not yet been played out — there are many moves yet to make. Both Rust and Go add some new and interesting ideas which, like languages before them, will initially attract programmers, but will ultimately earn both languages their share of derision, just as there are plenty of detractors for C++ and Java today. They nonetheless serve to advance the art and we can look forward to the new ideas that will grow from the lessons learned today.

Comments (32 posted)

Brief items

Quotes of the Week

The trade is young. We build web sites like people without nails and lumber and other standard components build homes: We take whatever materials we have at hand and stack them on top of each other until it resembles a structure.
Justin Crawford

The novice thought for a few moments, then asked: “Surely some of these could be made more consistent, so as to be easier to remember in the heat of coding?”

Master Git snapped his fingers. A hobgoblin entered the room and ate the novice alive. In the afterlife, the novice was enlightened.

Steve Losh

Comments (6 posted)

GDB 7.6 released

Version 7.6 of the GDB debugger is out. New features include 64-bit ARM support, an update of the C++ ABI to the GNU v3 version (" This has been the default for GCC since November 2000"), some Python scripting improvements, a number of new options, and more.

Full Story (comments: none)

Wayland development plans posted

Wayland developer Kristian Høgsberg has sent out a message detailing his plans for Wayland 1.2 and beyond. "What I'd like to do is to release a new major version every quarter. So we'll be aiming for 1.2 end of June, 1.3 end of September and so on. The motivation for this is that we have a lot of new features and new protocol in the works and a time-based release schedule is a good way to flush out those features. Instead of dragging out a release while waiting for a feature to become ready, we release on a regular schedule to make sure the features that did land get released on time." See the full text for details on the desired features for the 1.2 release.

Full Story (comments: 1)

Open Build Service version 2.4 released

Open Build Service (OBS) 2.4 has been released. "With OBS 2.4 it is now possible to build packages in the PKGBUILD format used for instance by the popular Arch Linux distribution. This is the third package format, after RPM and DEB, supported by the OBS which makes it feasible to build and ship software for all the major Linux distributions that use a binary package format. Another popular demand for build servers these days is the support for signing individual files (bootloader, driver etc.) inside packages with a cryptographic key to support standards like UEFI secure boot. In version 2.4 the OBS sign daemon has been extend to handle this security feature. And with the rise of App-Stores as means to distribute software to end users this OBS release brings support for the cross-distribution application metadata standard AppStream."

Comments (10 posted)

Newsletters and articles

Development newsletters from the past week

Comments (none posted)

Development Update, Schedule, and Funds (OpenShot blog)

Hot on the heels of a successful Kickstarter campaign, the OpenShot video editor project has announced its schedule (still targeting December 2013, more detailed plan coming soon) and a switch to Qt 5. "One of the biggest decisions we have made so far is to switch the user interface toolkit that OpenShot uses from GTK+ to Qt. We have carefully considered our options, and our team believes it is the only reasonable path forward. A few big contributing factors to choosing Qt was the performance of embedding HTML and JavaScript (for our timeline and curve editing widgets), native-looking widget rendering (on Mac, Windows, and Linux), improved tools for designing interfaces, and the easy ability to use OpenGL to display our video preview widget."

Comments (20 posted)

Packard: Shared Memory Fences

Keith Packard has posted another update on the ongoing work to revise the Direct Rendering Infrastructure in X. This post covers the GPU serialization mechanism implemented in SyncFences. "SyncFences are the same as Events as provided by Python and other systems. Of course all of the names have been changed to keep things interesting. I’ll call them Fences here, to be consistent with the current X usage."

Comments (none posted)

Mozilla: Push notifications for the Open Web

Mozilla has unveiled Simple Push, a new web API for "push" notifications. A blog post explaining the API notes several important design choices, such as the fact that developers do not need to pre-register their applications in order to receive notifications, and that users do not need to log in to a third-party service in order to receive notifications. "Web applications are freed from repeatedly polling for updates, leading to a better experience for everybody. Users get better battery life and more responsive applications. Developers don’t have to re-implement polling logic in every application they write. Mobile devices benefit from intelligent scheduling of notifications to reduce network usage and further improve battery life."

Comments (none posted)

Page editor: Nathan Willis

Announcements

Brief items

The SFC aims to create better non-profit accounting software

The Software Freedom Conservancy has announced a campaign to raise money and hire a developer to produce a useful, free-software accounting system aimed at the needs of non-profit organizations. "Indeed, Conservancy reached out into the broader fiscal sponsorship community beyond the FLOSS NPO community and discovered that many larger fiscal sponsors — even those willing to use proprietary components — have cobbled together their own unique systems, idiosyncratically tailored to their specific environments. Thus, good, well-designed, and reusable accounting software for non-profit fiscal sponsorship is not just missing in the software freedom community; it's missing altogether." The goal is to raise $75,000 for the first year's worth of work.

Comments (8 posted)

FSF certifies ThinkPenguin USB Wifi adapter

The Free Software Foundation has awarded Respects Your Freedom (RYF) certification to the TPE-N150USB Wireless N USB Adapter, sold by ThinkPenguin. "The RYF certification mark means that the product meets the FSF's standards in regard to users' freedom, control over the product, and privacy."

Full Story (comments: none)

Upcoming Events

Workshop on Essential Abstractions in GCC, 2013

The Workshop on Essential Abstractions in GCC will take place June 29-July 3 in Bombay, India.

Full Story (comments: none)

Events: May 2, 2013 to July 1, 2013

The following event listing is taken from the LWN.net Calendar.

Date(s)EventLocation
May 1
May 3
DConf 2013 Menlo Park, CA, USA
May 2
May 4
Linuxwochen Wien 2013 Wien, Austria
May 9
May 12
Linux Audio Conference 2013 Graz, Austria
May 10 Open Source Community Summit Washington, DC, USA
May 10 CentOS Dojo, Phoenix Phoenix, AZ, USA
May 14
May 15
LF Enterprise End User Summit New York, NY, USA
May 14
May 17
SambaXP 2013 Göttingen, Germany
May 15
May 19
DjangoCon Europe Warsaw, Poland
May 16 NLUUG Spring Conference 2013 Maarssen, Netherlands
May 22
May 23
Open IT Summit Berlin, Germany
May 22
May 24
Tizen Developer Conference San Francisco, CA, USA
May 22
May 25
LinuxTag 2013 Berlin, Germany
May 23
May 24
PGCon 2013 Ottawa, Canada
May 24
May 25
GNOME.Asia Summit 2013 Seoul, Korea
May 27
May 28
Automotive Linux Summit Tokyo, Japan
May 28
May 29
Solutions Linux, Libres et Open Source Paris, France
May 29
May 31
Linuxcon Japan 2013 Tokyo, Japan
May 30 Prague PostgreSQL Developers Day Prague, Czech Republic
May 31
June 1
Texas Linux Festival 2013 Austin, TX, USA
June 1
June 2
Debian/Ubuntu Community Conference Italia 2013 Fermo, Italy
June 1
June 4
European Lisp Symposium Madrid, Spain
June 3
June 5
Yet Another Perl Conference: North America Austin, TX, USA
June 4 Magnolia CMS Lunch & Learn Toronto, ON, Canada
June 6
June 9
Nordic Ruby Stockholm, Sweden
June 7
June 8
CloudConf Paris, France
June 7
June 9
SouthEast LinuxFest Charlotte, NC, USA
June 8
June 9
AdaCamp San Francisco, CA, USA
June 9 OpenShift Origin Community Day Boston, MA, USA
June 10
June 14
Red Hat Summit 2013 Boston, MA, USA
June 13
June 15
PyCon Singapore 2013 Singapore, Republic of Singapor
June 17
June 18
Droidcon Paris Paris, France
June 18
June 20
Velocity Conference Santa Clara, CA, USA
June 18
June 21
Open Source Bridge: The conference for open source citizens Portland, Oregon, USA
June 20
June 21
7th Conferenza Italiana sul Software Libero Como, Italy
June 22
June 23
RubyConf India Pune, India
June 26
June 28
USENIX Annual Technical Conference San Jose, CA, USA
June 27
June 30
Linux Vacation / Eastern Europe 2013 Grodno, Belarus
June 29
July 3
Workshop on Essential Abstractions in GCC, 2013 Bombay, India

If your event does not appear here, please tell us about it.

Page editor: Rebecca Sobol


Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds