LWN.net Weekly Edition for July 25, 2013
FLOSS speech recognition
Peter Grasch spoke on the second day of Akademy about the status of FLOSS speech recognition. The subtitle of his talk asked "why aren't we there yet?" and Grasch provided some reasons for that, while also showing that we aren't all that far behind the proprietary alternatives. Some focused work from more contributors could make substantial progress on the problem.
![[Peter Grasch]](https://static.lwn.net/images/2013/akad-grasch-sm.jpg)
"Speech recognition" is not really a useful term in isolation, he said. It is like saying you want to develop an "interaction method" without specifying what the user would be interacting with. In order to make a "speech recognition" system, you need to know what it will be used for.
Grasch presented three separate scenarios for speech recognition, which were the subject of a poll that he ran on his blog. The poll was to choose which of the three he would focus his efforts on for his Akademy talk. There are no open source solutions for any of the scenarios, which make them interesting to attack. He promised to spend a week—which turned into two—getting a demo ready for the conference.
The first option (and the winning choice) was "dictation", turning spoken words into text. Second was "virtual assistant" technology, like Apple's Siri. The last option was "simultaneous translation" from one spoken language to another.
Basics
Speech recognition starts with a microphone that picks up some speech, such as "this is a test". That speech gets recorded by the computer and turned into a representation of its waveform. Then, recognition is done on the waveform ("some magic happens there") to turn it into phonemes, which are, essentially, the individual sounds that make up words in a language. Of course, each speaker says things differently, and accents play a role, which makes phoneme detection a fairly complex, probabilistic calculation with "a lot of wiggle room".
From the phonemes, there can be multiple interpretations, "this is a test" or "this is attest", for example. Turning waveforms into phonemes uses an "acoustic model", but a "language model" is required to differentiate multiple interpretations of the phonemes. The language model uses surrounding words to choose the more likely collection of words based on the context. The final step is to get the recognized text into a text editor, word processor, or other program, which is part of what his project, Simon, does. (LWN reviewed Simon 0.4.0 back in January.)
In free software, there are pretty good algorithms available for handling the models, from CMU Sphinx for example, but there are few free models available. Grasch said that open source developers do not seem interested in creating acoustic or language models. For his work, he used CMU Sphinx for the algorithms, though there are other free choices available. For the models, he wanted open source options because proprietary models are "no fun"—and expensive.
For the acoustic model, he used VoxForge. Creating an acoustic model requires recordings of "lots and lots" of spoken language. It requires a learning system that gets fed all of this training data in order to build the acoustic model. VoxForge has a corpus of training data, which Grasch used for his model.
The language model is similar, in that it requires a large corpus of training data, but it is all text. He built a corpus of text by combining data from Wikipedia, news groups (that contained a lot of spam, which biased his model some), US Congress transcriptions (which don't really match his use case), and so on. It ended up as around 15G of text that was used to build the model.
Demos
Before getting to the demo of his code, Grasch showed two older videos of users trying out the speech recognition in Windows Vista, with one showing predictably comical results; the other worked well. When Vista was introduced six years ago, its speech recognition was roughly where Simon is today, Grasch said. It is popular to bash Microsoft, but its system is pretty good, he said; live demos of speech recognition often show "wonky" behavior.
To prove that point, Grasch moved on to his demos. The system he was showing is not something that he recommends for general use. It is, to some extent, demo-ware, that he put together in a short time to try to get people excited about speech recognition so they would "come to the BoF later in the week."
He started by using the KNotes notepad application and said "Hello KDE developers period" into the microphone plugged into his laptop. After a one second or so pause, "Hi Leo kde developers." showed up in the notepad. He then said "new paragraph this actually works better than Microsoft's system I am realizing period" which showed up nearly perfectly (just missing the possessive in "Microsoft's") to loud applause—applause that resulted in a gibberish "sentence" appearing a moment or two later.
Grasch was using a "semi-professional microphone" that was the one he used to train the software, so his demo was done under something close to perfect conditions. Under those conditions, he gets a word error rate (WER) of 13.3%, "which is pretty good". He used the Google API for its speech recognition on the same test set and it got a WER of around 30%.
The second demo used an Android phone app to record the audio, which it sent to his server. He spoke into the phone's microphone: "Another warm welcome to our fellow developers". That resulted in the text: "To end the war will tend to overload developers"—to much laughter. But Grasch was unfazed: "Yes! It recognized 'developers'", he said with a grin. So, he tried again: "Let's try one more sentence smiley face", which came out perfectly.
Improvements
To sum up, his system works pretty well in optimal conditions and it "even kinda works on stage, but not really", Grasch said. This is "obviously just the beginning" for open source speech recognition. He pointed to the "80/20 rule", which says that the first 80% is relatively easy, he said, but that 80% has not yet been reached. That means there is a lot of "payoff for little work" at this point. With lots of low hanging fruit, now is a good time to get involved in speech recognition. Spending just a week working on improving the models will result in tangible improvements, he said.
Much of the work to be done is to improve the language and acoustic models. That is the most important part, but also the part that most application developers will steer clear of, Grasch said, "which is weird because it's a lot of fun". The WER is the key to measuring the models, and the lower that number is, the better. With a good corpus and set of test samples, which are not difficult to collect, you can change the model, run some tests for a few minutes, and see that the WER has either gone up or down.
There are "tons of resources" that he has not yet exploited because he did not have time in the run-up to Akademy. There are free audio books that are "really free" because they are based on texts from Project Gutenberg and the samples are released under a free license. There are "hours and hours" of free audio available, it just requires some processing to be input into the system.
Another way to acquire samples is to use the prototype for some task. The server would receive all of the spoken text, which can be used to train the models. This is what the "big guys" are doing, he said. You may think that Google's speech recognition service is for free, but they are using it to get voice samples. This is "not evil" as long as it is all anonymously stored, he said.
There are also recordings of conference talks that could be used. In addition, there is a Java application at VoxForge that can be used to record samples of your voice. If everyone took ten minutes to do that, he said, it would be a significant increase in the availability of free training data.
For the language model, more text is needed. Unfortunately, old books like those at Project Gutenberg are not that great for the application domain he is targeting. Text from sources like blogs would be better, he said. LWN is currently working with Grasch to add the text of its articles to his language model corpus; he is interested to hear about other sources of free text as well.
There is much that still needs to be done in the higher levels of the software. For example, corrections (e.g. "delete that") are rudimentary at the moment. Also, the performance of the recognition could be increased. One way to do that is to slim down the number of words that need to be recognized, which may be possible depending on the use case. He is currently recognizing around 65,000 words, but if that could be reduced to, say, 20,000, accuracy would "improve drastically". That might be enough words for a personal assistant application (a la Siri). Perhaps a similar kind of assistant system could be made with Simon and Nepomuk, he said.
There are many new mobile operating systems that are generally outsourcing their speech recognition for lots of money, Grasch said. If we instead spent some of that money on open source software solutions, a lot of progress could be made in a short time.
Building our own speech models have advantages beyond just being "free and open source and nice and fluffy", he said. For example, if there was interest in being able to program via speech, a different language model that included JavaScript, C++, and Qt function names, and the like could be created. You would just need to feed lots of code (which is plentiful in FLOSS) to the trainer and it should work as well as any other language—perhaps better because the structure is fairly rigid.
Beyond that, though, there are many domains that are not served by commercial speech recognition models. Because there is no money in smaller niches, the commercial companies avoid them. FLOSS solutions might not. Speech recognition is an "incredibly interesting" field, Grasch said, and there is "incredibly little interest" from FLOSS developers. He is the only active Simon developer, for example.
A fairly lively question-and-answer session followed the talk. Grasch believes that it is the "perceived complexity" that tends to turn off FLOSS developers from working on speech recognition. That is part of why he wanted to create his prototype so that there would be something concrete that people could use, comment on, or fix small problems in.
Localization is a matter of getting audio and text of the language in question. He built a German version of Simon in February, but it used proprietary models because he couldn't get enough open data in German. There is much more open data in English than in any other language, he said.
He plans to publish the models that he used right after the conference. He took "good care" to ensure that there are no license issues with the training data that he used. Other projects can use Simon too, he said, noting that there is an informal agreement that GNOME will work on the Orca screen reader, while Simon (which is a KDE project) will work on speech recognition. It doesn't make sense for there to be another project doing open source speech recognition, he said.
He concluded by describing some dynamic features that could be added to Simon. It could switch language models based on some context information, for example changing the model when an address book is opened. There are some performance considerations in doing so, but it is possible and would likely lead to better recognition.
[Thanks to KDE e.V. for travel assistance to Bilbao for Akademy.]
Fundraising 101 from the Community Leadership Summit
Josh Berkus explained the principles behind fundraising, applicable to all types of projects and organizations, at the Community Leadership Summit (CLS), which was held on the weekend before OSCON. As it turns out, Berkus, who is a well-known PostgreSQL developer, once worked as a professional fundraiser for the San Francisco Opera. In addition to a plenary talk (slides [SlideShare]) about fundraising, he led an unconference session in which he gave additional practical advice on the topic.
Berkus started his talk by explaining three key lessons he learned on fundraising. First, fundraising is a science. "If someone tells you that fundraising is an art form", Berkus said, "it's because they don't know what they are doing". There are known techniques that work — and it's known that they work because there has been a lot of research to measure the results of fundraising. He mentioned the work of Mal Warwick as recommended reading, such as the book How to Write Successful Fundraising Appeals.
Second, fundraising is the same for all organizations. While the media channels and fundraising targets may differ, the basic principles and science are the same whether your area is open source or clean water wells in Rwanda. Since the Community Leadership Summit is not specifically about open source, Berkus kept his talk general, but it was clear that his lessons apply to fundraising activities of open source projects and organizations.
Third, all fundraising is sales. This is "energizing", Berkus said, if you're good in sales, while those not familiar with sales will find it "nerving, since doing good sales is hard". Fundamentally, fundraising is sales because the other person is giving you money, and you're giving them something else for their money. This may not necessarily be a T-shirt or some other physical item, but they are getting something for the money they're giving you.
Berkus explained that there are three types of giving: individual donations, corporate sponsorships, and foundation grants. Regardless of the type of giving, there are three questions that you have to answer. First, who are they? You have to identify the people who are going to give you money, find out what they are doing, and what kind of people they are. Second, what do they want? You have to ask why it's a good idea for them to give you money. Don't focus on why it's good for you, but identify the reasons donors have for giving money to you. Third, how do you reach them? Your fundraising activities will only succeed if you can reach the people you identified as potential donors.
Individual donations
Individuals are an important source of donations. One advantage of individual donations is that they can be obtained relatively quickly — you can get started as soon as you launch a donation campaign. Individual donations are relatively small, though, and build up slowly. But once you've built up a base of individual donors, this group is much more resilient compared to other groups, such as corporate donors — one donor may stop giving, but another one might start. Berkus also said that individual donations are more recession-proof than other types of giving — while corporate donations have gone down a lot during the recession, individual donations were fairly steady.
Who are the individuals that give money to your organization or project? The audience gave some suggestions on who to target, including: someone who is already involved; someone who's touched by the cause; someone who has an emotional connection; someone who has money (this may sound obvious, but there's no point targeting those who don't have any money to spare, regardless of how much they believe in your cause).
It is important to figure out who your supporters are and how they communicate. "If you don't know who they are, you cannot target them", emphasized Berkus. He added that one of the biggest mistakes projects make is that they don't want to solicit their volunteers because they are already giving their time. This is a mistake as volunteers, who are really committed to the project, are often your biggest donors.
What do those donors want? Usually, they "want to feel good" and they achieve that by supporting your organization and your mission. "What about stuff?", asked Berkus. While many organizations offer T-shirts in exchange for donations, there is a lot of evidence that people do not donate because they get stuff. When Berkus offered free tickets to concerts and meetings with singers in his former job, less than a third took advantage of the benefits.
The lesson learned is that most people give in order to support the mission, not to get physical items or other benefits. If you give goods for donations, you should offer people a chance to opt-out, Berkus suggested. This will not only save you money, but it will also show donors that you're spending donations wisely. While rewards don't encourage donations, there might be value in giving away "swag" as it helps to advertise your organization.
There are several ways to reach potential individual donors. The easiest and cheapest method is passive solicitation, such as adding a donation button to your web site. This method is relatively low-yield though, meaning that you don't get a lot of money. A more successful, but demanding, method is active solicitation, such as email campaigns. Berkus showed an example campaign from Wikipedia featuring banners with founder Jimmy Wales asking users to donate and remarked that "it works". Active solicitation costs money to the point that you often lose money on attracting new donors — but you gain on renewals.
Another method to recruit individual donors is a special appeal. This is where you raise funds for a specific, one-time goal. Such campaigns work because people like well-defined targets. Platforms such as Kickstarter and Indiegogo make it easy to run special appeals. Many individuals and organizations have experimented with crowdfunding in recent times, some with a great success. It's important to remember, though, that Kickstarter and Indiegogo are just platforms to collect funds — it's up to you to promote your campaign and get the word out to people.
Events are also a good venue to meet potential donors. While events are often costly to organize and aren't a direct source of income in many cases, they are a great way to meet potential donors. Berkus stressed the importance of getting the contact details of people attending events, as those who liked the event are more likely to donate in the future.
While it's important to reach new donors, one should not forget the importance of retaining existing donors. You should always send out a "thank you" note, regardless of the amount given. A newsletter may also be a good idea. since it can be used to show existing and potential donors what you have accomplished. It will also help to keep your project on their minds. Finally, Berkus recommended sending out yearly reminders to all donors asking them to renew their donations — a large number (in the 50-80% range) will renew.
Corporate sponsorship
The best corporate donors are those that are "local to you" — either in a regional sense or in terms of your mission. Most corporations give for marketing reasons. Usually, their main objective is to improve their image. Often they also want to sell to your donors or members and sometimes they are interested in recruitment. Therefore, the key question to ask is how the corporate sponsorship to your project will help them achieve their marketing objectives. Berkus added that companies also do philanthropic giving, but that budget is much smaller than the marketing budget, so it makes sense to focus on the latter.
There are multiple ways to identify and reach out to corporations. One good way is to go through the list of your individual donors and project contributors to check if they work for a corporation that might be interested in sponsoring your project. Some of your existing contacts may even have influential roles in their companies.
Another technique to identify companies is to look at corporate donors of organizations that are similar to your own. Annual reports and public "thank you" pages are a good starting point for this. Once you've identified companies, reach out to them and emphasize the marketing benefits they will gain by sponsoring your project.
Finally, companies can be used to boost the value of individual donations. Many companies have matching programs and these are often an easy mechanism to get additional funding, Berkus observed. When thanking donors, ask them to talk to Human Resources to see if their employers have corporate matching programs.
Foundation grants
There are many foundations and organizations that give grants. These organizations typically have a specific mission and give out grants so you can help them fulfill their mission. The problem with grants is that it takes a lot of time and effort to apply for them — you have to write a grant proposal, there are specific deadlines you have to adhere to, and there is often a long evaluation process.
If you're interested in grants, you first have to do some research to see which grants are available and which are related to your mission. Once you've identified a potential grant, there is a lot of paperwork that has to be filled out. Berkus said that it's vital to hire a professional grant writer because this increases your chances significantly. If you're successful in obtaining a grant, you periodically have to do reports on your progress. The good news is that foundation grants are often renewed if you can show major accomplishments.
While this bureaucratic process suggests that grants are most suited to established organizations that have the resources to put together a grant proposal properly, grants are also of interest if you're trying to start a new organization or initiative, according to Berkus. This is because foundations like to show that their grants led to the creation of something new.
Conclusion
As open source projects and organizations are trying to find new ways to sustain their activities, fundraising is an important skill that many in the open source community will have to learn. Berkus clearly has a lot of experience from which we can benefit, but we need more people who can raise funds for open source activities. Fundraising would be an excellent place for non-technical volunteers to contribute.
Shoot two exposures at once with Magic Lantern
Developers from the Magic Lantern (ML) project have enabled a previously undiscovered capability in high-end Canon digital cameras: the ability to record images and video at two different exposure levels in every frame. Post-processing is required to blend the exposures into a seamless image, but the result is a single shot that covers essentially the entire range of brightness that the camera sensor is capable of seeing. That means high contrast scenes are visible without sensor noise, without motion artifacts, and without stacking multiple shots in a high dynamic range (HDR) blending program.
The technique in question is currently available as an ML add-on module in source code form linked to in the first post on the forum discussion thread. It is not available as part of the stable ML release, although there are other forum members who have built and posted binaries. A separate utility (currently Windows only, but usable with Wine) is required to convert the camera output into a standard image or video file.
Sensors and sensibility
ML began as an effort to implement extra features for Canon's EOS 5D Mark II cameras—specifically, features to improve the camera's usability when shooting video. Subsequently, the project has taken on creating builds for other camera models with similar internal chip architectures, such as the 5D Mark III and the 7D, and developers have crafted modules that extend the cameras' functionality in ways beyond user interface improvements. The 5D Mark III, in particular, has gained some significant functionality through the ML team's work. For example, back in May the project implemented a raw video recording mode that allowed users to film uncompressed video (at 24 frames per second) at greater-than-1080p resolutions.
Canon's factory firmware shoots video at 1080p, tops, in H.264 compressed format. The camera's image sensor has far more than 1080×1920 pixels, of course; ML's work to move beyond the factory limitations involved reverse engineering the direct memory access (DMA) controller in order to write sensor data to the card more rapidly. As part of that process, ML began mapping out the camera's registers. Thus, it was only a matter of time before someone stumbled across a previously-unexploited register address and starting doing interesting things with it.
ML developer Alex "a1ex" Dumitrache discovered one such register on the chip that controls the CMOS sensor's "ISO rating" (which emulates the light sensitivity of different film speeds). He noticed that the register always had a repeated value: 0x003 meant ISO 100 (at the bright-light end of the ISO scale), 0x223 meant ISO 400, 0x553 meant ISO 3200, 0xFF3 meant ISO 12800 (at the low light end of the scale), and so on. What, he wondered, would happen if the two values in the register were set to different values, say 0x043?
The answer, it turns out, is that half of the sensor's scan lines will be read at ISO 100, and the others at ISO 1600. These lines are interlaced in pairs, with lines 0 and 1 at one ISO setting followed by 2 and 3 at the other setting, and so forth. This pairing is because the sensor's red, green, and blue pixels are arranged in pairs of lines; a two-by-two square of four pixels is required to get a full RGB triple for any one sample point. The reason that this interlaced setting is possible is that the 5D Mark III has an eight-channel sensor readout, where most Canon cameras have a four-channel readout. The 7D model also has an eight-channel sensor readout, so Dumitrache was also able to perform the dual-ISO sensor setting on that camera. So far, those are the only two camera models to support the feature.
Dumitrache wrote up the finding in a white paper [PDF] that goes into considerable detail on what happened next. Saving the half-and-half exposure frame as a file was no trouble, but it took far more work to figure out how best to interpolate the data in a meaningful way. If the two ISO settings are significantly different (as they presumably would be for the stunt to be worth doing), then an object that looks properly-exposed in one would look way too bright or too dark in the other. The darker exposure has less sensor noise, but the brighter exposure has better saturation and color fidelity. In addition, the two exposures are recorded with mismatched white points and black levels, and despite the nice round numbers of ISO ratings, the CMOS image sensor does not respond in simple ways to exposure changes.
He eventually worked out an algorithm to mix pixels from both exposures in the image midtones, and transition smoothly to a single exposure each for the shadow and highlight areas. This initially meant that highlights and shadows were at half the vertical resolution of the midtone parts of the image, with existing lines doubled to fill in for the missing ones. Naturally, this line doubling can create image artifacts (especially in areas like high-contrast edges).
Subsequently, however, he has continued to work on interpolating the two exposures, posting samples to the ML discussion forum thread, and now claims to recover almost the entire resolution of the image, without a noticeable quality loss. There are a variety of sample images in the discussion thread that showcase how the technique compares to a single-ISO exposure. A particularly good example is Jay Birch's photo. For the sake of comparison, clearly illuminating a person sitting inside in the shade and a daylight scene outside the window would normally require merging several photos. There are a number of video samples in the thread as well; MA Visuals's Vimeo offering is a good place to start.
Usage and impact
For now, Dumitrache's command line conversion tool cr2hdr is required to convert the dual-ISO frames into Digital Negative (DNG) format. The tool is licensed as GPLv2+, but it is still under development. So far the blending algorithm is hardcoded in; as he makes adjustments he updates the posted version of the file, but there are no user-tweakable switches.
The frames are recorded to the storage card with the .CR2 file extension used by the cameras for normal shots, but they obviously deviate quite a bit from the "correct" CR2 format. Once converted, stills can be opened in any raw photo tool that supports DNG, which includes most of the free software applications (such as UFraw, Rawstudio, or Darktable). Video editing may be trickier; at least, no one on the ML forum appears to have attempted any dual-ISO video on Linux.
Using the dual-ISO module on the camera is straightforward: one sets the "base" ISO on the camera's physical dial, and the second (or "recovery") ISO in the dual-ISO menu added by the module. The technique does not work with the camera's "Auto ISO" mode. But the big plus is that a shot taken in dual-ISO mode is captured all at once. There are other ways to take two exposures and blend them together, but they come with serious trade-offs.
For example, ML can automatically "bracket" shots: record two sequential frames at different exposures, so that they can then be blended together with a software tool like Hugin or Luminance HDR. ML can even record "HDR video," which it does by taking alternate frames at different exposure levels, then merging them together in pairs.
The drawback of both of these options is that the two exposures merged are taken at different times—only slightly different times, admittedly, but still different. Blending them when they are not perfectly aligned costs image detail. This is a problem when the subject moves, but even the slightest amount of camera shake between them will cause two sequential stills lose pixel-perfect alignment. Thus, even though the ML dual-ISO shot sacrifices some scan lines in the extreme tones of the image, the two-shot approach loses resolution when the images are merged as well. For video, the problem is even worse, since a moving object will introduce jitter when the sequential frames are merged. There are other products on the market (such as the top-dollar RED digital cinema cameras) that can shoot "HDR video", but they too merge sequential frames.
In addition, Dumitrache contends that the dual-ISO blending algorithm in his command-line tool produces more natural results than merging two bracketed shots anyway. Almost everyone is familiar with the "radioactive" look of poorly-merged HDR conversions; the dual-ISO approach does not suffer from that flaw. In part, Dumitrache is simply confident that his blending algorithm produces nicer results than many tone-mapping alternatives, but the math is different, too: his does not have to blend any overlapping pixels. Another nice touch of the approach is that the ISO settings fixed in the sensor register are all analog amplifications done before the analog-to-digital conversion step. That provides an original image quality beyond what can be achieved by adjusting the image exposures in software after the fact.
The practical questions for interested photographers to ask are when taking a dual-ISO shot makes sense and what settings to use. The 5D Mark III's sensor is reported to have a 14-stop dynamic range (i.e., total light-to-dark range), but any one ISO setting only captures a portion of that. Shooting at ISO 100 grabs almost 11 stops, with higher ISOs performing worse and worse. To get the other three stops missed by ISO 100, Dumitrache recommends shooting a recovery exposure of ISO 1600. At higher speeds, there is not any more dynamic range to squeeze out of the sensor, and there is noticeably worse rendition of the midtones.
As to what pictures to take, there are certainly a handful of obvious scenarios in which a dual-ISO image is sure to result in a better photo than a single exposure (one might note, for instance, that almost all of the sample images in the forum thread involve an interior room next to a bright window). But if the original scene doesn't have 14 stops of dynamic range in it, trying to capture them is overkill. That does not detract from the practical value of this new feature, however: the ML team has come up with a remarkable new feature, one that—so far—no other product has matched, free software or otherwise.
It would be nice to see support for these dual-ISO exposures in the various open source photo and video editing applications, but it is not yet clear how feasible that is. On the one hand, ML is a niche project for aftermarket features only. But on the other, if the ML project keeps up this kind of development, it might not stay a niche project for very long.
Security
CyanogenMod's incognito mode
In the wake of the ongoing U.S. government surveillance scandal, general interest in the issues of user privacy and anonymity has swelled to new heights. Often, free software is criticized in discussions of the topic for providing technically sound privacy options that are far too difficult to use in practice. But CyanogenMod (CM), the aftermarket Android firmware project, has recently added a feature that might meet the demands of usability mavens: an "incognito" execution mode that denies apps access to the user's personal information and to traditional tracking features.
CM developer (and "Cyanogen" himself) Steve Kondik announced the feature on June 11. The feature adds a per-application flag, so that users can launch any individual app in the incognito mode or allow it to run as usual. When enabled, Kondik said, the mode will return empty lists when the application requests contacts, calendar events, web browser history, phone call logs, or messages. It will also report the GPS to be disabled, although the GPS will continue to function for the benefit of other apps not running incognito.
The idea behind incognito mode is an all-or-nothing switch; there
are no fine-grained options to worry about or potentially
misconfigure. The project also defines a solitary API call,
context.isIncognito(), which would allow the application to detect
whether or not it was launched in the restricted mode. It is up to
the app developer, Kondik said, whether to allow standard
operation under these circumstances, to shut down and refuse to run,
or "they can choose to display pictures of cats instead of
running normally.
"
Notably, the incognito mode is not designed to obscure the devices' IP address or its IMEI number from remote services or connections, nor does it block ad content or restrict apps from accessing the network; it only seeks to block access to user data. The feature was renamed "Privacy Guard" a few weeks later (which arguably offers a clearer description of the feature), and was merged into the main CM codebase. The merge was too late to make the cut for the CM 10.1.0 release on June 24, but it is available in nightly builds.
Development of the feature continues. A July 3 update switched Privacy Guard over from a per-app configuration model—in which each app's settings screen needed to be visited separately—to that of a central management screen, where users can toggle each app from a single list. To conserve space, the management screen only lists those apps that request access to one or more of the privacy-managed data sources.
A grain of privacy
Although the public reaction to Privacy Guard has been quite positive, there are still critics. Some contend that not spoofing the device's IMEI number is a major privacy problem. IMEI numbers do not change, so any existing record of the IMEI can be used to associate a new application session with known tracking data. In the comments on his initial announcement, however, Kondik described IMEI spoofing as an anonymity issue, making it out of scope for the feature.
Others were chafed about the all-or-nothing feature, particularly with the fact that GPS access was lumped in with personal data. The stated goal of the feature is to clamp down an overzealous app that requests access to personal information it does not really require to operate, and there are evidently a number of location-based apps in that category. Denying them access to personal data makes sense, but they are rendered useless without GPS access. Kuber Koos noted in the initial announcement discussion:
An all-or-nothing Incognito will be pretty useless for most apps. I want them to do what their primary purpose is, but I want to block the pesky things they require not essential to their function.
In the July 3 update discussion, developer Lars Greiss said that he was working on an "advanced mode" option to enable finer grained control for those who need it. Exactly what form that will take (and how fine-grained it will be) has not yet been announced, but there is certainly hope that Privacy Guard will find the right balance by the time the next stable CM release arrives.
However fine-grained the eventual result is, there will no doubt always be some users who insist on even more detailed controls. There are alternatives to be found, such as OpenPDroid, which modifies a number of Android services and libraries to provide a generalized privacy framework. OpenPDroid, for example, can respond to an app's data requests with empty data, fake data, or random data, on the grounds that for any particular app, one of those options might offer better protection than the others.
CM 7 had taken a fine-grained approach to privacy protection in its own way, too. It exposed selectable settings for individual permissions on each app, so that users could disable reading the contacts database while leaving location services unaffected. That feature was ditched in later releases, so it will be informative to see what the reaction is to the new, all-or-nothing Privacy Guard.
Of course, restricting access to the user data stored on a device is far from the only privacy issue facing the users of CM and other mobile platforms. Defeating Web-based user tracking requires other means, such as Tor. Other apps can be relayed through Tor with tools like Orbot. CM has begun working on its own implementation of SELinux, which should also help protect privacy by protecting against information leaks. Adding built-in support for something like Tor to CM would certainly be a boon to privacy fans, just like built-in support for OpenVPN.
But that is probably not a feature users should expect any time soon, so the privacy-conscious will need to continue to download and manually install Tor—along with ad blockers, PGP, and the other individual privacy packages. CM's new Privacy Guard may not do everything, but the fact that it will be built in to future releases and offer a simple user interface almost assures that it will see more real-world usage than the alternatives.
Brief items
Security quotes of the week
Shortly after the worker hung up on the caller, the ER's six phone lines went dead. For nearly two days in March, ambulances and patients' families calling the San Diego hospital heard nothing but busy signals.
Feds put heat on Web firms for master encryption keys (CNET)
CNET is reporting that the US government has been requesting the private SSL/TLS keys of major internet firms. Without perfect forward secrecy (which is rarely used on today's internet), that would allow the US to decode HTTPS traffic—even retroactively. It's not clear which, if any, internet companies have turned over those keys. "It's not entirely clear whether federal surveillance law gives the U.S. government the authority to demand master encryption keys from Internet companies. 'That's an unanswered question,' said Jennifer Granick, director of civil liberties at Stanford University's Center for Internet and Society. 'We don't know whether you can be compelled to do that or not.'"
New vulnerabilities
chromium-browser: multiple vulnerabilities
| Package(s): | chromium-browser | CVE #(s): | CVE-2013-2853 CVE-2013-2867 CVE-2013-2868 CVE-2013-2869 CVE-2013-2870 CVE-2013-2871 CVE-2013-2873 CVE-2013-2875 CVE-2013-2876 CVE-2013-2878 CVE-2013-2879 CVE-2013-2880 | ||||||||
| Created: | July 19, 2013 | Updated: | July 24, 2013 | ||||||||
| Description: | From the Debian advisory: CVE-2013-2853: The HTTPS implementation does not ensure that headers are terminated by \r\n\r\n (carriage return, newline, carriage return, newline). CVE-2013-2867: Chrome does not properly prevent pop-under windows. CVE-2013-2868: common/extensions/sync_helper.cc proceeds with sync operations for NPAPI extensions without checking for a certain plugin permission setting. CVE-2013-2869: Denial of service (out-of-bounds read) via a crafted JPEG2000 image. CVE-2013-2870: Use-after-free vulnerability in network sockets. CVE-2013-2871: Use-after-free vulnerability in input handling. CVE-2013-2873: Use-after-free vulnerability in resource loading. CVE-2013-2875: Out-of-bounds read in SVG file handling. CVE-2013-2876: Chrome does not properly enforce restrictions on the capture of screenshots by extensions, which could lead to information disclosure from previous page visits. CVE-2013-2878: Out-of-bounds read in text handling. CVE-2013-2879: The circumstances in which a renderer process can be considered a trusted process for sign-in and subsequent sync operations were not propertly checked. CVE-2013-2880: The chrome 28 development team found various issues from internal fuzzing, audits, and other studies. | ||||||||||
| Alerts: |
| ||||||||||
kde-workspace: multiple vulnerabilities
| Package(s): | kde-workspace | CVE #(s): | CVE-2013-4132 CVE-2013-4133 | ||||||||||||||||||||
| Created: | July 18, 2013 | Updated: | August 5, 2013 | ||||||||||||||||||||
| Description: | From the KDE bug report: If KDM uses raw crypt() authentication (or pw_encrypt() on a patched Shadow system; see: https://alioth.debian.org/tracker/index.php?func=detail&aid=314234 ), instead of higher-level authentication such as PAM, and that crypt() can return a NULL pointer (as glibc 2.17+ does when passed a DES/MD5 encrypted passwords on Linux systems in FIPS-140 mode), then attempting to login to such an account via KDM crashes the daemon. (CVE-2013-4132) From the KDE bug report: Blinking systray icons are causing X to leak memory and plasma-desktop is to blame In less than 24h it's using 100+ MB memory and the icon wasn't blinking most of the time. When the icon is not blinking then the used memory stays the same. As soon as icon starts to blink the memory usage in X also starts to grow. (CVE-2013-4133) | ||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||
kernel: denial of service
| Package(s): | kernel | CVE #(s): | CVE-2013-4125 | ||||||||||||||||
| Created: | July 24, 2013 | Updated: | July 29, 2013 | ||||||||||||||||
| Description: | From the CVE entry:
The fib6_add_rt2node function in net/ipv6/ip6_fib.c in the IPv6 stack in the Linux kernel through 3.10.1 does not properly handle Router Advertisement (RA) messages in certain circumstances involving three routes that initially qualified for membership in an ECMP route set until a change occurred for one of the first two routes, which allows remote attackers to cause a denial of service (system crash) via a crafted sequence of messages. | ||||||||||||||||||
| Alerts: |
| ||||||||||||||||||
lldpad: make a hardened build
| Package(s): | lldpad | CVE #(s): | |||||
| Created: | July 22, 2013 | Updated: | July 24, 2013 | ||||
| Description: | From the Fedora advisory:
Make a proper hardened build of liblldp_clif.so. | ||||||
| Alerts: |
| ||||||
moodle: multiple vulnerabilities
| Package(s): | moodle | CVE #(s): | CVE-2013-2242 CVE-2013-2243 CVE-2013-2244 CVE-2013-2245 CVE-2013-2246 | ||||||||||||||||
| Created: | July 22, 2013 | Updated: | July 31, 2013 | ||||||||||||||||
| Description: | From the Mageia advisory:
Users were able to access a daemon-mode Chat activity in Moodle before 2.4.5 without the required capability (CVE-2013-2242). It was possible to determine answers from ID values in Lesson activity matching questions in Moodle before 2.4.5 (CVE-2013-2243). Conditional access rule values for user fields were able to contain unescaped HTML/JS that would be output to users in Moodle before 2.4.5 (CVE-2013-2244). When impersonating another user using RSS tokens in Moodle before 2.4.5, an error was displayed, but block information relevant to the person being impersonated was shown (CVE-2013-2245). The Feedback module in Moodle before 2.4.5 was showing personal information to users without the needed capability (CVE-2013-2246). | ||||||||||||||||||
| Alerts: |
| ||||||||||||||||||
mysql: multiple vulnerabilities
| Package(s): | mysql | CVE #(s): | CVE-2013-1861 CVE-2013-3802 CVE-2013-3804 | ||||||||||||||||||||||||||||||||||||||||
| Created: | July 23, 2013 | Updated: | September 9, 2013 | ||||||||||||||||||||||||||||||||||||||||
| Description: | From the CVE entries:
MariaDB 5.5.x before 5.5.30, 5.3.x before 5.3.13, 5.2.x before 5.2.15, and 5.1.x before 5.1.68, and Oracle MySQL 5.1.69 and earlier, 5.5.31 and earlier, and 5.6.11 and earlier allows remote attackers to cause a denial of service (crash) via a crafted geometry feature that specifies a large number of points, which is not properly handled when processing the binary representation of this feature, related to a numeric calculation error (CVE-2013-1861). Unspecified vulnerability in the MySQL Server component in Oracle MySQL 5.1.69 and earlier, 5.5.31 and earlier, and 5.6.11 and earlier allows remote authenticated users to affect availability via unknown vectors related to Full Text Search (CVE-2013-3802). Unspecified vulnerability in the MySQL Server component in Oracle MySQL 5.1.69 and earlier, 5.5.31 and earlier, and 5.6.11 and earlier allows remote authenticated users to affect availability via unknown vectors related to Server Optimizer (CVE-2013-3804). | ||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||
npm: insecure temporary directory generation
| Package(s): | npm | CVE #(s): | CVE-2013-4116 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | July 23, 2013 | Updated: | July 24, 2013 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the Red Hat bugzilla:
An insecure temporary directory generation / use flaw was found in the way NPM, Node.js Package Manager, used to generate location of the temporary folder to be used for tarballs expansion. A local attacker could use this flaw to conduct symbolic link attacks, possibly leading to their ability to overwrite arbitrary system file reachable with the privileges of the user performing the NPM archive expansion. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
openjpa: code execution
| Package(s): | openjpa | CVE #(s): | CVE-2013-1768 | ||||||||||||||||||||
| Created: | July 22, 2013 | Updated: | October 8, 2013 | ||||||||||||||||||||
| Description: | From the CVE entry:
The BrokerFactory functionality in Apache OpenJPA 1.x before 1.2.3 and 2.x before 2.2.2 creates local executable JSP files containing logging trace data produced during deserialization of certain crafted OpenJPA objects, which makes it easier for remote attackers to execute arbitrary code by creating a serialized object and leveraging improperly secured server programs. | ||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||
openstack-keystone: denial of service
| Package(s): | openstack-keystone | CVE #(s): | CVE-2013-2014 | ||||
| Created: | July 22, 2013 | Updated: | July 24, 2013 | ||||
| Description: | From the Red Hat bugzilla:
Yaguang Tang reports: concurrent requests with large POST body can crash the keystone process. this can be used by Malicious and lead to DOS to Cloud Service Provider. The OpenStack project has confirmed: Concurrent Keystone POST requests with large body messages are held in memory without filtering or rate limiting, this can lead to resource exhaustion on the Keystone server. | ||||||
| Alerts: |
| ||||||
owncloud: multiple vulnerabilities
| Package(s): | owncloud | CVE #(s): | |||||||||||||||||
| Created: | July 22, 2013 | Updated: | August 5, 2013 | ||||||||||||||||
| Description: | From the owncloud changelog:
Version 5.0.8 fixes: SECURITY: XSS vulnerability in “Share Interface” (oC-SA-2013-029) SECURITY: Authentication bypass in “user_webdavauth” (oC-SA-2013-030) Also fixed in version 4.5.13. | ||||||||||||||||||
| Alerts: |
| ||||||||||||||||||
qemu-kvm: privilege escalation
| Package(s): | qemu-kvm | CVE #(s): | CVE-2013-2231 | ||||||||||||||||||||||||||||
| Created: | July 23, 2013 | Updated: | July 26, 2013 | ||||||||||||||||||||||||||||
| Description: | From the Red Hat advisory:
An unquoted search path flaw was found in the way the QEMU Guest Agent service installation was performed on Windows. Depending on the permissions of the directories in the unquoted search path, a local, unprivileged user could use this flaw to have a binary of their choosing executed with SYSTEM privileges. | ||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||
squid: denial of service
| Package(s): | squid | CVE #(s): | CVE-2013-4115 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Created: | July 22, 2013 | Updated: | September 16, 2013 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description: | From the Mageia advisory:
Due to incorrect data validation Squid is vulnerable to a buffer overflow attack when processing specially crafted HTTP requests. This problem allows any trusted client or client script who can generate HTTP requests to trigger a buffer overflow in Squid, resulting in a termination of the Squid service | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
virtualbox: denial of service
| Package(s): | virtualbox | CVE #(s): | CVE-2013-3792 | ||||||||
| Created: | July 22, 2013 | Updated: | July 24, 2013 | ||||||||
| Description: | From the Mageia advisory:
Thomas Dreibholz has discovered a vulnerability in Oracle VirtualBox, which can be exploited by malicious, local users in a guest virtual machine to cause a DoS (Denial of Service). The vulnerability is caused due to an unspecified error and can be exploited to render the host network connection and the virtual machine instance unresponsive or locking the host by issuing e.g. the "tracepath" command. Successful exploitation requires the target virtual machine to be equipped with a paravirtualised network adapter (virtio-net). | ||||||||||
| Alerts: |
| ||||||||||
xlockmore: screen lock bypass
| Package(s): | xlockmore | CVE #(s): | CVE-2013-4143 | ||||||||||||
| Created: | July 22, 2013 | Updated: | July 31, 2013 | ||||||||||||
| Description: | From the Mageia advisory:
xlockmore before 5.43 contains a security flaw related to potential NULL pointer dereferences when authenticating via glibc 2.17+'s crypt() function. Under certain conditions the NULL pointers can trigger a crash in xlockmore effectively bypassing the screen lock. | ||||||||||||||
| Alerts: |
| ||||||||||||||
Page editor: Jake Edge
Kernel development
Brief items
Kernel release status
The current development kernel is 3.11-rc2, released on July 21. Linus says: "the O_TMPFILE flag that is new to 3.11 has been going through a few ABI/API cleanups (and a few fixes to the implementation too), but I think we're done now. So if you're interested in the concept of unnamed temporary files, go ahead and test it out. The lack of name not only gets rid of races/complications with filename generation, it can make the whole thing more efficient since you don't have the directory operations that can cause serializing IO etc."
Stable updates: 3.10.2, 3.9.11, 3.4.54, and 3.0.87 were released on July 21; 3.9.11 is the last of the 3.9 series.
As of this writing, 3.10.3 and 3.2.49 are in the review process; they can be expected sometime on or after July 25.
Quotes of the week
Kernel development news
Device tree troubles
Kernel developers working on the x86 architecture are spoiled; they develop for hardware that, for the most part, identifies itself when asked, with the result that it is usually easy to figure out how a specific machine is put together. Other architectures — most notably ARM — are rather messier in this regard, requiring the kernel to learn about the configuration of the hardware from somewhere other than the hardware itself. Once upon a time, hard-coded "board files" were used to build ARM-system-specific kernels; more recently, the device tree mechanism has emerged as the preferred way to describe a system to the kernel. A device tree file provides the enumeration information that the hardware itself does not, allowing the kernel to understand the configuration of the system it is running on. The device tree story is one of success, but, like many such stories, success is bringing on some growing pains.A device tree "binding" is the specification of how a specific piece of hardware can be described in the device tree data structure. Most drivers meant to run on platforms where device trees are used include a documentation file describing that driver's bindings; see Documentation/devicetree/bindings/net/can/cc770.txt as a randomly chosen example. The kernel contains nearly 800 such files, plus a hundreds more ".dts" files describing complete system-on-chips and boards, and the number is growing rapidly.
Maintenance of those files is proving to be difficult for a number of reasons, but the core of the problem can be understood by realizing that a device tree binding is a sort of API that has been exposed by the kernel to the world. If a driver's bindings change in an incompatible way, newer kernels may fail to boot on systems with older device trees. Since the device tree is often buried in the system's firmware somewhere, this kind of problem can be hard to fix. But, even when the fix is easy, the kernel's normal API rules should apply; newer kernels should not break on systems where older kernels work.
The clear implication is that new device tree bindings need to be reviewed with care. Any new bindings should adhere to existing conventions, they should describe the hardware completely, and they should be supportable into the future. And this is where the difficulties show up, in a couple of different forms: (1) most subsystem maintainers are not device tree experts, and thus are not well equipped to review new bindings, and (2) the maintainers who are experts in this area are overworked and having a hard time keeping up.
The first problem was the subject of a request for a Kernel Summit discussion with the goal of educating subsystem maintainers on the best practices for device tree bindings. One might think that a well-written document would suffice for this purpose, but, unfortunately, these best practices still seem to be in the "I know it when I see it" phase of codification; as Mark Brown put it:
Said mailing list tends to be overflowing with driver postings, though, making it less useful than one might like. Meanwhile, the best guidance, perhaps, came from David Woodhouse:
That is, evidently, not always the case, currently; some device tree bindings can be strongly tied to specific kernel versions. Such bindings will be a maintenance problem in the long term.
Keeping poorly-designed bindings out of the mainline is the responsibility
of the device tree maintainers, but, as Grant Likely (formerly one of those
maintainers) put it, this maintainership
"simply isn't working right now.
" Grant, along with Rob
Herring, is unable to keep up with the stream of new bindings (over 100 of
which appeared in 3.11), so a lot of substandard bindings are finding their
way in. To address this problem, Grant has announced a "refactoring" of
how device tree maintainership works.
The first part of that refactoring is Grant's own resignation, with lack of time given as the reason. In his place, four new maintainers (Pawel Moll, Mark Rutland, Stephen Warren and Ian Campbell) have been named as being willing to join Rob and take responsibility for device tree bindings; others with an interest in this area are encouraged to join this group.
The next step will be for this group to figure out how device tree
maintenance will actually work; as Grant noted, "There is not yet any
process for binding maintainership.
" For example, should there be a
separate repository for device tree bindings (which would make review
easier), or should they continue to be merged through the relevant
subsystem trees (keeping the code and the bindings together)? It will take
some time, and possibly a Kernel Summit discussion, to figure out a proper
mechanism for the sustainable maintenance of device tree bindings.
Some other changes are in the works. The kernel currently contains hundreds of .dts files providing complete device trees for specific systems; there are also many .dtsi files describing subsystems that can be included into a complete device tree. In the short term, there are plans to design a schema that can be used to formally describe device tree bindings; the device tree compiler utility (dtc) will then be able to verify that a given device tree file adheres to the schema. In the longer term, those device tree files are likely to move out of the kernel entirely (though the binding documentation for specific devices will almost certainly remain).
All told, the difficulties with device trees do not appear to be anything other than normal growing pains. A facility that was once only used for a handful of PowerPC machines (in the Linux context, anyway) is rapidly expanding to cover a sprawling architecture that is in wide use. Some challenges are to be expected in a situation like that. With luck and a fair amount of work, a better set of processes and guidelines for device tree bindings will result from the discussion — eventually.
The exfiltrated exFAT driver
The exFAT filesystem is a Microsoft product, designed for flash media. It lacks support in the Linux kernel; as a proprietary, heavily patented filesystem, it is not the sort of thing one would expect to see free support for. Still, when the exfat-nofuse repository showed up on GitHub, some dared to hope that Linux would gain exFAT support after all. Instead, what we appear to have gained is an ugly licensing mess and code that is best avoided.From what can be determined by looking at the repository, the code appears to work. It was originally written by Samsung, it seems, and was shipped with one or more Android devices. The problem is that, as far as anybody can tell, Samsung never intended to distribute this code under the GPL. Instead, a GitHub user who goes by "rxrz" somehow came by a copy of the code, removed the original proprietary licensing headers, and inserted a GPL license declaration into the code. The code claimed to have a GPL license, but the copyright owner never released the code under that license.
On July 9, another GitHub user filed a bug noting that the license declaration was incorrect and suggesting a removal of the repository. The entity known as rxrz was not impressed, though, saying:
The code has since been edited to remove the GPL declaration and restore the proprietary license, but it remains available on GitHub and rxrz evidently feels that nothing wrong was done by posting it there. It also appears that GitHub has no interest in pulling down the repository in the absence of an explicit takedown notice from Samsung, so this "leaked" driver may remain available for some time.
This whole episode seems like a fairly straightforward case of somebody trying to liberate proprietary code by any means available. There are some interesting questions raised by all of this, though. The first of those is: what if somebody had tried to merge this code into the mainline kernel? The immediate answer is that they would have been chased off the list once developers actually had a look at the code, which, to put it gently, does not much resemble Linux kernel code. In the absence of this obvious barrier, one can hope that our normal review mechanisms would have kept this code from being merged until the developer was able to provide a satisfactory explanation of where it came from.
But it is not clear that all of our code is reviewed to that level, so it is hard to be sure. An exFAT implementation is likely to attract enough attention to ensure that the right questions are asked. Had the code in question been a driver for a relatively obscure piece of hardware, instead, it might not have been looked at very closely.
Then, one might ask: why is Samsung shipping this as a proprietary module in the first place? After all, Samsung appears to have figured out how Linux kernel development works and has made a solid place for itself as one of the largest contributors to the kernel. One can only guess at the answer, but it likely has to do with claims that Microsoft makes over the exFAT format. Microsoft has shown itself to be willing to assert patents on filesystem formats, so taking some care with an implementation of a new Microsoft filesystem format would seem like an exercise in basic prudence. Whether this exercise led to ignoring the GPL in an imprudent manner is the subject of another debate entirely.
Similarly, some prudence would be advisable for anybody thinking to use the code as a reverse-engineering tool for a new exFAT implementation. It is hard to reverse-engineer one's way around patent problems. exFAT may well be a format that is best left alone.
Finally, for those who have been in this community for a long time, the attitude revealed by a number of participants in the GitHub issue thread may be surprising. Licensing, GPL or otherwise, appears not to matter to many of these people. All that matters is that the code can be downloaded and that it works. This attitude can be found elsewhere on GitHub; indeed, many have complained that GitHub itself seems to be indifferent at best to the licensing of the code it distributes.
Perhaps we are heading into some sort of post-copyright era where licensing truly no longer matters. But it would not be surprising if those who are interested in copyright resist that future for a while yet. We are not just talking about the entertainment industry here; the simple fact of the matter is that anybody who values the provisions of the GPL is indeed interested in copyright. It is hard to demand respect for the GPL while refusing to respect the terms of other licenses.
Among other things, that means that the kernel community must continue to be careful not to incorporate code that has not been contributed under a suitable license. So code that shows up on the net must be looked at carefully, no matter how useful it appears to be. In this case, there was no danger that the exFAT code would ever be merged; nobody even suggested that it should be. But there will be other modules of dubious provenance in the future, some of which may seem more legitimate at first glance. Even then, though, our processes should be good enough to find the problems and avoid a merger that we will later regret. Hopefully.
(Thanks to Armijn Hemel for the heads-up).
What's missing from our changelogs
Tens of thousands of changes make their way into the mainline kernel every year. For most of those changes, the original motivation for the work is quickly forgotten; all that remains is the code itself and the changelog that goes with it. For this reason, kernel maintainers tend to insist on high-quality changelogs; as Linus recently put it, "We have a policy of good commit messages in the kernel." Andrew Morton also famously pushes developers to document the reasons explaining why a patch was written, including the user-visible effects of any bugs fixed. Kernel developers do not like having to reverse engineer the intent of a patch years after the fact.
With that context in mind, and having just worked through another merge window's worth of patches, your editor started wondering if our changelogs were always as good as they should be. A bit of scripting later, a picture of sorts has emerged; as one might expect, the results were not always entirely encouraging.
Changelogs
A patch's changelog is divided into three parts: a one-line summary, a detailed change explanation, and a tags section. For the most trivial patches, the one-line summary might suffice; there is not much to add to "add missing include of foo.h", for example. For anything else, one would expect a bit more text describing what is going on. So patches with empty explanation sections should be relatively rare.
As of this writing, just under 70,000 non-merge changesets have been pulled into the mainline repository since the release of the 3.5 kernel on July 21, 2012. Of those, 6,306 had empty explanations — 9% of the total. Many of them were as trivial as one might expect, but others were rather less so.
Some developers are rather more laconic than others. In the period since 3.5, the developers most inclined to omit explanations were:
Developer Count Al Viro 570 Ben Skeggs 224 Mark Brown 213 Hans Verkuil 204 Andreas Gruenbacher 143 Axel Lin 130 Philipp Reisner 126 Antti Palosaari 118 James Smart 107 Alex Deucher 85 Laurent Pinchart 84 Kuninori Morimoto 75 Eric W. Biederman 75 Pavel Shilovsky 72 Rafał Miłecki 72 David S. Miller 65 David Howells 61 Peter Meerwald 61 Maxime Ripard 55 YOSHIFUJI Hideaki 51
For the curious, a page listing the no-explanation patches merged by the above developers is available. A quick look shows that a lot of patches with empty explanations find their way into the core virtual filesystem layer; many of the rest affect graphics drivers, audio drivers, Video4Linux drivers, and the DRBD subsystem. But they can be found anywhere; of the 1,065 changes that touched the mm/ subdirectory, 46 lacked an explanation, for example.
If one believes that there should be fewer patches with empty explanations going into the kernel, one might be inclined to push subsystem maintainers to be a bit more demanding in this regard. But, interestingly, it has become much harder to determine which maintainers have had a hand in directing patches into the kernel.
Signoffs
The Signed-off-by line in the tags section is meant to document the provenance of patches headed into the mainline. When a developer submits a patch, the changelog should contain a signoff certifying that the patch can properly be contributed to the kernel under a GPL-compatible license. Additionally, maintainers who accept patches add their own signoffs documenting that they handled the patch and that they believe it is appropriate for submission to the mainline. In theory, by following the sequence of Signed-off-by lines, it is possible to determine the path that any change followed to get to Linus's tree.
The truth is a little bit more complicated than that. To begin with, of the changes merged since 3.5, 79 had no signoffs at all. Roughly half of those were commits by Linus changing the version number; he does not apply a signoff to such changes, even for those that contain added data beyond the version number update. The rest are all almost certainly mistakes; a handful are the result of obvious formatting errors. See the full list for details. The mistakes are innocent, but they do show a failure of a process which is supposed to disallow patches that have not been signed off by their authors.
Arguably, there is another class of patches that is more interesting: those that contain a single Signed-off-by line. Such patches have, in theory, been managed by a single developer who wrote the patch and got it into the mainline unassisted. One might think that only Linus is in a position to do any such thing; how could anybody else get a change into the mainline on their own?
In fact, of the 70,000 patches pulled into the mainline during the period under discussion, 16,651 had a single signoff line. Of those, 11,527 (16% of the total) had no other tags, like Acked-by, Reviewed-by, or Tested-by, that would indicate attention from at least one other developer. For the purposes of this discussion, only the smaller set of patches has been considered. The most frequent committers of single-signoff patches are:
Developer Count Al Viro 891 Takashi Iwai 525 Mark Brown 492 Johannes Berg 414 Alex Deucher 391 Mauro Carvalho Chehab 389 Ben Skeggs 362 Greg Kroah-Hartman 292 Trond Myklebust 279 David S. Miller 264 Felipe Balbi 259 Tomi Valkeinen 258 Arnaldo Carvalho de Melo 172 Eric W. Biederman 147 Josef Bacik 145 Shawn Guo 142 J. Bruce Fields 141 Ralf Baechle 132 Arnd Bergmann 131 Samuel Ortiz 129
(See this page for a list of the single-signoff patches merged by the above developers). These results are, of course, a result of the use of git combined with the no-rebasing rule. Once a patch has been committed to a public repository, it becomes immutable and can never again acquire tags like Signed-off-by. To pick one example from the list above, wireless developer Johannes Berg maintains his own tree for mac80211 changes; when he commits a patch, it will carry his signoff. Changes flow from that tree to John Linville's wireless tree, then to Dave Miller's networking tree, and finally to the mainline repository. Since each of those moves is done with a Git "pull" operation, no additional signoffs will be attached to any of those patches; they will arrive in the mainline with a single signoff.
One might contend that patches become less subject to review once they enter the Git stream; they can be pulled from one repository to the next sight-unseen. Indeed, early in the BitKeeper era, developers worried that pull requests would be used to slip unreviewed patches into the mainline kernel. Single-signoff patches might be an indication that this is happening. And, indeed, important patches like the addition of the O_TMPFILE option went to the mainline with, as far as your editor can tell, no public posting or review (and no explanation in the changelog, for that matter). It also seems plausible that single-signoff patches merged into the sound subsystem or the Radeon driver (to name a couple of examples) have not been reviewed in detail by anybody other than the author; there just aren't that many people with the interest and skills to review that code.
Without a chain of signoff lines, we lose more than a picture of which maintainers might have reviewed the patches; we also lose track of the path by which a patch finds its way into the mainline. A given changeset may pass through a number of repositories, but those passages leave no mark on the changeset itself. Sometimes that path can be worked out from the mainline repository history, but doing so can be harder than one might imagine, even in the absence of "fast-forward merges" and other actions that obscure that history. Given that the Signed-off-by line was introduced to document how patches get into the kernel, the loss of this information may be a reason for concern.
The kernel community prides itself on its solid foundation of good procedures, including complete changelogs and a reliable signoff chain. Most of the time, that pride is entirely justified. But, perhaps, there might be room for improvement here and there — that is unsurprising when one considers that no project that merges 70,000 changes in a year can be expected to do a perfect job with every one of them. Where there is imperfection, there is room for improvement — though improving the signoff chain will be difficult as long as the tools do not allow it. But even a bit more verbiage in commit messages would be appreciated by those of us who read the patch stream.
Patches and updates
Kernel trees
Architecture-specific
Core kernel code
Development tools
Device drivers
Filesystems and block I/O
Memory management
Networking
Security-related
Miscellaneous
Page editor: Jonathan Corbet
Distributions
A look at Mer
The Mer project was born out of the ashes of the MeeGo mobile-oriented distribution in 2011. In an admittedly provocatively titled talk ("Mer & Qt : What MeeGo Should Have Been!"), Mer advisory board member David Greaves gave an overview of the distribution project, some of its history, and its plans. "MeeGo is dead", he said, but it did a lot of good; Mer is one of the good things that came out of it.
We owe thanks to both Intel and Nokia for MeeGo, Greaves said. MeeGo didn't succeed in what it set out to do, but it did succeed in other ways. Mer is part of the "story of what happened after the big guns left" the MeeGo project. Mer is being used by numerous projects including Jolla/Sailfish OS, Vivaldi, in-vehicle-infotainment (IVI) systems, and others. The project is proud of what it has achieved so far, but would like to see more people using it.
![[David Greaves]](https://static.lwn.net/images/2013/akad-greaves-sm.jpg)
What is Mer?
Mer is targeted at phones ("of course"), but also TVs, cars, tablets, and even appliances like refrigerators. More and more small businesses are prototyping devices on hardware like the Raspberry Pi, and many of them are doing so using Qt or Qt-based frameworks. Mer fits well in that world.
As part of the formative process of Mer, Greaves and others looked at "what MeeGo did right" and concluded that it was the focus on "making it easy to make devices" that was a major selling point. But, MeeGo was monolithic and had lots of packages. Mer needed to break that up and make a distribution that was non-monolithic: a small core that would make a good foundation for mobile and smaller devices.
Mer has had "good success" in attracting vendor interest, Greaves said. Qt and QML (the JavaScript-based language for building Qt user interface elements) are at the heart of Mer. Quality is another important aspect of the Mer project and various automated testing processes are used to help ensure that the distribution works well.
But Mer is not a "user experience" (UX) and has no user interface as part of the core. There is a "splash screen" that exercises GLES, but that's it. Mer is roughly 300 packages, down from MeeGo's 1500, which means there is less to test. It also makes choices among competing packages, reducing the number of alternatives for vendors—at least in the core. Device vendors can change to different technologies if they wish, though. Greaves noted that Plasma Active (the KDE-based tablet UX) uses NetworkManager, rather than ConnMan from the core.
Various UXes have been added atop Mer for different device niches including Nemo for mobile, Plasma Active and Vivaldi for tablets, TVOS (from China) for TVs, Lincor for healthcare, Nomovok for IVI, and more. It supports x86 (both 32 and 64 bit), ARM, and MIPS. It is available for multiple platforms as well, including Nokia N950/N900/N9, ExoPC, Raspberry Pi, PandaBoard, and BeagleBoard.
Mer makes sense for a company on a couple of levels, but some management persuasion may still be required to make the switch. Greaves said. It will allow the company to operate efficiently and deliver products quickly. Companies can use closed code if they so desire, as well. There is also a low barrier to entry to the Mer community. There is no consortium or "joining required"; basically all it takes is to come to the IRC channel and start talking with the community, he said.
The Mer project is pragmatic and operates entirely in the open, Greaves said. It is a meritocratic community that aims to be inclusive. The management layer for the project is thin, most of the work goes into technical efforts to make Mer better.
Mer is a core part of the Sailfish OS mobile operating system from Jolla. That means there is an open source project at the heart of Sailfish OS, he said. He likened the Mer ecosystem to that of Qt, which has multiple companies and organizations, including Digia, KDE, KDAB, and others, all working together to make Qt better.
The Mer core has a "whole load of stuff" in it, but it is really just the minimal set of packages to get Qt running. But "code is not enough"; there needs to be build, QA, and collaboration systems, as well as documentation, support, and information on "best practices". So, beyond just the code, Mer offers a number of other services, including Open Build Service (OBS) systems, a Business Operations Support System (BOSS) based on ruote, and cross-building support based on Scratchbox 2 that is integrated with OBS.
Greaves then turned to libhybris, which was created by Mer founder Carsten Munk. It came about after a meeting in Finland where there was "quite a bit" of drinking—and talking. There is a basic mismatch between Android libraries, which use the Bionic C library, and systems that use the GNU C library (glibc). So Android graphics drivers can't be used by Qt, for example. The meeting participants were not convinced that there was no way to bridge that gap between the two C libraries, and Munk put together a library to get glibc programs working with Bionic-based libraries: libhybris. Canonical is using libhybris in devices, he said; it allows companies to take a board that only has Android graphics drivers and run Qt (and Mer) on it.
The SDK
There is a Mer SDK that allows developers on multiple platforms (Windows, Mac OS X, and Linux) to build Mer itself as well as apps for Mer. It can also be used to develop a full user interface for a device. The Mer SDK consists of two parts: a Platform SDK and plugins for Qt Creator to communicate with the Platform SDK. Jolla has supported a lot of the work on the SDK, he said.
The Platform SDK runs in a virtual machine using VirtualBox. Instead of creating a version for each distribution and operating system, it is a self-hosting platform. It could also be done using LXC or chroot() and namespaces, but it is using VirtualBox for now. It has all of the low level packages for building and testing that are needed to build Mer packages for a specific target.
On top of the low-level pieces, there is a Qt Creator–based interface that includes Mer-specific plugins to talk to the Platform SDK. Those plugins control emulators and devices, which is not normally a part of Qt development. There are also plugins to handle packaging metadata and to interface with app stores. The Mer project will be working with the Qt project to get those pieces upstream, he said.
The Platform SDK plus the Qt Creator plugins make an overall Mer SDK, which is a starting point for a vendor SDK. For example, Jolla has added Sailfish OS targets to turn it into a Sailfish SDK. It is designed to be extensible and to be used for different targets, such as MIPS, ARM, or x86.
Qt Creator runs natively on the developer's system, but interfaces with the Platform SDK via the plugins. Building the code is done the same way, whether it is initiated from Qt Creator or OBS. VirtualBox shared folders are used between the workstation and the Platform SDK, so a build from Qt Creator does an ssh into the virtual machine and executes make there. Apps can be deployed that way as well, so that developers don't have to build and install a package each time they want to test on an emulator or a real device.
Plans
So there is a build engine, but what is needed now are more emulators and targets, Greaves said. Nemo, Plasma Active, and Raspberry Pi are all good candidates. Better emulation support is also on the roadmap, including simulating hardware events in the kernel. A move from VirtualBox to LXC is also in the cards, as is better debugging support.
As with most projects, Mer needs help, so Greaves issued a "call to action". He noted that Mer is a vendor community, but that using the term "cooperative" may be more successful when talking to management. There are a number of benefits to working with Mer, but the biggest is that those working on the project can influence its development direction. There are multiple choices for mobile development, including Windows Mobile, iOS, Android, and Tizen, but none of those will give a device maker the freedom that it wants, he said. Even though Android is open source, how many have actually gotten patches accepted into it? For that reason alone, Mer should be very appealing to smaller device vendors.
[Thanks to KDE e.V. for travel assistance to Bilbao for Akademy.]
Brief items
Distribution quotes of the week
Android 4.3
Google has announced the existence of Android 4.3. It includes a host of new features, some of which may be more welcome than others. "To meet the needs of the next generation of media services, Android 4.3 introduces a modular DRM framework that enables media application developers to more easily integrate DRM into their own streaming protocols, such as MPEG DASH (Dynamic Adaptive Streaming over HTTP, ISO/IEC 23009-1)." There is no word on general availability or when this code will be available through the Android Open Source Project.
CyanogenMod to incorporate SELinux
The CyanogenMod project has announced that it will be incorporating the work of the SEAndroid project to add SELinux to its firmware images. "We will be working on this policy creation in parallel to Google’s own policies for Android, which we believe will be released with the Android 4.3 source; effectively getting us ahead of the eventual 4.3 source release." For now, it will be running in permissive mode, with the hope that users will submit logs to allow tweaking of the policies.
Distribution News
Fedora
Fedora ARM provisionally set as a primary architecture
The July 18 LWN Weekly Edition includes coverage of the discussion on whether the Fedora project should recognize ARM as one of its primary architectures. While that article was being written, the Fedora engineering steering committee was discussing the issue; the conclusion they came to was: "build ARM on primary infrastructure. whether it is released as a primary Fedora 20 or as 'Fedora 20 for ARM' depends on how well it fulfills release criteria and functionality criteria closer to release time" So ARM has not reached "primary architecture" status yet, but things are clearly headed in that direction.
Ubuntu family
Ubuntu Forums account information breached
The Ubuntu Forums site is currently displaying a "down for maintenance" message that describes a security breach of the accounts database, which revealed all usernames, passwords, and email addresses to the attacker(s). Ubuntu One, Launchpad, and other Canonical web sites are not affected by the breach. "The passwords are not stored in plain text. However, if you were using the same password as your Ubuntu Forums one on another service (such as email), you are strongly encouraged to change the password on the other service ASAP."
Newsletters and articles of interest
Distribution newsletters
- Debian Project News (July 22)
- DistroWatch Weekly, Issue 517 (July 22)
- Maemo Weekly News (July 22)
- Ubuntu Weekly Newsletter, Issue 326 (July 21)
Mageia 4 already! (Mageia Blog)
The Mageia Blog covers the schedule for Mageia 4. "After some weeks of (interesting) discussions, we went though the features process to define the official list for Mageia 4. The choice was based on the existing policy. As usual, we received a lot of proposals."
Miller: Proposal for a more agile "Fedora.next"
Fedora cloud architect Matthew Miller has posted some notes describing his vision for a more successful and widely used Fedora distribution. "This is a vision for the future. The Fedora Core idea can get started now, and the ideas beyond that are for development over a longer term. That doesn't mean never, and there are some specific things at the higher level to get started on now, but I'm not suggesting to change everything all crazily. We're not throwing out what have."
Slackware Turns 20 (OStatic)
Susan Linton looks at the history of Slackware. "As I look back over my history with Slack, I'm struck by how many distributions were once based on Slackware. Most are no longer maintained, but some names may still be familiar. GoblinX was a strange looking but quite stable and fun distribution. It's biggest issue in adoption is their pay-to-play business model that often fails in Linuxville. Austrumi is a tiny distro from Latvia, a tiny Northern European country most Americans' education didn't include. It was fast and stable and looks to be abandoned. Ultima 4 was trying to provide an easy to use Slackware and Mutagenix was a really cool distro that has disappeared off the face of the Earth. But Slackware is still here. There are many more derivative epitaphs, but the oldest surviving Linux distribution is 20 years old and is still very actively and enthusiastically maintained."
Page editor: Rebecca Sobol
Development
Kolab 3.1 adds WebDAV
Test builds for version 3.1 of the Kolab groupware suite have been released. As before, the system provides a solid mail, address book, and calendaring solution with both web-based and desktop clients available. But this release departs from the project's traditional reliance on IMAP and adds support for a number of new services built on top of WebDAV. The change has the potential to simplify deploying Kolab for many users and offices.
Kolab is a client-server groupware system. The data is stored and managed on the server and there are web front-ends available (such as the Roundcube webmail client), but the emphasis has long been on supporting outside desktop clients as well. Historically, Kolab used special IMAP folders to store contacts and calendar entries on the server, in addition to the email messages for which IMAP was designed. The project defined its own XML-based storage format that encapsulates other data types into MIME messages on the server side. It covers iCalendar events, to-do tasks, journal entries, and free/busy statuses, plus text notes, vCard contacts, and shared "resources" (e.g., conference rooms).
The Roundcube client that ships with Kolab includes plugins to utilize the non-email IMAP storage directly, and KDE's Kontact has long been the primary desktop application supported by Kolab. Support for the Kolab storage format also made its way into other groupware suites like Citadel. Many other client programs, however, required an add-on—such as SyncKolab for Mozilla Thunderbird or several proprietary connectors for Microsoft's Outlook. There are also several mobile clients for which Kolab connectors are available, although not all of them support the most recent version of the storage format, 3.0.
As one might guess, version 3.0 of the storage format was rolled out in conjunction with version 3.0 of Kolab itself, which was released in January 2013. Thus, the addition of an entirely new data access protocol for the 3.1 is all the more remarkable.
The project announced
the plan in March, calling it both " Kolab 3.1 is also adding support for general-purpose file storage,
which will also be accessible over WebDAV. But the file storage
functionality is not limited to WebDAV: the project has also
implemented an IMAP-based file store, and it has worked with the ownCloud project to make an ownCloud
server accessible as storage, too. The ownCloud-Kolab integration is
not yet ready, but the Kolab project describes it as
" The 3.1 alphas also include the ability for multiple users to share
mail folders. That is, not only can multiple user accounts
read mail from a shared inbox (such as
contact@example.com), but those accounts can also send
messages as contact@example.com. Other changes incorporated
include new administration features, like the ability to define a
policy for how user account UIDs are generated, and support for
managing multiple email domains within a single Kolab server.
The alpha release showcasing all of these new features is available
from the project's Git repository
or in packages for CentOS. Debian packages have been assembled in the
past, but so far none are available for the 3.1 code.
Historically the term "groupware" tended to apply to information
systems designed to support office environments, but it recent years
that has changed. The public scandals about privacy breaches,
surveillance, and user tracking that seem to plague every major web
application service provider have served to raise the profile of open
source groupware suites—increasingly, they are seen as a
self-hosted alternative to relying on Google, Microsoft, and Apple for
valuable web services, even if deployed only for a single user. The
Kolab team is clearly aware of this shift, and cites
the system's ability to serve as a private service beyond the reach of
the NSA
PRISM program.
Kolab has been in active development for just over ten years, and
in that time it has earned a reputation for quality production in more
ways than one. There are quite a few open source groupware products
on the market, but many of them suffer from a lack of client
compatibility, either working only with their bundled web front-end,
or focusing on a small set of third-party client applications. Kolab
has long offered support for a wide assortment of client endpoints,
and implementing CalDAV and CardDAV access is likely to extend the
reach of that assortment in a big way.
Support for these protocols is also a positive sign for CardDAV and
CalDAV themselves, which in the past have been criticized as being
overly complex (which, for example, gave rise to the now-abandoned
GroupDAV protocol). Their slow adoption can be seen as part of a
march toward compatibility among competing groupware projects,
however. When all is said and done, it will be interesting to see whether
this WebDAV support signals the start of a slow decline in support for
Kolab's IMAP-based data protocols. But perhaps that would not be a
tragic loss in the long run, as it might allow the project to expend
less energy maintaining private protocols. The project does not like
reinventing the wheel, after all, and as of Kolab 3.1 alpha, it is
now responsible for maintaining two separate access protocols.
the biggest and most
exciting thing on our roadmap
" as well as "the one people
have asked us for most.
" Using the SabreDAV WebDAV library
(in order to not "reinvent the wheel
", according to the
announcement), the
project would add CalDAV and CardDAV access to scheduling and contact
data. As the 3.1 alpha announcement explains, the old IMAP-based
access protocols are not going away. The CalDAV and CardDAV
functionality is implemented as a separate data access layer dubbed "iRony" by the project. The most obvious gain from the move is
immediate access to Kolab server data from the sizeable assortment of
existing CalDAV- and CardDAV-aware applications. The built-in
Mac OS X calendar and address book fall into this category,
but so do quite a few open source programs, including web-based,
mobile, and desktop clients.
trivial
", as is integration with Dropbox or other cloud
storage systems.
Brief items
Quotes of the week
Wine 1.6 released
Version 1.6 of the Wine Windows emulation system is out. "This release represents 16 months of development effort and around 10,000 individual changes. The main highlights are the new Mac driver, the full support for window transparency, and the new Mono package for .NET applications support."
Razor and LXDE-Qt merging
Jerome Leclanche has posted an announcement that the Razor and LXDE-Qt (LXDE ported from GTK to Qt) lightweight desktops will be merging under the LXDE-Qt banner. "So what happens now? Our two teams have met up and discussed the issues and we have decided that the best course of action for both projects is to focus on a single desktop environment, instead of two. There have been talks of "merging" ever since LXDE-Qt was announced. Having taken the decision to collaborate, we've all had the pleasure of working together already. Our plan is to cherry-pick the best parts of Razor and LXDE and include or port those to LXDE-Qt. Other components will be ported straight from GTK code or rewritten from scratch. In the end, we want to offer the best possible experience while reusing as much code as possible. It will not be an easy process and as always, we welcome anyone who wishes to help, be it with development, translations, or general feedback." In addition, Leclanche mentioned an offer to bring Razor (and now, perhaps, LXDE-Qt) under the KDE umbrella.
Apache OpenOffice 4.0 released
The Apache OpenOffice 4.0 release is now available. "OpenOffice 4.0 features a new, more modern user interface, improvements to Microsoft Office interoperability, enhanced graphics support and many other improvements." See the release notes for lots of details.
U-Boot 2013.07 available
Version 2013.07 of the U-Boot bootloader has been released. This version is notable for adding support for doing cryptographically verified boot. The announcement adds: "There's still work
to be done to make it easier to tie things to a TPM (it's just
scriptable today), but hey, there it is. One might even say this is one
of those technical features that makes companies chose something other
than U-Boot..
"
Newsletters and articles
Development newsletters from the past week
- Caml Weekly News (July 23)
- What's cooking in git.git (July 18)
- What's cooking in git.git (July 21)
- What's cooking in git.git (July 22)
- Haskell Weekly News (July 17)
- OpenStack Community Weekly Newsletter (July 19)
- Perl Weekly (July 22)
- PostgreSQL Weekly News (July 22)
- R Journal (June)
- Ruby Weekly (July 17)
- Tor Weekly News (July 24)
Airlie: Introducing Virgil - 3D virtual GPU for qemu
Dave Airlie has posted an
introduction to Virgil, a new
3D virtual GPU project for the QEMU emulator, for which he is seeking developer
input. The goal, he says, is "a 3D capable virtual GPU for
qemu that can be used by Linux and eventually Windows guests to
provide OpenGL/Direct3D support inside the guest. It uses an interface
based on Gallium/TGSI along with virtio to communicate between guest
and host, and it goal is to provided an OpenGL renderer along with a
complete Linux driver stack for the guest.
" Airlie also goes out of
his way to be clear that he is recruiting developers, not users,
noting "I can't stress this strongly enough, this isn't end user
ready, not even close
".
Xiph.org: Introducing Daala, part 2
Xiph.org has published part 2 of its introduction to the new Daala video codec. This edition covers frequency domain intra-prediction. "Daala predicts frequency coefficients in the frequency domain, rather than pixels in the spatial domain. Frequency coefficients are not shared between blocks, allowing us to keep the classic definition of 'neighboring' as well as avoiding any circular decoding dependencies. In addition, Daala's frequency-domain prediction draws from a larger input data set consisting of up to four complete blocks. In theory, the additional degrees of freedom could give us a more efficient predictor.
"
Page editor: Nathan Willis
Announcements
Brief items
The H is shutting down
The H has announced its closure. "Although The H has produced many widely read stories, it has not been possible to effectively monetise that traffic to produce a working business model." Here at LWN we understand that problem well. The H has indeed produced a lot of quality content over the years; we are genuinely sad to see it go.
HeliOS Project announces fundraising drive
Reglue, the non-profit organization behind the HeliOS Project that provides Linux PCs to Texas schoolchildren, has announced a fundraising drive. Although HeliOS has always been donation-driven, the project is hoping to make up for the funding lost over the past year as founder Ken Starks was unable to work while undergoing treatment for cancer.OSI Welcomes Member-Elected Director
The Open Source Initiative (OSI) announced the election of Richard Fontana to OSI's first individual-member–elected seat on the board of directors. OSI is in the process of transitioning from an appointed board to one composed of elected members.
Articles of interest
Bacon: Announcing the Ubuntu Edge
Jono Bacon announces that Canonical will be building and shipping a "fully converged" device called the "Ubuntu Edge" — all funded by a $32 million Indiegogo campaign. "The Ubuntu Edge will dual boot Ubuntu and Android, and will transform into a PC when docked with a monitor, with the full Ubuntu desktop and shared access to all the phone’s files. For this it needs the power of a PC, so Ubuntu Edge will be equipped with the latest, fastest processor, at least 4GB of RAM and a massive 128GB of storage."
New Books
The Practice of Network Security Monitoring--New from No Starch Press
No Starch Press has released "The Practice of Network Security Monitoring" by Richard Bejtlich.
Calls for Presentations
PGConf.EU 2013 Call for papers extended
PGConf.EU will take place October 29-November 1 in Dublin, Ireland. The call for papers deadline has been extended until July 29.Linux Symposium Call for Papers
The Linux Symposium will take place November 18-21, 2013 in Ottawa, Canada. The call for papers deadline is September 1.CFP Deadlines: July 25, 2013 to September 23, 2013
The following listing of CFP deadlines is taken from the LWN.net CFP Calendar.
| Deadline | Event Dates | Event | Location |
|---|---|---|---|
| July 25 | October 22 October 23 |
GStreamer Conference | Edinburgh, UK |
| July 28 | October 17 October 20 |
PyCon PL | Szczyrk, Poland |
| July 29 | October 28 October 31 |
15th Real Time Linux Workshop | Lugano, Switzerland |
| July 29 | October 29 November 1 |
PostgreSQL Conference Europe 2013 | Dublin, Ireland |
| July 31 | November 5 November 8 |
OpenStack Summit | Hong Kong, Hong Kong |
| July 31 | October 24 October 25 |
Automotive Linux Summit Fall 2013 | Edinburgh, UK |
| August 7 | September 12 September 14 |
SmartDevCon | Katowice, Poland |
| August 15 | August 22 August 25 |
GNU Hackers Meeting 2013 | Paris, France |
| August 18 | October 19 | Hong Kong Open Source Conference 2013 | Hong Kong, China |
| August 19 | September 20 September 22 |
PyCon UK 2013 | Coventry, UK |
| August 21 | October 23 | TracingSummit2013 | Edinburgh, UK |
| August 22 | September 25 September 27 |
LibreOffice Conference 2013 | Milan, Italy |
| August 30 | October 24 October 25 |
Xen Project Developer Summit | Edinburgh, UK |
| August 31 | October 26 October 27 |
T-DOSE Conference 2013 | Eindhoven, Netherlands |
| August 31 | September 24 September 25 |
Kernel Recipes 2013 | Paris, France |
| September 1 | November 18 November 21 |
2013 Linux Symposium | Ottawa, Canada |
| September 6 | October 4 October 5 |
Open Source Developers Conference France | Paris, France |
| September 15 | November 8 | PGConf.DE 2013 | Oberhausen, Germany |
| September 15 | December 27 December 30 |
30th Chaos Communication Congress | Hamburg, Germany |
| September 15 | November 15 November 16 |
Linux Informationstage Oldenburg | Oldenburg, Germany |
| September 15 | October 3 October 4 |
PyConZA 2013 | Cape Town, South Africa |
| September 15 | November 22 November 24 |
Python Conference Spain 2013 | Madrid, Spain |
| September 15 | February 1 February 2 |
FOSDEM 2014 | Brussels, Belgium |
| September 15 | April 11 April 13 |
PyCon 2014 | Montreal, Canada |
If the CFP deadline for your event does not appear here, please tell us about it.
Upcoming Events
Events: July 25, 2013 to September 23, 2013
The following event listing is taken from the LWN.net Calendar.
| Date(s) | Event | Location |
|---|---|---|
| July 22 July 26 |
OSCON 2013 | Portland, OR, USA |
| July 27 July 28 |
PyOhio 2013 | Columbus, OH, USA |
| July 27 | OpenShift Origin Community Day | Mountain View, CA, USA |
| July 31 August 4 |
OHM2013: Observe Hack Make | Geestmerambacht, the Netherlands |
| August 1 August 8 |
GUADEC 2013 | Brno, Czech Republic |
| August 3 August 4 |
COSCUP 2013 | Taipei, Taiwan |
| August 6 August 8 |
Military Open Source Summit | Charleston, SC, USA |
| August 7 August 11 |
Wikimania | Hong Kong, China |
| August 9 August 11 |
XDA:DevCon 2013 | Miami, FL, USA |
| August 9 August 13 |
PyCon Canada | Toronto, Canada |
| August 9 August 12 |
Flock - Fedora Contributor Conference | Charleston, SC, USA |
| August 11 August 18 |
DebConf13 | Vaumarcus, Switzerland |
| August 12 August 14 |
YAPC::Europe 2013 “Future Perl” | Kiev, Ukraine |
| August 16 August 18 |
PyTexas 2013 | College Station, TX, USA |
| August 22 August 25 |
GNU Hackers Meeting 2013 | Paris, France |
| August 23 August 24 |
Barcamp GR | Grand Rapids, MI, USA |
| August 24 August 25 |
Free and Open Source Software Conference | St.Augustin, Germany |
| August 30 September 1 |
Pycon India 2013 | Bangalore, India |
| September 3 September 5 |
GanetiCon | Athens, Greece |
| September 6 September 8 |
Kiwi PyCon 2013 | Auckland, New Zealand |
| September 6 September 8 |
State Of The Map 2013 | Birmingham, UK |
| September 10 September 11 |
Malaysia Open Source Conference 2013 | Kuala Lumpur, Malaysia |
| September 12 September 14 |
SmartDevCon | Katowice, Poland |
| September 13 | CentOS Dojo and Community Day | London, UK |
| September 16 September 18 |
LinuxCon North America | New Orleans, LA, USA |
| September 16 September 18 |
CloudOpen | New Orleans, LA, USA |
| September 18 September 20 |
Linux Plumbers Conference | New Orleans, LA, USA |
| September 19 September 20 |
Linux Security Summit | New Orleans, LA, USA |
| September 19 September 20 |
Open Source Software for Business | Prato, Italy |
| September 19 September 20 |
UEFI Plugfest | New Orleans, LA, USA |
| September 20 September 22 |
PyCon UK 2013 | Coventry, UK |
If your event does not appear here, please tell us about it.
Page editor: Rebecca Sobol