LWN.net Weekly Edition for December 20, 2007
Insufficiently free?
Watching an extended flame war between Richard Stallman and Theo de Raadt is an interesting experience. The realization that one can sit back and watch without having to really care about the result brings a sense of profound tranquility and relief. Along the way, one gets to learn things like how mean Theo can be, or that Richard does not use a web browser. It all seems like good fun. Even so, when the discussion reaches levels like this:
it becomes impossible not to wonder if one hasn't wandered into an elementary school yard by mistake. Most observers would probably conclude that Mr. Stallman has chosen to express himself with less childish terms than Mr. de Raadt. Still, this conversation came about as a result of a statement made by Mr. Stallman, one which upset the OpenBSD community greatly. It is worthwhile to look at where the disagreement was.
In particular, Richard Stallman started the discussion by saying that he cannot "recommend" OpenBSD because the "ports" system they use facilitates the installation of certain non-free packages. His reasoning comes down to this:
There are all kinds of things which can be said about the OpenBSD community, but statements that they lack a proper appreciation for freedom are not among them. This community's view of what makes a system truly free differs from that of the Free Software Foundation, but what they produce is undeniably free software. It is, arguably, one of the most free systems available, with careful attention paid to the licensing of even things like firmware blobs which are not part of the system itself. So folks in the OpenBSD community resent this sort of claim, even if they profess to care little about the opinions of the person making it.
Of course, it's not only OpenBSD which fails to pass Mr. Stallman's test. The list of recommended distributions from the GNU web site has grown recently; it now contains gNewSense, Ututo, Dynebolic, Musix, BLAG, and GNUstep. True statistics are hard to come by, of course, but your editor would be most surprised if the combined installed base of these distributions added up to a full 1% of the Linux systems in use. Most of us, in other words, are using systems which Mr. Stallman is unable to recommend.
Many of us will be using distributions like Fedora or Debian which are strongly committed to the creation of free systems. The developers behind these distributions have gone to considerable trouble to be sure that everything which is part of their system is truly free software, even when, as has happened at times, the result has been trouble for users. These distributors have clearly advanced the cause of free software greatly through their efforts over many years. One might well wonder just why Mr. Stallman cannot bring himself to recommend the result of this work.
The OpenBSD developers, though, have been asking a different question: why is the GNU project happy to enable its software to be installed on non-free systems? That is where the charges of hypocrisy come from. Mr. Stallman answered both questions together. It seems that, in his view, there is little risk of leading users astray by letting them install programs like Emacs on proprietary systems:
Thus, the risk of leading people to use a non-free system by making a free program run on it is small.
It would appear, however, that proprietary applications carry a much higher degree of risk:
It is not all that hard to see, embodied within a statement like this, a somewhat condescending view of computer users, who have to be "led" to install the right software. It is a position which disallows the recommendation of completely-free operating systems which most of us use. It places a sort of ideological purity above the vast amounts of work which have gone into the creation of a variety of free systems available for all to run.
It is, in other words, an unreasonable position - as can be seen by the fact that almost no free software users actually follow Mr. Stallman's advice when they choose their systems. Before condemning this unreasonable position, though, it's worth a quick review of the famous George Bernard Shaw quote:
There is no doubt that we have benefited from Mr. Stallman's lengthy,
sometimes unreasonable campaign. Certainly he
has no doubt on that score, saying "Free operating systems exist
today because of the campaign which I started in 1983.
" But it's
worthwhile to remember that free operating systems also exist because
thousands of others have put in hard work for many years. It seems
appropriate to wonder whether telling those people that their work
still is not free enough really helps the cause of free software.
On the other hand, one need not wonder about the value of responding to a "refusal to recommend" with an extensive attack which ventures into pure character assassination. Vitriolic flaming helps nobody's cause. One may not agree with Mr. Stallman's position in this discussion, but one thing should be said: he kept his cool, remained respectful and stayed on-topic when others lost it completely. That is the way to promote free software.
Ruby on Rails releases 2.0

Rails (aka Ruby on Rails or RoR) is a framework for building web applications. It has gotten a lot of attention – some would say hype – over the past few years as easy to use and learn, while allowing the creation of complex database-backed web services. In the year since Rails 1.2, the team has not been idle, with their work culminating in the release of Rails 2.0 this month.
RoR is based around the idea of using the model-view-controller (MVC) pattern to cleanly separate the user interface from the application logic and data storage. All of the Ruby code written or generated for a Rails application is organized into a directory hierarchy based on what part of the MVC they implement. All of the parts of the application know how to find the others because of this convention, which is in keeping with the two principles that guided the development of RoR.
Fundamentally, RoR is built around two principles. The first is "convention over configuration", which is the idea that only things that deviate from standard practices need to be specified via configuration. One can get surprisingly far by sticking with these standard practices. The other principle is "don't repeat yourself", which means that there is a single place to go to specify something about the application; other places that need it or things derived from it, retrieve it from the canonical place. This is most evident in the specification of database table and column names; they are described in the model and other parts of the application retrieve them as required.
The principles are interrelated, of course, and are two of the innovations that RoR has popularized for web application frameworks. Many previous attempts required a huge amount of configuration information to be specified, often nearly identically in multiple places. Simplifying this configuration headache was explicitly a goal for Rails. It can take a bit of time to come to grips with the conventions used, but once that is done it is straightforward to use the framework.
Generating code to handle simple modifications to the database data, known as scaffolding, is another technique popularized by RoR. From the specification of the data model, Rails will generate an interface to create, read, update, and delete data in that model. It can also generate "migrations" which contain the SQL necessary to create or modify the database tables to reflect changes in the model. Migrations can be used in both a forward and backward direction to keep the database in sync with the state of the application as changes are made.
Rails itself is broken up into multiple components implementing each piece of the MVC architecture: ActiveRecord for the model, ActionPack for the view and controller, along with a number of lesser players. It provides extensive test harness facilities that allow testing of the web application without using a browser or network at all. RoR is a comprehensive solution, with a large number of very vocal supporters.
The new release provides a number of new features, some performance enhancements, as well as the requisite bug fixes. The bulk of the changes in 2.0 are in the controllers. The first is better support for "representational state transfer" (REST) style web application APIs, which were introduced in Rails 1.2. Better support for multiple different views based on application criteria were also added, allowing the interface to change based on the device accessing it, for example.
Security enhancements were made as well, with code being added to help protect against cross-site scripting and cross-site request forgery attacks. These two web application flaws are becoming rather popular to exploit, so any assistance a web framework can give is welcome. The default session objects have changed to be cookie-based, rather than stored in a file or the database. This allows snooping of the session data, but the data is hashed to prevent forgery.
Performance and scalability have been the traditional knocks against Rails, and though there were some enhancements, especially to ActiveRecord, that should provide some boost, it is not clear how well Rails handles huge sites. It is something the Rails team is aware of, so, over time, those kinds of problems should be solved. RoR is a very capable framework and the 2.0 release looks very good. The Rails community should find much of use.
Looking back at 2007
Consistent with our usual practice, LWN will not be publishing a Weekly Edition during the last full week of the year. This is thus the last such for 2007; the next weekly will be published on January 3, 2008. Also consistent with usual practice, you editor will look back on the year which is about to end, with an emphasis on evaluating how his predictions made at the beginning of the year came out. There is amusement to be had in exposing the flaws in one's crystal ball, but there is also value in seeing how one's view of the world has changed over the course of the year.Your editor bravely predicted that GPLv3 would be finalized and adopted by the FSF; sure enough, that happened right on schedule. Your editor also admitted to having "no clue" of how the FSF would respond to the criticism of the anti-DRM provisions of GPLv3. Certainly it would have been hard to predict the addition of the "user product" language and associated exemptions. So far, the impact of GPLv3 has been relatively small, but use of this license will surely grow over time.
Another prediction said that somebody would be sued for the distribution of proprietary kernel modules. That did not happen - at least, not in a way that the public (or your editor) heard about it. What your editor did not foresee was the burst of energy coming from the Software Freedom Law Center on behalf of the BusyBox developers. Thus far, GPL enforcement activities continue to focus on the relatively clear-cut cases. They also continue to have a very high success rate. Still, going after a company like Verizon is an ambitious move; it will be interesting to see how that one settles out.
The end of SCO was predicted. Your editor thought it might happen in March, when new dispositive motions would once again be entertained by Judge Kimball. Instead, the clear end of SCO happened in August when the court ruled that Novell still owned the Unix source and that SCO owed Novell a chunk of money. Like a fish thrown on the shore, SCO will continue to flop around for a while, but there can be little doubt about its ultimate fate.
The prediction that there would be serious talk of patent reform did not really come through. There were a couple of U.S. court decisions in 2007 which, arguably, raised the bar slightly for patent trolls. In general, though, the software patent situation remains unchanged - and as dangerous as ever.
There were a couple of predictions about closed hardware, together saying, essentially, that the situation would get better but that the problem would not go away. Things clearly got better when AMD decided to open up information about ATI's video hardware and assist with the creation of free drivers for that hardware. The progress toward a viable Atheros wireless chipset driver for Linux is also a happy development. The situation has improved, and will continue to do so.
[PULL QUOTE: Your editor predicted a serious war on bloat as people got tired of running out of memory. Wishful thinking, it seems, is alive and well. END QUOTE] Your editor predicted a serious war on bloat as people got tired of running out of memory. Wishful thinking, it seems, is alive and well. In practice, people just bought more memory; even the OLPC project decided it had to increase the amount of memory in its XO system. Your editor will not be repeating this prediction for 2008.
"Fedora will come into its own as a free, community-oriented distribution" has, beyond any doubt, come true. The Fedora 7 release brought community developers in from the margins, and Fedora 8 solidified the new process. The bulk of the packages in Fedora are now maintained by community developers. Red Hat's controlling hand, while still clearly present, is weaker than before. Fedora leader Max Spevack has presided over a crucial transformation of this important project; he will be moving on to other challenges early in 2008, but will be leaving behind a distribution in far better shape than the one he inherited a few years ago.
Predicting Debian releases is a dangerous business, but, in this case, Debian Etch was close enough to make it a relatively safe proposition. Your editor had also suggested (facetiously) that the Debian developers would subsequently go back to arguing about firmware in the kernel; that quite clearly did not happen.
The prediction that free software would play a larger role in online gaming was, for the most part, wishful thinking again. The release of the Second Life client code was a step in the right direction, but not much happened after that. Your editor still hopes that free software will be at the core of the games of the future, or he may never see his children again.
The Microsoft/Novell deal, predicted your editor, would blow over with relatively few consequences. In many ways that was true. One could argue that the whole "235 patents" routine would have come out anyway - we heard similar claims before Novell signed this deal. Your editor failed to guess that a whole stream of companies (Samsung, Xandros, LG Electronics, Linspire, Turbolinux) would follow Novell into similar agreements, though.
Your editor suggested that the "open source" term would suffer as a result of companies trying to retain higher levels of control over "open source" code. Certainly the OSI's approval of the CPAL "badgeware" license will not have helped in this regard. On the other hand, SugarCRM decided to just go with the GPLv3 in favor of its attribution-required license. As a whole, "open source" means almost what it meant one year ago.
Contrary to prediction, there have not been OLPC systems distributed to millions of children - though thousands should start getting them soon. We are still waiting to see what impact the OLPC project will really have - on free software, and on the world as a whole. Stay tuned.
Finally, the growth of desktop Linux was predicted, though your editor refrained from saying that 2007 would be the year of the Linux desktop. Clearly, progress has been made in that direction - we now have major vendors like Dell selling desktop systems, Wal-Mart's desktop offering sold out in days, and the number of pocket-sized "desktops" running Linux continues to grow.
Perhaps the biggest thing which your editor missed entirely was the fight over Microsoft's proposed OOXML standard. This issue came to light in January of this year, though it had been simmering for a little while before - the ECMA TC45 committee was already considering this proposal in the middle of 2006. The fight over the fast-tracking of OOXML and the ensuing questions on just how the community should work with the standards practice will continue to echo into 2008.
Overall, your editor feels like the predictions went reasonably well. Too well, perhaps; next year's predictions may need to be a little more adventurous. Those predictions will be posted in the January 3 edition. In the mean time, your editor wishes for a great holiday season and new year for everybody in the community; we have accomplished much over the last year and have many things to celebrate.
Security
The backdooring of SquirrelMail
SquirrelMail advertises itself as "webmail for nuts." It is a PHP-based package which is in wide use; most distributions include a SquirrelMail package. Security problems in SquirrelMail are certainly not unheard-of; even so, the announcement that the source distribution for version 1.4.12 had been compromised raised some eyebrows. Initially the project downplayed the problem:
It only took one day, though, before Uwe Schindler pointed out that, in fact, the changes made to the source opened a remote-execution back door into deployed SquirrelMail systems. Somewhere along the way, the project discovered that the 1.4.11 release had also been tampered with. The SquirrelMail developers released version 1.4.13 to close the vulnerabilities.
There have not been any public reports of systems being compromised by way of this vulnerability. Additionally, it would appear that all of the distributors which shipped the affected versions got their version of the code prior to the attack. So the episode would appear to have ended reasonably well - as far as we know. There are some lessons that one can take from this attack, though.
The downplaying of the problem initially was a potentially fatal mistake. If somebody has been tampering with the sources, there is no excuse not to go into red-alert mode immediately, even if the developers involved do not understand the attack. When a project has been compromised at such a fundamental level, one must assume the worst.
The compromise was discovered after a user noticed that the tarballs on the download site did not match the posted MD5 checksums. Your editor suspects that very few of us actually verify checksums in the packages they take from the net. Doing so more often would be a good exercise in software hygiene for all of us.
That said, the project got lucky this time around. A smarter attacker would have replaced the checksums after adding the back door, making the changes harder to detect. Longer-term, the increasing doubts about the security of MD5 suggest that relying on it to detect changes to tarballs might not be entirely safe. Far better to use public-key signatures; they should have a longer shelf life, and, if the keys are managed properly, they are impossible to replace. It seems that the project has posted GPG signatures for 1.4.13, though the Wayback Machine suggests that this is a recent practice. Your editor was unable to find the public key needed to verify the signatures.
The modifications to the tarballs were done using a compromised developer's account. The specific changes made were not put into the SquirrelMail source repository. The project has said nothing, though, about what has been done to ensure that no other changes were made there. Some sort of statement from the project along these lines would be most reassuring to SquirrelMail's users.
Perhaps the most encouraging conclusion, though, is this: there have been several attempts to compromise source distributions over the years. Many of them have succeeded in getting bad code into high-profile packages. But none of these attacks - so far as we know - have escaped detection for any significant period of time, and none of them have led to any sort of wide-scale exploit. As a whole, we would appear to be reasonably resistant to this kind of attack, even when the front-line defenses fail. With luck, and continued vigilance, that trend will continue. Both will be required, though: there is no doubt that the attackers will keep trying.
Brief items
'You've Got Cross-Site Scripting' (Dark Reading)
Dark Reading reports on a new email alert service for cross-site scripting bugs. "So XSSed.com, a site dedicated to archiving publicly disclosed XSS bugs, is now offering a free email alert service that notifies you as soon as an XSS vulnerability affecting your Website gets indexed to its archive. XSSed claims to have the industry's largest XSS archive, with over 17,000 disclosed vulnerabilities as of this posting."
New vulnerabilities
clamav: integer overflow and off-by-one
Package(s): | clamav | CVE #(s): | CVE-2007-6335 CVE-2007-6336 | ||||||||||||||||||||||||||||||||||||
Created: | December 19, 2007 | Updated: | July 17, 2008 | ||||||||||||||||||||||||||||||||||||
Description: | ClamAV contains integer overflow and off-by-one errors which could be exploited (via specially-crafted email) to execute arbitrary code. | ||||||||||||||||||||||||||||||||||||||
Alerts: |
|
cups: multiple vulnerabilities
Package(s): | cups | CVE #(s): | CVE-2007-5849 CVE-2007-6358 CVE-2007-4352 CVE-2007-5392 CVE-2007-5393 | ||||||||||||||||||||||||||||||||||||||||
Created: | December 19, 2007 | Updated: | October 16, 2008 | ||||||||||||||||||||||||||||||||||||||||
Description: | The cups 1.3.5 release fixes a number of vulnerabilities in the PDF filters. Additionally, there is a buffer overflow in the SNMP code and a temporary file vulnerability. | ||||||||||||||||||||||||||||||||||||||||||
Alerts: |
|
flash-plugin: lots of problems
Package(s): | flash-plugin | CVE #(s): | CVE-2007-5275 CVE-2007-4324 CVE-2007-4768 CVE-2007-6242 CVE-2007-6243 CVE-2007-6244 CVE-2007-6245 CVE-2007-6246 | ||||||||||||||||||||||||||||||||||||
Created: | December 19, 2007 | Updated: | November 14, 2008 | ||||||||||||||||||||||||||||||||||||
Description: | A vast number of vulnerabilities exists in the proprietary Flash plugin for Firefox. | ||||||||||||||||||||||||||||||||||||||
Alerts: |
|
IRC Services: denial of service
Package(s): | ircservices | CVE #(s): | CVE-2007-6122 | ||||
Created: | December 14, 2007 | Updated: | December 19, 2007 | ||||
Description: | loverboy reported that the "default_encrypt()" function in file encrypt.c does not properly handle overly long passwords. A remote attacker could provide an overly long password to the vulnerable server, resulting in a denial of service. | ||||||
Alerts: |
|
kdebase: denial of service
Package(s): | kdebase | CVE #(s): | CVE-2007-5963 | ||||||||
Created: | December 18, 2007 | Updated: | January 19, 2009 | ||||||||
Description: | The kdebase package is vulnerable to a denial of service in which a local user can render KDM unusable for logins by any user or cause KDM to exceed system resource limits. | ||||||||||
Alerts: |
|
kernel: denial of service
Package(s): | kernel | CVE #(s): | CVE-2007-5966 | ||||||||||||||||||||||||||||||||||||
Created: | December 19, 2007 | Updated: | February 3, 2010 | ||||||||||||||||||||||||||||||||||||
Description: | A bug in high-resolution timers (prior to kernel 2.6.22.15) can cause very long sleeps when large timeout values are used. | ||||||||||||||||||||||||||||||||||||||
Alerts: |
|
libexif: integer overflow
Package(s): | libexif | CVE #(s): | CVE-2007-6352 | ||||||||||||||||||||||||||||||||||||||||
Created: | December 19, 2007 | Updated: | October 15, 2008 | ||||||||||||||||||||||||||||||||||||||||
Description: | From the Red Hat advisory: An integer overflow flaw was found in the way libexif parses Exif image tags. If a victim opens a carefully crafted Exif image file, it could cause the application linked against libexif to execute arbitrary code, or crash. | ||||||||||||||||||||||||||||||||||||||||||
Alerts: |
|
libexif: denial of service
Package(s): | libexif | CVE #(s): | CVE-2007-6351 | ||||||||||||||||||||||||||||||||||||
Created: | December 19, 2007 | Updated: | October 15, 2008 | ||||||||||||||||||||||||||||||||||||
Description: | From the Red Hat advisory: An infinite recursion flaw was found in the way libexif parses Exif image tags. If a victim opens a carefully crafted Exif image file, it could cause the application linked against libexif to crash. | ||||||||||||||||||||||||||||||||||||||
Alerts: |
|
libgd2: buffer overflow
Package(s): | libgd2 | CVE #(s): | CVE-2007-3996 | ||||||||||||||||||||||||||||
Created: | December 19, 2007 | Updated: | October 13, 2009 | ||||||||||||||||||||||||||||
Description: | The GD library does not perform proper bounds checking when creating images; as a result, an attacker could, via crafted input, potentially execute arbitrary code. | ||||||||||||||||||||||||||||||
Alerts: |
|
mysql: privilege escalation
Package(s): | mysql | CVE #(s): | CVE-2007-6303 | ||||||||||||||||||||||||||||||||
Created: | December 19, 2007 | Updated: | April 7, 2008 | ||||||||||||||||||||||||||||||||
Description: | From the CVE entry: MySQL 5.0.x before 5.0.52, 5.1.x before 5.1.23, and 6.0.x before 6.0.4 does not update the DEFINER value of a view when the view is altered, which allows remote authenticated users to gain privileges via a sequence of statements including a CREATE SQL SECURITY DEFINER VIEW statement and an ALTER VIEW statement. | ||||||||||||||||||||||||||||||||||
Alerts: |
|
portage: information disclosure
Package(s): | portage | CVE #(s): | CVE-2007-6249 | ||||
Created: | December 14, 2007 | Updated: | December 19, 2007 | ||||
Description: | Mike Frysinger reported that the "etc-update" utility uses temporary files with the standard umask, which results in the files being world-readable when merging configuration files in a default setup. A local attacker could access sensitive information when configuration files are being merged. | ||||||
Alerts: |
|
squid: denial of service
Package(s): | squid | CVE #(s): | CVE-2007-6239 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Created: | December 18, 2007 | Updated: | March 25, 2009 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Description: | A flaw was found in the way squid stored HTTP headers for cached objects in system memory. An attacker could cause squid to use additional memory, and trigger high CPU usage when processing requests for certain cached objects, possibly leading to a denial of service. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Alerts: |
|
wpa_supplicant: stack-based buffer overflow
Package(s): | wpa_supplicant | CVE #(s): | CVE-2007-6025 | ||||
Created: | December 14, 2007 | Updated: | December 19, 2007 | ||||
Description: | A stack-based buffer overflow in driver_wext.c in wpa_supplicant 0.6.0 allows remote attackers to cause a denial of service (crash) via crafted TSF data. | ||||||
Alerts: |
|
Xfce: buffer overflows
Package(s): | xfce4 | CVE #(s): | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Created: | December 19, 2007 | Updated: | December 19, 2007 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Description: | The Xfce desktop contains a number of buffer overflow vulnerabilities; they have been fixed in the 4.4.2 release. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Alerts: |
|
Page editor: Jake Edge
Kernel development
Brief items
Kernel release status
The current 2.6 patch remains 2.6.24-rc5; no new -rc releases have been made over the last week. Fixes do continue to find their way into the mainline git repository, though.The current -mm tree is 2.6.24-rc5-mm1. Recent changes to -mm include some significant device model changes; a number of subsystem trees have been dropped from this release due to patch conflicts.
The current stable 2.6 kernel is
2.6.23.10 2.6.23.11 2.6.23.12. The big patch is 2.6.23.10, released on
December 14, with several dozen fixes. The 2.6.23.11 (December 14) and
2.6.23.12 (December 18)
releases contain small fixes for problems caused by 2.6.23.10.
For older kernels: 2.6.22.15 was released on December 14 with quite a few fixes.
2.4.36-rc1 was released on December 17 with a number of security-related fixes. The 2.4.35.5 release also contains those fixes.
Kernel development news
Quotes of the week
- 1,400-odd open bugzilla reports
- 719 emails saved away in my emailed-bug-reports folder, all of which need to be gone through, asking originators to retest and re-report-if-unfixed.
- A big ugly email titled "2.6.24-rc5-git1: Reported regressions from 2.6.23" in my inbox.
All of which makes it a bit inappropriate to be thinking about intrusive-looking new features.
Ho hum. Just send me the whole lot against rc5-mm1 and I'll stick it in there and we'll see what breaks.
Short subjects: kerneloops, read-mostly, and port 80
Kerneloops. Triage is an important part of a kernel developer's job. A project as large and as widely-used as the kernel will always generate more bug reports than can be realistically addressed in the amount of time which is available. So developers must figure out which reports are most deserving of their attention. Sometimes the existence of an irate, paying customer makes this decision easy. Other times, though, it is a matter of making a guess at which bugs are affecting the largest numbers of users. And that often comes down to how many different reports have come in for a given problem.Of course, counting reports is not the easiest thing to do, especially if they are not all sent to the same place. In an attempt to make this process easier, Arjan van de Ven has announced a new site at kerneloops.org. Arjan has put together some software which scans certain sites and mailing lists for posted kernel oops output; whenever a crash is found, it is stuffed into a database. Then an attempt is made to associate reports with each other based on kernel version and the call trace; from that, a list of the most popular ways to crash can be created. As of this writing, the current fashion for kernel oopses would appear to be in ieee80211_tx() in current development kernels. Some other information is stored with the trace; in particular, it is possible to see what the oldest kernel version associated with the problem is.
This is clearly a useful resource, but there are a couple of problems which make it harder to do the job properly. One is that there is no distinctive marker which indicates the end of an oops listing, so the scripts have a hard time knowing where to stop grabbing information. The other is that multiple reports of the same oops can artificially raise the count for a particular crash. The solution to both problems is to place a marker at the end of the oops output which includes a random UUID generated at system boot time. Patches to this effect are circulating, though getting the random number into the output turns out to be a little harder than one might have expected. So, for 2.6.24, the "random" number may be all zeroes, with the real problem to be solved in 2.6.25.
Read-mostly. Anybody who digs through kernel source for any period of time will notice a number of variables declared in a form like this:
static int __read_mostly ignore_loglevel;
The __read_mostly attribute says that accesses to this variable are usually (but not always) read operations. There were some questions recently about why this annotation is done; the answer is that it's an important optimization, though it may not always be having the effect that developers are hoping for.
As is well described in What every programmer should know about memory, proper use of processor memory caches is crucial for optimal performance. The idea behind __read_mostly is to group together variables which are rarely changed so they can all share cache lines which need not be bounced between processors on multiprocessor systems. As long as nobody changes a __read_mostly variable, it can reside in a shared cache line with other such variables and be present in cache (if needed) on all processors in the system.
The read-mostly attribute generally works well and yields a measurable performance improvement. There are concerns, though, that this feature could be over-used. Andrew Morton expressed it this way:
Combining frequently-written variables into shared cache lines is a good way to maximize the bouncing of those cache lines between processors - which would be bad for performance. So over-aggressive segregation of read-mostly variables to minimize cache line bouncing could have the opposite of the desired effect: it could make the kernel's cache behavior worse.
The better way, says Andrew, would have been to create a "read often" attribute for variables which are frequently used in a read-only mode. That would leave behind the numerous read-rarely variables to serve as padding keeping the write-often variables nicely separated from each other. Thus far, patches to make this change have not been forthcoming.
I/O port delays. The functions provided by the kernel for access to I/O ports have long included versions which insert delays. A driver would normally read a byte from a port with inb(), but inb_p() could be used if an (unspecified) short delay was needed after the operation. A look through the driver tree shows that quite a few drivers use the delayed versions of the I/O port accessors, even though, in many cases, there is no real need for that delay.
This delay is implemented (on x86 architectures) with a write to I/O port 80. There is generally no hardware listening for an I/O operation on that port, so this write has the sole effect of delaying the processor while the bus goes through an abortive attempt to execute the operation. It is an operation with reasonably well-defined semantics, and it has worked for Linux for many years.
Except that now, it seems, this technique no longer works on a small subset of x86_64 systems. Instead, the write to port 80 will, on occasion, freeze the system hard; this, in turn, generates a rather longer delay than was intended. One could imagine the creation of an elaborate mechanism for restarting I/O operations after the user resets the system, but the kernel developers, instead, chose to look for alternative ways of implementing I/O delays.
In almost every case, the alternative form of the delay is a call to udelay(). The biggest problem here is that udelay() works by sitting in a tight loop; it cannot know how many times to go through the loop until the speed of the processor has been calibrated. That calibration happens reasonably early in the boot process, but there are still tasks to be performed - including I/O port operations - first. This problem is being worked around by removing some delayed operations from the early setup code, but some developers worry that it will never be possible to get them all. It has been suggested that the kernel could just assume it's running on the fastest-available processor until the calibration happens, but, beyond being somewhat inelegant, that could significantly slow the bootstrap process on slower machines - all of which work just fine with the current code.
The real solution is to simply get rid of almost all of the delayed I/O port operations. Very few of them are likely to be needed with any hardware which still works. In some cases, what may really be going on is that the delays are being used to paper over driver bugs - such as failing to force a needed PCI write out by doing a read operation. Just removing the delays outright would probably cause instability in unpredictable places - not a result most developers are striving for. So the task of cleaning up those calls will have to be done carefully over time. Meanwhile, the use of port 80 will probably remain unchanged for 2.6.24.
revoke() returns
LWN last looked at Pekka Enberg's revoke() patch in July, 2006. The purpose of this proposed system call is to completely disconnect all processes from a specific file, thus allowing a new process to have exclusive access to that file. There are a number of applications for this functionality, such as ensuring that a newly logged-in user is the only one able to access resources associated with the console - the sound device, for example. There are kernel developers who occasionally mutter ominously about unfixable security problems resulting from the lack of the ability to revoke open file descriptors - though they tend, for some reason, to not want to publish the details of those vulnerabilities. Any sort of real malware scanning application will also need to be able to revoke access to files determined to contain Bad Stuff.Pekka has recently posted a new version of the patch, so a new look seems warranted. The first thing one notes is that the revoke() system call is gone; instead, the new form of the system call is:
int revokeat(int dir_fd, const char *filename);
This call thus follows the form of a number of other, relatively new *at() system calls. Here, filename is the name of the file for which access is to be revoked; if it is an absolute pathname then dir_fd is ignored. Otherwise, dir_fd is an open file descriptor for the directory to be used as the starting point in the lookup of filename. The special value AT_FDCWD indicates the current working directory for the calling process. If the revokeat() call completes successfully, only file descriptors for filename which are created after the call will be valid.
There is a new file_operations member created by this patch set:
int (*revoke)(struct file *filp);
This function's job is to ensure that any outstanding I/O operations on the given file have completed, with a failure status if needed. So far, the only implementation is a generic version for filesystems; it is, in its entirety:
int generic_file_revoke(struct file *file) { return do_fsync(file, 1); }
In the long term, revokeat() will need support from at least a subset of device drivers to be truly useful.
Disconnecting access to regular file descriptors is relatively straightforward; the system call simply iterates through the list of open files on the relevant device and replaces the file_operations structure with a new set which returns EBADF for every attempted operation. (OK, for almost every attempted operation - reads from sockets and device files return zero instead). The only tricky part is that it must iterate through the file list multiple times until no open files are found; otherwise there could be race conditions with other system calls creating new file descriptors at the same time that the old ones are being revoked.
The trickier part is dealing with memory mappings. In most cases, it is a matter of finding all virtual memory areas (VMAs) associated with the file, setting the new VM_REVOKED flag, and calling zap_page_range() to clear out the associated page table entries. The VM_REVOKED flag ensures that any attempt to fault pages back in will result in a SIGBUS signal - likely to be an unpleasant surprise for any process attempting to access that area.
Even trickier is the case of private, copy-on-write (COW) mappings, which can be created when a process forks. Simply clearing those mappings might be effective, but it could result in the death of processes which do not actually need to be killed. But it is important that the COW mapping not be a way to leak data written to the file after the revokeat() call. So the COW mappings are separated from each other by a simple (but expensive) call to get_user_pages(), which will create private copies of all of the relevant pages.
There has been relatively little discussion of this patch so far - perhaps the relevant developers have begun their holiday breaks and revoked their access to linux-kernel. This is an important patch with a lot of difficult, low-level operations, though; that is part of why it has been so long in the making. So it will need some comprehensive review before it can be considered ready for the mainline. Given the nature of the problem, it would not be surprising if another iteration or two were needed still.
What is RCU, Fundamentally?
[Editor's note: this is the first in a three-part series on how the read-copy-update mechanism works. Many thanks to Paul McKenney and Jonathan Walpole for allowing us to publish these articles. The remaining two sections will appear in future weeks.]
Part 1 of 3 of What is RCU, Really?
Paul E. McKenney, IBM Linux Technology Center
Jonathan Walpole, Portland State University Department of Computer
Science
Introduction
Read-copy update (RCU) is a synchronization mechanism that was added to the Linux kernel in October of 2002. RCU achieves scalability improvements by allowing reads to occur concurrently with updates. In contrast with conventional locking primitives that ensure mutual exclusion among concurrent threads regardless of whether they be readers or updaters, or with reader-writer locks that allow concurrent reads but not in the presence of updates, RCU supports concurrency between a single updater and multiple readers. RCU ensures that reads are coherent by maintaining multiple versions of objects and ensuring that they are not freed up until all pre-existing read-side critical sections complete. RCU defines and uses efficient and scalable mechanisms for publishing and reading new versions of an object, and also for deferring the collection of old versions. These mechanisms distribute the work among read and update paths in such a way as to make read paths extremely fast. In some cases (non-preemptable kernels), RCU's read-side primitives have zero overhead.
Quick Quiz 1: But doesn't seqlock also permit readers and updaters to get work done concurrently?
This leads to the question "what exactly is RCU?", and perhaps also to the question "how can RCU possibly work?" (or, not infrequently, the assertion that RCU cannot possibly work). This document addresses these questions from a fundamental viewpoint; later installments look at them from usage and from API viewpoints. This last installment also includes a list of references.
RCU is made up of three fundamental mechanisms, the first being used for insertion, the second being used for deletion, and the third being used to allow readers to tolerate concurrent insertions and deletions. These mechanisms are described in the following sections, which focus on applying RCU to linked lists:
- Publish-Subscribe Mechanism (for insertion)
- Wait For Pre-Existing RCU Readers to Complete (for deletion)
- Maintain Multiple Versions of Recently Updated Objects (for readers)
These sections are followed by concluding remarks and the answers to the Quick Quizzes.
Publish-Subscribe Mechanism
One key attribute of RCU is the ability to safely scan data, even
though that data is being modified concurrently.
To provide this ability for concurrent insertion,
RCU uses what can be thought of as a publish-subscribe mechanism.
For example, consider an initially NULL
global pointer
gp
that is to be modified to point to a newly allocated
and initialized data structure.
The following code fragment (with the addition of appropriate locking)
might be used for this purpose:
1 struct foo { 2 int a; 3 int b; 4 int c; 5 }; 6 struct foo *gp = NULL; 7 8 /* . . . */ 9 10 p = kmalloc(sizeof(*p), GFP_KERNEL); 11 p->a = 1; 12 p->b = 2; 13 p->c = 3; 14 gp = p;
Unfortunately, there is nothing forcing the compiler and CPU to execute
the last four assignment statements in order.
If the assignment to gp
happens before the initialization
of p
's fields, then concurrent readers could see the
uninitialized values.
Memory barriers are required to keep things ordered, but memory barriers
are notoriously difficult to use.
We therefore encapsulate them into a primitive
rcu_assign_pointer()
that has publication semantics.
The last four lines would then be as follows:
1 p->a = 1; 2 p->b = 2; 3 p->c = 3; 4 rcu_assign_pointer(gp, p);
The rcu_assign_pointer()
would publish the new structure, forcing both the compiler
and the CPU to execute the assignment to gp
after
the assignments to the fields referenced by p
.
However, it is not sufficient to only enforce ordering at the updater, as the reader must enforce proper ordering as well. Consider for example the following code fragment:
1 p = gp; 2 if (p != NULL) { 3 do_something_with(p->a, p->b, p->c); 4 }
Although this code fragment might well seem immune to misordering,
unfortunately, the
DEC
Alpha CPU [PDF]
and value-speculation compiler optimizations can, believe it or not,
cause the values of p->a
, p->b
, and
p->c
to be fetched before the value of p
!
This is perhaps easiest to see in the case of value-speculation
compiler optimizations, where the compiler guesses the value
of p
, fetches p->a
, p->b
, and
p->c
, then fetches the actual value of p
in order to check whether its guess was correct.
This sort of optimization is quite aggressive, perhaps insanely so,
but does actually occur in the context of profile-driven optimization.
Clearly, we need to prevent this sort of skullduggery on the
part of both the compiler and the CPU.
The rcu_dereference()
primitive uses
whatever memory-barrier instructions and compiler
directives are required for this purpose:
1 rcu_read_lock(); 2 p = rcu_dereference(gp); 3 if (p != NULL) { 4 do_something_with(p->a, p->b, p->c); 5 } 6 rcu_read_unlock();
The rcu_dereference()
primitive can thus be thought of
as subscribing to a given value of the specified pointer,
guaranteeing that subsequent dereference operations will see any
initialization that occurred before the corresponding publish
(rcu_assign_pointer()
) operation.
The rcu_read_lock()
and rcu_read_unlock()
calls are absolutely required: they define the extent of the
RCU read-side critical section.
Their purpose is explained in the
next section,
however, they never spin or block, nor do they prevent the
list_add_rcu()
from executing concurrently.
In fact, in non-CONFIG_PREEMPT
kernels, they generate
absolutely no code.
Although rcu_assign_pointer()
and
rcu_dereference()
can in theory be used to construct any
conceivable RCU-protected data structure, in practice it is often better
to use higher-level constructs.
Therefore, the rcu_assign_pointer()
and
rcu_dereference()
primitives have been embedded in special RCU variants of Linux's
list-manipulation API.
Linux has two variants of doubly linked list, the circular
struct list_head
and the linear
struct hlist_head
/struct hlist_node
pair.
The former is laid out as follows, where the green boxes represent
the list header and the blue boxes represent the elements in the
list.
Adapting the pointer-publish example for the linked list gives the following:
1 struct foo { 2 struct list_head list; 3 int a; 4 int b; 5 int c; 6 }; 7 LIST_HEAD(head); 8 9 /* . . . */ 10 11 p = kmalloc(sizeof(*p), GFP_KERNEL); 12 p->a = 1; 13 p->b = 2; 14 p->c = 3; 15 list_add_rcu(&p->list, &head);
Line 15 must be protected by some synchronization mechanism (most
commonly some sort of lock) to prevent multiple list_add()
instances from executing concurrently.
However, such synchronization does not prevent this list_add()
from executing concurrently with RCU readers.
Subscribing to an RCU-protected list is straightforward:
1 rcu_read_lock(); 2 list_for_each_entry_rcu(p, head, list) { 3 do_something_with(p->a, p->b, p->c); 4 } 5 rcu_read_unlock();
The list_add_rcu()
primitive publishes
an entry into the specified list, guaranteeing that the corresponding
list_for_each_entry_rcu()
invocation will properly
subscribe to this same entry.
Quick Quiz 2:
What prevents the list_for_each_entry_rcu()
from
getting a segfault if it happens to execute at exactly the same
time as the list_add_rcu()
?
Linux's other doubly linked list, the hlist, is a linear list, which means that it needs only one pointer for the header rather than the two required for the circular list. Thus, use of hlist can halve the memory consumption for the hash-bucket arrays of large hash tables.
Publishing a new element to an RCU-protected hlist is quite similar to doing so for the circular list:
1 struct foo { 2 struct hlist_node *list; 3 int a; 4 int b; 5 int c; 6 }; 7 HLIST_HEAD(head); 8 9 /* . . . */ 10 11 p = kmalloc(sizeof(*p), GFP_KERNEL); 12 p->a = 1; 13 p->b = 2; 14 p->c = 3; 15 hlist_add_head_rcu(&p->list, &head);
As before, line 15 must be protected by some sort of synchronization mechanism, for example, a lock.
Subscribing to an RCU-protected hlist is also similar to the circular list:
1 rcu_read_lock(); 2 hlist_for_each_entry_rcu(p, q, head, list) { 3 do_something_with(p->a, p->b, p->c); 4 } 5 rcu_read_unlock();
Quick Quiz 3:
Why do we need to pass two pointers into
hlist_for_each_entry_rcu()
when only one is needed for list_for_each_entry_rcu()
?
The set of RCU publish and subscribe primitives are shown in the following table, along with additional primitives to "unpublish", or retract:
Category | Publish | Retract | Subscribe |
---|---|---|---|
Pointers | rcu_assign_pointer() |
rcu_assign_pointer(..., NULL) |
rcu_dereference() |
Lists | list_add_rcu() list_add_tail_rcu() list_replace_rcu() |
list_del_rcu() |
list_for_each_entry_rcu() |
Hlists | hlist_add_after_rcu() hlist_add_before_rcu() hlist_add_head_rcu() hlist_replace_rcu() |
hlist_del_rcu() |
hlist_for_each_entry_rcu() |
Note that the list_replace_rcu()
, list_del_rcu()
,
hlist_replace_rcu()
, and hlist_del_rcu()
APIs add a complication.
When is it safe to free up the data element that was replaced or
removed?
In particular, how can we possibly know when all the readers
have released their references to that data element?
These questions are addressed in the following section.
Wait For Pre-Existing RCU Readers to Complete
In its most basic form, RCU is a way of waiting for things to finish. Of course, there are a great many other ways of waiting for things to finish, including reference counts, reader-writer locks, events, and so on. The great advantage of RCU is that it can wait for each of (say) 20,000 different things without having to explicitly track each and every one of them, and without having to worry about the performance degradation, scalability limitations, complex deadlock scenarios, and memory-leak hazards that are inherent in schemes using explicit tracking.
In RCU's case, the things waited on are called
"RCU read-side critical sections".
An RCU read-side critical section starts with an
rcu_read_lock()
primitive, and ends with a corresponding
rcu_read_unlock()
primitive.
RCU read-side critical sections can be nested, and may contain pretty
much any code, as long as that code does not explicitly block or sleep
(although a special form of RCU called
"SRCU"
does permit general sleeping in SRCU read-side critical sections).
If you abide by these conventions, you can use RCU to wait for any
desired piece of code to complete.
RCU accomplishes this feat by indirectly determining when these other things have finished, as has been described elsewhere for RCU Classic and realtime RCU.
In particular, as shown in the following figure, RCU is a way of waiting for pre-existing RCU read-side critical sections to completely finish, including memory operations executed by those critical sections.
However, note that RCU read-side critical sections that begin after the beginning of a given grace period can and will extend beyond the end of that grace period.
The following pseudocode shows the basic form of algorithms that use RCU to wait for readers:
- Make a change, for example, replace an element in a linked list.
- Wait for all pre-existing RCU read-side critical sections to
completely finish (for example, by using the
synchronize_rcu()
primitive). The key observation here is that subsequent RCU read-side critical sections have no way to gain a reference to the newly removed element. - Clean up, for example, free the element that was replaced above.
The following code fragment, adapted from those in the
previous section,
demonstrates this process, with field a
being the search key:
1 struct foo { 2 struct list_head list; 3 int a; 4 int b; 5 int c; 6 }; 7 LIST_HEAD(head); 8 9 /* . . . */ 10 11 p = search(head, key); 12 if (p == NULL) { 13 /* Take appropriate action, unlock, and return. */ 14 } 15 q = kmalloc(sizeof(*p), GFP_KERNEL); 16 *q = *p; 17 q->b = 2; 18 q->c = 3; 19 list_replace_rcu(&p->list, &q->list); 20 synchronize_rcu(); 21 kfree(p);
Lines 19, 20, and 21 implement the three steps called out above. Lines 16-19 gives RCU ("read-copy update") its name: while permitting concurrent reads, line 16 copies and lines 17-19 do an update.
The synchronize_rcu()
primitive might seem a bit
mysterious at first.
After all, it must wait for all RCU read-side critical sections to
complete, and, as we saw earlier, the
rcu_read_lock()
and rcu_read_unlock()
primitives
that delimit RCU read-side critical sections don't even generate any
code in non-CONFIG_PREEMPT
kernels!
There is a trick, and the trick is that RCU Classic read-side critical
sections delimited by rcu_read_lock()
and
rcu_read_unlock()
are not permitted to block or sleep.
Therefore, when a given CPU executes a context switch, we are guaranteed
that any prior RCU read-side critical sections will have completed.
This means that as soon as each
CPU has executed at least one context switch, all
prior RCU read-side critical sections are guaranteed to have completed,
meaning that synchronize_rcu()
can safely return.
Thus, RCU Classic's synchronize_rcu()
can conceptually be as simple as the following:
1 for_each_online_cpu(cpu) 2 run_on(cpu);
Here, run_on()
switches the current thread to the
specified CPU, which forces a context switch on that CPU.
The for_each_online_cpu()
loop therefore forces a
context switch on each CPU, thereby guaranteeing that all prior
RCU read-side critical sections have completed, as required.
Although this simple approach works for kernels in which preemption
is disabled across RCU read-side critical sections, in other
words, for non-CONFIG_PREEMPT
and CONFIG_PREEMPT
kernels, it does not work for CONFIG_PREEMPT_RT
realtime (-rt) kernels.
Therefore, realtime RCU uses
a different approach based loosely on reference counters.
Of course, the actual implementation in the Linux kernel is much more complex, as it is required to handle interrupts, NMIs, CPU hotplug, and other hazards of production-capable kernels, but while also maintaining good performance and scalability. Realtime implementations of RCU must additionally help provide good realtime response, which rules out implementations (like the simple two-liner above) that rely on disabling preemption.
Although it is good to know that there is a simple conceptual
implementation of synchronize_rcu()
, other questions remain.
For example, what exactly do RCU
readers see when traversing a concurrently updated list?
This question is addressed in the following section.
Maintain Multiple Versions of Recently Updated Objects
This section demonstrates how RCU maintains multiple versions of lists to accommodate synchronization-free readers. Two examples are presented showing how an element that might be referenced by a given reader must remain intact while that reader remains in its RCU read-side critical section. The first example demonstrates deletion of a list element, and the second example demonstrates replacement of an element.
Example 1: Maintaining Multiple Versions During Deletion
To start the "deletion" example, we will modify lines 11-21 in the example in the previous section as follows:
1 p = search(head, key); 2 if (p != NULL) { 3 list_del_rcu(&p->list); 4 synchronize_rcu(); 5 kfree(p); 6 }
The initial state of the list, including the pointer p
,
is as follows.
The triples in each element represent the values of fields a
,
b
, and c
, respectively.
The red borders on
each element indicate that readers might be holding references to them,
and because readers do not synchronize directly with updaters,
readers might run concurrently with this entire replacement process.
Please note that
we have omitted the backwards pointers and the link from the tail
of the list to the head for clarity.
After the list_del_rcu()
on
line 3 has completed, the 5,6,7
element
has been removed from the list, as shown below.
Since readers do not synchronize directly with updaters,
readers might be concurrently scanning this list.
These concurrent readers might or might not see the newly removed element,
depending on timing.
However, readers that were delayed (e.g., due to interrupts, ECC memory
errors, or, in CONFIG_PREEMPT_RT
kernels, preemption)
just after fetching a pointer to the newly removed element might
see the old version of the list for quite some time after the
removal.
Therefore, we now have two versions of the list, one with element
5,6,7
and one without.
The border of the 5,6,7
element is
still red, indicating
that readers might be referencing it.
Please note that readers are not permitted to maintain references to
element 5,6,7
after exiting from their RCU read-side
critical sections.
Therefore,
once the synchronize_rcu()
on
line 4 completes, so that all pre-existing readers are
guaranteed to have completed,
there can be no more readers referencing this
element, as indicated by its black border below.
We are thus back to a single version of the list.
At this point, the 5,6,7
element may safely be
freed, as shown below:
At this point, we have completed the deletion of
element 5,6,7
.
The following section covers replacement.
Example 2: Maintaining Multiple Versions During Replacement
To start the replacement example, here are the last few lines of the example in the previous section:
1 q = kmalloc(sizeof(*p), GFP_KERNEL); 2 *q = *p; 3 q->b = 2; 4 q->c = 3; 5 list_replace_rcu(&p->list, &q->list); 6 synchronize_rcu(); 7 kfree(p);
The initial state of the list, including the pointer p
,
is the same as for the deletion example:
As before,
the triples in each element represent the values of fields a
,
b
, and c
, respectively.
The red borders on
each element indicate that readers might be holding references to them,
and because readers do not synchronize directly with updaters,
readers might run concurrently with this entire replacement process.
Please note that
we again omit the backwards pointers and the link from the tail
of the list to the head for clarity.
Line 1 kmalloc()
s a replacement element, as follows:
Line 2 copies the old element to the new one:
Line 3 updates q->b
to the value "2":
Line 4 updates q->c
to the value "3":
Now, line 5 does the replacement, so that the new element is
finally visible to readers.
At this point, as shown below, we have two versions of the list.
Pre-existing readers might see the 5,6,7
element, but
new readers will instead see the 5,2,3
element.
But any given reader is guaranteed to see some well-defined list.
After the synchronize_rcu()
on line 6 returns,
a grace period will have elapsed, and so all reads that started before the
list_replace_rcu()
will have completed.
In particular, any readers that might have been holding references
to the 5,6,7
element are guaranteed to have exited
their RCU read-side critical sections, and are thus prohibited from
continuing to hold a reference.
Therefore, there can no longer be any readers holding references
to the old element, as indicated by the thin black border around
the 5,6,7
element below.
As far as the readers are concerned, we are back to having a single version
of the list, but with the new element in place of the old.
After the kfree()
on line 7 completes, the list will
appear as follows:
Despite the fact that RCU was named after the replacement case, the vast majority of RCU usage within the Linux kernel relies on the simple deletion case shown in the previous section.
Discussion
These examples assumed that a mutex was held across the entire update operation, which would mean that there could be at most two versions of the list active at a given time.
Quick Quiz 4: How would you modify the deletion example to permit more than two versions of the list to be active?
Quick Quiz 5: How many RCU versions of a given list can be active at any given time?
This sequence of events shows how RCU updates use multiple versions to safely carry out changes in presence of concurrent readers. Of course, some algorithms cannot gracefully handle multiple versions. There are techniques [PDF] for adapting such algorithms to RCU, but these are beyond the scope of this article.
Conclusion
This article has described the three fundamental components of RCU-based algorithms:
- a publish-subscribe mechanism for adding new data,
- a way of waiting for pre-existing RCU readers to finish, and
- a discipline of maintaining multiple versions to permit change without harming or unduly delaying concurrent RCU readers.
Quick Quiz 6:
How can RCU updaters possibly delay RCU readers, given that the
rcu_read_lock()
and rcu_read_unlock()
primitives neither spin nor block?
These three RCU components allow data to be updated in face of concurrent readers, and can be combined in different ways to implement a surprising variety of different types of RCU-based algorithms, some of which will be the topic of the next installment in this "What is RCU, Really?" series.
Acknowledgements
We are all indebted to Andy Whitcroft, Gautham Shenoy, and Mike Fulton, whose review of an early draft of this document greatly improved it. We owe thanks to the members of the Relativistic Programming project and to members of PNW TEC for many valuable discussions. We are grateful to Dan Frye for his support of this effort. Finally, this material is based upon work supported by the National Science Foundation under Grant No. CNS-0719851.
This work represents the view of the authors and does not necessarily represent the view of IBM or of Portland State University.
Linux is a registered trademark of Linus Torvalds.
Other company, product, and service names may be trademarks or service marks of others.
Answers to Quick Quizzes
Quick Quiz 1: But doesn't seqlock also permit readers and updaters to get work done concurrently?
Answer:
Yes and no.
Although seqlock readers can run concurrently with
seqlock writers, whenever this happens, the read_seqretry()
primitive will force the reader to retry.
This means that any work done by a seqlock reader running concurrently
with a seqlock updater will be discarded and redone.
So seqlock readers can run concurrently with updaters,
but they cannot actually get any work done in this case.
In contrast, RCU readers can perform useful work even in presence of concurrent RCU updaters.
Quick Quiz 2:
What prevents the list_for_each_entry_rcu()
from
getting a segfault if it happens to execute at exactly the same
time as the list_add_rcu()
?
Answer: On all systems running Linux, loads from and stores
to pointers are atomic, that is, if a store to a pointer occurs at
the same time as a load from that same pointer, the load will return
either the initial value or the value stored, never some bitwise mashup
of the two.
In addition, the list_for_each_entry_rcu()
always proceeds
forward through the list, never looking back.
Therefore, the list_for_each_entry_rcu()
will either see
the element being added by list_add_rcu()
, or it will not,
but either way, it will see a valid well-formed list.
Quick Quiz 3:
Why do we need to pass two pointers into
hlist_for_each_entry_rcu()
when only one is needed for list_for_each_entry_rcu()
?
Answer: Because in an hlist it is necessary to check for
NULL rather than for encountering the head.
(Try coding up a single-pointer hlist_for_each_entry_rcu()
.
If you come up with a nice solution, it would be a very good thing!)
Quick Quiz 4: How would you modify the deletion example to permit more than two versions of the list to be active?
Answer: One way of accomplishing this is as follows:
spin_lock(&mylock); p = search(head, key); if (p == NULL) spin_unlock(&mylock); else { list_del_rcu(&p->list); spin_unlock(&mylock); synchronize_rcu(); kfree(p); }
Note that this means that multiple concurrent deletions might be
waiting in synchronize_rcu()
.
Quick Quiz 5: How many RCU versions of a given list can be active at any given time?
Answer: That depends on the synchronization design. If a semaphore protecting the update is held across the grace period, then there can be at most two versions, the old and the new.
However, if only the search, the update, and the
list_replace_rcu()
were protected by a lock, then
there could be an arbitrary number of versions active, limited only
by memory and by how many updates could be completed within a
grace period.
But please note that data structures that are updated so frequently
probably are not good candidates for RCU.
That said, RCU can handle high update rates when necessary.
Quick Quiz 6:
How can RCU updaters possibly delay RCU readers, given that the
rcu_read_lock()
and rcu_read_unlock()
primitives neither spin nor block?
Answer: The modifications undertaken by a given RCU updater will cause the corresponding CPU to invalidate cache lines containing the data, forcing the CPUs running concurrent RCU readers to incur expensive cache misses. (Can you design an algorithm that changes a data structure without inflicting expensive cache misses on concurrent readers? On subsequent readers?)
Patches and updates
Kernel trees
Architecture-specific
Core kernel code
Device drivers
Documentation
Filesystems and block I/O
Memory management
Networking
Security-related
Virtualization and containers
Miscellaneous
Page editor: Jonathan Corbet
Distributions
News and Editorials
Distributions 2007 review
This is the last LWN weekly for 2007, so it must be time to reflect on what's happened during the past year. Also this is a slow time of year, so there hasn't been much new news.Debian GNU/Linux: Debian Etch (4.0) was released in April, as was the sixth revision of Sarge (3.1r6). The first Etch revision (4.0r1) was released in August. Debian development is focused on Lenny, now in the testing branch. Overall a good year, but it's unfortunate that the Dunc-Tank experiment of late 2006 - early 2007 seems to have caused the demise of the Debian Weekly News.
Fedora: Fedora made great
strides in becoming true community distribution with the merger of Core and
Extras. 2007 saw the release of both Fedora 7 and Fedora 8, both excellent
desktops/workstations. Max Spevack led the project through the merger and
announced his resignation at the end of the
year. This week's DistroWatch had the comment
that "despite all these positives, the distribution still fails to
attract first-time Linux users who sometimes complain about the lack of a
central configuration utility or the overly technical nature of the
operating system.
" This led to a discussion
on the Fedora Marketing list. There seems to be some agreement that Fedora
does expect its users to be somewhat clueful, and that's the way we like
it.
Gentoo Linux made one release this year. The year is not over so it's still possible for 2007.1 to make it in 2007. Gentoo saw quite a bit of developer churn this year, which may have led to a delayed release. Then again, releases aren't always that important. Gentoo works great for developers.
Mandriva Linux released in the spring and in the fall, or if you are down under it's the fall and the spring. The company is in recovery following the financial problems and lay-offs of previous years. Mandriva is friendly to new users, with a helpful community on mailing lists and forums to help you through any rough spots.
openSUSE released 10.3 this year. There's also an early alpha for 11.0 available. Like Fedora, openSUSE is a community project with an Enterprise sponsor. This has been a good year for the project. There has been quite a bit of new infrastructure like the Build Service, new mailing lists, style guidelines, and a new manager.
Slackware Linux: Slackware 12.0 was released in July. The Slackware current changelog remains active. There's not much else to say, Slackware continues. Slackware may not the most newbie friendly, but its very good at what it does. It's hard to imagine the Linux landscape without Slackware.
Ubuntu remains strong. Deals with Dell haven't hurt. Ubuntu, and its derivatives Edubuntu, Kubuntu and Xubuntu continue to gain users. Releases for this year include Feisty Fawn (7.04) and Gutsy Gibbon (7.10), as well as the first alpha for the Hardy Heron (8.04). To see Ubuntu's popularity, just look at all the other distributions that are using it for a base. (MEPIS, Geubuntu, gOS, Linux Mint, Symphony OS, Fluxbuntu, gNewSense, Arabian Linux, Kiwi, Impi, Guadalinex, MoLinux, nUbuntu, ProTech, Linux for Clinics, Mythbuntu, Pyramid, UbuntuCE, UbuntuME, Ubuntu Studio, ubuntutrinux, BeaFanatIX, PUD, and andLinux). These can be found by searching for Ubuntu in the Distribution List.
New Releases
NetBSD 4.0 released
NetBSD 4.0 is out. "Major achievements in NetBSD 4.0 include support for version 3 of the Xen virtual machine monitor, Bluetooth, many new device drivers and embedded platforms based on ARM, PowerPC and MIPS CPUs. New network services include iSCSI target (server) code and an implementation of the Common Address Redundancy Protocol. Also, system security was further enhanced with restrictions of mprotect(2) to enforce W^X policies, the Kernel Authorization framework, and improvements of the Veriexec file integrity subsystem, which can be used to harden the system against trojan horses and virus attacks."
Mandriva 2008.1 Alpha 1 released
Mandriva has released an alpha version of Mandriva Linux 2008 Spring (2008.1). Some of the major new features in this pre-release include PulseAudio, X.org 7.3, KDE 4.0 RC2, kernel 2.6.24 rc5, and UUID-based drive mounting. "You are encouraged to test and comment on this pre-release. Feedback should be posted in the form of bug reports to Bugzilla, or if it is not a type of feedback that can be expressed as a bug report, to the Cooker mailing list or to the Mandriva Forums."
Debian-Edu releases a lenny test CD for testing
Debian-Edu/Skolelinux has released the first test release based on Debian lenny. Click below to see some of the known problems with this release. For those interested in the package installation failure because usplash needs debian-edu-artwork-usplash issue, there is a possible workaround.
Distribution News
Fedora
The search for a new Fedora leader
Max Spevack, who has led the Fedora project through a period of great change and improvement, has announced that the time has come to move on to other (Fedora-related) challenges. So the project is looking for a new leader. "The Fedora Project Leader is a full-time Red Hat position, and so we need to go through a full interview process, etc. None of this is being done ad-hoc or randomly. The Fedora Board is part of the process, as is Red Hat's CTO and other managers within the engineering organization and human resources."
Fedora board election results
Matt Domsch has been elected to the Fedora board. "On the "appointed" side, we are pleased to announce that Bill Nottigham has renewed his seat for another term, and that Bob McWhirter, the JBoss community manager, has accepted a seat on the Fedora Board that previously belonged to Chris Blizzard."
KDE-SIG weekly report (48/2007)
Click below for a report from Fedora's KDE Special Interest Group (SIG). Items on the agenda for week 48 include Trolltech's Phonon GStreamer backend, kdemultimedia3 compat package?, API documentation, Live images for KDE4, and development progress: the road to kde4.
Gentoo Linux
Council meeting summary for 13 December 2007
A summary of the December 13th Gentoo council meeting has been released. Some of the topics discussed at the meeting include new USE documentation, Code of Conduct enforcement.And even more new stages... (Funtoo)
Daniel Robbins, founder of Gentoo, has announced the availability of fresh stages for AMD64, i686 and x86 for Gentoo users. "Barring any build issues from upstream, I plan to offer fresh Gentoo stages that are no more than a week old at http://www.funtoo.org/linux/, so the next time you need a fresh stage tarball, please give one of mine a try. It will save you quite a bit of "emerge -u world" time. And thanks :)"
Ubuntu family
Countdown to Hardy Alpha 2
Ubuntu's Hardy Heron Alpha 2 is expected to be released on Thursday, December 20, 2007. "With the DebianImportFreeze now in effect, it's time to nudge another baby heron out of the nest and hope it flies better than this broken metaphor: it's time for Hardy Alpha 2."
Distribution Newsletters
Fedora Weekly News Issue 113
The Fedora Weekly News for December 10, 2007 is out. "In Announcement, we have "Samba Security Updates For FC6" In Planet Fedora, we have "Talks with Mark: RHM Video", "F8 on the PS3", "Back from India: FOSS.in", "A good flip-flop: FUDCon Raleigh 2008", "Re-spinning Fedora" and "Succession Planning"" Plus several other topics.
Ubuntu Weekly Newsletter #70
The Ubuntu Weekly Newsletter for December 15, 2007 covers the countdown to Hardy Alpha 2, new MOTU & community members, Ubuntu Forums interview, Bazaar 1.0 release, and much more.DistroWatch Weekly, Issue 233
The DistroWatch Weekly for December 17, 2007 is out. "Yes, it's that time of the year when DistroWatch takes a brief look at the events that shaped the distribution world during the past 12 months. Who were the winners and losers in 2007? Which distributions impressed most? Were there any major surprises? Read more in our feature story. In the news section, Mandriva enters a new development process with Cooker Alpha 1, Max Spevack resigns as Fedora Project Leader, MEPIS updates its artwork for the upcoming release of SimplyMEPIS, Daniel Robbins announces updated "stage" tarballs, and Ulteo delivers the first of its online services. Finally, many thanks to all our loyal readers and best wishes for the festive season! See you all in 2008!"
Distribution meetings
FUDCon Raleigh 2008
FUDCon Raleigh 2008 will be held as a Bar Camp, an un-conference. Everyone with an interest in Fedora is invited to join. This is a three day event, January 11-13, 2008 held in Raleigh, North Carolina at the NC State University Campus and Red Hat Headquarters.Second call for talks for the Debian DevRoom at FOSDEM 2008
The second call for talks for the Debian DevRoom at FOSDEM 2008 is out. "FOSDEM is the Free and Open Source Developers' European Meeting, which traditionally takes place at the Campus Solbosch of the Université Libre de Bruxelles (ULB) in Brussels, Belgium, during the last weekend of February."
Ubuntu Live Conference: Call for Proposals Open
The call for proposals for the second Ubuntu Live conference is out. The conference is co-presented by Canonical, Ltd and O'Reilly Media is slated to take place July 21-22, 2008 at the Oregon Convention Center in Portland, Oregon. The call for participation will be open until February 4, 2008.
Interviews
Paludis, Gentoo and Ciaran McCreesh uncensored (Obsethryl's Lab)
George Makrydakis talks with Ciaran McCreesh about Paludis. "As a project, paludis combines a lot of what you will see in F/LOSS, in both social structure and relations to the "fathering" project. Instead of presenting Paludis myself and why it is preferable to use it in a Gentoo system instead of portage, I took the liberty of asking Mr. Ciaran McCreesh, Chief developer among the Paludis team about a relatively gentle introduction to the Paludis world, why it became a necessity, its design and goals. Also the relation with Gentoo is examined, but also a glimpse at how F/LOSS can be a socially complicated issue emerges from this text. This was an email Q & A with Mr. McCreesh and the replies laid here are uncensored."
Distribution reviews
The World Series of Linux (CMP Channel)
CMP Channel compares Linux desktop distributions. Part 1 looks at the Debian based distributions Ubuntu, Freespire and Xandros. Part 2 looks at RPM based distributions SLED 10, Fedora 7 and PCLinuxOS. Part 3 pits the winners of part 1 and part 2 (Ubuntu and Fedora) against each other. "A close call, but Ubuntu wins the game and the title of Best Desktop Linux."
Review: Opensuse 10.3 Linux distribution (vnunet)
vnunet has a short review of openSUSE 10.3. "You will need to decide on what desktop to use. KDE is the default desktop and a preview of the upcoming KDE 4 implementation is included, as well as the current 3.5.7 version. The classic all-green Suse colour scheme is employed, with the enterprise version of Kontact now also included."
Page editor: Rebecca Sobol
Development
Hardware Fun with the Arduino board
Several weeks ago, your author took a look at the SquidBee project, which involves making a wireless remote sensor network from building blocks made of open-hardware components. At the heart of each of the SquidBee nodes is an Atmel AVR 8 bit RISC microprocessor, which sits on an Arduino Diecimila circuit board. This week, we'll take a look at the Arduino project:
![[Arduino Diecimila]](https://static.lwn.net/images/ns/ArduinoDiecimila.jpg)
AVR chips programmed with the Arduino on-board library software are available in a number of different hardware configurations. The Arduino Diecimila board is the one of the more popular variations, it features a USB host connection which provides power and allows for software downloads. The Diecimila name comes from the fact that 10,000 Arduino boards have been sold, making is a fairly popular development platform. Arduino Diecimila boards are available from a number of vendors for around $35. The board was purchased online and arrived in the mail several days later.
In addition to the basic processor board, there are numerous open-design shield boards available. Shield board functions that are currently available include: motor control, biosensor interface, prototyping, XBee interface, Phidget sensor interface, and potentiometer interface. Upcoming shield boards include: sensor amplifier, external memory, external display controller, Bluetooth interface and multi-sensor interface.
To work with the Arduino board, it is necessary to install some software on a host machine. Your author used his main Athlon 64 which runs Ubuntu 7.04. There is a special Ubuntu installation document that walks the user through the package installation (and removal) steps, and explains the software setup procedure.
Running the Arduino IDE was a simple matter of typing ./arduino on the command line, which caused the IDE window to pop up. The IDE defaulted to the Diecimila board type, it was necessary to define the USB connection in the Tools/Serial Port pulldown. The first attempt at running an LED blinker test program resulted in a bit of operator confusion. The board is apparently shipped with this particular software example installed, so installing the same test software does not change the appearance of the already blinking LED.
The Blinker software was pulled into the IDE with the File->Sketchbook->Examples->Digital->Blink menu sequence. The software was built with no trouble using the Verify button and copied to the board using the Upload button. The LED started blinking again. Tweaking the delay times in the example code, then building and uploading the changed code verified that indeed, changes were being sent to the board. There is another slightly confusing interface aspect to the IDE, there are tape recorder style run/stop buttons at the top of the screen, but the run button is really the Verify (compile) function and the Stop button didn't seem to stop the running code.
The software that the Arduino board runs is written in the Arduino programming language, which looks a lot like C/C++ and is based on the wiring language. Making a few changes to the blinking LED example was so intuitive that it was not even necessary to consult the documentation. The Button example was also tried, digital input to the board worked as advertized.
Further testing of the I/O functions of the Arduino Diecimila board will require some hardware construction, which is beyond the scope of this (first) article. Your author has been building simple and complicated microcontroller projects for a number of decades; his initial impression of Arduino is that it has a very quick learning curve and provides a lot of powerful features. The Atmel AVR microcontroller provides a lot of useful I/O functionality and enough memory to build many interesting devices.
If you are looking for a convenient way to design a microcontroller based hardware project, extend the I/O capabilities of your desktop system, or just play with some cool hardware, Arduino is a quick and easy way to get started.
System Applications
Database Software
Firebird 1.5.5 released
Version 1.5.5 of the Firebird DBMS has been announced. "This bug-fix release adds no new features but addresses a few bugs and security issues that have turned up in the 11 months since v.1.5.4."
Postgres Weekly News
The December 16, 2007 edition of the Postgres Weekly News is online with the latest PostgreSQL DBMS articles and resources.SQLite 3.5.4 released
Version 3.5.4 of SQLite, a light weight DBMS, has been announced. Changes include critical bug fixes, standardization of ORDER BY, improvements to VACUUM, IN operator expression improvements and more.
Networking Tools
NagVis: 1.2 final released (SourceForge)
Version 1.2 final of NagVis has been announced. "NagVis is a visualization addon for the well known network managment system Nagios. NagVis can be used to visualize Nagios Data, e.g. to display IT processes like a mail system or a network infrastructure. I'm proud to present you NagVis 1.2 in the final version. It's nearly 3 month since the final release of NagVis 1.1, there have been some interesting changes."
NWrapper: first release (SourceForge)
The initial beta release of NWrapper has been announced. "NWrapper was built to be a quick wrapper for storing and executing multiple NMap commands (using SQLite), but it can do a lot more. Also, it was a way for me to start learning C (hence the lack of data structures or anything fancy)."
Printing
CUPS 1.3.5 is out
Version 1.3.5 of CUPS has been announced. "CUPS 1.3.5 is now available from the CUPS web site and fixes some SNMP and PDF filter security issues, some USB printing issues, and several scheduler issues."
Web Site Development
faceCart: Maintenance Build Released (SourceForge)
A new maintenance build of Face Cart has been announced. "face cart is AJAX powered shopping cart presenting unique user experience.e-commerce system designed in the patterns of oscommerce,face cart is Java 5EE e-commerce solution. The shopping cart provides unmatchable speed.Supports all database servers. It is recommended to download the new Build of faceCart. It fixes several problems related to the deploy process."
Intranet for alumni communities: available for download (SourceForge)
The initial release of Intranet for alumni communities has been announced. The software is a: "Web "Portal" with secured services as : directory, job offers, etc ... especially for the use by Alumni associations".
ZK: 3.0.1 Released (SourceForge)
Version 3.0.1 of ZK has been announced. "ZK is Ajax framework enriching Web apps with little programming. With event-driven and markup languages, development is as simple as programming desktops and authoring HTML/XUL pages. ZK supports scripting lang including Java, JavaScript, Ruby, Groovy... Over 38 new features and 58 bugs fixed, ZK 3.0.1 focuses mainly on fixing bugs and improving performance. New feature include GenericComposer, GenericEventListener, data-binding supports Map, integration with EJB, and etc."
Desktop Applications
Business Applications
JasperReports: 2.0.3 released (SourceForge)
Version 2.0.3 of JasperReports has been announced, it adds some new capabilities and includes some bug fixes. "JasperReports, the market leading open source business intelligence and reporting engine. This project is being moved to http://www.jasperforge.org/. This project is the home for all things Jasper, Reports, Analysis, Server, and Intelligence."
Calendar Software
pcal 4.11.0 released
Version 4.11.0 of pcal, a program which generates PostScript or HTML calendars, has been released. "Changes include fixes for all known bugs, support for new languages (Hawaiian and Slovak), support for moon icons and Julian dates on yearly-format calendars, support for a new preposition ('on') for certain calendar events, additional sample calendar event files, support for the Amiga platform, and other minor improvements."
Desktop Environments
GNOME Software Announcements
GNOME Software Announcements The following new GNOME software has been announced this week:- Deskbar-Applet 2.21.4 (new feature, bug fixes and translation work)
- Empathy 0.21.4 (new features, bug fixes and translation work)
- Epiphany 2.21.4 (new features, bug fixes and translation work)
- Eye of GNOME 2.21.3 (bug fixes and translation work)
- gcalctool 5.21.4 (bug fixes, documentation and translation work)
- Glade 3.4.1 (bug fixes and translation work)
- Gnome Games 2.21.4 (code cleanup, bug fixes and translation work)
- gnome-keyring 2.21.4 (new features, bug fixes and translation work)
- GNOME Power Manager 2.20.2 (bug fixes, documentation and translation work)
- GNOME Power Manager 2.21.1 (bug fixes and translation work)
- GnomePythonDesktop 2.21.1 (new features and bug fixes)
- gnome-speech 0.4.17 (bug fix)
- Gnome Subtitles 0.7.1 (bug fixes and translation work)
- GnuCash 2.2.2 (new features, bug fixes and translation work)
- Gtk2-Perl 2.21.4 (new features, bug and build fixes)
- gtk-engines 2.13.2 (bug fixes and translation work)
- libepc 0.3.1 (new feature and bug fixes)
- libgnomekbd 2.21.4.1 (bug fixes and translation work)
- libspectre 0.1.0 (first public release)
- MonoDevelop 1.0 beta 3 (new features)
- Orca 2.21.4 (new features, bug fixes and translation work)
- pypoppler 0.6.1 (new feature and code cleanup)
- seahorse 2.21.4 (new features, bug fixes and translation work)
- Swfdec 0.5.5 (new features and bug fixes)
- Tomboy 0.9.2 (new features and bug fixes)
KDE Commit-Digest for 9th December 2007 (KDE.News)
The December 9, 2007 edition of the KDE Commit-Digest has been announced. The content summary says: "The "simple menu" (similar to the menu found in the KDE 3 series) becomes usable. The clock receives a popup-based calendar widget, with KRunner becoming multi-threaded in Plasma. Work continues the long-awaited update of KBugBuster, with important development milestones reached. Version Control and other general work in KDevelop. Start of a DirectShow (for Windows) backend for Phonon, and the integration of this backend in Amarok 2.0..."
KDE Software Announcements
The following new KDE software has been announced this week:- Amarok cover to Kopete avatar 1.0B (beta release)
- amaroK Web Collection 1.0.1 (new features and bug fixes)
- amaroK Web Collection V xmms x1.0.1 (unspecified)
- cueIt .03 (bug fix)
- eric4 4.0.4 (bug fixes)
- indywiki 0.9.4 (new features and bug fixes)
- KCheckGMail 0.5.7.4 (bug fixes and translation work)
- KDbg 2.1.0 (new feature)
- KDelicious 3.0 (unspecified)
- KleanSweep 0.2.9 (feature removal and bug fixes)
- konqil.icio.us 2.1 (new features, bug fixes and translation work)
- KPS 0.4 (maintenance release)
- KPV 0.3.3 (new feature and bug fix)
- kReiSSy 0.2.1 (unspecified)
- KWorkGroup 0.3.0 (unspecified)
- lrcShow-II 0.4.0 (initial release)
- MBPAR 1 (bug fix)
- PokerTH 0.6 (new features, bug fixes and translation work)
- 'Q' DVD-Author RC 3 (new features and bug fixes)
- Qsynth 0.3.2 (bug fixes)
- QtiPlot 0.9.3-rc1 (new feature and bug fixes)
- QtiPlot 0.9.3-rc2 (new features and bug fix)
- Qtpfsgui 1.9.0 (unspecified)
- Traverso 0.42.0 (new features, bug fixes and translation work)
- Zhu3D 3.3.6 (new features, bug fixes and code cleanup)
Xorg Software Announcements
The following new Xorg software has been announced this week:- xf86-input-joystick-1.3.1 (new features and bug fixes)
- xorg-server 1.4.0.90 (new features and bug fixes)
Desktop Publishing
LyX 1.5.3 is released
Version 1.5.3 of LyX, a GUI front-end to the TeX typesetting system, is out. "This is a maintenance release that further improves the stability and the performance. Besides numerous crashes, the display problems that slipped into 1.5.2 with the performance fixes (on the Mac and on Windows) as well as problems entailed to the reworked document classes were fixed. Furthermore, LyX 1.5.3 comes with speed improvements that should pay off especially on the Mac and other UNIXes. Finally, this version also provides some new features."
Electronics
Electric 8.06 announced
Version 8.06 of the Electric VLSI Design System has been announced. "This release includes many improvements and bug fixes. Two notable features are the new Thin Film technology (tft) and an improved technology editing facility."
Financial Applications
SQL-Ledger 2.8.10 released
Version 2.8.10 of SQL-Ledger, a web-based accounting system, has been announced. The changes include: "added audit trail for statements, fixed lineitem reordering for previously saved orders and quotations, # fixed missing function call for payments batch, added mid-commit to voucher posting routine to override PostgreSQL's 8+ constraint bug, added reference to yearend procedure if none is supplied and added missing function call to destroy statement handle".
Fonts and Images
Movable Type released under the GPL
Back in 2004, LWN covered the fuss surrounding a license change for Movable Type which had the effect of requiring payments from many site operators. Our point at the time was that this software had never been made available as free software, so that kind of change was always a possibility. No longer: Movable Type is now available under GPLv2. "Like many of us on the team, some of you have been waiting for this moment for years. For a business, an open source license affects boring things like how a product is created, updated, and distributed. But the open source movement has always been about something more important: Freedom."
Games
Robocode: 1.5.1 fixes a security issue (SourceForge)
Version 1.5.1 of Robocode has been announced. "Robocode is a Java programming game, where the goal is to develop a robot battle tank to battle against other tanks. The robot battles are running in real-time and on-screen. The motto of Robocode is: Build the best, destroy the rest!"
Interoperability
Samba 4.0.0alpha2 released
Version 4.0.0alpha2 of Samba has been released. "Samba 4 is the ambitious next version of the Samba suite that is being developed in parallel to the stable 3.0 series. The main emphasis in this branch is support for the Active Directory logon protocols used by Windows 2000 and above. Samba 4 is currently not yet in a state where it is usable in production environments. Note the WARNINGS below, and the STATUS file, which aims to document what should and should not work."
Wine 0.9.51 released
Version 0.9.51 of Wine has been announced. Changes include: "A bunch of WinHelp improvements, Better Japanese font support,A ton of rpcrt4 fixes,Several Alsa capture fixes,Improved support for screen resolution changes and Lots of bug fixes."
Mail Clients
Claws Mail 3.2.0 announced
Version 3.2.0 of Claws Mail has been announced. This release adds many new features and bug fixes.
Medical Applications
Freemed-YiRC V1.00 Released (LinuxMedNews)
LinuxMedNews has announced version 1.00 of Freemed-YiRC. "Freemed-YiRC is an open source software project intended for use as a complete information system by child caring agencies. Freemed-YiRC originally started out as an intention to add child care functions into FreeMED, however it was quickly realized that the needs of child caring agencies were different and the project was forked. Hence, the Freemed-YiRC software project was born. YiRC = Youth in Residential Care."
PatientOS 0.26 released (LinuxMedNews)
Version 0.26 of PatientOS, a healthcare information system, has been announced. "This version marks the start of upgrade support for installation by providing a clean database and adding code to upgrade the database schema, data contents, server and client. Issues are now being logged in Jira. Scheduling setup and configuration tools have been added to build Resources and Appointment Types. A new registration form was added configured to streamline data entry."
Proteus Clinical Guidelines to go Open Source (LinuxMedNews)
LinuxMedNews notes plans to release the Proteus intelligent clinical guidelines tools as open-source software. "Lighting up the AMIA os-wg and OpenHealth e-mail discussion lists comes news that the Proteus 'intelligent clinical guidelines' tools are going to be open sourced with an as yet to be announced Free/Open Source license".
Music Applications
dssi-vst 0.5 announced
Version 0.5 of dssi-vst, a DSSI plugin wrapper for Win32 VST effects and instruments, is out. "The 0.5 release now comes with Javier Serrano Polo's VST-compatibility header, as previously distributed in LMMS. (Actually, this header was already compatible with dssi-vst -- no modifications to dssi-vst were necessary -- it's just that the header is now included in the package.) This permits it to be compiled without the official VST SDK and distributed under pure GPL."
Rubber Band 1.0.1 released
Version 1.0.1 of Rubber Band, an audio time-stretching and pitch-shifting library and utility, is out. "This small update (v1.0.1) fixes an option parsing bug and a dodgy bit of #ifdef nesting. The core code is the same as in 1.0."
Web Browsers
Mozilla Links Newsletter
The December 13, 2007 edition of the Mozilla Links Newsletter is online, take a look for the latest news about the Mozilla browser and related projects.
Word Processors
OxygenOffice Professional 2.3.1 released (SourceForge)
Version 2.3.1 of OxygenOffice Professional has been released, it features bug fixes. "OxygenOffice Professional (was: OpenOffice.org Premium) is an enhanced version of OpenOffice.org what is a multi-platform office productivity suite. OxygenOffice Professional contains more extras like templates, cliparts, samples, fonts and VBA support."
Languages and Tools
Caml
Caml Weekly News
The December 18, 2007 edition of the Caml Weekly News is out with new articles about the Caml language.
Java
Explore from here: A minor-MAJOR version (SourceForge)
A new version of Explore from here has been announced. "this NetBeans module add a custom action to filesystem nodes to launch an OS explorer pointing the directory referenced by the node. It can be customized specifying an implementation of net.sf.efhnbm.Launcher or a command."
Perl
Parrot 0.5.1 released (use Perl)
Version 0.5.1 of Parrot has been announced. "On behalf of the Parrot team, I'm proud to announce Parrot 0.5.1 "Hatachi." Parrot is a virtual machine aimed at running all dynamic languages."
Perl 5.10.0 is released
Version 5.10.0 of Perl has been announced. "Perl 5.10.0 is now out, the first in the 5.10.x major version series, after a five year long development process. It's currently being mirrored on CPAN."
Python
Python-URL! - weekly Python news and links
The December 18, 2007 edition of the Python-URL! is online with a new collection of Python article links.
Version Control
monotone 0.38 released
Version 0.38 of monotone has been announced. "A new release! 0.38 has few but important changes and bug fixes."
Page editor: Forrest Cook
Linux in the news
Recommended Reading
Video: Alan Cox on the state of free software
![[Alan Cox]](https://static.lwn.net/images/ns/AlanCoxVideo.png)
Companies
iPlayer for Linux wins approval of open sourcers (ZDNet UK)
ZDNet UK looks at the BBC's move to make its iPlayer online on-demand TV service available for streaming on Linux systems. "Following a meeting with the OSC [Open Source Consortium], the BBC's independent governing body, the BBC Trust, restated its commitment to make the download version of iPlayer "platform agnostic"."
Canonical releases Version 1.0 of 'Bazaar' version control tool (Linux-Watch)
Linux-Watch covers the release of Bazaar 1.0, a distributed version control system in Launchpad. "In a Linux-Watch interview, Shuttleworth explained that by making it easier to work in independent branches, which can then be easily adopted into the main code tree, Bazaar encourages developers to explore new ideas within a project rather than forking their new idea into another, related open-source project. This, in turn, "lets new developers start contributing immediately and working on new ideas even when they can't get buy-in with old guard." Thus, "this discourages forks and helps with the social conflicts between new and older project developers. It makes it easier for people to learn, work and have fun together on a project.""
Hospital software vendor McKesson uses Linux to heal IT budgets (Computerworld)
Computerworld reports on McKesson's move to from mainframes to Linux. "Today, San Francisco-based McKesson offers about 50 of its 70 most popular health care applications -- dealing with everything from billing to pharmacy records, staffing, admissions, physician order entry systems and surgery scheduling -- on Linux, reducing costs for hospitals and medical offices. The move was solidified in February, when McKesson partnered with Linux vendor Red Hat Inc. to unveil the Red Hat Enterprise Healthcare Platform, which was customized to meet the needs of the health care industry." (Found on LinuxMedNews).
Red Hat delays new software for PCs until January (Reuters)
Here's a brief Reuters article on the latest delay in Red Hat's desktop product. "Late on Monday, spokeswoman Leigh Day said the company planned to release the software in January, five months after the original target date of August that it had promised customers. She said Red Hat was postponing the product's release again because it has yet to resolve problems getting the right to distribute software for playing music and viewing videos with the Linux software."
Trolltech Hosting Phonon backends in KDE subversion repository (KDE.News)
KDE.News covers the release of the Phonon backends by Trolltech. "Trolltech announced today that the Phonon backends, which they have been developing for inclusion in Qt, are being transferred into the KDE source code repository. Phonon is the KDE 4 API for multimedia and is also set to be part of Qt 4.4, scheduled for the end of Q1 2008. You heard it right folks, a part of Qt will be officially hosted and developed inside KDE's very own Subversion repository, from whose loins Phonon first sprung, and be freely available to all under the LGPL."
Linux at Work
Linux gives the NYSE lower costs and independence (CNET)
Over at CNET, Matt Asay discusses the recent news about the New York Stock Exchange rolling out more Linux and less proprietary UNIX. "For those who believe they need to earn their living and make the difficult decisions that turn IT into a functional part of one's business, however, there are better options. Open source is one of them, of course, but it need not be the exclusive option. Sometimes a proprietary system will better fit a CIO's requirements. That's fine. But the point is that it should be the CIO who makes that decision, not the vendor."
Legal
Opera Files Antitrust Complaint Against Microsoft with EU Commission (Groklaw)
Groklaw reports that Opera has filed an antitrust complaint against Microsoft. "If you use any browser but Microsoft's you already know all about the problems you encounter. Nor is this an Opera theoretical. Remember this story from 2001, where Opera was allegedly directly targetted by Microsoft, locked out of Microsoft's MSN portal? Then again in 2003? After you read all that, next read these boldly inaccurate excuses Microsoft first tried to peddle about HTML standards and why Opera didn't work. Well, now the chickens have come home to roost."
Resources
Samba Domaincontroller For Small Workgroups With SWAT On Fedora 8 (Howtoforge)
Howtoforge works with SWAT on Fedora 8. "This document describes how to set up and configure a Samba Domaincontroller for small workgroups (up to 250 users) on Fedora 8 with the Samba Web Administration Tool. The resulting system provides an easy to manage domaincontroller for your Windows network."
Reviews
Firefox 3 beta 2 Arrives with More Speed and Fewer Bugs (Wired)
Wired reviews the second Firefox3 beta. "Linux users will be happy to note that beta 2 brings in the native GTK theme for Firefoxs default icons, buttons, and menu styles. Firefox finally looks like every other Gnome application and if the Linux platform is any indication, the final release of Firefox 3 will look perfectly native regardless of what OS you're using."
A first look at KDE 4.0 release candidate 2 (ars technica)
ars technica reviews the second KDE 4.0 release candidate. "The second KDE 4 release candidate illuminates the extent to which KDE 4 has matured since the earlier betas, but a massive infusion of debugging and polish is needed before the release next month. Heavy development on KDE 4 will obviously continue after the KDE 4.0 release, so whatever pieces are still missing are sure to be filled in eventually. Some critics point to the deficiencies of KDE 4 and argue that drastic reinvention of basic desktop components might not have been a good idea. After experiencing KDE 4 myself, I have to disagree."
Linux Networking Cookbook: Tasty Linux recipes (Linux-Watch)
Linux-Watch reviews the Linux Networking Cookbook. "In her book, [Carla] Schroder delivers exactly what she promises: recipes for creating tasty and useful Linux and TCP/IP networking setups. Want to know how to build a VOIP (voice over IP) server with Asterisk? How to create a single sign-on for hybrid Linux/Windows LANs? Or, how to create a real VPN with OpenVPN a Linux-based PPTP (Point-to-Point Tunneling Protocol) server? It's in there."
Page editor: Forrest Cook
Announcements
Non-Commercial announcements
Another BusyBox lawsuit settled
The Software Freedom Law Center has sent out a press release announcing the settlement of the GPL-infringement suit filed on behalf of the BusyBox developers against Xterasys. "Once SFLC verifies that the complete source code is available, Xterasys' full rights to distribute BusyBox under the GPL will be reinstated. Additionally, Xterasys has agreed to appoint an internal Open Source Compliance Officer to monitor and ensure GPL compliance, and to notify previous recipients of BusyBox from Xterasys of their rights to the software under the GPL. Xterasys will also pay an undisclosed amount of financial consideration to the plaintiffs."
European Broadcast Union adopts FLAC
The FLAC audio compression project notes the adoption of the FLAC format by the European Broadcast Union. The EBU's FAQ explains: "The EBU Musipop is an audio file transfer system. All concerts on the Euroradio satellite channels, Ravel & Verdi, are recorded by the EBU Geneva Musipop PC as WAV files. They are subsequently converted into FLAC files and sent via satellite over a dedicated 4.8Mbit/s xtranet channel to member stations. Concerts are stored on your local Musipop PC for up to 2 months (memory with standing) from where they are transferred to your local hard drive playout systems."
Commercial announcements
Novell reports fourth quarter financial results
Novell, Inc. has announced its fourth quarter 2007 financial results. "For the quarter, Novell reported net revenue of $245 million, which excludes $6 million of revenue from its Swiss-based business consulting unit, which Novell agreed to sell during the quarter. This compares to net revenue of $234 million for the fourth fiscal quarter 2006. The loss from operations for the fourth fiscal quarter 2007 was $13 million, compared to income from operations of $4 million for the fourth fiscal quarter 2006."
xTuple Debuts Managed Server Network
xTuple has announced XTN, the xTuple Network service. "xTuple, the leader in open source enterprise resource planning software, is pleased to announce the general availability of XTN, the xTuple Network service, for users of the xTuple Applications, PostBooks and OpenMFG. The xTuple Applications are advanced ERP software solutions built with open source components, such as the PostgreSQL database, the Qt toolkit for C++, and the OpenRPT report writer. The fully integrated packages include Inventory Management, Product Definition and Costing, Work Order Management, Manufacturing, Purchasing, Sales, Shipping and Receiving, Project Management, Sales Analysis, Accounts Payable, Accounts Receivable, a full General Ledger, and Customer Relationship Management. Both packages are fully multi-currency, multi-lingual, and support a range of multi-layered taxation structures."
New Books
Digital Astrophotography--New from Rocky Nook
Rocky Nook has published the book Digital Astrophotography by Stefan Seip.The Django Book is done
The Django Book has been announced. "The Django Book started shipping last week, and we've put the full text online for free. We put a draft of the book up about a year ago for comments, and were amazed by the quality (and quantity!) of responses. We read each of the comments (around 2500) as we revised the book towards a final print release. That print release has been available in stores for about a week, and we've put the text up for you to read for free."
Learning ActionScript 3.0--New from O'Reilly Media
O'Reilly has published the book Learning ActionScript 3.0 by Rich Shupe and Zevan Rosser.X Power Tools--New from O'Reilly Media
O'Reilly has published the book X Power Tools by Chris Tyler.
Resources
FSFE Newsletter
The December 17, 2007 edition of the FSFE Newsletter is online with the latest Free Software Foundation Europe news. Topics include: United Nations Internet Governance Forum (IGF), STACS meeting in London, Trophees du Libre 2007 in Soissons, Training Courses in Stockholm and Nijmegen, FTF events in Linz, Lausanne, Nijmegen and Dusseldorf, Foundation activities in Sweden, Berlin Fellowship discusses Free Software mobile phones, FSFE revisiting software patent information, SELF public beta and bug fixing and Interview with Werner Koch.The Perl Review, Winter 2007 (use Perl)
The Winter, 2007 edition of the The Perl Review has been announced. "The Winter 2007 issue of The Perl Review is here, and it has a wonderful cover picture that Eric Maki made with a combination of the B modules and GraphViz. Wonder what Perl's really doing with your program? Map it and find out! That's just the cover, and there is a lot more Perl on the inside."
An analysis of the SonyBMG rootkit disaster
Deirdre K. Mulligan and Aaron K. Perzanowski have posted a 76-page paper [PDF] on the causes of the SonyBMG rootkit fiasco. "This Article aims to identify the market, technological, and legal factors that appear to have led a presumably rational actor toward a strategy that in retrospect appears obviously and fundamentally misguided." There's also a couple of detailed suggestions on (U.S.) legal changes which could help make such episodes less likely in the future.
Calls for Presentations
Embedded Linux Conference 2008 - call for sessions
The CE Linux Forum has put out a Call for Presentations for the 2008 Embedded Linux conference to be held in Mountain View, CA April 15-17. The conference will be held at the Computer History Museum and presentations are being sought for many different topics of interest to embedded Linux developers. Click below for more information.SOA in Health Care Conference -- A Call for Abstracts (LinuxMedNews)
A Call for Abstracts has been posted for the SOA in Health Care Conference. The submission deadline is December 31. "The HSSP effort is pulling together an industry conference entitled "SOA for Health Care". Note that the event is focused on case-studies around SOA in health care, and is not about the standards themselves."
SyScan'08 Call For Paper/Training
Two SyScan'08 events have been announced, along with calls for papers. "The Symposium on Security for Asia Network aims to be a very different security conference from the rest of the security conferences that the information security community in Asia has come to be so familiar and frustrated with. SyScan is a non-product, non-vendor biased security conference. It is the aspiration of SyScan to congregate in Asia the best security experts in their various fields, to share their research, discovery and experience with all security enthusiasts in Asia." SyScan'08 Hong Kong will take place on May 29-30, 2008, SyScan'08 Singapore will take place on July 3-4, 2008.
Upcoming Events
Linux Foundation to host a symposium in Beijing
The Linux Foundation has announced that it will be co-hosting (with the Chinese OSS Promotion Union) a Linux developer symposium in Beijing, China. Speakers will include Dave Neary, Andrew Morton, Matt Mackall, and a certain LWN editor. "Similar to the Linux Foundation's Japanese Symposia, this event is intended to educate and promote cross-collaboration among Linux kernel developers and local developers in the region, resulting in increased kernel involvement and patch submissions."
Program Unveiled for MySQL Conference and Expo
The program for the 2008 MySQL Conference & Expo has been announced. "Registration is now open for the sixth annual MySQL Conference & Expo. Co-presented by MySQL AB and O'Reilly Media, the conference will take place April 14-17, 2008, in Santa Clara, California. The event is expected to bring together over 1,600 open source and database users from some of the most exciting and fastest-growing companies in the world, as well as from the large and active MySQL Community. The program for 2008 will include keynote presentations by Jacek Becla of Stanford Linear Accelerator and MySQL CEO Marten Mickos."
PostgreSQL Conference East 2008
The PostgreSQL Conference East 2008 has been announced. The event will take place in College Park, Maryland on March 29-30, 2008. "The conference series is designed to be a geographically strategic series of conferences that allow contributors, current users and future users/developers to learn and network. Each conference is held in an Academic facility, students and educators are free. Our goal is to establish a series of forums for local developers, administrators and users to mingle with leading PostgreSQL contributors. Initially these forums and conferences will be held in the U.S."
YAPC::NA 2008 Dates and Venue (use Perl)
The YAPC::NA 2008 Perl conference has been announced. "The Chicago Perl Mongers are excited to officially announce the location, dates, and website for YAPC::NA 2008. The conference will be held June 16th-18th 2008 at the Illinois Institute of Technology in Chicago, IL."
Events: December 27, 2007 to February 25, 2008
The following event listing is taken from the LWN.net Calendar.
Date(s) | Event | Location |
---|---|---|
December 27 December 30 |
24th Chaos Communication Congress | Berlin, Germany |
December 31 | Israeli Perl Workshop | Ramat Efal, Israel |
January 11 January 13 |
FUDCon Raleigh 2008 | Raleigh, NC, USA |
January 16 January 17 |
QualiPSo Conference 2008 | Rome, Italy |
January 17 January 19 |
KDE 4 release event | Mountain View, CA, USA |
January 24 | Federal DBA Day | Washington DC, USA |
January 28 February 2 |
Linux.conf.au 2008 | Melbourne, Australia |
January 28 February 1 |
Ruby on Rails Bootcamp with Charles B. Quinn | Atlanta, Georgia, USA |
January 29 January 31 |
Solution Linux 2008 | Paris, France |
February 1 | Open Island | Belfast, United Kingdom |
February 6 February 10 |
O'Reilly Money:Tech Conference | New York, NY, USA |
February 7 | Frozen Perl 2009 | Minneapolis, United States |
February 8 February 10 |
Southern California Linux Expo | Los Angeles, USA |
February 10 February 13 |
NDSS Symposium 2008 | San Diego, CA, USA |
February 11 | Florida Linux Show 2008 | Jacksonville, Florida, USA |
February 11 | Open Source Software (OSS) and the U.S. Department of Defense (DoD) | Alexandria, VA, USA |
February 13 February 15 |
German Perl-Workshop | Regionales Rechenzentrum Erlangen, Germany |
February 16 | Frozen Perl 2008 Workshop | Minneapolis, USA |
February 19 February 20 |
Linux Developer Symposium | Beijing, China |
February 19 February 20 |
Files and Backup | London, UK |
February 22 February 24 |
freed.in/2008 | Delhi, India |
February 23 February 24 |
Free/Open Source Developers' European Meeting 2008 | Brussels, Belgium |
February 23 February 26 |
Linux World Mexico | Mexico City, Mexico |
If your event does not appear here, please tell us about it.
Web sites
Free Your Media: A new blog on Linux multimedia
Pawel Wolniewicz, a Polish user of open source multimedia applications, has announced his new blog, Free Your Media. Articles so far include Traverso 0.42.0 released, Podcasting with Linux Command Line Tools and Audacity, 7 Alternatives to Flickr, and much more.
Miscellaneous
Mozilla Foundation Directed Giving Enables Specific Donations for Bugzilla, Camino, SeaMonkey and Accessibility (MozillaZine)
mozillaZine covers the Mozilla Foundation's directed giving program where donors choose one of four specific projects to direct their donation to. "While the Mozilla Foundation has accepted donations since shortly after its establishment in 2003, it has not previously been possible for supporters to specify how they want their money to be spent. With the launch of the directed giving program, Mozilla donors can now allocate their funds to be spent on Bugzilla (the open-source bug tracking software used by many software development projects), Camino (the Mozilla-powered native Mac OS X browser), SeaMonkey (the community-driven continuation of the Mozilla Application Suite) or the Mozilla Accessibility Community (which aims to make Mozilla software easier to use for users with disabilities)."
Page editor: Forrest Cook