Sponsored link
Serve your customers, not your servers, with VERIO Linux VPS. Full-access test-drive here.
| Weekly edition | Kernel | Security | Distributions | Search |
| Archives | Calendar | Subscribe | Write for LWN | LWN.net FAQ |
LWN.net Weekly Edition for April 10, 2008
OpenSSH bug falls through the cracks
Linux distributions often patch the software they distribute, to fix bugs or add features. Anything they add is pushed upstream to the project responsible for the package—at least in theory. When that theory is not borne out in practice, it can lead to the kind of unhappiness and finger pointing that went along with a recent OpenSSH release. The release notes point at Debian for failing to report it upstream, but the bug was actually fixed much earlier, in Red Hat Enterprise Linux 4 (RHEL4).
The bug in question is rather nasty, allowing a local attacker to hijack X Windows programs of a user who logged in using ssh with X forwarding enabled. Under those circumstances, the ssh client and server arrange that any X programs started on the logged-in machine actually display on the client machine's desktop. This is very useful for running X programs across the internet as the X traffic is encrypted as part of the ssh session.
Due to a broken interaction with Internet Protocol version 6 (IPv6)—the next generation protocol for internet traffic—ssh can get confused about the port number of the X server. If a particular port (which maps to the X DISPLAY environment variable) is not available to be used under IPv4—the protocol in use today—but is available under IPv6, the ssh server will incorrectly set DISPLAY. If it is an attacker's program that is listening on the IPv4 port, it will be able to hijack X programs that get run.
Up until sometime in the last several years, this would not have happened for most Linux boxes because IPv6 was generally not enabled. In that case, the ssh server would recognize that it could not get the port it wanted and try another, eventually setting DISPLAY correctly. Because IPv6 is much newer, these kinds of bugs may exist in other network programs. This bug should serve as a reminder to developers to closely check their IPv6 support.
Clearly, though, the bug fell through the cracks. The OpenSSH team shows its annoyance in the release notes:
It was reported in January to the Debian bug tracking system, but not fixed and released until late March. OpenSSH does releases every six months or so, with 4.9 being released on March 30. Having to turn around another release four days later to fix a problem that was known for a few months could certainly make for annoyed developers. So how did the bug get fixed in Debian, with a Common Vulnerabilities and Exposures (CVE) number being assigned, but without notifying the OpenSSH team?
The Debian bug entry is instructive, because it documents some of the steps that led to the hurried release. In particular, Phil Miller thought he had done the right thing to report the problem in February:
That email must have gotten lost or been eaten by a spam filter as de Raadt would presumably have gotten it to the right people had he seen it. The bug description clearly puts it in the realm of a security problem, but the bug was not classified that way in the Debian system. Had it been, it would have been handled differently, possibly triggering an email to the proper place. But the bug report also shows that Red Hat fixed it in 2005.
It was reported to Red Hat by a customer and got entered into their bugzilla as bug #163732. Unfortunately, that bug report is confidential because it contains potentially sensitive customer information. This makes it difficult to track further. Indications are that it was not seen as a security problem and that it was believed to have been already known as an OpenSSH bug. Apparently no one checked to make sure the OpenSSH folks knew of it though.
Closer cooperation between the OpenSSH maintainers for Red Hat and the upstream team would probably have helped. Red Hat has been carrying the patch along for quite some time. Because the security implications were not clear and the patch is quite simple, it may not have seemed to be all that necessary to get it upstream. Though, there are more than twenty patches listed in the fedora OpenSSH CVS repository for rawhide, which will become Fedora 9.
The OpenSSH team would be well served by paying closer attention to various distribution patches to their code as well. It is certainly plausible that those interested in finding security holes to exploit might start by seeing if any patches floating around for critical services like OpenSSH were useful. By being more proactive, OpenSSH might have found and fixed this bug much earlier. The way this particular bug avoided notice seems to be mostly happenstance; if there is blame to be placed, there is plenty to go around.
RHEL and other "enterprise" distributions have long support cycles which means that the versions of various packages being maintained are well behind the upstream project. It doesn't take very many bug reports getting shot down because they have already been fixed in a more recent version before distribution maintainers lose enthusiasm for making those reports. But it is an essential part of the process. The OpenSSH team has the reputation of being somewhat difficult to work with, which may have helped this particular problem get overlooked.
It is a difficult problem to solve fully. Distributions have their own set of requirements which may be in opposition to those of the upstream project. Those projects may also have policies and procedures that distributions are not up to speed on. The Linux kernel often sees the same kind of conflicts, which is why distributions often maintain their own set of kernel patches for features their customers need. But it is in everyone's best interest to work those problems out so that distributions carry along as few patches as possible while upstream projects do not miss out on bug fixes and features.
Discussing desktops at the Collaboration Summit
Your editor is typing this from the Linux Foundation's collaboration summit, currently in progress in Austin, Texas. The day's agenda includes giving a talk on the state of the kernel during the evening reception; beer-fueled hecklers would appear to be in your editor's near future. The first day, though, included a rather more sober panel on the state of the Linux desktop which revealed some interesting thoughts on where things are going.This panel, moderated by Steven Vaughan-Nichols, featured John Hull from Dell, David Liu (gOS), Jim Mann (HP), Timothy Chen (Via), Kelly Fraser (Xandros), Grégoire Gentil (Zonbu), Ellis Wang (Asus), Debra Kobs-Fortner (Lenovo), and a representative from Everex whose name your editor did not catch. Together, they represented a wide range of industries, from component makers and operating system vendors to providers of complete systems. They take different approaches to the Linux desktop, but they are all optimistic about where it is heading - though some are more so than others.
So how are these vendors doing with desktop Linux? While all of the vendors were optimistic, some were more guarded than others. Dell states that sales have "met expectations," but are aimed mostly at niche markets so far. There is, they say, a lot of interest in emerging markets, where users can start with Linux from the outset and do not have to migrate from other platforms. HP was also moderate in its enthusiasm, saying that its sales are "right about at the industry average." Lenovo was cautiously optimistic; their Thinkpad offerings are targeted at business users, which is a slower market to get into. According to Lenovo, most of their Linux-based sales are custom products designed for specific businesses.
Rather more enthusiasm came from gOS, the company which supplied the distribution for Wal-Mart's low-end PC. Sales, they say, are "very good." Asus is clearly happy with the success of the Eee PC. That success, they say, comes from the effort put into designing a complete solution for users, with features like quick booting and solid-state storage: "you drop it, it still works." Everex says that "sales are brisk"; the company is pleased and will continue to offer Linux-based products - including the "MyMiniPC", a small system aimed specifically at MySpace users. Via's components are found in a number of small Linux systems, including the Eee PC, so Via is happy. It's too early for real results from Zonbu, which is trying to use Linux-based systems for a "computers as a service" business model. But, says Zonbu, Linux is the best platform for companies trying new models. Finally, Xandros also is optimistic, especially about "new form factors" for the desktop, a place where Microsoft, they say, "stumbled."
The panel was asked what the development community can do to help these desktop businesses; in response, Arjan van de Ven piped up from the audience, asking what the companies are doing for the kernel community. From Lenovo, the word is that developers can work to get drivers into enterprise distributions as soon as possible. That request, of course, gets back to the tension between enterprise distributions and the desire for current code; this subject was not pursued further here, though. Dell would like to see more collaboration with other vendors in the production of drivers. The Via representative came straight out and said that "we don't do much" to support the community, but insisted that their intentions are good. He said that community support is hard for a Taiwanese company to do, but didn't say why. Via does plan to open a community site at linux.via.com.tw with driver code and more, but this site is not yet in place.
[PULL QUOTE: There would appear to be some tension between providing a truly open device and keeping support costs down. END QUOTE] Support of users came up briefly. The HP representative said that the company expects distributors to provide backup support, but the first call will always go to the vendor of the hardware. That can be a problem, especially for the small devices which are seeing so much success at the moment; a single support call can wipe out any profit on the sale of one of those systems. Selling "constrained systems" which only do a few things helps; but, earlier, Mr. Mann had also talked about the difficulty of installing additional applications on these systems. There would appear to be some tension between providing a truly open device and keeping support costs down. The word from Asus is that a system like the Eee PC generates a lot of relatively trivial calls - things like "how do I search on the web?" So there is a real need to train users which has little to do with Linux itself.
On the subject of applications, the gOS representative discussed a strategy of putting as much as possible on the web. The problem with local applications which look like Microsoft products is that users then expect those applications to behave like Microsoft products. It is better to have something which is obviously different and, presumably, better. Xandros called for better style guides and consistency throughout the interface; clones of other products are not what the market needs. On the HP side, the biggest request was "don't make people open a terminal."
Perhaps the most amusing comment came from the Via representative, who described a "Maddog/Shuttleworth" choice. He asserted that his grandparents would find Jon "maddog" Hall (who was in the audience) to be a rather scary presence, while Mark Shuttleworth comes across as a friendly gentleman. Our interfaces, he says, need to look more like Mark Shuttleworth. Your editor, who has always found Maddog to be one of the friendliest people he knows, does not entirely buy into this analogy. But perhaps there is something to be said for clean-shaven interfaces.
There was some talk of asking suppliers to provide hardware which is supported by free software. Perhaps the most telling comment came from Lenovo, which, apparently, has been asking for Linux-supported hardware "for a number of years." Free drivers are not a priority, though; the first priority is just having things work. So there is still some work to be done in this direction.
Arguably the most interesting theme which came from this discussion - and from the first day of the summit as a whole - is that nobody is really pushing all that hard to get Linux into traditional desktop settings. The real action at the moment would appear to be in small devices like the Eee PC. These "greenfield" areas where there is no established presence to compete against offer vendors a market where they are not trying to migrate users away from other products. They would appear to be convinced that Linux can be a strong contender there - maybe the strongest. So soon we may truly see the year of the Linux desktop - for specific types of "desktop."
Video forums for free software
Over the last few years, we have seen the rise of video content on the web, but much of that content has been locked up in non-free formats. Patented video codecs are a big part of the problem, though there are free alternatives (Theora and Dirac for example), they are not widely used. Free software projects often use videos as part of their marketing and documentation, using screencasts to highlight interesting or exciting features of the program for example. But the choices for collecting and distributing video content leave much to be desired for free software advocates.
The Fedora project has been looking into this problem lately, in support of its FedoraTV project. A recent thread on the fedora-advisory-board mailing list looks at various alternatives now that the original host of FedoraTV content, luluTV, has gone out of business. Greg DeKoenigsberg outlines the problem:
The question that follows: "we've got lots of people who are excited about making Fedora videos. What's the best way, in the short term, to gather those videos together to make them accessible?"
He goes on to outline the criteria for finding a near-term solution, starting with the absolute requirements: Ogg Theora format, one-click download, and a robust, stable hosting site. Also important, but not as critical are things like the ability to extract static screenshots for posting in various places, an easy way for community members to know when new videos are available (an RSS feed for example), and a way for uploaders to easily associate a license with their video. These should resonate with most projects that have an interest in providing a video forum for their community as they are likely to have many of the same needs.
Transcoding the videos to Flash to reach the largest possible audience is DeKoenigsberg's "controversial" criteria. It is an unfortunate truth that, even for fairly strong free software proponents, the Flash browser plugin provides the simplest route to viewing online videos. Other solutions exist and work, but require a great deal more effort to enable additional software repositories so that the proprietary or patented codecs can be installed. Interestingly, there were no arguments presented against the transcoding suggestion.
For Fedora, where Theora—or other free codec—viewers are easily available, Flash transcoding might be less of a requirement. Other projects, especially those that are cross-platform, may find that a large part of their community is either unable or unwilling to install additional software to view videos. Users of non-free operating systems are largely unaware of the video codec problems; their OS comes with a no-extra-cost video viewer that just works. Because of that, transcoding to Flash does at least provide a way to present videos that can be relatively easily viewed by free and non-free systems alike.
Various solutions to the hosting problem were discussed, from partnering with archive.org to rolling their own using MediaWiki, Plumi, or some of the technology released by luluTV. One of the suggestions that got the most attention was to create a Miro channel hosted, at least temporarily, on Fedora project servers. Miro has a lot of promise as a viewer and organizer of videos, with a BitTorrent client built-in, but it doesn't solve the other half of the problem: how to allow the community to contribute.
There is, it seems, a growing need for a free community video forum, both from a code and a hosting perspective. The bandwidth and storage requirements of video are enormous, so covering the actual cost will be a big challenge. Places like YouTube allow short videos to be uploaded, but they can only be played back via Flash. In addition, their software is not free, so they only solve parts of the problem.
There are no obvious free solutions, yet, but it is a problem that we will be facing more frequently. Somehow leveraging Miro as a free, cross-platform video delivery system may make the most sense. Providing a way for the community to upload video content into the channels would make for a mostly working FedoraTV and other projects like that. Miro supports free codecs as well, which might help to start weaning people away from their current non-free codec addiction. Then we can start figuring out how to pay for the network and hard disk capacity required.
Page editor: Jonathan Corbet
Security
Backscatter increase clogs inboxes
Backscatter, also known as blowback, is the result of a spammer forging the sender address on an email that is sent to a non-existent address. Many mail servers do not reject invalid addresses when they receive the email and instead generate a bounce message sometime later. The unfortunate victim, then, is the one whose address was forged as the sender. Sometimes, hundreds or thousands of bounce messages can be generated which flood the inbox of an innocent bystander.
Backscatter seems to be on the rise recently, the LWN inbox has seen a huge increase in the number of bounces over the last week or so. There may be some connection to some Google domains contributing to the problem, but that cannot explain all of it. One basic problem is that many mail servers are generating the bounce messages after accepting mail for invalid addresses, rather than rejecting it while the SMTP transaction is still in progress.
When a mail server gets a connection from a sending machine, it gets several pieces of information about the email in addition to its contents. Both a "from" and "to" address are included in this extra information, which is usually called the envelope, for obvious reasons. After receiving each piece of the envelope, a mail server has the opportunity to reject the message. Typically this isn't done for valid-looking sender addresses, except in limited blacklist situations, but it certainly can and should be done when the recipient address is invalid.
Due to a variety of mail server configuration issues, many mail servers do not avail themselves of rejecting mail for invalid senders. Instead, they defer their decision until sometime later. Servers that relay mail will not know whether some of the addresses they relay are valid, while other servers (qmail for example) separate the SMTP conversation program from the local delivery program for security reasons and thus do not have that information available. Other valid or semi-valid reasons exist, but once the mail has been accepted, the proper means of indicating a bad address is no longer available.
In the days before spam—remember those?—a mail server could generally trust that the sender address in the envelope was the real sender. So an incorrectly addressed email could be bundled up in a bounce message and sent to the sender. If the sender address is valid, it is very little different than a bounce that is generated by the sender's machine when the mail gets rejected at SMTP time. Unfortunately, the majority of sender addresses these days are forged.
But spammers don't want to use just any forged address, they want to use something that is valid or appears valid. Mail servers have gotten better at testing sender addresses for validity before accepting mail from them. So, where does an enterprising spammer get a valid email address? They pick one at random from their list of "500,000 guaranteed opt-in email addresses" that they bought from some other miscreant. They use those lists to send their spam to as well as using them to choose sender addresses to use.
As might be guessed, the SpamAssassin mailing lists have been discussing the problem recently, especially trying to find ways to reduce the amount received. SpamAssassin does have the VBounce plugin to recognize bounce messages. By default, it doesn't increase the score of bounces by much as it is meant to be used with procmail to put bounces in a separate place from spam.
Another idea floated on the list is to use SPF or DKIM records for a domain. The belief is that spammers avoid using those domains because it is likely to cause their message to be immediately classified as spam. Anecdotal evidence seems to indicate that backscatter can be significantly reduced in this way.
Security news
Bruce Schneier reviews Access Denied
Bruce Schneier takes a look at Access Denied, a new book on internet censorship from the MIT Press. "Today, things are very different. Internet censorship is flourishing. Organizations selectively block employees' access to the Internet. At least 26 countries -- mainly in the Middle East, North Africa, Asia, the Pacific and the former Soviet Union -- selectively block their citizens' Internet access. Even more countries legislate to control what can and cannot be said, downloaded or linked to. 'You have no sovereignty where we gather,' said Barlow. Oh yes we do, the governments of the world have replied."
New Massive Botnet Twice the Size of Storm (Dark Reading)
Dark Reading reports from the RSA conference on an enormous botnet that is currently active with roughly 400,000 bots. "The so-called Kraken botnet has been spotted in at least 50 Fortune 500 companies and is undetectable in over 80 percent of machines running antivirus software. Kraken appears to be evading detection by a combination of clever obfuscation techniques, including regularly updating its binary code and structuring the code in such a way that hinders any static analysis, says Paul Royal, principal researcher at Damballa."
OpenPacket.org 1.0 Is Live
From Richard Bejtlich's weblog comes the news that OpenPacket.org is open for business. "The mission of OpenPacket.org is to provide quality network traffic traces to researchers, analysts, and other members of the digital security community. One of the most difficult problems facing researchers, analysts, and others is understanding traffic carried by networks. At present there is no central repository of traces from which a student of network traffic could draw samples. OpenPacket.org will provide one possible solution to this problem."
Security reports
Freezing More Than Bits: Chilling Effects of the OLPC XO Security Model
A paper that will be presented at the USENIX Usability, Psychology and Security (UPSEC) conference takes a look at the OLPC Bitfrost security model [PDF]. "In this paper, we discuss Bitfrost, the security model developed by the One Laptop Per Child project for its XO laptop computers. Bitfrost implements a number of security measures intended primarily to deter theft and malware, but which also introduce severe threats to data security and individual privacy. We describe several of the technical provisions in Bitfrost, outline the risks they enable, and consider their legal ramifications and the psychological impact posed for children and society."
New vulnerabilities
alsaplayer: arbitrary code execution
| Package(s): | alsaplayer | CVE #(s): | CVE-2007-5301 | |||
| Created: | April 7, 2008 | Updated: | April 9, 2008 | |||
| Description: | From the Debian advisory: Erik Sjölund discovered a buffer overflow vulnerability in the Ogg Vorbis input plugin of the alsaplayer audio playback application. Successful exploitation of this vulnerability through the opening of a maliciously-crafted Vorbis file could lead to the execution of arbitrary code. | |||||
| Alerts: |
| |||||
audit: privilege escalation
| Package(s): | audit | CVE #(s): | CVE-2008-1628 | |||||||||
| Created: | April 9, 2008 | Updated: | April 25, 2008 | |||||||||
| Description: | From the Red Hat bugzilla entry: A vulnerability has been reported in Linux Audit, which potentially can be exploited by malicious, local users to gain escalated privileges. The vulnerability is caused due to a boundary error within the "audit_log_user_command()" function in lib/audit_logging.c. This can be exploited to cause a stack-based buffer overflow via an overly long "command" argument and potentially execute arbitrary code with the privileges of the application using libaudit. | |||||||||||
| Alerts: |
| |||||||||||
comix: arbitrary code execution
| Package(s): | comix | CVE #(s): | CVE-2008-1568 | |||||||||
| Created: | April 9, 2008 | Updated: | April 28, 2008 | |||||||||
| Description: | From the NVD entry: comix 3.6.4 allows attackers to execute arbitrary commands via a filename containing shell metacharacters that are not properly sanitized when executing the rar, unrar, or jpegtran programs | |||||||||||
| Alerts: |
| |||||||||||
flash-plugin: multiple vulnerabilities
| Package(s): | flash-plugin | CVE #(s): | CVE-2007-6637 CVE-2007-6019 CVE-2007-0071 CVE-2008-1655 CVE-2008-1654 | |||||||||
| Created: | April 9, 2008 | Updated: | April 18, 2008 | |||||||||
| Description: | From the Red Hat advisory: Several input validation flaws were found in the way Flash Player displayed certain content. These may have made it possible to execute arbitrary code on a victim's machine, if the victim opened a malicious Adobe Flash file. (CVE-2007-0071, CVE-2007-6019) A flaw was found in the way Flash Player established TCP sessions to remote hosts. A remote attacker could, consequently, use Flash Player to conduct a DNS rebinding attack. (CVE-2007-5275, CVE-2008-1655) A flaw was found in the way Flash Player restricted the interpretation and usage of cross-domain policy files. A remote attacker could use Flash Player to conduct cross-domain and cross-site scripting attacks. (CVE-2007-6243, CVE-2008-1654) A flaw was found in the way Flash Player interacted with web browsers. An attacker could use malicious content presented by Flash Player to conduct a cross-site scripting attack. (CVE-2007-6637) | |||||||||||
| Alerts: |
| |||||||||||
gnome-ssh-askpass, openssh: privilege escalation
| Package(s): | gnome-ssh-askpass | CVE #(s): | CVE-2008-1657 | ||||||||||||
| Created: | April 7, 2008 | Updated: | May 7, 2008 | ||||||||||||
| Description: | From the Gentoo advisory: OpenSSH will execute the contents of the ".ssh/rc" file even when the "ForceCommand" directive is enabled in the global sshd_config (CVE-2008-1657). | ||||||||||||||
| Alerts: |
| ||||||||||||||
konversation: arbitrary code execution
| Package(s): | konversation | CVE #(s): | CVE-2007-4400 | ||||||
| Created: | April 9, 2008 | Updated: | April 9, 2008 | ||||||
| Description: | From the Red Hat bugzilla: Media script (/usr/share/apps/konversation/scripts/media) that is distributed with konversation package reportedly does not escape tags from media files corr[e]ctly allowing command injection into IRC channel. | ||||||||
| Alerts: |
| ||||||||
m4: execution of arbitrary code
| Package(s): | m4 | CVE #(s): | CVE-2008-1687 CVE-2008-1688 | |||
| Created: | April 8, 2008 | Updated: | April 9, 2008 | |||
| Description: | m4-1.4.11 fixes two issues with possible security implications. A minor security fix with the use of "maketemp" and "mkstemp" -- these are now quoted to prevent the (rather unlikely) possibility that an unquoted string could match an existing macro causing operations to be done on the wrong file. Also, a problem with the '-F' option (introduced with version 1.4) could cause a core dump or possibly (with certain file names) the execution of arbitrary code. | |||||
| Alerts: |
| |||||
nx nxnode: multiple vulnerabilities
| Package(s): | nx nxnode | CVE #(s): | ||||
| Created: | April 7, 2008 | Updated: | April 9, 2008 | |||
| Description: | From the Gentoo advisory: Multiple integer overflow and buffer overflow vulnerabilities have been discovered in the X.Org X server as shipped by NX and NX Node (vulnerabilities 1-4 in GLSA 200801-09). A remote attacker could exploit these vulnerabilities via unspecified vectors, leading to the execution of arbitrary code with the privileges of the user on the machine running the NX server. | |||||
| Alerts: |
| |||||
otrs: SOAP command execution
| Package(s): | otrs | CVE #(s): | CVE-2008-1515 | ||||||
| Created: | April 4, 2008 | Updated: | April 17, 2008 | ||||||
| Description: | A bug in the trouble ticket system OTRS allowed a remote attacker to get remote access without specifying a valid user name via the SOAP interface. | ||||||||
| Alerts: |
| ||||||||
pdns-recursor: DNS cache poisoning
| Package(s): | pdns-recursor | CVE #(s): | CVE-2008-1637 | ||||||||||||
| Created: | April 9, 2008 | Updated: | April 18, 2008 | ||||||||||||
| Description: | From the Red Hat bugzilla entry: Amit Klein of Trusteer discovered and documented weakness in a way PowerDNS Recursor generates DNS queries and transaction IDs used in DNS queries. This weakness can be used to predict transaction IDs used in a subsequent queries after observing certain amount of consequent previous queries, leading to a high possibility of performing a successful cache poisoning attack. | ||||||||||||||
| Alerts: |
| ||||||||||||||
pecl-apc: arbitrary code execution
| Package(s): | pecl-apc | CVE #(s): | CVE-2008-1488 | ||||||
| Created: | April 9, 2008 | Updated: | April 10, 2008 | ||||||
| Description: | From the Gentoo advisory: Daniel Papasian discovered a stack-based buffer overflow in the apc_search_paths() function in the file apc.c when processing long filenames. A remote attacker could exploit this vulnerability to execute arbitrary code in PHP applications that pass user-controlled input to the include() function. | ||||||||
| Alerts: |
| ||||||||
PolicyKit: authentication bypass
| Package(s): | PolicyKit | CVE #(s): | CVE-2008-1658 | ||||||
| Created: | April 9, 2008 | Updated: | April 17, 2008 | ||||||
| Description: | From the Red Hat bugzilla entry: Format string vulnerability was discovered in the PolicyKit grant helper. User may specify password with formatting sequences and cause polkit-grant-helper to crash or bypass authentication. | ||||||||
| Alerts: |
| ||||||||
silc-toolkit: buffer overflow
| Package(s): | silc-toolkit | CVE #(s): | CVE-2008-1552 | ||||||
| Created: | April 4, 2008 | Updated: | April 24, 2008 | ||||||
| Description: | A flaw in processing PKCS#1-Messages in silc-toolkit could lead to a buffer overflow. Remote attackers could exploit that to crash the server. | ||||||||
| Alerts: |
| ||||||||
wireshark: multiple vulnerabilities
| Package(s): | wireshark | CVE #(s): | CVE-2008-1561 CVE-2008-1562 CVE-2008-1563 | ||||||||||||||||||
| Created: | April 4, 2008 | Updated: | May 8, 2008 | ||||||||||||||||||
| Description: | Multiple flaws in wireshark could lead to crashes when certain packets are processed. | ||||||||||||||||||||
| Alerts: |
| ||||||||||||||||||||
Page editor: Jake Edge
Kernel development
Release status
Kernel release status
The current 2.6 development kernel remains 2.6.25-rc8; there have been no kernel releases over the last week. The patch rate into the mainline repository has slowed considerably, but the current regression list suggests that the 2.6.25 release is not imminent quite yet.
Kernel development news
Quotes of the week
A Linux Driver Project status report
Greg Kroah-Hartman has sent out a lengthy report on the state of the Linux Driver Project. "The main problem is a lack of projects. It turns out that there really isn't much hardware that Linux doesn't already support. Almost all new hardware produced is coming with a Linux driver already written by the company, or by the community with help from the company."
Memory allocation failures and scary warnings
People who put their Linux systems under a certain amount of memory stress - and who look at their logfiles - may notice an occasional message indicating that a "page allocation failure" has occurred, followed by a scary backtrace. These people may also notice that, despite the apocalyptic appearance of this message, the world often fails to end. In fact, the system tends to carry on just fine. For this reason, Dave Jones, who probably gets ten emails for every backtrace generated on a Fedora system, has suggested that these messages are simply noise which should be removed. Whether that should really happen is not entirely clear, though; understanding why requires a bit of background.In general, the kernel's memory allocator does not like to fail. So, when kernel code requests memory, the memory management code will work hard to satisfy the request. If this work involves pushing other pages out to swap or removing data from the page cache, so be it. A big exception happens, though, when an atomic allocation (using the GFP_ATOMIC flag) is requested. Code requesting atomic allocations is generally not in a position where it can wait around for a lot of memory housecleaning work; in particular, such code cannot sleep. So if the memory manager is unable to satisfy an atomic allocation with the memory it has in hand, it has no choice except to fail the request.
Such failures are quite rare, especially when single pages are requested. The kernel works to keep some spare pages around at all times, so the memory stress must be severe before a single-page allocation will fail. Multi-page allocations are harder, though; the kernel's memory management code tends to fragment pages, making groups of physically-contiguous pages hard to find. In particular, if the system is under pressure to the point that there is not much free memory available at all, the chances of successfully allocating two (or more) contiguous pages drops considerably.
Multi-page allocations are not often used in the kernel; they are avoided whenever possible. There are situations where they are necessary, though. One example is network drivers which (1) support the transmission and reception of packets too large to fit into a single page, and which (2) drive hardware which cannot perform scatter/gather I/O on a single packet. In this situation, the DMA buffers used for packets must be larger than one page, and they must be physically contiguous. This is a situation which will become less pressing over time; scatter/gather capability in the hardware is increasingly common, and drivers are being rewritten to make use of this capability. With sufficiently smart hardware, the need for multi-page allocations goes down considerably.
But all of that skirts around the main point, which is that kernel code is supposed to handle allocation failures properly. There is never any guarantee that memory will be available, so kernel code must be written defensively. Allocation failures must be handled without losing any more capability than is strictly necessary. If one assumes that kernel code is written correctly, there should be no need to issue warnings on allocation failures. Things should just continue to work, perhaps without users noticing at all.
And, in fact, things often do just work. But the discussion resulting from Dave's suggestion makes it clear that few developers are confident that all kernel code does the right thing in the face of memory allocation problems. In cases where an allocation failure is not dealt with correctly, the system may go down in random places, leaving few clues as to what really happened. In that kind of situation, the allocation failure warning may be the only useful information which survives the crash. For this reason, some people want to see the warnings left in place.
As it happens, the memory allocator supports a special bit (__GFP_NOWARN) which causes the warning not to be emitted if a specific allocation fails. So it has been suggested that the allocations made from code which is known to handle failures properly have __GFP_NOWARN set. That would kill the warnings in code known to do the right thing while leaving it for all other callers, presumably limiting the warnings to places where there might truly be a problem. Jeff Garzik strongly opposed this idea, though, saying that it clutters up the code and "punishes good behavior."
The other reason given for keeping the warnings in place is to make it clear when a system is running under persistent memory pressure. Such systems will not be performing optimally; often there are changes which can be made to relieve the pressure and help the system to run more smoothly. So it has been suggested that the warning could be reduced in frequency and made less scary. Nick Piggin suggests:
An alternative idea would be to keep some sort of counter somewhere which could be queried by curious system administrators.
Of course, the real solution is to ensure that all kernel code is robust in the face of allocation failures. This can be hard to do, since the error recovery paths in any code are not often exercised or tested. Fortunately, the fault injection framework can help in this situation. Kernel developers can use this framework to simulate allocation failures in specific regions of code, then watch to see what happens. Your editor's impression, though, is that relatively few developers are using this tool. So confidence in the kernel's handling of allocation failures may remain low, and the desire to keep the warning around may remain high.
Improving syncookies
Back in 1997 TCP SYN flood attacks were all the rage among script kiddies. A SYN flood is a denial of service attack that uses up server resources by initiating, but not completing, a connection. Attacks via this method still remain a problem today though they are now more likely to be launched by sophisticated botnets rather than an individual. A first line defense against SYN floods is the syncookie. The syncookie was not designed for Linux specifically but found its way into kernel 2.1.44 via a patch from Andi Kleen.
This long-time feature generated some recent discussion when a patch was submitted adding syncookie support to IPv6. The patch has now been queued for acceptance but in discussion along the way the community also began to tackle some longstanding limitations of syncookies and reaffirmed how relevant the feature continues to be.
To fully describe syncookies some background on how TCP uses a three way handshake to establish a connection is in order. The first packet of any TCP session received by the server is known as the SYN packet because it carries the synchronize control flag. The SYN flag indicates that its sender wishes to open a new connection. That flag is only used during the opening sequence. The server responds with a packet also containing the SYN flag because the connection needs to be opened in both directions. This second packet also carries the ACK flag and is known as the SYN-ACK. It serves to both open the connection from the server to the client and to acknowledge receipt of the opening packet from the other host. Finally, the client sends a bare ACK packet to the server to acknowledge receipt of server-to-client SYN-ACK and the connection is then fully established.
During a SYN flood a server receives the first packet of the three-way TCP handshake and responds with a SYN-ACK but no further data is ever received from the initiating client. When the SYN-ACK is generated most servers will also create an entry in the SYN queue. This queue is the waiting area for half-open connections awaiting handshake completion. The attacker intentionally orphans those entries and instead generates more SYN packets which in turn take up more entries in the queue. The server needs to wait for a long timeout before giving up and recovering the connection resources. During this time the attacker can flood it with many more half-open connections. Eventually the server runs out of resources and cannot accept any new connections without dropping some, perhaps legitimate, connection from the queue. Simple solutions such as placing a quota on the number of partially open connections per peer or using dynamically adjusted packet filters do not work because the SYN packets are easy to forge with fake source addresses.
A syncookie allows the server to defer using up any resources until the third packet in the three-way handshake has been received. At that time the peer's address has been mildly authenticated because the final packet in the handshake contains a reference to the sequence number that was sent by the server in the second packet. With this assurance, packet filters and resource quotas keyed to the peer's address will again be useful defenses against resource attacks.
The basic mechanism of the syncookie works by carefully manipulating the initial sequence number value of the connection instead of choosing it at random. Upon receiving a SYN the server carefully encodes the vital information that would have been stored as state in the SYN queue. This encoded information is cryptographically hashed with a secret key to form the sequence number of the SYN-ACK and sent to the client. The third packet of a legitimate handshake, which is the ACK from the client back to the server, contains this sequence number (plus one) in its acknowledgment number field. In this way all the information necessary to fully open the connection is presented back to the server without having to maintain state while the handshake is being completed.
The major downside to syncookies is that they only have space to encode the most basic of TCP handshake options. At the time of initial syncookie deployment this was not a large problem because the only option prominently in use at the time was the Maximum Segment Size (MSS) option. This option is provided to help the peer avoid unnecessary fragmentation by sending packets that the other end of the connection knows a priori are too large to cross its network. This is exactly the kind of information that is normally stored as state in the SYN queue. The syncookie designers knew that this option was important to performance and found 3 bits for it in the encoded syncookie. These bits are used to approximate the real value of the option to one of 8 common values.
In the intervening years new options have come into prominence and these are not syncookie compatible. The most important of these are the window scaling and Selective Acknowledgment (SACK) options. These features respectively allow the TCP congestion control window to grow beyond 64KB and be more efficient in the case of minor packet losses from those large windows. Without using these features it is impossible to get good transfer rates on networks with large bandwidth or large latency. Many household broadband links require at least the window scaling option to fully utilize the network connection. Due to this limitation, and the modest computation overhead of the cryptographic hash, the Linux stack only resorts to syncookie based connections when the number of half-open connection exceeds a high watermark controlled by the net.ipv4.tcp_max_syn_backlog sysctl. These connections are less featureful than normal connections but they are only resorted to when the queue would otherwise require active pruning.
It turns out that the cookie mechanism is only implemented for IPv4. Recently, Dave Griffin posted patches that add IPv6 support for syncookies. Andi Kleen, author of the original syncookie patch, wondered if the mechanism should be continued at all much less added to IPv6:
Andi's argument was three pronged. His first point was about the reduced abilities of cookie initiated connections as already described in this article. Over time the value of these options has increased and therefore the cost of using syncookies has increased too. His second point was that Linux no longer uses all of the memory necessary for a full connection until the new connection is fully open. Instead it uses a "minisock" for that period. The minisock is a 96 byte struct tcp_request_sock structure holding the minimum state necessary to get the connection fully opened. The fully established struct tcp_sock is 1616 bytes. Both structure size measurements refer to a 64-bit kernel. Finally, Andi points out that the queue management routines for an overloaded SYN queue are more sophisticated now than the dumb head drop algorithm that was in place when syncookies were first deployed. The suggestion was that in aggregate these advances might make Linux robust enough without syncookies so that they could therefore be removed all together.
Instead of engaging in a theoretical discussion some readers set up and ran their own experiments. One of the best parts of the Linux community is the tendency to put real data behind their arguments. While there is often disagreement over the realism of the measured scenarios, the data points always help us better understand the dynamics of kernel code.
This data compellingly supports the continued value of the syncookie and that position seems to have won the day. The IPv6 syncookie patches are now queued within the network 2.6.26 development tree.
However, the biggest news is probably that this discussion brought renewed energy to the problem of lost handshake options. Florian Westphal and Glen Griffin have recently presented a solution to the most damaging aspect of that problem too.
Their solution is to leverage the echoed TCP timestamp option in a way similar to the way classic syncookies leverage the echoing of the SYN-ACK sequence number in the subsequent ACK. The timestamp option was introduced with RFC 1323 and is widely deployed on modern Linux, Windows, and FreeBSD (including OS X) systems. Its main purpose is to be able to increase the frequency of round trip time measurements in the presence of large congestion control windows.
Using the timestamp to preserve the window scale and SACK option values requires modifying the timestamp of the SYN-ACK packet to include the state necessary to support them. During a normal handshake the client will echo the modified timestamp value of the SYN-ACK packet back to the server as part of the timestamp option on the third part of the handshake and thus propagate the SACK and window scale information without keeping any state on the server.
In order to make room in the timestamp for this new information the least significant 9 bits of the timestamp are shaved off. The encoded representation of the window scale and SACK options are then transferred back and forth at the minor cost of reduced granularity of TCP timestamps during the handshake exchange. Timestamps lose their least significant 512 jiffies with this approach.
Below are two different TCP handshakes completed with syncookies and the timestamp patch. Note that the lowest bits of the SYN-ACK timestamp are the same in each handshake even at different points in time because each handshake uses the same SACK and window scaling options. As a result the timestamp values in each SYN-ACK are different but the lower nine bits share the same 0x166 value.
13:51:04.582464 IP 127.0.0.1.57985 > 127.0.0.1.4050: S 1061746051:1061746051(0)
win 32792 <mss 16396,sackOK,timestamp 0xfffea013 0,nop,wscale 6>
13:51:04.582478 IP 127.0.0.1.4050 > 127.0.0.1.57985: S 2800702917:2800702917(0)
ack 1061746052 win 32768 <mss 16396,sackOK,timestamp 0xfffe9f66 0xfffea013,nop,wscale 6>
13:51:04.582480 IP 127.0.0.1.57985 > 127.0.0.1.4050: .
ack 1 win 513 <nop,nop,timestamp 0xfffea013 0xfffe9466>
13:59:19.047306 IP 127.0.0.1.45979 > 127.0.0.1.4050: S 218483035:218483035(0)
win 32792 <mss 16396,sackOK,timestamp 0x0001bed4 0,nop,wscale 6>
13:59:19.047320 IP 127.0.0.1.4050 > 127.0.0.1.45979: S 1141094138:1141094138(0)
ack 218483036 win 32768 <mss 16396,sackOK,timestamp 0x0001bd66 0x0001bed4,nop,wscale 6>
13:59:19.047322 IP 127.0.0.1.45979 > 127.0.0.1.4050: .
ack 1 win 513 <nop,nop,timestamp 0x0001bed4 0x0001bd66>
While there is no guarantee that the timestamp option will be supported by every TCP peer, timestamps are widely deployed on the most common operating systems. Additionally, because timestamps, window scaling, and selective acknowledgments are all features related to high latency and bandwidth networks it would be unlikely to find an implementation that supported only a subset of these options.
One shortcoming of the scheme is that it is not general enough to be future-proof as new handshake based options may continue to be deployed. At this time the MSS, SACK, window scaling, and timestamp options are the only handshake options seen with any regularity other than the NOP option which is just used for packet alignment. However, the whole point of an extensible option scheme is to leave room for future improvements. The IANA registry that records option values was last updated in February 2007 to reserve option code 27 for use with Experimental RFC 4782 "Quick Start for TCP and IP". Only time will tell if that particular option will be the next challenge to the syncookie scheme or if something else will rise first.
The timestamp patch has only been posted very recently, and there has been little discussion of it beyond the developers who worked directly on it. It is not clear whether or not it will be accepted right away into the mainline, but it certainly seems to address a well known core problem with the syncookie at a minor cost.
With the updates for IPv6 and modern TCP option schemes syncookies appear primed to keep providing sweet relief in their somewhat esoteric networking security niche. Perhaps they will keep chugging away for another 10 years without having to be re-baked.
vringfd()
One of the core features of the (now stalled) kevent subsystem was a circular buffer intended for efficient movement of data between the kernel and user space. Kevent may have run out of steam, but the ring buffer idea is back via a different path. Rusty Russell is now proposing a new system call (called vringfd()) which turns some of the virtio work into a new kernel-to-user ring buffer interface. The submitted patch is breathtaking in its lack of documentation on this new system call, especially considering that its author is quite good with that sort of writing. Your editor has taken this omission as a personal challenge and, as a result, has set about reverse engineering the (somewhat complex) vringfd() interface.A user-space process which wishes to set up a vring for communication with the kernel must create a slightly complicated data structure first. One starts by deciding how many entries the ring should have; this number must be a power of two which fits into an unsigned, 16-bit value. Given this number (we'll call it RING_SIZE), the data structure looks like this:
struct messy_vring_thing {
struct vring_desc descriptors[RING_SIZE];
struct vring_avail available;
char padding[up-to-next-page-boundary];
struct vring_used used[RING_SIZE];
};
The page alignment for the used array is important - that array might be mapped separately into kernel space. The array must fit into a single page, which puts a practical limit of 256 entries for RING_SIZE on systems with 4096-byte pages. If this API goes forward, chances are good that a way will be found to raise this limit.
Individual descriptors in the ring are described with this structure:
struct vring_desc
{
__u64 addr; /* Address of the buffer */
__u32 len; /* Length of the buffer */
__u16 flags; /* some flags */
__u16 next; /* Next buffer in the chain */
};
For a simple buffer, the application would simply point addr at the beginning and set len to the appropriate value. If the buffer is to be written to by the kernel, the application should also set VRING_DESC_F_WRITE in the flags field.
Things can get more complicated than that, though, in that the vringfd() interface supports multipart scatter/gather buffers. To set up such a buffer, user space would use one vring_desc entry for each segment of the buffer. For all but the final segment, the VRING_DESC_F_NEXT flag (saying "use the next descriptor too") should be set, and next should be the index of the next descriptor. When the kernel grabs a buffer, it will follow the chain and use all segments found until the final one (which lacks the VRING_DESC_F_NEXT flag) is encountered.
Before the kernel will use buffers set up by the application, though, user space must indicate that the buffer is ready. That is done through the vring_avail structure:
struct vring_avail
{
__u16 flags;
__u16 idx;
__u16 ring[RING_SIZE];
};
The ring array holds indexes into the descriptors array. The idx field should always be the index of the last valid entry in ring. When a new buffer is ready for transfer to or from the kernel, the application will store the index of the first descriptor into ring[idx+1], then increment idx. When the ring is first established, the kernel remembers the position of idx, so the first buffer should be added here after the vringfd() system call is made.
The kernel will consume buffers from the available ring as needed. Once the requested operation has been performed on the buffer and the kernel is done with it, the buffer will show up in the used area, which is structured this way:
struct vring_used_elem
{
__u32 id;
__u32 len;
};
struct vring_used
{
__u16 flags;
__u16 idx;
struct vring_used_elem ring[RING_SIZE];
};
In the vring_used structure, idx is the index of the next entry in ring which may be written by the kernel; it will be incremented after the ring is updated. When a buffer is placed in the used ring, the id field will be the index of the descriptor, and len will be the actual length of the data transferred.
Note that the flags fields in the vring_avail and vring_used structures appear to be unused.
Once the application has this whole data structure set up, it can establish the ring buffer with the kernel with the new system call:
long vringfd(void *addr, unsigned int ring_size, u16 *last_used);
Here, addr is the base address of the data structure described above, ring_size is the number of descriptors in the ring, and last_used is a 16-bit unsigned integer indicating which entry in the used ring was last consumed by the application. Failure to keep last_used current will not slow things down, but it will keep poll() from working properly.
The return value will be a file descriptor associated with the ring.
Creating the vring is only part of the job, though. The next step is to connect it with a kernel subsystem for the transfer of data. Rusty's patch includes vring support in the tun virtual network driver; to use that support, an application makes a special ioctl() call to provide the vring file descriptor to the tun driver. Any other subsystem will need a similar mechanism to support vring.
If the application is using the ring to transfer data into the kernel, it must (1) set up one or more descriptors for full data buffers in the available ring, then (2) make a write() call to the vring file descriptor. The buffer and length passed to write() are ignored; all that matters is that a write was done to that file descriptor. When write() returns the operation will have been set in motion, but it cannot be considered to be complete until the ring descriptors show up in the used ring.
For data transfers from the kernel to user space, the application simply puts buffers into the available ring, then waits until they show up in the used ring. A poll() on the vring file descriptor will block until buffers are available. The kernel determines whether unconsumed buffers exist in used by comparing the vring_used->idx index against the application-supplied last_used value. It's worth noting that, depending on how the relevant kernel subsystem works, buffers may not actually make it into the used ring until the poll() call is made.
On the kernel side, a developer wanting to add vring support to a subsystem will start by creating a set of vring_ops:
struct vring_ops
{
void (*destroy)(void *);
int (*pull)(void *);
int (*push)(void *);
};
All of these functions take a private pointer given when the subsystem attaches to the vring (to be described shortly). The pull() callback is invoked when the application calls poll(); if there is any descriptor processing which must be done with user space accessible, this is the place to do it. If pull() adds any buffers to the used ring, it should return the number of buffers; it can also return a negative error code. push() is called from a write() call indicating that there are buffers ready to be transferred into the kernel; it returns zero or a negative error code. The destroy() callback is called when the vring file descriptor is closed. All of these callbacks are optional.
Attaching to a vring is done with:
struct vring_info *vring_attach(int fd, const struct vring_ops *ops,
void *data, bool atomic_use);
For this call, fd is a file descriptor corresponding to a vring, ops is the operations structure described above, data is a private data pointer which is passed into the vring_ops callbacks, and atomic_use is nonzero if the kernel needs to be able to add buffers to the used ring in atomic context. The return value is a pointer to an internal vring data structure or an ERR_PTR() value if something goes wrong.
To obtain a buffer from the available ring, a call is made to:
int vring_get_buffer(struct vring_info *vr,
struct iovec *in_iov,
unsigned int *num_in, unsigned long *in_len,
struct iovec *out_iov,
unsigned int *num_out, unsigned long *out_len);
This function will fill in an array of iovec structures corresponding to the next available buffer. If the kernel expects to write to the buffer, it should set in_iov to the iovec array, num_in pointing to the length of in_iov, and in_len pointing to a location to store the total length of the buffer (or NULL if that information is not useful). For transfers into the kernel, out_iov, num_out, and out_len should be set similarly. Note that the addresses stored in the iovec arrays are user-space addresses; vring_get_buffer() does not validate them, so the caller must do so.
It is possible to set pass both in_iov and out_iov; in this case, one of the two will be set, depending on whether the next buffer in the available ring has the VRING_DESC_F_WRITE flag set. In most cases, though, only one of the two sets of parameters will have non-NULL values. The apparent intent of the API is that, if bidirectional transfers between user space and the kernel are needed, two separate vrings should be used.
The return value from vring_get_buffer will be one of (1) a positive descriptor index, (2) zero, indicating that no buffers are available, or (3) a negative error code.
The descriptor index should be saved the the final step, which is indicating that the kernel is done with a specific buffer:
void vring_used_buffer(struct vring_info *vr, int id, u32 len);
void vring_used_buffer_atomic(struct vring_info *vr, int id, u32 len);
Either one of these functions indicates that the buffer indicated by id should be put into the used ring; len is the amount of data actually transferred. If sleeping is not possible, vring_used_buffer_atomic() should be used - but the vring must have been attached with the atomic_use flag set.
There does not appear to be a way for a subsystem to detach from a vring; it must, instead, wait for the application to close the associated file descriptor.
This interface is in an early stage, and the code has a number of limitations and FIXME comments. So things seem likely to evolve before vringfd() is seriously considered for merging into the mainline kernel. The idea of a ring buffer for this kind of communication seems to come around on a regular basis, though, so it would seem that there is a demand for this kind of API.
Patches and updates
Kernel trees
- Oliver Pinter: linux v2.6.22.21-op1. (April 5, 2008)
- Daniel Walker: fully bisectable real time series. (April 9, 2008)
Core kernel code
- Peter Zijlstra: fast_gup for shared futexes. (April 5, 2008)
- Mike Travis: cpumask: reduce stack pressure from local/passed cpumask variables v3. (April 5, 2008)
- Rusty Russell: vringfd syscall. (April 5, 2008)
- Paul E. McKenney: Add call_rcu_sched(). (April 7, 2008)
- David Howells: [PATCH] PRINTK: Decouple input locking from output locking and move render outside lock. (April 9, 2008)
Development tools
- Vegard Nossum: kmemcheck v7. (April 4, 2008)
- Szabolcs Szakacsits: Linux POSIX file system test suite. (April 4, 2008)
Device drivers
- Stefan Richter: What's in linux1394-2.6.git?. (April 4, 2008)
- Dmitry Baryshkov: Clocklib: generic clocks framework. (April 4, 2008)
- Rafael J. Wysocki: PM: New suspend and hibernation callbacks. (April 4, 2008)
- Markus Becker: Update: [RFC][PATCH] mrv8k: Driver for "Marvell 88w8335 [Libertas]". (April 4, 2008)
- Herton Ronaldo Krzesinski: Add Realtek 8187B support. (April 9, 2008)
- Darius: I2C driver for IMX. (April 9, 2008)
Documentation
- Randy Dunlap: documentation: build source files in Documentation sub-dir. (April 4, 2008)
Filesystems and block I/O
- Miklos Szeredi: vfs: add helpers to check r/o bind mounts. (April 4, 2008)
- Mark Fasheh: Ocfs2 updates for 2.6.26-rc1. (April 4, 2008)
- joern@logfs.org: LogFS take five. (April 4, 2008)
- Akira Fujita: ext4 online defrag (ver 0.8). (April 4, 2008)
- Jan Kara: Ordered mode rewrite patch. (April 9, 2008)
Memory management
- Nick Piggin: SLQB: YASA. (April 3, 2008)
- Balbir Singh: Add an owner to the mm_struct (v6). (April 3, 2008)
- Johannes Weiner: Generic show_mem(). (April 4, 2008)
- Andrea Arcangeli: [PATCH 0 of 8] mmu notifiers #v10. (April 4, 2008)
- Andrea Arcangeli: mmu notifier #v11. (April 5, 2008)
- Andrea Arcangeli: [PATCH 0 of 9] mmu notifier #v12. (April 9, 2008)
- Christoph Lameter: [patch 00/10] [RFC] EMM Notifier V3. (April 5, 2008)
- Nitin Gupta: compcache: compressed caching v2. (April 9, 2008)
Networking
- YOSHIFUJI Hideaki / =?iso-2022-jp?B?GyRCNUhGIzFRTEAbKEI=?=: [IPV6] MROUTE: Support multicast routing.. (April 4, 2008)
- YOSHIFUJI Hideaki / =?iso-2022-jp?B?GyRCNUhGIzFRTEAbKEI=?=: [IPV6] SIT: ISATAP Updates.. (April 4, 2008)
- Rusty Russell: net: add destructor for skb data.. (April 5, 2008)
- Juliusz Chroboczek: Stochastic Fair Blue queue discipline. (April 9, 2008)
Architecture-specific
- Huang, Ying: AES: x86_64 asm implementation optimization. (April 9, 2008)
- Markus Metzger: x86, ptrace: PEBS support. (April 9, 2008)
- Johannes Berg: introduce HAVE_EFFICIENT_UNALIGNED_ACCESS Kconfig symbol. (April 9, 2008)
- Glauber Costa: integrate dma_ops. (April 9, 2008)
- Mathieu Desnoyers: Text Edit Lock and Immediate Values for 2.6.25-rc8-mm1. (April 9, 2008)
Security-related
- Tetsuo Handa: TOMOYO Linux 1.6.0 released. (April 4, 2008)
Virtualization and containers
- Satoshi UCHIDA: Yet another I/O bandwidth controlling subsystem for CGroups based on CFQ. (April 4, 2008)
- Matt Helsley: Container Freezer: Reuse Suspend Freezer. (April 4, 2008)
- Avi Kivity: [PATCH 00/35] KVM updates for the 2.6.26 merge window (part II). (April 4, 2008)
- Avi Kivity: kvm-65 release. (April 7, 2008)
- Jeremy Fitzhardinge: [PATCH 0 of 6] [RFC] another attempt at making hotplug memory and xen play together. (April 4, 2008)
- sukadev-r/Jw6+rmf7HQT0dZR+AlfA@public.gmane.org: Clone PTY namespaces. (April 4, 2008)
- sukadev@us.ibm.com: Clone PTS namespace. (April 9, 2008)
- Paul Menage: Control Groups Roadmap ideas. (April 9, 2008)
Benchmarks and bugs
- Rafael J. Wysocki: 2.6.25-rc8-git2: Reported regressions from 2.6.24. (April 4, 2008)
- Rafael J. Wysocki: 2.6.25-rc8-git7: Reported regressions from 2.6.24. (April 9, 2008)
- Arjan van de Ven: Kernel Warning/Oops report for the week of April 4th, 2008. (April 4, 2008)
Page editor: Jonathan Corbet
Distributions
News and Editorials
Distribution-friendly projects Part 2
[Editor's note: This article, which looks at the interactions of software projects and distribution providers, is presented in three parts. Part 1 introduces the concepts found here, in part 2.]
Technical needs
Under the name technical needs we're going to see a series of requests that distributors often have to make to the original developers of the software they want to package. Not all these requests are made by all distributors. Some will care more about one particular aspect than another. Some might apply only on non-mainstream distributions, and some distributions might just want to take care of philosophical needs and leave the technical side entirely alone, even if similar distributions aren't exactly common.Most of the technical needs described in this article are present in the policies set forth by Debian (written), Gentoo (mostly unwritten), and apply to other distributions as well. Some of these needs won't be encoded in any policy and are often not requested explicitly by the developers. Those are mostly details that make a distributor's life easier. These details may not be mandatory, but it's still worth considering them. The easier the life of the downstream maintainer is, the easier it is for the software to be packaged.
Also, it's important to note that when a distribution makes a request, it might not be alone. Other distributions might want to take advantage of the same change, but they didn't have time to request it, or simply preferred to wait before packaging the software until some issues were resolved. Don't just ignore the request because the distribution which contacted you already took care of the issue by patching your software. Acknowledge the request and apply the patch, it will make both your and their life easier on the long term.
Sane version information
Distributions often rely on the version information provided by the original software developers. This usually means that they don't expect huge changes between version $x.y.z$ to version $x.y.z+1$.One very common scheme for versions is the major, minor, micro version, which in the example above would be respectively $x$, $y$ and $z$ (it's a common misconception that $y$ is the major version component).
The way this kind of scheme is usually applied relates to the compatibility of the programming interface (API and ABI). Changes in the software warrant increments of various version components depending on the amount of changes in the interfaces:
- adding zero or more interfaces, without changing or removing previous interfaces, or the behaviour expected from them - meaning the software is entirely compatible with the older version - usually only require an increment of micro version;
- changing or removing interfaces, usually deprecated - in such a way that older software might require to be adapted, but not rewritten - usually require an increment of minor version;
- changing the interface entirely - requiring users of the software to rewrite their code, or otherwise do major structural changes - usually require an increment of the major version.
There might be other components, too. For instance if the source archive has to be regenerated without any code change (missing file, updated addresses for the maintainers or the homepages), rather than changing the version entirely, a suffix might just be added at the end of the version, making it, for instance $1.2.3a$ or $1.2.3c$. If just a security issues has been fixed, it could also be expressed by adding a nano component to the version, like $1.3.34.1$, to emphasize that there is no change other than the security fix.
The source archives for the software should be named after both the project and the version, resulting in names like foobar-1.3.4.tar.gz. Having different versions of the same software that don't have the same naming causes confusion.
It is quite important for the distributions that source archives not be changed without changing the name: distributions usually make sure that the checksum (usually MD5, but often nowadays SHA1) of the archive is the one they recorded, and changing the tarball without notice often leads to failed builds.
There is a similar issue with the naming of the directory inside the archive. Most distributions assume that the source is included inside a directory with the same name of the archive (minus the extension), but often enough the archive contains sources not organised in a directory, or a directory with the name of the project without version. Similarly, if possible the directory should also contain eventual suffixes, to avoid adding extra cases in their presence.
Distribution methods like Ruby Gems and Python Eggs mandate similar version schemes for their packages for the same reason Free Software distribution would prefer them: it makes it easier to compare versions, and know when something has to be updated.
Internal libraries
One common issue considered by both Debian and Gentoo policies relates to the use of internal copies of libraries. Sometimes the software needs some uncommon libraries to work properly. These libraries are unlikely to be found on users' systems, which would require them to download and install them separately. Such a task is not easy for new users. A few projects will keep an internal copy of the libraries they want to use for that reason, and will use that internal copy unconditionally.Adding an internal copy of a library seems cheap to the original developers, and it's convenient for users to download and install a single package, however this causes a large number of problems to the distributors. The first problem is that they might have to patch the same bug several times. Let's all think of zlib as a practical example, a very common library implementing the classic deflate algorithm of compression. It's a very small library, that a lot of projects imported internally over the years. Not too long ago, a serious security issue was found in the code of zlib, and all the distributors had to patch it out as fast as they could. In a perfect world, patching zlib, and eventually rebuilding everything that linked to it would have sufficed. Unfortunately, we're not in a perfect world. More software was packaged with internal copies of the library, requiring each of those packages to be patched to make sure the issue was solved.
There are many other implications with using internal bundled copies of libraries, and most of them are critical for distributors. These problems increase their complexity when the internal copies of libraries are modified to suit better the use the application has for them. In those cases, even though the source might be advertised as being part of another library, they are actually different from that library, and their replacement might be impossible, or may cause further problems.
- The code is no longer shared between programs: not only the source code, which requires extra work to fix bugs and security issues, but also executable code and data. When shared libraries are used, the memory used by processes loading them is reduced, as they will share code and part of the data. This cannot be done when using static libraries or, worse, internal copies of libraries.
- Symbols may collide during the loading: modern Linux and Unix systems use the ELF format for programs and libraries. This format provides a so-called flat namespace for the symbols (data and functions) to be found. When using internal copies of other libraries in a library, the two definitions of the same symbol might collide, and just one of them can be used. If the interface used by the library changed subtly, it is possible that this will lead the program in an execution path that was not intended and is not safe.
- Distribution-specific changes need to be duplicated: as it will be discussed later on, sometimes distributions need to make changes to source code, to fix bugs (security related and not), or change paths of files for instance. Internal copies require downstream maintainers to repeat these changes multiple times.
By allowing the distributions to use their own copies of libraries, the developers are still preserving the ability for the user not to install extra dependencies, but also giving the distributions the power they need, to avoid changing the original code, sometimes in a conflicting way. It is important for the upstream authors to not change the behaviour of bundled libraries, as the distributions will most likely want to use a shared system library instead. Modifications made to a bundled library will likely cause problems for users who use get the package from their distribution's repository where it has been built with a shared system library.
An easy choice for optional dependencies
Almost all distributions prefer having a choice about the optional dependencies of a package. Source-based distributions (like Gentoo and FreeBSD's ports system) offer the same (or more) choices as the original project. Gentoo's USE flags or FreeBSD's knobs offers the user options on which options will be enabled. Binary distributions (like Debian or RedHat) might want to choose options to ensure that the final binary package does not try to use dependencies that are not present in their official repositories.Again, if a project does not provide an easy way to control whether some optional dependency is used, most distributions will either try to workaround that problem (by forcing cache discovery variables) or change the build system themselves to get the choice to disable or enable some dependency. This creates problems similar to the ones discussed above: different distributions might use slightly different changes, which may cause errors when merging them in, and they might make errors that introduce new bugs.
As above, it's just a matter of providing a switch in the build system (like a --disable-feature or --without-feature in autoconf, or a WITHOUT_FEATURE knob for make). If the software has a plug-in infrastructure, binary distributions might also just package the different plug-ins in different packages, allowing the user to choose which ones to install. Software without plug-in structures might require building different packages with different feature sets. For instance, if a software can use either OpenSSL or GnuTLS as implementation of SSL/TLS layers, then the distribution might create two packages, linking to one or the other. The user could then choose between the two.
When some optional dependencies are discovered by the build system, used if present and ignored if not, without a way to tell the software to not build the optional feature that uses a library that is present on the system, we're talking about an automagic dependency. Automagic dependency is a term used to indicate when a package, optionally using another, discovers its presence automatically, without allowing for the user (or the downstream maintainer) to ask not to use it. This kind of dependency is usually a problem just for source distributions, as they build the software on users' systems, which may or may not have the same configuration as the developer working on the build scripts. Binary distributions on the other hand build their code in controlled environments having only the stated dependencies installed. This might actually confuse one of their developers in thinking that a given dependency is mandatory, seeing it enabled in their local build, and not finding an option to disable it.
In general, automagic dependencies should be avoided; having a soft failure default is usually equivalent for the user passing by - you enable the dependency if found, disable it if not found, but still give a way to tell the build system to disable it even when found. This preserves the behaviour intended by the original developers, but also provides the control that (source) distributions want to have over what is built.
Control over how the software is built
Another problem shared both by binary and source distribution is having control on how the software is built. For binary distributions this usually means being able to impose options to the compiler, linker and other tools during the package build process, so they respect their standard options. For source distributions, this means allowing the user to choose the options to provide to the compiler, linker, assembler and other build tools, on a package-by-package basis.This does not mean that the distributions want to force-feed extra optimisations into software that might be fragile. This seems to be the biggest concern of developers for not wanting to provide a way to change the options used at compile time.
Distributions might want to reduce the optimisations used, or they might just wish to enable (or disable) warnings to more easily spot eventual problems with their packages. Distributions might also want to build debug information, or remove debug messages, and so on. There are a huge amount of possible combinations.
When the distributions want to reduce optimisation, that might be because the need to create packages which work on lower architectures not compatible with these optimisations. Or they know that some of these optimisations are not going to work with their environment. They might know that their version of the compiler does not support the optimisation, or there could be other reasons. Usually, the distribution knows the best way to handle the package for their own environment.
This also leads to a compromise between upstream developers and downstream maintainers: the former should provide their own default options and optimisations, leaving a way to override these defaults as the distributions see fit. On the other hand, distributions should try their best to determine when eventual problems might be caused by their own choice of optimisations. Distributions should not expect upstream developers to fix problems that they have caused with their choice of optimisations. This way, it's usually possible to keep the relationship between upstream and downstream in good terms even when the set of optimisations used is totally different.
More times than not, the problem is not even of willingness of the developers to provide an override, but rather a problem of actually having such an override working. While most distribution developers can fix these problems with relative ease, original developers would probably want to facilitate the work of their distributors by checking their own releases so that setting very minimal options to the compiler will work as intended. A common mistake is hard-setting CFLAGS (or similar variables) in the configure.ac file for autoconf (which otherwise has proper support for user-chosen options).
While we're talking about compiler optimisations it's important to note that for some software, e.g. number crunching software (multimedia applications, cryptography tools, etc.) enabling extra optimisations is desirable. Even so, it should be possible to disable extensive optimisation. These optimisations are usually fragile, and only work in particular environments (compiler type and version, and architectures), so having a way for distributors to decide what they actually want to enable is a very real need.
But having a way to provide options to compiler (C and C++, respectively CFLAGS and CXXFLAGS) is not all that is needed: most modern distributions might want access to the options used by the linker (LDFLAGS) to change the kind of hash tables to be generated, or to enforce particular security measures. For custom-prepared build systems, it's a common mistake to ignore this need, or to support it in the wrong way. Linker options should go before the list of object files, which in turn should go before the list of libraries to link to. This is another common mistake that distributors can fix with relative ease, but it would be better taken care of by the original developers, as it would require repeating the same steps for (almost) all distributions. [This ends part 2 of this article. Stay tuned for part 3, which will cover the philosophical concerns and present some conclusions.]
New Releases
Fedora Rawhide 20080404 Snapshot Released
A development snapshot of Fedora Rawhide was released April 4, 2008. It's available via bit torrent. "Please us bugzilla to report any problems you find (after making sure that somebody else hasn't already reported the issues). The Beta release notes ( http://fedoraproject.org/en/f9-beta-relnotes ) still mostly apply."
GNUSTEP CD 1.6 released
The GNUstep project has released version 1.6, with GNUstep development environment. "As a bonus you get some classic games like nethack, and quite a few network and system recovery and administrator tools. There is also a few 3D and audio programs on it. It's based on the 2.6.24.x Linux kernel, and on the Debian Linux distribution, created using the live-helper package. It is available for i386 and amd64."
Mandriva Linux 2008 Spring released!
Mandriva Linux 2008.1 has been released. "lots of super shiny new features like Eee compatibility and easy synchronization and all sorts of good stuff."
openSUSE-Education 1.0 RC2 for openSUSE 10.3 is Ready
The openSUSE-Education release for the openSUSE 10.3 release candidate is available. "The latest release includes new LTSP 5 packages, an update to Tuxpaint, added NetBeans 6.0.1, and a slew of other changes and improvements.." See this page for more information.
Slackware 12.1 RC1 available
From the April 3rd Slackware-current changelog: "OK, we're going to call this Slackware 12.1-rc1, though there is still some more minor work to do. Please help test! And if we're missing anything major, please let me know at volkerdi[-at-]slackware.com. Thanks. :-)"
News Flash: download Ulteo Application System Beta!
The Ulteo Application System Beta1 is available for download. "The Ulteo Application System is a FREE installable version of Ulteo, that ships with hundreds applications and innovative features..."
Distribution News
Debian GNU/Linux
Release Update: architecture status, release goal status, BSPs
Here's a Debian Lenny release update. Armel architecture looks good, Arm and hppa less so. Bug squashing parties are helping to keep the number of RC bugs down, but there are still too many left. gcc-4.3 is now the default on many archs. Click below for details on these and other release topics.
Surveying the Debian community!
Paul Wise writes: "As a Debian developer I have on occasion felt a bit out of touch with doing things with Debian and out of touch with other users. I mentioned to some folks at DebConf7 that I felt I focused too much on working on Debian and not actually connected to what the benefit of working on Debian is." To get into touch with the Debian community, Paul has some questions for Debian users and new contributors.
Let's resurrect Debian Weekly News
A call has gone out to help resurrect the Debian Weekly News. "If you like to become an editor or proof reader, please subscribe to the debian-publicity mailing list. But even if you consider your english to be not good enough you can still be of help! Of course it's impossible to be at any place and watch everything, so we need your help to report noteworthy things to the debian-publicity mailing list. That includes watching our mailing list as well as news sites, blogs and other mailing lists about Debian, Linux and IT in general."
Second call for votes for the Debian Project Leader Elections 2008
The second call for votes has gone out for the 2008 Debian Project Leader Election. "Votes must be received by 23:59:59 UTC on Saturday, April 12th, 2008"
Fedora
Fedora Board Recap 2008-APR-01
The Fedora board met on April 1, 2008. Click below for a brief look at the topics discussed. Notably, after ruling against codeina previously, there has been a reversal and the latest Fedora 8 version has been added to rawhide for Fedora 9.
Fedora: Call for Stories
The Fedora Project has issued a call for stories. "We're looking for Fedora Stories -- a person who wants to talk about how Fedora has enabled them to do something interesting or exciting with the innovative technology we provide. We want to use these stories as part of the bigger Marketing Plan for Fedora."
Mandriva Linux
Notice of discontinued support for Mandriva Linux 2007.0
Mandriva Linux 2007.0 will reach its scheduled end-of-life on April 13, 2008. "At the same time, Mandriva Linux 2007.1 will be moving into the "base" support phase and will no longer be receiving updates for desktop-related components; only core packages (such as the kernel, etc.) and other networked components (web server, database, etc.) will receive updates for an additional 6 months, until October 13, 2008."
SUSE Linux and openSUSE
Advance notice of discontinuation of SUSE Linux 10.1
SUSE Linux 10.1 will soon reach its end of life. "Having provided security-relevant fixes for more than two years, vulnerabilities found in SUSE Linux 10.1 after May 15th 2008 will not be fixed any more for this product. We expect to release the last updates around May 30th 2008. Please do not confuse SUSE Linux 10.1 with the SUSE Linux Enterprise 10 family of products, these are different products and follow different policies."
Distribution Newsletters
Ubuntu Weekly Newsletter #85
The Ubuntu Weekly Newsletter for April 5, 2008 covers Hardy Final``Freeze approaching, New MOTU members, Ubuntu Live registration info, Hardy Release Party Flyers, Launchpad OpenID, Forum News, Matt Zimmerman Interview, New Ubuntu related websites, and much more.
PCLinuxOS Magazine Issue 20
The April edition of PCLinuxOS Magazine is out, with the latest news about PCLinuxOS and other topics.
OpenSUSE Weekly News/17
This week the openSUSE Weekly News covers openSUSE-Education 1.0 RC2 for openSUSE 10.3, Tips and Tricks: Quick host-to-host transfer, Stephan Binner: openSUSE's KDE 4.0.3 Packages, Greg Kroah-Hartman: Linux Driver Project Status Report as of April 2008, Reminder: openSUSE project meeting, Event: LugRadio Live USA 2008, and several other topics.
Fedora Weekly News Issue 127
This week's Fedora Weekly News looks at announcements for "Rawhide 20080404 Snapshot Released", "Call for Stories", "Announcement list for Fedora Translation Community", "Purging old Fedora test releases", "Fedora Mirrors Wanted" and "Fedora Unity releases updated Fedora 8 Re-Spin", plus several other topics.
DistroWatch Weekly, Issue 247
The DistroWatch Weekly for April 7, 2008 is out. "It was slow news week for distributions, but developers have been quite busy. There were lots of developmental releases last week, including a Slackware 12.1 release candidate. openSUSE and Mandriva announced discontinued support, Gentoo released a beta, and a Debian developer is trying to bring back the Debian Weekly News. I took a look at the new Dreamlinux 3.0 release and while it remained pretty and added some new features, I had mixed results."
Miscellaneous Articles
Unattended Fedora 8 Installation With NFS And Kickstart (HowtoForge)
HowtoForge installs Fedora 8 with kickstart and NFS. "This document describes how to set up an installation environment with kickstart and NFS. With the resulting system you will be able to run unattended Fedora 8 installations on the client systems in your LAN - additionally, you will save a lot of Internet bandwidth. The whole client configuration can be included into the kickstart file (especially the post-installation script) so you, the admin, will also save a vast amount of time."
10 things to consider when choosing a Linux distribution (TechRepublic)
Over at TechRepublic, Jack Wallen lists ten items to think about before choosing a Linux distribution. His items cover things like whether the distribution is 100% free, security, intended use, community, and more. "Of course, times and opinions change. For nearly 10 years I [rode] the Red Hat/Fedora wagon. And then, after considerable thought, I jumped over to Ubuntu. Why? Because it fit my evolving needs. Many will argue that one Linux distribution is just like another — and I agree, on fundamentals. But when it comes down to everyday use, each distribution is different from the next. So why would you want to use Debian vs. Fedora or Ubuntu vs. Mandriva? Let’s dive into this and find out."
Interviews
Interview: Jeremy Katz on Fedora Live CDs (Red Hat Magazine)
Jonathan Roberts talks with Jeremy Katz about improvements to Fedora Live CDs. "Are there any other improvements to the Live CDs for Fedora 9, or do you have any that you'd like to get implemented for