The current stable 2.6 release is 184.108.40.206
was released on
April 7 with a fair number of fixes; 220.127.116.11
- each containing a
single security fix - both came out on April 11, and 18.104.22.168
, with a pair of x86-64
fixes, was released on April 12.
The current 2.6 prepatch remains 2.6.17-rc1; no prepatches have been
released over the last week. 2.6.17-rc2 would appear to be imminent,
however, and may be out by the time you read this.
The patches merged since 2.6.17-rc1 are mostly fixes, but there are a few
more substantive changes, including a simplified form of the scheduler starvation avoidance
patch, some tweaks to the memory overcommit algorithm, the removal of
the obsolete blkmtd driver, the removal of the unmaintained Sangoma WAN
drivers, and a new "kernel internal pipe" object (and other changes) for the
splice() system call.
The current -mm tree is 2.6.17-rc1-mm2. Recent changes to
-mm include a new set of patches for 64-bit resource tracking and some core
dump code tweaks.
Comments (none posted)
Kernel development news
"Virtualization" is the act of making a set of processes believe that it
has a dedicated system to itself. There are a number of approaches being
taken to the virtualization problem, with Xen, VMWare, and User-mode Linux
being some of the better-known options. Those are relatively heavy-weight
solutions, however, with a separate kernel being run for each virtual
machine. Often, that is exactly the right solution to the problem; running
independent kernels gives strong separation between environments and
enables the running of multiple operating systems on the same hardware.
Full virtualization and paravirtualization are not the only approaches
being taken, however. An alternative is lightweight virtualization,
generally based on some sort of container concept. With containers, a
group of processes still appears to have its own dedicated system, but it
is really running in a specially isolated environment. All containers run
on top of the same kernel. With containers, the ability to run different
operating systems is lost, as is the strong separation between virtual
systems. Thus, one might not want to give root access to processes running
within a container environment. On the other hand, containers can have
considerable performance advantages, enabling large numbers of them to run
on the same physical host.
There is no shortage of container-oriented projects. These include
relatively simple efforts like the BSD jail module through more
thorough efforts like Linux-VServer, OpenVZ, and the proprietary Virtuozzo (based on OpenVZ) offering. Many of these
projects would like to get at least some of their code into the kernel and
shed the load of carrying out-of-tree patches. There is little
interest, however, in merging code which only supports some of these
projects. The container people are going to have to get together and work
out some common solutions which they can all use.
It appears that this is exactly what the container developers are doing. A
loose agreement has been put in place
wherein developers from a few projects will discuss proposed changes and
jointly work them into a form where they meet everybody's needs. Once a
particular patch has reached a point where all of the developers are
willing to sign off on it, it can be forwarded for eventual merging into
The more complex and intrusive changes, such as PID virtualization, appear to be
on hold for now. Instead, it looks like the first jointly-agreed patch
might be the UTS namespace
virtualization patch. The aim of the patch is relatively straightforward:
it allows each container (as represented by a family tree of processes) to
have its own version of the utsname structure, which holds the
node name, domain name, operating system version, and a few other things.
In essence, it replaces a single global structure with multiple structures
attached at various places in the process tree. It still requires a
five-part patch, with every reference to the global system_utsname
structure replaced by a call to the new utsname() function.
Longer-range plans call for the virtualization of every global namespace in
the kernel, including SYSV IPC, process IDs, and even netfilter rules.
There was an interesting discussion on the virtualization of security
modules; some think that each container should be able to load its own
security policy, while others argue in favor of a single system security
policy which is aware of (and able to use) containers. Unsurprisingly,
SELinux is already equipped with a type hierarchy mechanism which can be
used with containers in the single-policy approach.
Containers might still prove to be a hard sell with some developers, who
will see them as complicating access to many internal kernel data structures
without adding a whole lot of value. It is clear, however, that there is a
demand for this sort of lightweight virtualization - OpenVZ, alone, claims to be running over 300,000 virtual
environments. So the pressure to standardize this code and move it into
the mainline will only grow over time. Once they are clean enough to
satisfy the development community, pieces of the container concept are
likely to be merged.
Comments (6 posted)
The new splice() system
was covered here last week. As was predicted then, this new
kernel API has continued to evolve; many of the non-fix patches going into
the post-2.6.17-rc1 mainline involved changes to splice()
For starters, the prototype of the splice() system call has
long splice(int fd_in, loff_t *off_in, int fd_out, loff_t *off_out,
size_t len, unsigned int flags);
The two offset values (off_in and off_out) are new; they
indicate where each file descriptor should be positioned prior to beginning
the transfer of data. Note that these offsets are passed via pointers;
user space can use a NULL pointer to indicate that the current offset
should be used. Note also that these offsets do not work like the
offsets in pread() or pwrite(): they will actually change
the offset associated with the file descriptor. Providing an offset for a
file descriptor associated with a pipe is an error.
Internally, the splice() code has seen a couple of interesting
changes. One of them (in the regular pipe code, actually) is the creation
of a new pipe_inode_info structure to represent the core machinery
behind a pipe. This structure can stand apart from the normal
inode structure. Many of the internal interfaces have been
changed to use the new structure, including the new methods in the
ssize_t (*splice_write)(struct pipe_inode_info *pipe,
struct file *out, size_t len,
unsigned int flags);
ssize_t (*splice_read)(struct file *in, struct pipe_inode_info *pipe,
size_t len, unsigned int flags);
Since there are still few implementations of these methods in the kernel,
the changes are not particularly disruptive.
Next in the list is support for directly splicing two file descriptors
where neither is a pipe. This functionality is not (yet) available to user
space via splice(), but it is used internally to implement
sendfile() with the splice() mechanism. The direct
splicing is implemented using a hidden pipe_inode_info structure
(i.e. a pipe); it is stored in the new splice_pipe field of the
task structure, so each process can only have one such connection running
at any given time.
One patch which has not been merged - and will likely wait until 2.6.18 at
this point - is the
tee() system call:
long tee(int fdin, int fdout, size_t len, unsigned int flags);
This call requires that both file descriptors be pipes. It simply connects
fdin to fdout, transferring up to len bytes
between them. Unlike splice(), however, tee() does not
consume the input, enabling the input data to be read normally later on by the calling
process. Jens Axboe provides an example implementation of the user-space
tee utility, which comes down to a couple of calls:
len = tee(STDIN_FILENO, STDOUT_FILENO, INT_MAX, SPLICE_F_NONBLOCK);
splice(STDIN_FILENO, out_file, len, 0);
The input data will be written both to the standard output and the file
represented by out_file without ever being copied to or from user
To be sure of copying the entire input data stream, the application must
perform the above calls within a loop, of course; see the full example at
the end of the tee()
This call is quite new, and may well change before it makes it into the
mainline. Among other things, it might get a new name, since something as
simple as tee() may already be in use in a number of
Comments (6 posted)
Once upon a time, setting up a Linux system was a long and problematic
process, with no assurance that a given system would work without great
pain. Most of those problems have been overcome for years, and, to a great
extent, a system can be expected to "just work" with Linux. A few
problematic areas remain, however, and wireless networking is one of them.
Even when the available hardware is supported (often not the case), making
a wireless connection work in a fully-featured way can often be a
A lot is happening in the wireless area, however. To help things along,
the folks at the Open Source Development Labs hosted a summit for
wireless networking developers on April 6
and 7 at OSDL headquarters in Portland. This event brought
together a diverse mix of developers from around the world, many of whom
had never met before. Its purpose - to chart a path forward for the
creation of a reasonable Linux wireless networking implementation -
appeared to have been largely achieved.
Your editor was fortunate enough to be able to attend this meeting. The
following report is an attempt to summarize the conclusions from the summit
- it is not a set of detailed minutes, and your editor will engage in some
chronological reordering along the way. Hopefully the result will provide
a sense for where things stand, and where they are likely to go in the near
As John Linville (the recently-named wireless networking maintainer) noted
in a conversation with your editor, early wireless adapters were marketed
as "wireless Ethernet," and Linux kernel developers treated them as a sort
of slow, unreliable, fiddly Ethernet adapters. But wireless is not
Ethernet in any way - it is a completely different set of networking
standards with its own quirks, special features, and distinct needs.
Treating wireless as a form of Ethernet slowed support for those special
features, and, more importantly, impeded the development of any sort of
internal kernel support for wireless. Each developer who set out to
write a driver for a wireless adapter ended up implementing everything from
the beginning. So there was no general wireless API, no comprehensive
support of wireless features, and a great deal of divergence and
duplication of code between drivers.
In 1997, Jean Tourrilhes decided to do something about this situation. The
result was WE-1 - the first version of the Linux wireless extensions.
There was still no 802.11 standard at the time, but the WE API enabled the
configuration and operation of wireless adapters with a single set of
tools. Jean's wireless tools are still the core utilities for managing
wireless adapters, though the graphical interfaces are replacing the
wireless tools for most users.
Development of the wireless extensions continued, with WE-9 - supporting
the new 802.11 standard - being released in 1999. WE-18, merged last year,
added support for WPA ("WiFi Protected Access"). The current revision,
WE-20, adds a new, netlink-based interface as a future replacement for the
current ioctl() API.
Though development continues, there appears to be a general, shared feeling
that the wireless extensions are heading toward the end of their useful
life. A replacement API - which does not exist yet - would work with the
entire wireless networking stack, rather than being an interface directly
to the low-level drivers. Regardless of how that plays out, however, the
wireless extensions are likely to be around for a long time to come.
The current status
The current effort to create a proper wireless stack for Linux started in
2004, when Jeff Garzik announced the creation of a
special wireless tree, initially seeded with the HostAP code. The merging
of HostAP enabled support of some relatively current networking cards and
the use of a Linux system as a wireless access point. The creation of this
tree did help to get things going, but HostAP has not turned out to be
everything that had been hoped for. In particular, there is no support in
HostAP for cards which need software MAC ("softmac") implementations. But
many contemporary cards rely upon the host software for many low-level
operations; these cards can not be supported by the wireless stack found in
the current (2.6.16) kernel.
The result is, as John Linville put it, a Linux wireless implementation
which supports "anything which is obsolete." Some cards are supported,
some better than others. Most noteworthy among current hardware is the set
of Intel IPW drivers, which, thanks to Intel, have very good support in the
kernel - but these adapters do not need softmac support.
What is lacking, at this point, is a small list of mildly desirable
features, including support for much widely-used hardware. Ease of use is
also lacking - despite improvements in the graphical tools, configuring a
wireless connection can still be a painful procedure. Perhaps the best
demonstration of these two problems was to be found at the summit itself,
where about 25% of the participants ended up using an Ethernet cable to
plug directly into the OSDL network.
Other problems include consistency (or the lack thereof) across hardware -
there are still a number of adapter-specific APIs in the kernel and in the
out-of-tree drivers. The documentation of APIs is, well, nonexistent; a
complaint was heard that Linux Device Drivers does not describe how to
write a wireless driver. There is no coordinated process for extending
APIs. Quality of service support is not present - an issue we'll return to
shortly. There are no driver test suites in general circulation. And the
whole regulatory issue looms over the wireless networking arena, and is the
largest single cause of out-of-tree (or nonexistent) wireless drivers.
Many vendors simply do not feel that they can release programming
information or free drivers and remain compliant with regulatory regimes
Meanwhile, the upcoming 2.6.17 kernel will see some improvements in its
wireless support. John merged one of the many softmac implementations out
there, on the theory that it was one of the most active projects and that
it would help to support driver development. The bcm43xx (Broadcom)
driver, which uses softmac, was also merged, and there are a couple of
other softmac-based drivers under development. Even so, the consensus
appears to be that softmac is not the way forward; that, instead, the
Devicescape stack is the real future of Linux wireless.
Devicescape is a company which offers
a number of products and services around wireless networking. In
developing it offerings, Devicescape created its own, Linux-based 802.11
stack with a number of nice features - including good softmac and WPA
support . This stack was recently released under the GPL and has been
fixed up for the kernel by Jiri Benc. It is regarded by many as
being the best of the available free stacks.
When Jeff Garzik maintained the wireless tree, he took a firm position
against moving to the Devicescape stack, stating instead that the in-kernel
code should be evolved toward the needed capabilities. He appears to have
found himself in the minority, however, and John Linville seems poised to
merge this stack for a future kernel release. He maintains a separate
development tree which includes Devicescape, and some drivers (notably
bcm43xx) have been ported to this stack. Nobody at the summit was heard to
argue against merging Devicescape.
Devicescape hacker Simon Barber talked about this code for a bit,
and a separate breakout session addressed it as well. This stack is a
large body of code. The freely-released code available now includes the
802.11 stack, the "openap" access point code, and a link-layer bridging
module. Work which will be released soon includes improvements to the
hostapd daemon (802.11g support, among other things; this code is being
merged now); bridging and VLAN integration, and various improvements to
Ethereal for wireless developers. There is also "a complete home gateway
distribution" in the works. There is the inevitable web portal being put
together to provide access to all this code.
Quite a bit of work is foreseen for the Devicescape stack. It is composed,
internally, of a long list of handler functions which deal with frames
(both data packets and 802.11 management frames) on their way to and from
the adapter. Future plans call for enabling loadable modules to plug in
their own handler functions. More of the management code may also
eventually be moved out to user space. To that end, some additional
management capabilities will be added to the hostapd daemon, which handles
authentication and management tasks. Merging hostapd with wpa_supplicant, which
handles the client side of the authentication process, is envisioned;
evidently a number of things become easier when the two functions are
merged into the same process.
There is also a great deal of complexity coming with the long list of
future 802.11 standards. These standards will require support as they are
One interesting area of development has to do with quality of service
support. 802.11 defines four service levels: "voice," "video," "best
effort," and "background." There is a priority range for each service
level, and the ranges overlap. All voice packets will go out before any
background packets, but the rest of the levels will share the available
bandwidth. With proper QoS support, a wireless user can carry on a
voice-over-IP conversation, stream video of the latest "breaking news"
celebrity sighting from CNN, grab a new kernel by FTP, and distribute
materials (best not to ask what) via Bittorrent. Each activity can operate
at its own quality of service level, and all should get the best available
Some wireless network adapters have quality of service support in the
form of four separate transmit queues. If the host places each packet in
the appropriate queue, the adapter will divide the available bandwidth
between them in a way which respects each level's service quality. The
problem is that the Linux networking stack only supports one transmit queue
per device. This presents a problem when one of the four device-level
queues fills up. There is no way to tell the kernel that no more
background packets can be queued, but there is still space for voice
packets, for example; the only thing the driver can do is to stop the queue
for all packets.
The Devicescape hackers have worked around this problem using the traffic
control mechanism built into the networking stack, which normally operates
at a level not seen by driver code. By creating a separate internal queue
for each service level, the Devicescape stack can, for all practical
purposes, implement a separate transmit queue for each service level. Even
better, it becomes possible to configure policy - which types of traffic
get which service level - from user space using the normal traffic control
tools. What would be nice, however, would be to generalize this use of the
queueing discipline code, and to make it available for other sorts of
hardware as well.
Another area requiring work is user-space API definition. There is no
well-understood API which, for example, can be used by a graphical wireless
management utility to talk with the networking stack and with processes like
hostapd. There isn't really even a discussion of how such an API should
look at the moment.
Other open issues include the usual regulatory hassles, the lack of a
user-space MAC-layer management environment, the need for better scanning,
support for adapters which perform MAC management in hardware,
power management support, and a rework of the configuration interface.
Configuration is handled by way of ioctl() calls and a
/proc interface. It was noted, in a pointed manner, that the
Devicescape code will not make it into the mainline as long as it contains
/proc files. It seems that the Devicescape stack also needs some
work before it will operate properly on SMP systems.
Finally, adding proper wireless support to the kernel will involve the
creation of a specific net_device type for 802.11 devices. An
802.11-specific sk_buff structure should also be defined. Current
code still uses the Ethernet types and drags along the extra needed
information on the side.
The biggest open issue, however, may be this: what happens to the
just-merged softmac code when Devicescape is merged? There is much
duplication of functionality there, and nobody is thrilled by the idea of
having to maintain two separate 802.11 stacks indefinitely into the
future. There is a clear parallel with the OSS and ALSA sound drivers;
ALSA was supposed to replace OSS, but removing the OSS drivers has proved
to be a difficult thing to do. It is not clear what can be done to make
removing softmac any easier.
The summit was mostly attended by kernel-oriented developers, but there was
also some discussion of user-space tools; NetworkManager
hacker Daniel Williams was present. It is recognized by all that,
while the quality of the available tools has improved significantly in the
last couple of years, there is some ground to cover yet. In particular,
configuring an interface can be relatively painless when things go well,
but, as soon as something doesn't quite work, the whole experience falls
Improving the situation will require support from the kernel side. When
things go wrong, user space needs to know just what the problem is. But
there is no consistent set of error codes returned by the kernel to
indicate, for example, that the required adapter firmware is not present,
or the provided WEP key is not valid. Some drivers support more of the
current API than others, which does not help, and API documentation is
generally not available. Better scanning support would also be useful.
While getting the networking stack and user-space tools into shape is
necessary, improving hardware support is also a necessary step toward a
Linux wireless implementation which truly "just works." Some hardware
(Intel, others) is well supported now, others (Broadcom) will be supported
soon. Some, such as the Atheros chipset, may be a long time in coming.
The existing Atheros driver (as found in OpenBSD) appears to be severely
tainted by code of questionable origin, to the point that its chances of
being merged into Linux are about zero. There is an effort to
document the Atheros hardware from the currently-available code, enabling a
clean-room driver implementation in the future, but there is quite a bit of
work yet to be done.
The regulatory compliance issue came up again in this context. Some
adapters (such as Atheros) are, for all practical purposes, general-purpose
radios which can be programmed to operate far out of the 802.11
specification. When a free driver is developed for such hardware, it would
be a Good Thing to be sure that it runs the hardware in a manner compliant
with the applicable regulations, even if it cannot necessarily be certified
as such. That sort of testing requires specialized equipment, however, and
is evidently a multi-day process. The necessary equipment does exist at
companies like Nokia and at some universities, but there is currently no
process for obtaining access to that equipment for compliance testing.
Much of the current driver work is done outside of the mainline tree, and
the kernel developers would like to see that changed. Once code gets into
the mainline, it is easier for others to review and improve. Greg
Kroah-Hartman encouraged driver developers to merge their code as early as
possible, even if it doesn't work yet.
Communications regarding wireless drivers, it was agreed, would remain on
the netdev mailing list for now. If, at some point, that conversation
threatens to overwhelm other traffic on netdev, a new list can be created.
There will also likely be a web set put together for wireless driver
information in the near future.
One purpose behind the summit was simply to try to pull more of the
relevant developers into the wider kernel process. To that end, there was
a talk on source control systems - git and quilt in particular. The "merge
early" approach was advocated many times.
Stephen Hemminger gave a talk on the state of the bridging code.
Bridging is of interest to wireless developers - it can be used for
connection sharing and mesh networking applications. To that end, the
bridging code is likely to be reworked and much of it moved to user space.
Just like routing is mostly handled by user-space daemons now, bridging
management - including the spanning tree maintenance - will move to user
space in the future.
Some representatives of the Personal
Telco Project were brave enough to compete with a delivery of pizza for
the developers' attention at lunch time. These folks have put together a
network of over 100 Linux-based free wireless hotspots around Portland.
They had a number of requests of the kernel developers, including
free Atheros drivers which don't crash the system and good,
zero-configuration mesh networking. This is an interesting project which
shows the power of what a few "unemployed geeks" can do.
Overall, the wireless summit was an optimistic event. While the
shortcomings of Linux wireless support were well recognized and understood,
there was also a clear sense that not only could the problems be solved,
but that many of the solutions were already well advanced. If all goes
according to plan, the day when Linux wireless "just works" is not that far
Comments (30 posted)
Patches and updates
Core kernel code
- Jens Axboe: sys_tee.
(April 11, 2006)
Filesystems and block I/O
Virtualization and containers
Page editor: Jonathan Corbet
Next page: Distributions>>