LWN.net Weekly Edition for October 16, 2008

OpenOffice.org releases 3.0, faces new challenges

A new version of the popular free software office application suite, OpenOffice.org (OOo) 3.0, was released this week to lots of press and enough download traffic to bring down its webserver. While the release isn't a huge leap forward in terms of features, it does provide some compelling enhancements. Perhaps the most interesting is the increased focus on extensions, a la Firefox, that don't require modifying the core OOo code. This may help combat the problem—or perceived problem—that Sun is stifling OOo development through its bureaucratic procedures for adding new functionality.

The first thing one notices when starting up OOo 3.0 is the new splash screen, but it appears for only a short time. One of the major complaints about the suite has been how long it takes to start up—something that has been addressed in 3.0. The application opens to a new welcome screen (seen at left) that presents a more friendly appearance, rather than an empty window, for new users. Once past that point, the various tools look much as they did in OOo 2.4 and earlier versions.

The other changes are mostly under the covers; they will be noticed by power users, but are not immediately obvious to basic users. These include:

• Writer (word processor) has a new slider for zooming
• Writer allows multi-page display and editing
• Calc (spreadsheet) allows up to 1024 columns per sheet
• Draw (drawing) can handle poster-size files
• Impress (presentation) supports multiple monitors for presentations
• Writer has additional editing modes for multi-lingual support as well as wiki document editing
• Calc has a new equation solver
• Chart (graphing) has improved graphical output

The OOo extensions repository has many different kinds of add-ons for OOo, that provide new or enhanced functionality for users. The most popular is the PDF import extension which allows loading PDF files into the application for editing. Given that OOo has long had the ability to natively export PDFs, importing them is an excellent addition.

Clearly Sun and the OOo project see extensions as a fertile ground for innovation by folks who are not necessarily OOo "contributors"—as they have not signed the Sun Contributor Agreement (SCA) [ PDF, currently unavailable due to the download traffic problems ]. Sun's community manager for OOo, Louis Suarez-Potts, puts it this way:

One of the new features that OOo developers are most excited about won't affect Linux users at all. OOo 3.0 has a native Mac OS X look and feel, rather than the earlier X11-based interface. A native Windows version has always been a part of OpenOffice (and its precursor, StarOffice), but the new default theme is said to be particularly attractive on that platform.

There are various new features aimed at those currently using—or needing to interoperate with—Microsoft Office. There is support for Access database files as well as improved Visual Basic for Applications (VBA) macro support. Somewhat controversially, OOo 3.0 has added the ability to read (but not write) Office Open XML (OOXML) files. OOXML is the newly minted standard for office documents that Microsoft and Ecma pushed through the ISO standardization process earlier this year.

Support for OOXML is one of the contentious areas surrounding OOo. There are two (vocal) developer camps, one Sun-centric, the other Novell-centric; unsurprisingly they tend to clash over OOXML as well as development pace and direction issues. It has gotten to the point where a fork, called Go-OO, has come about, led by Novell's Michael Meeks. Go-OO's version of OOo has been adopted by several distributions leading some to see it as a "hostile" fork.

Sun's chief open source officer, Simon Phipps, clearly sees Go-OO (and the related OO-Build) as an attempt by Novell to control OOo:

Meeks most recent look at OOo development is the proximate cause of much of the current sniping in various blogs. Meeks analyzes commits to the OOo codebase to try to extract trends in the development of the tool. His conclusion is stark—undoubtedly inflammatory to those in the Sun camp—"Crude as they are - the statistics show a picture of slow disengagement by Sun, combined with a spectacular lack of growth in the developer community."

While there have been various responses to the analysis—including this LWN comment thread—there has, as yet, been no real counter-analysis that comes to a different conclusion. Perhaps there are other ways to slice and dice the data that look more favorable to growth in the OOo community, but if not, the conclusion is worrisome. OOo is a very useful tool, that is used by many, which offers a way out of Microsoft lock-in. Because of Novell's close association with Microsoft, people worry that Go-oo is an underhanded means for another kind of lock-in—this time to Novell.

In what seems almost a taunt—as well as a validation of the accusation of a hostile fork—Meeks adds a postscript to his analysis:

There have long been complaints about the pace of OOo development, along with calls for creating a foundation to oversee it. It would seem that OOo is at a bit of a crossroads. If Sun's commitment is reduced, without a corresponding increase in contributions from others, OOo could stagnate—or Go-oo could take over.

Ostensibly, the SCA is one of the sticking points for some contributors. They do not trust Sun not to take their contributions in a proprietary direction. But the conflict is really rooted in issues of control and development direction—two things likely to lead to forking. While two forks is suboptimal, perhaps, it may lead to improvements in both the code and the development process for OOo.

There are legitimate concerns on both sides of the issue—undoubtedly the mostly silent user community has yet another perspective—but there is enough bad blood between them that it is hard to see it resolving in some relatively amicable way. The office application suite is an extremely lucrative product, at least in the proprietary world. One gets the sense that both Sun and Novell are seeing dollar signs which are clouding their vision. A neutral foundation of some kind might be a good first step towards reconciliation.

Comments (33 posted)

LK2008: Embedded and Mobile Linux

The first of these is OpenSourceEmbedded, presented by uClinux developer Jeff Dionne. He opened with a statement that, ten years ago, Linux-based embedded systems were nearly unknown. Now those systems are everywhere, with hundreds of millions of deployments. Embedded systems, he says, make up the largest installed base of Linux systems.

All is not perfect, though, in the embedded sphere. Linux still has an uncomfortably large footprint for embedded use. There is also no unified distribution for embedded use; instead, the industry is full of homemade solutions made by vendors. He would like to address this situation through the creation of a next-generation platform. It would take the form of a kit that developers could start with which comes equipped with design examples for a number of applications: telephones, digital video recorders, etc.

There are two hardware platforms being targeted initially by this effort. One is a Plasma MIPS processor - a very simple device which can be implemented with an FPGA. A simulator for this processor runs about 600 lines of code. The other, more advanced platform is a LEON 2/3 SPARC processor, a full system with a memory management unit and which supports multiprocessor configurations. Examples of the first processor include a RealTek MIPS system, while the LEON SPARC CPU is similar to current SuperH 3 processors. The Plasma and LEON SPARC processors are being designed now, with the intent of producing them as open hardware designs.

On top of these processors will be a base operating system layer with a "mini-POSIX" environment. There will be an interesting packaging system which stores components as separate "blocks" in flash, outside of any filesystem. The running system will be assembled from the blocks by the boot loader. This organization is designed to avoid bricking; any bad or corrupted components can simply be bypassed without affecting the functioning of the rest of the system. This, evidently, is how PalmOS did things.

The next challenge is creating a community around this whole effort. To that end, resources are to be put up at opensourceembedded.org - though nothing is available as of this writing. The site will include project hosting, along with the ability to download the development kits. Jeff says that the uClinux experience has shown that the kit approach works; with a ready-to-use code base like that, a community can come together.

There are also plans to create an organization behind this effort which, among other things, can enter into non-disclosure agreements with hardware manufacturers. This organization will also work to help vendors ship GPL-compliant products.

OpenSourceEmbedded appears to be in an early state, so it's hard to make any guesses about how successful it will be. For more information, see Jeff's slides [PDF].

Mobile Linux

The closing session at the 2008 Linux-Kongress was a talk by Dirk Hohndel, who began by noting that Linux-Kongress is, in fact, the oldest Linux event. It was first held in 1994, and hosted many of the kernel developers who were active at that time; Dirk estimates that about half of the development community was to be found in a single room. It would take a rather larger room to accomplish that now. Dirk complimented the event on its avoidance of commercialism and its sustained focus on the technology.

The technology that Dirk came to talk about was mobile Linux. He started by expressing his disappointment with desktop Linux. It has become a collection of poorly-integrated applications which are somehow trying to replicate Windows 95. The result does not work well on the desktop, and it most certainly is not optimized for the mobile environment.

But, says Dirk, mobile Linux is not really embedded Linux either. Embedded Linux evokes images of access points and other single-application boxes which are not meant to be extended past a single function. They are not concerned with the user's experience, and they are not concerned with mobility. The subject here is devices with a screen, and which can have new applications installed onto them. So some sort of desktop-like interface is needed, but current desktop Linux does not fill the bill.

According to Dirk, the problem with desktop Linux is the fundamental approach: developers are not the target audience for this software, but they are making all the interface decisions. What's needed is input from people who are specialized in interface design and human-computer interaction. That leads to a "scary thought": interface specialists are generally not coders, but they will be making decisions that coders are expected to implement. That is not a normal mode of operation in the free software community, but it is needed here.

Other problems include the proliferation of "80% done" projects. Much of the work has been done, but nobody wants to do the work to finish the job. There's also far too many choices; in general, says Dirk, people do not like it if they have to choose between more than two alternatives. When dealing with the Linux desktop, it's hard to find situations where there are fewer than six choices. And, overall, the Linux desktop lacks consistency. That, says Dirk, is why he uses an Apple laptop. Apple enforces a consistent design across the application space and, he says, the result is very nice.

Devices should be simple and natural to use; such devices are increasingly hard to find anywhere. As an example, he held up a paper notebook. The device boots very quickly, has a nice "touch-based" pencil-oriented interface. No manuals or explanations are needed. Linux-based devices should be just as easy to use. But, at the same time, they need to offer an experience which is close to what people expect from an ordinary, desktop computer. It should have access to the Internet, and users should be able to install software.

Dirk then pulled out an Eee PC system and gave the five-second boot demonstration. This work, he says, is an example of what is being done by Intel in support of the Moblin project. Intel is trying to solve some of the hardest problems in the mobile space, contributing the results for everybody to use.

To that end, Moblin is working toward the creation of a base distribution for mobile systems. The user interface will be based on the GNOME mobile work, but with a lot of enhancements. The end goal is the creation of a Linux distribution for mobile devices which is far better than the state of the art today. It is not, he says, an attempt to compete with distributors; instead, Moblin is providing a base which the distributors can build on. Intel's effort will naturally focus on Intel processors, but contributions for any architecture are welcome at Moblin.

In conclusion, Dirk noted that Linux's success on the server side was relatively easy. The mobile problem is much harder. Intel is hoping that others will join in to help Moblin reach its rather ambitious goals.

Comments (6 posted)

Fedora checking community health with EKG

Measuring the health of communities is an interesting, difficult task. The Fedora project has recently started using a new tool, called EKG, to try to get an overview of the demographics of the free software projects that are sponsored by the distribution. EKG is still young, but already provides some interesting information. Because it is GPL-licensed, as is the Fedora norm, it can be picked up by other distributions or interested parties to target their own projects.

At its core, EKG is a few Ruby scripts that process mailing list data so that graphs can be produced. Currently, it produces both pie charts and line graphs that indicate the number of Red Hat posters versus those from elsewhere. A portion of the most recent set of graphs can be seen at right.

Red Hat's Michael DeHaan has taken on development of EKG to use as a tool to measure how well various projects are building a community separate from Red Hat. There are lots of free software projects that have been released by Red Hat—or Fedora, which often amounts to the same thing—but may or may not be seen as useful tools outside of Fedora. By looking at the mailing list traffic, particularly over time, some idea of which projects are building a community, and which aren't, can be derived. As the project page puts it:

Mailing lists are just one measure of the health of a project, of course, so DeHaan is looking at other metrics. Commits to the project repository—along with the identities of the commiter—would seem an obvious choice. Better graphs with more useful information on each axis as well as time series of the pie charts are also on the "to do" list. He is also looking at derived statistics that will allow direct comparison of different projects by using equations that in some way model success.

It is difficult to draw any conclusions from the limited graphs that are currently available. One thing that does stand out, though, is the popularity of gmail.com email addresses, which seem to account for around one-quarter of posts. One can also certainly see projects that are completely dominated by "inside" (i.e. Red Hat) folks. The JBoss lists are a good example.

Projects are trying various ways to measure how well they are doing their job; EKG is another way to do that. For the kernel, the statistics on each release are gathered by LWN, as well as over longer periods by the Linux Foundation. Ubuntu has its Upstream Report which looks at how well bugs are getting to upstream bug trackers. Undoubtedly other projects have their own ways of trying to measure their impact.

As yet, there is no mailing list for EKG development. We look forward to the day when EKG is applied to its own development list. It would seem that some kind of "metahealth" measurement of the community might be able to be derived from that data.

Comments (none posted)

LK2008: The values of the Linux community

James started with a bit of history, noting the the initial value placed on software was entirely commercial. Once the industry realized that software could be worth far more to its users than it costs to create, the proprietary mode became dominant - and that has affected the evolution of programming in general. The value placed on the code by its developers became irrelevant, leading to "paycheck coding." There is no value placed on creativity, and such a model leads to bad code.

Eventually Richard Stallman came along and challenged the commercial view of software. But, during this time, about the only alternative to commercial software was the BSD Unix distribution, and that got caught up in the lawsuit by ATT. So closed software took over; Windows won on commodity platforms, but proprietary software also became dominant in the Unix arena.

In 1991, Linux hit the scene; since then, it has become the most popular and vibrant free software operating system available. In a sense, this is interesting, in that Linux is licensed under the GPL, a license that many companies hate. Apple explicitly chose BSD as the base for MAC OS to avoid GPL-licensed code. But, despite this antipathy, lots of companies use Linux, and even contribute to its development. It is interesting, James says, to look at why that is.

The reason is the Linux community's values. In particular, the community prizes technical merit above all other considerations - including small things like what any company or user would like to have. Also prized is passion; code supported by a developer who clearly cares about it will generally fare better in the review process. If the code quality and the passion are there, the community does not care about much of anything else. Factors like the source of the code or who might benefit from its incorporation don't really matter.

In particular, contributors to the kernel are not required to sign on to any particular belief system or any specific view of freedom. A contributor may have an FSF-like belief in free software, or, instead, be a corporate developer who does not care about software freedom at all. Even the BSD community requires acquiescence with a specific view of freedom. A Linux contributor, instead, need only be willing to contribute the code under the share-alike rules of the GPL.

As a result, anybody can play with Linux, regardless of philosophy or corporate status. We have a community which is defined by contributions, not by a specific set of values regarding software freedom. That has allowed the formation of a very diverse community with a specific shared interest: creating the best kernel we can.

There are some significant benefits from this approach. It forces companies to recognize their engineers' values; that, in turn, makes for more motivated developers. Developers who are interested in improving Linux can get resources and support from corporations. Users get high-quality code from developers who care about what they are doing. Companies get the ability to focus on their little piece of the problem while taking advantage of the community-maintained kernel for the rest; they can also offload their older code to the community for long-term maintenance.

James compared the Linux way of doing things with the US constitution. That document only mentions freedom three times, yet it has become a blueprint which has supported freedom for over 200 years. It is a relatively short document. The proposed EU constitution, instead, is about 20 times the length, before taking into account other documents which are referenced. That document would appear to be somewhat bloated; the goals would be better served by a more concise formulation.

Similarly, the Linux community spends little time talking about freedom. Instead, the focus is on a set of brief principles involving code quality and passion. Freedom is not legislated; it arises as an emergent value inherent in the Linux way of doing things. Linux has managed to bring about software freedom without talking about it, and without imposing a view of software freedom on its contributors. In the process, Linux has succeeded in creating something which is as free - or more free - than the GNU system envisioned by the Free Software Foundation.

During the question period, James wished for a free software advocate who would argue the point with him, but no such person emerged. He will, it seems, have to repeat the talk in a different venue before he can have that debate.

Comments (49 posted)

Page editor: Jake Edge

Security

SELinux permissive domains

Readers of this page—along with the kernel page—will not find it surprising that SELinux is a complex beast. It is, however, the dominant security framework for Linux, pushed hard by Red Hat, but also being adopted, slowly, by SUSE, Ubuntu, and others. Over the years, through lots of hard work, it has become somewhat less complex, at least for administrators; a new feature, called permissive domains will help further ease the administration of SELinux-enabled systems.

These days, SELinux has two modes, the aptly named enforcing and permissive modes. When in enforcing mode, SELinux will not allow operations that are not permitted by the policy, whereas in permissive mode, a violation is just logged and the operation is allowed to continue. Administrators trying to track down an SELinux problem with an application—whether a real security issue or just a problem with the policy—can put the system into permissive mode, then study the logs to determine what policies are being violated. Or they can use audit2allow to make those policy changes for them.

Until permissive domains, though, the choice between permissive and enforcing was binary for the entire system. By putting a system into permissive mode, various attacks that SELinux might normally stop on other applications would instead just be logged. With permissive domains, a single process, or group of related processes, can be marked as permissive, while the rest of the system stays in enforcing mode.

Red Hat SELinux hacker Dan Walsh, describes permissive domains on his blog. One of the motivations is to help third-party software developers feel more comfortable about shipping SELinux policy with their application:

Permissive domains are available in recently updated Fedora 9 systems and will come standard with Fedora 10. As Walsh shows, enabling permissive mode for a domain is trivial:

    # semanage permissive -a httpd_sys_script_t

    # semanage permissive -d httpd_sys_script_t

This is definitely a nice step forward for assisting with policy development, but there is still a lingering problem with the recommended way to generate SELinux policies. Walsh describes how that is done:

The problem, of course, is that blindly using audit2allow is extremely dangerous. It assumes that the application has no security problems, that all of its accesses should be permitted—if that can be assumed, what is SELinux for? By taking all of the violations and turning them into policy changes, the application, rather than the policy developer, decides on the access it requires. Using audit2allow correctly is much more complex, requiring a good understanding of SELinux and the existing policies and domains.

To be fair to Walsh, in a related post, he does warn:

The audit2allow manpage is even more explicit:

Using audit2allow is, unfortunately, the way that most SELinux policy is developed. There aren't enough SELinux experts—there may never be enough—to actually look at the code for applications and determine a priori what the policy should look like. So, testing applications by running them to determine what permissions they require is the only sane way to do it, error-prone though it may be.

Comments (4 posted)

New vulnerabilities

cups: several vulnerabilities

Comments (none posted)

dbus: denial of service

Comments (none posted)

exiv2: denial of service

Comments (none posted)

kernel: several vulnerabilities

Comments (none posted)

mon: insecure temp files

Comments (none posted)

portage: privilege escalation

Comments (none posted)

ruby: multiple vulnerabilities

Comments (none posted)

Events

OWASP Summit set for November in Portugal

Full Story (comments: none)

Page editor: Jake Edge

Kernel development

Brief items

Kernel release status

The 2.6.27 kernel is out, released by Linus on October 9. For those just tuning in, 2.6.27 includes (among many other things) UBIFS, support for integrity checking in the block layer, multiqueue networking, the ftrace tracing framework, the lockless page cache, the relocation of a lot of firmware, the GSPCA webcam driver set, and a number of extended system calls. See the always-excellent KernelNewbies summary for lots more information about this release.

The 2.6.28 merge window is currently open with around 4100 changesets merged at the time of this writing. See the article below for a summary of what has been added to the kernel so far in this development cycle.

The current stable kernels are 2.6.25.18 and 2.6.26.6 which were released on October 8. Both contain a long list of important fixes throughout the kernel tree.

Comments (4 posted)

Kernel development news

Quotes of the week

On Tue, 14 Oct 2008, Jean Delvare wrote:
> 
> Marek Vasut (1):
>       i2c/tps65010: Vibrator hookup to gpiolib

Comments (none posted)

Quotes of the week (part 2)

Comments (1 posted)

Merged for 2.6.28

User-visible changes for 2.6.28 include:

• There are new drivers for Analog Devices SSM2602, AD1882A and AD1980 codecs, Freescale MPC5200 I2S audio devices, Texas Instruments TLV320AIC26 codecs, Tascam US-122L USB Audio/MIDI interfaces, Wolfson Micro WM8580, WM8900, WM8903, and WM8971 audio devices, Blackfin SPORT peripheral interface controllers, NVIDIA HDMI HD-audio codecs, Toshiba RBTX4939 MIPS boards, Atheros L2 10/100 network adapters, Cisco 10G Ethernet adapters, JMicron JMC250 chipset-based network adapters, QLogic QLGE 10Gb Ethernet adapters, SMSC LAN95XX based USB 2.0 10/100 ethernet devices, AFEB9260 ARM-based boards (an open source board design), Arcom/Eurotech VIPER boards, AT91SAM9X watchdog devices, ITE IT8716, IT8718, IT8726, and IT8712 Super I/O watchdogs, W83697UG/W83697UF watchdog devices, TLV320AIC23 codecs, Micron MT9M111 camera chips, Magic-Pro DMB-TH tuners, Afatech AF9015 and AF9013 DVB-T USB2.0 receivers, Conexant cx24116/cx24118 tuners, DVB cards based on SDMC DM1105 PCI chip, Silicon Laboratories SI2109/2110 demodulators, ST STB6000 DVBS Silicon tuners, numerous Fujifilm FinePix cameras, ALi video camera controllers, WM8400 AudioPlus HiFi codecs, and SGS-Thomson M48T35 Timekeeper RAM chips.

• Support for the old Sun 4 architecture and ColdFire serial ports has been removed.

• There is a new sysfs file (unload_heads) which can be used by a user-space process to tell an ATA disk to retract its heads and prepare for an impact. When used in conjunction with an accelerometer, this feature could be used to attempt to preserve a disk in a falling laptop.

• Improved support for ptrace() - and support for precise event-based sampling in particular - has been added for the x86 architecture.

• The crypto subsystem has gained support for deterministic ANSI X9.31 A.2.4 pseudo-random number generation.

• The SMACK security module can now be configured to enforce mandatory access control rules on privileged processes.

• There is a script which can be used to generate a minimal "dummy" policy for SELinux. The smallest workable policy, it seems, is 587 lines long.

• Some sound devices can detect the presence of audio devices on input and output jacks. The ALSA layer now allows drivers for those devices to register those jacks and report the presence of devices attached to sound cards through the input layer.

• Work with multiqueue networking continues; 2.6.28 will include the ability to associate a separate queueing discipline with each internal packet queue.

• The wireless regulatory compliance subsystem has been merged.

• The kernel now supports the Phonet packet protocol used by Nokia cellular modems. See networking/phonet.txt in the kernel documentation directory for more information.

• Also added to core networking is support for the Distributed Switch Architecture protocol, with initial support for a number of Marvell switch chips.

• The netfilter layer has been augmented to support network namespaces.

• The ext4 system has lost the "ext4dev" name; this is a signal that the developers are getting ready to declare it ready for production use. Ext4 has also gained a set of static tracepoints for use with SystemTap or other tracing tools.

• The FIEMAP ioctl() for extent mapping has been added.

• Xen has added CPU hotplugging support.

• Version 4 of the rpcbind protocol is now supported; this enables the kernel to offer RPC services via IPv6.

• The OCFS2 filesystem has gained a number of features, including POSIX locks, extended attributes, and use of the JBD2 journaling layer.

Changes visible to kernel developers include:

• Discard request and request timeout handling have been added to the block layer; a number of other internal API changes have been made as well. See this article for details.

• Video4Linux2 drivers no longer have their open() function called with the big kernel lock held. The lock_kernel() calls have been pushed down into individual drivers within the mainline tree; external drivers will need to be fixed.

The merge window is likely to remain open until approximately October 24.

Comments (none posted)

Block layer: solid-state storage, timeouts, affinity, and more

Solid-state storage devices

There are some enhancements aimed at improving the kernel's support of solid state storage devices. One of those, the discard API, has been covered here before. This API allows high-level block subsystem users (filesystems) to indicate that a particular range of blocks no longer contains useful data. That allows the low-level device to incorporate those blocks into its garbage collection scheme and to stop worrying about their contents when performing wear leveling.

Since the initial LWN article, though, the API has changed a little. The way to issue a discard request is now:

    int blkdev_issue_discard(struct block_device *bdev, sector_t sector,
			     unsigned nr_sects);

The end_io() parameter seen in previous versions of the API is no longer present. There is no way for callers to know when the request completes, or, indeed, if the request completes at all. Since the caller is indicating a lack of interest in the given sectors, it really should not matter what the device does thereafter.

There is a filesystem-level function for creating discard requests:

    static inline int sb_issue_discard(struct super_block *sb,
				       sector_t block, 
				       unsigned nr_blocks);

Here, the interface is expecting block numbers using the filesystem block size, rather than 512-byte sectors.

User-space programs can issue discard requests with the new BLKDISCARD ioctl() call. Needless to say, such operations should be done with care; about the only logical user of this ioctl() would be mkfs programs.

Block drivers which support discard requests will provide a suitable function to the block layer:

    typedef int (prepare_discard_fn) (struct request_queue *queue, 
    	    			      struct request *rq);

    void blk_queue_set_discard(struct request_queue *q, 
    	                       prepare_discard_fn *dfn);

In the absence of a "prepare discard" function, discard requests for the device will fail.

The block layer has also added a flag by which drivers can indicate that a device is not rotating storage, and, thus, does not suffer from seek delays. By setting QUEUE_FLAG_NONROT (with queue_flag_set() or queue_flag_set_unlocked()), a driver tells the block layer that it is working with a solid state device. I/O schedulers can use that information to avoid plugging the queue - a useful technique for combining requests to rotating storage devices, but a useless operation when there is no seek penalty to avoid.

Request affinity

On large, multiprocessor systems, there can be a performance benefit to ensuring that all processing of a block I/O request happens on the same CPU. In particular, data associated with a given request is most likely to be found in the cache of the CPU which originated that request, so it makes sense to perform the request postprocessing on that same CPU. With 2.6.28, sysfs entries for block devices will include an rq_affinity variable. If it is set to a non-zero value, CPU affinity will be turned on for that device. According to the patch changelog, turning this feature on can reduce system time by 20-40% on some benchmarks.

Timeout handling

Robust device drivers typically have to be written to handle cases where devices fail to complete operations they have been instructed to do. In a few cases, higher-level code helps with this task; the networking layer, for example, can track outgoing packets and let a driver know when a transmit operation has taken too long. In most other drivers, though, it's up to the driver itself to notice when an operation seems to be taking too long.

Like the network subsystem, the block layer manages queues of requested operations. As of 2.6.28 the block layer will, again like networking, have a mechanism for notifying drivers about request timeouts; that, in turn, will allow a bunch of timeout-related code to be removed from the lower layers. Timeout handling in the block layer can be more complex, though, and the associated API reflects that complexity.

A block driver must register a function to handle timed-out requests:

    typedef enum blk_eh_timer_return (rq_timed_out_fn)(struct request *);

    void blk_queue_rq_timed_out(struct request_queue *q, 
				rq_timed_out_fn *fn);

The amount of time a request should be outstanding before timing out is set up with:

    void blk_queue_rq_timeout(struct request_queue *q, 
    	 		      unsigned int timeout);

The tracking of per-request timeouts is done within the block layer; the timer for any individual request is started when that request is dispatched to the driver by the I/O scheduler. Should a request fail to complete before the timeout period passes, the driver's timeout function will be called with a pointer to the languishing request. The driver then can do one of three things:

• Figure out that, in fact, the request was completed as expected, but that completion had not been noticed by the driver. A dropped interrupt could bring out such a situation, for example. In this case, the driver returns BLK_EH_HANDLED, and the request will be marked as completed.

• Decide that the request needs more time, perhaps because it has been re-issued by the driver. A BLK_EH_RESET_TIMER will start the timer again for this request.

• Punt and return BLK_EH_NOT_HANDLED. The block layer currently does nothing at all when it gets this return code; future plans appear to include aborting the request within the block layer when this return value is encountered.

If things look bad, the driver may decide to abort any outstanding requests, reset the device, and start over. There are a couple of new functions which can help with this task:

    void blk_abort_request(struct request *req);
    void blk_abort_queue(struct request_queue *q);

These functions will abort the given request, or all requests on the queue, as appropriate. Part of that process involves calling the driver's timeout handler for each aborted request.

Other changes in brief

Some other block-layer changes include:

• The handling of minor numbers has been changed, allowing disks to have an essentially unbounded number of partitions. The cost of this change is that minor numbers may be attached to a different major number, and they might not all be contiguous; for this reason, drivers must set the GENHD_FL_EXT_DEVT flag before the extended numbers will be used. See this article for more information on this change.

• The prototypes of blk_rq_map_user() and blk_rq_map_user_iov() have changed; there is now a gfp_mask parameter. This allows these functions to be used in atomic context.

• kblockd_schedule_work() has an additional parameter specifying the relevant request queue.

• The new function bio_kmalloc() behaves much like bio_alloc(), but it does not use a mempool to guarantee allocations and can thus fail.

It is, all told, one of the busier development cycles for the block layer in recent times.

Comments (1 posted)

Patches and updates

Kernel trees

• Linus Torvalds: Linux 2.6.27. (October 13, 2008)

• Greg KH: Linux 2.6.26.6. (October 9, 2008)

• Steven Rostedt: 2.6.26.6-rt10. (October 13, 2008)

• Greg KH: Linux 2.6.25.18. (October 9, 2008)

• Steven Rostedt: 2.6.24.7-rt21. (October 9, 2008)

Core kernel code

• Ingo Molnar: scheduler updates for v2.6.28. (October 13, 2008)

• Ingo Molnar: fastboot tree for v2.6.28. (October 13, 2008)

• Arjan van de Ven: fastboot: Introduce an asynchronous function call mechanism. (October 13, 2008)

• Ingo Molnar: RCU updates for v2.6.28. (October 13, 2008)

• Dave Hansen: Track in-kernel when we expect checkpoint/restart to work. (October 15, 2008)

• Heiko Carstens: Convert stop_machine to use a workqueue. (October 15, 2008)

Development tools

• Paul E. McKenney: rudimentary tracing for Classic RCU. (October 9, 2008)

• Paul E. McKenney: v3 rudimentary tracing for Classic RCU. (October 15, 2008)

• Torok Edwin: ftrace: add userspace stacktrace support and semaphore-latency tracer. (October 15, 2008)

Device drivers

• Jesse Barnes: What's in pci-2.6.git. (October 9, 2008)

• Jiri Kosina: HID merge plans for 2.6.28. (October 9, 2008)

• Mauro Carvalho Chehab: V4L/DVB updates. (October 15, 2008)

• Alan Cox: First block of the tty backlog. (October 15, 2008)

• Greg KH: STAGING patches for 2.6.28. (October 15, 2008)

• Alan Cox: TTY updates for 2.6.28. (October 15, 2008)

• Alex Chiang: PCI: let the core manage slot names. (October 15, 2008)

• Yu Zhao: PCI: Linux kernel SR-IOV support. (October 15, 2008)

Filesystems and block I/O

• Theodore Ts'o: New ext4 patchset: 2.6.27-rc9-ext4-1. (October 9, 2008)

• Evgeniy Polyakov: Distributed storage release.. (October 15, 2008)

• Szabolcs Szakacsits: Stable NTFS-3G 1.5012 released. (October 15, 2008)

• Edward Shishkin: Reiser4 for Linux-2.6.27. (October 15, 2008)

• Marco Stornelli: PRAMFS with XIP support. (October 15, 2008)

Networking

• Herbert Xu: gre: Ethernet over GRE. (October 15, 2008)

• Thomas Graf: pkt_sched: Control group classifier. (October 15, 2008)

Architecture-specific

• Ingo Molnar: What's new in x86.git for v2.6.28. (October 9, 2008)

Security-related

• Herbert Xu: Crypto Update for 2.6.28. (October 13, 2008)

• James Morris: Security related updates. (October 13, 2008)

• Kentaro Takeda: TOMOYO Linux. (October 15, 2008)

• Mimi Zohar: integrity. (October 15, 2008)

• Eric Paris: [PATCH -v2 00/16] fanotify: novel all file access notification and permissions system. (October 15, 2008)

Page editor: Jake Edge

Distributions

News and Editorials

What is Ulteo?

In the past two years the project has had some time to mature and with the announcement that OpenOffice.org 3.0 is available through Ulteo.com it seemed like a good time to revisit the project.

Ulteo is aimed at Windows users, and gives them a slow and easy to way to convert to Linux using the first of several several sub-projects; the Ulteo Online Desktop. Many Linux applications are available through a Java enabled web browser such as Firefox or Internet Explorer. OpenOffice.org, KPdf, Kopete, Skype, Thunderbird + Enigmail, Gimp and Digikam, Inkscape and Scribus and many other applications are available in the Online Desktop without installing any new software on the PC. A subscription to Ulteo Premium provides extra storage for documents and other benefits.

Once the user becomes comfortable with Linux applications they could be ready for the Ulteo Application System which is an installable system for the PC. The Application System features automatic document synchronization/backup, automatic updates and upgrades, and all the applications included in the online desktop.

The Ulteo Virtual Desktop seems to be much the same as the Online Desktop. It is designed to run under Windows and allows the use of both Linux and Windows applications. The Virtual Desktop uses coLinux to provide the Linux desktop on Windows.

The final Ulteo product, for now at least, is the Documents Synchronizer. This, like the Virtual Desktop, is Windows software but it can be used with the Online Desktop to backup and retrieve documents, whether these are produced locally with Windows applications or with Linux applications using Online Desktop.

Ulteo is not something that will be of immediate interest to the average LWN reader. Presumably most readers are already knowledgeable about running Linux and its applications. However most of us probably do know someone who is not ready to run Linux natively. At least some of those people could start using the Online Desktop and become more familiar with various Linux applications without having to download and install those applications. Who knows where they might go after that.

Comments (2 posted)

New Releases

Mandriva Linux 2009 has been released

Comments (none posted)

Fedora 10 Snapshot 1 Released

Full Story (comments: none)

Announcing the General Availability of XO Software Release 8.2.0

Full Story (comments: 2)

Distribution News

Debian GNU/Linux

Bits from the Debian Eee PC team

Full Story (comments: 9)

Bits from the Debian CD team

Full Story (comments: none)

Debian bugs #600000 and #1000000 contest

Full Story (comments: 2)

Fedora

Fedora Board Meeting Recap 2008-10-07

Full Story (comments: none)

First steps of the transition from Livna to RPM Fusion begins now for livna-devel users!

Full Story (comments: none)

Distribution Newsletters

Arch Linux Newsletter

Comments (none posted)

DistroWatch Weekly, Issue 274

Comments (none posted)

Fedora Weekly News #147

Full Story (comments: none)

Ubuntu Weekly Newsletter #112

Full Story (comments: none)

Newsletters and articles of interest

Mandriva Linux 2009 'optimised' for mini laptops (vnunet)

Comments (none posted)

Distribution reviews

Foresight Kid's can inspire young minds (Linux.com)

Comments (3 posted)

Sidux grows on you (Linux.com)

Comments (none posted)

Page editor: Rebecca Sobol

Development

Connecting to Microsoft Exchange with OpenChange

Working with a Windows network from Linux has never been a smooth ride. While Samba, Wine and OpenOffice.org have made many components workable, connecting to the Microsoft Exchange email server has remained unreliable. Now the OpenChange developers hope to change that, providing the same capabilities as Microsoft Outlook in a range of Linux-native clients like Kontact and Evolution. OpenChange is not yet workable, but partial operation can demonstrate its potential.

If you want to connect to Exchange at the moment, you have a few options. Evolution can connect using a hack with Outlook Web Access, providing email, shared folders, calendars and contacts. But it's far from reliable; I tried to get by with it at the office, warts and all, and managed it for a couple of weeks before resigning myself to Windows. The other options are even worse -- just use the webmail client, or use the IMAP server for email and hacks such as this one to get at other data in a manner similar to Evolution. Working from home on Kubuntu, I find it easier to just use the webmail client.

OpenChange is taking a much more sensible approach. At the heart of the project is a MAPI-compatible API, which allows clients to talk directly to Exchange and access all of its functionality. The code is still being actively developed, but some application developers have started playing around with it; the first code for Evolution came out in January 2008. According to Brad Hards, an OpenChange and Kontact developer, "OpenChange can do most of the Exchange tasks now, though it can't currently do free/busy."

For the curious, OpenChange developer Julien Kerihuel has written a simple command-line client. It's currently available in Ubuntu Intrepid and Debian Experimental, though you're better off compiling it yourself as it is changing quite rapidly. It isn't especially well documented, and the manpage implies some functionality that Kerihuel is still working on, but I did have some success.

First, you need to set-up a new profile:

tom@dell-desktop:~$ mapiprofile --profile=tom.chance -w=dell-desktop \
  -I=192.168.0.11 -D=BIOREG -u=tom.chance -p=mypassword --create

You can check if it has worked by listing your mailboxes:

tom@dell-desktop:~$ openchangeclient --mailbox --profile=tom.chance \
  + Mailbox - Tom Chance

|---+ Calendar :      (Total: 774 / Unread: 0 - Container class: IPF.Appointment) [FID: 0xdadb000000000001]  
|---+ Contacts :      (Total: 59 / Unread: 0 -  Container class: IPF.Contact) [FID: 0xdbdb000000000001]
|---+ Deleted Items : (Total: 0 / Unread: 0 -   Container class: IPF.Note) [FID: 0x3efc000000000001] 
|---+ Drafts :        (Total: 0 / Unread: 0 -   Container class: IPF.Note) [FID: 0xdcdb000000000001]   
|---+ Inbox :         (Total: 474 / Unread: 0 - Container class: IPF.Note) [FID: 0x3bfc000000000001]
|---+ General :       (Total: 10 / Unread: 1 -  Container class: IPF.Note) [FID: 0xe00240000000003]

I managed to send a test email, which I picked up in Outlook without problems. When I opened the same email in KMail, however, it has a "winmail.dat" binary file attached, which you wouldn't normally get in emails from Outlook.

tom@dell-desktop:~$ openchangeclient -p=tom.chance --to="tom.chance@bioregional.com" \
  --subject="OpenChange test" --body="Hmm, did this work?" --sendmail

sendmail MAPI_E_SUCCESS (0x0)

You can also interrogate folders, send emails, create and delete contacts, calendar appointments and access most of the other Exchange functionality. Kerihuel: "Openchangeclient is a test case for libmapi, it's a useful way to test if a problem is in the client application or in libmapi, and there is a plugin for sugarcrm, so it may remain in future." There's a proxy server using Samba too, for those who want yet another way of connecting.

For Kontact users, usable integration is probably a good 6 months away. The akonadi resource can deal with most of OpenChange's functionality, "at least a bit", accord to Hards, though "Kontact can't currently make use of it because it isn't converted to akonadi yet." KDE 4.2 should come out with akonadi integration, but the OpenChange functionality might not yet be stable enough for large quantities of important data. Hards thinks KDE 4.3 is probably "the sweet spot."

Until then, Ballmer's mantra remains relevant; OpenChange and its client implementations could do with developers, developers, developers. Cracking this nut could throw open Exchange to a new range of clients, and as Kontact and its peers become stable on Windows and MacOSX, an entrenched Windows server will pose less of a threat to free software migrations on desktops.

Comments (3 posted)

System Applications

Clusters and Grids

VIFF 0.7.1 released

Full Story (comments: none)

Database Software

PostgreSQL Weekly News

Full Story (comments: none)

Device Drivers

LIRC 0.8.4 released

Comments (none posted)

Filesystem Utilities

Stable NTFS-3G 1.5012 released

Full Story (comments: none)

Mail Software

managesieve 0.4 released

Full Story (comments: none)

Networking Tools

libnetfilter_conntrack 0.0.97 released

Full Story (comments: none)

Printing

CUPS 1.3.9 released

Comments (none posted)

Web Site Development

Plone 3.1.6 released

Full Story (comments: none)

Plone 3.2a1 released

Full Story (comments: none)

Desktop Applications

Audio Applications

Ardour: Finally, "move Tracks Up/down" (and Other News)

Comments (none posted)

mp3splt 0.5.2 released (SourceForge)

Comments (none posted)

Business Applications

Chandler Desktop 1.0.2 released

Full Story (comments: none)

Openbravo ERP: R2.40 version is available (SourceForge)

Comments (none posted)

CAD

Sweet Home 3D: version 1.4 released (SourceForge)

Comments (none posted)

Data Visualization

DISLIN 9.4 released

Full Story (comments: none)

Desktop Environments

GNOME Software Announcements

• Libgda and Libgnomedb 3.99.5 (new features, bug fixes and translation work)
• Orca 2.24.1 (bug fixes and translation work)

Comments (none posted)

KDE Software Announcements

• Elltube 0.2.2 (new features and bug fixes)
• KBib 0.6.5 (new features and bug fix)
• kipi-plugins 0.1.6 (bug fixes)
• libkdcraw 0.1.6 release for KDE3 (new features)
• Palapeli 0.3 (unspecified)
• Qwipe 0.1 (initial release)
• rpCalc 0.6.0 (new features)
• SMILE 0.8.4 (bug fixes)

Comments (none posted)

Xorg Software Announcements

• xorg-server 1.5.2 (bug fixes and code cleanup)

Comments (none posted)

Desktop Publishing

GNU PDF Blog

Comments (none posted)

Electronics

introducing CGRAN: 3rd party GNU Radio application support

Full Story (comments: none)

Encryption Software

M2Crypto 0.19.1 released

Full Story (comments: none)

Games

Humerus 2.0 released

Full Story (comments: none)

Geographical Software

uDig GIS: A First Look (Linux Journal)

Comments (8 posted)

Interoperability

Wine 1.1.6 announced

Comments (none posted)

Mail Clients

Claws Mail 3.6.1 is out

Full Story (comments: none)

Medical Applications

MedinTux needs French to English translators to spread worldwide (LinuxMedNews)

Comments (1 posted)

Multimedia

Elisa Media Center 0.5.14 released

Full Story (comments: none)

Music Applications

Virtual MIDI Piano Keyboard 0.2.1

Full Story (comments: none)

Office Applications

Task Coach: Release 0.71.1 available (SourceForge)

Full Story (comments: none)

Office Suites

OpenOffice.org 3.0 released

Full Story (comments: 26)

Meeks: Measuring the true success of OpenOffice.org

Comments (36 posted)

Science

mpmath 0.10 released

Full Story (comments: none)

Video Applications

Gnash 0.8.4 released

Full Story (comments: 1)

Web Browsers

Firefox 3.1 beta 1 is now available for download

Full Story (comments: none)

Geode in Firefox 3.1: Lost in Linux (Linux Magazine)

Comments (11 posted)

Languages and Tools

C++

dlib C++ Library: 17.10 Released (SourceForge)

Comments (none posted)

Caml

Caml Weekly News

Full Story (comments: none)

Java

IcedTea6 1.3 released

Full Story (comments: none)

Whiteoak 2.1 released (SourceForge)

Comments (none posted)

Python

Mock 0.4.0 released

Full Story (comments: none)

posix_ipc 0.1 is available

Full Story (comments: none)

python-aspects 1.2 released

Full Story (comments: none)

Python-URL! - weekly Python news and links

Full Story (comments: none)

Tcl/Tk

Tcl-URL! - weekly Tcl news and links

Full Story (comments: none)

IDEs

ZK: 3.5.1 released (SourceForge)

Comments (none posted)

Libraries

Pantheios: 1.0.1 (beta 165) released (SourceForge)

Comments (none posted)

Miscellaneous

Bouml: 4.7 released (SourceForge)

Comments (none posted)

Page editor: Forrest Cook

Linux in the news

Recommended Reading

Linux at 17 - What Windows promised to be (the Register)

Comments (7 posted)

Livin' La Vida Linux (ZDNet)

Comments (1 posted)

Trade Shows and Conferences

Linux Summit will preview new advanced file system (SearchEnterpriseLinux.com)

Comments (57 posted)

Companies

Linux opportunity buried in Unix market share data (CNET)

Comments (none posted)

Linux Adoption

Differing trends for Linux on netbooks

Laptop Mag's article Ubuntu Confirms Linux Netbook Returns Higher than Anticipated quotes Canonical's marketing manager Gerry Carr: "“We don’t know what the XP return rates are. But I will say that the return rate is above normal for netbooks that offer open-source operating systems,” Carr echoed. Carr highlighted a few reasons why Ubuntu-running netbooks are returned more often. “Unclear selling is happening, typically online. The customer will get their netbook sent to their home and they imagine to find something like a Microsoft desktop, but they see a brown Ubuntu version. They are unwilling to learn it and they were expecting to have Windows.”"

Comments (6 posted)

Linux in The Economic Meltdown (IT Management)

Comments (none posted)

Linux at Work

Video and photos show Linux booting on the Brazilian voting machines (BR-Linux.org)

Comments (66 posted)

Legal

Beta blockers? (Nature)

Comments (8 posted)

Interviews

Interview: Miguel de Icaza (DesktopLinux.com)

Comments (34 posted)

Resources

Running the Chromium browser on Ubuntu 8.04 with CrossOver Chromium (HowtoForge)

Comments (2 posted)

Reviews

Multi-Head, Multi-User Killer GNU/Linux App Languishes (LinuxMedNews)

Comments (7 posted)

Miscellaneous

Ideas Wanted: Creative Heat Recycling from Servers (Linux Journal)

Comments (21 posted)

Top Agenda Setters for 2008 (Silicon.com)

Comments (none posted)

Page editor: Forrest Cook

Announcements

Non-Commercial announcements

EFF challenges bogus patent on Internet music files

Full Story (comments: none)

The Linux Foundation's FOSSBazaar Nearly Doubles its Membership

Comments (none posted)

Commercial announcements

ASF announces Apache Tuscany 1.3.2

Comments (none posted)

AtMail releases AtMail 5.5

Comments (none posted)

Atmel, Cisco and SICS collaboratively release uIPv6

Comments (none posted)

GarageGames Looking to Fund New Studios in Austin, Vancouver, and Seattle

Comments (none posted)

ITTIA introduces compact DBMS for portable media applications

Comments (none posted)

Cloud Computing Adoption Model unveiled by rPath

Full Story (comments: none)

New SSH Tectia Solution for Linux on IBM System z

Full Story (comments: none)

OpenLogic expands professional service offerings

Full Story (comments: none)

SourceForge.net launches Hosted Apps

Full Story (comments: none)

Timesys Announces Embedded Linux ReadyKits

Comments (none posted)

New Books

Maven: The Definitive Guide - New from O'Reilly

Full Story (comments: none)

Packt Publishing releases CUPS Administrative Guide

Full Story (comments: none)

Resources

Linux Foundation releases LSB 4.0 beta

Comments (10 posted)

Contests and Awards

USAID Launches Open Source Development 2.0 Challenge

Comments (none posted)

Meeting Minutes

Perl 6 Design Minutes (use Perl)

Comments (none posted)

Calls for Presentations

CISSE 2008 - paper submission deadline extended

Full Story (comments: 3)

FOSS.IN/2008: CfP announced

Full Story (comments: none)

Linuxchix Miniconf @linux.conf.au 2009: Call for Presentations

Full Story (comments: none)

O'Reilly RailsConf 2009 opens Call for Participation

Full Story (comments: none)

O'Reilly Where 2.0 2009 Conference opens call for participation

Full Story (comments: none)

Upcoming Events

MySQL European Customer Conferences 2008

Comments (none posted)

European PGDay 2008

Comments (none posted)

Web 2.0 Expo Europe Examines Key Industry Themes

Full Story (comments: none)

Events: October 23, 2008 to December 22, 2008

If your event does not appear here, please tell us about it.

Web sites

KDE Launches User Forum

Full Story (comments: none)

Audio and Video programs

O'Reilly Webcast - When Times Gets Tough, the Tough Get Tuning

Full Story (comments: none)

Page editor: Forrest Cook