Red Hat's directory server

The larger proprietary software vendors - Microsoft, Sun, Novell, etc. - have long offered directory server products aimed at large network ("enterprise") deployment. These products not only make basic user information available network-wide; they can also be used to distribute a wider array of information. Directory servers are a useful and necessary tool, and the competition in this area is fierce.

Red Hat has set itself up to compete directly with the other "enterprise" software companies. To that end, Red Hat has put together a number of valuable products and services, but, so far, it has not been able to offer a directory server as part of its solution. That gap in Red Hat's offerings has increasingly looked like a liability, especially as Novell increases its efforts to compete in the same space. So Red Hat needed a directory server. It found one, some time ago, when it acquired many of the remaining bits of Netscape from AOL. Since the acquisition, however, little has been heard about the former Netscape's offerings.

Until now. On June 1, Red Hat announced the availability of its directory server product. The (now) Red Hat Directory Server is fast, with an impressive array of capabilities; for the full list, see the product sheet [PDF]. The directory server product is sold like Red Hat Enterprise Linux: by subscription. Pricing is not yet available.

The Red Hat Directory Server also resembles RHEL in another way: it has a Fedora equivalent. The Fedora Directory Server Project is where the development work will be done; the site offers source, documentation, mailing lists, etc. It is, in other words, just another free software development project.

At the Fedora site, one can see that, in fact, not all of the directory server code has been released - yet. The server itself is available under a special GPL+Exception license. The code is generally governed by the terms of the GPL, with the exception that plugin modules can remain proprietary. Those modules, however, must restrict themselves to a carefully-specified set of interfaces; anything linking to any other part of the server can only be distributed under the GPL. Other parts of the system - the management console and admin server components - remain non-free, though they are available in binary format. Red Hat plans to free that code as well, but some work is involved; those components are written in Java, and do not play well with the free Java implementations.

The Fedora project has some ambitious goals; the best description of what they have in mind can be found in Christopher Blizzard's weblog. The project claims to want to bring in outside developers, and to make them "feel that they are equals." Given all that the directory server hackers want to do, they will almost certainly need some help from outside. Consider this:

To some readers, this vision sounds like the Windows registry - except that it's a nightmare, monster central registry for thousands of users. The "everything lives in the directory server" approach clearly will not be for everyone. But, for people wanting to create a single, integrated environment across a large organization, this vision will have some appeal. It is truly a view of the network as a single, large computer, with a minimum of boundaries. It promises to reduce the cost of administering large numbers of systems. One can see why Red Hat thinks it needs to go in this direction to remain competitive in the future.

High-end directory servers have, so far, been the domain of expensive, proprietary software. The freeing of the Netscape server, if handled well, could bring an end to that era. So this move by Red Hat is important, and deserving of support. High-quality free infrastructure is a good thing.

Comments (24 posted)

A survey of RSS aggregators

Over the years, the proliferation of news sites, weblogs and other sites with daily updates has made it nearly impossible for the average user to visit every site of interest in a timely fashion. For those of us who want or need to keep informed on a variety of topics, RSS, RDF and Atom feeds have become a nearly indispensable tool to skim the headlines for many sites at once without having to spend more than an hour per day clicking through bookmarks. However, this raises the question of how to manage news feeds effectively.

There are a fair number of RSS aggregator projects on Freshmeat, but we decided to limit our scope to applications that are fairly mature, have been updated recently (many RSS aggregator projects listed on Freshmeat have not been updated in years) and run on the desktop. In particular, we were looking for aggregators that handle a large number of feeds, make it easy to manage feeds and integrate well with the Linux desktop and the average user's workflow.

For some time now, this writer has used the Bloglines service to browse RSS feeds. For this article, the feed list from Bloglines, containing about 130 RSS/RDF and Atom feeds, was exported as an OPML file and imported that into each of the aggregators to see how they performed.

RSSOwl

The first aggregator we'll look at is RSSOwl. This aggregator is written in Java, using the SWT graphic library. RSSOwl has a fairly flexible interface, and opens up tabs for each new feed that the user opens from the list of "favorites."

There are a few interesting features in RSSOwl. First, RSSOwl has an export feature, which can be used to export a feed or individual article to PDF, Rich Text (RTF) or HTML. This might be handy for saving feeds and entries for later. RSSOwl also supports AmphetaRate, a centralized ratings service for rating articles found in news feeds.

Oddly, it seems to display feeds as plain text rather than rendering the HTML. We're not sure if this is a glitch in RSSOwl or if we missed a step in setting it up. Otherwise, RSSOwl's performance was very good, and it handled a large number of feeds without any problems.

Snownews

The Snownews aggregator is unique in this list, because it's not a graphical application. Snownews is a console-based feed-reader that uses ncurses, and is a fairly straightforward application with few frills.

Snownews does not support OPML directly, but there is an "opml2snow" script that comes with Snownews to convert OPML into the format that Snownews likes. It's a little more of a hassle than the easy-import offered by other readers, but it gets the job done. Snownews displays headlines and feeds inline. To follow the feed URL, one must use an external browser. It works fairly well with GUI browsers, but works best (at least in this writer's opinion) with a text-mode browser like w3m or Lynx.

It's probably not going to be the first choice for most users, but those who prefer browsing in w3m or other text-mode browsers should definitely check it out.

Liferea

One reader that seems to be getting a lot of attention at the moment is the Linux Feed Reader, Liferea. This is a nicely-designed newsreader that's easy to use. It imported our OPML file with no problems, and gives the user the option of rendering HTML with Mozilla or GtkHTML2. It spawns an external browser for full articles rather than displaying them within the Liferea window. This works well if you prefer to browse content in Firefox, Epiphany or another browser, but we would like it if Liferea would give the option of displaying the entire article inside Liferea itself.

One interesting feature with Liferea is the ability to create a new feed from a Feedster search. This can be quite handy if you're interested in finding feeds on a specific topic from a variety of sources.

If one wishes to be alerted, or interrupted, with updates from subscribed feeds, Liferea has a feature that will pop up a notification window at regular intervals with new headlines. We enabled this feature briefly, but turned it off after an hour or so, finding it quite distracting.

We also found Liferea to be a bit less than stable, at least the 0.9.0 release that is available in Ubuntu Hoary. Liferea crashed a few times when doing something as simple as deleting a feed. Overall, its performance was quite good, and the interface is excellent -- but it might need to stabilize a bit before being our first choice of the available aggregators.

Blam

Blam is a aggregator written in C# using Mono and GTK#. It's a little more basic than Liferea or Snownews, but it serves well as a basic newsreader. Headlines and summaries are displayed within Blam, but it requires an internal browser to follow links.

At first, Blam would not import the OPML from Bloglines. We tried subscribing a few feeds manually and then exporting Blam's list to OPML to find out what was different. The difference was that Bloglines uses "title" for the name of each feed, and Blam expects "text" -- after doing a quick search and replace in Vim, changing "title" to "text," Blam imported the list of feeds just fine.

Blam is a good choice for users who want a very basic newsreader that's fast and light.

Akregator

KDE users are probably already familiar with Akregator. This reader uses KHTML to display full articles in tabs within the Akregator interface, at least by default. Akregator can also be configured to use an external browser for those who prefer Firefox or another browser to Konqueror/KHTML.

For users who prefer Konqueror for Web browsing, Akregator is an excellent choice. Konqueror auto-discovers feeds on pages, and makes it easy to add those feed subscriptions to Akregator. Akregator has fewer frills than Liferea or RSSOwl, but it integrates very well with KDE and performs well.

Firefox and Thunderbird

We should also mention Firefox and Thunderbird. While not dedicated aggregators, both applications allow users to read and manage news feeds. However, they lack a number of features that many users would want, at least natively. The advantage of using Firefox as an aggregator is that Firefox makes it very easy to create a "Live Bookmark" to subscribe to feeds, when the browser discovers the feed in a page.

If Firefox doesn't detect the feed, that complicates things greatly. Firefox supports adding a bookmark manually, but does not support adding a feed manually. The Live Bookmark also doesn't allow the user to preview the content or full text, just the headlines from a feed. Firefox doesn't support importing OPML files natively, so users with large subscription lists would have to go through a lot of work to re-subscribe to sites using Firefox.

Of course, it is possible to extend Firefox's capabilities with extensions. We tried the Sage extension with Firefox, and were quite pleased with it. The Sage extension adds a sidebar to Firefox much like the Bookmarks and History sidebars. There are two panes in the sidebar, a list of subscriptions and lower pane that lists headlines from the selected feed.

The integration with Firefox makes it a convenient aggregator for those of us who use Firefox exclusively or extensively. Sage had no problem importing the OPML list exported from Bloglines, and its performance was quite acceptable. There are a number of other news reading extensions for Firefox for those who are interested.

Thunderbird, by itself, is also limited in its abilities to import and manage feeds. For users who spend a lot of time in their e-mail client, and who have a fairly limited number of feeds, it would work well -- but this writer would not like to have to import 100 or more feeds using the "Manage Subscription" dialog for Thunderbird. The advantage to using Thunderbird for feeds is the ability to mail links from subscribed feeds.

We found the Forumzilla extension for Thunderbird, which adds OPML import and other features to Thunderbird. Unfortunately, it consistently crashed Thunderbird when trying to import the OPML exported from Bloglines.

Summary

After spending time with each of these aggregators, this writer prefers Liferea and Sage, though any of the aggregators would do in a pinch. Given the variety and maturity of the various options, Linux users should not have much trouble finding an aggregator that works well for them.

Comments (11 posted)

IP Software Compliance Tools -- Who Needs Them and Why?

Now there is a second such product, Palamida's IP Amplifier, and it's clear there is a market for such products. Cisco, for one, has just signed on with Palamida. Who really needs products like this, and why? And is there a difference between them?

Who Needs Software Compliance Tools?

Now that Free and Open Source software has hit the mainstream of the enterprise, businesses need to be certain that they are not taking on legal liabilities with the code. There are many licenses, and making sure a company is abiding by them all is complex. That's one reason you are hearing so many voices calling for simplifying and settling on fewer licenses. But it goes deeper than that.

"Everyone who distributes software should know what goes into it," attorney Lawrence Rosen explains. "And almost everyone who distributes software wants to comply with the relevant licenses. Most reputable software-based businesses recognize that playing fast-and-loose with copyright claims isn't worthwhile."

While most businesses today are pleased to adopt and incorporate open source products into their products and services, they want to know what licenses apply so that they can comply with the terms.

"That's what Black Duck and Palamida make possible," Rosen adds. "A distributor or user can know what open source software is in its own software and act accordingly, early in the cycle. It's now possible to evaluate license compatibility for specific component sets and plan appropriate combinations for use in products to be developed."

Unfortunately, developers sometimes use GPL code (or other licensed FOSS code) without telling management, thinking it's public domain. It isn't. And with outsourcing, sometimes developers are in other countries that may have more relaxed views on copyright and this can cause problems. So when developers let things happen they shouldn't (such as making unauthorized copies or derivative works), companies have an automated way to catch some of that and react appropriately before much bigger problems can develop.

Software practices are also changing. Application development today is becoming more like an assembly line, more a matter of assembling bits of code from open source projects and from outsourced firms and incorporating them into proprietary products than handcrafting 100% custom software. This isn't a bad thing, because it makes it possible to avoid having to reinvent the wheel -- one of the advantages of Open Source -- but it also means that checking on license terms and making sure you are complying with them all is vital to the process.

And there is no doubt that enforcement of GPL violations is increasing, as Fortinet learned recently when a German court banned their U.K. subsidiary from further distribution of their firewall and antivirus products until they complied with the GPL, which they promptly did.

Then there is the Sarbanes-Oxley Act [PDF], and its requirements for IT audits.

"The SEC’s new rules on heightened corporate responsibility for public company reporting known as Sarbanes-Oxley require public companies to abide by internal procedures that are sufficient to provide reasonable assurance that the financial and non-financial information required to be disclosed in its periodic and current reports is accurate," says Karen Copenhaver, executive vice president and general counsel for Black Duck Software.

"Specifically, Sarbanes creates two new corporate governance requirements: assessment of internal controls over financial reporting (required by section 404 of the Act), and heightened corporate responsibility for financial reports (required by section 302 of the Act). It would be hard to overestimate the burden that compliance with these new rules has placed on public companies in the first few years since their enactment.

"Even before Sarbanes, public companies were required to address intellectual property matters in their current and periodic reports. A reporting company traditionally discloses the importance of its intellectual property assets to the company’s business and any third-party intellectual property encumbrances on the company’s ability to conduct its business. To the extent that a failure to identify or comply with third party license obligations has an effect on the accuracy of any of this information, public companies will be concerned about compliance with their obligations under Sarbanes."

Obviously, Sarbanes-Oxley has upped the ante considerably. But most businesses and developers want to do the right thing anyway, apart from outside pressures. The tools don't set policy for a company, but they surely make it easier to make sure policies are observed.

What Do the Tools Offer?

Before automated software compliance tools were available, due diligence in checking software for infringing code was done by assigning the tedious task to senior software programmers in the company, who, together with lawyers laboriously looked through the code. The problem with such a system, aside from the time it required and the drudgery, is that no one person knows all the Free and Open Source projects available by sight, let alone all the proprietary products you are not allowed to see without complex legal arrangements.

Automated systems are an obvious answer. What they provide is a Google-like collection of code. They've collected it all for you. Both tools scan for copyright infringement and can spot more than verbatim matches. But they do more than scan. Palamida says its IP Amplifier product automatically detects, manages and reports on the third party, commercial and open source components that may exist in their software code base. It consists of two key modules -- the Compliance Library and the Detector. Using an automated collection system, the Compliance Library contains billions of source code snippets and millions of files of the most commonly used open source projects found in the market.

Palamida: "The Palamida IP Amplifier uses three different types of technologies to automate detection, source code fingerprinting, file digest matching, and for Java files, namespace matching. This means the software is able to conduct both source code and binary code analysis. So for companies whose developers download whole libraries, compiled code, XML files, icons, text files, and include those resources into their code base, the software will still detect their usage even though their source code is not available and even if we do not have the components listed in our database."

Next, there is a "layer of analysis that is beyond just code matching for reduction of false positives. We call this technology CodeRank™. CodeRank™ looks at the code matches and evaluates the results on multiple levels, including uniqueness, coverage and clustering. How unique is that match to what is in the Palamida database? How much of a customer file matches a file in Palamida’s database? How dense are the matches – do they look like a continuous cut and paste or does it look like two engineers coded against the same API?"

After their software evaluates the code matches, Palamida assigns a CodeRank number to the matches; the higher the CodeRank number the higher the chances of copying. In the scan results, users will see a list of all code that has matches and a list of all the third party products that they most likely came from, with the most likely on top.

Reports identify all components that include open source and list their licenses, text and license information, in addition to the CodeRank. All the information and data is exportable in XML data format, allowing users to create custom reports, as well as via HTML reports.

Black Duck too offers a great deal more than just code scanning. Black Duck's Copenhaver: "We do more than just scan code. Our product provides a full suite of services covering project planning, code analysis and detection, license analysis and management, auditing and archival capabilities for the complete life cycle of software projects.

"From an open source perspective," Coperhaver adds, "we help developers manage the origins and obligations of code that they use so they can meet the expectations of the industry and community. But everything we do works for both open source and proprietary or commercial code. Users can add code prints and licenses into the system to manage their internal proprietary code along with open source.

"Our product helps people manage the introduction of licensed materials into their code bases, understand the obligations associated with that code (and combinations of components from different sources), provide an environment for controlled remediation of issues that arise and create an archivable record of the actions that were taken by the team along the way. Our products are designed to bring together developers, lawyers and business decision makers into a collaborative environment."

Black Duck offers an analysis 'engine' that processes licenses at a detailed level and alerts users to license conflicts and obligations of both software source and binary components and their combinations. The ProtexIP Knowledgebase contains detailed breakdowns of 500+ software licenses for automated comparison of license terms and notification of collective obligations, and the data is remotely updated frequently with new licenses as they come to market. It recently added what they call Custom Code Prints, which gives ProtextIP support for proprietary source code.

Palmida claims a database of 40,000 of the most commonly used OSS projects and their associated licenses, monitoring more than 38 million open source files and billions of source code snippets. The Knowledge Base also contains all pertinent information regarding the open source projects: name, version number, project name, licensor, licensor information (when available), license, license text, and project URL, all using an automated collection toolset that incorporates information on all the new projects released on the major OSS repositories for real time updates.

The Palamida database takes up less than 10 Gb disk space, thanks to a compression algorithm, and it's all kept on a customer's own servers, behind their firewall. Its code is written in Java. IP Amplifier can be configured to search daily or weekly and has a set of configuration tools to integrate it into build systems.

Are There Any Differences?

The biggest differentiator is cost. IP Amplifier 3.0 is licensed on an annual subscription basis, for unlimited number of users, at prices that begin at $50,000 and go up to $250,000 per year, depending on the customer's development environment. There is a 30-day Free Trial offer.

Black Duck now offers two options. You can pay an annual licensing fee for its multiuser ProtextIP product, at $25,000 per year, and then add additional charges based on the amount of code you have. Or, you can use their new hosted ProtextIP/OnDemand product, an online system for a single user, single project, 90-day sessions, for which you pay based on the amount of code you wish to scan. It costs $3,000 for 10 MB of code and costs scale up to $25,000 for 100 MBs. A company thinking of acquiring another might wish to use the online tool, rather than purchase more costly version.

Both products still require human analysis, naturally. There can be false matches, if two independent developers happen to write software that is very much the same, even if there has been no copying, just because there are only so many ways of writing the same instruction. Both tools provide not only identical matches but also flag similarities in your source code to others' programs that are worth your further investigation and list issues for review. It's important to realize, however, that the tools scan and analyze copyright issues and licensing issues, not patent infringement. That is an entirely separate ballgame. But for what they are designed to do, unquestionably they have simplified, organized, and improved the due diligence process.

Comments (12 posted)

A Look at The Onion Router (Tor)

Last week we promised a look at Tor, a system for anonymous Internet communication, primarily developed by Nick Mathewson and Roger Dingledine. Current development is supported by the Electronic Frontier Foundation (EFF), but Tor was originally developed as part of the U.S. Naval Research Laboratory's Onion Routing program.

As the Tor web page explains, Tor is a "toolset for a wide range of organizations and people that want to improve their safety and security on the Internet". What does that mean? In a nutshell, Tor is a client/server application that anonymizes traffic by routing it from the client through a series of nodes to hide the origin of a request. It can also be used to protect services against denial of service attacks and the like by hiding their origin.

Tor routes traffic through nodes that "know" about the previous node and the next node -- but not the rest of the network. By routing traffic through a series of "onion routers" Tor makes it difficult for the receiver, observers and even other Tor routers to detect the source of traffic. A more complete description of Tor's design can be found in the design paper; a protocol specification is also available for those who wish to build compatible software.

Tor works as both a server and as a client. By default, Tor runs as a client only, but it can be configured to allow other users to connect to your system as a Tor node. In addition, Tor can be used to run "hidden" services that do not reveal your IP address to others at all. The "hidden wiki" maintains a list of hidden services that users can see as an example. Finally, it's possible to set up one's own Tor network that does not interact with the public Tor network, for those who want to test the protocol but may lack access to the Internet.

To achieve best results, one may need to use Tor in conjunction with other applications. For example, users who wish to browse anonymously would use Tor in conjunction with Privoxy. Other applications may require use of tsocks or ProxyChains.

To see what Tor had to offer, we installed it on a Ubuntu Hoary machine, along with Privoxy, tsocks and ProxyChains. Configuring services to work with Tor is not terribly difficult, and there is a relatively detailed HOWTO for users who wish to configure specific applications like Gaim, X-Chat, SSH or BitTorrent with Tor.

It should be noted that using Tor can have an impact on performance for client applications. Using Tor and Privoxy together for browsing, for example, introduced a notable lag. Firefox users may be interested in using the SwitchProxy Tool extension to switch Proxy use on and off, reserving Tor for specific sites rather than for all web browsing. Users should also be prepared for some odd behavior on some sites -- for example, we kept being redirected to country-specific versions of Google, rather than Google's main site, when using Tor and Privoxy. Tor itself didn't seem to have much of an impact on system performance overall.

Tor is not completely foolproof. It could be possible for someone who's running a Tor server to modify Tor or use other software to monitor traffic going through the server. Traffic coming out of the "exit node" (the last hop in the Tor "circuit") is not encrypted, so a malicious user could set up a Tor server and browse traffic coming out of their machine. (It is possible to specify your exit node in the Tor configuration.) There are also potential JavaScript issues, and there are other ways to analyze traffic that passes through Tor.

Interested users should also have a look at the EFF's legal issues page about Tor. Though Tor can be used for things like BitTorrent, it is not designed to assist copyright infringement or other illegal activity.

There is still a lot of development ahead for Tor, but it is definitely worth a look for users who are interested in anonymous communication on the Internet. Users with bandwidth to spare are also encouraged to set up and run a Tor server to help test its scalability and to help provide a larger Tor network. See the download page for Tor packages and source code.

Comments (9 posted)

apache-utils: htpasswd buffer overflow

Comments (none posted)

gxine: format string vulnerability

Comments (none posted)

ImageMagick: xwd coder denial of service

Comments (none posted)

Mailutils: multiple vulnerabilities in imap4d and mail

Comments (none posted)

Kernel release status

The current -mm tree is 2.6.12-rc5-mm2. Recent changes to -mm include the pluggable congestion avoidance modules patch, some filesystem namespace patches, some scheduler tweaks, and lots of fixes.

The current stable 2.6 kernel is 2.6.11.11, released on May 27.

The current 2.4 kernel is 2.4.31, released by Marcelo on May 31. 2.4.31 contains quite a few fixes and some driver updates, but new features are no longer being added to 2.4.

Comments (none posted)

The ongoing Philips webcam driver saga

Comments (5 posted)

Time to remove LSM?

The LSM patch ran into some difficulties on its way into the kernel, but it is now an established part of the internal API. So some developers were surprised recently when James Morris suggested that perhaps the time has come to remove the LSM framework. His arguments are simple: there is only one serious module using the LSM framework in the intended manner, while unrelated projects are trying to use it in inappropriate ways.

James asks: rather than forcing SELinux to conform to a general-purpose API (of which it is the sole user), why not just wire SELinux directly into the kernel, get rid of LSM, and be done with it?

SELinux is not truly the only security module out there, of course. The kernel includes a couple of other modules: a reimplementation of the capabilities mechanism and "root plug," a module which prevents processes from running as root unless a specific USB device is plugged in. There are out-of-tree modules, such as the BSD securelevels patch and Trustees Linux. The Immunix (now Novell) AppArmor product includes a module which uses the LSM framework. AppArmor is a proprietary offering, but the security module portion of it is GPL-licensed (as is necessary, since the functions for loading security modules are exported GPL-only).

There does not appear to be a groundswell of support for the idea of removing the LSM framework from the kernel at this time. That could change over time, however: increasingly, out-of-tree code is held to be irrelevant when decisions are made. If SELinux remains the only significant in-tree user of the LSM framework, LSM will look like useless baggage to more and more developers. If there are security modules out there which are reasonable alternatives to SELinux, their developers may want to think about getting them into the mainline sometime in the not-too-distant future.

Comments (5 posted)

Files with negative offsets

As it turns out, there is a situation where a negative file position makes sense. Special files (such as /dev/mem and /dev/kmem) provide a window into the system's main memory. The "position" within these files corresponds to the address of the memory of interest. The interesting thing is that, on the x86_64 platform, addresses can be negative numbers.

This comes about as follows: this architecture currently uses a 48-bit address space. The hardware sign-extends the uppermost bit, however, so any address with that bit set will turn into a negative number. The x86_64 Linux port uses the upper bit to mark kernel space, so kernel addresses are, in fact, negative. A quick look at /proc/kallsyms confirms this:

    ffffffff80100000 T startup_32
    ffffffff80100100 T startup_64
    ffffffff801001a0 T initial_code
    ffffffff801001a8 T init_rsp
    ffffffff801001b0 T early_idt_handler
    ...

The end result is that using /dev/kmem on an x86_64 system is difficult; any attempt to seek into kernel space will yield an error.

The clear fix is to modify the VFS layer to let negative file positions be passed through to the underlying filesystem or device driver. The problem with doing that in a general way, however, is that not all code (especially in drivers) is prepared to deal with a negative offset. Suddenly exposing that code to negative offsets could open up no end of bugs and security problems. So the real solution, as worked out by Al Viro and Linus Torvalds, is to add a new flag for the file structure called FMODE_ANY_OFFSET. This flag can only be set within the kernel; user space has no access to it. So the /dev/kmem driver will be able to set the flag and work with the full range of offsets, but, for the rest of the system, nothing will change.

Comments (10 posted)

The beginning of the realtime preemption debate

The realtime preemption patches attempt to provide a guaranteed maximum response time for high-priority user-space processes - just like a "real" realtime operating system would. This goal is achieved by making everything in the kernel preemptible. No matter what the kernel is doing on a given processor, if a higher-priority process becomes runnable, it will be scheduled immediately. Many changes are required to make the whole kernel preemptible; the core parts are:

• New locking primitives. The spinlocks used by the kernel can cause any number of processors to stall while waiting for a lock to become free. Code which holds a spinlock cannot be preempted, or a deadlocked kernel could result. The realtime preemption patches introduce a new mutual exclusion type (the rt_mutex) which does not spin, and, thus, will not stall a processor. The spinlocks and semaphores currently used in the kernel are all converted over to the new rt_mutex type, and all code which runs with spinlocks held becomes preemptible. The rt_mutex type also implements priority inheritance, so that a low-priority process will not block a higher-priority process (for long, at least) by losing the processor while holding an important lock.

• Threaded interrupt handlers. Interrupt handlers can create latencies by monopolizing the processor for long periods of time. The realtime preemption patch moves interrupt handling into kernel threads, which contend for the processor with all other processes in the system. If a certain realtime task is more important than interrupt handling, its priority can be set accordingly.

• Various other mutual exclusion mechanisms, including read-copy-update, per-CPU variables, and seqlocks, require that preemption be disabled. All of these mechanisms are changed for the realtime preemption mode, usually by making them look more like regular spinlocks.

The realtime preemption patch set (at version -RT-2.6.12-rc5-V0.7.47-10 as of this writing) is clearly large and intrusive - it would be hard to make fundamental changes like those listed above any other way. It should be noted that Ingo has gone out of his way to minimize this intrusiveness, however: the patch is written to minimize code changes, and the kernel functions as always if realtime preemption is not selected at configuration time. The merging of this patch set would not force the new preemption model on users.

According to Lee Revell, the realtime preemption patches are already seeing some serious use:

Certainly the discussions that inevitably follow the release of a new version of the patch set indicate that there is an active user community out there. Some members of the community are starting to wonder why the realtime preemption patches have not been merged, and when (if ever) that might change. The biggest reason is that Ingo has not yet requested that the patches be included - though many small pieces and fixes from the realtime patch set have found their way into the mainline. If and when Ingo does push for inclusion, however, there will be some opposition.

To some developers, the realtime patch seems like a set of questionable and widespread changes aimed at the needs of a very small user community. Changing spinlocks into mutexes and moving interrupt handlers into threads are fundamental changes to how the kernel does things with the potential for the creation of subtle bugs and performance problems. Reworking things and adding complexity at that level is not a task that should be undertaken without a strong need - and many developers do not see a sufficiently strong need.

There are some concerns about the performance impact of these changes. Acquiring an uncontended spinlock is a very fast operation; the rt_mutex type, with its wait queues and priority inheritance mechanisms, is bound to be slower. There is some anecdotal evidence that there is a performance hit to realtime preemption, but little in the way of real benchmarking has been done. In any case, the performance penalty should only affect users who have actually enabled the realtime preemption mode.

Finally, not everybody is convinced that the realtime preemption approach can solve the real problem: providing an ironclad guarantee that a realtime process will be scheduled within a given maximum latency. Ingo believes that this guarantee can be made by eliminating all code within the kernel which can delay a reschedule; others feel that, to make a guarantee that can truly be trusted, the entire kernel must be audited and verified. They have a point: how strong a guarantee would you want before running realtime Linux in your car's braking system?

Those who want true realtime guarantees, along with developers who simply do not want to clutter the kernel with realtime mechanisms, argue that a different approach should be taken. The most commonly suggested alternative is RTAI-Fusion, which works (at its core) by interposing a "nanokernel" between Linux and the bare hardware. The nanokernel guarantees latency by taking the lowest-level scheduling decisions out of the Linux kernel's hands; it is kept small and easy to verify. Another project taking a similar approach is Iguana, which is based on the L4 microkernel.

Since the realtime preemption patch is not being proposed for merging at this time, no decisions are likely to result from the current, lengthy discussion. If Ingo has his way, there may never be one big decision; instead, pieces of the patch will be merged if and when it makes sense.

There may be some interesting realtime-related sessions at next month's Kernel Summit in Ottawa, however. Meanwhile, should anybody wish to plow through the entire thread on linux-kernel, here is the starting point.

Comments (9 posted)

KANOTIX - The Knoppix Improved

• KANOTIX has improved considerably Klaus Knopper's hardware auto-detection and auto-configuration module. Users report that KANOTIX is one of the very few distributions that boots into a graphical desktop with touchpads on most notebooks working straight away. Many digital cameras, PDAs, and storage devices are configured on the fly, while wireless networking, both for network access and printing, is also ready for use right after the boot. Granted, some of these modules are experimental and might cause problems in certain hardware configurations - this is why Klaus Knopper has been reluctant to include them in Knoppix. But for many novice and medium-skilled Linux users such superb autoconfiguration of their hardware is a blessing, and a welcome surprise, compared to most other distributions.
  
  
• KANOTIX has been optimized for notebooks. We have mentioned the automatic configuration of touchpads, but that's not the whole story. The distribution also includes PowerNowd, a daemon which works as a client of the CPUFreq driver and which has an ability to adjust the speed of the CPU depending on system load.
  
  
• KANOTIX can be installed on one's hard disk with a simple graphical installer. While this program was originally developed for Knoppix, the KANOTIX maintainers have made a number of behind-the-scenes improvements. As a result (and in the true spirit of the GPL), the KANOTIX installer is now used by both KANOTIX and Knoppix, where it has been given preference over the original installer.
  
  
• As with most live CDs nowadays, KANOTIX also provides a way to save user's configuration and preferences to a variety of storage media, including floppy disks, USB pen drives, and hard disks. An interesting variation of this task is that if the settings are saved to a hard disk partition, specifying "myconfig=scan" at boot time will make KANOTIX search all hard disks for the settings file and load it automatically, without the user having to remember the location. Needless to say, the ability to set a persistent /home directory is also present in KANOTIX.
  
  
• The latest version of KANOTIX uses the Unionfs file system. While this is also true for the more recent releases of Knoppix, the KANOTIX developers have made a number of unusual enhancements to the concept. Firstly, Unionfs is optional; to activate it, "unionfs" has to be added to the boot prompt. However, if the KANOTIX hardware detection program detects the presence of ATI or NVIDIA drivers, it will enable Unionfs automatically - this way the users can easily install any proprietary ATI or NVIDIA graphics drivers, if they so desire. Unionfs can also be loaded either as read-only (unionro) to prevent files from being overwritten, or as read-write (unionrw), and it can even be activated after the CD is fully booted, although this will only set the Unionfs into the home directory.
  
  
• KANOTIX comes with a selection of applications that is probably better suited for an average home user that those included in Knoppix. Firefox has been the default browser for several releases and many will find the presence of the Guarddog firewall or the Skype telephony application a pleasant surprise. Firefox also comes pre-configured with the Java plugin. The best of all, however, is the ability to install more applications through klik, a sophisticated web-based interface for installing software on Knoppix and KANOTIX. New programs can be installed either with a single mouse click while browsing the applications on the above-mentioned web site, or by typing "klik://program_name" in Firefox. The range of available software is surprisingly large, including many non-free programs, such as the Flash plugin, Real Player, Acrobat Reader, NVIDIA and ATI drivers, just to mention the more obvious ones. Beta versions of certain popular software packages, e.g. OpenOffice.org 2.0, are also provided for one-click download and installation.
  
  
• The developers of KANOTIX have also written a large number of specialist bash scripts for various tasks. These are located in /usr/local/bin and although they are not well commented (or documented), most of them have self-explanatory names. The scripts vary from configuring hardware drivers to starting services, fixing problems, and installing software.
  
  
• A 64-bit edition of KANOTIX has also been released. This is one of the very few live CDs providing native support for the AMD64 and Intel EM64T processors. The live CD is based on the "pure64" branch of Debian Sid, with a compatibility layer to run certain 32-bit applications, such as OpenOffice.org.

Comments (4 posted)

Preparation of the next stable Debian GNU/Linux update (IV)

Full Story (comments: none)

Debian sarge release update

Full Story (comments: 9)

Cybernet Systems Sponsors the NetMAX Desktop Project

Comments (none posted)

Trustix Secure Linux now available via BitTorrent

Full Story (comments: none)

Two new Gentoo based Live CD distributions

Comments (none posted)

Debian Weekly News

Full Story (comments: none)

Gentoo Weekly Newsletter

Full Story (comments: none)

DistroWatch Weekly, Issue 102

Comments (none posted)

Fedora updates

Comments (none posted)

Trustix Secure Linux updates

Full Story (comments: none)

Linspire 5.0, The Linux Desktop For The Masses (LinuxElectrons)

Comments (none posted)

My Workstation OS: Scientific Linux (NewsForge)

Comments (none posted)

Review: FreeBSD 5.4 (NewsForge)

Comments (none posted)

Anyterm: A Terminal Anywhere

    * Some Javascript on a web page.
    * An XmlHTTP channel to the web server.
    * An Apache module that receives the XmlHTTP requests and feeds them
      to an emulated terminal, and thence to a shell or whatever.

The how it works document sheds light on the internal operation of an Anyterm session and the deployment document describes a number of possible configuration arrangements. The documentation also addresses a number of potential security concerns when running Anyterm.

Anyterm stable version 1.0 and development version 1.1.0 were just announced: "This week the stable branch has reached the milestone of version 1.0, as I think that this is now good enough for widespread use. There's also a development branch where I'll be adding more experimental features, starting with WAP support in version 1.1.0 which was released today. So you can now get a shell prompt on your mobile phone. Some work is needed to make it useable though. Future plans include merging my QWAZERTY keyboard-layout mapping code."

Dependencies include version 2 of the Apache web server and the ROTE terminal emulation library. Anyterm development is Debian-based, your editor was able to get Anyterm to build on a Fedora Core 3 system by adding some file paths various lines of several include files. The installation instructions provided sufficient information for getting the software up and running.

The configuration instructions bring one issue to light: "If you're using a system with SE-Linux security features, such as Fedora Core 3, you may find that they prevent anygetty from invoking /bin/login. This probably just needs a slight change to a configuration file somewhere to make it work; if someone knows what is required please get in touch." A bit of SE-Linux configuration knowledge would be a useful addition to the documentation.

To get a look at Anyterm in action, you can try running the Tetris clone "bastet" from the Anyterm web site.

Comments (1 posted)

Mailman 2.1.6 Released

Full Story (comments: none)

popa3d 1.0 announced

Full Story (comments: none)

Knettools 1.0 (stable) released

Comments (none posted)

OpenSSH 4.1 released

Full Story (comments: none)

Twisted 2.0.1 Released

Comments (none posted)

CUPS 1.2.x Weekly Snapshot, r4528

Comments (none posted)

SSL-Explorer 0.1.11 released! (SourceForge)

Comments (none posted)

Caravel CMS version 2.3 released (SourceForge)

Comments (none posted)

SchoolBell 1.1 Released

Full Story (comments: none)

Constructing Services with J2EE (O'ReillyNet)

Comments (none posted)

PythonCAD release 25

Full Story (comments: none)

Eman 1.7 Released

Comments (none posted)

KDE 3.4.1 released

Full Story (comments: none)

GNOME Software Announcements

• Drivel 2.0 Beta 1 (new features)
• G-Inspector 20050529 (new features and bug fixes)
• Glom 0.8.29 (new features and translation work)
• GnomePythonExtras 2.10.2 (bug fixes)
• nautilus-python 0.4.0 (initial release)

Comments (1 posted)

KDE Software Announcements

• K3b 0.12beta2 (bug fixes)
• Kexi 0.9 Stable (new features and bug fixes)

Comments (none posted)

KDE Project Offers Kolab Groupware Services to its Contributors (KDE.News)

Comments (2 posted)

KDE Commit Digest (KDE.News)

Comments (none posted)

mnemo-0.5 released (SourceForge)

Comments (none posted)

Oscilloscope plugin 0.2.0 announced

Full Story (comments: none)

Qocs 0.0.6 Released

Comments (1 posted)

XCircuit 3.3.14 Released

Comments (none posted)

New version of HLA Adventure for Linux/Red Hat

Full Story (comments: none)

OpenEMR 2.7.2 Released (LinuxMedNews)

Comments (none posted)

BEAST/BSE version 0.6.6 released (GnomeDesktop)

Comments (none posted)

OpenOffice.org Newsletter

Full Story (comments: none)

Mozilla Deer Park Alpha 1 released

Comments (4 posted)

Minutes of the mozilla.org Staff Meeting (MozillaZine)

Comments (none posted)

Caml Weekly News

Full Story (comments: none)

Issue Three of The Monad.Reader

Full Story (comments: none)

SBCL 0.9.1 released

Full Story (comments: none)

This Week in Perl 6 (O'Reilly)

Comments (none posted)

Dr. Dobb's Python-URL!

Full Story (comments: none)

Ruby Weekly News

Comments (none posted)

Ruby Weekly News

Comments (none posted)

Bochs 2.2 released (SourceForge)

Comments (none posted)

OProfile 0.9 released

Comments (none posted)

How The Kernel Development Process Works (Groklaw)

Comments (none posted)

Underground showdown (Register)

Comments (8 posted)

Stallman: Nokia's announcement next to nothing (NewsForge)

Comments (13 posted)

India's Upcoming Free Software, Free Society Conference (Linux Journal)

Comments (none posted)

Linux named "platform for the future" by PalmSource keynote speaker (DesktopLinux.com)

Comments (1 posted)

Big-business technologists talk up Linux (Computerworld Australia)

Comments (none posted)

IT giants accused of exploiting open source (News.com)

Comments (2 posted)

Cyber fixers now at PC near you! (Hindustan Times)

Comments (4 posted)

Novell reports loss as older business shrinks (News.com)

Comments (2 posted)

Detroit high school opens its desktops (NewsForge)

Comments (9 posted)

The Meeks shall inherit the earth (GnomeDesktop)

Comments (none posted)

Interview with KDE-PIM Hacker Daniel Molkentin (KDE.News)

Comments (none posted)

Interview with KDE PIM Hacker Cornelius Schumacher (KDE.News)

Comments (none posted)

An evening with the Guru of Python: Guido van Rossum (TuxJournal.net)

Comments (1 posted)

The Daemon, the GNU and the Penguin - Ch. 10 (Groklaw)

Comments (none posted)

Developing GNOME Applications with Java (Linux Journal)

Comments (13 posted)

Rexx: Power Through Simplicity (O'ReillyNet)

Comments (2 posted)

Programming Tools: UML Tools (Linux Journal)

Comments (7 posted)

The Small Computer System Interface (SCSI) standard (IBM developerWorks)

Comments (1 posted)

Linux in Government: Optimizing Desktop Performance, Part III (Linux Journal)

Comments (none posted)

EU puts funds toward global research on open source (News.com)

Comments (4 posted)

Google's summer of code

Comments (1 posted)

BitMover announces BitKeeper to CVS converter

Comments (none posted)

CAC Media Announces Digital Entertainment Devices

Comments (none posted)

EarthLink and Microtel Partner to Offer $69.99 PC

Comments (none posted)

Nokia Makes Donation to GNOME Foundation

Comments (9 posted)

Novell's "Mono Kickstart" program

Full Story (comments: 2)

SCO's second quarter results

Comments (3 posted)

Win4Lin Products Available for Linspire

Full Story (comments: none)

Killer Game Programming in Java - O'Reilly's Latest Release

Full Story (comments: none)

Table of analogs of Windows software in Linux

Full Story (comments: none)

Edition 2 of MyOSS Magazine

Full Story (comments: none)

Open source and the commoditization of software

Comments (8 posted)

First-Round Voting in 2005 Readers' Choice Awards (Linux Journal)

Comments (none posted)

Debian Day at LinuxTag 2005

Full Story (comments: none)

Europython 2005 update

Comments (none posted)

Fedora Talk at USC Los Angeles, CA

Full Story (comments: none)

Joint Call for Participation: IEEE Conference on Web Services

Full Story (comments: none)

Embedded Technology 2005, Yokohama, Japan

Comments (none posted)

Events: June 2 - July 28, 2005

Comments (none posted)

The gift of volunteering

Dear Editor
 
I have been using windows for a number of years and am certified in
system administration. However when I became unemployed I thought this
was an opportunity to research Linux to see what I could use it for and
gain some skills to try to re-enter the labour market.
I needed to start gently with a GUI distro and found IPCop. I was
pleasantly surprised at how useful it is and the quality of the addons
and documentation. However having never been involved in the Open Source
community I did not really know what to expect in terms of support and
contribution to the community.
I continued to explore the Open Source community through IPCop and found
something quite unexpected. By reading the docmentation manuals and
implementing the services of the distro I really began to get an
appreciation of the high standard and professionalism of the work
countless volunteers had put in and this helped me develop an
understanding of what the Open Source community is really all about.
It was at this stage I noticed that a section of the documentation had
not be written up and when I enquired with the community what was
happening with it, I was invited by the Lead Documentation volunteer to
contribute. While still a Linux novice I thought that this would be an
ideal opportunity to learn more about the VPN features.
Through this activity I was able to learn more about system
administration than I did through certification because I found it an
enjoyable form of edu-tainment. I became introduced to other projects
and their developers from different parts of europe. I now regulary
beta test new versions of TauVPN and Linsys, which are Open Source
windows IPSec clients. I enjoyed this so much that I volunteered to
write up a how-to for TauVPN.
After responding to questions on the IPCop forums I noticed that users
were posting queries related to issues arising when using more than one
firewall and were trying to find firewalling technologies that were
complimentary to their IPCop or existing Router.
I started to explore other firewall distributions and firewall
technologies. I had become a CCNA in february of this year and this gave
me an understanding of switching and bridging so I was very interested
when I came across Linux bridging firewall technology. I signed up to
the ebtables mailing list and established that a Linux bridging firewall
can be very effective against attacks when configured without an IP
address. When further combined with an Intrusion Prevention System such
as snort-inline it can detect attacks and prevent them.
Having discovered this technology and how useful it would be to
compliment an existing firewall/router. I then went about researching
an easy to use GUI distro with these features and found distros for
nearly everything but no bridging firewall IPS distro. If anyone knows
of an easy to use GUI bridging firewall distro that I can install on a
older computer then I would appreciate details on it.
 
If there is none I would be interested in collaborating with
other volunteers in the development of an easy to use bridging firewall.
 
Christiaan Theron
christiaan@wonderport.com
 
 
 
 
 
 

Comments (2 posted)

Letter to the Editor: Setback for Linux

Forbes' website's feedback form gives no indication of a successful
submission, so... you all have my permission to publish this:
 
    http://www.forbes.com/business/2005/05/25/cz_dl_0525linux...
 
---------------------8<-----cut-here-----8<------------------------
 
Daniel Lyons seems to be in the habit of being late, and dead wrong.
 
Here, the kernel developers (and specifically Linus) developed a
replacement tool in a matter of days or weeks, which is more closely
tailored to their way of doing things and therefore - at least in
principle - going to be even more effective than BK at maintaining
Linux's pace and security.
 
To the delight of many of the kernel hackers and onlookers, Linux is now
both ahead of the game and free of any proprietary encumbrances.
 
Daniel must have a deep-seated misunderstanding of how Open Source works
or is carrying a chip on his shoulder if he can spin that rather
excellent news to be somehow negative. And it seems to have taken him
more than a month to do it.
 
Perhaps he can do a story on how it took Microsoft years longer than
Linux to run reliably 64 bits wide on AMD's chips? I'd be interested to
see how long it takes him find a negative in that for Linux. It's an
odd spectator sport, I admit, but harmless and predictable.
 
Cheers; Leon
Perth, Western Australia
 
--
http://cyberknights.com.au/ Modern tools; traditional dedication
http://plug.linux.org.au/ Member, Perth Linux User Group
http://slpwa.asn.au/ Member, Linux Professionals WA
http://osia.net.au/ Member, Open Source Industry Australia
http://linux.org.au/ Member, Linux Australia

Comments (none posted)