Leading items

File monitoring with Mortadelo and SystemTap

SystemTap is a tool to help gather information about running Linux systems which has been available for some time now. But applications that use the tool have been few and far between. Mortadelo is a GUI tool that uses SystemTap to observe and record system calls. It is more of a proof-of-concept than a complete application—though it is useful in its current form—but it does start to show some of the things that can be done using SystemTap.

Mortadelo specifically intercepts system calls that deal with accessing files, collecting the arguments to the calls as well the return codes. It is patterned after the Windows Filemon program, which is used in much the same way that a Linux user might use strace—only with a GUI. Problems with permissions or files that do not exist are the kinds of things that Mortadelo could be used to diagnose.

The data collected is displayed in a list in the GUI (shown at left), which can then be filtered using regular expressions to pull out the information of interest. Because it uses SystemTap, Mortadelo gathers information from all running processes at once, allowing the user to choose which parts they are interested in. The filtering is somewhat primitive, in that particular fields cannot be chosen to filter on, but still useful because it searches each entry fully.

System calls that return an error are highlighted in red making it easy to pick them out. By choosing appropriate strings to filter on, all permission errors in the system or every access of a particular filename can be seen. The GUI allows one to start and stop the recording as well as to save the captured data to a file. Each entry includes a timestamp, the process name and pid, the system call, return code, and arguments.

The application is written in C#, using the Mono framework; one of the authors has an interesting weblog entry comparing Mono and Python for developing this kind of tool. Mortadelo's interface to SystemTap is fairly straightforward, it spawns a stap command and sends it the probe points and code via stdin. It then reads the stap output, parsing it and displaying it in the window.

There were some tricks to getting it to build and run, but Eugene Teo's instructions for running it on Fedora 8 were quite helpful. Part of the problem was in getting SystemTap going on the system, which is a problem we have mentioned before. There were some other small hurdles as well, but Teo's hints and proper application of grep were enough to get past those.

Mortadelo's impact isn't so much in the application itself as it is in some of the ideas behind it. Using SystemTap for GUI tools will help users and administrators, especially those who are not command-line savvy. If Mortadelo, or some descendant of it, becomes popular, that will help make SystemTap use more widespread. Distributors will start packaging it in more readily usable forms, perhaps installing it by default. That will in turn help anyone tasked with keeping a Linux system smoothly functioning, whether they are GUI-centric or not.

Comments (9 posted)

Ryzom returns?

Toward the end of 2006, a company called Nevrax went out of business. Nevrax was the operator of an online multiplayer game called Ryzom which had developed a dedicated (if insufficiently lucrative) following. A group of free software developers, former Nevrax employees, and assorted Ryzom players sensed an opportunity here: perhaps the source for Ryzom could be obtained from the failing company and turned into free software. It seemed like a winning solution for all sides: Nevrax's creditors could get whatever money could be raised for the code, Ryzom players would continue to have a game, and the free software community would get an extensive new code base. All that was needed was to convince the relevant bankruptcy court that this was a good idea.

To that end, the Free Ryzom project raised some €170,000 in pledges - an impressive amount of money. The Free Software Foundation offered $60,000 toward this goal. But, in court, another suitor (Gameforge) won out with a plan to keep the game proprietary. The Free Ryzom folks became the Virtual Citizenship Association and faded from view; it seemed that this story was done.

Only it seems it's not done. In February, the project sent out a news update on what had been happening over the past year. It seems that Gameforge stopped paying its employees in June, 2007, and, by August, was not paying its creditors. In October, Gameforge France went back into the bankruptcy process; then, last February, the Ryzom servers were shut down. This particular plan to save Ryzom, it seems, was not as successful as one might have liked.

So it seems that the Ryzom source might, once again, be up for grabs. A news update suggests that the process is moving quickly, but the project could make a try for the code if it is able to come up with a large (at least €230,000) bid in the immediate future. As of this writing, the Free Ryzom folks are examining their options and trying to come to a decision on the best course to take.

There can be no doubt that this code would be a valuable acquisition. Despite the fact that some of the very first multiplayer online games were free software (consider Netrek, for example, which occupied rather too much of your editor's time some 15 years ago, or some of the early MUD and MOO systems), free software does not have much to offer in that area now. The lack of competitive offerings in this area is one of the biggest motivations for people to use Windows. A free Ryzom could be a strong step toward better online gaming with free software.

[PULL QUOTE: One has to wonder why we seem to be unable to put together a competitive game without relying on a huge infusion of source from the proprietary world. END QUOTE] That said, one has to wonder why we, the larger free software community, seem to be unable to put together a competitive game without relying on a huge infusion of source from the proprietary world. There are certainly projects out there; consider Battle for Wesnoth or WorldForge, for example. Wesnoth is an addictive game with basic multiplayer capability and an active developer community, but it is a turn-by-turn game with relatively rudimentary graphics - though the graphics and soundtracks are quite nice by free software standards. WorldForge has high ambitions and a lot of infrastructure, but it never really seems to get out of that pre-alpha state. A look at WorldForge's CVS logs suggests that very few developers are actively contributing to the project.

There are critics of the free software community who would argue that gaming is the sort of program that free software just cannot do as well as proprietary software. A certain amount of planning and direction is required to pull together a coherent virtual world, quite a bit of artistic work (artwork, sounds, etc) is required, and so on; a project without a business-based revenue stream just cannot compete in this area. There might be some truth to this claim - but not that much. When one looks all all that we have accomplished, it does not seem like an online multiplayer game - challenging though it might be - should be beyond our capabilities.

What seems more likely is that we just haven't gotten the project management right yet. Anybody who has hung around with people who are interested in computing knows that game playing is certainly an itch that many feel the need to scratch. We just haven't yet made it easy enough for that scratching to happen.

What's needed is a relatively simple core upon which people can easily create virtual worlds. It should be straightforward for people who are not developers - artists, musicians, script writers - to contribute to the system, and their contributions should be made welcome. The desktop projects have had a certain amount of success in bringing in non-developer contributors; a look at how they have done that could be worthwhile.

Arguably, we should have most of the pieces we need. Battle for Wesnoth has shown that it's possible to put together a community which goes beyond just software developers. WorldForge seems to have a good start on some important pieces of infrastructure. There may be some useful code to be had from the Second Life client, which has been free for a year now. We are a large and talented community, we certainly have the ability to do something interesting in this area. It should not be necessary to wait until we get a code dump from a dead proprietary software company.

Comments (31 posted)

NDISwrapper dodges another bullet

Hardware compatibility has long been a problem for Linux—though it has gotten much better over the years—so it will be surprising to some to see a kernel change that will make some hardware cease working. For others, who follow kernel development a bit more closely, it will come as no great surprise that NDISwrapper was disabled by a change made to the kernel back in January. NDISwrapper has never been very popular with kernel hackers, but, because it is GPL licensed and allows more hardware to be used, there are folks on both sides of the argument. For a while, it looked like NDISwrapper had lost that argument, but the 2.6.25-rc4 release restores the functionality it requires.

NDISwrapper is a kernel module that is used to load Windows-only drivers into Linux. For some hardware, notably wireless network cards, it is the only way to support them because the manufacturer provides neither specifications nor a working Linux driver. Unfortunately, many of these cards are installed in laptops where it is difficult or impossible to replace them with Linux-friendly alternatives. This is what led to implementing the Network Device Interface Specification (NDIS) for Linux. NDIS is an ancient—it was originally developed by Microsoft and 3Com for MS-DOS in the mid to late 1980s—interface for networking devices, which is still in use today.

The NDISwrapper code has been around since 2003, but always as a separate module that must be built by the user (or distribution) and loaded into the kernel. It is not part of the mainline kernel, nor will it ever be; maintaining a glue layer that allows proprietary, closed-source drivers to be linked into the kernel is not high on anyone's list. But, NDISwrapper is GPL. Its code is available for inspection or modification by all, so that is not the problem, it is the intent that matters.

When a binary-only driver—the NVidia video driver for example—is loaded into the kernel, a "taint" flag is set, indicating that the kernel is tainted by code that cannot be examined. Bug reports for tainted kernels are routinely ignored, unless they can be reproduced in an untainted kernel. Life, it seems, is too short to try and diagnose problems that could easily have been created by a buggy driver that cannot be debugged. Originally, the taint flag was just a means to detect and ignore those bug reports, but over time it has become part of a mechanism to restrict which symbols a module can access.

Some kernel symbols are considered so integral that any module using them must be a derivative work. Therefore, modules that want to use them must be GPL. Modules declare their license using the MODULE_LICENSE macro, while symbols are exported using either EXPORT_SYMBOL or EXPORT_SYMBOL_GPL. Any module that doesn't have a compatible license doesn't get access to the GPL-only symbols.

Few would argue for a GPL module which existed to re-export all of the GPL-only symbols to non-GPL modules. But that is not what NDISwrapper does; instead it implements NDIS, but in order to do that, needs access to GPL-only symbols, mostly for USB and workqueue interfaces. It would be hard to contend that NDIS drivers are derivative of the Linux kernel, they were written for an entirely different system using an interface that predates Linux. This is why NDISwrapper developers and users think that an exception should be made for it. Clearly the Windows drivers taint the kernel, but accessing a subset of the GPL-only functionality through NDISwrapper should be allowed, they argue.

Since NDISwrapper itself is GPL, the normal module loading rules would allow it to access GPL-only symbols, except that an explicit check for NDISwrapper was added to the 2.6.16 kernel. The question, then, revolves around what should be done when the kernel detects it being loaded. NDISwrapper has always been careful to mark the drivers that it loads as tainted, but the recent patch marks the module itself as tainted, disallowing access to the GPL-only symbols and breaking NDISwrapper. Absent that patch, only the kernel is marked as tainted—the module itself is not.

A similar situation occurred back in October 2006, which LWN covered on the Kernel page, when a stricter interpretation of tainting started to be enforced. At that point, NDISwrapper stopped working and it looked like it might stay that way, until Andrew Morton stepped in with objections to breaking NDISwrapper with no warning. Shortly thereafter, a patch was merged that only marked the kernel as tainted when NDISwrapper is loaded. At that point, the issue fell by the wayside, until now.

Part of the problem is that marking a symbol as GPL-only means different things to different developers. For some, it is a means to warn proprietary driver developers that they are straying into territory that makes distribution of their drivers very likely to be a violation of the GPL, while others want to use it to completely eliminate binary-only kernel drivers. There is no policy that clearly delineates which interpretation is "correct". Meanwhile, NDISwrapper has been in use by many for four years or more; breaking it now, with little or no warning, is likely to create some very unhappy users.

Linus Torvalds clearly thinks there are no licensing issues with NDISwrapper:

Jon Masters, the author of the patch that inadvertently made this change, had an excellent suggestion that should be pursued to try and reduce these kinds of problems in the future:

Another interesting battle will be that surrounding exporting init_mm() which was removed in early versions of 2.6.25, but then restored in 2.6.25-rc4. It is fairly clearly a low-level kernel interface that is unused by any in-tree driver, so its export was removed. One rather glaring exception is that the out-of-tree NVidia binary drivers do use it. Its export has been restored for one more development cycle, but it is clearly seen as something that should not be touched by drivers. It could be quite a struggle between the developers and users of a very popular driver and the kernel hackers that don't want to see kernel API abuse.

Issues surrounding the GPL are always contentious on linux-kernel; this one is no different. While NDISwrapper is an out-of-tree driver, it has hardly been invisible, so complaints when it breaks should come as no surprise. A simple renaming will avoid the current kernel check, so breaking it that way will mostly be an annoyance to users rather than a real barrier to its use. Since there is no real consensus amongst kernel hackers on the binary driver issue, it is hard to see one emerging with regards to NDISwrapper, but that would be the best outcome. One way or another, it needs to be decided, NDISwrapper shouldn't come under a periodic threat of breaking. If it is determined to be a violation of the kernel interfaces, that should be clearly indicated and its users should be given some warning so they can find alternatives.

Comments (34 posted)

Page editor: Jonathan Corbet
Next page: Security>>