|
LWN.net Weekly Edition for January 3, 2008The 2007 Linux and free software timeline Welcome to the tenth annual LWN Linux and free software timeline. In what has become a longstanding tradition, LWN finishes each year with a collection of the most important events from the last twelve months.This is version 0.9 of the 2007 timeline. There are certainly errors and omissions; if you find any, please send them to timeline@lwn.net rather than posting them as comments. The development of the LWN.net Linux Timeline was supported by LWN subscribers; if you like what you see, please consider subscribing to LWN.
Thanks to the following people for suggestions which have improved this year's timeline:
For the historically minded, the timelines for the previous nine years remain available:
Wild predictions for 2008 It's that time of year again: the beginning of the new year - along with the lack of much else going on - inspires editors to make predictions about what they think may happen in the coming months. Your editor is not immune to these forces, and he has long since ceased to fear the possibility of looking like a fool in front of thousands of people. He's used to looking like a fool in front of thousands of people. So, without further ado, here's a set of wild guesses about what may await us in 2008.
DevelopmentSupport for Flash media will reach a usable state in 2008 - at least, on the playback side. The ability to waste time on video sites using only free software will doubtless prove appealing for many Linux users, while the ability to display Flash-based advertising may prove less so. But Flash is an important medium for video content and various types of interaction; good, free support for this medium is an important prerequisite for true World Domination. Arguably even more important is the ability to create Flash media on Linux, but that will take a little longer to come around. KDE 4.0 will be released early in the year. This is a huge, milestone release for the KDE development community, but the developers who have worked so hard toward this goal may find the user community's response a little disappointing. For all of the great work which has gone into 4.0, it remains a dot-zero release, and a big one at that. The remaining bugs and missing features are certain to put off some early adopters. One need only think back to the early GNOME 2.x releases, though, to realize that this is a normal part of the development process and that things will get much better quickly. The focus on power consumption will intensify this year, continuing a trend from 2007. Linux should, by all rights, consume less power than competing systems on the same hardware - but it doesn't. We now have the tools to identify and track down the worst offenders in this area, and we have the low-level support needed to make low-power Linux possible. Mobile applications may continue to drive this push, but there may be even more low-hanging fruit on fixed systems. There is just no end of reasons to reduce power consumption on all systems running Linux, and we're now in a position to get that job done. The merging of the realtime Linux tree will be substantially complete by the end of the year. Your editor is out on a limb here; the remaining realtime code includes some of the most intrusive changes. But distributors are shipping this code now, and it has been well tested in a number of environments. So it seems likely that, by the end of 2008, the mainline Linux kernel will be fully capable of running in a realtime mode.
Legal issues and related overheadThe OOXML standardization debate will continue, and Microsoft may well prevail in getting its document format recognized as a standard by the end of the year. The free software community will react as it always has - it's just another data format to support. More projects will move to GPLv3 in 2008, creating occasional fallout at the distributor level when newly-created licensing conflicts are found. The most interesting potential change is the GNU C library, which remains at LGPL 2.1 as of this writing. A GPLv3-licensed glibc would have to be user-replaceable, which could be problematic on locked-down devices. So, if this change happens, expect a increased interest in alternative C libraries for embedded applications. GPL enforcement activities will continue and may even increase. Patience with companies which use the code without complying with its license is at a low point, and that will not change. Chances are that, once again, almost every company which is confronted on GPL-violation issues will come into compliance without going to court. There will be no more Microsoft patent deals, at least with companies of any significance. Those who are inclined to make such agreements have already done so; the holdouts are unlikely to change their minds at this point.
Commercial and relatedThe OLPC project will start to think seriously about the successor to the XO. There will be many opportunities to build a platform which can be even more empowering for small children; for example, the GNU Radio folks are already pondering ways to bring software-defined radio capabilities to this machine. Meanwhile, deployments of the XO will continue to happen and we'll see the first effects of putting truly free systems into the hands of children. Some of those effects will certainly surprise us. The days of hardware support hassles will be over. By the end of 2008, we should have good support for ATI graphics adapters, Atheros wireless chipsets, and even, via the Nouveau project, NVidia adapters. There will always be exceptions, but the rule will be clear: we will be able to buy hardware secure in the knowledge that it will work with our Linux systems. Competition between distributors will grow in intensity. We saw some hints of this in the sniping between Red Hat and Novell toward the end of 2007; there will be more as these businesses increase their focus on the bottom line. Ubuntu will also push harder, though, interestingly, it often seems like that distributor's biggest perceived competitor is Fedora. Your editor believes (and hopes) that cooperation at the development level will remain strong despite increasing drama at the public relations level. Along these lines, expect intensified competition from Sun, which will continue to try to aggressively push Solaris into Linux shops while simultaneously presenting a friendly face to the community. We may also see more of the less-friendly side of the BSD community for similar reasons.
CommunityThere will be a major technical Linux event in the United States - the first in some years. The Linux Plumbers Conference, planned for mid-September, will be unique in its focus on the kernel and the software layers immediately surrounding it. Getting the "greater kernel ecosystem" together in one place is an overdue move which should help integration and development of the plumbing we all depend on. Participation in the development community will grow. That, of course, has been true every year for at least two decades. In 2008, though, we can expect to see a stronger push to encourage developers from parts of the world which traditionally have not contributed so strongly to our community; Asia, in particular, should continue to increase its presence. We will also continue to see companies in the embedded systems area figure out that, if they do not participate in the development of the code they use, others will have a much stronger influence on how that development goes. Tolerance for anti-social behavior on mailing lists, IRC channels, etc. will continue to drop as development communities try to attract and provide a welcoming environment for more participants. Many communities have formal codes of conduct now; others may well try to adopt them. But even less-formal groups will increasingly understand that a harsh and unfriendly environment hurts the project as a whole. As usual, we'll come back to these predictions at the end of the year and mock them without mercy. Until then, best wishes for a great 2008 from the LWN editorial team!
The Grumpy Editor's video journey part 3: DVD authoring As readers of the first part of this series will remember, your editor has set out on a project to digitize a set of old video tapes and turn them into properly-formatted DVD media suitable for handing out to the grandparents. Part 1 was about the task of capturing this data to disk; part 2 covers the video editors available for turning the captured data into something watchable, and part 3 covers the task of creating a DVD from the edited video.Attentive readers may have noticed that part 2 has not yet been written; there are more editors available than your editor had expected (currently under review are Cinelerra CV, Kino, PiTiVi, LiVES, and Avidemux), so that process is taking longer than expected. For the purposes of this article, let us assume that your editor has a disk full of video clips which have been edited and properly formatted into the MPEG2/AC3 video object files expected by DVD players. There will be a discussion of the best ways to get those files there in the near future, promise. Many of us have burned CDs and found the process to be relatively straightforward - the biggest obstacle is often just getting past the grumpiness built into cdrecord and its latter-day derivatives. Creating data DVDs is not a whole lot harder. So one might be inclined to approach the task of creating a video DVD with a "this will be easy" attitude. It is, in fact, a task just about anybody can learn to do, but it is on a different order of complexity than creating a CD full of music. A video DVD is, in truth, a program complete with its own hierarchical structure, menus, and code written for the simple virtual machine lurking within every DVD player. Creating a playable DVD requires writing that program. If DVDs are programs, then the one compiler available for Linux systems is the command-line dvdauthor tool. Regardless of how one builds a DVD, dvdauthor will be involved in the process at some point. This tool requires a collection of video objects representing the actual video titles and also implementing the menus, subtitles, and more. It's all tied together via a complex XML file (example) which is compiled by dvdauthor to create the final product. It is possible to create all of these pieces by hand, and, doubtless, Real Linux Video Jocks would not do it any other way. One can use dvdauthor to help with the generation of parts of the XML file. There is documentation which seems fairly complete, if a bit terse. But the fact of the matter is that most people attempting to use this tool directly will give up in despair. There is no reason why DVD authors should have to work at this level; dvdauthor is essentially an assembler which, while being absolutely essential to do most of the heavy lifting, should be hidden from most polite company. DVD creation is a visual task; there should be visually-oriented tools for this job. The good news is that these tools do, indeed, exist.
DVDStyler
As a grumpy aside, your editor must note that the directory browser uselessly starts at $HOME. One need not work with much video data before realizing that special provisions must be made for its storage; video objects are unlikely to be kept in the home directory. Your editor has a hard time understanding why tools like this are unable to start file searches in the current working directory, which is a much more likely place to find things of interest. Switching to $HOME is not just a least-surprise violation; it actively makes things harder for the user. The "Backgrounds" tab helpfully offers a dozen or so canned background images which can be used for the DVD menus. They are nice backgrounds, and they might just be useful for somebody struggling through the process of creating a DVD for the first time. Your editor, though, suspects that most users, by the time they create their second (working) DVD, might just want to supply their own background images. They will look for that option under the "Backgrounds" tab in vain, though. It is possible to supply a custom image: go to the large (video screen) pane, right-click, select "properties," and set an image there. It's easy, once you've figured it out. But one would think that, having gone to the trouble to provide an entire mode dedicated to background images, the developer would have thought to toss in a "none of the above" button. The hardest part of creating a DVD (once one has suitable video in place, obviously) is getting the menus to work. DVDStyler starts with an empty main menu in place; it is up to the user to add entries which will do interesting things. That is done by way of the "Buttons" tab. There's a selection of arrows available, as well as the ability to add basic text buttons. The button of interest can be simply dragged to the right spot on the menu, sized appropriately, and configured to do the right thing. There are also "empty" buttons for more complicated situations where the real button text (or image) is found on the menu's background image.
Also required is a specification of what happens when one of the directional arrows is pressed. The default "auto" setting leaves that up to the player, which will probably do the right thing - the down arrow, for example, will move the focus to the next button below the current one. Anybody who is concerned about the user interface provided by the resulting DVD will probably want to set these actions explicitly, though - a somewhat tedious and time-consuming task. Eventually, the time comes to actually create the DVD. Most first-time users will probably go to the DVD menu for this task, but the "burn" option is not there - it's under the "file" menu instead. The resulting dialog works nicely, giving the user the option to stop after generating the ISO image or to run a preview application (xine by default) before actually writing to the disk. Underneath this dialog is a whole set of helper commands which are run; those can be configured if need be, but most users will not tread there. All told, your editor found DVDStyler to be the easier tool to use for quickly putting together a video disk. There is just one little problem: those disks never quite worked right on your editor's ancient DVD player. Somehow, a misunderstanding about how the menus should work crept in. Your editor suspects, perhaps, that overlapping buttons may have something to do with it; the other application reviewed by your editor (QDVDAuthor) detected and corrected that situation, but DVDStyler did not. In any case, newer players had no problem with the generated disks, so this may not be a problem that most people need to be concerned with.
'Q' DVD-Author
Qdvdauthor provides a three-paned window with areas for the current set of audio/video objects, the DVD hierarchy, and the menu designer. The audio/video pane, on the left end, is clearly a work in progress. There is a thumbnail area which shows the opening frame of the associated video - sometimes. Other times it stays green and qdvdauthor silently leaves an mplayer process desperately cranking away in the background. It was only when the load average on your editor's system got to around 20 that he figured that one out. There is a "play" button which pops up a cheery "not yet implemented" button. The run time of each video title is also displayed. All told, it is a more useful display than what DVDStyler offers, with the potential to be quite a bit better yet. The middle pane shows the current hierarchy of objects making up the DVD. It is a helpful display, given that DVDs truly are hierarchical objects. It likes to reset itself to the top, though, making it necessary to scroll repeatedly toward the bottom when the DVD gets more complex. The right pane shows one of the DVD menus - or a couple of other things we'll see later on. One very nice feature is the little display at the bottom showing how much data has been committed to the DVD so far and how much room remains. Video titles are easily added using the prominent "add movie" button. Once attention turns to the menu creation process, one notices that there is no separate "backgrounds" tab - but there is a button for adding a custom background image, which is what is really needed anyway. Your editor found that dragging a thumbnail from the video pane over to the menu area created a picture button which would play the associated title - a nice feature.
Eventually, the time comes to commit all of that work to an actual DVD. A click on the associated button gets that process going. If one has been sloppy in drawing out buttons, the first thing to come up will be a warning that some of the buttons overlap, accompanied by an offer to fix the problem automatically. One can also decline the offer (aborting the process) to fix the problem manually. This is as good a point as any to note that moving and resizing buttons in qdvdauthor is a real exercise in pain. The button areas have the usual grab points for moving, dragging edges and corners, or rotating the button. But none of those are visible until the user has clicked the mouse and committed himself to doing something. The end result is that attempts to drag a button often do something else - like rotating them to some strange angle. The basic interaction modes for operating on graphical objects in a display have been well understood for years; one can only imagine that whoever designed this interface was engaging in some sort of sadistic exercise which was sponsored by purveyors of strong drink.
There's a few other nice features hidden in this application. The menu pane can be made to show the XML file which will be generated for dvdauthor; it can also be put into a garish and complex dialog which facilitates the addition of subtitles. There is a template mechanism for menus, and a network-based repository from which qdvdauthor can download new templates. There is an operation which will convert the entire DVD between the NTSC and PAL formats - your editor has not yet exercised this option, but, given that some of the grandparents for whom this work is intended live in Europe, it will eventually come in handy. There is a little-used plugin mechanism and a theme feature as well; long-neglected Motif users will be glad to know there is a style for them. The addition of audio to menus and intro/outro sequences to titles is relatively straightforward. There is also an option to make DVD slideshows out of a series of still images.
ConclusionEither one of these applications can get the job done. They both show the best of how an application on a Unix-like system can add power by using existing tools. Neither DVDStyler nor qdvdauthor actually does much of the work of creating menus or burning DVDs; they mostly just put together fiendishly-complex command lines and call out to the tools which have been designed to do that work well. Overall, the combination works reasonably well. A feature which is lacking from both tools is a "hold my hand" mode for people who are not - and do not want to be - experts in DVD creation. A sequence of screens which would set up an initial menu, import titles, and create buttons for each would be most helpful in this regard. As it is, users must have their own internal checklist in mind when creating DVDs, and it is easy to miss things. Your editor, while certainly slower than most, is unlikely to be the only one to have created an impressive pile of coasters before finally producing a DVD which actually worked as intended. While the tools edited here are, in your editor's opinion, the best available for Linux for this task, there are some others to be aware of:
Of all these tools, it must be said that qdvdauthor is, at this time, the most complete and capable. It provides access to almost any capability supported by current DVD players, is relatively easy to use, and works most of the time. With luck, the developers (who released the 1.0.0 version reviewed here in November, 2007) will devote themselves to smoothing out the remaining rough edges, leaving us with a tool which DVD authors at any level can use.
Page editor: Jonathan Corbet Security The future of unencrypted web traffic Hypertext transfer protocol (http) is the heart of the web, providing the means to retrieve content from remote servers. It is an unencrypted, text-based protocol which allows malicious intermediaries to snoop on and potentially modify the traffic. Unfortunately, internet service providers (ISPs) are getting increasingly bold in manipulating the traffic that they carry. This has lead some to call for the elimination of http, in favor of encrypted http (aka secure http or https). An ISP is perfectly situated to gather an enormous amount of information about its users, their website preferences and habits (often called clickstream data). Some have reportedly been selling some of that data in a thinly-anonymized form to advertisers and others. As AOL's well-intentioned, but poorly implemented, release of search queries showed, it is rather easy to analyze this kind of data and pierce the anonymity, deriving the specific user. Another recent ISP trick is to modify a retrieved web page to display other information – under the control of the ISP – which looks like it comes from the website itself. Canadian ISP Rogers Internet has been testing a system to add content to the Google homepage for their customers who are near their monthly bandwidth limits. There are also plans afoot for ISPs to use clickstream data to target advertising – though just where those ads would show up is far from clear. This kind of manipulation is unlikely to be what internet users expect – to the extent they think about it all. The model folks tend to use is that of a phone company; we do not expect them to sell our call records to the highest bidder, nor do we give them license to modify our calls. Various telecommunications privacy laws protect that data, but those laws have not (yet) been applied to internet traffic. In addition, ISPs tend to have a monopoly or near-monopoly, which restricts alternative, less-intrusive ISPs from competing. Fortunately, there are technical solutions possible in the internet realm that would be difficult or impossible to implement network-wide in the phone system. Encrypting website traffic will go a long way towards eliminating this kind of ISP abuse, though it is no panacea. As more of these kinds of privacy invasions occur, we should see more routine use of https by websites. Currently, https is almost exclusively used for e-commerce transactions; typing in credit card numbers and the like. Authentication via username and password is another area that sees widespread encrypted pages. Sites may start to use https for their entire site to combat clickstream and page rewriting abuse – though there will still be some information leakage as the ISPs can still see what sites are being visited. In order to make an https connection, the server must have a certificate with its public key. Typically those are signed by an authority recognized by browsers which allows the browser to authenticate that the certificate belongs to the host visited. Getting signed certificates is a bit cumbersome, costs some money, and they need to be renewed periodically – all of which adds up to a headache for a site, especially a small, non-commercial site, that wants to switch to using https. Self-signed certificates are an alternative, but because they are susceptible to man-in-the-middle attacks, browsers warn their users when they receive one. Another problem with this approach is the extra processing required on the server to support encrypting each and every request. There is a non-trivial amount of extra work that must be done per request and cannot be cached. Sites that wish to avoid the problems that some ISPs are introducing will just have to bear that cost. Pushing bits is not very glamorous, but that is really what one hires an ISP to do. Since they seem to be finding new and exciting ways to interfere with those bits – Comcast messing with BitTorrent traffic for example – internet users will have to find ways to thwart their schemes and encryption will be a big part of that effort. Using https site-wide is only one step, other services will also need to be protected from ISP abuse. What if an ISP started manipulating the results returned from DNS queries, perhaps routing some to a server they control?
LWN adds a Security index LWN has added a new index to complement the existing Kernel index. The Security index covers security articles we have published since the start of 2007. Hopefully this will be a useful resource for our readers and, as always, we value your comments. Please send them to lwn-AT-lwn.net.
New vulnerabilities autofs: privilege escalation
bind: insecure permissions
clamav: mystery vulnerability
exiftags: multiple vulnerabilities
exiv2: integer overflow
gallery2: multiple vulnerabilities
Ganglia: cross-site scripting
imlib: denial of service
kernel: information leak, denial of service
mt-daapd: multiple vulnerabilities
mysql: denial of service
peercast: buffer overflow
syslog-ng: denial of service
typo3-src: SQL injection
wireshark: multiple vulnerabilities
wireshark: lots of dissector vulnerabilities
Updated vulnerabilities cairo: integer overflow
Django: denial of service
MySQL: privilege escalation
Sun JDK/JRE: multiple vulnerabilities
apache2: information disclosure
apache: multiple vulnerabilities
apache: cross-site scripting
httpd: denial of service, cross-site scripting
apache2: denial of service
asterisk: possible SQL injection
autofs: insecure default configuration
cacti: SQL injection vulnerability
cacti: denial of service
clamav: denial of service
clamav: multiple vulnerabilities
clamav: integer overflow and off-by-one
cups: denial of service
cups: multiple vulnerabilities
debian-goodies: privilege escalation
dovecot: privilege escalation
dovecot: directory traversal
e2fsprogs: integer overflows
eggdrop: stack-based buffer overflow
emacs: buffer overflow
emacs: command execution via local variables
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||