User: Password:
|
|
Subscribe / Log in / New account

LWN.net Weekly Edition for November 23, 2011

That newfangled Journal thing

By Jonathan Corbet
November 20, 2011
Certain topics, it seems, are just meant to stress-test the LWN comment system; after the unveiling of "the Journal" on November 18, "the works of Lennart Poettering" will have to be added to that list. Lennart (along with Kay Sievers) has clearly hit a nerve with his latest proposal. There is probably enough in this concept to upset just about anybody, but the Journal deserves serious consideration, even if it represents a break from how logging has been handled in the past.

Start with this 2010 quote from Andrew Morton:

The kernel's whole approach to messaging is pretty haphazard and lame and sad. There have been various proposals to improve the usefulness and to rationally categorise things in way which are more useful to operators, but nothing seems to ever get over the line.

The point Lennart and Kay are trying to make is that "haphazard and lame and sad" doesn't just apply to kernel messaging; it describes the approach taken by Linux in general. Messages are human-readable strings chosen by developers, often without a whole lot of thought. These messages may have information useful to automated system monitoring tools, but their free-form nature makes them hard to find and the information contained therein hard to extract. A quick look at what goes on inside a tool like logwatch makes it clear that, whatever their benefits may be, free-form messages are not clean or easy for tools to deal with. The fact that developers can (and do) change messages at any time does not help matters either. Messages are stored in flat files that makes them hard to access, and they are subject to manipulation and editing by attackers. There must be a better way.

The Journal may or may not be a better way, but it does incorporate a number of interesting ideas - though some of them may be "interesting" in the Chinese curse sense. It is tied to systemd, which condemns it in some people's eyes even before the rest of the proposal can be considered. But hooking logging into systemd makes it easy to capture all messages from managed services and get them to their final resting place. It enables tricks like printing the last few messages from a service when querying its status. It also makes it possible for the Journal to verify the source of many messages and to add a bunch of additional metadata to those messages. From this point of view, at least, it looks like a more robust way of dealing with system logging.

When this subject was discussed at the 2011 Kernel Summit, one of the most controversial points was the request that developers attach 128-bit UUID values to most or all messages. Associating a unique identifier with each type of message does make some sense. Users (or their tools) can look up specific messages in a help database to learn what the message really means and how they should respond. Translation mechanisms can use the identifiers to localize messages. System monitoring utilities can immediately recognize relevant events and respond to them. But, while the value of the identifier is clear, there is still a lot of disagreement over its practicability. It is not clear that developers will bother to generate unique identifiers and to use them in a disciplined way.

All complaints about UUIDs were quickly overshadowed by another issue once the full proposal was posted: one might charitably say that there is not, yet, a consensus around the proposed new logfile format. In a sense, that is unsurprising, since that format is deliberately undocumented and explicitly subject to change at any time. But it is clearly a binary format, and a reasonably complex one at that; each field of a log message is stored separately, and there is a mechanism for compressing out duplicated strings, for example. Binary-blob data is allowed as a part of a log message. The Journal's format is clearly a significant departure from the classic, all-text flat-file format used by Unix-like systems since before many of us were born.

There is value in a new log format. Current log files quickly become large and unwieldy; extracting useful information from them can be a slow and painful task. The new format, beyond holding much more information, promises to be space-efficient and very quick to answer queries like "please give me all the disk error messages from the last day." System administrators could probably find a way to get used to the new features of this format in a hurry.

That said, the idea of using an undocumented binary file format for crucial system logging data sets off alarms in the head of just about anybody who has been managing systems for any period of time. The format isn't "secret," of course, but we still should not have to reverse-engineer the format of our logfiles from the Journal source. Unless it is very carefully done, this kind of format could lose a lot of logging data as the result of a relatively small corruption. If the file format really does shift over time, recovery of information from older logfiles could become difficult. It is not surprising that this proposal scares a lot of people; there may have to be some significant changes before the Journal becomes acceptable to a lot of system administrators.

In truth, it may make sense to separate systemd's role in the collection, verification, and annotation of log data from the role of a backend dedicated to distribution and storage of that data. The two tasks are different, and not all sites have the same needs in this area. Mixing the two risks limiting the usefulness of the Journal and muddying the debate about the concept as a whole.

Tradition

This proposal is a request for comments; it has garnered no end of those. What is interesting to see, though, is that a lot of these comments take the form of "syslog has been good for Unix for 40 years, this proposal is trying to change one of our grandest traditions." There is a lot to value in the Unix tradition, but we do need to be careful about sticking to it just because That's How Things Have Always Been Done. Even longstanding mechanisms are subject to improvement. In the case of logging, it seems fairly clear that what we have now is not a thing of timeless beauty. This article will take no position on whether the Journal is the right way to improve on Linux logging, but it will claim that the fact that people are looking at this problem and that they are willing to make significant changes to improve things is good for Linux as a whole.

For much of the history of the Linux system, the unwritten goal was probably something like "let's make something that looks sort of like SunOS 4.x, but which is free, works on current hardware and has the rough edges smoothed a bit." That goal - implement something firmly within the Unix tradition - served to unify Linux development for many years; in our own special way, we were all pulling in the same direction. But that goal has long since been met; there is no longer an obvious target for us to be chasing at this point.

That means that we are heading ever more deeply into new and uncharted territory; Linux is increasingly doing things that no Unix-like system has done before. In the absence of an obvious model, developers doing new things will do them in new ways. The results include Android, KDE4, GNOME3, and, yes, the collected works of Lennart Poettering. If the work is genuinely new, it will look different from what we are used to. Some of this work will certainly turn out to be a bad idea, while some of it will endure for a very long time. But none of it will exist unless developers are willing to take risks and rethink how things have always been done.

Unless a shinier SunOS 4.x is really all that we are after, we as a community are desperately in need of people who are willing to stir things up and try new things to solve problems. To restrict ourselves to what Unix did is an almost certain way to share Unix's fate. We can do better than that - indeed, we have done better than that - but, to do so, we need to create an environment where developers are willing to take some technical risks. Bad ideas need to be exposed as such whenever possible, but ideas should not be shot down just because they do not preserve the way our grandparents did things. The Journal may or may not turn into a sensible option, but the process that creates things like the Journal is one of the most important things we have.

Comments (251 posted)

Trinity Desktop Environment: Keeping KDE 3 alive

November 22, 2011

This article was contributed by Bruce Byfield

At this point, the Trinity Desktop Environment (TDE) hardly needs a conventional review. After all, as the numbering of the new 3.5.13 release suggests, TDE is a continuation of the KDE 3 series, and anyone who is even remotely interested must have seen the basic desktop many times. Instead, TDE raises some questions. For instance, how well can a desktop first released nearly a decade ago adapt to modern computing? Just as importantly, what are its chances of survival, and what directions might its development take in the future? To judge from the new release, these are important questions.

The latest version of TDE is available as packages for recent releases of Debian, Ubuntu, Red Hat, and Fedora. However, packages for releases made in the last month or so, such as Fedora 16, are not yet available, while packages for Slackware, openSUSE and Mandriva are mentioned as being in development. Alternatively, you can download a Kubuntu Live CD or build TDE from source.

If you install TDE, note that its packages are not interchangeable with packages from other sources for the KDE 3 series. Nor can KDE 4 releases run them. Installing TDE requires dependencies not found in any version of mainstream KDE, including patched versions of the libcaldav and libcarddav libraries and the version of Qt3 that Trinity maintains as separate projects.

Thankfully, all Trinity packages have a -trinity suffix attached to them, and the desktop maintain settings in a ~/.trinity directory separate from the ~/.kde directory used by KDE 4, so TDE can coexist safely beside other versions of KDE. The largest problem you are likely to encounter is finding TDE at the login screen, where it is listed under "TDE" rather than "KDE" or "Trinity," as you might first expect.

Running TDE

For long time KDE users, TDE is a step back in time. The first login starts with the KPersonalizer wizard, complete with Konqi, the now seldom-used dragon mascot of KDE. However, this nostalgia is not an unalloyed delight.

The first impression is that TDE is generally faster than KDE 4. This impression waivers when you open KDE 4 apps and seems inconsistent when running applications not designed for a specific desktop, but the general impression remains for routine operations like logging in or out.

[TDE desktop]

If you are familiar with the KDE 3 release series, the next thing you are likely to notice is how little it has visibly changed, even though 3.5.13 is the third TDE release, 3.5.12 having been released in October 2010, and 3.5.11 in April 2010. Perhaps the most visible improvements in the 3.5.13 version, which was released November 1, is a Monitor and Display dialog that supports multiple monitors, the option to run a KDE-4-style menu, and a few themes. You have to go into the repositories to notice the addition of a few TDE-specific applications, such as kbookreader, kdbusnotification, kmymoney, and kstreamripper. Otherwise, most of the enhancements in the new release are behind the scenes, and largely unnoticeable. Ranging from bug-fixes to improvements in security, these enhancements are welcome, but, to the casual eye, TDE seems pretty much identical to the last of the KDE 3 series.

Although interface differences are obvious, in terms of basic features TDE compares favorably with the latest KDE 4 releases. In fact, although rearrangement of configuration options make comparisons difficult, TDE's Control Center actually appears to have more administrative options than the latest KDE 4.7 release, especially for hardware peripherals. For many, an interface designed to the standards of a decade ago might seem at first a small price to pay for the speed and administration tools.

[TDE control center]

However, TDE also has some serious shortcomings. For example, according to the release notes, Bluetooth functionality is absent. So, too, is the ability to edit HTML messages and display images in KMail, which is the subject of a $2500 bounty. TDE is also at an obvious disadvantage in its lack of counterparts to advanced KDE 4 features such as Folder Views and Activities. Nor does it integrate any online applications or services into the desktop, the way that an increasing number of desktop environments do.

In addition, TDE would benefit from the ability to migrate email, addresses, and settings from other versions of KDE. Non-KDE applications such as Firefox can be used interchangeably in TDE and KDE, but not KDE-specific applications. Start KMail in TDE, for example, and you have to configure it separately from other versions of the same application. Admittedly, migration of information might be difficult, since KDE 4 applications such as KMail and Amarok store information in databases, but, without some sort of assistance, TDE installations are practical mainly on a fresh machine or else as experiments not intended for serious work.

Even more seriously, while the last releases of the KDE 3 series had a reputation for stability, TDE's patches seem to have taken a toll. Every now and then on my Debian installation, TDE takes three or four times as long as usual to log out. Random crashes also occur. For some reason, too, changing the theme creates a new panel on the left side of the screen, while logging in always starts the audio mixer.

Questions of viability

Judging by the ongoing complaints about the KDE 4 series, a market should exist for a continuation of KDE 3. Yet, for some reason, TDE remains a niche project. It has only four main developers, although, according to project lead Timothy Pearson, others have reported bugs and submitted patches. The same names appear over and over on TDE's mailing lists, and the project's Community Styles and Themes page includes just a single contribution.

Furthermore, in the last eleven months, the project's download page has averaged just 214 visits per month, with a high of 492 in February 2011. Even with the new release, this month's visits were only 252 with the month two-thirds over — and probably not all of those visits were followed by an installation.

Little wonder, then, that the project's home page includes a plea for help in every aspect of development. The current team has undertaken an ambitious project with limited resources, and is running hard just to stay in the same place.

That might lead to misgivings about TDE's viability, but there are some other things to consider. The project has managed to do three releases, with a fourth planned for April 2012. It has also successfully navigated major design decisions, avoiding switching to the Qt4 framework by taking over the maintenance of Qt3, and continuing to rely on DCOP, invoking D-Bus only when necessary for such operations as running KDE 4 applications.

Moreover, project members are well aware that much remains to be done. Asked about the priorities for the future, Pearson mentions a replacement for HAL, but emphasizes that the main concern will be bug-fixes. "TDE inherited a lot of bugs from KDE 3.5.10, and has added some of its own over the years," he acknowledged. The next release, he added, "is going to primarily be a bug fix release, although I imagine some new features will probably creep in between now and the release date."

Despite the challenges, Pearson remains optimistic that TDE will survive and define itself. Since forking from KDE 3, he said,

The two projects' goals have diverged significantly. We are not trying to compete with KDE 4 (or any other desktop for that matter) — instead, we are working on an alternative desktop environment with a different perspective on human-computer interaction compared to the environments currently available.

In other words, like Xfce, TDE is designed for those who view the desktop primarily as a place from which to launch local applications. To those who use online applications or services, or whose workflow now depends on enhanced features like KDE 4's Activities and alternative desktop arrangements, that may seem like an obsolete goal. Yet the continuing mutter of controversy about KDE 4 suggests that there are at least some who wish to continue working as they already have for a decade.

For these reasons, in its own quixotic way, TDE seems likely to struggle on, at least for the immediate future. For those who were happy with KDE 3, and opposed to the direction of KDE 4, the project seems a perfect opportunity for getting involved and making a difference.

Comments (16 posted)

Thoughts on conferences

By Jake Edge
November 22, 2011

Over the last four years or so, I have attended numerous conferences in many different locations. It has been, really without any exceptions, an incredible experience. Conferences are one of the main ways that our communities come together and meet face-to-face—something that's important to counterbalance the standard email and IRC development environment.

In that time, I have also seen many different ways to organize, schedule, and produce those conferences, and, as is the case with free software projects, there are bits and pieces that conferences can learn from each other. What follows is my—fairly opinionated obviously—distillation of what works well and less well, which will hopefully be useful as new conferences spring up, or as existing ones plan for next year.

Location location location

Conferences are generally located in interesting cities throughout the world. This is a good thing as it means that folks get a chance to see different cities and countries. Moving the conferences around every year also means that geographic distance will become less of a factor as the conference will be "nearby" every once in a while. Because of that, those on a limited budget still have a chance to attend. Smaller, regional conferences may not move around very much (or at all), but their focus is typically more on gathering people from the region, rather than trying to attract a large presence from elsewhere.

Regardless of which city the conference is held in, it is very important that it be held in a "downtown" location, close to restaurants, bars, and sightseeing. One of the more important parts of any conference is the so-called "hallway track", where informal gatherings of attendees take place. Nearby bars and restaurants are an integral part of the hallway track. In a well-situated conference setting, it is not unusual to stumble into several different discussions during the conference, when eating dinner or visiting a pub. Requiring taxis, public transportation, or rental cars to access food choices is definitely sub-optimal, especially for lunch breaks.

Being in a downtown location also likely increases the options for hotels or other accommodations. While the rates at the conference hotel are generally pretty reasonable from the standpoint of other, similar hotels, they certainly aren't cheap. Folks that have a limited budget will appreciate having other options that are still within easy walking (or public transport) distance of the conference site.

Putting together a map with the conference venue, the various accommodation options, restaurants, pubs, clubs, and so on, is extremely useful. Adding information about how to use any public transport systems, directions on getting to and from the airport or train stations, as well as a bit of sightseeing information is invaluable as well.

Scheduling

The conference lineup is getting larger each year it seems, which makes it difficult to avoid date conflicts. It's obviously beneficial to do so, but may ultimately be hard to pull off. Even when they don't overlap, it's not uncommon for two conferences, in different places, to bump right up against each other, or only be a few days apart. That will make it difficult or impossible for folks to attend both—making it important to at least try to avoid conflicts between conferences with closely related topics or themes. A recent movement to collect up multiple conferences and run them back-to-back (or even concurrently) in a single location certainly has its advantages, but it also leads to attendees being away from home for up to two weeks—and possibly falling prey to conference burnout part way through. Obviously, there are various pros and cons here.

Something that is more easily controlled by the conference organizers is the actual schedule of talks. There are tradeoffs there as well, of course, but it is important to find the right balance between time for presentations, time for hallway chats, and time to move between various parts of the venue. If there are long distances between the farthest meeting rooms, it may make sense to have larger breaks between sessions. It is frustrating for attendees to be sitting in on a good talk, perhaps one that is running a little bit late, and having to watch the clock so that there is enough time to get to another session all the way across the venue. Worse yet, arriving on-time sometimes means the room is already full.

Timing is tricky too, though, because a staggered schedule (with, say, 50-minute slots and 20 minutes in between) makes it harder for attendees to keep track of when the next session starts. On the other hand, starting each session on the hour (or half-hour) may lead to shortened slots or very short room switching times. One possible solution is to have longer and shorter slots that are alternated in some clear pattern—it is sometimes the case that a speaker doesn't really have enough material for a full slot anyway. For hallway track purposes, 30-minute breaks in the morning and afternoon, as well as a 90-minute lunch break, should be strongly considered. All of that is pretty difficult to balance and still get in plenty of talks—at least in a one or two-day conference—but fewer, higher-quality talks with lots of discussion and social time makes for a better conference in my experience.

Talks

While meeting up with friends and colleagues, and visiting an interesting city, are certainly part of the draw for a conference, it is the presentations that provide much of the impetus for attending. Make no mistake, there have been some excellent talks with great presenters at all of the conferences I have attended. Keynote talks are another story unfortunately. Some conferences choose keynote speakers based on their merits, but some of the bigger conferences give keynote slots to sponsors, which can be something of a mixed bag.

There is nothing inherently wrong with providing keynote slots to sponsors, but those talks often end up being extended advertisements for the company and its products. There are exceptions, and even some of the ad-oriented talks are still interesting, but it is unfortunate that they are often some of the weakest talks at the conference. It may be unavoidable, as large corporate sponsors expect to get something for the money they put in, but there is no reason that the technical level of the talks couldn't be better. There certainly is a lot to be gained both for the company and for the attendees if the talks are genuinely interesting and informative—even if they must stray a ways into marketing.

Conference organizers are largely at the mercy of the presentation proposals that they get, but it is nice to have related talks collected up into themes (or whole tracks if there are enough). It's also important to try to ensure that related subjects are not scheduled at the same time. Scheduling is undoubtedly a tricky balancing act at times, but having several related talks arranged back-to-back in the same room makes it that much easier for attendees.

A grab bag of other thoughts

Conferences often hand out bags with a schedule/program, a T-shirt, and a few handouts (flyers, pens, notebooks, etc.) from sponsors. Most of it is fairly useful (though the flyers generally don't do much for me) and can be used or given to a friend or family member—except the bag itself. One has to wonder how many of the bags—often fairly attractive, made of cloth, with a conference logo—never make it home with attendees.

The reason is that most of those bags are essentially useless for any purpose other than carrying around the small amount of stuff that originally comes in them. In general, they are much too small to do anything with, so they get tossed out—or re-gifted to people who don't know any better. Either spending less money on a paper/plastic bag or, better still, handing out a bag that can be used for other purposes (perhaps a laptop bag or even a cloth grocery bag) would be good. It just seems like most conference bags are an expensive waste. For some, T-shirts may fall into the same category, but at least those can be donated to someone else or to a local organization for the needy.

While schedule apps for mobile phones (or tablets) are all the rage, it's a little hard to see what they offer beyond what a simple—up-to-date!—web page could. If the schedule is changing frequently, an app may remind you to update its data, unlike a web page, but that value is lost in the cumbersome navigation that the apps tend to have. Is it really so hard to recognize what day it is and make that the default? Bonus points would be awarded for noting the time and auto-scrolling to the right place in the schedule. It seems wrong somehow to have a separate app per event that needs to be downloaded before and deleted after, but I guess that's part of what apps are all about.

Of course, either apps or a web page require reasonable WiFi at the conference venue. For the most part, WiFi works pretty well at conferences these days. It can be overwhelmed when the majority of attendees are in one place (like for keynotes), but the rest of the time it seems to work pretty reliably, which stands in (sometimes stark) contrast to five (or even fewer) years ago. It's important for attendees to be able to keep up with email and such, though it can be a distraction as well. It's pretty easy to recognize a talk that isn't going very well by the number of head-down, typing attendees.

In closing ...

Here are some of the things that I noted about some of the conferences I have attended in 2011. It is by no means comprehensive, but just some quick-hit thoughts about things that each did well that other conferences should at least consider as part of their planning. For example, one of the things that struck me about the Prague conferences in October (LinuxCon Europe, Embedded Linux Conference Europe, GStreamer Conference, and Kernel Summit) was the large open area in the center of the venue between all of the rooms. There was plenty of space for chatting, raiding the snack/beverage tables, and even sitting down (if you were quick or paying close attention anyway).

While it probably complicated scheduling even further, I certainly appreciated the 10am start for the second day of the GStreamer Conference. Staying up too late at conferences, whether carousing or writing/editing, is all too common, so a later start (with perhaps a correspondingly late finish) is welcome. Like many conferences, Ubuntu Developer Summit in Budapest had an excellent location, with plenty of nearby restaurants and a walking mall with lots of food and beverage choices just two blocks away. Not to mention the fact that it was close enough to walk to some of the sightseeing highlights of the city.

The Desktop Summit in Berlin was memorable for many reasons, but one thing that really stood out was the wiki with a great deal of useful information about the city, how to get around, and so on. The Embedded Linux Conference Europe had videos (in free formats) available soon after the conference from Free Electrons (the GStreamer conference also had prompt videos that were made by UbiCast). It should also be noted that the Linux Foundation either runs or assists with many of the conferences that I have attended and it does a uniformly great job in putting those conferences on, from the venues to the program and keeping everything running smoothly as well.

We are lucky to have so many high quality conferences for Linux and free software. The schedule is packed, and there are good choices for most any area of our communities. While there are clearly areas where conferences could do better, it's hardly a weak point. It's certainly worth finding the time and budget to attend those that are in your areas of interest.

[ I'd like to thank the various sponsors, LWN subscribers in particular, for making it possible for me to attend—and report on—a wide variety of conferences. ]

Comments (39 posted)

Page editor: Jonathan Corbet

Inside this week's LWN.net Weekly Edition

  • Security: Sovereign Keys; New vulnerabilities in bind, chromium, freetype, squid, ...
  • Kernel: Drivers as documentation; The pin control subsystem; POSIX_FADV_VOLATILE.
  • Distributions: Ubuntu, Banshee, and default applications; Chrome OS Linux, Shuttleworth interview, Mint, openSUSE.
  • Development: Mozilla Popcorn: Enabling interactive video elements; Cappucino, Boxes, PyPy, ...
  • Announcements: Koha creators asking for help in trademark dispute, Interview with Andrew Tanenbaum,Open Source and the Open Road.
Next page: Security>>

Copyright © 2011, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds