LWN.net Weekly Edition for January 4, 2013

LWN's 2013 Predictions

It's that time of year again: a new year has begun, so your editor must duly come up with a set of predictions that, by the end of the year, will look either embarrassing or painfully obvious — or both. It is a doomed exercise, but, then, we all need some comic relief sometimes. So, without further ado, here's a few thoughts on what can be expected this year.

In 2012, the shape of a solution for the UEFI secure boot problem came into view. In 2013, the full messiness of the secure boot situation will come to the fore. There are already reports of some secure-boot-enabled systems refusing to work right with the Linux "shim" solution; others will certainly come out. We are, after all, at the mercy of BIOS developers who only really care that Windows boots properly. We are also at the mercy of Microsoft, which could decide to take a more hostile stance at anytime; there have already been snags in getting the Linux Foundation's bootloader signed.

UEFI secure boot bears watching, and we owe thanks to the developers who have been laboring to make Linux work in that environment. But the problem of locked-down systems is much larger — and much older — than UEFI secure boot, and many of the systems in question already run Linux. Maintaining access to "our" hardware will continue to be a problem this year, just like it has been in the past. "Runs Linux" will continue to mean something different than "the owner can run their own software on it," and UEFI secure boot does not really change the situation all that much, especially for non-x86 systems.

Maintaining access to our software will also be a problem; the usual hassles with software patents will continue, as will attempts to impose draconian surveillance laws, take over the management of the Internet, and so on. Such is life in the 21st century. One could hope that the patent issue, at least, would eventually get to the point where legislators feel the need to improve the situation even slightly; recent reports that a patent troll is shaking down companies for the novel act of using a scanner might suggest that we are getting closer. But the legal system excels at tolerating (and abetting) absurdity; don't expect any significant fixes in 2013.

The 3.12 kernel release will happen on November 20, 2013, or, at worse, by the beginning of December. The kernel development process has become a well-tuned machine with a highly predictable cycle; the longest cycle in 2012 (3.3) was only twelve days longer than the shortest (3.5). In the absence of significant externally imposed stress, it is hard to see anything changing that in 2013.

On the other hand, even your editor would not dream of trying to predict which features will be added in the 3.12 development cycle.

The community will continue to become less tolerant of unpleasant behavior from even its most prominent members. The history of free software resembles that of many frontier environments; at the outset there is a small set of explorers who work mostly below the radar. As the frontier is settled — free software becoming successful and often commercially driven — the small and young community begins to lose its "wild west" feel. In our rather larger and older community, the standards for behavior are becoming more stringent. At this point, almost nobody is seen as being so indispensable that we have to put up with them regardless of their behavior.

So, in 2013, we may well see more episodes where community members call out others for what they say or do and suggest that others refuse to associate with them. To an extent, that may lead to a more friendly and inclusive community environment. But a consensus on what constitutes acceptable behavior does not always exist. So we may also see energy going into personal fights that might be better directed toward more positive activities.

In a similar vein, recent resignations of GNU maintainers have made it clear that there is some disagreement within the organization on how decisions should be made. It may be time for a change of management in the Free Software Foundation and the GNU Project in 2013. Richard Stallman will certainly keep his place as the philosophical leader of both organizations, but it may become increasingly clear that his limited energies need not be absorbed by administrative matters. If these projects are to survive his eventual departure (nobody lasts forever), they would do well to better establish their processes and identities separate from their founder now.

"Vertical integration" will be heard often in 2013. There is no shortage of developers and companies asserting that the "distribution of a collection of disparate chunks of software" model is holding Linux back. Instead, they want to create a system following an overall design from top to bottom. The market performance of platforms like Android suggests that success may be found this way; now we have projects like GNOME OS, Firefox OS, and Ubuntu following similar paths. This kind of integration may well lead to a slicker result, but it also risks fragmenting the Linux world in a way that the traditional distributions did not.

This push toward vertical integration has generated a fair amount of conflict in our community; that will continue into 2013. Perhaps unsurprisingly, the strongest criticisms are reserved for those who are trying hardest to do this integration work as a community project rather than a company-controlled commercial product. As the Android developers have discovered, it can be a lot easier to just design a top-to-bottom system behind closed doors and release the result later. If we wish to minimize the fragmentation risk, we might want to engage more fully — and more constructively — with those who are doing their integration work in the open.

Some distribution will ship a release based on Wayland this year. It will be a painful experience for everybody involved. Wayland as a replacement for the venerable X Window System is certainly the future, but the future can sometimes take rather longer to arrive than one would expect.

Several new Linux-based platforms will ship on hardware in the coming year. We should see devices based on Firefox OS in some parts of the world. The Mer-based "Sailfish OS" may well be available in 2013. Samsung will likely ship Tizen-based handsets, the KDE-based "Vivaldi" tablet (or something descended from it) may actually ship, Ubuntu may show up on some mobile devices, and something totally unpredictable will probably materialize as well. There will be a wealth of interesting Linux-based choices, though they cannot all be expected to do well in the market.

Finally, LWN will celebrate its 15th anniversary on January 22. As we were thinking about what LWN might be in 1997, we could never have imagined that we would still be at it all these years later. Some people never learn, evidently. But, having come this far, we certainly don't plan to stop now. So, as we wish all of you a great 2013, we would also like to thank you for fifteen years of support, and all the years yet to come. We could not possibly ask for a better audience.

Comments (38 posted)

The 5.0 BlueZ

The BlueZ project, which provides the official Bluetooth support for Linux, unveiled version 5.0 on December 24, with the new major version number signifying changes in the API — changes found in several places in the Bluetooth stack. Only relatively recent kernels are supported due to reliance on some new kernel interfaces, several multimedia modules have been moved out to GStreamer or into separate projects, and there are new tools and supported profiles. The most significant changes are in the D-Bus API, however. Most (if not all) BlueZ applications will require a bit of porting, but the new interfaces should reduce work in the long run.

Interfaces

Linux 3.4 introduced a new Bluetooth Management kernel interface for interacting with Bluetooth adapters. BlueZ 5.0 now uses the new interface to manage local hardware. The new interface allows the kernel to manage all Host Controller Interface (HCI) traffic, as opposed to the 4.x stack's approach, with the kernel splitting that duty with user-space processes hooking into raw HCI sockets — and all of the synchronization and security problems that followed.

However, a recurring complication in the Bluetooth world is the standard's revisions and extensions. In particular, Bluetooth 4.0 added the Low Energy (LE) protocol designed to work with sensors and other Bluetooth devices capable of running off of small, "coin cell" batteries. Not only is the protocol different, but new adapter hardware is required on the host side as well. Support for Bluetooth LE adapters was not introduced until kernel 3.5; BlueZ 5.0 supports Bluetooth LE functionality only for this and subsequent kernels.

In user space, the most visible changes to working with BlueZ come from the D-Bus API. Version 5.0 has been rewritten to transform a number of previously custom interfaces into their standard D-Bus equivalents. For example, the BlueZ 4.x series provided its own methods for retrieving and setting an object's properties (GetProperties and SetProperty), and the PropertyChanged event to monitor an object for changes. In BlueZ 5.0, the standard D-Bus Properties interface is used instead.

Similarly, the generic D-Bus ObjectManager interface replaces BlueZ 4.x's custom interfaces for the tasks of adding, removing, and managing objects on the BlueZ D-Bus service. For the most part, this migration is simply the removal of duplicate, BlueZ-only methods from the code, however there are some significant changes as well. BlueZ 4.x had a org.bluez.Manager interface that enabled applications to find the attached Bluetooth adapters. BlueZ 5.0 has no equivalent interface, because D-Bus's generic ObjectManager provides an ObjectManager.GetManagedObjects method. This method returns every managed object in the BlueZ service, but, as the BlueZ 5.0 porting guide explains, applications need only scan through the returned objects to find an org.bluez.Adapter1 instance.

The "1" appended to org.bluez.Adapter1 is another change debuting in BlueZ 5.0; it is the version number of the new Adapter interface. All of BlueZ's interfaces now have a version number attached, with the project committing to supporting the two most recent versions of each interface. Unfortunately, this backward-compatibility pledge does not apply retroactively; the BlueZ 4.x interfaces — and their predecessors — have been dropped.

Devices and profiles

The changes outlined above all concern interacting with the locally-attached Bluetooth host adapter. But, as much fun as that is, a single Bluetooth dongle is not particularly useful on its own — users will eventually want to pair or connect to other Bluetooth devices.

BlueZ 5.0 introduces new methods for discovering and connecting to Bluetooth devices, again retiring a number of custom methods in favor of simpler, more general D-Bus APIs. For example, the CreateDevice and CreatePairedDevice methods were used to explicitly instantiate devices in BlueZ 4.x, but they have been removed entirely. Instead, the ObjectManager automatically creates org.bluez.Device1 objects for each device discovered during a scan, and automatically removes unused objects every few minutes.

There is also a new general-purpose Device1.Connect method that applications can call to connect to a discovered device without knowing in advance which Bluetooth profiles the device supports. This simplifies the process considerably, particularly when dealing with devices that support multiple profiles (such as audio devices, which for example could serve as hands-free phone accessories or music headsets).

API changes are not all that the update provides, however. The new release adds support for five new profiles, all of them of the Bluetooth 4.0 LE variety. Cycling speedometers and heart-rate monitors are the newly-supported hardware device profiles, and new functional profiles support Bluetooth LE's alert-notification, scan-parameter, and HID-over-GATT services. The alert notification and scan parameter profiles are precisely what one would guess given a few minutes to brainstorm; the first is a framework for sending alert messages of various types (such as "new message" alerts sent from a computer to a Bluetooth watch), and the second is a framework for servers and wireless sensors to agree on connection and timing settings, which are important for Bluetooth LE's advertised power savings. The HID-over-GATT profile is a way for low-energy devices to serve as human interface devices (HIDs) over Bluetooth LE's Generic Attribute (GATT) profile.

With these additions, BlueZ's supported profiles number more than 25 (the list at the project's site is out of date at the moment, but a more complete list is found in the Bluez git repository). The math can get a little fuzzy because some profiles are layered on top of other profiles, and it may make more sense to implement the higher-level profiles in other applications. For example, a 2011 Google Summer of Code project implemented the Phonebook Access Profile (PBAP) — which runs on top of the generic object exchange (OBEX) profile — in Evolution Data Server. BlueZ 5.0 pushes a few more profiles out of the BlueZ project itself; audio-video elements implementing the A2DP and AVRCP profiles were pushed upstream to GStreamer, and the telephony profiles HFP and HSP were dropped in favor of the external implementations in oFono.

Both the GStreamer and oFono profiles implement BlueZ's new org.bluez.Profile1 interface, which is designed for connecting to external processes that implement a Bluetooth profile. The interface expects some basic information common to all profiles, such as authentication behavior and service discovery protocol (SDP) records, but implementing the rest of the profile's behavior is up to the application. It is not clear how many external profile projects one could reasonably expect to appear, but BlueZ does still have a handful of unimplemented profiles to choose from (such as the blood pressure monitor profile BLP). Furthermore, it is certainly likely that the Bluetooth Special Interest Group (SIG) will come up with more profiles as Bluetooth LE continues to grow in popularity.

The BlueZ project did a bit of refactoring in this release as well, moving its implementation of the Sub-band codec (SBC) into a standalone project, and adding some new tools for testing and monitoring Bluetooth functionality.

For end users, BlueZ 5.0's immediate gains are the LE protocol support and the new profiles. These will enable users to pair and use the latest generation of Bluetooth devices — at least, until the next revision of the Bluetooth specification is turned loose. For application developers, the simplified APIs are no doubt welcome news (particularly removing custom interfaces in favor of standardized D-Bus alternatives).

This is especially helpful when it comes to implementing new device profiles not yet supported by BlueZ itself — because Bluetooth products vary so much and new ones are constantly popping up without warning. One might well argue that Bluetooth's design makes life easier from a hardware maker's perspective by cramming as much intelligence into the protocol and the software stack as possible; Bluetooth devices are cheap, plentiful, and widely interoperable as a result. But the downside is increased complexity required of the operating system and application developer. BlueZ 5.0 manages to simplify things for Linux, which is a significant accomplishment — hopefully one that will bear fruit in the coming months.

Comments (6 posted)

Previewing digiKam 3.0.0

DigiKam, the KDE-based photo organization tool, is slated to release version 3.0.0 at the end of January (to coincide with the release of KDE 4.10). The digiKam project has a rapid release schedule — with four or more stable releases per year being typical in recent years — which can make staying up to date a challenge. But the circumstances of my personal photo collection dictated an upgrade, so I spent some time at the end of December working with the latest build. DigiKam is not quite a one-stop-solution for photographers, although each release adds more features, but it still excels at the task of managing a large collection of images.

The "circumstances" I refer to are not unique to me; they are a common issue with digital photography and open source software. It started with the purchase of a new (as in just-released) DSLR camera. New hardware is an area where Linux photographers still suffer at the hands of the camera-makers, because the companies routinely change details in their cameras' raw photo formats. These companies typically provide their own Windows and OS X software for working with the new formats, but open source projects like dcraw and LibRaw must reverse-engineer those formats by examining donated sample images. Consequently, I was forced to use pre-release versions of various raw conversion applications in order to use the files produced by the new camera.

Working with pre-release editing software is not particularly painful, since daily builds and snapshots are standard practice for most of the projects — but then again, raw converters have the distinct advantage of being non-destructive, never touching the original file. It is riskier (at least to a degree) to use unstable code to manage the database of archived images, because data loss could occur and potentially harm many files at once. As a result, I ended 2012 with several thousand images that I had not imported into a digiKam collection. When the digiKam 3.0.0 release candidate was announced on December 29, I decided it was time to take the plunge (on a subset of the new images, to simplify matters).

DigiKam relies on a sizable number of external libraries, including several that are specific to KDE, so it can be an undertaking to compile if you do not run a recent version of KDE. That said, it did not require the absolute latest versions of its KDE dependencies, so it does not require installing bleeding-edge dependencies. In addition, digiKam does not impose nearly as many dependencies as some other KDE applications, so there is not significant overhead for those users running a GNOME or LXDE environment.

For the uninitiated, digiKam offers four major modes: the main collection manager, an image editor, a batch processor, and a "light table." The light table is a comparison tool designed to help you inspect multiple images in detail. In one sense it is just another view into the image collection, but it functions in its own window. In practice, a user would locate a set of images in the collection manager, then drag-and-drop them into the light table to zoom in and compare similar options side by side. The digiKam image editor is not as full-featured as any of the standalone Linux photo editors (such as Darktable or Rawstudio), and in particular it offers fewer features for working with raw image formats. Whether it meets any individual user's needs for a particular task is going to vary. The same would be said of the batch processor; most of the raw photo editors offer batch processing as well, and digiKam's batch support does not include every feature available in the editor, but it does include quite a few.

What's new in editing and export

In fact, the 3.0.0 release adds batch-processing support for several new effects and tools. Perhaps the most important is raw demosaicing, which allows the user a choice among multiple methods for transforming a raw image file's data into standard RGB pixels. Raw formats are in theory wrappers around minimally-processed sensor data, which often means that they incorporate different numbers of red, green, and blue pixels, arranged in specific patterns, and at unusual bit-depths. There is a never-ending disagreement on how best to interpolate this raw data into more traditional RGB triples; hence there are multiple demosaicing algorithms to choose from. The average user might not notice the difference, but sticklers for detail will enjoy the ability to choose.

Several other new batch-processing options have been added, including color transformation effects and cropping, both of which are more common tasks than choosing a custom raw demosaicing algorithm. It is also noteworthy that the batch processor was rewritten to parallelize image operations. As a result, processing a batch queue where several operations are performed on each file should be noticeably faster on multi-core machines by pipelining the images.

There is just one noteworthy addition to the image editing tools, automatic noise reduction, which was implemented as a student project during 2012's Summer of KDE program. This function uses digiKam's existing wavelet-based noise reduction feature, but it estimates the amount of noise in the open image and attempts to hone in on an appropriate level of denoising that will not adversely soften important image features. This feature may prove useful for batch processing, when the user cannot spare the time to inspect every image file, but it is also handy to simply start off with a good guess at the amount of noise needing to be removed.

DigiKam offers users a variety of export options beyond the simple still image, and the 3.0.0 release adds to the list. First, the application's database can now import metadata tags for video files, and users can incorporate videos into "slideshow" output. That does stretch the definition of "slide," but since most digital cameras support recording video, it is undoubtedly a useful feature. Second, at some point in the recent past digiKam evidently lost the ability to export photos directly to KDE's wallpaper collection (the environment's Plasma framework allows for automatic wallpaper rotation and other fanciful effects); 3.0.0 restores this functionality. Last but certainly not least, digiKam can now act as a Digital Living Network Alliance (DLNA) media server, which enables users to discover and browse image collections by album on any DLNA-aware product, such as the many "smart" TVs on the market.

Import and management

With the image editing and export tools, it is certainly possible to work entirely within digiKam, but I have always preferred to make use of its collection management features and jump back and forth between an array of other applications for editing. The 3.0.0 release only introduces one change to the image management tools: the ability to pre-assign labels and tags to a set of images at the time they are imported from a camera and into the collection. This is a time-saver; as memory card capacities increase, importing a card-full of photos takes longer and longer. In my experience, the chief reason most people do not maintain good metadata about the location and subject of their photos is that the tools make it too difficult. Adding tag metadata while importing them may not solve the problem entirely, but it helps — after all, the user is guaranteed to be present when attaching the camera and starting the import.

One of digiKam's biggest strengths is that is offers so much flexibility in where an image collection is stored and how it can be searched. Unlike most photo organizers, it can track images stored locally, on remote servers, and on removable media all in the same database. The search options include geotagging, time and date information, user-supplied text tags, a broad assortment of metadata options, and some more esoteric options like "image fingerprints." Fingerprints allow the user to sketch out an image, which is then compared against a mathematical decomposition of the images in the collection — it is essentially a find-by-visual-similarity search.

On the other hand, there is also one oft-requested feature that still has not made it into digiKam of 3.0.0: face recognition. Face recognition is a tricky task, to be sure, but in digiKam's case the feature was a Google Summer of Code project that was started but dropped by the student before it was complete. Cynics might suggest that this is an inherent problem with the Summer of Code method for feature-addition, but it is not preventable. After all, there is nothing that prevents a non-student contributor from dropping out of a project either. You can still manually tag individuals in an image, which is a feature that the facial-recognition search will presumably hook into. On its own, though, tagging an individual in the People search tab does not offer any advantage over putting the person's name into a text tag.

The 3.0.0 release candidate offers a nice set of new features for digiKam users, but it is probably still wise for the average user to wait until the final release. In my limited test, the application failed to import and convert the existing, older version of digiKam's collection database to the new schema used by 3.0.0. Although there are workarounds, such as manually moving the database with the application's built-in database migration tool, corrupting the database of a hefty collection is a major problem. The automatic schema conversion problem has been reported, so it will hopefully be fixed before 3.0.0 is released. Once the kinks are worked out and the final release is available, it is certainly worth a look.

Comments (none posted)