LWN.net Weekly Edition for January 24, 2013

Fifteen years of LWN

Sometime around mid-1997, your editor and Liz Coolbaugh were discussing ways in which one might go about starting a business around the Linux operating system. As we watched the Unix wars and awaited the arrival of the universally-proclaimed operating system of the future (a thing called "Windows NT"), we realized that Linux was the best hope there was for those who wished to avoid a future dominated by proprietary, closed systems. Linux seemed to be going somewhere in a time when Unix-based systems as a whole were beginning to struggle, and we wanted to help it get there. And besides, working with Linux was a lot of fun.

The idea we went with was to form a support company joined into Red Hat's ill-fated support partner network. But how were we going to attract customers — and keep busy while waiting for those customers to show up? The idea we came up with was simple enough: start a web-based newsletter to help the world keep up with what was happening in the insanely fast-moving Linux world (the linux-kernel list sometimes carried a shocking 100 messages in a single day back then) and, at the same time, inform that world of just how clever and on top of the situation we were.

So that is what we set out to do. The first LWN.net server went on the net in January, 1998, though we would not acquire that domain until much later that year. It ran on an old machine in your editor's basement and served its content over a single ISDN line. We published the January 22 issue when we had something that looked somewhat reasonable, thus establishing the Thursday publication cycle without any conscious thought on the matter. One week later, with a second issue written (headlined by the timely announcement that the Netscape browser would be open-sourced), we sent a message to the comp.os.linux.announce newsgroup telling the world of our existence, and life was never the same thereafter.

Like many business plans, ours failed to survive contact with the real world; a number of its successors fared no better. But, through it all, we kept LWN going. It didn't take long for the ISDN line to prove inadequate, even on a site with almost no image content at all. Linux began to take off for real as it led the final wave of the dotcom boom; LWN's readership rose with it. Eventually we realized that, while our various business schemes never seemed to get far, people were always interested in LWN. Far too late, we figured out that, perhaps, LWN was the business we'd been trying to build all along.

So, toward the end of 1999, we set ourselves to that task in earnest. Our long-suffering readers have heard much about our ups and downs over the years, but, by one obvious metric, LWN is a success: fifteen years after that first issue, LWN.net is still here. There is no shortage of work to do or things to improve, but somehow we seem to have found a way to do enough right to stick around.

We have watched Linux grow from a "hobbyist" system that few took seriously into the platform on which much of the world's computing is based. When we started, the number of people paid to work on Linux could perhaps have been tracked efficiently with an eight-bit variable; now it would be hard to even begin to guess how big the Linux employment market is. We have seen companies try to FUD Linux out of existence; others have tried to claim ownership of it. And we've seen Linux survive these challenges and more; Linux, too, is still here.

When LWN started, the community had no real idea of how to run a free software project involving hundreds or thousands of people. Those that tried often ran into trouble; the kernel process choked several times while others, like the project to make a workable browser out of the Netscape code, often seemed on the verge of collapsing under their own weight. The evolution of our software over the last fifteen years has been impressive, but the evolution of our community is doubly so. We can now take on projects that seemed unattainable even in the middle of dotcom boom optimism.

Fifteen years ago, we were a small, youthful band that thought it could change the world and have fun in the process. It is fair to say that both objectives were achieved nicely. Now we are numerous, older, professional, and tightly tied into the market economy; the wild-west days are mostly behind us. There will be plenty of work to do on Linux for a long time, but one might well ask: are our days of changing the world done?

The answer to that question is almost certainly "no." We have, at this point, succeeded in the creation of a large body of software that is not under the control of any one person or company. That software now forms the platform used for the growing swarm of ubiquitous devices; as these devices get smaller and cheaper, they will only become more prevalent. We have established the expectation that the code for these devices should be available and free, and we have promoted the idea that the devices themselves should be open and hackable. But we have not yet fully created the basis for free computing and, with it, a more free society. There is a lot of work to be done yet in that area.

When LWN got its start, our community's objective was simple, create a freer, better Unix. We have long since crossed that one off the list; now we need a better operating system for the devices — and the challenges — of the future. The problem is that we don't yet know what that operating system needs to look like. Unix embodies a great many solid design principles, but a system that was designed for slow terminals on minicomputers cannot be expected to be ideal for a phone handset, much less for hardware that we cannot yet envision. The system must evolve, perhaps in ways that cause it to diverge considerably from its Unix roots. Guiding that evolution without fragmenting our community or losing our focus on freedom will be one of our biggest challenges in the coming years.

The next fifteen years, in other words, promise to be just as interesting as the last fifteen were; here at LWN, we plan to continue to be a part of our community as those years play out. LWN, too, will need to evolve to best meet the community's needs, but, like Linux, we will evolve while keeping that community's values at heart. Thousands of you, the best readers one could possibly ask for, have sustained us for these years and helped to keep us honest. It is our hope to serve all of you even better in the coming years. It has been quite a ride; thank you all for letting us be a part of it. We are looking forward to seeing where it takes us next.

Comments (43 posted)

LightZone reborn as free software

One of the first high-quality raw photo editors available for Linux desktops was LightZone, but although it was (initially) free of charge, it was a proprietary product. Unfortunately the small company behind it eventually folded, and both the free and paid versions went away, as did the updates required to support newer cameras. The company shut its doors for good in 2011, but the software has made a sudden—and unexpected—comeback as an open source project. Fans of the original will be pleased, but the nascent effort still has considerable work ahead before it grows into a self-sustaining community project.

Flashback

LightZone was launched in mid-2005, first for Mac OS X, followed a few months later by Windows. But the application was written in Java, and in 2006 a developer at parent company Light Crafts began building it for Linux as well, posting the bundles (with permission) on his personal web site. The response was positive enough that Light Crafts soon began providing LightZone for Linux as an official release—one which, unlike the offerings for proprietary operating systems, was free. Perhaps that situation was bound to change (after all, there was evidently money to be made), and Light Crafts did eventually start charging for licenses on Linux, too.

But 2006 was also the year that resident 800-pound gorilla Adobe dove into the raw photo editor space with Lightroom and Apple's Aperture (which had been around in less-feature-filled, 1.0 form since 2005) really took off. Before Apple and Adobe entered the market, many small companies offered raw photo converters, but the heavyweights captured market share quickly. New point releases of LightZone continued to arrive, but with few major additions to the tool set. The last new version was LightZone 3.9, released in early 2010. Light Crafts shut down in 2011.

But the application's fans were still there; users seemed especially fond of LightZone's unique tools, which offered editing options not found in competing applications. These included an exposure tool designed around Ansel Adams's zone system and the ability to apply adjustments to one part of an image only by outlining regions directly on the canvas—plus a general reputation for ease-of-use. A user community emerged at the site LightZombie.org, providing updated versions of the dcraw library (on which LightZone's raw file decoding functionality depends), support files for new camera models, and (after the Light Crafts site went offline) Internet Archive links to the installer packages. Customers who had purchased a license key could still install and activate the archived packages, or use the built-in 30-day trial period.

Reboot

After Light Crafts closed up shop, visitors to the LightZombie site began lobbying to have the source code released. The site's administrators discussed the idea privately with former Light Crafts executives, but never made any progress—until December of 2012, when LightZombie's Doug Pardee posted a cryptic announcement that "In a few days, the LightZombie Project will be replaced by something much grander." There were other hints that the code might be released after all, such as the announcement that Anton Kast, the developer who had made the initial Linux port while at Light Crafts, had joined the project.

On December 22, Kast announced that he had convinced the rights holders to release the source code, and made it available at GitHub. Initially Kast made a direct import of the 3.9.x codebase, complete with the license-key-activation modules, without any documentation, and designed for the proprietary build system in use at the company. The LightZombie site was renamed LightZoneProject.org, and maintainers put out a call for volunteers in January, to which several Linux and Windows developers responded.

In the weeks since the initial release, the focus has been on getting the application to build and install successfully with free tools. The commercial product was packaged for distribution with Install4J, although as Kast pointed out on the developers' discussion forum (which at the moment seems to require membership in order to view messages ... ) that there may be little real need for an extra packaging layer, since wrapper scripts were used to launch the application on all three platforms. The IzPack tool was suggested as a plausible open source replacement, although so far it remains an open topic of discussion.

A bigger issue is the version of Java required. The commercial 3.9 release bundled its own version of Sun's Java 1.6, which was already out of date when Light Crafts ceased operations. It also relied on several Sun image processing classes that are no longer available, and some classes imported from Java Advanced Imaging (JAI) that were not part of the official JAI release at the time of 3.9's development. In addition, some Linux developers expressed an interest in getting the application to run on OpenJDK since it is the default on several major Linux distributions.

Over the following two weeks, though, the developers managed to successfully replace the Sun classes with modern equivalents, and used Apache Ivy to automatically pull in a current version of JAI at build time—a strategy employed by other open source projects. For now, Pavel Benak's branch is the focus of development, and the Linux port currently builds on Ubuntu and Arch Linux, either with OpenJDK 6 or 7 or with Oracle's Java 6, 7, or 8. The Windows build is reported to be working as well, albeit only with Oracle's Java 6. The Mac OS X platform, however, has seen no development so far due to a lack of volunteers.

Let there be zones

As advertised, the codebase on Github is essentially unchanged since the last commercial release of LightZone. Pardee has updated the dcraw library and support files, so newer cameras are supported, but the application still asks for a license key at start-up. However, the 30-day trial period is still enabled as well—a time period that can be reset at will.

The majority of the tools will feel familiar to anyone who has used another raw photo editor; just like the competition, LightZone allows the user to stack together a string of image adjustments by adding them to a list on the right hand side of the window. But LightZone does offer some tools not found in other open source photo editors. One example is ZoneMapper, the Ansel Adams–inspired tool mentioned earlier. Adams's "zones" are essentially ten stops between absolute black and absolute white. ZoneMapper presents a rectangle with ten handles on it corresponding to each of the zones, the user can drag the cut-off points up or down and the zones on either side are compressed or expanded as a result. The same effects could be performed with the traditional Levels or Curves tools, but ZoneMapper is much easier to use.

I stopped using the proprietary version of LightZone when it was no longer free (there were, after all, free software alternatives). As a result there are several new features that were new to me, although commercial LightZone customers will find them familiar. One of these is the Relight tool, which automatically brightens underexposed portions of the image. This, too, is an operation that can be done by hand with other tools, but what makes it worth mentioning is that it works quite well without manual intervention.

Not everything in LightZone is perfect; for example, the noise reduction tool crashed the application every time I tried it. Still, it is encouraging to see how well LightZone works considering that the open source project is barely a month old. End users may wish to wait until stable packages are available, but LightZone will hold its own against Rawstudio, UFRaw, Darktable, and RawTherapee. Developers from those competing projects may find the source to be interesting reading as well; in addition to the unusual tools, LightZone enjoyed a reputation for high-quality grayscale conversion and for good tool presets.

The road ahead

Obviously working out the bugs is important, but LightZone as a self-sustaining project has a more difficult task ahead of it in other areas. For starters, the project needs to formally decide on a license. The copyright document in the source tree was imported with the rest of the code; it bears a short, BSD-like copyright statement from Light Crafts' founder Fabio Riccardi and a 2011 date, but the project will need to make this clear. Moving forward, as Tex Andrews posted on the new project site, the group will have to start work on documentation, translations, and discuss "certain organizational issues that now confront us."

Andrews and Pardee, who managed the LightZombie project for more than a year, have cultivated an enthusiastic user base. That will supply the new project with momentum, but it does not guarantee that it will thrive. Keeping a community project alive takes considerable effort, as just about anyone in free software will attest. At the moment, the volunteers have updated dependencies and repaired the build system in short order, but the real work of refactoring the code and improving it has yet to start (case in point being the license-key activation, which should be straightforward enough to remove).

Nevertheless, the new project is a rare gift to software users. Many a small commercial application has vanished for financial reasons and had its disappointed users lobby for the release of the source code. Most of these appeals are unsuccessful. But these lobbying efforts have the potential to turn into successes—consider Blender, for instance. LightZone will at least get a second chance to win over users, which is an opportunity few applications see.

Comments (10 posted)

A brief survey of Linux audio session managers

Most of my music and sound work requires only a few applications. If I'm recording a song, I use a digital audio workstation (DAW) with a few plugins launched from within the sequencer, and if I make something in Csound, it's usually done completely in the AVSynthesis environment. The DAW and AVSynthesis programs are self-contained; they need no external software to store and recall their internal and external connections, plugin states, MIDI controller settings, and so forth. When I work in those environments, I know that I can open a previously saved session and have all former connections, states, and settings restored to their last conditions.

Recently I've been experimenting with a more heterogeneous workspace with several applications connected in a modular system. The JACK server allows flexible inter-application data exchange, and some powerful setups can be configured with the many JACK-compliant sequencers, synthesizers, and plugins. Unfortunately, the reconnection and reconfiguration of a complex modular system becomes a real problem when you've opened a DAW — with plugins, of course — and half a dozen standalone synthesizers are active with an equal number of standalone audio processors, each with its own internal settings and configuration details. The resulting maze of connections and configurations takes time to re-establish, valuable time that could be better spent on making music. Fortunately, help is here, thanks to Linux audio session managers.

A session manager (or "session handler") is a utility that stores and recalls every aspect of your audio production session. Typically each session has a unique identifier, so a click or two can reload, reconfigure, and reconnect all the pieces of your session exactly as you saved it. For smaller sessions, a manager may be a neat convenience, but for large-scale sessions with dozens of components the session manager is indispensable.

Technicalities

Session managers work in a few different ways, though all do the same things. Some can only manage applications that are JACK-compliant or that subscribe to a particular session protocol; others are more accommodating. Most managers include a graphical interface, but, since this is Linux, there are non-graphical solutions too. In fact, even the bash shell makes it easy to write a script that will relaunch an entire session. Such scripts are powerful and effective, but interaction is limited and it may be difficult to monitor and store state changes. Changes to your basic setup configuration may require additions to the script outside the session.

Session management protocols are designed to handle these matters. A basic session handler should be able to restore a session exactly as the user left it, but the handler must deal with a mash-up of harsh realities, including the internal and external states of host and clients (with display positions and sizes), the instantiation of all active plugins with their selected presets and internal edits, the loading of all necessary files used by the project, and the connections between the components in the system, including audio and MIDI I/O, track and bus assignments, and any other routing considerations. The session manager must be toolkit- and distribution-independent and it must handle the full variety of plugin and sound file formats. It should also be indifferent to the audio backend, but it should be aware of system services such as D-Bus. Finally, the manager should provide robust error reporting when things go wrong.

Session managers typically provide a library of services and a daemon to monitor and report process changes. An application that is linked to the manager's library can be monitored by the session daemon and update its state data for saving and recall. Obviously, the efficiency of any session management software depends on the services provided by the library and the client's implementation of the library services (if any).

In this article we'll look at the current crop of session managers for Linux audio production. We begin with a little history to understand how we've come to the present state of the art.

From LADCCA to LASH

The first attempt at Linux audio session management appeared as LADCCA, the Linux Audio Developer's Configuration and Connection API. Developer Bob Ham's LADCCA Reference Manual indicates that, even at the earliest stage, developers were aware of the basic problems and issues. LADCCA's development group expanded and the project eventually became LASH, the Linux Audio Session Handler. Focus remained on managing applications written for JACK and/or plain ALSA, with expanded services via D-Bus, improved signal handling, backward compatibility with previous APIs (including LADCCA), and other amenities. LASH developer Dave Robillard's LASH Manual provides more details for programmers. LASH became a popular solution for complex sessions and is still found in many software repositories.

On to LADISH

LADISH — the Linux Audio Developers (Integrated? Interesting? Indefatigable?) Session Handler — is the current successor to the LADCCA/LASH lineage. The project goals remain JACK-centric, with sophisticated client identification according to "levels" that define the detail of state storage and recall. D-Bus and jack-dbus are required, and the LASH and JACK Session APIs are supported.

The concept of levels is neatly summarized in the LADISH documentation, quoted below:

The use of these levels is an attempt to sort and regulate the various possible conditions for any Linux audio application. Those conditions include the degree of JACK compliance, any WINE or DOS requirements, network operation, the multiplicity of existing APIs, and so forth.

Like the original LASH project, LADISH includes a GUI (called gladish) for configuring LADISH and your session management preferences:

Gladish works perfectly, providing a neat concept of virtual Studios, Rooms, and Projects for organizing your session components. As an alternative, users of the KXStudio system can also choose to use its excellent Claudia utility, a custom-made frontend for the LADISH System:

A "getting started with LADISH" tutorial is on-line at the LADISH Wiki. Further excellent tutorial material can be found at Lampros Liontos's wonderful Penguin Producer web site.

The Non Session Manager

The Non Session Manager is a component in the Non* suite of audio/MIDI software by Jonathan Liles. It can be used as a standalone program without the rest of the suite, but you will need to install the core NonToolKit library first. Check your distribution's software repositories for an installable Non suite. If it hasn't been packaged for your system, you'll have to visit the Non Web site and follow the instructions there regarding access to the project's Git repositories. The suite is easy to compile, with only a few dependencies, all commonly found in any modern Linux distribution repositories.

After installing the program it's ready to run. The command-line invocation takes two possible options, one to specify the URL of the Non Session Manager daemon (nsmd) and one to indicate a particular root path for your sessions. When the session manager starts, it requires no further configuration.

The Non Session Manager is easy to use. The top button bar exposes most of its functions, though there are a few hidden features. The "New" button starts a new session, "Add Client" adds an application to the session, "Save" saves the session, and "Close" closes it. The "Open" button opens an existing session, or you can click on the session name in the list box, where we encounter hidden feature number one: clicking on the session name will immediately close the running session and load your selection, so be sure to save your work first. The "Duplicate" button copies a running session as a template for later similar sessions.

When you add a client to a session you might discover another hidden feature, the nsm-proxy client. This convenient feature lets you add clients built without Non Session Manager or other session management support, though the addition is limited to opening and closing the particular client. If a client has LADISH level-1 support (or another method of saving its internal state), you can also select a "Save State" signal, though the Non Session Manager may not directly handle the state data.

Yet another hidden feature: When adding clients you can add a special client named JACKPatch that will store your JACK audio and MIDI connections. You can save a JACK connection map as a Patchbay in QJackCtl, but of course you need to be running QJackCtl. The Non Session Manager JACKPatch client provides a handy means of storing and restoring your connections without an external application.

The Non Session Manager also has features of interest to users of networked audio systems. Its documentation states that the Non Session Manager is the only session manager capable of saving and restoring session configurations of networked machines. Alas, I was unable to test a networked session, so I must refer the reader to the Non Session Manager Web site for more information regarding the handler's implementation over a network.

Of the session handlers I tested, the Non Session Manager gets top honors for features, flexibility, ease of use, and transparency. It does its job, otherwise it stays out of the way. It doesn't interfere with other processes, and it can be reconfigured interactively during a session. The Non Session Manager isn't perfect — I'd like to have a more direct method of entering nsm-proxy clients, and I might like to be able to rename sessions, but these are minor issues. Otherwise the Non Session Manager is highly recommended Linux audio software.

QJackCtl

The image below shows QJackCtl's JACK Session manager at work.

If your modular configuration includes only JACK-compliant applications and you use QJackCtl to manage connections, you might consider using it to manage your sessions as well. Operation is simple enough — just click on the Session button and follow the prompts for saving your work. The manager saves all aspects of its clients and restores them exactly. Incidentally, Ardour subscribes to the JACK session management protocol, making it a simple matter to include its considerable powers into a modular setup.

Programmer's documentation for the JACK session API can be perused at jackaudio.org. For normal users, the Tango Studio project has provided some excellent documentation on its helpful JACK Session tutorial page.

Chino

In my research for this article, I discovered Chino, a scripted session manager with no ties to previous management protocols. According to its author, Chino is "... a framework and toolset to build and manage a meta-application consisting of the user's favorite modular JACK audio and MIDI tools, each started and interconnected in predefined ways." Chino's developer is rather modest about his project, but I found it to be a well-designed tool that may be just the item you need if the "heavier" solutions don't suit your project requirements and you're comfortable with the Linux command line.

Installation and configuration are covered in Chino's documentation. No unusual dependencies are required, but you'll need the jack_snapshot utility to store and recall your JACK audio and MIDI connections. You can also save an SVG graphic display of your session's status. You'll need the Xsvg viewer to view it from within a Chino session, though any SVG-capable viewer can be used to view the image outside the session.

Chino defines a session in a method/application hierarchy. According to the documentation, methods are "categories for applications that are to be handled in similar ways. Every application belongs to one method." Following that definition, the documentation informs us that applications can be "(almost) anything... one or more (audio)-programs, audio/MIDI hardware or netjack ports — many things can be crammed into an application." In practice it's all simple enough. Start with this command to start a new session:

	chino -n newsession

Follow the prompts, and be sure to use the h (Help) and w (Write) options liberally. You'll find your way around Chino quickly enough.

If your saved session is named newsession.sdef you'll reopen it with this command:

	chino -o newsession.sdef

Chino is easy to use and does its job quickly and efficiently. However, it is still in development, with a few rough edges yet to polish. I created a test session from its default settings for the amSynth and Yoshimi synthesizers, with normal audio/MIDI connections made with QJackCtl. I saved the session, but, on closing, amSynth had to be manually stopped. On reopening the session, the synthesizers appeared as expected, but only amSynth's connections were restored. I must emphasize that this is a single test case and in no way stands as a general estimation. I'll need to spend some more time with Chino — there's much more to it than what I've presented, and it is definitely worth watching.

Summing the signals

When I started writing this article I had little experience with session managers. I'm still no expert, still having a lot to learn about these projects, but I'll hazard an opinion or two. Currently I prefer the Non Session Manager for the greatest number of my projects. The LADISH-based handlers are advised for sessions with only JACK-aware applications, though QJackCtl's session handler may be preferred when using that utility. Chino is available for command-line users, as is the humble and powerful bash scripting language.

Modular music systems are difficult to configure and restore, but by definition what is difficult is not impossible. Apparently the developers of these programs and utilities have a similar attitude towards difficult things, and I'm happy to report the ever-improving states and conditions of their projects.

Comments (2 posted)

Page editor: Jonathan Corbet

Inside this week's LWN.net Weekly Edition

• Security: HTTPS interception in Nokia's mobile browser; New vulnerabilities in ganglia, kernel, mysql, nagios, ...
• Kernel: Supporting variable-sized huge pages; GPIO in the kernel: future directions; Making EPERM friendlier.
• Distributions: Jolla and the Hildon Foundation struggle to cooperate; CentOS, Fedora, SolusOS, ...
• Development: More on PID namespaces; Trinity 1.1; Parrot 5.0; Firefox OS developer phones; ...
• Announcements: FSF: End Software Patents, FOSDEM speaker interviews, ...

Next page: Security>>