LWN.net Logo

Development

Using Linux to manage a large audio collection

Recently, your author decided to dig into a long-delayed project, the consolidation of a large collection of music onto a hard drive-based archive. Over the years, a large collection of music has been built up, the majority of which is live concert recordings from various tape trading networks. Prices of hard drives in the several hundred gigabyte size range have been steadily falling, making an online audio archive possible and affordable.

Music recording technology has gone through an amazing number of technology changes in the last 30 years. First there were reel-to-reel tapes, then audio cassettes. VHS-HiFi was a short-lived medium that offered improvements over cassettes until the audio DAT tape showed up. After that was CD-R media. Add to that vinyl records, commercial CDs, sound tracks from video tapes, and live recordings from multi-track digital recorders. Recently, a large number of BitTorrent sites have been good sources of live music recordings.

Your author is somewhat picky about audio standards, although he routinely shuns audiophile audio hardware. The Microsoft .wav file format seemed like the universal standard to use as an archive medium. The 44.1K sampling rate was chosen due to its compatibility with audio CDs. Compressed formats like Ogg Vorbis and MP3 have their uses for portable players, but for archiving purposes, .wav is for the most part, the best sounding and most universally workable standard.

Taking that one step further, .wav files can be bit-for-bit converted to and from FLAC, the Free Lossless Audio Codec. FLAC typically squashes music by about 50% of its original size, and .flac files can be played directly with music players such as XMMS. It is trivial to re-convert .flac files to .wav for burning CD-Rs.

The standard Linux filesystem was chosen as way of storing the archive. It offers a wealth of handy command-line utilities for management, and GUI-based interfaces for those who prefer the that mode of operation. Some conventions were chosen for representing the music from a typical live concert. Each concert gets stored in a unique directory named ArtistYYYYMMDD. Individual songs were stored in their own .wav files with a similar naming scheme: ArtistYYYYMMDDd#T##.wav where YYYYMMDD is the date, d# is the (optional) disk number from a multi-CD set and T## is the track number. Additionally, a file called ArtistYYYYMMDD.txt can be included to contain a textual description of the audio source, processing information, song lists, and other information. It would be nice to have a meta-data file such as an XML file that contains more information in a computer readable format, but that's for the future.

A big collection of CD-burning source material was recently rescued from an old computer system and a big pile of backup tapes from the same machine. The majority of this data consisted of 1-2 GB .wav files that were created by copying audio DATs into a CMI8732-based sound card with a lossless S/PDIF digital audio interface. The audio stream was converted to .wav files with the obsolete SoundRecorder utility, or Cinelerra. Sound from analog sources was fed into the computer using an external Flying Cow Analog to S/PDIF converter.

Many of the audio DATs were recorded with a 48Khz audio sampling rate; those were converted to 44.1Khz files with SoX. Most of the source material was stereo volume normalized with Ecasound and some custom audio scripts. Volume normalization is a bit of an art, it usually works best on large parts of the source .wav file, broken up by concert sets, or where the person running the recorder tweaked the recording level. Normalizing groups of songs gets rid of annoying volume changes from song to song.

Although Cinelerra is primarily a video editor, it has the critical ability of being able to digest a 2GB audio file without choking. It also has the ability to mark audio edit points throughout the source material, and bulk-render individual .wav files as marked by the edit points.

After all of that, the music archive is showing many signs of improvement. All of the long gaps and audience chatter has been removed from the source material. The volume of the .wav files is fairly constant from concert to concert. The songs are now accessible individually. Your author was lucky in that he discovered how poor the CD-R medium is for long time archiving before he deleted his source .wav files. CD-R media degrades with heat, dirt, and repeated handling.

Now that the music archive is coming together, some big advantages are beginning to surface. It is possible to copy the entire archive to another computer with a one line ssh/tar command. This is extremely powerful for backing up the data or copying sections to a friend's computer. A spare computer can easily be retrofitted with a large hard drive, then turned into a music library audio appliance. Although not as portable as an mp3 player, the entire archive can be copied to a laptop for listening away from home.

This is very much a work in progress, it is also a process that will never be finished. The archive is up to 45Gb and is growing daily. Some software remains to be written. A random music player should be easy to write with just a few lines of Python code. That could be extended to include more advanced features such as noting song groups that should always be played together, and skipping files that contain short spoken segments such as a band introduction. In the source material, files are occasionally split into two pieces due to the editing out of a bad section in the middle, your author is still searching for a way to join two .wav files into one to fix that problem.

Linux and the wide variety of open-source tools have made this entire process a breeze, if somewhat time consuming. Audio recording has gone through a series of diverging technologies, this distillation effort has reversed that trend.

Comments (19 posted)

System Applications

Database Software

MySQL 5.1.7-beta has been released

Version 5.1.7-beta of the MySQL database is available for testing. "This is the first published Beta release in the 5.1 series. All attention will continue to be focused on fixing bugs and stabilizing 5.1 for later production release."

Full Story (comments: none)

pgAdmin III v1.4.2 Released

Version 1.4.2 of pgAdmin has been released. "The pgAdmin Development Team are pleased to announce the release of version 1.4.2 of pgAdmin, the Open Source administration and development platform for PostgreSQL 7.3 and above. pgAdmin can be run on Linux, FreeBSD, Mac OSX, Solaris and Windows."

Comments (none posted)

PostgreSQL Weekly News

The March 5, 2006 edition of the PostgreSQL Weekly News is online with new PostgreSQL information and events.

Full Story (comments: none)

Mail Software

Bogofilter 1.0.2 released

Version 1.0.2 of Bogofilter, a Bayesian spam filter, is available. "This release fixes has some minor configuration script scripts, some minor option errors, suppresses multiple messages when the database nears its maximum size, has an emacs VM entry in the FAQ and updated emacs VM support."

Full Story (comments: none)

Gotmail 0.8.8 Released

Version 0.8.0 of Gotmail, a Perl interface to hotmail.com, is out. "This is a simple maintenance release with only a few basic bug fixes applied including better dealing with spaces in folder names and extra lines in config files."

Full Story (comments: none)

Web Site Development

Campsite 2.5 announced

Version 2.5 of Campsite, an open-source multi-lingual content management system for news websites, is out. "Version 2.5 is a major feature release."

Full Story (comments: none)

Using Ajax from Perl (O'Reilly)

Dominic Mitchell discusses the use of Ajax from Perl on O'Reilly. "If you're even remotely connected to web development, you can't have failed to have heard of Ajax at some point in the last year. It probably sounded like the latest buzzword and was one of those things you stuck on the "must read up on later" pile. While it's definitely a buzzword, it's also quite a useful one. Ajax stands for "Asynchronous JavaScript and XML." It's a term coined by Jesse James Garret in "Ajax: A New Approach to Web Applications." Ignore the football team, they're mere impostors. ;-) What does that actually mean? In short, it's about making your web pages more interactive."

Comments (none posted)

Scheduling Jobs in a Java Web Application (O'ReillyNet)

Chris Hardin discusses Java Web Application scheduling on O'Reilly. "Web application frameworks are built to service requests when they come in, typically from web users. This seems fine, but what if you need to execute code at specific times (for example, to generate reports in the middle of the night when CPU use is low)? Quartz provides best-of-breed Java scheduling functionality, and in this article, Chris Hardin shows how to get Struts to load up Quartz and your scheduled work."

Comments (none posted)

Miscellaneous

Open Graphics schematics posted

The Open Graphics project is working toward the development of a completely open 3D graphics adaptor for use with free systems. Last week, the project announced its intent to release the first set of schematics for wider review. The schematic [PDF] and a slightly outdated bill of materials are now available. Quite a bit of work remains to turn this design into an actual product, but the developers are working with the apparent hope of getting something out this year.

Comments (12 posted)

New Live CD showcasing XGL (FootNotes)

If you are curious about what can be done with all the 3D work going on: FootNotes has a blurb about the Kororaa live CD, a Gentoo-based live system with all the Xgl goodies on it.

Comments (11 posted)

Desktop Applications

Audio Applications

FUPlayer 0.2.1 is available

Stable version 0.2.3 of FUPlayer has been announced, it features bug fixes. "FUPlayer is a full featured music manager and player for the GNOME desktop. With it, you can play music from your hard drive, create playlists, do real file management using its Trash, and play, rip, and burn audio CD's. It features an interface similar to those of many modern manager-style players, but with many improvements, such as true non-modal search and browse functionality, find-as-you-type, and drag destination highlighting." See the CHANGELOG file for more information.

Comments (none posted)

QjackCtl 0.2.20 released

Version 0.2.20 of QjackCtl, a Qt interface to the JACK Audio Connection Kit (JACK) is out with a number of new capabilities.

Full Story (comments: none)

Rhythmbox needs YOU! (GnomeDesktop)

The Rhythmbox project needs volunteers to work on its music player playlist feature. "Rhythmbox 0.9.3 gained support for all mass-storage audio players (auto-detected if HAL knows about them, or via .is_audio_player). What it doesn't have is support for playlists on all those players. Last night we committed support for reading playlists from PSPs, and we want to add support for the rest. If you have an audio player which uses playlists (and isn't an iPod or PSP), you need to run to bugzilla and tell us about it."

Comments (none posted)

CAD

Twenty-ninth release of PythonCAD available

Release 29 of PythonCAD has been announced. "The twenty-ninth release of PythonCAD contains various improvements to the internal entity creation and manipulation code. The routines for transferring entities between layers has been reworked, as have the routines for deleting entities. This code rework flushed out a number of bugs and sub-optimal code issues which have been resolved."

Full Story (comments: none)

Desktop Environments

GNOME 2.14 Release Candidate (2.13.92) Released!

GNOME 2.13.92 has been released. "This is our last unstable release before the big .0 release. Lots of new features and bug fixes have been added during this cycle, probably more than what you can remember if you've been running all the unstable releases so far."

Full Story (comments: 1)

GARNOME 2.13.92 (aka 2.14 RC) is out

Version 2.13.92 of GARNOME, the GNOME testing distribution, is out. "This release includes all of GNOME 2.13.92 (aka 2.14 Release Candidate) plus a whole bunch of updates that were released after the GNOME freeze date."

Full Story (comments: none)

GNOME Software Announcements

The following new GNOME software has been announced this week: You can find more new GNOME software releases at gnomefiles.org.

Comments (none posted)

KDE Software Announcements

No new KDE software announcements were received this week. You can find the latest KDE software releases at kde-apps.org.

Comments (2 posted)

Electronics

gEDA 20060123 announced

The OpenCollector Database has an announcement for a new release of the gEDA suite, a collection of electronic design automation (EDA) tools. "This release includes schematic backup and autosaving, a new "L" net drawing mode, improved PNG export, new file selection dialog boxes (when using at least GTK+ 2.4.x), embedded picture support, spice-sdb backend improvements, and many bugfixes."

Comments (none posted)

Kicad 2006-03-07 released

Release 2006-03-07 of Kicad, a printed circuit and electronic circuit CAD application for KDE, is available with bug fixes and other improvements.

Comments (none posted)

Financial Applications

Release of GnuCash 1.9.2 (unstable) (GnomeDesktop)

Unstable version 1.9.2 of GnuCash, a financial management application, has been announced. "The GnuCash development team proudly announces GnuCash 1.9.2 aka "With extra flavor enhancements", the third of several unstable 1.9.x releases of the GnuCash Open Source Accounting Software which will eventually lead to the stable version 2.0.0. This release contains many bugfixes since the second release but is still only intended for developers and adventurous testers who want to help tracking down bugs."

Comments (none posted)

Games

Ember 0.4.0 released

Version 0.4.0 of Ember has been announced on the WorldForge game site. "Ember is a fully functional 3d client for the WorldForge project. ItÂ’s meant to be as extensible as possible, to allow for future world builders to adapt it to their worlds or games. This release adds a lot of new features and refactored code. One of the main changes is the addition of scripting language support. Many of the existing gui components and widgets have been converted to Lua code. Together with a build in code editor and a new more robust widget look this makes for a solid foundation for further gui development."

Comments (none posted)

Interoperability

Wine 0.9.9 released

Version 0.9.9 of Wine has been announced. Changes include: Many new features and improvements in Richedit, More Web browser support, Recursive directory change notifications, Wine installation is now fully relocatable, Direct3D 8 and 9 now use the same code, Many debugger improvements, Systray is now handled by the explorer process and Lots of bug fixes.

Comments (none posted)

Wine Weekly Newsletter

The March 5, 2006 edition of the Wine Weekly Newsletter is out with the latest Wine project news.

Comments (none posted)

Music Applications

dssi-vst 0.4 released

Version 0.4 of dssi-vst, a DSSI plugin wrapper for Win32 VST plugins, is out. "The main change since the 0.3.1 release is that dssi-vst now builds with newer versions of the Wine tools. Wine 0.9.5 or newer is now required. This release also builds with version 2.4 of the VST SDK, although it should still work with the older 2.3 as well."

Full Story (comments: none)

Qsynth 0.2.5 released

Version 0.2.5 of Qsynth, a Qt interface to FluidSynth, is out with new features and bug fixes.

Full Story (comments: none)

Office Suites

OpenOffice.org 2.0.2 released

OpenOffice.org 2.0.2 is out. This release adds integrated spellchecking dictionaries, some new import filters, KDE address book interoperability, and a new set of icons.

Full Story (comments: 6)

Miscellaneous

File permissions in Nautilus (GnomeDesktop)

GnomeDesktop.org points to a blog about Nautilus file permission modifications. "Christian Neumair has posted an overview of some of the work he is doing with regards to changing file permissions within Nautilus. A very good read for those that are interested."

Comments (none posted)

Languages and Tools

Caml

Caml Weekly News

The March 7, 2006 edition of the Caml Weekly News is out with new Caml language articles.

Full Story (comments: none)

Java

GNU Classpath 0.90 released

Version 0.90 of the GNU Classpath essential Java libraries have been released. Changes include: "JTables can be rearranged and resized. Free Swing text components support highlighting and clipboard. Much improved styled text. Fast event dispatching and lower memory consumption. Better support for mixing lightweight and heavyweight components in AWT containers. GNU Crypto and Jessie cryptographic algorithms have been added providing ssl3/tls1 and https support. Unicode 4.0.0 support. GIOP and RMI stub and tie source code tools. XML validaton support for RELAX NG and W3C XML schemas. New file backend for util.prefs. Updated gnu.regexp from POSIX to util.regex syntax."

Full Story (comments: none)

PHP

PHP OpenID 1.0.0 Released

Version 1.0.0 of the PHP OpenID library is available. "JanRain, Inc. is proud to announce the first stable release of our OpenID library for PHP! This release incorporates improved documentation, bugfixes, a configuration helper for the server example, and support for installation with the PEAR command-line installer."

Full Story (comments: none)

PHP Weekly Summary for February 27, 2006

The PHP Weekly Summary for February 27, 2006 is out. Topics include: True labelled break goes off-list, portable string API in HEAD, segfault recovery, life without $this, stack overflow prevention, deprecation macros in 5_1, and late static binding patch.

Comments (none posted)

Python

PyCon Python IDE review

Jonathan Ellis presents a Python IDE review from the recent PyCon convention. "I presented an IDE review at PyCon last Friday. It was basically a re-review of what I thought were the 3 most promising IDEs from the Utah Python User Group IDE review, to which I added SPE, which was by far the most popular of the ones we left out that time. The versions reviewed are: PyDev 1.0.2, SPE 0.8.2.a, Komodo 3.5.2, Wing IDE 2.1 beta 1".

Comments (none posted)

Dr. Dobb's Python-URL!

The March 6, 2006 edition of Dr. Dobb's Python-URL! is online with a new collection of Python article links.

Full Story (comments: none)

Ruby

Ruby Weekly News

The March 5, 2006 edition of the Ruby Weekly News looks at the latest discussions from the ruby-talk mailing list.

Comments (none posted)

Tcl/Tk

Dr. Dobb's Tcl-URL!

The March 6, 2006 edition of Dr. Dobb's Tcl-URL! is online with new Tcl/Tk articles and resources.

Full Story (comments: none)

Miscellaneous

SchemaSpy 2.1.2 released (SourceForge)

Version 2.1.2 of SchemaSpy has been announced. "SchemaSpy analyzes schema metadata, letting you click through the hierarchy of your tables' parent/child relationships either graphically or through HTML tables. It works with just about any RDBMS given an appropriate JDBC driver. SchemaSpy also identifies several common schema anomalies."

Comments (none posted)

Page editor: Forrest Cook
Next page: Linux in the news>>

Copyright © 2006, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds