Lessons from PostgreSQL's Git transition

The PostgreSQL Project finally switched from CVS to Git in September 2010, and did its first release based on the new Git repository on October 5. Making the switch happen took years and resulted in at least one near-disaster. Other projects that are contemplating, or working on, a transition in their version control system may find useful lessons in how PostgreSQL fared.

A History of CVS

Switching version control systems is a relatively straightforward process for a young project, but not for an old one. From 1996 through mid-September 2010, the PostgreSQL Project was developed using CVS. In fact, the date the CVS server went live — July 8, 1996 — is generally considered the "birthday" of the open-source project, which came out of the ten-year-old university project. Twenty-one major releases and 154 minor releases were committed, branched, and packaged on CVS. A large web and development infrastructure existed around CVS, as well as a multi-step, multi-role release procedure.

In 2004, as Subversion was beginning to become popular, PostgreSQL contributors first started to argue about switching away from CVS. However, Subversion was not seen as sufficiently mature at the time, or as offering enough advantages over CVS. This discussion, and occasional flamewars, continued to crop up on the main "pgsql-hackers" mailing list. Those who wanted to migrate off of CVS split into multiple camps based on the system they preferred, including Subversion, Arch, and Monotone. And, of course, a large group of developers saw no reason to change at all.

As with many projects which operate by rough consensus, where there is no consensus, there is no action. The community verdict was to wait for one version control system to become the clear leader. In retrospect, this turned out to have been a good decision.

First Git Mirror

In 2007, one of the git-cvsimport maintainers from New Zealand decided to set up a persistent and frequently updated Git mirror for the PostgreSQL CVS tree. This mirror was extremely popular and one Google Summer of Code student even did his project using the Git mirror instead of the CVS repository. An increasing number of PostgreSQL contributors started using Git mirrors.

In 2008 a few members of the PostgreSQL web team decided to set up an "official" git.postgresql.org, using FromCVS for conversion, and Gitweb with custom administration scripts for the web version. This was done without obtaining community consensus, and caused some controversy. But once the flames died out, many people started using the repository. However, the mirror became a source of frustration for the developers who depended on it. Synchronization with CVS was often undependable, with changes made in CVS failing to show up in Git.

While one or two Subversion mirrors were created well before the Git mirror, and there was even a Bazaar mirror on Launchpad, none of these were at all popular. By mid-2008, it was clear that a majority of PostgreSQL developers favored an eventual switch to Git.

Evaluation

pgCon is the annual international PostgreSQL conference in Ottawa, Canada. By pgCon 2009, PostgreSQL was one of only a handful of major projects still using CVS. It was time to switch to something, and a long discussion ensued at the developer meeting. With a version of Git available for Windows, a major hurdle had been cleared, but several issues remained to be solved, most of them having to do with project infrastructure.

First, the project has an automated regression testing infrastructure called the PostgreSQL Buildfarm. This network of donated servers and virtual machines does daily or hourly CVS checkouts, builds PostgreSQL, and runs regression tests. The Buildfarm would need to be updated to use Git, which was a challenge because not all of the operating systems represented in the build farm (such as UnixWare and AIX) had Git packages available.

The second challenge was the code review process. Using CVS, PostgreSQL contributors submitted their patches in context diff format to the pgsql-hackers mailing list and the CommitFest application. The committers did not want to change this process.

It was also unclear whether it would be possible to recreate past releases from Git.

Decision to Switch

By pgCon 2010, these issues had been resolved. The Buildfarm code had been patched, and the web team planned to set up a CVS mirror of Git for the build farm members who could not run Git. Developers had tested several back branches, and tweaked the conversion process until it was possible to produce a back-branch release which was identical to the one produced by CVS.

The Buildfarm developers, primarily Andrew Dunstan, worked on transitioning build farm members to using the Git mirror in advance of the switchover. However, the git.postgresql.org mirror proved too unreliable for this purpose, and Andrew had to set up another mirror on Github using some custom scripts. That took longer than the changes to the Buildfarm client code. But it worked, and the build farm servers started to convert to Git.

Accordingly, those at the developer meeting set a date: on August 18th, 2010, PostgreSQL would switch over. The Git repository would become the canonical code base and the CVS repository would become the mirror. This date was based on the assumption that 9.0 would be released by then, and August 18th would be after the first Alpha release for 9.1. Suitably, at pgCon Andrew did a talk on "Git for PostgreSQL developers".

Failure to Switch Over

By August 13th, things were looking somewhat different; 9.0 was not released yet. But everything else was scheduled, so the conversion went ahead. First, the developers froze the CVS tree. Next, Magnus Hagander employed the cvs2git tool to do a final conversion from CVS to Git. Then all of the PostgreSQL developers were asked to test the new repository. Things looked OK.

Then, the day before the Git repository was expected to be opened to new patches, committer Robert Haas noticed a problem:

2010-02-28
        PostgreSQL...
        This commit was manufactured by cvs2svn to create branch REL8_3_STABLE

It seems that, where we had ported patches from later versions to earlier versions, cvs2git had manufactured inappropriate merge commits. Nobody had noticed because it didn't affect the head of any branch, just the history.

We reverted to CVS, and postponed switchover until after 9.0 was released.

The Second Conversion

Over the next few weeks, web team member Magnus Hagander worked with CVS2git developers Max Bowsher and Michael Haggerty to clean up issues with the conversion. In addition to the false merge commits, there were quite a number of other artificial commits and weird history in the converted version, such as branches which didn't exist and reappearing deleted files. A good portion of this was due to the fact that not only were we running an old version of CVS, but repository steward Marc Fournier and others had also done "CVS surgery" a number of times to fix problems over the years.

A second conversion test broke the ability to recreate any release. We discovered the new Debian server we were running it on did not default to ISO date format and had changed all the date strings for the releases. Fixed. A fair amount of "dirty history" in CVS simply could not be converted and needed to be patched in CVS before conversion. That was patched by Tom Lane.

Magnus, Max, Robert, and Tom stuck with it and resolved all of the issues which were considered roadblocks. cvs2git's next release will contain several improvements which are a result of our conversion. Several community members also added documentation to the PostgreSQL wiki on how we would use Git for development, including how to use git.postgresql.org, and how to commit.

On September 20, we released version 9.0 of PostgreSQL. Since Magnus was not heavily involved in the 9.0 release, he was able to spend his time preparing. So, on September 21st, we switched to Git.

Not Done Yet

However, the actual day of conversion was hardly the end. There was still a lot of minor cleanup to do. Contributors had to prune fictitious branches, move tags which appeared in the wrong place, and clean up many other issues.

Because the community wanted to preserve old releases exactly as they had been in CVS, we needed to preserve the file header tags in each file and remove them only from the "tip" of each branch. These are the comments which CVS automatically creates in each file which look like this:

    * $PostgreSQL: pgsql/src/tools/fsync/test_fsync.c,v 1.27 2010/02/26 02:01:39 momjian Exp $

Git doesn't use these tags since they don't work with atomic multi-file commits, so they needed to be removed from current development without removing them from the history.

Administrators also had to fix people's usernames, since contributors had freely used idiosyncratic nicknames and incomplete user information for git.postgresql.org before it was the canonical repository. These were changed to their main e-mail addresses with their real names. And, most of all, hackers who had waited until the conversion to learn Git were asking for command help on the mailing lists.

After the conversion, the project had to resolve policies and contents for .gitignore files, which tell Git what kinds of files to ignore. These are used to prevent developers from accidentally sending in patches containing editor backup files or build artifacts.

One issue which still isn't resolved is the CVS mirror for the Git repository. git-cvsserver turned out to have serious scalability limits, which makes it too limited to support the build farm servers. As a result, Andrew Dunstan required all of the build farm machine owners to migrate their nodes to Git immediately rather than gradually. However, there are still a few machines which are very valuable for testing and cannot run Git and, so far, there is no solution for those.

No Merge Commits

PostgreSQL had introduced a new workflow for reviewing patches in 2008, called "CommitFests" — a bi-monthly process where committers and reviewers clear out the pending patch queue. By 2010, the bugs had been worked out of the CommitFests and everyone thought they were working well. Among other things, the new workflow was helping train new contributors. So nobody wanted to change it. Also, many committers were okay with the migration only if they could still review patches the way they were used to.

The result was the adoption of a policy which will surprise veteran Git users. Per Magnus's blog:

Yes, that's correct. No merge commits. To submit a patch, extract it as a context diff and e-mail it. Committers are to apply the patch under their own names, without branch history. The project has decided, more-or-less, to use Git like it was CVS as far as commits to the main repository are concerned. Rather than adapt the PostgreSQL project's workflow to Git, Git would be adapted to the project's workflow.

It's possible, even likely, that eventually the PostgreSQL project will move towards the "normal Git workflow" where branches and merges would be used for feature work. But it definitely won't happen this year.

The First Release

The conversion wasn't really complete until the project did a release from it. That happened on Tuesday, October 5th with a combined security update. Unsurprisingly, there was a problem.

Security releases usually have fairly tight timing; from the moment someone commits the security patches, the issues fixed are publicly visible. Due to that, and limited familiarity with the Git commands, the source packager missed the final commit on the latest branch (9.0.1). That caused the release to be delayed by a day.

Git can't be blamed for everything, though. The release was delayed for another 2 hours because the Subversion repository which holds the www.postgresql.org web code locked up. But the release did go out.

Benefits of Git

Now that the migration is mostly done, many people are discovering the benefits of working on an up-to-date, distributed, version control tool.

Robert Haas has documented how to get commit summaries and sizes from Git. He wrote a perl script (which Tom Lane improved) that allows you to produce a changelog suitable for release notes from Git. Andrew Dunstan also found new ways to sync his local repository.

The pgAdmin project has also switched to Git and the pgWeb project will hopefully soon follow.

In the future, Git should both allow developers to work on longer-lived forks more easily and to test them more fully; we've already seen this with synchronous replication and SE-Postgres. Hopefully the translation teams will also be able to take advantage of forks on git.postgresql.org to collaborate on translating the docs and messages. Most importantly, Git should help prevent bit-rot in features which take months or years to develop.

Best of all, most PostgreSQL developers were able to continue hacking away without being involved in the switchover at all.

Lessons Learned

Assuming that there are any projects out there who have not yet switched to their distributed version control system of choice, here's a few things to learn from our migration:

• Start with a Git mirror.
• Designate a specific "Git migration team". Make sure they have lots of free time.
• Your first attempt to migrate will probably fail, so you need to be prepared for more than one.
• Changing your infrastructure, workflow, and build tool dependencies is harder than the repository conversion.
• Make friends with the conversion tool authors.
• Write lots of docs about the new tools and workflow.
• The more history you have on your current system, the more work conversion is going to be.
• Things which are broken in your current history are not going to fix themselves when you migrate.
• When testing the conversion, make sure to look at more than HEAD and branch-tips.

The biggest lesson, though, is not to be in a hurry! It was over three years from PostgreSQL's first Git mirror to final conversion, and 16 months of actual preparation. If you take your time and are ready to retry things that don't work the first time, you should be able to have a successful migration to Git.

Comments (64 posted)

UTOSC: Tayler on learning from failure

Web comic author and illustrator Howard Tayler looked at success and failure in business, particularly from the perspective of free content business models, in his keynote talk at the Utah Open Source Conference (UTOSC). Tayler has successfully turned his comic, Schlock Mercenary, into his day job for the past six years, even though the comic is freely available daily on the web. Arriving at a successful business plan is mostly a matter of learning from mistakes, he said, either your own or those of your peers. We live in a time where it is much easier to find out about mistakes, thanks to the internet.

Tayler previously worked at Novell for 11 years, doing everything from tech support to product management. He started Schlock Mercenary in 2000, while he was there, and built a following for the comic. In 2004, he left Novell to do the comic full-time. Though there have been ups and downs, he has succeeded in supporting his family since that time, primarily with book sales of the comic, along with advertising on the web site.

This is not the first time Tayler has spoken to UTOSC. In a presentation at UTOSC 2008, Tayler focused more on his particular business model, and at that time he saw it much differently than he does today. He likened his earlier model of creating a business plan to "making grizzly bear soup", which has a hard part, actually killing the grizzly, and "everything else is just soup recipes". For him, the hard part was to make the comic itself, and the other, moneymaking parts kind of just fell out of that. But, "business plans are not that cut and dried", he said.

We are all storytellers, Tayler said, of one kind or another. His stories are 1000 years in the future, but he also tells himself the same kinds of stories that those of us who aren't comic space opera authors tell. These are the stories that often govern our behavior ("if I work hard I will be able to retire at 65" or "if I always take off my shoes at the airport, I am safe from terrorist attack" and so on). As we keep telling ourselves these stories, some of them become business plans. Unfortunately, those stories are not really reflecting reality.

In The Black Swan, author Nassim Nicholas Taleb attributes success "much more to luck than we would be comfortable with", Tayler said. That book "blows my mind" and he strongly recommended it. One of the examples that Taleb gives is Casanova, who succeeded and failed seven different times in his life, going from prosperity to poverty. To many this is "proof" of the adage "never give up", but Taleb points out that there are likely many more folks who tried just as hard but never achieved that final success, so we don't know anything about them.

The failure stories would be valuable, so that we can learn from them. But people are "disinclined to talk about the failures", so we lose that information. But now, we have lots of ways to find about failures. People love to sweep their failures under the rug, but "there are now spiders under the rug recording those failures", and we can access that information through search engines.

An example that Tayler gave was the logo redesign that Tropicana orange juice did last year. The logo changed from an orange with a straw in it to a glass of orange juice, and within three weeks Tropicana knew they had made a mistake because they were losing millions of dollars. Brands that made "terrible orange juice", like Donald Duck brand, were suddenly outselling Tropicana. Marketing had decided that the brand needed a refresh, but it was a bad decision that looked good at the time it was made. Tropicana went back to the old logo, and in pre-internet days, that controversy would have faded. But you can still find lots of information about that failure on the web today.

On the other hand, people imitate each other's successes all the time. In his talk two years ago, he encouraged people to follow the same path he had as a business plan. But, he said, that kind of plan won't work for everyone as "'web cartoonist' pays better than 'web novelist'". For whatever reason, changing the web comic business model to a product that is strictly words doesn't work.

He has been lucky in that he "hasn't had enough failures to learn from", at least yet. When he left Novell, he set a failure condition: if they had to tap credit cards to pay bills for two consecutive months, it was time for him to go look for a job. Thousands of people don't recognize that their business has crossed a failure boundary, and they keep going hoping that the business is going to work out and pay them back. It is very important to set some failure conditions when starting a business so that failure can be recognized and flexibility can kick in.

Taleb and "dozens of authors" have suggested that business planning needs to be both robust and flexible. It must be robust enough to absorb a few mistakes, while being flexible enough to learn from those failures. Large organizations tend to come to this realization late, he said—if they realize it at all.

Evolution is the ultimate teacher of flexibility, robustness, and learning from failure. Evolution does not move toward a goal, it moves away from things that didn't work. It is, "improving things by making accidental random changes", which results in things like sharks—but also zedonks. Zedonks are a cross between donkeys and zebras that are ornery, sterile, and will "live to a ripe old age on a pony farm biting children". They are one of evolution's mistakes, he said.

While we may not have the time to make changes to business plans the way that evolution does it, we can use the information from various failed plans to guide our choices. You can't try out all of the different possible plans yourself, but new businesses are being started all the time. The ability to see their plans—what works and what doesn't—is an enormous advantage.

There is a flipside to the "law of large numbers", he said. If you roll a six-sided die many times, the average is 3.5, but rolling it long enough will produce long strings of consecutive sixes. By looking at the failures of others, "we can game the system", and make it more likely that our plans will come up sixes. "Lightning wants to hit something", he said, so "put up a lightning rod at the leading edge of a storm".

Tayler then took questions from the audience, most of which centered on the specifics of his particular business model. He brings in roughly 30% of his revenue from web advertising and 50-60% from the sale of several different books. There are some revenue sources he hasn't tapped yet, like speaking engagements, mostly because he likes talking for free. He cautioned against having any single revenue stream that makes up more than 40% of your income. At one point, he was in that position with Google AdSense and was concerned that Google could just "randomly turn that off", which could have been catastrophic.

Community is very important to his business. When he sells something, he wants that customer to "feel like they are part of a club". That requires good customer service, and making a connection with people at conferences and shows. There are certain cities where he sells much more than the population would indicate, like Austin and Detroit, because he has been there multiple times interacting with his fans.

In response to a question about telling good ideas from bad, Tayler pointed to a meeting that he attended in 1997 when he was working for Novell. In that meeting, then-CEO Eric Schmidt said that the internet needed a "directory", which was a great idea, he said, but one that Novell couldn't deliver. So Schmidt went to a startup (Google) and helped make it happen.

It is important to get a lot of feedback on an idea; "the more eyes on your idea, the better". Many in his field make the mistake of believing that their customers are the same as they are, so they design their products and such for themselves. Getting others to review product plans and other business ideas will help avoid that trap.

There may be a big change coming for publishing with the advent of e-books, he said. There is a belief that a "tipping point" exists once e-book sales reach 25% of printed book sales. Unlike some others in the publishing business, though, he is cautiously optimistic about his prospects if that happens. If it doesn't happen, he will be fine because he sells paper books. If it does happen, he has 200,000 readers that already read things online, so he is confident that he can find a way to sell content to them.

He has already done some e-publishing, starting with an app for the iPhone. He chose that market to start with because he heard that "iPhone users have big fat open wallets and low IQs", he joked. The truth is that iPhone users have a reputation for being more willing than Android users, for example, to pay for content. After a year with a subscription-model iPhone app, he made $1200, which he split with the developer. That was not enough income to continue that experiment, so he now has free ad-supported apps for iPhones, Android, and the iPad. The iPad is "the way to read my comic", he said, and believes that it will be a big player in the e-comics world.

Comments (6 posted)

Bradley Kuhn dives in full-time at the Software Freedom Conservancy

The Software Freedom Conservancy (SFC) has been in business for four years, so it seems about time for the organization to have a full-time employee. But is the organization ready to support a full-time employee? What started as a small offshoot of the Software Freedom Law Center (SFLC) is now home to more than 20 open source projects, with others waiting in the wings. It has a solid set of services for its organizations, but fundraising may be a challenge for now full-time Executive Director Bradley Kuhn.

Kuhn, known for his work as Executive Director of the Free Software Foundation and later with the SFLC, has been the driving force behind the Conservancy since its inception. As he writes in his introductory post, Kuhn has been doing the work to keep the Conservancy running in his "spare" time outside of his work with the SFLC. As a part-time volunteer, Kuhn has done a remarkable job of bootstrapping the Conservancy, but additional skills will be needed to make the Conservancy self-sufficient enough to carry one or more employees.

The SFC grew out of the SFLC, and was launched in April 2006. While the SFLC was equipped to provide pro-bono legal services for FOSS projects, there was an obvious need for more than just legal services.

What the SFC Does

The conservancy provides several services that projects need, but that developers are not ordinarily well-equipped or enthused to do themselves. The SFC provides a non-profit umbrella for projects, so that they can receive tax-deductible donations as a 501(c)(3), and provides fundraising assistance. The SFC also helps with contract negotiation, which comes in handy when a project wants to work with a contributor company, hold a conference, or any other activities that require contracts. If desired by the project, the SFC can also hold physical and legal assets, ranging from trademarks to servers donated to a project.

In other words, the SFC performs all manner of legal and financial work on behalf of projects so that contributors can focus on the project and not administrivia.

How do the client projects feel about the SFC? Matt Mackall of the Mercurial project, says that he's pleased: "Things have been good so far. They're fairly responsive and take a load of paperwork off our plate."

Mackall, primary author and project lead for Mercurial, says that the decision to join the SFC was spurred by a threat from BitMover to one of Mercurial's developers. Mackall recently decided to work on Mercurial full-time so, to enable that, donations come in for Mercurial to the SFC. He then receives payment as an independent contractor from the conservancy.

Aside from handling donations, Mackall says that the Conservancy has been helpful in many other ways ranging from making it possible to get T-shirts for contributors, to supporting Mercurial's licensing move from GPLv2 to GPLv2+, and assistance with developer sprints in Paris and Chicago.

The SFC isn't the only organization that does this, nor is it the first. For instance, Software in the Public Interest (SPI) has been serving in a similar role for Debian and other projects for many years. Kuhn sees it as a good thing that several non-profit homes for projects exist:

Next Steps for the Conservancy

Rapid growth is not in the cards for the Conservancy. In its four years, the SFC has signed up an impressive set of projects that run the gamut from Linux distributions to media players. The list includes more than 20 projects, from major and well-established projects like Samba to smaller projects like Libbraille, which provides a shared library for easily developing software for Braille displays.

The current projects aren't the only projects that wish to join the Conservancy. Kuhn says that there's a waiting list to get in and he hopes to have the resources to accept most of those in waiting. The SFC has standardized on letting in new members twice per year, to coincide with the biannual director's meeting.

Likewise, Kuhn isn't planning to expand the services offered by the SFC just yet.

One of the things that the Conservancy does offer is help with license compliance. The SFC has been busy in protecting the rights of BusyBox, which is often shipped in embedded devices without vendors complying with the requirements of the GPLv2. The SFC worked with the SFLC to file suit in December of 2009 against Best Buy, Samsung, Westinghouse, JVC, and several other companies for violating the GPLv2 in shipping BusyBox.

The SFC received a default judgment [PDF] against Westinghouse in July for $90,000 in damages and $50,000 to cover costs of the suit. However, Kuhn says that Westinghouse is trying to avoid paying the judgment, so it may be some time before the Conservancy can recognize the income. When it does, Kuhn says that the any money for damages are earmarked for the member project, while money for costs have gone to the SFLC to help it continue to serve its members.

Expect more of the same. Kuhn says that GPL enforcement is "definitely" an area where the SFC will be more active in 2011:

Funding

Finally, Kuhn says he wants to increase the Conservancy's available operating funds, but that will depend heavily on the generosity of the community and corporate sponsors. Right now, he says that the SFC has "about six months of operating expenses in cash for 2011, so my first goal is to bring that cash fund up to a year's worth of expenses and make sure I can deliver on a sustainable budget, including one employee."

Kuhn is the first full-time employee of the SFC, which raises the question of how funds will be raised to pay Kuhn's salary going forward. How does the Conservancy plan to raise money for that and other necessary services? First off, Kuhn is "bootstrapping," the move by being a full-time volunteer until 2011. He says that the SFC will be paying only for expenses until 2011.

Will the Conservancy be tapping member projects for funds? Kuhn says yes, but only for those that wish to contribute. "As part of Conservancy's fundraising strategy, we also ask our member projects to give a small percentage of funds raised in their earmarked account back to Conservancy generally. We hope to never have to make that mandatory, although we do have to tell very large projects that are in need of a lot of services from Conservancy staff that it will be difficult to assure ongoing services without this component."

Even if all projects choose to contribute, it still won't cover all the funding needs for a full-time salary — much less all of the services offered by the Conservancy. To that end, Kuhn says the Conservancy will seek "a diversified" donor base, and has been encouraging FOSS supporters to give to FOSS-related non-profits and not just the SFC:

Kuhn says that he doesn't want to depend too heavily on corporate donors, and doesn't have a specific plan to target donations from companies. He does talk to potential sponsors at conferences, and expects a contribution for 2011 from Google. So far the Conservancy appears to have done well by its members and on a shoestring budget, but a more aggressive and targeted fundraising plan would be more comforting. This is, perhaps, an area where the Conservancy should actively seek contributors to work on a fundraising plan and specific funding targets.

Funding concerns aside, the SFC has achieved quite a bit in its four years without the benefit of having a full-time employee. Providing the SFC can meet its fundraising goals to sustain Kuhn full-time, 2011 should be a very interesting year for the organization.

Comments (4 posted)

Fedora accepting YubiKey one-time passwords

Some Fedora hosts and services have recently added a new authentication mechanism which uses a USB device that generates one-time passwords (OTPs). The YubiKey doesn't require any client-side software because it acts like a USB keyboard. In addition, the server-side authentication software, as well as a client-side utility to update the YubiKey, are open source. Like most security measures, hardware-based OTPs have advantages and disadvantages. But, when an OTP is combined with another authentication method—a regular password for example—it can provide a two-factor authentication that is much more secure.

YubiKeys came into Fedora to provide two-factor authentication for access to sensitive servers, like those that contain the keys for signing Fedora packages. Now that the infrastructure team has gotten that working, it decided to allow other Fedora users to use their own YubiKeys for single-factor authentication to things like ssh access to fedorapeople.org or signing in to the Fedora Community site.

The YubiKey uses symmetric encryption, which means that both the YubiKey and the server must share an AES secret key. The YubiKey comes from the factory with a pre-installed key that is shared with Yubico's server. For Fedora's purposes, users will run fedora-burn-yubikey when they first get the device. That script will generate a key on the server that also gets burned into the YubiKey.

To use the device, users plug it into the USB port on the machine they are using, put the cursor in the password prompt field, and press a button on the YubiKey. That will generate an OTP and a carriage return into the field. The OTP contains a serial number that gets incremented each time the YubiKey is used. Invalidating an exposed (but unused) OTP is done by properly authenticating with the server, thus incrementing the serial number past the exposed password.

The server ensures that each password can only be used once by tracking the serial number for each AES key it knows about. Even if an attacker were able to capture the AES key somehow—the YubiKey device has no means to read it out directly—he must also use a serial number that is greater than the last one used by the owner. If that happened, the next legitimate attempt to authenticate using the YubiKey would fail (because its serial number is now too low), which would be a clear indication of key compromise.

Obviously, physical security of the YubiKey is paramount. An attacker that gets access to it for even a short time can easily—largely undetectably—authenticate as the owner. Since OTPs that have been used are no longer valid, sniffing password entry on the wire or "keystroke" logging won't be of any assistance to an attacker—by the time the password is known, it's no longer useful.

After Mike McGrath announced YubiKey support on the fedora-devel mailing list, there was some discussion of the device and possible attacks against it. One of the concerns raised was with the potential sharing of AES keys between servers. If a user wanted to use the same device with multiple YubiKey-enabled sites, they would need to share their AES key with all of the servers. If any one of those servers was compromised, it would allow an attacker to authenticate to any of the others.

For a number of reasons, this attack scenario was considered to be unlikely. It would take explicit action by the user—or a compromised client machine—when writing the AES key to the YubiKey to record the key before it gets written. Presumably users who are purchasing YubiKeys will be security-conscious enough to recognize that it is a bad idea to record the AES key. McGrath also made the window of exposure quite small:

In addition, newer YubiKeys allow for two different keys to be stored in them and will generate OTPs from one or the other based on how long the button is pressed. Toshio Kuratomi puzzled out the magic incantation required to write the second key and observed that holding the button for longer than 2.5 seconds would send an OTP based on the second key. Using that feature would allow sharing the device with two separate services, but if YubiKey OTPs become more popular, some other scheme for handling multiple sites will probably be required.

In some ways, this is all just a prelude to more widespread use of multi-factor authentication. In the next few years, OTPs are likely to become much more widely used for accessing sensitive sites and applications. The YubiKey idea is interesting, particularly because it doesn't require client-side support and doesn't lock users into some proprietary OS. A similar device that used public key, rather than symmetric, encryption would be worth trying as well.

Comments (10 posted)

Security quotes of the week

Comments (1 posted)

Schneier on Stuxnet

Comments (33 posted)

Flaw in libc implementation threatens FTP servers (The H)

Comments (8 posted)

Gilmore on the "computer health certificate" plan

Full Story (comments: 55)

IcedTea6 1.7.5, 1.8.2, 1.9.1 Released

Full Story (comments: none)

acroread: multiple vulnerabilities

Comments (none posted)

kernel: denial of service

Comments (none posted)

kernel: multiple vulnerabilities

Comments (none posted)

MRG Messaging: denial of service

Comments (none posted)

openswan: code execution

Comments (none posted)

subversion: restriction bypass

Comments (none posted)

wireshark: stack overflow

Comments (none posted)

xpdf: code execution

Comments (none posted)

Kernel release status

As of this writing, the final 2.6.36 release is not out, but it can be expected at almost any time.

Stable updates: there have been no stable updates in the last week.

Comments (none posted)

Quotes of the week

Comments (13 posted)

Linsched for 2.6.35 released

Full Story (comments: 6)

No fanotify for 2.6.36

Along came Tvrtko Ursulin, who pointed out a problem with the fanotify system calls; he then followed up with a second issue. It seems that the results of permission decisions were not always being handled correctly, and that the fanotify_init() system call had, somewhere along the way, lost the intended priority argument. The second issue, in particular, is serious because it affects the user-space ABI, which must be maintained indefinitely.

Eric acknowledged the problems and started to ponder ways to get around them before the 2.6.36 release, but Alan Cox advised a more cautious approach:

Eric, not entirely pleased with the idea, carried on the discussion for a while. Eventually, though, he sent in a patch disabling the fanotify system calls:

Linus took the patch, so, while the fanotify code will be present in the 2.6.36 release, it will not be accessible from user space. Whether the problems can be fixed in a way which is suitable for a 2.6.36.y stable release remains to be seen.

Comments (none posted)

ARM's multiply-mapped memory mess

First, some background. Contemporary processors do not normally address memory directly; instead, memory accesses are mediated through mappings created in the hardware's memory management unit. Depending on the processor, those mappings may be controlled through segment registers, page table entries, or some other means. The mapping will translate a virtual address into a physical address, but it also controls how the processor will access that memory and, perhaps, cache its contents.

As explained by ARM maintainer Russell King, ARM processors have a number of attributes which affect how memory mappings work. There is the concept of a memory type; "normal memory" is subject to reordering of reads and writes, while "device memory" is not, for example. There is also a bit indicating whether memory can be shared between processors; unshared memory is faster because there is no need to worry about cross-processor cache coherency. Then, like many CPUs, ARM processors can specify different caching behavior in the mapping; RAM might be mapped with writeback caching enabled, while device memory is uncached.

The ARM kernel maps RAM as normal memory with writeback caching; it's also marked non-shared on uniprocessor systems. The ioremap() system call, used to map I/O memory for CPU use, is different: that memory is mapped as device memory, uncached, and, maybe, shared. These different mappings give the expected behavior for both types of memory. Where things get tricky is when somebody calls ioremap() to create a new mapping for system RAM.

The problem with these multiple mappings is that they will have differing attributes. As of version 6 of the ARM architecture, the specified behavior in that situation is "unpredictable." Users, as a rule, are not enamored of "unpredictable' behavior, especially when their data is involved. So it would make sense to avoid multiple memory mappings with differing attributes. The ARM architecture has traditionally allowed this kind of mapping, though, and a number of drivers, as a result, rely on being able to remap RAM in this way.

Back in April, Russell raised an alarm about this issue, and posted a patch causing ioremap() to fail when the target is system RAM. This change avoids the potential data corruption issue, but at the cost of breaking every driver using ioremap() in this way. There were complaints at the time, so the patch sat out the 2.6.35 development cycle, but Russell merged it for 2.6.36. There it sat until, with the release imminent, Felipe Contreras posted a patch backing out the change, saying:

Russell was not impressed. In his view, remapping RAM in this way is a dangerous technique which will lead to data corruption sooner or later. Despite being warned six months ago, driver developers have not fixed the problem - there are as many broken drivers now as there were before. So, he says, there is no benefit to waiting any longer; the dangerous behavior should be stopped before somebody gets burned.

On the other side, driver developers point out that everything "seems to work" as it is, so there is no urgent need for change. Furthermore, Russell's patch looks to them like an API change, and the normal rule of kernel development is that anybody making internal API changes is charged with cleaning up any resulting messes. Fixing the drivers is not a trivial task, and it's Russell's contention that they have always been broken, so he is not willing (or necessarily able) to make them all work again.

The situation looked stalled, with a reversion of the patch looking like the only way forward. But, in fact, it looks like there is a way out. The first is to allow those mappings for one more cycle, but to put in a user-visible warning when they happen. As Andrew Morton put it:

It is the "worried users" who have been missing from the equation so far; they can provide a type of pressure which, seemingly, is unavailable to worried subsystem maintainers.

The other piece of the solution is to give driver developers a way to obtain a chunk of physically-contiguous RAM which can be remapped in this way. Such memory cannot be mapped simultaneously as system RAM. One nice idea would be to simply unmap system memory when it is put to a device's use, but that proves to be difficult to implement. The alternative is to just set aside some memory at boot time and never let the kernel use it for any other purpose; drivers can then allocate from that pool when they need to. Russell has posted a patch which makes this kind of memory set-aside possible.

So this particular situation will probably have a happy outcome, presuming that the above outcome happens and that that no users are burned by unpredictable mappings with the 2.6.36 kernel. But it highlights some ongoing problems. It can be very hard to get developers to fix things, especially if the current code "seems to work." Those developers also became aware of the change at a very late date - if, indeed, they are even aware of it now. It seems that testing of -rc kernels by developers is not happening as much as we would like. Still, the development process seems to work, and problems like this are overcome eventually.

Comments (15 posted)

Synaptics multitouch coming - maybe

The Linux driver for Synaptics touchpads does not currently support multitouch mode for the usual reason: Synaptics has not deigned to tell the world how to actually use its hardware. There is hope for change, though: Chase Douglas has recently posted a set of Synaptics driver patches which add touchpad support based on information obtained through reverse engineering. There are evidently still a few glitches to work out, but the mode appears to work. Alas, that does not necessarily mean we'll have Synaptics multitouch support in the near future; the real difficulties may be outside of the technical realm.

When Chase posted his patches, Takashi Iwai - better known for his ALSA sound driver work - responded that he had also worked on multitouch support:

In the ensuing discussion, it became clear that Chase's patch was, in fact, Takashi's patch with some improvements added. Takashi had apparently posted the patch set once before Novell legal called a halt to the exercise; that work had been stashed in a Launchpad page until Chase stumbled across it, made some improvements, and resubmitted the code. (Just to be clear: it does not appear that Chase was trying to take credit for somebody else's work; he just hadn't understood the original source of the code).

Evidently, Takashi sees the independent posting of the code as being sufficient to get around Novell legal's objections to its merging; he responded with enthusiasm despite being allegedly on vacation. Chase's kernel patches have been topped up with a series of X.org patches taking advantage of the new kernel support and adding user-level support for nice things like three-finger and "pinch" gestures. All of this code has seemingly been waiting for its chance to escape into the wild; all it took was for somebody else to start pushing the kernel-side code.

So it seems that the floodgates are open and multitouch support on Synaptics devices will be available to all. But there could yet be a catch. As Chase noted: "If you're the originator of the work, and my patch is accepted, I think we'll need your SOB on it." Without a signoff from Takashi, this code may not be accepted into the mainline. Takashi has suggested that his signoff from the initial posting could be used, but he appears to be unwilling to repost the code with a signoff now.

And that's where the trouble could come: your editor has had no contact with Novell's legal department and has no special knowledge, but it would not be surprising to learn that the concerns that department had about this code might not be swept aside quite so easily. It's possible that the new signoff from Takashi might not be forthcoming. Or, possibly, distributors will get cold feet for the same reason that Novell legal did and decide not to enable the feature. This code has been in the wild for some months now, but it has not found its way into users' systems; the increasing attention being paid to it now may not be enough to change that fact.

The inability to use Takashi's code - if, indeed, it comes to that - would not be a huge problem. The important thing is the information on how the hardware works; given that, some energetic hacker would undoubtedly reimplement the changes in short order. Patent concerns could be harder to work around, though. Without knowledge of which patents were at issue, it's hard to say how big an obstacle they could be. By some accounts, multitouch interfaces in general are patented, though that does not seem to have stopped some companies from incorporating such interfaces into their products. If it stops nervous Linux distributors, though, Linux users as a whole will be the losers.

That, of course, is the nature of the software patent system. But the scenario described above is highly speculative at this point. The important thing is that the code (along with the associated hardware information) is out there and available for those who would incorporate it into their systems. Hopefully it will be more widely distributed soon. Unfortunately, the wait for those nice nice wall-size displays may be just a little bit longer.

Comments (12 posted)

Statistics for the 2.6.36 development cycle

The 2.6.36 kernel will incorporate about 9400 changesets contributed by 1159 developers. It thus continues a recent trend toward less active development cycles; here is what we have seen over the course of the last year or so:

Release	Changes
2.6.31	10,883
2.6.32	10,998
2.6.33	10,871
2.6.34	9,443
2.6.35	9,801
2.6.36	~9,400

The work which pushed up the changeset numbers in previous development cycles (shoveling out-of-tree code into the staging directory being at the top of the list) continues to wind down, as does work in other areas (like new filesystems). As a result, the kernel is going through a period of relatively low flux - but only relative to the last couple of years - and stabilization. That said, it's worth noting that, unless something unexpected happens, the 2.6.36 development cycle will be one of the shortest in recent memory; as a result, the number of changesets merged per day is the highest since 2.6.30.

Perhaps more interesting is this set of numbers: in 2.6.36, the development community added 604,000 lines of code and deleted 651,000 - for a total loss of almost 47,000 lines of code. This is the first time since the beginning of the git era that the size of the kernel source has gone down. Given that, perhaps it is appropriate to start with a look at who has been so busily removing code from the kernel:

Most lines removed - 2.6.36
Sam Ravnborg	205813	31.6%
Benjamin Herrenschmidt	133666	20.5%
Amerigo Wang	19145	2.9%
Tony Luck	8418	1.3%
Greg Kroah-Hartman	7094	1.1%
Kiran Divekar	4487	0.7%
Palash Bandyopadhyay	4457	0.7%
Vincent Sanders	3467	0.5%
Dave Jones	2600	0.4%
Christoph Hellwig	2163	0.3%

Sam Ravnborg and Ben Herrenschmidt both got to the top of the list through the removal of a bunch of defconfig files, part of a general cleanup inspired by some grumpiness from Linus back in June; Sam also finished up some SPARC unification work. Amerigo Wang removed a number of old and unused drivers. Between the three of them, they got rid of almost 360,000 lines of code - a laudable bit of work.

Looking at code changes in general for the 2.6.36 development cycle yields this picture:

Most active 2.6.36 developers

By changesets Vasiliy Kulikov 160 1.7% Eric Paris 124 1.3% Dan Carpenter 122 1.3% Chris Wilson 117 1.3% Eric Dumazet 108 1.2% Uwe Kleine-König 103 1.1% Axel Lin 98 1.0% Johannes Berg 97 1.0% Al Viro 96 1.0% Julia Lawall 89 1.0% Tejun Heo 88 0.9% Joe Perches 83 0.9% Christoph Hellwig 73 0.8% Alex Deucher 71 0.8% Ben Skeggs 69 0.7% John W. Linville 68 0.7% Stefan Richter 64 0.7% Stephen M. Cameron 62 0.7% Felix Fietkau 60 0.6% Randy Dunlap 59 0.6%

By changed lines Sam Ravnborg 208270 19.4% Benjamin Herrenschmidt 134811 12.5% Chris Metcalf 53204 4.9% Omar Ramirez Luna 51087 4.8% Amerigo Wang 19191 1.8% Jarod Wilson 16020 1.5% Felix Fietkau 11898 1.1% Alan Olsen 11650 1.1% Mike Thomas 11087 1.0% Lars-Peter Clausen 10795 1.0% Tony Luck 9351 0.9% Tetsuo Handa 7955 0.7% Casey Leedom 7888 0.7% John Johansen 7591 0.7% Greg Kroah-Hartman 7195 0.7% Charles Clément 6864 0.6% Dmitry Kravkov 6754 0.6% Kiran Divekar 6753 0.6% Ben Collins 6540 0.6% Christoph Hellwig 6045 0.6%

On the changesets side, Vasiliy Kulikov leads with a long list of mostly small fixes, mostly in device driver code. The bulk of Eric Paris's work is the addition of the fanotify subsystem - work which, as of this writing, will not be enabled for the 2.6.36 release due to user-space ABI concerns. Dan Carpenter is another master of small fixes, usually for problems identified by static analysis tools. Chris Wilson had a large number of changes to the Intel i915 driver - and seemingly an even larger number fixing the resulting problems. Eric Dumazet's changes were a large number of fixes and improvements to the networking subsystem.

Three of the top five in the "lines changed" column have already been mentioned above. The other two are Chris Metcalf, who added the new "Tile" architecture, and Omar Ramirez Luna, who added the TI dspbridge driver to the staging tree.

Only one top-five developer (Dan Carpenter) was also in the top five for 2.6.35; there are a lot of new faces on the list this time around.

There were 184 employers (that we could identify) who contributed code to the 2.6.36 kernel. The top corporate supporters were:

Most active 2.6.36 employers

By changesets (None) 1524 16.3% Red Hat 1129 12.1% (Unknown) 865 9.2% Intel 691 7.4% Novell 404 4.3% IBM 374 4.0% Nokia 212 2.3% Texas Instruments 189 2.0% (Academia) 178 1.9% Samsung 178 1.9% Fujitsu 160 1.7% NTT 151 1.6% Pengutronix 145 1.6% AMD 131 1.4% Google 125 1.3% (Consultant) 109 1.2% Societe Francaise de Radiotelephone 108 1.2% QLogic 107 1.1% Atheros Communications 99 1.1% MiTAC 98 1.0%

By lines changed (None) 299115 27.8% IBM 151386 14.1% Red Hat 76455 7.1% (Unknown) 71662 6.7% Tilera 64825 6.0% Texas Instruments 63521 5.9% Intel 55167 5.1% Novell 25798 2.4% Samsung 14619 1.4% NTT 12187 1.1% Marvell 10769 1.0% Chelsio 10560 1.0% (Academia) 10345 1.0% QLogic 9873 0.9% Google 9503 0.9% Broadcom 8391 0.8% ST Ericsson 8390 0.8% Canonical 8354 0.8% Nokia 8060 0.7% Atheros Communications 7762 0.7%

For the most part, this list looks the way it has for most development cycles, but there are a couple of new names here. One is Tilera, the company behind the Tile architecture, which got its support merged for 2.6.36. The other name appearing here for the first time is Canonical, which got the AppArmor security module code merged at last. Meanwhile, one should not forget the other 164 companies which do not appear on the above list; the commercial ecosystem around the Linux kernel remains strong and diverse.

Finally, your editor decided to rerun an old experiment to look at the longevity of code in the kernel. Every line in the kernel source was mapped back to the kernel release where it was last changed, then the totals for each release were plotted. The resulting picture looks like this:

At 1.6% of the total, 2.6.36 represents a relatively small piece of the total code base - the smallest for a long time. Almost 29% of the kernel code still dates back to the beginning of the git era, down from 31% last February. While much of our kernel code is quite new - 31% of the code comes from 2.6.30 or newer - much of it has also hung around for a long time.

All told, 2.6.36 was a relaxed development cycle with relatively few big new features and a fair amount of cleanup. That is certainly part of how it was able to be stabilized in a shorter-than-usual period, and with fewer than the usual number of regressions (56 reported as of October 10, as opposed to 100 for 2.6.35-rc6). Whether 2.6.36 represents a new norm for a slightly slower kernel development process remains to be seen. As of this writing, the linux-next tree contains 5850 changesets, most of which are presumably intended for 2.6.37. Quite a few changes still typically do not appear in linux-next prior to the opening of the merge window, so we should see more changes than that merged for 2.6.37. Still, current linux-next does not look like a huge wave of pent-up changes waiting to fly into the mainline; 2.6.37 may or may not exceed 2.6.36 in the number of changes, but it does not look like it will be breaking any records.

Comments (9 posted)

The Ubuntu font and a fresh look at open font licensing

When Ubuntu 10.10 was released on October 10, it marked the "official" debut of the Ubuntu Font, commissioned by Canonical from type foundry Dalton Maag. Free software licensing aficionados will no doubt pick up on the fact that the font was released under the "Ubuntu Font License" (UFL) rather than one of the established options. Canonical takes pains to point out that the UFL is an interim license only, though, and the chain of events that led up to its release illustrate the dynamic landscape that is taking shape in the open font community.

The two most widely used licenses for free and open fonts are the Open Font License (OFL), created and stewarded by SIL International, and the General Public License (GPL) with the Font Exception clause. Canonical felt that neither one precisely met the requirements it set out for the ongoing development of the Ubuntu font family, and started private discussions with several members of the open font development community. Ultimately, the need to release the font under clear terms for inclusion Ubuntu 10.10 dictated that the company adopt an interim license, and continue working on a more permanent solution.

The Ubuntu Font Family as a project

Part of the difficulty in finding a "perfect match" license stemmed from the fact that Ubuntu is keen on continuing to develop the font family as a community project, with large contributions from designers outside Canonical and Dalton Maag, but branded under the Ubuntu label. While the initial font release includes Regular, Bold, Italic, and Bold Italic faces, and covers the Latin, Greek, and Cyrillic alphabets, the long-term plan is to add additional scripts and writing systems in regular point-releases.

Paul Sladen, who is a contractor working on the font family for Canonical, explained the basic plan and structure of the project. Dalton Maag is only contracted to deliver five scripts — Latin, Cyrillic, Greek, Arabic, and Hebrew — although Sladen hopes that the foundry will remain involved. That wide Unicode coverage will establish a base style for the font family, and more and more community material will be merged in as time goes by. The first such contribution has actually been merged: the Indian Rupee symbol; its inclusion makes Ubuntu 10.10 the first operating system to ship with out-of-the-box support for the currency symbol.

The font family is a project hosted on Launchpad, where contributions are beginning to trickle in for additional character sets. For new scripts (and presumably other large-scale enhancement efforts), Sladen said that the current plan, though not set in stone, "is that each LoCo group find themselves two type designers, document the additions necessary for their script, and in order to maintain stylistic harmony across the font family one of those type designers would be mentored by Dalton Maag. Each script would be focused on in turn, on a two-month rotation (eg. potentially six new written scripts or languages per year)." Point releases (1.10, 1.20, etc.), he added, would add beta support for the new scripts, while assuring that the font metrics for those glyphs that were already in the stable (x.00) release remain unaltered, to protect existing documents against sudden repagination and alignment problems.

This structured development plan is distinctly different from the process seen in previous open font commissions, such as the Liberation fonts developed by Ascender for Red Hat. Development has continued on the Liberation family, but without plans specifically targeting greater Unicode coverage.

Licensing decisions

There are derivatives of Liberation that do set out to build wide support for other writing systems though, notably the DejaVu fonts project. This highlights one of the principle issues that separate different open font licensing options: simply examining the names, a user would have no idea that the DejaVu family is related to Liberation.

Canonical considered both the OFL and GPL with Font Exception when beginning its search for a license under which to release the Ubuntu font family. For those unfamiliar, although both licenses are designed to permit modification and redistribution of fonts, they do feature several important differences.

First, the OFL does not require availability of source code. In practice, though, it is easy to modify fonts based only on their binary representation (such as .TTF or .OTF files). For derivative fonts, OFL requires the derivative maker to obtain permission from the original copyright holder in order to re-use the "Reserved Font Name" of the original. Finally, OFL includes a clause that does not allow the font to be re-sold by itself. The latter two elements illustrate that the OFL is designed to meet the expectations of type designers. In particular, the "Reserved Font Name" preservation is an important issue because copyright and patent law varies greatly between different countries in regard to fonts; as a result many type designers rely on protecting the name of the original as the primary line of defense against "brand" dilution.

The GPL with Font Exception, as one might expect, does require source code availability and does not restrict commercial sale of the font. It does not, however, explicitly address renaming derivative works. The Font Exception clause itself simply specifies that documents created with the font are not considered derivative works of the font. The OFL has a similar clause.

While planning for the release of the font, Canonical undertook discussions that reportedly involved OFL co-author Nicolas Spalinger and the Open Font Library's Dave Crossland, seeking to find a licensing approach that would meet Ubuntu's precise needs — a set of concerns that seemed to fit neither OFL nor the GPL with Font Exception. Lacking a perfect fit, Canonical then drafted the interim UFL for the font, following the same broad structure as the OFL.

All of the parties were quick to emphasize that there was nothing adversarial about the discussions, nor the eventual decision to create the UFL. Sladen posted a diff between the interim UFL and the latest version of the OFL (1.1) that shed some light on what Canonical's needs were, however. There are slight differences in wording (such as the use of "propagate" instead of "distribute") that seem to place the license more in line with the GPL and LGPL used for Ubuntu software, but the biggest significant change is in the UFL's treatment of derivative font naming.

First, while the OFL establishes the concept of a "Reserved Font Name" that original copyright holder retains control over, the UFL specifies three distinct degrees of font derivative, and applies specific naming instructions to each. Unmodified versions of the original font must retain the original name, modified versions that are not "Substantially Changed" must be renamed to append the derivative name to the original name, and modified versions that are "Substantially Changed" must be renamed entirely.

The term "Substantially Changed" is defined in the license as "Modified Versions which can be easily identified as dissimilar to the Font Software by users of the Font Software comparing the Original Version with the Modified Version." The distinct degrees of modification and the explicit requirements for each take the place the OFL's requirement that recipients of the font seek explicit permission from the copyright holder on naming questions. The UFL further specifies that the license does not grant any rights to trademarks associated with the original font.

Consequently, the UFL allows "substantial" and "unsubstantial" forks of the Ubuntu font family, but attempts to encode in the license itself what the fork creators must do regarding the font name, and it simultaneously seeks to protect Canonical's trademark on the Ubuntu name itself.

Future licenses, and future code

Although Canonical is clearly satisfied that the terms of the UFL perform their intended function, the company maintains that the UFL is only a temporary solution, and that within several months, a permanent license will be selected. The trouble is that Canonical is not interested in being a license steward, feeling both that license stewardship is outside of its core mission, and that licenses should be maintained by wider community efforts.

While no one is in favor of license proliferation, it is clear that different people and groups involved in open font development represent a spectrum of different beliefs. Crossland believes that this spectrum would best be served by a clear set of license choices, along the lines provided by Creative Commons for artistic works. The Open Font Library's goal of creating strong copyleft fonts, for example, might best be supported by the GPL with Font Exception. SIL's interest in attracting professional type designers to work on modifiable, redistributable fonts is best supported by the OFL.

Thus it seems like a new open font license is inevitable, whether it is eventually maintained by SIL, the FSF, or some other party altogether. Who that might be is entirely up in the air. SIL might seem like the obvious choice, given its long history with the open font movement, but one should not expect a "L"OFL to appear suddenly. Spalinger discussed the UFL and Ubuntu font family project on his blog, and although he emphasizes that he is not speaking on behalf of SIL, he makes it a point that stewardship of another license is not a task that SIL is likely to undertake lightly, nor without support and input from the wider open font community.

Spalinger also has considerable praise for the practical benefits that the Ubuntu font project have brought about, including the fonttest web application, the ongoing efforts of Dalton Maag and Canonical's Design Team to publicly document the font design and testing process on its blog, and Canonical's plan to continue to the develop the font family in the open.

Regarding Dalton Maag, Sladen was also happy that the foundry (and Bruno Maag in particular) was reacting warmly to the interaction with the Ubuntu community. Many foundries, he said, are not used to the long, iterative feedback-and-tweak cycle that this particular commission carried with it — typically, a foundry finishes the design, delivers it to the client, deposits its check, and is finished. "It was a slow start, it is a complete mindset change for them," he said, "I think Dalton Maag have realised the value of the publicity."

At the same time, Dalton Maag's involvement with the project is also having an impact on the open source font creation tool set. Initially, Sladen said, Canonical tried to find a type foundry that worked with open source tools — and there were none.

Instead, Dalton Maag is providing constructive feedback on FontForge and other open source font applications, from the perspective of a full-time type foundry. "Ubuntu focuses quite a lot on usability and user-interaction; before you can fix things, you need to work out what is broken, and precisely the same applies to the libre-font creation stack." He cited several pieces missing from the open source type design process, such as digital signatures and visual hinting and layout tools that the community needs to work together to replace.

Crossland said that type design is in the midst of a revolution at the moment, after years of stability, brought about by Web Font support. As a result, the open font community has grown significantly in recent years, but a side effect of that growth is the differentiation of different participants' goals and needs. One license will probably never fit all. On the bright side, the artistic community has thrived with Creative Commons' palette of licenses, and the free software movement has never been held back by supporting multiple license options. Fonts and their unique set of issues are simply coming of age.

Comments (36 posted)

Distribution quotes of the week

Comments (none posted)

Ubuntu 10.10 released

Full Story (comments: 25)

openSUSE Announces Second 11.4 Development Milestone

Comments (1 posted)

Neuroimaging research in Debian

Comments (none posted)

bits from the DPL: let's Squeeze, GR, delegations, events, ...

Full Story (comments: none)

delegation for the Debian auditors

Full Story (comments: none)

Bits from the NM Process

Full Story (comments: none)

Minutes from the Ubuntu Technical Board meeting

Full Story (comments: none)

Mageia Blog: Mageia news: another busy week!

Comments (none posted)

Distribution newsletters

• DistroWatch Weekly, Issue 375 (October 11)
• Fedora Weekly News Issue 246 (October 6)
• openSUSE Weekly News, Issue 144 (October 9)
• Ubuntu Weekly Newsletter, Issue 213 (October 10)

Comments (none posted)

Jaeger: systemd - and #osc2010

Comments (51 posted)

Linux Mint 9 review (PC Advisor)

Comments (none posted)

Tiny Core Linux offers a complete Linux solution in 11MB (Geek.com)

Comments (1 posted)

UTOSC: Inexpensive audio and video recording

Clint Savage set out to inexpensively record audio from Linux user group (LUG) meetings that could be easily shared. Soon after he started making podcasts available, though, he started getting requests for video. That led him to investigate cheap video hardware as well as free software solutions for video. At the Utah Open Source Conference (UTOSC), Savage described his journey and the solutions he has found.

Near the beginning of his talk, Savage stressed that he is not an audio/video engineer, just an interested hobbyist, and that he was presenting tools and equipment that worked for him—not necessarily the "best" way to handle audio and video on Linux. He started out very simply, using a $7 microphone from Radio Shack to record LUG meetings. He used Audacity to add in some introductory audio and then made the audio available as podcasts.

Once he had that working, he investigated live streaming with a Shoutcast-compatible server. He tried Icecast, but it "took a long time to set up". In the meantime, his listeners wanted video, and he wanted to stay with free and open solutions while not breaking the bank, so he started looking into cheap video hardware.

For cameras, he settled on a Canon ZR80 MiniDV digital camcorder, but said you can also use almost any webcam, including those built-in to laptops. He wanted to be able to do screen capture for presentation slides and screencasts, which led him to the vga2usb device from Epiphan. While that works well, it costs around $300, so he found a much cheaper solution using a scan converter and EZCap, which does a reasonable job for around $60. Lowering the cost allowed him to outfit each of the half-dozen UTOSC presentation rooms with systems capable of recording audio synced with each talk's slides.

For video capture software he looked, briefly, at the proprietary offerings for Windows and Mac OS X, but was quickly put off by the price. Not only is there the "Apple tax" on hardware, which makes it more expensive than the equivalent Windows or Linux box, but then you need to spend something like $3000 for video software. Windows has similar offerings, but it is just as expensive.

For Linux, there are multiple choices for video editing (Kdenlive, Kino, PiTiVi) and for screen capture (Istanbul, gtk-recordMyDesktop), but none of those directly provide streaming solutions. Flumotion is a free streaming server, but he spent three days and "never got it working". Most of those tools use GStreamer under the hood, though, so he started looking into that.

GStreamer is a library and set of command-line tools for handling multimedia content. Savage showed several examples of using GStreamer from the command line, starting with a very simple:

    $ gst-launch videotestsrc ! ximagesink

    $ gst-inspect | more

For doing more interesting things with GStreamer, Savage pointed to gst-python, which are Python bindings for the GStreamer library. Using gst-python allows adding and removing filters or adjusting audio/video inputs and outputs in real-time under the control of a program. It has been used to build various applications including audio players like Rhythmbox and Amarok.

Savage is quite excited about another, fairly new tool that uses gst-python. Freeseer is an "open source video studio in a backpack", according to its web page, that is built on GStreamer, Python, and KDE. It originally used the FFmpeg command line, but switched over to gst-python after a fairly short time. It is designed for recording conferences, but is more versatile than that.

Freeseer has a simple interface and, as Savage demonstrated, just hitting the "record" button starts recording the screen. It can record from various sources, including audio sources, and is set up to tag the multimedia files with things like which room is being recorded, title and presenter from an RSS feed, and so on. There are some features that Freeseer is missing—streaming, multiple video sources for picture-in-picture, and automatic upload for example—but development is moving quickly so those things, and undoubtedly others, will be addressed soon.

Savage was the executive director for this year's UTOSC, so he set up systems using Freeseer to do the audio and slide recording for each presentation room. He borrowed refurbished, low-end systems from the Electroregeneration society, and added in around $100 worth of equipment (screen grabber and microphone) for each room. He recommended a $50 Radio Shack wireless lapel microphone as the best he had found for presentations, and recording systems only require a Pentium 4 or better. So, for "a few hundred dollars", anyone can put together a system to record presentations, and it wouldn't take that much more to add in live video.

That was clearly a major part of Savage's intent with his presentation: to get other conferences, LUGs, and user groups of all kinds to do more recording of the presentations that they have. From a financial standpoint, it is well within the budget of almost any group, and the free tools are available and getting better. The financial barriers to entry for any group—technical or not—are pretty low, which is quite a change from where things stood five or even three years ago.

Comments (4 posted)

Quotes of the week

Comments (1 posted)

Evolutionary Algorithms in Python (0.6)

Full Story (comments: none)

Firefox 4 beta for Android and Maemo

Comments (12 posted)

Call for bids: GTK+/MeeGo Handset integration work

Full Story (comments: none)

The Linspotting project - Linux video editing workflow ready for action

Comments (4 posted)

Pogo 0.1 released

Full Story (comments: 2)

Development newsletters from the last week

• Caml Weekly News (October 12)
• Linux Audio Monthly Roundup (October)
• LibreOffice development summary (October 11)
• PostgreSQL Weekly News (October 10)

Comments (none posted)

Seigo: activities as homonyms

Comments (16 posted)

IBM joins Oracle for OpenJDK work

Comments (18 posted)

GTK+/MeeGo Handset integration work, call for bids

Full Story (comments: none)

HTC Willfully Violates the GPL in T-Mobile's New G2 Android Phone (Freedom to Tinker)

Comments (28 posted)

Hall: LPI and My First International Proctoring "Job"

Comments (none posted)

Motorola Mobility sues Apple

Comments (15 posted)

The Perl Language Reference Manual v5.12.1 - printed edition

Full Story (comments: none)

Enterprise JavaBeans 3.1, 6th Edition--New from O'Reilly Media

Full Story (comments: none)

Linux Foundation survey on enterprise Linux use

Comments (1 posted)

Red Hat Summit and JBoss World Call For Papers Now Open

Comments (none posted)

Linux Plumbers Conference keynote and event lineup

Comments (none posted)

Events: October 21, 2010 to December 20, 2010

If your event does not appear here, please tell us about it.

Announcing: The virt-tools.org website

Comments (none posted)