Leading items

A new open source foundation

A foundation to promote increased participation by commercial software companies in free software projects would normally be seen as a good thing. That the foundation begins with a budget of $1 million would also seem to bode well for its success, at least in the near term. But when that foundation is sponsored—and funded—by Microsoft, it is met with a healthy dose of skepticism, which is exactly what greeted the recent announcement of the formation of the CodePlex Foundation.

While the new foundation is not strictly related to CodePlex.com—Microsoft's SourceForge-like repository for open source code—they share the name from the "CodePlex brand", as well as supporting the "same mission", according the foundation's FAQ. Like CodePlex.com, the CodePlex Foundation is geared towards commercial software developers who are, evidently, not being served by existing organizations. Though it is far from clear what is missing from the existing sites and organizations, the FAQ makes it clear that CodePlex intends to fill that hole:

One would think that existing foundations—for GNOME, Linux, Mozilla, Apache, et al.—might also be interested in bringing commercial and free software developers together. In fact, those foundations have been fairly successful in doing so. There may be value in having a more all-encompassing foundation, which doesn't serve a particular sub-genre of free software, but it is a bit hard to see the CodePlex Foundation filling that role.

To start with, the foundation's board will be Microsoft-dominated, with three current Microsoft employees on the six-member board. In addition, Sam Ramji, up until recently the leader of Microsoft's open source efforts, will serve as interim president of the foundation and chairman of the board. Ramji recently announced that he is leaving Microsoft for a cloud computing startup in Silicon Valley, but, as a former employee, his detachment from the Microsoft view is unclear. The two outside members are Mono/Moonlight developer Miguel de Icaza of Novell and DotNetNuke developer Shaun Bruce Walker, both of whom have fairly close ties to Microsoft.

At some level, the make up of the board of directors is unsurprising, but it is rather puzzling that the board of advisors would be similarly stacked with Microsoft employees. Six of the twelve members of the advisory board (listed to the right of the board on its web page linked above) are employed by Microsoft. There are some seemingly independent voices on the advisory board, including Larry Augustin of SugarCRM and MySQL developer Monty Widenius of Monty Program AB. Since the advisory board is just meant to advise the board of directors, who will make any decisions, also packing the advisory board certainly gives the appearance that the foundation will just be a Microsoft mouthpiece.

In a lengthy blog posting, Andy Updegrove analyzed the by-laws [PDF] and make up of the foundation. Updegrove is a lawyer known for his reporting on the ODF and OOXML standards process—and its apparent subversion by Microsoft—and is the director of standards strategy for the Linux Foundation, so he is hardly a disinterested observer. He notes, though, that he has helped set up many "consensus based consortia and foundations" over the years, which gives him a good basis to evaluate the CodePlex Foundation.

There are several areas that Updegrove notes as problematic in the foundation as it stands. In order for an organization to be seen as neutral, it's best to have multiple partners or members, but the foundation is not set up as a membership organization. The by-laws mandate a new five-member board be elected in 100 days, but, since there are no members who can vote on board seats, the interim board will be doing the electing. Those five new board members will have complete control of the foundation, and will have been elected for as many as four years.

In addition, because a 2/3 majority is needed to change many parts of the by-laws (size of the board, text of the contributor agreements, etc.), any two members of the board can effectively block those changes. So, even if the permanent board only has one Microsoft employee—which seems unlikely—and one "friendly" member, the company effectively has a veto over any fundamental changes the board might want to make.

The draft Contribution Agreement [PDF] and Copyright Assignment Agreement [PDF] may give a look into what the foundation is trying to achieve. They both grant wide-ranging rights—including licenses to any patents held by the contributor that cover the contributed code—to the foundation and those who get code from it. The introductory slides specifically mention passing those rights to "downstream developers", but are mute when it comes to upstream. That may be an oversight, but it might also be an attempt to isolate the GPL from the CodePlex ecosystem.

Exactly why anyone would want to contribute code to the foundation is unspecified. There are sometimes good reasons to do copyright assignment, but the foundation hasn't articulated what benefit developers would get by doing that. As Updegrove puts it:

Like many newcomers to free software—along with some established players—Microsoft clearly does not feel comfortable working with existing organizations and communities. It wants to create its own playing field, where it can dominate, and control if necessary. By using contributor agreements and by-laws that are favorable to its interests, potentially at the expense of those who participate, it is creating something that it feels comfortable with.

In some ways, it is similar to the control that Sun has, so far, exercised in the OpenSolaris and Java communities. For both Sun and Microsoft, there is a corporate aversion to allowing the community to drive the direction of projects. But, that is one of the hallmarks of successful free software development. Various companies have learned this, over time, so it is possible that Microsoft will as well. Five, or even three, years ago, it would have been unimaginable that it would create a foundation for open source; clearly some progress has been made.

Microsoft's penchant for co-opting others' ideas, then twisting them to their own ends (e.g. "Embrace, Extend, Extinguish") leads many to scrutinize the CodePlex Foundation more carefully than they might one created by a different company. There is a level of distrust within the free software community that can only be erased through fundamental changes to Microsoft's behavior over a prolonged period. It's certainly possible that the foundation is one step on that path, but the conflicting signals it continues to send about free software gives one pause. Updegrove voices just that concern:

While Updegrove is rather critical of the current foundation structure, he also has specific suggestions on changes that could be made. Things like increasing the size of the board, and decreasing the number of Microsoft employees on it, as well as taking back 3/4 of the initial funding and replacing it with contributions from other corporate members, are among his suggestions. The idea behind those suggestions is for the foundation to truly be a neutral party, rather than just an arm of Microsoft:

Overall, the foundation idea seems half-baked. Its announcement may have been rushed to offset some recent negative publicity—in the form of patents that Microsoft tried to sell to patent trolls—or to blunt the impact of Ramji leaving the fold, but it certainly doesn't have a very polished look. Over time, it may evolve into something useful to the free software community, but only time will tell.

That, really, is the crux of the matter. Speculating about Microsoft's motives and plans is certainly reasonable, but we will have to wait and see what the foundation actually does. The make up of the new board, along with any changes to the advisory board, will likely signal where the foundation is headed. How many commercial software developers "join" the foundation and start contributing code, will also be interesting to watch. Based on what we see today, though, it would not be terribly surprising to see the foundation die quietly on the vine a few years down the road.

Comments (13 posted)

Tornado and Grand Central Dispatch: a quick look

Two traditionally proprietary companies made open source releases recently: Facebook released a Python-based web server and application framework called Tornado, and Apple released a thread-pool management system called Grand Central Dispatch. It is not the first open source code release for either company, but both projects are worth examining. Tornado is designed to suit specific types of web applications and is reportedly very fast, while Grand Central Dispatch may cause some developers to re-think task-parallelism.

This Tornado serves you

Tornado is actually a product of FriendFeed, the social-networking-aggregator acquired by Facebook in August. It consists of a web server and surrounding framework (all written in Python), tailored to handle a very large number of established, open connections. The web server component (tornado.web) is "non blocking" — meaning that it is event-driven, designed around the Linux kernel's epoll facility, and can thus maintain large numbers of open TCP sockets without tying up excessive memory and without large numbers of threads.

Event-driven Web servers like Tornado are single-threaded; each thread can manage potentially thousands of open connections as long as the application does not block while it waits for data from the socket — the thread simply polls them each in turn. Additional connections can be handled by running multiple server processes on SMP systems. In contrast, traditional web servers are blocked from handling additional connections while they wait for I/O, or must spawn additional threads to handle additional connections at the cost of context-switching and increased memory use.

In addition to the web server itself, the Tornado release includes a suite of modules used to build web applications, including XHTML, JSON, and URL decoding, a MySQL database wrapper, a localization and translation module, a Python templating engine, an HTTP client, and an authentication engine. The latter supports third-party schemes such as OAuth and OpenID, plus site-specific schemes used by Facebook, Yahoo, and Twitter.

The Tornado code is hosted on GitHub and is available under the Apache 2.0 license. Tornado works with Python 2.5 and 2.6, and requires PycURL and a working JSON library. Documentation is available on tornadoweb.org, and a live demo "chat" application is running on http://chan.friendfeed.com:8888/.

FriendFeed's Bret Taylor announced the release on his blog, comparing Tornado to web.py and Google webapp. He claims that in Apache Benchmark tests, Tornado was able to handle four times the number of requests per second (or more) of competing frameworks, including web.py, Django, and CherryPy.

Taylor's post, and the subsequent discussion, sparked some controversy among users and developers of the Twisted framework, who objected to disparaging comments about Twisted's code maturity and suitability. Twisted founder Glyph Lefkowitz posted a lengthy response responding to the claims made about Twisted, but, overall, approving of the Tornado release itself. Matt Heitzenroder posted his own head-to-head performance tests that show Tornado beating Twisted.web, but not dramatically.

Aside from performance numbers, many in the open source community seemed impressed by what Tornado offers — a simple framework for building "long polling" web applications, including support for everything from templating to cookie management to localization in a single package. Since Tornado has proven itself viable as the framework underlying FriendFeed, it is likely to pick up a significant following as an open source project.

Invisible threads

Apple's Grand Central Dispatch (GCD) is an operating system-level feature that debuted in the recent release of OS X 10.6 ("Snow Leopard"). GCD is essentially a mechanism to allow application developers to parallelize their code, but let the OS worry about intelligently managing the threads. GCD determines the maximum number of concurrent threads for the system and manages the queues for all running applications. Thus the application developer only needs to write GCD-capable code, and trusts the OS to take optimal advantage of multiple cores and multiple processors.

Apple's source code release consists of the Apache-licensed user space API library libdispatch and changes to the XNU kernel, Apple's open source Mach-based kernel common to OS X and Darwin. The XNU changes reportedly improve performance of the event notification interface Kqueue. GCD also relies on a non-standard extension to C, C++, and Objective-C known as "blocks," however, so blocks support in the compiler is a prerequisite for application developers wishing to take advantage of GCD. Blocks are supported for the LLVM compiler through the compiler-rt project.

Because GCD abstracts thread creation from the application developer, it is most similar to OpenMP or Intel's Threading Building Blocks (TBB). All three allow the developer to designate portions of code as "tasks" to be parallelized in some fashion. GCD is different in that it leverages a language feature (blocks) rather than the preprocessor directives of OpenMP or templates of TBB. In addition, TBB is limited to C++, though OpenMP is available for C, C++, and Fortran.

Blocks are essentially inline-defined, anonymous functions. They are designated by a caret (^) in place of a function name, take arguments like any function, and can optionally return a value. Blocks are different in that they have read-only access to variables from their parent scope (a feature similar to "closures" in languages such as Ruby). Consequently, in replacing a for loop with GCD's parallel equivalent, dispatch_apply, the developer can write a block containing the loop's contents without the hassle of passing extra arguments to it just to access variables that were available to the loop.

From Apple's Concurrency Programming Guide, the following example loop iterates count times:

    for (i = 0; i < count; i++) {
        printf("%u\n",i);
    }

    dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);

    dispatch_apply(count, queue, ^(size_t i) {
       printf("%u\n",i);
    });

When executed, GCD creates count tasks, one for each iteration of the block, placing them on a task queue. GCD makes a default queue available through dispatch_get_global_queue(), but developers can create private queues if they wish; to serialize access to a shared data structure, for example. In the traditional parallelizing-a-for-loop example, tasks are queued asynchronously, but GCD provides several mechanisms for monitoring completion of tasks, such as callbacks and semaphores.

Apple provides a basic introduction to GCD and programming with blocks on its developers' site. In addition, the OS X scientific research community at MacResearch.org has a detailed tutorial complete with GCD examples and the equivalent code written for OpenMP. MacResearch.org has basic performance numbers posted for its tutorial code, and Apple has posted a benchmarking sample that compares GCD against serialized code and native POSIX threads.

So far, GCD is only implemented for Mac OS X, but reaction from the developer community has been positive. Having the operating system worry about the details of thread pool management seems like a winning idea; most of the discussion on Mac forums has revolved around the wisdom of relying on a language extension such as blocks. Ars Technica commented on places where Linux could benefit from a native GCD implementation, such as in higher-level frameworks like QtConcurrent, but notes that use of the Apache license limits integration to projects using GPL version 3 and later.

Impact

Apple and Facebook have a history of making periodic releases of code projects under open source terms, even though both enjoy a reputation for maintaining "walled gardens" around their core products. As is predictable when large proprietary companies release open source code, considerable energy has been expended on the web speculating as to what each company hoped to "gain" from the release. A leading theory for GCD is that Apple hopes to further the adoption of blocks into standard C and C++, but no consensus has yet emerged for why Tornado was released.

In fact, neither Tornado nor GCD has made major waves in the open source community, but if the initial reaction is a good indicator, both are solid and valuable products. GCD is the likelier of the two to stir up passionate debate going forward, as fully assimilating it into mainstream Linux would require touching not one but two of the fundamental pillars of the community: the kernel and the compiler. Although LLVM has its fans, the Linux community is still predominantly a GCC ecosystem. Pushing Apple code into the Linux kernel and into GCC won't happen lightly.

Comments (42 posted)

FOSS license compliance in the consumer electronics market

[ This article is an opinion piece and does not contain legal advice. The authors are not lawyers. ]

[ Editor's note: This is part 1 in a series of three. Part 2 looks at compliance engineering and part 3 looks at license compliance for companies. ]

Free and Open Source Software (FOSS) license compliance is a contentious topic. There are different perspectives about when and how license terms apply, about which licenses can be used together, and about how potential issues should be resolved. The consumer electronics market is an area where FOSS license compliance is particularly problematic. This is primarily attributable to economic reasons rather than dishonesty, but in a market worth more than $335 Billion in 2008, it is an issue worth exploring.

Due to the relative youth of the FOSS ecosystem, there is a lack of case law and best practice information available. In the past, one of the few resources available to the community was Debian Legal, and businesses had little beyond Open Bar (USA) and ifrOSS (EU) to support them.

That situation is improving. Organizations like FSF's Free Software Licensing and Compliance Lab, gpl-violations.org, FSFE's Freedom Task Force and Software Freedom Law Center (SFLC) have helped push professional legal and business approaches to the forefront of FOSS discourse. The recent launch of the International Free and Open Source Software Law Review has provided a neutral platform for future discussions. As FOSS has matured so too has the level of information accessible to support businesses and projects.

The consumer electronics business

Consumer electronics are sold in high volumes for low margins, and competition in the market is fierce. The majority of sales take place during the first three months after launch and consumers focus on price and functionality when selecting new technology. Products are developed in Asia by original device manufacturers (ODMs) and original equipment manufacturers (OEMs) and shipped in completed form. There are few western companies doing their own development, and even those with in-house skills are unlikely to build a finished product themselves.

ODM/OEMs may develop products for competing western companies using a single board to save money. A board design and Software Development Kit (SDK) is provided by an upstream supplier like the chip vendor, the ODM/OEM will add hardware or software functions, and the finished system is placed into customized casings. Purchasing companies can label these variants as their own by adapting control panels, contact information, and documentation.

During this process issues can arise regarding license compliance. FOSS originating from a chip vendor may be supplied with incomplete source code to the ODM/OEM. If the source is supplied in complete form it may later be customized by the ODM/OEM and only partially re-integrated into source tree. The marketing team may forget to place licenses or written offers for source code in the product manuals. The list of potential points of failure is lengthy.

The fundamental issue is simple. If FOSS code and changes to that code are not integrated into source releases, or if other terms of popular licenses are not met, then compliance issues can occur. This problem is compounded when one board with a problem appears in devices supplied to a number of western companies. A host of violation reports spanning a dozen European and American businesses may eventually point towards a single mistake during development at an Asian supplier.

Why violations occur

There are many types of FOSS compliance issues. The specific issues depend on the license being used, but may include people forgetting to add a copy of the license text to products, forgetting to include the source code with shipped binaries, or having no policy to handle source code requests after providing a written offer promising this service. There is often a disconnect between support, website maintenance, and legal departments, so even correctly prepared material gets lost in the shuffle. At first glance it can appear daunting to perform due diligence.

However, FOSS compliance is not inherently more complex than proprietary compliance, and compliance in general is not so difficult as to be excusably ignored. There is even a field called compliance engineering where external specialists or in-house staff check that code shipped in products meets the required license terms. The problem for the consumer electronics market is that compliance engineering is perceived to endanger profit. There are two reasons for this.

The first reason is that market timing is extremely important, and a delay reaching consumers could mean being beaten by the competition. Compliance engineering with any reasonable fidelity will take a few days, and when companies will only have one or two test samples of the product available for checking functionality, it's hard to find a way to schedule time for compliance checking. Furthermore, any questions raised will have to be answered by the supplier and potentially other parties in the supply chain. Any missing source code will have to be located and integrated in the SDK. If there is missing code or a supplier in the chain who simply won't release required code (and this happens more than you might imagine), then it is possible that a device will face months of delays.

The second reason is that the cost of compliance engineering may drive a product out of profitability. A transaction cost of €1,200 for checking one device is reasonable given the current market rates, and this sum is a lot of money in the consumer electronics market. The initial release of a product is often a test run to check demand, and may consist of as few as 200 devices being made available to the public. A compliance check at this stage would raise the price of the product by €6, and while justified by law - license compliance is not based on quantity shipped - it may be difficult from an economic perspective. Even after the test run is complete and additional orders are made, if the company plans to ship 10,000 or fewer devices the cost per unit is still at least 12 cents.

Because of these two pressures the companies involved often don't spend too much time trying to understand FOSS licensing or putting the infrastructure in place to ensure compliance. They may see themselves as facing a choice of shipping non-compliant software and risking a court case or facing a market loss from missed sales. With court cases relatively rare in FOSS today, risking a legal challenge may appear to be a less painful option than the alternative.

Whether this perception will continue is debatable. Gpl-violations.org has made what appear to be permanent changes to how businesses approach FOSS in Europe since 2004, and SFLC have started to become pro-active in seeking compliance for projects in the USA. Community tolerance for negligent behavior by commercial entities is waning.

This market adjustment is predictable given the status of FOSS technology. The European Commission estimated that the ecosystem of FOSS applications with reasonable quality control and distribution in 2007 was worth around €12 billion. The cost of obtaining this code is adherence to the license terms, and with rising value creators are naturally less tolerant of misuse then they may have been when FOSS was still in its infancy.

What developers can do to protect their rights

Developers who own the copyright on code have various ways to ensure people obey the licenses. If you are not a copyright holder on code but have found clear evidence of a violation it is a good idea to tell the copyright holders. Ensuring fair play with using the licenses helps maintain confidence in the FOSS ecosystem. It means people can make a decision about how their code will be used and be reasonably sure everyone will stick to the terms.

Perhaps the most important thing when assessing violations is to get the facts right. SFLC's Legal issues primer for Open Source and Free Software projects can assist with this, as can its Practical guide to GPL compliance. The second most important thing is to be fair and professional. Emotion or lack of understanding won't help correct a problem and it certainly won't help foster a potentially useful working relationship.

If you are pretty sure a violation has taken place you can decide what route to take regarding enforcement (if you are a copyright holder) or informing the code owners (if you are not a copyright holder). The first step for everyone is probably to document everything carefully. FSFE and gpl-violations.org published a brief document on reporting and fixing license violations that explains some of the key points that you need to cover. The suggestion is that you should make a report with:

• The name of the product affected
• The reason why a violation is believed to exist
• The name of the project code that may have been violated
• A statement regarding what license this code is under
• A link to the project site

This information can then be used by you, the affected project, your lawyers, the infringing company, or a third party like gpl-violations.org, FSFE, FSF or SFLC, to examine the situation as applicable. Avoid doing things like forwarding email threads or inserting commentary as this makes it difficult to assess the situation.

For copyright holders there is an established formal mechanism to enforce copyright through legal action. This can be done by taking an infringing party to court or by seeking an out-of-court settlement. There is no doubt this approach is effective, as members of the gpl-violations.org project can attest, though it can also be costly in time and initial fees. Other avenues for correcting misuse of licenses also exist and may be quicker in some circumstances. For example, informal discussions can work with accidental infringement, and mediation by FSFE's Freedom Task Force or FSF's Free Software Licensing and Compliance Lab has also proven to be effective in the past. When it comes to legal advice, independent professionals like Carlo Piana provide excellent advice for both developers and companies with concerns.

Gaining compliance is most often an educational exercise. FOSS has a lot of a new adopters and many of the commercial entities using code in the consumer electronic market come from a proprietary background. That's no excuse for not understanding the licenses, but it is a strong case for considering how these companies can be turned into good community citizens. Productive compliance efforts should use carrots and/or sticks to encourage people to communicate and cooperate with the code creators, projects, and other businesses in this area.

Punishment is not the name of the game. Working together in good faith is.

About the authors

Armijn Hemel is a technology consultant with Loohuis Consulting in The Netherlands and the primary engineer for the gpl-violations.org project.

Shane Coughlan is a business and technology consultant with Opendawn in Japan. He is an expert in Free/Open Source Software licensing, standardization, communication methods and business development

Comments (11 posted)

Page editor: Jonathan Corbet
Next page: Security>>