There would be obvious value to a mechanism by which hardware purchasers could know which products are truly open without having to dig around on the net or risk buying the wrong device. There is clearly interest in this information; look at the extensive lists of routers which can run distributions like OpenWRT, for example, or the long list of Android phones offered through online auction sites which are marked as being rooted. But hardware is, for all practical purposes, never labeled as being open in a useful way, even when the hardware is indeed open. There is an information gap here: manufacturers are not informing their customers of an attribute which could make their products more appealing.
With that in mind, your editor looked at the Free Software Foundation's recently-announced endorsement criteria for its new "Respects Your Freedom" mark. Attempts to identify free-software-friendly hardware have come and gone in the past, but the FSF might just have the staying power to make something stick. Unfortunately, in your editor's opinion, this initiative is flawed in a way which has doomed a number of FSF initiatives. Someday we may have a mark which makes open hardware easy to identify, but it won't be this one.
There is much that is good in the FSF's criteria. At the beginning is the obvious requirement that the device should work with 100% free software - though even the FSF makes exceptions for "auxiliary processors." So a cellular handset, with a closed baseband processor, should still qualify. It must also be possible to replace the running software using only free tools; devices with locked-down memory or cryptographic signature verification need not apply. These requirements are an obvious description of what's required for a piece of hardware to be truly open.
Interestingly, the device is allowed to implement DRM mechanisms - but only in free software, so the DRM can be removed by a suitably skilled and motivated user. On the other hand, the device is not allowed to phone home with identify or location information except when the user has asked for that behavior. One could argue that, for the purposes of judging the openness of hardware, phoning home could be seen in the same light as DRM: OK as long as it can be ripped out of the device. If the goal is "respects your freedoms," though, it is not that hard to make a case for mandating more respectful treatment of personal information from the outset. So far so good.
Consider, though, this aspect of the "100% free software" requirement:
The FSF has never been content to work toward the creation of free software and advocacy for its use; it has also made an overt effort to ensure that, like an Orwellian "unperson," proprietary software is never even mentioned. So a mobile device running a system like MeeGo might qualify for the FSF's endorsement, assuming it's open, lacking binary drivers, etc. But if the application installer lists a popular proprietary Flash plugin or network telephony application, it may be deemed to be "steering users" toward non-free code. That would cost it the endorsement, despite the fact that it's a fully open and respectful device.
Let it be said: your editor does not believe that "respect your freedoms" includes hiding information about available options. Free software should be able to win on its own merits; it doesn't require attempts to create ignorance about proprietary alternatives. Viewing users as needing to be "steered" in the right direction does not seem respectful.
This requirement, alone, is probably enough to drive otherwise friendly manufacturers away from seeking endorsement for any kind of device which will have an associated application store. But it gets worse:
It's a rare manufacturer indeed who will not mark a Windows-compatible product as being compatible with Windows. A requirement like this would force such a manufacturer to sell the same product in two different packages - an expense that is unlikely to be made up through extra sales of FSF-endorsed devices. Manufacturers cannot usually afford to ignore the existence of large, lucrative markets, so they will inevitably decide to do without the FSF endorsement, even if their product would otherwise qualify.
Finally, the criteria require "cooperation with FSF and GNU public relations," described this way:
This requirement has little to do with respect for a user's freedoms and everything to do with promoting the FSF's particular agenda and world view. To obtain the FSF's endorsement for a specific device, a company must train all of its representatives in the use of Stallmanesqe newspeak. It is a mixing of two entirely different objectives - promoting open hardware and promoting the FSF's world view - that seems likely to be detrimental to both. Companies have learned to be careful in how they use each others' trademarks, but that does not extend to wider restrictions on "approved terminology." One can easily see a corporate lawyer balking at such a requirement; as a result, an endorsement mark which would have carried the FSF's name and URL will not appear.
We owe a lot to the Free Software Foundation for helping to make the free software explosion happen when it did. Without the FSF, things would have happened differently and more slowly, and the world would certainly have been worse. The FSF still serves an important role; we need a no-compromises advocate for free software out there. But, by conflating free software with control over language and options, the FSF often seems to work counter to its stated goals. That is certainly the case here; your editor predicts that the number of products carrying the FSF's endorsement will be easily counted without running out of fingers. An opportunity to recognize and promote freedom-supporting hardware has been lost, and that is a sad thing.
Over the last few years, Red Hat has taken the lead in investigating ways to apply the concepts behind free software to various other fields. That has led to things like the book The Open Source Way and the opensource.com web site. Karsten Wade of Red Hat's community architecture team came to the Utah Open Source Conference to formalize some of the ideas that underpin open source and describe how they can be applied more widely. It is, he said, an effort to "decouple open source from technology", so that other people can "remap" those ideas onto other fields.
The underlying idea behind free and open source software is the four freedoms embodied in GNU's free software definition, and the idea that "none of us are free until all of us are free", Wade said. There is very little difference between the free software and open source camps, but there is an "artificially constructed fight" between the two. This conflict is sustained, largely by the press, to make it look like there is some "deep inherent argument" between the supporters of each.
But if we remove all of the labels, both sides agree on 85-95% of the issues, he said, and the differences are largely in what the priorities are. The idea of "free software" has worked well for hackers, while "open source" has been popular with businesses, as "it resonates with them". That has led to there being a really strong brand around the open source term, which is why Red Hat (and others) talk about "the open source way". They are "riding on the coattails of a well-known brand", Wade said.
Certain elements must be present in any open source endeavor. There needs to be an infrastructure set up that fosters participation, as well as an infrastructure to share the results of any work. Obviously, there has to be something to share, along with people to share it, i.e. participants. These elements are also present in what is known as a "community of practice"—a sociological concept that shares much with the open source way. Looking at communities of practice can help to understand the communities that already exist for FOSS, as well as to help shape new communities as they arise.
A community of practice is a group of people that come together because they share a concern or passion. There is a specific domain that the community of practice operates in—the concern or passion of its members—along with a "practice", which is how they address problems in that domain. These are much like—perhaps identical to—the requirements for using open source techniques. In addition, communities of practice have a number of principles that help separate them from other kinds of groups.
It is important that a community of practice be designed for evolution—it is not just thinking about what is going on now, but allowing for new ideas that will help define its future. A dialog between those inside and outside of the community is important, so that the group doesn't become insular. There should be different levels of participation available, which will allow anyone who is not harming the group to still do what they want: "if they want to fold napkins, let them". Often these peripheral participants learn how to get things done within the group by finding out who to talk to; it is a stepping stone.
Communities of practice also develop in both public and private spaces. It is essential that they are governed in public, but private talks are important as well. People need to be able to get together privately, over drinks for example, to discuss things outside of the public sphere. These communities also focus on value, because "people want to belong to something that makes a difference". If these things sound familiar, it is because FOSS projects are almost always communities of practice.
Wade used barn-raising as an example of where some of the ideas behind open source comes from. If you want to get people together to build a barn, you don't just stack up wood, cement bags, and shovels, then ask everyone to dive in. You first need to survey the land, build the foundation, and get it ready for lots of people to participate. "The infrastructure had to be there so that everyone could do the common work", he said. FOSS projects are much the same way.
Another analogy he used was that of musicians coming together on the village green. Each musician brings their style and tunes to the common space, and each is ready to learn from what the others bring. As the "jam" progresses, there is a friendly competition between the participants to try to outdo each other. That is similar to how FOSS project participants work together.
While many believe that FOSS works on the "Tom Sawyer" model, where one group or organization takes advantage of the work of others (much as Tom took advantage of his friends' fence-painting work), that is not the open source way, Wade stressed. Red Hat and others are often accused of that, but that's "not how it goes"; the community will notice freeloaders. Some may get away it for a while, but eventually it will be noticed.
Michael Tiemann's experience showing his daughter how to work the resonant pendulum at San Francisco's Exploratorium was Wade's final analogy. In order to move that pendulum, it takes regular, small tugs on the weak magnet, but eventually the 350-pound pendulum will be swinging four feet in either direction. In FOSS, Wade said, "sometimes it seems like we have to wrap a rope around it and give it a huge tug", but we don't need that, and little incremental things (like regular releases) can make all the difference.
As the talk wound down, Wade surveyed the audience for additional examples of communities of practice and open source techniques being used in other fields. He also showed two videos from opensource.com that described how two very different fields (seed banks and film-making) were recognizing that open source ideals and techniques can be—and are being—used in their work. The open source way is a powerful tool that has been in use for a very long time, and in a wide variety of places. Talks like this one can only help to spread that word, so that it can be applied even more widely.
For years, MySQL has been the highest-profile open source relational database system, but with the Sun (and, later, Oracle) acquisition of MySQL's corporate parent MySQL AB, the development community has split in several directions. Now, a few years later, both of the leading community-driven forks of MySQL, Drizzle and MariaDB, have made important new releases. Drizzle, the light-and-lean database system designed for web and cloud applications, unveiled its first beta release — complete with MySQL migration tools — and MariaDB, the full-featured database system positioned as a direct competitor to MySQL, made a "gamma" release, and picked up an important endorsement.
Drizzle build 1802 was released at the end of September, and was dubbed the "Drizzle7 beta release" in the accompanying announcement. In addition to the usual assortment of speed-ups, bug fixes, and new options, three major features grabbed the headlines. One is the introduction of Sphinx-based documentation. Sphinx is a documentation system based around the reStructuredText markup format, which is intended to make it easier to integrate application documentation inline within the source code itself. Indeed, Drizzle is taking advantage of this feature, storing its documentation in its source tree.
Of more importance to database users, however, are two features that simplify the transition from MySQL to Drizzle. First, the drizzledump backup and restore utility now has the ability to detect when it is run against a MySQL database, and export a dump of the database in a Drizzle-compatible format. For the slightly more daring, it can also dump the MySQL database and import the data and structures directly into a Drizzle database in a single command. Either way, it eliminates the need to run a costly conversion between the two applications.
Second, Drizzle can now speak MySQL's native TCP/IP protocol. By default, Drizzle uses the same TCP port reserved for MySQL, 3306. Future plans are to develop a separate Drizzle protocol running on TCP port 4427, but for the time being, the ability to use MySQL's network protocol has the effect of making it much easier to port applications written for MySQL over to Drizzle.
This feature includes the network protocol only; Drizzle does not support Unix sockets as a connection method, which is part of the project's stripped-down philosophy. Drizzle was started by former MySQL architect Brian Aker in 2007 as a response to what he felt was MySQL's increasing focus solely on enterprise applications, abandoning many of the project's original constituents, web application developers. Aker has said on several occasions that he wants to develop Drizzle as a community project, in contrast to the final days of his involvement with MySQL, when virtually all of the MySQL developers were employed working on the project full-time, and patches from outsiders dropped to virtually zero.
Drizzle adopted a smaller, faster "microkernel" architecture, stripping out advanced functionality such as views, triggers, and query caching, while pushing many of the remaining functions (such as logging or authentication) into pluggable modules. In several places, it simplifies MySQL's multi-faceted design, such as offering only one type of binary blob, specifying UTF-8 as the text format, and UTC as the only timestamp "time zone" format. The result is a faster database management system around one-third the size of MySQL, and one that Aker hopes will be easier for new developers to understand and contribute to.
The project allows all contributors to retain their own copyright on their contributions. The project uses Bazaar as its source code management system, making incremental releases every two weeks. Aker stated in 2009 that there were more than 100 contributors to the project, which is roughly in line with the size of the Drizzle-developers team on Launchpad.net. According to Launchpad, there are 325 active branches under development, owned by 66 developers or teams. Also telling is that the developers hail from a variety of different employers, including Canonical, Google, Oracle/Sun, and Rackspace.
In addition to the community focus, Drizzle is optimized for "cloud" and web application usage in a number of ways. As mentioned earlier, it provides TCP/IP as its only connection method. It is also optimized for 64-bit processors and "massive concurrency" over multi-core and multi-CPU machines, including sharding across multiple nodes. Finally, it is built for Unix-like servers only (offering no Windows version), and supports external stored procedures in scripting languages like Ruby, Perl, and PHP.
While Drizzle is an attempt to hone the MySQL code base into a lean-and-mean database manager, MariaDB takes nearly the opposite approach, building a system with an array of high-end options well suited for enterprise usage. MariaDB was started by MySQL creator Michael "Monty" Widenius in 2009, with the goal of developing a community-driven project that could serve as a drop-in replacement for the official MySQL.
The 5.2.2-gamma release of MariaDB was also announced at the end of September, and is described as a "release candidate" marking the end of the 5.2 development cycle. The list of new features is tellingly longer than that of Drizzle's, including a reworked version of the default InnoDB storage engine and two entirely new storage engines: OQGRAPH, which is designed for storing tree-like structures and complex graphs, and Sphinx, a text-oriented storage engine (this Sphinx bears no relation to the Sphinx documentation system used in Drizzle; chalk it up squarely to coincidence).
Also new is support for virtual columns (fields containing expressions that are evaluated upon retrieval), segmented key caches for the MyISAM engine (which allow multiple threads to fetch keys simultaneously without locking the entire cache), the ability to CREATE tables with storage-engine-specific attributes, an extended user statistics system, and pluggable authentication.
MariaDB 5.2.2 is based on the MySQL 5.1.50 source code, but several of the new features mentioned above — such as extended user statistics and segmented key caching — come from other sources. On top of that, some of the new functionality is still in development for Oracle's MySQL, including virtual columns. The official MySQL's pluggable authentication system is available only to Oracle customers with commercial support contracts. MariaDB comes with an authentication module that allows the system to use existing MySQL user accounts, thus easing the transition between the two products.
On the whole, however, MariaDB aims at compatibility with MySQL. Widenius's new venture is partly an attempt to rebuild the community-based development approach that MySQL enjoyed in its early days, and partly an attempt to build a different business model around database development. Unlike MySQL AB, which was sold to Sun in a deal that he engineered, Widenius describes his new business Monty Program AB as a "hacker business model" where revenue from its support contracts go directly back into maintaining the code. He also founded the non-profit Open Database Alliance with other MySQL service providers to attract various independent support providers and database resellers.
Like Drizzle, the MariaDB source code is hosted at Launchpad.net. In contrast, however, contributors must sign a contributor agreement that assigns joint ownership of the contribution to Monty Program AB. Monty Program AB employees also review all patches and contributions and approve membership in the Maria-captains team that has commit rights. According to Launchpad, there are 21 Maria-captains members, and 149 in the larger Maria-developers group, all working on 62 active branches.
Despite intentionally following the official MySQL development series, MariaDB has started to attract attention on its own. Last week, a number of former MySQL executives launched SkySQL, a database support company competing head-to-head with Oracle's services — including support for MariaDB alongside support for MySQL. SkySQL executive Kaj Arno told InternetNews.com "If you are a MySQL customer and your bug is fixed in MariaDB, I think it might make sense to move," though he added that encouraging customers to migrate was not the company's goal.
To say that Oracle's acquisition of Sun and the open source projects it stewarded has been poorly-received by the community would be quite the understatement. This month, the big debate is over OpenOffice.org (OOo) fork LibreOffice, and Oracle has taken a hard line: renewing its public commitment to OOo and threatening to excommunicate OOo community council members who do not distance themselves from the new project.
Looking at how well the MySQL forks have matured, however, it does not look like LibreOffice supporters have too much to fear. Drizzle and MariaDB are both prepared to help any interested MySQL users migrate away from the platform. Drizzle is taking shape as a fast and light replacement for the large market segment of customers whose MySQL database is primarily designed to serve as the back-end of a web application, while MariaDB is actually ahead of MySQL on supporting high-end features for enterprise customers. In either case, MySQL may not enjoy its current position as the default database of choice for much longer.
Page editor: Jonathan Corbet
Copyright © 2010, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds