A report from the (not only) MySQL conference 2011

The MySQL Conference and Expo in Santa Clara, the largest open source conference in the San Francisco Bay area since the decline of LinuxWorld, was unusual this year in several ways. First was the inclusion of many non-MySQL databases in the conference. The second was the mostly-friendly rivalry of the several MySQL forks, who presented co-equal keynotes. The dominant discussion at the conference, however, was the stormy relationship between the MySQL community, the conference, and Oracle.

PostgreSQL and more

The biggest official change this year is that conference organizers O'Reilly and Associates decided to bring in non-MySQL open source databases in order to expand the scope of the conference. Most notable among these was the long-time MySQL "rival" project, PostgreSQL. The PostgreSQL support company EnterpriseDB was the top sponsor of the conference, and dominated the expo floor with a huge presentation area. There were two PostgreSQL tutorials and seven talks in the conference program.

In addition to PostgreSQL, several other open source database projects were invited to give talks at the conference, including MongoDB, CouchDB, Cassandra, Redis and HBase. MongoDB was particularly active on the trade show floor, and their ubiquitous coffee mugs turned up all over the conference.

The PostgreSQL presence at MySQLCon included at keynote in which EnterpriseDB staff went over features for the recently released version 9.0 and the upcoming version 9.1. The 9.1 features will be covered in a future LWN article.

Hot-and-cold Oracle

Oracle's relationship with the conference bordered on the schizophrenic. According to a member of the conference committee, Oracle sponsored the conference but refused to allow their name to be listed as a sponsor. For several months Oracle refused to allow their staff to submit talks to the conference or attend, relenting at the last minute and sending several speakers and a small marketing crew, according to a member of the Oracle staff.

Oracle did sponsor a party on Wednesday night of the conference. However, Oracle also sponsored a new MySQL track at IOUG's Collaborate conference at the same time, in Florida, 3000 miles away. While by all reports attendance of the MySQL track at Collaborate was poor (some witnesses reporting as few as 50 people at the keynotes), MySQL luminaries employed at Oracle, as well as MySQL-using Oracle partners, were obligated to go to Florida and not to California.

This cannot have helped attendance at the MySQL Conference. Indeed, attendance was down to around 1100 compared with over 1500 last year according to conference organizers.

MySQL

On Tuesday, Tomas Ulin, Oracle's MySQL engineering manager presented the recent improvements which Oracle has introduced in version 5.5, what is planned for 5.6, and some of the changes that have been made to MySQL in their first year of stewardship of the project.

One of the biggest changes to MySQL 5.5 is that InnoDB, the primary and most mature transactional database engine for MySQL, is now the default. This is welcome news since one of the chief causes of inexperienced MySQL users losing data is use of the older, non-crash-safe, MyISAM engine. The MySQL and InnoDB teams at Oracle have also been merged, "as they always should have been" according to Ulin.

Other 5.5 features include substantially improved performance on Windows, enhanced partitioning, and the performance_schema, which is a tool to collect runtime performance data about MySQL queries.

Ulin also announced a "development release" of MySQL 5.6. Oracle's goal is to make development releases very stable, so that they can move from the development releases to final release quickly. Features for 5.6 include more improvements to partitioning and additional views in performance_schema. MySQL is also adding additional query optimizations to improve performance on large databases, such as multi-range reads, sort optimizations, and pushdown of predicates into subqueries.

MySQL 5.6 may also include enhanced integration with Memcached. This includes the ability to use the Memcached protocol in order to access the InnoDB storage engine directly, effectively making InnoDB a NoSQL and SQL database at the same time.

Ulin also went over Oracle's plans for MySQL Cluster, otherwise known as NDB. MySQL Cluster is a specialized database engine aimed at telecommunications companies, and has been commercially successful in that market. "If you make a phone call today, MySQL cluster is probably involved somewhere," said Ulin.

Version 7.2 will include support for some kinds of JOINs across clustered tables, which NDB has not previously had. It will also include increases in the number of columns per table it can support, and replication of user privileges for faster access. It is also expected to support dynamic switching between SQL queries and direct object-oriented access to the database engine.

MariaDB

The two successive acquisitions of MySQL, by Sun and then by Oracle, have spawned a number of forks, each of which is pursuing its own course of development and community. On Wednesday, Monty Widenius, founder of MySQL, presented MariaDB, his MySQL successor database — or fork — which his company in Finland has been developing for the last two years. Several of the original MySQL developers are now working on MariaDB.

MariaDB is primarily meant to be a drop-in replacement for MySQL 5.1. Its main advantage is being pure open source and not owned by Oracle. MariaDB has also released an LGPL C-language driver for MySQL, which according to Widenius resolves some of the licensing issues with Oracle MySQL drivers.

Beyond that, MariaDB 5.2 was released recently with a number of useful features not present in mainstream MySQL. Widenius announced that for the first time Sphinx full text search is available directly as a storage engine called SphinxDB. MariaDB contains multiple improvements to MyISAM storage and supports pluggable authentication. It also has added "virtual columns", which hold automatically updated calculated values based on the other columns in the table.

MariaDB 5.3 will include an extensive rewrite of the query optimizer which is supposed to improve response time on more complex queries by orders of magnitude. It will also support a new, faster form of group commit for faster database writes.

Widenius also announced that MySQL support company SkySQL would be offering support for MariaDB. SkySQL is a new MySQL support company in Finland created by a group of former MySQL AB staff and MySQL co-founder David Axmark.

Drizzle

Probably the most conspicuous MySQL fork at MySQL Conference and Expo, due to the number of speakers, was Drizzle. Drizzle is optimized for usage on cloud hosting, as well as being extensively rewritten to clean up the code. The project's team is made up of both former MySQL developers and new contributors with Rackspace as their primary commercial sponsor.

On Wednesday, Brian Aker took the stage for Drizzle. His big (if rather belated) announcement was the general availability release of Drizzle, which is now ready for its first production use as of around a month ago. The second announcement was that MySQL support and services company Percona had announced commercial support for Drizzle.

The Drizzle team has spent the last three years redeveloping MySQL around a "micro-kernel" architecture. This means that they've taken many things in the MySQL core code, simplified them, and converted them to "plugins", allowing individual users to reconfigure how Drizzle works. Their refactor of the code has also eliminated many longstanding MySQL "gotchas" regarding Unicode support, timestamps, constraints, Cartesian joins, and more.

Since Drizzle is built "for the Cloud", a strong part of its focus is on replication. Drizzle supports row-based replication using protobufs, an open format created by Google. The new open replication format supports integration with a variety of other tools such as ApacheMQ, Memcached, and Hadoop. It also supports multiple masters, partial replication, and sharding. In development are virtualized database containers using database "catalogs".

Aker also announced libDrizzle, a client library which works with MySQL and SQLite as well as Drizzle. Since this driver is BSD-licensed, he believes it will be attractive to users who have legal concerns about the MySQL driver licensing.

The future of open source databases and MySQL

The conference ended on Thursday with talks from Baron Schwartz of Percona and Mike Olson of Cloudera and BerkeleyDB on the future of databases. The two talks were remarkably similar in their predictions:

• databases will replicate and cluster seamlessly
• data will grow to petabytes and more, even for smaller organizations
• databases will integrated better with the rest of the stack
• databases will support a variety of different data formats
• people are using a variety of special-purpose databases now, but future databases will be more all-purpose
• data caching and databases will stop being completely separate layers, but will be fused

The main difference between the two presentations was on the place of relational vs. non-relational databases in the future of databases.

The future of MySQL was rather more of a source of anxiety for the attendees. As a PostgreSQL geek, I got asked repeatedly where PostgreSQL would be in five years by MySQL users who were clearly wondering the same thing about MySQL. All of the forks and the love/hate relationship with Oracle have undermined confidence in MySQL, and sent users looking for alternatives, or for reassurance.

As for the future of MySQL Conference and Expo, the rumor at the conference was that O'Reilly plans to move it away from Santa Clara. An attendee named Olaf even ran vote-by-Twitter poll for a new location.

Videos from the conference are available on blip.tv.