Leading items [LWN.net]

A look at PostgreSQL

July 7, 2004

This article was contributed by Joe 'Zonker' Brockmeier.

With the 7.5 release of PostgreSQL not too far away, and news of new features sponsored by Fujitsu and Software Research Associates (SRA), we decided to take a look at the PostgreSQL project and what users might be able to expect in the coming months. We spoke to PostgreSQL steering committee member Bruce Momjian about the upcoming 7.5 release, and the "state" of PostgreSQL. According to Momjian, "the project is doing very well."

We're very organized and thorough in the way we do stuff. That's kind of paid off [in that] every three or four months it seems like we're making another kind of milestone in what we can do with Postgres in terms of adoption and features. It's kind of hard to put it into words, I've stopped getting surprised at how successful it's been.

Though each new release is a milestone, Momjian said that the 7.5 release would have an unusual number of new features. In part, that's thanks to Fujitsu and SRA underwriting the development of tablespaces, nested transactions development and support for Java server-side programming. Momjian is employed by SRA to work with PostgreSQL and the community, and says the company approached him to broker arrangements with developers already working on those features:

Big missing functionality typically takes weeks to develop, very hard for developers to spend weeks volunteering, they've got to put food on the table. Fujitsu would supply X amount of money for the amount of time they're spending working on these features, [which were] very slow going because they were only spending a few hours a week... the infusion of cash allowed them to commit weeks.

The tablespace feature will allow a database to be spread across multiple storage devices. Currently, PostgreSQL requires all of a database to exist on a single filesystem. This can be a problem for performance and space reasons. In 7.5, by default, PostgreSQL will continue to store everything on the same filesystem, but Momjian said that an administrator will be able to use tablespaces to move a table or entire database to another filesystem. Even better, Momjian says that this will not impact an application using the database -- so existing applications will not need to be rewritten to use a database that takes advantage of tablespaces.

Oracle users and developers will know nested transactions by the name "savepoints." This feature in 7.5 will give developers "better control over failure cases with multi-statement locks" and allow developers a better option than simply causing an entire transaction to fail if one statement fails. Momjian noted that PostgreSQL already had "a robust system" but that developers porting applications from Oracle needed finer control than the current PostgreSQL system allows. "Some applications needed logic that would say 'I want to try inserting, but if that fails, I want to do something else.'"

Another feature in 7.5 of interest to many users will be point-in-time recovery. With point-in-time recovery, PostgreSQL will allow users to recover information "up to the instant of hardware failure."

Of course, not all PostgreSQL users are defectors from the Oracle camp. The focus of late for many open source projects seems to be on the "enterprise" features, which might lead hobbyist and small business users to wonder whether those projects will continue to be suitable for their use. We asked whether focus on enterprise features might detract from the "little guy," and he said that while PostgreSQL 7.5 will have many features that are aimed directly at the enterprise users, the PostgreSQL project isn't losing sight of the small-scale users. In fact, there are several features that are directly aimed at the little guy rather than enterprise users.

One of those features is direct import of comma-separated value (CSV) files. Momjian said that many users have asked for the ability to directly import a CSV file produced by a spreadsheet program or other utility. Prior to 7.5, users would have to convert those files into a suitable format for PostgreSQL to import using a Perl script or other utility -- but with 7.5 users will be able to "load CSV natively right into Postgres."

Another "little guy" feature of interest in 7.5 is the ability to change the data type of a column. In prior versions of PostgreSQL, it would be necessary to add a new column, import data from the existing column into the new column, drop the old column and then rename the new column to change the data type. In 7.5, users will be able to simply alter the data type of a column in one easy step.

Momjian also said that the Postgres developers do worry about "bloat," and that "we've managed to come very far with adding features, without impacting performance or readability [of the PostgreSQL code.]" On average, he said that PostgreSQL adds "maybe 50,000 lines every year to the code...no feature goes in unless it fits like a glove."

Though not part of the 7.5 release, the recently announced Slony-I replication system bears mentioning as well. The Slony-I replication system, sponsored by Afilias, does asynchronous master-to-slave replication, slave promotion and failover.

In addition to the obvious new features, there's also a little work underneath the hood that will benefit PostgreSQL users as well. Momjian told LWN that the PostgreSQL team had done a "major redesign" in the way that PostgreSQL buffers disk writes, which will result in a "serious performance improvement" in the next release.

Though perhaps of little interest to the LWN readership, Momjian also pointed out that 7.5 will be the first version of PostgreSQL to have a native port to Win32:

We feel that the Windows port is important to highlight the accomplishments of open source to the people running on the Windows platform. You can't show how good open source is if it's not running on their platform.

There is no set date for the 7.5 release yet, but he said that it should be out be out by the end of the year, once the project has been able to conduct extensive testing of all the new features. After the release, he predicts "increased migration from proprietary databases," and notes that the PostgreSQL project is already seeing 1,000 to 2,000 downloads per week of the unofficial, unadvertised testing release of PostgreSQL for Windows.

In all, the next release of PostgreSQL should be quite impressive, and allow a number of organizations to dump expensive proprietary databases for an open source alternative.

Comments (13 posted)

Looking forward to OLS

The 2004 Ottawa Linux Symposium starts on July 21. The content this year looks as good as ever: the list of presentations includes well-known Linux developers from all over the world. As usual, the talks place OLS at the forefront of kernel-oriented Linux conferences, with some don't-miss desktop topics thrown in as well. It will be a great gathering for anybody interested in where Linux is going, or who just wants to hang out with a lot of developers and drink too much beer. At least, for anybody who has registered; OLS is sold out and is no longer accepting registrations.

Once again, OLS will be preceded by the invitation-only Kernel Summit. At the same time, the Desktop Developer's Conference will be happening upstairs; registration for that event is still open.

The 2004 event will be the sixth annual Ottawa Linux Symposium. We talked briefly with OLS founder and organizer Andrew Hutton about the event.

LWN: The sixth Ottawa Linux Symposium will be happening next month. Can you tell us how this event got its start? What inspired you to create OLS?

After attending Linux Expo in North Carolina in 1998 and 1999 and the Atlanta Linux Showcase I noticed that the technical events were in danger of being overshadowed by the Dot.Com inspired multi-million dollar marketing events that were beginning to happen at that time. Nobody I knew would voluntarily go to one of these new marketing events. At about 4am one morning while thinking about this problem I asked Alan Cox if he'd consider coming to Ottawa and doing the keynote for a new event on the other end of the spectrum, a pure technical event. He said something like 'sure haven't been to Canada yet, why not' and 3 months later we had the first Linux Symposium.

LWN: OLS has become one of the definitive gatherings of free software developers, especially in the kernel area. How is it that OLS is able to attract such an impressive list of participants - many of whom have to travel a long way to get there - every year?

Content, content, content. Above all else we try to attract the best leading edge content we can. The goal is to create an environment in which nobody goes to a presentation without learning something new about the subject.

LWN: This year, the Desktop Developers Conference will be happening immediately prior to OLS. Can you tell us a little about this event and your expectations for it?

The goal is to bring together the various parties involved in a functional free desktop from kernel people, to X developers, distribution builders, desktop infrastructure people (GNOME/KDE/etc) and application developers to share experiences and discuss the areas in which future cooperation is possible.

LWN: The 2004 Kernel Summit will also be happening just before OLS. Do you expect to host more such events in the future, along the lines of the successful "miniconfs" which accompany Linux.Conf.Au?

For smaller groups we've encouraged this for years. The Desktop Developers' Conference will be the first of the more public ones though. It may or may not remain adjacent to the Linux Symposium in the future. The main reason it is this year is that despite all the buzz you've heard about the future of the desktop, there isn't a lot of support for it yet and this makes it easier for people to justify attending both at this time.

LWN: Another Linux.Conf.Au idea that seems to work well is moving the conference to a different city every year. Might we ever be able to look forward to the Jasper or Victoria Linux Symposium?

Probably not. We discuss this every year and people just enjoy coming to Ottawa ever year. Ottawa is a nice tourist town these days, and has the facilities we require all within walking distance. One of the great things about OLS is never needing a car.

LWN: The Symposium is currently limited to about 500 attendees. Do you think you may ever allow OLS to become larger? Why?

There are two main reasons. Space and communications overhead. It is nice to have time to find and sit and chat with all the people you're looking for during the event. We do end up a bit larger than 500 some years, but for now the space we have isn't suitable either. To keep things productive keeping it small is key.

As usual, LWN editor Jonathan Corbet will be present at OLS and the Kernel Summit this year.

Comments (none posted)

Europatent preview: Godado patents search engines

Anybody who is curious about what benefits software patents might bring to Europe need look no further than UK patent GB2362971, entitled "A method of searching the internet and an internet search engine." This patent, held by the Italian company Godado Italia Srl, was first filed in May, 2000; it was assigned last February.

What does this patent cover?

Upon receipt of a search signification, a search is conducted for web sites having a textual match with the search signification. In addition, the thesaurus database is searched to determined the category of meaning to which the search signification belongs and the meaning of the search signification thus determined is used to identify related significations having a correlation with the meaning of the search signification. The enquirer is then provided with a list of web sites having a textual match with the search signification and with a list of related significations as a suggestion for supplementary research.

In other words, a search engine with the advanced capability of looking up additional search terms in a thesaurus and telling the user about those terms.

Godado is not content to sit on this patent. The company has applied with the EPO for a Europe-wide patent, and has also filed a claim in Italy. With those in hand, Godado has selected its first target: the financial portal Portalino. For the curious, Portalino has posted Godado's demand letter (in Italian); your editor has created an English translation to go along with it. Essentially, the letter accuses Portalino of the heinous crime of running a search engine, claims that said search engine is an infringement of Godado's patent, and demands that the search engine be shut down immediately.

One might assume that Godado does not intend to content itself with harassing Portalino; according to this Punto Informatico article, the patent has already been filed in Spain, Portugal, Germany, and France (along with the UK and Italy). A new litigation company, it would seem, has been turned loose in Europe.

This patent was not filed until 2000; chances are that, with a bit of (yes) searching, sufficient prior art can be found to invalidate it. This will not be the last shakedown attempt by a company wielding a suspect patent, however, especially if the European Union blesses software patents in their full glory. Godado shows that U.S.-style software patent hassles can become part of the European landscape. Unless, of course, the EU manages to avoid the imposition of union-wide software patents.

Comments (8 posted)

Page editor: Jonathan Corbet
Next page: Security>>