LWN.net Logo

PostgreSQL 8.0.0 goes beta

PostgreSQL 8.0.0 goes beta

Posted Aug 11, 2004 2:47 UTC (Wed) by zone (guest, #3633)
In reply to: PostgreSQL 8.0.0 goes beta by danpb
Parent article: PostgreSQL 8.0.0 goes beta

Slony-I is not a production quality replication solution, and never will be.

It's a third-party trigger based solution. It does not have access to PostgreSQL's internal data structures and therefore has very limited decision making abilities compared to real replication systems. It cannot replicate schema changes, for example, because that is something Postgres doesn't expose (nor should it). It also uses standalone daemons which can die independently of the database, and separate configuration files that can get out of sync.

In order to be competitive, Postgres needs a built-in replication system capable of replicating everything Postgres is capable of using it's native transaction and point in time facilities rather than reinventing them outside of Postgres. Replication is still a gap that Postgres needs to fill, and I'm hoping for sooner rather than later.


(Log in to post comments)

PostgreSQL 8.0.0 goes beta

Posted Aug 11, 2004 12:20 UTC (Wed) by danpb (subscriber, #4831) [Link]

It has full access to the data dictionary so can detect inconsistencies in the schemas between master & slave. For production systems, schema changes would generally be done with managed upgrade scripts, so there is minimal admin overhead running those scripts on both master & slaves.

Whether its a standalone daemon or part of main PG daemon is pretty irrelevant - a forked process in the main server can just as easily die as a standalone process, with the added bonus that it could take out the entire DB rather than just the replication daemon. If a standalone replication daemon died, your monitoring software (eg Nagios, BigSister, many others) would alert production support & it could be re-started, at which point it'll seemlessly catch up to where it left off.

Separating configuration files from the main server config is actually desireable in some circumstances. For example, Slony-1 quite happily allows master & slave to be running different PG server versions (if the app in question supports it of course), so having replication config merged with main server config would actually increase admin burden in this case. The question of configuration file consistency is something that would be addressed OS-wide, not just for slony & PG. cf-engine is one possible tool, but there are a great many others.

PostgreSQL 8.0.0 goes beta

Posted Aug 11, 2004 12:26 UTC (Wed) by JanniCash (guest, #23933) [Link]

Just because the main DB server process starts the replication engines thread or process doesn't make it more robust. The best example for that is MySQL reporting replication healthy because the threads do exist, while no data is replicated at all any more.

A replication solution that is designed on purpose to support version upgrades from existing installations of PostgreSQL can hardly be embedded in the server engine. This concept fixed ties one version of the replication system to one version of the database. That gives zero flexibility to future enhancements of the replication system while maintaining backward database support.

I don't know where the idea to replicate arbitrary schema changes automatically comes from. A lot of people think this is a must have, while I think it is obsolete or even limits the usefullness of a replication solution. I have not seen many developers who need production quality replication for their prototyping- and test-system. Schema changes on production servers go through the same QA process as the application itself, so in a professional environment the Slony support for DDL scripts is absolutely sufficient.

Where the idea of "replicating everything" really starts to get annoying is when you intend to use certain replicas for special purposes. A replica that is serving search request might need a couple additional indexes that would only eat CPU and IO on all other nodes. The ability to replicate a subset of a databases objects and have additional triggered funcitonality on either the origin or a replica is entirely gone with the blind "copy all" approach.

Slony-I is trigger based, yes, but the triggers are implemented in C and have full access to the PostgreSQL backend internal structures. They do not write some external binary file but rather rely on the PostgreSQL crash recovery mechanism to ensure that what is committed will be replicated. The only other way to guarantee that for an asynchronous replication system (something utilizing 2PC isn't) would be to read the write ahead log. An approach done by an external process in the replication solution DataPropagator for IBM DB2. This program (capture) then inserts the extracted log information into capture data tables, read via select by an external process (apply) on the remote replica and applied via insert, update and delete operations to same. No arbitrary DDL support either. Would you claim that DataPropagator for IBM DB2 is not a production grade replication solution or could you point out in what respect Slony-I cannot compete with it?

Sincerely,
Jan Wieck
Slony-I lead developer

PostgreSQL 8.0.0 goes beta

Posted Sep 27, 2004 3:20 UTC (Mon) by nextdude (guest, #24999) [Link]

> I don't know where the idea to replicate arbitrary schema changes
> automatically comes from.

How about an object-oriented persistence framework that runs on a server than needs 100% uptime? I've got an object-oriented application where the structure of existing object classes may change (occasionally) or where new object classes are added (frequently). In either scenario, I'd like a replication solution that can transparently replicate the necessary schema changes on the fly to minimize downtime with no further intervention required at the database level.

How can SLONY-I support that?

Copyright © 2008, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds