User: Password:
Subscribe / Log in / New account

PostgreSQL pain points

PostgreSQL pain points

Posted Mar 27, 2014 3:34 UTC (Thu) by zblaxell (subscriber, #26385)
In reply to: PostgreSQL pain points by jberkus
Parent article: PostgreSQL pain points

> Database blocks become difficult to resize and move.


> Can't keep up with hardware advances in a timely fashion.

Most of that happens below the block device level, so filesystems and raw partitions get it at the same time.

> Can no longer use standard tools like "rsync" with database files.

Databases tend to have their own. You often can't use rsync with a live database file on a filesystem either.

> The database project now needs a staff to maintain what's basically their own filesystem

That private filesystem doesn't have to do much that the database wasn't doing already. You could skip an indirection layer.

Unlike a filesystem, a database is not required to support legacy on-disk data across major releases (your DBA must replicate, dump/restore, or in-place upgrade instead). This means the private database filesystem could adapt more quickly to changes in storage technology compared to a kernel filesystem.

> Throwing away all of the good stuff developed by Linux IO and FS geeks over the last 20 years.

You are assuming that FS geeks are developing stuff that is relevant for databases. A database might be better off freed from the constraints of living with a filesystem layer (and legacy filesystem feature costs) between it and its storage.

OTOH a filesystem might be better after all--but that has to be proven, not assumed.

> Clobbering all other IO-using software on the same machine.

That's also true in the filesystem case.

(Log in to post comments)

PostgreSQL pain points

Posted Mar 27, 2014 6:22 UTC (Thu) by amacater (subscriber, #790) [Link]

A database might be better off ...

In a slightly different context - IBM Clearcase did/does something similar.
Softtware snapshotting and versioning by intercepting file system calls and writing to a custom intermediate file system level.

Result: everyone's worst nightmare if a large disk fails - IBM _might_ be able to recover your life's work if you can send them the entire filesystem.

And yes, dirty pages and flushing are fun :(

PostgreSQL pain points

Posted Mar 27, 2014 8:00 UTC (Thu) by marcH (subscriber, #57642) [Link]

A database needs a backup strategy built-in anyway.

About ClearCase: anyone who has used it (and used other things) knows it was one of the worst pieces of engineering ever. So, if you want to be convincing I suggest not using it as an example in any point you are trying to make.

Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds