Coming soon: KMyMoney 1.0

Posted Aug 27, 2009 23:57 UTC (Thu) by dlang (guest, #313)
In reply to: Coming soon: KMyMoney 1.0 by astrophoenix
Parent article: Coming soon: KMyMoney 1.0

using a database instead of a flat file will be far safer and faster to update.

you can implement all this yourself (but not easily in a ascii xml file), but at that point you have duplicated the hard stuff that a database does.

once you take the effort of converting your software to work with a database (delegating the data integrity, searching, indexing, etc tasks to the database) with an embedded database like sqlite, it's a small step to move to a full database that can offer you additional features (like replication and failover) for no additional application complexity (beyond the change to talk to that sort of database)

the application should have the ability to do import and export through flat files, but it makes a _lot_ of sense to use a database instead of flat files (and especially instead of ascii XML files) for the main datastore.

Coming soon: KMyMoney 1.0

Posted Aug 28, 2009 15:29 UTC (Fri) by astrophoenix (guest, #13528) [Link] (1 responses)

interesting. I see what you're saying. it makes theoretical sense.

but I wonder where the crossover point is: i.e., how big does your dataset have to be before a user using the gui app will notice any difference between a database backend or an xml backend?

because the other tradeoff is that it's a LOT simpler for a USER of the data to do unexpected things with the data if it's ascii.

I suppose I could just export it to the ascii, do what I want with it, and then get used to the routine of importing it back from the ascii each time I run the gui app.

before I was really thinking about the tradeoffs, the idea of a Free software project putting the data in a hard-for-the-user-to-use format seemed a bit jarring; technically the database format is still useable, but getting database redundancy and failover is a tad bit more work and more fragile than committing a text file to git or mercurial.

Coming soon: KMyMoney 1.0

Posted Aug 28, 2009 16:23 UTC (Fri) by dlang (guest, #313) [Link]

I see the biggest is the data safety issue

when making a change (no matter how trivial) you would need to write a modified version of the entire file and do the appropriate fsyncs (for the file and the metadata)

with a database you just tell the database what changes you want done and it makes the changes (probably fare more efficiantly than you will, as well as being far less likely to have missed some corner case)

searching a xml text file involves reading the file, parsing it, then walking the data in memory (or reading the file, parsing it as you go, and aborting the parsing when you have the data you need)

databases have many tricks that they can use to speed the data access (indexes, binary data representation, etc)

as for locking the data in a 'hard for the user to use' format, I would say that putting it in a standard database that many different tools can readily access (in many cases making it look like a spreadsheet, but including dump/restore to/from flat text files) makes it pretty easy to get at the data.

also, keep in mind that one commonly used database for this is sqlite. that database engine keeps all the data in one file (just as easy to check into a version control system), and does not require communication to a seperate daemon. it basicly acts as a file manipulation library in your software, so it has very low overhead.

as for redundancy and failover, that can mean many things. if you want to have the data stored on a central box (rather than sitting on someone's laptop that can break, get lost/stolen, etc) databases can do failovers to backup boxes transparently to the user

for a single home user this (usually) does not matter, for a small business this can be critical.

In addition, with a database back-end it is relativly easy to allow multiple people to share access to the data at once. again, not a feature that a single home user normally cares about.