| |||||||||||||
Coming soon: KMyMoney 1.0Coming soon: KMyMoney 1.0Posted Aug 27, 2009 23:01 UTC (Thu) by astrophoenix (guest, #13528)Parent article: Coming soon: KMyMoney 1.0
what in the world is the big move to database backends? I like having the data stored in an ascii xml file...
'gramps' has already moved to a database storage model, and I hate it. I liked doing crazy ascii type stuff with the data, like being able to read it! or tweak stuff by hand occasionally. or manage it using git or mercurial. now if I simply open gramps, then close it, I get a bunch of binary changes that are meaningless; I didn't actually change any data.
can anyone explain the advantage of using a database instead of an xml file for something like a personal accounting app??
frustrated, and loving ascii,
(Log in to post comments)
Coming soon: KMyMoney 1.0 Posted Aug 27, 2009 23:57 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link]
using a database instead of a flat file will be far safer and faster to update.
you can implement all this yourself (but not easily in a ascii xml file), but at that point you have duplicated the hard stuff that a database does.
once you take the effort of converting your software to work with a database (delegating the data integrity, searching, indexing, etc tasks to the database) with an embedded database like sqlite, it's a small step to move to a full database that can offer you additional features (like replication and failover) for no additional application complexity (beyond the change to talk to that sort of database)
the application should have the ability to do import and export through flat files, but it makes a _lot_ of sense to use a database instead of flat files (and especially instead of ascii XML files) for the main datastore.
Coming soon: KMyMoney 1.0 Posted Aug 28, 2009 15:29 UTC (Fri) by astrophoenix (guest, #13528) [Link]
interesting. I see what you're saying. it makes theoretical sense.
but I wonder where the crossover point is: i.e., how big does your dataset have to be before a user using the gui app will notice any difference between a database backend or an xml backend?
because the other tradeoff is that it's a LOT simpler for a USER of the data to do unexpected things with the data if it's ascii.
I suppose I could just export it to the ascii, do what I want with it, and then get used to the routine of importing it back from the ascii each time I run the gui app.
before I was really thinking about the tradeoffs, the idea of a Free software project putting the data in a hard-for-the-user-to-use format seemed a bit jarring; technically the database format is still useable, but getting database redundancy and failover is a tad bit more work and more fragile than committing a text file to git or mercurial.
Coming soon: KMyMoney 1.0 Posted Aug 28, 2009 16:23 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
I see the biggest is the data safety issue
when making a change (no matter how trivial) you would need to write a modified version of the entire file and do the appropriate fsyncs (for the file and the metadata)
with a database you just tell the database what changes you want done and it makes the changes (probably fare more efficiantly than you will, as well as being far less likely to have missed some corner case)
searching a xml text file involves reading the file, parsing it, then walking the data in memory (or reading the file, parsing it as you go, and aborting the parsing when you have the data you need)
databases have many tricks that they can use to speed the data access (indexes, binary data representation, etc)
as for locking the data in a 'hard for the user to use' format, I would say that putting it in a standard database that many different tools can readily access (in many cases making it look like a spreadsheet, but including dump/restore to/from flat text files) makes it pretty easy to get at the data.
also, keep in mind that one commonly used database for this is sqlite. that database engine keeps all the data in one file (just as easy to check into a version control system), and does not require communication to a seperate daemon. it basicly acts as a file manipulation library in your software, so it has very low overhead.
as for redundancy and failover, that can mean many things. if you want to have the data stored on a central box (rather than sitting on someone's laptop that can break, get lost/stolen, etc) databases can do failovers to backup boxes transparently to the user
for a single home user this (usually) does not matter, for a small business this can be critical.
In addition, with a database back-end it is relativly easy to allow multiple people to share access to the data at once. again, not a feature that a single home user normally cares about.
Coming soon: KMyMoney 1.0 Posted Aug 28, 2009 16:43 UTC (Fri) by jordanb (guest, #45668) [Link]
One thing to keep in mind about sqlite is that the 'database' is just a file on the file system, and you can use the 'sqlite3' tool to open up and poke around in any sqlite file.
Actually if you're trying to manipulate the data 'out of band', or make an external tool that acts on a program's data, then sqlite is a dream format since it gives you a very powerful structured data manipulation interface (the SQL language).
If you want to keep the data in version control, you can do a data dump from the database file into a text file and then check *that* into VCS:
echo ".dump" | sqlite3 grampsdb > my_gramps.sql
You can also do that, of course, and use your regex masher on that sql file, and then import it back into the database. ;)
Coming soon: KMyMoney 1.0 Posted Aug 28, 2009 16:48 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
one advantage to manipulating things in SQL rather than in a flat file is that you don't have to worry as much about the structural elements (field seperators, allowed characters, field lengths, etc) as the app can (and should) have this data managed by the database engine.
|
|||||||||||||