User: Password:
Subscribe / Log in / New account

Data Warehousing 101

Data Warehousing 101

Posted Jul 21, 2011 17:37 UTC (Thu) by jeremiah (subscriber, #1221)
Parent article: Data Warehousing 101

Nice article. Now if I could find some kind of DB that's good at storing deconstructed XML or JSON with version information I'd be a happy camper. I'm always drawn to map-reduce, but always find myself using going back to RDBMS w/ recursive queries.

I've got a situation where every attribute and element is version-ed and stored separately. Everything is treated as a node, and the relationships between nodes are defined by the query. Needless to say, it's a little trippy. Or to say it another way, we can store to different directed graphs, and define the structure of a third, and have the DB/code fill in the values of the 3rd, using the first two.

Anyone know of some kind of db architecture that would be inherently good at this?

(Log in to post comments)

Data Warehousing 101

Posted Jul 21, 2011 19:41 UTC (Thu) by jberkus (subscriber, #55561) [Link]


If only there were! There's a few different implementations of graph databases, of which Neo4J is probably the leader, but every one I've seen is hard to use and has some substantial scalability limitations.

I think Postgres & WITH RECURSIVE, plus pl/perl or pl/python stored procedures for the decomposition may be your current best bet.

Data Warehousing 101

Posted Jul 21, 2011 20:28 UTC (Thu) by jeremiah (subscriber, #1221) [Link]

Except for the languages listed, that's what I've got. I found Neo4J difficult because of the lack of flexibility in defining relationships, ie they couldn't be dynamic. At times I even toy with the idea of doing something on my own at the file system level, but it's just not worth the effort, esp trying to redo a lot of the features that the Postgres folks have worked so hard over the years to develop. And I'm sure I could never approach their level of scalability on my own. Thanks for the input.

Data Warehousing 101

Posted Aug 4, 2011 0:12 UTC (Thu) by Lennie (guest, #49641) [Link]

Copyright © 2018, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds