LWN: Comments on "CouchDB and SQLite creators introduce UnQL, a NoSQL query language (The H)"

Death To SQL And All Its Children

incase — Mon, 05 Sep 2011 10:22:04 +0000

Nice comment. But I wonder:
Is there any true open source implementation of NFNF (or non-1NF) databases?
I wasn't able to locate one.
Also: As NFNF databases are still structured dbs, how are they manipulated and queried?
Furthermore, I assume the following would be a valid structure of a non-1NF database.

table songs
   id int
   title string

table broadcasts
   id int
   songs list of int references table songs

If so: How do I query all broadcasts which played the song with id=1?
I don't see how your praised hierarchical search comes into play here.

Death To SQL And All Its Children

bronson — Thu, 11 Aug 2011 22:16:50 +0000

Come to think of it, if 1NF is first normal form, then everything less normal should be N1NF. Oh well. Naming things is the second hardest thing in programming.

Death To SQL And All Its Children

bronson — Thu, 11 Aug 2011 22:11:12 +0000

Non First Normal Form. Good call on looping and not recursing!

Death To SQL And All Its Children

ceswiedler — Thu, 11 Aug 2011 22:02:59 +0000

I googled "NFNF database" to try to understand what the hell you're talking about, and this article came up as the first hit. I'm still trying to exit that loop.

Death To SQL And All Its Children

Wol — Tue, 02 Aug 2011 10:35:49 +0000

> Relational algebra doesn't imply normalization, or is your point about something else?

But isn't normalisation all about coercing information into a format suitable for applying relational algebra? Or do you understand me as meaning converting the data into *first* normal form, which I most definitely DON'T mean! Maybe I've got it wrong, but as I understand it, if your data isn't normalised, then it's unstructured, and you can't apply relational algebra.

> I'd say you are confusing the crap engineering you've seen for relational algebra ;)

Relational algebra is maths - it has nothing to do with engineering ;)

If by "crap engineering" you mean C&D's 12 rules defining what a relational database is, I'd agree with you :-) A relational database, as implemented by C&D, is crap engineering.

That is my point. I can show, from an engineering standpoint, that a post-relational NFNF database can mimic a relational first normal form database and have near-as-dammit identical performance characteristics. I can show that any optimisation you can apply to an RDBMS, I could apply the same to NFNF. So far we're equal. But because NFNF is not constrained by C&D, I can apply further optimisations *at* *the* *engineering* *level* that means NFNF blows relational out of the water.

For example, I've thrown out this challenge repeatedly - "please *store* a list in an RDBMS". Yes it's easy to *model* a list in an RDBMS, but that's just shoved a load of complexity into the database that shouldn't be there! That's why NFNF is a far superior data store. Because it can store a list AS A LIST, it doesn't need to store "position in list" as data in a table (indeed, doing so is not only necessary with a first normal form database, but also extremely bad engineering because you are - from the application's point of view at least - mixing up data and metadata in the same table with no way of telling which is which, other than in the programmer's head!). Another advantage is that it stores the key *once* for the entire list, not once per element of the list! Those two factors mean an NFNF datastore probably needs half or less "real estate" on disk to store the information. Okay - that's irrelevant to theory, but from an engineering POV, reducing the need to read from disk makes a massive difference to the speed - and by being able to cache more data in RAM that difference is amplified!

Basically, as you can see, my point is that you CANNOT implement a relational database, as defined by C&D, without C&D enforcing a whole bunch of crap engineering decisions - if you don't implement those crap decisions then you are not C&D-compliant.

(By the way, I would always use relational theory to design my databases, but I would then put it in an NFNF database precisely so I could optimise it in ways forbidden by C&D.)

(Another aside - NFNF databases don't have query optimisers. A basic totally-unoptimised relational database of any size will have extremely inefficient query performance - basically just a brute-force search of the entire database. An NFNF database can do a hierarchy-search. As a result, if a relational database query engine wastes maybe 10% of its power doing an optimisation, it can make massive gains in efficiency. An NFNF database, on the other hand, only has about 3 to 5% headroom before it hits perfect efficiency, so to achieve any noticeable gain in query efficiency could easily cost 50 - 100% extra. Just not worth the effort!)

Cheers,
Wol

Death To SQL And All Its Children

servilio-ap — Tue, 02 Aug 2011 03:46:54 +0000

> And then use a database that is NOT first normal form.

Relational algebra doesn't imply normalization, or is your point about something else?

> Relational algebra is great maths. It's just that when it's implemented as a relational database it's crap engineering ... :-)

I'd say you are confusing the crap engineering you've seen for relational algebra ;)

Not sure if that's a good idea

knobunc — Tue, 02 Aug 2011 00:54:50 +0000

More recent SQL makes that problem more apparent.

SELECT a.id, b.id
FROM table_a a 
JOIN table_b b XXXX
WHERE a.some = 123
  AND b.some > 1435

The syntax error from the missing text at XXXX would catch the problem.

Not that I am a SQL apologist by any stretch.

UnQL, a NoSQL query language

nix — Mon, 01 Aug 2011 14:41:39 +0000

Quite. Lisp was merely a mathematical formalism (not even intended as a computer language per se) until Steve Russell implemented the first interpreter for it.

Not sure if that's a good idea

ehabkost — Mon, 01 Aug 2011 14:36:30 +0000

Yes, the existence of multiple languages makes the problem harder; but the problem still exists. I still miss a standard API to deal with SQL (but I have no hopes that there will be one) and I'm sure I'll miss a standard API to deal with UnQL in case it becomes popular.

Not sure if that's a good idea

nix — Mon, 01 Aug 2011 14:19:59 +0000

A standard API... for what language? If you define a C one, it'll be ugly and out-of-place for C++. If you define a C++ one, it'll be almost unusable from C and just try using it from Haskell.

(In any case, an API is needed regardless, but if the majority of what goes over it is in a different domain-specific language, i.e. SQL, at least that API can be simple.)

Not sure if that's a good idea

Cyberax — Mon, 01 Aug 2011 05:38:20 +0000

There are more subtle ways for it, you can join sub-trees using XQuery expressions and then use built-in function to create Cartesian products.

Not sure if that's a good idea

endecotp — Sun, 31 Jul 2011 15:08:54 +0000

Hmm, OK, I see that you can have multiple "for" clauses (e.g. the last example at http://cs.au.dk/~amoeller/XML/querying/flwrexp.html).

I would still rather have something that is more like raw relational algebra though. I always find it surprising that such a thing doesn't already exist in the mainstream. I guess the problem is that SQL is "good enough".

Not sure if that's a good idea

HelloWorld — Sun, 31 Jul 2011 14:18:30 +0000

> Only because the "Object Oriented" purists rule the programming language world.
As ehabkost already pointed out, this has nothing to do with the issue I mentioned.

Not sure if that's a good idea

Cyberax — Sun, 31 Jul 2011 13:37:42 +0000

Yes, it can do joins (even recursive ones) - it's trivial to prove that it's relationally complete.

In its present form FLWOR is not suited for relational databases, but I'm not claiming that we should just replace SQL with FLWOR.

However, it would be nice to see a language for database queries based on the same ideas.

Not sure if that's a good idea

endecotp — Sun, 31 Jul 2011 13:28:42 +0000

> Want an example of a GOOD query language? Here it is:
> http://en.wikipedia.org/wiki/FLWOR

It seems to be an XML thing, not relational. Can it do joins?

Not sure if that's a good idea

ehabkost — Sun, 31 Jul 2011 12:50:25 +0000

I think ORMs have nothing to do with the criticism from user HelloWorld: the problem appears when you are _not_ using an ORM and you have to "write code to build SQL queries" when you could just have a simple API.

There are libraries that help on that, such as SQLAlchemy; the point is that those libraries shouldn't have been necessary in the first place. Defining standard API would be much more valuable to me than defining a standard query language.

CouchDB and SQLite creators introduce UnQL, a NoSQL query language (The H)

rsidd — Sun, 31 Jul 2011 10:18:10 +0000

No joke. Larry Wall has himself called Perl "the first postmodern computer language."

Death To SQL And All Its Children

Cyberax — Sun, 31 Jul 2011 08:04:08 +0000

Well, as I've pointed earlier, http://en.wikipedia.org/wiki/FLWOR is quite nice. It seems to solve most of SQL's problems and remain a pure declarative language, suitable for deep optimizations.

Not sure if that's a good idea

euske — Sat, 30 Jul 2011 23:56:16 +0000

My point is that, to many programmers, anything that gets their job done without much effort (i.e. learning new stuff) is considered sensible. Many people probably understand that SQL is horrible, but that does not change their behavior until there's an easier solution. It's hard for many people to "unlearn" the way of doing things once they've accustomed themselves to it.

Death To SQL And All Its Children

Wol — Sat, 30 Jul 2011 21:36:27 +0000

And then use a database that is NOT first normal form.

Relational algebra is great maths. It's just that when it's implemented as a relational database it's crap engineering ... :-)

Cheers,
Wol

CouchDB and SQLite creators introduce UnQL, a NoSQL query language (The H)

Wol — Sat, 30 Jul 2011 21:32:59 +0000

"post-relational" aka Pick all use roughly the same query language, called originally ENGLISH or in its other variants RETRIEVE, PERFORM and probably more. It is very english-like hence the name, but bear in mind it is a *query* language only. It was meant to be an update language too, like SQL, but that fell by the wayside as being "too complicated". And it is complicated. Horribly.

(I understand SQL was originally SEQL, "Simple English Query Language", but they changed it when it became complicated and un-english-like :-)

Cheers,
Wol

Death To SQL And All Its Children

rstreeks — Sat, 30 Jul 2011 18:45:38 +0000

You hit the nail on the head by stating that SQL is not equal to Relational algebra. But scores of developers do think that this is the case. We real should teach them first relational algebra and then SQL. But I am sorry to say that is not the case.

CouchDB and SQLite creators introduce UnQL, a NoSQL query language (The H)

yootis — Sat, 30 Jul 2011 15:59:19 +0000

Hilarious. Made my day!

Not sure if that's a good idea

Cyberax — Sat, 30 Jul 2011 14:13:01 +0000

Actually, no. SQL is NOT a sensible language.

It's a relational query language, but it tries hard to hide its nature. For example, in a _true_ relational language such mistake should be immediately noticeable: "select a.id, b.id from table_a a, table_b b where a.some=123 and b.some>1435". It should be immediately obvious that a Cartesian product is being created (probably accidentally).

Also, SQL is way too verbose (especially treatment of NULLs), incomplete (nested grouping), real queries depend on vendor-specific features, CTEs are a poorly-designed bolt-on for tree-structured queries, etc. In short, SQL is horrible.

Want an example of a GOOD query language? Here it is: http://en.wikipedia.org/wiki/FLWOR

Ultimately, NoSQL movement may finally push the industry to design a better query language.

Not sure if that's a good idea

euske — Sat, 30 Jul 2011 11:51:23 +0000

SQL *is* a sensible language to those who don't understand (or don't bother to take a time to understand) quoting, string escaping, etc. and its consequences. This is not an isolated problem. It is a part of a much greater problem in software industry; new ways are accepted only when they're quick, easy and dirt cheap. Otherwise, it will never replace traditional (i.e. quick, dirty and insecure) methods.

Not sure if that's a good idea

HelloWorld — Sat, 30 Jul 2011 10:21:20 +0000

> Uhm. We have such APIs.
Yes, we have them, there are also SQLAlchemy and DBIx::Abstract etc.. But SQL injection attacks still happen, so as it seems many people don't know them. So the question is, why do we need a query language at all? Wouldn't it be more sensible to define just an API? They did it with the DOM after all, even across programming languages.

CouchDB and SQLite creators introduce UnQL, a NoSQL query language (The H)

flewellyn — Sat, 30 Jul 2011 06:36:36 +0000

You mean Perl?

Not sure if that's a good idea

Cyberax — Sat, 30 Jul 2011 06:17:35 +0000

Uhm. We have such APIs.

My favorite one is LINQ. My second favorite one is QueryDSL. They both allow to build queries like this:

>Query q = session.from(qSomeClass)
> .where(qSomeClass.firstname.eq("John"))
> .where(qSomeClass.lastName.eq("Doe"))
> .where(qSomeClass.someRelation.someField.in(1,2,3))
>;
>if (someCondition)
> q.where(qSomeClass.birthDate.lt(someDate));
>List<SomeClass> res=q.list();

So there are no places for SQL injection AND it's easy to dynamically construct queries.

Oh, and DBAs insisting on using pure SQL should be put against the wall second (after lawyers) come the revolution.

Death To SQL And All Its Children

ldo — Sat, 30 Jul 2011 04:54:26 +0000

Shame to see them slavishly copying the limitations of SQL, like not treating collections (in SQL, tables/relations/views) as first-class objects.

Why doesnt someone go back to relational algebra and implement a language that offers the full power of that?

UnQL, a NoSQL query language

pr1268 — Sat, 30 Jul 2011 03:53:23 +0000

Having read the title and article header only, my initial reaction is that this is some sort of strange esoteric or theoretical programming language "invented" to solve a specific set of problems.

On the one hand, my reaction sounds snide and unenlightened, but on the other hand, I kind of remember hearing that LISP was merely a "paper language" and then someone (a student of the paper's author, perhaps) went out and implemented the whole thing from scratch, or so the story goes. Maybe I shouldn't be so quick to judge...

Not sure if that's a good idea

sbergman27 — Sat, 30 Jul 2011 03:19:08 +0000

"A query language intuitively looks like a sensible thing, but SQL ought to have taught us a lesson: It's just a PITA to integrate a query language with a programming language in a sensible way."

Only because the "Object Oriented" purists rule the programming language world. SQL is, all these years later, still so powerful that it boggles the mind. I've spent way more time mucking with ORMs to get them to do what I want than I would have if I'd just built the SQL queries.

Just say "no" to ORMs. Or use the ORM for tasks that are truly trivial, and write code to build SQL queries if it takes you more than 5 minutes to figure out how to do it through the ORM. Set a definite time limit, and stick to it.

CouchDB and SQLite creators introduce UnQL, a NoSQL query language (The H)

tstover — Sat, 30 Jul 2011 02:34:22 +0000

My immediate reaction is that the query language is a large part of the problem nosql is trying to avoid. However, Hipp is a bad ass so I'll keep an open mind.

Not sure if that's a good idea

HelloWorld — Sat, 30 Jul 2011 00:16:10 +0000

A query language intuitively looks like a sensible thing, but SQL ought to have taught us a lesson: It's just a PITA to integrate a query language with a programming language in a sensible way. SQL injection vulnerabilities are a direct consequence of the fact that there is no proper API to create SQL queries in terms of data structures instead of strings. Prepared statements help, but they're not a complete solution. I don't think that this mistake should be repeated again.

CouchDB and SQLite creators introduce UnQL, a NoSQL query language (The H)

jcorgan — Sat, 30 Jul 2011 00:13:17 +0000

"...post-modern applications."

WTF? Next thing we'll hear about is deconstructionist compilers that output obscurantist object code.

CouchDB and SQLite creators introduce UnQL, a NoSQL query language (The H)

josh — Fri, 29 Jul 2011 23:22:02 +0000

Definitely not; a handful of non-relational databases use MapReduce, but definitely not all of them or even a majority of them. Others provide a simple key-value interface, graph query languages, custom languages, or custom API.

CouchDB and SQLite creators introduce UnQL, a NoSQL query language (The H)

SEJeff — Fri, 29 Jul 2011 22:55:39 +0000

Isn't the common "query language" for NoSQL K/V stores called Map / Reduce?