CouchDB and SQLite creators introduce UnQL, a NoSQL query language (The H) [LWN.net]

CouchDB and SQLite creators introduce UnQL, a NoSQL query language (The H)

Posted Jul 29, 2011 22:55 UTC (Fri) by SEJeff (guest, #51588) [Link] (2 responses)

Isn't the common "query language" for NoSQL K/V stores called Map / Reduce?

CouchDB and SQLite creators introduce UnQL, a NoSQL query language (The H)

Posted Jul 29, 2011 23:22 UTC (Fri) by josh (subscriber, #17465) [Link] (1 responses)

Definitely not; a handful of non-relational databases use MapReduce, but definitely not all of them or even a majority of them. Others provide a simple key-value interface, graph query languages, custom languages, or custom API.

CouchDB and SQLite creators introduce UnQL, a NoSQL query language (The H)

Posted Jul 30, 2011 21:32 UTC (Sat) by Wol (subscriber, #4433) [Link]

"post-relational" aka Pick all use roughly the same query language, called originally ENGLISH or in its other variants RETRIEVE, PERFORM and probably more. It is very english-like hence the name, but bear in mind it is a *query* language only. It was meant to be an update language too, like SQL, but that fell by the wayside as being "too complicated". And it is complicated. Horribly.

(I understand SQL was originally SEQL, "Simple English Query Language", but they changed it when it became complicated and un-english-like :-)

Cheers,
Wol

CouchDB and SQLite creators introduce UnQL, a NoSQL query language (The H)

Posted Jul 30, 2011 0:13 UTC (Sat) by jcorgan (subscriber, #47213) [Link] (3 responses)

"...post-modern applications."

WTF? Next thing we'll hear about is deconstructionist compilers that output obscurantist object code.

CouchDB and SQLite creators introduce UnQL, a NoSQL query language (The H)

Posted Jul 30, 2011 6:36 UTC (Sat) by flewellyn (subscriber, #5047) [Link] (2 responses)

You mean Perl?

CouchDB and SQLite creators introduce UnQL, a NoSQL query language (The H)

Posted Jul 30, 2011 15:59 UTC (Sat) by yootis (subscriber, #4762) [Link] (1 responses)

Hilarious. Made my day!

CouchDB and SQLite creators introduce UnQL, a NoSQL query language (The H)

Posted Jul 31, 2011 10:18 UTC (Sun) by rsidd (subscriber, #2582) [Link]

No joke. Larry Wall has himself called Perl "the first postmodern computer language."

Not sure if that's a good idea

Posted Jul 30, 2011 0:16 UTC (Sat) by HelloWorld (guest, #56129) [Link] (15 responses)

A query language intuitively looks like a sensible thing, but SQL ought to have taught us a lesson: It's just a PITA to integrate a query language with a programming language in a sensible way. SQL injection vulnerabilities are a direct consequence of the fact that there is no proper API to create SQL queries in terms of data structures instead of strings. Prepared statements help, but they're not a complete solution. I don't think that this mistake should be repeated again.

Not sure if that's a good idea

Posted Jul 30, 2011 3:19 UTC (Sat) by sbergman27 (guest, #10767) [Link] (4 responses)

"A query language intuitively looks like a sensible thing, but SQL ought to have taught us a lesson: It's just a PITA to integrate a query language with a programming language in a sensible way."

Only because the "Object Oriented" purists rule the programming language world. SQL is, all these years later, still so powerful that it boggles the mind. I've spent way more time mucking with ORMs to get them to do what I want than I would have if I'd just built the SQL queries.

Just say "no" to ORMs. Or use the ORM for tasks that are truly trivial, and write code to build SQL queries if it takes you more than 5 minutes to figure out how to do it through the ORM. Set a definite time limit, and stick to it.

Not sure if that's a good idea

Posted Jul 31, 2011 12:50 UTC (Sun) by ehabkost (subscriber, #46058) [Link] (2 responses)

I think ORMs have nothing to do with the criticism from user HelloWorld: the problem appears when you are _not_ using an ORM and you have to "write code to build SQL queries" when you could just have a simple API.

There are libraries that help on that, such as SQLAlchemy; the point is that those libraries shouldn't have been necessary in the first place. Defining standard API would be much more valuable to me than defining a standard query language.

Not sure if that's a good idea

Posted Aug 1, 2011 14:19 UTC (Mon) by nix (subscriber, #2304) [Link] (1 responses)

A standard API... for what language? If you define a C one, it'll be ugly and out-of-place for C++. If you define a C++ one, it'll be almost unusable from C and just try using it from Haskell.

(In any case, an API is needed regardless, but if the majority of what goes over it is in a different domain-specific language, i.e. SQL, at least that API can be simple.)

Not sure if that's a good idea

Posted Aug 1, 2011 14:36 UTC (Mon) by ehabkost (subscriber, #46058) [Link]

Yes, the existence of multiple languages makes the problem harder; but the problem still exists. I still miss a standard API to deal with SQL (but I have no hopes that there will be one) and I'm sure I'll miss a standard API to deal with UnQL in case it becomes popular.

Not sure if that's a good idea

Posted Jul 31, 2011 14:18 UTC (Sun) by HelloWorld (guest, #56129) [Link]

> Only because the "Object Oriented" purists rule the programming language world.
As ehabkost already pointed out, this has nothing to do with the issue I mentioned.

Not sure if that's a good idea

Posted Jul 30, 2011 6:17 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link] (9 responses)

Uhm. We have such APIs.

My favorite one is LINQ. My second favorite one is QueryDSL. They both allow to build queries like this:

>Query q = session.from(qSomeClass)
> .where(qSomeClass.firstname.eq("John"))
> .where(qSomeClass.lastName.eq("Doe"))
> .where(qSomeClass.someRelation.someField.in(1,2,3))
>;
>if (someCondition)
> q.where(qSomeClass.birthDate.lt(someDate));
>List<SomeClass> res=q.list();

So there are no places for SQL injection AND it's easy to dynamically construct queries.

Oh, and DBAs insisting on using pure SQL should be put against the wall second (after lawyers) come the revolution.

Not sure if that's a good idea

Posted Jul 30, 2011 10:21 UTC (Sat) by HelloWorld (guest, #56129) [Link] (8 responses)

> Uhm. We have such APIs.
Yes, we have them, there are also SQLAlchemy and DBIx::Abstract etc.. But SQL injection attacks still happen, so as it seems many people don't know them. So the question is, why do we need a query language at all? Wouldn't it be more sensible to define just an API? They did it with the DOM after all, even across programming languages.

Not sure if that's a good idea

Posted Jul 30, 2011 11:51 UTC (Sat) by euske (guest, #9300) [Link] (7 responses)

SQL *is* a sensible language to those who don't understand (or don't bother to take a time to understand) quoting, string escaping, etc. and its consequences. This is not an isolated problem. It is a part of a much greater problem in software industry; new ways are accepted only when they're quick, easy and dirt cheap. Otherwise, it will never replace traditional (i.e. quick, dirty and insecure) methods.

Not sure if that's a good idea

Posted Jul 30, 2011 14:13 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link] (6 responses)

Actually, no. SQL is NOT a sensible language.

It's a relational query language, but it tries hard to hide its nature. For example, in a _true_ relational language such mistake should be immediately noticeable: "select a.id, b.id from table_a a, table_b b where a.some=123 and b.some>1435". It should be immediately obvious that a Cartesian product is being created (probably accidentally).

Also, SQL is way too verbose (especially treatment of NULLs), incomplete (nested grouping), real queries depend on vendor-specific features, CTEs are a poorly-designed bolt-on for tree-structured queries, etc. In short, SQL is horrible.

Want an example of a GOOD query language? Here it is: http://en.wikipedia.org/wiki/FLWOR

Ultimately, NoSQL movement may finally push the industry to design a better query language.

Not sure if that's a good idea

Posted Jul 30, 2011 23:56 UTC (Sat) by euske (guest, #9300) [Link]

My point is that, to many programmers, anything that gets their job done without much effort (i.e. learning new stuff) is considered sensible. Many people probably understand that SQL is horrible, but that does not change their behavior until there's an easier solution. It's hard for many people to "unlearn" the way of doing things once they've accustomed themselves to it.

Not sure if that's a good idea

Posted Jul 31, 2011 13:28 UTC (Sun) by endecotp (guest, #36428) [Link] (3 responses)

> Want an example of a GOOD query language? Here it is:
> http://en.wikipedia.org/wiki/FLWOR

It seems to be an XML thing, not relational. Can it do joins?

Not sure if that's a good idea

Posted Jul 31, 2011 13:37 UTC (Sun) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

Yes, it can do joins (even recursive ones) - it's trivial to prove that it's relationally complete.

In its present form FLWOR is not suited for relational databases, but I'm not claiming that we should just replace SQL with FLWOR.

However, it would be nice to see a language for database queries based on the same ideas.

Not sure if that's a good idea

Posted Jul 31, 2011 15:08 UTC (Sun) by endecotp (guest, #36428) [Link] (1 responses)

Hmm, OK, I see that you can have multiple "for" clauses (e.g. the last example at http://cs.au.dk/~amoeller/XML/querying/flwrexp.html).

I would still rather have something that is more like raw relational algebra though. I always find it surprising that such a thing doesn't already exist in the mainstream. I guess the problem is that SQL is "good enough".

Not sure if that's a good idea

Posted Aug 1, 2011 5:38 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]

There are more subtle ways for it, you can join sub-trees using XQuery expressions and then use built-in function to create Cartesian products.

Not sure if that's a good idea

Posted Aug 2, 2011 0:54 UTC (Tue) by knobunc (guest, #4678) [Link]

More recent SQL makes that problem more apparent.

SELECT a.id, b.id
FROM table_a a 
JOIN table_b b XXXX
WHERE a.some = 123
  AND b.some > 1435

The syntax error from the missing text at XXXX would catch the problem.

Not that I am a SQL apologist by any stretch.

CouchDB and SQLite creators introduce UnQL, a NoSQL query language (The H)

Posted Jul 30, 2011 2:34 UTC (Sat) by tstover (guest, #56283) [Link]

My immediate reaction is that the query language is a large part of the problem nosql is trying to avoid. However, Hipp is a bad ass so I'll keep an open mind.

UnQL, a NoSQL query language

Posted Jul 30, 2011 3:53 UTC (Sat) by pr1268 (guest, #24648) [Link] (1 responses)

Having read the title and article header only, my initial reaction is that this is some sort of strange esoteric or theoretical programming language "invented" to solve a specific set of problems.

On the one hand, my reaction sounds snide and unenlightened, but on the other hand, I kind of remember hearing that LISP was merely a "paper language" and then someone (a student of the paper's author, perhaps) went out and implemented the whole thing from scratch, or so the story goes. Maybe I shouldn't be so quick to judge...

UnQL, a NoSQL query language

Posted Aug 1, 2011 14:41 UTC (Mon) by nix (subscriber, #2304) [Link]

Quite. Lisp was merely a mathematical formalism (not even intended as a computer language per se) until Steve Russell implemented the first interpreter for it.

Death To SQL And All Its Children

Posted Jul 30, 2011 4:54 UTC (Sat) by ldo (guest, #40946) [Link] (9 responses)

Shame to see them slavishly copying the limitations of SQL, like not treating collections (in SQL, tables/relations/views) as first-class objects.

Why doesnt someone go back to relational algebra and implement a language that offers the full power of that?

Death To SQL And All Its Children

Posted Jul 30, 2011 18:45 UTC (Sat) by rstreeks (subscriber, #1018) [Link] (7 responses)

You hit the nail on the head by stating that SQL is not equal to Relational algebra. But scores of developers do think that this is the case. We real should teach them first relational algebra and then SQL. But I am sorry to say that is not the case.

Death To SQL And All Its Children

Posted Jul 30, 2011 21:36 UTC (Sat) by Wol (subscriber, #4433) [Link] (6 responses)

And then use a database that is NOT first normal form.

Relational algebra is great maths. It's just that when it's implemented as a relational database it's crap engineering ... :-)

Cheers,
Wol

Death To SQL And All Its Children

Posted Aug 2, 2011 3:46 UTC (Tue) by servilio-ap (subscriber, #56287) [Link] (5 responses)

> And then use a database that is NOT first normal form.

Relational algebra doesn't imply normalization, or is your point about something else?

> Relational algebra is great maths. It's just that when it's implemented as a relational database it's crap engineering ... :-)

I'd say you are confusing the crap engineering you've seen for relational algebra ;)

Death To SQL And All Its Children

Posted Aug 2, 2011 10:35 UTC (Tue) by Wol (subscriber, #4433) [Link] (4 responses)

> Relational algebra doesn't imply normalization, or is your point about something else?

But isn't normalisation all about coercing information into a format suitable for applying relational algebra? Or do you understand me as meaning converting the data into *first* normal form, which I most definitely DON'T mean! Maybe I've got it wrong, but as I understand it, if your data isn't normalised, then it's unstructured, and you can't apply relational algebra.

> I'd say you are confusing the crap engineering you've seen for relational algebra ;)

Relational algebra is maths - it has nothing to do with engineering ;)

If by "crap engineering" you mean C&D's 12 rules defining what a relational database is, I'd agree with you :-) A relational database, as implemented by C&D, is crap engineering.

That is my point. I can show, from an engineering standpoint, that a post-relational NFNF database can mimic a relational first normal form database and have near-as-dammit identical performance characteristics. I can show that any optimisation you can apply to an RDBMS, I could apply the same to NFNF. So far we're equal. But because NFNF is not constrained by C&D, I can apply further optimisations *at* *the* *engineering* *level* that means NFNF blows relational out of the water.

For example, I've thrown out this challenge repeatedly - "please *store* a list in an RDBMS". Yes it's easy to *model* a list in an RDBMS, but that's just shoved a load of complexity into the database that shouldn't be there! That's why NFNF is a far superior data store. Because it can store a list AS A LIST, it doesn't need to store "position in list" as data in a table (indeed, doing so is not only necessary with a first normal form database, but also extremely bad engineering because you are - from the application's point of view at least - mixing up data and metadata in the same table with no way of telling which is which, other than in the programmer's head!). Another advantage is that it stores the key *once* for the entire list, not once per element of the list! Those two factors mean an NFNF datastore probably needs half or less "real estate" on disk to store the information. Okay - that's irrelevant to theory, but from an engineering POV, reducing the need to read from disk makes a massive difference to the speed - and by being able to cache more data in RAM that difference is amplified!

Basically, as you can see, my point is that you CANNOT implement a relational database, as defined by C&D, without C&D enforcing a whole bunch of crap engineering decisions - if you don't implement those crap decisions then you are not C&D-compliant.

(By the way, I would always use relational theory to design my databases, but I would then put it in an NFNF database precisely so I could optimise it in ways forbidden by C&D.)

(Another aside - NFNF databases don't have query optimisers. A basic totally-unoptimised relational database of any size will have extremely inefficient query performance - basically just a brute-force search of the entire database. An NFNF database can do a hierarchy-search. As a result, if a relational database query engine wastes maybe 10% of its power doing an optimisation, it can make massive gains in efficiency. An NFNF database, on the other hand, only has about 3 to 5% headroom before it hits perfect efficiency, so to achieve any noticeable gain in query efficiency could easily cost 50 - 100% extra. Just not worth the effort!)

Cheers,
Wol

Death To SQL And All Its Children

Posted Aug 11, 2011 22:02 UTC (Thu) by ceswiedler (guest, #24638) [Link] (2 responses)

I googled "NFNF database" to try to understand what the hell you're talking about, and this article came up as the first hit. I'm still trying to exit that loop.

Death To SQL And All Its Children

Posted Aug 11, 2011 22:11 UTC (Thu) by bronson (subscriber, #4806) [Link] (1 responses)

Non First Normal Form. Good call on looping and not recursing!

Death To SQL And All Its Children

Posted Aug 11, 2011 22:16 UTC (Thu) by bronson (subscriber, #4806) [Link]

Come to think of it, if 1NF is first normal form, then everything less normal should be N1NF. Oh well. Naming things is the second hardest thing in programming.

Death To SQL And All Its Children

Posted Sep 5, 2011 10:22 UTC (Mon) by incase (guest, #37115) [Link]

Nice comment. But I wonder:
Is there any true open source implementation of NFNF (or non-1NF) databases?
I wasn't able to locate one.
Also: As NFNF databases are still structured dbs, how are they manipulated and queried?
Furthermore, I assume the following would be a valid structure of a non-1NF database.

table songs
   id int
   title string

table broadcasts
   id int
   songs list of int references table songs

If so: How do I query all broadcasts which played the song with id=1?
I don't see how your praised hierarchical search comes into play here.

Death To SQL And All Its Children

Posted Jul 31, 2011 8:04 UTC (Sun) by Cyberax (✭ supporter ✭, #52523) [Link]

Well, as I've pointed earlier, http://en.wikipedia.org/wiki/FLWOR is quite nice. It seems to solve most of SQL's problems and remain a pure declarative language, suitable for deep optimizations.