Kuhn: Copyleft Won't Solve All Problems, Just Some of Them

Posted Mar 21, 2022 11:20 UTC (Mon) by Wol (subscriber, #4433)
In reply to: Kuhn: Copyleft Won't Solve All Problems, Just Some of Them by Wol
Parent article: Kuhn: Copyleft Won't Solve All Problems, Just Some of Them

I forgot. The other massive advantage going for Pick, is that the abstraction maps to the underlying hardware, enabling you to run scientific experiments.

Because relational explicitly forbids you from knowing the actual implementation, it becomes pure maths with no way to run experiments and specify real-world requirements.

Cheers,
Wol

Kuhn: Copyleft Won't Solve All Problems, Just Some of Them

Posted Mar 21, 2022 15:43 UTC (Mon) by nix (subscriber, #2304) [Link] (1 responses)

This is nonsense. Of *course* you *can* know specific underlying implementations -- and even if the source is unavailable you can run experiments to determine what the RDBMS is doing. What RDBMSes allow is for the implementations to improve things over time without requiring you to rewrite everything.

This is exactly the same as the potential speedup you can get from writing something in a language with a runtime smart enough to do JIT-compiling versus one like C where that stuff isn't provided (unless you implement it by writing such a language implementation in C). Determining the performance of Java code, or Lisp code, or JavaScript code isn't suddenly impossible, and reasoning about it isn't "pure maths with no way to run experiments". It means that running experiments (or reasoning about the JITter) becomes *necessary* to make such judgements. This is probably harder than reasoning about the performance of something written in C -- but the latter's performance isn't going to respond to the data being processed and improve over time, no matter what you do.

Kuhn: Copyleft Won't Solve All Problems, Just Some of Them

Posted Mar 21, 2022 20:46 UTC (Mon) by Wol (subscriber, #4433) [Link]

> This is probably harder than reasoning about the performance of something written in C -- but the latter's performance isn't going to respond to the data being processed and improve over time, no matter what you do.

You're missing the elephant in the room. Can you improve the performance of a Huffman Coder written in C? Not speed, but compression? Can you improve the performance of a fridge that is already capable of reaching 1K? If your rocket is already doing 0.9c, can you make it go any faster?

Instead of c, the speed of light, the *maximum* *possible* speed attainable under our understanding of Physics, let's define I - the speed of information - the maximum speed possible for a database to transfer information from OS to application.

As I'm sure you'll admit, a completely naive unoptimised FNF database is hopelesly inefficient, and that hopelessness grows with the size of the database. That's part of your frustration with "why can't the optimiser do any better?". Running at pedestrian, Newtonian speeds, there is absolutely oodles of room for improvement with an optimiser.

But that's why I did that Gedanken experiment. Go back and look at it. Pick achieves a near-naive speed of 0.95I ! Yep, as programmer I need to pre-define the indexes to assist my queries, but that's not hard. (Define indices for your relational database - I won't object :-)

There's just no point having a Pick optimiser - there is no headroom whatsoever to make it worth it. You said we can't do complex queries - well if you want to do it in SQL, I could run my Pick query, retrieve the *exact* dataset you're looking for, pass it through my SQL layer, and sit back and relax before your SQL Optimiser has even worked out where to start.

Please, DON'T compare Pick to other databases. Compare it to Information Theory. You can't cool a fridge below 0K. You can't accelerate a rocket beyond c. You can't accelerate a database beyond I.

Cheers,
Wol