Tratt: Fast Enough VMs in Fast Enough Time

I think I'd argue that the thing that's innovative in RPython is not the tracing JIT compiler (you could extend LLVM to have a tracing JIT compiler/interpreter for the LLVM IR, after all - it's "just" a matter of engineering time), but the application of a tracing JIT compiler to the problem of making writing a JIT compiler easy.

The leap involved is immense - most programmers are capable of writing a simple interpreter for a language of their own design. Not as many programmers are capable of writing a compiler from their language to an IR. RPython says "if you can write your interpreter in this subset of Python, and add these annotations, you get a JIT for free." As it happens, they've chosen a subset of Python that's reasonably easy to live with, and the annotations are ones that should be natural to anyone writing an interpreter.

To give you an idea of how different this is, imagine writing a modern implementation of BBC BASIC, absent the commands beginning "*", and the CALL and USR facilities for leaving BASIC to call arbitrary machine code. You'd probably tokenize the language the way the BBC Micro did in the 80s, then directly interpret the tokenized "bytecode" (which has a one-to-one correspondence to the original source).

Now try and JIT this - to use LLVM, you need to rework your interpreter to translate to LLVM IR (which is a non-trivial task), then you check it works by using the LLVM interpreter to run it, then you add the JIT bits. In contrast, to use RPython, you rewrite your interpreter in RPython (trivial if it was already written in Python, not that hard if it was written in something else), and check it still works as an interpreter. You then annotate a few key points in your interpreter, and it's done; you have a JIT compiler.

Does that apply to tracing JIT more than JIT compilers generally?

This is problem with any JIT, but it's worse for tracing one.

Surely the main use for JIT compilers is situations when static compiling is not practical, like when the precise target platform is not known at the time when the code is circulated, or when the person running the code wants more control over it than they get from a precompiled binary.

The main use is for the languages which don't support static compilation.

Think web applications - do you really want to precompile your application for every CPU it might run on, and would people really want to run your precompiled binary locally without good reason to trust you?

Well, we all use JITs every time we run the program (our CPUs don't execute x86 code directly, it translates it to RISC-like µops on the fly) thus the question becomes: "do we need yet another JIT" and sometimes the answer is yes, but more often then not it's no and people use managed code because it has high buzzword-compliance value.

This code could not be generated at static compile time because the program input is not available then. Even if you can predict a static path at static compile time (doubtful) you still don't have the actual data values to take advantage of.

You have a point. P4 was purely tracing JIT and contemporary CPUs dropped that approach because it's hard to combine both in a single chip.

As P4 story shows pure tracing JIT is a disaster and will probably always be a disaster. But combination or regular JIT with tracing JIT may sometimes make sense as was already pointed out. The question is: how often will it make sense?

DEC's FX!32 and IBM's DAISY were similar projects.

And here you are missing the point again. DAISY, Dynamo, FX!32 and IA-32 EL - they all play with architectures which explicitly tried to switch from JIT implemented in hardware to "intelligent compiler". This idea flew like a lead balloon. In the vain hope to save it they tried software JIT - but it does not work as well.

They weren't tracing JITs but they could just as well have been.

I'm not so sure. The general problem of tracing JITs (which killed NetBurst, too) will probably be fatal for these projects.

Now, the use of tracing JIT as "superbooster" for rare case may be interesting, but these projects never went so far.

P.S. Actually architecture of contemporary CPUs make work of JIT creators very frustrating. It's easy to show real-world case where JIT wins over pure static compiler. But since contemporary CPUs always use JIT we don't have a "JIT vs compiler" competetion. We have "“JIT+compiler” vs “JIT+JIT”" competition - and this combination rarely makes sense. Especially when two stacked JITs are created by different teams which don't cooperate at all.