Interpreters and JITs

The other day I came across this interesting article about the improvements to the Android runtime. Summarizing, it will start to run an application via the interpreter. Then some parts will be JITted, injecting some profiling code, so that when the machine is idle and charging the hot sections of the code will be recompiled. This recompilation can be applied multiple times, and I think the compiled code is saved, not just kept in memory, so further executions already benefit from this.

Years ago I would think of a world split between Interpreters and JITs. It was reading about the HotSpot JVM when I found that both worlds could be mixed. You start by interpreting the code so that you don't have any initial delay because of compilation, and when it's detected that a method is run very often the JIT compiles it. The astonishing feature provided by the HotSpot JVM is that once a method has been compiled it can be further optimized based on runtime information and "hot swapped". Another interesting feature that I have just learnt is that it can also use OSR (on stack replacement). This means that a method that is being run only once, but for a long while (a big loop) will be detected as a Hot Spot and replaced even when it is running!!! Another cool feature is that as in some cases optimizations can have taken wrong decisions, a method can be replaced by a deoptimized version. You'll find this paragraph interesting:

Remember how HotSpot works. It starts by running your program with an interpreter. When it discovers that some method is "hot" -- that is, executed a lot, either because it is called a lot or because it contains loops that loop a lot -- it sends that method off to be compiled. After that one of two things will happen, either the next time the method is called the compiled version will be invoked (instead of the interpreted version) or the currently long running loop will be replaced, while still running, with the compiled method. The latter is known as "on stack replacement", or OSR.

The .Net runtime does not use an interpreter, just a JIT (and also de AOT compilation via ngen). I could be wrong, but this gives you the impression that it's less advanced than the JVM HotSpot. In .Net 4.6 a new, more performant, JIT (ryujit) is used. From what I've read it's a traditional JIT, neither interpretation nor hot swapping has been added to the mix. Bearing in mind that Ryujit has been on the works for quite a few years and that Microsoft has put a lot of effort on it, I assume they consider that hot swapping is not necessary for huge performance gains.

Reading about all this has woken up my interest on how mondern javascript engines work (Interpreter, JIT, both...).

• V8: (chrome, standard node.js) Rather than an interpreter and a JIT, it includes a fast, not optimized JIT to start with, and then hot methods are compiled via a slower, optimized JIT. Copy pasted from somewhere on the net:

  V8 never interprets, it always compiles. The first compiler is a very fast, very slim compiler that starts up very quick. The code it produces isn't very fast, though. This compiler also injects profiling code into the code it generates. The other compiler is slower and uses more memory, but produces much faster code, and it can use the profiling information collected by running the code compiled by the first compiler.

• Mozilla Spider Monkey: Mozilla's runtime has quite evolved over the years. It started as an interpreter, then they added a particular case of JIT, a tracing JIT (sequences of code are compiled rather than whole methods), and then they replaced it with a conventional per-method JIT (along with the interpreter).
• Microsoft's ChakraCore: From this overview it seems pretty advanced. It uses an interpreter for fast start-up and 2 multithreaded JITs, a fast one and an optimized one. It seems that apart from JITting methods it can also JIT specific loops. Of course the compiled code can be hot-swapped.

If you want to read more about Interpreters and JITs, this makes a good reading.