JIT Compilation and Adaptive Optimization

Learning Objectives

Explain why JITs exist; how interpreters profile hot functions and loops with invocation and back-edge counters; baseline versus optimising compilers; tiered, method-based and tracing JITs.

Describe JIT IRs, CFGs, SSA and data-flow analysis; type feedback and inline caches (monomorphic, polymorphic, megamorphic); inlining, constant folding and propagation, dead-code elimination, common-subexpression elimination, loop optimisations, escape analysis, scalar replacement and devirtualisation.

Explain speculation, guards, deoptimisation and on-stack replacement; code caches, compilation cost, warm-up, profiling overhead and optimisation cliffs; debugging optimised code; executable-memory security; JIT-less modes; benchmarking JIT runtimes correctly.

Plain-Language Explanation

An interpreter is flexible but slow. A JIT compiler watches the program run, notices which functions are busy and what kinds of values they receive, and compiles those functions to fast machine code that *assumes* things will stay the same — "these are always small integers". If an assumption ever turns out false, the runtime throws away the fast code and goes back to the interpreter (deoptimisation), later perhaps compiling again with better knowledge.

Precise Technical Explanation

Why JIT compilation exists Run-time information (actual types, hot paths, constant values, which classes are loaded) enables optimisations an AOT compiler cannot safely make for dynamic languages or open-world programs, while keeping start-up fast by compiling only hot code.

Interpreted execution and profiling The interpreter records how often functions run, which branches are taken and which types appear at each operation (V8's feedback vectors — the [0], [1] slots in Module 10's Ignition listing — HotSpot's MethodData objects).

Hot functions and hot loops Functions called often, or loops iterating often, are compilation candidates.

Invocation counters and back-edge counters Counters incremented on calls and on loop back-edges; crossing thresholds triggers compilation (HotSpot tier thresholds; V8 uses budgets of interpreted work).

Translates bytecode to machine code quickly with few optimisations, removing dispatch overhead (V8 Sparkplug, HotSpot C1 at tier 1–3, .NET tier 0).

Builds an SSA IR, uses feedback, optimises aggressively; slow to compile, fast code (HotSpot C2, V8 Maglev and TurboFan, .NET tier 1, Graal).

Combining tiers: interpret → baseline (with profiling) → optimised; .NET uses tier-0 code then recompiles hot methods at tier 1 (with on-stack replacement for loops).

Compiles whole methods, inlining hot callees (HotSpot, V8, .NET).

Records a hot loop's actual path as a linear trace with guards and compiles it (LuaJIT, PyPy); side exits handle paths not recorded.

Intermediate representations used by JITs Sea-of-nodes graphs (C2, TurboFan), CFG-based SSA (Maglev, Graal's graph IR is also sea-of-nodes), linear trace IRs (LuaJIT).

Basic blocks and edges, as in Module 10, built from bytecode.

SSA and data-flow analysis SSA makes def-use chains explicit; analyses compute ranges, types, nullness and liveness over the graph.

Observed types per operation: "the + at offset 2 has only seen small integers (Smis)".

Per-call-site caches remembering the last lookup result for a given object shape or receiver class, turning dynamic lookups into a shape check plus a direct access.

Monomorphic, polymorphic, and megamorphic call sites One shape seen (monomorphic — fastest), a few (polymorphic — a small dispatch), many (megamorphic — generic lookup, slow, and often not optimised).

Copying a callee's body into the caller: removes call overhead and exposes the callee to the caller's optimisations — the most important JIT optimisation.

Constant folding and propagation Computing with known constants and substituting them into uses.

Removing computations whose results are unused or branches proven impossible.

Common subexpression elimination Computing a repeated expression once (global value numbering).

Loop-invariant code motion, strength reduction, unrolling, range-check elimination, vectorisation.

Proving an object never escapes the method (not stored globally, not returned, not passed to unknown code).

Replacing non-escaping objects by their fields in registers — eliminating the allocation entirely.

Turning a virtual call into a direct (inlinable) call when the receiver's class is known or only one implementation is loaded (class-hierarchy analysis), with a guard or a dependency that triggers deoptimisation if a new subclass loads.

Compiling for the observed case only: "a and b are small integers".

Guards and assumptions Cheap checks (type, shape, overflow) at the start of speculative code; when one fails, execution leaves the optimised code.

Deoptimization and reconstructing interpreter state The runtime maps the optimised frame's machine state back to interpreter frames (bytecode offset, values of locals and stack) using metadata recorded at compile time, and resumes in the interpreter. The function may be reoptimised later with broader assumptions.

On-stack replacement (OSR) Switching a *running* function into compiled code in the middle of a long loop (entering at the loop header), and the reverse for deoptimisation.

Memory region holding compiled code; it can fill up (HotSpot ReservedCodeCacheSize), which stops further compilation.

JIT compilation cost CPU time and memory spent compiling; background compiler threads reduce its impact on latency.

Performance improves over the first seconds as code moves up the tiers; benchmarks must exclude or report this phase.

Counters and feedback cost time in lower tiers; that is why the top tier drops profiling.

Small code changes causing large slowdowns (a site turning megamorphic, a function becoming too large to inline, repeated deoptimisation leading the engine to give up optimising).

Debugging optimized code Inlining and register allocation hide variables and frames; debuggers rely on deoptimisation metadata or disable optimisation while debugging.

Security restrictions on executable memory W^X policies: memory is writable or executable, not both; JITs flip permissions or use dual mappings; some platforms forbid JIT entirely for third-party apps.

JITless modes and constrained environments V8's --jitless,.NET's interpreter mode on iOS-like platforms, HotSpot's -Xint: everything runs in the interpreter — slower but compatible with no-JIT policies.

Benchmarking JIT runtimes correctly Warm up, run many iterations, avoid dead-code elimination of the measured work (use results), watch for OSR and deoptimisation, report distributions, and use harnesses (JMH for Java, BenchmarkDotNet for.NET).

Example — optimisation concept A dynamic-language runtime may observe that add is called hot with numbers, optimise it under that assumption with a guard, and deoptimise when called with strings. A particular engine is not guaranteed to optimise this function or to use a specific tier — but V8 in Node.js 24.18 did exactly this in the real run below.

function add(a, b) {
  return a + b;
}

Why It Matters in Real Systems

JIT behaviour explains warm-up, latency spikes after deploys, benchmarks that lie, and why "type-stable" code is faster in JavaScript and Python implementations with JITs.

Key Terms

Recorded types per operation Per-site cache of lookup results by shape/class

Check protecting a speculative assumption Leaving optimised code back to the interpreter

Switching code tiers in the middle of a running loop Time until steady-state performance

Syntax or Instruction Format

No dedicated source syntax; this is a runtime or architecture concept. Inspection flags: Node --trace-opt --trace-deopt, HotSpot -XX:+PrintCompilation, .NET DOTNET_JitStdOutFile/DOTNET_JitDisasmSummary=1 (on recent runtimes).

Complete Example

V8 optimising and then deoptimising add (Node.js 24.18 / V8 with --trace-opt --trace-deopt; a launcher filters the trace to add and removes addresses — tiers, order and timing are engine-, version- and run-specific):

Output (one real run):

HotSpot's tiered compilation of a hot method and an on-stack replacement of a hot loop (Temurin JDK 21 with -XX:+PrintCompilation; a Python launcher keeps the lines for this program's own methods). Columns: timestamp (ms), compile id, % = OSR compilation, tier (3 = C1 with profiling, 4 = C2), method; "made not entrant" means superseded code is retired:

Output (one real run; ids and timestamps vary):

const { spawnSync } = require("node:child_process");
const { writeFileSync } = require("node:fs");

// the child prints its own results to stderr so they cannot interleave with V8's trace on stdout
writeFileSync("jit_child.js", `
function add(a, b) { return a + b; }
let s = 0;
for (let i = 0; i < 200000; i++) s = add(s, i) % 1000;   // hot, always small integers
console.error("after numbers:", s);
console.error("strings now:", add("a", "b"));           // violates the speculation
for (let i = 0; i < 200000; i++) s = add(s, i) % 1000;
console.error("done:", s);
`);
const run = spawnSync(process.execPath, ["--trace-opt", "--trace-deopt", "jit_child.js"], { encoding: "utf8" });
for (const line of run.stderr.trim().split(/\r?\n/)) console.log("program:", line);
for (const line of run.stdout.split(/\r?\n/)) {
  if (!line.includes("<JSFunction add ")) continue;
  let m;
  if ((m = line.match(/^\[marking .* for optimization to (\w+).*reason: ([^\]]+)\]/))) console.log(`add: marked for optimization → ${m[1]} (${m[2]})`);
  else if ((m = line.match(/^\[compiling method .*\(target (\w+)\)/))) console.log(`add: compiling with ${m[1]}`);
  else if ((m = line.match(/^\[completed compiling .*\(t
…

program: after numbers: 0
program: strings now: ab
program: done: 0
add: marked for optimization → MAGLEV (hot and stable)
add: compiling with MAGLEV
add: compiled (MAGLEV)
add: DEOPTIMIZED (deopt-eager, reason: not a Smi)
add: marked for optimization → MAGLEV (hot and stable)
add: compiling with MAGLEV
add: compiled (MAGLEV)

import os, subprocess, tempfile

JDK = r"D:\EngineerHub-tools\jdk21\bin"
src = """public class Hot {
    static int add(int a, int b) { return a + b; }
    public static void main(String[] args) {
        int s = 0;
        for (int i = 0; i < 2_000_000; i++) s = add(s, i) & 1023;
        System.out.println(s);
    }
}
"""
d = tempfile.mkdtemp()
open(os.path.join(d, "Hot.java"), "w").write(src)
subprocess.run([os.path.join(JDK, "javac.exe"), "Hot.java"], cwd=d, check=True)
out = subprocess.run([os.path.join(JDK, "java.exe"), "-XX:+PrintCompilation", "Hot"], cwd=d, capture_output=True, text=True).stdout
for line in out.splitlines():
    if "Hot::" in line or line.strip().isdigit():
        print(line.rstrip())

50   71       3       Hot::add (4 bytes)
50   72       4       Hot::add (4 bytes)
50   71       3       Hot::add (4 bytes)   made not entrant
51   73 %     3       Hot::main @ 4 (34 bytes)
51   74       3       Hot::main (34 bytes)
51   75 %     4       Hot::main @ 4 (34 bytes)
53   73 %     3       Hot::main @ 4 (34 bytes)   made not entrant
54   75 %     4       Hot::main @ 4 (34 bytes)   made not entrant
448

Line-by-Line Explanation

V8: after enough calls with small integers ("hot and stable"), add is compiled by Maglev, V8's mid-tier optimising compiler. Calling it with strings fails the "Smi" (small integer) guard: the trace reports a deoptimisation with reason "not a Smi", and execution continues in the interpreter (which handles strings correctly — the program prints ab). Later, add is reoptimised with feedback that now includes strings.

HotSpot: Hot::add is compiled at tier 3 (C1 with profiling), then tier 4 (C2); the tier-3 version is "made not entrant". Hot::main @ 4 with % is an OSR compilation entering at bytecode index 4, the loop header — main runs only once, so OSR is the only way its loop gets compiled.

Expected Output

The program's own output is deterministic (after numbers: 0, strings now: ab, done: 0; HotSpot prints 448); the JIT events, their order and timings vary by run and version.

Step-by-Step Runtime Behavior

Interpreter runs add → feedback: Smi + Smi → budget exhausted → mark for optimisation → background compile → swap in optimised code → guard fails on strings → deoptimise: reconstruct the interpreter frame at the bytecode offset of + → generic addition in the interpreter → later reoptimise.

Common Misconceptions

"JIT-compiled code is always correct for all inputs." It is correct because guards and deoptimisation handle the inputs it did not specialise for.

"Once optimised, always optimised." Deoptimisation and reoptimisation happen throughout a program's life.

"Microbenchmarks measure steady state." Without warm-up they measure the interpreter and the compiler.

Common Bugs and Failure Modes

Polymorphic or megamorphic hot sites, functions that change object shapes, deoptimisation loops, benchmarks optimised away, code cache exhaustion, performance cliffs after small refactors.

Corrected or Safer Patterns

Keep hot code type-stable, initialise objects with the same properties in the same order, use benchmark harnesses, monitor deoptimisation and compilation logs in performance investigations.

Debugging and Inspection Techniques

--trace-opt, --trace-deopt, --trace-ic (V8); -XX:+PrintCompilation, -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining, JITWatch (HotSpot); DOTNET_TieredCompilation=0 to compare; perf/async-profiler with JIT symbol maps.

Performance Considerations

Compilation competes for CPU at start-up; tiered compilation and background threads amortise it; AOT or snapshots remove it for start-up-critical workloads.

Security Considerations

JIT compilers have been a major source of browser vulnerabilities (type-confusion bugs in speculative optimisations); hardened environments disable JITs or isolate them; W^X and control-flow integrity limit damage.

Portability and Version Caveats

V8's tiers (Ignition, Sparkplug, Maglev, TurboFan), HotSpot's tiers and .NET's tiering change between versions; flags and log formats are diagnostic interfaces, not stable APIs.

Real-World Use Cases

Long-running servers on the JVM and .NET, browsers, Node.js services, PyPy for long-running Python workloads, LuaJIT in games and networking.

Best Practices

Write type-stable hot paths, measure after warm-up, read the runtime's own logs before guessing, and consider AOT where start-up matters more than peak.

Exercises

E1 · Beginner. Why did V8 deoptimise add?

Answer: The optimised code assumed small-integer arguments (guarded); calling it with strings failed the guard ("not a Smi"), so V8 discarded that assumption and returned to the interpreter.

E2 · Intermediate. What does % mean in HotSpot's PrintCompilation output, and why was it needed for main?

Answer: An on-stack-replacement compilation. main is called once, so a normal method compilation would never be used; OSR compiles the hot loop and transfers the running frame into it at the loop header.

E3 · Advanced challenge. How does escape analysis remove the allocation in int len(int x, int y) { Point p = new Point(x, y); return p.x * p.x + p.y * p.y; }?

Answer: p never escapes len (not stored, returned or passed to unknown code), so the JIT replaces p.x and p.y by the scalar values x and y (scalar replacement) and deletes the allocation.

Quiz

? What happens when a speculative guard fails in optimised code?
+ The runtime deoptimises: rebuilds interpreter state and continues in the interpreter. :: Correctness is preserved.
- The program crashes. :: Guards exist to avoid that.
- The wrong result is returned. :: Guards prevent that.
- The JIT recompiles before continuing. :: It continues in the interpreter first.

? What is a megamorphic call site?
+ A site that has seen many different receiver shapes or classes. :: Inline caches give up; lookups are generic.
- A site called very often. :: That is "hot".
- A recursive call. :: Unrelated.
- A call to a native function. :: Unrelated.

? Which are JIT optimisations? (select all that apply)
+ Inlining :: The most important one.
+ Escape analysis with scalar replacement :: Removes allocations.
- Lexing :: Front-end work.
+ Devirtualisation guarded by class-hierarchy assumptions :: Enables inlining of virtual calls.

Summary and Key Takeaways

JITs profile in lower tiers, compile hot code speculatively with guards, and deoptimise when assumptions fail; inlining, escape analysis and devirtualisation are their biggest wins. Warm-up, cliffs and deoptimisation make performance a property of a running system — measure it with the runtime's own tools.