If you’ve ever wondered why a Java application feels sluggish for its first few seconds and then suddenly speeds up, you’ve experienced just-in-time (JIT) compilation firsthand. JIT compilation is the piece of the Java Virtual Machine (JVM) that decides, while your program is running, which parts of the code are worth turning into fast native machine code — and it’s one of the main reasons Java has been able to compete with compiled languages on raw performance for over two decades.
Explore what a JIT compiler is, how it differs from ahead-of-time (AOT) compilation and pure interpretation, and how Azul’s Falcon JIT compiler builds on the standard approach to close some of the gaps that show up in modern, latency-sensitive, cloud-native workloads.
What Is JIT Compilation?
“JIT” stands for just-in-time. A JIT compiler translates code into native machine instructions while the program is executing, rather than before it starts (ahead-of-time) or one instruction at a time as it runs (interpretation). The JVM ships Java bytecode — a platform-independent intermediate format produced by javac — and it’s the JVM’s job to turn that bytecode into instructions the underlying CPU can actually run.
A pure interpreter would read and execute that bytecode line by line, every time, forever. That’s simple and starts instantly, but it’s slow for anything that runs a loop or a hot method millions of times. A JIT compiler instead watches the program as it runs, identifies the “hot” code paths — the methods and loops that execute often enough to matter — and compiles just those paths into optimized native code. The rest of the program can stay interpreted, since compiling code nobody calls repeatedly would waste time and memory for no benefit.
How Does a JIT Compiler Work?
Most production JVMs, including OpenJDK’s HotSpot, use a tiered approach:
- Interpretation — the method runs as interpreted bytecode the first few times it’s called, while the JVM counts invocations.
- Profiling — the JVM tracks which methods and loops are “hot” (called frequently) and collects type and branch information along the way.
- Compilation — once a method crosses an invocation threshold, a background compiler thread compiles it to native machine code, often applying speculative optimizations based on the profile data collected (e.g., assuming a branch always goes one way, or a call site always resolves to one implementation).
- Re-optimization — if a speculative assumption turns out to be wrong (a branch that was always true suddenly isn’t), the JVM can de-optimize back to interpreted code and recompile with better information.
This is why JIT-compiled programs often get faster the longer they run: the JVM keeps learning about actual runtime behavior and keeps recompiling hot paths with better information, something a compiler working ahead of time simply can’t do because it has no runtime data to work from.
Tiered compilation: C1, C2, and the compilation levels in between
HotSpot doesn’t jump straight from “interpreted” to “fully optimized.” It uses several intermediate compilation levels, commonly described as tiered compilation:
- Level 0 — pure interpretation, with basic invocation counters running.
- Level 1 (C1, no profiling) — a quick, lightly optimized compile used for methods that appear to run briefly and don’t need deep optimization.
- Level 2–3 (C1, with profiling) — a fast client compiler (C1) produces a reasonably optimized version while continuing to collect richer profile data (branch frequencies, actual argument types at call sites, etc.).
- Level 4 (C2, fully optimized) — the server compiler (C2) uses the accumulated profile to apply aggressive optimizations: inlining hot call chains into a single compiled unit, eliminating redundant null checks and bounds checks, unrolling loops, and allocating objects on the stack instead of the heap when it can prove they never escape the method.
The point of staggering compilation this way is to avoid two failure modes at once: compiling everything immediately with the expensive C2 optimizer would burn CPU and delay startup, while never escalating past a quick C1 compile would leave real performance on the table for code that runs millions of times. Tiered compilation lets short-lived methods get a “good enough” compile cheaply while long-running hot methods eventually earn the full C2 treatment.
Inlining: the optimization behind most of the others
Method inlining — replacing a call to a small method with the method’s actual body — is arguably the single most consequential JIT optimization, because it’s what makes many of the others possible. A getter method called inside a hot loop, once inlined, exposes its internals to the surrounding optimizer: dead branches can be eliminated, bounds checks can be proven redundant, and values can be kept in registers instead of being written back to memory. None of that is visible if the call stays a call. A JIT compiler can make inlining decisions no ahead-of-time compiler safely can, because it knows from live profile data which of several possible implementations a virtual call site actually resolves to at runtime — often just one, even when the code is technically polymorphic — and it can always fall back to a real virtual call if that assumption is ever violated.
JIT vs. AOT vs. Interpretation
| Approach | When compilation happens | Strength | Trade-off |
|---|---|---|---|
| Interpretation | None — reads and executes bytecode directly | Instant startup, no compile step | Slowest steady-state execution |
| JIT (just-in-time) | During execution, based on real runtime profile | Best steady-state peak performance; adapts to actual behavior | Warmup time before hot paths are compiled; compilation uses CPU/memory at runtime |
| AOT (ahead-of-time) | Before execution, based on static analysis | Instant peak performance from the first call; smaller runtime footprint | No runtime profile to optimize against; can’t adapt to how the code actually behaves in production |
Is Java JIT compiled? Yes — by default, and it’s the reason Java’s steady-state throughput is often competitive with statically compiled languages, even though the JVM starts from portable bytecode. Is a JIT a compiler or an interpreter? Strictly, a JIT is a compiler; production JVMs simply combine a JIT compiler with an interpreter so the two techniques cover each other’s weaknesses — interpretation for fast startup, JIT compilation for fast steady-state execution.
AOT compilation is the mirror image of that trade-off. Ahead-of-time compilers — the GraalVM Native Image toolchain is a well-known Java example — compile methods before the program ever starts, based only on static analysis. That gets rid of warmup entirely: the very first request is already running compiled code. The cost is that an AOT compiler is guessing at how the code will behave, because it has no live profile to work from, and it usually has to make more conservative assumptions (or restrict dynamic language features like reflection) to compile safely ahead of time.
Why JIT Warmup Matters More in the Cloud-Native Era
JIT compilation was designed for a world of long-running application servers: warm up once, run for weeks. Cloud-native deployment inverts that assumption. Autoscaling, rolling deployments, and serverless functions all create new JVM instances constantly, and every one of them starts back at square one — interpreted bytecode, cold profile data, invocation counters at zero. A service that autoscales to handle a traffic spike spins up new pods that are, for the first seconds to minutes of their life, running slower interpreted or lightly-compiled code exactly when the extra capacity is needed most.
This shows up as a few concrete problems in production: elevated tail latency (p99, p999) right after a deployment or scaling event, wasted cloud spend on instances that are running below their eventual steady-state efficiency, and health checks or load balancers that need extra tuning to avoid routing traffic to instances that technically “passed” startup but haven’t warmed up yet. None of this is a flaw in JIT compilation as a technique — it’s a mismatch between an optimization strategy designed for long-lived processes and an infrastructure model built around short-lived ones.
Where Falcon Improves on Standard JIT Compilation
Azul Prime replaces OpenJDK’s default JIT compiler (C2) with Falcon, a JIT compiler built on LLVM — the same industrial-strength compiler infrastructure used by Clang, Rust, and Swift. That matters for two reasons.
A more advanced optimizing backend
LLVM has decades of investment in advanced compiler optimizations — vectorization, instruction scheduling, register allocation — poured into it by a much larger community than any single JVM vendor could fund alone. Falcon inherits that work, which can translate into improved peak throughput on compute-heavy, loop-intensive code compared to C2, without requiring any code changes.
Continuing to re-optimize instead of settling
Standard JIT compilers do re-optimize when a speculative assumption breaks, but the practical ceiling on how aggressively they keep revisiting already-compiled code is constrained by compile-time budget. Falcon is designed to keep using live production profile data to refine hot methods further, which matters most for long-running services where the JVM has hours or days of runtime behavior to learn from — exactly the profile of most cloud-native Java services.
Falcon doesn’t eliminate JIT warmup by itself — for that, Azul Prime pairs it with ReadyNow, which persists a JIT profile from a previous run (or a training run) and replays it on startup so hot methods get compiled immediately instead of waiting for the usual invocation counters to trip. Together, Falcon and ReadyNow target the two costs that standard JIT compilation accepts as a fact of life: slower peak optimization and a cold start every time a new instance boots.
Offloading compilation with the Optimizer Hub
There’s a third lever Prime pulls on the JIT side: JIT compilation itself, even Falcon’s more advanced version of it, still consumes CPU and memory on the instance that’s running the application, competing directly with the application’s own workload for resources right when that instance needs them most. Azul’s Optimizer Hub (formerly Cloud Native Compiler) moves that work off-box, running Falcon compilation as a separate, centrally managed service that application JVMs offload to over the network. Application instances get compiled code back without ever spending their own CPU cycles compiling it, and because the compilation service is shared across many JVM instances, it can also reuse profile and compilation work across instances running the same application — something no single-JVM JIT compiler can do on its own.
Frequently Asked Questions
How does a JIT compiler work?
It profiles a running program to find frequently executed code, compiles that code to native machine instructions in the background, and swaps it in — sometimes re-optimizing or de-optimizing later if runtime behavior changes.
What does a JIT compiler do?
It bridges portable bytecode and native performance: the JVM starts running bytecode immediately, and the JIT compiler converts the parts of the program that run often into optimized machine code without requiring a separate, up-front compile step.
Is Java JIT compiled?
Yes. The standard JVM (HotSpot) interprets bytecode initially and JIT-compiles hot methods as it runs. Java can also be AOT-compiled (e.g., with GraalVM Native Image), but JIT compilation is the default and most common execution mode.
Is JIT a compiler or an interpreter?
A JIT is a compiler — it produces native machine code. It’s typically paired with an interpreter inside the same JVM so the program can start running immediately while the JIT compiles the hot paths in parallel.
What is an AOT compiler in Java?
An ahead-of-time (AOT) compiler compiles Java code to native machine instructions before the program runs, using only static analysis rather than a live runtime profile. GraalVM Native Image is the most common Java AOT toolchain; it trades away JIT’s warmup period and runtime adaptability in exchange for instant peak startup performance.
Does JIT compilation slow down my application?
Not in steady state — the whole point of JIT compilation is to make hot code faster than pure interpretation. The cost shows up as warmup: for a window after startup (or after a code path first becomes hot), the JVM is spending some CPU on profiling and compiling in the background while that code is still running interpreted or at a lower compilation tier.
Can I see what my JVM’s JIT compiler is doing?
Yes — flags like -XX:+PrintCompilation show which methods are being compiled and at what tier, and -XX:+PrintInlining shows inlining decisions. Azul Prime environments can also surface this through Azul Mission Control for a more detailed, production-safe view of compilation and de-optimization activity.
The Bottom Line
JIT compilation is the reason Java can ship portable bytecode and still deliver native-competitive performance: interpret first for instant startup, watch the running program to find what actually matters, then compile just those hot paths into optimized machine code — and keep refining that decision as long as the program keeps running. Falcon doesn’t change that fundamental model; it changes how good the compiled code is and how long the JVM keeps improving on it, by swapping in an LLVM-based backend with a broader optimization toolkit than C2 and by continuing to act on live production profile data well past the point where standard JIT compilers settle. Paired with ReadyNow to remove the cold-start penalty and Optimizer Hub to take compilation off the application instance entirely, Falcon is built specifically for the reality of modern Java deployment: short-lived instances, autoscaling, and workloads where both warmup time and peak throughput are on the line at the same time.
Related Reading
For a closer look at how ahead-of-time compilation compares in practice, see JIT Performance: Ahead-of-Time versus Just-in-Time.