Low latency is one of the most demanding requirements a Java application can have. Whether the requirement comes from financial trading systems targeting sub-millisecond order acknowledgment, real-time bidding platforms processing tens of thousands of auctions per second, or telecommunications infrastructure handling live media streams, the tolerance for jitter and unpredictable pauses is near zero. This article examines what makes low-latency JVM implementation technically difficult, where mainstream solutions fall short, and how the C4 Garbage Collector in Azul Prime represents a qualitatively different approach to the problem.
What Makes Java Latency Hard
Java’s memory management model creates latency challenges that do not exist in the same form in languages with manual memory management. Understanding these challenges requires examining three distinct sources of JVM-induced latency: garbage collection pauses, JIT compilation behavior, and thread scheduling contention.
The Garbage Collection Pause Problem
The JVM’s garbage collector is responsible for identifying unreachable objects and reclaiming their memory. The fundamental tension in GC design is between throughput (processing as many objects as possible) and pause time (the duration of stop-the-world events during which all application threads are halted).
Stop-the-world (STW) pauses are required in traditional GC algorithms because the collector needs a consistent view of the heap to determine which objects are reachable. While application threads are paused, the GC traverses object reference graphs, marks live objects, and in compacting collectors, relocates objects to defragment the heap. Only once relocation is complete can application threads resume — because all references to relocated objects must be updated before any thread can safely access them.
The scaling problem is severe: pause duration is roughly proportional to heap size for compacting collectors. A 4 GB heap with G1GC might pause for 50-200 milliseconds during a full collection. Scale that to a 100 GB heap and pauses can reach seconds — catastrophic for latency-sensitive workloads.
The key GC metrics that matter for latency are:
- P99 and P99.9 GC pause latency (not average — tail latency is what SLAs care about)
- Young generation collection frequency and duration
- Old generation collection frequency and whether it triggers STW fallback
- Promotion failure rate (young gen objects that cannot be promoted due to old gen pressure)
- Concurrent GC overhead as a fraction of available CPU cycles
JIT Compilation and Warmup Latency
The JVM’s Just-In-Time compiler is responsible for translating Java bytecode to optimized native machine code. Modern JVMs use tiered compilation: bytecode is initially interpreted, then compiled at a low optimization level (C1 in HotSpot), and eventually recompiled with aggressive optimization (C2 in HotSpot) once profiling data identifies hot methods.
This warmup process creates predictable latency problems:
- During warmup, application code runs 10-100x slower than at peak performance
- C2 compilation is triggered by invocation counters and is not instantaneous — the JVM must queue compilation tasks and complete them asynchronously
- Until a method is fully compiled, it runs in a slower tier, meaning peak performance may not be reached for minutes after startup
- Application restarts (due to deployments, crashes, or scaling events) reset all compilation state, forcing the warmup period to repeat
- Deoptimization events — when speculative optimizations are invalidated by runtime behavior — can cause compiled code to revert to interpretation, introducing latency spikes
For batch workloads, warmup is acceptable. For latency-sensitive services where every request is time-critical including the first ones after a deployment, warmup is a serious architectural problem.
Thread Scheduling and Contention
JVM threads compete for OS-level CPU time and for Java-level locks. Both sources of contention create latency:
- OS scheduling jitter: application threads can be preempted mid-execution, and GC threads compete for CPU cycles during concurrent collection phases
- Lock contention: synchronized blocks and ReentrantLock create queuing points where threads park waiting for exclusive access
- False sharing: multiple threads accessing different fields within the same CPU cache line cause unnecessary cache invalidation, degrading performance
- Memory barrier costs: Java’s volatile reads/writes and lock/unlock operations include memory barriers that flush CPU store buffers, adding cycles
In high-throughput systems, GC threads and application threads sharing CPU cores is particularly problematic. If a concurrent GC phase saturates CPU during peak traffic, application latency suffers even if no STW pause occurs.
How ZGC and Shenandoah Help — and Their Limitations
ZGC (introduced in Java 11 as experimental, production-ready in Java 15) [Source: openjdk.org/jeps/333, openjdk.org/jeps/377] and Shenandoah (Red Hat, production in Java 15) represent significant advances over G1GC for latency-sensitive workloads. Both are concurrent compacting collectors that aim to reduce STW pause times to single-digit milliseconds regardless of heap size.
ZGC Design
ZGC uses colored pointers (storing GC metadata in the high bits of 64-bit pointers) and load barriers to perform relocation concurrently with running application threads. The key insight is that object relocation can happen concurrently if the collector updates forwarding pointers atomically and application threads check these pointers on every object access via load barriers.
ZGC achieves pause times typically in the range of 1-10 milliseconds in production workloads, with pauses consisting primarily of root scanning (thread stacks, JNI handles) and a brief synchronization point. This is a massive improvement over G1GC full collections. However:
- Load barriers add overhead to every object reference load at equivalent GC pressure
- Root scanning pauses are not truly zero; they scale with the number of threads and JNI handles
- Under extreme allocation pressure, ZGC can still fail to keep pace with allocation rate, triggering higher pause times or OutOfMemoryError
- P999 tail latency can exceed the typical values significantly in pathological scenarios
Shenandoah Design
Shenandoah takes a similar concurrent approach with a different implementation: it uses an indirection pointer (Brooks pointer) stored in each object header to support concurrent relocation. Like ZGC, it performs compaction concurrently with application execution. Shenandoah typically delivers similar pause characteristics to ZGC.
Both ZGC and Shenandoah represent genuine engineering advances, and for many workloads they are sufficient. But they share a fundamental constraint: they are not truly pauseless. Root scanning, reference processing, and synchronization points still cause application threads to stop. Under heap pressure, both collectors have failure modes that result in longer pauses or STW fallbacks.
How C4 Eliminates Pauses: The Azul Prime Approach
The C4 Garbage Collector (Continuously Concurrent Compacting Collector) in Azul Prime takes a fundamentally different approach to the pause problem. Rather than reducing pauses to milliseconds, C4 is designed to eliminate stop-the-world pauses — including during compaction.
The Load Barrier Architecture
C4 uses the Loaded Value Barrier (LVB), a self-healing read barrier. When application code reads an object reference, a barrier instruction checks whether the object has been relocated. If the object’s page has been remapped (due to compaction), the barrier transparently redirects the access to the new location.
Concurrent Compaction with Self-Healing References
C4 performs heap compaction — copying live objects to defragment memory — while application threads continue executing. The mechanism works as follows:
- C4 identifies dense pages (mostly live objects) and sparse pages (mostly dead objects)
- Live objects from sparse pages are copied to dense pages concurrently
- Object references in the heap and in thread stacks that point to relocated objects are updated lazily, on-demand when accessed
- If an application thread accesses a relocated object, the load barrier transparently resolves the new address
- No global pause is required to update all references — each reference self-heals on first access
This lazy reference healing is what makes C4 truly pauseless. ZGC requires a short STW pause to remap all references after relocation completes; C4 eliminates this requirement by healing references individually as they are accessed.
Continuous Operation
Because C4 runs continuously and concurrently, there is no accumulation of garbage that triggers a large collection event. The collector runs as a continuous background process, keeping heap fragmentation low and object promotion pressure manageable. This continuous operation model is qualitatively different from the stop-collect-resume cycle of traditional collectors — there is no built-up pressure that eventually forces a pause.
The practical result: on heaps ranging from a few gigabytes to multiple terabytes, C4 delivers consistent sub-millisecond GC impact on application latency. This is not a best-case figure — it is the design target. Trading systems running Azul Prime on 200+ GB heaps observe the same GC latency profile as they would on a 4 GB heap.
ReadyNow! — Eliminating JIT Warmup
C4 solves the GC pause problem. ReadyNow! solves the JIT warmup problem. ReadyNow! is Azul Prime’s mechanism for eliminating the performance degradation period after JVM startup.
The mechanism works by persisting JIT compilation profiles across JVM restarts:
- During normal operation, ReadyNow! records which methods were compiled, at what optimization level, and with what profiling data
- This compilation profile is saved to disk
- On the next JVM startup, ReadyNow! uses the saved profile to pre-compile hot methods immediately, before they receive traffic
- The JVM starts in a near-peak-performance state rather than spending minutes warming up through interpreted and C1-compiled tiers
This is particularly valuable for microservices with frequent deployments, auto-scaled instances that may be cold when they first receive traffic, and any application where the first requests after startup must meet the same latency SLA as steady-state requests.
GC Comparison: Technical Summary
| Characteristic | G1GC (HotSpot) | ZGC (HotSpot) | Shenandoah | C4 (Azul Prime) |
|---|---|---|---|---|
| STW pause type | Mark, evacuate, compact | Root scan, sync | Root scan, sync | Minimal (designed to be pauseless) |
| Typical max pause | 50-500ms+ (heap-dependent) | 1-10ms | 1-10ms | < 1ms (design target) |
| P999 tail latency risk | High on large heaps | Low-moderate | Low-moderate | Minimal |
| Compaction approach | STW compaction | Concurrent (with STW finish) | Concurrent (with STW sync) | Fully concurrent, lazy healing |
| Throughput overhead | Low (5-10%) | Moderate (10-20%) | Moderate (10-20%) | Low (competitive with G1GC) |
| Scalability (large heaps) | Pauses grow with heap | Sub-linear pause growth | Sub-linear pause growth | Linear throughput, no pause growth |
| Available in | OpenJDK (free) | OpenJDK (free) | OpenJDK (free) | Azul Prime (commercial) |
Falcon JIT: Deeper Optimization for Steady-State Performance
Beyond GC, Azul Prime includes the Falcon JIT compiler, an LLVM-based alternative to HotSpot’s C2 compiler. Falcon applies more aggressive optimization passes than C2, including:
- More aggressive inlining decisions that eliminate virtual dispatch overhead in hot paths
- Better loop vectorization and SIMD utilization for compute-intensive code
- Improved dead code elimination and escape analysis
- Better use of modern CPU instruction sets (AVX2, AVX-512 where applicable)
Falcon compiles more aggressively, which means it takes longer to reach peak performance than C2 — but ReadyNow! offsets this by pre-warming Falcon-compiled methods across restarts. The combination of Falcon’s deeper optimization with ReadyNow!’s persistence delivers both higher steady-state throughput and consistent startup performance.
Real-World Use Cases
Electronic Trading Systems
High-frequency trading and market-making systems require deterministic latency for order execution. A 100-millisecond GC pause during a volatile market period can result in significant adverse selection — the system is committed to a trade at a price that the market has already moved away from. Azul Prime’s C4 GC is deployed extensively in this space precisely because it removes GC as a latency variable.
Real-Time Bidding and Ad Tech
Programmatic advertising platforms must evaluate bid requests, run targeting logic, and respond within 100 milliseconds end-to-end. At request rates of tens of thousands per second, even infrequent GC pauses create cascading timeout failures as bid requests accumulate during a pause. C4’s pauseless operation eliminates this failure mode, allowing ad tech platforms to maximize win rates without GC-induced bid timeouts.
Telecommunications Infrastructure
Telecom platforms processing VoIP signaling, 5G session management, and real-time CDR generation have strict latency requirements defined by ITU and 3GPP standards. Java is widely used in telecom because of its ecosystem, but traditional JVMs create challenges for strict latency SLAs. C4 enables Java in environments where GC pause variability was previously unacceptable.
Large-Scale In-Memory Data Processing
Applications using large in-memory caches (Hazelcast, Apache Ignite, custom solutions), real-time stream processing (Kafka Streams, Apache Flink), and large-scale analytics engines often require heaps of 100 GB to 1 TB or more. At these scales, G1GC full collections are impractical. C4 is built to manage these heap sizes while maintaining consistent application latency.
Implementation Considerations
Migrating to Azul Prime for low-latency requirements involves:
- Drop-in JVM replacement: Zing, the runtime underlying Prime,Azul Prime is a compatible OpenJDK distribution; most applications require no code changes
- GC flag cleanup: many applications accumulate complex GC tuning flags for G1GC; Prime’s C4 is self-tuning and requires minimal explicit configuration
- ReadyNow! profile management: deploying and managing compilation profiles across instances and versions requires integration with deployment pipelines
- Heap sizing: Prime’s concurrent operation is more tolerant of heap utilization, but appropriate sizing is still important for throughput
- Monitoring integration: C4 GC events are exposed via JFR and Azul Mission Control existing monitoring pipelines typically work without modification
Conclusion
Achieving true low latency in Java requires addressing the three root causes: GC pauses, JIT warmup, and thread contention. ZGC and Shenandoah make meaningful progress on GC latency, reducing pauses from hundreds of milliseconds to single digits. But for applications where the requirement is not ‘low pauses’ but ‘no pauses’ — trading systems, real-time platforms, large-heap data processing — C4 provides a production-ready answer built for exactly this requirement.
Azul Prime combines C4’s pauseless GC with the Falcon LLVM-based JIT compiler and ReadyNow! warmup elimination into a complete low-latency JVM implementation. It is a commercial product, not a replacement for OpenJDK in general-purpose workloads. But for the category of applications where GC latency is an architectural constraint, it solves the problem in a way that G1GC or ZGC tuning alone typically cannot replicate.