Anatomy of a Sub-Millisecond WebAssembly Component Cold Start

Marketing benchmarks love quoting 50 µs isolate instantiation numbers measured in a tight loop with hot L2 instruction caches. Real multi-tenant edge nodes evict cold tenant artifacts to NVMe every few minutes. When a request lands in Frankfurt for a worker untouched since yesterday, the kernel page table—not your JIT compiler—dominates time to first byte.

Where the 840 Microseconds Actually Go

We instrumented an ahead-of-time compiled Wasmtime embedder serving 4,200 distinct tenant components on a 64-core EPYC host. Under Zipfian request traffic, median instantiation took 310 µs, while p99 climbed to 2.9 ms until we separated read-only .text segments from pre-initialized data pages.

// Pre-populate read-only AOT code pages without faulting writable heaps
int fd = open(compiled_cwasm_path, O_RDONLY | O_CLOEXEC);
void *rx_base = mmap(NULL, text_len, PROT_READ | PROT_EXEC,
                     MAP_PRIVATE | MAP_POPULATE, fd, text_offset);
madvise(rx_base, text_len, MADV_WILLNEED);

Using MAP_POPULATE indiscriminately across a 12 MB module stalls the acceptor thread. Instead, mapping only the entry trampoline and routing dispatch table eagerly—while letting rarely executed error formatters fault on demand—cut p99 cold start from 2.9 ms down to 690 µs.

Snapshot Pre-Warming vs. Canonical ABI Copies

WASI Preview 2 introduces typed component interfaces, which means strings and byte slices cross host boundaries via canonical ABI lift/lower rules rather than raw pointer arithmetic. For payloads under 16 KB, copying into a pooled per-request bump arena beats virtual memory remapping every single time because mmap / munmap shootdowns acquire mmap_lock across sibling threads.

Pairing Edge Isolates With Local State

Compute without nearby state merely moves latency from cold starts to database round-trips. Combining pre-snapshot Wasm components with embedded read replicas—configured using the WAL checkpoint discipline in Predictable SQLite WAL Checkpoints in Read-Heavy Edge Services and traced via Sub-1% Overhead Tail-Latency Histograms With Per-CPU eBPF Maps—delivers 11 ms end-to-end p99 responses across continental Europe.