Content-Addressed Monorepo Build Caches Without Silent CI Drift
Why timestamps, unsorted directory globs, and leaked host `PATH` variables poison remote build caches—and a 120-line digest verifier that pushed our cache hit rate from 61% to 96.4%.

Principal Systems Architect · Stockholm

A remote build cache is only as trustworthy as its action input Merkle tree. Teams adopt task runners expecting five-minute pull requests to drop to forty seconds, only to discover two weeks later that macOS laptops never hit artifacts compiled on Linux runners—or worse, a stale .d.ts bundle ships to production because an undeclared environment variable changed.
Four Silent Sources of Cache Misses and Poisoning
Auditing 11,400 CI executions across a 90-package TypeScript and Rust repository exposed four recurring culprits:
- Locale-dependent file glob ordering:
readdirreturns inode order on ext4 and APFS-normalized order on macOS unless explicitly sorted by UTF-8 byte code points. - Ambient environment pass-through: Hashing
process.envwholesale busts every task wheneverGITHUB_RUN_NUMBERincrements; omitting env vars letsNODE_ENVdrift poison outputs. - Embedded archive mtimes:
tarheaders embed modification timestamps unless--mtime='UTC 2020-01-01'and--sort=nameare enforced. - Absolute diagnostic paths: Compiler source maps emitting
/home/runner/work/...instead of workspace-relative../../packages/...paths.
export function computeTaskDigest(inputs: {relPath: string; blake3: string}[], envAllowlist: Record<string, string>): string {
const sortedFiles = [...inputs].sort((a, b) => (a.relPath < b.relPath ? -1 : a.relPath > b.relPath ? 1 : 0));
const sortedEnv = Object.keys(envAllowlist).sort().map((k) => `${k}=${envAllowlist[k]}`);
return hashPayload(JSON.stringify({ v: 2, files: sortedFiles, env: sortedEnv }));
}
Double-Execution Determinism Gate
Once every night, run a clean build twice in two isolated temporary directories (/tmp/build-a and /tmp/build-b) with TZ=UTC and compare output hashes byte-for-byte. That single nightly check kept our frontend accessibility checks in Top-Layer HTML Dialogs and Tactile Braille Focus Discipline and Wasm AOT artifacts in Anatomy of a Sub-Millisecond WebAssembly Component Cold Start reproducible across every developer machine.
Connected Architecture Notes

Henrik Lindqvist
Stockholm · UTC+1Principal distributed systems architect writing weekly benchmarks on consensus protocols, storage engines, WebAssembly isolates, and tactile web accessibility.
Reader Field Notes
0 verified notesNo published field notes yet on this essay. Share a production observation or benchmark note below.
Continue Reading
All essays →
Deterministic Raft Lease Quorums Under Asymmetric Cross-AZ Jitter
Oct 28, 2026

Anatomy of a Sub-Millisecond WebAssembly Component Cold Start
Oct 14, 2026

Pacing LSM-Tree Compaction Debt on High-Queue NVMe Drives
Sep 27, 2026
Get next Sunday’s benchmark by email
Full essay in plain HTML, zero tracking pixels, one-click unsubscribe.