ccache
is great on your laptop.
sccache
adds a daemon and a remote cache.
distcc
farms compiles across machines. They all assume the worker is trusted shared-kernel infrastructure. hpcc assumes it isn't.
curl -fsSL https://hpcc.dev/install.sh | bash
Linux & macOS · interactive · pins a release via HPCC_VERSION=v0.1.0-alpha bash.
Windows worker setup: tools/userdata/install-worker.ps1.
A bwrap sandbox is not. A KVM boundary is. That single distinction is the entire reason hpcc exists — and the reason every existing OSS distributed compiler stalls at the regulated-enterprise door.
The worker is trusted shared-kernel infrastructure. Isolation is best-effort: a bubblewrap namespace, a Linux user, a chroot. Defensible on a laptop, indefensible in a regulated multi-tenant environment.
The worker is hostile-by-default, multi-tenant, and on the audit trail. One Firecracker microVM per tenant session. Separate kernel. KVM boundary. No NIC. Every compile produces a single audit row reproducible from one line of a TSV.
A cell-by-cell view of the boundary your auditor can recognise — and the operational pieces that fall out of it.
| ccache | sccache-dist | distcc | hpcc | |
|---|---|---|---|---|
| Worker isolation | none local user, no remote |
bwrap shared kernel namespace |
none worker is the host |
Firecracker · KVM per-tenant microVM, separate kernel |
| Network on worker | — | full NIC | full NIC | no NIC · vsock only |
| Toolchain identity | compiler binary hash | compiler binary hash | operator-managed | OCI image digest |
| Source staging | client preprocess | client preprocess | client preprocess | PREPROCESSED or CAS · content-addressed manifest · 1-RPC cache hit · cross-developer sharing via .hpcc marker |
| Audit trail | — | logs | logs | per-job row · 9-tuple · reproducible |
| Reproducibility | by ceremony | by ceremony | by ceremony | auto-injected flags · pinned env |
| Cache poisoning | laptop-trusted | client writes cache | — | paranoid mode · worker-only writes |
| Windows path | — | — | — | Hyper-V isolated · hpcc-agent over HvSocket · no SMB across the partition boundary |
Driven directly by hpcc — no firecracker-containerd dependency. The VM stays warm across compiles in a per-tenant pool, torn down on idle. Separate kernel. KVM boundary.
There is no exfiltration argument to have, because there is no network device. Full stop. Host↔guest is one vsock device carrying a single bidirectional gRPC stream.
No "hash the gcc binary" dance. 50 developers sharing one image produce one cache bucket. CI and laptops cannot silently diverge.
The scheduler is a lookup service — never touches compile payloads or artifact bytes. The client dials the worker directly. No proxy hop, no bottleneck, no scheduler holding cache keys it doesn't need.
Auto-injected per family: GCC/Clang get -ffile-prefix-map=/src=. + -Werror=date-time; MSVC gets /d1trimfile:/src + /PDBSourcePath:/src. The per-Exec staging dir is stripped from .o/.obj/.pdb embedded paths so two Execs of the same source produce byte-identical outputs. Pinned locale, timezone, hostname inside the VM round it out.
(image_digest, source_digest, flags, output_digest, tenant, worker, vm, duration, exit) — every compile, reproducible from a single line. The table format auditors want to see.
Default source_mode = "cas": client ships a content-addressed manifest, the worker probes its compile cache by manifest digest — 1 RPC, ~32 bytes, .o back. A .hpcc project marker normalises paths so two developers' identical checkouts hash to the same key. Bazel's ActionCache.GetActionResult, without the rest of REAPI's surface. "preprocessed" remains selectable as a fallback.
Preprocessed C++ compresses 5–10×. The single largest perf lever, and it's on by default.
Cache reads and writes happen only on the worker. Clients never touch cache stores, never hold remote-store credentials. A compromised laptop cannot poison the cache.
Each tenant in the scheduler config declares its own IdP (Okta, Keycloak, Auth0, anything OAuth2-compliant). Clients know only their tenant_id + scheduler URL; an unauthenticated GetTenantIdP RPC returns the OAuth endpoints for that tenant so laptops never hardcode IdP coordinates. The scheduler validates the password-grant JWT against the named tenant's JWKS — an IdP configured for tenant A is never asked to verify a token labeled as tenant B — then signs a short-lived routing token tenant-, image-, and worker-scoped. The same tenant_id scopes the storage namespace, the per-job audit row, and (when wired) the per-tenant upload quota.
The default Microsoft path for sharing host directories into a Hyper-V isolated container is VSMB — the same SMB protocol family that's absorbed EternalBlue and a two-decade tail of kernel-mode RCEs. Stapling an SMB parser across a boundary whose pitch is "auditors recognise this kernel+VM line" gives the threat model back. hpcc mirrors the Linux side instead: hpcc-agent.exe as PID 1, listening on HvSocket (the Hyper-V analogue of vsock). One bidi AgentService.Exec stream carries header + input file chunks in, stdio + result + output file chunks out. Wire protocol we own — not an industry-standard filesystem protocol with a CVE-of-the-month history.
The scheduler returns a worker address and TLS trust info — it never touches compile payloads. The client dials the worker directly. The worker dispatches into a tenant-pinned Firecracker VM via a single bidirectional gRPC stream over a vsock device. No NIC ever appears in the guest.
Every job emits a 9-tuple. hpcc inspect <hash> reads it back. hpcc explain <file> tells you which header or flag broke the key — not a log to grep.
When a build that should have hit the cache misses, the question stops being "where do I start?" and starts being "what changed?" hpcc answers the second question directly.