Archive — history, not state. Kept for its reasoning and its evidence; its plan is closed.

Derivations — the shop’s build model, rebuilt on closures

Written 2026-08-02 from John’s ruling, verbatim in spirit: “I do not understand why we have lists of any files at all. There should be no lists of files. Basically ever. You list packages and the shop uses them.” This plan runs ALONGSIDE plans/one-binary.md and is orthogonal to it: one-binary owns the floor (the fused binary, forge, the boot section); this plan owns how the shop turns source into runnable artifacts and how it decides what is already made.

The verdict on the current machinery

The cache/hash layer is built on two wrong premises, and every recent fix — the relower cache, the compiler/press salt split, SEED_BOOT_FLOOR_SRC, the key_inputs stage tags — is a patch on top of them:

1. Eager, global invalidation events. A builder edit moves a global salt and the world is invalid at once; the seed reweld makes it a synchronous full rebuild. Nix is not slow, Erlang’s reloading is not slow, and we are combining them — the slowness is ours: invalidation should not be an event at all. It should be a per-file cache miss discovered lazily, on demand.
2. Hand-declared identity. key_inputs.cm (STAMP_DIRS, STAMP_FILES, MACH_SOURCE_FILES, EXEMPT), the Makefile’s SEED lists, gate A proving the hand lists complete — all of it re-states, by hand, facts the shop can walk: a closure is discoverable from any root by reading imports. Hand lists drift; the boot-floor gap (2026-08-02) was not carelessness, it was the structural failure mode of every hand list. Tools may LIST what feeds an actor on request; nothing keeps such a list as an input.

The consequence of both: a tiny change anywhere caused everything to recompile, sequentially — sequential being its own absurdity in an actor system where spawning fifty compilers is trivial.

The dev build, as it should run (the ps story)

The binary carries its boot cart — the ONLY cart the system runs from — with precompiled actors in one or more mach blobs (trending toward one), and it understands package paths like compiler to launch a compiler cold. After boot it mounts packages or a store; the first mount is the shop bundle (the qop — NOT legacy), which seeds it with mcode: most of the mcode that made the cart in the first place, plus more. Iteration is faster before anything else happens, because the store opens warm.

Request ps:

1. Resolve. ps is a bare name; the shop searches its bin packages and finds the owning package. It reads that package’s package.json for link replacements. (A package describing ITSELF — its natives, its endowments — is not a hand list; it is the package’s own manifest. The sin is central cross-package file lists.)
2. Walk the closure. Compile ps far enough to see its use imports and endowments (an import-scan, cached per file like everything else). Then their imports and endowments, transitively. The closure is nothing more than a set of C files and CM files from a variety of packages — say 10 C and 50 CM.
3. Policy. The closure yields the endowment list; the shop can refuse to compile at all right here. Dev builds continue.
4. Diff against the store, per file. No closure-wide hash decides anything (at most, one lookup: “is ps itself already made?”). Each member is checked individually: maybe 30 of the 50 CM files already exist as mcode in the qop — untouched. Maybe 5 of the 10 C files already exist as object files for this platform — untouched.
5. Build the misses IN PARALLEL. Launch 20 mcode compilers for the 20 missing units and 5 C compilers for the missing objects — trivial with requestors. This is the piece we are missing entirely today.
6. Lower and link, also parallel. Each unit lowers to its own mach blob (the typical dev-machine link plan), 50 spawned lowerings; objects become dylibs; the executable manifest is assembled from the results.

Principles

1. No file lists. You list packages; the shop uses them. Identity is walked (closures, package self-manifests, the build plan’s rendered manifest for C), never declared in a side list. Tools answer “what feeds this actor?” on demand.
2. Per-file derivations. Every artifact — a unit’s mcode, a C object, a dylib, a lowered blob, an executable manifest — is keyed on ITS OWN inputs. Logger changes → logger’s derivation changes → ps’s manifest gains one new row → relink. Nothing else recompiles, ever.
3. No invalidation events. Nothing is ever marked stale; there is no salt to move and no reseed to trigger. A change simply means the next demand for a dependent derivation misses and rebuilds — lazily, only for what is actually requested, like Nix after a toolchain bump.
4. Parallel by default. Misses in one closure build concurrently on a spawned compiler fleet. The builder is not one actor working a queue.
5. Freshness is a question, not a stamp. “Is this actor out of date, for this platform?” = walk its closure, check each member’s derivation against the store. The shop answers it; make/forge ask it; no sidecar stamp files.

Derivation inputs (discovered, never listed)

• Unit mcode: source hash, compile options (kind, link lane, rep — and PIT_STREAMLINE_DISABLE becomes an explicit option, not ambient env), and the identity of the producing compiler. That identity is the compiler’s own executable manifest — the machine-derived closure identity every actor already has (P0’s mechanism). This is how a compiler fix propagates with no global event: edit streamline → the compiler’s manifest moves → future unit demands miss and rebuild, per file, in parallel. Edit logger → the compiler’s manifest does not move, because logger is not in the compile pipeline’s walked closure.
• C object: source content, cc identity, flags. Already correct today (cc-content keys); stays.
• Dylib: its objects, composition from the package’s own natives declaration.
• Lowered mach blob: the unit’s mcode content, the lowering’s identity. Pre-P5 that identity is the binary (its C closure is the plan-rendered manifest — machine-derived); post-P5 it is the Pit lowering actor’s manifest like any other.
• Executable manifest: its rows. The manifest hash is the actor’s identity — this already exists (P0) and is the model everything above copies.

Determinism gates (fixpoint press, byte-identical regeneration, the differential fuzzer) are per-derivation properties and survive unchanged — they are Nix-shaped too.

THE IMMEDIATE JOB (F) — generations, not per-start realization

(fix the stampede and ratify the generation model in one landing)

Diagnosed 2026-08-02 from a fleet prototype (worktree 4ab8): every compile operation $started a builder; every named start freshness-REALIZES (P0); realizing the builder needs a builder → recursive realization stampede. 5,285 building entries for one key, one built, 100% CPU, 3.4 GiB RSS. The prototype found the hole before D2 shipped it. John’s ruling on the fix:

Realize once, instantiate many. Realization (produce/refresh an executable artifact — expensive, freshness-gated) and instantiation (spawn an actor from that artifact — a mapped-pool start, milliseconds, NO freshness question) are different operations, and the start path must know which one it is doing. A builder PER compilation is the intended shape — what must never happen is a realization per compilation.

Freshness is decided at generation boundaries, never per start, for infrastructure. The generation model, ratified: the cart’s generation-G fleet boots; when mounted source is newer, G’s compiler compiles the G+1 compiler and hands off (P0’s derivation_verified handoff IS this); G+1 does all further compilation and its infrastructure is PINNED for the generation. User programs keep per-start freshness (that is P0’s point); the machinery that ANSWERS freshness questions never asks one about itself mid-generation. Generation boundaries today: daemon boot and clerk handoff. When D1’s cheap closure-walk lands, the clerk can detect its own staleness per request and trigger a handoff — boundaries become automatic without ever re-entering per-start realization.

The work items:

• F1 — instantiate-don’t-realize for the fleet. The clerk realizes the builder executable ONCE per generation (at startup/handoff — compiled by the PREVIOUS generation’s builder when stale, which is the cart-compiler-compiles-the-new-compiler two-step). The fleet requestor starts instances by SUPPLYING that realized executable (the existing msg.executable start lane — it already bypasses catalog and realization). The named-start-through-freshness path for infrastructure dies.
• F2 — single-flight per realization key in shop_realize: between cache miss and stored result the key is claimed; concurrent requesters join the in-flight build instead of starting their own. Protects everything (two clients realizing one program), not just builders.
• F3 — width is an OPTION, never a law (John’s ruling). Requestor fleets accept an optional width; the default is unbounded. Backpressure is a knob for constrained targets, not a design assumption.
• F4 — honest observability. Log the generation identity at every handoff (one line: the manifest hash) so “which compiler compiled this” is never a mystery; fix the scheduler’s strike 1/3 line on cooperative pauses (cooperative yields accrue no strikes — the log should say paused (cooperative)).
• F5 — timeout ownership matches process ownership. The launcher kills a daemon IT spawned this invocation when readiness fails; a shared long-lived daemon is never killed by a client’s timeout.

F1+F2 are the fix; F3–F5 ride along. This lands before or with D2 — the requestor fleet is built ON these semantics, not patched to them.

What this deletes

• shoplib/key_inputs.cm — all of it: STAMP_DIRS, STAMP_FILES, MACH_SOURCE_FILES, EXEMPT, and gate A as a completeness proof (a residual assertion that the pipeline roots are the right roots may survive at two lines).
• prime_toolchain_stamp, set_builder_salt, the builder/compiler/press salts — including the salt split merged 2026-08-02 (c9d504f5), which is hereby scaffolding with a named expiry: it keeps keys honest until derivation-input identity lands, then dies.
• The Makefile’s seed hash lists — SEED_INPUT_HASH’s file globs, SEED_MACH_SRC (the “Makefile cannot read a .cm” restatement), SEED_BOOT_FLOOR_SRC (added 2026-08-02, expiry now named) — replaced by asking the shop the freshness question (the forge seed-check item in one-binary’s gates table; these two plans meet exactly there).
• boot/seed.stamp / boot/cart.stamp as sidecar stamp files.
• The oracle half of tests/store_freshness.ce that replays hand lists; it becomes a walk-and-compare against the same closure the shop walks.
• The single sequential builder lane as the only compile path: the one-actor builder_worker, the batch protocol and its sizing/timeout machinery, and the builder_worker_ref ? send : inline_lower() dual path (see D2’s audited inventory). The seed press (bootstrap_artifacts) and cement consume the same requestors, so the seed parallelizes for free — no separate seed lane survives.

What this speeds up, concretely

• A fleet edit rebuilds exactly its file plus dependents’ relinks — measured this session at its best (2 units) and worst (~50 units because pit-linker/build.cm sat in a stamped directory it could never affect). Under this plan the worst case converges to the best case.
• Compiler edits stop being world events; the world rebuilds lazily, only what is demanded, N-wide in parallel.
• The qop-as-warm-store means a fresh shop starts with most of its closure already derived.
• Reseeds stop existing as a dev-loop concept (one-binary’s unweld started this; the freshness question finishes it).

Increments

• D1 — the closure walk as THE freshness primitive. One implementation (the import-scan closure, per-file store diff) behind one query: is this actor current, for this platform? Wire pit shop status / forge’s seed-check to it. No behavior change to keys yet.

• D2 — the requestor decomposition (parallel build fan-out). The build path becomes a small algebra of requestors, each usable alone and composed with parallel():

  • unit_mcode(locator) → the mcode. THE atom: resolve, check the store, compile on miss. The shop uses it 50 times in parallel.
  • closure(locator) → the locator set. The import walk, itself parallel per import edge.
  • unit_blob(locator, target) → a lowered mach blob. Composes unit_mcode then lowers.
  • pool([locators], plan) → one pool. parallel(unit_blob …) then the link fold (the link itself is legitimately ONE job — N fragments to one pool; the parallelism lives upstream of it).
  • executable(locator, plan) → the manifest. closure → parallel(unit_blob …) + parallel(object …) → link → manifest.
  • object(c_file) / dylib(package) → the C side, which already half-does this (pit-linker/build.cm runs parallel(jobs, 4)) — widths stop being hardcoded and follow core count.

  The sequential inventory this deletes (audited 2026-08-02): builder_worker.ce do_compile_batch — one actor compiling a batch in a while loop, and the fleet holds exactly ONE builder_worker (set_builder_worker, a single ref): fifty units is fifty sequential compiles in one mailbox. Lowering (do_mach) rides the SAME single mailbox, so compiles and lowerings also serialize against each other. The whole batch protocol exists only because round-trips to the one worker were the cost — a fleet deletes batching, its result-count checks, and BUILDER_COMPILE_TIMEOUT sizing. The builder_worker_ref ? send : inline_lower() dual path dies with it — one way: requestors. Also suspicious and to be re-derived or deleted with a reason: parallel(jobs, pkg == RUNTIME_PACKAGE ? 1 : 4) in build.cm forces runtime C compiles to width 1.

  Pure speed, key-neutral, can land before any deletion elsewhere.

• D3 — derivation-input identity. Unit keys fold the producing compiler’s manifest identity instead of the global salt; lowering keys fold the lowering’s identity; PIT_STREAMLINE_DISABLE becomes a parameter. The pipeline closure is verified effect-free (the two-line residue of gate A).

• D4 — the deletion. key_inputs, salts, stamps, Makefile lists, the oracle replay — everything in the list above, in one landing, behind the full gate.

Ordering with one-binary: D1/D2 are independent and can start now; D3 wants P0’s manifest identity everywhere it already is (done) and should land before P5 gives the lowering its own manifest; D2 should also land before P5 — the Pit lowering should be ported into the requestor fleet, not into the single-worker shape and then reworked (same economy as “L2b before P5”); D4 is the joint landing with one-binary’s forge seed-check, where the last Makefile list dies. Cross-plan gates in full: C4→D1 (clerk’s link uniformity reuses D1’s freshness primitive) is the only hard wait; D3/D2-before-P5 are economy orderings; D4≡seed-check is one landing with two names.

The shop-perf rows (E) — ruled 2026-08-03

A full-pipeline investigation (run against the dev tip of 2026-08-02, pre-K_mach) measured the first make smoke at ~10 minutes with ~300 GiB of transient allocation — 80% of it text, and ~79% of ALL allocation inside shoplib/canonical.cm’s recursive encoder. The object store came out ~75% pit.mach.origin@1 JSON: provenance 38× larger than the Mach pools it describes. Two of the report’s findings were already fixed on this branch before it arrived: the missing pre-lowering Mach cache lookup (shop_build.cm finish_unit_mach checks cache_mach_result(null, derivation) before lower_mach_unit; key folds the runtime mach ABI since eabacbe67) and the cold-boot adopt path (3m42s → 13.8s, d7357d516). The rest of the disease is real and this plan owns it, because every row below is this file’s thesis applied: the work was never needed, or its representation manufactures orders of magnitude more bytes than the artifact contains.

The frame (John): priority 1 is eliminating work, priority 2 is speeding up what remains, priority 3 is cache robustness at the top. The budget case is the WORST case — compiler source changed, everything must honestly recompile — and that case must be fast, not hand-waved into a cache.

Pushback resolved: content addressing is NOT the cost and is not on the table. Blake2 is GB/s; the cost is manufacturing the bytes being hashed — recursive UTF-32 JSON, at every depth, for artifacts repeated per closure. Determinism and byte-identity proofs survive every row here untouched.

The rulings (John, 2026-08-03, in conversation):

1. Origin maps become a lazy join (E1). pit.mach.origin@1 stops being materialized and stored. mach_pool_emit.cm’s own comment is the proof it was always derivable: “the map is the join, and nothing in it is invented.” The press stores only the compact per-function origins column (one small integer per instruction — low single-digit MB for the whole tree, against 334 MB of expanded JSON); debug_resolve performs the pool-PC → unit/function/site/span join on demand, at first symbolication. First stack trace through a pool pays a small join; the seed stops encoding provenance entirely.
2. Record artifacts go straight to canonical binary (E4) — one landing, one reseed. No streaming-JSON intermediate: pit.mcode.unit@3 and the other record artifacts move from canonical JSON to a canonical binary (nota-family) encoding, per the standing “nota metadata + binary pools” ruling. Content hashes move, so this is its own landing with a reseed and the docs/spec/artifact-formats.md “canonical records” section rewritten. Kills both encode and decode cost structurally — the multi-MB-per-unit JSON is paid today on every store, every hash, and every hydrate.
3. The encoder stays Pit. No C native for canonical encoding/hashing. The new encoder is a streaming Pit byte-builder: write bytes into one growing blob, no per-subtree text values, hash over the finished bytes. The amplification dies with the representation, not with a language swap.
4. This plan is the home. These rows execute alongside D1–D4/F1–F5; the streamline-IR question stays in one-binary P4.

The rows:

• E1 — origin maps become a lazy join. As ruled above. Touches mach_pool_emit.cm (emit the column, drop origin_map from the press path), shop_store.cm (cache_mach_origin_value stores the column; mach_origin_key unchanged as the address), shoplib/debug_resolve.cm (the join moves here, on demand). The K_mach sidecar row shrinks with it.
• E2 — a cache hit never decodes the unit. Today the full-hit path (shop_build.cm with_source) parses the entire multi-MB unit JSON via parse_json_blob just to hand the artifact on, and hot-ps hydration decodes ~184 MB per invocation for the same reason. The fix is a slim per-unit metadata record (imports, kind, source/content/mach hashes) — which IS D1’s import-scan record; one artifact serves both. Hits and closure walks touch the slim record; only a real compile or lowering touches the full unit. E4 shrinks what “full unit” costs; E2 stops paying it at all on the hot path.
• E3 — the scratch-image check moves ahead of hydration. shop_actor.ce’s image_from_executable consults scratch_images only after load_executable_ref + hydrate_executable have paid the decode. Key the image by catalog ref / executable identity and check it before loading anything. A hot ps should be a map-and-go.
• E4 — canonical binary artifacts. As ruled above. Sequence AFTER E1 so the biggest JSON producer is already gone and the reseed diff is honest about what the format move buys.
• E5 — hit/miss telemetry and budget gates. K_mcode and K_mach hit/miss counters in the compile plan; phase events (read, inflate, decode, lower, encode, store) with elapsed + allocation deltas; make budget rows for: steady smoke wall + allocation, hot-ps root allocation, store composition split (Mach vs mcode vs provenance bytes), and a redundant-lowering count that fails on repeats without distinct keys. Regressions in this territory must be loud — “one compiler change made it slow again” should name the key edge that moved.
• E6 — the smoke convergence cycle (found 2026-08-03, needs one diagnostic session). After a compiler edit, the make/forge/seed/cement staleness checks re-trigger each other: measured three forge invocations inside ONE smoke, plus a reseed plus a recement, in smokes of 3m45s and 1m19s AFTER the 8m20s seed had already run (“C sources are newer than it” → rebuild → “daemon predates binary” → seed derivations stale → regenerate → generated inputs get fresh mtimes → repeat). Mtime-keyed checks with no single ordering; the fix direction is content-keyed freshness under one owner, which is D4’s remaining half. Caveat: measured in a git-archive copy whose flattened mtimes may amplify the cycle — reproduce in a real worktree before treating the magnitude as real.

The early-cutoff subtlety — the one D3 residue (recorded so the win is not lost): the split key SHAPE landed (c9d504f5e: K_mcode folds compiler_identity, K_mach folds lowering_identity + mcode content + mach ABI), but clerk.ce feeds the SAME value into both — the whole builder executable’s closure hash (pin_generation_builder and the clerk startup both do it; the one-binary P2 finding). The builder’s closure contains the compiler AND the press, so the honest-but-coarse identity over-invalidates in both directions: a press-only edit recompiles all mcode; a compiler edit re-lowers every pool even when the recompiled mcode is byte-identical, because lowering_identity moved with it. The fix is per-entry sub-closure identities sliced from the manifest the builder already carries: compiler_identity = the closure reachable from the compile entry, lowering_identity = the closure reachable from mach_lower (the mach ABI is already folded C-side). That lands the convergent worst case: compiler edit → mcode recompiles (honest, parallel under the landed D2 fleet) → unchanged-output units hit K_mach and never re-lower → pools relink only where inputs moved.

LANDED 2026-08-03. pit-shop/builder_identity.cm does the slice and pin_generation_builder feeds it. The walk is a reachability question inside the ONE manifest the pinned builder already carries: bindings rows are the resolver’s actual import edges, densely numbered against units, so a BFS from each op’s entry names that op’s sub-closure without re-walking source or decoding a single unit. compiler_identity = the closure reachable from pit-compiler/compiler (what compile_one calls); lowering_identity = the closure reachable from pit-shop/mcode_lower + pit-shop/mach_lower (what do_mach/do_fragment call; the mach ABI stays folded C-side and is not re-folded here). The identity hashes each unit’s source_hash, not its mcode hash — “the file changed” is the honest trigger, and it is what keeps the K_mach cutoff below it from being circular. Cost: one manifest object read, one decode, one BFS, per GENERATION — never per compile request. Cadence is enforced by placement: the pin IS the generation boundary. A unit that is actually in both sub-closures moves both; a lowering-only edit (mach_lower, shoplib::mach_pool_emit) moves only lowering. Each half falls back to the coarse whole-closure hash independently when the manifest cannot be read or carries no such entry, so the degraded path is exactly the pre-split behaviour — over-invalidating, never stale. The bootstrap window in start_current_clerk stays deliberately coarse: before the builder executable exists there is no manifest to slice. Gated by tests/builder_identity.ce (baseline): the two identities differ in a normal generation, a lowering-only edit moves only lowering, a compiler-only edit moves only compiler, a shared dep moves both, and an edit to the builder’s own message vocabulary moves neither. NOTE: the first reseed after this lands is a full one — every K_mcode and K_mach key moves once, and pit.seed.derivations@1’s recorded builder identities are stale until make seed rewrites them.

mach_press closure verdict — separable (C-line close-out, 2026-08-03). The manifest edge is real, but it is not a compile-time input to mcode. pit-compiler/compiler.cm imports shoplib::mach_press and shoplib::mach only for compile_to_blob, compile_pipeline, run_ast, and run_ast_noopt. The production builder’s compile/floor messages call mcode_unit_result and compile_result; neither reaches those helpers. The four helpers are used by tests and the differential runner, where executable pool production belongs.

Proposed, not implemented: extract those helpers and their two imports into a separate pit-compiler::runner module, then retarget tests/suite.cm, tests/lto.cm, and shop_tools/internal/diff_runner.cm. The compiler entry’s sub-closure would then stop at target-neutral mcode production, while the runner would explicitly join compiler + press for tools that want to execute freshly compiled code. A press-only edit would move lowering_identity only, leave K_mcode hot for every source unit, and avoid recompiling all mcode before the honest K_mach invalidation and re-press.

What the worst case becomes when E1/E4/D2/D3 are in: parse+compile of the true closure, fanned out across cores; lowering only for changed mcode; hashing over compact binary bytes produced once; no provenance materialization; GC pressure collapsing with the allocation it was serving. The ceiling left standing is the self-hosted compiler on the interpreter — and that ceiling is seconds for a 50k-line tree, not minutes. If, after these rows, streamline still dominates, one-binary P4 (interned IR) is the next lever — gated on a per-pass time/allocation spike, not taken on faith.

Ordering within E: E3 and E5’s counters are small and immediate; E1 next (biggest eliminate, no format politics — the stored map is pure derived data); E2 rides D1’s record; E4 is its own reseed landing after E1. Nothing in E blocks D1/D2/D3; D2/D3 remain the parallel-fan-out and identity halves of the same worst case.

E7-E9 — from the 2026-08-03 phase decomposition (agent-proposed, anchored)

• E7 — eliminate the bootstrap-driver realization. shop_tools/bootstrap.ce realizes a 66-unit ordinary Mach executable (97.31s wall, 70.55s canonical encode, 66 compile+mach misses) before asking the already-fresh coordinator to seed. Make it a thin resident/preseeded protocol client. Biggest single eliminate left.
• E8 — one resolved closure, shared. Compilation, manifest construction, realization dependency collection, and seed-object enumeration each re-walk the closure: 1,032 node touches, 17.61s (14.36s in the actual seed). Resolve once, hand the result around.
• E9 — speed the residual encoder. After E7/E8, ~8.35s of canonical encode remains; add artifact-class subcounters, optimize the dominant class, never changing sorted-key identity or bytes.
• Budget rows stay owed: they need allocation sampling, byte-classified store counters, and derivation-key tracking to be honest (agent, 2026-08-03).

2026-08-03 E7-E9 measurement pass (exact base b8c2de2d6)

The valid measurements below ran from one git archive in an isolated shop, outside the socket-restricted agent sandbox, after ps aux | grep pit plus a ten-second CPU-time sample showed the other build-named processes were idle daemon owners rather than an active P3 press. An earlier sandbox-denied socket.bind attempt is discarded, not counted as a run.

The second edit is still diagnostic: its completed clerk plan took 92.842s, including 63.986s canonical encode, 69 K_mcode misses, 102 graph touches / 6.076s, and zero Mach lowerings. But 158.84s is not a successful seed and must not be presented as the campaign headline. The source comments lived only in the scratch archive.

E9: go; do not close it. Residual canonical encode is 66.147s, not under the ~10s no-go threshold. The dominant artifact class is portable pit.mcode.unit@3, not Mach provenance: the 61.511s cold clerk plan had 63 K_mcode misses and zero Mach lowering, and the second-edit clerk repeated the same shape at 63.986s. Target shoplib/canonical.cm’s mcode hot path: write Nota integer scalars and repeated record keys directly into the existing output blob (or cache their exact scalar/key encodings) instead of allocating one nota.encode blob per scalar. Gate the change with byte-for-byte parity; sorted-key identity and stored bytes do not move.

2026-08-03 E10 — the D2 atom owns its artifact tail

Boundary ruling: unit_mcode(locator) is the per-artifact requestor and owns the whole miss journey — resolve, K_mcode probe, compile, canonical encode, content hash, and content-addressed object write. On a miss, the builder actor that compiled the unit now also encodes and writes it; the portable unit never crosses back through the clerk merely to be encoded there. The builder result’s artifact contract is exactly {key, content_hash} (beside scalar telemetry). The successful reply is the commit certificate; the clerk does not read the object back to validate work the atom just completed, and publishes K_mcode.

The shared residue is deliberately only:

• the derivation catalog edge;
• F2 single-flight bookkeeping at the realization boundary; and
• policy.

None of compile, encode, hash, or object materialization is coordinator work. The metadata edge remains monotone under a race: an incomplete direct-compile sidecar cannot replace a complete import-scan sidecar for identical unit bytes. It is artifact-internal tail state, published by the builder after writing the metadata object. Complete and compile-only sidecars use separate derived keys, and readers prefer the complete lane, so last-writer-wins catalog replacement cannot erase a completed scan.

The first post-move profile also exposed a coordinator-local multiplier rather than shared artifact work: shoplib/path.cm::join used two regex replacements per call and accounted for 3.6 GB of clerk allocation during the seed. Path joining and reverse separator scans now use direct scalar boundary scans. This does not move the closure requestor’s ownership; it removes accidental regex encoding work from the shared resolver that composes the atoms.

The first per-actor profile made the same ruling necessary for the composed unit_blob(unit_mcode(locator)) path: leaving mcode decode, Mach lowering, sidecar encoding, and object writes in the clerk still charged it for the largest replies. The lowering builder now receives the mcode content address, reads and decodes that object, lowers it, and stages the Mach payload and sidecars. It returns only mach_hash; the clerk publishes only K_mach. Thus a composition of artifact atoms preserves the boundary instead of pulling an inner artifact back through the coordinator between stages.

The object-write seam was audited before moving the work. Every writer stages to a unique temporary name and bootstrap_mv commits with the platform’s atomic rename/replace primitive. Since the destination name is the hash of the bytes, racing writers for one destination necessarily carry identical bytes; replacement is therefore logically write-once. The permanent concurrent writer regression verifies exact bytes and object_verify after two writers race the same address.

The lasting speed budget is the same boundary: one locator-to-mcode journey through the resident clerk and configured builder, measured cold and then warm. That end-to-end latency, rather than an internal compiler-only timer, is the honest unit of system speed.

The measure-first pass on exact pre-E10 dev 33b310765 did not produce a number: profiling the archive-cold seed exposed an existing unrooted-value fault in pit_text_replace and the daemon stopped with SIGSEGV after 452.314s. That is an invalid measurement, not a cold-seed result. The runtime seam is now GC-rooted and has a 2,000-replacement regression in both the ordinary and VM suites. The crash record is retained at /private/tmp/pit_e10_baseline.aIpWBQ/.pit/log/crash-1785819103.log.

The one counted post-E10 pass used executable source tree 2f765f631, an isolated git archive shop, and low-overhead all-actor profiling (100ms sample interval, 64KiB stride, depth one, allocation sampling disabled). Results:

Builder aggregation is exact rather than sample-scaled: 78 stopped builders contributed their lifecycle cpu_ms; the non-overlapping remaining 22 used the profiler’s exact per-turn accumulators (21 one-turn actors and one zero-turn actor). The permanent actor_stop.cpu_ms field exists because a short-lived builder can finish before a requested sampling profile is applied.

The placement ruling passes, but the single-digit clerk target does not: 15.34% is the measured answer. The clerk’s remaining 36.145s splits into 25.420s of callback turns, 9.308s of park turns, 1.417s of message turns, and negligible delay. Artifact encode/hash/object-write CPU is now in builders; the next coordinator reduction must attack requestor orchestration rather than silently pulling the artifact tail back across the boundary. The raw counted profile is retained at /private/tmp/pit_e10_cpu.F7uwz0/e10_seed_profile_final.json.

2026-08-04 E9 reconciliation — parity banked, fast path rejected

The candidate direct-integer and repeated-key encoder passed its identity gate: an independent copy of the old encoder produced byte-identical output for integer boundaries through +/-2^53, nested/Unicode scalar fixtures, and 12 real store artifacts (six mcode and six other), 7,376,595 bytes total. Reference time was 0.841684s, candidate cold time 0.820535s, and candidate warm time 0.837206s: only 2.51% cold and 0.53% warm. That does not pay for a native Nota-integer primitive plus actor-local key-cache state. The fast paths are therefore removed; the permanent old-vs-current byte-parity corpus remains.

The suspicious counted pass remains part of the record:

The first reconciliation question was duplication. Exact K_mcode-key maps in one prepared-boot, store-cold seed observed 132 miss keys across all emitted plans. They produced exactly 132 keyed compiler executions and 132 mcode encodes, with 132 distinct keys globally and zero redundant-key executions. Thus this seed contains no evidence of a broken claim or duplicate compile race, and no speculative claim-protocol change is taken. This is a measured workload result, not a proof that arbitrary simultaneous requestors can never race. The counted run completed in 204.48s and is retained at /private/tmp/pit_reconcile_count.EjQsCQ.

The second question was observer cost. An identical-source, identical-boot, store-cold pair with the C binary already prepared took 183.99s with plan timers, counters, key maps, and plan JSON enabled, and 183.92s with the same source under a scratch-only PIT_BUILD_TELEMETRY=0 diagnostic switch. The 0.07s / 0.04% delta is noise-sized. Telemetry did not manufacture the 301.68s result. This prepared-binary control is not a new campaign cold anchor; the landed E7/E8 anchors remain 155.34s cold and 49.35s for edit 1.

The journey/wall gap is now accounted for. The 21.051ms row compiles a fresh two-line module after the clerk and builder generation are resident, with no fleet contention, service realization, QOP construction, or native forge. In the 301.68s run, approximately 95.66s precedes the main seed process, including the cold floor and builder-generation readiness; 43 stale resident-builder closure checks span 30.73s there. The main process then spends 8.914s before the seed/bootstrap plan, 150.447s in that plan, and about 46.66s after it on QOP/final-forge work.

Inside the 150.447s seed plan, 21.621s precedes the clerk realization, 100.698s is the clerk executable plan, and about 27.76s is the serial tail of service/tool realizations. The clerk plan itself is 5.998s graph walk, a 91.097s compile window, and about 3.603s residue. Its 63 distinct units carry 2,086.005s aggregate compile elapsed and 4,370.091 builder-active seconds (47.97 average width); the 74.723s canonical figure is concurrent builder elapsed, not clerk CPU or additive wall. The earlier exact actor profile agrees with that placement: builders consumed 1,015.067 CPU-seconds, versus 36.145s in the clerk and 30.703s in the root shop actor. The missing wall is therefore resident-generation/floor readiness, contended real-unit compilation, serial realization orchestration, and final QOP/native-forge work—not duplicate K_mcode atoms and not telemetry logging.

The owed edit-2 number remains valid: its clerk plan finished in 108.447s with 69 K_mcode misses, zero Mach lowerings, and 141.553s aggregate builder canonical elapsed. The daemon survived, completed seed/bootstrap, wrote the QOP, and returned from the final forge. The original counted shop and logs are retained at /private/tmp/pit_e9_counted.WAIhYD.

E5’s source-only budget rows landed with these contracts and first samples:

• steady-smoke wall: enabled, 5.0s ceiling; a smoke that fails is a failure, and a root restart never becomes an allocation sample;
• steady-smoke allocation: named but disabled. Summing live actors before and after omits allocations by smoke actors born and reaped between snapshots, so the observable is a lower bound until the runtime exposes a lifetime total or the observer collects lifecycle allocation totals;
• hot-ps root allocation: enabled, 2 MiB ceiling; first exact back-to-back sample was 1,682,408 bytes;
• physical store composition (post-verification sample): 105,111,024 total bytes = 9,393,840 Mach + 86,556,568 mcode + 3,886,716 provenance + 5,273,900 other, with zero read errors. The composition sum is exact; a total-byte ceiling is named but disabled because this store is append-only and has no reachability/collection epoch, so physical bytes are not an honest bounded live set;
• distinct-key redundant lowering: enabled. The fresh-then-repeat probe saw one lowering under one distinct K_mach key, one warm hit, and zero repeated, unkeyed, or no-miss lowerings. Repeating a lowering without a new distinct K_mach key fails the gate.

Source: plans/archive/derivations.md