Archive — history, not state. Kept for its reasoning and its evidence; its plan is closed.

Seed speed — attribution round, 2026-08-04 night

Branch cp/seed-speed-probe, worktree .claude/worktrees/night-seedspeed, base dev @ 7c02555dd. Machine: 15 logical cores (10 P + 5 E), AC power. Other agents’ pit daemons were resident but idle during the timed runs.

Cold protocol: rm -rf .pit build cold-build pit && make, then make seed. The wipe was required — CP9.6’s reader defect is real on this tip.

Headline. The seed is not slow because the compiler is slow. It is slow for two reasons that have nothing to do with compilation quality:

  1. A runtime-global mutex is taken and dropped on every image function call. Under a parallel press, 69.6% of all sampled frames are blocked in __psynch_mutexwait/__psynch_mutexdrop beneath pit_runtime_mach_image_pin_count, against 6.0% in the interpreter. /usr/bin/time -l on one press: 181 s user, 532 s sys, 16.3 M involuntary context switches.
  2. The forge presses the whole tree three times into three throwaway shops whose derivation keys are 100% identical (149 / 149 / 149, all pairwise intersections 149). Presses 2 and 3 are a bit-for-bit repeat of press 1.

Streamline is still ~90–98% of a unit compile, and that is the honest cost — about 181 core-seconds for the whole 69 k-line closure. Everything above that is overhead.


1. End-to-end: make and make seed

Cold make (C floor + candidate binary) — 21.4 s

/usr/bin/time -l: 21.41 real, 11.18 user, 2.48 sys, 194 MB max RSS. Its one compile plan (cake/forge.ce, the cold floor):

fieldvalue
elapsed_s11.23
compile_count / distinct keys31 / 31
compile_elapsed_s (summed)83.22
fleet_average_width11.29
compile_average_width10.71
canonical_encode179 encodes / 4.69 s
store_write348 writes / 0.67 s
graph_walk31 walks / 1.12 s
mach_lower31 / 0.14 s
link_press31 / 1.45 s

make is not the problem. Note it later runs at ~0.1 s because .pit/build object caching makes the C compile free on repeat.

Cold make seed — 396.6 s

Stage timeline, from timestamped stdout (scratchpad/cold_seed.log):

stages% of 396.6
shop setup, daemon restart, realize cake/forge.ce2.50.6%
C candidate build #1 (first)0.050.0%
press-1 bootstrap (compile the seed)112.928.5%
press-1 qop pack12.83.2%
C build #2 (clean) + 23 C tests9.32.3%
press-2 bootstrap116.729.4%
press-2 qop pack10.12.5%
C build #3 (proof) + tests6.51.6%
press-3 bootstrap112.128.3%
press-3 qop pack11.72.9%
fixpoint compare, C build #4 (final), smoke, promote1.90.5%

Rolled up: press bootstraps 341.7 s (86.2%), qop packs 34.6 s (8.7%), C compile + C tests 17.9 s (4.5%), everything else 2.4 s (0.6%).

Corpus: the seed closure is 127 source files / 69,178 lines (extracted from the candidate shops’ observe.jsonl), 149 distinct mcode derivations.

Per-press telemetry (identical in all three candidate shops)

Aggregated over each shop’s 14 pit.compile.plan.telemetry@1 records:

fieldshop-1shop-2shop-3
distinct K_mcode miss keys149149149
compile_count151151151
compile_elapsed_s (summed over workers)2324.32417.22310.1
canonical_encode982 / 60.2 s982 / 59.7 s982 / 63.9 s
store_write914 / 0.63 s914 / 0.28 s914 / 0.31 s
graph_walk398 / 13.5 s398 / 15.3 s398 / 13.4 s
link_press— / 5.70 s— / 5.28 s— / 5.52 s
mach_lower— / 0.06 s— / 0.06 s— / 0.07 s
k_mcode hits / misses573 / 149573 / 149573 / 149
compile_redundant_key_count000

shop-1 ∩ shop-2 = shop-1 ∩ shop-3 = shop-2 ∩ shop-3 = 149. Union = 149.

The largest single plan is pit-shop/clerk: elapsed 78.73 s, compile_window_s 70.73, 77 units, fleet_peak 77, compile_average_width 34.0, graph_walk_elapsed_s 6.02.


2. Where a press’s CPU actually goes

Cold press into a fresh shop (proof/pit … bootstrap, 119.93 s wall), /usr/bin/time -l:

119.93 real   181.18 user   532.47 sys
3,268,149,248  maximum resident set size      (3.27 GB)
    1,819,117  page reclaims
   16,295,133  involuntary context switches
3,202,952,946,659  instructions retired
3,030,444,958,555  cycles elapsed             (IPC 1.06)

74.6% of the CPU is kernel time. A 20-second sample of the same workload (258,869 leaf samples across all threads):

leafsamples%
__psynch_mutexwait115,34744.6%
__psynch_mutexdrop48,94618.9%
mach_msg2_trap (idle threads)25,89810.0%
Pit_CallRegisterVMFunction (the interpreter)15,6576.0%
__workq_kernreturn / kevent / __psynch_cvwait (idle)38,84615.0%
pit_runtime_mach_image_pin_count (self)2,2320.9%
everything else~11,9434.6%

Excluding the idle threads, 92.8% of runnable-thread samples are mutex wait/drop. Every one of those stacks is the same shape:

actor_runner → actor_turn → Pit_ResumeVM → Pit_CallRegisterVMFunction
  → mach_exec_from_function+196
    → pit_runtime_mach_image_pin_count+44   → _pthread_mutex_firstfit_lock_slow
    → pit_runtime_mach_image_pin_count+100  → _pthread_mutex_firstfit_unlock_slow

Per compile thread this is 5,617 of 5,681 samples (98.9%) at one call site and 5,536 of 5,592 (99.0%) at another.

The code

source/mach_vm.c:438

static int mach_binding_is_pinned(PitContext *ctx, PoolBinding *binding) {
  return ctx && binding && binding->runtime == ctx->rt && binding->image &&
         pit_runtime_mach_image_pin_count(ctx->rt, binding->image) > 0;
}

called from mach_exec_from_function (static inline, on the call path of every PIT_FUNC_KIND_IMAGE function), and source/registry.c:139

uint32_t pit_runtime_mach_image_pin_count(PitRuntime *rt,
                                          const MachImageView *image) {
  if (!rt || !image) return 0;
  uint32_t count = 0;
  sys_mutex_lock(&rt->image_lock);     /* RUNTIME-GLOBAL, shared by all actors */
  for (size_t i = 0; i < rt->pinned_mach_image_count; i++) {  }
  sys_mutex_unlock(&rt->image_lock);
  return count;
}

rt->image_lock is per-runtime, not per-actor. In a fused/resident press essentially all executing code lives in images, so every call in every one of the 28–77 builder actors serializes here. It is invisible in the dev daemon because pit compile profile runs one compile at a time and the lock is uncontended.

Cross-check that this is the whole gap. 69,178 lines at the measured uncontended throughput (536–617 lines/s, §4) is ≈ 120 core-seconds of honest compile. Measured user time for a press is 181 s — the same order. The 532 s of sys is pure lock overhead. Wall per press is 113 s on a 15-core machine that should have finished 181 core-seconds in ~15 s.


3. The three-press multiplication, and the warm-press control

cake/forge.cm:press() runs candidate_press three times (press-1/2/3), each preceded by prepare_candidate_shop creating a fresh, empty shop-N/objects/blake2 + shop-N/catalog. Nothing is shared between them.

Measured, same binary + same content + same shop, back to back:

runwallusersysCPU
cold shop (fresh shop-X)119.9 s181.2 s532.5 s595%
warm shop, nothing changed (press-3-warm2)21.3 s18.1 s5.6 s111%

So a press against a warm store is 5.3× faster and burns 30× less CPU. Because the three shops’ key sets are provably identical, sharing one store would turn 341.7 s of press bootstraps into ≈ 113 + 21 + 21 = 155 s.

A second, separate finding from the same experiment

An intermediate control run (press-3-warm) executed the identical command against warm shop-3 after make seed promoted the new boot.qop, boot/root.cart and ./pit into the workspace. It took 113 s and its 108 K_mcode keys had zero overlap with the original press-3’s 153 keys. The next run (nothing changed) hit and took 21.3 s. So the seed’s own promotion step invalidates every downstream compile key — the derivation key is sensitive to a workspace artifact that is not compiler input. Worth its own row; it means “reseed then rebuild” always pays full price.


4. Phase split inside one unit compile (uncontended, dev daemon)

pit compile profile <file> --no-pass-timings:

filelinestotaltokenizeparserequirementsfoldmcodestreamlineresidual
pit-compiler/callresolve.cm7991490.5 ms32.513.76.75.026.81461.2 (98.0%)−55.3
pit-compiler/streamline.cm54618855.3 ms152.6111.669.944.4213.77952.1 (89.8%)311.1

Throughput 536–617 lines/s. Front end (tokenize + parse + requirements + fold + mcode) is 5–7%. Canonical encode, store writes, mach lower and graph walk are all sub-3% of aggregate builder time (§1 table). The only phase worth optimizing inside the compiler is streamline.

Streamline outer phases

pit-compiler/streamline.cm (7952 ms region):

phasems%
initial_optimize_and_lower2728.334.3%
detail_initial_optimize_functions1904.4(nested)
detail_initial_lower_and_stone803.8(nested)
range_reconciliation1779.322.4%
checked_store_facts498.06.3%
compress_slots479.36.0%
inline_round_1379.74.8%
switch327.54.1%
compact_final_nops248.73.1%
decode_ir206.72.6%
record_shape_facts201.32.5%
call_resolution193.32.4%
licm145.61.8%
encode_ir94.41.2%
mark_closure_writes76.01.0%
final_int_ranges47.30.6%
tco4.20.1%

callresolve.cm agrees on shape: initial_optimize_and_lower 482.9 (33.0%), range_reconciliation 375.4 (25.7%), licm 133.9 (9.2%).

CP9.1 is confirmed landed and working. Counters: streamline.cmparent_map_cache_hits 69, parent_map_rebuilds 12, detail_inline_run_make_ctx_parent_map 52.9 ms of 7952 ms. callresolve.cm — hits 14, rebuilds 2, parent map 0.44 ms. The 5,500 ms / 44% line item from the 2026-08-04 measurement round is gone.

The new #2 is range_reconciliation (pit-compiler/streamline.cm:5331-5385). Per function containing any int-overflow op it runs analyze_local_arraysinfer_slot_write_types(optimistic)int_ranges.inferinfer_slot_write_types(sound)eliminate_type_checks, and int_ranges then runs a third time in final_int_ranges.


5. The double-parse: refuted where the plan said, confirmed elsewhere

Refuted in mcode_unit_result. pit-compiler/compiler.cm:1623 calls parse_result exactly once and hands the parse to compile_result_from(parsed, …), which threads it through mcode_result_fromanalyze_result_from without re-tokenizing. The in-code comment says so and the call graph agrees (compiler.cm:1580, :361, :250, :215). Nothing parses twice inside a unit compile.

Confirmed in the closure walk. pit-shop/shop_fetch.cm:458, inside analyze_locator, runs compiler.requirements_result(source_text, locator, …) on every source file to discover imports. That is a full tokenize + parse + analysis.collect. Minutes later the builder fleet tokenizes and parses the identical text again. It is cached under mcode_cache_key, so a fresh candidate shop pays it once per file per press.

Cost, measured: graph_walk_elapsed_s in the pit-shop/clerk plan is 6.02 s of that plan’s 78.73 s wall (7.6%), and it is serial — the plan’s compile_window_s is 70.73 s, i.e. the 8 s difference is the walk running before any compile is dispatched. Per press the summed graph walk is 13.4–15.3 s; across the seed ≈ 40 s (10%). The scan’s parallel is cooperative within one actor (shop_fetch.cm:508, drain()), so it uses one core.

Sanity check against the per-unit numbers: tokenize + parse + requirements is 52.8 ms for callresolve.cm’s 799 lines → ~4.6 s for 69 k lines single-threaded. That is the 6 s.


6. CP9.5 — the July 10→14 streamline regression

git log --since=2026-07-09 --until=2026-07-16 -- pit-compiler/streamline.cm returns 24 commits. The decisive one is visible in file history rather than in streamline.cm itself:

$ git log --follow --pretty='%h %ad %s' -- pit-compiler/callresolve.cm
…
0247a4147 2026-07-11 inliner: shared callee resolution (upvalue + reuse-robust)
                     + hot-size + call facts        ← file created here

callresolve.cm did not exist before 2026-07-11. It is the file that introduced make_ctx’s per-child rebuild of the parent’s fn_slot_fids map — the loop CP9.1 measured at 5,767 ms / ~44% of a 13.1 s vm_suite unit compile. 0247a4147 is the guilty change for the bulk of the 1.0 s → 5.4 s move, and it is already fixed on dev (a6a73b60f, verified live by the counters in §4).

What is left of the 5.4× is not one commit. The residue is the optimizer campaign’s new analyses, which now dominate: range_reconciliation (22–26%) rests on 4215f553b (integer range facts, 2026-07-09), 142e4a172 (ranges in large functions), 0890f148d (immutable parameter facts) and f3bad5657/ddd093d27 (guarded/fused array stores), all inside the window. Those are optimizations with output value, so the row becomes “make them cheaper”, not “find the regression” — the hunt is closed.


7. Redundancy and scheduler answers

Are units recompiled that shouldn’t be? Inside a press: no. compile_redundant_key_count = 0, mcode_encode_redundant_count = 0, mach_lower_redundant_count = 0 in all three shops. Across presses: everything — 149 identical keys × 3.

Is canonical encoding run more than once per unit? 982 encodes for 151 compiles (6.5 per compile), but zero redundant keyed encodes — the extra encodes are manifests, identity arms and metadata records, not repeats of the same value. Summed across parallel workers it is ~60 s per press against ~2,324 s of summed compile: 2.6%. Not a target.

Are store writes synchronous on the compile path? Yes — builder_worker.ce’s tail calls shop_store.stage_mcode_unit inline before replying. It costs 914 writes / 0.28–0.63 s per press. Negligible; leave it alone.

Does the fleet saturate cores? It over-saturates. pit-shop/shop_build.cm:1710 defaults width to null and :1368 calls parallel(jobs, state.width) — unbounded. In the clerk plan fleet_peak = 77 = compile_count (every unit dispatched at once) with compile_average_width 34.0 on 15 logical cores. Peak RSS 3.27 GB. This is not an occupancy indictment — it is the fuel for the §2 lock contention.


8. Ranked fixes

#fixest. winriskkindfiles
1Get pit_runtime_mach_image_pin_count off the per-call pathpress 113 s → ~25–40 s; seed 396 → ~150 smediummechanical + one design choicesource/mach_vm.c, source/registry.c, source/pit_internal.h
2One shared object store across the three candidate presses342 s → ~155 s (−187 s, 47% of seed)highneeds design (gate semantics)cake/forge.cm
3Make range_reconciliation cheaper~20% of every compilemediumneeds design (soundness)pit-compiler/streamline.cm, pit-compiler/int_ranges.cm
4Cap fleet width at core countsmall alone; removes 3.3 GB RSS + 16.3 M ctx switcheslowmechanicalpit-shop/shop_build.cm
5Stop parsing every file twice (closure scan)~6 s serial/press, ~40 s of seed (10%)mediumneeds designpit-shop/shop_fetch.cm, pit-compiler/compiler.cm
6Instrument detail_initial_optimize_functionsunknown; it is 24% of streamline unattributednonemechanical (measurement)pit-compiler/streamline.cm
7Workspace promotion invalidates all compile keysone full extra rebuild per reseedmediumneeds diagnosisderivation-key inputs, pit-shop/shop_store.cm

1 — the image-pin lock. Mechanical, with one design choice.

mach_binding_is_pinned is a safety check: it makes a function that outlived its mapping fail as an invalid cursor instead of reading freed address space. The check must survive; only the lock has to go. Options, cheapest first:

  • Stamp the pin state onto the PoolBinding at pin/unpin time — both already hold rt->image_lock and both already own the range array — and have mach_binding_is_pinned read the stamp with a relaxed atomic load.
  • Or make pin_count an atomic on the range and index it from the binding, so the read is one atomic load and no scan.

Either keeps the invariant “a call into an unpinned image refuses”. Regressions already exist: source/mach_image_lifetime_test.c, source/mach_pool_test.c:1285. Do not weaken the check to buy the win. Expected: user time unchanged (~181 s), sys time collapses from 532 s, wall per press falls toward the 15–25 s the honest work implies.

2 — the three-press store. Needs a ruling before code.

Measured fact that should drive the ruling: all three presses derive the identical 149 K_mcode keys, so today’s gate is already not proving “two different binaries compile the same way” — the compiler identity is content-derived from the compiler sources, which do not change between candidates. What the fixpoint gate actually proves is that the press/pack machinery is stable. If that is the intended claim, one shared content-addressed store across shop-1/2/3 is sound (objects are addressed by their canonical bytes; a hit cannot silently change the answer) and buys 187 s. If the claim is meant to be stronger, the alternative is to cut to two presses or share only press-1 → press-2. cake/forge.cm:1488-1600 is the whole seam; prepare_candidate_shop (:1170) is what creates the empty store.

3 — range reconciliation. Needs design.

pit-compiler/streamline.cm:5331. Per qualifying function it runs the write-type inference twice and int_ranges.infer once, then final_int_ranges runs the range analysis a third time over the same function at :5400. The second inference is what makes the int facts sound (the first is optimistic), so it cannot simply be deleted — the target is sharing one analysis result between the reconcile pass and final_int_ranges, and skipping functions whose optimistic and conservative tables already agree. Soundness bar is the int_ranges precedent: differential fuzz on/off must agree, plus corpus byte identity.

4 — fleet width. Mechanical.

pit-shop/shop_build.cm:1710 — default width to the core count instead of null. This alone will not make the seed fast (fix 1 is what does), but it removes the 3.27 GB peak and the 16.3 M context switches, and it is a one-line change with an existing plumbed parameter.

5 — the closure-scan parse. Needs design.

pit-shop/shop_fetch.cm:458. Either the scan’s parse travels to the builder that will compile the same text, or the scan stops being a full parse (an import-only scan over the token stream). The BFS in drain() depends on the scan’s results to expand the frontier, so the walk cannot simply be moved onto the fleet without restructuring. 10% of the seed; take it after 1 and 2.

7 — the promotion-invalidates-everything row.

Diagnosed but not chased: after make seed promoted boot.qop, boot/root.cart and ./pit, an otherwise-identical press re-missed 108/108 keys with zero overlap against its own previous run. The next run hit. mcode_cache_key (pit-shop/shop_store.cm:97) folds compiler_identity, which is the builder generation identity, which is derived from the builder executable’s source closure — something in that derivation is picking up a promoted artifact. Find it: it costs a full extra recompile on every post-reseed build.


Reproduction

  • scratchpad/cold_make.log, scratchpad/cold_seed.log — timestamped stage logs.
  • scratchpad/parse_plans.py — extracts pit.compile.plan.telemetry@1 from observe.jsonl.
  • scratchpad/press_sample2.txt — 20 s sample of a cold press.
  • Candidate-shop telemetry survives at build/forge-press/shop-{1,2,3}/log/observe.jsonl*.
  • Warm/cold press control:
    L=$PWD/build/forge-press
    PIT_INPROCESS=1 PIT_CONTENT=$L/press-2/boot.qop \
      $L/proof/pit --shop $L/shop-3 --content $L/press-2/boot.qop \
      bootstrap --out $L/press-3-warm2
    

Image-pin lock fix — landed, cp/pin-lock @ 51d250f4e (base dev @ 0c6f6aacd)

Row 1 of §8 is done. mach_binding_is_pinned no longer takes rt->image_lock.

The design, and why it is the sound one

Three shapes were on the table. What ruled between them is the lifetime of the thing the check reads, which the provider fixes exactly:

  • A PoolBinding lives inside a PitMachPoolBindingHandle (source/image_provider.c:41). provider_binding_for_placement (:556) acquires a pin before it builds the handle, and handle->active is never cleared for a pool binding — only provider_close_pin (:788) clears active, and that is the pin class, not the binding class. So a live binding always holds its own pin, and provider_detach_unused_locked (:127) cannot free the placement while pin_refs is non-zero.
  • Therefore, wherever binding is safe to dereference, binding->image — which is &placement->view, storage owned by the runtime, not by the read-only mapping — is safe to dereference too.

That killed the “cache pinned-ness on the binding” option (b): the direct-binding regressions construct a stack PoolBinding and pin the view separately (source/mach_pool_test.c:1277-1285, and the same shape at :489 :544 :1387 :1445 :1524 :1798 :2001), so a binding-scoped flag would answer a different predicate and those tests would have had to be rewritten to keep passing. It also killed a seqlock over the registry: a press pins ~149 images, so a lock-free linear scan is worse single-threaded than the lock it replaces. The answer has to be O(1) and it has to hang off the image pointer.

Chosen: a mirror word on MachImageView.

/* source/pit_internal.h — in MachImageView */
_Atomic uintptr_t pin_owner;   /* the runtime holding >=1 pin, or 0 */

registry.c owns every write, all of them under rt->image_lock, beside the authoritative range->pin_count they mirror: published on both pin success paths, retracted when unpin drops the count to 0. pit_runtime_mach_image_pin_count is untouched and still locked — its cold callers (the C regressions, dbg_code_from_function at source/runtime.c:10021) keep the runtime-scoped counting read. Only the hot gate changed:

/* source/mach_vm.c:438 */
return ctx && binding && binding->runtime == ctx->rt && binding->image &&
       pit_mach_image_pinned_by(binding->image, ctx->rt);

Diff: 53 insertions, 1 deletion, across three files.

The memory-ordering argument

  • Publication. The store is memory_order_release, made under rt->image_lock and after the placement the view describes is fully built. The read is memory_order_acquire. A reader that sees this runtime therefore also sees the section table it is about to read. Nothing is published by the mirror that was not already published by the lock.
  • The drain race is not new. A pin can drop to zero the instant after the load, so the gate can read “held” for a moment after it stopped being true. That was equally true of the locked version: pit_runtime_mach_image_pin_count unlocks before it returns, so its caller always acted on a snapshot another thread could already have invalidated. The lock bought data-race-freedom on the array scan and nothing else; the scan is gone, so there is nothing to race on.
  • What actually keeps a running image alive is ownership, not this check. An image function traces its owner, the owner traces the binding handle, and the handle holds a pin for its whole life. The count cannot fall to zero underneath a call in progress. The gate is the belt on those braces and it keeps exactly the strength it had — an unpinned binding still refuses, as direct_execution_test asserts before and after.
  • Runtime scoping. The mirror names the runtime, not just a count, so pinned into rt is one comparison rather than an argument. Two runtimes pinning the same view object would clobber each other’s owner — that cannot happen for provider views, which are embedded in a per-runtime placement list, and no test does it. If it ever did, the loser fails closed (refuses to run), never open.
  • Initialization. Pit_MachPoolOpen memsets the view (source/mach_pool.c:507,510), so a fresh view is unpinned; provider_adopt_mapping copies only freshly opened views, never a pinned one.

Before / after

15 logical cores, AC power. Caveat that matters: other night agents were building throughout, load average 12–17 in both arms — so the absolute numbers are higher than the §2 attribution round’s (which had a quiet machine). The A/B is paired, back to back, same worktree, same protocol.

Cold pressproof/pit … bootstrap into a shop whose objects/blake2 and catalog were emptied, PIT_INPROCESS=1, /usr/bin/time -l:

metricbefore (dev)after (fix)Δ
wall330.50 s224.29 s−32.1%
user385.71 s310.26 s−19.6%
sys135.24 s31.18 s−77.0%
involuntary ctx switches8,605,3411,105,510−87.2%
voluntary ctx switches3,2602,430−25.5%
instructions retired8.572e127.531e12−12.1%
cycles elapsed2.152e121.391e12−35.4%
page reclaims14,277,80313,067,455−8.5%
max RSS7.43 GB9.16 GB+23%

RSS goes up on purpose: the fleet was previously blocked, so it never got wide. Row 4 of §8 (cap fleet width) is now the row that pays, not the row that was masked.

Cold make seed (whole three-press seed, full rm -rf .pit build cold-build pit first):

beforeafterΔ
wall607.75 s386.44 s−221.3 s, −36.4%

(/usr/bin/time -l cannot attribute CPU here — the presses are daemon-spawned, not children of make. Wall is the honest column.)

Cold make (C floor + candidate binary; contains no press, so no change is expected — this is the no-regression control):

beforeafter
real37.13 s15.32 s
user11.52 s11.55 s
sys2.43 s2.67 s

User and sys are identical; the real-time gap is machine load, not the fix.

10 s sample of the in-seed press, t = 75 s into make seed, all threads:

leafbefore%after
__psynch_mutexwait61,96448.4%not in top 15
__psynch_mutexdrop20,51316.0%not in top 15
mach_msg2_trap (idle)12,81410.0%15,043
Pit_CallRegisterVMFunction7,3285.7%5,029
__workq_kernreturn (idle)6,4075.0%14,325
kevent (idle)6,4075.0%7,519
__psynch_cvwait (idle)5,9514.6%107,966
pit_runtime_mach_image_pin_count (self)1,1770.9%0

pit_runtime_mach_image_pin_count appears 165 times anywhere in the baseline sample’s call tree and zero times in the fixed one.

Deeper into a cold press (t = 100 s), the top non-idle leaf is now the interpreter itself, which is what §2 said the honest workload should look like:

__psynch_cvwait (idle)            89,007
Pit_CallRegisterVMFunction        14,509   <- the interpreter
mach_msg2_trap                    14,454
__workq_kernreturn                 7,225
kevent                             7,225
pit_key_equal                        886
mach_resolve_outer_frame             854
pit_mach_record_get_hashed           529

Verification

gateresult
scripts/gate.sh "vm suite" ./pit test run tests/vm_suite.ceOK
./pit test suite475 passed, 0 failed
./pit fuzz 1002,466 cases, 0 failed (opt/unopt 820, linked/unlinked 826 + shapes 6, join_types 820)
./pit test run tests/compile.cm187 passed, 0 failed
ARMS="force-gc" scripts/check_arms.sh18 passed, 5 skipped, 0 failed — ALL GREEN
ARMS="validate-gc" scripts/check_arms.sh18 passed, 5 skipped, 0 failed — ALL GREEN
two concurrent ./pit test suite against one daemon475/0 and 475/0

The force-gc and validate-gc arms are the ones that matter for this change: they build and run the C regressions — mach_pool_test, mach_image_lifetime_test, resident_image_provider_test — which are exactly the pin-lifetime tests §8 said not to weaken. direct_execution_test still proves an unpinned binding refuses.

What this moves in §8

Row 1 is closed. With the lock gone the seed’s remaining shape is what §1 and §3 predicted: three presses of identical work (row 2, still the biggest single win at ~187 s), streamline inside a unit compile (row 3), and an unbounded fleet that now actually spends the cores it takes (row 4, whose RSS cost is visible above).

Reseed identity + width

Branch cp/reseed-identity, worktree .claude/worktrees/night-identity, base dev @ 0c6f6aacd. Cold protocol: rm -rf .pit build cold-build pit && make, then make seed. Three to five other agents’ seeds ran concurrently on the same 15-core machine throughout, so wall times are contended; they are only ever compared against another run from the same session. Key counts are not affected by contention.

Row 7 reproduces, and it is not the promotion

Row 7 read “workspace promotion invalidates all compile keys”. The 108/108 zero-overlap re-miss is real and reproduces exactly — but on a reseed that promoted byte-identical artifacts, so the promotion cannot be the cause.

The leaked field is compiler_identity, and what leaks into it is which of two boot lanes pinned the builder.

pit-shop/clerk.ce pins one generation two ways:

clerk bootpathwhat K_mcode gets
its own source visiblepin_generation_builder -> builder_identity.for_bundlethe D3 slice: hash over {locator, source_hash} of the COMPILE sub-closure — bea2ba88…
its own source not visiblepin_resident_generation_builderthe coarse whole-closure hash (resident_derivation.source_closure, i.e. the builder’s executable-MANIFEST hash) handed to both halves — 573dd10e…

Which lane a shop takes is not a property of the shop. A fresh shop’s first boot is source-less, because the runtime mount opens behind the clerk (shop_actor.ce starts boot-image services, then open_content); every later boot into that same directory is not. Straight out of one make seed, one directory, two consecutive processes:

build/forge-press/shop-3/log/boot.log   (press-3 bootstrap, FIRST boot)
  clerk source-less resident      blake2:bc39ebb6…
  resident builder generation     blake2:573dd10e…          <- coarse, BOTH halves

build/forge-press/shop-3/log/boot.log   (the qop pack, second boot)
  clerk resident current          blake2:bc39ebb6…
  builder generation blake2:573dd10e…
    compiler blake2:bea2ba88…  lowering blake2:ae2d7f56…     <- sliced

Same binary, same tree, same generation NAME — two compiler_identity values. Everything the first boot compiled is unreachable to every later boot.

Measurement — before

The reseed was verified to be a no-op: forge: two clean boot presses are byte-identical, and boot/root.cart (md5 454612a7…) and boot.qop (md5 397cd591…) were unchanged across it. press-1, press-2 and press-3 all produced the committed root.cart byte-for-byte, so the press is deterministic and the tree was already at its fixpoint.

Three presses into the same warm shop-3, same binary (build/forge-press/proof/pit), same content (press-2/boot.qop):

pressboot laneidentity fed to K_mcodedistinct K_mcode misseshitswallmax RSSinvol. ctx switchesfleet_peak
press-3 (1st boot, fresh shop)source-lesscoarse 573dd10e149573in-seed77
warm A (2nd boot)sourcesliced bea2ba88112596289.3 s5.94 GB7.82 M74
warm B (3rd boot)sourcesliced bea2ba88068435.8 s6.52 GB0.66 M2

press-3 ∩ warm A = 0 keys. Warm A is a full recompile of what press-3 had just finished compiling — 289 s thrown away because the two boots keyed it differently. Warm B is what a warm press costs when the keys line up: 36 s.

That is row 7’s 153-then-108-with-zero-overlap, reproduced and explained. It costs one full extra compile of the tree the second time any shop is used, and make seed creates three fresh shops.

The fix

The obvious fix — slice at boot in the resident lane — does not work, and that was measured rather than assumed: a fused shop cannot read the builder’s executable-manifest OBJECT. It is neither in the shop’s own store nor in the content bundle’s object layer; object_get_value on 573dd10e… returns null even in a fully booted daemon with content mounted. A first attempt shipped exactly that and logged (one coarse identity) on every press.

The manifest is readable in exactly one place: at press time, in the shop that has just realized the executable (the object lands 9 s before the cart is cemented). So the slice is computed there and carried in the cart:

  • pit-shop/boot_cart.cmcemented_identities(exe) -> {compiler, lowering} per exec, using builder_identity.for_bundle.
  • pit-shop/cement.cmcemented_derivation(exe, identities) merges them into the resident row’s derivation record. This module computes nothing: builder_identity reaches shop_store, and cement.cm is in shop_actor’s closure — the root actor must not grow the store to carry a fact the press already knows. boot_cart.cm is press-side and already reaches it.
  • pit-shop/builder_identity.cmfor_resident(derivation, coarse) reads them back, falling back to reading the manifest, and failing that to the coarse identity in BOTH halves. A cart pressed before this landed behaves exactly as it did: over-invalidating, never stale.
  • pit-shop/clerk.cepin_resident_generation_builder uses it.

Measurement — after

press-3’s first boot now logs the slice on the source-less lane:

shop resident: clerk source-less resident blake2:c2c31de5…
shop resident: resident builder generation blake2:573dd10e…
  compiler blake2:bea2ba88…  lowering blake2:ae2d7f56…

Same protocol, same shop, straight after the seed:

beforeafter
distinct K_mcode misses, warm press after a cold press1120
K_mcode hits596708
overlap with the cold press’s key set0 of 149n/a — nothing missed
wall289.3 s32.0 s
involuntary context switches7.82 M0.67 M
max RSS5.94 GB6.48 GB

9.0x on the wall for the second use of any shop, and the 289 s of duplicate compile is gone entirely.

Regression tests

  • tests/builder_identity.ceresident_lane_checks(): one generation pinned on either lane yields ONE answer; a lowering-only builder change leaves compiler_identity alone through the resident lane; both fallbacks (no cemented slice with an unreadable manifest -> coarse; no cemented slice with a readable one -> the read) hold.
  • tests/store_freshness.cegeneration_lane_cache_checks(), counter-based on pit.compile.plan.telemetry@1: compile one source with the builder pinned the way a first boot pins it (asserting k_mcode_misses == 1), then pinned the way every later boot pins it, and assert that plan reports k_mcode_hits == 1, k_mcode_misses == 0. The source carries a guid so the first compile is always cold and the second one’s hit means what it says.

Fix 2 — fleet width clamp

pit-shop/shop_build.cm defaulted state.width to null, and null is parallel’s UNBOUNDED. The default is now the core count from runtime::internal/sysinfo’s ncpus() (internal/sysinfo.c -> sys_cpu_count), read once and memoised. An explicit width on the request still wins, and a runtime that cannot answer keeps the historic unbounded behaviour rather than guessing. Same change at pool_derivation’s parallel. No oversubscription factor, no per-plan heuristic — a clamp, not a knob.

Paired presses, same corpus and same miss count, one binary apart:

presswidth defaultfleet_peakdistinct missessummed compile
seed A shop-3unbounded77149272.0 s
seed B shop-2ncpus() = 1515149271.1 s

Peak concurrency 77 -> 15 at no throughput cost (the summed compile time is a wash). On the warm-press pair above, involuntary context switches fall 7.82 M -> 0.67 M, though that pair also differs in how much it compiles.

Seed wall times are reported for completeness only and are not comparable — every run shared the machine with other agents’ seeds: seed A 9:45, seed C 10:56.

What this does NOT show

A byte-identical reseed does not, by itself, move compiler_identity: the coarse hash is the builder’s manifest hash and it was unchanged (573dd10e…) across all three seeds in this session. Row 7’s “workspace promotion” framing was a misattribution of the boot-lane flip; nothing needs to be fixed about promotion.

Source: plans/archive/night-2026-08-04/seed-speed.md