Archive — history, not state. Kept for its reasoning and its evidence; its plan is closed.
Seed speed — attribution round, 2026-08-04 night
Branch cp/seed-speed-probe, worktree .claude/worktrees/night-seedspeed, base
dev @ 7c02555dd. Machine: 15 logical cores (10 P + 5 E), AC power. Other
agents’ pit daemons were resident but idle during the timed runs.
Cold protocol: rm -rf .pit build cold-build pit && make, then make seed.
The wipe was required — CP9.6’s reader defect is real on this tip.
Headline. The seed is not slow because the compiler is slow. It is slow for two reasons that have nothing to do with compilation quality:
- A runtime-global mutex is taken and dropped on every image function
call. Under a parallel press, 69.6% of all sampled frames are blocked in
__psynch_mutexwait/__psynch_mutexdropbeneathpit_runtime_mach_image_pin_count, against 6.0% in the interpreter./usr/bin/time -lon one press: 181 s user, 532 s sys, 16.3 M involuntary context switches. - The forge presses the whole tree three times into three throwaway shops whose derivation keys are 100% identical (149 / 149 / 149, all pairwise intersections 149). Presses 2 and 3 are a bit-for-bit repeat of press 1.
Streamline is still ~90–98% of a unit compile, and that is the honest cost — about 181 core-seconds for the whole 69 k-line closure. Everything above that is overhead.
1. End-to-end: make and make seed
Cold make (C floor + candidate binary) — 21.4 s
/usr/bin/time -l: 21.41 real, 11.18 user, 2.48 sys, 194 MB max RSS.
Its one compile plan (cake/forge.ce, the cold floor):
| field | value |
|---|---|
| elapsed_s | 11.23 |
| compile_count / distinct keys | 31 / 31 |
| compile_elapsed_s (summed) | 83.22 |
| fleet_average_width | 11.29 |
| compile_average_width | 10.71 |
| canonical_encode | 179 encodes / 4.69 s |
| store_write | 348 writes / 0.67 s |
| graph_walk | 31 walks / 1.12 s |
| mach_lower | 31 / 0.14 s |
| link_press | 31 / 1.45 s |
make is not the problem. Note it later runs at ~0.1 s because
.pit/build object caching makes the C compile free on repeat.
Cold make seed — 396.6 s
Stage timeline, from timestamped stdout (scratchpad/cold_seed.log):
| stage | s | % of 396.6 |
|---|---|---|
shop setup, daemon restart, realize cake/forge.ce | 2.5 | 0.6% |
C candidate build #1 (first) | 0.05 | 0.0% |
| press-1 bootstrap (compile the seed) | 112.9 | 28.5% |
| press-1 qop pack | 12.8 | 3.2% |
C build #2 (clean) + 23 C tests | 9.3 | 2.3% |
| press-2 bootstrap | 116.7 | 29.4% |
| press-2 qop pack | 10.1 | 2.5% |
C build #3 (proof) + tests | 6.5 | 1.6% |
| press-3 bootstrap | 112.1 | 28.3% |
| press-3 qop pack | 11.7 | 2.9% |
fixpoint compare, C build #4 (final), smoke, promote | 1.9 | 0.5% |
Rolled up: press bootstraps 341.7 s (86.2%), qop packs 34.6 s (8.7%), C compile + C tests 17.9 s (4.5%), everything else 2.4 s (0.6%).
Corpus: the seed closure is 127 source files / 69,178 lines (extracted from
the candidate shops’ observe.jsonl), 149 distinct mcode derivations.
Per-press telemetry (identical in all three candidate shops)
Aggregated over each shop’s 14 pit.compile.plan.telemetry@1 records:
| field | shop-1 | shop-2 | shop-3 |
|---|---|---|---|
| distinct K_mcode miss keys | 149 | 149 | 149 |
| compile_count | 151 | 151 | 151 |
| compile_elapsed_s (summed over workers) | 2324.3 | 2417.2 | 2310.1 |
| canonical_encode | 982 / 60.2 s | 982 / 59.7 s | 982 / 63.9 s |
| store_write | 914 / 0.63 s | 914 / 0.28 s | 914 / 0.31 s |
| graph_walk | 398 / 13.5 s | 398 / 15.3 s | 398 / 13.4 s |
| link_press | — / 5.70 s | — / 5.28 s | — / 5.52 s |
| mach_lower | — / 0.06 s | — / 0.06 s | — / 0.07 s |
| k_mcode hits / misses | 573 / 149 | 573 / 149 | 573 / 149 |
| compile_redundant_key_count | 0 | 0 | 0 |
shop-1 ∩ shop-2 = shop-1 ∩ shop-3 = shop-2 ∩ shop-3 = 149. Union = 149.
The largest single plan is pit-shop/clerk: elapsed 78.73 s,
compile_window_s 70.73, 77 units, fleet_peak 77, compile_average_width
34.0, graph_walk_elapsed_s 6.02.
2. Where a press’s CPU actually goes
Cold press into a fresh shop (proof/pit … bootstrap, 119.93 s wall),
/usr/bin/time -l:
119.93 real 181.18 user 532.47 sys
3,268,149,248 maximum resident set size (3.27 GB)
1,819,117 page reclaims
16,295,133 involuntary context switches
3,202,952,946,659 instructions retired
3,030,444,958,555 cycles elapsed (IPC 1.06)
74.6% of the CPU is kernel time. A 20-second sample of the same workload
(258,869 leaf samples across all threads):
| leaf | samples | % |
|---|---|---|
__psynch_mutexwait | 115,347 | 44.6% |
__psynch_mutexdrop | 48,946 | 18.9% |
mach_msg2_trap (idle threads) | 25,898 | 10.0% |
Pit_CallRegisterVMFunction (the interpreter) | 15,657 | 6.0% |
__workq_kernreturn / kevent / __psynch_cvwait (idle) | 38,846 | 15.0% |
pit_runtime_mach_image_pin_count (self) | 2,232 | 0.9% |
| everything else | ~11,943 | 4.6% |
Excluding the idle threads, 92.8% of runnable-thread samples are mutex wait/drop. Every one of those stacks is the same shape:
actor_runner → actor_turn → Pit_ResumeVM → Pit_CallRegisterVMFunction
→ mach_exec_from_function+196
→ pit_runtime_mach_image_pin_count+44 → _pthread_mutex_firstfit_lock_slow
→ pit_runtime_mach_image_pin_count+100 → _pthread_mutex_firstfit_unlock_slow
Per compile thread this is 5,617 of 5,681 samples (98.9%) at one call site and 5,536 of 5,592 (99.0%) at another.
The code
source/mach_vm.c:438
static int mach_binding_is_pinned(PitContext *ctx, PoolBinding *binding) {
return ctx && binding && binding->runtime == ctx->rt && binding->image &&
pit_runtime_mach_image_pin_count(ctx->rt, binding->image) > 0;
}
called from mach_exec_from_function (static inline, on the call path of
every PIT_FUNC_KIND_IMAGE function), and source/registry.c:139
uint32_t pit_runtime_mach_image_pin_count(PitRuntime *rt,
const MachImageView *image) {
if (!rt || !image) return 0;
uint32_t count = 0;
sys_mutex_lock(&rt->image_lock); /* RUNTIME-GLOBAL, shared by all actors */
for (size_t i = 0; i < rt->pinned_mach_image_count; i++) { … }
sys_mutex_unlock(&rt->image_lock);
return count;
}
rt->image_lock is per-runtime, not per-actor. In a fused/resident press
essentially all executing code lives in images, so every call in every one of
the 28–77 builder actors serializes here. It is invisible in the dev daemon
because pit compile profile runs one compile at a time and the lock is
uncontended.
Cross-check that this is the whole gap. 69,178 lines at the measured uncontended throughput (536–617 lines/s, §4) is ≈ 120 core-seconds of honest compile. Measured user time for a press is 181 s — the same order. The 532 s of sys is pure lock overhead. Wall per press is 113 s on a 15-core machine that should have finished 181 core-seconds in ~15 s.
3. The three-press multiplication, and the warm-press control
cake/forge.cm:press() runs candidate_press three times (press-1/2/3),
each preceded by prepare_candidate_shop creating a fresh, empty
shop-N/objects/blake2 + shop-N/catalog. Nothing is shared between them.
Measured, same binary + same content + same shop, back to back:
| run | wall | user | sys | CPU |
|---|---|---|---|---|
cold shop (fresh shop-X) | 119.9 s | 181.2 s | 532.5 s | 595% |
warm shop, nothing changed (press-3-warm2) | 21.3 s | 18.1 s | 5.6 s | 111% |
So a press against a warm store is 5.3× faster and burns 30× less CPU. Because the three shops’ key sets are provably identical, sharing one store would turn 341.7 s of press bootstraps into ≈ 113 + 21 + 21 = 155 s.
A second, separate finding from the same experiment
An intermediate control run (press-3-warm) executed the identical command
against warm shop-3 after make seed promoted the new boot.qop,
boot/root.cart and ./pit into the workspace. It took 113 s and its 108
K_mcode keys had zero overlap with the original press-3’s 153 keys. The
next run (nothing changed) hit and took 21.3 s. So the seed’s own promotion
step invalidates every downstream compile key — the derivation key is
sensitive to a workspace artifact that is not compiler input. Worth its own
row; it means “reseed then rebuild” always pays full price.
4. Phase split inside one unit compile (uncontended, dev daemon)
pit compile profile <file> --no-pass-timings:
| file | lines | total | tokenize | parse | requirements | fold | mcode | streamline | residual |
|---|---|---|---|---|---|---|---|---|---|
pit-compiler/callresolve.cm | 799 | 1490.5 ms | 32.5 | 13.7 | 6.7 | 5.0 | 26.8 | 1461.2 (98.0%) | −55.3 |
pit-compiler/streamline.cm | 5461 | 8855.3 ms | 152.6 | 111.6 | 69.9 | 44.4 | 213.7 | 7952.1 (89.8%) | 311.1 |
Throughput 536–617 lines/s. Front end (tokenize + parse + requirements + fold + mcode) is 5–7%. Canonical encode, store writes, mach lower and graph walk are all sub-3% of aggregate builder time (§1 table). The only phase worth optimizing inside the compiler is streamline.
Streamline outer phases
pit-compiler/streamline.cm (7952 ms region):
| phase | ms | % |
|---|---|---|
initial_optimize_and_lower | 2728.3 | 34.3% |
├ detail_initial_optimize_functions | 1904.4 | (nested) |
└ detail_initial_lower_and_stone | 803.8 | (nested) |
range_reconciliation | 1779.3 | 22.4% |
checked_store_facts | 498.0 | 6.3% |
compress_slots | 479.3 | 6.0% |
inline_round_1 | 379.7 | 4.8% |
switch | 327.5 | 4.1% |
compact_final_nops | 248.7 | 3.1% |
decode_ir | 206.7 | 2.6% |
record_shape_facts | 201.3 | 2.5% |
call_resolution | 193.3 | 2.4% |
licm | 145.6 | 1.8% |
encode_ir | 94.4 | 1.2% |
mark_closure_writes | 76.0 | 1.0% |
final_int_ranges | 47.3 | 0.6% |
tco | 4.2 | 0.1% |
callresolve.cm agrees on shape: initial_optimize_and_lower 482.9 (33.0%),
range_reconciliation 375.4 (25.7%), licm 133.9 (9.2%).
CP9.1 is confirmed landed and working. Counters:
streamline.cm — parent_map_cache_hits 69, parent_map_rebuilds 12,
detail_inline_run_make_ctx_parent_map 52.9 ms of 7952 ms.
callresolve.cm — hits 14, rebuilds 2, parent map 0.44 ms.
The 5,500 ms / 44% line item from the 2026-08-04 measurement round is gone.
The new #2 is range_reconciliation (pit-compiler/streamline.cm:5331-5385).
Per function containing any int-overflow op it runs
analyze_local_arrays → infer_slot_write_types(optimistic) →
int_ranges.infer → infer_slot_write_types(sound) → eliminate_type_checks,
and int_ranges then runs a third time in final_int_ranges.
5. The double-parse: refuted where the plan said, confirmed elsewhere
Refuted in mcode_unit_result. pit-compiler/compiler.cm:1623 calls
parse_result exactly once and hands the parse to compile_result_from(parsed, …), which threads it through mcode_result_from → analyze_result_from
without re-tokenizing. The in-code comment says so and the call graph agrees
(compiler.cm:1580, :361, :250, :215). Nothing parses twice inside a
unit compile.
Confirmed in the closure walk. pit-shop/shop_fetch.cm:458, inside
analyze_locator, runs compiler.requirements_result(source_text, locator, …)
on every source file to discover imports. That is a full tokenize + parse +
analysis.collect. Minutes later the builder fleet tokenizes and parses the
identical text again. It is cached under mcode_cache_key, so a fresh
candidate shop pays it once per file per press.
Cost, measured: graph_walk_elapsed_s in the pit-shop/clerk plan is
6.02 s of that plan’s 78.73 s wall (7.6%), and it is serial — the plan’s
compile_window_s is 70.73 s, i.e. the 8 s difference is the walk running
before any compile is dispatched. Per press the summed graph walk is 13.4–15.3 s;
across the seed ≈ 40 s (10%). The scan’s parallel is cooperative within one
actor (shop_fetch.cm:508, drain()), so it uses one core.
Sanity check against the per-unit numbers: tokenize + parse + requirements is
52.8 ms for callresolve.cm’s 799 lines → ~4.6 s for 69 k lines single-threaded.
That is the 6 s.
6. CP9.5 — the July 10→14 streamline regression
git log --since=2026-07-09 --until=2026-07-16 -- pit-compiler/streamline.cm
returns 24 commits. The decisive one is visible in file history rather than in
streamline.cm itself:
$ git log --follow --pretty='%h %ad %s' -- pit-compiler/callresolve.cm
…
0247a4147 2026-07-11 inliner: shared callee resolution (upvalue + reuse-robust)
+ hot-size + call facts ← file created here
callresolve.cm did not exist before 2026-07-11. It is the file that
introduced make_ctx’s per-child rebuild of the parent’s fn_slot_fids map —
the loop CP9.1 measured at 5,767 ms / ~44% of a 13.1 s vm_suite unit compile.
0247a4147 is the guilty change for the bulk of the 1.0 s → 5.4 s move, and
it is already fixed on dev (a6a73b60f, verified live by the counters in §4).
What is left of the 5.4× is not one commit. The residue is the optimizer
campaign’s new analyses, which now dominate: range_reconciliation (22–26%)
rests on 4215f553b (integer range facts, 2026-07-09), 142e4a172
(ranges in large functions), 0890f148d (immutable parameter facts) and
f3bad5657/ddd093d27 (guarded/fused array stores), all inside the window.
Those are optimizations with output value, so the row becomes “make them
cheaper”, not “find the regression” — the hunt is closed.
7. Redundancy and scheduler answers
Are units recompiled that shouldn’t be?
Inside a press: no. compile_redundant_key_count = 0,
mcode_encode_redundant_count = 0, mach_lower_redundant_count = 0 in all
three shops. Across presses: everything — 149 identical keys × 3.
Is canonical encoding run more than once per unit? 982 encodes for 151 compiles (6.5 per compile), but zero redundant keyed encodes — the extra encodes are manifests, identity arms and metadata records, not repeats of the same value. Summed across parallel workers it is ~60 s per press against ~2,324 s of summed compile: 2.6%. Not a target.
Are store writes synchronous on the compile path?
Yes — builder_worker.ce’s tail calls shop_store.stage_mcode_unit inline
before replying. It costs 914 writes / 0.28–0.63 s per press. Negligible;
leave it alone.
Does the fleet saturate cores?
It over-saturates. pit-shop/shop_build.cm:1710 defaults width to null and
:1368 calls parallel(jobs, state.width) — unbounded. In the clerk plan
fleet_peak = 77 = compile_count (every unit dispatched at once) with
compile_average_width 34.0 on 15 logical cores. Peak RSS 3.27 GB. This is not
an occupancy indictment — it is the fuel for the §2 lock contention.
8. Ranked fixes
| # | fix | est. win | risk | kind | files |
|---|---|---|---|---|---|
| 1 | Get pit_runtime_mach_image_pin_count off the per-call path | press 113 s → ~25–40 s; seed 396 → ~150 s | medium | mechanical + one design choice | source/mach_vm.c, source/registry.c, source/pit_internal.h |
| 2 | One shared object store across the three candidate presses | 342 s → ~155 s (−187 s, 47% of seed) | high | needs design (gate semantics) | cake/forge.cm |
| 3 | Make range_reconciliation cheaper | ~20% of every compile | medium | needs design (soundness) | pit-compiler/streamline.cm, pit-compiler/int_ranges.cm |
| 4 | Cap fleet width at core count | small alone; removes 3.3 GB RSS + 16.3 M ctx switches | low | mechanical | pit-shop/shop_build.cm |
| 5 | Stop parsing every file twice (closure scan) | ~6 s serial/press, ~40 s of seed (10%) | medium | needs design | pit-shop/shop_fetch.cm, pit-compiler/compiler.cm |
| 6 | Instrument detail_initial_optimize_functions | unknown; it is 24% of streamline unattributed | none | mechanical (measurement) | pit-compiler/streamline.cm |
| 7 | Workspace promotion invalidates all compile keys | one full extra rebuild per reseed | medium | needs diagnosis | derivation-key inputs, pit-shop/shop_store.cm |
1 — the image-pin lock. Mechanical, with one design choice.
mach_binding_is_pinned is a safety check: it makes a function that outlived
its mapping fail as an invalid cursor instead of reading freed address space.
The check must survive; only the lock has to go. Options, cheapest first:
- Stamp the pin state onto the
PoolBindingat pin/unpin time — both already holdrt->image_lockand both already own the range array — and havemach_binding_is_pinnedread the stamp with a relaxed atomic load. - Or make
pin_countan atomic on the range and index it from the binding, so the read is one atomic load and no scan.
Either keeps the invariant “a call into an unpinned image refuses”. Regressions
already exist: source/mach_image_lifetime_test.c,
source/mach_pool_test.c:1285. Do not weaken the check to buy the win.
Expected: user time unchanged (~181 s), sys time collapses from 532 s, wall per
press falls toward the 15–25 s the honest work implies.
2 — the three-press store. Needs a ruling before code.
Measured fact that should drive the ruling: all three presses derive the
identical 149 K_mcode keys, so today’s gate is already not proving
“two different binaries compile the same way” — the compiler identity is
content-derived from the compiler sources, which do not change between
candidates. What the fixpoint gate actually proves is that the press/pack
machinery is stable. If that is the intended claim, one shared
content-addressed store across shop-1/2/3 is sound (objects are addressed by
their canonical bytes; a hit cannot silently change the answer) and buys 187 s.
If the claim is meant to be stronger, the alternative is to cut to two presses
or share only press-1 → press-2. cake/forge.cm:1488-1600 is the whole seam;
prepare_candidate_shop (:1170) is what creates the empty store.
3 — range reconciliation. Needs design.
pit-compiler/streamline.cm:5331. Per qualifying function it runs the write-type
inference twice and int_ranges.infer once, then final_int_ranges runs the
range analysis a third time over the same function at :5400. The second
inference is what makes the int facts sound (the first is optimistic), so it
cannot simply be deleted — the target is sharing one analysis result between
the reconcile pass and final_int_ranges, and skipping functions whose
optimistic and conservative tables already agree. Soundness bar is the
int_ranges precedent: differential fuzz on/off must agree, plus corpus byte
identity.
4 — fleet width. Mechanical.
pit-shop/shop_build.cm:1710 — default width to the core count instead of
null. This alone will not make the seed fast (fix 1 is what does), but it
removes the 3.27 GB peak and the 16.3 M context switches, and it is a one-line
change with an existing plumbed parameter.
5 — the closure-scan parse. Needs design.
pit-shop/shop_fetch.cm:458. Either the scan’s parse travels to the builder
that will compile the same text, or the scan stops being a full parse (an
import-only scan over the token stream). The BFS in drain() depends on the
scan’s results to expand the frontier, so the walk cannot simply be moved onto
the fleet without restructuring. 10% of the seed; take it after 1 and 2.
7 — the promotion-invalidates-everything row.
Diagnosed but not chased: after make seed promoted boot.qop,
boot/root.cart and ./pit, an otherwise-identical press re-missed 108/108
keys with zero overlap against its own previous run. The next run hit.
mcode_cache_key (pit-shop/shop_store.cm:97) folds compiler_identity,
which is the builder generation identity, which is derived from the builder
executable’s source closure — something in that derivation is picking up a
promoted artifact. Find it: it costs a full extra recompile on every
post-reseed build.
Reproduction
scratchpad/cold_make.log,scratchpad/cold_seed.log— timestamped stage logs.scratchpad/parse_plans.py— extractspit.compile.plan.telemetry@1fromobserve.jsonl.scratchpad/press_sample2.txt— 20 ssampleof a cold press.- Candidate-shop telemetry survives at
build/forge-press/shop-{1,2,3}/log/observe.jsonl*. - Warm/cold press control:
L=$PWD/build/forge-press PIT_INPROCESS=1 PIT_CONTENT=$L/press-2/boot.qop \ $L/proof/pit --shop $L/shop-3 --content $L/press-2/boot.qop \ bootstrap --out $L/press-3-warm2
Image-pin lock fix — landed, cp/pin-lock @ 51d250f4e (base dev @ 0c6f6aacd)
Row 1 of §8 is done. mach_binding_is_pinned no longer takes rt->image_lock.
The design, and why it is the sound one
Three shapes were on the table. What ruled between them is the lifetime of the thing the check reads, which the provider fixes exactly:
- A
PoolBindinglives inside aPitMachPoolBindingHandle(source/image_provider.c:41).provider_binding_for_placement(:556) acquires a pin before it builds the handle, andhandle->activeis never cleared for a pool binding — onlyprovider_close_pin(:788) clearsactive, and that is the pin class, not the binding class. So a live binding always holds its own pin, andprovider_detach_unused_locked(:127) cannot free the placement whilepin_refsis non-zero. - Therefore, wherever
bindingis safe to dereference,binding->image— which is&placement->view, storage owned by the runtime, not by the read-only mapping — is safe to dereference too.
That killed the “cache pinned-ness on the binding” option (b): the direct-binding
regressions construct a stack PoolBinding and pin the view separately
(source/mach_pool_test.c:1277-1285, and the same shape at :489 :544 :1387 :1445 :1524 :1798 :2001), so a binding-scoped flag would answer a different
predicate and those tests would have had to be rewritten to keep passing. It also
killed a seqlock over the registry: a press pins ~149 images, so a lock-free
linear scan is worse single-threaded than the lock it replaces. The answer has to
be O(1) and it has to hang off the image pointer.
Chosen: a mirror word on MachImageView.
/* source/pit_internal.h — in MachImageView */
_Atomic uintptr_t pin_owner; /* the runtime holding >=1 pin, or 0 */
registry.c owns every write, all of them under rt->image_lock, beside the
authoritative range->pin_count they mirror: published on both pin success
paths, retracted when unpin drops the count to 0. pit_runtime_mach_image_pin_count
is untouched and still locked — its cold callers (the C regressions,
dbg_code_from_function at source/runtime.c:10021) keep the runtime-scoped
counting read. Only the hot gate changed:
/* source/mach_vm.c:438 */
return ctx && binding && binding->runtime == ctx->rt && binding->image &&
pit_mach_image_pinned_by(binding->image, ctx->rt);
Diff: 53 insertions, 1 deletion, across three files.
The memory-ordering argument
- Publication. The store is
memory_order_release, made underrt->image_lockand after the placement the view describes is fully built. The read ismemory_order_acquire. A reader that sees this runtime therefore also sees the section table it is about to read. Nothing is published by the mirror that was not already published by the lock. - The drain race is not new. A pin can drop to zero the instant after the
load, so the gate can read “held” for a moment after it stopped being true.
That was equally true of the locked version:
pit_runtime_mach_image_pin_countunlocks before it returns, so its caller always acted on a snapshot another thread could already have invalidated. The lock bought data-race-freedom on the array scan and nothing else; the scan is gone, so there is nothing to race on. - What actually keeps a running image alive is ownership, not this check. An
image function traces its owner, the owner traces the binding handle, and the
handle holds a pin for its whole life. The count cannot fall to zero underneath
a call in progress. The gate is the belt on those braces and it keeps exactly
the strength it had — an unpinned binding still refuses, as
direct_execution_testasserts before and after. - Runtime scoping. The mirror names the runtime, not just a count, so
pinned into rtis one comparison rather than an argument. Two runtimes pinning the same view object would clobber each other’s owner — that cannot happen for provider views, which are embedded in a per-runtime placement list, and no test does it. If it ever did, the loser fails closed (refuses to run), never open. - Initialization.
Pit_MachPoolOpenmemsets the view (source/mach_pool.c:507,510), so a fresh view is unpinned;provider_adopt_mappingcopies only freshly opened views, never a pinned one.
Before / after
15 logical cores, AC power. Caveat that matters: other night agents were building throughout, load average 12–17 in both arms — so the absolute numbers are higher than the §2 attribution round’s (which had a quiet machine). The A/B is paired, back to back, same worktree, same protocol.
Cold press — proof/pit … bootstrap into a shop whose objects/blake2 and
catalog were emptied, PIT_INPROCESS=1, /usr/bin/time -l:
| metric | before (dev) | after (fix) | Δ |
|---|---|---|---|
| wall | 330.50 s | 224.29 s | −32.1% |
| user | 385.71 s | 310.26 s | −19.6% |
| sys | 135.24 s | 31.18 s | −77.0% |
| involuntary ctx switches | 8,605,341 | 1,105,510 | −87.2% |
| voluntary ctx switches | 3,260 | 2,430 | −25.5% |
| instructions retired | 8.572e12 | 7.531e12 | −12.1% |
| cycles elapsed | 2.152e12 | 1.391e12 | −35.4% |
| page reclaims | 14,277,803 | 13,067,455 | −8.5% |
| max RSS | 7.43 GB | 9.16 GB | +23% |
RSS goes up on purpose: the fleet was previously blocked, so it never got wide. Row 4 of §8 (cap fleet width) is now the row that pays, not the row that was masked.
Cold make seed (whole three-press seed, full rm -rf .pit build cold-build pit first):
| before | after | Δ | |
|---|---|---|---|
| wall | 607.75 s | 386.44 s | −221.3 s, −36.4% |
(/usr/bin/time -l cannot attribute CPU here — the presses are daemon-spawned,
not children of make. Wall is the honest column.)
Cold make (C floor + candidate binary; contains no press, so no change is
expected — this is the no-regression control):
| before | after | |
|---|---|---|
| real | 37.13 s | 15.32 s |
| user | 11.52 s | 11.55 s |
| sys | 2.43 s | 2.67 s |
User and sys are identical; the real-time gap is machine load, not the fix.
10 s sample of the in-seed press, t = 75 s into make seed, all threads:
| leaf | before | % | after |
|---|---|---|---|
__psynch_mutexwait | 61,964 | 48.4% | not in top 15 |
__psynch_mutexdrop | 20,513 | 16.0% | not in top 15 |
mach_msg2_trap (idle) | 12,814 | 10.0% | 15,043 |
Pit_CallRegisterVMFunction | 7,328 | 5.7% | 5,029 |
__workq_kernreturn (idle) | 6,407 | 5.0% | 14,325 |
kevent (idle) | 6,407 | 5.0% | 7,519 |
__psynch_cvwait (idle) | 5,951 | 4.6% | 107,966 |
pit_runtime_mach_image_pin_count (self) | 1,177 | 0.9% | 0 |
pit_runtime_mach_image_pin_count appears 165 times anywhere in the
baseline sample’s call tree and zero times in the fixed one.
Deeper into a cold press (t = 100 s), the top non-idle leaf is now the interpreter itself, which is what §2 said the honest workload should look like:
__psynch_cvwait (idle) 89,007
Pit_CallRegisterVMFunction 14,509 <- the interpreter
mach_msg2_trap 14,454
__workq_kernreturn 7,225
kevent 7,225
pit_key_equal 886
mach_resolve_outer_frame 854
pit_mach_record_get_hashed 529
Verification
| gate | result |
|---|---|
scripts/gate.sh "vm suite" ./pit test run tests/vm_suite.ce | OK |
./pit test suite | 475 passed, 0 failed |
./pit fuzz 100 | 2,466 cases, 0 failed (opt/unopt 820, linked/unlinked 826 + shapes 6, join_types 820) |
./pit test run tests/compile.cm | 187 passed, 0 failed |
ARMS="force-gc" scripts/check_arms.sh | 18 passed, 5 skipped, 0 failed — ALL GREEN |
ARMS="validate-gc" scripts/check_arms.sh | 18 passed, 5 skipped, 0 failed — ALL GREEN |
two concurrent ./pit test suite against one daemon | 475/0 and 475/0 |
The force-gc and validate-gc arms are the ones that matter for this change: they
build and run the C regressions — mach_pool_test, mach_image_lifetime_test,
resident_image_provider_test — which are exactly the pin-lifetime tests §8 said
not to weaken. direct_execution_test still proves an unpinned binding refuses.
What this moves in §8
Row 1 is closed. With the lock gone the seed’s remaining shape is what §1 and §3 predicted: three presses of identical work (row 2, still the biggest single win at ~187 s), streamline inside a unit compile (row 3), and an unbounded fleet that now actually spends the cores it takes (row 4, whose RSS cost is visible above).
Reseed identity + width
Branch cp/reseed-identity, worktree .claude/worktrees/night-identity, base
dev @ 0c6f6aacd. Cold protocol: rm -rf .pit build cold-build pit && make,
then make seed. Three to five other agents’ seeds ran concurrently on the
same 15-core machine throughout, so wall times are contended; they are only
ever compared against another run from the same session. Key counts are not
affected by contention.
Row 7 reproduces, and it is not the promotion
Row 7 read “workspace promotion invalidates all compile keys”. The 108/108 zero-overlap re-miss is real and reproduces exactly — but on a reseed that promoted byte-identical artifacts, so the promotion cannot be the cause.
The leaked field is compiler_identity, and what leaks into it is which
of two boot lanes pinned the builder.
pit-shop/clerk.ce pins one generation two ways:
| clerk boot | path | what K_mcode gets |
|---|---|---|
| its own source visible | pin_generation_builder -> builder_identity.for_bundle | the D3 slice: hash over {locator, source_hash} of the COMPILE sub-closure — bea2ba88… |
| its own source not visible | pin_resident_generation_builder | the coarse whole-closure hash (resident_derivation.source_closure, i.e. the builder’s executable-MANIFEST hash) handed to both halves — 573dd10e… |
Which lane a shop takes is not a property of the shop. A fresh shop’s first
boot is source-less, because the runtime mount opens behind the clerk
(shop_actor.ce starts boot-image services, then open_content); every later
boot into that same directory is not. Straight out of one make seed, one
directory, two consecutive processes:
build/forge-press/shop-3/log/boot.log (press-3 bootstrap, FIRST boot)
clerk source-less resident blake2:bc39ebb6…
resident builder generation blake2:573dd10e… <- coarse, BOTH halves
build/forge-press/shop-3/log/boot.log (the qop pack, second boot)
clerk resident current blake2:bc39ebb6…
builder generation blake2:573dd10e…
compiler blake2:bea2ba88… lowering blake2:ae2d7f56… <- sliced
Same binary, same tree, same generation NAME — two compiler_identity values.
Everything the first boot compiled is unreachable to every later boot.
Measurement — before
The reseed was verified to be a no-op: forge: two clean boot presses are byte-identical, and boot/root.cart (md5 454612a7…) and boot.qop (md5
397cd591…) were unchanged across it. press-1, press-2 and press-3 all
produced the committed root.cart byte-for-byte, so the press is deterministic
and the tree was already at its fixpoint.
Three presses into the same warm shop-3, same binary
(build/forge-press/proof/pit), same content (press-2/boot.qop):
| press | boot lane | identity fed to K_mcode | distinct K_mcode misses | hits | wall | max RSS | invol. ctx switches | fleet_peak |
|---|---|---|---|---|---|---|---|---|
| press-3 (1st boot, fresh shop) | source-less | coarse 573dd10e | 149 | 573 | in-seed | — | — | 77 |
| warm A (2nd boot) | source | sliced bea2ba88 | 112 | 596 | 289.3 s | 5.94 GB | 7.82 M | 74 |
| warm B (3rd boot) | source | sliced bea2ba88 | 0 | 684 | 35.8 s | 6.52 GB | 0.66 M | 2 |
press-3 ∩ warm A = 0 keys. Warm A is a full recompile of what press-3 had
just finished compiling — 289 s thrown away because the two boots keyed it
differently. Warm B is what a warm press costs when the keys line up: 36 s.
That is row 7’s 153-then-108-with-zero-overlap, reproduced and explained. It
costs one full extra compile of the tree the second time any shop is used, and
make seed creates three fresh shops.
The fix
The obvious fix — slice at boot in the resident lane — does not work, and
that was measured rather than assumed: a fused shop cannot read the builder’s
executable-manifest OBJECT. It is neither in the shop’s own store nor in the
content bundle’s object layer; object_get_value on 573dd10e… returns null
even in a fully booted daemon with content mounted. A first attempt shipped
exactly that and logged (one coarse identity) on every press.
The manifest is readable in exactly one place: at press time, in the shop that has just realized the executable (the object lands 9 s before the cart is cemented). So the slice is computed there and carried in the cart:
pit-shop/boot_cart.cm—cemented_identities(exe)->{compiler, lowering}per exec, usingbuilder_identity.for_bundle.pit-shop/cement.cm—cemented_derivation(exe, identities)merges them into the resident row’s derivation record. This module computes nothing:builder_identityreachesshop_store, andcement.cmis inshop_actor’s closure — the root actor must not grow the store to carry a fact the press already knows.boot_cart.cmis press-side and already reaches it.pit-shop/builder_identity.cm—for_resident(derivation, coarse)reads them back, falling back to reading the manifest, and failing that to the coarse identity in BOTH halves. A cart pressed before this landed behaves exactly as it did: over-invalidating, never stale.pit-shop/clerk.ce—pin_resident_generation_builderuses it.
Measurement — after
press-3’s first boot now logs the slice on the source-less lane:
shop resident: clerk source-less resident blake2:c2c31de5…
shop resident: resident builder generation blake2:573dd10e…
compiler blake2:bea2ba88… lowering blake2:ae2d7f56…
Same protocol, same shop, straight after the seed:
| before | after | |
|---|---|---|
| distinct K_mcode misses, warm press after a cold press | 112 | 0 |
| K_mcode hits | 596 | 708 |
| overlap with the cold press’s key set | 0 of 149 | n/a — nothing missed |
| wall | 289.3 s | 32.0 s |
| involuntary context switches | 7.82 M | 0.67 M |
| max RSS | 5.94 GB | 6.48 GB |
9.0x on the wall for the second use of any shop, and the 289 s of duplicate compile is gone entirely.
Regression tests
tests/builder_identity.ce—resident_lane_checks(): one generation pinned on either lane yields ONE answer; a lowering-only builder change leavescompiler_identityalone through the resident lane; both fallbacks (no cemented slice with an unreadable manifest -> coarse; no cemented slice with a readable one -> the read) hold.tests/store_freshness.ce—generation_lane_cache_checks(), counter-based onpit.compile.plan.telemetry@1: compile one source with the builder pinned the way a first boot pins it (assertingk_mcode_misses == 1), then pinned the way every later boot pins it, and assert that plan reportsk_mcode_hits == 1, k_mcode_misses == 0. The source carries a guid so the first compile is always cold and the second one’s hit means what it says.
Fix 2 — fleet width clamp
pit-shop/shop_build.cm defaulted state.width to null, and null is
parallel’s UNBOUNDED. The default is now the core count from
runtime::internal/sysinfo’s ncpus() (internal/sysinfo.c ->
sys_cpu_count), read once and memoised. An explicit width on the request
still wins, and a runtime that cannot answer keeps the historic unbounded
behaviour rather than guessing. Same change at pool_derivation’s parallel.
No oversubscription factor, no per-plan heuristic — a clamp, not a knob.
Paired presses, same corpus and same miss count, one binary apart:
| press | width default | fleet_peak | distinct misses | summed compile |
|---|---|---|---|---|
seed A shop-3 | unbounded | 77 | 149 | 272.0 s |
seed B shop-2 | ncpus() = 15 | 15 | 149 | 271.1 s |
Peak concurrency 77 -> 15 at no throughput cost (the summed compile time is a wash). On the warm-press pair above, involuntary context switches fall 7.82 M -> 0.67 M, though that pair also differs in how much it compiles.
Seed wall times are reported for completeness only and are not comparable — every run shared the machine with other agents’ seeds: seed A 9:45, seed C 10:56.
What this does NOT show
A byte-identical reseed does not, by itself, move compiler_identity: the
coarse hash is the builder’s manifest hash and it was unchanged (573dd10e…)
across all three seeds in this session. Row 7’s “workspace promotion” framing
was a misattribution of the boot-lane flip; nothing needs to be fixed about
promotion.
Source: plans/archive/night-2026-08-04/seed-speed.md