Working state — a note taken while the work happens, not a specification. The system as it is meant to be is in Architecture.

A standing performance measurement suite

Status: DESIGN + partial prototype. Written 2026-08-05 against c13443301. Prototype harnesses landed: benchmarks/bench_suite.ce, benchmarks/start_latency.ce.

The problem this closes: every performance investigation in this repo has re-derived its own harness, so numbers from different sessions are not comparable. plans/archive/night-2026-08-04/switchbench.ce is the canonical example — a good harness with a hard-coded scratch path from a worktree that no longer exists (switchbench.ce:16), presented by switch-report.md:305 as “the harness that produced every row” and unrunnable as committed.

The one rule this design is built on, taken from plans/archive/night-2026-08-04/benchmarks.md:3-8: a wall-clock number on a loaded machine is not a measurement. Every row below declares which class it is in. CI gates the counter class and only reports the wall class.

0. Reading the numbers in this document

Every number I took is from this worktree on 2026-08-05, on a machine carrying other agent sessions. Where a number is wall-clock I say so and give its spread. Where I did not verify something I say that too.

One environment caveat, corrected after the fact — the original diagnosis in this section was wrong, and the wrong remedy costs six minutes:

• ./pit is not tracked (git ls-files says so); it is a build product. This worktree’s copy is dated Aug 3 while the checkout is Aug 5, so it is simply a stale binary — two days behind its own tree.
• The committed boot/root.cart at c13443301 is not stale and does not name pit-shop/development_shop (strings boot/root.cart | grep -c → 0; it names pit-shop/clerk). The rename in d6fdc3122 is an ancestor of HEAD.
• The remedy is make, not make seed. boot/root.cart is in FORGE_PLAN_INPUTS (Makefile:53-55), so a cart newer than ./pit makes the binary out of date and make re-forges it against the current cart.

But there is a trap in front of that, and it is the real finding. A foreign daemon can own this worktree’s socket. When make was run here, the process holding .pit/pit.sock was .claude/worktrees/lane-cp/pit ps (pid 12600) — left by this very measurement session. ./pit forge darwin --out build is a client call, so the daemon performed the forge, in the daemon’s worktree: the output line read forge: promoted …/worktrees/lane-cp/build/pit, and this worktree’s ./pit was never touched. The re-forged binary was byte-identical to lane-cp’s existing one, so nothing was harmed — but a build silently landing in another worktree is a much sharper version of the stale-daemon problem than the build-signature refusal, because it succeeds.

Correct recovery in a worktree whose socket may be foreign-owned:

lsof -t .pit/pit.sock          # whose daemon is this?
ps -o command= -p <pid>        # confirm the binary path
./pit down                     # or kill that exact pid
make

Measurements in this document were taken with the lane-cp binary at the same clean commit c13443301, which boots correctly. That does not affect any number here — it is the same source, and every counter-class row reproduced exactly across runs.

1. Inventory — what exists today

1.1 Trustworthy and already gating

1.2 Runtime counters available to pit code

Correcting the task brief: actor.op_counts() and actor.vm_stats() no longer exist on actor. They moved to deniable endowments — source/pit_actor.c:652-654 records the move, and tests/actor_module_surface.ce:25-28 locks them off actor.

• $vm.vm_stats() — std_endowments/vm.c:721-777, exactly 21 fields. alloc_bytes, alloc_count, gc_count, gc_ms, minor_count, major_count, minor_promoted_bytes, minor_survivor_copied_bytes, minor_max_us, minor_p50_us, minor_p95_us, nursery_bytes, old_alloc_peak_bytes, … All always-on and free (source/pit_internal.h:2170) except instructions, which is zero unless $pgo.count_instructions(true). Caveat worth knowing: minor_p50_us/p95_us come from log2-microsecond buckets (vm.c:702-716) and are therefore powers of two, not interpolated. minor_max_us and minor_total_ms are exact.
• $pgo.op_counts() — std_endowments/pgo.c:24-26, built at source/mach_vm.c:232-243. Keys are opcode mnemonics, zeros skipped. Gated by ctx->instr_count_on (pit_internal.h:2178), checked once per decode at mach_vm.c:3122 / :3159. The 15-35% figure in plans/archive/compiler-perf.md:594 is about the enabled cost; disabled it is one predictable not-taken branch.
• $inspect.snapshot() — std_endowments/inspect.c:24-178, ~100 fields per actor, backed by scheduler_snapshot() (source/scheduler.c:2935). Installs no hook and keeps no counter (inspect.c:5-8) — it is a formatter over state the scheduler already has. Carries heap_used, heap_size, static_size, boot_mach_size, gc_bytes_copied, total_turns, total_messages, max_turn_s, avg_turn_s, msg_queue_length.
• $vm.memory() — std_endowments/vm.c:618-677. The one runtime-wide allocator view: total_mapped, actor_heap_used_bytes, actor_offheap_bytes, shared_offheap_bytes, pool table.
• Clock — time.monotonic() only (pitlib/internal/time.c:16-19 → platform/posix-runtime/source/sys_os_posix.c:61-72). Nanosecond source resolution, delivered as a double of seconds since process origin; because the origin is subtracted it keeps ~ns precision. actor.monotonic() is the same clock (source/pit_actor.c:463-465) — deliberately, so harness timings are comparable with max_turn_s and alive_s. There is no CPU-time clock and no cycle counter reachable from pit.

1.3 Benchmarks that exist

• benchmarks/native_call_dispatch.ce — the style exemplar. Min-of-9, ns/iter, and critically reports differences against two empty base loops (:79-84). Its header (:7-9) already carries the contention warning.
• benchmarks/link_inline.ce — zero wall clock, pure counters, reports one-line JSON (:167-177). The best existing model for a CI-safe row.
• benchmarks/benches/*.cm (17 kernels) — driven by shop_tools/test.ce:1047-1086 through shop_tools/internal/bench_runner.cm: calibrated batch size, 3 warm-ups, 11 samples, reports mean/median/min/max/stddev/p95/p99_ns and ops_per_sec (bench_runner.cm:163-168). Wall clock only, no counters. This is a well-built harness that should be extended with a counter bracket rather than replaced.
• Three benchmarks are broken: benchmarks/nota.ce, wota.ce, wota_nota_json.ce all call time.now_ns(), which is deliberately removed (tests/time.ce:13 asserts it must disrupt). nota.ce:82 also reads a benchmarks/nota.json that does not exist.
• aot_bench/ — a large, well-designed dual-lane instrument, parked by the standing no-AOT ruling (plans/archive/todo.md:35). Last touched d27b0701c, 2026-07-31. Preserved, not broken. Leave it.

1.4 A claim in the plans that the code and the measurements contradict

perf/baselines/2026-08-04-cp6-dev.json records cold_ps_s 19.686 and hot_ps_s 1.693. perf/baselines/2026-08-04-lane-cp-postp3.json records cold_ps_s 0.42. Neither matches what I measured today: cold 0.13 s, hot 0.02 s (§5.3). plans/archive/compiler-perf.md:649-663 already diagnoses the postp3 baseline as not like-for-like (the clerk booted source-less because .pit/sources/pit-shop had not been snapshotted). The 19.7 s figure is almost certainly the same artefact in the other direction.

The lesson is not “the baselines are wrong” — it is that cold_ps_s as currently defined does not name what it measures. A cold-start row must record the shop’s source-snapshot state, or it is not comparable to any other cold-start row. That goes into the schema in §4.

2. The design

2.1 Two classes, declared in the data

Every row carries a class field. There are exactly two values.

counter — derived from a count the runtime or the OS already keeps, or from a deterministic property of an artifact. Reproduces bit-for-bit on a loaded machine. CI may gate these. Examples: instruction counts, guard counts, artifact bytes, alloc bytes, GC cycles, bytes copied, turns, messages, instructions retired from rusage.

wall — a time.monotonic() or shell-level elapsed reading. Contended. CI reports these and never gates them. Every wall row must carry median_ms, min_ms, max_ms, samples and spread_pct so a reader can see the machine’s state in the artifact itself. A wall row without dispersion is not admissible.

There is a useful third thing that is nearly counter-class: a ratio of two walls taken in the same window (e.g. gc_share_pct = GC ms ÷ workload ms). Contention scales both terms, so the ratio survives load far better than either term. These are marked class: "counter" only when both terms come from the same uninterrupted window; otherwise they stay wall.

2.2 The finding that most changes how this should be built

/usr/bin/time -l on this platform reports instructions retired and maximum resident set size per process. Measured over 8 identical pit ps invocations:

So the whole CLI-level lane can be made counter-class: measure pit ps by instructions retired and peak RSS, not by wall. That is a load-immune, CI-gateable number for items 2 and 3, and it captures C-level and kernel work that mach_instr_total cannot see.

Portability caveat I did not verify: instructions retired is an Apple Silicon rusage extension. On Linux the equivalent needs perf stat or getrusage + PMU access, and CI runs on a self-hosted runner whose platform I did not check. The suite must degrade to wall + RSS where the counter is absent, and record which it used.

2.3 Daemon state is a first-class variable, not noise

This bit me and it will bite anyone who does not design for it. Same binary, same command, same machine:

A 6-10× swing with nothing changed but what the daemon had been doing. Any shell-level or in-daemon timing row is meaningless without the fleet state attached. Every record therefore carries a fleet block (actor count, summed heap_used, summed heap_size, total GC count) sampled immediately before the measurement — see §4.

Notably, item 1 was robust to this (spawn 4.28 ms dirty vs 4.52 ms clean), which is itself worth knowing: actor breeding does not scale with fleet heap.

2.4 Shape of the suite

Three lanes, run in this order, each emitting one record:

1. bench.cli — outside the daemon. Cold daemon boot, hot round trip, measured by rusage counters + wall. Needs exclusive control of the daemon, so it runs first and alone.
2. bench.run — inside the daemon (benchmarks/bench_suite.ce). Compile, link, artifact size, GC, actor memory, runtime memory. Mostly counters.
3. bench.start — inside the daemon (benchmarks/start_latency.ce). Start latency decomposition. Wall, but with tight dispersion.

CI appends each record to perf/trend/<schema>.jsonl, one line per run, keyed by commit. A trendline is then jq over that file. Nothing needs a database.

3. John’s ten items, one by one

Item 1 — actor start latency, in-process. MEASURABLE TODAY. Prototyped.

benchmarks/start_latency.ce. Decomposes into three intervals, all in-daemon so process start and daemon connect are excluded by construction:

• spawn — $start() call → shop’s start reply carrying the child.
• first_response — start reply → the child’s first reply to {type:"help"}. This is John’s question in the strict sense.
• stop — $stop(child) → death notice.

Iteration 0 realizes and is reported separately, never folded into the medians.

Sensitive to: whether the program is already realized (that is the whole point of separating the cold iteration); not meaningfully sensitive to fleet heap. Class: wall, but with 24% spread on spawn and tight agreement between two independent instruments (§5.1).

Do NOT use pit time --profile for this. It measures a different path — see the defect in §6.

Item 2 — shell round trip. MEASURABLE TODAY. Prototyped (shell-level).

./pit ps end to end. Should be taken as instructions retired + peak RSS (counter class) with wall alongside. Subtracting item 1 is not quite the right arithmetic: the right subtraction is wall(./pit ps) − elapsed_s(pit time ps), which isolates process start + connect + IPC from everything in-daemon.

Item 3 — cold daemon start. MEASURABLE TODAY. Prototyped (shell-level).

./pit down → ./pit ps. Two distinct things hide under “cold” and the schema must separate them:

• warm-page-cache cold — daemon down, binary and boot.qop still in the OS page cache. 0.13 s.
• cold-page-cache cold — first boot after the file was written. I observed 0.49 s on the session’s first boot.

Also must record whether the shop has a source snapshot, per §1.4.

Item 4 — actor switching cost. NOT HONESTLY MEASURABLE TODAY.

This is the item that needs real runtime work, and I want to be precise about why rather than hand-wave.

The architecture in the brief is confirmed. source/scheduler.c:2501-2518 creates sys_cpu_count() workers (PIT_WORKERS can only lower it, never raise: wn >= 1 && wn < n). actor_runner (:2452) takes engine.lock (:2454), waits on engine.wake_cond, and dequeues at :2465. One refinement: it is one global queue set, not one list — engine.q_head[PQ_COUNT] with four priority bands (PQ_READY/REFRESHED/EXHAUSTED/SLOW, :52-56), all under the one mutex. No work stealing, no per-worker queue.

The engine struct (source/scheduler.c:75-89) contains not one counter. Concretely absent: run-queue depth, global turn count, context switches, work steals, wakeups, worker idle/busy time, engine.lock contention or hold time, enqueue counts, per-band distribution. Even engine.num_workers (:88) has no pit-visible reader.

What exists is all per-actor: total_turns (scheduler.c:4721), total_messages (:4571), total_callbacks, max_turn_s (:4723), avg_turn_s (derived :3019), and msg_queue_length computed by walking arrlen(a->letters) at snapshot time (:3148).

So today the best available answer is an assortment, and each number means something narrower than “switching cost”:

What is missing and what it would take. Four counters in engine, incremented under the lock that is already held, would make this a real measurement: enqueue_total, dequeue_total, wakeup_total, and a running queue_depth_sum sampled at dequeue (giving time-averaged depth by the same integral trick build_telemetry.cm:274-286 already uses for fleet width). Plus engine.num_workers exposed. Estimate: ~40 lines of C in source/scheduler.c plus ~15 in std_endowments/inspect.c, no new locking, no per-turn cost beyond four increments inside an already-held mutex. Lock contention specifically (wait time on engine.lock) is a bigger job and I would not do it first.

Until that lands, any single number called “actor switching cost” in this repo is not defensible, and the suite should say so in the artifact rather than emit a plausible-looking figure.

Item 5 — compile throughput. MEASURABLE TODAY. Prototyped. §5.4.

compiler.mcode_unit_result(source, path, opts) timed directly, no shop, no fetching. Sensitive to: compiler profile (ship here), and strongly to which file — throughput is not linear in source size (§5.4).

Item 6 — link time. MEASURABLE TODAY. Prototyped. §5.4.

mcode_link.link_result(rows, plan). Prototype links the corpus with imports deliberately unresolved, so it measures the linker and not the resolver.

Item 7 — artifact size. PARTIALLY MEASURABLE. Prototyped with a named gap.

• Unit mcode — canonical.encode(unit), the pit.mcode.unit@3 artifact. Exact and deterministic. Working.
• Linked mcode — same over program.value.groups + each program.value.units[i].unit. Working.
• Pressed mach pool — press.press() returns a blob (so length() gives bits, ÷8 for bytes — this trips everyone; switchbench.ce:177 divides by 8 for the same reason). It works for a synthetic unit (576 B for a trivial loop) but refuses for the real corpus units in my harness, which currently reports pressed_pool_bytes: null and a count of failures rather than a fake zero. press needs an identity it can derive (mach_press.cm:110-116) and I did not chase which of mcode_hash/unit/ stamp the corpus units are missing. Small fix, not a design gap.

Item 8 — actor memory footprint. PARTIALLY MEASURABLE. Needs runtime work for the honest total.

John is right that one number will not do. What is honest today, per actor, all from $inspect.snapshot():

• heap_used — live bytes in this actor’s own heap. The single best number.
• heap_size — reserved capacity (a near-idle actor sits in a 2 MB block).
• static_size — non-heap cost: constant pool + context (pit_internal.h:2816).
• boot_mach_size — the only per-actor image-bytes reading.
• frame_stack_bytes, heap_object_region_bytes, and a full breakdown by object type (array/blob/text/record/function/code/frame).

What does not exist: any per-actor attribution of pinned mach-image or shared stone bytes. The runtime tracks pinned image count (pit_runtime_pinned_mach_image_count, source/pit_internal.h:440) but no byte total, and nothing is exposed to pit. The ct_main_used_bytes / ct_overflow_bytes / ct_index_bytes fields exist in PitRuntimeMemorySnap (pit_internal.h:2899-2902), are filled by the snapshot, and are then silently dropped by pit_vm_memory — they never reach pit code. In this build ct_pool_used / ct_pool_total read 0 for every actor.

So the suite reports per-actor own-heap + static + boot image, and separately the runtime-wide aggregate, and explicitly refuses to add them into one “footprint” because they double-count. To get the number John is actually asking for — “own heap plus attributable pinned/shared” — needs:

1. exposing the four ct_* byte fields through $vm.memory() (~6 lines, the data is already collected); and
2. a byte total alongside pinned_mach_image_count, attributed per actor by which views that actor’s heap references (this is the real work — attribution is a policy question, since a view pinned by three actors is not three copies).

Item 9 — GC cost. MEASURABLE TODAY. Prototyped. §5.5.

$vm.vm_stats() bracketing a fixed churn workload gives cycles, alloc bytes, alloc count, promoted bytes, survivor bytes, gc_ms, and minor_max_us. Derived: bytes-per-GC, µs-per-GC, and GC share of the window (a ratio, so load-tolerant).

Two honest gaps:

• The major-collect pause distribution is collected and unreadable. major_max_s (pit_internal.h:2235) and pause_hist_major[32] (:2239) are maintained; vm.c:757-761 passes pause_hist_minor for both percentile calls and never touches the major histogram. ~6 lines to fix and it is the cheapest high-value addition in this whole document.
• The p50/p95 that do exist are log2 buckets (powers of two). Report minor_max_us and minor_total_ms as the exact figures.

Also: in this build (gc=copy, nursery off) every generational field reads 0. The suite must record the GC profile in the record or the row is unreadable.

Item 10 — stone pool blob size. PARTIALLY MEASURABLE.

$vm.memory().shared_offheap_bytes (1,075,861 B today) is the only runtime-level shared/stone figure reachable from pit. is_stone is a blob immutability flag (source/blob.h:76-78), not a separate arena with its own counter; stone text lives in the constant pool. The per-actor proxies (ct_pool_used/ct_pool_total) read 0 in this build. The real constant-pool byte fields are the ones dropped by pit_vm_memory (item 8). Same ~6-line fix unlocks both.

Additions I think belong in a standing suite

1. Artifact identity, not just size. The IR census already gates output quality. Add the hash of the pressed corpus to the record. A byte-identical output across a refactor is the strongest possible regression proof, and plans/archive/compiler-perf.md:636-638 already relies on it informally.
2. Instructions retired for pit ps (§2.2) — the only genuinely load-immune end-to-end number available.
3. Store steady-state bytes — .pit/objects size. Already in pit.cp6.baseline@1; cheap, and it is the number that catches an artifact explosion.
4. The existing pit.compile.plan.telemetry@1 stream. It is already emitted on every build into observe.jsonl and nothing reads it in CI. Folding its compile_average_width / fleet_average_width into the trend costs nothing and directly measures build parallelism.
5. A measurement_protocol block in every record, copying pit.compiler.profile.report@1’s self-documenting habit (compile.ce:719-720). A record that explains its own method survives the session that produced it.

4. Proposed artifact format

Consistent with pit.compile.plan.telemetry@1 (build_telemetry.cm:16) and pit.cp6.baseline@1 (cp6.py:277): a flat schema field, *_count / *_elapsed_s pairing, canonical encoding via shoplib/canonical.cm when hashed.

Three schemas, one per lane, sharing a common envelope.

pit.bench.envelope@1        (embedded in each, not emitted alone)
  schema          text
  commit          text        git HEAD
  build_id        text        e.g. "forge-host-nan64-v1"
  profile         text        "host-nan64-v1;rep=nan64;text=utf32;gc=copy;..."
  host            {os, arch, cpu_count, workers}
  taken_at        number      wall epoch, for ordering only — never for timing
  fleet           {actors, heap_used, heap_size, gc_count}   ← §2.3, REQUIRED
  measurement_protocol  {…}   free-form, self-documenting strings
  contended       bool        set by the runner when it cannot claim a quiet machine

pit.bench.cli@1             lane 1, outside the daemon
  cold  {class:"wall", boot_ms, page_cache:"warm"|"cold",
         source_snapshot: bool,                    ← §1.4
         instructions_retired, max_rss_bytes}      ← class:"counter"
  hot   {class:"wall", median_ms, min_ms, max_ms, samples, spread_pct,
         instructions_retired, cycles_elapsed, max_rss_bytes,
         involuntary_ctx_switches}
  in_daemon_elapsed_ms      from `pit time ps`
  cli_overhead_ms           hot.median_ms - in_daemon_elapsed_ms

pit.bench.start@1           lane 3  (IMPLEMENTED)
  program, iterations
  cold  {class:"wall", spawn_ms, first_response_ms}
  warm  {spawn|first_response|stop:
           {class:"wall", median_ms, min_ms, max_ms, p90_ms, samples, spread_pct}}

pit.bench.run@1             lane 2  (IMPLEMENTED)
  compile {units:[ {path, source_bytes, mcode_bytes, pressed_pool_bytes,
                    expansion_ratio, functions, instructions, guards,
                    guard_pct, literals,           ← all class:"counter"
                    wall:{…}, bytes_per_s} ],      ← class:"wall"
           totals:{…}}
  link    {units, linked_mcode_bytes, functions, instructions, guards,
           guard_pct, literal_rows, wall:{…}}
  gc      {class:"counter", repeats, alloc_bytes, alloc_count, gc_count,
           gc_ms, bytes_per_gc, us_per_gc, minor_*, major_count,
           workload_wall_ms, gc_share_pct}
  actor_memory   {class:"counter", totals:{…}, actors:[{…}]}
  runtime_memory {class:"counter", total_mapped, actor_heap_used_bytes,
                  actor_offheap_bytes, shared_offheap_bytes, …}

Storage. One JSON line per run appended to perf/trend/<schema>.jsonl. Append-only, git-tracked, merge=ours like boot.qop so concurrent lanes do not conflict. A trendline is jq -r '[.commit,.compile.totals.instructions]|@tsv'.

Gating. A separate perf/bench_gate.py, modelled exactly on ir_census.py: compare the newest record’s counter-class fields only against a committed reference, ceilings per field, any increase fails, corpus drift fails. Wall fields are printed in the summary and never compared. This is the piece that makes the suite a gate rather than a diary.

5. Prototype and real numbers

Contention caveat, stated once and applying to every wall number below: these were taken on 2026-08-05 on a machine carrying other agent sessions. Wall figures are usable for order-of-magnitude and for the paired comparisons they appear in; they are not a quiet-machine baseline. Counter figures (instructions, guards, bytes, alloc, GC cycles) reproduced exactly across every repeat run, which is the point of the two-class split.

Harnesses: benchmarks/bench_suite.ce, benchmarks/start_latency.ce.

5.1 Item 1 — actor start latency (clean daemon, ps already realized, 11 warm iterations)

Cold (realizing) iteration: spawn 5.27 ms, first response 0.13 ms.

Cross-check: pit time --profile ps independently reports help_s = 67-69 µs, against this harness’s 54 µs median / 82 µs p90 for the same exchange. Two instruments, same order, no contradiction.

Robustness: on a dirtied daemon (one actor at a 512 MB heap) the same harness gave spawn 4.28 ms / first 47.5 µs — i.e. actor breeding does not scale with fleet heap. Good news, and worth keeping as a standing check.

5.2 Item 2 — shell round trip (clean daemon, 15 samples)

./pit ps wall: 0.02 s on every one of 15 samples (the shell’s time quantises here; the rusage counters below are the real signal).

In-daemon portion (pit time ps elapsed_s): 5.78 – 5.94 ms. ⇒ CLI + IPC + process start ≈ 20 − 5.8 ≈ 14 ms.

On a dirtied daemon the same two numbers were 0.12 s and 56 ms — see §2.3.

5.3 Item 3 — cold daemon start

./pit down then pit ps, three cycles: 0.13, 0.13, 0.14 s; hot immediately after each: 0.02 s. ⇒ daemon boot ≈ 110 ms. The session’s very first boot (cold OS page cache) was 0.49 s.

Contradicts perf/baselines/2026-08-04-cp6-dev.json (cold_ps_s 19.686) by two orders of magnitude — see §1.4. Trust the measurement; fix the definition.

5.4 Items 5, 6, 7 — compile throughput, link time, artifact size

Corpus: 5 real units, ship profile, link_mode: "pool", 5 runs each.

• Expansion ratio 27.9× source bytes → canonical mcode bytes, and it varies 16.3×–36.4× per unit. Throughput is not linear in source size: mcode.cm (147 KB) compiles at 52.8 KB/s while streamline.cm (193 KB) manages 27.4 KB/s. Size alone does not predict compile cost — a per-unit row is mandatory, a corpus average would hide this.
• Link (item 6): 5 units → median 7.99 s (min 7.86, spread 6.3%), producing 10,322,845 linked bytes, 160,238 instructions (−1,123 vs the sum of units) and 23,451 guards (−229). Linking costs about as much as compiling the whole corpus (8.0 s vs 11.4 s) — link is a first-class cost, not a rounding error, which I did not expect going in.
• Pressed pool bytes currently null for these units — the named gap in item 7. Works on a synthetic unit (576 B).
• Every counter column above reproduced identically across three separate runs. The wall column moved 0.1–2%.

5.5 Item 9 — GC (12 repeats of a 400k-object churn)

alloc_bytes   422,606,808     alloc_count   9,600,250
gc_count                3     gc_ms              67.2
bytes_per_gc  140,868,936     us_per_gc      22,403.5
workload_wall_ms    244.7     gc_share_pct       27.5
minor_count/major_count/nursery_bytes: 0   (nursery off in this gc=copy build)

The first draft of this harness used a 4 ms workload in a 512 MB heap, collected zero times, and reported all-zero GC fields — a reminder that a GC row with gc_count == 0 is not a measurement. GC_REPEATS is sized so the window contains collections, and the harness should assert that.

5.6 Item 8 / 10 — memory

Smallest real actor (pit-shop/system, 0 turns): heap_used 641,312 B live inside a heap_size of 2,097,152 B reserved, static_size 81,738 B non-heap, boot_mach_size 69,360 B of shared boot image. So “a small actor that does nearly nothing” is honestly ~626 KB live + ~80 KB static, in a 2 MB reservation, plus a 69 KB image view that is shared and must not be summed across actors.

Runtime-wide: actor_heap_used_bytes 90.3 MB over actor_heap_capacity_bytes 144.7 MB; actor_offheap_bytes 837,091; shared_offheap_bytes 1,075,861 (the item-10 number); total_mapped 385,875,968.

6. A defect found while building this

pit time --profile <p> measures a path ~80× more expensive than a normal start, and nothing says so. On a clean daemon, same program, same fleet:

run_profiled (shop_tools/time.ce:318-371) issues a separate realize request to get the phase split; run_timed (:288-316) does not. The realize request appears to re-walk the dependency graph rather than hit the resident derivation, costing ~330 ms even fully warm. So --profile’s breakdown describes the realize lane, not the lane a normal start takes.

I have not confirmed the mechanism (resident-derivation cache bypass is my hypothesis, from the bootstrap_policy: true in start_request at time.ce:222 and the separate realize at :320-326); I am reporting the reproducible measurement, not the cause. Either way, pit time --profile must not be used as the item-1 instrument, and the ~330 ms warm realize is worth a look on its own.

7. Ranked build order

Deliberately not proposed: rebuilding benchmarks/benches/*.cm + bench_runner.cm (already good — extend with a counter bracket instead), anything in aot_bench/ (parked by ruling), and a second copy of the cp6 corpus measurement (compiler_perf.cm already does it well).

Source: plans/measurement-suite.md