Archive — history, not state. Kept for its reasoning and its evidence; its plan is closed.

pit-native (AOT) vs mach — benchmark findings

ARC CLOSE 2026-07-10 (through commit d73f47fd) — final scoreboard after the full perf arc (harness, B+C pre-sizing, native call/closure/float fixes, inliner, sound type facts, interning, element-type facts, magic division, registerization, de-rooting sweep, grow-store fp fix, range facts + untagged register regions). Native lane, all match=Y, make check green:

bench	mach	native	vs the field
arith_int	31.5	10.1	OCaml 9.96 · C# 8.4 · LuaJIT 15.6 — matches OCaml, beats LuaJIT 1.5×
arith_poly	26.8	7.9	OCaml 7.11 · C# 6.3 · LuaJIT 10.9 — matches OCaml
loop_nested	31.2	11.8	OCaml 11.5 · C# 9.5 — matches OCaml
call_hot	22.8	7.8	LuaJIT 9.6 · OCaml 6.6 — beats LuaJIT
array_read	34.1	12.0	LuaJIT 14.0 · OCaml 10.0 — beats LuaJIT, sound
array_grow	4.3	2.1	OCaml 2.5 — beats OCaml
string_concat	0.33	0.27	fastest of all 9 runtimes measured
record_field	137	88	was 172 mach / native hang
record_new	38.9	22.9	Lua-parity; OCaml/C# ~1.2 needs nursery+representation
gc_churn	10.7	8.1	OCaml 0.18 / C# 0.6 — generational-GC territory
float_math	20.8	12.4	field ~1.2 — needs num promotion into untagged FP regs
fib / closure	83 / 70	88 / 66	call machinery — leaf-path arc next

Field-access columns (2026-07-10, commit 131e2ca5) — d_field (pure field access, 1.8M ops) and record_field (field-heavy RMW, 2M iters) ported to all harnesses + node/bun/s7 added:

runtime	d_field ms	ns/field-op	record_field ms
C# / node / bun	0.82–1.05	~0.5	11.7–13.1
OCaml / LuaJIT	1.42–1.46	~0.8	18.3–18.6
qjs	5.85	3.3	47.9
Lua 5.5	8.20	4.6	39.1
s7	10.21	5.7	39.4
pit-native	14.9	8.3	88
CRuby	19.9	11.1	95.1
pit-mach	25.1	13.9	137.7
mruby	28.7	15.9	123.8

The remaining pit gap is the per-field-op helper call + hash/probe in the native lane (Lua’s interpreter runs its probe inline at ~4.6ns; JIT/AOT offset access is ~0.5-0.8ns). Identified next step [native, emitter-only]: inline the record probe in emitted IL with a precomputed key hash — helper only on miss. s7 notes: fib is special-cased self-recursion (4.8ms — near node!) but indirect/closure calls are ~8x slower (39-43ms) and arithmetic is its weak spot (arith_poly 55ms); an s7 GC bug at safety=0 was found and worked around (safety=1) in bench.scm.

arith_int hot loop: ~50 → ~29 executed instrs/iter (untagged registers, zero frame stores, magic modulo = 17 of the 29). Correctness: 6 real bugs found+fixed this arc; the record_field/fib “hang” was re-diagnosed as a pit CLIENT stream-teardown artifact (daemon actor completes; read observe.jsonl for ground truth). Permanent gates added: overflow trio, record_grow_keys, msg_bench. Remaining levers: num promotion (floats), native leaf calls, per-actor nursery (the one runtime-design conversation left), mach fused field ops (parked by design).

Update 2026-07-09 late (through commit 20cd0b31) — the perf-arc wave merges: native call/closure/float fixes, inliner, sound type facts, interning, array element-type facts, constant-divisor magic division, slot registerization. Post-merge native lane (all match=Y, gates green): arith_int 14.7 (2.05x), arith_poly 11.4 (2.38x), loop_nested 16.5, float_math 12.7 (was garbage), call_hot 8.3 (2.63x — beats LuaJIT’s 9.6), array_read 13.5 (2.83x, sound — element-type facts), arrfor_sum 25.6, record_new 24.7, gc_churn 7.9 (was slower than mach), array_grow 2.1, string_concat 0.27. mach lane: call_hot 21.7 (was 59.7, inliner), arrfor_sum 57 (was 148). Key lesson from registerization: arith_int’s loop is now fully register-resident (zero frame stores, sdiv eliminated) yet ~14.7ms stands — the remaining cost is tag/untag + int32-overflow checks (~50 instrs/iter vs OCaml’s ~10). The path below 10ms is untagged register regions + range-fact overflow elision. Still open: native-frame GC bug (record_field hang, fib flake — fix in flight), de-rooting sweep in flight. Details in the sections below; pre-merge baselines kept for history.

Update 2026-07-09 (commit 2b78c007) — first perf merges landed, make check green: native record/array literals now pre-size (B) and MACH_PUSH grew an inline fast path (C). Post-merge: record_new native 38.8→23.5ms (nat_alloc 75→42MB, = mach; now ~Lua-speed), gc_churn native 13.4→8.5ms (flipped from 0.94x — slower than mach — to 1.41x faster), d_rec3 native 33→18.4ms, d_arr3 alloc halved, array_grow mach 5.2→4.3ms. Canaries (arith_int, array_read) unchanged. Numbers below this line are the pre-merge baseline unless marked otherwise.

Measured 2026-07-09 on this branch (macOS arm64, Opus session). Harness: aot_bench/vs.ce (dual-lane driver) over aot_bench/suite/*.ce (15 fixtures). Each fixture is compiled through the same streamlined IR and run both under mach (mach.mach_load) and native (compile_native_ir → dylib_open → native_module_load); results are cross-checked (native == mach) and per-run $runtime.vm_stats() deltas record instructions / allocation / GC for each lane.

Run it: restart the daemon with PIT_ALLOW_NATIVE_LOAD=1, then BENCH_ONLY=<name> PIT_ALLOW_NATIVE_LOAD=1 ./pit aot_bench/vs.ce (one fresh daemon per fixture isolates the fixtures that currently crash native).

TL;DR

• AOT is 2–3× on compute, ≤1× on allocation. The 2–3× is real but it is the ceiling of a codegen that still spills every slot to frame memory each iteration; the ≤1× on memory-bound code is a floor set by the shared C allocator/GC that native cannot touch.
• Native boxes integers on stores — record_new/gc_churn allocate ~1.8× more bytes under native than mach, adding GC pressure that erases the speedup. Same representation bug corrupts float_math.
• The interp-vs-native differential is not in the gate, and it matters. This driver found 6 emitter bugs (calls, recursion, closures, floats, record store, module calls) in one pass. The opt/noopt fuzzer is blind to all of them.

Runtime: mach vs native

RUNS=7 median, 3M/2M/… iteration loops. mach_instr = VM instructions dispatched (mach); native dispatches ~0 (it is native). alloc/gc are per single run.

Why compute tops out at ~2×

mach costs 0.73 ns / instruction (independently confirmed by the budget tool). arith_int is 13 instructions/iteration. Native removes 100 % of the VM dispatch (mach 39M instrs → native 51) yet only halves wall time, because the generated code re-stores every live slot to the frame on every iteration and re-materializes loop-invariant constants — QBE keeps those stores because %fp is a parameter it can’t prove non-escaping (GC scans it, get/put reach it, the resume dispatcher reads it). The IL bears this out: arith_int’s 9 source lines expand to 1425 IL lines, dominated by str/ldr against the frame.

array_read hits 2.82× — the highest — precisely because index reads are the case where mach’s dispatch overhead is the largest fraction of per-op cost, so removing it helps most.

Why memory-bound code is floored (≤1×)

gc_churn is slower under native. The work is almost entirely Pit_NewRecord/Pit_NewArray + GC, which is the same C code in both lanes; native removes dispatch that was never the bottleneck and adds a little overhead. string_concat (0.74×) is memcpy-bound in the shared runtime. record_new (1.17×) and array_grow (1.49×) scale with how alloc-heavy they are.

Half of a representative language workload is memory-bound, and on that half AOT structurally cannot help. That is the single biggest reason the headline number feels underwhelming.

Native over-allocates (correctness-adjacent perf bug)

record_new: mach 42 MB/run, native 75 MB/run (1.78×). gc_churn: 18 MB vs 32 MB (1.83×). Native is heap-boxing integer field values on store_field/store_index (streamline types them int as a math type; native treats that as float and boxes). ~110 extra bytes per 3-field record ≈ 3 boxed ints. This both inflates GC and is the same representation unsoundness that makes float_math return garbage.

Emitter correctness gaps (found by this driver)

These contradict earlier notes that calls/recursion/closures “work” — they work for tiny top-level cases (fib(15)) but fail in hot loops / at depth / with type facts on. Prime suspect: the P1a guard-elision (reads fn._write_types) is unsound on the call/float paths, and there is no interp-vs-native gate to catch it. compare_aot (the intended oracle) is itself broken — it fails identically on fixtures that run fine here, so it verifies nothing.

Compile speed

Frontend (mach path), real files — aggregate ~4 klines/s end-to-end, but wildly phase-skewed:

Streamline is the compile bottleneck by 10–30×. (tokenize.cm: 528 lines, 136.6 ms total, 103 ms of it streamline.)

Native/AOT compile — qbe_emit (pure-pit codegen) is fast: 0.5–1.2 ms even producing ~1500 IL lines. The cost is the C toolchain: ~57 ms fixed per file (os.qbe assemble is in-process and cheap; the cc -c and link subprocesses dominate). For a 9-line fixture that’s ~28× the frontend. AOT compile is entirely toolchain/process-spawn bound and roughly fixed-cost per file, so throughput in klines/s is meaningless at this size — the number to cut is the per-file 57 ms.

Cross-language comparison

Same 12 algorithms + iteration counts ported to Lua/LuaJIT/QuickJS/CRuby/mruby (aot_bench/cross/bench.{lua,js,rb}), each self-timed (calibrate to ≥50ms, min of 3, warm — same shape as the pit numbers). Result values match pit across all runtimes (arith_int=144, fib(30)=832040, array_read=630533…), which validates the ports — and Lua/JS/Ruby all compute float_math=525855, confirming pit-native’s garbage there is a real bug. ms per workload, lower = faster:

(Lua uses CPU os.clock; others wall — equal for single-thread compute. mruby caps array size, so array_grow N/A. native ✗ = the emitter bugs above. C# = dotnet run -c Release, RyuJIT tiered, default workstation GC, class-typed records = heap allocation.)

C# (RyuJIT) is fastest on 11 of 12 — it’s the performance ceiling of this set — and it sharpens the diagnosis below. It loses only string_concat, where its immutable-string + is O(n²) (23ms) and pit/QuickJS’s amortized append win.

OCaml 5.4.1 (ocamlopt, no flambda) — added 2026-07-09 as the AOT-native gold standard (tagged 63-bit ints, offset-field records, bump-allocated minor heap + generational GC, no JIT — the closest architecture to what pit-native can become):

OCaml recalibrates targets: pit-native compute is ~1.4x off OCaml (frame stores), arrays/grow are already OCaml-class, and the alloc-heavy 30-70x gaps are representation (offset fields) + generational GC, not codegen.

Reading it

• pit-mach is a strong interpreter — Lua-5.5-class on integer compute (arith_int 29.3 vs 28.2), and faster than QuickJS/CRuby/mruby on nearly everything. Not slow.
• pit-native is in the JIT tier on straight-line compute — it beats LuaJIT on arith_int (14.0 vs 15.6), loop_nested (16.1 vs 19.3), and array_read (12.9 vs 14.0), and trails only C# (~1.6–1.8×). pit’s mach→native speedup (1.8–2.8×) equals Lua→LuaJIT on the same code. The “2–3×” is not the disappointment.
• The whole gap is allocation + abstraction machinery, exactly where pit is worst-in-field and pit-native is broken:
  • allocation — record_new: pit 45 mach / 39 native, dead last; LuaJIT 1.5, C# 1.2, even CRuby 17. gc_churn: pit 12.6/13.4 vs C# 0.6 (22×). C# is the tell: it does NOT do allocation sinking (it heap-allocates the class), yet it’s 34× faster than pit-native — purely from a generational GC with bump-pointer gen0 allocation. So closing most of this gap does not require LuaJIT’s escape analysis; it requires a generational GC. pit’s non-generational GC + int-boxing-on-store (75MB vs 42MB) is the root cause.
  • closures — LuaJIT/C# 0.5; pit-mach 78 = slowest of all 8 runtimes (mruby does it in 46), native broken.
  • recursion — fib C# 1.1, LuaJIT 3.3; pit-mach 84.9 = slowest of all 8 (mruby 45), native broken.
  • calls — call_hot C# 5.4, LuaJIT 9.6; pit-mach 59.7 (beats only mruby), native broken.
  • floats — pit-mach 21.6 (~2× Lua), native broken.
• pit strengths: integer loops (JIT-tier in native), array reads (native beats LuaJIT), and string append (0.3ms — pit and QuickJS use amortized/rope append; Lua/Ruby/LuaJIT and even C# pay O(n²) on immutable strings — C# is slowest here at 23ms, a genuine pit win).

What this re-prioritizes

pit’s straight-line codegen is already JIT-class; the losses are all allocation + abstraction machinery (calls, closures, records, floats), which coincide exactly with where pit-native is unfinished. Highest-leverage work:

1. Generational GC + cheap young-gen allocation — the single biggest win. C# proves it closes ~30× on alloc-heavy code with no escape analysis. Add a bump-pointer nursery + generational collection, and stop int-boxing on store_field/store_index (also fixes the native over-alloc and float bug). Helps mach and native equally.
2. Call + closure path — pit-mach’s call is ~4× Lua’s (fib 31 ns/call vs 8), closures worst-in-field. Fix the mach path (helps now) and the native call/closure codegen (currently crashes).
3. Record representation — pit records are slower than Lua tables and Ruby hashes; profile Pit_NewRecord + field load/store.
4. (later) escape analysis / allocation sinking — LuaJIT-style, for the last factor once a generational GC exists. And slot registerization to push the already-competitive compute path past C#.

record_new decomposition: representation, not GC

Controlled ladder (aot_bench/suite/d_*.ce, all 300k iters, mach lane; mutator = mach_ms − gcms):

record_new (147 ns/record) splits ~70% field ops / ~16% allocation / ~14% GC. The proof is d_field: the exact same field traffic with zero allocation and zero GC still costs 31ms — 70% of record_new. Fixing the GC entirely takes 44→38ms; it is not the bottleneck. d_arr3 (identical shape as an array, i.e. offset access) is 2.4× faster and 3.6× smaller than the record.

Mechanism (confirmed in C): pit records are open-addressing hash tables (rec_find_slot/rec_get in runtime.c), and mach field access has no inline cache — every r.x runs pit_mach_record_get (pit_module_internal.c:45): pointer-chase + type-check + 2× Pit_Push/PopGCRef + key interning (pit_key_from_string) + hash + probe. ~15–22 ns/field vs C#’s ~1 ns fixed offset. The table also over-allocates (8 slots × 16 B for 3 fields → ~140 B/record vs an array’s ~40 B), which is the 42 MB vs 11.7 MB above.

So for record-heavy code the top lever is record representation — a shape/hidden-class with fixed-offset fields + an inline cache on MACH_LOAD/MACH_STORE — not the GC. (Generational GC still matters for allocation-rate-bound churn like gc_churn, where GC was ~25%.) This refines the C#-comparison conclusion below: C#’s record_new win conflated offset-fields (the big part) with its generational GC; the decomposition separates them.

Ranked levers

1. Slot registerization — promote purely-local int/float slots to QBE SSA temps + phis so they never touch frame memory; QBE then registerizes across the loop back-edge and hoists invariants. Needs escape analysis (get/put → stay in frame), GC-safety (pointer-holding-across-alloc → stay in frame; int-only safe), phi insertion for loop-carried slots. Breaks the compute ceiling — target 4–10× on arith/array_read. Biggest single win.
2. Representation-sound integer stores — stop float-boxing int-typed values on store_field/store_index (and generally). Fixes the 1.8× over-allocation on record/array code and the float_math miscompile. Helps the memory axis and correctness at once.
3. Fix call / recursion / closure codegen — everything real calls functions; these must be sound before AOT ships anything beyond microbenchmarks.
4. Put the interp-vs-native differential in the gate — aot_bench/vs.ce found 6 bugs the fuzzer can’t see. Wire it into make check so codegen edits can’t silently miscompile.
5. Leaf-function fast path — skip the resume-dispatch prologue for functions with no invoke points; cuts per-call overhead (call_hot pattern).
6. Attack the shared runtime floor (helps mach and native) — inline record/array alloc fast-path, bump allocation, cheaper GC. This is the only way alloc-heavy code gets faster; AOT can’t do it.
7. Unboxed float slots — same registerization idea for doubles; helps float /graphics workloads (nbody, spectral-norm).
8. Cut AOT compile toolchain cost — the 57 ms/file is cc+link subprocess; batch, cache, or emit machine code directly. Only matters for the dev loop.

Source: plans/archive/perf-2026-07/pit-native-benchmarks.md