Archive — history, not state. Kept for its reasoning and its evidence; its plan is closed.

Native raw signed-int self recursion experiment

Date: 2026-07-13

Measurement base: e0a4a67e

Composed integration base: 637d7204

Integration branch: codex/integrate-raw-self-call

Result

This experiment proves that carrying a signed integer in a private native ABI through a recursively hot region is a material lever. On the exact base, native fib(30) falls from 30.460 ms to 24.201 ms median: 20.5% less time (1.259x throughput). The language ABI, Pit frames, suspension contract, moving GC behavior, and non-selected functions remain unchanged.

This is deliberately a narrow proof, not yet a general integer calling convention. The current recognizer accepts only the fully proven shape:

if (n < 2) return n
return self(n - 1) + self(n - 2)

It is structural, not source-name based. It walks the optimized instruction trace and verifies the constants, relation, both self calls, both decrements, numeric addition, and returned sum. It fails closed if the trace changes. It also recognizes the generic numeric guards retained when the function value escapes, but follows those edges only because the public wrapper has already proved a tagged integer in the safe range.

Composed integration

The experiment replayed without textual conflicts onto 637d7204, which also contains native record templates and structured raw numeric TCO. Focused composition checks found no target-code interference:

• compiler suite: 105 passed, 0 failed;
• fib still emits $pit_fn_0_rawi and Mach/native both return 832040;
• the boundary matrix still emits the raw entry and both lanes return 78;
• record_template emits its native descriptors and $__new_record_template_ss calls; both lanes return 853729;
• record_template_suspend returns 813729 in both lanes and also passes the native suspension harness with forced moving GC;
• tco_self retains %slotreg_0 in the raw integer lane and %slotreg_1 in the raw-double lane; both lanes return 12500002500000;
• the structured-TCO suspension fixture returns 1250025000 across 48 forced suspensions/collections at maximum native depth 2;
• emitted QBE IL for record_template and tco_self is byte-for-byte identical to pristine 637d7204.

The only target-code changes observed are functions matching the exact proven Fibonacci CFG (the suite fixture and its dedicated correctness/suspension variants). Record-template allocation, structured TCO, and their helper/data emission remain unchanged.

What changed

For a selected nan64 function, QBE now emits three entries:

• the existing public tagged entry name, now a small guard/wrapper;
• the complete, untouched tagged body under a private alternate name;
• a private body taking and returning one unboxed signed integer.

The raw body does not bypass actor fairness. Every logical recursive call still uses pit_rt_self_prep, constructs the normal tagged and GC-scanned Pit frame, charges the actor, and can suspend before entering the child. The first result is packed into its ordinary frame slot before the second call, so that frame is the sole continuation state if the native host stack unwinds. Resume segments 16385 and 16386 are tested by exact equality; the wrapper does not claim later tagged segments.

The public wrapper checks:

1. fresh entry or one of the two exact raw resume segments;
2. tagged integer input on fresh entry;
3. n <= 46, because fib(46) is the last result fitting signed int32;
4. n < 2, which can return without consulting the self binding;
5. for recursive inputs, the captured self slot still contains this closure.

Anything outside that proof executes the original tagged body. This matters because the outer fib binding is mutable even when a saved closure remains callable. Checking the binding once dominates all private calls: the accepted hot trace contains no callback, mutation, or other user-code re-entry.

Private results reserve INT64_MIN as a control-unwind token and INT64_MIN + 1 as a wrapper-only defensive fallback token. Neither intersects the proven int32 domain. Negative results are repacked with zero extension of the int32 bit pattern before shifting, exactly matching nan64’s canonical tagged integer constructor; sign-extending before the shift would be wrong.

No mcode operation or language behavior changed. The emitter gained an optional function-name override so the pre-existing tagged body can be retained verbatim.

Correctness and fallback matrix

aot_bench/regress/native_raw_self_int_boundaries.ce checks every result individually before returning its golden value:

Mach and native both return the golden 78. Dumped IL confirms that this escaped-function fixture actually receives $pit_fn_0_rawi; it therefore tests the wrapper instead of silently exercising only the tagged body.

The compiler tests additionally prove:

• the exact Fibonacci CFG is selected;
• escaped generic numeric scaffolding is selected only with the wrapper guard;
• exact resume-segment tests are emitted and there is no broad >= claim;
• self(n - 3) is rejected;
• observable work before the base split is rejected.

Suspension, disruption, and moving GC

The private body treats a null return from pit_rt_self_prep as either pre-entry suspension or disruption. Suspension calls pit_rt_native_control_return with the correct continuation segment; disruption unwinds without falsely popping a child frame.

The native suspension harness passed these final binaries:

Budget 1 forces the pre-entry path on essentially every recursive activation. Depth 34 verifies that the host stack can be discarded and reconstructed well beyond the shallow case.

Performance

Three interleaved fresh-process perf_one runs were made in this worktree and an untouched worktree at exact base e0a4a67e. Each row is itself the median of 11 timed executions after three warmups.

The medians imply a 6.259 ms or 20.5% reduction. Mach medians were 52.300 ms and 51.936 ms respectively (+0.7%, ordinary run noise); this is a QBE-only change and the Mach program is identical.

The deep fib(34) oracle measured 352.641 ms Mach and 166.681 ms native on the experiment. It is primarily a correctness and suspension-stress fixture, not a base comparison.

The four language shootouts were also run and returned exact matching values:

More importantly, current emitted QBE IL for all four is byte-for-byte identical to the exact base. None matches the narrow recognizer, so this experiment cannot change their executable performance.

Code and memory tradeoffs

For fib.ce on Apple arm64:

The 948 linked bytes are 308 bytes of public wrapper plus 640 bytes of raw body; the original 6,496-byte tagged function is byte-for-byte unchanged. The Darwin dylib file grows from 54,448 B to 71,056 B because the extra text crosses a 16 KiB Mach-O segment boundary. Static/console builds should be judged by the 948 bytes of actual text, not that host packaging discontinuity.

There is no added target heap object, side table, profile state, or wider Pit frame. Every recursive activation retains the same normal Pit frame. On this arm64 build the private native prologue reserves 48 bytes of host stack versus 64 bytes in the tagged body; the top-level wrapper reserves 32 bytes once. Thus deep recursion uses 16 fewer host-stack bytes per private activation, not more.

The emitter implementation is large for the very narrow coverage: roughly 400 source lines, mostly explicit QBE for the wrapper and two resumable calls. That cost is compiler source/compile work, not target runtime memory, but it argues for generalizing the mechanism rather than accumulating one shape at a time.

This optimization is nan64-only. f32 and fixed builds emit only the tagged body and both representation golden suites pass. That makes the experiment safe for those builds but gives no speedup to a future 32-bit Playdate/GBA representation yet.

Original branch validation

• compiler suite: 96 passed, 0 failed;
• full local suite: 1,918 passed, 0 failed (rerun outside the filesystem/network sandbox after the sandbox denied loopback binds);
• fuzz: 3,733 passed, 0 failed, 500 cases, seed 20260713;
• nan64, f32, and fixed representation fixtures: all match their goldens;
• f32 and fixed IL: tagged-only, no _rawi entry;
• boundary oracle: Mach/native match at 78;
• forced-suspension, moving-GC, and stack-limit tests: pass as above;
• git diff --check: pass.

Failed/revised approaches during the experiment

The first prototype selected fixed instruction indices. It was rejected before final validation because ordinary optimizer changes would make that proof both fragile and potentially unsound. The final code uses labels, operand identity, and a consumed structural trace.

The first private return path checked both control and fallback sentinels after every child. A private child is only entered on a freshly prepared frame after the public proof, so it cannot produce the wrapper-only fallback. Removing that comparison/branch reduced the raw body from 668 to 640 linked bytes.

The escaped boundary fixture initially did not select the optimization because escape analysis correctly retained generic numeric error paths. The recognizer now understands only the proven integer normal edges, and the compiler test plus IL assertion prevent this oracle from becoming a tagged-only false positive.

Recommendation

The speed/space exchange is attractive for recursion-heavy native code: 20.5% less time, 948 bytes of text for one selected function, no heap cost, smaller per-call host stack, and the full suspension contract preserved. The result supports a general raw signed-integer region/calling convention.

The current exact-Fibonacci recognizer should be treated as the executable proof and benchmark anchor, not the final architecture. The next implementation should derive raw parameter/result facts for arbitrary direct-safe self-recursive regions, preserve tagged spill state at suspension boundaries, and use the same public guard/tagged fallback contract. This would turn the demonstrated lever into useful language-wide coverage while amortizing the wrapper/resume machinery.

Tooling friction

BENCH_ONLY=... ./pit aot_bench/vs.ce did not filter the matrix: a later typed selection path overrides the environment value. The focused perf_one.ce harness was used instead. The benchmark harness should make one authoritative selection mechanism so a requested one-row run cannot silently become the full matrix. No tooling change was made in this branch.

Source: plans/archive/perf-2026-07/perf-native-raw-self-int.md