Archive — history, not state. Kept for its reasoning and its evidence; its plan is closed.

Compact linked value/origin/effect spine

Date: 2026-07-14

Status: isolated Gate-1 representation and benchmark-consumer experiment. It is not imported by streamline, Mach, QBE, the linker, or the runtime. The later artifact audit deleted the rejected linked-call pairer, origin sidecar, tests, and probes described historically below; this document preserves their measurements, not live executable infrastructure.

Decision being tested

The accepted structured analysis proved that stable values across finalized control flow can materially improve generated native code. Its object graph is not an acceptable durable linked representation: sampled compiler corpus units retain about 22-23 times their raw mcode footprint, and the original prototype reached a sampled 512 MiB compiler heap.

pit-compiler/compact_spine.cm projects that existing view into eleven packed blobs. Dense IDs are valid only within one linked compilation. Portable identity remains:

(unit content hash, local function id | main, finalized semantic mcode PC)

Adding or reordering a linked unit can renumber dense IDs without changing the portable origin. Unit hashes are sorted before IDs are assigned. Operations carry exhaustive effect bits; functions aggregate them. Packed tables retain functions, operations, operand slices, values, blocks, normal/exception edges, dominance, phis and inputs, natural loops and member sets, and exact cross-unit links. Boundary APIs decode records only when a compiler consumer asks for one. The spine retains neither semantic mcode nor the record view.

build_views is important: QBE already constructs the accepted structured view. A linked compiler can project that same view once and release it rather than constructing the expensive graph a second time.

Gate-1 retained measurements

The first complete packed-only pass covered the five self-host compiler units:

This passes the retained-layout gate of at most twice raw mcode and is roughly 24 times smaller than the old graph. It is a compiler-only cost; no table is carried by a generated Mach or native executable.

The first shared-view incremental samples were:

The projection still allocates temporary Pit arrays/maps while it writes the blobs. Most die when build_views returns; the exact retained footprint above is authoritative. A production direct packed builder should remove those temporary maps after executable levers are known.

This does not solve the existing structured builder’s peak. The first packed-only corpus command allocated 1,276,367,120 bytes in total because it also constructed the record view. A follow-up attempted to measure actor.value_footprint for streamline’s full view; the diagnostic traversal ran for several minutes at about one full CPU, starved the shared worktree’s daemon, and blocked an unrelated pit ps. The exact probe PID was stopped. The probe now bounds detailed record-footprint traversal to functions below a corpus-size threshold. Large probes must run in an isolated process after other work releases the node.

First executable consumer

The first consumer must not repeat static export caching. Its target is the call after the exported value has already been cached: a linked graph proves that one call origin always invokes one exact Pit function body in another unit.

pit-compiler/cross_unit_inline_experiment.cm implements a narrow pre-streamline falsification. It directly splices a provider leaf into the consumer; it creates no function object, new opcode, target metadata, or runtime state. The proof is deliberately fail-closed:

• direct call only, with exact instruction length, numeric line/column tail, integer arity zero through four, and legal under-application;
• at most 24 provider operations;
• no closure cells, nested functions, regex/data-table ids, handler, call, invoke/apply, context, heap read/write, allocation, or disruption;
• a positive opcode/operand schema for numeric constants, immediate literals, pure arithmetic/guards/comparisons, modeled jumps, and return;
• the shared structured effect schema must independently classify every admitted nonconstant operation as pure and nonescaping;
• every local slot position and control label is remapped, and caller/provider origins are retained as a side fact.

prove_export establishes the provider side from real optimized module mcode, rather than trusting a member name. The module must construct one literal mutable record, store the named member from one exact function value, and return it. The proof follows only branches whose condition it establishes itself (currently the ordinary is_stone guard around a fresh record store); the panic path is therefore unreachable in the proof. The resolved function must pass the leaf proof. Captures, unknown branches, dynamic module initialization, mutation through calls, and ambiguous writes remain opaque.

The compiler-side importer seam is now prototyped. Mcode retains an instruction-external _linked_call_origins fact only for a direct call through a statically cached use export:

{consumer final function/PC, provider request/unit/export,
 cache slot/level/callee slot, inline context}

This does not alter semantic instructions. The fact is isolated per nested function, stamped with final local function identity, remapped through stone and concat insertions, cloned with source/caller provenance when a local body is inlined, transported through final nop compaction, and updated through local and ancestor slot compression. Canonical unit generation packs it into a linker-only sidecar, then strips it from ordinary target mcode. Malformed claims are discarded or rejected rather than repaired by guesswork.

pit-compiler/linked_call_pairing_experiment.cm now pairs that real generated consumer fact with prove_export over a separately compiled real provider. The constant provider produces one exact link and splices successfully. pit-compiler/linked_call_sidecar.cm stores eight u32 words per fact (exactly eight logical u32 words (64 raw PitValue payload bytes on the current JSON artifact ABI; future packed storage is 32 bytes), deduplicates provider request/unit/export texts once, and stores rare inline provenance in five-word context rows. The pairer can consume the stripped canonical unit plus this sidecar. The remaining missing production seam is shop/link orchestration: it must collect provider unit code, invoke the pairer, then discard the sidecar. The earlier performance tool still uses an explicit synthetic consumer claim over an equivalent array-hidden local closure; it isolates call-path benefit, while the new pairing tests establish that both real endpoint proofs join.

Focused tests cover executable Mach equivalence, actual call removal, no function cloning, real provider export proof, captured/oversized rejection, and rejection of unmodeled control, exception, unit-relative constant, heap, call, allocation, and malformed-tail forms. Compact-spine and cross-unit splice modules each pass 4/4; importer-origin and real-pairing coverage pass 5/5. The existing compiler suite passed 132/132 before the concurrent dense record experiment changed its focused matrix.

The real importer/provider constant pair measured 41 -> 46 finalized main mcode instructions and 3,408 -> 3,536 Mach bits, 3.76% code growth, inside the 10% gate. A generic exported add(a,b) is honestly rejected: its optimized body still retains dynamic type fallback, context/log call, and disruption. Arithmetic cross-unit inlining therefore needs either exact linked argument type proof that makes the fallback unreachable, or a separately reviewed full semantic splice; the pure-leaf prototype does not pretend it is solved.

The follow-up flat-mcode numeric-arm matcher was rejected and deleted. It attempted to recognize the frontend expansion of a real generated two-argument + function and replace an exact numeric linked call with the existing semantic add. The provider is about 170 raw / 156 streamlined operations because the source function also implements text concatenation and type-error construction; the safe numeric arm is only six operations. Even after admitting exact safe postlude variants, the matcher could not recognize the compiler’s own current 156-operation form: only fail-closed and tamper-rejection cases passed (4/10), while all positive specialization and fallback execution cases failed. It never reached a trustworthy timing gate. Weakening the matcher would have coupled correctness to incidental instruction shape, so its module, tests, probe, benchmark fixture, and matrix row were removed.

The replacement design is semantic rather than syntactic. A linked value graph must identify the exact exported function and its operation semantics. For exact int/float argument facts, the linker can clone and specialize the whole function (or inline its semantic body) and then let normal streamline remove the unreachable text/error arms. Honest dynamic, nullable, mixed, and text calls retain the original function. This gives cloning/inlining a stable place to carry origin facts for later PGO. It requires no new runtime opcode and must preserve ordinary disruption, suspension, and actor-freeze boundaries. Benchmarking resumes only after positive semantic-specialization correctness is green.

The first two-million-call Mach A/B uses an exact cross-unit exported zero-argument Pit leaf returning a constant. It hides the function behind an array at the consumer call site, so ordinary unit-local resolution cannot inline it. After two warmups and seven measured executions:

The exact result was 7 in both arms. This is a 50.25% Mach reduction even though the conservative splice retained five extra semantic instructions and grew the Mach blob by 144 bits. The direct evidence is therefore call/frame removal, not a loop-body instruction-count trick. It is deliberately a narrow constant-return leaf: ordinary arithmetic functions still contain dynamic fallback/disruption paths before type proof and remain ineligible.

A second complete run, including ordinary QBE native compilation and module load, confirmed the lever:

The two Mach medians put the repeatable benefit at 42-50%; the native result is the larger prize because the entire dynamic call/frame path disappears and QBE retains the loop scalar directly. Native code grew slightly: QBE logged 27,609 B IL / 25,684 B assembly for baseline and 28,353 B IL / 26,109 B assembly for candidate, a 425 B (1.65%) assembly increase in this fixture. A third complete stats run measured Mach 26.342 -> 13.923 ms (-47.14%) and native 22.906 -> 5.387 ms (-76.48%). The native candidate varied more between runs, but both native runs remain decisive (76-83% faster). Per execution, baseline allocated 720 B in 5 allocations versus candidate 728 B in 5 allocations; both collected zero times. The current conservative splice thus costs one extra 8-byte frame slot, retained only for the live top frame, not per call.

The compiler spine, profile, and link sidecar are not runtime allocations. Ordinary target mcode now contains zero linked-call records. The unit-wrapper sidecar has eight words per fact. Because the current artifact cache is JSON, those words are a flat PitValue array: 64, 640, and 6,400 raw payload bytes for one, ten, and one hundred calls on the 64-bit ABI, plus array headers. A future binary artifact codec can pack the same rows to 32, 320, and 3,200 bytes. Their shared provider text table remained 25 bytes, and these fixtures had no inline-context rows. Before this table, the same record facts cost 173-175 bytes each in canonical JSON. A deliberately conservative boot-source model treating every analyzed member as a linked export had found:

This upper bound motivated the flat table. At the current 64 raw payload bytes/fact its rows would be 0, 7,232, and 192 bytes respectively, plus array headers and one deduplicated text table per unit. Packed binary storage would halve those row payloads. The linker must consume and discard it before Mach/native target artifacts. The A/B tool is tools/cross_unit_inline_probe.ce; metadata/code-size probes are in tools/linked_call_origin_probe.ce.

Disposition gates

Keep the packed representation only if the full corpus remains at or below 2x raw retained and a real consumer wins. Keep the exact-callee consumer only if:

• Mach or native call time improves materially on the linked-call shape;
• normal generated-program heap use does not grow;
• code growth from cross-unit inlining is measured and budgeted;
• real static-binding metadata proves both endpoints without source-name assumptions;
• provider source locations survive into diagnostics/debug sidecars; and
• full, fuzz, moving-GC, and forced-suspension tests remain exact.

If the call pilot does not win, retain the packed origin/effect layout result as research evidence but do not integrate the spine merely for architectural neatness.

Source: plans/archive/perf-2026-07/perf-compact-linked-spine.md