Building C

Everything below the confined layer is C: the VM, the allocator, the operating-system seams, the endowment providers, and any native module a package ships. Two pieces decide what that C is and turn it into binaries — cake, which answers what one file needs to compile, and the runtime builder, which derives the file list from a recipe, drives cake over it, and links the result.

Cake

Cake’s job, in one sentence: one file in, one object out, with the flags its package declares. It answers what a source unit compiles to and what it needs to get there; it does not decide which units exist, and it does not link.

There is no second build system underneath cake holding a parallel copy of the file lists, and nothing generates input for one.

There are no mapping manifests anywhere in it. A module’s name is where the file is, so there is nothing for a table to say, and cake’s answers are computed from the tree rather than read from a list somebody maintains. C symbols come from one authority, shoplib::symbols, and every consumer reads the symbol it was given rather than assembling one.

A target is a recipe

Nothing in cake, in the shop, or anywhere else takes a target name to mean “these things.” Building is pulling levers: these packages available as endowments, these embedded whole, this collector, this value representation, these fallback and bin lists.

A recipe is a named bundle of lever settings, committed as a file in recipes/. The name is something the recipe declares — a label for its own settings — and never something cake interprets. “I want to build for ios” must never reach code that knows ios means darwin_file; it reaches the ios recipe, which says so itself.

The endowment lists live here rather than shop-wide, which is what lets one recipe draw from a posix package and another not — see Endowments.

A recipe describes the runtime binary and nothing else. What a boot section or bundle carries — which actors, which assets, which payload form — is chosen when that product is composed — the composing invocation’s own arguments (--payload, --with), with the standard invocations committed in the makefiles and scripts. Those are per-build answers, and putting them in a recipe would make them per-target answers to questions that are not.

The file’s schema, the two lever groups that never mix, and what the platform package declares for itself instead are Recipes. shoplib::recipe is the one reader; the runtime builder and cake both ask it rather than keeping tables of their own.

The one sanctioned place a target name appears outside a recipe is a package’s own module arms — {"darwin": "internal/file_darwin", …} — where the package is declaring how it varies, matched against the name the recipe supplied. The package speaks; cake only carries. Anything else that branched on a name would be a second target-branching mechanism, and there is only one.

Cake gets its per-package answers two ways — a standard one that needs no code, and a program for the cases the standard cannot express.

The standard path

Most packages declare nothing and write no build code at all: the closure names the modules, and the directory names the files. A package.json carries only what a directory cannot say, and it has three fields:

• aliases — short names this package’s modules may write for other packages.
• modules — a logical module name mapped to a per-target implementation. This is what makes std::file’s use('file_provider') reach internal/file_darwin on macOS and internal/file_linux on Linux: the package names the arms and the recipe’s target name selects one. It is the only target-branching mechanism above the C boundary, and it serves .cm and .c alike. A name here needs no file of its own — the arms can be the whole definition — and putting the implementations under internal/ is what makes the abstraction enforced rather than merely intended.
• compilation — CFLAGS and LDFLAGS, with per-target overrides.

An include/ directory is added to the include path automatically, and everything in source/ is support C. A normal package build brings it in with that package’s modules; a runtime recipe brings it in whenever the package is selected by linked_packages or a provider list. Both facts come from position rather than a declared source-file list.

There is no name field: package identity comes from the shop resolution that selected the package. There is no source-membership list: a C file’s package-relative path is its module name, and source/ is support C by position. There is no members field: the shop supplies the selected package identities, and Cake resolves only those identities. It never walks a checkout to invent packages from nested package.json files.

From those, cake computes each module’s implementing file for the target and its flags. For most packages that is the whole story, and there is no cake.ce.

The program path

Some builds need a decision the manifest cannot state — refusing a target outright, choosing an implementation by probing the host, or handing back a list of vendored sources. For those, a package puts a cake.ce beside its package.json, and the shop starts it and sends it questions. The package’s build program is a program, in the same spirit as a rakefile — code the tool runs, rather than a data format it parses.

Three questions:

A cake.ce overrides the standard answer rather than replacing the system: it starts from the same computed file and adjusts it, and a package that has nothing to say for a phase says so and the standard result stands. pit-qbe is the real example — it refuses non-desktop systems, points its module at the backend source, and puts its vendored QBE files in source/.

The directory itself

Before either of those, cake reads the tree. A module’s name is where it is: the file internal/<stem>.c in a package provides that package’s module internal/<stem>, so cake finds a package’s C modules by listing it — listing the directory is reading the list, and nothing restates it. Adding a module is adding a file, and adding hardware support is publishing a package and a shop naming it, never editing a build table.

internal/ is the package-private module directory. What lives in it is named from inside the owning package and nowhere else, so privacy is a fact about where the file sits rather than a rule someone has to check.

source/ is the one directory that yields no modules. If a package compiles any C module, it is compiled with everything in that package’s source/ — the folder is the unit, so there is no per-file support list to keep and no way for a helper and a module to disagree about which one a file is.

The runtime’s own plan

The runtime is built the same way at scale: cake/plan.cm is one module that is the runtime’s build program, returning its file lists, the collector and text arms, and the link flags — each answered for the recipe it was handed rather than enumerated per target. It is what the target matrix is rendered from, and the matrix is a record of recipes people have written rather than a list cake consults.

Declaring C without writing a cake

Most packages need no cake.ce and no declaration of any kind. Putting render.c at the package’s top level is what makes <pkg>::render exist; the shop’s c_native list is what permits it to be compiled, and it permits by naming the package. A development shop names broadly and a hardened one names narrowly, but both name: absence is never permission.

Where one module has genuinely different implementations per target, the manifest’s modules arms say so:

{
  "modules": {
    "render": { "emscripten": "render_web", "*": "render" }
  }
}

The "*" arm is the default. Naming a system explicitly overrides it, and a package that supplies nothing for a system says so, which keeps that module out of the build for that target entirely.

The same target-arm shape appears at two other scales, so one idea covers all of them:

• alias arms select a whole package per target;
• compilation blocks carry per-triple CFLAGS and LDFLAGS.

A module that needs an optional host library says so in the C file, for the same reason its name is where it is:

PIT_USE_PROBE(libgit2)

The build reads that declaration and compiles the module only where it detects the library. There is no manifest column for it — the requirement belongs to the file that has it. Three more facts follow the same rule and are scanned by the same pass: PIT_USE_BOOT(<fn>) names a bring-up function, PIT_USE_LANE(<lane>) says this provider must run on a particular lane, and PIT_USE_LEASE(<slot>) says it holds an exclusive OS slot. Each is a fact about the C, so each lives in the C.

What cake produces

Cake renders its answers into build manifests, one per recipe. They are committed to the repository, and that is deliberate: it means a machine with a C compiler and nothing else can build the runtime, because the file list is already there to be read. That machine is who they are for — the runtime builder asks cake directly, since it is running inside a pit and has no reason to read a rendering of an answer it can have.

(A build manifest is cake’s rendered output. A build description is the composer’s invocation of the bundling tools — arguments, not a file. One says how to compile the runtime; the other says what rides on it.)

They are rendered outputs and never an input anyone maintains. A regenerated manifest carries file lists, flags, and each module’s key and full C symbol — the symbol as data, exactly as shoplib::symbols produced it, so that no consumer of the manifest ever assembles one from parts. What it does not carry is a mapping: nothing in it says which file provides which module, because a file’s name is its path and there is nothing left to say.

It also carries the recipe’s levers, rendered as compiler flags — one row per flag, in the order cake emits them, with a row kind per group because the groups never mix:

stamp|-DPIT_PROFILE_NAME="n64-nan32-v1"
stamp|-DPIT_VALUE_BITS=32
stamp|-DPIT_WOTA_BITS=32
stamp|-DPIT_PROFILE_RECORD="hash"
stamp|-DPIT_REP_NAN32=1
build|-DPIT_SINGLE_THREAD=1
build|-DACTOR_AR_S=60
build|-DACTOR_FAST_TIMER_S=1
build|-DACTOR_SLOW_TIMER_S=60
build|-DACTOR_SLOW_STRIKES_MAX=3

A stamp| row is a fact every artifact repeats and a mount compares; a build| row is a fact about the binary alone. The compiler cannot tell them apart and should not — a consumer concatenates both kinds into its flags — but a reader can, and the split is what keeps a lever from drifting into the wrong group unnoticed. The scheduler arrangement and the lifetime budgets are build| facts: they select provider files and set the binary’s own timers, and change no byte in any artifact.

This is how the recipe’s authority reaches a build with no pit in it. The runtime binary refuses to compile unless the build states its own profile name, value width, wota width and record lookup, and the recipe is where those come from — but the cold floor and the cross build scripts cannot read a recipe, because reading one means running a validator and they have no runtime to run it in. So the rendering carries the decision, exactly as it already carries the file lists and the symbols. A consumer concatenates these rows into its compiler flags; it never parses one, so no second place in the tree knows what a stamp lever means. The collector and text-body arms are selected from these same rows, which is what makes it impossible for a build to compile one collector while stamping itself with the other.

The same rendering produces a cold bootstrap script: POSIX shell plus a C compiler, self-contained, no pit binary required. That script is the floor the whole system stands on, and everything above it is built by pit.

The runtime builder

Pit source has a compiler and an assembler; so does C. Cake is the C compiler driver — one file in, one object out, flags from the package. The runtime builder is the assembler: it reads the recipe, derives the translation-unit list from the platform package and the lists, drives cake once per unit, links the result, and renders the static extension table — the generated C that lets a build reach its native modules by symbol — from shoplib::symbols. It invokes the host compiler, or a cross toolchain when the target is not the machine you are on. There is no second build system underneath it.

It is called forge, it lives beside cake in the same package, and it is a command:

pit forge darwin
pit forge n64 --out build-n64 --cc mips64-elf-gcc
pit forge darwin --tests --set stamp.text=kim8

The target is a recipe name and nothing else. --out places the products, --cc names the compiler, --jobs bounds the compile fan-out, --opt sets the optimization flags, --flags carries the toolchain environment (a cross triple, an SDK root — facts about the machine doing the compiling, which no committed file can know), --define adds a preprocessor switch, --tests also builds and runs the C tests, and --force ignores the object store. There is no --gc, no --text, no --numrep: those are lever positions, and a lever position is a recipe.

--set stamp.<lever>=<value> is not a way back in. Committed recipes are suggestions, so a build is allowed to differ from every one of them, and --set is a build saying which lever it moved — validated by the same rules a file is, so the two groups still never mix and a build lever written under stamp is still refused by name. It exists because the profile sweep varies thirteen positions that are nobody’s target: they last for one QA run and describe no machine, so they are arguments. A position someone actually builds for is a recipe, and writing one is how you make it one.

The arms

make check-arms is the sweep that keeps the profile space honest: make check compiles one point of it, so every other collector, text body and value word would otherwise go uncompiled. Each arm is the host’s own recipe with one or two levers moved, built and C-tested in its own directory, and the sweep reports every arm rather than stopping at the first red one.

What it derives, and where each answer comes from

Every row is a read. The builder contains no branch on a target name, and the one fact it supplies itself is which machine is doing the compiling — the host architecture that goes into PIT_TARGET_ARCH.

The toolchain is the gate

The builder keeps no list of what a host has. It compiles every unit the plan names, and a unit whose C declares an optional host library with PIT_USE_PROBE is allowed to fail: when it does, that module leaves the static table and its package’s link flags leave the link line, together. A unit that declared no probe and failed stops the build with the compiler’s own error, because that error is the answer. Nothing is maintained, so nothing drifts.

Objects are content, not timestamps

Each object is named by the hash of its command line, its source bytes, and the state of every header the build could include, and it is stored in the shop’s build directory beside every other content-addressed artifact. A rebuild that would produce the same object finds it. Timestamps are consulted nowhere: a checkout can land older content with a newer mtime, and a build that believed the timestamp would link the previous idea of the runtime.

One binary

The product is one self-contained executable. Forge generates a tiny assembly source whose body is an .incbin of the chosen boot image, with a fixed start/end symbol pair, and links that object with the runtime. There is no pit.bare, append step, repack target, or runtime file lookup for boot bytes. Module dylibs resolve the runtime’s symbols from the host process rather than linking against a runtime library, so a static runtime plus the platform’s dynamic-export flag serves the dynamic lane exactly as a shared library did — and it leaves the builder with one link command instead of a per-platform spelling of shared libraries, install names, and run paths.

Forge never builds in place. It writes a candidate binary and candidate boot bytes in the build directory, boots that candidate, runs its smoke subset, and has it reproduce its own boot image. The clean candidate promotes only when the reproduced bytes compare exactly. Promotion replaces the executable atomically and updates the committed boot input explicitly; an interrupted or failed candidate leaves the installed pair untouched.

Before the Pit lowering exists, an ABI move adds one internal step: forge builds the candidate with read acceptance for exactly the previous Mach generation, uses that candidate’s C lowering to press the new generation, then rebuilds and gates without acceptance. This is a forge-internal build mode, never an environment variable or user option, and no bridge-produced byte is promoted. P5 removes it after Pit owns lowering.

Its interesting decision is how a native module reaches the runtime, and there are two answers:

Dynamically. The module becomes a shared library, and the runtime loads it when an actor claims it. This is the development arrangement: rebuilding one module does not relink anything else, so the edit-run loop stays short.

Statically. The module is compiled into the runtime binary and reached through the generated extension table. This is what shipping looks like — one binary, everything it needs already inside it, nothing to find at run time. It is also the only arrangement available on a target that has no dynamic loading, which is most of the constrained ones.

The same C source serves both; the difference is how it is linked and how the runtime finds it.

The C tests

The C the builder compiles has tests that have to run underneath a runtime rather than inside one — the object ABI, the pool reader, the collector’s card table. pit forge --tests builds and runs them against the very objects it just linked, so a test can never be testing a differently-configured runtime than the one it reports on.

A test is a file: <name>_test.c in the runtime package’s source/ or tests/, or in any directory this build already takes platform C from. Listing those is reading the list, so nothing enumerates tests and adding one is adding a file. It also settles the per-platform question structurally: the posix daemon-lifecycle fixture is found on the platforms whose package names posix files and is simply not there on the ones that do not.

Every test compiles under every arm. A test with nothing to assert under this build’s profile says so itself and exits 77, which reports as a skip — it does not ask to be left out of the build, because a test the build leaves out is a test nobody can see is missing. What a test needs beyond being compiled and run is likewise a fact about that C, so it is declared in that C: PIT_TEST_SOURCE names an extra translation unit, PIT_TEST_CFLAGS and PIT_TEST_LDFLAGS add flags, PIT_TEST_STANDALONE says it links no runtime object, and PIT_TEST_RUN gives one environment to run under (repeat it to run more than once). The same scanner reads them that reads PIT_USE_PROBE.

The two builds meet at the extension spec

The boot-image build and the runtime build meet inside forge. From its own closure of actors, the boot-image build emits the exact set of C modules and endowment providers those actors can reach — package, file, full symbol, and the facts each C file declares about itself. That is the extension spec.

A dev runtime links from the shop’s whole lists and takes no spec: it does not yet know what will be built against it. A ship runtime links exactly the spec, trimmed to the claims, and nothing else. So the bootstrap order is the order of what is knowable — dev runtime from the lists, boot image from its closure, ship runtime from the extension spec — and each step names its input rather than guessing it.

Compiling the runtime itself

Both mechanisms above describe C that a package contributes. The runtime is built the same way, from the same recipe: the VM, the collector chosen by the GC axis, the text implementation chosen by the text axis, the platform package’s sources, and whichever capability levers the recipe set. A runtime binary is a target profile made concrete — see Target Profiles.

That is why a build for another machine is a build of a different runtime, not just different application code: the same plan, answered for a different target, produces a different set of C.