Architecture

Golem is a straight pipeline. Source flows one way; nothing loops back.

Emet source
   │  emetc build
   ▼
binary manifest  ◀── scroll-format (shared model) ──▶  golemd
   │  golemctl apply (POST /manifest)                    │
   ▼                                                     ▼
golemd selects this host's scroll ───▶ diff ───▶ reversible reconcilers ───▶ the box

emetc writes the manifest and golemd reads it. Both compile against the scroll-format crate, which owns the model, so the two ends stay in agreement.

emetc — the compiler (writer)

Emet is a typed, Elm-modeled language (apps/emet/). A program is a set of top-level declarations with Hindley-Milner inference; main : List Scroll is the fleet. The compiler’s stages:

lexer  → layout → parser → infer (Algorithm W) → eval → manifest

eval produces Vec<Scroll> — one scroll per host, each a tree whose every level holds either a leaf’s glyphs or named sub-scrolls. emetc build then content-addresses each scroll and assembles a Manifest. Its default output is the binary manifest; --text is a readable plan and --json a debug view (see CLI).

Everything a config computes — a rendered nginx site, a firewall rule, a container quadlet — is evaluated at this stage, on the author’s machine. A glyph reaches the manifest finished. The one value a glyph carries unresolved is a sealed secret, and opening it is the whole of what a host works out for itself: an interpolated Secretspec.get compiles to literal chunks around a hole that golemd opens at enact, so a manifest can be stored and shipped without the plaintext. See Trust model.

scroll-format — the shared contract

libs/scroll-format/ owns the wire model and nothing else: Glyph, Scroll, AddressedScroll, Manifest, ContentId, and the pure functions that turn scrolls into manifest bytes and back. It names no I/O, no filesystem, no network — pure data and pure functions.

Both emet and golemd depend on it; it depends on neither. Dependencies point toward this small, stable centre, so there is exactly one definition of the manifest bytes and the two ends cannot drift. The compiler re-exports the model through emet::ir. See Manifest format for the schema and the determinism guarantees.

golemctl — the operator CLI

golemctl apply <source> <addr>:

1. If <source> ends in .emet, shell out to emetc build and capture the binary manifest from stdout. Otherwise read the file as prebuilt manifest bytes.
2. POST the bytes to the agent’s /manifest, which answers 202 with a reconcile_id.
3. Follow that reconcile through /reconciles/:id, drawing the units as they settle. --reattach skips the POST and picks up the newest attempt instead, so a dropped connection costs the view and not the run.

golemctl plan posts the same bytes to /plan and prints the diff without writing anything. state, history, and show read back the agent’s applied scroll and journal.

golemctl fleet apply|plan|status are those verbs fanned out over a TOML inventory, concurrently, one connection per host. The fan-out lives entirely in the client: no golemd knows about another. See CLI.

golemd — the agent (reader)

golemd runs on each box (apps/golemd/). On a manifest:

1. Ingest. scroll_format::from_bytes decodes the manifest and checks format_version.
2. Select. Pick the AddressedScroll whose scroll.name matches this agent’s --host. A node enacts only its own scroll.
3. Diff (pure). Compare the desired scroll’s glyphs against the last-applied outcomes in the journal, keyed by glyph key and versioned by content id, producing an ordered list of GlyphOps: Install / Remove / Replace / Noop. This is a pure fold — desired state in, an ordered plan out, no side effects.
4. Enact. Run each op through the Reconciler port. Leaf units drain a bounded worker pool — workers = 4 by default, workers = 1 for the fully-serial walk — because the authored model promises no ordering between units; within a unit, glyphs still enact in source order (ADR 0034 §3). Apply captures the prior host state into an Inverse receipt; the port speaks glyph vocabulary, never apt-get. A leaf that exhausts its retries is rolled back or kept by its own policy, and its siblings settle either way.
5. Journal. Append a Reconcile revision embedding the content id, the ordered ops, and the reversal receipts.

The core (steps 3–5) is pure and testable with a fake in-memory reconciler — zero host I/O. The real host effects live in adapters. See Reversible reconcile for the reconciler contract and the four concrete reconcilers.

golemctl plan --against-host adds a second source beside the diff, not inside it. golemd probes the live host for every glyph the diff already named — reusing the same read-only check apply runs before touching anything — and the report renders the two side by side: what the journal says, and what the host says. The diff in step 3 stays exactly the pure fold it already was, unmodified and un-consulted by the probe; nothing the probe observes reaches apply, changes an op, or is journaled. It is read, rendered, and otherwise inert.

Why split it this way

Concern	Lives in	Changes	Failure mode
What runs on the fleet	Emet source	Daily	Compile error, before shipping
How a shape lowers to glyphs	Emet library functions	As you build abstractions	Compile error
The wire bytes	scroll-format	Rarely (a format_version bump)	Typed decode error on read
What lands on a box	golemd reconcilers	Almost never	Reversible; the failing leaf rolls back, its siblings settle

The agent is the part that can change a real box, so it is deliberately the smallest: four reconcilers over four glyphs, each capturing enough to undo itself.

Where to look in the source

Component	Path
Emet compiler	apps/emet/
Shared wire model	libs/scroll-format/
Operator CLI	apps/golemctl/
Agent	apps/golemd/
Reconciler port	apps/golemd/src/reconciler.rs
Host reconcilers	apps/golemd/src/reconcilers.rs
Fake reconciler (tests)	apps/golemd/src/fake_reconciler.rs
Journal / store	apps/golemd/src/journal.rs, planroom.rs
Lichess fleet (real Emet)	examples/lichess/