Golem

What it is

Golem configures bare-metal servers. It does the job Ansible, Chef, and Puppet do: packages, services, files, and lines in files, on machines you own and keep.

Golem is not a cluster scheduler, and it does not replace Kubernetes. On a traditional server, systemd is already the orchestrator — it starts units, orders them, restarts them, and supervises containers through Quadlet. Golem puts those units on the box, with the packages and the files they need, and keeps them at the state you wrote. The two together run a Debian trixie fleet well, with no control plane to operate.

You write the fleet in Emet, a typed functional language with generics, records, pattern matching, and exhaustiveness checking. emetc runs your program to completion on your own machine and emits a manifest — the finished desired state, with every function applied, every branch taken, and every value computed. A host receives desired state that has already type-checked and already evaluated.

A scroll is one host’s share of that manifest, shaped as a tree. Its leaves hold glyphs, and each leaf is enacted as its own unit: it retries on its own schedule, rolls back on its own, and settles independently of its siblings. The branches above the leaves group them by subsystem and hand down retry settings and reload obligations. Every scroll carries a BLAKE3 content id, so golemd can see which parts of a host changed and leave the rest alone.

Four glyph kinds cover the whole surface: aptPackage, systemdService, the filesystem glyph (file, directory, symlink), and lineInFile.

golemd is the per-host agent. It takes the manifest, selects its own scroll, compares it against the journal of what it applied last time, and enacts the difference. Each apply first records the prior state of what it is about to touch, so the edit can be put back exactly — and golemd reverses only edits it recorded itself, leaving whatever was on the box before it arrived untouched. Both ends compile against the same shared scroll-format crate, so the writer and the reader stay in agreement.

Evaluating everything up front and being able to reverse it afterward is what makes experiments cheap. Say state X works and you edit it into X+1. If a unit of X+1 fails and its policy says roll back — the default — that unit returns to the state it held under X, a working one, while the rest of the host settles on X+1. You can try a change without being sure of it first.

Higher-level shapes — a container workload, a service behind a firewall, an ingress — are Emet functions that return glyphs. Some ship with the toolchain, like the Quadlet library; the rest you write for your own fleet and reuse across hosts. They are ordinary code you can read, copy, and adapt, and they all land on the same four glyphs, which is what keeps the agent small.

Why it’s shaped this way

Abstractions live in the language

A “workload” or a “service” is an ordinary Emet function that returns a List Glyph. You can parameterize it, share it across hosts, and see exactly what it produces with golemctl plan --detail.

Evaluated before it ships

Emet is Hindley-Milner typed with generics, records, case/if, and exhaustiveness checking, and emetc runs the whole program on your machine. A missing field or a type mismatch is a compile error, before any bytes leave your laptop.

Content-addressed versioning

Every scroll is identified by a BLAKE3 hash of its deterministic bytes. Same bytes, same id, no-op. A changed field is a new id, and an upgrade. The manifest is versioned by format_version, not by hand-typed numbers.

Reversible reconcile

Applying a glyph captures what it changed, and golemd reverses only the edits it recorded. One mechanism drives upgrade (reverse the old, apply the new), removal (reverse toward an empty scroll), and a failed unit’s rollback.

The pipeline in one line

Emet source ──emetc build──▶ binary manifest ──golemctl apply──▶ golemd
  ──diff by content id──▶ reversible reconcilers ──▶ the box ──▶ journalled revision

The diff is why an apply that changes nothing costs nothing, and the journal is what lets the next one be undone.

See Architecture for each stage.

Start here

• Install the toolchain — build emetc, golemctl, and golemd.
• The config — write your first Emet program and watch it compile to glyphs.
• Applying changes — compile to a manifest and reconcile a box.

Guides

• A first glyph — the smallest fleet: one host, one package, one unit.
• A service abstraction — a thin helper on the shipped Quadlet library that lowers to a container quadlet, a unit, and a firewall fragment.
• An app behind Traefik — an application and its database on a private network, one hostname published over HTTPS through the shipped Traefik library.
• A maintenance page — a parameterized Emet function built from file and lineInFile.
• A tour of the lichess fleet — the real multi-host lichess deploy: the shipped library, a fleet’s helpers on it, and the four glyphs it lowers to.

Tutorials

Hands-on walkthroughs against ephemeral Debian VMs — golemd’s real reconcilers, the way we ran them.

• Bring up the fleet — boot the VMs, deploy golemd, and prove that an ssh forward and a bearer token are the only way in.
• A failing unit — one broken leaf, eight siblings that settle anyway.
• A registry on the fleet — run a container registry on one VM; push to it from another.
• The website loop — golem building, storing, and serving golem’s own docs.

What is not built yet

What golem does not do today — some of it planned, some of it a deliberate limit:

• Per-user identity. The agent authorizes with one shared secret over an SSH tunnel, so it knows a caller was authorized, never which person submitted. SSO and an audit trail are designed, not built. Manifest signing is not on the roadmap at all. See Trust model.
• Server-side fleet coordination. golemctl fleet fans a verb out over an inventory, but the fan-out is entirely client-side: one connection per host, each golemd acting only on its own scroll. No coordinator, no ordered drain, no health-gated cross-host rollback.
• Container and DNS resource kinds in the agent. These are the deliberate limit: golemd enacts the four glyphs, and shapes like these are Emet abstractions written on top of them. Secrets went a different way — a value type on the wire, sealed at compile time, described in the Trust model.

See Status for the full breakdown.