Bare metal

A Linux host without containers: the installer places verified binaries, writes systemd units and /etc/scafo, and leaves starting the node to you.

scripts/install.sh installs four binaries, scafo, celld, caddy and esbuild, verifies every download by SHA-256 before it changes anything on the host, and, when it runs as root on a systemd host, writes the node and ingress units plus their configuration. It does not start a node: the bucket credentials go in first.

The node API listens on the host's loopback, 127.0.0.1:7400, for the colocated Caddy and nothing else. Nothing widens it, no firewall rule names it, and the CLI reaches the fleet through the platform apex like on every other recipe.

#Prerequisites

  • Linux on x86_64 or aarch64, with glibc. The installer refuses musl hosts (Alpine and similar) because every binary it installs is linked against glibc; run the container image there instead.
  • wireguard-tools from the operating system, if this node will use the WireGuard transport. Scafo never generates key material itself.
  • curl or wget, plus uname, mktemp, install, sha256sum, tar and gzip.
  • An S3-compatible bucket supporting If-Match and If-None-Match conditional writes and Range reads.
  • Apex and wildcard DNS for the platform domain pointing at this host, and TCP 80 and 443 open.

#Run the installer

curl -fsSL https://scafo.dev/install.sh | sudo sh

This recipe runs the node and its ingress as system services, and that is what the sudo is for: the two systemd units, the scafo and caddy service accounts, /etc/scafo and /var/lib/scafo, which the next steps rely on. Without it the installer still installs and verifies the four binaries, under ~/.local/bin, and prints the manual equivalent of those units instead; see install.

It selects the latest release unless SCAFO_VERSION names an exact one:

curl -fsSL https://scafo.dev/install.sh | sudo SCAFO_VERSION=1.2.3 sh

#Configure the node

sudoedit /etc/scafo/node.env

SCAFO_S3_ENDPOINT, SCAFO_S3_BUCKET, SCAFO_S3_ACCESS_KEY and SCAFO_S3_SECRET_KEY are the required four; SCAFO_PLATFORM_DOMAINS names the domains applications are served under, and applications get no host without it.

#Start the node and its ingress

sudo systemctl enable --now scafo-node.service
sudo systemctl status scafo-node.service caddy.service

The node unit carries Wants=caddy.service and Before=caddy.service, so the ingress comes up with it and after it.

#Verify

curl -sS https://apps.example.com/.well-known/scafo/health
curl -sS -o /dev/null -w '%{http_code}\n' https://nobody.apps.example.com/

The first answers { "ok": true, "version": ..., "node": ... } from the node, through Caddy, with a certificate Caddy obtained for the apex on that first request. The second is 404: a host nobody has claimed is answered by Caddy, never by a fallback application. Before the DNS record resolves, add --resolve apps.example.com:443:<host ip> to the first command.

#Deploy

Follow first deploy from your own machine, not on this host. The bucket values it asks for are the ones in /etc/scafo/node.env, prefix included; the endpoint is https:// plus SCAFO_PLATFORM_DOMAINS.

scafo remote add prod --endpoint https://apps.example.com --register
scafo deploy ./my-app --name shop

If you turned the apex off, the API is still only on this host's loopback: see without the apex.

#What the installer does

Checks the platform: Linux only, x86_64 or aarch64 only, glibc only.
Resolves a destination: SCAFO_INSTALL_BIN_DIR if set, otherwise /usr/local/bin when it has root or non-interactive sudo, otherwise ~/.local/bin.
Downloads and verifies, into a temporary directory, before touching the host:
  • scafo-linux-<arch> from the GitHub release, latest or v$SCAFO_VERSION. The checksum comes from SCAFO_SHA256 if set, otherwise from the .sha256 file beside the download. With neither, it refuses to install.
  • celld 0.6.2 from the denoland/celld release, against a checksum pinned in the script.
  • scafo-caddy-linux-<arch> from the latest sandros94/scafo-ingress release, checked against SCAFO_CADDY_SHA256 or the sidecar .sha256. There is no fallback to upstream Caddy: it carries none of the scafo modules the generated configuration relies on: the scafo app, caddy.storage.scafo, the scafo dynamic-upstreams source and tls.permission.scafo.
  • esbuild 0.28.2, from its platform npm package, against a pinned checksum.
Installs the four binaries mode 0755 into the destination directory.
When it is root on a running systemd host and SCAFO_INSTALL_SKIP_UNITS is not 1:
  • creates the system users scafo and caddy if they do not exist, and /var/lib/scafo mode 0700 owned by scafo;
  • writes /etc/scafo/node.env mode 0600 and /etc/scafo/caddy.json mode 0644, never overwriting either if it already exists;
  • writes /etc/systemd/system/scafo-node.service and /etc/systemd/system/caddy.service, then runs systemctl daemon-reload;
  • records what it made in /var/lib/scafo/install.manifest, for the uninstaller.

Otherwise it prints the equivalent manual setup and installs nothing but the binaries.

#Environment variables

VariableEffect
SCAFO_VERSIONExact Scafo release to install; unset means latest.
SCAFO_SHA256Pins the Scafo checksum instead of trusting the sidecar file.
SCAFO_INSTALL_BASEAlternative artifact base URL for the Scafo binary.
SCAFO_CADDY_BASEAlternative artifact base URL for scafo-caddy.
SCAFO_CADDY_SHA256Pins the scafo-caddy checksum.
SCAFO_INSTALL_BIN_DIRDestination directory for the four binaries.
SCAFO_INSTALL_MIRRORMirrors the pinned third-party artifacts; checksums still apply.
SCAFO_INSTALL_SKIP_UNITS1 installs binaries only and writes no systemd units.

Note

Only the stable channel is served here: GitHub releases, latest or pinned by SCAFO_VERSION. The edge channel ships as the ghcr.io/sandros94/scafo:edge image, with its binaries kept as run artifacts; point SCAFO_INSTALL_BASE at a mirror of those to install one.

#Files on the host

PathWhat it is
/usr/local/bin/{scafo,celld,caddy,esbuild}The installed binaries.
/etc/scafo/node.envBucket credentials and every other node setting. Mode 0600.
/etc/scafo/caddy.jsonThe ingress configuration: this node declared once as the fleet Caddy serves, and every scafo module reading it. Generated once; operator edits are never overwritten.
/var/lib/scafoNode state, mode 0700, owned by scafo. It also holds the working directory and the ports of every runtime process this node supervises, under celld/<application>/, and must be kept between restarts and reinstalls, which is what the unit's StateDirectory=scafo gives it. See durability and the state directory.
/var/lib/scafo/node.jsonNode identity, mode 0600.
/var/lib/scafo/ingress-tokenThe token the ingress presents to /v1/ingress and /v1/storage, minted on the node's first boot. Mode 0600.
<bin dir>/scafo-uninstallThe release's uninstall.sh, installed beside the binaries.
/var/lib/scafo/install.manifestWhat this install made, for scafo-uninstall.
/var/lib/caddyCaddy's own state directory, and the ingress's ACME account. The certificates are not here; they are in the bucket, through the node.
/var/log/scafo-ingress/Where Caddy logs what it served, when the node is told to count it.
/etc/systemd/system/scafo-node.serviceThe node unit.
/etc/systemd/system/caddy.serviceThe ingress unit.

#/etc/scafo/node.env

The generated file, with everything the node reads:

# Configuration of this scafo node, read by scafo-node.service.
#
# The bucket is the whole registry: every node pointed at the SAME bucket and
# prefix is one installation, and nothing else makes two nodes a fleet.
# Required — the node refuses to boot without them.
SCAFO_S3_ENDPOINT=
SCAFO_S3_BUCKET=
SCAFO_S3_ACCESS_KEY=
SCAFO_S3_SECRET_KEY=
SCAFO_S3_REGION=us-east-1
# Optional: keeps this installation inside one prefix of a shared bucket.
#SCAFO_S3_PREFIX=

# The names applications are served under: <name>.<domain>. Comma-separated.
SCAFO_PLATFORM_DOMAINS=

# The node API. Loopback needs no further statement about the network; widening
# it makes the node refuse to boot until SCAFO_TRANSPORT says what that network
# is — "wireguard" to tunnel it, "none" to state that it is already private.
SCAFO_LISTEN=127.0.0.1:7400
#SCAFO_ADVERTISE=
#SCAFO_TRANSPORT=

# The ingress is not configured from here. It READS the fleet's routing off this
# node's API, at /v1/ingress/routes behind the ingress token, and follows it.

# What this machine IS, for applications that ask for a class of node.
#SCAFO_NODE_LABELS=tier=fast,region=eu-west-1

# Upstream Celld policy for every runtime process, comma-separated. Scafo adds
# no durability, idle-eviction or memory defaults. If SHUTDOWN_TOTAL_MS is
# raised above 40000, also raise TimeoutStopSec in scafo-node.service to at
# least that value plus 10 seconds.
#SCAFO_CELLD_ENV=CELLD_MAX_RSS_MB=512,CELLD_MAX_RESIDENT_CELLS=64

# Observing this node: the first switch makes it record its own events, and the
# second — which is REFUSED without the first — counts what Caddy actually
# served. Read both back with "scafo metrics" and "scafo traces".
#SCAFO_OBSERVABILITY=1
#SCAFO_INGRESS_ACCESS_LOG=/var/log/scafo-ingress/access.log
Read more in Config.

#/etc/scafo/caddy.json

The generated ingress configuration. It names the node in one place, the scafo app's single fleet, and the three modules that talk to it, storage, the dynamic_upstreams source and the on-demand permission, take no options and read that fleet:

/etc/scafo/caddy.json
{
  "storage": { "module": "scafo" },
  "logging": {
    "logs": {
      "default": { "exclude": ["http.log.access.scafo"] },
      "scafo": {
        "writer": {
          "output": "file",
          "filename": "/var/log/scafo-ingress/access.log",
          "mode": "0644"
        },
        "encoder": { "format": "json" },
        "include": ["http.log.access.scafo"]
      }
    }
  },
  "apps": {
    "scafo": {
      "fleets": [
        {
          "name": "default",
          "endpoint": "http://127.0.0.1:7400",
          "token_file": "{env.CREDENTIALS_DIRECTORY}/ingress-token",
          "prefer": "self"
        }
      ]
    },
    "tls": {
      "automation": {
        "policies": [{ "on_demand": true }],
        "on_demand": {
          "permission": { "module": "scafo" }
        }
      }
    },
    "http": {
      "servers": {
        "scafo": {
          "listen": [":80", ":443"],
          "routes": [
            {
              "handle": [
                {
                  "handler": "reverse_proxy",
                  "dynamic_upstreams": { "source": "scafo" },
                  "load_balancing": {
                    "selection_policy": { "policy": "first" },
                    "retries": 2,
                    "try_duration": "5s"
                  },
                  "health_checks": {
                    "passive": { "fail_duration": "30s", "max_fails": 1 }
                  }
                }
              ]
            }
          ],
          "errors": {
            "routes": [
              {
                "match": [{ "vars": { "scafo.route": ["unknown", "denied"] } }],
                "handle": [
                  {
                    "handler": "static_response",
                    "status_code": 404,
                    "body": "No deployment answers to this name.\n"
                  }
                ]
              }
            ]
          },
          "logs": { "default_logger_name": "scafo" }
        }
      }
    }
  }
}
  • apps.scafo is the fleet: the node's loopback API, the ingress token as token_file, and prefer: "self", which puts this node's own processes at the head of every route.
  • storage keeps certificates, their private keys and the issuance locks in the bucket, through the node's /v1/storage. The ACME account is not a fleet's, so it stays in Caddy's own state directory, /var/lib/caddy.
  • tls issues every certificate on demand, and the scafo permission agrees only to a name the fleet's routing document names exactly.
  • http is one catch-all route whose upstreams come from that routing document, in priority order, this node first. A host nobody has claimed, and /v1/storage or /v1/ingress/routes at the API apex, fall through to the error route and get the same 404.

token_file is $CREDENTIALS_DIRECTORY/ingress-token, the private copy the ingress unit's LoadCredential= makes. When the installer runs without writing units, the configuration it prints for a hand-run Caddy names the node's own /var/lib/scafo/ingress-token instead, which that Caddy's user must be able to read.

Read more in Docs > Ingress Caddy.

#The systemd units

#scafo-node.service

[Unit]
Description=scafo node daemon
Documentation=https://github.com/sandros94/scafo
Wants=network-online.target
After=network-online.target
# Pulled in and started BEFORE the ingress; see the comment in the installer.
Wants=caddy.service
Before=caddy.service

[Service]
Type=exec
User=scafo
Group=scafo
EnvironmentFile=/etc/scafo/node.env
Environment=SCAFO_STATE_DIR=/var/lib/scafo
# Default Celld source builds to the bundled esbuild binary.
Environment=CELLD_ESBUILD=/usr/local/bin/esbuild
Environment=SCAFO_CELLD_BIN=/usr/local/bin/celld
ExecStart=/usr/local/bin/scafo sail
Restart=on-failure
RestartSec=5s

# Signal the daemon first so it drains runtimes; kill remaining cgroup processes on timeout.
KillMode=mixed
KillSignal=SIGTERM
SendSIGKILL=yes
TimeoutStopSec=50s
# Keep runtimes in this cgroup for timeout containment.
Delegate=no

StateDirectory=scafo
StateDirectoryMode=0700
NoNewPrivileges=yes
ProtectSystem=full
ProtectHome=yes
PrivateTmp=yes
LimitNOFILE=1048576

[Install]
WantedBy=multi-user.target

KillMode=mixed signals the daemon alone so it drains its runtimes in order, and kills whatever is left in the cgroup on timeout. TimeoutStopSec=50s is the shutdown grace; raise it past CELLD_SHUTDOWN_TOTAL_MS + 10s if that Celld value is increased.

#caddy.service

[Unit]
Description=Caddy, the ingress a scafo node routes through
Documentation=https://caddyserver.com/docs/
Wants=network-online.target
After=network-online.target scafo-node.service
# The token is unavailable on first boot until the node starts. LoadCredential=
# fails before ExecStartPre, so disable start-rate limiting and retry on failure.
# Do not add Requires=: a running ingress may continue through a node restart.
StartLimitIntervalSec=0

[Service]
Type=notify
User=caddy
Group=caddy
# This gives the ingress a private copy without exposing the source token.
LoadCredential=ingress-token:/var/lib/scafo/ingress-token
ExecStart=/usr/local/bin/caddy run --config /etc/scafo/caddy.json
ExecReload=/usr/local/bin/caddy reload --config /etc/scafo/caddy.json --force
Restart=on-failure
RestartSec=5s
TimeoutStopSec=5s

# Caddy's own state directory. The certificates are NOT here — they are in the
# bucket, reached through the node — but the ACME account is: it is this
# ingress's, not the fleet's, and losing it only means registering again.
StateDirectory=caddy
Environment=XDG_DATA_HOME=/var/lib/caddy
Environment=XDG_CONFIG_HOME=/var/lib/caddy
# What the node counts requests out of, when it is told to.
LogsDirectory=scafo-ingress
LogsDirectoryMode=0755

# Binding 80 and 443 without running as root.
AmbientCapabilities=CAP_NET_BIND_SERVICE
NoNewPrivileges=yes
ProtectSystem=full
ProtectHome=yes
PrivateTmp=yes
LimitNOFILE=1048576

[Install]
WantedBy=multi-user.target

The node mints the ingress token 0600 inside a 0700 directory on its first boot. LoadCredential= is how Caddy gets it without loosening either mode: systemd reads the file as root and places a copy only Caddy's UID can open in $CREDENTIALS_DIRECTORY, which is what the fleet's token_file in /etc/scafo/caddy.json names.

Important

LoadCredential= fails before any of the unit's own commands run, so on the very first boot, before the token exists, the ingress unit fails and retries. That is what StartLimitIntervalSec=0 with Restart=on-failure is for: do not add Requires=, and do not widen the token file's permissions.

#Container classes

The installer installs no container engine and no docker client. A node that takes a deployment declaring a container class builds or pulls its image through a Docker-compatible engine's client, docker on this node's PATH or the one CELLD_DOCKER names, so both the engine and that client must be on this machine before such a deployment can land. CELLD_DOCKER and CELLD_CONTAINER_PLATFORM go in the unit's Environment=, beside CELLD_ESBUILD. The installer says so again when it finishes.

Whether the scafo user may reach the engine's socket is yours to decide, and the unit above grants it no access to one. The daemon already runs in the host's network namespace, which is where Celld has to be to reach the containers it starts. A daemon installed this way has not yet been run with a container class; see what is verified.

#Uninstalling

scafo-uninstall, which the installer placed beside the binaries, undoes the install (curl -fsSL https://scafo.dev/uninstall.sh | sudo sh -s -- --dry-run runs the same script off the wire if it is gone):

sudo scafo-uninstall --dry-run
sudo scafo-uninstall

It stops the ingress before the node, then removes the units, the four binaries and /etc/scafo. A plain run preserves /var/lib/scafo, /var/lib/caddy, /var/log/scafo-ingress and the accounts the install created.

FlagEffect
--purgeAlso removes the state directory (the node identity and the ingress token, which nothing can reissue, and the runtime working directories, which may still hold writes this fleet has acknowledged and the bucket has not got yet) plus Caddy's state directory with its ACME account, the ingress logs, and the system users this install created.
--dry-run, -nPrints every removal and performs none of them.
--bin-dir <dir>Names a non-default binary directory; SCAFO_INSTALL_BIN_DIR does the same.

Neither mode changes the bucket or its registry. The install manifest makes the removal exact; without it, the script derives safe defaults.

#Building the binary yourself

pnpm compile

produces build/scafo-linux-amd64 and build/scafo-linux-arm64, each with a .sha256 file. Celld, an ingress and WireGuard tools remain separate prerequisites; pnpm compile builds Scafo alone.