# Caddy modules

> `scafo-ingress`: the `scafo` app and the modules that read it, their builds, images, options, request variables and every environment variable.

Five modules in one repository and one binary. The `scafo` **app** is where the fleets are configured, once; the other four take no options and read it. A **fleet** is one Scafo installation: one node endpoint, one bearer token, one routing document, one bucket. One Caddy can stand in front of one fleet or several, and both are the same configuration and the same code path.

Caddy holds an endpoint URL and a bearer token per fleet, which authorize that node's `/v1/storage` and `/v1/ingress/routes` and nothing else. The node holds its bucket credentials and does the reading and writing, so there is no path from a compromised Caddy to the rest of any bucket.

| Module | Caddy module ID | What it does |
| --- | --- | --- |
| The app | `scafo` | Where the fleets are, which token opens them, which DNS providers exist |
| TLS storage | `caddy.storage.scafo` | Keeps certificates in each fleet's own bucket, routed per name |
| Dynamic upstreams | `http.reverse_proxy.upstreams.scafo` | Resolves a request's backends across every fleet |
| On-demand permission | `tls.permission.scafo` | Decides, at handshake time, which names may get a certificate |
| Plain-HTTP matcher | `http.matchers.scafo_plain_http` | Matches a host whose fleet asked not to be redirected to HTTPS |

Module path: `github.com/sandros94/scafo-ingress`. License: MIT.

## Building

```sh
xcaddy build --with github.com/sandros94/scafo-ingress            # every module here
xcaddy build --with github.com/sandros94/scafo-ingress/storage    # the storage one and the app
xcaddy build --with github.com/sandros94/scafo-ingress/ingress    # the HTTP ones and the app
xcaddy build --with github.com/sandros94/scafo-ingress/permission # the permission one and the app
```

The repository root imports every module in it, so naming the root compiles them all in; naming a single package compiles that module and the `scafo` app it reads, and nothing else.

### Dockerfile build arguments

The repository's `Dockerfile` does the same inside the official `caddy:<version>-builder` image and copies the result into the matching runtime image.

| Build argument | Default | Meaning |
| --- | --- | --- |
| `CADDY_VERSION` | `2.11.4` | Selects both the xcaddy builder and the runtime base, so the two never drift apart. It is the canonical Caddy pin for the repository: `go.mod`'s Caddy requirement is kept equal to it |
| `DNS_PROVIDERS` | empty | Whitespace-separated list of pinned Go module paths to compile in as well. Empty is the lean flavour |

```sh
docker build -t my-caddy .
docker build -t my-caddy --build-arg CADDY_VERSION=2.10.0 .
docker build -t my-caddy --build-arg DNS_PROVIDERS=github.com/caddy-dns/gandi@v1.0.3 .
```

One Dockerfile and one xcaddy invocation stand behind both published flavours, so a custom build is the published build with a different argument.

### Images, tags and binaries

Published for `linux/amd64` and `linux/arm64`, in two flavours. **Lean** is Caddy with only the modules above. **`-dns`** is the same build with a curated set of DNS-01 provider plugins compiled in, for the managed wildcard certificates — a Caddy plugin can only arrive at compile time.

| | Image | Release binaries |
| --- | --- | --- |
| Lean | `ghcr.io/sandros94/scafo-caddy:latest`, `:X.Y.Z`, `:edge` | `scafo-caddy-linux-{amd64,arm64}` |
| With providers | `:latest-dns`, `:X.Y.Z-dns`, `:edge-dns` | `scafo-caddy-dns-linux-{amd64,arm64}` |

Each binary ships a `.sha256` alongside it and is the same binary that is inside the image of its flavour, lifted out of that build's builder stage. Every release also carries one `caddy-version` file naming the Caddy both flavours embed.

::note
Take the lean flavour unless you want managed wildcards. HTTP-01 needs no plugin, and neither does the on-demand TLS that covers the names a fleet routes exactly. The providers are there for the wildcard. Naming a DNS provider this build does not carry fails startup with an error listing the ones it does.
::

### DNS providers in the `-dns` flavour

Exact pins, from `dns-providers.txt`, which is the whole list.

| Provider | Go module | Caddy module |
| --- | --- | --- |
| Bunny | `github.com/caddy-dns/bunny@v1.2.0` | `dns.providers.bunny` |
| Cloudflare | `github.com/caddy-dns/cloudflare@v0.2.4` | `dns.providers.cloudflare` |
| DigitalOcean | `github.com/caddy-dns/digitalocean@v0.0.0-20250606074528-04bde2867106` | `dns.providers.digitalocean` |
| Hetzner | `github.com/caddy-dns/hetzner/v2@v2.0.1` | `dns.providers.hetzner` |
| OVH | `github.com/caddy-dns/ovh@v1.1.0` | `dns.providers.ovh` |
| Route 53 | `github.com/caddy-dns/route53@v1.6.2` | `dns.providers.route53` |

The last path element, minus any `/vN` suffix, is the Caddy module name. The release build asserts that every entry registered under that name, so a provider that breaks the convention fails the build rather than going missing quietly. The DigitalOcean pin is a Go pseudo-version because that repository has never cut a tag.

A provider that is not in the list is added the same way the curated ones arrive — `--with github.com/caddy-dns/gandi@v1.0.3` to xcaddy, or the same string in `DNS_PROVIDERS`.

## The `scafo` app

The app is configured in one of two ways, and never both at once:

- **From the environment.** The published image ships a default Caddyfile that declares an empty `scafo` block, and the app fills itself in from the process environment. No Caddyfile to write, no volume to bind. See [every environment variable](#every-environment-variable).
- **In the configuration**, as below. When the configuration declares at least one fleet, the environment is **not consulted for fleets at all**. The same rule applies to DNS providers on their own. The two can never be half-merged.

`scafo` is a global option, so it goes in the leading braces block:

::code-group
```caddyfile [Caddyfile]
{
	scafo {
		fleet client-a {
			endpoint http://10.0.0.1:7400
			token    {env.TOKEN_A}
		}
		fleet client-b {
			endpoint      http://10.0.0.2:7400
			token_file    /run/secrets/client-b
			http_redirect off
		}

		dns cloudflare {
			api_token {env.CLOUDFLARE_API_TOKEN}
		}

		acme_delegation acme.example.com
		acme_email      ops@example.com
	}
}
```

```json [JSON]
{
  "apps": {
    "scafo": {
      "fleets": [
        { "name": "client-a", "endpoint": "http://10.0.0.1:7400", "token": "{env.TOKEN_A}" }
      ]
    }
  }
}
```
::

In JSON every option keeps its Caddyfile name, and a fleet's name is its `name` member.

### `scafo` options

| Option | Description |
| --- | --- |
| `fleet <name> { … }` | One fleet. Repeatable. The name is a slug: lowercase alphanumerics separated by single hyphens |
| `dns <module> { … }` | The default DNS provider. Every option in the block is one of the provider module's own JSON fields, except `zones` |
| `dns <name> <module> { … }` | A named DNS provider, with `zones <zone…>` saying which zones it holds. The name is a slug, like a fleet's, and is not the module name — an operator may hold two accounts at one provider |
| `acme_delegation <fqdn>` | The fixed name a delegated DNS challenge record is written at. See [wildcard certificates](/docs/wildcard-certificates) |
| `acme_email <email>` | The contact address registered with the ACME CA for the managed wildcards |
| `acme_ca <url>` | The ACME directory URL for the managed wildcards. Unset uses Caddy's default |

A value written in the configuration wins over the environment for `acme_delegation`, `acme_email` and `acme_ca`; the environment fills in only what the configuration left empty.

### `fleet` options

| Option | Type | Default | Description |
| --- | --- | --- | --- |
| `endpoint` | URL | — | Base URL of that fleet's node API, for example `http://127.0.0.1:7400`. A path prefix is allowed and is preserved. Must use `http` or `https`. **Required** |
| `token` | string | — | The bearer token, inline. Trimmed of surrounding whitespace |
| `token_file` | path | — | Path to a file containing the bearer token. Read once at startup and trimmed, so a trailing newline is fine |
| `prefer` | string | `self` | Whose upstreams should head every route. `self` means the node answering the request. Any other value is a node id, sent through verbatim. `none` asks for nothing — it is read here and not sent, and the node then leaves the fleet's own order alone |
| `poll_interval` | duration | `5s` | How long to wait between fetches of this fleet's routing document. Must be positive |
| `timeout` | duration | `10s` | Bounds one fetch of the routing document. Must be positive |
| `storage_timeout` | duration | `30s` | Bounds one storage request to this fleet's node. Must be positive. It does not bound a lock acquisition, which blocks until the lock is granted or the caller's context is done |
| `http_redirect` | `on`/`off` | `on` | `off` serves this fleet's hosts over plain HTTP as well, and leaves every other fleet alone |

::warning
**Exactly one of `token` and `token_file`.** Setting both is a configuration error, and so is setting neither; both fail at startup, naming both options. Once resolved the two are indistinguishable.
::

### DNS providers

The `dns` block mirrors the environment form exactly, and every field value is a string:

```caddyfile
dns cloudflare {
	api_token {env.CLOUDFLARE_API_TOKEN}
}

dns other cloudflare {
	zones     a.example b.example
	api_token {env.OTHER_CLOUDFLARE_API_TOKEN}
}
```

A wildcard base is solved by the named provider whose `zones` entry is its longest suffix, matched at a label boundary; otherwise by the default provider, the one with no `zones` of its own; otherwise, when zones are declared somewhere and `acme_delegation` is set, by the delegation. When no zone is declared anywhere, the default provider takes everything.

::read-more{to="/docs/wildcard-certificates"}
What the delegation is, and the records a customer creates.
::

### Placeholders

Every value accepts Caddy's global placeholders, most usefully `{env.VAR}`. The app is provisioned before it serves anything, so request-scoped placeholders are not available — only environment and system values.

| Form | Behaviour |
| --- | --- |
| `{env.SCAFO_INGRESS_TOKEN}` | Resolved while the app is provisioned and never written back into the configuration. `caddy adapt` emits the placeholder verbatim, and the admin API's `GET /config/` returns it verbatim, so only the running process holds the token |
| `{$SCAFO_INGRESS_TOKEN}` | Substituted **before** parsing, so it bakes the secret into the adapted JSON. Do not use it for a token |

The endpoint is expected to be loopback or an equally private network path. Do not expose a node's API publicly on the strength of the token alone; a node reached over a shared network carries the token over that network, so keep it private or use `https`.

## `caddy.storage.scafo`

`storage` is a global option. It takes **no options**: the fleets are the app's.

::code-group
```caddyfile [Caddyfile]
{
	scafo {
		fleet default {
			endpoint http://127.0.0.1:7400
			token    {env.SCAFO_INGRESS_TOKEN}
		}
	}
	storage scafo
}
```

```json [JSON]
{
  "storage": { "module": "scafo" }
}
```
::

An option inside `storage scafo` is refused at startup with an error that says where fleets are configured instead.

### Where each key goes

One Caddy has one certificate keyspace, and every key in it that names a domain is routed to the fleet that owns that domain — through that fleet's node, with that fleet's token. A certificate never crosses a fleet. Ownership follows [the same resolution](#resolving-a-host) routing uses, plus one rule for a `*.base` certificate: it belongs to the fleet whose document carries `*.base` as a route, answers on `base` as its platform apex, or asks for `base` in its `wildcards`.

| Key | Belongs to |
| --- | --- |
| `certificates/<issuer>/<domain>/<domain>.{crt,key,json}` | the fleet owning `<domain>` |
| `ocsp/<domain>-<hash>` | the fleet owning `<domain>` |
| `acme/<issuer>/challenge_tokens/<domain>.json` | the fleet owning `<domain>` |
| lock `issue_cert_<domain>` | the fleet owning `<domain>` |
| `acme/<issuer>/users/…` — the ACME account | the ingress's local disk |
| `last_clean.json`, `rw_test_<n>` | the ingress's local disk |
| locks `storage_clean`, `register_acme_account…`, `ari_<id>` | the ingress's local disk |
| Caddy's own `pki/…`, `ech/…`, anything else | the ingress's local disk |

**Keys that name no domain have no fleet**, so they live on the ingress's own disk, in Caddy's default file storage under its data directory. The ACME account is the important one: it is the ingress's, not any fleet's, and putting it in one fleet's bucket would let that fleet impersonate the ingress to its certificate authority. Keep the data directory (`/data` in the image) on a volume.

Two consequences:

- CertMagic's storage check writes and reads back an `rw_test_<random>` key before an issuance. It names no domain, so it checks the local disk rather than a fleet's bucket; a fleet's bucket is exercised by the certificate writes themselves.
- The renewal lock CertMagic takes while fetching ARI names no domain, so it is taken locally: a lock between the goroutines of this ingress rather than between several of them.

**A domain nobody owns gets no default fleet.** Falling back to "the first one" would put one customer's private key in another customer's bucket. A read of such a key is reported as missing, and a write is refused with an error naming the domain.

A listing of a prefix that names a domain is sent to that fleet alone. A listing whose children name domains (`certificates/<issuer>`, `ocsp`) is sent to every fleet and merged, keeping a key only if it came back from the fleet that owns its domain — so a stale key left in one fleet's bucket for a name another fleet owns is never loaded, renewed or deleted through the wrong node.

Each fleet's storage requests use a dedicated HTTP client bounded by that fleet's `storage_timeout`. There are no retries other than the lock poll.

::read-more{to="/reference/ingress-protocol"}
::

## `http.reverse_proxy.upstreams.scafo`

It goes inside a `reverse_proxy` block as `dynamic scafo`, and takes **no options**.

::code-group
```caddyfile [Caddyfile]
reverse_proxy {
	dynamic scafo
	lb_policy first
}
```

```json [JSON]
{
  "handler": "reverse_proxy",
  "dynamic_upstreams": { "source": "scafo" },
  "load_balancing": { "selection_policy": { "policy": "first" } }
}
```
::

An option inside `dynamic scafo` is refused at startup with an error that says where fleets are configured instead. `prefer`, `poll_interval` and the timeouts are per fleet, so two site blocks cannot disagree about a fleet.

### Resolving a host

Every fleet has its own poller and its own cached routing document; a request is resolved against all of them at once, so it costs a map lookup and no round trip. The request's host, lowercased and without its port, is tried in this order, and the first match answers:

1. a route naming it exactly, in any fleet;
2. a fleet API host naming it exactly;
3. a wildcard route covering it — the host with its first label replaced by `*`, looked up once.

An exact route beats a wildcard across fleets, just as it does within one. When two fleets publish the same host, the one that published it **first** keeps it for as long as it goes on publishing it, and the conflict is logged once per change rather than once per poll. Two fleets claiming a host in the very same refresh have no "first", so the order they were configured in decides: declaration order in a Caddyfile, alphabetical by name when they came from the environment.

A fleet whose node is unreachable affects only its own hosts: its last document keeps answering.

### Request variables

The module cannot choose a status code — Caddy's answer to *no upstreams* is `503` — so it records why in four request variables and leaves the answer to a `handle_errors` route.

| Variable | Value |
| --- | --- |
| `scafo.route` | `ok`, `denied`, `unknown` or `unavailable` |
| `scafo.host` | The normalized host the lookup was keyed by: the request host, minus any port, lowercased |
| `scafo.matched` | The document key that answered: a route host, a wildcard such as `*.example.com`, or a fleet API host. Empty when nothing matched |
| `scafo.fleet` | The name of the fleet that answered. Empty when none did |

`scafo.host` is always the request's own host; `scafo.matched` tells a wildcard's traffic apart from a route named for one application, and `scafo.fleet` tells one installation's traffic from another's.

| `scafo.route` | Meaning |
| --- | --- |
| `ok` | Some fleet's document named upstreams for this request. A `503` with `ok` means they were all unreachable |
| `unknown` | No fleet serves this host |
| `denied` | The host is a fleet API host and the path is one the ingress must answer itself rather than proxy. Enforced whatever the error route says |
| `unavailable` | **Not one** fleet has a routing document yet, so nothing is known about this host either way. Deliberately not `unknown`: a cold start is not a missing deployment. Once any fleet has answered, a host nobody names is `unknown` |

These are **request** variables. A `vars` matcher branches on them, and any handler that expands request placeholders — `respond`, `header`, `rewrite` — can read `{http.vars.scafo.fleet}`. Caddy's access-log encoder does not run in a request context, so a fleet name reaches the log only by way of something that does, such as a response header the log already records.

Turning the first two into a `404` takes built-in directives only:

```caddyfile
handle_errors {
	@missing vars scafo.route unknown denied
	handle @missing {
		respond "No deployment answers to {http.vars.scafo.host}." 404
	}
	respond "{err.status_code} {err.status_text}" {err.status_code}
}
```

`denied` is answered the same way as `unknown` on purpose: a distinct status would confirm that the path exists.

### The pollers

**One poller per fleet**, not one per site block: however many handlers use `dynamic scafo`, a node is asked once per `poll_interval`. A fetch whose bytes are identical to the last one's is dropped rather than republished, so the cross-fleet view is rebuilt, and a conflict logged, only when something actually changed. A document whose `v` is not `1` is refused whole, and the last one understood keeps answering.

| Situation | Behaviour |
| --- | --- |
| First fetch, during provisioning | Attempted, so a healthy pair begins already routing. **Its failure does not fail startup** — it is logged and the poller carries on |
| Later fetch failures | Cost nothing: the last document that parsed keeps answering, and the log says so once per state change rather than once per tick |
| A configuration error | Fails startup. A missing, unreadable or empty token aborts with an error naming the option |
| Reload | The incoming configuration takes its reference to each poller **before** the outgoing one gives its own back, so polling never pauses and the last document keeps answering across the swap — even when a node happens to be down. A poller stops only when the last configuration using it has gone |

Each fetch is bounded by the fleet's `timeout`, and the interval is waited *between* attempts rather than on a schedule, so a slow answer delays the next request instead of stacking up behind it.

### Load balancing

Each node returns a route's upstreams in **priority order, never as a weighting**. `lb_policy first` takes that at its word: with `prefer self`, the head of the list is the process on this very node, so a request that arrives here stays here whenever it can.

```caddyfile
reverse_proxy {
	dynamic scafo

	lb_policy       first
	lb_retries      2
	lb_try_duration 5s

	fail_duration 30s
	max_fails     1
}
```

| Setting | Value | Effect |
| --- | --- | --- |
| `lb_policy` | `first` | Uses the head of the node's order and reaches for the rest only when it fails |
| `max_fails` | `1` | **Must stay at 1.** `first` picks the first upstream not yet marked down, and a failed dial does not mark one down until it has failed `max_fails` times — so with a higher value every retry of the same request goes back to the same dead process |
| `fail_duration` | e.g. `30s` | How long a failed upstream stays skipped, and what lets it back in once it is up |
| `lb_retries`, `lb_try_duration` | e.g. `2` and `5s` | Bound how long one request keeps trying. Caddy stops at whichever it reaches first |

::warning
A request to an unknown host takes the same "no upstreams" path as a route whose backends are all down, so it spends the same retry budget before the 503 can become a 404. A dynamic upstream that resolved to nothing resolves to nothing again on the next attempt, so keep the retries small.
::

## `tls.permission.scafo`

The on-demand TLS permission. It goes in the `on_demand_tls` global option and takes **no options**.

::code-group
```caddyfile [Caddyfile]
{
	on_demand_tls {
		permission scafo
	}
}
```

```json [JSON]
{
  "apps": {
    "tls": {
      "automation": {
        "on_demand": {
          "permission": { "module": "scafo" }
        }
      }
    }
  }
}
```
::

A name is approved only when some fleet's **current** routing document names it as an exact route host or as a fleet API host. The answer comes from the documents already in memory, so a handshake costs a map lookup and no HTTP call — which matters, because this runs on the handshake path of every name the ingress has no certificate for, and a stranger decides how often that is.

- **A name covered only by a `*.` entry is refused.** An on-demand certificate per arbitrary subdomain is a door anyone can knock on: point a thousand names at the ingress and each handshake asks the CA for another certificate under a customer's domain, until that customer's weekly issuance limit is gone. The wildcard those names are covered by is obtained once, through DNS-01, and covers all of them.
- **A name in no document is refused**, and so is every name while no document has arrived yet. "I do not know" is not "yes", and a cold start must not become an issuance for whatever connects first.

## `http.matchers.scafo_plain_http`

A request matcher with **no options**. It matches a host whose fleet is configured with `http_redirect off`; a host no fleet owns does not match. Caddy decides HTTP-to-HTTPS redirects per site block from the host names written in the configuration, and this ingress writes none down — it serves whatever the routing documents say, and that changes without a reload — so the decision is made per request instead:

::code-group
```caddyfile [Caddyfile]
http:// {
	@plain scafo_plain_http
	handle @plain {
		reverse_proxy {
			dynamic scafo
		}
	}
	handle {
		redir https://{host}{uri} 308
	}
}
```

```json [JSON]
{
  "match": [{ "scafo_plain_http": {} }]
}
```
::

The default is the redirect, and only a fleet that asked for plain HTTP gets it. The ACME HTTP challenge is unaffected either way: Caddy answers it in the server itself, before any route or matcher runs.

## Every environment variable

Read by the `scafo` app when the configuration declares no fleet, which is how the published image is meant to run. `<NAME>` is `[A-Z0-9_]+`, and becomes a fleet or provider name lowercased with underscores turned into hyphens: `SCAFO_FLEET_CLIENT_A_ENDPOINT` is fleet `client-a`.

### Fleets

| Variable | Description |
| --- | --- |
| `SCAFO_FLEET_ENDPOINT` | Node API base URL of the single fleet, which is named `default` |
| `SCAFO_FLEET_TOKEN` | Its bearer token, inline |
| `SCAFO_FLEET_TOKEN_FILE` | Its bearer token, as a path. Exactly one of the two |
| `SCAFO_FLEET_PREFER` | `self` (default), a node id, or `none` |
| `SCAFO_FLEET_POLL_INTERVAL` | Between fetches of the routing document. Default `5s` |
| `SCAFO_FLEET_TIMEOUT` | Bounds one fetch. Default `10s` |
| `SCAFO_FLEET_STORAGE_TIMEOUT` | Bounds one storage request. Default `30s` |
| `SCAFO_FLEET_HTTP_REDIRECT` | `on` (default) or `off`. Also accepts `true`/`false`, `yes`/`no`, `1`/`0` |
| `SCAFO_FLEET_<NAME>_…` | The same eight, for a fleet called `<NAME>` |

Durations are Caddy durations (`5s`, `1m30s`) and must be positive; `0s` is an error rather than "use the default".

::warning
**The single and the named forms are mutually exclusive.** `SCAFO_FLEET_TOKEN` beside `SCAFO_FLEET_A_TOKEN` fails startup naming both variables, and so does `SCAFO_FLEET_ENDPOINT` beside `SCAFO_FLEET_DEFAULT_ENDPOINT`. A `SCAFO_FLEET_*` variable this build does not recognize fails startup naming it and listing the ones it does, and a fleet with an endpoint and no token fails too.
::

### DNS providers

| Variable | Description |
| --- | --- |
| `SCAFO_DNS_PROVIDER` | The default provider's Caddy module name, as `caddy list-modules` prints it minus the `dns.providers.` prefix |
| `SCAFO_DNS_ZONES` | Comma-separated zones the default provider holds. Optional; listing them is what enables the delegation and what makes bases that appear later work without a reload |
| `SCAFO_DNS_<FIELD>` | One of the provider module's own JSON fields, lowercased: `SCAFO_DNS_API_TOKEN` is `api_token` |
| `SCAFO_DNS_<NAME>_PROVIDER` | A named provider's module. Declaring this is what makes `<NAME>` a namespace |
| `SCAFO_DNS_<NAME>_ZONES` | Comma-separated zones that provider holds |
| `SCAFO_DNS_<NAME>_<FIELD>` | One of that provider's own JSON fields |

`ZONES` is reserved and cannot be a provider field name. A `SCAFO_DNS_*` variable with no provider declared anywhere is an error naming every one of them. Without `SCAFO_DNS_<NAME>_PROVIDER`, `SCAFO_DNS_BN_API_KEY` could not be told apart from a default-provider field called `bn_api_key` — declaring the name is what carves out the namespace.

### ACME and the image

| Variable | Description |
| --- | --- |
| `SCAFO_ACME_DELEGATION` | The fixed name a delegated DNS challenge record is written at |
| `SCAFO_ACME_EMAIL` | The contact address registered with the ACME CA. Optional |
| `SCAFO_ACME_CA` | The ACME directory URL. Optional; unset uses Caddy's default |
| `SCAFO_ACCESS_LOG` | Read by the image's default Caddyfile only: the access log file path. Default `/dev/stdout`. The log is JSON and the file mode is `0644` |

## The image's default Caddyfile

What the published image runs at `/etc/caddy/Caddyfile`, with its comments shortened. Mounting a Caddyfile of your own over it replaces it entirely; the module repository's `examples/Caddyfile` is the same file with the `scafo` block written out, and the place to start.

```caddyfile [/etc/caddy/Caddyfile]
{
	# Every fleet, DNS provider and ACME setting comes from the environment.
	scafo {
	}

	# Certificates live in the bucket of the fleet that owns each name. The
	# ACME account and Caddy's own housekeeping stay under the data directory.
	storage scafo

	# Only a name some fleet's routing document names exactly gets a
	# certificate on demand.
	on_demand_tls {
		permission scafo
	}
}

(scafo_log) {
	log {
		output file {$SCAFO_ACCESS_LOG:/dev/stdout} {
			mode 0644
		}
		format json
	}
}

(scafo_proxy) {
	reverse_proxy {
		dynamic scafo

		lb_policy first
		lb_retries 2
		lb_try_duration 5s

		fail_duration 30s
		max_fails 1
	}
}

(scafo_errors) {
	handle_errors {
		@missing vars scafo.route unknown denied
		handle @missing {
			respond <<HTML
				<!DOCTYPE html>
				<html lang="en">
				<head><meta charset="utf-8"><title>Not found</title></head>
				<body style="font: 16px/1.6 system-ui, sans-serif; max-width: 32rem; margin: 12vh auto; padding: 0 1.5rem">
				<h1 style="font-size: 1.25rem">Nothing is served here</h1>
				<p>No deployment answers to <code>{http.vars.scafo.host}</code>.</p>
				<p style="color: #666">scafo</p>
				</body>
				</html>
				HTML 404
		}
		respond "{err.status_code} {err.status_text}" {err.status_code}
	}
}

http:// {
	import scafo_log

	@plain scafo_plain_http
	handle @plain {
		import scafo_proxy
	}
	handle {
		redir https://{host}{uri} 308
	}

	import scafo_errors
}

https:// {
	import scafo_log

	tls {
		on_demand
	}

	import scafo_proxy
	import scafo_errors
}
```

## Operational notes

A node must be reachable when Caddy **starts** for its fleet to start already routing, but its absence does not keep Caddy from starting. A **running** Caddy is not affected by node downtime: TLS handshakes are served from the in-memory certificate cache, requests are routed from the last document that parsed, and only background work fails and is retried later — renewals, OCSP staple refreshes, a certificate for a new name, a route that has changed since the last successful poll — and only for that node's fleet.

A Caddy that starts while every node is down has an empty certificate cache and no routing document, so TLS fails and every request is `unavailable` until one answers:

- Start the nodes before Caddy and stop them after, so Caddy never outlives them.
- Compose: share a node's network namespace where there is one node, and `depends_on` it.
- systemd: order the Caddy unit `After=` the node's, with `Restart=on-failure` and `StartLimitIntervalSec=0`.
- Kubernetes: run a node as a sidecar in the Caddy pod with a startup probe.

Several Caddy instances may share one fleet's bucket; coordination relies on that node's lock endpoints, and expiring locks left behind by a crashed holder is the node's job. The ARI renewal lock is local to each ingress rather than shared.

### Rotating the token

The ordering rule: **the node first, then Caddy.** A Caddy reload refills the certificate cache from storage, so it must find a node that already accepts the token it is about to present. Serving is unaffected throughout, and only that fleet's renewals and route updates stall while Caddy holds a token the node rejects.

1. Give the node both tokens, new first: `SCAFO_INGRESS_TOKEN="new,old"`, and restart it. Both are accepted.
2. Reload each Caddy with the new token, at leisure. Reloads are graceful, not restarts.
3. Contract the node to `SCAFO_INGRESS_TOKEN="new"` and restart it. The old token stops working.

A single node and a single Caddy can skip the expand step: set the new value on both, restart the node, reload Caddy.

::read-more{to="/docs/ingress-caddy"}
::