Errors
The problem-document format, every error code, and how each surface renders one.
Every failure scafo surfaces carries one of twenty-three codes. The code is the stable contract: messages may change, codes may not.
#Problem documents
The node API answers every failure as an RFC 9457 problem document.
HTTP/1.1 409 Conflict
content-type: application/problem+json
{
"type": "urn:scafo:application-blocked",
"title": "This application is blocked",
"status": 409,
"detail": "\"shop\" has been blocked by an admin of this fleet: it is served nowhere and accepts no deploys until the block is lifted, and nothing of it has been deleted — ask whoever administers this installation"
}| Member | Type | Notes |
|---|---|---|
type | string | urn:scafo:<code>. Always present |
title | string | The failure's message. Always present |
status | number | The HTTP status. Always present |
detail | string? | Human-readable specifics, safe to expose. Present when the throw site supplied one |
One answer is not a scafo failure:
| Situation | Document |
|---|---|
| A transport-layer HTTP error the handler did not raise | { "type": "about:blank", "title": …, "status": … } |
A failure the node did not expect is one: it is answered as internal, with the real error written to the node's log and never exposed.
A deploy-overloaded answer additionally carries retry-after: 1.
#Code table
| Code | HTTP | Meaning, and where it is raised |
|---|---|---|
auth-invalid | 401 | Authentication failed. Every token failure, whatever the cause: a missing or malformed Authorization header, a wrong alg, an unknown kid, a bad signature, an expired or not-yet-valid token, a lifetime over 60 s, a replayed jti, a wrong ingress token, a workload token that does not verify, an application with no trust policy, an issuer that application does not name, or an operator whose registry record is no longer current |
auth-scope | 403 | Authorization by tier or application scope. An application outside an operator key's scope; a caller that is neither an operator nor a node of this fleet on a route that takes both; a node token on an operator-only route; registering a key beyond the caller's tier or reach; granting * as a non-admin; superseding a key that is not the caller's own; revoking a key an operator-tier caller was not registered to replace; blocking or labelling an application as a non-admin; claiming a domain with its subdomains on a key that does not reach <name>-*; freezing deploys as a non-admin |
trust-mismatch | 403 | The token holds up, but the application's trust policy does not admit its issuer or subject. The one rejection that is authorization rather than authentication, and the one a CI job can act on. Its detail names the issuer, the subject the token carried and the application it was presented for, and ends with the scafo trust add command that admits exactly that subject. It never lists what the application already trusts, and it prints no command for a subject carrying a *, a quote or whitespace |
cas-conflict | 409 | A conditional write was refused by the store; a named lock is already held; or a state a command asserted did not hold after the call (an application still blocked, or still retired, after the call that should have changed it) |
capability-missing | 409 | This node cannot do the thing at all: no deploy runner configured, no file-backed artifact stager, no signing key with which to reach the coordinator, a publish command that could not be run, a POSIX process group that could not be created, stopped or observed, or — at scafo sail start — a Celld binary that could not be run, whose version could not be read, or that is older than 0.5.0. Also scafo deployment prune and scafo app lease when no answering node reports a Celld release this build was verified against |
placement-unavailable | 409 | Placement cannot be satisfied: no live node, or no free port pair in the configured range |
version-skew | 409 | Another answering node serving the application, or crash-looping on it, reports a different Celld version, and the two are not a rolling pair upstream declared (the list is 0.5.0 ↔ 0.5.1, 0.6.0 ↔ 0.6.1 and 0.6.1 ↔ 0.6.2). A node neither starts that application's process nor accepts a deploy for it until every answering node serving it runs a release it may share the fleet with. The detail names every node that has to stop and the rolling pairs this build allows. A conflict rather than a missing capability: every binary involved is one scafo runs, and what is wrong is the fleet's state |
application-blocked | 409 | An admin of the fleet has stopped the application: it takes no deploy and no domain claim. A conflict rather than a forbidden: nothing is wrong with the caller or its key, and the same request works unchanged the moment the block is lifted. Releasing a domain stays open |
deploys-frozen | 409 | An admin has stopped deploy intake for the whole fleet, for a maintenance or an upgrade. Refused before anything is created; the detail names when, by whom, why, and how it is lifted. Serving carries on |
draining | 503 | Nothing is refusing you — this node cannot answer right now. The fleet's routing is not known yet (no reconcile pass has completed); no node is coordinating; or a forwarded deletion landed on a node that does not coordinate either |
deploy-overloaded | 429 | The node's deploy queue or staged-byte capacity is full — a node publishes one deploy at a time and queues the rest. Answered with retry-after: 1; scafo deploy honours that number, backs off with jitter and comes back until --queue-timeout has passed, unless --no-wait was given |
deploy-timeout | 408 | A deploy receive, queue or task deadline elapsed |
store-nonconforming | 500 | The bucket is not behaving as the registry requires: a stored object that is not valid JSON, registry meta of an unknown version, a store that does not honour conditional writes or ranged reads, a listing cut short, a missing ETag, a TLS object that is not a storage envelope, or a publish that reported no version |
store-unavailable | 503 | The store stopped answering: a write it met with a transient error was repeated until the tries ran out. Nothing about an answer was wrong — there was no answer |
publish-failed | 502 | celld deploy ended badly. Celld reports a failed publish as an exit code and prose only, so the cause may be the artifact, the bucket or the node's container engine alike, and scafo does not guess: the detail is the tail of Celld's own output. A container class deployed through a node with no engine is one such case, with Celld saying it could not build the container image. When Celld had already named a version, the detail says it may have published |
internal | 500 | A failure scafo has no name for. It is the node's own and never the caller's; what it was is in the node's log. A deploy that failed on one is recorded with this code — an artifact entry the accepting node could not write among them |
not-found | 404 | The thing named does not exist: an object, a deploy id, a stored TLS key or prefix, a domain the named application does not hold — said the same way whether somebody else holds it or nobody does — an ingress format this node does not render, a remote that is not registered, or a node that is not answering |
immutable | 409 | Something that may only be written once was written again, or an identity was changed: a node id, an application's v or name, an existing deployment, or an existing deploy id |
lease-lost | 409 | Authority to write was lost partway: another node has taken the coordinator lease, a placement fence is stale, a named lock is no longer held, or publish authority was lost. The holder renewing its own lease while the work runs is not that, and does not interrupt it |
reference-held | 409 | The name or thing is spoken for: an id belonging to another identity, a node id belonging to an operator key (or the reverse), a domain another application claimed — or that another application's claim already covers, either way round — or the last trusted operator key / last admin key, which cannot be revoked. A domain refusal names the holder only to a caller whose scope reaches it. Also a Celld node lease that is still held, which scafo app lease drop refuses without --force |
resource-limit | 413 | An input exceeded a configured bound: a JSON body over api.jsonBytes, an artifact over artifactBytes, an archive with too many entries, or an archive that expands too large |
invalid-slug | 400 | A name or id does not match its rule: an application name, a content id, a node id, an operator id, a deploy id, or a --name/--env pair whose join is no application name |
invalid-input | 400 | Everything else the caller got wrong: missing or malformed configuration, missing required arguments, malformed request bodies and headers, a bad storage key, an unsupported storage protocol version, a self-revocation, an artifact without a runtime config, an artifact entry whose path or link leaves the directory it is unpacked into, a tar compressed twice, a host that is no host to claim or that lies under a platform domain, an application already holding its 64 domains, and the fallback for an unrecognized problem type |
#Reading a problem document back
A client turns an answer back into the failure it describes rather than re-labelling it — flattening trust-mismatch and auth-invalid into one code would throw away the distinction the API is careful to keep.
Every member is checked rather than trusted, because what arrives may have come from a proxy, a load balancer or an error page:
| Situation | Result |
|---|---|
type is urn:scafo:<known code> | That code |
type is urn:scafo:<code this build has never heard of> | invalid-input, the fallback. A newer node naming a code this one does not know is a real thing and not worth crashing over |
type is missing or is not a string | invalid-input |
title is missing or is not a string | "Request failed" |
status is not an integer | The default status of the resolved code |
The answer carries no type and title at all | invalid-input, Node answered <status>, with the method and URL as its detail |
#CLI rendering
Every command wraps its work in one reporter.
| Outcome | stderr | Exit code |
|---|---|---|
| A scafo failure | scafo: <title>[: <detail>] [urn:scafo:<code>] | 1 |
| Anything else | scafo: unexpected failure: <message chain> | 1 |
The unexpected-failure chain follows cause up to four levels deep, because fetch reports TypeError: fetch failed and puts the refused connection in its cause — a renderer stopping at the top would print nothing an operator could act on.
A failure answered by a node is re-rendered from that node's own problem document, so the code printed is the code the node chose.
#Ingress-side failures
#The Caddy module
The dynamic-upstreams module cannot choose a status code — Caddy's answer to no upstreams is 503 — so it records why in scafo.route and leaves the answer to a handle_errors route.
scafo.route | Meaning |
|---|---|
ok | The document named upstreams for this request. A 503 with ok means they were all unreachable |
unknown | No fleet serves this host |
denied | The host is a platform API host and the path is one the ingress must answer itself. Enforced whatever the error route says |
unavailable | Not one fleet's routing document has arrived yet. Deliberately not unknown: a cold start is not a missing deployment |
scafo.host carries the normalized host the lookup was keyed by, scafo.matched the routing entry that answered and scafo.fleet the fleet that answered. Answering denied the same way as unknown is deliberate: a distinct status would confirm that the path exists.
For storage, a 404 is translated into an error wrapping io/fs.ErrNotExist, which is what CertMagic checks with errors.Is to tell a missing key from a broken backend.
#Traefik
Traefik's http provider logs a failed fetch and keeps its previous configuration. A 503 urn:scafo:draining from a node that has not finished its first reconcile pass is therefore a pause in the provider rather than a withdrawal of every route — treat it as any other fetch failure, never as an empty document.
The renderer answers {} rather than empty maps when there is nothing to publish, because empty maps make Traefik retain stale provider configuration.