Writing pipelines that survive the night

A pipeline that works when you run it by hand can still fail at 3 a.m., silently. On top of the schema and the references, pipelines validate applies five fixed rules to the shell text of each pipeline: when[].shell guards, run: of a shell step and run: of an always: entry. Each rule comes from a real incident.

The five rules

Rule	What it catches	Why
R1	A `{{...}}` inside single quotes or a `<<'EOF'` heredoc	Bash doesn't expand the engine's safe variable there, and the step gets an empty value.
R2	`\s` in a `grep -E` pattern	It isn't POSIX, and macOS `grep` doesn't accept it. Use `[[:space:]]`. This one cost ten silent nights.
R3	A multi-line `run:` that doesn't start with `set -euo pipefail`	Without it, a command that fails halfway leaves the step marked as successful.
R4	`triggers:` and `notify:` without `notify.stale_after`	A pipeline held back by a guard has no error to report, only absence.
R5	An assignment `x=$(… grep …)` without `\|\|` inside the substitution	Under `set -e`, a `grep` with no matches kills the step before it reaches the fallback.

R1: a reference where bash doesn't expand it

The engine replaces each {{...}} with ${__PIPELINES_REF_n} and passes the real value through the child process's environment. Inside single quotes, or in a heredoc with a quoted delimiter (<<'EOF'), bash doesn't expand that variable, and the step receives the literal variable name instead of the value. Typical symptom: a path that silently arrives empty.

R2: `\s` inside `grep -E`

It isn't POSIX ERE (it's a GNU extension) and macOS's grep doesn't accept it; you need [[:space:]]. A guard built on it can make a pipeline skip silently for days.

R3: a multi-line run without `set -euo pipefail`

Without it, a command that fails midway through a block leaves the step successful, and the real failure shows up later and somewhere else. Not required in when: guards: a guard is a one-line expression evaluated by its exit code, not a script.

R4: triggers and notify without stale_after

Without it, a pipeline held back by a guard tells nobody: there is no error to notify, only absence. See Notifications.

R5: `x=$(… grep …)` without `||`

Under set -e (and pipefail if there's a pipe) the assignment takes the substitution's exit code, and grep exits with 1 when it doesn't match. That's exactly the "no match" case the fallback on the next line meant to handle, so the step dies before reaching it. The rule only looks at assignments at the start of a line (with or without export, local or readonly): if x=$(grep …) is guarded by construction.

Why the list is closed

There's no rule registry and no configurable severity: adding a sixth rule means editing the engine. That's on purpose: the moment the linter accepts configuration, it stops being a list of incidents and becomes a framework nobody maintains.

Validating a draft with --file

bash

pipelines validate --file path/to/pipeline.yaml

--file validates a standalone pipeline.yaml, before it lives under pipelines/<name>/. It frees the draft from three requirements of an installed pipeline:

name: doesn't have to match a containing directory.
Prompts are resolved relative to the file's own directory. A missing one is a warning, not an error, but then that prompt's references haven't been checked, and the warning says so.
notify.channel is checked only if a pipelines.yaml is found walking up from the file's directory; if none is found, that's also a warning.

No warning changes the exit code, and validate --file always lists what it couldn't check.

validate, with or without --file, collects all the problems it finds before failing instead of stopping at the first one. Two checks still abort on their own before the rest run: a cycle in the dependency graph and, outside --file, a missing agent prompt.

validate checks that the YAML is correct and that its references resolve; it doesn't execute anything. A pipeline can pass without a single warning and still fail at 3 a.m. because of something no static check can see, a tail -n +4 that miscounts a blank line, for example. The risk narrows; it doesn't close.

Conventions every pipeline should follow

Every variable value goes through params. Nothing hardcoded in a run: or in a prompt.
Secrets by name only (requires.env), with the value in .env. A {{secrets.*}} reference may only appear in the prompt of an agent step; in a run: it's a validation error.
What the agent claims, a shell step verifies. An agent saying it saved a screenshot doesn't prove the file exists: follow the agent step with a shell step that checks it.
A when: guard doesn't clean anything up. It's evaluated before any step runs; if it depends on state that only a step clears, the pipeline blocks itself forever. Cleanup belongs inside a step.
Use always: for anything ephemeral: closing browsers, emptying work directories, deleting temporary clones.
Make stale_after somewhat longer than the throttle, so that silence raises an alert.
A step whose predecessor was skipped is skipped in cascade: a missing reference in a step-level when: counts as false, not as an error.
Pipelines shouldn't work in your own checkouts. Each pipeline gets its own workspace, passed in through a param; to see what it produced, git fetch in your checkout.

The environment a step sees

A shell step, a shell: guard and a notification channel receive a narrow, fixed environment: PATH, HOME, SHELL, USER, TZ and TMPDIR inherited, plus the declared secrets, never the whole environment of the process.

The locale is pinned to C.UTF-8, not inherited. That way a pipeline behaves the same when launched by hand as from launchd, systemd or cron, which don't export LANG. It matters more than it seems: in the C locale, a regex . matches a byte, not a character, so any pattern over accented text fails only under cron. C.UTF-8 gives UTF-8 semantics in regexes while keeping bytewise collation, so no sort changes order.