agentic-pipelines documentation

agentic-pipelines is an engine for pipelines that run unattended. You declare a pipeline in YAML as a graph of steps: plain shell steps, and agent steps that hand the work to an AI agent (through the Claude Agent SDK) inside a filesystem sandbox that the engine itself enforces, not the SDK. launchd on macOS, or a systemd user timer on Linux, fires each pipeline on schedule; the engine runs it, stores every run on disk and notifies you when it finishes.

The engine ships no pipelines of its own. Yours live in a separate data repo that you create, and the engine finds it at run time by locating its pipelines.yaml. If you'd rather start from something that already works, pipelines-starter has seven ready-to-use pipelines.

Status. Version 0.1.0. It runs on macOS (scheduled with launchd) and on Linux with systemd (scheduled with user timers). There is no crontab backend, and Windows through WSL2 hasn't been verified.

How it fits together

Architecture of agentic-pipelines: the operator, or the scheduler (launchd or systemd), calls the pipelines CLI, which hands the pipeline to the runner. The runner runs shell steps and agent steps; every tool call of an agent step goes through the engine's filesystem guard before reaching an MCP server or the workspace. The runner reads the data repo and writes runs to .runs/, and calls the notification channel when the run closes. The read-only web dashboard reads .runs/ and the scheduler's state. — The engine's architecture, from the engine repo.

Three separate places are involved, and it pays to keep them apart in your head:

Place	What it holds	Who changes it
The engine	Schema, runner, CLI, filesystem guard, linter	Whoever adds a capability to the engine
The data repo (yours)	One directory per pipeline with its `pipeline.yaml`, the prompts for its `agent` steps, the agents, and the notification channel scripts	Whoever adds or changes a pipeline
Outside both	The `launchd` plists or `systemd` units, each pipeline's workspaces, the `.runs/` history	The engine, when installing and running

The third one is what you won't see by reading code, and it's where to look when something doesn't add up. What lives on disk lists every path.

The engine assumes nothing about what a pipeline does: it only knows how to execute the contract that pipeline.yaml describes. Each step declares its inputs and outputs, the engine builds the graph, validates every output against its contract before passing it to the next step, and a set of guards decides whether the run starts at all.

What happens during a run

Trigger. launchd or the systemd timer at the cron time, or someone running pipelines run.
Preflight. Three checks, in this order:
- requires checks whether this machine can run the pipeline (binaries, agents, secrets, network). If anything is missing, the run fails.
- The per-pipeline lock. If a run is already in progress, the command exits with code 3.
- The when guards check whether today is the day (throttle, changed, shell, between). If any of them doesn't pass, the run ends as skipped, with exit code 0. That's not an error: it's the normal outcome on half the nights of a pipeline with a throttle.
Steps. In the order the graph dictates. A shell step runs with sh -c (bash --posix -c on Linux) in a narrow, fixed environment; an agent step opens a governed session (see Agent steps and the sandbox). A step with outputs: must write exactly the JSON of its contract to stdout.
Wrap-up. The steps under always: run every time, on success or failure. Then the notify channel receives the result, if notify.on includes it or if the pipeline has gone too long without a success (stale_after).
Result. A success anchors the throttle of the next run. A failed run stores the last lines of the failing step's stderr in that step's error, and can be resumed with pipelines resume.

Everything lands in .runs/<pipeline>/<run-id>/ inside the data repo: a run.json with the full record and one log per step, with secrets already redacted.

Where to go next

Install on macOS or Install on Linux.
Your first pipeline: a shell-only pipeline, then one with an agent step, checked against the real CLI.
The pipeline.yaml reference for every field, and Commands for every command and exit code.

Known limitations

There is no pipelines init. Creating a new data repo (its pipelines.yaml, agents/, .env) is still manual.
launchd and systemd only. install and uninstall know nothing about crontab or other schedulers. Windows through WSL2 hasn't been verified.
stdio MCP servers only. SSE and HTTP are not supported.
kind: playwright or kind: declared in mcp_servers. playwright uses the engine's governance table. declared is for a server of your own whose tools the engine doesn't know: you list what's allowed in allow and vouch that none of those tools runs arbitrary code or writes where it shouldn't.
No dependency cache between runs. The engine doesn't install a target repo's dependencies (a yarn install, for example); preparing each environment is up to whoever prepares it.