Your first pipeline

A data repo is a directory with a pipelines.yaml at its root. Here you'll create one with two pipelines: one that's shell only, and one that adds an agent step. Both examples have been checked against the real CLI.

1. A minimal data repo

mkdir -p my-pipelines/pipelines/hello-world my-pipelines/agentscd my-pipelinesecho 'channels: {}' > pipelines.yamlprintf '.env\n.params.local.json\n.runs/\n' > .gitignore

pipelines.yaml can be almost empty, but it has to exist: it's the marker the engine uses to find the repo root by walking up directories. The three entries in .gitignore are files the engine generates or reads, and they must never be committed.

Once it grows, a data repo looks like this.

2. A shell-only pipeline

pipelines/hello-world/pipeline.yaml:

name: hello-worlddescription: Minimal example pipeline, shell only.version: 1params:  name:    description: Who to greet.    default: worldrequires:  bin: [jq]steps:  - id: greet    type: shell    run: |      set -euo pipefail      echo "greeting {{params.name}}" >&2      jq -n --arg n "{{params.name}}" '{message: ("hello, " + $n)}'    outputs:      message: string  - id: count    type: shell    inputs: [greet.message]    run: |      set -euo pipefail      n=$(printf '%s' "{{greet.message}}" | wc -c | tr -d ' ')      jq -n --arg n "$n" '{chars: ($n | tonumber)}'    outputs:      chars: number

Three things here are conventions in every pipeline:

• name matches the directory name. If it doesn't, validate fails.
• A step with outputs: writes only its contract's JSON to stdout. Everything else goes to stderr with >&2. Here jq -n builds the JSON.
• {{...}} is the only way to pass values between steps and from params. In a run: block the engine doesn't do text interpolation: it replaces each reference with a safe environment variable, so the value is never reinterpreted as shell syntax.

Now, from any subdirectory of the data repo:

pipelines validate# schema, graph, references and the lint rulespipelines doctor hello-world# can this machine run it?pipelines run hello-world --set name=Danipipelines status hello-worldpipelines show hello-world <run-id>

run prints one line per step and then the final status with the run id:

  ✓ greet                success  ✓ count                successsuccess — 2026-10-05T15-57-44-383Z

show dumps the full run.json: you'll see outputs.message: "hello, Dani" on the first step and outputs.chars: 11 on the second.

3. Adding an agent step

An agent is a Markdown file with front matter in agents/. This is the minimal one, agents/summarizer.md:

---description: Example agent that answers with a minimal JSON object.---You are an example agent. Reply with exactly the JSON the step asks for,with no extra text.

The prompt for each agent step lives next to the pipeline, by convention under steps/. pipelines/hello-agent/steps/summarize.md:

Read the file `notes.txt` in your working directory and return a JSON objectwith a `summary` field (one sentence) and a `lines` field (the number oflines in the file).

And pipelines/hello-agent/pipeline.yaml:

name: hello-agentdescription: A shell step that prepares data and an agent step that summarizes it.version: 1params:  workdir:    description: The agent's working directory. It is the only write root.    required: truerequires:  agents: [summarizer]  bin: [jq]steps:  - id: prepare    type: shell    run: |      set -euo pipefail      mkdir -p "{{params.workdir}}"      printf 'one\ntwo\nthree\n' > "{{params.workdir}}/notes.txt"      jq -n '{ready: true}'    outputs:      ready: boolean  - id: summarize    agent: summarizer    prompt: steps/summarize.md    cwd: "{{params.workdir}}"    inputs: [prepare.ready]    tools: [Read]    max_cost_usd: 0.5    outputs:      summary: string      lines: number

What changes compared to a shell step:

• agent: and prompt: instead of run:. agent is the implicit type.
• cwd is the sandbox's write root. The engine does not create it, which is why the previous step does.
• tools: [Read] is everything the agent is allowed to do. Without tools: it gets Read, Grep and Glob.
• max_cost_usd caps the spend of this particular step.

agent steps authenticate with your Anthropic API key, read from the data repo's .env (gitignored):

echo 'ANTHROPIC_API_KEY=sk-ant-...' >> .env

pipelines validate hello-agentpipelines doctor hello-agent# also checks the agent and your Anthropic API keypipelines run hello-agent --set workdir=/tmp/hello-agent

What's next

With this you have the building blocks of any real pipeline. The next leap is declaring triggers, when and notify:

• Scheduling: triggers and pipelines install.
• Notifications: notify and channels.
• Surviving the night: the five lint rules and the conventions that keep a pipeline honest when nobody is watching.
• The pipeline.yaml reference: the full contract for every field.