dev-harness

A Docker-sandboxed, orchestrator-verified ticket-in / PR-out coding harness. Give it a task (a CLI prompt or a labeled GitHub issue) and it plans, implements, verifies, reviews, and delivers a reviewed pull request — or a patch for a local folder — with no human involvement before review.

Staged model routing is fully configurable. Each stage (plan, implement, repair, review) runs on one of two providers: claude — your Claude subscription via Claude Code — or api — the Anthropic API or any Anthropic-compatible endpoint, with an optional model override. Run everything on one provider or split stages across the two; the default router plans and reviews on claude and implements and repairs on api.

How it works

  trigger                sandbox (one Docker container per task)
  ───────                ─────────────────────────────────────────────
  harness run ─┐         ┌───────────────────────────────────────────┐
               ├──▶ queue │  provision → plan → implement             │
  gh issue     │  (SQLite)│   (claude)   (api)                        │
  labeled      ┘    │     │       │          │                        │
  `agent`           │     │       ▼          ▼                        │
  (daemon poll)     └────▶│    verify ◀── commit (orchestrator)       │
                          │       │                                   │
                          │   pass │ fail                             │
                          │       │   └─▶ repair (api) ──┐            │
                          │       │        (≤ 2 rounds)   │            │
                          │       │        ▲──────────────┘            │
                          │       ▼   fail after 2 → escalate (ntfy)   │
                          │    review (claude, structured verdict)     │
                          │       │                                    │
                          │       ▼                                    │
                          │    deliver ─▶ PR (repo) / patch (folder)   │
                          └───────────────────────────────────────────┘
                                  │
                                  ▼
                       host: git push + gh pr create   (credential-free
                             (operator credentials)     container never
                                                         sees a token)

Verification runs from the host via docker exec, never through the agent, so the agent cannot fake a green gate. The commit, push, and PR are all made by the orchestrator with the operator's credentials; the container holds no git or model credentials except the single per-stage model token passed at exec time.

Research lineage: this harness draws on the patterns catalogued in research/ — Stripe's blueprint-driven runs and hard repair cap, Spotify's separation of the agent runtime from the verification runtime, Ramp's warm dependency caches, and the credential-free sandbox model.

Install

Prerequisites (host): Docker 24+, GitHub CLI (gh) 2.9x authenticated (gh auth login), Node 22+, pnpm, and git.

One-line install. While the repo is private, install via authenticated gh:

gh api repos/mike-diff/dev-harness/contents/install.sh -H "Accept: application/vnd.github.raw" | sh

Once the repo is public, the plain curl form works too:

curl -fsSL https://raw.githubusercontent.com/mike-diff/dev-harness/main/install.sh | sh

The installer tracks main and lays out a single install root:

~/.harness/
  repo/          cloned source + build (dist/) + node_modules
  bin/harness    launcher on your PATH (auto-loads state/.env)
  state/         queue.db, sessions/, config.json, .env   (HARNESS_HOME)
  VERSION        installed commit

It adds ~/.harness/bin to your PATH (idempotently, in your shell rc), migrates an existing ~/.dev-harness into state/, and seeds state/.env from the template. Open a new shell (or export PATH="$HOME/.harness/bin:$PATH") and the harness command works from any directory. harness self update / self uninstall / self doctor manage the install (see the command reference).

Configure credentials in ~/.harness/state/.env (seeded from .env.example):

Variable	How to obtain
CLAUDE_CODE_OAUTH_TOKEN	claude setup-token (Claude subscription; needed when any stage routes to claude)
ANTHROPIC_API_KEY or ANTHROPIC_AUTH_TOKEN	auth for the api provider — an Anthropic API key, or your provider's token (set exactly one)
ANTHROPIC_BASE_URL	optional; empty targets the Anthropic API, else your provider's Anthropic-compatible endpoint
ANTHROPIC_MODEL	optional model override for the api provider (empty uses the endpoint's default)
NTFY_URL	optional; your ntfy topic URL for escalation notifications (empty disables)
HARNESS_HOME	optional; the launcher sets it to ~/.harness/state

Never commit .env. Each stage exec receives exactly one provider's credentials — never both.

Verify and build the sandbox image, then run a task:

harness self doctor                         # checks prereqs, docker, tokens
harness build-image                         # harness-base image (~1.8 GB, once)
harness run "add an isEven function with tests" --repo owner/name

From a source checkout (development)

If you are working on the harness itself rather than installing it:

pnpm install
pnpm build                                  # compiles src/ to dist/
pnpm dev run "<task>" --repo owner/name      # run from TypeScript, no build step

A source checkout reads .env from the project directory (via node --env-file), independent of any ~/.harness install.

Command reference

Command	Purpose
harness run "<task>" --repo <owner/name>	Enqueue a task targeting a GitHub repo.
harness run "<task>" --path <abs-folder>	Enqueue a task targeting a local folder (delivered as a patch). Exactly one of --repo/--path.
harness execute <id>	Run a queued task through the full pipeline to a PR or patch.
harness issue <n> --repo <owner/name>	One-shot: enqueue an agent-labeled issue (same guards and write-backs as the daemon) and run it to a PR. --force re-runs a seen issue.
harness list	List all tasks with id, state, source, target, and age.
harness status <id>	Show a task's full record plus its event log.
harness approve <id>	Apply a delivered patch to the original folder (folder tasks), after a diff-stat preview and confirmation. --yes skips the prompt.
harness daemon	Run the long-lived daemon: poll configured repos for agent-labeled issues, claim and execute queued tasks (concurrency 3). Clean shutdown on SIGINT.
harness build-image	Build (or --force rebuild) the harness-base sandbox image.
harness spike <claude|api>	Run a one-turn provider check in a scratch container (validates credentials/endpoint).
harness sandbox --repo/--path	Provision a one-off sandbox and print its container id (--keep to retain).
harness self update	Pull the latest main, reinstall deps, and rebuild. Leaves state/ untouched; notes if the sandbox image needs a rebuild.
harness self uninstall	Remove the install and its PATH entry. Keeps state/ (queue, config, secrets) unless --purge.
harness self doctor	Check prerequisites, docker, the sandbox image, and token presence (never prints values).
harness self version	Print the installed commit and install location.

Configuration

Optional config.json under HARNESS_HOME (~/.harness/state/config.json for an installed harness, or ~/.dev-harness/config.json from a source checkout):

{
  "router": {
    "plan": "claude",
    "implement": "api",
    "repair": "api",
    "review": "claude",
    "claudeFallbackToApi": false
  },
  "repos": {
    "owner/name": {
      "poll": true,
      "owner": "your-github-login",
      "includeCollaborators": ["trusted-login"],
      "verify": ["pnpm run lint", "pnpm run test"]
    }
  }
}

• router — which provider (claude or api) runs each stage. claudeFallbackToApi routes claude stages to the api provider when set (e.g. during a rate-limit window).
• repos["owner/name"] — per-repo settings:
  • poll — the daemon polls this repo for agent-labeled issues (every 45s).
  • owner — the GitHub login whose issue text is trusted for prompt assembly (see the injection guard below).
  • includeCollaborators — additional logins whose comments may enter the prompt.
  • verify — explicit verification commands, overriding auto-detection.

Verification commands are otherwise auto-detected: package.json scripts (lint/typecheck/test via pnpm or npm per the lockfile), or Python (ruff check / pytest), else none.

Run as a GitHub Action

The harness also runs with zero infrastructure as a GitHub Action: label an issue agent and an ephemeral runner executes the full pipeline, ending in a PR on your repo. GitHub is the queue in this mode — the label event replaces the daemon's poll loop (harness issue is the one-shot equivalent of its poll-claim-execute), and since applying a label requires triage permission, the label doubles as the execution permission gate.

Copy examples/github-actions/agent-issue.yml into .github/workflows/ in your repository, add your provider credentials as Actions secrets, and label an issue agent. Setup details, routing options, and caveats are documented in the template itself. The workflow pulls the prebuilt sandbox image from GHCR (published by this repo's publish-image workflow; the :actions tag matches GitHub-hosted runners' uid) instead of building it. One caveat worth repeating: PRs created with the default GITHUB_TOKEN do not trigger your other workflows — use a PAT or GitHub App token if CI must run on harness PRs.

Security model

• Credential-free containers. The sandbox never receives git or GitHub credentials. The only secret entering a container is the single per-stage model token, passed via docker exec -e and never written to disk. Cloned repos have their local credential helper neutralized so no config in the sandbox can request host credentials.
• The host owns push and verification. The workspace is a host bind mount, so git push and gh pr create run host-side with the operator's identity, and verification runs from the host via docker exec — the agent cannot fake a green gate or push on its own.
• Push target integrity. The push targets a URL derived from the trusted task.repo (not the agent-writable workspace remote). Global/system git config is excluded for the push, and if the workspace config carries any url.* rewrite rule (insteadOf / pushInsteadOf), delivery aborts and escalates rather than push — no agent-writable config can change where the credentialed push lands.
• Issue-prompt injection guard. When a task is triggered by a GitHub issue, only text authored by the configured owner (plus opted-in collaborators) enters the prompt. Third-party comment bodies are excluded by default, so an attacker cannot inject instructions into the agent via an issue comment.
• Secret redaction on egress. Runner error output, ntfy notifications, and issue comments are scrubbed of the in-scope provider token values before they leave the process.

Escalation

• Bounded repair. A failing verification triggers at most 2 repair rounds (the counter lives in the database, not agent context). Still failing after round 2 → the task is escalated.
• Notifications. Every terminal failure — escalated or failed — POSTs a scrubbed message to your ntfy topic. Issue-triggered tasks also receive a failure comment on the issue. A review verdict that does not approve (or a critical finding) drives one extra repair round against the cap, else escalates.

Development

pnpm build       # compile src/ to dist/ (tsconfig.build.json)
pnpm test        # unit + integration tests (add RUN_DOCKER_TESTS=1 for @docker tests)
pnpm typecheck   # strict TypeScript, zero any
pnpm lint        # (tsc --noEmit)