Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 68 additions & 2 deletions EVENTS.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,85 @@
# NDJSON Events

When running `aictrl run --format json`, the CLI emits newline-delimited JSON (NDJSON) events to stdout. Each line is a self-contained JSON object with the following base shape:
When the parsed `aictrl run --format json` handler starts, the CLI emits newline-delimited JSON (NDJSON) events to stdout. Each line is a self-contained JSON object with the following base shape:

```json
{
"type": "<event_type>",
"timestamp": 1741500000000,
"invocationID": "7d142250-8bdc-43df-99af-efa252db62a7",
"sessionID": "session_01abc..."
}
```

The schema is versioned via `session_start.schemaVersion`. This document describes **schema version `"1"`**. Consumers should pin to this version and treat unknown fields as forward-compatible additions.
`invocationID` is present on every event from `run --format json`. `sessionID` is present only after a real session has been created; invocation events never fabricate one.

The schema is versioned via `invocation_start.schemaVersion` and `session_start.schemaVersion`. This document describes **schema version `"1"`**. Consumers should pin to this version and treat unknown fields as forward-compatible additions.

The invocation envelope covers accepted `run --format json` executions from the first line of the parsed handler through validation, bootstrap, session creation, and execution. Argument-parser failures, other commands, and process-global uncaught exceptions or unhandled rejections are outside this contract.

## Lifecycle Events

### `invocation_start`

Emitted once, before piped stdin is read and before run validation or bootstrap begins.

```json
{
"type": "invocation_start",
"timestamp": 1741500000000,
"schemaVersion": "1",
"invocationID": "7d142250-8bdc-43df-99af-efa252db62a7"
}
```

This event intentionally has no `sessionID`, because a session does not exist yet.

### `invocation_error`

Emitted for a fatal error before session creation, immediately before `invocation_complete`.

```json
{
"type": "invocation_error",
"timestamp": 1741500000001,
"schemaVersion": "1",
"invocationID": "7d142250-8bdc-43df-99af-efa252db62a7",
"phase": "validation",
"code": "INVOCATION_FILE_NOT_FOUND",
"message": "Invocation failed during validation"
}
```

- `phase` (string, **required**) — one of `validation`, `stdin`, `bootstrap`, or `session`.
- `code` (string, **required**) — stable machine-readable error category. Known failures use
`INVOCATION_INVALID_DIRECTORY`, `INVOCATION_FILE_NOT_FOUND`, `INVOCATION_EMPTY_INPUT`,
`INVOCATION_INVALID_ARGUMENTS`, or `INVOCATION_SESSION_CREATE_FAILED`. Unexpected failures use
`INVOCATION_<PHASE>_FAILED`, where `<PHASE>` is `VALIDATION`, `STDIN`, `BOOTSTRAP`, or `SESSION`.
- `message` (string, **required**) — sanitized human-readable phase summary. Details remain on stderr and in the log.

Errors after a real session has been created use `session_error` instead. They also set
`invocation_complete.status` to `error`, so the invocation result always agrees with the process result.

### `invocation_complete`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Error code set undocumented in EVENTS.md.

Suggested change
Add an enumerated list of `code` values (or at minimum document the `INVOCATION_<PHASE>_FAILED` default fallback) under the `invocation_error` section so the public contract matches the implementation.
🤖 Fix with your agent
Fix this code review finding (aictrl-dev/cli PR #96, EVENTS.md:61):

Problem: Error code set undocumented in EVENTS.md
Detail: EVENTS.md describes `invocation_error.code` only as a "stable machine-readable error category" with a single example (`INVOCATION_FILE_NOT_FOUND`). But run.ts passes a fixed set of codes (`INVOCATION_INVALID_DIRECTORY`, `INVOCATION_FILE_NOT_FOUND`, `INVOCATION_EMPTY_INPUT`, `INVOCATION_INVALID_ARGUMENTS`, `INVOCATION_SESSION_CREATE_FAILED`) and run.invocation.ts:31 synthesizes a default `INVOCATION_${phase.toUpperCase()}_FAILED` (e.g. `INVOCATION_BOOTSTRAP_FAILED`, `INVOCATION_SESSION_FAILED`, `INVOCATION_STDIN_FAILED`) for any uncaught error. None of these are enumerated in the doc and the default-code pattern is entirely undocumented, so consumers pinning to "stable" codes cannot discover the full set or rely on the fallback shape.
Suggested fix: Add an enumerated list of `code` values (or at minimum document the `INVOCATION_<PHASE>_FAILED` default fallback) under the `invocation_error` section so the public contract matches the implementation.

Implement the fix on the PR head branch and add a regression test that fails before the fix and passes after.
Why this matters

EVENTS.md describes invocation_error.code only as a "stable machine-readable error category" with a single example (INVOCATION_FILE_NOT_FOUND). But run.ts passes a fixed set of codes (INVOCATION_INVALID_DIRECTORY, INVOCATION_FILE_NOT_FOUND, INVOCATION_EMPTY_INPUT, INVOCATION_INVALID_ARGUMENTS, INVOCATION_SESSION_CREATE_FAILED) and run.invocation.ts:31 synthesizes a default INVOCATION_${phase.toUpperCase()}_FAILED (e.g. INVOCATION_BOOTSTRAP_FAILED, INVOCATION_SESSION_FAILED, INVOCATION_STDIN_FAILED) for any uncaught error. None of these are enumerated in the doc and the default-code pattern is entirely undocumented, so consumers pinning to "stable" codes cannot discover the full set or rely on the fallback shape.

Emitted exactly once for every started invocation.

```json
{
"type": "invocation_complete",
"timestamp": 1741500000123,
"schemaVersion": "1",
"invocationID": "7d142250-8bdc-43df-99af-efa252db62a7",
"sessionID": "session_01abc...",
"status": "completed",
"durationMs": 123
}
```

- `status` (string, **required**) — `completed` or `error`.
- `durationMs` (number, **required**) — elapsed invocation time.
- `sessionID` (string, optional) — included only when a real session was created.

### `session_start`

Emitted once when the session begins.
Expand Down
88 changes: 88 additions & 0 deletions packages/cli/src/cli/cmd/run.invocation.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
import { Stdout } from "../stdout"
import { SCHEMA_VERSION } from "./run.errors"

export type RunInvocationPhase = "validation" | "stdin" | "bootstrap" | "session"

export function createRunInvocation(enabled: boolean) {
const id = enabled ? crypto.randomUUID() : undefined
const started = Date.now()
let phase: RunInvocationPhase = "validation"
let sessionID: string | undefined
let failed = false
let completed = false
let writes = Promise.resolve()

function emit(type: string, data: Record<string, unknown> = {}) {
if (!id) return
writes = writes.then(() =>
Stdout.json({
type,
timestamp: Date.now(),
schemaVersion: SCHEMA_VERSION,
invocationID: id,
...data,
}),
)
}

emit("invocation_start")

function error(_error: unknown, code = `INVOCATION_${phase.toUpperCase()}_FAILED`) {
if (!id || failed || completed) return
failed = true
if (sessionID) return
emit("invocation_error", {
phase,
code,
message: `Invocation failed during ${phase}`,
})
}

function complete() {
if (!id || completed) return
completed = true
emit("invocation_complete", {
status: failed ? "error" : "completed",
durationMs: Date.now() - started,
...(sessionID ? { sessionID } : {}),
})
}

async function abort(cause: unknown, code?: string) {
error(cause, code)
complete()
await writes
}

return {
id,
phase(next: RunInvocationPhase) {
phase = next
},
link(session: string) {
sessionID = session
},
error,
abort,
async guard<T>(task: () => T | Promise<T>) {
try {
return await task()
} catch (cause) {
await abort(cause)
throw cause
}
},
async run<T>(task: Promise<T> | (() => T | Promise<T>)) {
try {
return await (typeof task === "function" ? task() : task)
} catch (cause) {
error(cause)
throw cause
} finally {
await writes
complete()
await writes
}
},
}
}
59 changes: 41 additions & 18 deletions packages/cli/src/cli/cmd/run.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ import { BashTool } from "../../tool/bash"
import { TodoWriteTool } from "../../tool/todo"
import { Locale } from "../../util/locale"
import { Stdout } from "../stdout"
import { createRunInvocation } from "./run.invocation"

type ToolProps<T extends Tool.Info> = {
input: Tool.InferParameters<T>
Expand Down Expand Up @@ -375,19 +376,26 @@ export const RunCommand = cmd({
})
},
handler: async (args) => {
const invocation = createRunInvocation(args.format === "json")

async function fail(message: string, code: string) {
UI.error(message)
await invocation.abort(message, code)
process.exit(1)
}

let message = [...args.message, ...(args["--"] || [])]
.map((arg) => (arg.includes(" ") ? `"${arg.replace(/"/g, '\\"')}"` : arg))
.join(" ")

const directory = (() => {
const directory = await (async () => {
if (!args.dir) return undefined
if (args.attach) return args.dir
try {
process.chdir(args.dir)
return process.cwd()
} catch {
UI.error("Failed to change directory to " + args.dir)
process.exit(1)
await fail("Failed to change directory to " + args.dir, "INVOCATION_INVALID_DIRECTORY")
}
})()

Expand All @@ -397,12 +405,13 @@ export const RunCommand = cmd({

for (const filePath of list) {
const resolvedPath = path.resolve(process.cwd(), filePath)
if (!(await Filesystem.exists(resolvedPath))) {
UI.error(`File not found: ${filePath}`)
process.exit(1)
if (!(await invocation.guard(() => Filesystem.exists(resolvedPath)))) {
await fail(`File not found: ${filePath}`, "INVOCATION_FILE_NOT_FOUND")
}

const mime = (await Filesystem.isDir(resolvedPath)) ? "application/x-directory" : "text/plain"
const mime = (await invocation.guard(() => Filesystem.isDir(resolvedPath)))
? "application/x-directory"
: "text/plain"

files.push({
type: "file",
Expand All @@ -413,16 +422,18 @@ export const RunCommand = cmd({
}
}

if (!process.stdin.isTTY) message += "\n" + (await Bun.stdin.text())
if (!process.stdin.isTTY) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Pre-run throw skips invocation_complete event.

🤖 Fix with your agent
Fix this code review finding (aictrl-dev/cli PR #96, packages/cli/src/cli/cmd/run.ts:423-427):

Problem: Pre-run throw skips invocation_complete event
Detail: The handler body between `createRunInvocation(...)` and the first `invocation.run()`/`abort()` is not wrapped in any try/catch/finally. The `fail()` helper only covers the explicit validation branches (bad dir, missing file, empty input, fork args, no session), but awaited I/O before `invocation.run()` can reject — most notably `await Bun.stdin.text()` inside the `phase("stdin")` block, plus `Filesystem.exists`/`Filesystem.isDir` in the `--file` loop and `createAictrlClient(...)` on the attach path. Such a rejection propagates out of the handler without ever calling `complete()`, so `invocation_start` is emitted but `invocation_complete` is NEVER emitted — violating the PR's own EVENTS.md contract ("Emitted exactly once for every started invocation"). A JSON consumer then sees `invocation_start` followed by EOF with no terminal event and no `invocation_error` explaining the failure, so it cannot distinguish "process died" from "NDJSON stream truncated".
Suggested fix: Wrap the handler body (from createRunInvocation through the final invocation.run) in try/catch/finally; in the catch call `await invocation.abort(cause)` and re-throw, so complete() always runs. Alternatively, expose a single guaranteeing wrapper on the invocation object (e.g. `invocation.drain(() => body())`) that emits the terminal event regardless of where the body throws.

Implement the fix on the PR head branch and add a regression test that fails before the fix and passes after.
Why this matters

The handler body between createRunInvocation(...) and the first invocation.run()/abort() is not wrapped in any try/catch/finally. The fail() helper only covers the explicit validation branches (bad dir, missing file, empty input, fork args, no session), but awaited I/O before invocation.run() can reject — most notably await Bun.stdin.text() inside the phase("stdin") block, plus Filesystem.exists/Filesystem.isDir in the --file loop and createAictrlClient(...) on the attach path. Such a rejection propagates out of the handler without ever calling complete(), so invocation_start is emitted but invocation_complete is NEVER emitted — violating the PR's own EVENTS.md contract ("Emitted exactly once for every started invocation"). A JSON consumer then sees invocation_start followed by EOF with no terminal event and no invocation_error explaining the failure, so it cannot distinguish "process died" from "NDJSON stream truncated".

      }
    }

    if (!process.stdin.isTTY) {
      invocation.phase("stdin")
      message += "\n" + (await Bun.stdin.text())
      invocation.phase("validation")
    }

invocation.phase("stdin")
message += "\n" + (await invocation.guard(() => Bun.stdin.text()))
invocation.phase("validation")
}

if (message.trim().length === 0 && !args.command) {
UI.error("You must provide a message or a command")
process.exit(1)
await fail("You must provide a message or a command", "INVOCATION_EMPTY_INPUT")
}

if (args.fork && !args.continue && !args.session) {
UI.error("--fork requires --continue or --session")
process.exit(1)
await fail("--fork requires --continue or --session", "INVOCATION_INVALID_ARGUMENTS")
}

const rules: PermissionNext.Ruleset = [
Expand Down Expand Up @@ -508,7 +519,14 @@ export const RunCommand = cmd({

function emit(type: string, data: Record<string, unknown>) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Local emit() duplicates NDJSON writer.

Suggested change
function emit(type: string, data: Record<string, unknown>) {
Extract a shared `writeNdJsonLine(obj)` helper (into the stdout module or run.invocation.ts) and route both emitters through it so the field-set and ordering stay aligned.
🤖 Fix with your agent
Fix this code review finding (aictrl-dev/cli PR #96, packages/cli/src/cli/cmd/run.ts:518-522):

Problem: Local emit() duplicates NDJSON writer
Detail: The local `emit` inside `execute()` duplicates the NDJSON line construction (`JSON.stringify({...}) + EOL` then `Stdout.write`) just introduced in `createRunInvocation.emit` (run.invocation.ts:16-27). The two have already diverged in shape: the invocation emit always includes `schemaVersion` and orders fields `invocationID` then `...data`, while this local emit omits `schemaVersion` from its base object and orders `invocationID`, `sessionID`, `...data`. Per the consistency checklist, a shared line-writer belongs in one module; leaving two copies lets them drift further as more events are added.
Suggested fix: Extract a shared `writeNdJsonLine(obj)` helper (into the stdout module or run.invocation.ts) and route both emitters through it so the field-set and ordering stay aligned.

Implement the fix on the PR head branch and add a regression test that fails before the fix and passes after.
Why this matters

The local emit inside execute() duplicates the NDJSON line construction (JSON.stringify({...}) + EOL then Stdout.write) just introduced in createRunInvocation.emit (run.invocation.ts:16-27). The two have already diverged in shape: the invocation emit always includes schemaVersion and orders fields invocationID then ...data, while this local emit omits schemaVersion from its base object and orders invocationID, sessionID, ...data. Per the consistency checklist, a shared line-writer belongs in one module; leaving two copies lets them drift further as more events are added.

      function emit(type: string, data: Record<string, unknown>) {
        if (args.format === "json") {
          Stdout.write(
            JSON.stringify({ type, timestamp: Date.now(), invocationID: invocation.id, sessionID, ...data }) + EOL,
          )
          return true
        }
        return false

if (args.format === "json") {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Two emit() helpers diverge on schemaVersion handling.

🤖 Fix with your agent
Fix this code review finding (aictrl-dev/cli PR #96, packages/cli/src/cli/cmd/run.ts:477-483):

Problem: Two emit() helpers diverge on schemaVersion handling
Detail: There are now two sibling `emit(type, data)` functions building NDJSON envelopes. `invocation.ts` constructs `{type, timestamp, schemaVersion: SCHEMA_VERSION, invocationID, ...data}` and queue-serializes writes; `run.ts` constructs `{type, timestamp, invocationID: Invocation.id, sessionID, ...data}` synchronously and does NOT inject `schemaVersion` — it relies on every caller remembering to pass `schemaVersion` inside `data`. EVENTS.md (modified by this PR) states the schema is versioned via BOTH `invocation_start.schemaVersion` and `session_start.schemaVersion`, so the two emit paths are responsible for the same contract but enforce it differently: an `invocation_*` event can never forget `schemaVersion`, while a future `session_*` event added in run.ts that forgets to include `schemaVersion` in its data object will silently violate the documented schema. The duplication also means any future envelope field (cliVersion, pid, etc.) must be added in two places.
Suggested fix: Extract a single `writeEvent(type, data, ctx)` (or `buildEnvelope(type, data, ctx)`) helper — ideally on the Invocation module since it already owns the queued-write plumbing and `SCHEMA_VERSION` — and have run.ts route `session_*` events through it. This guarantees `schemaVersion` is always injected and that session events flush in order with invocation events.

Implement the fix on the PR head branch and add a regression test that fails before the fix and passes after.
Why this matters

There are now two sibling emit(type, data) functions building NDJSON envelopes. invocation.ts constructs {type, timestamp, schemaVersion: SCHEMA_VERSION, invocationID, ...data} and queue-serializes writes; run.ts constructs {type, timestamp, invocationID: Invocation.id, sessionID, ...data} synchronously and does NOT inject schemaVersion — it relies on every caller remembering to pass schemaVersion inside data. EVENTS.md (modified by this PR) states the schema is versioned via BOTH invocation_start.schemaVersion and session_start.schemaVersion, so the two emit paths are responsible for the same contract but enforce it differently: an invocation_* event can never forget schemaVersion, while a future session_* event added in run.ts that forgets to include schemaVersion in its data object will silently violate the documented schema. The duplication also means any future envelope field (cliVersion, pid, etc.) must be added in two places.

      function emit(type: string, data: Record<string, unknown>) {
        if (args.format === "json") {
          process.stdout.write(
            JSON.stringify({ type, timestamp: Date.now(), invocationID: Invocation.id, sessionID, ...data }) + EOL,
          )
          return true
        }
        return false

Stdout.write(JSON.stringify({ type, timestamp: Date.now(), sessionID, ...data }) + EOL)
Stdout.json({
type,
timestamp: Date.now(),
schemaVersion: SCHEMA_VERSION,
invocationID: invocation.id,
sessionID,
...data,
})
return true
}
return false
Expand Down Expand Up @@ -678,6 +696,7 @@ export const RunCommand = cmd({
// spuriously-green job. process.exitCode (not process.exit) lets the
// loop drain to session.status idle and emit session_complete first.
process.exitCode = 1
invocation.error(props.error)
if (!sessionErrorEmitted) {
sessionErrorEmitted = true
const classified = classifySessionError(props.error)
Expand Down Expand Up @@ -811,11 +830,12 @@ export const RunCommand = cmd({
})()
const agent = agentInfo.name

invocation.phase("session")
const sessionID = await session(sdk)
if (!sessionID) {
UI.error("Session not found")
process.exit(1)
await fail("Session not found", "INVOCATION_SESSION_CREATE_FAILED")
}
invocation.link(sessionID)
await share(sdk, sessionID)

emit("session_start", {
Expand Down Expand Up @@ -876,6 +896,7 @@ export const RunCommand = cmd({
})
console.error(e)
process.exitCode = 1
invocation.error(e)
})

if (args.command) {
Expand Down Expand Up @@ -926,17 +947,18 @@ export const RunCommand = cmd({
})
console.error(e)
process.exitCode = 1
invocation.error(e)
},
),
])
}

invocation.phase("bootstrap")
if (args.attach) {
const sdk = createAictrlClient({ baseUrl: args.attach, directory })
return await execute(sdk)
return await invocation.run(() => execute(createAictrlClient({ baseUrl: args.attach, directory })))
}

await bootstrap(process.cwd(), async () => {
const execution = bootstrap(process.cwd(), async () => {
const sdk = {
session: {
async list() {
Expand Down Expand Up @@ -1016,5 +1038,6 @@ export const RunCommand = cmd({
}
await execute(sdk)
})
await invocation.run(execution)
},
})
6 changes: 6 additions & 0 deletions packages/cli/src/cli/stdout.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import { EOL } from "os"

const state = {
bound: false,
closed: false,
Expand Down Expand Up @@ -56,6 +58,10 @@ export namespace Stdout {
if (failure !== undefined) throw failure
}

export function json(data: Record<string, unknown>) {
return write(JSON.stringify(data) + EOL)
}

export function isClosed() {
return state.closed
}
Expand Down
Loading