Skip to content

relay: workflows created pre-#1369 are orphaned — undeletable, undisableable, and fire forever #1593

Description

@AaronGoldsmith

Summary

Workflows created while a relay ran the pre-#1369 code path (≤ relay-v0.1.1) are permanently unmanageable — invisible orphans that keep firing on their cron schedules and cannot be disabled or deleted through any client-visible ID. We hit this in production (staging relay sprout-oss) and it took several days and a code-level investigation to stop twice-daily ghost messages.

Background: how the orphans are created

Before #1369 ("Harden relay attack surfaces", merged 2026-06-30), handle_workflow_def called create_workflow, which generated a random server-side UUID (Uuid::new_v4()) for the DB row and ignored the kind:30620 event's d-tag:

// crates/buzz-db/src/workflow.rs (relay-v0.1.1)
pub async fn create_workflow(...) -> Result<Uuid> {
    let id = Uuid::new_v4();   // <-- never matches the event d-tag
    ...
}

Clients (CLI buzz workflows list, desktop) list workflows by querying kind:30620 events and reading the d-tag as the workflow ID. So every workflow ID a user has ever seen never matched the actual workflows row. Consequences:

  1. Disable is a no-op: re-submitting the definition with enabled: false (same d-tag) created a new DB row under yet another random UUID. The original enabled: true row keeps firing.
  2. Delete is a no-op: kind:5 with a-tag 30620:<owner>:<visible-uuid> looks up a UUID that doesn't exist in the DB. Worse, the relay returns accepted: true with no error, so the user believes the deletion worked (side-effect failures are logged server-side only).
  3. After the relay was upgraded past Harden relay attack surfaces #1369 (upsert-by-d-tag), all new operations behave correctly — but the legacy orphan rows remain, enabled, firing forever, with UUIDs no client can discover.

How we escaped

handle_a_tag_deletion has a name-based fallback: when the a-tag d-value is not a UUID, it resolves via find_workflow_by_owner_and_name(community, actor, name). This reaches orphans regardless of UUID. The stock CLI rejects non-UUID values (parse_uuid), so we patched cmd_delete_workflow locally to emit 30620:<owner>:<name> and swept the orphans (repeating, since the lookup is LIMIT 1). Verified with webhook canaries (202 → delete → 404).

Proposed fixes

  1. Orphan reconciliation (the actual long-term fix). A migration or startup/admin sweep that deletes (or at minimum disables + logs) workflows rows whose id does not match any live kind:30620 event's d-tag for the same community. Every community that created workflows before the relay crossed Harden relay attack surfaces #1369 has these landmines.
  2. Surface side-effect failures for deletions. kind:5 processing returns accepted: true even when the workflow deletion hits NotFound. The OK/response message should indicate "no matching workflow" so users don't believe a no-op succeeded. (Same applies to disable-via-update flows.)
  3. CLI: support name-based deletion. The relay already supports it and it's the only recovery path for ID-mismatch situations: buzz workflows delete --name <workflow-name>.
  4. Name-based deletion ignores the a-tag pubkey. handle_a_tag_deletion's name branch looks up by actor_bytes (the deletion signer), so the is_agent_owner allowance in validate_standard_deletion_event (humans managing agent-owned content, feat(relay): allow agent owners to edit/manage agent-owned content #1403) is dead code on this path — a human owner cannot clean up their agent's orphans by name. The lookup should honor the a-tag's pubkey component after ownership validation passes.

Related

Repro sketch

Not directly reproducible on current main (creation is fixed); the failure mode requires rows created under ≤ relay-v0.1.1. To simulate: insert a workflows row with enabled=true and a random id unrelated to any 30620 event d-tag, then attempt to disable/delete it through the CLI.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions