flaky-api

An adversarial mock API for stress-testing frontend cache strategies.

Most mock servers help you build the happy path. flaky-api does the opposite — it deliberately simulates the messy real-world conditions that break optimistic updates, invalidation logic, and pagination normalization in production. Point your React Query, SWR, Apollo, or RTK Query app at it and watch your assumptions fail under controlled conditions.

Deep-dive articles on the patterns this tool is designed to test live at frontendcache.com.

---

Status

Early but runnable. Fourteen named scenarios across all six categories are implemented today, with a proxy server, deterministic seeded RNG, YAML config, and a CLI. The full scenario catalog below is the design target — scenarios marked [implemented] work today; the rest are the roadmap.

Install & run

flaky-api is not published to a package registry — you clone and build it, then run the CLI from the local build:

git clone https://github.com/frontendcache/flaky-api.git
cd flaky-api
npm install
npm run build      # compiles TypeScript to ./dist

# In one terminal: run flaky-api in front of your real backend
node dist/cli.js --port 4000 --upstream http://localhost:3001 --scenarios ./flaky.config.yaml

# Or kick the tires with the built-in echo terminal (no real backend needed)
node dist/cli.js --port 4000 --scenarios ./examples/optimistic-rollback-race.yaml --demo-upstream

# Then point your React Query / SWR / Apollo / RTK Query client at http://localhost:4000

Prefer not to build? Run straight from source with tsx via the dev script — swap node dist/cli.js for npm run dev -- anywhere in this README:

npm run dev -- --port 4000 --scenarios ./examples/optimistic-rollback-race.yaml --demo-upstream

A minimal config:

# flaky.config.yaml
upstream: http://localhost:3001
seed: 42            # optional, makes runs reproducible

scenarios:
  - id: mutation.fail-after-delay
    match: { method: [POST, PATCH], path: "/todos/**" }
    probability: 0.25
    config: { ms: 2000, status: 503 }

  - id: read.stale-after-write
    match: { path: "/todos/**" }
    config: { windowMs: 5000 }

List the built-in scenarios:

node dist/cli.js --list-scenarios

---

The problem

Cache-layer code looks correct in development because development is too kind. You don't see:

• A mutation POST that takes 12 seconds because of a backend retry storm — long enough that the user fires three more.
• A GET /users/42 that returns { updated_at: "2024-01-01" } immediately after a PATCH that should have moved updated_at forward (read-your-writes violation under replica lag).
• A cursor-paginated list where page 3 silently overlaps page 2 by two items because the underlying list mutated between requests.
• A 200 OK response whose body is a partial entity because an upstream microservice timed out and the gateway returned what it had.

Each one of these is a one-line config in flaky-api. Each one will reveal a different bug in a naive cache implementation.

---

Scenario catalog

These are the failure modes the tool models. Every scenario is independently togglable; combine them to reproduce realistic compound failures. Each entry links to the relevant deep-dive on the companion site.

Legend: [implemented] = works today, no marker = roadmap.

Latency & throughput

Scenario	What it does	Cache pattern it stresses
latency.fixed [implemented]	Adds N ms to every response	Loading-state UI, suspense boundaries
latency.jitter [implemented]	Random latency within a configured range	Debouncing, request deduplication
latency.bimodal [implemented]	95% fast, 5% slow (long-tail simulation)	Stale-while-revalidate, timeout handling
latency.slow-start	First request after N idle seconds is slow	Cold-cache UX, prefetching

→ See: Background refetch strategies

Mutation hazards

Scenario	What it does	Cache pattern it stresses
mutation.slow [implemented]	Delays write responses to expose optimistic UI flicker	Optimistic updates, rollback
mutation.fail-after-delay [implemented]	Accepts the mutation, then returns 5xx late	Rollback correctness, retry semantics
mutation.partial-success [implemented]	200 OK with a body that omits the changed field	Cache merge logic, write conflict detection
mutation.out-of-order	Reorders two concurrent mutations server-side	Mutation queue, last-write-wins handling
mutation.duplicate-effect	Applies the mutation twice (idempotency-key violation)	Idempotent mutation design

→ See: Mutation sync & rollback patterns

Read consistency

Scenario	What it does	Cache pattern it stresses
read.stale-after-write [implemented]	Reads from replica that hasn't seen the write yet	Read-your-writes, cache-after-mutation invalidation
read.version-skew [implemented]	Returns mixed-version entities in one response	Normalized cache merging, schema migration
read.partial-entity [implemented]	Returns an entity with some fields missing	Defensive selectors, fallback values
read.phantom-entity	Entity briefly exists, then 404s	Detection logic, refetch-on-missing

→ See: Reference vs value storage models

Pagination

Scenario	What it does	Cache pattern it stresses
pagination.overlap [implemented]	Returns items already seen on a previous page	Cursor-based dedup, normalized list
pagination.gap [implemented]	Skips items between pages (item inserted then deleted)	Item-not-found handling, gap detection
pagination.shifted	List mutates between requests, so cursors drift	Cursor stability strategies
pagination.partial-page	Returns N items instead of the requested page size	Loading-more UI, end-of-list detection

→ See: Normalizing cursor-based pagination

Invalidation & tags

Scenario	What it does	Cache pattern it stresses
invalidation.cascade	One write invalidates many entity types	Tag-based invalidation, dependency graphs
invalidation.stale-tag	Tag header points at an entity that no longer exists	Tag GC, orphaned-tag handling
invalidation.echo	Server emits invalidation for an entity the client doesn't have	Defensive invalidation handling

→ See: Tag-based invalidation systems

Network & transport

Scenario	What it does	Cache pattern it stresses
network.drop [implemented]	Random connection resets mid-response	Retry policy, partial-body handling
network.5xx-burst [implemented]	Brief window of 5xx errors then recovery	Exponential backoff, circuit breaker
network.429 [implemented]	Rate-limit responses with Retry-After	Backoff honoring, request scheduling
network.cdn-divergence	Different responses from different "edge nodes"	Edge cache awareness, request affinity

---

Bundled examples

Five runnable example configs live in ./examples:

• basic.yaml — jittered latency on everything plus a 25%-probability late mutation failure.
• optimistic-rollback-race.yaml — POST/PATCH/DELETE on /todos/** returns 503 after 2 s. Watch your optimistic UI roll back.
• read-your-writes.yaml — after a write to /todos/**, GETs return the pre-write body for 5 s.
• pagination-overlap.yaml — successive GETs to /feed overlap by 2 items, exposing dedup bugs in cursor pagination.
• compound-replica-lag.yaml — a realistic compound failure stacking latency.bimodal, read.version-skew, read.partial-entity, and mutation.partial-success to mimic a lagging read replica.

Try one in two commands (from the clone, after npm run build):

node dist/cli.js --scenarios ./examples/optimistic-rollback-race.yaml --demo-upstream &
curl -X POST http://127.0.0.1:4000/todos/42 -d '{}'        # 503 after 2 s
curl     http://127.0.0.1:4000/anything-else                # 200 instant

Programmatic usage

After npm run build, import from the local build (./dist) — there is no registry package:

import { startServer, type FlakyConfig } from "./dist/index.js";

const config: FlakyConfig = {
  upstream: "http://localhost:3001",
  scenarios: [
    { id: "mutation.fail-after-delay", match: { method: "POST" }, config: { ms: 1000 } },
    { id: "read.stale-after-write", match: { path: "/todos/**" }, config: { windowMs: 5000 } },
  ],
};

const server = await startServer({ config, port: 4000 });
// ... run your tests against server.url
await server.close();

Roadmap

These are designed but not implemented yet (PRs welcome — each is small and self-contained):

• MSW adapter (@flaky-api/msw) — wrap MSW handlers with scenario failures rather than running a separate server.
• Playwright route adapter — await page.route('**', flakyApi({ preset: 'optimistic-rollback-race' })).
• The scenarios in the catalog above that aren't marked [implemented].

---

Design notes

Why a separate tool, not just MSW handlers? MSW handlers are imperative and per-test. The failure modes here are reusable, named, parameterized scenarios — closer to fault-injection primitives than per-endpoint mocks. You compose them; you don't rewrite them.

Why YAML config? So scenarios can be checked into the repo, shared across teams, and referenced from CI runs and bug reports. "Reproduce with node dist/cli.js --scenarios ./bug-1247.yaml" is a useful artifact.

Why not just use chaos-mesh / toxiproxy? Those operate at the network layer. flaky-api operates at the HTTP-semantic layer — it understands the difference between a POST /todos and a GET /todos/42, so it can simulate read.stale-after-write and pagination.overlap which network-level chaos cannot.

Determinism. Every scenario accepts a seed. Bug reports should be reproducible.

---

Related reading

Deep-dive articles covering the cache patterns this tool is built to stress-test:

• State architecture & cache fundamentals
• Cache invalidation & server synchronization
• Data normalization & query key design

---

Contributing

The scenario catalog is the heart of this project. If you've hit a cache bug in production that isn't covered above, open an issue describing the failure mode. New scenarios are a few dozen lines each — see src/scenarios/ for templates.

Development:

npm install
npm test          # node:test under tsx
npm run typecheck
npm run build
npm run dev -- --scenarios ./examples/basic.yaml --demo-upstream

License

MIT