feat(ui-test): Playwright UI E2E suite — helpers + feature flows (stage 2) by cjlg-soloio · Pull Request #2196 · kagent-dev/kagent

cjlg-soloio · 2026-07-09T17:31:54Z

Description

Stage 2 of the Playwright UI E2E suite — the Stage 1 driver foundation plus the full set of multi-step feature flows built on top of it. Rebased directly on Stage 1 (#2180), so this branch stacks cleanly: once #2180 merges, this PR narrows to just the 9 commits below.

Foundation (helpers, stage 1):

Scenario-override engine in the stub backend — POST /__mock/scenario sets a per-endpoint { status, body } override; POST /__mock/reset clears them. Default stays the happy path.
Typed control seam (mocks/control.ts) surfaced as a mock fixture: mock.noAgents(), mock.agentsError(), etc. The fixture resets the stub before every test, so scenarios never leak between tests.
Driver helpers — page.ts (loadPage, expectNoErrors, expectToast), nav.ts (dropdown-aware gotoView / gotoCreate), select.ts (combobox driver), and a2a.ts (chat/stream fixtures).

Feature flows (stage 2): one spec per core UI journey, each asserting both UI state and the payload POSTed/PUT to the backend, plus empty/error/validation edge states.

Spec	Flow
`tests/agents/create-agent.spec.ts`	Declarative create-agent wizard
`tests/chat/chat-session.spec.ts`	Chat send / streamed reply / tool call / feedback / session load
`tests/models/models.spec.ts`	Model config create, edit-save, delete, gateway-key path
`tests/mcp/mcp-server.spec.ts`	Remote + stdio MCP server create, delete, filter
`tests/agents/create-harness.spec.ts`	Harness agent create
`tests/agents/agent-lifecycle.spec.ts`	Agent delete flow
`tests/prompts/prompt-libraries.spec.ts`	Prompt library create / edit / delete
`tests/onboarding/onboarding.spec.ts`	Onboarding wizard completion + skip

Product code changes are limited to a handful of additive data-testid hooks (chat input/send, tool-call card, combobox/dialog anchors) where role/text selectors were ambiguous — no behavior changed.

Testing

cd ui
npm install        # picks up @mcp-ui/client from the rebased base
npm run test:pw

Playwright boots the stub backend and dev server automatically and drives each flow through happy, empty, error, and validation states. Runs serially (workers: 1) against the shared stub. Full suite: 43 passed locally.

Change type

test — extends the Playwright E2E suite. Product changes limited to additive data-testid hooks; no runtime behavior changed.

Changelog

NONE

Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: cjlg-soloio <carlos.logrono@solo.io> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

- Force single Playwright worker unconditionally (shared stub + Next server) - Default req.method/req.url before parsing in the stub server - Fail fast on stub reset failure in CI; stay non-fatal locally - Drop unused pull-requests:write permission from the workflow Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

Pairs fullyParallel with workers: 1 so both stay off until per-test data isolation lands, preventing tests from silently racing against the shared stub backend if the worker count is raised. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

- Pin STUB_PORT on the stub webServer so a shell-set STUB_PORT can't misbind it - Stop reusing the dev server so BACKEND_INTERNAL_URL always applies (no silent stub bypass) - Type ToolServerListEntry as the real RemoteMCPServerResponse for compile-time drift detection - Drop no-op **/*.mjs from the playwright tsconfig include Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

…ght video Resolve Charlesthebird's review on PR kagent-dev#2180: - Playwright workflow now reads node-version-file: ui/.nvmrc instead of a hardcoded major version. - Remove Cypress entirely: delete cypress.config.ts and cypress/e2e/smoke.cy.ts, drop the cypress and now-unused start-server-and-test devDeps plus the test:e2e/test:e2e:cypress scripts, and remove the dead CYPRESS_INSTALL_BINARY env from the ui-playwright, ci, and ui-chromatic workflows and the Dockerfile. - Enable Playwright video recording (video: "on"); artifacts already upload via the existing test-results step. - De-Cypress the Playwright README (Jest, Vitest, Storybook untouched). data.ts is intentionally kept — it is consumed by the stacked stage-2 PR. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

Builds on the Stage 0 foundation to unlock non-happy-path testing and give specs a small, proven driver surface: - Scenario-override engine in the stub backend: POST /__mock/scenario sets a per-endpoint { status, body } override; /__mock/reset clears them. Default stays the happy path. - Typed control seam (mocks/control.ts) surfaced as a `mock` fixture (mock.noAgents(), mock.agentsError(), ...); the fixture resets the stub before every test so scenarios never leak between tests. - page.ts (loadPage, expectNoErrors, expectToast) and nav.ts (dropdown-aware gotoView/gotoCreate) driver helpers. - home spec (happy / empty / error) proving the engine, and a nav spec proving dropdown routing. Form/select/dialog helpers are deferred to Stage 2 (built demand-driven with the create-agent flow). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

First Stage 2 feature spec — the declarative create-agent flow: - happy path: fill name/description, pick the model, submit -> asserts the redirect to /agents and the captured POST /api/agents payload - validation: empty required fields block submit (no request sent) - failure: forced create error surfaces an error toast, no redirect Groundwork introduced here (demand-driven, reused by later sub-stages): - stub captures mutation (POST/PUT/DELETE) bodies and exposes them via GET /__mock/requests; supports per-mutation error overrides; reset clears both - control.ts: setMutation, agentsCreateError, capturedRequests, lastRequest - helpers/select.ts: Radix <Select> driver Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

Second Stage 2 feature (commit 2/8) — the chat streaming flow: - happy path: send a message, mock the A2A SSE reply -> asserts the user message, the streamed agent text, and a rendered tool-call block - empty state: a new chat shows "Start a conversation" - failure: an aborted stream surfaces an error toast, no agent reply Groundwork introduced here: - helpers/a2a.ts: mockAgentReply / mockAgentStreamError — page.route mock of the browser-side A2A SSE call (text/event-stream JSON-RPC frames) - stub: dynamic GET routes (agent detail, sessions-for-agent) + POST /api/sessions returning a session with an id (needed for the streamed contextId) - data-testid hooks: chat-input, chat-send (ChatInterface), tool-call (ToolDisplay) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

Third Stage 2 feature (commit 3/8) — model config lifecycle: - create: pick provider (OpenAI) + model + API key -> asserts redirect to /models, success toast, and the captured POST /api/modelconfigs payload - edit-save: open the seeded config via the edit-model hook -> Save Changes -> asserts redirect + captured PUT /api/modelconfigs/<ref> Groundwork: - stub: /api/models keyed "OpenAI" (was lowercase); stock/configured provider endpoints (/api/modelproviderconfigs/*); GET /api/modelconfigs/<ns>/<name> detail - data-testid hooks: model-provider-select, model-select, model-api-key-input Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

Fourth Stage 2 feature (commit 4/8) — create a RemoteMCPServer tool server: - happy: fill server name + URL, submit -> asserts redirect to /mcp and the captured POST /api/toolservers payload (type, url, name) - failure: forced 500 shows the inline "Couldn't create server" alert, no redirect Groundwork: stub GET /api/toolservertypes (blocking gate on /mcp/new). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

Fifth Stage 2 feature (commit 5/8) — the harness create variant at /agents/new-harness: - happy: name + model, submit -> asserts redirect to /agents and the captured POST /api/agentharnesses (kind AgentHarness, backend openclaw) - validation: empty required fields block submit (no request) Groundwork: stub POST /api/agentharnesses returns data.agent (required by createAgentHarnessFromForm). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

Sixth Stage 2 feature (commit 6/8) — delete an agent from the list: - happy: Agent options menu -> Delete -> confirm -> asserts the dialog closes and DELETE /api/agents/<ns>/<name> was captured - cancel: dismissing the dialog issues no DELETE No stub change (the generic mutation capture covers DELETE). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

Seventh Stage 2 feature (commit 7/8) — prompt libraries: - empty state: /prompts (namespace kagent) with no libraries - create: name + a fragment key/content, submit -> asserts redirect to the detail page, success toast, and the captured POST /api/prompttemplates payload Groundwork: stub GET /api/prompttemplates (list) + GET /api/prompttemplates/<ns>/<name> (detail). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

Eighth Stage 2 feature (commit 8/8) — complete the first-run onboarding wizard: walks Welcome -> Model (select existing) -> Agent Setup -> Tools -> Review -> Finish, asserting the POST /api/agents at finalize, that the wizard dismisses to the Agents list, and that localStorage['kagent-onboarding'] flips to "true". No new stub routes (all endpoints already seeded). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

cjlg-soloio · 2026-07-15T18:31:51Z

Superseded by #2264. Rebased onto the stage-1/foundation branch (#2262) and moved to a paren-free branch cjlg-soloio/playwrightTestsStage2 (the old branch name's parentheses broke the Docker buildx cache-scope in CI). Same commits; all 43 Playwright specs pass locally.

github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Jul 9, 2026

cjlg-soloio and others added 16 commits July 13, 2026 15:12

config for playwright

fa4de65

Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

Run with 1 worker

631d1c3

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: cjlg-soloio <carlos.logrono@solo.io> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

docs: explain defensive req.method/url defaults in stub server

083bcbf

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

cjlg-soloio force-pushed the feat(ui)--Playwright-tests-foundation-stage-2 branch from 1da788e to 3154d21 Compare July 13, 2026 22:13

cjlg-soloio changed the title ~~feat(ui-test): Playwright ui tests helpers + per-test scenarios stage 1~~ feat(ui-test): Playwright UI E2E suite — helpers + feature flows (stage 2) Jul 13, 2026

github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Jul 13, 2026

cjlg-soloio closed this Jul 15, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(ui-test): Playwright UI E2E suite — helpers + feature flows (stage 2)#2196

feat(ui-test): Playwright UI E2E suite — helpers + feature flows (stage 2)#2196
cjlg-soloio wants to merge 16 commits into
kagent-dev:mainfrom
cjlg-soloio:feat(ui)--Playwright-tests-foundation-stage-2

cjlg-soloio commented Jul 9, 2026 •

edited

Loading

Uh oh!

cjlg-soloio commented Jul 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

cjlg-soloio commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing

Change type

Changelog

Uh oh!

cjlg-soloio commented Jul 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

cjlg-soloio commented Jul 9, 2026 •

edited

Loading