Skip to content

ci(reporter): post test results to GitHub PR comments via @testomatio/reporter#5659

Open
DavertMik wants to merge 1 commit into
4.xfrom
ci/testomatio-github-pr-comments
Open

ci(reporter): post test results to GitHub PR comments via @testomatio/reporter#5659
DavertMik wants to merge 1 commit into
4.xfrom
ci/testomatio-github-pr-comments

Conversation

@DavertMik

Copy link
Copy Markdown
Contributor

What

Enables the @testomatio/reporter GitHub pipe across the mocha-based test workflows so each job posts a pass/fail summary directly as a PR comment.

The reporter (v2.9.0) was already a devDependency and already wired into every mocha script (--reporter @testomatio/reporter/mocha). This PR only supplies the CI environment the pipe needs.

Changes (6 workflows)

Added to test.yml, plugin.yml, playwright.yml, puppeteer.yml, webdriver.yml, appium_Android.yml:

permissions:
  contents: read
  pull-requests: write

env:
  GH_PAT: ${{ github.token }}                 # activates the GitHub pipe (matches existing webdriver.yml convention)
  TESTOMATIO: ${{ secrets.TESTOMATIO }}       # activates TMS reporting once the secret is set; dormant no-op until then
  • Added the missing --reporter @testomatio/reporter/mocha flag to the bare Playwright/Puppeteer helper unit runs.
  • Dropped the now-redundant per-step GH_PAT in webdriver.yml (covered by workflow-level env).

Verification

Simulated a PR environment locally — the mocha reporter activated the GitHub pipe and built the exact comment CI will post:

GitHub Pipe: Enabled
Pipes: GitHub Reporter, Debug Reporter
🟢 UNIT-TESTS PASSED

With no secrets (local/non-PR runs) it's a safe no-op: Pipes: No pipes enabled.

Notes

  • PR comments work immediately on same-repo PRs via github.token. To also stream results into Testomat.io TMS, add the repo secret TESTOMATIO.
  • Fork PRs won't receive comments — GitHub makes GITHUB_TOKEN read-only and withholds secrets on pull_request runs from forks (platform limitation).
  • One comment per workflow-job, updated in place across re-runs (keyed by a hidden marker).
  • Scope is mocha tests only; CodeceptJS acceptance runs (./bin/codecept.js run) are intentionally untouched. appium_iOS.yml was skipped (disabled if: false, 3.x-only push).

🤖 Generated with Claude Code

…/reporter

Enable the reporter's GitHub pipe across the mocha-based test workflows so
each job posts a pass/fail summary as a PR comment. The reporter was already
a devDependency and wired into every mocha script; this only supplies the
CI environment it needs:

- permissions: pull-requests: write (so github.token can comment)
- GH_PAT: ${{ github.token }} (activates the GitHub pipe; matches webdriver.yml)
- TESTOMATIO: ${{ secrets.TESTOMATIO }} (activates TMS pipe once the secret is set;
  dormant no-op until then)

Also add the missing --reporter @testomatio/reporter/mocha flag to the bare
Playwright/Puppeteer helper unit runs, and drop the now-redundant per-step
GH_PAT in webdriver.yml (covered by workflow-level env).

Scope is mocha tests only; CodeceptJS acceptance runs are untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Testomat.io Report 🟢 BUILD PASSED
Tests ✔️ 1 tests run
Summary 🟢 1 passed; 🟡 0 skipped
Duration 🕐 8 seconds
Job 🗂️ Plugins tests / build
Operating System 🖥️ Linux X64

🐢 Slowest Tests

  • CodeceptJS plugin: should initialize the coverage plugin and attempt coverage collection (8.219 seconds)

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Testomat.io Report 🟢 UNIT-TESTS PASSED
Tests ✔️ 39 tests run
Summary 🟢 39 passed; 🟡 0 skipped
Duration 🕐 2 seconds
Job 🗂️ Run Unit tests / unit-tests
Operating System 🖥️ Linux X64

🐢 Slowest Tests

  • ApiDataFactory: should create multiple posts and cleanup after (1.114 seconds)
  • ApiDataFactory: should not remove records if cleanup:false (0.507 seconds)
  • ApiDataFactory: should create a new post (0.186 seconds)
  • ApiDataFactory: should cleanup created data (0.109 seconds)
  • REST - Form upload: should show error when file size exceedes the permit (0.039 seconds)

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Testomat.io Report 🟢 UNIT-TESTS-WINDOWS PASSED
Tests ✔️ 33 tests run
Summary 🟢 33 passed; 🟡 0 skipped
Duration 🕐 0 seconds
Job 🗂️ Run Unit tests / unit-tests-windows
Operating System 🖥️ Windows X64

🐢 Slowest Tests

  • Container: should load TypeScript steps_file that imports other TS files (0.682 seconds)
  • Container: should load TypeScript helper that imports another TypeScript file without extension (0.015 seconds)
  • Container: should properly execute methods from TypeScript steps_file (0.01 seconds)
  • Container: should handle TypeScript files that use require() (0.01 seconds)
  • Container: should create Russian translation (0.009 seconds)

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Testomat.io Report 🟢 APPIUM PASSED
Tests ✔️ 9 tests run
Summary 🟢 9 passed; 🟡 0 skipped
Duration 🕐 1 minute, 14 seconds
Job 🗂️ Appium Tests - Android / appium
Operating System 🖥️ Linux X64

🐢 Slowest Tests

  • Appium: should react on swipeUp action @second (11.803 seconds)
  • Appium: should react on swipeDown action @second (11.725 seconds)
  • Appium: should append field value @second (11.384 seconds)
  • Appium: should fill field by accessibility id (10.272 seconds)
  • Appium: should fill field by xpath (9.596 seconds)

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Testomat.io Report 🟢 RUNNER-TESTS PASSED
Tests ✔️ 277 tests run
Summary 🟢 275 passed; 🟡 2 skipped
Duration 🕐 7 minutes, 47 seconds
Job 🗂️ Run Unit tests / runner-tests
Operating System 🖥️ Linux X64

🐢 Slowest Tests

  • CodeceptJS Workers Runner: should handle large worker count without inflating statistics (17.295 seconds)
  • Failure in before: should trigger skipped events (11.445 seconds)
  • CodeceptJS Workers Runner: should report accurate test statistics in pool mode (10.61 seconds)
  • CodeceptJS Workers Runner: should handle single vs multiple workers statistics consistently in pool mode (10.521 seconds)
  • CodeceptJS Workers Runner: should report correct test counts with grep filtering in pool mode (10.49 seconds)

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Testomat.io Report 🟢 BUILD PASSED
Tests ✔️ 397 tests run
Summary 🟢 373 passed; 🟡 24 skipped
Duration 🕐 10 minutes, 55 seconds
Job 🗂️ Puppeteer Tests / build
Operating System 🖥️ Linux X64

🐢 Slowest Tests

  • Puppeteer: should show correct number key when Shift modifier is active (31.178 seconds)
  • Puppeteer: should wait for text after timeout (30.017 seconds)
  • Puppeteer: should show correct numpad or punctuation key when Shift modifier is active (15.143 seconds)
  • Puppeteer: fills large multi-paragraph content (12.208 seconds)
  • Puppeteer: fills large multi-paragraph content (12.091 seconds)

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Testomat.io Report 🟢 BUILD PASSED
Tests ✔️ 446 tests run
Summary 🟢 414 passed; 🟡 32 skipped
Duration 🕐 11 minutes, 5 seconds
Job 🗂️ Playwright Tests / build
Operating System 🖥️ Linux X64

🐢 Slowest Tests

  • Playwright: should wait for text after timeout (30.478 seconds)
  • Playwright: should show correct numpad or punctuation key when Shift modifier is active (14.151 seconds)
  • Playwright: fills large multi-paragraph content (12.087 seconds)
  • Playwright: fills large multi-paragraph content (11.927 seconds)
  • Playwright: fills large multi-paragraph content (11.853 seconds)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant