ci: add QA protocol enforcement checks#138
Conversation
📝 WalkthroughWalkthroughThe backend workflow now restricts pull-request activity types, runs isolated tests with scoped coverage thresholds, executes API contract tests through the isolated runner, and adds reusable credential scanning and pull-request template linting jobs. ChangesBackend workflow validation
Estimated code review effort: 4 (Complex) | ~45 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
⚔️ Resolve merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 94d53b6638
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| env: | ||
| WORKSTREAM_TEST_DATABASE_URL: postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test | ||
| WORKSTREAM_TEST_DATABASE_URL: postgresql+**********************************************/workstream_test |
There was a problem hiding this comment.
In the GitHub Actions backend job, pytest reads WORKSTREAM_TEST_DATABASE_URL through backend/tests/conftest.py, and the API-contract step below validates WORKSTREAM_DATABASE_URL with assert_local_database_url. This replacement is not a SQLAlchemy/asyncpg URL at all (it has no ://, host, or credentials), so the first DB access or contract URL check on every CI run will fail instead of connecting to the Postgres service.
Useful? React with 👍 / 👎.
|
|
||
| credential-scan: | ||
| name: Credential Scanner | ||
| uses: Flow-Research/.github/.github/workflows/credential-scanner.yml@ci/qa-protocol-enforcement |
There was a problem hiding this comment.
This new reusable workflow is referenced by the mutable branch ci/qa-protocol-enforcement (and the PR linter uses the same pattern), while this repo's PR checklist and existing workflow pattern require no unpinned new GitHub Action. If that branch moves or is deleted, Workstream CI can change or break without a reviewed commit in this repo; pin these reusable workflows to immutable commit SHAs.
Useful? React with 👍 / 👎.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/backend.yml:
- Around line 70-75: Update the reusable workflow references in the
credential-scanner and pr-lint jobs to use full 40-character commit SHAs instead
of the mutable ci/qa-protocol-enforcement branch. Pin both Flow-Research/.github
workflow calls to the intended commit and preserve the existing workflow paths
and job behavior.
- Around line 68-70: Update the credential-scan job to declare the minimum
explicit GITHUB_TOKEN permissions required by the credential scanner, rather
than inheriting repository defaults, and replace the mutable reusable workflow
ref in the uses value with its immutable commit SHA. Preserve the existing
Flow-Research credential-scanner workflow.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: e1057705-4149-4840-9d1e-5493a411952b
📒 Files selected for processing (1)
.github/workflows/backend.yml
| credential-scan: | ||
| name: Credential Scanner | ||
| uses: Flow-Research/.github/.github/workflows/credential-scanner.yml@ci/qa-protocol-enforcement |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== backend.yml excerpt ==\n'
sed -n '1,140p' .github/workflows/backend.yml
printf '\n== reusable workflow reference search ==\n'
rg -n "credential-scanner.yml|ci/qa-protocol-enforcement|permissions:" .github/workflows .github -SRepository: Flow-Research/workstream
Length of output: 3124
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== backend.yml full file with line numbers ==\n'
cat -n .github/workflows/backend.yml | sed -n '1,180p'Repository: Flow-Research/workstream
Length of output: 2913
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== locate referenced reusable workflow in repo metadata ==\n'
rg -n "credential-scanner.yml" -S .Repository: Flow-Research/workstream
Length of output: 222
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== inspect git status and diff stat for workflow files ==\n'
git status --short
git diff --stat -- .github/workflows/backend.yml .github/workflowsRepository: Flow-Research/workstream
Length of output: 221
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== show any workflow-level permissions in backend.yml ==\n'
python3 - <<'PY'
from pathlib import Path
p = Path('.github/workflows/backend.yml')
text = p.read_text()
for i, line in enumerate(text.splitlines(), 1):
if 'permissions:' in line or 'uses:' in line or 'credential-scan' in line:
print(f"{i}: {line}")
PYRepository: Flow-Research/workstream
Length of output: 607
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== check whether the external ref is a branch or tag name in GitHub semantics is not possible locally; inspect for other pins ==\n'
rg -n "`@ci/qa-protocol-enforcement`|`@main`|`@master`|`@v`[0-9]" .github/workflows -SRepository: Flow-Research/workstream
Length of output: 558
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== candidate lines around 68-77 ==\n'
nl -ba .github/workflows/backend.yml | sed -n '60,90p'Repository: Flow-Research/workstream
Length of output: 238
🏁 Script executed:
#!/bin/bash
set -euo pipefail
nl -ba .github/workflows/backend.yml | sed -n '1,120p'Repository: Flow-Research/workstream
Length of output: 202
🏁 Script executed:
#!/bin/bash
set -euo pipefail
for f in .github/workflows/agent-gates.yml .github/workflows/loop-memory.yml; do
printf '\n== %s ==\n' "$f"
sed -n '1,140p' "$f"
doneRepository: Flow-Research/workstream
Length of output: 1639
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
for path in [Path('.github/workflows/agent-gates.yml'), Path('.github/workflows/loop-memory.yml')]:
print(f"\n== {path} permissions/uses lines ==")
for i, line in enumerate(path.read_text().splitlines(), 1):
if 'permissions:' in line or 'uses:' in line:
print(f"{i}: {line}")
PYRepository: Flow-Research/workstream
Length of output: 474
Add explicit permissions to credential-scan and pin the reusable workflow ref.
The job currently inherits repository defaults, and @ci/qa-protocol-enforcement is not pinned to an immutable commit. Use the minimum token scope this scanner needs.
🧰 Tools
🪛 zizmor (1.26.1)
[warning] 68-70: overly broad permissions (excessive-permissions): default permissions used due to no permissions: block
(excessive-permissions)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/backend.yml around lines 68 - 70, Update the
credential-scan job to declare the minimum explicit GITHUB_TOKEN permissions
required by the credential scanner, rather than inheriting repository defaults,
and replace the mutable reusable workflow ref in the uses value with its
immutable commit SHA. Preserve the existing Flow-Research credential-scanner
workflow.
Source: Linters/SAST tools
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5605c721b0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| WORKSTREAM_DATABASE_URL: postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test | ||
| run: python scripts/api_contract_e2e.py | ||
|
|
||
| credential-scan: |
There was a problem hiding this comment.
For a PR consisting of this workflow update, the pre-existing Internal review evidence gate step in this same workflow still runs before tests and scripts/check_internal_review_evidence.py treats .github/ changes as requiring evidence. Since this commit changes only .github/workflows/backend.yml and adds no .agent-loop/initiatives/.../*-internal-review-evidence.md, running the gate now exits 1 with required tracks senior engineering, qa/test, security/auth, product/ops, ci integrity, so the backend workflow will block the PR before reaching the new checks.
Useful? React with 👍 / 👎.
|
@catchmeifyoucaan please fix the conflict and add me to the reviewer in other to merge the PR |
|
Closing and reopening to trigger CI run |
5605c72 to
da09a25
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/backend.yml:
- Around line 11-13: Update the test job definition in the workflow to
explicitly declare read-only permissions, using the minimum required scope
rather than inheriting repository defaults. Keep the existing runner and timeout
configuration unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 2a3439ac-8ba3-44ec-8c0a-05e85b085c03
📒 Files selected for processing (1)
.github/workflows/backend.yml
| test: | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 240 |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Explicitly restrict the test job to read-only permissions.
The job inherits repository defaults, so the workflow does not guarantee the stated read-only behavior. Declare the minimum scope explicitly.
🔒 Proposed fix
test:
+ permissions:
+ contents: read
runs-on: ubuntu-latest📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| test: | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 240 | |
| test: | |
| permissions: | |
| contents: read | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 240 |
🧰 Tools
🪛 zizmor (1.26.1)
[warning] 11-143: overly broad permissions (excessive-permissions): default permissions used due to no permissions: block
(excessive-permissions)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/backend.yml around lines 11 - 13, Update the test job
definition in the workflow to explicitly declare read-only permissions, using
the minimum required scope rather than inheriting repository defaults. Keep the
existing runner and timeout configuration unchanged.
Source: Linters/SAST tools
Wires two reusable workflows from Flow-Research/.github: - credential-scanner: flags plaintext credentials in source files - pr-template-linter: validates PR body has required sections from template
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
dbea1cd to
d4c8fb2
Compare
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2 participants
Scope
Add organization QA workflow enforcement to Workstream CI and pin reusable workflows to immutable revisions.
Invariants
Design
Use the centralized Flow Research credential scanner and PR-template linter as reusable workflows.
Duplicate and idempotency review
CI checks are read-only and repeatable. No application writes or data migrations are introduced.
Privacy and security review
Reusable workflow references are pinned to reviewed commit SHAs. Credential scanning remains enforced.
Verification
Summary by CodeRabbit