Skip to content

fix: harden context ranking and release v0.5.1#58

Merged
aryamthecodebreaker merged 1 commit into
mainfrom
codex/external-eval-ranking
Jul 18, 2026
Merged

fix: harden context ranking and release v0.5.1#58
aryamthecodebreaker merged 1 commit into
mainfrom
codex/external-eval-ranking

Conversation

@aryamthecodebreaker

Copy link
Copy Markdown
Owner

What changed

  • fixes context ranking regressions exposed by a frozen six-repository external benchmark
  • makes exact path evidence win, maps build-output paths back to TypeScript sources, preserves seed priority through imports, and caps proximity boosts below direct evidence
  • adds small priors against examples and declaration files for runtime tasks
  • adds focused ranking regressions and grows core coverage to 44 tests
  • records exact external top-five outputs and fails scheduled/release evaluation on result drift
  • adds a root GitHub Action entrypoint while retaining the nested metadata for compatibility
  • prepares v0.5.1 packages, changelog, curated release notes, docs, and an evidence-based 5,000-star launch kit

Measured result

Frozen external benchmark, before → after:

  • Top-1: 33.3% (2/6) → 50.0% (3/6)
  • Top-3: 33.3% (2/6) → 83.3% (5/6)
  • Top-5: 66.7% (4/6) → 83.3% (5/6)

The remaining Zod miss is documented rather than hidden or relabeled.

Internal benchmark: Top-1 62.5% (5/8), Top-3 87.5% (7/8).

Verification

  • npm run ci
  • Action 5 tests, CLI 41 tests, Core 44 tests
  • typecheck, lint, production build, bundle-drift checks, CLI/Action/workspace smoke tests
  • node scripts/evaluate-external.mjs --gate --check-recorded
  • core and CLI npm pack --dry-run
  • root/nested Action metadata synchronization
  • v0.5.1 package/dependency/lock/MCP metadata validation
  • independent ranking, release-integrity, and upstream-truth reviews: no remaining blockers

Release note

This makes the repository eligible for GitHub Marketplace listing through its root action.yml; actual Marketplace publication remains an owner-side release UI step and is not claimed here.

@aryamthecodebreaker
aryamthecodebreaker merged commit 48710b5 into main Jul 18, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant