Add K1e answer replay harness

[2002yy]

Scope

K1e adds a reproducible answer replay harness on top of merged K1a-K1d.

• reuses the existing Study Agent LLM provider owner and model profiles
• replays the existing 10 answer-quality gold cases through the production local-knowledge path
• preserves K1b evidence eligibility, K1c refusal, and K1d adaptive source coverage
• reuses the existing answer evaluator
• records corpus/prompt fingerprints, provider/model identity, latency, and available usage metadata
• keeps raw replay reports under the ignored output/ directory

Provenance

The runner determines provenance from the concrete production adapter. A test double cannot label itself as a completed real-provider replay by changing a string field. Synthetic tests remain synthetic_test; unavailable or failed real calls do not receive fabricated quality scores.

Current status

The harness and regression coverage are implemented. No completed real-provider benchmark is claimed in this PR yet; CI validates the harness with synthetic providers and keeps the deterministic K1a-K1d contracts green.

K2 structural parsing and chunking remains out of scope.

Merge gate

The latest head must pass full CI, and project status must continue to distinguish harness readiness from an actually executed real-provider replay.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4224d74b50

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Make the documented replay command import the project package

Running the documented command python tools/run_rag_provider_replay.py ... from the repository root fails before it can initialize a provider: Python puts tools/ (not the repository root) on sys.path, so this new sibling-package import raises ModuleNotFoundError: No module named 'src'. Use a module invocation in the documentation/entry point, install the package, or add an intentional bootstrap so the advertised replay harness can run without a caller-provided PYTHONPATH.

Useful? React with 👍 / 👎.