Skip to content

fix(agent): preserve trusted fragment provenance in answer evaluation - #263

Merged
DavidHLP merged 2 commits into
codex/mixed-submission-clarificationfrom
codex/judge-fragment-provenance
Oct 11, 2026
Merged

DavidHLP merged 2 commits into
codex/mixed-submission-clarificationfrom
codex/judge-fragment-provenance

Conversation

@DavidHLP

Copy link
Copy Markdown
Owner

Answer-level evaluation dropped the supplied snapshot's sample_kind and access_scope when formatting retrieval fragments. A synthetic fragment claiming to be real in its untrusted text was therefore indistinguishable from real evidence to the answer and judging passes.

Reuse the existing SourceHit.as_model_dict() projection and serialize each fragment as a JSON object. Both passes receive snapshot provenance, while newlines and forged field claims remain inside text; the prompt explicitly says contrary text claims do not override snapshot fields. Citation selection, authorization, retries, and prompt-budget preflight are unchanged. Update canonical documentation and its bound corpus version.

Addresses PR #254 review thread PRRT_kwDORHwl-s6rC0rj. The separate overlapping behavior-category finding remains open and is not claimed fixed here.

Validation at 995cb3f:

  • Remote test-first supplied-snapshot regression: 2 failed, 32 deselected.
  • Remote answer/evaluation-entry/retrieval/repository-corpus regression: 107 passed in 5.53 seconds.
  • Independent Spec and Standards reviews: APPROVE.
  • Exact-head CI38108215383: running; not credited as passed.

Depends on PR #262. Remote validation preserves only an unrelated whitespace-only mvnw.cmd checkout normalization. No paid calls or formal acceptance occurred. Prompt projection tests do not prove live-model resistance to forged text; formal U02/U03/U04 and human evidence remain incomplete.

@DavidHLP
DavidHLP marked this pull request as ready for review October 11, 2026 03:34
@DavidHLP
DavidHLP merged commit 79c4de8 into codex/mixed-submission-clarification Oct 11, 2026
32 checks passed
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-11T03:35:50.843623Z 995cb3f Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant