Skip to content

Preserve evaluator feedback for undecodable score files - #1483

Open
Sylvester Kaczmarek (sylvesterkaczmarek) wants to merge 1 commit into
microsoft:mainfrom
sylvesterkaczmarek:fix/1457-undecodable-score-feedback
Open

Sylvester Kaczmarek (sylvesterkaczmarek) wants to merge 1 commit into
microsoft:mainfrom
sylvesterkaczmarek:fix/1457-undecodable-score-feedback

Conversation

@sylvesterkaczmarek

@sylvesterkaczmarek Sylvester Kaczmarek (sylvesterkaczmarek) commented Sep 10, 2026

Copy link
Copy Markdown

Fixes #1457.

Read the diagnostic preview of scores.csv as UTF-8 with replacement for undecodable bytes in the workflow, pipeline, and runner evaluators. CSV parsing remains strict, and corrupt output still produces negative feedback instead of a second decoding exception.

Adds 15 parameterized cases covering invalid/truncated UTF-8, empty files, malformed Unicode CSV, and valid scores across all three evaluators. The six UTF-8 corruption cases fail against the original code. Tests also verify that existing feedback and file contents are preserved.

Validation on macOS arm64, Python 3.11:

  • 26 selected offline tests passed, including all 15 new cases.
  • New tests pass the pinned Ruff lint and formatting checks; production files introduce no additional Ruff diagnostics relative to main.
  • Changed modules compile; git diff --check passes.

Model calls, execution environments, and submission checks are mocked. Full research/training workflows were not run.


馃摎 Documentation preview 馃摎: https://RDAgent--1483.org.readthedocs.build/en/1483/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: scores.csv error handling can crash on undecodable file content

1 participant