Skip to content

fix+test: ensemble missing-key crash and eligibility boundary condition - #24

Merged
cschanhniem merged 2 commits into
mainfrom
feat/eligibility-boundary-ensemble-config-validation
Jun 28, 2026
Merged

cschanhniem merged 2 commits into
mainfrom
feat/eligibility-boundary-ensemble-config-validation

Conversation

@cschanhniem

Copy link
Copy Markdown
Collaborator

Summary

Bug fix: ensemble.py silent crash on missing score key

# Before (raises KeyError if scores is missing a weighted key)
value = sum(scores[name] * weight for name, weight in weights.items()) / total_weight

# After (degrades gracefully, treats missing scorer as 0 contribution)
value = sum(scores.get(name, 0.0) * weight for name, weight in weights.items()) / total_weight

If any upstream scorer fails to produce a key that appears in weights, the old code would raise KeyError and crash the entire pipeline run (no JSONL output, no certificates, no manifest). With .get(), the missing scorer contributes 0.0 to the ensemble — a safe degradation that still produces ranked output so a human can review.

New tests

  • test_missing_score_key_defaults_to_zero — weights has "safety" but scores doesn't; verifies result is 0.5714 (weighted avg with safety=0.0) not a crash
  • test_extra_score_key_not_in_weights_is_ignored — extra keys in scores (e.g. boman_activity) that aren't in weights are ignored; result is unchanged
  • test_eligibility_novelty_at_exact_minimum_passes — eligibility filter uses >= so a candidate with novelty=0.0 and min_novelty=0.0 must be selected; confirms boundary operator is inclusive

Test plan

  • pytest tests/test_scoring_ensemble.py tests/test_pipeline_filters.py — 41 passed
  • Full suite pytest — 777 passed
  • Pre-push hook passed

Context

Iteration #24 of the pre-wet-lab quality loop. The ensemble missing-key crash is a real reliability risk: if a physchem feature is NaN or a scorer throws, the whole run dies silently instead of producing degraded but usable output.

🤖 Generated with Claude Code

…#24)

ensemble.py: scores[name] → scores.get(name, 0.0)
  Previously, if a scorer failed to produce a key that appears in weights,
  ensemble_score would raise KeyError crashing the entire pipeline run.
  Using .get() degrades gracefully (treats missing scorer as 0 contribution).

New tests:
- test_missing_score_key_defaults_to_zero: weights has "safety" but scores
  doesn't → result is weighted average treating missing key as 0.0, not crash
- test_extra_score_key_not_in_weights_is_ignored: extra keys in scores are
  silently ignored, consistent with weighted average semantics
- test_eligibility_novelty_at_exact_minimum_passes: verifies the eligibility
  filter uses >= (not >): novelty=0.0 with min_novelty=0.0 should be selected
A missing key was silently treated as 0.0, which would hide YAML config
typos (e.g. 'activty' → correct score 0.91 vs wrong score 0.34). Now
emits warnings.warn(UserWarning) listing the missing keys so the issue
is auditable in logs without crashing. Test updated to use pytest.warns()
to assert the warning is actually emitted.
@cschanhniem
cschanhniem merged commit 3fb8ee6 into main Jun 28, 2026
3 checks passed
@cschanhniem
cschanhniem deleted the feat/eligibility-boundary-ensemble-config-validation branch June 28, 2026 02:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant