Skip to content

Report weighted target loss attribution in calibration fit - #160

Merged
anth-volk merged 5 commits into
mainfrom
calibration-follow-up
Aug 18, 2026
Merged

Report weighted target loss attribution in calibration fit#160
anth-volk merged 5 commits into
mainfrom
calibration-follow-up

Conversation

@anth-volk

@anth-volk anth-volk commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Summary

  • consume authoritative per-target final-loss attribution from Microcosm diagnostics schema version 6
  • reconstruct the audited historical weighting doctrines at read time for supported releases without backfilling published artifacts
  • fail closed when reported or reconstructed attribution is incomplete or inconsistent
  • replace the former Loss Sources calculation with weighted target error and add a separate target-weight view
  • use weighted mean capped error for attribution-view coloring and update the overview metrics, tooltips, and presentation
  • document the app's distinct loss and error concepts

Cross-repository contract

Producer implementation: PolicyEngine/microcosm#716 (merged; fixes PolicyEngine/microcosm#715).

The producer and consumer share the five row fields target_loss_weight, target_loss_weight_share, target_loss_scale, final_capped_scaled_error, and final_loss_contribution, plus the schema-version-6 target_loss_basis. Both sides use the same ordered-basis hash and a 1e-12 absolute or relative contribution tolerance.

Consumer fixtures mirror Microcosm's uniform, unequal/custom-scale, and structured-unavailable outputs. Cross-repository tests verify hash agreement, warning handling, reported attribution precedence, and contribution reproduction.

User impact

Calibration Fit can now show which targets carry the most calibration importance and which contribute most to weighted target error. Historical releases use deterministic, version-pinned fallbacks where exact reconstruction is possible; unsupported attribution remains unavailable without exposing internal provenance in the normal UI.

The External datasets page and its independent evaluation loss are unchanged.

Validation

  • GitHub CI — passed
  • bun test — 184 passed
  • bun run lint — passed
  • bun run build — passed
  • git diff --check — passed
  • local Calibration Fit presentation visually confirmed by the user

@vercel

vercel Bot commented Aug 18, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
calibration-diagnostics Ready Ready Preview Aug 18, 2026 8:26pm

Request Review

@anth-volk
anth-volk marked this pull request as ready for review August 18, 2026 20:29
@anth-volk
anth-volk merged commit 19463d7 into main Aug 18, 2026
4 checks passed
@anth-volk
anth-volk deleted the calibration-follow-up branch August 18, 2026 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Publish per-target final loss attribution in calibration diagnostics

1 participant