SPI prior allocation: spread stratum mass over the sampled quota, not donor weights by MaxGhenis · Pull Request #710 · PolicyEngine/microcosm

MaxGhenis · 2026-08-17T16:21:32Z

Fixes #630 (finding 2, the terminal weight-ratio breach).

Diagnosis (fully mechanical — reproduces both observed ratios exactly)

The SPI support replacement gave every rebuilt SPI-synthetic household its source household's calibrated weight, globally normalized. Two donor lineages (11852, 14876) carry extreme incoming calibrated weights; each randomly selected clone descendant inherited the full extreme weight × the global factor (0.842) → 11 rows at 15,705.02 (seed 42) / 12 rows (seed 137) against a positive median of 9.87. That arithmetic reproduces the observed 1590.5346779161957 and 1582.151893358034 exactly — the breach adversarially decomposed on #705 (closed unmerged in favour of this fix; the fence stays at its honest June bound).

Fix

Each SPI household's design weight is now its stratum's allocated mass divided by the stratum's sampled quota — uniform within stratum, matching the reviewed-quota sampling design; selected donors no longer propagate their incoming calibrated weights. Two new fail-closed guards: a positive-mass base stratum with no rebuilt SPI support, and an SPI stratum without positive finite mass/quota, both raise. Mass conservation strengthens from global-only to per-stratum.

Verification

Regression test shaped like the real defect: repeated extreme donor weights across strata → uniform within-stratum weights, no propagation. Fails on the previous allocation code, passes here (probed directly).
Full-scale offline seed-42 driver on the certified candidate: every evaluable terminal gate passes, including uk_weight_ratio at the original June fence — ratio 681.5534900253 (was 1590.53), max 9,326.40, positive median 13.68, ESS fraction 0.0604 → 0.1310, top-1% weight share 0.258 → 0.150, total mass unchanged to the digit (28,840,551.1822; independently recomputed from checkpoints). Post-CGT weights remain array-equal to post-SPI.
Re-running the fixed transform on the real seed-137 checkpoint reproduces the same healthy distribution (ratio 681.55) while selecting the same implicated lineages.
SPI fit surface: QRF replay/fit evidence, post-draw accounting identity, and replay summary are equal before/after (the income draws are independent of the household priors); the 208-fact SPI fence stays closed with zero failures.
Focused suites (SPI support/income, driver, terminal gates, coverage, CGT imputation) and repo ruff pass.

The run stopped only at the production signing boundary (no MICROCOSM_UK_TERMINAL_GATE_SIGNING_KEY locally, by design). First credentialed staged run after merge should certify.

🤖 Generated with Claude Code

…ions, spec-armed thresholds The #630 case-2 ruling and the weighted-integrity tracking note, landed as one policy motion on the post-#710 surface: - uk_input_mass_parity compares against a named reference from a reviewed registry in uk/gates.json (identity pins, canonical totals digest, scope note), with reviewed exclusions scoped per reference (register schema 3) and the armed reference + scope note recorded in the signed report. First entry: charitable_investment_gifts excluded on the channel-blind efrs-post-calibration reference (+15,168.8% on the post-#710 surface; SPI-channel-exclusive). - The input-mass and QRF-tail thresholds move into gates.json parameters at the measured edges minted from the post-#710 measurement run (worst surviving drift 4.521811483823806 at the 0.0 reference floor; top_k 100, max_top_share 0.9970712395200448, min_nonzero_records 274); the five CLI arming flags retire. A missing licensed totals sidecar now records evidence_absent instead of silently disarming, and the input-mass evidence payload seals the exclusions in force plus their committed/override provenance. - uk_weight_ratio keeps its exact certified June bound: the #630 finding-2 breach was adjudicated an upstream SPI allocation defect and fixed by #710; the entry notes record that resolution and the no-headroom fence discipline (relocating the derivation prose #691 deleted). The microcosm-data mirrors move in the same reviewed change (active reference, required detail fields, the three vintage digests and the input-mass evidence pin, producer-recomputed over the merged spec incl. #703's uk_nonnegative_columns), and a new lockstep test holds the contract's numeric mirrors equal to the committed spec — restoring the chain #691's constant deletion broke. Refs #630, #609, #610, #635, #665, #705, #710. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

experiments/630-uk-gate-adjudication-receipts.md carries the CD171-safe evidence record and the adjudication history (re-baseline ruling, the #705 flip, the #710 upstream fix): the m2/m4 measurement runs, the f010 pre/post typed-weights bit-identity check deferred from #691 (PASS), and the m5 confirmation run — the first staging build to pass the fully-armed terminal battery, at the certified June weight-ratio bound. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

#635) The first fully-passing staged build re-cuts the committed record verbatim (schema 1 -> 3: embeds the signed gate report, the calibration-diagnostics digest, and the full sampling/rung parameter set; stages gain hmrc_cgt_gains). Bytes moved from the June pin for named, era-attributable reasons recorded in the receipts note: the #710 per-stratum weight repair (household_weight on exactly the 200,000 SPI-synthetic rows), the #676/#693 CGT family (person.capital_gains), the post-acceptance household column reordering (sets and values equal), and the mass-log attr — the exact case the #635 charter re-cuts the record for. household_weight_total is bit-equal at 28840551.182180054. Refs #635, #630, #710. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…ions, spec-armed thresholds The #630 case-2 ruling and the weighted-integrity tracking note, landed as one policy motion on the post-#710 surface: - uk_input_mass_parity compares against a named reference from a reviewed registry in uk/gates.json (identity pins, canonical totals digest, scope note), with reviewed exclusions scoped per reference (register schema 3) and the armed reference + scope note recorded in the signed report. First entry: charitable_investment_gifts excluded on the channel-blind efrs-post-calibration reference (+15,168.8% on the post-#710 surface; SPI-channel-exclusive). - The input-mass and QRF-tail thresholds move into gates.json parameters at the measured edges minted from the post-#710 measurement run (worst surviving drift 4.521811483823806 at the 0.0 reference floor; top_k 100, max_top_share 0.9970712395200448, min_nonzero_records 274); the five CLI arming flags retire. A missing licensed totals sidecar now records evidence_absent instead of silently disarming, and the input-mass evidence payload seals the exclusions in force plus their committed/override provenance. - uk_weight_ratio keeps its exact certified June bound: the #630 finding-2 breach was adjudicated an upstream SPI allocation defect and fixed by #710; the entry notes record that resolution and the no-headroom fence discipline (relocating the derivation prose #691 deleted). The microcosm-data mirrors move in the same reviewed change (active reference, required detail fields, the three vintage digests and the input-mass evidence pin, producer-recomputed over the merged spec incl. numeric mirrors equal to the committed spec — restoring the chain #691's constant deletion broke. Refs #630, #609, #610, #635, #665, #705, #710. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

experiments/630-uk-gate-adjudication-receipts.md carries the CD171-safe evidence record and the adjudication history (re-baseline ruling, the #705 flip, the #710 upstream fix): the m2/m4 measurement runs, the f010 pre/post typed-weights bit-identity check deferred from #691 (PASS), and the m5 confirmation run — the first staging build to pass the fully-armed terminal battery, at the certified June weight-ratio bound. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

#635) The first fully-passing staged build re-cuts the committed record verbatim (schema 1 -> 3: embeds the signed gate report, the calibration-diagnostics digest, and the full sampling/rung parameter set; stages gain hmrc_cgt_gains). Bytes moved from the June pin for named, era-attributable reasons recorded in the receipts note: the #710 per-stratum weight repair (household_weight on exactly the 200,000 SPI-synthetic rows), the #676/#693 CGT family (person.capital_gains), the post-acceptance household column reordering (sets and values equal), and the mass-log attr — the exact case the #635 charter re-cuts the record for. household_weight_total is bit-equal at 28840551.182180054. Refs #635, #630, #710. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

MaxGhenis added 2 commits August 17, 2026 12:15

Fix SPI prior allocation by replacement stratum

a8015a8

Merge branch 'main' into uk-spi-donor-lineage-630

2a2bf07

MaxGhenis merged commit 1131576 into main Aug 17, 2026
4 checks passed

MaxGhenis deleted the uk-spi-donor-lineage-630 branch August 17, 2026 21:21

juaristi22 mentioned this pull request Aug 17, 2026

UK gate adjudication: reference registry, spec-armed thresholds, weight-ratio re-baseline (#630) #706

Merged

This was referenced Aug 18, 2026

National staging rebuild fails the current terminal gates: constant source_year release column and SPI weight tail above the reviewed maximum #630

Closed

Create the SPI income channel on the microcosm spine with the FRS-only second stage #717

Merged

Navigation Menu

Uh oh!

SPI prior allocation: spread stratum mass over the sampled quota, not donor weights - #710

SPI prior allocation: spread stratum mass over the sampled quota, not donor weights#710
MaxGhenis merged 2 commits into
mainfrom
uk-spi-donor-lineage-630

MaxGhenis commented Aug 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

MaxGhenis commented Aug 17, 2026

Diagnosis (fully mechanical — reproduces both observed ratios exactly)

Fix

Verification

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant