Skip to content

Use recurring objects instead of DINOv2 embeddings for annotator-sourced FP sequences - #37

Merged
Chouffe merged 11 commits into
mainfrom
feat/recurring-object-fp-identity
Aug 14, 2026
Merged

Chouffe merged 11 commits into
mainfrom
feat/recurring-object-fp-identity

Conversation

@Chouffe

@Chouffe Chouffe commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Closes #35.

The annotator cohort's identity — "same recurring artefact on the same camera" — no longer gets reconstructed from geometry and DINOv2 distance: it is read from recurring_objects.json, which records it authoritatively. Embed only what we cannot identify. Design, with the decisions recorded: docs/specs/2026-08-14-recurring-object-fp-identity-design.md.

What changes

  • embed_fp_for_selection.py skips source: pyro-annotator entries. Their embedding path was degenerate anyway — score-less five-field labels tie best_scoring_frame on every frame, so the crop was whatever directory traversal yielded first (FP selection ranks annotator sequences on a confidence column their labels do not have #29). Per-split embeddings_meta.json now holds only sequences whose identity must be inferred.
  • build_fp_yolo_dataset.py pins one background image per recurring object ahead of the clustering (new --recurring-objects arg; the ledger joins the stage deps). The mapping inverts each ledger entry's ingested_folders, restricted to registered pinned folders; a pinned folder missing from the ledger, or claimed twice, is a hard error. Frame pick is deterministic: lexicographically first folder of the object, then highest score / first filename.
  • build_sequential_dataset.py keeps pinning every sequence, and drops the now-dead filter that excluded annotator rows from the embedding items.
  • Shared helpers (partition_pinned, remaining_quota, new folder_to_recurring_object) move to pyro_dataset.fp.selection.

Verification against a pre-change baseline snapshot

  • Embeddings: exactly 50 (train) / 8 (val) / 0 (test) rows dropped; test split bit-identical; remaining pool vectors differ by at most 3.7e-07 (GPU batch-composition noise on 16 of 6,448 rows).
  • Equivalence proof: fed the baseline vectors minus annotator rows, the new sequential build reproduces the baseline FP selections byte-for-byte in all three splits — the refactor is behaviour-preserving.
  • Real rebuild: sequential differs by exactly one train folder (a same-camera, same-azimuth sibling 76 minutes apart — the measured float noise, not the code). YOLO quotas exact (two_stage+50_pinned / two_stage+8_pinned), all 58 pinned objects present, test split unchanged; train/val clustered remainder re-rolls as expected since k changed.
  • test_data_leakage: 17 passed. Full suite: 156 passed.

Merge with #36 (append-only test growth)

Main merged the test-growth implementation mid-review; this branch merges it back in (71f3da6) and the two designs compose:

  • build_sequential_dataset.py resolves to Grow the sequential test set from annotator imports, append-only #36's structure — the test split copies the frozen lockfile verbatim, no selection — with this PR's train/val simplifications inside the else-branch (helpers imported from pyro_dataset.fp.selection, dead pinned_folders filter gone).
  • The one semantic tension between the two specs resolves in this PR's favour by construction: freeze_test_selection.py pins annotator test FPs straight from the registry (no embeddings needed), and its clustering fill simply cannot see un-embedded surplus pins — they defer until new test smoke opens slots, instead of competing by embedding distance. Recorded in the spec; the test-growth spec's §5 wording ("surplus eligible for two-stage selection") is weakened accordingly.
  • Grow the sequential test set from annotator imports, append-only #36's four new frozen_test_selection tests are kept in tests/test_build_sequential_pinning.py; the pinning-helper tests live in tests/test_fp_pinning.py where this PR moved them.
  • Doc staleness the merge created is fixed: the spec no longer claims annotator sequences cannot enter test (that refusal was repealed by Grow the sequential test set from annotator imports, append-only #36), and CLAUDE.md's pinning bullet is scoped to train/val with the freeze-quota path described for test.

Post-merge verification (all local, real stages):

  • Full dvc repro on the merged tree: frozen test negatives materialise exactly (built sequential_test FP set == lockfile, 151/151); leakage gate passes with Grow the sequential test set from annotator imports, append-only #36's new recurring-object-in-test assertions (19 tests); dvc status clean; 189 unit tests pass.
  • Determinism byte-match: re-running the merged builder into a scratch directory reproduces both committed sequential datasets byte-for-byte (diff -qr across 3,208 train + 330 val + 302 test sequence folders, images and labels — zero differences).

Notes

  • Data for the rebuilt outputs is pushed to the DVC remote (229 objects — content-addressed dedup keeps the delta small).

@Chouffe
Chouffe merged commit 23ec90c into main Aug 14, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use recurring objects instead of DINOv2 embeddings for annotator-sourced FP sequences

1 participant