diff --git a/CLAUDE.md b/CLAUDE.md
index c858c6f..0090b38 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -145,9 +145,13 @@ What differs from the platform loop:
the next `dvc repro`, or the build errors on the stale lockfile.
Design: `docs/specs/2026-08-14-annotator-test-growth-design.md`.
- **False-positive boxes are class 99**, not `0`: they mark where the detector
- fired, not smoke. `build_sequential_dataset.py` pins annotator-sourced FP
- sequences ahead of its clustering, so the ones chosen upstream actually reach
- the built dataset.
+ fired, not smoke. Annotator-sourced FP sequences are never embedded — their
+ identity comes from `recurring_objects.json` — and in train and val both
+ builders pin them ahead of clustering: the sequential build takes every
+ sequence, the YOLO build one image per recurring object. In test they reach
+ `sequential_test` only through `freeze_test_selection.py`, up to the quota —
+ surplus pins are deferred until new test smoke opens slots. See
+ `docs/specs/2026-08-14-recurring-object-fp-identity-design.md`.
- Each folder carries a `meta.json` with every lane's full track, including boxes
left out of `labels/`, so an object-level dataset can be derived later without
re-exporting.
diff --git a/data/reporting/leakage_results.xml b/data/reporting/leakage_results.xml
index 0526fd6..c9de1af 100644
--- a/data/reporting/leakage_results.xml
+++ b/data/reporting/leakage_results.xml
@@ -1 +1 @@
-
+
diff --git a/docs/specs/2026-08-13-annotator-export-sequential-import-design.md b/docs/specs/2026-08-13-annotator-export-sequential-import-design.md
index 53f624e..54c5ad1 100644
--- a/docs/specs/2026-08-13-annotator-export-sequential-import-design.md
+++ b/docs/specs/2026-08-13-annotator-export-sequential-import-design.md
@@ -434,6 +434,10 @@ This does not displace anything. The import adds as many smoke sequences as the
holds (58 here), which raises `quota = n_wf` by the same number — the pinned FPs consume
exactly the slots their own positives created.
+Since [2026-08-14-recurring-object-fp-identity-design.md](2026-08-14-recurring-object-fp-identity-design.md),
+pinning extends to the YOLO build (one image per recurring object) and the
+cohort is no longer embedded at all.
+
### 7. Changes to existing pyro-dataset code
Everything else is new code. Exactly two existing files change, and both changes are
diff --git a/docs/specs/2026-08-14-recurring-object-fp-identity-design.md b/docs/specs/2026-08-14-recurring-object-fp-identity-design.md
index 35601c3..a4b6696 100644
--- a/docs/specs/2026-08-14-recurring-object-fp-identity-design.md
+++ b/docs/specs/2026-08-14-recurring-object-fp-identity-design.md
@@ -136,8 +136,11 @@ score, ties broken by filename. Annotator labels are score-less, so in practice
the first labeled frame by name — but the rule stays correct if a scored sequence ever
carries a `source`.
-Test split: untouched. Annotator sequences never enter test
-(`assignments_from_splits` refuses it), so round-robin is unaffected.
+Test split: untouched by this design. When it was written, annotator sequences
+could not enter test (`assignments_from_splits` refused it); the test-growth
+design has since repealed that refusal, so annotator test FPs will eventually
+appear in the round-robin pool — accepted, as the test-growth design declares
+detector-test behaviour out of its scope.
### `scripts/build_sequential_dataset.py`
@@ -196,8 +199,12 @@ Integration verification, in order:
beyond fewer sequences passing through KMeans.
- Interaction with
[2026-08-14-annotator-test-growth-design.md](2026-08-14-annotator-test-growth-design.md)
- (designed separately): it opens `split: test` to annotator imports, repealing the
- "test is never assigned" premise this document leans on, and its §5 makes surplus
- annotator test FPs "eligible for two-stage selection" — which presumes embeddings
- this design stops producing. That reconciliation belongs to the test-growth
- implementation, not to this one.
+ (designed separately, merged since): it opens `split: test` to annotator imports,
+ repealing the "test is never assigned" premise this document originally leaned on.
+ Its §5 makes surplus annotator test FPs "eligible for two-stage selection", which
+ presumes embeddings this design stops producing. How that lands in practice:
+ `freeze_test_selection.py` pins annotator test FPs from the registry — no
+ embeddings needed — and its clustering fill simply cannot see the surplus, so a
+ deferred sequence waits for future slots instead of competing by embedding
+ distance. Weaker than §5's letter, consistent with both designs' spirit: exact
+ identity is never traded back for estimated identity.
diff --git a/dvc.lock b/dvc.lock
index 77a0ff1..e84130a 100644
--- a/dvc.lock
+++ b/dvc.lock
@@ -592,18 +592,18 @@ stages:
build_fp_yolo_dataset:
cmd: uv run python scripts/build_fp_yolo_dataset.py --registry data/raw/fp/registry.json
--data-dir data/raw/fp/data --wf-dataset data/processed/wildfire_yolo --embeddings-dir
- data/interim/fp_sequence_embeddings --output data/processed/fp_yolo --random-seed
- 0
+ data/interim/fp_sequence_embeddings --recurring-objects data/raw/pyro-annotator/recurring_objects.json
+ --output data/processed/fp_yolo --random-seed 0
deps:
- path: data/interim/fp_sequence_embeddings/train
hash: md5
- md5: 51edec8cc1a3f1c8b93e7af595f80717.dir
- size: 20216589
+ md5: d7c1ce9531de9fc9d89bf6f4305ba12d.dir
+ size: 20043098
nfiles: 2
- path: data/interim/fp_sequence_embeddings/val
hash: md5
- md5: aac0d5327b7b6ea3e05a66dccf154749.dir
- size: 2491101
+ md5: 35a4487a1ba6a41ca2b529c6a699e6c3.dir
+ size: 2463314
nfiles: 2
- path: data/processed/wildfire_yolo
hash: md5
@@ -619,19 +619,23 @@ stages:
hash: md5
md5: 7983fceca0836f734bdf7e6bf032063e
size: 1318880
+ - path: data/raw/pyro-annotator/recurring_objects.json
+ hash: md5
+ md5: ecc9988f92c56db6f9516052b8aae959
+ size: 50832
- path: scripts/build_fp_yolo_dataset.py
hash: md5
- md5: ca087d24c82e3a93ff766c015e852a37
- size: 10573
+ md5: 7017e8e7f6d9d2d9e1ca981b63f25e7c
+ size: 13137
- path: src/pyro_dataset/fp/selection.py
hash: md5
- md5: e9d2065653a9aafdc035c700fa2d678f
- size: 7999
+ md5: e8b6b6673a571baeba4f94e18faeb98a
+ size: 10121
outs:
- path: data/processed/fp_yolo
hash: md5
- md5: cf82b8283d478d11ae8458335327f778.dir
- size: 332638599
+ md5: 218aa610b9887a65ec9b27009d6a6f6f.dir
+ size: 332994040
nfiles: 6002
merge_yolo_dataset:
cmd: uv run python scripts/merge_yolo_dataset.py --wf-dataset data/processed/wildfire_yolo
@@ -640,8 +644,8 @@ stages:
deps:
- path: data/processed/fp_yolo
hash: md5
- md5: cf82b8283d478d11ae8458335327f778.dir
- size: 332638599
+ md5: 218aa610b9887a65ec9b27009d6a6f6f.dir
+ size: 332994040
nfiles: 6002
- path: data/processed/wildfire_yolo
hash: md5
@@ -660,8 +664,8 @@ stages:
nfiles: 5281
- path: data/processed/yolo_train_val
hash: md5
- md5: 038270d9336432fa9001cf92069731b2.dir
- size: 2027739474
+ md5: a43f8eae91dd781aa91984d92bbe65d2.dir
+ size: 2028094915
nfiles: 33635
visualize_yolo_train_val:
cmd: uv run python scripts/visualize_dataset.py --dir-dataset data/processed/yolo_train_val
@@ -669,8 +673,8 @@ stages:
deps:
- path: data/processed/yolo_train_val
hash: md5
- md5: 038270d9336432fa9001cf92069731b2.dir
- size: 2027739474
+ md5: a43f8eae91dd781aa91984d92bbe65d2.dir
+ size: 2028094915
nfiles: 33635
- path: scripts/visualize_dataset.py
hash: md5
@@ -679,8 +683,8 @@ stages:
outs:
- path: data/reporting/viz/yolo_train_val
hash: md5
- md5: 7b3748c47c77eb11844bf0628927c3b4.dir
- size: 3658088575
+ md5: 922d0a257ec915d91bb299f11df8ae27.dir
+ size: 3658189215
nfiles: 16817
visualize_yolo_test:
cmd: uv run python scripts/visualize_dataset.py --dir-dataset data/processed/yolo_test
@@ -710,13 +714,13 @@ stages:
deps:
- path: data/interim/fp_sequence_embeddings/train
hash: md5
- md5: 51edec8cc1a3f1c8b93e7af595f80717.dir
- size: 20216589
+ md5: d7c1ce9531de9fc9d89bf6f4305ba12d.dir
+ size: 20043098
nfiles: 2
- path: data/interim/fp_sequence_embeddings/val
hash: md5
- md5: aac0d5327b7b6ea3e05a66dccf154749.dir
- size: 2491101
+ md5: 35a4487a1ba6a41ca2b529c6a699e6c3.dir
+ size: 2463314
nfiles: 2
- path: data/raw/fp/data
hash: md5
@@ -742,16 +746,16 @@ stages:
size: 348611
- path: scripts/build_sequential_dataset.py
hash: md5
- md5: 7cb5dd1ef282a34d5cd0436a020f664f
- size: 12785
+ md5: a279800d6a6e0a1c507b77256174d805
+ size: 11749
- path: src/pyro_dataset/fp/lockfile.py
hash: md5
md5: cfd051c856b57783e3201555d1b4b8c6
size: 2623
- path: src/pyro_dataset/fp/selection.py
hash: md5
- md5: e9d2065653a9aafdc035c700fa2d678f
- size: 7999
+ md5: e8b6b6673a571baeba4f94e18faeb98a
+ size: 10121
outs:
- path: data/processed/sequential_test
hash: md5
@@ -760,9 +764,9 @@ stages:
nfiles: 15320
- path: data/processed/sequential_train_val
hash: md5
- md5: 9faac139ef157197191b84bc24bd1598.dir
- size: 8555476683
- nfiles: 144452
+ md5: ef44e3f9519456b587f424e27b5ad8a2.dir
+ size: 8556786817
+ nfiles: 144472
build_toy_dataset:
cmd: uv run python scripts/build_toy_dataset.py --yolo-src data/processed/yolo_train_val
--seq-src data/processed/sequential_train_val --yolo-output data/processed/yolo_toy
@@ -770,13 +774,13 @@ stages:
deps:
- path: data/processed/sequential_train_val
hash: md5
- md5: 9faac139ef157197191b84bc24bd1598.dir
- size: 8555476683
- nfiles: 144452
+ md5: ef44e3f9519456b587f424e27b5ad8a2.dir
+ size: 8556786817
+ nfiles: 144472
- path: data/processed/yolo_train_val
hash: md5
- md5: 038270d9336432fa9001cf92069731b2.dir
- size: 2027739474
+ md5: a43f8eae91dd781aa91984d92bbe65d2.dir
+ size: 2028094915
nfiles: 33635
- path: scripts/build_toy_dataset.py
hash: md5
@@ -790,8 +794,8 @@ stages:
nfiles: 7728
- path: data/processed/yolo_toy
hash: md5
- md5: 9bab5ff7b4ca24749be1120daa85f88a.dir
- size: 101768902
+ md5: 257cdce256f23272ee72010f90fca5cd.dir
+ size: 101552974
nfiles: 1683
test_data_leakage:
cmd: uv run pytest tests/test_data_leakage.py -v --junit-xml=data/reporting/leakage_results.xml
@@ -803,9 +807,9 @@ stages:
nfiles: 15320
- path: data/processed/sequential_train_val
hash: md5
- md5: 9faac139ef157197191b84bc24bd1598.dir
- size: 8555476683
- nfiles: 144452
+ md5: ef44e3f9519456b587f424e27b5ad8a2.dir
+ size: 8556786817
+ nfiles: 144472
- path: data/processed/wildfire_yolo
hash: md5
md5: 7e25bea7a3afa9a9bf704b8e42c56a97.dir
@@ -818,8 +822,8 @@ stages:
nfiles: 5281
- path: data/processed/yolo_train_val
hash: md5
- md5: 038270d9336432fa9001cf92069731b2.dir
- size: 2027739474
+ md5: a43f8eae91dd781aa91984d92bbe65d2.dir
+ size: 2028094915
nfiles: 33635
- path: data/raw/fp/registry.json
hash: md5
@@ -857,12 +861,12 @@ stages:
size: 1318880
- path: scripts/embed_fp_for_selection.py
hash: md5
- md5: 27c76f4126615f2be3fb4b798e8c55ef
- size: 8024
+ md5: ba8c3fb00f8f15b129aa481889d816b8
+ size: 8525
- path: src/pyro_dataset/fp/selection.py
hash: md5
- md5: e9d2065653a9aafdc035c700fa2d678f
- size: 7999
+ md5: e8b6b6673a571baeba4f94e18faeb98a
+ size: 10121
outs:
- path: data/interim/fp_sequence_embeddings/test
hash: md5
@@ -871,13 +875,13 @@ stages:
nfiles: 2
- path: data/interim/fp_sequence_embeddings/train
hash: md5
- md5: 51edec8cc1a3f1c8b93e7af595f80717.dir
- size: 20216589
+ md5: d7c1ce9531de9fc9d89bf6f4305ba12d.dir
+ size: 20043098
nfiles: 2
- path: data/interim/fp_sequence_embeddings/val
hash: md5
- md5: aac0d5327b7b6ea3e05a66dccf154749.dir
- size: 2491101
+ md5: 35a4487a1ba6a41ca2b529c6a699e6c3.dir
+ size: 2463314
nfiles: 2
materialise_annotator_sequences:
cmd: uv run python scripts/materialise_annotator_sequences.py --export-dir data/raw/pyro-annotator/export
diff --git a/dvc.yaml b/dvc.yaml
index 5c88903..260d466 100644
--- a/dvc.yaml
+++ b/dvc.yaml
@@ -39,6 +39,7 @@ stages:
--data-dir data/raw/fp/data
--wf-dataset data/processed/wildfire_yolo
--embeddings-dir data/interim/fp_sequence_embeddings
+ --recurring-objects data/raw/pyro-annotator/recurring_objects.json
--output data/processed/fp_yolo
--random-seed 0
deps:
@@ -46,6 +47,7 @@ stages:
- src/pyro_dataset/fp/selection.py
- data/raw/fp/registry.json
- data/raw/fp/data
+ - data/raw/pyro-annotator/recurring_objects.json
- data/processed/wildfire_yolo
- data/interim/fp_sequence_embeddings/val
- data/interim/fp_sequence_embeddings/train
diff --git a/scripts/build_fp_yolo_dataset.py b/scripts/build_fp_yolo_dataset.py
index 8977404..04e7560 100644
--- a/scripts/build_fp_yolo_dataset.py
+++ b/scripts/build_fp_yolo_dataset.py
@@ -10,7 +10,10 @@
test : FP = 50% of total → n_fp = n_wf
Sampling strategy:
- • train, val — TWO-STAGE clustering (selects diverse sequences):
+ • train, val — annotator-sourced sequences first: one image per recurring
+ object (identity from data/raw/pyro-annotator/recurring_objects.json,
+ deterministic frame pick), then TWO-STAGE clustering fills the rest
+ (selects diverse sequences):
1. Bbox-overlap grouping per camera (NMS top-1 main bbox; intra-camera
union-find with IoU > 0.7) → "atoms" (each ≡ same recurring artefact
on the same camera).
@@ -58,9 +61,16 @@
import logging
import random
import shutil
+from collections import defaultdict
from pathlib import Path
-from pyro_dataset.fp.selection import load_embeddings, two_stage_select
+from pyro_dataset.fp.selection import (
+ folder_to_recurring_object,
+ load_embeddings,
+ partition_pinned,
+ remaining_quota,
+ two_stage_select,
+)
SPLITS = ["train", "val", "test"]
FP_RATIO = {"train": 0.1, "val": 0.1, "test": 0.5}
@@ -84,6 +94,12 @@ def make_cli_parser() -> argparse.ArgumentParser:
default=Path("data/interim/fp_sequence_embeddings"),
help="Per-split DINOv2 embeddings root (read for train/val two-stage selection).",
)
+ parser.add_argument(
+ "--recurring-objects",
+ type=Path,
+ default=Path("data/raw/pyro-annotator/recurring_objects.json"),
+ help="Recurring-object ledger; identity for annotator-sourced (pinned) sequences.",
+ )
parser.add_argument("--output", type=Path, default=Path("data/processed/fp_yolo"))
parser.add_argument("--random-seed", type=int, default=0)
parser.add_argument(
@@ -184,6 +200,49 @@ def round_robin_sample(
return selected
+def best_pinned_image(seq_path: Path) -> Path | None:
+ """Deterministic representative frame for a pinned sequence.
+
+ Annotator labels carry no score, so every frame ties at 0.0 and the sort
+ falls through to the filename. The rule stays correct for scored labels.
+ """
+ candidates = scored_images(seq_path)
+ if not candidates:
+ return None
+ candidates.sort(key=lambda c: (-c[0], c[1].name))
+ return candidates[0][1]
+
+
+def select_pinned_images(
+ pinned_seqs: list[dict], ledger: dict, data_dir: Path
+) -> list[Path]:
+ """One background image per recurring object, ledger-identified.
+
+ Several sequences of one artefact are distinct temporal negatives for the
+ sequential build but near-duplicate backgrounds here, so the object — not
+ the sequence — is the unit. Representative: lexicographically first
+ folder that yields a labeled frame.
+ """
+ ro_by_folder = folder_to_recurring_object(
+ ledger, {s["folder"] for s in pinned_seqs}
+ )
+ folders_by_ro: dict[str, list[str]] = defaultdict(list)
+ for s in pinned_seqs:
+ folders_by_ro[ro_by_folder[s["folder"]]].append(s["folder"])
+ images: list[Path] = []
+ for ro_id in sorted(folders_by_ro):
+ image = None
+ for folder in sorted(folders_by_ro[ro_id]):
+ image = best_pinned_image(data_dir / folder)
+ if image is not None:
+ break
+ if image is None:
+ logging.warning(f"{ro_id}: no labeled frame in any pinned folder")
+ continue
+ images.append(image)
+ return images
+
+
# Two-stage helpers live in src/pyro_dataset/fp/selection.py and are imported above.
@@ -206,6 +265,8 @@ def round_robin_sample(
sequences = load_registry(registry_path)
logging.info(f"Loaded {len(sequences)} FP sequences from registry")
+ ledger = json.loads(args["recurring_objects"].read_text())
+
wf_counts = count_wf_images(wf_dataset)
quotas = {
split: int(wf_counts[split] * FP_RATIO[split] / (1 - FP_RATIO[split]))
@@ -246,24 +307,28 @@ def round_robin_sample(
for split in SPLITS:
quota = quotas[split]
if split in TWO_STAGE_SPLITS:
+ # Annotator-sourced sequences are pinned: one image per recurring
+ # object, chosen deterministically, ahead of the clustering.
+ pinned_seqs, _ = partition_pinned(by_split_seqs[split])
+ pinned = select_pinned_images(pinned_seqs, ledger, data_dir)
emb, items = load_embeddings(embeddings_dir, split)
selected_idx = two_stage_select(
items=items,
embeddings=emb,
data_dir=data_dir,
- quota=quota,
+ quota=remaining_quota(quota, pinned),
nms_iou=nms_iou,
match_iou=match_iou,
seed=seed,
)
- selected = [
+ selected = pinned + [
data_dir
/ items[i]["sequence_folder"]
/ "images"
/ items[i]["image_name"]
for i in selected_idx
]
- strategies[split] = "two_stage"
+ strategies[split] = f"two_stage+{len(pinned)}_pinned"
else:
selected = round_robin_sample(by_split_candidates[split], quota, rng)
strategies[split] = "round_robin"
diff --git a/scripts/build_sequential_dataset.py b/scripts/build_sequential_dataset.py
index 7cfb9e9..ed9f653 100644
--- a/scripts/build_sequential_dataset.py
+++ b/scripts/build_sequential_dataset.py
@@ -63,7 +63,12 @@
from pathlib import Path
from pyro_dataset.fp.lockfile import load_lockfile, validate_lockfile
-from pyro_dataset.fp.selection import load_embeddings, two_stage_select
+from pyro_dataset.fp.selection import (
+ load_embeddings,
+ partition_pinned,
+ remaining_quota,
+ two_stage_select,
+)
SPLITS = ["train", "val", "test"]
@@ -127,28 +132,6 @@ def copy_sequence(src: Path, dst: Path, dry_run: bool) -> None:
shutil.copytree(src_sub, dst_sub, dirs_exist_ok=True)
-ANNOTATOR_SOURCE = "pyro-annotator"
-
-
-def partition_pinned(sequences: list[dict]) -> tuple[list[dict], list[dict]]:
- """Split registry entries into pinned (annotator-sourced) and the rest.
-
- Adding a sequence to data/raw/fp does not put it in the built dataset:
- two-stage selection picks `quota` representatives from the whole pool and
- may not choose it. Annotator false positives were selected deliberately
- upstream — one per recurring object, hard negatives first — so they are
- included before clustering fills what remains.
- """
- pinned = [s for s in sequences if s.get("source") == ANNOTATOR_SOURCE]
- rest = [s for s in sequences if s.get("source") != ANNOTATOR_SOURCE]
- return pinned, rest
-
-
-def remaining_quota(quota: int, pinned: list[dict]) -> int:
- """Slots left for clustering once the pinned sequences take theirs."""
- return max(quota - len(pinned), 0)
-
-
def frozen_test_selection(
lockfile_path: Path, quota: int, registered: set[str], data_dir: Path
) -> list[Path]:
@@ -252,22 +235,19 @@ def frozen_test_selection(
f"pinned FP sequence folder not found: {fp_data_dir / seq['folder']}"
)
pinned_seqs.remove(seq)
- pinned_folders = {s["folder"] for s in pinned_seqs}
pinned_paths = [fp_data_dir / s["folder"] for s in pinned_seqs]
quota = remaining_quota(quota, pinned_seqs)
- # Load DINOv2 embeddings + per-sequence metadata for this split.
+ # Load DINOv2 embeddings + per-sequence metadata for this split. The
+ # annotator cohort is absent by construction — embed_fp_for_selection
+ # skips it — so only folders missing from disk are dropped here.
emb, items = load_embeddings(embeddings_dir, split)
- # Filter to items whose folder still exists on disk, dropping pinned
- # ones so they cannot be selected twice; preserve alignment.
keep_mask = []
items_kept: list[dict] = []
for it in items:
- ok = (fp_data_dir / it["sequence_folder"]).is_dir() and it[
- "sequence_folder"
- ] not in pinned_folders
+ ok = (fp_data_dir / it["sequence_folder"]).is_dir()
keep_mask.append(ok)
- if not (fp_data_dir / it["sequence_folder"]).is_dir():
+ if not ok:
fp_missing_total += 1
if ok:
items_kept.append(it)
diff --git a/scripts/embed_fp_for_selection.py b/scripts/embed_fp_for_selection.py
index ed6d11e..ef744e9 100644
--- a/scripts/embed_fp_for_selection.py
+++ b/scripts/embed_fp_for_selection.py
@@ -1,5 +1,9 @@
"""Compute one DINOv2-base embedding per FP sequence using the highest-scoring
labeled frame. Used by the FP YOLO dataset build pipeline (val + train).
+Annotator-sourced sequences (source: pyro-annotator) are skipped: their
+identity is recorded exactly in the recurring-object ledger, so the builders
+pin them without embeddings (see
+docs/specs/2026-08-14-recurring-object-fp-identity-design.md).
For each sequence in the chosen split(s):
1. Pick the labeled frame with the highest detection score (6th column of
@@ -28,6 +32,8 @@
from tqdm import tqdm
from transformers import AutoImageProcessor, AutoModel
+from pyro_dataset.fp.selection import partition_pinned
+
HF_REPO = "facebook/dinov2-base"
PADDING = 0.2
OUT_SIZE = 224
@@ -209,6 +215,9 @@ def main() -> None:
for split in args.splits:
seqs = by_split.get(split, [])
+ # Annotator-sourced sequences are never embedded: their identity comes
+ # from the recurring-object ledger, and the builders pin them outright.
+ _, seqs = partition_pinned(seqs)
if not seqs:
print(f"[{split}] no sequences; skipping")
continue
diff --git a/src/pyro_dataset/fp/selection.py b/src/pyro_dataset/fp/selection.py
index a005bbe..86e6953 100644
--- a/src/pyro_dataset/fp/selection.py
+++ b/src/pyro_dataset/fp/selection.py
@@ -20,11 +20,61 @@
import json
from collections import defaultdict
+from collections.abc import Sequence
from pathlib import Path
import numpy as np
from sklearn.cluster import KMeans
+ANNOTATOR_SOURCE = "pyro-annotator"
+
+
+def partition_pinned(sequences: list[dict]) -> tuple[list[dict], list[dict]]:
+ """Split registry entries into pinned (annotator-sourced) and the rest.
+
+ Adding a sequence to data/raw/fp does not put it in a built dataset:
+ two-stage selection picks `quota` representatives from the whole pool and
+ may not choose it. Annotator false positives were selected deliberately
+ upstream — one per recurring object, hard negatives first — so they are
+ included before clustering fills what remains, and they are never
+ embedded: their identity comes from the recurring-object ledger.
+ """
+ pinned = [s for s in sequences if s.get("source") == ANNOTATOR_SOURCE]
+ rest = [s for s in sequences if s.get("source") != ANNOTATOR_SOURCE]
+ return pinned, rest
+
+
+def remaining_quota(quota: int, pinned: Sequence) -> int:
+ """Slots left for clustering once the pinned sequences take theirs."""
+ return max(quota - len(pinned), 0)
+
+
+def folder_to_recurring_object(ledger: dict, folders: set[str]) -> dict[str, str]:
+ """Invert the ledger's ingested_folders, restricted to `folders`.
+
+ The ledger may list folders that were planned but never registered
+ (record_ingested stamps at planning time), so only requested folders are
+ mapped. A requested folder missing from the ledger, or claimed by two
+ objects, is a hard error: falling back silently would re-introduce the
+ estimated identity this mapping replaces.
+ """
+ mapping: dict[str, str] = {}
+ for ro_id, entry in ledger.items():
+ for folder in entry.get("ingested_folders", []):
+ if folder not in folders:
+ continue
+ if folder in mapping:
+ raise ValueError(
+ f"{folder}: listed under both {mapping[folder]} and {ro_id}"
+ )
+ mapping[folder] = ro_id
+ missing = folders - mapping.keys()
+ if missing:
+ raise ValueError(
+ f"no recurring object recorded for: {', '.join(sorted(missing))}"
+ )
+ return mapping
+
def iou_xyxyn(a, b) -> float:
ax1, ay1, ax2, ay2 = a[:4]
diff --git a/tests/test_build_fp_pinning.py b/tests/test_build_fp_pinning.py
new file mode 100644
index 0000000..1378228
--- /dev/null
+++ b/tests/test_build_fp_pinning.py
@@ -0,0 +1,72 @@
+"""The YOLO FP build pins one background image per recurring object.
+
+Annotator labels are score-less (five fields, class 99), so every frame ties:
+the representative must be chosen by a deterministic rule, not directory
+traversal order. Within an object: lexicographically first folder; within a
+folder: highest label score, ties broken by filename.
+"""
+
+import importlib.util
+from pathlib import Path
+
+SCRIPT = Path("scripts/build_fp_yolo_dataset.py")
+
+spec = importlib.util.spec_from_file_location("build_fp_yolo_dataset", SCRIPT)
+module = importlib.util.module_from_spec(spec)
+spec.loader.exec_module(module)
+
+BOX = "99 0.5 0.5 0.1 0.1\n"
+SCORED = "0 0.5 0.5 0.1 0.1 0.42\n"
+
+
+def make_seq(tmp_path, folder, labels):
+ seq = tmp_path / folder
+ (seq / "images").mkdir(parents=True)
+ (seq / "labels").mkdir()
+ for stem, text in labels.items():
+ (seq / "images" / f"{stem}.jpg").write_bytes(b"jpg")
+ (seq / "labels" / f"{stem}.txt").write_text(text)
+ return seq
+
+
+def test_scoreless_labels_pick_the_first_frame_by_name(tmp_path):
+ make_seq(tmp_path, "seq", {"b": BOX, "a": BOX, "c": BOX})
+ assert module.best_pinned_image(tmp_path / "seq").name == "a.jpg"
+
+
+def test_a_scored_label_outranks_name_order(tmp_path):
+ make_seq(tmp_path, "seq", {"a": BOX, "z": SCORED})
+ assert module.best_pinned_image(tmp_path / "seq").name == "z.jpg"
+
+
+def test_sequence_without_labeled_frames_yields_none(tmp_path):
+ seq = tmp_path / "seq"
+ (seq / "images").mkdir(parents=True)
+ (seq / "labels").mkdir()
+ assert module.best_pinned_image(seq) is None
+
+
+def test_one_image_per_recurring_object(tmp_path):
+ make_seq(tmp_path, "cam_2", {"x": BOX})
+ make_seq(tmp_path, "cam_1", {"y": BOX})
+ make_seq(tmp_path, "other_1", {"z": BOX})
+ ledger = {
+ "ro_00001": {"ingested_folders": ["cam_1", "cam_2"]},
+ "ro_00002": {"ingested_folders": ["other_1"]},
+ }
+ pinned = [{"folder": "cam_1"}, {"folder": "cam_2"}, {"folder": "other_1"}]
+ images = module.select_pinned_images(pinned, ledger, tmp_path)
+ # ro_00001 is represented by its lexicographically first folder, cam_1
+ assert [i.name for i in images] == ["y.jpg", "z.jpg"]
+
+
+def test_object_falls_to_its_next_folder_when_the_first_has_no_labels(tmp_path):
+ empty = tmp_path / "cam_1"
+ (empty / "images").mkdir(parents=True)
+ (empty / "labels").mkdir()
+ make_seq(tmp_path, "cam_2", {"x": BOX})
+ ledger = {"ro_00001": {"ingested_folders": ["cam_1", "cam_2"]}}
+ images = module.select_pinned_images(
+ [{"folder": "cam_1"}, {"folder": "cam_2"}], ledger, tmp_path
+ )
+ assert [i.name for i in images] == ["x.jpg"]
diff --git a/tests/test_build_sequential_pinning.py b/tests/test_build_sequential_pinning.py
index fa323eb..7b64c09 100644
--- a/tests/test_build_sequential_pinning.py
+++ b/tests/test_build_sequential_pinning.py
@@ -1,8 +1,12 @@
-"""The sequential build must include annotator-sourced FP sequences.
+"""The sequential build copies the frozen test negatives verbatim.
-Adding a sequence to data/raw/fp does not put it in the built dataset: the
-build picks `quota` representatives by clustering the whole pool and may
-simply not choose it. Sequences the importer selected deliberately are pinned.
+The lockfile IS the test FP selection: the build performs no selection for
+test and errors on any mismatch, so the test set only ever grows
+(docs/specs/2026-08-14-annotator-test-growth-design.md §3).
+
+The pinning helpers this file used to cover (`partition_pinned`,
+`remaining_quota`) moved to `pyro_dataset.fp.selection` and are tested in
+tests/test_fp_pinning.py.
"""
import importlib.util
@@ -18,40 +22,6 @@
spec.loader.exec_module(module)
-def test_partition_pinned_splits_on_source():
- pinned, rest = module.partition_pinned(
- [
- {"folder": "a", "source": "pyro-annotator"},
- {"folder": "b"},
- {"folder": "c", "source": "platform"},
- ]
- )
- assert [s["folder"] for s in pinned] == ["a"]
- assert [s["folder"] for s in rest] == ["b", "c"]
-
-
-def test_partition_pinned_handles_an_empty_pool():
- assert module.partition_pinned([]) == ([], [])
-
-
-def test_pinned_sequences_consume_quota_before_clustering():
- pinned, _ = module.partition_pinned(
- [{"folder": f"a{i}", "source": "pyro-annotator"} for i in range(4)]
- )
- assert module.remaining_quota(quota=10, pinned=pinned) == 6
-
-
-def test_remaining_quota_never_goes_negative():
- pinned, _ = module.partition_pinned(
- [{"folder": f"a{i}", "source": "pyro-annotator"} for i in range(12)]
- )
- assert module.remaining_quota(quota=10, pinned=pinned) == 0
-
-
-def test_remaining_quota_is_untouched_without_pinned_sequences():
- assert module.remaining_quota(quota=10, pinned=[]) == 10
-
-
def make_lock(tmp_path, folders):
path = tmp_path / "lock.json"
path.write_text(json.dumps({"folders": folders}))
diff --git a/tests/test_fp_pinning.py b/tests/test_fp_pinning.py
new file mode 100644
index 0000000..9895c15
--- /dev/null
+++ b/tests/test_fp_pinning.py
@@ -0,0 +1,82 @@
+"""Both dataset builds must include annotator-sourced FP sequences.
+
+Adding a sequence to data/raw/fp does not put it in a built dataset: the
+builds pick `quota` representatives by clustering the whole pool and may
+simply not choose it. Sequences the importer selected deliberately are
+pinned, and the helpers implementing that live in pyro_dataset.fp.selection
+so the YOLO build, the sequential build and the embedding script share them.
+Identity for the pinned cohort comes from the recurring-object ledger.
+"""
+
+import pytest
+
+from pyro_dataset.fp.selection import (
+ folder_to_recurring_object,
+ partition_pinned,
+ remaining_quota,
+)
+
+
+def test_partition_pinned_splits_on_source():
+ pinned, rest = partition_pinned(
+ [
+ {"folder": "a", "source": "pyro-annotator"},
+ {"folder": "b"},
+ {"folder": "c", "source": "platform"},
+ ]
+ )
+ assert [s["folder"] for s in pinned] == ["a"]
+ assert [s["folder"] for s in rest] == ["b", "c"]
+
+
+def test_partition_pinned_handles_an_empty_pool():
+ assert partition_pinned([]) == ([], [])
+
+
+def test_pinned_sequences_consume_quota_before_clustering():
+ pinned, _ = partition_pinned(
+ [{"folder": f"a{i}", "source": "pyro-annotator"} for i in range(4)]
+ )
+ assert remaining_quota(quota=10, pinned=pinned) == 6
+
+
+def test_remaining_quota_never_goes_negative():
+ pinned, _ = partition_pinned(
+ [{"folder": f"a{i}", "source": "pyro-annotator"} for i in range(12)]
+ )
+ assert remaining_quota(quota=10, pinned=pinned) == 0
+
+
+def test_remaining_quota_is_untouched_without_pinned_sequences():
+ assert remaining_quota(quota=10, pinned=[]) == 10
+
+
+def test_folder_to_recurring_object_inverts_ingested_folders():
+ ledger = {
+ "ro_00001": {"ingested_folders": ["cam-a_1"]},
+ "ro_00002": {"ingested_folders": ["cam-b_1", "cam-b_2"]},
+ }
+ assert folder_to_recurring_object(ledger, {"cam-a_1", "cam-b_2"}) == {
+ "cam-a_1": "ro_00001",
+ "cam-b_2": "ro_00002",
+ }
+
+
+def test_folder_missing_from_ledger_is_a_hard_error():
+ ledger = {"ro_00001": {"ingested_folders": ["cam-a_1"]}}
+ with pytest.raises(ValueError, match="cam-x_9"):
+ folder_to_recurring_object(ledger, {"cam-x_9"})
+
+
+def test_folder_under_two_objects_is_a_hard_error():
+ ledger = {
+ "ro_00001": {"ingested_folders": ["dup_1"]},
+ "ro_00002": {"ingested_folders": ["dup_1"]},
+ }
+ with pytest.raises(ValueError, match="dup_1"):
+ folder_to_recurring_object(ledger, {"dup_1"})
+
+
+def test_unrequested_ledger_folders_are_ignored():
+ ledger = {"ro_00001": {"ingested_folders": ["staged-never-registered", "cam-a_1"]}}
+ assert folder_to_recurring_object(ledger, {"cam-a_1"}) == {"cam-a_1": "ro_00001"}