Skip to content

Write generic keys instead of dive_-prefixed ones - #1866

Open
mattdawkins wants to merge 2 commits into
mainfrom
dev/coco-generic-keys
Open

Write generic keys instead of dive_-prefixed ones#1866
mattdawkins wants to merge 2 commits into
mainfrom
dev/coco-generic-keys

Conversation

@mattdawkins

Copy link
Copy Markdown
Member

Per-detection attributes, track attributes, notes and confidence pairs are generic annotation concepts — nothing about them is specific to DIVE — so export should not brand them. Export now writes attributes, track_attributes, notes and confidence_pairs.

Import already fell back to the unprefixed names for attributes and notes (kwcoco.py:345,349, coco.ts:452,462,466); this makes those the preferred spelling and keeps the dive_-prefixed ones as legacy aliases, so files DIVE wrote earlier keep importing. confidence_pairs gains the same treatment on both readers.

Kept prefixed, because they really are DIVE-specific:

  • info.dive_dataset_info — DIVE's dataset metadata block
  • info.dive_extensions — DIVE describing which optional keys it used (its contents are now the generic names)

This changes DIVE's output format. Anything downstream reading dive_detection_attributes, dive_track_attributes, dive_notes or dive_confidence_pairs off a DIVE export needs updating; the readers here are covered, third parties are not. info.dive_extensions still advertises which keys a given file uses.

Tests: export assertions moved to the generic keys; existing prefixed documents were left as-is so they now serve as legacy-import coverage, and both suites gained an explicit generic-vs-prefixed pair. docs/DataFormats.md updated, including the worked example and the legacy-alias notes.

Verified: 44 server tests pass locally, and a real export/import round trip writes ['attributes', 'confidence_pairs', 'notes', 'prob', 'track_attributes'] and reads every value back.

Not run: vitest, eslint and tsc — no node_modules in this checkout.

Stacked context: #1865 renames videos[].fps to annotation_fps and is still open. Both touch these files, so whichever lands second will need a trivial rebase.

Attributes, notes and confidence pairs are not DIVE concepts, so export
writes attributes, track_attributes, notes and confidence_pairs. Import
still reads the prefixed spellings, so older files keep working.
The repo's eslint config disallows for...of.
@BryonLewis

Copy link
Copy Markdown
Collaborator

I'm hesitant about dropping the dive_ prefix on these annotation keys, even though the concepts (attributes, notes, confidence pairs) aren't DIVE-only.

The concepts being generic doesn't make the wire encoding a shared convention. DIVE's shapes are specific: notes as a string array, confidence_pairs as an ordered sparse vector with explicit-zero vs absent semantics, attributes / track_attributes as free-form dicts with DIVE reserved keys. Writing those under bare names reads like "this is how everyone should spell this in COCO," when it's really still DIVE's profile. We already acknowledge that with info.dive_extensions and dive_dataset_info listing unprefixed keys inside a DIVE-owned extensions list is a bit inconsistent.

More practically, unprefixed names are collision-prone. attributes and notes are exactly the kind of keys another exporter (or a future KWCOCO/COCO convention) might already use with a different schema. Once we prefer the generic spelling on import, a foreign file that happens to carry attributes or notes gets silently interpreted as DIVE's. The prefix was cheap insurance against that; reclaiming a common name after we've published it as our write path is hard.

annotation_fps was a different case — VIAME and DIVE aligned on a shared name for a shared meaning. Here we're unilaterally claiming common slots without that coordination, and it's a breaking change for anything that already reads dive_* off our exports.

@PaulHax

PaulHax commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator

Perhaps we migrate notes to export to kwcoco's existing caption. kwcoco already defines ANNOTATION.caption as annotation-level text

Upstream the rest to https://gitlab.kitware.com/computer-vision/kwcoco

Upstream proposal:

  • attributes / track_attributes — free-form key/value dicts on
    annotations and tracks. kwcoco has no attributes concept at all today.
  • category_scores — a sparse [{category_id, category?, score}, ...] to
    replace the dense prob, using the id-plus-name-hint pattern kwcoco already
    uses for keypoints. Worth asking to add category_scores on both annotation
    and track. The track case answers
    kwcoco #7.

@BryonLewis

Copy link
Copy Markdown
Collaborator

I really want to hold off on complicating this structure with some back and forth with KWCOCO specification right now because DIVE is moving way to quick with functionaltiy and features. I think once there is a settling point we can discuss things that may need to be upstreamed to KWCOCO.

Currently our annotations for Notes are at a track level, not at an annotation level. KWCOCO captions appear to be at the individual annotation level. Within DIVE a note at annotation level would probably fall under detection attribute and not the claude created Note field which probably should have fell under a reserved Track Attribute (because we already had the infrastructure to connect metadata to tracks).

This also gets into an argument of the semantics of what a caption is vs a note and if they represent the same thing or possibily different things? KWCOCO references it as either "A video level text caption", or "An annotation-level text caption":

  • does a caption specifically reference visual qualities about the annotation: "It's a fish off axis with the camera 45 degrees"
  • can a note be something that is a reference for why the annotation was made: "system created annotation"

The back and forth of figuring these out is just going to take up time and I'd shelve it until there is a definitive examples of how these fields are being used (I don't mean one-offs a of a single person utilizing it).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants