Ego4D-OSCA Dataset Overview

We introduce the Ego4D-OSCA dataset, which relies on a subset of the large-scale Ego4D dataset and aims to serve as a benchmark for evaluating methods for the newly defined task of vision-based Object State Change Anticipation (OSCA).

---

Annotation Files

Two .csv files are generated for the training and test subsets:

• Ego4D-OSCA_annotations_train
• Ego4D-OSCA_annotations_test

These files provide annotations required for the OSCA task.
Below, we summarize the structure of these CSV files and describe how each attribute connects to corresponding fields in the original Ego4D dataset.

Selected videos and clips are retrieved from the Ego4D LTA and OSCC benchmarks.
Reference: https://ego4d-data.org/docs/data/annotations-schemas/

---

Attribute Descriptions

• video_uid, clip_id, clip_uid
  Retrieved from the Ego4D main data file. These identifiers reference the source video and clip used in OSCA.

• clip_start, clip_end
  Define the temporal segment for which the near-future object state change (in the next temporal segment) is to be anticipated.
  (Ego4D main data file)

• interval_start, interval_end
  Define the temporal segment within the clip time period in which the object state change occurs.
  (Ego4D LTA data file)

• clip_parent_start, clip_parent_end
  Indicate the temporal segment in which the annotated action label (concatenation of structured_verb and structured_object) occurs.
  (Ego4D LTA data file)

• structured_verb
  Indicates the state-modifying motion motif (verb) that occurs in the current clip.
  Retrieved and processed based on the corresponding structured verb values in the Ego4D main data file.

• structured_object
  Indicates the object on which the state-modifying verb occurs in the current temporal segment.
  Retrieved and processed based on the corresponding structured object values in the Ego4D main data file.

• history_binary_osc
  Lists, in chronological order, the history of state changes that occurred up to the current clip.
  This representation is binary and does not include categories of state change.
  (Ego4D main data file)

• history_osc
  Lists, in chronological order, the specific categories of state changes that occurred up to the current clip.
  (Ego4D main data file)