Repository navigation
Add Viser frame controls for trajectory replay - #466
Merged
Merged
Conversation
Synchronize control-mode trajectory replay with a latest-only browser frame slider while keeping simulation writes on the replay thread.
yuecideng
added
enhancement
Greptile SummaryThe PR adds browser-based frame seeking and synchronization for trajectory replay while retaining replay-state mutations on the replay thread and Viser updates on its worker thread.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| embodichain/lab/scripts/run_env.py | Multiplexes terminal and browser replay commands, centralizes clamped seeking, and publishes slider visibility and progress. |
| embodichain/lab/visualization/runtime.py | Adds latest-only replay-state and command queues while preserving backend updates on the visualization worker. |
| embodichain/lab/visualization/backends/viser.py | Adds the Replay control slider, lifecycle handling, command dispatch, and suppression of server-originated callback feedback. |
| embodichain/lab/visualization/backends/base.py | Extends the backend contract with replay command-sink registration and replay-state publication. |
| tests/lab/scripts/test_run_env.py | Verifies that a browser seek is consumed while terminal replay is paused. |
| tests/visualization/test_runtime.py | Verifies latest-only replay transport and visualization-thread publication. |
| tests/visualization/test_viser_backend.py | Covers client seeks, suppression of server-originated updates, slider synchronization, and removal. |
| docs/source/guides/run_env.md | Documents browser-based control replay and its interaction with auto-play. |
| docs/source/overview/sim/viser_visualization.md | Documents the Viser Replay control panel and cross-thread command behavior. |
Sequence Diagram
sequenceDiagram
participant Browser
participant Viser as Viser worker
participant Queue as Latest-only queues
participant Replay as Replay thread
Browser->>Viser: Change Frame slider
Viser->>Queue: Enqueue newest requested step
Replay->>Queue: Drain browser seek
Queue-->>Replay: Requested step
Replay->>Replay: go_to_step(step)
Replay->>Queue: Publish replay state
Queue-->>Viser: Current step and range
Viser->>Browser: Synchronize Frame slider
Reviews (2): Last reviewed commit: "wip" | Re-trigger Greptile
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds trajectory replay controls to the Viser visualization flow, enabling browser-based frame seeking during replay and synchronizing state between terminal replay control and the Viser UI.
Changes:
- Introduces a replay “Frame” slider in
ViserBackendand an event sink for browser seek commands. - Extends
VisualizationRuntimeto publish replay slider state asynchronously and coalesce browser seek commands. - Updates
run-envreplay control loop to consume browser seeks (including while paused) and keep UI state synchronized; documents the feature.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/visualization/test_viser_backend.py | Adds unit tests for replay slider creation, seek emission, progress tracking, and hiding behavior. |
| tests/visualization/test_runtime.py | Adds tests validating replay control state publishing and coalescing of seek commands. |
| tests/lab/scripts/test_run_env.py | Adds a test ensuring browser seeks are consumed while terminal replay is paused. |
| embodichain/lab/visualization/runtime.py | Adds runtime queues / APIs for replay control state publishing and seek command draining. |
| embodichain/lab/visualization/backends/viser.py | Implements replay control folder/slider UI, event handling, and state updates in Viser. |
| embodichain/lab/visualization/backends/base.py | Adds backend hooks for replay control seeks and replay slider state publishing. |
| embodichain/lab/scripts/run_env.py | Integrates visualization runtime replay controls into terminal replay loop and updates headless messaging. |
| docs/source/overview/sim/viser_visualization.md | Documents the Replay control panel behavior and synchronization semantics. |
| docs/source/guides/run_env.md | Updates control-mode replay guide to mention terminal-or-Viser usage and behavior details. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+324
to
+335
| def read_key(timeout: float | None) -> str | None: | ||
| nonlocal terminal_active | ||
| if not terminal_active: | ||
| time.sleep(timeout or _REPLAY_CONTROL_POLL_INTERVAL) | ||
| return None | ||
| try: | ||
| return control_input.read_key(timeout=timeout) | ||
| except EOFError: | ||
| if visualization_runtime is None: | ||
| raise | ||
| terminal_active = False | ||
| return None |
Comment on lines
+1163
to
+1169
| elif event.category == "replay_control": | ||
| state = self._replay_control_state | ||
| sink = getattr(self, "_replay_control_command_sink", None) | ||
| if state is not None and state[2] and sink is not None: | ||
| target_step = max(0, min(int(event.value), state[1])) | ||
| self._replay_control_slider.value = target_step | ||
| sink(target_step) |
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.
Suppressed comments (1)
embodichain/lab/visualization/backends/viser.py:1531
publish_replay_control()applies_apply_gui_events()before syncingself._replay_control_slider.value. If a browser seek event is processed there,_apply_gui_events()sets the slider to the requestedtarget_step, but the laterself._replay_control_slider.value = stepcan immediately overwrite it with a stale server-sidestep(until the replay thread catches up). This can make the slider jump back while dragging.
One simple fix is to process queued GUI events after updating/removing/registering the replay controls so browser-originated updates win the last write for that tick.
elif self._replay_control_slider is None or previous_max_step != max_step:
self._remove_replay_control()
self._register_replay_control()
else:
self._replay_control_slider.value = step
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds an expanded Replay control panel to Viser when
run-envuses--replay_mode control --viser. Its integer frame slider seeks directly to recorded trajectory steps, follows terminal-driven playback, and pauses auto-play when the browser requests a frame.Browser callbacks feed a latest-only integer queue, while
ReplayWrapper.go_to_step()remains on the replay thread. Viser handle creation and updates remain on the visualization worker thread. The implementation intentionally avoids new protocol exports and external dependencies.Dependencies: None.
Issue reference: N/A.
Type of change
Screenshots
Not included. The Viser slider lifecycle and browser callback path are covered by backend tests.
Validation
black .black --check --diff --color ./pytest -q tests/visualization tests/lab/scripts/test_run_env.py -x— 60 passedpytest -q tests/gym/envs/test_replay.py::test_control_loop_uses_single_keys_and_auto_can_be_interrupted -m slow— 1 passedsphinx-build -b html docs/source <temp-dir>— succeeded with 668 pre-existing warningsChecklist
black .command to format the code base.