Share render-state publication across active visual consumers

agent_context/topics/simulation-system/rendering.md

@@ -5,12 +5,30 @@ and readiness reporting. Return to the [simulation overview](simulation-system.m
 
 ## Rendering does not advance physics
 
-`NewtonPhysicsCfg.sync_to_renderer=None` preserves DexSim's consumer-aware
-per-step policy. Camera rendering and reset observations explicitly synchronize
-on demand; correctness must not depend on continuous render sync in headless
-training. `SimulationManager.sync_render_state()` and backend render hooks
-publish state without advancing physics time. Marker publication while paused
-uses `capture_visualization(force=True)`.
+`SimulationManager.render_frame()` owns a read-only consumption phase after
+physics, interval events or reset writes. An open native window, nonempty camera
+groups, due simulation-time recording and due Viser captures share one state
+publication in that phase. Empty camera groups alone are not consumers; an open
+window remains a consumer even with `NewtonPhysicsCfg.sync_to_renderer=False`.
+Viser decides whether a frame is due before invoking its publication callback.
+
+`update()` renders after each substep by default. Gym uses
+`update(render_final_step=False)` and enters `render_frame()` around observations
+after interval events, so the final camera, recording and Viser reads share the
+post-event state. Standalone callers combining those consumers can use the same
+sequence. Reset completes all writes before a new frame with forced Viser capture.
+An independent camera/capture call still publishes fresh state. No publication
+cache survives a frame, so paused edits and direct writes between frames do not
+need mutation counters. State edits inside a read-only frame require an explicit
+`sync_render_state()` before further reads; that API always publishes.
+
+The Newton backend step hook suppresses automatic DexSim publication during
+manager-owned updates to avoid publishing twice. The manager honors explicit
+`sync_to_renderer=True` even without consumers; `None`/`False` publish only for
+consumers. Raw DexSim `World.update()` retains its configured automatic policy.
+Render-thread recording only consumes already-published state and must never
+call the blocking physics-to-render bridge. Marker publication while paused uses
+`capture_visualization(force=True)` without advancing physics.
 
 Viser forces headless mode and is mutually exclusive with the native window.
 An empty server may start before assets are declared; it must not finalize

docs/source/api_reference/embodichain/embodichain.lab.sim.cfg.rst

@@ -104,3 +104,27 @@ Constructors and ``from_dict()`` accept only the current fields. Surface-element
 properties are grouped under ``attrs.surface_props``. Volume elasticity uses
 ``youngs`` and ``poissons``. Configuration dictionaries serialize this same
 schema; no legacy aliases or field migration are provided.
+
+World-level configuration module
+--------------------------------
+
+The same world-level configuration types and backend selectors are available
+from ``embodichain.lab.sim.cfg.simulation``. For manager-owned Newton steps,
+``sync_to_renderer=None`` or ``False`` publishes only for visual consumers;
+``True`` additionally requests publication on steps without consumers. An open
+window remains a consumer regardless of this automatic policy.
+
+.. currentmodule:: embodichain.lab.sim.cfg.simulation
+
+.. autosummary::
+
+   DLSSCfg
+   RenderCfg
+   GPUMemoryCfg
+   PhysicsBackendCfg
+   DefaultPhysicsCfg
+   NewtonCollisionPipelineCfg
+   NewtonPhysicsCfg
+   physics_cfg_for_backend
+   physics_backend_from_cfg
+   validate_physics_cfg

docs/source/api_reference/embodichain/embodichain.lab.sim.sim_manager.rst

@@ -29,6 +29,31 @@ not write drive targets. Set this field to ``None`` to opt out or select
 prevents automatic recreation. ``GizmoCfg(ik_start_enabled=True)`` activates
 native IK on the first update with an open window, as used by the robot tutorial.
 
+Physics and visual consumption
+------------------------------
+
+State-only headless stepping does not publish Newton state to the renderer.
+An open native window, camera groups, due recording and due Viser captures
+request publication when they consume state. ``render_frame()`` shares that
+publication within one read-only consumption phase. Gym arranges this phase
+after physics and interval events automatically. Standalone callers combining
+camera observations with recording or Viser can use:
+
+.. code-block:: python
+
+   sim.update(physics_dt, step=1, render_final_step=False)
+   # Complete direct state edits before entering the read-only frame.
+   with sim.render_frame():
+       sim.render_camera_group(camera_group_ids)
+       # Due recording and Viser captures run on context exit.
+
+The phase never advances physics and never caches publication across frames.
+Independent camera/capture calls request fresh publication. If state changes
+inside a frame, call ``sync_render_state()`` explicitly before further reads.
+Opening a window and rendering while paused also request publication. Native
+render-thread recording consumes already-published state without invoking the
+blocking physics-to-render bridge from that thread.
+
 .. rubric:: Classes
 
 .. autosummary::

embodichain/lab/gym/envs/base_env.py

@@ -602,7 +602,11 @@ def _advance_physics(self) -> None:
             sensor for sensor in self.sensors.values() if sensor.requires_substep_update
         ]
         if not sensors:
-            self.sim.update(self.physics_dt, self.cfg.sim_steps_per_control)
+            self.sim.update(
+                self.physics_dt,
+                self.cfg.sim_steps_per_control,
+                render_final_step=False,
+            )
             return
         for sensor in sensors:
             sensor.begin_control_step()
@@ -615,6 +619,7 @@ def sample_substep(dt: float) -> None:
             self.physics_dt,
             self.cfg.sim_steps_per_control,
             after_substep=sample_substep,
+            render_final_step=False,
         )
 
     def _update_sim_state(self, **kwargs):
@@ -930,11 +935,9 @@ def reset(
                 self._initialize_episode(reset_ids, **options)
             self._elapsed_steps[reset_ids] = 0
 
-            self.sim.sync_render_state()
-            self.sim.capture_visualization_safely(force=True)
-
-            with self._profiler.section("get_obs"):
-                obs = self.get_obs(**options)
+            with self.sim.render_frame(force_visualization=True):
+                with self._profiler.section("get_obs"):
+                    obs = self.get_obs(**options)
             with self._profiler.section("get_info"):
                 info = self.get_info(**options)
 
@@ -1009,8 +1012,9 @@ def step(
             with self._profiler.section("update_sim_state"):
                 self._update_sim_state(**kwargs)
 
-            with self._profiler.section("get_obs"):
-                obs = self.get_obs(**kwargs)
+            with self.sim.render_frame():
+                with self._profiler.section("get_obs"):
+                    obs = self.get_obs(**kwargs)
             with self._profiler.section("get_info"):
                 info = self.get_info(**kwargs)
             with self._profiler.section("reward"):

embodichain/lab/sim/cfg/simulation.py

@@ -32,6 +32,19 @@
 
 from embodichain.utils import configclass, logger
 
+__all__ = [
+    "DLSSCfg",
+    "RenderCfg",
+    "GPUMemoryCfg",
+    "PhysicsBackendCfg",
+    "DefaultPhysicsCfg",
+    "NewtonCollisionPipelineCfg",
+    "NewtonPhysicsCfg",
+    "physics_cfg_for_backend",
+    "physics_backend_from_cfg",
+    "validate_physics_cfg",
+]
+
 if TYPE_CHECKING:
     from dexsim.engine.newton_physics import NewtonCfg
     from dexsim.engine.newton_physics.solvers_cfg import NewtonSolverCfg
@@ -521,11 +534,14 @@ class NewtonPhysicsCfg(PhysicsBackendCfg):
     """
 
     sync_to_renderer: bool | None = None
-    """Automatic Newton-to-render sync policy for each physics update.
-
-    ``None`` preserves DexSim's consumer-aware default, ``True`` always syncs,
-    and ``False`` disables per-step sync. Camera rendering still performs an
-    explicit on-demand sync through :class:`SimulationManager`.
+    """Newton-to-render publication policy.
+
+    ``None`` and ``False`` let :class:`SimulationManager` publish on demand
+    for windows, cameras, recording and browser visualization. ``True`` also
+    requests publication on steps without a visual consumer. The manager batches
+    consumers within each render frame. Direct DexSim ``World.update`` callers
+    retain the underlying automatic policy: window-aware for ``None``, always
+    for ``True``, and disabled for ``False``.
     """
 
     debug_mode: bool = False

embodichain/lab/sim/physics/base.py

@@ -119,6 +119,15 @@ def sync_render_state(self, result: "dexsim.scene.Scene") -> None:
         """
         del result
 
+    def _step(self, physics_dt: float) -> None:
+        """Advance physics; the manager owns subsequent visual consumption."""
+        self._manager._world.update(physics_dt)
+
+    @property
+    def _requires_step_render_sync(self) -> bool:
+        """Whether the caller explicitly requested publication on every step."""
+        return False
+
     def prepare_for_teardown(self) -> None:
         """Release backend-owned views before Spawn releases their parents.
 

embodichain/lab/sim/physics/newton.py

@@ -111,6 +111,14 @@ def sync_render_state(self, result: "dexsim.scene.Scene") -> None:
             )
         backend.sync_to_dexsim(result.world)
 
+    def _step(self, physics_dt: float) -> None:
+        """Let the manager batch window and offscreen consumers after stepping."""
+        self._manager._world.update(physics_dt, sync_to_dexsim=False)
+
+    @property
+    def _requires_step_render_sync(self) -> bool:
+        return self._manager.sim_config.physics_cfg.sync_to_renderer is True
+
     def prepare_for_teardown(self) -> None:
         """Release Newton render views while Spawn still owns their parents."""
         if self._runtime_device is not None and self._runtime_device.startswith("cuda"):