Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions agent_context/MAP.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ topics:
- id: simulation-system
title: Simulation System
aliases: [simulation system, simulation manager, Newton physics backend, motion module, sim motion, camera attachment, 运动能力模块, 仿真系统, 仿真管理器]
keywords: [SimulationManager, SimulationManagerCfg, RenderCfg, DLSSCfg, DLSS, frame_time_delta_ms,
offscreen_dlss_enabled, explicit physics stepping, arena, DexSim, PhysicsBackendCfg,
keywords: [SimulationManager, SimulationManagerCfg, RenderCfg, DenoisingCfg, DenoisingMode, DLSSCfg,
NRDCfg, DLSS, NRD, frame_time_delta_ms, explicit physics stepping, arena, DexSim, PhysicsBackendCfg,
DefaultPhysicsCfg, NewtonPhysicsCfg, AutoSolverCfg, DexUniSolverCfg, DexUni, runtime control, deformable, particle set,
ArticulationJointKinematics, get_parent_joint_chain, asset_physics_mode, joint_drive_props, enable_gravity,
fixed-cadence playback, quaternion, xyzw]
Expand Down
29 changes: 17 additions & 12 deletions agent_context/topics/simulation-system/rendering.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# Rendering, attachment and startup diagnostics

Read this for physics/render synchronization, native-window/DLSS configuration
and readiness reporting. Return to the [simulation overview](simulation-system.md).
Read this for physics/render synchronization, native-window/offscreen image
processing and readiness reporting. Return to the
[simulation overview](simulation-system.md).

## Rendering does not advance physics

Expand Down Expand Up @@ -51,22 +52,26 @@ Completion is marked only after all attachments succeed, so partial failures
retry. The camera registry is the sole attachment-intent store. Detailed
sensor behavior belongs to [sensors](../sensor-system/sensor-system.md).

## Renderer and DLSS configuration
## Renderer, denoising and reconstruction configuration

`cfg/simulation.py:RenderCfg.apply_to_dexsim_config()` translates rendering and
`DLSSCfg` into WorldConfig after automatic renderer resolution. Forward the
DLSS master switch even when false, and retain config during headless startup
for offscreen cameras or a later window. Native window/offscreen behavior can
differ; settings not authored by EmbodiChain retain DexSim defaults.
`cfg/simulation.py:RenderCfg.apply_to_dexsim_config()` translates rendering,
`DenoisingCfg`, `DLSSCfg` and `NRDCfg` into WorldConfig after automatic renderer
resolution. `DenoisingCfg` independently selects the window and offscreen
pipelines from `off`, `optix`, `dlss` and `nrd`; `dlss` maps to DLSS Ray
Reconstruction and `nrd` maps to standalone NRD RELAX. Native SR/REBLUR
variants are not part of the EmbodiChain public contract. Retain the
complete configuration during headless startup for offscreen cameras or a later
window.

The actual camera/window owns output dimensions; compatibility target fields
must not resize it. Internal dimensions/upsample ratio affect FastRT/OfflineRT
windows; hybrid/offscreen sizes derive from output size and quality.
`frame_time_delta_ms` is render cadence, not physics/control cadence.
`gym/utils/gym_utils.py:config_to_cfg()` decodes nested DLSS mappings. Scalar
validation runs at construction and conversion after mutable edits. Configuration
tests do not prove GPU/NGX support, which initializes on a rendered frame.
Read exact renderer/quality defaults in the config source.
`gym/utils/gym_utils.py:config_to_cfg()` decodes all three nested mappings.
Scalar validation runs at construction and conversion after mutable edits.
Configuration tests do not prove GPU/NGX/NRD runtime support, which initializes
on a rendered frame. Read exact renderer and algorithm defaults in the config
source.

## Startup summaries

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ tests need non-symmetric rotations to expose component-order mistakes.
| Backend/device selection, source overlays, collision/mass policy | [Physics configuration](physics-configuration.md); `cfg/`, `spawn/descriptors.py` |
| Joint state order, drive lowering, mimic coupling, native adaptation | [Articulation and batch adapters](articulation-adapters.md); `objects/backends/`, `objects/articulation.py` |
| Volume/surface state, topology and kinematic nodes | [Deformables](deformables.md); `cfg/deformable.py`, `objects/deformable/` |
| Camera synchronization, native window, DLSS, startup readiness | [Rendering and diagnostics](rendering.md); manager, `cfg/simulation.py`, `_startup_summary.py` |
| Camera synchronization, native window/offscreen denoising, startup readiness | [Rendering and diagnostics](rendering.md); manager, `cfg/simulation.py`, `_startup_summary.py` |
| Robot definition, control parts, robot presets | [Robot system](../robot-system/robot-system.md) |
| Camera attachment resolution or contact sensors | [Sensor system](../sensor-system/sensor-system.md) |
| Browser export or native gizmos | [Visualization](../sim-visualization/sim-visualization.md), [native gizmos](../sim-visualization/native-gizmos.md) |
Expand Down
58 changes: 51 additions & 7 deletions docs/source/api_reference/embodichain/embodichain.lab.sim.cfg.rst
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,17 @@ UV seams and render detail.
.. autosummary::

AssetPhysicsMode
DenoisingMode
MeshCollisionApproximation

.. rubric:: Classes

.. autosummary::

RenderCfg
DenoisingCfg
DLSSCfg
NRDCfg
PhysicsBackendCfg
DefaultPhysicsCfg
NewtonPhysicsCfg
Expand Down Expand Up @@ -105,20 +109,60 @@ properties are grouped under ``attrs.surface_props``. Volume elasticity uses
``youngs`` and ``poissons``. Configuration dictionaries serialize this same
schema; no legacy aliases or field migration are provided.

World-level configuration module
--------------------------------
Rendering configuration module
------------------------------

Rendering types live in ``embodichain.lab.sim.cfg.rendering``. The public
``dlss`` path maps to DLSS Ray Reconstruction and ``nrd`` maps to standalone
NRD RELAX.

.. currentmodule:: embodichain.lab.sim.cfg.rendering

.. autosummary::

DenoisingMode
DenoisingCfg
DLSSCfg
NRDCfg
RenderCfg

Physics configuration module
----------------------------

Physics backend types live in ``embodichain.lab.sim.cfg.physics``. The legacy
``embodichain.lab.sim.cfg.simulation`` module remains a compatibility facade.
For manager-owned Newton steps, ``sync_to_renderer=None`` or ``False`` publishes
only for visual consumers; ``True`` additionally requests publication on steps
without consumers.

.. currentmodule:: embodichain.lab.sim.cfg.physics

.. autosummary::

GPUMemoryCfg
PhysicsBackendCfg
DefaultPhysicsCfg
NewtonCollisionPipelineCfg
NewtonPhysicsCfg
physics_cfg_for_backend
physics_backend_from_cfg
validate_physics_cfg

Compatibility facade
--------------------

The same world-level configuration types and backend selectors are available
from ``embodichain.lab.sim.cfg.simulation``. For manager-owned Newton steps,
``sync_to_renderer=None`` or ``False`` publishes only for visual consumers;
``True`` additionally requests publication on steps without consumers. An open
window remains a consumer regardless of this automatic policy.
The historical ``embodichain.lab.sim.cfg.simulation`` module re-exports the
rendering and physics configuration types for callers that have not migrated
to the split modules.

.. currentmodule:: embodichain.lab.sim.cfg.simulation

.. autosummary::

DenoisingMode
DenoisingCfg
DLSSCfg
NRDCfg
RenderCfg
GPUMemoryCfg
PhysicsBackendCfg
Expand Down
22 changes: 17 additions & 5 deletions docs/source/api_reference/embodichain/embodichain.lab.sim.rst
Original file line number Diff line number Diff line change
Expand Up @@ -146,15 +146,27 @@ Utility

embodichain.lab.sim.utility

DLSS Configuration
------------------
Ray-Tracing Denoising and Reconstruction
----------------------------------------

.. currentmodule:: embodichain.lab.sim

Configure window and offscreen Ray Reconstruction and Super Resolution through
``SimulationManagerCfg.render_cfg.dlss``. Output resolution remains owned by the
window or camera configuration.
Select independent window and offscreen image-processing paths through
``SimulationManagerCfg.render_cfg.denoising``. The public ``dlss`` path maps to
DLSS Ray Reconstruction and ``nrd`` maps to standalone NRD RELAX. DLSS and NRD
tuning remain in their dedicated nested configs; output resolution stays owned
by the window or camera configuration.

.. autodata:: DenoisingMode

.. autoclass:: DenoisingCfg
:members:
:undoc-members:

.. autoclass:: DLSSCfg
:members:
:undoc-members:

.. autoclass:: NRDCfg
:members:
:undoc-members:
14 changes: 9 additions & 5 deletions docs/source/overview/sim/sim_manager/rendering/configuration.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# Rendering Configuration

The {class}`~embodichain.lab.sim.cfg.RenderCfg` class controls the renderer,
ray-tracing sample count, tone mapping, and DLSS settings used by
ray-tracing sample count, scoped denoising/reconstruction, tone mapping, DLSS,
and NRD settings used by
{class}`~embodichain.lab.sim.sim_manager.SimulationManager`.

## Core options
Expand All @@ -10,13 +11,15 @@ ray-tracing sample count, tone mapping, and DLSS settings used by
| :--- | :--- | :--- | :--- |
| `renderer` | `str` | `"auto"` | Renderer backend: `auto`, `hybrid`, `fast-rt`, or `rt`. |
| `spp` | `int` | `1` | Samples per pixel for ray-traced rendering. Must be at least `1`. |
| `denoising` | `DenoisingCfg` | `DenoisingCfg()` | Independent `window`/`offscreen` choices: `off`, `optix`, `dlss`, or `nrd`. |
| `tone_mapping_enabled` | `bool` | `False` | Apply modified Reinhard tone mapping to RGB output. |
| `tone_mapping_exposure` | `float` | `1.0` | Fixed linear exposure multiplier used before tone mapping. |
| `dlss` | `DLSSCfg` | `DLSSCfg()` | NVIDIA DLSS settings. See {doc}`dlss`. |
| `nrd` | `NRDCfg` | `NRDCfg()` | NVIDIA NRD settings for the public `nrd` path. |

Ray-traced output uses DexSim's OptiX denoiser. Tone mapping affects RGB
output only; depth, segmentation masks, normals, and position buffers remain
unchanged.
The public `dlss` path maps to DexSim `DLSS_RR`; the public `nrd` path maps to
standalone DexSim `NRD_RELAX`. Tone mapping affects RGB output only; depth,
segmentation masks, normals, and position buffers remain unchanged.

## Renderer selection

Expand Down Expand Up @@ -44,12 +47,13 @@ SimulationManager.set_default_renderer("fast-rt")

```python
from embodichain.lab.sim import SimulationManagerCfg
from embodichain.lab.sim.cfg import RenderCfg
from embodichain.lab.sim.cfg import DenoisingCfg, RenderCfg

sim_config = SimulationManagerCfg(
render_cfg=RenderCfg(
renderer="fast-rt",
spp=4,
denoising=DenoisingCfg(window="dlss", offscreen="nrd"),
tone_mapping_enabled=True,
tone_mapping_exposure=1.0,
)
Expand Down
48 changes: 13 additions & 35 deletions docs/source/overview/sim/sim_manager/rendering/dlss.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,36 +9,24 @@ constructing a configuration object does not by itself initialize DLSS.

## Processing model

DexSim manages Ray Reconstruction (RR) for denoising and reconstruction,
and Super Resolution (SR) for upscaling. Offscreen DLSS and RR use DexSim's
native defaults. `upscale_enabled` controls standalone SR when RR is disabled.
DexSim manages Ray Reconstruction (RR) for the public `dlss` path. The public
`nrd` path is configured separately through `RenderCfg.nrd` and maps to
standalone NRD RELAX.

## EmbodiChain configuration

The following fields are available under `RenderCfg.dlss`:

| Parameter | Default | Description |
| :--- | :---: | :--- |
| `dlss_enabled` | `True` | Master switch for DLSS on window and offscreen targets. |
| `upscale_enabled` | `True` | Enables standalone SR when RR is disabled. |
| `dlss_quality` | `2` | Quality preset: `-1` auto, `0` ultra performance, `1` performance, `2` balanced, `3` quality, `4` ultra quality, or `5` DLAA. |
| `render_width`, `render_height` | `0` | Optional internal dimensions for FastRT/OfflineRT windows. Zero derives the dimensions from the quality preset. |
| `target_width`, `target_height` | `0` | Compatibility fields. Set the actual output size on the window or camera instead. |
| `upsample_ratio` | `None` | Optional FastRT/OfflineRT ratio used to derive unset internal dimensions. |
| `exposure_compensation` | `1.0` | Positive exposure multiplier used by the RR bridge. |
| `frame_time_delta_ms` | `0.0` | Render-frame interval in milliseconds. Zero selects DexSim's automatic measurement; a positive value supplies a fixed interval. |

`frame_time_delta_ms` is deliberately defaulted to `0.0`, matching DexSim's
native `DLSSConfig` default. DexSim measures the elapsed time between rendered
frames and uses it in the temporal path. This value describes render cadence,
not the physics or control timestep, so it should normally remain `0.0`.
Specify a positive value only when the application intentionally renders at a
known fixed cadence.

EmbodiChain currently mirrors the core DLSS controls only. DexSim's advanced
multi-camera tiled controls are intentionally not duplicated in
`DLSSCfg`; their native defaults remain in effect while the engine manages
camera-group rendering.
`DLSSCfg.tiled_enabled` controls whether compatible multi-camera targets use a
single tiled atlas. Atlas gutter and dimension limits remain owned by DexSim
and use its native defaults.

## Quality and resolution

Expand Down Expand Up @@ -67,31 +55,21 @@ preset.
Configure DLSS quality in a headless simulation:

```python
from embodichain.lab.sim import DLSSCfg, SimulationManagerCfg
from embodichain.lab.sim.cfg import RenderCfg
from embodichain.lab.sim import SimulationManagerCfg
from embodichain.lab.sim.cfg import DenoisingCfg, DLSSCfg, RenderCfg

sim_config = SimulationManagerCfg(
headless=True,
render_cfg=RenderCfg(
renderer="hybrid",
denoising=DenoisingCfg(window="dlss", offscreen="dlss"),
dlss=DLSSCfg(
dlss_enabled=True,
dlss_quality=3,
),
),
)
```

Use a fixed 60 FPS render interval only when the rendering cadence is known
and intentionally fixed:

```python
render_cfg = RenderCfg(
renderer="hybrid",
dlss=DLSSCfg(frame_time_delta_ms=16.667),
)
```

Configure an OfflineRT window with an explicit internal resolution:

```python
Expand All @@ -105,10 +83,10 @@ sim_config = SimulationManagerCfg(
)
```

The same settings are available in task JSON/YAML under
`render_cfg.dlss` (decoded into `env_cfg.sim_cfg.render_cfg.dlss`). Set
`dlss_enabled: false` explicitly when a task must use the standard renderer
path.
The same settings are available in task JSON/YAML under `render_cfg.denoising`
and `render_cfg.dlss` (decoded into the corresponding nested config objects).
Use `denoising.window: off` or `denoising.offscreen: off` to select the raw
ray-traced path for a scope.

## Availability and fallback

Expand Down
6 changes: 6 additions & 0 deletions embodichain/lab/gym/utils/gym_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -464,12 +464,14 @@ def config_to_cfg(
"""

from embodichain.lab.sim.cfg import (
DenoisingCfg,
RobotCfg,
RigidObjectCfg,
RigidObjectGroupCfg,
ArticulationCfg,
LightCfg,
DLSSCfg,
NRDCfg,
RenderCfg,
physics_cfg_for_backend,
)
Expand Down Expand Up @@ -655,8 +657,12 @@ class ComponentCfg:
env_cfg.enable_sensor = config.get("enable_sensor", True)

render_config = deepcopy(config.get("render_cfg", {}))
if isinstance(render_config.get("denoising"), dict):
render_config["denoising"] = DenoisingCfg(**render_config["denoising"])
if isinstance(render_config.get("dlss"), dict):
render_config["dlss"] = DLSSCfg(**render_config["dlss"])
if isinstance(render_config.get("nrd"), dict):
render_config["nrd"] = NRDCfg(**render_config["nrd"])
if "renderer" in config:
# Keep the existing flat renderer option as the command-line override.
render_config["renderer"] = config["renderer"]
Expand Down
5 changes: 4 additions & 1 deletion embodichain/lab/sim/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
VisualMaterialInst,
ReuseSegmentState,
)
from .cfg import DLSSCfg
from .cfg import DenoisingCfg, DenoisingMode, DLSSCfg, NRDCfg
from .common import BatchEntity
from .profiler import Profiler, ProfilerCfg

Expand All @@ -43,7 +43,10 @@
"ProfilerCfg",
"SimulationManager",
"SimulationManagerCfg",
"DenoisingCfg",
"DenoisingMode",
"DLSSCfg",
"NRDCfg",
"SIM_CACHE_DIR",
"MATERIAL_CACHE_DIR",
"CONVEX_DECOMP_DIR",
Expand Down
Loading
Loading