Skip to content

feat(rendering): add scoped denoising pipeline config - #666

Merged
yuecideng merged 13 commits into
mainfrom
codex/enhance-render-denoising-config
Oct 3, 2026
Merged

yuecideng merged 13 commits into
mainfrom
codex/enhance-render-denoising-config

Conversation

@yuecideng

@yuecideng yuecideng commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Description

Add an EmbodiChain-owned denoising/reconstruction configuration layer for the ray-tracing pipeline introduced by DexSim !1416, updated for DexSim 0.5.1rc1.

Summary

  • add DenoisingCfg with independent window and offscreen selections
  • expose only off, optix, dlss, and nrd; dlss maps to DLSS_RR and nrd maps to standalone NRD_RELAX
  • add NRDCfg for NRD algorithm tuning while keeping native RELAX/REBLUR selection out of the public mode strings
  • align DLSSCfg with DexSim's 0.5.1rc1 mode-based API and tiled multi-camera settings
  • decode nested denoising, DLSS, and NRD mappings from Gym YAML/JSON configs
  • add an executable configuration example under examples/sim/rendering/denoising_config.py
  • add a reproducible isolated-process offscreen benchmark with a generated material/edge/motion stress scene, image metrics, timing, VRAM, comparison PNG, and Markdown report
  • update public API docs and simulation-system agent context

Dependency and compatibility

  • Depends on DexSim !1416 and pins dexsim_engine==0.5.1rc1. Local validation used DexSim 0.5.1rc1 at 93229769e1.
  • This is a configuration breaking change: DLSSCfg.dlss_enabled and DLSSCfg.upscale_enabled are replaced by the selected denoising/reconstruction mode.
  • The PR branch is based on the latest EmbodiChain main and includes the DexSim 0.5.1rc1 compatibility update.

Type of change

  • Enhancement
  • Breaking change (existing rendering configuration requires migration)
  • Documentation update

Screenshots

The benchmark writes comparison.png with OptiX/DLSS/NRD frames and amplified difference images under outputs/benchmarks/.

Validation

  • black --check --diff --color ./
  • python docs/scripts/check_api_docs.py — 2397/2397 exports documented
  • rendering/Gym/benchmark tests on DexSim 0.5.1rc1 — 440 passed
  • context and API documentation tests — 48 passed
  • Sphinx dummy build — succeeded with existing repository warnings
  • example:
    python examples/sim/rendering/denoising_config.py --window optix --offscreen nrd --dlss-quality 3 --nrd-history-frames 24
  • benchmark:
    python -m scripts.benchmark.rendering.offscreen_denoising

Offscreen benchmark (RTX 5090)

FastRT, 640×360, 1 spp for every mode, DLSS quality 3, 12 warmup frames, 40 measured frames, and 12 quality frames along the same moving-camera/object path. OptiX at 1 spp is the user-selected quality reference, so PSNR/SSIM measure reference consistency rather than absolute ground-truth fidelity. This run uses DexSim 0.5.1rc1.

mode median ms P95 ms FPS peak process VRAM MiB PSNR dB vs OptiX SSIM MAE temporal delta MAE
optix 2.679 3.039 373.318 6369 reference 1.000000 0.000000 0.000000
dlss 1.681 2.621 594.928 6602 30.5496 0.985792 0.017983 0.008905
nrd 1.486 1.847 672.817 6427 25.8215 0.961245 0.034748 0.023640

On this scene, standalone NRD is 11.6% faster than DLSS by median latency and reaches 13.1% higher FPS. DLSS remains closer to the OptiX reference: +4.73 dB PSNR, +0.0245 SSIM, and 48.2% lower MAE than NRD. NRD uses 175 MiB less peak process VRAM in this run.

Full-suite caveats:

  • bare pytest is blocked during collection because the local environment lacks optional gradio for two GenSim UI test modules
  • excluding that optional directory reaches an unrelated Newton baseline failure in TestArticulationNewton.test_reset_restores_default_link_mass_properties on a separate preexisting world-mask path

Agent context review

  • Revised simulation-system rendering guidance for the 0.5.1rc1 native mode names and public EmbodiChain mapping.
  • env-framework, robot-system, and differentiable-env required no prose change; their existing ownership boundaries remain accurate.

Checklist

  • I have run the black . command to format the code base.
  • I reviewed affected documentation and agent context, updated it where needed, or explained why no update was needed.
  • Public API changes are reflected in the API docs (python docs/scripts/check_api_docs.py).
  • I have added tests that prove the feature works.
  • Dependencies are documented; the DexSim 0.5.1rc1 pin is included.

@yuecideng yuecideng added rendering Things related to rendering (eg, performace, efficiency, bug) dexsim Things related to dexsim breaking labels Sep 21, 2026
@yuecideng
yuecideng marked this pull request as ready for review October 3, 2026 09:28
@greptile-apps

greptile-apps Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 4/5

[High risk] Refactors rendering and physics config into separate modules.

The PR should satisfy the repository’s public-annotation requirement before merging; no new behavioral defect was established.

Fix All in CodexFindings

  1. P2 Missing guarded type import ▶
Fix with agent prompt
### Issue 1
embodichain/lab/sim/cfg/physics.py:26-31
`physics.py` uses `NewtonSolverCfg` in a public annotation but never imports it. The developer guide requires public APIs to be fully annotated and circular imports to be guarded with `TYPE_CHECKING`. Add the guarded import to satisfy this repository requirement before merging.

```suggestion
from typing import Any, Literal, Sequence, TYPE_CHECKING

import numpy as np
import torch

from embodichain.utils import configclass, logger

if TYPE_CHECKING:
    from dexsim.engine.newton_physics.solvers_cfg import NewtonSolverCfg
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The PR adds independent window and offscreen denoising selections, DLSS and NRD tuning, Gym configuration decoding, documentation, and an offscreen benchmark. Since the previous review, it also adds DexUni legacy-option translation and a render-mode compatibility fallback. The previously reported missing export list and global SSIM calculation are fixed.

Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Gym YAML/JSON or Python config] --> B[RenderCfg]
  B --> C[DenoisingCfg: window and offscreen]
  B --> D[DLSSCfg and NRDCfg]
  C --> E[DexSim WorldConfig]
  D --> E
  E --> F[Window rendering]
  E --> G[Offscreen cameras]
Loading

Reviews (4) · Last reviewed commit: "fix(sim): bridge DexUni and rendering co..."

Comment thread embodichain/lab/sim/cfg/physics.py
Comment thread scripts/benchmark/rendering/offscreen_denoising.py Outdated
Comment on lines +26 to +31
from typing import Any, Literal, Sequence, TYPE_CHECKING

import numpy as np
import torch

from embodichain.utils import configclass, logger

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Missing guarded type import physics.py uses NewtonSolverCfg in a public annotation but never imports it. The developer guide requires public APIs to be fully annotated and circular imports to be guarded with TYPE_CHECKING. Add the guarded import to satisfy this repository requirement before merging.

Suggested change
from typing import Any, Literal, Sequence, TYPE_CHECKING
import numpy as np
import torch
from embodichain.utils import configclass, logger
from typing import Any, Literal, Sequence, TYPE_CHECKING
import numpy as np
import torch
from embodichain.utils import configclass, logger
if TYPE_CHECKING:
from dexsim.engine.newton_physics.solvers_cfg import NewtonSolverCfg

Context Used: CLAUDE.md (source)

Prompt To Fix With AI
This is a comment left during a code review.
Path: embodichain/lab/sim/cfg/physics.py
Line: 26-31

Comment:
**Missing guarded type import** `physics.py` uses `NewtonSolverCfg` in a public annotation but never imports it. The developer guide requires public APIs to be fully annotated and circular imports to be guarded with `TYPE_CHECKING`. Add the guarded import to satisfy this repository requirement before merging.

```suggestion
from typing import Any, Literal, Sequence, TYPE_CHECKING

import numpy as np
import torch

from embodichain.utils import configclass, logger

if TYPE_CHECKING:
    from dexsim.engine.newton_physics.solvers_cfg import NewtonSolverCfg
```

**Context Used:** CLAUDE.md ([source](https://github.com/dexforce/embodichain/blob/main/CLAUDE.md))

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Codex Fix in Claude Code

@yuecideng
yuecideng merged commit 2f8f204 into main Oct 3, 2026
9 checks passed
@yuecideng
yuecideng deleted the codex/enhance-render-denoising-config branch October 3, 2026 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking dexsim Things related to dexsim rendering Things related to rendering (eg, performace, efficiency, bug)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant