Skip to content

Bridge: cut per-frame IPC cost of the D3D9 bridge - #158

Open
Kim2091 wants to merge 1 commit into
NVIDIAGameWorks:mainfrom
Kim2091:feat/20261015-upstream-bridge-perf
Open

Kim2091 wants to merge 1 commit into
NVIDIAGameWorks:mainfrom
Kim2091:feat/20261015-upstream-bridge-perf

Conversation

@Kim2091

@Kim2091 Kim2091 commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Bridge: cut per-frame IPC cost of the D3D9 bridge

Most of a bridged frame went to IPC overhead:

  • The once-per-frame event query GetData was a blocking round trip.
    It drained the whole command queue and forced lockstep between client
    and server.
  • Each of the ~28,000 commands per frame in HL2 RTX paid the full
    per-command cost.

Measured in HL2 RTX:

  • Frame time: ~14-15 ms -> ~8-8.6 ms
  • Commands per frame: ~28,000 -> ~4,000

Changes:

  • Non-blocking queries (client.asyncQueries, default True)

    • Event queries are answered on the client from the UID of the last
      Issue the server executed, published in shared memory.
    • Other small query types send GetData without waiting. The server
      writes the result to a shared per-query slot.
    • D3DQUERYTYPE_RESOURCEMANAGER and slot exhaustion keep the blocking
      path.
    • Fixes: the blocking path reads the data blob even when pData is
      null, and the server no longer crashes on queries whose create
      failed.
  • Redundant state filtering (client.dropRedundantState, default True)

    • The client tracks the state it sent and skips setters whose value
      the server already holds (21 setters).
    • Shader constants are narrowed to the registers that changed.
    • Tracking is reset on state block Apply, Reset and
      SetRenderTarget(0).
    • Replaces eliminateRedundantSetterCalls, which compared against
      game-visible state.
  • State packets (client.batchState, default True)

    • Consecutive setters travel as records of one Bridge_StatePacket.
    • The packet closes before any other command, so ordering is
      unchanged.
  • Spin-then-block queues (ipcSpinMicroseconds, default 100)

    • Queue waits spin, then block on a named event that the other side
      signals only while the waiter sleeps.
    • pull() copies the element out before releasing its slot.
    • The unused semaphore-based queue is removed.
  • Present semaphore fix (presentSemaphoreMaxFrames, default 1)

    • Both sides create it with presentSemaphoreMaxFrames as the
      initial count, so the setting takes effect. Before, it was always
      lockstep.
  • Source engine defaults (hl2.exe)

    • Creates no longer wait for responses.
    • Fixes the trailing space in the presentSemaphoreMaxFrames key.
    • The client sends its response policy at the handshake so both
      sides agree.
    • Failed creates are logged, and uploads to them are skipped.
  • Perf counters (perfLogIntervalFrames, default 0 = off)

    • Logs per-frame command, wait and query statistics on both sides.
  • Unit tests: sent_state, query_results, ipc_queue_wait,
    upload_parts.

Data queue fixes

These fix hangs and crashes on large texture uploads. The first two come
from xoxor4d's fork; the squashed commit credits them as co-author.

  • No stall at the batch start (xoxor4d)
    • The client no longer stalls when its data ring wraps while the
      server waits at the current batch start. A server waiting there has
      consumed everything, so nothing can be overwritten.
  • Absolute ring positions (xoxor4d)
    • The overwrite protection compares absolute word positions instead
      of ring offsets plus a reset flag.
    • The flag could strand both processes in a wait that never ended.
  • Surface uploads in 1 MB blobs
    • A blob that does not fit before the end of the ring restarts at
      word 0, and the skipped tail counts against its command.
    • A single 64 MB blob (4096x4096 A8R8G8B8) could not fit in the
      default 96 MB ring. Before the fix, the queue deadlocked and the
      client then overwrote its own command header.
  • Uploads over 16 MB split into several commands
    • A lock larger than the ring (for example a 256 MB A32B32G32R32F
      level) used to need a bigger clientChannelMemSize.
    • The server locks the surface on the first part and unlocks it on
      the last, so the runtime still sees one upload.
    • Pending locks are tracked per surface.
  • Not covered: volume texture uploads and large server-to-client
    readbacks still go in one piece.

The protocol changed: deploy the client d3d9.dll and the server
NvRemixBridge.exe together.

Co-Authored-By: Claude Opus 5.5 (1M context) noreply@anthropic.com

Most of a bridged frame went to IPC overhead:
- The once-per-frame event query `GetData` was a blocking round trip.
  It drained the whole command queue and forced lockstep between client
  and server.
- Each of the ~28,000 commands per frame in HL2 RTX paid the full
  per-command cost.

Measured in HL2 RTX:
- Frame time: ~14-15 ms -> ~8-8.6 ms
- Commands per frame: ~28,000 -> ~4,000

Changes:

- **Non-blocking queries** (`client.asyncQueries`, default `True`)
  - Event queries are answered on the client from the UID of the last
    `Issue` the server executed, published in shared memory.
  - Other small query types send `GetData` without waiting. The server
    writes the result to a shared per-query slot.
  - `D3DQUERYTYPE_RESOURCEMANAGER` and slot exhaustion keep the blocking
    path.
  - Fixes: the blocking path reads the data blob even when `pData` is
    null, and the server no longer crashes on queries whose create
    failed.

- **Redundant state filtering** (`client.dropRedundantState`, default `True`)
  - The client tracks the state it sent and skips setters whose value
    the server already holds (21 setters).
  - Shader constants are narrowed to the registers that changed.
  - Tracking is reset on state block `Apply`, `Reset` and
    `SetRenderTarget(0)`.
  - Replaces `eliminateRedundantSetterCalls`, which compared against
    game-visible state.

- **State packets** (`client.batchState`, default `True`)
  - Consecutive setters travel as records of one `Bridge_StatePacket`.
  - The packet closes before any other command, so ordering is
    unchanged.

- **Spin-then-block queues** (`ipcSpinMicroseconds`, default `100`)
  - Queue waits spin, then block on a named event that the other side
    signals only while the waiter sleeps.
  - `pull()` copies the element out before releasing its slot.
  - The unused semaphore-based queue is removed.

- **Present semaphore fix** (`presentSemaphoreMaxFrames`, default `1`)
  - Both sides create it with `presentSemaphoreMaxFrames` as the
    initial count, so the setting takes effect. Before, it was always
    lockstep.

- **Source engine defaults** (`hl2.exe`)
  - Creates no longer wait for responses.
  - Fixes the trailing space in the `presentSemaphoreMaxFrames` key.
  - The client sends its response policy at the handshake so both
    sides agree.
  - Failed creates are logged, and uploads to them are skipped.

- **Perf counters** (`perfLogIntervalFrames`, default `0` = off)
  - Logs per-frame command, wait and query statistics on both sides.

- **Unit tests**: `sent_state`, `query_results`, `ipc_queue_wait`,
  `upload_parts`.

Data queue fixes for large texture uploads:

- **No stall at the batch start**
  - The client no longer stalls when its data ring wraps while the
    server waits at the current batch start. That server has consumed
    everything pushed, so nothing can be overwritten.

- **Absolute ring positions**
  - The overwrite protection compares absolute word positions (skipped
    tail slots included, identical on both sides) instead of ring
    offsets and the serverResetPosRequired flag, which could leave both
    processes waiting forever.
  - The client waits on the data semaphore in 2 ms slices and re-checks
    the reader position itself.

- **Surface uploads in 1 MB blobs, split above 16 MB**
  - A blob that does not fit before the ring end restarts at word 0 and
    its skipped tail counts against the command, so one 64 MB blob
    (4096x4096 A8R8G8B8) could not fit in the default 96 MB ring, and a
    lock larger than the ring (256 MB A32B32G32R32F) never could.
  - Rows travel in 1 MB blobs, in consecutive UnlockRect commands of at
    most 16 MB each. The server locks the surface on the first part and
    unlocks it on the last, so the runtime still sees one upload.
    Pending locks are kept per surface.

The protocol and shared memory layout changed: deploy the client
`d3d9.dll` and the server `NvRemixBridge.exe` together.

Co-authored-by: xoxor4d <45299104+xoxor4d@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@Kim2091
Kim2091 force-pushed the feat/20261015-upstream-bridge-perf branch from 0eff415 to 917b537 Compare October 19, 2026 07:36
@MarkEHenderson

Copy link
Copy Markdown
Collaborator

REMIX-6297 for tracking

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants