Fix cuDNN 9 libraries not being preloaded on Windows CUDA builds

[gijreyneveld]

Description

I use an open-source desktop application for bioimage analysis (QuPath) that depends on DJL for GPU inference.
Our community has been struggling to set this up, because libraries are somehow not pre-loaded (torch_cuda.dll UnsatisfiedLinkError).

We made a workaround to add a directory that supplies the DJL chache directory to PATH before launching, (or sometimes a conda environment). However, I wanted to find the root cause of this so that we might make the GPU inference in our application more user friendly. This is what I found;

LibUtils.loadLibTorch() walks the native library directory and explicitly skips every cuDNN file (so not everything as we first expected):

&& !name.startsWith("cudnn")

cuDNN is then instead preloaded separately, by hardcoded filename;

if (Files.exists((libDir.resolve("cudnn64_8.dll")))) { ... }
else if (Files.exists((libDir.resolve("cudnn64_7.dll")))) { ... }

However, it is only compatible with cuDNN6 v7 and cuDNN v8, whereas the cu128/win-x86_64 bundle for PyTorch 2.7.1 that DJL 0.36.0 downloads comes with cuDNN 9. Therefore, it matches nothing and the files are passed over a second time and then torch_cuda.dll oads with nothing behind it.

I made the edit to hardcode cuDNN9 and tested it with the desktop application and the command line. I had to tweak build.gradle.kts to reproduce it in the command line, see second change below.

Longer term, cuDNN version 10 will run into the same thing. Something version-agnostic would avoid that, though I didn't want to change the loading behaviour of the existing cuDNN 8/7 paths without understanding why the explicit ordering of cuDNN 8 and 7 is there.

Changes

1. LibUtils.loadLibTorch() — adds a cuDNN 9 branch ahead of the existing cuDNN 8 / 7 checks. The older version paths are unchanged. After patching this locally and rebuilding pytorch-engine, the PyTorch engine initialises with CUDA and GPU inference runs with no PATH changes at all. Verified on two Windows machines with CUDA 12.8 and from command line

2. engines/pytorch/pytorch-engine/build.gradle.kts — this was needed in order to reproduce the above from the command line on Windows. I didn't pro-actively think about making this piece system-agnostic, so it can be a separate thing.

The test task sets PATH like this:

environment("PATH" to "src/test/bin:${environment["PATH"]}")

Gradle's environment map is case-sensitive, but the inherited Windows variable is named Path. So environment["PATH"] is null and interpolates as the literal string "null", and PATH gets added as a second key alongside Path. Windows
is case-insensitive when creating the process and uses PATH. Dumping System.getenv() from inside a test worker shows:

KEY[PATH] = src/test/bin:null
KEY[Path] = C:\Program Files\Eclipse Adoptium\...;...CUDA\v12.8\bin;...

So test workers on Windows run with no usable PATH, and CUDA loading fails at cublas64_12.dll long before it ever reaches the cuDNN code. Maybe this is why the issue above was never caught by the Windows tests?

System.getenv("PATH") looks up case-insensitively and returns the good Path value, so this is invisible from inside Java unless you dump the raw keys.

Change sets PATH with the ; separator. It leaves the duplicate Path key in place.

Reproduction

Windows, CUDA 12.8, with no directory on PATH supplying cuDNN. Change 2 is needed for the test to get far enough to show the bug.

set PYTORCH_FLAVOR=cu128
gradlew.bat :engines:pytorch:pytorch-engine:test --tests "*PtNDArrayTest*" --rerun-tasks

Before — note there is no cudnn* line anywhere in the sequence:

Loading native library: ...\zlibwapi.dll
Loading native library: ...\fbgemm.dll
Loading native library: ...\caffe2_nvrtc.dll
Loading native library: ...\torch_cpu.dll
Loading native library: ...\c10_cuda.dll
Loading native library: ...\torch_cuda.dll

PtNDArrayTest > testLargeTensor FAILED
  Caused by: java.lang.UnsatisfiedLinkError: ...\torch_cuda.dll: Can't find dependent libraries

After — the cuDNN 9 libraries load and initialisation completes:

Loading native library: ...\zlibwapi.dll
Loading native library: ...\cudnn_graph64_9.dll
Loading native library: ...\cudnn_engines_precompiled64_9.dll
Loading native library: ...\cudnn_engines_runtime_compiled64_9.dll
Loading native library: ...\cudnn_heuristic64_9.dll
Loading native library: ...\cudnn_ops64_9.dll
Loading native library: ...\cudnn_cnn64_9.dll
Loading native library: ...\cudnn_adv64_9.dll
Loading native library: ...\cudnn64_9.dll
Loading native library: ...\fbgemm.dll
Loading native library: ...\caffe2_nvrtc.dll
Loading native library: ...\torch_cpu.dll
Loading native library: ...\c10_cuda.dll
Loading native library: ...\torch_cuda.dll
Loading native library: ...\torch.dll

PtNDArrayTest > testLargeTensor PASSED
PtNDArrayTest > testStringTensor PASSED

Related issues

#3480 reports the same error on 2.4.0-cu124. There's no load log attached so I can't confirm it, but the workaround that resolved it there was adding the DJL cache directory to PATH.

#3740 same problem as the log shows DJL downloading all eight cuDNN 9 files and then not loading any of them before torch_cuda.dll fails. Here, adding the DJL cache directory PATH resolved it as well.

Notes

• Change is not backward compatible
• The individual cuDNN 9 library load order is random, but it works
• checkstyleMain passes but couldn't run formatJava

[Copilot]

Pull request overview

This PR addresses Windows CUDA initialization failures for the PyTorch engine by ensuring cuDNN 9 DLLs are preloaded before torch_cuda.dll, and by fixing the Gradle test PATH setup so Windows tests can reproduce and catch the issue.

Changes:

• Extend LibUtils.loadLibTorch() to preload cuDNN 9 DLLs (ahead of existing cuDNN 8/7 handling) on Windows.
• Update pytorch-engine Gradle test configuration to correctly construct PATH on Windows by reading the inherited environment via System.getenv("PATH").

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
engines/pytorch/pytorch-engine/src/main/java/ai/djl/pytorch/jni/LibUtils.java	Adds explicit cuDNN 9 preload sequence to avoid torch_cuda.dll dependency failures on Windows CUDA builds.
engines/pytorch/pytorch-engine/build.gradle.kts	Adjusts test worker PATH setup to avoid losing the inherited Windows PATH due to case-sensitive Gradle environment map lookup.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[xyang16]

@gijreyneveld Thanks for the contribution! Could you please take care of the review comments and fix CI?