VitisAI EP (VAIML/BF16): silent miscompilation of SRVGGNetCompact with trained PReLU weights

[tomhda]

Summary

Compiling a BF16 (cast-format) SRVGGNetCompact super-resolution model with trained PReLU weights through VitisAIExecutionProvider's VAIML flow succeeds with no warning, runs fast, and returns numerically exploded output: max|diff| > 1500 vs the fp32 CPU reference (PSNR is negative; output should be roughly [0,1]). The failure is deterministic and weight-dependent — the identical graph with random weights compiles and runs correctly.

The concerning part is the failure mode: silent wrong results, with no compile-time or run-time diagnostic.

Environment

• Ryzen AI 7 PRO 350 (XDNA2 NPU), Windows 11
• NPU driver 32.0.203.314
• Ryzen AI Software 1.7.1 (conda), onnxruntime-vitisai 1.23.3.dev20260320
• quark 0.11rc1; BF16 conversion via python -m quark.onnx.tools.convert_fp32_to_bf16 --format with_cast
• Model: realesr-animevideov3 (SRVGGNetCompact: 17x [Conv3x3(64) + per-channel PReLU] -> Conv -> DepthToSpace(CRD, 4x) + nearest-Resize skip Add; weights BSD-3-Clause)

Reproduction

Complete package (models + 30-line script): https://github.com/tomhda/vaiml-bf16-prelu-repro

git clone https://github.com/tomhda/vaiml-bf16-prelu-repro
cd vaiml-bf16-prelu-repro
python repro.py

Observed:

max|diff| = 1552.0256  (expected < 0.05)
psnr vs fp32 = -39.9 dB  (expected > 30 dB)

Bisection evidence

All rows: same conversion command, same EP options, same machine. "OK" = PSNR > 30 dB vs fp32 CPU.

Case	Result
Single ops in isolation (Conv / Conv+PReLU / Conv+DepthToSpace(CRD) / DepthToSpace+Resize+Add)	OK (51-69 dB)
Same topology at depths 2/8/16, PReLU slopes = 0.25 (default init)	OK (~59 dB)
Same topology, random weights, after onnx-simplifier	OK (59.3 dB)
Trained weights, with or without onnx-simplifier	BROKEN (identical -39.9 dB both times -> deterministic)
Synthetic PReLU slopes ~ U(-1.4, 1.7), matching the trained slope range	OK but degraded (35 dB vs 59 dB) and compile time inflated 38 s -> 1005 s
Trained weights with PReLU replaced by the equivalent ReLU(x) - w*ReLU(-x)	OK (38.4 dB)

Notes:

• The trained model's per-channel PReLU slopes span -1.38 … +1.69 (negative and >1 values, 17 layers). Conv biases reach ~28.
• Out-of-range slopes alone reproduce a fidelity drop plus a ~30x compile-time blowup; the full numeric explosion additionally requires the trained weight values (possibly interacting with the large biases).
• The equivalent PReLU decomposition avoids the issue entirely and is what we ship as a workaround, so we are not blocked — reporting because the silent-corruption failure mode seems worth fixing.

Expected behavior

Numerically correct compilation of PReLU with arbitrary trained slopes, or a hard compile error / CPU fallback when a kernel's supported parameter range is exceeded. Silent garbage output should never be possible.

[tomhda]

Update: I re-tested the same model (trained PReLU weights, non-decomposed) through Windows ML's catalog-delivered VitisAI EP — package WindowsWorkload.EP.AMD.VitisAI.1.8 (NPU driver 32.0.203.314, Krackan Point, Windows 11 26200) — and the miscompilation does not reproduce there. The non-decomposed fp32 model compiles and runs correctly: PSNR ~59.7 dB vs fp32 CPU reference, versus the catastrophic ~-40 dB degradation with the Ryzen AI SW 1.7.1 native VAIML flow reported above.

Two observations that may still be useful:

• The PReLU-decomposed variant still compiles to a noticeably faster schedule than the non-decomposed model on the same EP 1.8 (133 ms vs 274 ms per 256x256 x4 tile on my machine), so the decomposition workaround remains relevant for performance even where the accuracy issue is fixed.
• Ryzen AI SW 1.7.1 (native VAIML) still reproduces the bug as described. It would be good to know whether the fix in the Windows ML EP 1.8 toolchain is planned to land in a native Ryzen AI SW release as well.