Uint8 output differs from CPU by 255 (255 vs 0)

[FFchopon]

Describe the issue

With the attached ONNX model and identical inputs, output 2 differs between CPUExecutionProvider and DmlExecutionProvider. On both providers it has dtype uint8 and shape [1, 32, 32, 256], but at a differing element CPU returns 255 while DirectML returns 0; the maximum absolute difference is 255. Outputs 0 and 1 match.

To reproduce

Attach the model as model.onnx, install onnxruntime-directml==1.23.0 and NumPy, then run:

import numpy as np
import onnxruntime as ort

def make_session(provider):
    options = ort.SessionOptions()
    options.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL
    if provider == "DmlExecutionProvider":
        options.add_session_config_entry("session.disable_cpu_ep_fallback", "1")
    return ort.InferenceSession(
        "model.onnx", options, providers=[provider]
    )

cpu = make_session("CPUExecutionProvider")
dml = make_session("DmlExecutionProvider")

rng = np.random.default_rng(2024)
inputs = {
    "input": rng.standard_normal((1, 1, 1, 3)).astype(np.float32),
    "input_token_0": rng.standard_normal((1, 3)).astype(np.float32),
}
cpu_out = cpu.run(None, inputs)
dml_out = dml.run(None, inputs)

for i, (a, b) in enumerate(zip(cpu_out, dml_out)):
    diff = np.abs(a.astype(np.float64) - b.astype(np.float64))
    index = np.unravel_index(np.argmax(diff), diff.shape)
    print(i, a.dtype, a.shape, "max_abs_diff:", diff[index],
          "CPU:", a[index], "DML:", b[index])

Urgency

No response

Platform

Windows

OS Version

Windows build 26200

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.23.0

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response