[dependabot]

Bumps onnxruntime-genai from 0.9.0 to 0.18.0.

Release notes

Sourced from onnxruntime-genai's releases.

v0.17.0

Version 0.17.0 makes repeated prompts faster, adds new vision and audio models, and brings more ways to tune the Engine for different GPUs.

Highlights

• Faster repeated prompts with prefix caching - The dynamic Engine can now reuse matching prompt prefixes across requests. Prefix caching is enabled automatically for supported paged KV-cache layouts.
• Runtime profiles for different GPUs - A single model package can select Engine settings and model variants based on available GPU memory. New capability APIs also make it easier for applications to discover Engine limits.
• New LFM2 models - Added support for LFM2-VL and LFM2.5-VL vision models, LFM2-MoE, and LFM2-Audio and LFM2.5-Audio with speech input and output.
• Moonshine streaming speech recognition - Added Moonshine small and tiny streaming ASR models, designed for fast transcription with low CPU use.
• Faster DFlash decoding - DFlash now supports CUDA graph capture, prepacked weights, optional projection fusion, sampled speculative decoding, and direct Qwen 3.8 package export.
• Lower GPU memory pressure - Embedding lookups can be offloaded to CPU while keeping CUDA graph capture enabled for decoding.
• Less work during generation - Paged decoders can compute logits only for the token rows needed for generation and speculative verification.
• Safer conversation editing - Engine requests can rewind to the start of an earlier turn and continue from there without creating a new request.
• A better Model Builder configuration - The new structured configuration organizes target, drafter, speculative decoding, runtime, and search settings in one validated format.
• More Engine flexibility - Added configurable paged block sizes, improved speculative draft controls, and APIs for maximum request length.
• Reliability improvements - Includes fixes for WebGPU graph capture, decoder pipeline KV-cache shapes, paged prefill settings, legacy execution providers, and prefill logits.

v0.16.0

This is a big release! Version 0.16.0 brings major upgrades for high-throughput serving, faster decoding, and the latest generation of Qwen models.

Highlights

• A much more capable Engine - Continuous batching now supports chunked prefill, paged attention, multi-turn conversations, mixed workloads, and smarter token scheduling. This improves throughput while making better use of GPU memory.
• Faster generation with speculative decoding - Added support for draft models, n-gram speculation, MTP self-speculation, and DFlash2 and DSpark block drafting.
• Tool calling and constrained decoding - The Engine can generate tool calls, continue after tool results without replaying the full prompt, and enforce structured output with guidance.
• Better multi-turn control - Requests can retain their KV cache between turns. Sampling, token limits, seeds, stop strings, and guidance can be changed for each turn.
• More Qwen support - The model builder now supports Qwen2.5-VL, Qwen3-VL, and Qwen3.5 dense and MoE models, with additional export and speculative-decoding support for Qwen3.6 and Qwen3.8.
• More efficient KV caches - Added paged, windowed, sparse, and quantized KV-cache support, including INT4, INT8, and FP8 paths.
• New quantized model formats - Added native support for NVFP4 and FP8 checkpoints from NVIDIA Model Optimizer and compressed-tensors, plus the quant_auto format.
• New hardware support - Added the AMDGPU execution provider and WebGPU paged-attention exports.
• New model support - Added Granite models and improved Marian batching, multimodal prefill, Nemotron audio, and multi-image vision handling.
• Stronger reliability - This release includes broad validation and recovery improvements across the Engine, speculative decoding, vision and audio inputs, cache management, and process shutdown.

v0.15.2

No release notes provided.

v0.15.1

No release notes provided.

v0.15.0

What's Changed

• Add per_layer_inputs support for Gemma4 and fix pixel_position_ids by @​apsonawane in microsoft/onnxruntime-genai#2188
• Enable Qwen3.5 35B-A3B MoE text export for TRT-RTX EP by @​yen-shi in microsoft/onnxruntime-genai#2186
• Multilingual model ids expansion in the sample/example based on new Nvidia's README by @​nenad1002 in microsoft/onnxruntime-genai#2190
• Fix webgpu shared buffer issue by @​apsonawane in microsoft/onnxruntime-genai#2191
• Fix nightly build by @​kunal-vaishnavi in microsoft/onnxruntime-genai#2193
• Clean up base constructor in model builder by @​kunal-vaishnavi in microsoft/onnxruntime-genai#2192
• Fix CUDA illegal memory access crash in TopK when inputs contain NaN by @​jiafatom in microsoft/onnxruntime-genai#2195
• Fix: Skip CPU provider in SetProviderSessionOptions by @​jiafatom in microsoft/onnxruntime-genai#2179
• Add README file for testing folder by @​kunal-vaishnavi in microsoft/onnxruntime-genai#2205

... (truncated)

Commits

• See full diff in compare view

[Copilot]

🟡 Changes recommended

The new version is incompatible with the targeted Python 3.10 branch, and the lockfile remains stale.

2 open findings

• Preserve GenAI 0.9.0 for Python 3.10 compatibility · New
• Regenerate uv.lock to record the updated GenAI dependency · New

What changed in this PR

Updates the optional Python ONNX GenAI dependency.

Changes:

• Bumps onnxruntime-genai from 0.9.0 to 0.17.1 for Python 3.10.

File	Description
python/​pyproject.toml	Updates the Python 3.10 ONNX GenAI dependency.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.