[wkutak]

DO NOT MERGE

Summary

• Reasoner notebook (reasoner/run_with_vllm.ipynb)
  • Add --trust-request-mm-kwargs to the Cosmos3-Super-FP8 launch on vllm/vllm-openai:v0.31.0. The video examples send media_io_kwargs (num_frames, fps), and without the flag the server rejects every one with HTTP 400. Image requests were not affected.
  • Publish the server on 127.0.0.1 only (-p 127.0.0.1:8001:8000) in all three launch cells. The query cells already use localhost.
  • Document next to the flag that it is for trusted clients only (per-request media overrides can increase CPU, GPU and memory use), and that serving remote clients needs additional measures such as --api-key and network-level access controls.
• Quantization cookbook: pin nvidia-modelopt[torch]==0.44.0 in the README and the three notebooks. This is the ModelOpt version the published FP8 checkpoints were quantized with.
• vLLM-Omni launch guidance (cosmos3-generate-media/references/vllm-omni.md): set VLLM_OMNI_VIDEO_SYNC_TIMEOUT=3600 in the Nano, Super and FP8 launch commands and explain why. The video examples call the synchronous /v1/videos/sync endpoint, whose default limit is 600 s. The first video after a server start includes a one-time warm-up that exceeds it. Edge and the 4-step launches are unchanged.

Why

• On the published nvidia/Cosmos3-Super-FP8 with vllm/vllm-openai:v0.31.0 at TP=4, launch command as written, the reasoner notebook passed all image cells but failed all 10 video cells with HTTP 400.
• --trust-request-mm-kwargs lets any client that can reach the server choose its own media-processing settings. The launch cells published port 8001 on all host interfaces with no API authentication, so the flag is now paired with a loopback-only binding.

Validation

All runs on one node with B200 GPUs, against freshly verified checkpoints (Super-FP8 at HF head f5d7c44f, Nano-FP8 at 313dbe58).

Reasoner (Super-FP8, vllm/vllm-openai:v0.31.0, TP=4, 4 GPUs)

• The notebook cells ran end to end: all 25 code cells without errors, the 12 video cells returned content, and the server log shows 18 successful chat completions with 0 HTTP 400s and 0 5xx.
• The loopback binding and the flag were both in place for this run.

vLLM-Omni (vllm/vllm-omni:cosmos3, vLLM 0.31.0 + vLLM-Omni 0.32.dev2, 1 GPU each)

• Nano-FP8: 5 of 5 audiovisual cases passed (t2i, t2v, t2vs, i2v, i2vs). Super-FP8: 3 of 3 passed (t2i, t2v, i2v).
• The server detected modelopt FP8 from the checkpoint and applied the checkpoint's activation policy (first_steps=3 last_steps=3 reasoner=a16).
• Outputs were checked with PyAV: images and videos are 1280x720, videos have 189 frames, audio is present only for the audio cases, and frames are not flat.
• First video after a server start: about 9 minutes (Nano) and about 28 minutes (Super), then about 2 minutes (Nano) and about 6.5 minutes (Super). With the default 600 s limit, the first Nano video (t2v) and the next (t2vs) returned 504 Gateway Timeout. Super with the default limit was not run; its 1767 s first request implies it would fail the same way.
• The cause of the one-time warm-up is not identified. It starts at the first native-FP8 denoising step. It's likely due to Quack kernels enabled on Blackwell GPUs.

[satishra-ai]

Tested PR #341 at 39eb806: PASS with coverage limitations.
Reasoner: 18/18 requests passed, including 12 video requests, at TP=2.
Nano/Super generation: 8/8 cases passed.
Source checks: 22 passed; ModelOpt 0.44.0 CPU smoke passed.
No confirmed product defects. TP=4 and full-model requantization remain unverified.

@lfengad