Repository navigation
Clarify raw vLLM video sampling requests - #349
sylvesterkaczmarek wants to merge 2 commits into
Conversation
Signed-off-by: Sylvester Kaczmarek <16242628+sylvesterkaczmarek@users.noreply.github.com>
pengcuo
left a comment
There was a problem hiding this comment.
Review notes (docs only; I did not run anything).
-
The raw HTTP example uses a model id that vLLM will not serve.
payload = {"model": "Cosmos3-Super", ...}. The setup guide serves withvllm serve nvidia/Cosmos3-Super(no--served-model-name), so the served id isnvidia/Cosmos3-Superand a request forCosmos3-Superreturns "model does not exist". The OpenAI snippet above it correctly usesclient.models.list().data[0].id. Usenvidia/Cosmos3-Superhere (ornvidia/Cosmos3-Nano, since the quickstart uses Nano on port 8000). -
It contradicts the notebook this README documents.
run_with_vllm.ipynbsamples video withextra_body={"media_io_kwargs": {"video": {"num_frames": -1, "fps": 4}}}. It does not usemm_processor_kwargs. The PR says no live vLLM request was run, so it is unverified which shape is right, and the two docs now describe different mechanisms. Please pick one and check it against a running server. If both are valid, say what each controls. Ifmedia_io_kwargsalready samples at 4 fps, addingdo_sample_frames: Truemay sample twice. -
Note
reasoner_prompt_guide.mdalready says "the notebook uses frame sampling throughextra_body...mm_processor_kwargs". That sentence is also not what the notebook does, so it should be fixed together with this.
Signed-off-by: Sylvester Kaczmarek <16242628+sylvesterkaczmarek@users.noreply.github.com>
|
Thanks for the detailed review, @pengcuo. You were right about the raw model ID and the mismatch with the actual notebook. Commit |
Resubmits NVIDIA/cosmos#337 at the new Cosmos Framework cookbook path after NVIDIA/cosmos#384 moved the file to
.agents/skills/.The Reasoner quickstart and prompt guide now show video frame sampling via
media_io_kwargs={"video": {"num_frames": -1, "fps": 4}}in both OpenAI clientextra_bodyand direct/v1/chat/completionsJSON. This matchesrun_with_vllm.ipynb(and avoids suggesting two independent sampling paths at once). The raw JSON uses the model IDnvidia/Cosmos3-Super, consistent with the documented vLLM launch. Addresses NVIDIA/cosmos#190.Validation: Documentation-only; checked the actual notebook cells, model-serving instructions and guide for consistency, parsed the Python examples, and passed
git diff --check. No live vLLM inference or model-serving claim. Maintainer verification on a running server is still welcome.DCO signed off. Thanks @yaoxu-crypto for the migration guidance and invitation to resubmit.