From 0e470afe99b98f9059bdf017b49043e635d404f8 Mon Sep 17 00:00:00 2001 From: Sylvester Kaczmarek <16242628+sylvesterkaczmarek@users.noreply.github.com> Date: Sat, 31 Oct 2026 01:03:17 +0000 Subject: [PATCH] Clarify action dimensions and AgiBotWorld URDF scope Signed-off-by: Sylvester Kaczmarek <16242628+sylvesterkaczmarek@users.noreply.github.com> --- .../cosmos3/generator/action/README.md | 25 +++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/.agents/skills/cookbooks/cosmos3/generator/action/README.md b/.agents/skills/cookbooks/cosmos3/generator/action/README.md index c89eb9f5f..3a1fb3194 100644 --- a/.agents/skills/cookbooks/cosmos3/generator/action/README.md +++ b/.agents/skills/cookbooks/cosmos3/generator/action/README.md @@ -65,8 +65,33 @@ fingers. | UMI | End-effector pose (9D) + gripper grasp state (1D) | 10D | Meter | Normalization | 16 frames @ 20FPS | | Human hand pose | Ego pose (9D) + right wrist (9D) + right fingertips (15D) + left wrist (9D) + left fingertips (15D) | 57D | Meter | Wrist-frame alignment, normalization | 16 frames @ 15FPS | +Action dimensions are tied to the selected embodiment. `domain_name` chooses a registered +action domain, and Cosmos Framework validates forward-dynamics input width against that +domain's canonical raw action dimension. Changing a 10D DROID vector to 12D or 18D, for +example, is not supported by the released checkpoints simply by changing the input shape. +A custom action layout requires a compatible registered embodiment/domain and model weights +trained for that action representation. Some dataset-defined domains, such as hand pose and +LIBERO, can resolve their width from dataset configuration for forward dynamics; this does +not make arbitrary widths interchangeable within a trained domain. See the +[Cosmos Framework action-domain registry](https://github.com/NVIDIA/cosmos-framework/blob/main/cosmos_framework/data/generator/action/utils/domain_utils.py) +for the currently registered canonical widths. + Action data samples across different embodiments can be inspected interactively in the [Cosmos3 Action Viewer](https://huggingface.co/spaces/nvidia/Cosmos3-Action-Viewer) Hugging Face Space. +### AgiBotWorld robot description + +The public Cosmos Framework AgiBotWorld forward-kinematics path currently uses the +committed [`G1_omnipicker_calibrated.urdf`](https://github.com/NVIDIA/cosmos-framework/blob/main/cosmos_framework/data/generator/action/robot_assets/G1_omnipicker_calibrated.urdf) +as its calibrated kinematics reference. The URDF contains `package://` references to +visual meshes, but those mesh files are not bundled in the framework's `robot_assets` +directory. They are not required by the action FK path: the loader removes `` +and `` elements before constructing its MuJoCo kinematics model. + +Accordingly, the committed file should be treated as the kinematics reference for the +registered omnipicker-based AgiBotWorld action path, not as a complete renderable robot +asset package or a generic description for every AgiBotWorld gripper variant. The public +framework does not currently include a separate CTEK URDF or the referenced mesh package. + ## Run with Cosmos Framework ### Quickstart