[xyao-nv]

Add robot-specific-configured VLM observation and action adapters with DROID. Provide default absolute-pose goal execution using measured pose and gripper state.

Why this change?

The upstream VLM policy provides step, chunk, and goal execution interfaces, but callers must implement observation selection, command decoding, and motion generation. Add reusable implementations so it can use these interfaces directly to control specific robot, like DROID.

What changes?

• Add state, image, and combined image/state observation adapters configured from the embodiment.

• Add concrete step, chunk, and goal action adapters with command schemas.

• Define explicit ABS_EE_POSE and ABS_JOINT_POSITION command types.

• Default VLMGoalPolicy to PoseTargetMotionGenerator for absolute pose goals. PoseTargetMotionGenerator holds targets until measured translation, rotation, and gripper closure meet their tolerances.

Usage examples

Step: Request one command from the VLM and execute it for one environment step.

  step_policy = VLMStepPolicy(
      config=config,
      inference_backend=backend,
      observation_adapter=ImageStateObservationAdapter(embodiment),
      action_adapter=StepActionAdapter(embodiment, CommandType.ABS_EE_POSE),
  )

Goal: Request a target and execute it until measured state reaches it or the step limit is reached; then request another target.

  goal_policy = VLMGoalPolicy(
      config=config,
      inference_backend=backend,
      observation_adapter=ImageStateObservationAdapter(embodiment),
      action_adapter=GoalActionAdapter(
          embodiment,
          CommandType.ABS_EE_POSE,
          motion_generator=PoseTargetMotionGenerator(embodiment),
      ),
      max_steps_per_goal=50,
  )

Chunk: Request N=8 commands together and execute one per environment step before requesting another chunk.

  chunk_policy = VLMChunkPolicy(
      config=config,
      inference_backend=backend,
      observation_adapter=ImageStateObservationAdapter(embodiment),
      action_adapter=ChunkActionAdapter(
          embodiment, CommandType.ABS_EE_POSE, chunk_size=8,
      ),
  )

[xyao-nv]

/review

[cvolkcvolk]

Left some suggestions. Feel free to pass to your agent.