[python] Add RoboMIND AgileX HDF5 pipeline

[YannByron]

Summary

Add a RoboMIND AgileX sample that uses the same HDF5 transform contract for local and Ray ingestion. The pipeline writes episode and frame tables, materializes canonical actions, and refreshes train-split statistics independently.

Changes

• Add pypaimon.ray.load_from_hdf5 with shared HDF5 discovery, transform validation, and a single coordinated Paimon commit.
• Add separate RoboMIND AgileX episode and frame transforms for local and Ray ingestion.
• Keep deletion vectors and Vortex vector storage enabled while storing image and depth payloads as raw bytes.
• Split canonical action materialization from statistics refresh and process frame updates by Paimon split.
• Generate multi-episode AgileX HDF5 fixtures in pytest, with an optional flag for downloaded customer data.
• Document the dataset layout, split semantics, table options, local usage, and Ray usage.

Testing

• python -m pytest -q pypaimon/tests/multimodal_hdf5_test.py pypaimon/tests/ray_hdf5_test.py pypaimon/tests/robomind_agilex_pipeline_test.py (29 passed, 1 skipped)
• Ruff on all changed Python files
• git diff --check origin/master...HEAD
• Customer dataset test with --robomind-agilex-input (requires a downloaded RoboMIND dataset)

Notes

This builds on the HDF5 DataSource ingestion merged in #9411. Repeating ingestion appends duplicate rows by design; a future merge-into mode can provide episode-level upserts.

[JingsongLi]

Inline findings on the current PR head.

[JingsongLi]

[P1] Read language metadata from the published sidecar annotations

The published AgileX trajectory.hdf5 layout contains the master, puppet, and observations datasets, but not language_raw or language_distilbert; RoboMIND publishes task-language annotations separately in CSV/JSON. Following the documented --input .../h5_agilex_3rgb path therefore fails on the first file here. The synthetic fixture masks this by adding both datasets itself. Please load/derive the instruction from the official sidecar/path metadata and make the embedding external or optional, then cover the actual published HDF5 shape in a test. References: official AgileX HDF5 layout, RoboMIND dataset card.

[JingsongLi]

[P1] Reuse one row-id file index across all splits

Each update_by_arrow_with_row_id call constructs a new TableUpdateByRowId; its constructor runs plan_for_write() and rebuilds the index of every current data file. Because that happens inside the split loop, N splits perform N full manifest/file scans, making the backfill approximately quadratic in split count. This is material for the full AgileX dataset (10,629 trajectories, generally hundreds of frames each). Please plan/index the snapshot once and reuse a batch-capable updater across the streamed splits while retaining the single final commit.

[JingsongLi]

[P2] Pin the intended vector file format

This sample, its tests, and the documentation expect Vortex, but current master changed the multimodal default to Parquet in #9444. Since TABLE_OPTIONS does not set vector.file.format, the merged code will inherit Parquet and the Vortex assertion will fail once CI runs against the updated base. Please either set vector.file.format=vortex explicitly (and document the pypaimon[vortex] / Python >= 3.11 requirement), or accept Parquet and update the tests and documentation accordingly.

[JingsongLi]

[P2] Make the schema-change stage retryable

alter_table durably adds the nullable action column before the row-id update is committed. If update generation, commit, or the subsequent snapshot lookup fails, a retry immediately hits _validate_backfill_target and rejects the existing column, leaving the warehouse stuck with an empty or incomplete action. Please treat an existing compatible column as a resumable state and rerun the deterministic row-id update (or persist explicit stage state), and add a failure-injection retry test.