[ooctipus]

Superseded by three focused PRs:

1. Separate recorder capture from batched episode storage #8822 — shared recorder capture and optional batched storage.
2. Add native offline NeRD for Cartpole and ANYmal C #8825 — offline NeRD with Cartpole and flat/rough ANYmal C.
3. Integrate native SuperMimic training and G1 tracking #8826 — native SuperMimic training and G1 tracking.

PRs #8825 and #8826 each depend on #8822; neither depends on the other. All branches are in ooctipus/IsaacLab and target upstream main. Their descriptions link feature-only comparisons while the shared dependency is unmerged.

The split passed 126 focused tests, full formatting/changelog gates and full warnings-as-errors documentation builds. The original branch is preserved, and the original implementation, training evidence and limitations remain below for reference.

---

Description

Integrate three learned-dynamics workflows into Isaac Lab: offline NeRD for Cartpole and ANYmal C, and online SuperMimic tracking for G1. This PR includes the shared recorder/capture implementation, learned Newton solver, task configurations and native training entrypoints together.

The workflow guide covers preparation, recording, dynamics fitting, policy training/evaluation, W&B checkpoint uploads and resume.

Architecture and review order

1. Shared capture: core RecorderManager owns batched episode storage with independent world indices. Native recorder terms publish preallocated transition tensors before autoreset; consumers choose streaming-only capture or episode retention. Replay and dataset writing stay outside capture. No TransitionCapture dispatcher or isaaclab_physicsnemo extension is introduced.
2. Shared offline NeRD: isaaclab_newton.physics.nerd owns the generalized-coordinate representation and physical model contract. NewtonNeRDManager owns learned physics through the normal solver lifecycle, with fixed history/reset storage and CUDA graphs enabled. Warp owns Newton state writes. isaaclab_rl.nerd owns offline fitting and episode-aware sampling. Live Newton contacts use the same PhysicsNeMo encoder at capture and deployment; fitting normalizes only valid continuous geometry while preserving body identities and masks.
3. Cartpole and ANYmal C: registered teacher, -Explore and -Learned configurations live in isaaclab_tasks.core. scripts/learning/nerd/record.py and train.py collect native HDF5 episodes and fit resumable models. Standard reinforcement-learning train.py / play.py handle PPO and transfer back to MuJoCo Warp. ANYmal's temporal network recipe lives beside its task. NeRD consumes only applied motor effort from Control.joint_f, copied immediately before the solver. Actuators own targets, internal state, damping and clipping. Canonical ANYmal tasks retain ANYdrive and execute four 5 ms learned physics steps per 20 ms policy action; Cartpole uses actuator-owned damping and clipping. Rough ANYmal adds live contact sets, native terrain-aware reset sampling, and teacher/exploration/learned configurations; the collector can use random actions or an exported feedforward policy.
4. SuperMimic G1: the native IsaacContrib-Motion-Tracking-G1 task owns motion, geometry, capture and differentiable task semantics. isaaclab_rl.supermimic owns replay and optimization; the existing RL entrypoints dispatch --rl_library supermimic. It reuses the recorder lifecycle and optional PhysicsNeMo ParallelNeRD backbone while retaining its distinct, validated state representation and online objective.

A file-by-file ownership audit and negative architecture gates reject external training forwarders, duplicate task stepping, task-to-learner imports and class-level Newton state mutation.

Core changes and dependencies

The branch includes the unmerged recorder/storage prerequisite and finite quaternion-gradient fix. Follow-up fixes preserve retained buffers across inference contexts and derive NewtonCfg.class_type when replacing a solver; existing Digit/custom-coupling workarounds are removed. Velocity playback also accepts a ground asset without a terrain importer. The branch targets upstream main directly from ooctipus/IsaacLab.

Breaking recorder API: get_episode() returns an owned tensor snapshot. Live writes use add_to_episodes() / set_success_to_episodes(); the annotation caller and migration guide are updated.

NeRD requires optional PhysicsNeMo Robotics. Validation used development revision a81160ef plus local attention-accumulation fix ee6d2e0, with PhysicsNeMo 2.1.1. The dependency fix disables autocast for FP32 attention reductions; a regression fails before it and passes afterward. This replaces the original integration's runtime class monkeypatch. The dependency repository denied push access, so that fix is not published yet; safe FP16 fitting remains dependent on it. A compatible released package combination also remains unresolved. SuperMimic's default MLP workflow needs neither PhysicsNeMo nor an external SuperMimic training package. Robot/motion data remain separately prepared inputs with pinned provenance and the upstream MIT notice.

Validation

• Four NeRD task/network combinations (Cartpole/ParallelNeRD, flat ANYmal/ParallelNeRD, flat ANYmal/VectorNeRD and rough ANYmal/ParallelNeRD with live contacts): actual terminal capture, exact live/stored tensor comparisons, HDF5 fitting, numerical optimizer resume, deployment with a different world count and visual-only geometry, eager/compiled prediction checks, repeated graph replay with asynchronous resets and intervening allocations, and rejection of changed physics/timing. Live versus stored capture is byte-exact; optimizer continuation is checked numerically. Regression checks independently verify damped/clipped actuator effort before post-step buffer reuse and reject target-conditioned descriptors and solver-side stiffness/damping drives. Mutation checks confirm that moving effort capture after the step or restoring a recorder control selector causes failures. Tests cover ParallelNeRD/VectorNeRD, FP16/scaler resume, four 5 ms effort-driven updates per policy step, and fixed-root/body-flag/collision-filter invalidation. Two-rank CPU training matches an independent combined-batch optimizer step and resumes distinct per-rank samplers; GPU/NCCL scaling is not claimed. The rough case also covers contact-conditioned resume, changed-terrain deployment, contact-history resets and changed-friction rejection.

• Fresh applied-effort-only CLI fitting completed with finite metrics and parameters on a local RTX 5090: flat ANYmal, 20,000 updates on 1,228,800 standing/walking/exploration transitions; Cartpole, 30,000 updates on 384,000 fresh transitions with actuator-owned damping. Held-out normalized MSE fell from 0.16785 at update 500 to 0.09169 for ANYmal, and from 0.001876 to 0.000003635 for Cartpole. These losses have different task normalizations and should not be compared across tasks.

• Both final checkpoints and all 16 dataset/configuration files were downloaded from W&B and verified byte-for-byte. Native CLI training resumed on the recovered data/checkpoints to updates 20,002 and 30,002, preserved normalization, and uploaded the next checkpoints to the same runs. Each recovered model then completed eight native RSL-RL PPO iterations in 32 worlds with CUDA graphs enabled and finite metrics/parameters. These are policy integration smokes, not policy convergence or wrench-accuracy results. Verification evidence includes scripts, commands, logs, hashes and PPO checkpoints. PPO logging was given an explicit WANDB_RUN_ID: the installed W&B 0.31.0 lacks wandb.util.generate_id, called by the existing Isaac Lab announcement helper.

• Full held-out flat-ground wrench evaluation of the new 20,000-update ANYmal checkpoint covers all 120,800 transitions (18,400 standing, 80,000 walking, 22,400 exploration). With actual motor effort fixed, experimental constrained reconstruction gives joint-force RMSE 1.96 / 12.98 / 9.92 N and joint-torque RMSE 0.121 / 2.014 / 1.031 Nm, respectively. Relative RMS errors are 2.71% / 12.85% / 11.42% for joint force and 1.63% / 14.88% / 8.85% for joint torque; body-parent force/torque results are included in the evaluation evidence. The earlier checkpoint on identical held-out frames gives walking force RMSE 12.65 N: this refit does not resolve walking wrench accuracy. The largest motor-balance residual is 0.0000444 Nm; the worst 0.1% of walking frames account for 55% of squared force error. This is one-step evaluation using recorded state/history and exact current contact geometry, referenced to MuJoCo Warp forward replay; free learned rollouts are not evaluated. The provisional 10% accuracy goal is unmet. The optimized standalone Warp implementation retains fixed applied effort and the same constrained objective, with FP64 matrix preparation and fused FP32 iterations. The sampled 128-world solve falls from 53.7 ms to 0.041 ms. Complete reconstruction medians over all held-out batches are 0.345 / 0.350 / 0.352 ms (standing/walking/exploration), versus native physics plus wrenches 0.300 / 0.681 / 0.762 ms. A separate 4,096-world sampled check gives 1.18 / 1.62 / 1.18 ms versus native 0.959 / 1.55 / 1.51 ms. CUDA graphs are enabled throughout; timings exclude neural inference. All 120,800 transitions pass the numerical gates: joint/body force RMSE changes by less than 0.001 N and torque RMSE by less than 0.0001 Nm, with maximum learned motor-balance residual 0.0000331 Nm. Independent original-Torch comparisons and synthetic tests cover 0–60 active constraint rows. Source, raw outputs, timing distributions and gates are archived in W&B as anymal-wrench-reconstruction-optimized; the summary was downloaded and verified byte-for-byte. This remains one-step prototype validation and does not enforce MuJoCo's soft-contact law. This diagnostic is not integrated into production sensors. Reports, raw per-frame arrays, source scripts and plots are archived with the evaluation.

• Output-component interventions on those same 120,800 held-out frames localize the flat-ground wrench error to velocity prediction. On walking, replacing only base linear velocity with recorded values reduces joint-force RMSE from 12.98 to 9.95 N; correcting all velocities reaches 2.83 N, close to the 2.82 N recorded-motion reconstruction baseline. Pose-only corrections have negligible benefit. A further 40,800-frame standing/exploration check identifies hip-flexion velocity as the dominant joint component for force error (1.96 to 0.38 N standing; 9.92 to 3.41 N exploration); exploration torque is more sensitive to hip-abduction velocity (1.03 to 0.62 Nm). These are conditional interventions, not additive error shares, and do not establish why the network misses those velocities. Diagnostics, raw predictions and plots are archived; production source is unchanged.

• Before the version-2 checkpoint change, Cartpole logged to W&B, uploaded a checkpoint, downloaded byte-identical model tensors, resumed from update 32 to 34 and published the next durable checkpoint to the same validation run.

• Original fixed-foot contact features were rechecked against [Experimental] Add isaaclab_physicsnemo extension for NeRD and FlashNeRD learned physics #8472: 64/64 snapshots and 14,336 scalars are bit-identical, including 937 active probe samples.

• Rechecked recorder/configuration/G1/custom-coupling/discovery boundaries. The recorder still matches the frozen implementation across 1,224 snapshots, 7,792 tensor pairs and 1,272 HDF5 datasets on CPU/CUDA.

• 4,096-world G1 recorder recheck retained one physical CUDA graph launch per control step. Median isolated capture was 0.089 ms; retention insertion was 0.117 ms. This shared-host check is not an isolated end-to-end throughput claim.

• The existing ANYmal applied-effort checkpoint produces bit-identical joint-position/velocity predictions on the same saved inputs across 32 steps and asynchronous history resets. Whole teacher rollouts are not bit-exact across repeated runs. With CUDA graphs enabled, the isolated four-world inference/history/guard/forward-kinematics benchmark is effectively unchanged: median 0.567 ms before versus 0.564 ms after; this excludes actuator and collision time.

• Existing SuperMimic validation: 351 focused tests, 1,594 byte-identical reference tensor pairs, compiled MLP/NeRD train-save-resume, W&B artifact recovery, fresh 40-motion asset preparation and native evaluation/video. The reference audit covers eager numerical semantics, not compiler bitwise equivalence.

• Full repository formatting/lint and changelog checks, git diff --check, and a clean warnings-as-errors Sphinx build passed.

• Rough-terrain training completed 60,000 updates on 716,800 transitions, mixing random exploration and policy-driven samples, with CUDA graphs enabled for physics. The training run finished normally. Its final W&B checkpoint (nerd-jpze1zti:v30) was downloaded and verified byte-for-byte against the local file. The native rough learned task also completed two PPO iterations with 16 worlds using the intermediate checkpoint; this is an integration smoke, not policy convergence.

• The final rough-terrain evaluation includes scripts, traces and plots. Removing contacts increases generalized-velocity RMSE from 0.243 to 3.106 on 22,400 fixed held-out transitions. On unseen terrain, teacher-velocity reconstruction gives joint-wrench RMSE 0.039 N / 0.013 Nm; NeRD gives 1,229 N / 349 Nm for policy-driven motion and 672 N / 171 Nm for random exploration. Body-parent wrench errors are also large (1,785 N / 577 Nm in the policy-driven case). Recorded-effort replay with predicted collisions reaches 0.315 m root-position RMSE at 0.5 s. Learned wrench accuracy fails this validation, and force sensors remain unsupported. These results establish working contact conditioning and native training integration, not reliable learned sensors or long-horizon policy performance.

Scope and migration

ANYmal C supports flat-ground fixed probes and a separate rough-terrain live-contact contract. On rough terrain Newton collision detection remains enabled and NeRD replaces the numerical dynamics solve; ANYdrive and forward kinematics remain. The checkpoint fixes robot parameters and uniform external contact material while permitting static terrain geometry to change through live contacts. Contact capacity overflow is an error. The rough policy observations are proprioceptive; independently moving external objects and per-surface materials are outside this contract. Offline fitting supports ParallelNeRD/VectorNeRD, FP32/BF16/FP16 and standard torchrun distributed training. Data remains device-resident on each rank. FP16 requires the dependency fix noted above. Position and velocity controllers are supported when they publish final clipped effort before physics. Solver-side stiffness/damping drives and MuJoCo-specific actuators are rejected because they generate motor effort beyond this boundary. CUDA graphs remain enabled; ANYdrive currently steps outside the physics graph.

The old flat ANYmal -Effort-Explore/-Effort-Learned IDs are removed; their behavior is now the canonical -Explore/-Learned path, without aliases. Applied-effort checkpoints retain their physical/input contract. Target-conditioned checkpoints are rejected. Cartpole data/checkpoints from the previous solver-side damping configuration require recollection/refitting.

Native HDF5/checkpoints replace the experimental Zarr/custom-artifact launchers without compatibility aliases or automatic conversion. Version-2 checkpoints add architecture, scaler and per-rank sampler state; datasets with the old incomplete physical fingerprint need recollection/refitting. Independent inverse reconstruction closely reproduces numerical teacher joint/body-parent wrenches using motion, the physical model and contact law. The final rough-terrain NeRD checkpoint fails wrench-accuracy validation; learned force and joint-wrench sensors remain unsupported. Historical nine-hour/full-corpus results are not validation of this new native path, and this PR does not claim the paper's zero-failure result. Existing backend/dependency deprecations and compiler/video advisories remain.

Type of change

• New feature
• Breaking recorder snapshot API change
• Bug fixes for quaternion gradients, recorder lifetime and solver configuration
• Documentation update

Checklist

• Read the contribution guidelines
• Ran repository formatting and lint checks
• Updated documentation
• No runtime warnings (dependency/backend warnings noted above)
• Validated changed behavior and applied the test-audit criteria
• Added changelog fragments for every touched package
• Contributor name already appears in CONTRIBUTORS.md

The file-by-file audit retains 74 files (+6,151/-185 against the branch base). The applied-effort follow-up and generated task-browser cleanup change 13 existing files (+117/-145) and add none. Runtime validation used a60e77c0b; current head fdfea65aa only adds the generated browser cleanup. Four NeRD integration cases, two ownership/registration checks, both targeted mutation checks, full repository format/lint/changelog checks and the warnings-as-errors Sphinx build pass. Current-head CI remains in progress; no full CI pass is claimed.