perf(rbd): flat 1-D narrow-phase dispatch — pack warps across batches - #21
perf(rbd): flat 1-D narrow-phase dispatch — pack warps across batches#21haixuanTao wants to merge 3 commits into
Conversation
The narrow-phase kernels dispatched [max_len/64, num_batches, 1]: one 64-lane workgroup per batch rounded up from that batch's live pair count. A batched robot env has ~7 pairs, so at 2048+ envs the GPU ran thousands of workgroups at ~11% lane occupancy and narrow-phase scaled ~linearly with env count (0.02 ms at 1 env -> 18.6 ms at 4096). gpu_flatten_batches_dispatch (one thread, same style as the existing max-scan init kernels) now builds exclusive prefix offsets over the per-batch work-lists plus a flat [total/64, 1, 1] grid; the classify, deferred and PFM kernels walk 0..total and recover (batch, item) with a binary search over the offsets. Warps fill with real pairs from consecutive batches. Buffer layout is unchanged — only the dispatch shape and index math moved. RTX 5090, 12-DOF biped batch, dt=5ms, steady-state contacts: narrow-phase 2048 envs: 10.31 -> 5.09 ms 4096 envs: 18.57 -> 6.94 ms whole step 2048 envs: 22.45 -> 16.50 ms/step (91k -> 124k env-steps/s) Physics bit-identical (robot trajectory + cube settle unchanged to printed precision). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…narrow phase Collision pairs carry env-local collider ids, and collider_parent is batch-strided like every other per-collider buffer (its construction comment says so: 'Env-local body slot; the kernels apply the per-batch stride') — but the classify and pfm_pfm kernels read it unsliced, resolving every batch's parents through batch 0's table. With identical environments the tables coincide and nothing observable goes wrong — which is why every bit-exactness check passed. With heterogeneous environments (the point of per-env MJCF insertion) contacts are silently mis-parented: a pair that is same-body in batch 0 gets skipped in batches where it isn't, and solved impulses can target the wrong bodies. Repro (now a stacking test): env0 = body with two glued boxes + a single box; env1 = single box dropped onto a two-glued-box body (equal body and collider counts, different parent tables). Pre-fix the falling box never rests on the stack; post-fix it settles on top (z=0.75). Identical-env scenes are bit-identical before/after, as the aliasing argument predicts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Pushed an additional commit with a correctness fix found while stress-testing heterogeneous environments: the classify and pfm_pfm kernels read |
The flat narrow-phase kernels (post-dimforge#21 flatten) thread purely off `global_invocation_id.x` with `num_threads = num_workgroups.x * WORKGROUP_SIZE` — the grid y/z dims are never read. But the FIXED fallback grid was still the stale 2-D `[x, num_batches, 1]` from before the flatten (the indirect path was updated to 1-D by flatten_batches; the fixed path was not). Every one of the `num_batches` y-slices re-ran the identical flat grid-stride loop, so the atomic-append emitters — deferred (site 7) and pfm_pfm (site 8) — wrote each contact `num_batches` times. That exploded the contact buffers and graph coloring: ~19s/step at 2048 batches with the GPU pinned (the long-standing "fixed-grid pathology" that forced sites 7+8 onto indirect dispatch; also the historical >16-batch illegal address). Fix: size the fixed grid 1-D over the whole flat capacity, matching the indirect path. Verified: sites 7+8 on fixed dispatch now run at 2.0 s/iter (was pathological), iter-0 falls within baseline noise. Flips both sites back to fixed dispatch, removing their per-step host sync. Also hoist the fused-color `num_colors_uniform` update out of a per-step `Tensor::scalar` realloc (a cudaMalloc inside step(), illegal under CUDA-graph capture) to an in-place stream-ordered `write_buffer`. Both are prerequisites for restoring graph capture on the unified stack. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2n9RqmxTJb8UG5d1Sjw4W
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(rbd): apply the per-batch stride to collider_parent reads in the narrow phase Replaces #21 Co-Authored-By: Haixuan Xavier Tao <tao.xavier@outlook.com> * feat: per-environment collision-pair capacity override Replaces #24 Co-Authored-By: Haixuan Xavier Tao <tao.xavier@outlook.com> * feat(rbd): make the narrow-phase contact prediction distance configurable Replaces #28 Co-Authored-By: Haixuan Xavier Tao <tao.xavier@outlook.com> * fix(rbd): thread the configurable prediction distance through the brute-force broad phase Completes #28 * feat(python): per-environment MJCF insertion Replaces #16 Co-Authored-By: Haixuan Xavier Tao <tao.xavier@outlook.com> * fix(python): drop the duplicated collisions-capacity setter and pass the RbdCoupling to insert_rigid_body_in Completes #16 * feat(python): per-step MJCF actuator control + multibody state readback Replaces #12 Co-Authored-By: Haixuan Xavier Tao <tao.xavier@1ms.ai> * fix(python): gate multibody control/readback on dim3, add the missing PyArray2 import Replaces #12 Co-Authored-By: Haixuan Xavier Tao <tao.xavier@outlook.com> * fix(rbd): decode the SoA link workspace for the multibody readback and drop the stale set_gravity copy Completes #12 * perf(rbd): dedupe shared TriMesh uploads in from_rapier Replaces #19 Co-Authored-By: Haixuan Xavier Tao <tao.xavier@outlook.com> * perf(rbd): optional GPU contact reduction, merging per-pair manifolds to <=4 points Replaces #17 Co-Authored-By: Haixuan Xavier Tao <tao.xavier@outlook.com> * fix(rbd): pass the prediction distance to manifold_reduction in the contact-reduction kernel Completes #17 * perf(rbd): flat 1-D narrow-phase dispatch, packing warps across batches Replaces #21 Co-Authored-By: Haixuan Xavier Tao <tao.xavier@outlook.com> * fix(rbd): restore the contacts capacity binding and import atomic_load_u32 for the flat dispatch Completes #21 * fix(rbd): drop the stale 2mm PREDICTION constant reintroduced by the flat-dispatch port Completes #21 * fix mpm feature-gating * feat(rbd): expose dof_state_mut, links_static, joint_constraints and link_of_body * feat(rbd): env-reset primitives, GPU motor scatter, contact sensors, actuator delay, encoded step, substep-refresh cadence and per-DoF armature/frictionloss * fix(rbd): guard against implicit-coriolis drifting from the batch_indices uniform * feat(rbd): cluster contact manifolds by normal, matching rapier, with a tunable threshold * feat(rbd): model multibody joint frictionloss as a constraint instead of a force * feat(rbd): seed per-DoF joint friction from rapier's Multibody::frictions * refactor(rbd): read the contact prediction distance from RbdSimParams instead of a dedicated uniform * chore: cargo fmt * refactor(rbd): move the contact merge cosine into RbdSimParams * refactor: move read_multibody_links onto NexusState and drive every env from control_multibody_motors * test(rbd): add a headless many-small-environments step-timing harness * revert(rbd): drop the flat 1-D narrow-phase dispatch Measured 7-21% slower on Metal. * chore: cleanup comments * fix: gate control_multibody_motors on dim3 so the 2D build still compiles * fix instability in joint-ball3 demo * chore: remove debug test files * chore: cleanups * fix(rbd): build the bench harness without the metal feature and only in 3D * fix(rbd): silence the clippy needless-borrow and unnecessary-mut lints * fix(rbd): split the joint-constraint back-solve into its own dispatch to fit 8 storage buffers * test(rbd): keep the bench harness under wgpu's default buffer-size limit * fix(rbd): split the batched env reset into pose and DoF passes to fit 8 storage buffers * chore: switch to the published rapier version * fix: make all envs share the same RbdSimParams * chore: clippy fixes --------- Co-authored-by: Haixuan Xavier Tao <tao.xavier@outlook.com> Co-authored-by: Haixuan Xavier Tao <tao.xavier@1ms.ai>
Problem
The narrow-phase kernels dispatch
[max_len/64, num_batches, 1]: one 64-lane workgroup per batch, rounded up from that batch's live pair count. A batched robot environment has ~7 collision pairs, so at 2048+ envs the GPU runs thousands of workgroups at ~11% lane occupancy, and narrow-phase time scales almost linearly with env count (0.02 ms at 1 env → 18.6 ms at 4096 on the scene below) — it goes from negligible to the single largest pass in the frame.Change
A new one-thread kernel (
gpu_flatten_batches_dispatch, same style and cost as the existing max-scan init kernels) builds exclusive prefix offsets over the per-batch work-lists plus a flat[total/64, 1, 1]indirect grid. The classify, deferred, and PFM kernels walk0..totaland recover(batch, item)with a binary search over the offsets (≤12 steps at 4096 batches). Warps fill with real pairs from consecutive batches instead of one batch's handful plus idle lanes.Buffer layout is unchanged — per-batch capacity-strided storage stays as is; only the dispatch shape and index math moved. The same flatten kernel is reused for both the collision-pair and the PFM work-lists, repurposing the two existing indirect buffers (nothing else consumes their batched form).
Results
RTX 5090, batched 12-DOF biped (one robot per env, mesh feet), dt = 5 ms, steady-state contacts, wgpu/Vulkan:
Physics is bit-identical before/after (robot trajectory and cube-settle checks match to printed precision), as expected — the same items are processed, only warp packing changes. Single-env scenes see no change (the flat grid degenerates to the old one).
The remaining narrow-phase cost is the per-pair GJK/EPA divergence itself; the same per-batch dispatch pattern also exists in the contact-solver kernels (
contacts_grid), which this PR deliberately leaves untouched — planned as a follow-up.🤖 Generated with Claude Code