Skip to content

Conveyor stack integration draft - #6952

Draft
maxkra15 wants to merge 127 commits into
isaac-sim:mainfrom
maxkra15:maximiliank/conveyor-franka-env
Draft

maxkra15 wants to merge 127 commits into
isaac-sim:mainfrom
maxkra15:maximiliank/conveyor-franka-env

Conversation

@maxkra15

@maxkra15 maxkra15 commented Aug 6, 2026 •

Copy link
Copy Markdown
Contributor

Conveyor integration draft

Do not merge this PR. Review the component PRs individually; PhysX is an optional final layer.

PR Scope Included head
#8327 Experimental conveyor support in Contrib 619e4d24e125
#8331 Racetrack transfer task b702f94a3074
#8332 Warehouse sorting task 8ff9bf90dbc2
#8794 Optional native CPU PhysX support fbe9220ded23

Integration: ef7889e0f737. Base: 259e1a8cf79d. #8326 is superseded: both backends use the existing in-memory mesh spawner. No new mesh API or terrain importer is needed.

Validation

  • 38 focused cases passed: 13 task/geometry, 4 warehouse assets, 15 Newton/spec, 5 optional PhysX and the existing native triangle-mesh owner case
  • Native mesh replacement matches 90 mesh inputs and 16 finalized Newton geometry/material arrays; stock cuboid colors match all six original callers
  • Single-pass visual stripping preserves physics ownership; normalized warehouse geometry/materials/cameras match after correcting the old duplicate robot height
  • 84 action/tool/reset tensors and 261 dispatcher tensors match; existing keepers reject duplicate robot elevation and cross-environment slot broadcast mutations
  • Newton GPU checkpoint: eight environments, 1200 steps, 15 transfers, zero failures/non-finite observations
  • Optional PhysX native probe: two CPU environments, 135 steps, 16 surfaces, no fall-through; partial/full/hard reset and repeated close passed
  • Full formatting/lint/changelog/LFS, lock consistency and generated environment-browser checks passed
  • Clean strict Sphinx HTML build passed; existing JPEG previews retained

Warehouse policy limitation: Warehouse checkpoint model_1100: paired corrected historical/new scenes each completed zero transfers in two environments over 2400 steps (seed 41), with zero failures/non-finite observations; sorting capability remains unvalidated.

Refresh with:

uv run --no-project python tools/conveyor_stack.py --check --push

Add --checkpoint /path/to/model.pt for bounded racetrack evaluation. Refresh fetches current main and exact component heads, validates a temporary checkout and stops on merge/test failure.

@maxkra15
maxkra15 marked this pull request as ready for review August 6, 2026 23:52
@maxkra15
maxkra15 requested a review from a team August 6, 2026 23:52
@github-actions github-actions Bot added the isaac-lab Related to Isaac Lab team label Aug 6, 2026
@greptile-apps

greptile-apps Bot commented Aug 6, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

[Medium risk] Adds conveyor task environments and surface velocity physics features.

The PR is not ready to merge because default video recording on PhysX selects a Newton-only capture path.

Findings

  1. P1 Parcel counters survive batch resets ▶

Summary

The PR adds shared conveyor surface controls and Franka racetrack-transfer and warehouse-sorting tasks, with a CPU PhysX reference. Since the previous review, it also revises training resets and checkpoint handling, adds bounded evaluation, and changes runtime, installation, and documentation workflows.

  • Warehouse batch resets now clear parcel counters.
  • Default video capture still needs to select a source compatible with PhysX.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Task configuration] --> B[Launch simulation]
  B --> C{Physics backend}
  C -->|Newton| D[Newton surface-force adapter]
  C -->|PhysX CPU| E[PhysX surface-velocity adapter]
  D --> F[Conveyor environment]
  E --> F
  F --> G[Transfer or sorting command]
  G --> H[Policy observations and actions]
Loading

Reviews (5) · Last reviewed commit: "Merge latest develop and migrate conveyo..."

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

The public custom-mesh spawner and Newton conveyor task are coherently structured, but three scoped fixes are needed before merge: limit the new displayColor behavior so existing mesh spawners do not change rendering semantics, preserve supported environment constructor keywords such as render_mode, and apply the repository-mandated copyright header to the new task files.

  • Design and architecture: The separation between procedural geometry, the Warp force driver, environment configuration, and the reusable MeshCustomCfg spawner is sound. However, renderer-independent color authoring was added to the shared mesh helper, unintentionally extending the behavior beyond custom meshes and creating inconsistent color/material handling for existing spawners.
  • API: MeshCustomCfg and spawn_mesh_custom are consistently exported and documented through the public stubs and changelog. The task environment constructor currently discards all Gym-provided keyword arguments, narrowing the base ManagerBasedRLEnv constructor contract and breaking paths that rely on render_mode; supported arguments should be forwarded.
  • Implementation: The custom-mesh validation, collision configuration, racetrack generation, and force-driver organization are internally coherent. Required corrections are to scope displayColor authoring appropriately, retain the kitless warning when material properties remain unapplied, and update the new files to the required 2022-2026 copyright header.

Minor fixes needed. Posted 3 actionable findings inline.

Automated review; human maintainers own approval decisions.

Comment thread source/isaaclab/isaaclab/sim/spawners/meshes/meshes.py Outdated
Comment thread source/isaaclab_tasks/isaaclab_tasks/contrib/conveyor_franka/__init__.py Outdated
@maxkra15

maxkra15 commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

@greptileai check again

@maxkra15
maxkra15 marked this pull request as draft August 8, 2026 00:56
@maxkra15
maxkra15 force-pushed the maximiliank/conveyor-franka-env branch from 6efde78 to 66de12d Compare August 10, 2026 23:04
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Aug 14, 2026
@maxkra15 maxkra15 changed the title Add force-driven Newton conveyor task Add conveyor Franka task with Newton and CPU PhysX backends Aug 14, 2026
@maxkra15
maxkra15 marked this pull request as ready for review August 15, 2026 00:14
@kellyguo11 kellyguo11 moved this to In progress in Isaac Lab Sep 2, 2026
@maxkra15 maxkra15 changed the title Add conveyor Franka task with Newton and CPU PhysX backends Add conveyor Franka tasks with Newton and CPU PhysX Sep 24, 2026
@maxkra15

Copy link
Copy Markdown
Contributor Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 28, 2026
Comment on lines +53 to +55
def reset(self, env_ids: Sequence[int] | torch.Tensor) -> None:
"""Restore reset-recipe slot identities for selected environments."""
self.slot_ids[env_ids] = torch.arange(CUBE_COUNT, device=self.slot_ids.device)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Parcel counters survive batch resets

When a warehouse environment resets for a new 24-parcel batch, this method restores its four slot IDs but leaves transfer_counts and assignment_counts from the previous batch. The new batch therefore inherits old transfer metrics, and those old assignment counts change which arriving parcels the dispatcher prioritizes. Clear both counters for the selected environments when resetting the slots.

Suggested change
def reset(self, env_ids: Sequence[int] | torch.Tensor) -> None:
"""Restore reset-recipe slot identities for selected environments."""
self.slot_ids[env_ids] = torch.arange(CUBE_COUNT, device=self.slot_ids.device)
def reset(self, env_ids: Sequence[int] | torch.Tensor) -> None:
"""Restore slot identities and clear batch counters for selected environments."""
self.slot_ids[env_ids] = torch.arange(CUBE_COUNT, device=self.slot_ids.device)
self.assignment_counts[env_ids] = 0
self.assignment_counts[env_ids, :CUBE_COUNT] = 1
self.transfer_counts[env_ids] = 0

@maxkra15 maxkra15 changed the title Add conveyor Franka tasks with Newton and CPU PhysX Add conveyor racetrack transfer and warehouse sorting tasks Oct 1, 2026
@maxkra15

Copy link
Copy Markdown
Contributor Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Oct 25, 2026
Integrate upstream main e907f82
to keep this branch current while preserving its feature scope.
Use the clone configuration owner for existing conveyor namespace
defaults. This preserves the public values while satisfying the current
clone-ownership check across the integrated task stack.

Recorded upstream main: e907f82

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation infrastructure isaac-lab Related to Isaac Lab team

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

4 participants