Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
e7580cd
Rename modifiers to tensor processors and add dictionary processing
mataylor-nvidia Oct 16, 2026
4d3c5e8
Centralize modifier deprecation compatibility
mataylor-nvidia Oct 16, 2026
93aa169
Replace modifier aliases with migration errors
mataylor-nvidia Oct 19, 2026
51ecf64
Remove the legacy observation modifiers field
mataylor-nvidia Oct 19, 2026
72f3f19
Integrate dictionary processors with observation groups
mataylor-nvidia Oct 19, 2026
32ad9f0
Apply dictionary processors to individual observation terms
mataylor-nvidia Oct 19, 2026
29edf1a
Finish environment teardown after processor cleanup errors
mataylor-nvidia Oct 19, 2026
b794e75
Fix observation type annotations for Sphinx imports
mataylor-nvidia Oct 19, 2026
0ae9669
Preserve dictionary observation schemas after validation failures
mataylor-nvidia Oct 19, 2026
12d30f4
Simplify dictionary observation Gym spaces
mataylor-nvidia Oct 19, 2026
1a73657
Allow optional output parameters on dictionary observation terms
mataylor-nvidia Oct 19, 2026
db9a1f9
Keep observation terms tensor-valued
mataylor-nvidia Oct 20, 2026
236c3ae
Add dictionary tensor processor configuration and interface
mataylor-nvidia Oct 20, 2026
9df69eb
Resolve nested dictionary processor parameters
mataylor-nvidia Oct 20, 2026
d44e2ba
Use tensor processor terminology consistently
mataylor-nvidia Oct 20, 2026
3ae0408
Merge branch 'main' into feat/tensor-processors
ooctipus Oct 20, 2026
9127465
Preserve modifier imports with deprecation warnings
mataylor-nvidia Oct 20, 2026
e5f4c0a
Restore None return type for Warp observation terms
mataylor-nvidia Oct 21, 2026
b924f27
Merge branch 'main' into feat/tensor-processors
mataylor-nvidia Oct 21, 2026
383f324
Migrate LEAPP export fixture to tensor processors
mataylor-nvidia Oct 21, 2026
85a3465
Unify tensor and structured data processors
mataylor-nvidia Oct 22, 2026
ee9d78d
Address tensor processor review feedback
mataylor-nvidia Oct 23, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/source/api/lab/isaaclab.managers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ On each recorded sample, each environment retains its lag with probability 0.8 a
one. Zero redraws on every sample. Holding the lag keeps the latency constant while frames continue to
advance; it does not freeze the returned frame. Reset always redraws the selected environments' lags.

The processing order is observation function, modifiers, noise, clipping, scaling, delay, then history.
The processing order is observation function, tensor processors, noise, clipping, scaling, delay, then history.
The ``delay_min_lag`` and ``delay_max_lag`` field names and delay placement follow
`mjlab's observation configuration <https://mujocolab.github.io/mjlab/main/source/observations.html>`_.
``delay_hold_prob`` follows mjlab's lag-retention semantics, with a default of 1.0 to preserve Isaac Lab's
Expand All @@ -115,7 +115,7 @@ For a delay buffer with no recorded sample after initialization or reset, the cu
without recording it. Once recording starts, delays exceeding the available history return the oldest
sample. Partial resets invalidate only the selected environments' histories.

Observation outputs are independent snapshots: later simulation steps, modifier updates, and resets
Observation outputs are independent snapshots: later simulation steps, processor updates, and resets
do not overwrite previously returned tensors. The manager automatically copies borrowed storage before
mutation or retention. Term and custom callback outputs are treated conservatively, without decorators
or task-level copy settings. Clipping and scaling create independent storage when needed, which later
Expand All @@ -131,7 +131,7 @@ The manager checks the signature once during initialization. Only an explicitly
``out=None`` enables destination writes; accepting arbitrary ``**kwargs`` does not. The manager probes
the ordinary call to infer shape, dtype and device, which must remain constant, then allocates a fresh
contiguous destination for each computation. ``out`` is reserved and must not appear in term ``params``.
Custom modifiers, noise and history retain their usual snapshot protections. Built-in
Custom tensor processors, noise and history retain their usual snapshot protections. Built-in
:class:`~isaaclab.envs.mdp.observations.image_rgb` terms use this interface automatically to avoid a
second full-image copy for normalized uint8 images. Existing observation configurations require no changes.

Expand Down
75 changes: 63 additions & 12 deletions docs/source/api/lab/isaaclab.utils.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,11 @@
logger
math
mesh
modifiers
noise
seed
sensors
string
tensor_processors
timer
types
version
Expand Down Expand Up @@ -124,17 +124,6 @@ Mesh operations
:imported-members:
:show-inheritance:

Modifier operations
~~~~~~~~~~~~~~~~~~~

.. automodule:: isaaclab.utils.modifiers
:members:
:imported-members:
:special-members: __call__
:inherited-members:
:show-inheritance:
:exclude-members: __init__, func

Noise operations
~~~~~~~~~~~~~~~~

Expand Down Expand Up @@ -166,6 +155,68 @@ String operations
:members:
:show-inheritance:

Tensor processors
~~~~~~~~~~~~~~~~~

Use :mod:`isaaclab.utils.tensor_processors` to transform tensors or nested tensor collections.
:class:`~isaaclab.utils.tensor_processors.TensorProcessor` and
:class:`~isaaclab.utils.tensor_processors.TensorProcessorCfg` support ``torch.Tensor``,
:class:`~isaaclab.utils.warp.ProxyArray`, and recursively nested lists, tuples, and dictionaries
through ``TensorProcessorData``. Each implementation defines the structures it accepts and returns;
the base class does not traverse collections automatically.

Set ``cfg.func`` to a function, a processor class, or a lazily resolved import string. Functions
receive ``cfg.params`` as keyword arguments; classes read settings from their configuration.
Optional ``inputs`` and ``outputs`` lists describe named keys but do not select or route data.
For example, a subclass can stack a nonempty list of dictionaries whose ``position`` tensors
share a shape, dtype, and device:

.. code-block:: python

import torch
from isaaclab.utils.tensor_processors import TensorProcessor, TensorProcessorCfg


class StackPositions(TensorProcessor):
def __init__(self, cfg: TensorProcessorCfg, data: list[dict[str, torch.Tensor]]) -> None:
super().__init__(cfg)
first = data[0]["position"]
self.output = torch.empty((len(data), *first.shape), dtype=first.dtype, device=first.device)

def __call__(self, data: list[dict[str, torch.Tensor]]) -> torch.Tensor:
return torch.stack([item["position"] for item in data], out=self.output)


data = [{"position": torch.zeros(3)}, {"position": torch.ones(3)}]
cfg = TensorProcessorCfg(func=StackPositions, inputs=["position"])
processor = cfg.func(cfg, data)
stacked_positions = processor(data)

Construction uses input metadata without reading values or advancing state. Subsequent calls use
the same number of input dictionaries and tensor shapes. Inputs are read-only and owned by the
caller. This subclass owns its persistent output, which remains valid until the next call; copy
results that must be retained. ``reset`` and ``close`` do nothing unless overridden. Processors
using Warp launch work on the caller's current Warp stream.

Observation terms continue to use shape-preserving single-tensor processors through
:attr:`~isaaclab.managers.ObservationTermCfg.tensor_processors`. The observation manager constructs
class processors with ``cfg.func(cfg, data_dim, device)``. Built-in filters retain this constructor
and their single-tensor behavior; structured processors are constructed directly as above.

Migration: import ``TensorProcessor`` and ``TensorProcessorCfg`` from
``isaaclab.utils.tensor_processors`` in place of ``ModifierBase`` and ``ModifierCfg`` from
``isaaclab.utils.modifiers``. Replace ``ObservationTermCfg.modifiers`` with
``ObservationTermCfg.tensor_processors``. The processing order and tensor shapes are unchanged.
Old imports remain supported and emit ``DeprecationWarning`` with migration instructions.

.. automodule:: isaaclab.utils.tensor_processors
:members:
:imported-members:
:special-members: __call__
:inherited-members:
:show-inheritance:
:exclude-members: __init__, func

Image operations
~~~~~~~~~~~~~~~~

Expand Down
5 changes: 5 additions & 0 deletions source/isaaclab/changelog.d/dict-tensor-processors.added.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
* Extended ``TensorProcessor`` and ``TensorProcessorCfg`` to support tensors, ``ProxyArray`` values,
and nested lists, tuples, and dictionaries through the shared ``TensorProcessorData`` type.
Added optional named input/output metadata and lifecycle hooks while preserving single-tensor
observation processing. Added manager resolution of nested scene entities and manager terms
in processor parameters.
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
* Deprecated ``ObservationTermCfg.modifiers`` in favor of ``ObservationTermCfg.tensor_processors``.
Retained the legacy field with a targeted warning and forwarding during manager initialization.
Setting both fields now raises ``ValueError`` to prevent silently overriding either pipeline.
Empty file.
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
* Deprecated ``isaaclab.utils.modifiers``, ``ModifierBase``, and ``ModifierCfg`` in favor of
``isaaclab.utils.tensor_processors``, ``TensorProcessor``, and ``TensorProcessorCfg``. Preserved
legacy imports and serialized callable paths with an import-time deprecation warning.
4 changes: 2 additions & 2 deletions source/isaaclab/isaaclab/managers/manager_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from ..physics import PhysicsEvent, PhysicsManager
from ..utils import string as string_utils
from ..utils import string_to_callable, to_dict
from ..utils.modifiers import ModifierCfg
from ..utils.tensor_processors import TensorProcessorCfg
from .manager_term_cfg import ManagerTermBaseCfg
from .scene_entity_cfg import SceneEntityCfg

Expand Down Expand Up @@ -434,7 +434,7 @@ def _resolve_param_value(
self._scene_entity_cfgs.append(value)
elif isinstance(value, ManagerTermBaseCfg):
self._process_term_cfg_at_play(f"{term_name}.{key}", value)
elif isinstance(value, ModifierCfg):
elif isinstance(value, TensorProcessorCfg):
for field in fields(value):
field_value = getattr(value, field.name)
resolved_value = self._resolve_param_value(
Expand Down
47 changes: 37 additions & 10 deletions source/isaaclab/isaaclab/managers/manager_term_cfg.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,16 @@

from __future__ import annotations

import warnings
from collections.abc import Callable
from dataclasses import MISSING
from typing import TYPE_CHECKING, Any, Literal

import torch

from ..utils import configclass
from ..utils.modifiers import ModifierCfg
from ..utils.noise import NoiseCfg, NoiseModelCfg
from ..utils.tensor_processors import TensorProcessorCfg
from .scene_entity_cfg import SceneEntityCfg

if TYPE_CHECKING:
Expand Down Expand Up @@ -162,20 +163,23 @@ class ObservationTermCfg(ManagerTermBaseCfg):
"""The function or :class:`ManagerTermBase` subclass to be called.

The manager instantiates class-based terms before calling them.
The function or the class's ``__call__`` method should take the environment object and any other parameters
as input and return the observation signal as torch float tensors of
shape (num_envs, obs_term_dim).
With the standard observation manager, the function or the class's ``__call__`` method takes the
environment object and any other parameters as input and returns the observation signal as a
torch tensor of shape (num_envs, obs_term_dim).

With the experimental Warp observation manager, it also takes a pre-allocated output buffer as
its second argument, writes the observations into that buffer, and returns None.
"""

modifiers: list[ModifierCfg] | None = None
"""The list of data modifiers to apply to the observation in order. Defaults to None,
tensor_processors: list[TensorProcessorCfg] | None = None
Comment thread
mataylor-nvidia marked this conversation as resolved.
"""The list of tensor processors to apply to the observation in order. Defaults to None,
in which case no modifications will be applied.

Modifiers are applied in the order they are specified in the list. They can be stateless
Processors are applied in the order they are specified in the list. They can be stateless
or stateful, and can be used to apply transformations to the observation data. For example,
a modifier can be used to normalize the observation data or to apply a rolling average.
a processor can be used to normalize the observation data or to apply a rolling average.

For more information on modifiers, see the :class:`~isaaclab.utils.modifiers.ModifierCfg` class.
For more information, see :class:`~isaaclab.utils.tensor_processors.TensorProcessorCfg`.
"""

noise: NoiseCfg | NoiseModelCfg | None = None
Expand Down Expand Up @@ -206,7 +210,7 @@ class ObservationTermCfg(ManagerTermBaseCfg):
delay_max_lag: int = 0
"""Maximum observation delay, counted in recorded samples. Zero disables delay.

Set both lag bounds equal for a constant delay. Delay is applied after modifiers, noise, clipping, and
Set both lag bounds equal for a constant delay. Delay is applied after tensor processors, noise, clipping, and
scaling, before observation history. Until enough samples have been recorded, the oldest available
sample is returned. After reset, no data from the previous episode is returned.
"""
Expand All @@ -233,6 +237,14 @@ class ObservationTermCfg(ManagerTermBaseCfg):
"""Whether or not the observation manager should flatten history-based observation terms to a 2-D (N, D) tensor.
Defaults to True."""

modifiers: list[TensorProcessorCfg] | None = None

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The old cfg also need deprecation shim?

"""Deprecated alias for :attr:`tensor_processors`, to be removed in a future release.

During manager initialization, a non-None value emits a :class:`FutureWarning`, is forwarded
to :attr:`tensor_processors`, and is cleared from this field. Set only one of the two fields;
providing both raises :class:`ValueError`, even for empty lists.
"""

def validate_config(self):
"""Validate observation delay bounds."""
if type(self.delay_min_lag) is not int or type(self.delay_max_lag) is not int:
Expand All @@ -242,6 +254,21 @@ def validate_config(self):
if not 0.0 <= self.delay_hold_prob <= 1.0:
raise ValueError("delay_hold_prob must be in [0, 1].")

def _migrate_modifiers(self) -> None:
"""Normalize the deprecated field after user edits and before manager preparation."""
if self.modifiers is None:
return
if self.tensor_processors is not None:
raise ValueError("Set only one of 'modifiers' and 'tensor_processors'.")
warnings.warn(
"ObservationTermCfg.modifiers is deprecated and will be removed in a future release."
" Use ObservationTermCfg.tensor_processors instead.",
FutureWarning,
stacklevel=5,
)
self.tensor_processors = self.modifiers
self.modifiers = None


@configclass
class ObservationGroupCfg:
Expand Down
Loading
Loading