[clean] document one capture instant per track() call

README.md

@@ -118,7 +118,7 @@ The accuracy and robustness of cuVSLAM can be influenced by several factors. If
 
 - **Intrinsic and Extrinsic Calibration**: Accurate camera calibration is crucial. Ensure your calibration parameters are precise. For more details, refer to our guide on [image undistortion](examples/euroc/README.md#distortion-models). If you're new to calibration, consider working with [experienced vendors](https://nvidia-isaac-ros.github.io/v/release-3.2/getting_started/hardware_setup/sensors/amr_extrinsic_calibration.html#extrinsic-calibration-of-sensors-in-custom-locations)
 
-- **Synchronization and Timestamps**: Accurate synchronization significantly impacts cuVSLAM performance. Make sure multi-camera images are captured simultaneously—ideally through hardware synchronization—and verify correct relative timestamps across cameras. Refer to our [multi-camera hardware assembly guide](examples/realsense/multicamera_hardware_assembly.md) for building a rig with synchronized RealSense cameras
+- **Synchronization and Timestamps**: Accurate synchronization significantly impacts cuVSLAM performance. Make sure multi-camera images are captured simultaneously—ideally through hardware synchronization—and verify correct relative timestamps across cameras. Refer to our [multi-camera hardware assembly guide](examples/realsense/multicamera_hardware_assembly.md) for building a rig with synchronized RealSense cameras. Each `track()` call must carry images captured at one instant (within 1 ms); cycling through cameras or camera groups across calls is not supported. Unsynchronized cameras are not supported either; see [Cameras that are not synchronized with each other](TROUBLESHOOTING.md#cameras-that-are-not-synchronized-with-each-other) for workarounds such as one tracker per camera
 
 - **Frame Rate**: Frame rate significantly affects performance. The ideal frame rate depends on translational and rotational velocities. Typically, 30 FPS is suitable for most "human-speed" motions. Adjust accordingly for faster movements
 

TROUBLESHOOTING.md

@@ -41,6 +41,7 @@ See [Requirements](README.md#performance).
   - [Try to use rectified images if possible](#try-to-use-rectified-images-if-possible)
   - [Disable SLAM and ground constraint](#disable-slam-and-ground-constraint)
 - [Step 6: Verify synchronization](#step-6-verify-synchronization)
+  - [Cameras that are not synchronized with each other](#cameras-that-are-not-synchronized-with-each-other)
 - [Step 7: Run stereo odometry tracking](#step-7-run-stereo-odometry-tracking)
 - [Step 8: Adjust odometry settings](#step-8-adjust-odometry-settings)
   - [Improve input images](#improve-input-images)
@@ -150,8 +151,9 @@ cuvslam::Odometry::Config::max_frame_delta_s
 image_jitter_threshold_ms
 ```
 
-In the Isaac ROS cuVSLAM node, there is also a synchronizer that can align multiple image streams with non-equal
-timestamps. Check Isaac ROS documentation and tune:
+In the Isaac ROS cuVSLAM node, there is also a synchronizer that groups images whose timestamps are within
+`sync_matching_threshold_ms` into one frame; it cannot synchronize free-running cameras. Check Isaac ROS documentation
+and tune:
 
 ```
 num_cameras
@@ -351,6 +353,20 @@ Example: Left and right eyes out of sync
 
 ![Example: Left and right eyes out of sync](doc/images/troubleshooting_desync.gif)
 
+### Cameras that are not synchronized with each other
+
+Each `track()` call must carry images captured at one instant (within 1 ms). A camera may be missing from a call,
+for example after a frame drop, but cycling through cameras or camera groups across calls is not supported, and neither
+is filling an empty slot with an older image or giving images from different instants one timestamp. cuVSLAM matches
+features across cameras only within a call. `track()` rejects a call whose image timestamps differ by 1 ms or more,
+but when such input passes this check, tracking accuracy is not guaranteed.
+
+If your cameras are not synchronized:
+
+1. Synchronize them in hardware (external trigger).
+2. If they mostly line up, pass only frame sets whose images are within 1 ms of each other and drop the rest.
+3. Otherwise, run one tracker per camera (or per synchronized stereo pair) and fuse their poses yourself.
+
 ## Step 7: Run stereo odometry tracking
 
 If mono left and mono right both work, run stereo tracking. Stereo reuses extrinsic calibration via the

cuvslam-skills/cuvslam-troubleshoot/SKILL.md

@@ -90,7 +90,7 @@ Key parameters to note once you have the config:
   cuNLS availability, `depth_camera_ids`, and pinhole calibration.
   Knowing this up front directs the diagnosis toward vision, depth, or IMU root causes.
 - `rectified_images` (Isaac ROS) / `rectified_stereo_camera` (C++ API) — wrong value flips the entire stereo pipeline
-- `image_jitter_threshold_ms` — tolerated timestamp jitter between left/right images; too tight a value causes frames to be dropped silently
+- `image_jitter_threshold_ms` — tolerated gap between consecutive frames (exceeding it only logs a warning), not a tolerance between cameras
 - `Tracker.Mode` plus the `async_sba` / `sync_mode` that must match it — affects reproducibility
 - `enable_slam` / loop closure settings — SLAM should only be tuned after odometry is solid
 - `denoise_input_images`, `imu_from_left` — sensor-specific flags
@@ -169,7 +169,7 @@ With the config and dataset in hand, verify these before touching the diagnostic
 
 1. **Accurate intrinsic calibration** — verify distortion model and params match the actual camera. See `references/edex-calibration-pitfalls.md` for common EDEX mistakes (e.g. `"fisheye"` with zero params ≠ `"pinhole"`).
 2. **Accurate extrinsic calibration** (stereo/multi-cam) — verify baseline axis and magnitude match the physical rig. See `references/edex-calibration-pitfalls.md` for wrong-axis baseline diagnosis.
-3. **Proper image synchronization** — left/right timestamps must match per frame.
+3. **Proper image synchronization** — each `track()` call carries images captured at one instant (within 1 ms), never cameras cycled across calls.
 4. **Sharp images, low noise** — frame rate matters more than resolution.
 5. **Textured, rigid scenes** — avoid repetitive patterns and dynamic objects.
 6. **Correct parameters and coordinate frames** — check `rectified_stereo_camera` flag.

cuvslam-skills/cuvslam-troubleshoot/commands/bag2edex.md

@@ -225,7 +225,7 @@ If any topic prints a frame_id belonging to a different camera, set `camera_opti
 
 **Wrong intrinsics** — intrinsics are read from `camera_info` messages in the bag. Verify the published `camera_info` matches the actual camera before converting.
 
-**Mismatched frame counts** — only synchronized frame pairs (within `sync_threshold_ns`) are saved. Increase `sync_threshold_ns` in the config if too many frames are dropped.
+**Mismatched frame counts** — a frame set is saved only if its image timestamps differ by less than `sync_threshold_ns` (default 1 ms); a difference equal to or greater than the threshold is rejected. Don't raise it to keep more frames: images 1 ms or more apart are not one capture instant.
 
 **No synchronized frames found** — the tool exits with an error listing per-topic frame counts. Check that image topics are correct and timestamps overlap within `sync_threshold_ns`.
 

cuvslam-skills/cuvslam-troubleshoot/references/isaac-ros-visual-slam-usage.md

@@ -56,6 +56,9 @@ ros2 launch isaac_ros_visual_slam isaac_ros_visual_slam_realsense_bag.launch.py
 
 ## Key Parameters
 
+Defaults are the `declare_parameter` values in release-4.3 `visual_slam_node.cpp`; check them there for the user's
+release before relying on one.
+
 | Parameter | Default | Notes |
 |---|---|---|
 | `num_cameras` | `2` | Number of input cameras (max 32) |
@@ -65,11 +68,11 @@ ros2 launch isaac_ros_visual_slam isaac_ros_visual_slam_realsense_bag.launch.py
 | `enable_localization_n_mapping` | `true` | `false` = odometry only, no loop closure |
 | `enable_image_denoising` | `false` | Enable for low-light/noisy cameras |
 | `enable_ground_constraint_in_slam` | `false` | Constrain to horizontal plane for wheeled robots |
-| `image_jitter_threshold_ms` | — | Tolerated timestamp jitter between left/right images |
-| `sync_matching_threshold_ms` | — | Tolerated timestamp offset across cameras for sync |
+| `image_jitter_threshold_ms` | `34.0` | Tolerated gap between consecutive frames; exceeding it only logs a warning |
+| `sync_matching_threshold_ms` | `5.0` | Grouping window only; `Odometry::Track` rejects images differing by 1 ms or more, so keep grouped timestamps under 1 ms |
 | `base_frame` | `base_link` | Robot base frame for TF |
 | `camera_optical_frames` | — | List of camera optical frame names (must match TF tree) |
-| `imu_frame` | `imu` | IMU frame (VIO mode only) |
+| `imu_frame` | `""` | IMU frame (VIO mode only) |
 | `save_map_folder_path` | `""` | Path to save SLAM map |
 | `load_map_folder_path` | `""` | Path to load a saved SLAM map |
 

libs/cuvslam/cuvslam2.h

@@ -592,13 +592,13 @@ class CUVSLAM_API Odometry {
    *
    * The track will output poses in the same coordinate system until a loss of tracking.
    *
-   * Image timestamps have to match. cuVSLAM will use timestamp from the camera 0 image.
+   * Image timestamps have to match. cuVSLAM will use the timestamp of images[0].
    * If a camera rig provides "almost synchronized" frames, the timestamps should be within 1 millisecond.
    * The number of images for tracker should not exceed rig->num_cameras.
    *
    * @param[in]  images  an array of synchronized images no more than rig->num_cameras.
-   * Must use ImageData::DataType::UINT8. Partial ImageSet is supported, for example due to frame drops. Corresponding
-   * cameras are identified by Image::camera_index.
+   * Must use ImageData::DataType::UINT8. Partial ImageSet is supported, for example due to frame drops, but cycling
+   * through cameras or camera groups across calls is not. Corresponding cameras are identified by Image::camera_index.
    * @param[in]  masks  (Optional) an array of corresponding masks no more than rig->num_cameras.
    * Must use ImageData::DataType::UINT8 and ImageData::Encoding::MONO. Non-zero pixels are excluded: no features are
    * selected or kept there, and in OdometryMode::RGBD and OdometryMode::Multisensor the depth at those pixels is

python/cuvslam2.cpp

@@ -715,7 +715,8 @@ NB_MODULE(pycuvslam, m) {
           "images etc. can still be either CPU or GPU arrays/tensors. The reverse does not hold:\n"
           "odometry with `use_gpu` disabled only accepts CPU arrays/tensors.\n\n"
           "The images etc. must be in the same order as cameras in the rig.\n"
-          "If data for a camera is not available, pass empty array or tensor for that camera image.\n\n"
+          "If data for a camera is not available, pass empty array or tensor for that camera image.\n"
+          "Cycling through cameras or camera groups across calls is not supported.\n\n"
           "Parameters:\n"
           "    timestamp: Image timestamp in nanoseconds\n"
           "    images: List of numpy arrays containing the camera images\n"
@@ -725,7 +726,7 @@ NB_MODULE(pycuvslam, m) {
           "    depths: Optional list of numpy arrays containing depth images\n"
           "    options: Optional per-frame options that override default settings for this call only\n\n"
           "Returns:\n"
-          "    :class:`PoseEstimate` object with the computed pose. If tracking fails, `is_valid` will be False.")
+          "    :class:`PoseEstimate` object with the computed pose. If tracking fails, `world_from_rig` is `None`.")
       .def(
           "register_imu_measurement",
           [](Odometry& self, uint32_t sensor_index, const ImuMeasurement& imu_measurement) {
@@ -1286,7 +1287,8 @@ NB_MODULE(pycuvslam, m) {
           "images etc. can still be either CPU or GPU arrays/tensors. The reverse does not hold:\n"
           "odometry with `use_gpu` disabled only accepts CPU arrays/tensors.\n\n"
           "The images etc. must be in the same order as cameras in the rig.\n"
-          "If data for a camera is not available, pass empty array or tensor for that camera image.\n\n"
+          "If data for a camera is not available, pass empty array or tensor for that camera image.\n"
+          "Cycling through cameras or camera groups across calls is not supported.\n\n"
           "Parameters:\n"
           "    timestamp: Images timestamp in nanoseconds\n"
           "    images: List of numpy arrays or tensors containing the camera images\n"