Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .agents/skills/isaaclab-auditing-an-issue
1 change: 1 addition & 0 deletions .agents/skills/isaaclab-triaging-issue-backlog
36 changes: 36 additions & 0 deletions .github/workflows/nightly-isaacsim-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,7 @@ jobs:
CURRENT_PIN: ${{ steps.pin.outputs.current }}
CANDIDATE_PIN: ${{ steps.pin.outputs.candidate }}
DIGEST: ${{ steps.pin.outputs.digest }}
BRANCH_CHANGED: ${{ steps.pin.outputs.branch_changed }}
run: |
set -euo pipefail

Expand All @@ -187,6 +188,8 @@ jobs:
title="[CI] Bump Isaac Sim image to $short_digest"
body_file="$RUNNER_TEMP/isaacsim-image-update.md"
{
echo "# Description"
echo
echo "This automated draft updates CI to the current Isaac Sim nightly image."
echo
echo "| Field | Value |"
Expand All @@ -199,6 +202,30 @@ jobs:
echo "Source: https://registry.ngc.nvidia.com/orgs/0947644777160149/teams/internal/containers/isaac-sim/tags"
echo
echo "New PRs are opened as drafts so maintainers can merge after the CI results are acceptable."
echo
echo "## Type of change"
echo
echo "- Infrastructure update"
echo
echo "## Release backport"
echo
echo '- [ ] <!-- backport-active-release --> Backport this pull request to the active release branch after it merges into `develop`'
echo
echo "## Screenshots"
echo
echo "Not applicable."
echo
echo "## Checklist"
echo
echo "Docker and GPU tests run on demand. This workflow comments \`run-ci\` after opening the pull request or updating its commit."
echo
echo '- [ ] I have read and understood the [contribution guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html)'
echo '- [ ] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format`'
echo '- [ ] I have made corresponding changes to the documentation'
echo '- [ ] My changes generate no new warnings'
echo '- [ ] I have added tests that prove my fix is effective or that my feature works'
echo '- [ ] I have added a changelog fragment under `source/<pkg>/changelog.d/` for every touched package (do **not** edit `CHANGELOG.rst` or bump `extension.toml` — CI handles that)'
echo '- [ ] I have added my name to the `CONTRIBUTORS.md` or my name already exists there'
} > "$body_file"

repository_owner=${REPOSITORY%%/*}
Expand All @@ -208,6 +235,7 @@ jobs:
-f head="$repository_owner:$UPDATE_BRANCH" \
--jq '.[0].number // empty')

pr_created=false
if [ -n "$pr_number" ]; then
pr_url=$(gh api --method PATCH "repos/$REPOSITORY/pulls/$pr_number" \
-f title="$title" \
Expand All @@ -223,10 +251,18 @@ jobs:
-F body=@"$body_file" \
-F draft=true \
--jq '.html_url')
pr_number=${pr_url##*/}
pr_created=true
echo "Opened draft PR: $pr_url"
echo "Draft PR: $pr_url" >> "$GITHUB_STEP_SUMMARY"
fi

if [ "$BRANCH_CHANGED" = "true" ] || [ "$pr_created" = "true" ]; then
gh api --method POST "repos/$REPOSITORY/issues/$pr_number/comments" \
-f body='run-ci' >/dev/null
echo "Triggered CI on PR #$pr_number."
fi

- name: Report no-op or dry run
if: ${{ steps.pin.outputs.changed != 'true' || inputs.dry_run }}
env:
Expand Down
142 changes: 142 additions & 0 deletions skills/developer/issue-audit/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
---
name: isaaclab-auditing-an-issue
description: Audits one open GitHub issue against current code to decide close or keep-open, and drafts the closing comment. Use when deciding whether a specific issue is still valid, whether a reported bug was fixed, or what to reply before closing.
audience: developer
status: experimental
owners:
- isaaclab-maintainers
---

# Auditing an Issue

## When To Use

Use this skill for a single issue: deciding whether it is still valid against current code, and drafting the reply if it is being closed.

For auditing many issues at once, use `isaaclab-triaging-issue-backlog`, which applies this skill per issue and adds the batch controls.

Do not close an issue purely because it is old. Age is not evidence.

## Core Principle

A comment that reads well is not evidence that its claims are true. Verify every factual claim against current code before posting, and ask what is lost by closing, not only whether closing is defensible.

## Workflow

1. Read the whole thread, not just the first post: `gh issue view NUMBER --repo isaac-sim/IsaacLab --comments`.
2. Identify each checkable claim you intend to make in the reply.
3. Verify each one using the table below. Prefer running code over reading it.
4. Check the Do Not Close list before deciding.
5. Draft the comment in the style below, then confirm every link resolves.

## Mandatory Checks

| Claim shape | Required check |
|---|---|
| "X is fixed / landed in #N" | `gh pr view N` returns MERGED, and its diff touches the reported behavior. A generic "Merges changes from main" PR is never an acceptable citation. |
| "Isaac Lab doesn't have X" | Grep `source/isaaclab_tasks/isaaclab_tasks/contrib/`, `source/isaaclab_contrib/` and the `*_experimental` packages. Features move to contrib and absence claims go stale. |
| "`SYMBOL` no longer exists" | `grep -rn "SYMBOL" source/` — any live hit refutes it. |
| "`BACKEND` supports X" | Check that backend's own package. Backends differ; several classes raise `NotImplementedError` on one and not another. |
| Behavioral claim about an error or a limit | Write a temporary script and run it with `uv run python`. Reading the raise site is not enough — the guard may be narrower than it looks. |
| A usage answer | Confirm it is still the best current advice, not merely that some answer was given. Check the relevant capability on the Isaac Sim side too. |
| Any URL | `curl -s -o /dev/null -w "%{http_code}" -L "URL"` returns 200. |

## URL Traps

- `github.com/isaac-sim/IsaacLab/blob/main/...` **404s** for anything under `source/isaaclab_newton/`, `source/isaaclab_ov/` or `source/isaaclab_contrib/` — those packages are not on `main`. Use `blob/develop/...`, or cite the path in backticks with no link. Package layout shifts over time (e.g. `isaaclab_ovphysx` was merged into `isaaclab_ov`), so verify the path exists on the ref you link.
- `isaac-sim.github.io/IsaacLab/main/...` is the stale 2.x docs build. Use `release/3.0.0/...`, and confirm the anchor still exists — several 2.x anchors were removed in the 3.x reorganisation.

## Do Not Close

Hold the issue when any of these apply, even if the stated close reason is technically defensible:

- A maintainer asked the reporter a question and the thread is still recent.
- It is pre-merge feedback on an open PR.
- It is a duplicate of an issue that was itself closed without a fix.
- The titled ask is unmet, even if a related bug in the thread was fixed.
- It is blocked on upstream work that is planned but not landed. Confirm the upstream issue exists; "file it upstream" is not a resolution if nobody has.

When holding, record the concrete next step and the evidence, so the next audit does not redo the work.

## Comment Style

- Thank the reporter by handle in the first sentence, specifically and briefly.
- State what actually resolves it, with file paths in backticks and a verified PR link.
- Invite a reopen if it still reproduces on a current release.
- Do not assign fault. Avoid "we never heard back" (blames the reporter) and "nothing on our side" (blames a sibling team). Describe where the behavior lives, and route rather than deflect.
- Do not name a colleague as the source of a negative finding. Crediting a helpful answer is fine.
- If the reply was AI-assisted, end with the disclaimer footer below, and do not claim a level of human review that did not happen.

### Disclaimer Footer

End an AI-assisted reply with a horizontal rule and a one-line footer:

```markdown
---
*🤖 This issue was reviewed as part of an AI-assisted triage of the Isaac Lab backlog, with automated verification against the current codebase. If we got this wrong, please reopen — we'd rather hear about it!*
```

Match the claim to what actually happened. "with automated verification against the current codebase" is accurate for a machine-checked reply. Only say "verified by a maintainer" if a maintainer read that specific comment.

### Worked Example

A close for a bug that was genuinely fixed:

```markdown
Thanks @Mitchell-Torok for catching that `reset_joints_by_offset` ignored the `joint_names` you passed in `SceneEntityCfg`.

It's fixed: both `reset_joints_by_offset` and `reset_joints_by_scale` in `isaaclab/envs/mdp/events.py` now index the default joint state and the soft-limit clamp with `asset_cfg.joint_ids` (broadcasting `env_ids[:, None]` for subsets), so only the named joints are touched. Shipped in https://github.com/isaac-sim/IsaacLab/pull/2899. Sorry for the delay — please reopen if extra joints still move on a current release.

---
*🤖 This issue was reviewed as part of an AI-assisted triage of the Isaac Lab backlog, with automated verification against the current codebase. If we got this wrong, please reopen — we'd rather hear about it!*
```

Each part is doing work: the handle and the specific symptom in sentence one, the mechanism plus file path so the comment stands alone in search results, a PR link confirmed MERGED, a three-word apology rather than a paragraph, and a reopen invitation tied to a current release.

### Fault Assignment

The rewrites below are the difference between a reply that reads as closure and one that reads as a brush-off.

| Instead of | Write |
|---|---|
| "We asked twice and never heard back." | "We weren't able to get to the bottom of this at the time, and a lot has changed since." |
| "There's nothing on our side to fix." | "That code path lives in `<component>`, so `<tracker>` is the better home for it." |
| "This is an Isaac Sim bug that their team owns." | "The `sim.reset()` requirement lives in that layer and is tracked there." |
| "As @SomeDev confirmed, it's broken." | State the finding directly, without attributing it to a colleague. |

## Common Mistakes

- Trusting a polished draft. Fluency correlates with neither accuracy nor completeness.
- Answering "was a reply given?" instead of "is that answer still correct and complete?"
- Citing a PR whose title matches the topic without reading its diff.
- Closing an issue whose stated reason is sound but which is the only remaining record of a live problem.

## Validation

Verify citations and links before posting:

```bash
gh pr view PR_NUMBER --repo isaac-sim/IsaacLab --json state,mergedAt,title
curl -s -o /dev/null -w "%{http_code}" -L "URL"
```

For a behavioral claim, write a temporary script and run it:

```bash
uv run python PATH_TO_SCRIPT
```

If skills changed, run:

```bash
uv run --no-project python tools/skills/cli.py check
```

## Maintenance

Keep the URL traps section synchronized with the current package layout. When a package moves between `main` and `develop`, or the docs build version changes, update it before it produces a dead link in a public comment.

## References

- [Backlog triage skill](../issue-backlog-triage/SKILL.md)
- [Contributing guide](../../../docs/source/refs/contributing.rst)
88 changes: 88 additions & 0 deletions skills/developer/issue-backlog-triage/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
---
name: isaaclab-triaging-issue-backlog
description: Triages many open GitHub issues at once into close/keep-open with drafted closing comments. Use when auditing the whole issue backlog, running a bulk close pass, or reporting how many issues can be retired.
audience: developer
status: experimental
owners:
- isaaclab-maintainers
---

# Triaging the Issue Backlog

## When To Use

Use this skill when auditing many issues at once: a backlog sweep, a bulk close pass, or a report on how much of the backlog can be retired.

For a single issue, use `isaaclab-auditing-an-issue` directly.

**REQUIRED SUB-SKILL:** Use `isaaclab-auditing-an-issue` for every individual issue. This skill only adds batch orchestration; it does not restate the per-issue checks.

## Core Principle

Scale multiplies confident errors. A batch of drafted comments will read uniformly well whether or not the claims are true, so verification must be adversarial and the human review gate is not optional.

## Workflow

1. Dump the backlog with metadata and split it into batches:

```bash
gh issue list --repo isaac-sim/IsaacLab --state open --limit 500 \
--json number,title,author,createdAt,updatedAt,labels,comments,assignees,url
```

2. Audit each issue per `isaaclab-auditing-an-issue`, recording a verdict, the evidence, and a confidence level.
3. Run an adversarial pass over every close recommendation. Instruct the reviewer to **refute**, and to return REFUTED or UNCERTAIN whenever it cannot independently confirm the evidence. Verify low-confidence recommendations as strictly as the rest — they refute at a far higher rate.
4. Sweep the drafted comments for claims that a per-issue audit misses across a batch: capability assertions, citations whose PR does not match, and dead links.
5. Review every comment with a maintainer, in batches, before anything is posted.
6. Post only what a maintainer explicitly approved.

## Batch Controls

- **Never infer approval.** Approve on an explicit statement, not on a reply that merely discusses a batch. Record approvals so a resumed session cannot post something unreviewed.
- **Keep one plan file** listing every issue to be closed with its final comment, plus appendices for held issues and for issues pulled out during review with their next steps.
- **Re-show anything edited after approval.** Later passes rewrite earlier comments; approval does not survive a rewrite.
- **Report what was excluded.** State how many issues were refuted, held, or pulled out, not only how many are closeable.

## Expect These Failure Modes

Measured over one full pass of the backlog; treat them as the default, not the exception.

| Failure mode | What it looks like |
|---|---|
| Stale absence claims | "Isaac Lab doesn't support X" when X moved to `contrib/`. |
| Mismatched citations | A merged PR whose title fits the topic but whose diff never touches the behavior. |
| Capability drift | Asserting a backend or release supports something it does not. |
| Answered-but-wrong | A usage answer that was correct when written and is now outdated. |
| Blame framing | "We never heard back" or "nothing on our side", which read badly in public. |
| Defensible but lossy | A close whose reason is sound but which discards the only record of a live problem. |

## Common Mistakes

- Treating a high-confidence label as a substitute for verification.
- Batching approval across issues that were edited at different times.
- Reporting a reduction figure without the refuted and held counts.
- Posting before a maintainer has read the specific comment being posted.

## Validation

Before posting, confirm the batch is internally consistent: every comment ends with the intended marker, every cited PR is merged, and every URL resolves.

```bash
gh pr view PR_NUMBER --repo isaac-sim/IsaacLab --json state,mergedAt,title
curl -s -o /dev/null -w "%{http_code}" -L "URL"
```

If skills changed, run:

```bash
uv run --no-project python tools/skills/cli.py check
```

## Maintenance

Keep the per-issue checks in `isaaclab-auditing-an-issue` and do not duplicate them here. Update the failure-mode table when a new class of error survives verification into a posted comment.

## References

- [Issue audit skill](../issue-audit/SKILL.md)
- [Contributing guide](../../../docs/source/refs/contributing.rst)
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Allowed standalone smoke tests that reached readiness before the startup deadline to complete their full soak interval.
2 changes: 1 addition & 1 deletion source/isaaclab/test/app/standalone_script_cases.py
Original file line number Diff line number Diff line change
Expand Up @@ -489,7 +489,7 @@ def read_available_output(timeout: float) -> bool:
_terminate_process_group(process)
returncode = process.poll()
break
if now - start_time >= startup_timeout:
if ready_at is None and now - start_time >= startup_timeout:
_terminate_process_group(process)
returncode = process.poll()
break
Expand Down
32 changes: 32 additions & 0 deletions source/isaaclab/test/app/test_standalone_scripts.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import sys
from dataclasses import replace
from pathlib import Path
from unittest import mock

import pytest
import standalone_script_cases as script_cases
Expand Down Expand Up @@ -371,6 +372,37 @@ def test_subprocess_supervisor_soaks_then_stops_process_group():
assert result.elapsed < 2.0


def test_subprocess_supervisor_completes_soak_after_startup_deadline(monkeypatch):
"""Readiness just before the startup deadline must still receive the full soak."""
process = mock.Mock(returncode=None)
process.poll.side_effect = lambda: process.returncode
process.communicate.return_value = (b"", None)
selector = mock.Mock()
now = 0.0
poll_times = iter((299.0, 300.0, 304.0))

def select(timeout):
nonlocal now
if timeout == 0.0:
return []
now = next(poll_times)
if now == 299.0:
return [(mock.Mock(fileobj=process.stdout), script_cases.selectors.EVENT_READ)]
return []

selector.select.side_effect = select
monkeypatch.setattr(script_cases.subprocess, "Popen", lambda *args, **kwargs: process)
monkeypatch.setattr(script_cases.selectors, "DefaultSelector", lambda: selector)
monkeypatch.setattr(script_cases.os, "read", lambda *args: b"READY\n")
monkeypatch.setattr(script_cases.time, "monotonic", lambda: now)
monkeypatch.setattr(script_cases, "_terminate_process_group", lambda process: setattr(process, "returncode", -15))

result = run_until_ready(["demo.py"], r"READY", startup_timeout=300.0, soak_time=5.0)
assert result.ready
assert result.stopped_after_soak
assert result.elapsed == 304.0


def test_subprocess_supervisor_ignores_fatal_output_after_intentional_teardown(monkeypatch):
"""Fatal-looking output caused by intentional teardown must not fail a healthy launch."""

Expand Down
Empty file.
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
Fixed
^^^^^

* Fixed :func:`~isaaclab_rl.utils.pretrained_checkpoint.get_published_pretrained_checkpoint` reporting
every download failure as ``A pre-trained checkpoint is currently unavailable for this task.``. A
checkpoint the asset server does not provide is still reported that way, but the message now names the
location that was tried, the task and backends it was derived from, and what to do instead.

Changed
^^^^^^^

* Changed :func:`~isaaclab_rl.utils.pretrained_checkpoint.get_published_pretrained_checkpoint` to raise
``RuntimeError`` when a published checkpoint cannot be downloaded, for instance when the
``.pretrained_checkpoints`` cache directory is not writable, instead of returning ``None``. The
originating error is chained as the cause. ``None`` is now returned only when the asset server does not
report the checkpoint, which covers both an unpublished checkpoint and a server that could not be
reached; callers that relied on ``None`` to mask local download failures must catch ``RuntimeError``.
Loading
Loading