Skip to content

Keep instance segmentation refreshes fast with many annotated components - #168

Merged
WouterJansen merged 1 commit into
Cosys-Lab:mainfrom
jcfurey:fix/segmentation-refresh-scaling
Oct 5, 2026
Merged

WouterJansen merged 1 commit into
Cosys-Lab:mainfrom
jcfurey:fix/segmentation-refresh-scaling

Conversation

@jcfurey

@jcfurey jcfurey commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Instance segmentation refreshes get slow with many annotated meshes: each refresh re-hid every annotation component in every camera's scene and lighting captures and the player's view with AddUnique (quadratic), never dropped destroyed components, and UpdateAnnotationComponents searched its list for every component.

  • Hiding uses a set and drops destroyed components.
  • Cameras hide only the components added since the previous refresh; the annotator tracks them, and refresh serial numbers make a new camera, or one that missed a refresh, hide all of them. The player's view is updated once per refresh instead of once per camera.

Found in HERCULES (a Cosys-AirSim fork, UE 5.8.3) with ~25k annotated streamed map tiles, where a refresh took ~48 ms of game-thread time; afterwards segmentation images were unchanged and the stutter gone. This tree's version also hides the lighting capture; built from it with UE 5.8.3 (Windows), segmentation images in Blocks render as before.

🤖 Generated with Claude Code

Each refresh hid every annotation component again in every camera's
scene and lighting captures and the player's view with AddUnique, which
is quadratic, and never dropped destroyed components; UpdateAnnotation-
Components also searched its list for every component. With thousands of
annotated meshes a refresh took tens of milliseconds to seconds of game
thread time.

- Hiding uses a set (linear) and drops destroyed components.
- The annotator remembers the components added since the previous
  refresh, and cameras hide only those; refresh serial numbers make a
  camera that is new, or missed a refresh, hide all of them instead. The
  player's view is updated once per refresh rather than once per camera.
- UpdateAnnotationComponents looks components up in a set.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JmWqXFCY6vDdpRe4VHoaQX
@jcfurey
jcfurey marked this pull request as ready for review October 4, 2026 05:46
@WouterJansen
WouterJansen merged commit ab98100 into Cosys-Lab:main Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants