Skip to content

Fix signal-terminated nvidia-smi queries - #5543

Open
fallintoplace wants to merge 1 commit into
NVIDIA:mainfrom
fallintoplace:fix/nvidia-smi-signal-exit-20261025
Open

fallintoplace wants to merge 1 commit into
NVIDIA:mainfrom
fallintoplace:fix/nvidia-smi-signal-exit-20261025

Conversation

@fallintoplace

Copy link
Copy Markdown
Contributor

Fixes #5542.

Description

What: Report signal-terminated nvidia-smi commands as failed GPU queries.

Why: Signal exits have negative return codes. The current check treats them as success and can report no GPUs.

Implementation: Check returncode != 0 and add regressions for exit codes 1, -9, and -15.

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • New tests added to cover the changes.

Testing

  • Focused pytest on the matching isolated module copy: 3 passed. The two signal cases fail before the fix.
  • Python syntax and line lengths checked. Full repository style and test scripts could not run without a checkout.

Risk

Low. Only previously accepted negative exit codes now raise, like positive failures.

How to revert

Revert the commit. No config or saved-state changes.

Signed-off-by: Minh Vu <vuhoangminh97@gmail.com>
@greptile-apps

greptile-apps Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Low impact] This PR appears safe to merge.

Summary

GPU queries now raise when nvidia-smi returns any nonzero exit code, including negative codes from signal termination.

  • GPU queries reject every nonzero nvidia-smi exit.

Reviews (1) · Last reviewed commit: "Fix signal-terminated nvidia-smi queries" · Reviewed by Greptile

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.66%. Comparing base (a0a3501) to head (b530a29).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5543      +/-   ##
==========================================
+ Coverage   71.65%   71.66%   +0.01%     
==========================================
  Files        1048     1048              
  Lines      112323   112323              
==========================================
+ Hits        80480    80493      +13     
+ Misses      31843    31830      -13     
Flag Coverage Δ
unit-tests 71.66% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Report signal-terminated nvidia-smi queries as failures

3 participants