Skip to content

cherry-pick(v2.1): clamp the nico-pxe file-descriptor soft limit to the runtime hard limit - #5643

Merged
nv-dmendoza merged 1 commit into
NVIDIA:release/v2.1from
shayan1995:v2.1-cherry-pick-ulimit
Sep 1, 2026
Merged

cherry-pick(v2.1): clamp the nico-pxe file-descriptor soft limit to the runtime hard limit#5643
nv-dmendoza merged 1 commit into
NVIDIA:release/v2.1from
shayan1995:v2.1-cherry-pick-ulimit

Conversation

@shayan1995

Copy link
Copy Markdown
Contributor

This PR backports the file-descriptor limit hardening (#5583) into v2.1, scoped to nico-pxe - the only chart on this line that carries the ulimit fail-fast entrypoint:

  • The requested soft limit is clamped to the runtime hard limit with a logged notice (reproduced failure: a hard limit of 65535 crashloops the pod against the 65536 default, with only dash's terse EINVAL as diagnostic).
  • A nulled or non-numeric fileDescriptorLimit no longer renders ulimit -Sn 0 (a silent zero-fd start): null falls back to the 65536 default, invalid values fail the render.
  • The hardware-health and ssh-console hunks from the original commit are dropped: those charts do not have the FD-limit entrypoint on release/v2.1.

Cherry-pick of 00a7a45 (#5583), conflicts resolved by scoping to nico-pxe.

Related issues

Backport of #5583 (fixes #5535 for the v2.1 line).

Type of Change

  • Add - New feature or capability
  • Change - Changes in existing functionality
  • Fix - Bug fixes
  • Remove - Removed features or deprecated functionality
  • Internal - Internal changes (refactoring, tests, docs, etc.)

Breaking Changes

  • This PR contains breaking changes

Testing

  • Unit tests added/updated
  • Integration tests added/updated
  • Manual testing performed
  • No testing required (docs, internal refactor, etc.)

helm unittest on the v2.1 tree: the FD-limit suite passes (5 cases: default+clamp shape, override, null fallback, non-numeric and negative render failures); render checks confirm null falls back to 65536 and invalid values fail with the guard message. The clamp behavior was verified live on dev6 (hard limit 65535) during #5583.

…imit (NVIDIA#5583)

Fixes NVIDIA#5535.

The ulimit fail-fast entrypoints crashloop on runtimes whose container
hard nofile limit is below 65536 (reproduced on dev6: hard limit 65535 →
all three deployments CrashLoopBackOff). The entrypoints now clamp to
`$(ulimit -Hn)` with a logged notice, and print both limits before
exiting on failure. Render-time validation replaces the silent-zero
mode: null falls back to the 65536 default, non-numeric/non-positive
values fail the render (previously rendered `ulimit -Sn 0`, which dash
accepts).

- [x] helm unittest: 5 cases per chart (default+clamp shape, override,
null fallback, non-numeric and negative render failures) — 15/15 pass
- [ ] dev6 verification with the site FD-limit workarounds removed (in
progress; results to follow as a comment)

(cherry picked from commit 00a7a45)
@shayan1995
shayan1995 requested a review from a team September 1, 2026 20:01
@shayan1995
shayan1995 requested a review from a team as a code owner September 1, 2026 20:01
@coderabbitai

coderabbitai Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 9f7de137-f8d0-4a69-91ab-36b5a9bfe1be

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-01T20:03:52.907746Z 13c84c3 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 13c84c3894

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ulimit -Sn {{ .Values.fileDescriptorLimit | int }} || exit 1
{{- $fdLimit := 65536 }}
{{- if not (kindIs "invalid" .Values.fileDescriptorLimit) }}
{{- $fdLimit = .Values.fileDescriptorLimit | int }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject fractional file-descriptor limits rather than truncating

When an operator supplies a fractional number through a values file or --set-json (for example, fileDescriptorLimit: 32768.9), Sprig's int conversion truncates it to 32768, after which the positivity check accepts it. This contradicts the newly documented positive-integer-only contract and silently changes invalid input instead of failing the render; validate the original value's numeric kind/integrality before converting it.

AGENTS.md reference: AGENTS.md:L241-L249

Useful? React with 👍 / 👎.

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown

🔐 TruffleHog Secret Scan

No secrets or credentials found!

Your code has been scanned for 700+ types of secrets and credentials. All clear! 🎉

🔗 View scan details

🕐 Last updated: 2026-09-01 20:04:55 UTC | Commit: 13c84c3

@nv-dmendoza nv-dmendoza left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, backports fix

@nv-dmendoza
nv-dmendoza merged commit 36201e7 into NVIDIA:release/v2.1 Sep 1, 2026
122 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants