Skip to content

fix(worker-utils): bump the worker library pin so the transport rotation ships - #1496

Open
balajinvda wants to merge 1 commit into
mainfrom
fix/bump-worker-library-pin
Open

fix(worker-utils): bump the worker library pin so the transport rotation ships#1496
balajinvda wants to merge 1 commit into
mainfrom
fix/bump-worker-library-pin

Conversation

@balajinvda

@balajinvda balajinvda commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Issues

Relates to #1382

Why

The four worker services consume src/libraries/go/worker as a pinned pseudo-version, not as local source:

src/compute-plane-services/{worker-utils,worker-init,worker-task,worker-llm-credentials}/go.mod
  github.com/NVIDIA/nvcf/src/libraries/go/worker v0.0.0-20260710034659-973443ac16c3

That pin is from 10 July, so every worker image built since then excludes anything merged into the library — including the QUIC transport rotation from #1383.

This was verified, not assumed. Two dev images built from different commits, one containing #1383, came out with identical digests:

gh.1136-748ba8d1   sha256:5a0a158a...
gh.1182-ce91506b   sha256:5a0a158a...

Extracting the layers and grepping the binary:

'failed to dial host'            1   <- control, the proxy package IS linked
'rotating quic transport'        0   <- #1383 absent
'consecutive dial failures'      0   <- #1383 absent

Deploying either image would have shown the failure unchanged and read as the fix not working, when it was never compiled in.

What changed

The pin moves to a commit containing #1383, in all four consumers plus their go.sum:

v0.0.0-20260710034659-973443ac16c3  ->  v0.0.0-20260902221359-b62eeeca37a7

All four are bumped together on purpose. Leaving any behind produces images that disagree about the library version.

Customer Release Notes

Not customer visible on its own. It is what allows the worker-side fix in #1383 to reach a built image.

Plan Summary

Not applicable.

Usage

Verify any worker image before drawing conclusions from a deploy:

skopeo copy docker://<image> dir:/tmp/x
for f in /tmp/x/*; do gzip -t "$f" 2>/dev/null && zcat "$f" || cat "$f"; done > all.bin
grep -ac 'rotating quic transport' all.bin      # must be >= 1

Testing

Built //src/compute-plane-services/worker-utils/worker:worker against the new pin and grepped the compiled archive:

proxy.a  'rotating quic transport'   2
         'consecutive dial failures' 2
         'failed to dial host'       2   (control)

Before the bump the first two were zero in the shipped image.

The module proxy serves the new pseudo-version (200 for the .info endpoint), so CI needs no special resolution. Local go get required GOPRIVATE because the proxy will not resolve the nested module path by SHA without it, but that is a workstation concern and not a build dependency.

Notes

The pin is a recurring trap: it is silent, and the failure mode is a build that succeeds and produces an image identical to the previous one. Worth considering whether these services should consume the library through the workspace instead, so library changes cannot be silently excluded. Not attempted here.

References

None

Related Pull Requests

#1383 is the rotation this makes shippable. #1492 adds diagnostics for it and is not included in this pin.

Dependencies

No new third-party dependencies. Only the in-repo library pin and the corresponding go.sum entries change.

Summary by CodeRabbit

  • Chores
    • Updated internal worker service components to use a newer worker library version.
    • No user-visible functionality changes were introduced.

…ion ships

The four worker services consume src/libraries/go/worker as a pinned
pseudo-version, not as local source, and that pin has been at a 10 July commit:

    v0.0.0-20260710034659-973443ac16c3

So every worker image built since then excludes anything merged into the
library, including the QUIC transport rotation from #1383. Verified rather than
assumed: two dev images built from different commits, one of which contained
#1383, had identical digests, and extracting the layers showed the rotation
strings absent while the surrounding proxy code was present.

    gh.1136-748ba8d1  sha256:5a0a158a...  'rotating quic transport' 0
    gh.1182-ce91506b  sha256:5a0a158a...  'rotating quic transport' 0
                                          'failed to dial host'     1  (control)

Deploying either would have shown no improvement and looked like the fix
failing, when the fix was never in the binary.

Bumping to a commit that contains it fixes that. Confirmed by building
worker-utils locally against the new pin and grepping the compiled archive:

    proxy.a  'rotating quic transport'   2
             'consecutive dial failures' 2
             'failed to dial host'       2  (control)

The module proxy serves the new pseudo-version, so CI needs no special
resolution. Local `go get` did need GOPRIVATE, because the proxy will not
resolve the nested module path by SHA without it; that is a workstation
concern, not a build dependency.

All four consumers are bumped together. Leaving any behind would produce images
that disagree about the library version.

Co-Authored-By: Balaji Ganesan <bganesan@nvidia.com>
@balajinvda
balajinvda requested a review from a team as a code owner September 2, 2026 23:23
@balajinvda
balajinvda requested a review from estroz September 2, 2026 23:23
@coderabbitai

coderabbitai Bot commented Sep 2, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e053e132-8fed-4758-8da3-f1f00ab13b74

📥 Commits

Reviewing files that changed from the base of the PR and between 6c62961 and 6302383.

⛔ Files ignored due to path filters (4)
  • src/compute-plane-services/worker-init/go.sum is excluded by !**/*.sum
  • src/compute-plane-services/worker-llm-credentials/go.sum is excluded by !**/*.sum
  • src/compute-plane-services/worker-task/go.sum is excluded by !**/*.sum
  • src/compute-plane-services/worker-utils/go.sum is excluded by !**/*.sum
📒 Files selected for processing (4)
  • src/compute-plane-services/worker-init/go.mod
  • src/compute-plane-services/worker-llm-credentials/go.mod
  • src/compute-plane-services/worker-task/go.mod
  • src/compute-plane-services/worker-utils/go.mod

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.


📝 Walkthrough

Walkthrough

The pull request updates the worker library dependency in four compute-plane Go modules from the July 2026 pseudo-version to the September 2026 pseudo-version.

Changes

Worker dependency alignment

Layer / File(s) Summary
Update worker module requirements
src/compute-plane-services/worker-init/go.mod, src/compute-plane-services/worker-llm-credentials/go.mod, src/compute-plane-services/worker-task/go.mod, src/compute-plane-services/worker-utils/go.mod
All four modules now require worker version v0.0.0-20260902221359-b62eeeca37a7.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Merge Risk: ⚪ Minimal · up to 63023

This change updates all four worker services to the library revision containing the transport fix, with matching checksum updates. No actionable merge-blocking risk remains beyond normal checks and review.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title follows Conventional Commits format and accurately describes the dependency pin update that delivers the QUIC transport rotation.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (4 skipped: 4 unsupported.)

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/bump-worker-library-pin

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant