Skip to content

Fixes CI base image pulls by not inheriting runner registry credentials #7330

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
myurasov-nv wants to merge 2 commits into from
Closed

Fixes CI base image pulls by not inheriting runner registry credentials #7330

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
117c243
Stop inheriting runner registry credentials in CI docker config
myurasov-nv Aug 25, 2026
b7b3615
Fail the deps hash step when the base image manifest is unreadable
myurasov-nv Aug 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions .github/actions/_lib/compute-deps-hash/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,15 +63,16 @@ runs:

# Resolve the actual base image digest so a new push of a mutable tag
# (e.g. latest-develop) invalidates the deps cache automatically.
base_image_digest=$(docker buildx imagetools inspect \
# Fail loudly: an unreadable manifest is an auth/registry problem, and
# silently falling back to the tag string stops the deps cache from
# tracking the base image at all.
if ! base_image_digest=$(docker buildx imagetools inspect \
"${ISAACSIM_BASE_IMAGE}:${ISAACSIM_VERSION}" \
--format '{{json .Manifest.Digest}}' 2>/dev/null | tr -d '"' || true)
if [ -n "${base_image_digest}" ]; then
base_image_uniq_id="${ISAACSIM_BASE_IMAGE}:${ISAACSIM_VERSION}:${base_image_digest}"
else
echo "🟠 Could not resolve base image digest, falling back to tag string"
base_image_uniq_id="${ISAACSIM_BASE_IMAGE}:${ISAACSIM_VERSION}"
--format '{{json .Manifest.Digest}}' 2>&1 | tr -d '"'); then
echo "::error::Cannot read manifest for ${ISAACSIM_BASE_IMAGE}:${ISAACSIM_VERSION}: ${base_image_digest}"
exit 1
fi
base_image_uniq_id="${ISAACSIM_BASE_IMAGE}:${ISAACSIM_VERSION}:${base_image_digest}"

mapfile -t manifest_files < <(git ls-files | grep -E "${deps_manifest_pattern}" || true)
file_hash=$(git ls-files -s "${deps_files[@]}" "${manifest_files[@]}" 2>/dev/null \
Expand Down
13 changes: 7 additions & 6 deletions .github/actions/_lib/setup-docker-config/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,18 +26,19 @@ runs:
exit 0
fi

# Seed a CLEAN config - never inherit the runner's registry credentials.
# A persistent `docker login nvcr.io` on the runner is stored inline
# (the credential helper is broken), and inheriting it makes NGC refuse
# to mint a token for the PUBLIC nvidia/* catalog for that org identity
# ("denied: Access Denied") instead of falling back to anonymous.
DOCKER_CONFIG_DIR=$(mktemp -d)
if [ -f "${HOME}/.docker/config.json" ]; then
python3 -c "import json; cfg=json.load(open('${HOME}/.docker/config.json')); cfg['credsStore']=''; cfg.pop('credHelpers',None); json.dump(cfg,open('${DOCKER_CONFIG_DIR}/config.json','w'))"
else
echo '{"credsStore":""}' > "${DOCKER_CONFIG_DIR}/config.json"
fi
echo '{"credsStore":"","auths":{}}' > "${DOCKER_CONFIG_DIR}/config.json"
export DOCKER_CONFIG="${DOCKER_CONFIG_DIR}"
echo "DOCKER_CONFIG=${DOCKER_CONFIG_DIR}" >> "$GITHUB_ENV"

if [ -n "${NGC_API_KEY:-}" ]; then
echo "🔵 Logging into nvcr.io..."
docker login -u '$oauthtoken' -p "${NGC_API_KEY}" nvcr.io
printf '%s' "${NGC_API_KEY}" | docker login -u '$oauthtoken' --password-stdin nvcr.io
else
echo "🟠 NGC_API_KEY not set - skipping nvcr.io login (normal for fork PRs)"
fi
Loading