Repository navigation
fix: pip-audit says when the network failed instead of passing or failing - #67
Conversation
| "run", | ||
| "--locked", | ||
| "--all-extras", | ||
| "--with", |
There was a problem hiding this comment.
CRITICAL (Constitution): The pip-audit invocation does not pin an exact version. The constitution requires every tool in the image to be pinned to an exact version so consumers receive a fixed set of tool versions.
Suggestion: Pin pip-audit to an exact version in the uv invocation, for example with --with pip-audit==X.Y.Z.
Evidence:
"--with",
"pip-audit",
"pip-audit",
There was a problem hiding this comment.
Fixed in 406f7a4
--with pip-audit=={BAKED["pip-audit"]}, which is the Containerfile's PIP_AUDIT_VERSION (2.9.0). With no baked version it is a 2. Both cases are tested.
…ling pip-audit 2.9.0 exits 1 for a PyPI read timeout, the same status as a vulnerability, and an egress path answering 404 makes it skip every package and exit 0. Raise --timeout to 60, retry a network failure twice with 10s/30s backoff, and report one that outlasts the retries as exit 2 (could not run). "Not found" for a package uv.lock takes from PyPI counts as a failed lookup. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R7Y1jckWHhAgXg2WBwkWbb
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R7Y1jckWHhAgXg2WBwkWbb
f62a5a3 to
4ac95c2
Compare
| "run", | ||
| "--locked", | ||
| "--all-extras", | ||
| "--with", |
There was a problem hiding this comment.
CRITICAL (Constitution): The pip-audit tool is fetched without an exact version pin. The constitution requires every tool to be pinned to an exact version; pip-audit is not one of the explicitly permitted consumer-pinned seams.
Suggestion: Pin pip-audit to an exact version in the uv run --with specification, and ensure the installed version is reported alongside the pin.
Evidence:
"--with",
"pip-audit",
"pip-audit",
There was a problem hiding this comment.
Fixed in 406f7a4
Same fix as the thread above: pinned to the image's baked pip-audit version, and an unknown version is a 2.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R7Y1jckWHhAgXg2WBwkWbb
There was a problem hiding this comment.
Gatehouse found no issues.
Reviewed 1 commit since 4ac95c2 (incremental).
Check pip-audit's "Found N known vulnerabilit" summary before classifying anything: it prints only after every lookup returned, so a finding beside a version PyPI no longer has, or beside uv's own fetch noise, stays a 1. A run that skipped every registry package installed in the venv is a failed lookup, which closes the exit-0 path behind a non-PyPI mirror. Test fixtures now use pip-audit 2.9.0's real columns layout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R7Y1jckWHhAgXg2WBwkWbb
There was a problem hiding this comment.
Gatehouse found no issues.
Reviewed 1 commit since 406f7a4 (incremental).
2 finding(s) dropped: their evidence is not in the diff.
What was wrong
pip-audit 2.9.0 has two network failure modes, and the wrapper reported both as verdicts:
PyPIService.querycatchesConnectTimeoutbut letsReadTimeout, connection resets and 5xxHTTPErrorescape as tracebacks. Every one exits 1, which is also the status for "found a vulnerability". On the busy Exedore runners the 15s default read timeout trips against a PyPI that answers in under a second (e.g. ashigaru run 44228385707).pypi.org/pypi/<pkg>/<ver>/jsonbecomesSkippedDependencyand pip-audit exits 0. If an egress path answers 404 for everything, every package is skipped and the scan "passes". The wrapper only printed a NOT AUDITED note. I believe this is the push-side "Passed", but I couldn't confirm it: the gateway refused the ashigaru log.The fix (
t_pip_auditinsdlc)--timeout 60(was pip's 15) and--progress-spinner off.uv.locksays a package comes frompypi.organd pip-audit reports it "not found on PyPI", that is a failed lookup, not an unauditable package. It is retried the same way. Local path and vendored dependencies (trentina'spetit-plus) still pass with the NOT AUDITED note, as before.CannotRunError, which exits 2, COULD NOT RUN: neither a pass nor a finding.Why could-not-run fails CI
Exit 2 fails
sdlc ciandsdlc push, the same as every other check here. I did not route it through the CodeQL-style DEFERRED path. A green CI that audited nothing is thepip-audit … || trueproblem this check was written to end. The retries absorb the transient timeouts that caused the reported failures (about 40s of extra wall time in the worst case before giving up), so what's left is a genuine outage, and that should stop a merge and say why. If persistent outages turn out to block pushes in practice, deferral can be added later as a separate decision.Tests
tests/test_pip_audit_resilience.py(also added to the container workflow's unit-test loop):--timeoutargument checkeduv.lock→ 2, pip-audit never runsRan locally: ruff 0.17.1 check and format, Gourmand 1.0.0, Gatehouse 0.26.0 (findings addressed in the diff), host contract,
test_plan.py,test_codeql_resilience.py, and the new test.🤖 Generated with Claude Code
https://claude.ai/code/session_01R7Y1jckWHhAgXg2WBwkWbb