From dd52d6d37d4304f34ebdacdf93313fb9b79e37e3 Mon Sep 17 00:00:00 2001 From: Scott McCarty Date: Wed, 4 Nov 2026 17:37:50 -0500 Subject: [PATCH 1/3] feat: jq in the image, and a stated boundary for the shell test toolbox Two repos in the fleet conversion failed `sdlc ci` on pytest for the same reason, and it was not their code: `jq: command not found`. - crunchtools/josui: all four entrypoint tests. The entrypoint is a shell script and calls jq at line 87. - crunchtools/claude-agent: 4 of 103 shell-hook tests. mypy, pip-audit, the constitution validator and CodeQL all pass there; the only external command the hook calls is jq. This is not one repo's quirk. Claude Code hands a hook its event as JSON on stdin, hooks are shell scripts, and jq is the smallest correct way to read JSON from bash. Both repos declare `Requires: jq` in their own package manifests (packaging/claude-agent.spec:16 and the debian control file; josui's Containerfile installs it). More hook-shipping repos are coming. The alternative was to run those tests inside each repo's own built image with the `pytest-in-image` kind. That is the right answer for a per-repo runtime dependency, and it is the answer for crunchtools/tig, whose tests need the image's install paths (crunchtools/tig#29). It is the wrong answer here: claude-agent's reference Containerfile installs the PUBLISHED package, so building it in the gate would test the last release rather than the pull request. Its real install test is already the 8-distro matrix in packages.yml. Those four are unit tests of a shipped script whose declared dependency is jq, and unit tests belong in the gate image. So jq goes in, and the boundary goes in as a comment above the `dnf install`, where someone adding the next package will read it: beyond the checks this image carries bash, coreutils, git, curl, python3 and jq, and nothing else. A test that reaches for socat, nc, node or a repo's own runtime is an integration test and belongs in `pytest-in-image` against that repo's image, or in a new kind. Without a written line this list becomes a host-tools grab bag one repo at a time. tests/contract.sh now asserts jq, curl and bash execute, alongside the checks it already proves. That makes the toolbox a tested property of the image rather than an accident of the base, which matters because the base is Hummingbird and deliberately thin -- `which` is already in this list for exactly that reason. RT#1552 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R7Y1jesjfgxHeqtKJCvay6 --- CHANGELOG.md | 26 ++++++++++++++++++++++++++ Containerfile | 18 +++++++++++++++++- IMAGE_TAG | 2 +- gatehouse-findings.toml | 22 ++++++++++++++++++++++ pyproject.toml | 2 +- tests/contract.sh | 5 ++++- 6 files changed, 71 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 98a772e..3254f95 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,32 @@ All notable changes to this project are documented here, following ## [Unreleased] +## [0.3.6] - 2026-11-04 + +### Added + +- **jq, and a stated boundary for the shell test toolbox.** Claude Code hands a + hook its event as JSON on stdin, hooks are shell scripts, and jq is the + smallest correct way to read JSON from bash. So a repo that ships such a hook + has unit tests that run the script as a subprocess -- and those failed here + with `jq: command not found`, which is not a verdict about the code. Two + repos hit it within a day of the fleet conversion (crunchtools/josui's + entrypoint tests; 4 of crunchtools/claude-agent's 103 shell-hook tests), and + both declare `Requires: jq` in their own package manifests. + + The alternative was to run those tests inside each repo's own image. That is + right for a per-repo runtime dependency and wrong here: claude-agent's + reference Containerfile installs the PUBLISHED package, so building it in the + gate would test the last release rather than the pull request. + + The boundary now lives as a comment above the `dnf install` where it is read: + beyond the checks, this image carries bash, coreutils, git, curl, python3 and + jq, and nothing else. A test that reaches for socat, nc, node or a repo's own + runtime is an integration test and belongs in `pytest-in-image` against that + repo's image, or in a new kind. `tests/contract.sh` now asserts jq, curl and + bash execute, so the toolbox is a tested property rather than an accident of + the base image. + ## [0.3.5] - 2026-11-04 ### Changed diff --git a/Containerfile b/Containerfile index 2953776..6073128 100644 --- a/Containerfile +++ b/Containerfile @@ -87,7 +87,23 @@ ENV SDLC_GOURMAND_VERSION=${GOURMAND_VERSION} \ # Hummingbird does not ship it. Without it `codeql database create` fails with # "Could not find a valid Python distribution" on an image that obviously has # one. -RUN dnf install -y git-core tar zstd which \ +# +# THE SHELL TEST TOOLBOX, and its boundary. Beyond the checks themselves this +# image carries exactly: bash, coreutils, git, curl, python3 and jq. Nothing +# else. jq is here because Claude Code hands hooks their event as JSON on +# stdin, hooks are shell scripts, and jq is the smallest correct way to read +# JSON from bash -- so a repo that ships such a hook has unit tests that run +# the script as a subprocess, and two of them (crunchtools/josui, +# crunchtools/claude-agent) declare `Requires: jq` in their own packages. That +# is a shared protocol, not one repo's quirk. +# +# A test that reaches for socat, nc, node, or a repo's own runtime is an +# integration test: it runs through the `pytest-in-image` kind against the +# repo's OWN built image, or it gets a new kind. It does not get a package +# here. Without that line this list becomes a host-tools grab bag, one repo at +# a time. tests/contract.sh asserts the toolbox, so it is a tested property +# rather than an accident of the base image. +RUN dnf install -y git-core tar zstd which jq \ && dnf clean all \ && rm -rf /var/cache/dnf diff --git a/IMAGE_TAG b/IMAGE_TAG index c2c0004..449d7e7 100644 --- a/IMAGE_TAG +++ b/IMAGE_TAG @@ -1 +1 @@ -0.3.5 +0.3.6 diff --git a/gatehouse-findings.toml b/gatehouse-findings.toml index f83927e..69fbbb7 100644 --- a/gatehouse-findings.toml +++ b/gatehouse-findings.toml @@ -31,3 +31,25 @@ rebuild is meant to move. The exact-pin rule is about the TOOLS whose verdicts \ this image produces, and all of those are pinned exactly -- the contract test \ asserts each reported version equals the version pinned at build time. """ + +[[findings]] +agent = "constitution" +category = "constitution" +file = "Containerfile" +evidence = "RUN dnf install -y git-core tar zstd which jq \\" +status = "not-a-bug" +reason = """ +Same reasoning as the base-image finding above, and the same four unpinned \ +neighbours: git-core, tar, zstd and which have never carried exact RPM \ +versions on this line. The Container Image profile requires a weekly rebuild \ +(`0 6 * * 1`) whose entire purpose is to pick up OS security updates; pinning \ +an OS package to an exact version is how you build an image that stops \ +receiving them, which is the opposite of what the cron is for. What the \ +constitution pins to exact versions is the CHECK tools -- ruff, mypy, semgrep, \ +pip-audit, CodeQL, Gourmand, Gatehouse -- because a verdict that changes under \ +you is not a verdict; those are pinned by ARG and reported by `sdlc versions`. \ +jq is not a check, it is part of the shell test toolbox, and its filter \ +language is stable across every version the fleet ships. Pinning it would \ +freeze a CVE-bearing RPM into the gate image for no gain in reproducibility \ +that the digest-addressed release does not already provide. +""" diff --git a/pyproject.toml b/pyproject.toml index f38c855..2f32bb5 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -10,7 +10,7 @@ [project] name = "python-sdlc" -version = "0.3.5" +version = "0.3.6" description = "One container image and one script for every CrunchTools Python SDLC check" requires-python = ">=3.14" license = "AGPL-3.0-or-later" diff --git a/tests/contract.sh b/tests/contract.sh index e487e95..570cad0 100755 --- a/tests/contract.sh +++ b/tests/contract.sh @@ -43,7 +43,10 @@ for probe in \ "gourmand:gourmand --version" \ "codeql:codeql version --format=terse" \ "uv:uv --version" \ - "git:git --version" + "git:git --version" \ + "jq:jq --version" \ + "curl:curl --version" \ + "bash:bash --version" do name=${probe%%:*} cmd=${probe#*:} From 232946f360c8659bec9aef4f6744aa52b601aabd Mon Sep 17 00:00:00 2001 From: Scott McCarty Date: Wed, 4 Nov 2026 17:43:13 -0500 Subject: [PATCH 2/3] fix: the release race that cost v0.3.5 its image `container.yml`'s concurrency group was keyed on `github.run_id`. That value is unique per run, so every run sat in a group of its own and the group controlled nothing at all. A merge to main and the tag `release.yml` creates from it are two pushes of the SAME commit. Both built it, both pushed the same `:` tag, the second replaced the first's manifest, and then the loser's "Verify the pushed digest" step failed with `manifest unknown` on a digest that no longer existed. v0.3.4 happened to win that race. v0.3.5 lost it: the main run moved `latest`, the tag run failed, and the `0.3.5` tag was never created. There is a git tag and a GitHub release for v0.3.5 with no image behind them, which is the one state a fleet pinning this image by exact release cannot survive -- `sdlc` exits 2 with "manifest unknown" and nothing is checked. The only reason no repo is broken by it is that the promote job which opens the version-bump pull requests lives in the run that failed, so the bump never went out. Keyed on `github.sha`, the two runs serialize and the second verifies the digest it just pushed. `cancel-in-progress` stays false off pull requests: cancelling the main run when its tag arrives would abandon a push half-finished. v0.3.5 is superseded by this release rather than backfilled. RT#1552 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R7Y1jesjfgxHeqtKJCvay6 --- .github/workflows/container.yml | 17 ++++++++++++++++- CHANGELOG.md | 21 +++++++++++++++++++++ 2 files changed, 37 insertions(+), 1 deletion(-) diff --git a/.github/workflows/container.yml b/.github/workflows/container.yml index 850bf99..5dc36e1 100644 --- a/.github/workflows/container.yml +++ b/.github/workflows/container.yml @@ -25,8 +25,23 @@ on: - cron: "0 6 * * 1" workflow_dispatch: +# Keyed on the COMMIT, not the run, so the push-to-main run and the tag run of +# the same commit SERIALIZE instead of racing. `github.run_id` is unique per +# run, so the old group put every run in a group of its own and controlled +# nothing: both builds pushed the same `:` tag, the second replaced the +# first's manifest, and the loser's "Verify the pushed digest" step then failed +# with `manifest unknown` on a digest that no longer existed. +# +# That is not theoretical. It cost v0.3.5 its image: the main run won, `latest` +# moved, the tag run lost, and the `0.3.5` tag was never created -- a GitHub +# release with no image behind it, which is the one state a pinned fleet cannot +# survive. v0.3.4 hit the same race and happened to win it. Serialized, the +# second run pushes and then verifies its own digest, which is the current one. +# +# cancel-in-progress stays false off pull requests: cancelling the main run +# when its tag arrives would abandon a push half-done. concurrency: - group: ${{ github.workflow }}-${{ github.event_name == 'pull_request' && github.ref || github.run_id }} + group: ${{ github.workflow }}-${{ github.event_name == 'pull_request' && github.ref || github.sha }} cancel-in-progress: ${{ github.event_name == 'pull_request' }} permissions: {} diff --git a/CHANGELOG.md b/CHANGELOG.md index 3254f95..031cec9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,27 @@ All notable changes to this project are documented here, following ## [0.3.6] - 2026-11-04 +### Fixed + +- **Two builds of the same commit raced and one of them lost its image.** + `container.yml`'s concurrency group was keyed on `github.run_id`, which is + unique per run -- so every run sat in a group of its own and the group + controlled nothing. A merge to main and the tag that `release.yml` then + creates are two pushes of the SAME commit, so both built it and both pushed + the same `:` tag; the second replaced the first's manifest, and the + loser's "Verify the pushed digest" step failed with `manifest unknown` on a + digest that no longer existed. + + v0.3.4 won that race. **v0.3.5 lost it**: the main run moved `latest`, the tag + run failed, and the `0.3.5` tag was never created -- a GitHub release and a + git tag with no image behind them, which is the one state a fleet pinning this + image cannot survive. Nothing had pinned it yet only because the promote job + that opens the bump pull requests is in the run that failed. + + The group is keyed on `github.sha` now, so the two serialize and the second + verifies the digest it just pushed. v0.3.5 is superseded by this release + rather than backfilled. + ### Added - **jq, and a stated boundary for the shell test toolbox.** Claude Code hands a From ec3c3f6ad3201c415c046390c9ccc0572b4e03e4 Mon Sep 17 00:00:00 2001 From: Scott McCarty Date: Wed, 4 Nov 2026 17:48:29 -0500 Subject: [PATCH 3/3] fix: the Containerfile version label, which the validator caught Constitution II: every file that states the version states the same one. I moved IMAGE_TAG and pyproject.toml to 0.3.6 and left `LABEL version="0.3.5"` behind, and `Constitution / Validate constitution` said so: VERSION: Containerfile says 0.3.5, CHANGELOG.md says 0.3.6 (II) Which is the check working. mcp-github's 2.0.0 was bumped by hand in two of its five places and could not be tagged; that is why the rule exists. RT#1552 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R7Y1jesjfgxHeqtKJCvay6 --- Containerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Containerfile b/Containerfile index 6073128..01515bb 100644 --- a/Containerfile +++ b/Containerfile @@ -44,7 +44,7 @@ LABEL description="Every CrunchTools Python SDLC check in one image: ruff, mypy, LABEL org.opencontainers.image.source="https://github.com/crunchtools/python-sdlc" LABEL org.opencontainers.image.description="Every CrunchTools Python SDLC check in one image." LABEL org.opencontainers.image.licenses="AGPL-3.0-or-later" -LABEL version="0.3.5" +LABEL version="0.3.6" # What the image carries, readable without starting anything. `sdlc versions` # prints these and a consuming repo's CI pins them.