Skip to content

Put the Docker image back on Node.js 24 and check that stated versions agree - #6

Open
fasharif wants to merge 7 commits into
mainfrom
fix/node-24-image-and-version-drift
Open

fasharif wants to merge 7 commits into
mainfrom
fix/node-24-image-and-version-drift

Conversation

@fasharif

@fasharif fasharif commented Oct 4, 2026 •

Copy link
Copy Markdown
Owner

What was wrong

  1. The Docker image no longer matched the published results. build(deps): Bump node from 24-trixie-slim to 26-trixie-slim #3, merged on 3 October 2026, moved the image's Node.js stage from node:24-trixie-slim to node:26-trixie-slim. The published time run had been measured in that image the day before with Node.js v24.21.0, and the README and docs/results.md give ./scripts/run-docker-timing.sh --cpus 2,4 --memory 2g as the way to reproduce it. That script builds the Dockerfile, and an image built from main reports Node.js v26.10.0. The Dockerfile's own header, ADR 16, the runbook, the study script and CI still said Node.js 24. Node.js 26 is also not a long-term support line yet: nodejs.org lists v26.10.0 with lts: false, and the Node.js release schedule makes it one on 28 October 2026.
  2. Nothing stopped it happening again. Dependabot was free to propose any newer line of the base images, and no check compared the Dockerfile with CI, the documents or the published run. CI does not build the image, so build(deps): Bump node from 24-trixie-slim to 26-trixie-slim #3 passed it.
  3. ruff targeted Python 3.10, while mypy, ADR 9, the Python worker's docstring and the study script all say Python 3.12 or later.
  4. summary.csv was written with CR LF line endings, unlike runs.csv and unlike the copy stored in git, so running the recorded report command on the committed run did not reproduce that file byte for byte.
  5. The README's link to Algorithm-Energy-C pointed at this repository's own front page, which is the README the reader is already on.

What changed

  • Dockerfile: the Node.js stage is node:24-trixie-slim again, at the digest that the published run recorded in data/runs/20261002T211256Z-docker/environment.txt (Docker Hub still serves that digest for the tag). The published results, the committed run data and CI's node-version are unchanged.
  • .github/dependabot.yml: for the Docker image, Dependabot ignores Node.js 25 and later and Python 3.13 and later, which leaves it the newer digests of the two lines the results were measured with.
  • tests/integration/test_versions.py (new, runs with make integration in the existing workers job; it only reads files): takes the Node.js and Python lines from the Dockerfile's FROM lines and fails when another place names a different one: the Dockerfile header, the workflow, pyproject.toml, the requirements headers, the study script, the runbook, ADR 9 and 16, study.md and the README. It also checks Dependabot's bounds against the Dockerfile, the oldest supported Node.js across package.json, CI, the study script and the README, and that the Dockerfile's base image tags are the ones the published Docker run recorded. It compares version lines and tags, not patch releases or digests.
  • pyproject.toml: ruff's target-version is py312. No code needed rewriting.
  • analysis/greenbench_analysis/report.py: summary.csv lines end with LF; the test reads the file as bytes and checks it.
  • README.md: the Algorithm-Energy-C link goes to the tree at 0bf73c8, the last commit on main before greenbench was merged; the testing section describes the version test, says that it does not compare digests, and says that CI does not build the Docker image.
  • docs/decisions.md: ADR 16 has a status note with what happened, what the version test does and does not compare, and the date Node.js 26 becomes LTS.

How it was verified

Everything below ran locally in Docker (Docker Desktop 4.93.0, engine 29.8.1, on Windows 11) on 4 October 2026. In review the same checks were run again, except actionlint, the Python 3.14 commands and the Node.js 18 and 20 tests, which CI ran. CI run 37216337524, on commit 6b5ad5f, finished with all eleven jobs successful (158 integration tests; 107 Python tests on 3.12 and on 3.14; 30 Node.js tests). The last commit, b515f0d, changes only the wording of README.md and docs/decisions.md; the version tests, make integration, make check-python and make lint-shell pass on it locally, and its CI run, 37218096304, also finished with all eleven jobs successful.

  • Image built from this branch: Node.js v24.21.0, npm 11.19.0, Python 3.12.14, gcc 14.2.0, the versions that docs/results.md names. Built from main: Node.js v26.10.0.
  • In that image, what the workflow's jobs run: make and make test with gcc and with clang; make sanitize; make integration PYTHON=python (158 passed: the 130 on main and 28 new cases); make smoke PYTHON=python; make check-python PYTHON=python (ruff, ruff format, mypy, 107 tests); make check-node (npm ci, ESLint, tsc, 30 tests); make lint-shell (ShellCheck 0.10.0). actionlint 1.7.12 from its image: clean.
  • The statistics job's four commands on Python 3.14.7 (python:3.14-slim-trixie): pass. node --test on Node.js 18.20.8 and 20.20.2: 30 tests pass on each.
  • The new tests fail without the fixes: test_versions.py run against main fails in 13 cases (the Node.js line, ruff's target, the missing Dependabot bounds, the base image tags), and the summary.csv assertions fail against main's report.py.
  • The recorded report command, run on data/runs/20261002T211256Z-docker/, reproduces README.md, docs/results.md, both charts and summary.csv byte for byte. On main the first four already matched and summary.csv did not.
  • docker run --rm -v "$PWD/results:/work/results" <image>, as in the README, writes the smoke report, and its meta.json records Node.js v24.21.0.
  • scripts/run-docker-timing.sh --rehearse --cpus 2,4 --memory 2g completes (run from a temporary copy with other image and container names, because the machine was shared). Its environment.txt records the same two base images as the published run, and its meta.json the same compiler and interpreter versions.
  • .github/dependabot.yml validates against the Dependabot schema (check-jsonschema 0.38.2, vendor.dependabot).
  • Dependabot's own code honours the bounds. I ran its configuration parser and Docker update checker from ghcr.io/dependabot/dependabot-updater-docker (digest sha256:6ba99ff33a22…) on this branch's Dockerfile. With this branch's dependabot.yml it reports node 24-trixie-slim as up to date and proposes only the newer digest of python:3.12-slim-trixie. With main's dependabot.yml it proposes node 26-trixie-slim and python 3.14-slim-trixie.
  • The log of Dependabot's Docker run of 2 October 2026 (run 37075475332) shows why only Node.js moved then: it skipped the 3.14-slim-trixie, 3.13-slim-trixie and 3.12-slim-trixie tags "due to cooldown period" and reported no update for Python. Without the Python bound, the update checker proposes 3.14-slim-trixie (the item above).

Not verified:

  • The time study was not measured again: it needs a machine doing nothing else. The published numbers are untouched, and they were measured with the image this pull request restores.
  • The hosted Dependabot service itself: the check above ran Dependabot's parser and update checker from its image, not the service, which reads dependabot.yml on GitHub's side. Its next weekly run will show the same or not. If another line gets through, the version test fails on that pull request.

Left alone, and why

  • Moving the study to Node.js 26. That is a new decision rather than a fix: ADR 16, CI, the runbook and the study script change together, and the published times have to be measured again on a quiet machine.
  • A CI job that builds the Dockerfile. It would not have caught build(deps): Bump node from 24-trixie-slim to 26-trixie-slim #3: the Node.js 26 image builds and passes every check above. It would catch a base image that stops building, but it is a new job, so I have left it as a separate choice.
  • @types/node 26.6.3 (build(deps-dev): Bump @types/node from 24.13.6 to 26.6.3 in /workers/node in the node-dev-tools group #4). The type definitions are now for Node.js 26 while CI checks and tests on Node.js 24, 20 and 18. Lint and type-check pass, and the worker tests pass on all three, so I did not revert it or add a rule for it.
  • The Python base image's digest, and digest refreshes in general. Docker Hub now serves a newer digest for python:3.12-slim-trixie than the one pinned, and it holds Python 3.12.15, where the pinned digest and the published results have 3.12.14. The pinned one is what the published run recorded, so I left the update to Dependabot, whose update checker proposes it (see above). Merging such a refresh changes the patch release that run-docker-timing.sh measures, and neither CI nor the version test objects: the test compares lines and tags on purpose. Whether to take digest refreshes while the published results name 3.12.14, or to hold the digests until the study is measured again, is a separate choice that I have left open; ADR 16 and the README now say what the test does not cover.
  • The repository's name and description, which still describe Algorithm-Energy-C: these are repository settings.
  • The energy study: it needs bare-metal Linux.
  • ubuntu-latest: an annotation on the latest CI run on main says that the label moves to Ubuntu 26 from 19 October 2026. I built and tested the C code, ran make sanitize and ran ShellCheck in ubuntu:26.04 (GCC 15.2.0, Clang 21.1.8, ShellCheck 0.11.0), and all pass, so I left the runner label as it is.

🤖 Generated with Claude Code

fasharif and others added 7 commits October 4, 2026 20:01
A dependency update (#3) moved the image's Node.js stage to
node:26-trixie-slim. It was merged the day after the published time run
had been measured in this image with Node.js v24.21.0. The README and
docs/results.md give ./scripts/run-docker-timing.sh as the way to
reproduce that run, and the script builds this image, so the command no
longer reproduced the versions the results name. The Dockerfile's own
header, ADR 16, the runbook, the study script and CI all still say
Node.js 24, and Node.js 26 does not become a long-term support line
until 28 October 2026 (the nodejs/Release schedule).

Put back the line the Dockerfile had before: the digest that the run
recorded in data/runs/20261002T211256Z-docker/environment.txt. Docker
Hub still serves that digest for the tag, and the image built from it
reports Node.js v24.21.0, Python 3.12.14 and gcc 14.2.0, as
docs/results.md does.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ruff still targeted Python 3.10. The Python worker once claimed to run
on 3.10 or later, which nothing tested; f66878c corrected ADR 9 and the
worker's docstring to 3.12 or later, which is what CI tests, and ruff's
target was left behind. mypy's python_version and the study script's
check already said 3.12. With the old target, ruff's upgrade rules held
the code to syntax that 3.10 understands.

ruff check and ruff format --check pass unchanged with the new target,
so no code needed rewriting.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Putting the base image back does not stop the same drift next week:
Dependabot would propose node:26-trixie-slim again, and nothing in CI
compares the Dockerfile with what the documents and the published run
say.

- Dependabot now ignores Node.js 25 and later and Python 3.13 and later
  for the Docker image, so it only refreshes the digests of the two
  lines that the published results were measured with. Moving to
  another line stays a decision for a person, because CI, the scripts
  and the documents have to change with it and the time study has to be
  measured again.
- tests/integration/test_versions.py reads the Dockerfile's two FROM
  lines and fails when another place names a different version: the
  Dockerfile's own header, the workflow, pyproject.toml, the
  requirements headers, the study script, the runbook, ADR 9 and 16,
  study.md and the README. It also checks that Dependabot's bounds are
  the lines after the Dockerfile's, that the oldest supported Node.js
  is the same in package.json, CI, the study script and the README, and
  that the Dockerfile's base image tags are the ones the published
  Docker run recorded in its environment.txt. It reads files only, and
  runs with the integration tests in the existing CI job.
- ADR 16 gets a status note with what happened and when Node.js 26
  becomes a long-term support line. The README describes the test and
  says that CI does not build the image.

On main the new test fails in 13 places (the Node.js line, ruff's
target and the missing Dependabot bounds); with the two commits before
this one it passes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The report wrote summary.csv through Python's csv module with its
default line ending, CR LF, while the harness writes runs.csv with LF
and .gitattributes stores every text file with LF. The copy committed
in data/runs/ therefore has LF endings, and running the recorded report
command again on the same run gave a summary.csv that differed from it
in every line ending, although docs/results.md, both charts and the
README block came out byte for byte the same.

Write LF. The test reads the file as bytes, because read_text converts
CR LF to LF and so could not see the difference; it fails without the
change.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The README's link to Algorithm-Energy-C pointed at this repository's
front page, which is the README the reader is already on: greenbench
replaced the C benchmark in the same repository. Link to the tree at
0bf73c8, the last commit on main before greenbench was merged, and say
that the benchmark lived here. A link to a commit keeps working if the
repository is renamed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The notes added with the Dependabot bounds described its behaviour as a
fact ("only refreshes the digests"). The bounds are configuration that
was validated against the Dependabot schema but cannot be run locally,
so the Dockerfile header, the comment in dependabot.yml and ADR 16 now
describe the configuration; the version test is what fails if another
line gets through.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The README said the version tests fail when the Dockerfile's base images
are not the ones that the published Docker run recorded, and ADR 16 that
they fail when the places name different versions. The test compares the
Node.js and Python lines (24, 3.12) and the base image tags, and leaves
patch releases and digests out, so that a digest refresh of the same tag
passes.

That matters for what the results name: the digest that Docker Hub served
for python:3.12-slim-trixie on 4 October 2026 holds Python 3.12.15, and
the pinned digest, which the published run used, holds 3.12.14. Say so in
both places, so that nobody reads the test as holding the exact images.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant