Skip to content

Latest commit

 

History

History
491 lines (389 loc) · 22.9 KB

README.md

File metadata and controls

491 lines (389 loc) · 22.9 KB

python-sdlc

Every CrunchTools Python check in one container image, run the same way on a laptop and in CI.

Constitution XII (v1.30.0) requires it for every Python repo — any repo with a pyproject.toml — except gourmand and this one, which build what the image contains. gatehouse runs it too, as of gatehouse#134 (constitution#138).

Three commands, three config blocks

Command Runs when Config block in pyproject.toml
sdlc commit the pre-commit hook [tool.sdlc.commit]
sdlc push the pre-push hook [tool.sdlc.push]
sdlc ci the CI job named sdlc ci [tool.sdlc.ci]
  • commit takes seconds, works offline and costs nothing: formatting, lint fixes, and Gourmand's structural review. Everything here is deterministic — the same tree gives the same answer, with no model and no network — so there is no reason to find it at push, where each rejection costs a six-to-eight minute cycle.
  • push renders every verdict, once, over origin/<default-branch>...HEAD. It never builds a container — that is minutes, and a push you wait minutes for is a push people stop making.
  • ci runs push's checks except Gourmand and Gatehouse, which CI already runs through their own workflows, plus the repo's own checks, such as building its image.

Because the laptop and CI run the same image and the same commands, a green push predicts a green CI.

Working a change

In the order that costs least:

sdlc commit && sdlc push          every verdict, before a CI cycle
sdlc pytest -- tests/x.py::test_y -n0    reproduce one failure, just that one
jq . .git/sdlc/last-push.json     what the last run cost, per check

Two things mislead people here. The Python running your tests is the image's, not your machine's, so a result from a host pytest says nothing about this gate — and bytecode your machine compiled is kept out of the container deliberately, so it cannot say anything either. And on a review finding, CI triage is the authority, not the local record: push predicts CI, it does not replace it.

Wiring a repo

A repo carries one file, .sdlc/bin/sdlc (this repo's bin/sdlc-bootstrap), and pins nothing. The launcher ships inside quay.io/crunchtools/sdlc; the bootstrap asks the image for it (sdlc launcher <ref>, which writes that ref into it) and runs it. So the launcher, the image and every tool inside it float together on :latest, for the laptop and CI both, and a release reaches every repo without a pull request.

.pre-commit-config.yaml is examples/pre-commit-floating.yaml: two language: system hooks, .sdlc/bin/sdlc commit and .sdlc/bin/sdlc push. CI is examples/ci.yml, whose job is still named sdlc ci, so the required check does not change:

      - name: sdlc ci
        env:
          SDLC_IMAGE: ${{ vars.SDLC_IMAGE }}
        run: .sdlc/bin/sdlc ci

scripts/convert_repo.py <checkout> converts a repo wired the old way -- the rev-pinned hooks below, or the standalone Gourmand and Gatehouse hooks -- and prints the diff; --apply writes it. It replaces a workflow only when the CI stub covers every job in it (sdlc ci, Gourmand), refreshes gatehouse.yml from examples/gatehouse.yml when it differs only in comments or in the guard's script (an older guard that does not cover .sdlc/), keeps everything else (publish, release, container builds), and refuses with exit 2 a shape it would have to guess at. So that a converted repo never meets GATE CANNOT RUN on its first run, it also writes skip-stacks for a stack this image cannot check yet (node, bootc, docs), and [tool.sdlc.ci] skip for the three container checks when the Containerfile needs build secrets (--mount=type=secret, subscription-manager, RHSM) that sdlc ci is not given; the publish workflow keeps building those with its secrets. A Python repo that names no stacks is left alone, and an existing key that does not already say this is refused rather than rewritten. scripts/fleet_wave.py runs it across the org: a plan by default, one auto-merging pull request per repo with --apply, a TSV ledger either way. It reads GH_TOKEN from the environment for the GitHub API (required with --apply), and pushes with the credentials git already has.

What floating trusts. Whoever can move quay.io/crunchtools/sdlc:latest now decides the host-side code every seat and runner runs, and that code is no longer read in git at a pinned rev before it runs. Only this repo's release workflow, after the contract test, moves the tag, and never backwards. Signing the image and verifying it on pull (a policy.json sigstore entry for quay.io/crunchtools/sdlc) would close the rest, and is not done yet.

An sdlc RPM, later. It installs the same bootstrap as /usr/bin/sdlc on hosts and seats, so a person can type sdlc tiers anywhere. Nothing in a repo changes when it lands: the hooks keep starting the repo's own copy, which is identical, and the RPM never needs a release of its own, because the launcher it fetches floats with the image.

The rev-pinned form still works, and is what a repo has until it converts:

default_install_hook_types: [pre-commit, pre-push]

repos:
  - repo: https://github.com/crunchtools/python-sdlc
    rev: v0.8.0
    hooks:
      - id: sdlc-commit
      - id: sdlc-push

with a CI step that clones this repo at that rev: and runs $RUNNER_TEMP/sdlc/bin/sdlc ci. That rev: pins the launcher, the image and every tool inside it.

A project in a subdirectory

If the repo's Python is not at the root -- a JavaScript repo with a Python service beside it, say -- name the directory in the ROOT pyproject.toml:

[tool.sdlc]
project = "image-server"

Only the uv-driven checks move: mypy, pytest, pip-audit and the sync they share. The constitution validator, Gourmand, Gatehouse, CodeQL and ruff check . stay repo-wide, and ruff's own config is hierarchical, so a nested pyproject.toml still applies to the files under it.

Stacks

One image serves every kind of repo. A stack is the set of checks that only mean something for one kind of repo, and which stacks a repo has is read off its files:

File at the root Stack Status
pyproject.toml python checked
Containerfile or Dockerfile container checked
package.json node not yet
a Containerfile whose FROM is a bootc image bootc not yet
none of these docs not yet

Gourmand, Gatehouse, the constitution validator and hooks-installed belong to no stack and run everywhere. A repo can add or drop a stack, and nothing more:

[tool.sdlc]
stacks = ["container"]
skip-stacks = ["node"]

A Python repo that names neither key runs the python stack alone, exactly as python-sdlc always has, whatever else is in it. Turning the container stack on under 36 pinned repos is its own decision, not a side effect of a bump. sdlc stacks shows what was detected and what runs.

A stack this image names but cannot check yet is refused with exit 2, not left out: a node-only repo judged by the constitution alone would go green on checks that never ran. skip-stacks is how a repo says it accepts that.

A repo with no pyproject.toml writes the same [tool.sdlc] tables in sdlc.toml. A repo with one configures sdlc there, and sdlc.toml beside it is refused.

The container stack

Check Command What it does
containerfile-lint push, ci hadolint over the Containerfile. Without a .hadolint.yaml the threshold is error and DL3007 (floating base tags) DL3041 (unpinned dnf) and SC1072 (raised on an exec-form RUN ["python3", "-c", …]) are ignored, as fleet conventions; warnings such as DL3002 are printed and do not fail. A repo's .hadolint.yaml replaces all of that. A Containerfile hadolint cannot parse (a RUN heredoc) is printed as ADVISORY and SKIPPED, not failed; container-build still builds it
container-build ci podman build, at the tag the next two read
container-smoke ci starts the built image detached, with no network and no capabilities, and reads its state from podman: still running after seconds, or exited 0, passes; any other exit, the OOM killer, or an entrypoint that cannot start (126/127) is a finding. Advisory by default: a finding is printed as FOUND SOMETHING, ADVISORY and does not fail the command; gate = true makes it gate. Exit 0 passes even when podman reports OOMKilled, which nested rootless podman does for any container; OOMKilled counts only beside a non-zero exit. Podman failing to create the container is still a 2. An image whose PID 1 is systemd (/sbin/init, systemd) is SKIPPED with its reason: systemd needs privileges the run withholds
container-scan ci grype over the built image's filesystem, mounted read-only by podman; fails on a fixable Critical by default, and the repo's .grype.yaml applies

Build, smoke and scan are one lane, beside everything else.

What a repo gets with no config at all

Command Runs
commit ruff-check, ruff-format, gourmand, hooks-installed
push lint, mypy, pytest, gourmand, gatehouse, semgrep, pip-audit, constitution, codeql
ci lint, mypy, pytest, semgrep, pip-audit, constitution, codeql

ruff-check and ruff-format fix what they find, so they pass on anything fixable. lint is the same two tools as a verdict — ruff check --no-fix and ruff format --check, changing nothing — and it is the only ruff that push and ci run.

sdlc tiers prints what each command runs in the repo you are in, and which lane each check runs in.

Lanes

A command's checks run in lanes. Steps of one lane run in sequence; lanes run at the same time, so a command takes about as long as its longest chain rather than the sum of everything in it.

The image decides the lanes, by three rules:

  • An image-build check and every pytest-in-image check that runs the image it tags are one lane — the build has to finish before the tests start.
  • Gatehouse is the review lane, always. It is the one check that does not read the worktree: its input is the origin/<default-branch>...HEAD diff, which bin/sdlc computes on the host and pipes in. So it can run beside anything, including a command that rewrites the tree.
  • Everything else is the main lane, in the order tiers prints. That is where uv sync happens, and one lane keeps two syncs out of one venv.

A command that rewrites the tree — anything running ruff-check, ruff-format or gourmand — gets no lanes beside it except that reviewer, because a build context reading those files while they change is judging a tree that no longer exists. That makes sdlc commit sequential by definition.

So sdlc push has two lanes: everything that computes, and the review. sdlc ci has one unless the repo builds an image, because CI gets Gatehouse from its own workflow.

The CPU budget is for the whole gate, not for each container: it is divided between the lanes that run containers, floor of one CPU each. A lane whose every step is a question bin/sdlc answers itself — the reviewer, which is one API call and then waiting — takes a single CPU and is left out of that division, so putting it beside the others costs them nothing. Two lanes are no busier on your laptop than one was.

The budget itself is half the machine, so a gate cannot take the desktop away from the person running it — except where there is no such person. In CI, or where a cpuset, a CPU affinity or a cgroup quota already narrows the machine, the whole allowance is used: something has already decided this process's share, and halving it again double-counts the same protection. Halving CI is what made lanes slower than no lanes on a 4-vCPU runner; see the v0.3.2 changelog entry.

Each lane's output is buffered and printed whole, in the order tiers lists the checks, so two lanes never cut into each other's log. Each block ends with how long its check took.

With one exception, which is the shape sdlc push has: when exactly one lane runs containers and the rest do not, that lane runs in the foreground and streams to your terminal as it always did, and the others are printed after it. Buffering the lane that is doing the work would mean watching a silent terminal for the length of the test suite, which reads like a hang.

What the last run found

Every command leaves a record, inside the git directory rather than in the worktree:

Path What is in it
<git-dir>/sdlc/last-<command>.json each check, its exit, its seconds, its log
<git-dir>/sdlc/logs-<command>/ what each buffered check printed
$ jq -r '.checks[] | "\(.seconds)s \(.check) exit \(.exit)"' .git/sdlc/last-push.json

A check's exit is its status: 0 passed, 1 found something, 2 could not run, 3 deferred to CI, 4 SKIPPED -- reported only by container-smoke, for a systemd PID 1 it cannot boot, and containerfile-lint, for a Containerfile hadolint cannot parse -- and 5 an ADVISORY finding from an ungated container-smoke. A 4 or a 5 never decides the command's own exit.

The checks appear in the order they finished, which with lanes is not the order they were listed in. A check that streamed to your terminal has no log file: the terminal had it.

The record is written, never read. Nothing in this tool consults it to decide anything, because a verdict comes from running the check — if a file could stand in for one, a stale or hand-edited file would be a way past a gate. Failing to write it never fails a gate either; a repo with nowhere to write simply has no record.

If no check finishes for SDLC_LANE_STALL seconds (default 3600) the printer stops waiting on the order, says so, waits for every lane to stop and then prints what is left. That guards against a lane killed outright — SIGKILL, the OOM killer, a cancelled CI job — which runs no cleanup and so never reports at all, leaving the printer waiting for a report that is never coming. One check honestly taking longer than the threshold reaches it too; raise the variable if that is your repo. Either way only the order is given up, never the verdict: every lane's real exit status is still collected.

Configuring it

One rule for all three blocks:

  • checks adds to that command's defaults; skip removes from them. What push skips, ci skips too.
  • [tool.sdlc.ci] skip may not name a push verdict — lint, mypy, pytest, semgrep, pip-audit, constitution or codeql. Constitution XII says CI runs push's checks except Gourmand and Gatehouse, and this tool decides which two those are. Skipping a verdict in ci alone leaves it on pre-push, which is a developer's machine and one --no-verify, while the required sdlc ci reports green. If a check should not run, take it out of [tool.sdlc.push] — then the choice is in one place. A check the repo added itself may be skipped in ci freely.
  • A check that needs settings gets its own table: [tool.sdlc.<command>.<check>].
  • A check that is not built in must say what kind it is.
  • Check names are unique, and sdlc <name> runs any one of them — the inner loop.

Paths are not set here. Each tool reads its own config: [tool.ruff] excludes, [tool.mypy] files, [tool.pytest.ini_options] testpaths.

A typical repo writes nothing. Trentina writes this:

[tool.sdlc.ci]
checks = ["image", "image-tests"]

[tool.sdlc.ci.image]
kind = "image-build"
build_args = { VERSION = "@project.version" }
labels = { "org.opencontainers.image.revision" = "@git.sha" }

[tool.sdlc.ci.image-tests]
kind = "pytest-in-image"
tests = ["tests/image/"]

Built-in checks and their settings

Check Settings
pytest env (table), args (default ["-q", "-rfE"])
semgrep config (default .semgrep), tests (default .semgrep/run-tests.sh), paths (default ["."])
codeql config (default .github/codeql/codeql-config.yml), baseline (default .sdlc/codeql-baseline.json)
constitution manifest (default .specify/memory/constitution.md)
ruff-check, ruff-format, lint, mypy, gourmand, gatehouse, pip-audit, hooks-installed none
containerfile-lint containerfiles (default: the root Containerfile or Dockerfile), threshold
container-build the image-build kind's settings; containerfile defaults to the root one
container-smoke gate (default false: findings are advisory), seconds (default 10), args, env, network (default false), image
container-scan fail_on (default critical), only_fixed (default true), image

Kinds, for a repo's own checks

image-build — podman build, leaving the image at a known tag for the next check and for a CI publish step.

Setting Type Default
containerfile path in the repo "Containerfile"
context path in the repo "."
build_args table none
labels table none
tag string "localhost/<repo>:sdlc"

pytest-in-image — runs tests inside a built image, locked down the way production runs it: no network, a read-only root, no capabilities. pytest is installed into the cache volume at the versions the repo's uv.lock pins, not added to the image.

Setting Type Default
tests list of paths required
mounts list of paths in the repo, mounted read-only ["tests"]
env table none
args list ["-q", "-rfE"]
image string "localhost/<repo>:sdlc"
network boolean false

Paths must stay inside the repository; .. or an absolute path is refused. Values in build_args, labels and env may be @project.version, @git.sha or @now.

There is no shell kind. A repo that needs something these cannot express gets a new kind here, so the config stays something a person can read.

Running main before a release

Every push to main publishes its tested image as quay.io/crunchtools/sdlc:main, the same name git and Gourmand use: what has merged and not yet been released. It is the digest the contract test ran against at :<sha>, read back after the copy, and it never moves back onto an older commit. Only a release moves latest and the version tags, and no pull request run touches a registry at all. The release is the gate.

A floating repo opts its CI into :main with the repository variable SDLC_IMAGE=main -- a setting only an admin can change, rather than a one-line edit a pull request could slip in. It is not a wall: a pull request from a branch can still edit ci.yml or .sdlc/bin/sdlc themselves, as it always could edit its workflows; an org ruleset's required workflow is what would close that. To run one on your machine, first matching wins:

How Value
SDLC_IMAGE=main git push a bare tag of quay.io/crunchtools/sdlc (main, or a release such as 0.8.0), or a full image reference
[tool.sdlc] image = "main" a bare tag only, written on its own line under [tool.sdlc]

Every command on anything but the pinned release prints which image it is and what chose it. A registry tag is pulled when newer, so :main is the current :main, not whatever was cached.

CI never honours [tool.sdlc] image. Under CI or GITHUB_ACTIONS, and for sdlc ci anywhere, the key is refused with exit 2. A pull request must not choose the image that judges it, and only the launcher can enforce that: the planner runs inside whichever image was chosen.

Nothing built on a laptop or a seat is ever pushed. To try a change to this image before it merges, build it locally (podman build -t localhost/python-sdlc:dev .) and run it with SDLC_IMAGE=localhost/python-sdlc:dev through this clone's bin/sdlc. A converted repo's .sdlc/bin/sdlc will not take it: the bootstrap runs what it pulls, so it pulls only quay.io/crunchtools/sdlc at main, latest, a release X.Y.Z or @sha256:<digest>, and refuses anything else with exit 2. sdlc has no push step, and only this repo's GitHub Actions on main publish to quay.io.

When :latest is broken

A floating repo runs whatever :latest is, so a bad release reaches every repo at once. Until a fixed release ships:

  • A local commit or push: git commit --no-verify / git push --no-verify. CI still judges the pull request.
  • CI, fleet-wide: set the organization variable SDLC_IMAGE to the last good release, X.Y.Z. Every floating repo's CI picks it up on its next run, with no pull request; a repository variable of the same name does the same for one repo. Clear it once the fix is on :latest.
  • A seat: SDLC_IMAGE=X.Y.Z git push.

The fallback needs a release that carries the launcher. The bootstrap asks the image for it, so a tag older than the first release with sdlc launcher cannot be pinned this way: it exits 2. Rolling back works only once two releases carry launcher -- the broken one and a good one before it.

Skipping a check

SDLC_SKIP=gatehouse git push skips one check, once, for a false positive. The finding still has to be answered on the pull request.

Exit codes

0   ran, found nothing
1   ran, found something
2   COULD NOT RUN

2 is never treated as a pass. A gate that cannot run and reports success is worse than no gate.

CodeQL fails only on new findings

codeql fails on any finding not in .sdlc/codeql-baseline.json. The baseline is the findings someone has looked at and accepted, committed where a reviewer sees it. sdlc codeql-baseline writes it from the current findings. See docs/codeql.md.

What is in the image

Tool Pinned by
ruff, semgrep, pip-audit this image
Gourmand, Gatehouse this image, copied from their official images
CodeQL this image (the Python-only bundle, 110 MB rather than 1.75 GB)
hadolint, grype this image, copied from their official images
constitution validator the repo — cloned at the release its own manifest names
mypy, pytest the repo's uv.lock, because they load the project's own plugins

sdlc versions prints what the image carries.

How a new Gourmand or Gatehouse arrives

Their releases trigger a rebuild here, which is released as a new python-sdlc version and promoted to sdlc:latest. Every converted repo floats on that tag, so the laptop then runs the reviewer CI runs, with no pull request per repo. A repo still pinned by rev: moves only when it converts or bumps by hand.

Host contract

A developer machine needs git, podman, pre-commit and gh. bin/sdlc is POSIX shell, runs only git and podman, and refuses any step that is not a podman build or podman run. tests/test_host_contract.sh proves it with a restricted PATH, and proves the lane executor there too: that the lanes really overlap, that a 2 in one beats a 1 in another whichever finishes first, that a failure in one lane does not stop the other, and that no two lanes' output interleaves.

Licence

AGPL-3.0-or-later. The CodeQL bundle is redistributed under GitHub's CodeQL CLI terms — see docs/codeql.md.