Skip to content

build(make): align make docs with CI, stop re-bootstrapping, run mypy once - #831

Merged
yeongseon merged 5 commits into
mainfrom
build/make-docs-strict-bootstrap-once
Oct 24, 2026
Merged

yeongseon merged 5 commits into
mainfrom
build/make-docs-strict-bootstrap-once

Conversation

@yeongseon

@yeongseon yeongseon commented Oct 24, 2026 •

Copy link
Copy Markdown
Owner

Summary

Every Make target depends on bootstrap, which ran pip install --upgrade pip and pip install hatch each time, and make check ran hatch run lint (already style plus typecheck) and then make typecheck again. With stub pip and Hatch programs on origin/main (9127b17), one make check made 6 pip calls and one make check-all made 12, and each ran the type-check script twice. make doctor exited 0 with Hatch and func missing (|| echo); it failed only by accident, with exit 2, when .venv/bin/python did not exist. make docs ran mkdocs build without the --strict the CI docs gate uses, so a broken link built with a warning and exit 0. Its CI branch also hid pip's output and errors (> /dev/null 2>&1, ; between the steps) and pinned mkdocstrings[python]<1.0, which cannot render api.md (Highlight.__init__() missing 1 required positional argument: 'md'), so CI=1 make docs never built the docs at all.

After this PR a make check-all with Hatch present makes 0 pip calls and runs the type check once; from nothing it makes one (pip install hatch). make doctor is a read-only check that fails when a required tool is missing, make docs builds like the CI docs gate (mkdocs build --strict with the docs extra) in both branches, and make install-ci PYTHON=<interpreter> installs into the interpreter you choose.

Makefile, tests and two docs pages only: no workflow, src/, tools/, scripts/ or policy file changes, and check-all and the security-* targets are untouched. This adapts the Makefile fix already merged in azure-functions-doctor-python#636 and azure-functions-scaffold-python#462 to this repository's targets.

Changes

  • Makefile
    • bootstrap: creates .venv only while it is missing and installs Hatch only while $(HATCH) is not an executable regular file (-x alone also accepts a directory). It never upgrades pip, and --disable-pip-version-check --quiet keeps pip's "run --upgrade pip" notice off the first install. It prints nothing when there is nothing to do (it printed two lines per target).
    • PYTHON ?= python3 replaces PYTHON := $(VENV_DIR)/bin/python, whose only user was the old doctor. It is the interpreter that creates .venv (a hard-coded python3 before, same default) and the one install-ci installs into. Hatch still provisions its own default environment (Python 3.11).
    • check: lint (was lint then typecheck): hatch run lint is style followed by typecheck. check-all is not edited; it runs the type check once because check does.
    • install-ci: refuses an interpreter older than Python 3.11, then $(PYTHON) -m pip install -e ".[dev]", the extra the workflows install inline today. It has no ensure-hatch, so it never goes through Hatch's default environment (pinned to Python 3.11) and a CI matrix cell keeps its own interpreter.
    • docs: mkdocs build --strict in both branches. The CI branch installs -e ".[docs]", the extra the CI docs gate installs, instead of three pins that had drifted (mkdocstrings[python]<1.0 against the extra's <2.0), joins its steps with && and no longer hides pip's output or errors. The local branch joins ensure-hatch and the build with &&. The pip install --upgrade pip line is gone.
    • doctor: read-only. A Python 3.11 or newer (the first of python3.14 ... python3.11, python3 on PATH that passes the version probe) and $(HATCH) are required: each prints ok ... or MISSING ... (required...), and any MISSING makes the target fail (recipe exit 1, make exits 2). func and azurite are listed ok or missing ... (optional ...) and never affect the status, because only a local Functions host needs them. A tool counts only if command -v resolves it to an executable regular file (one have helper), because dash's bare command -v accepts any existing path and -x alone is also true for a searchable directory. The old output (Python version, pip list, func --version, pre-commit hook state) is gone; nothing in the repository parses it (grep).
  • tests/test_make_contract.py (new, 21 cases): make -n expansions plus the real recipes run in a scratch directory whose PATH holds only stub programs that log their arguments, so nothing is installed, no network is used and no real tool can run. Hatch scripts are expanded recursively, because the second type check hid inside Hatch's lint script, not in make's output. It also asserts that check-all still contains each workflow lint and the Bandit evidence step exactly once.
  • docs/development.md, docs/contributing.md: the command tables said make check is "lint + typecheck", make docs "Build the MkDocs site" and make doctor "Show environment diagnostic info". Updated, plus install-ci and two short paragraphs in the development guide (when make install acts and what PYTHON selects; what make doctor requires and the one thing it cannot see, a Python 3.11 that only Hatch manages).

Net diff: 4 files, +344/-27 (the test file +279, the Makefile +46/-22, the two docs pages +19/-5). Nothing under src examples demo scripts tools changes.

Behavior changes to know about:

  1. make doctor's output and exit status change (above). Run before make install it fails on .venv/bin/hatch and says make install creates it; that is the Hatch this Makefile runs, not one on PATH.
  2. .venv's pip is no longer upgraded; it keeps the version bundled with the interpreter (22.0.2 for python3 3.10.12 here, 24.0 for 3.11.15). Hatch installs with both.
  3. make docs fails on any MkDocs warning, and its CI branch now installs the docs extra (needs the build backend and network, like the CI docs gate).
  4. PYTHON selects the interpreter that creates .venv.

Verification

Environment unless stated: Linux, GNU Make 4.3, /bin/sh = dash, Python 3.11.15 for the test runs, python3 = 3.10.12 (pip 22.0.2), Hatch 1.18.1, ruff 0.17.0, mypy 2.4.0, MkDocs 1.6.1. "main" is unmodified origin/main 9127b17; the fresh clones are of this branch at 297a0c4.

Stub-pip call counts. make VENV_DIR=<stub> on a scratch copy of the Makefile with a fake venv whose pip, hatch and python3 -m venv only append their arguments to a log (nothing real runs; CI=1 so install skips the pre-commit hook):

make ... state pip calls, main pip calls, this branch type-check runs, main this branch
check venv and Hatch present 6 (3 --upgrade pip, 3 install hatch) 0 2 (hatch run lint + hatch run typecheck) 1
check venv, no Hatch 6 1 (install ... hatch)
check no venv 6 and one python3 -m venv 1 and one -m venv
check-all venv and Hatch present 12 (6 + 6) 0 2 1
check-all venv, no Hatch 12 1
check-all no venv 12 and one -m venv 1 and one -m venv
lint present 2 0 1 1
test present 2 0
install present 2 0

make -n check: --upgrade pip 3 times on main, 0 here; make -n check-all: 6 and 0. Here make -n check is the bootstrap guard, one hatch run lint and the final echo.

make doctor.

Scenario main this branch
fresh checkout, no .venv exit 2: .venv/bin/python: not found exit 2: MISSING .venv/bin/hatch (required; make install creates it); ok for python3.14, func, azurite
.venv/bin/python present, Hatch and func missing (PATH=/usr/bin:/bin) exit 0 ("No hatch environment found", "func not found") exit 2: MISSING <venv>/bin/hatch (required; make install creates it); func and azurite listed missing ... (optional ...)
after make install the old recipe printed the Python version, pip list, func --version and the hook state four ok lines (Python, Hatch, func, azurite); exit 0
a directory, or a file without its executable bit, at .venv/bin/hatch n/a exit 2 MISSING, and bootstrap installs Hatch (stub pip: one install ... hatch call)

The bootstrap and doctor recipe text taken from make -n was run under dash, bash, bash --posix and busybox sh for an executable file, a directory, a non-executable file and a missing Hatch path: identical results in all four shells (only the executable file is ok and not reinstalled; the other three are MISSING and trigger pip install hatch). shellcheck 0.8.0 (-s sh, -s dash, -s bash) is clean on the bootstrap, doctor, install-ci and docs recipes.

make docs. A broken link ([a broken link](does-not-exist.md) appended to docs/faq.md) as the only change:

Branch of the recipe main this branch
local, through the real Hatch env prints the WARNING, "Documentation built", exit 0 prints it, then "Aborted with 1 warnings in strict mode!", exit 2
CI=1, fresh Python 3.11 venv, clean tree exit 2: TypeError: Highlight.__init__() missing 1 required positional argument: 'md' (the mkdocstrings[python]<1.0 pin) exit 0, "Documentation built"
CI=1, fresh venv, broken link crashes before the link check exit 2, "Aborted with 1 warnings in strict mode!"

Without the link both local runs exit 0. The pip install failure path is covered by test_docs_shows_a_failed_install_and_does_not_build_after_it (stub pip prints an error and exits 1: make exits non-zero, the error is on stderr, MkDocs never runs).

Contract tests. pytest tests/test_make_contract.py: 21 passed on Python 3.11.15, 3.12.13, 3.13.13 and 3.14.4. The same file against main's Makefile: 17 failed, 4 passed (the 4 are properties main already has: check-all keeps every gate, a failed MkDocs build fails the target in both branches, doctor writes nothing). One mutated copy of the Makefile per case, each run against the tests: baseline green, and all 43 mutations turn at least one test red:

  • bootstrap (7): an unconditional --upgrade pip; Hatch reinstalled every time; a directory counts as Hatch (-x only); a non-executable file counts (-f only); python3 hard-coded (ignores PYTHON); .venv recreated every time; the old PYTHON := .venv/bin/python restored.
  • check and check-all (8): check also runs typecheck; check drops lint; check also runs the tests; check-all drops security, the workflow lints, the tests or check; check-all runs check twice.
  • docs (8): --strict removed (local or CI); a failed build swallowed with || true (local or CI); a failed dependency install ignored (; for &&); pip's output and errors swallowed; --upgrade pip restored; the CI branch pins packages instead of the docs extra.
  • doctor (14): always exit 0; any command -v hit counts; a directory counts; a non-executable file counts; a missing optional tool fails; doctor depends on ensure-hatch; the version probe removed, inverted or lowered to 3.10; a missing Python or Hatch does not fail; Hatch looked up on PATH; only python3 scanned; doctor writes a file.
  • install-ci (6): through Hatch; ignores PYTHON; no version guard; guard inverted; installs without the dev extra; depends on ensure-hatch.

Quality gates.

Command Result
ruff check src tests; ruff format --check src tests All checks passed; 104 files already formatted
mypy src tests (strict, from pyproject.toml); make security-types no issues in 104 source files; no issues in 6 source files
python tools/lint_release_workflows.py; tools/lint_workflow_pins.py; tools/lint_hatch_matrix.py; git diff --check origin/main..HEAD all exit 0; clean
make security (the CI invocation, SECURITY_PYTHON = the dev interpreter) bandit: PASS {"files": 49, "loc": 8902, "errors": 0, "severity.LOW": 13, "confidence.MEDIUM": 2, "confidence.HIGH": 11, "blocking": 0, "stale_exceptions": 0}: the 13 line-pinned LOW findings of #817 still match, because nothing under src examples demo scripts tools changed
mkdocs build --strict exit 0
pytest tests (default addopts, coverage gate), dev-only env without the companion package 1526 passed, 30 skipped, 22 deselected, coverage 95.14% (gate 95%); main's 1505 plus the 21 new tests; the 30 skips are the existing companion-dependent ones

Fresh clones of this branch (git clone --branch build/make-docs-strict-bootstrap-once, HEAD = 297a0c4, no .venv, no Hatch environment):

Clone, command Result
A (default interpreter: python3 is 3.10.12) make doctor MISSING .venv/bin/hatch (required; make install creates it), exit 2
A make install exit 0 in 39 s: prints "Creating virtual environment..." and "Installing Hatch..."; Hatch 1.18.1 in the 3.10 venv, whose pip stays 22.0.2 (main's bootstrap takes a fresh 3.10 venv to pip 26.2.2, measured on a fresh clone of main); Hatch environment on Python 3.11.15
A make doctor; make install a second time exit 0 with four ok lines; exit 0 in 1 s with no "Creating" or "Installing" line
A make test exit 0: 1526 passed, 30 skipped, 22 deselected, coverage 95.14% in 23 s
A make check with this Makefile, then with main's Makefile exit 0 in 7 s; mypy summaries printed 2 (one typecheck run: src tests and the security tools) against 4 on main
A make check-all exit 0 in 26 s: ruff, format, mypy, 1526 passed at 95.14%, bandit: PASS with "blocking": 0, "stale_exceptions": 0
A make docs; the same with a broken link, this Makefile / main's exit 0 in 2 s; exit 2 / exit 0
B make install PYTHON=python3.11 exit 0 in 34 s: .venv/bin/python is 3.11.15, pip 24.0 (un-upgraded), Hatch 1.18.1
B make doctor; make check exit 0; exit 0 in 6 s
B make install-ci PYTHON=<Python 3.11 venv>/bin/python exit 0 in 27 s; azure_functions_openapi 0.30.1 imports from the clone's src in that interpreter
B make install-ci PYTHON=python3 (3.10.12); PYTHON=python3.99 install-ci needs Python 3.11 or newer, got 3.10.12, exit 2, before pip runs; python3.99: No such file or directory, exit 2

Not run: macOS or BSD make (the contract module skips itself on Windows); make demo*, make e2e-local and make e2e-azure; publish-pypi.yml (tag or manual dispatch only, and it calls make install in two jobs). docs.yml and maintenance.yml also call make install but do not run on pull requests, so they were dispatched by hand on this branch (see the CI section).

What remains of #811

Refs, not Closes: this PR covers the Makefile items below; the rest of #811's checklist is separate work.

#811 item State
make install without a per-run pip or Hatch upgrade done here
make install-ci PYTHON=<interpreter> installs into exactly that interpreter done here (the target; no workflow calls it yet)
make doctor read-only, non-zero when Python >= 3.11 or Hatch is missing, reports func/azurite done here
make check: mypy once done here. check is lint (ruff, format check, one type check); the Bandit scope and the workflow lints stay in check-all, which is unchanged
make docs equals the CI docs gate (mkdocs build --strict) done here, in both branches. The gate's Mermaid-label lint and screenshot-manifest checks are separate CI steps and stay outside make docs
make integration, make verify not in this PR
JUnit and coverage retained from failed CI lanes not in this PR

Not in this PR

  • AGENTS.md and CONTRIBUTING.md are policy-bearing files (R3 in docs/review-evidence-policy.md), so they are untouched. Nothing they say becomes wrong (they name make install, make check-all and make docs); a follow-up can add doctor and install-ci to their command lists together with the maintainer's authorization record.
  • No workflow is edited, so no job runs make install-ci or the new doctor yet. When one adopts install-ci, note that tools/lint_hatch_matrix.py matches \bmake\s+install\b, which is also a prefix of make install-ci, so it would count it as Hatch-routed.
  • Found while verifying, unchanged: SECURITY_PYTHON ?= hatch run python resolves hatch from PATH, not $(HATCH), so after make install alone make security (and therefore make check-all) fails with hatch: not found unless Hatch is also on PATH. The same happens on main. The security targets are out of scope here.
  • maintenance.yml runs hatch run lint and then hatch run typecheck, so mypy still runs twice in that scheduled job (visible in the dispatch run below). That is a workflow edit and is left for its own change.
  • Known limitation: a Python 3.11 that only Hatch manages is not on PATH, so make doctor reports it as missing; make install still works (Hatch provisions it).

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Refactoring (no functional changes)
  • Chore (maintenance, dependencies, CI, etc.)

Checklist

  • My code follows the project's code style
  • I have run make check-all and all checks pass (including tests and at least 95% coverage): exit 0 in a fresh clone of the branch at 297a0c4, coverage 95.14%
  • I have added tests for new functionality (if applicable)
  • I have updated documentation (if applicable)
  • My changes do not introduce new warnings

Related Issues

Refs #811

Review Evidence

  • Review source and reviewed commit SHA: none yet; author verification only, at head 297a0c4
  • Material findings and disposition (fixed / acknowledged-wont-fix / deferred-with-issue), or "none": none
  • Unresolved blockers, or "none": none
  • R3 (workflow permissions, release/publishing, Swagger UI credential/CSP handling, CLI import trust, review-policy files): no - Makefile, tests/ and two docs/ pages only; no workflow, permissions:, secrets, release/publishing, Swagger UI, CLI-import or policy-bearing file (AGENTS.md, CONTRIBUTING.md, the PR template, docs/agent-playbook.md, docs/review-evidence-policy.md, docs/release_process.md, docs/security.md, DESIGN.md, SECURITY.md) changes

CI on the head commit

297a0c41a7347f435babe38761bd2567e9cd7cbd (the head of this PR; main still 9127b17): 30 check-runs, all pages fetched (one page, total_count 30): 29 success, 1 skipped (dependabot-auto-merge, by design), 0 failed. The five required contexts (bandit, semgrep, Analyze, PR title, ci-required) succeeded on this exact SHA.

  • Test (Py 3.11, af latest): 1556 passed, 22 deselected, coverage 95.14% (gate 95%); the 3.12, 3.13 and 3.14 cells: 1556 passed. The af 1.21.0, af 1.24.0 and Minimum dependencies cells (no companion package): 1526 passed, 30 skipped. All of them include the 21 new tests.
  • bandit: bandit: PASS {"files": 49, "loc": 8902, "errors": 0, ... "blocking": 0, "stale_exceptions": 0} on the PR head.
  • make install is not on any pull-request path, so I dispatched the two workflows that call it on this branch's head. Docs (strict build check) (run 40118127178, workflow_dispatch): the log shows "Creating virtual environment...", "Installing Hatch...", the Hatch environment created, then hatch run mkdocs build --strict and "Documentation built"; success. Maintenance Checks (run 40118128903): the same bootstrap lines, hatch run lint, and make security with bandit: PASS ... "blocking": 0, "stale_exceptions": 0; success. Both appear among the 30 check-runs above (docs-check, maintenance).

yeongseon and others added 5 commits October 25, 2026 02:23
Every Make target depends on bootstrap, which ran `pip install --upgrade pip` and `pip install hatch` each time, and `make check` ran `hatch run lint` (already style plus typecheck) and then `make typecheck` again. With stub pip and Hatch programs, one `make check-all` made 12 pip calls and two type-check runs.

bootstrap now creates `.venv` only while it is missing and installs Hatch only while `.venv/bin/hatch` is not an executable regular file; it never upgrades pip. `check` is `lint`, so the type check runs once. `PYTHON` (default python3) selects the interpreter that creates `.venv`, and `make install-ci PYTHON=<interpreter>` installs `.[dev]` into exactly that interpreter, never through Hatch, and refuses Python older than 3.11. `check-all` and the security targets are unchanged.

tests/test_make_contract.py asserts the expansion with `make -n` and runs the real recipes against stub programs on a PATH that holds nothing else.

Refs #811

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
`make docs` ran `mkdocs build`, while the CI docs gate runs `mkdocs build --strict`, so a broken link built with a warning and exit 0. Its CI branch also hid pip's output and errors (`> /dev/null 2>&1`, `;` between the steps) and installed `mkdocstrings[python]<1.0`, which resolves to a mkdocstrings that cannot render api.md (`Highlight.__init__() missing 1 required positional argument: 'md'`), so `CI=1 make docs` never built the docs at all.

Both branches now run `mkdocs build --strict`. The CI branch installs `-e ".[docs]"`, the extra the CI gate installs, so the dependency pins live only in pyproject.toml, and it stops with pip's own error when the install fails instead of building with whatever was already installed. No target upgrades pip any more, and a test reads the Makefile to keep it that way.

Refs #811

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
`make doctor` could not fail for a missing Hatch or func (`|| echo`): it failed only by accident, when `.venv/bin/python` did not exist, and its `hatch env run pip list` creates the Hatch environment when it is missing, so the report wrote.

doctor now installs and writes nothing. It requires a Python 3.11 or newer on PATH and the `.venv/bin/hatch` this Makefile runs, each counted only as an executable regular file, and exits non-zero with a `MISSING` line when either is absent. func and azurite are listed as optional, because only a local Functions host needs them. The old output (Python version, pip list, func version, pre-commit hook state) is gone; nothing in the repository parses it.

Refs #811

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
…ingle type check

The command tables still described `make check` as lint plus typecheck, `make docs` as a plain MkDocs build and `make doctor` as an environment dump. They now say what each target does after this series, and the development guide gains two short paragraphs: when `make install` acts and what `PYTHON` selects, and what `make doctor` requires, what it treats as optional and the one thing it cannot see (a Python 3.11 that only Hatch manages).

AGENTS.md and CONTRIBUTING.md are policy-bearing files and stay untouched; nothing they say becomes wrong.

Refs #811

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
The contract file had grown to 369 lines over the series. The helpers now share one `make` runner, the stub interpreter only answers the version probes the Makefile runs, related cases are merged and the three-line given/when/then comments are gone in favor of blank-line blocks. The virtual-environment case now reads the expansion of `make -n bootstrap` instead of stubbing `-m venv`, and also asserts that `python3` stays the default interpreter.

Every one of 43 one-line mutations of the Makefile (bootstrap, check, check-all, docs, doctor and install-ci) still turns at least one test red.

Refs #811

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
@yeongseon

Copy link
Copy Markdown
Owner Author

R3 authorization for head 297a0c41a7347f435babe38761bd2567e9cd7cbd.

The maintainer delegated the merge decision to an Oracle review. The Oracle (AI) reviewed this change at head 297a0c41a7347f435babe38761bd2567e9cd7cbd through code inspection and independent probes and returned GO (fresh install/test/check-all/security pass; pip calls 12 to 0; doctor matrix; docs link breaks make docs (main exit 0); install-ci uses selected interpreter; 14 of 14 mutations detected; 1526 passed 95.14%; Bandit pins unchanged). Changes since that head are merges of main only; required checks were re-run on this exact head and are green. This is advisory AI evidence, not line-by-line human review or human approval. Refs #811 stays open: integration, verify, failed-lane artifacts and the check-includes-Bandit-and-lints part are deferred. The directory-at-Hatch-path test proves an install attempt, not repair. Reviewer-noted trims are optional.

@yeongseon
yeongseon merged commit e13bd68 into main Oct 24, 2026
34 checks passed
@yeongseon
yeongseon deleted the build/make-docs-strict-bootstrap-once branch October 24, 2026 19:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant