From 96495530bccc53b7cf3c0e34302a0dca8a5a0073 Mon Sep 17 00:00:00 2001 From: Scott McCarty Date: Wed, 4 Nov 2026 18:11:33 -0500 Subject: [PATCH] feat: `[tool.sdlc] project`, for a repo whose Python is in a subdirectory The launcher resolved every check against the mount root, so a repo holding its Python project one directory down could not be gated at all. `need_project` found the root pyproject.toml, `uv sync` ran where there was no lockfile, and the gate exited 2 while the code it was meant to check sat untouched below. `project = "image-server"` in the ROOT pyproject.toml moves the uv-driven checks -- mypy, pytest, pip-audit and the sync they share -- into that directory. Everything repo-wide stays at the root: the constitution validator, Gourmand, Gatehouse, CodeQL, and `ruff check .`, whose config is hierarchical and already applies a nested pyproject.toml to the files beneath it. Design decisions worth stating: - **The root pyproject, not an env var or a CI argument.** The pre-commit hook, the pre-push hook and the CI job all read that file, and all three must agree on where the project is. Three entry points means three places to forget. - **Validated when the config is read**, not when a check runs: relative, inside the repo, and holding a pyproject.toml. `sdlc_config()` already calls into the whole config on every command, so a bad value fails on the next commit rather than in CI. Seven tests in tests/test_plan.py cover the refusals and the default. - **The venv cache key includes the subdirectory.** One repo can hold two projects, and gates.sh already recorded what happens when two projects share a UV_PROJECT_ENVIRONMENT: uv rebuilds the venv on every invocation, in both directions, forever. - **A single string, not a list.** No repo needs two projects yet, and a list is a later change if one does. - **Default `"."`**, so nothing changes for the 26 repos already converted. `run()` gains a `cwd` parameter for this, defaulting to the repo root, and the five uv call sites pass the project directory. Nothing else moved. Prompted by crunchtools/rotv, the last repo in the RT#1552 conversion: a Node.js project whose `image-server/` is a deployed Python service with 19 modules, `strict = true` declared but never run, and six test files that no workflow has ever executed. The alternative was promoting image-server's config to the root, which the constitution's own `version_files` glob would then force to share rotv's release version, or declaring rotv exempt and leaving that service ungated. Neither is worth avoiding a six-line resolver. RT#1552 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R7Y1jesjfgxHeqtKJCvay6 --- CHANGELOG.md | 29 ++++++++++++++++++++ Containerfile | 2 +- IMAGE_TAG | 2 +- README.md | 15 +++++++++++ pyproject.toml | 2 +- sdlc | 66 +++++++++++++++++++++++++++++++++++++--------- tests/test_plan.py | 44 ++++++++++++++++++++++++++++++- 7 files changed, 143 insertions(+), 17 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 031cec9..8319ed7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,35 @@ All notable changes to this project are documented here, following ## [Unreleased] +## [0.3.7] - 2026-11-04 + +### Added + +- **`[tool.sdlc] project`, for a repo whose Python lives in a subdirectory.** + The launcher resolved every check against the mount root, so a repo holding a + Python project one level down could not be gated at all: `need_project` saw + the root `pyproject.toml`, `uv sync` ran where there was no lockfile, and the + gate exited 2 while the real code sat untouched below. + + `project = "image-server"` in the ROOT `pyproject.toml` moves the uv-driven + checks -- mypy, pytest, pip-audit and the sync they share -- into that + directory. Everything repo-wide stays at the root: the constitution + validator, Gourmand, Gatehouse, CodeQL and `ruff check .`, whose config is + hierarchical and already picks up a nested `pyproject.toml` for files under + it. The venv cache key includes the subdirectory, so two projects in one repo + do not fight over one environment. + + It lives in the root pyproject and nowhere else, because the pre-commit hook, + the pre-push hook and the CI job all read that file and must agree; an env + var or a CI argument would have to be told to three entry points separately. + Validated at config time -- relative, inside the repo, and holding a + `pyproject.toml` -- so a bad value is caught on the next commit rather than + in CI. Default `"."`, so nothing changes for the 26 repos already converted. + + Prompted by crunchtools/rotv: a Node.js repo whose `image-server/` is a + deployed Python service with 19 modules, `strict = true` on paper and six + test files that no workflow had ever run. + ## [0.3.6] - 2026-11-04 ### Fixed diff --git a/Containerfile b/Containerfile index 01515bb..6ac20af 100644 --- a/Containerfile +++ b/Containerfile @@ -44,7 +44,7 @@ LABEL description="Every CrunchTools Python SDLC check in one image: ruff, mypy, LABEL org.opencontainers.image.source="https://github.com/crunchtools/python-sdlc" LABEL org.opencontainers.image.description="Every CrunchTools Python SDLC check in one image." LABEL org.opencontainers.image.licenses="AGPL-3.0-or-later" -LABEL version="0.3.6" +LABEL version="0.3.7" # What the image carries, readable without starting anything. `sdlc versions` # prints these and a consuming repo's CI pins them. diff --git a/IMAGE_TAG b/IMAGE_TAG index 449d7e7..0f82685 100644 --- a/IMAGE_TAG +++ b/IMAGE_TAG @@ -1 +1 @@ -0.3.6 +0.3.7 diff --git a/README.md b/README.md index ce5b1ba..0f44d9c 100644 --- a/README.md +++ b/README.md @@ -61,6 +61,21 @@ CI, one job, which reads the release from that same `rev:`: That `rev:` pins the launcher, the image and every tool inside it, for the laptop and CI both. One pin. +### A project in a subdirectory + +If the repo's Python is not at the root -- a JavaScript repo with a Python +service beside it, say -- name the directory in the ROOT `pyproject.toml`: + +```toml +[tool.sdlc] +project = "image-server" +``` + +Only the uv-driven checks move: mypy, pytest, pip-audit and the sync they +share. The constitution validator, Gourmand, Gatehouse, CodeQL and +`ruff check .` stay repo-wide, and ruff's own config is hierarchical, so a +nested `pyproject.toml` still applies to the files under it. + ## What a repo gets with no config at all | Command | Runs | diff --git a/pyproject.toml b/pyproject.toml index 2f32bb5..08252b5 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -10,7 +10,7 @@ [project] name = "python-sdlc" -version = "0.3.6" +version = "0.3.7" description = "One container image and one script for every CrunchTools Python SDLC check" requires-python = ">=3.14" license = "AGPL-3.0-or-later" diff --git a/sdlc b/sdlc index c73be34..7198e42 100755 --- a/sdlc +++ b/sdlc @@ -185,12 +185,13 @@ RESERVED = (*COMMANDS, "plan", "tiers", "list", "versions", "pytest-deps", "code def sdlc_config() -> dict: """`[tool.sdlc]`, checked for shape before anything trusts it.""" cfg = pyproject().get("tool", {}).get("sdlc", {}) or {} - stray = sorted(set(cfg) - set(COMMANDS)) + stray = sorted(set(cfg) - set(COMMANDS) - {"project"}) if stray: cannot( - f"[tool.sdlc] has {', '.join(stray)}; the only blocks are " - f"[tool.sdlc.commit], [tool.sdlc.push] and [tool.sdlc.ci]" + f"[tool.sdlc] has {', '.join(stray)}; the only keys are " + f"[tool.sdlc.commit], [tool.sdlc.push], [tool.sdlc.ci] and `project`" ) + project_dir() # validated here, so a bad path is caught on the next commit # The WHOLE config is checked whichever command runs, so a typo in the ci # block is found on the next commit rather than in CI. for command in COMMANDS: @@ -571,7 +572,7 @@ PYTEST_PACKAGES = ("pytest", "pytest-asyncio", "pytest-xdist", "pytest-timeout") def t_pytest_deps() -> int: """Install the lock's pytest into the cache volume for `pytest-in-image`.""" need_lock_dep("pytest") - lock = tomllib.loads((REPO / "uv.lock").read_text(encoding="utf-8")) + lock = tomllib.loads((project_dir() / "uv.lock").read_text(encoding="utf-8")) pins = sorted( f"{pkg['name']}=={pkg['version']}" for pkg in lock.get("package", []) @@ -583,16 +584,46 @@ def t_pytest_deps() -> int: return CLEAN shutil.rmtree(target, ignore_errors=True) step(f"install {', '.join(pins)} for the in-image run") - rc, _ = run(["uv", "pip", "install", "--target", str(target), *pins], env=project_env()) + rc, _ = run( + ["uv", "pip", "install", "--target", str(target), *pins], + env=project_env(), + cwd=project_dir(), + ) if rc != 0: cannot("could not install pytest for the in-image run") stamp.write_text("\n".join(pins), encoding="utf-8") return CLEAN +def project_dir() -> Path: + """Where the uv project is, which is not always the repo root. + + `[tool.sdlc] project = "image-server"` in the ROOT pyproject.toml points the + uv-driven checks -- mypy, pytest, pip-audit and the sync they share -- at a + subdirectory. Everything repo-wide stays at the root: the constitution + validator, Gourmand, Gatehouse, CodeQL and `ruff check .`, whose own config + is hierarchical and already picks up a nested pyproject for files under it. + + Written in the root pyproject and nowhere else, because the two hooks and + the CI job all read that file and must agree; an env var or a CI argument + would have to be told to three entry points separately. + """ + raw = pyproject().get("tool", {}).get("sdlc", {}).get("project", ".") + if not isinstance(raw, str): + cannot(f"[tool.sdlc] project must be a string, not {type(raw).__name__}") + if Path(raw).is_absolute() or ".." in Path(raw).parts: + cannot(f"[tool.sdlc] project must be a relative path inside the repo; got {raw!r}") + target = (REPO / raw).resolve() + if not str(target).startswith(str(REPO.resolve())): + cannot(f"[tool.sdlc] project escapes the repo: {raw!r}") + if raw != "." and not (target / "pyproject.toml").is_file(): + cannot(f"[tool.sdlc] project {raw!r} has no pyproject.toml") + return target + + def need_project() -> None: """Assert this is a Python project, for the targets that require one.""" - if not (REPO / "pyproject.toml").is_file(): + if not (project_dir() / "pyproject.toml").is_file(): cannot( "pyproject.toml does not exist, so this target has no project to " "check. constitution, gourmand and gatehouse work on any repo; " @@ -608,7 +639,11 @@ def project_env() -> dict[str, str]: UV_PROJECT_ENVIRONMENT across different Python versions make uv rebuild the venv on every invocation, in both directions, forever. """ - name = os.environ.get("SDLC_PROJECT", REPO.name) + # The subdirectory is part of the key: one repo can hold two projects, and + # sharing a venv between them would make uv rebuild it on every invocation. + sub = project_dir().relative_to(REPO.resolve()) + default = REPO.name if str(sub) == "." else f"{REPO.name}-{str(sub).replace('/', '-')}" + name = os.environ.get("SDLC_PROJECT", default) pyver = f"{sys.version_info.major}.{sys.version_info.minor}" env = dict(os.environ) env.update( @@ -630,6 +665,9 @@ def run( env: dict[str, str] | None = None, stdin: bytes | None = None, tee: bool = False, + # Where the tool runs. Defaults to the repo root; the uv-driven checks + # pass the project directory, which [tool.sdlc] project may move. + cwd: Path | None = None, ) -> tuple[int, str]: """Run a tool and return (ITS exit status, captured output). @@ -642,7 +680,7 @@ def run( if tee: proc = subprocess.run( cmd, - cwd=REPO, + cwd=cwd or REPO, env=env if env is not None else dict(os.environ), input=stdin, check=False, @@ -656,7 +694,7 @@ def run( else: proc = subprocess.run( cmd, - cwd=REPO, + cwd=cwd or REPO, env=env if env is not None else dict(os.environ), input=stdin, check=False, @@ -684,7 +722,7 @@ def need_lock_dep(name: str) -> None: makes "the repo forgot to depend on mypy" a real case, and the honest answer to it is 2, not 0. """ - lock = REPO / "uv.lock" + lock = project_dir() / "uv.lock" if not lock.is_file(): cannot("uv.lock is missing; mypy and pytest resolve from the project lock") try: @@ -703,7 +741,7 @@ def need_lock_dep(name: str) -> None: def sync_project(env: dict[str, str]) -> None: """Build the project venv, or fail as 'could not run'.""" step("uv sync --locked --all-extras") - rc, _ = run(["uv", "sync", "--locked", "--all-extras"], env=env) + rc, _ = run(["uv", "sync", "--locked", "--all-extras"], env=env, cwd=project_dir()) if rc != 0: cannot( "uv sync --locked failed. The lockfile is out of date with " @@ -750,6 +788,7 @@ def t_mypy(s: dict) -> int: str(CACHE / "mypy" / project_name()), ], env=env, + cwd=project_dir(), )[0] @@ -774,9 +813,9 @@ def t_pytest(s: dict) -> int: "no:cacheprovider", *args, ] - if jobs != "0" and "pytest-xdist" in (REPO / "uv.lock").read_text(encoding="utf-8"): + if jobs != "0" and "pytest-xdist" in (project_dir() / "uv.lock").read_text(encoding="utf-8"): cmd += ["-n", jobs] - return run(cmd, env=env)[0] + return run(cmd, env=env, cwd=project_dir())[0] def t_gourmand(s: dict) -> int: @@ -849,6 +888,7 @@ def t_pip_audit(s: dict) -> int: rc, out = run( ["uv", "run", "--locked", "--all-extras", "--with", "pip-audit", "pip-audit"], env=env, + cwd=project_dir(), tee=True, ) unaudited = [ diff --git a/tests/test_plan.py b/tests/test_plan.py index c8f2189..0a755d2 100644 --- a/tests/test_plan.py +++ b/tests/test_plan.py @@ -105,10 +105,52 @@ def config_error(pyproject: str) -> str | None: msg = config_error("[tool.sdlc.post]\n") -check("a block that is not a command is refused", msg is not None and "only blocks are" in msg) +check("a block that is not a command is refused", msg is not None and "only keys are" in msg) msg = config_error('[tool.sdlc.ci]\nchecks = ["typo"]\n') check("a typo in ci is found on commit", msg is not None and "typo" in msg, str(msg)) +# --------------------------------------------------- project in a subdirectory +# +# The uv-driven checks can live one directory down. The value is validated when +# the config is read, so a bad path is caught on the next commit, not in CI. + +msg = config_error('[tool.sdlc]\nproject = "nope"\n') +check( + "a project directory with no pyproject.toml is refused", + msg is not None and "no pyproject.toml" in msg, + str(msg), +) +msg = config_error('[tool.sdlc]\nproject = "/etc"\n') +check("an absolute project path is refused", msg is not None and "relative path" in msg, str(msg)) +msg = config_error('[tool.sdlc]\nproject = "../elsewhere"\n') +check( + "a project path leaving the repo is refused", + msg is not None and "relative path" in msg, + str(msg), +) +msg = config_error("[tool.sdlc]\nproject = 3\n") +check("a non-string project is refused", msg is not None and "must be a string" in msg, str(msg)) +msg = config_error("[tool.ruff]\n") +check("no project key is fine, and means the repo root", msg is None, str(msg)) + + +def project_of(pyproject: str, sub: str | None = None) -> str: + """Where project_dir() resolves to, relative to the repo root.""" + with tempfile.TemporaryDirectory() as root: + Path(root, "pyproject.toml").write_text(pyproject) + if sub: + Path(root, sub).mkdir(parents=True) + Path(root, sub, "pyproject.toml").write_text("[project]\nname='x'\nversion='0'\n") + S.REPO = Path(root) + return str(S.project_dir().relative_to(Path(root).resolve())) + + +check("the default project is the repo root", project_of("[tool.ruff]\n") == ".") +check( + "a named project resolves to that directory", + project_of('[tool.sdlc]\nproject = "image-server"\n', "image-server") == "image-server", +) + # ----------------------------------------------------------- codeql baseline