Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .githooks/pre-push
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/bin/sh
# Opt in with an absolute core.hooksPath pointing to this tooling checkout.
# Keep Git's stdin intact: the validator checks the objects actually pushed.
set -eu
hook_dir=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
exec python3 "$hook_dir/../scripts/check_workflows.py" --pre-push
9 changes: 9 additions & 0 deletions .github/actionlint.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"version": "1.7.12",
"sha256": {
"darwin_amd64": "5b44c3bc2255115c9b69e30efc0fecdf498fdb63c5d58e17084fd5f16324c644",
"darwin_arm64": "aba9ced2dee8d27fecca3dc7feb1a7f9a52caefa1eb46f3271ea66b6e0e6953f",
"linux_amd64": "8aca8db96f1b94770f1b0d72b6dddcb1ebb8123cb3712530b08cc387b349a3d8",
"linux_arm64": "325e971b6ba9bfa504672e29be93c24981eeb1c07576d730e9f7c8805afff0c6"
}
}
26 changes: 26 additions & 0 deletions .github/workflows/workflow-lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: Workflow validation

on:
push:
pull_request:
workflow_dispatch:

permissions:
contents: read

jobs:
workflow-lint:
runs-on: ubuntu-24.04
timeout-minutes: 10
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
persist-credentials: false
- name: Install checksum-pinned actionlint
run: |
python3 scripts/install_actionlint.py --directory "$RUNNER_TEMP/actionlint"
echo "$RUNNER_TEMP/actionlint" >> "$GITHUB_PATH"
- name: Validate every workflow at the checked-out commit
run: python3 -B scripts/check_workflows.py --ref HEAD
- name: Test parser regression and exact-push-ref guard
run: python3 -B -m unittest discover -s tests -p test_workflow_validation.py -v
5 changes: 5 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,11 @@ For a parameter improvement, supply exact rational choices, the full dependency
argument, and an updated patch against the pinned source. Explain whether the
change stays within the parameter ceiling or changes one of its hypotheses.

Before publishing a branch, run the [workflow validation gate](docs/workflow-validation.md)
against its exact committed tree. The guide includes an opt-in pre-push hook and
a safe tooling-overlay procedure for branches with an intentionally frozen
scientific base. Workflow lint does not substitute for a scientific replay.

Keep `upstream/`, frozen source archives and pinned evidence unchanged. When
changing generated artifacts, edit their generators and include the regenerated
certificates and patches in the same change. Run the checks affected by your
Expand Down
161 changes: 161 additions & 0 deletions docs/workflow-validation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
# Validate workflows before publishing a branch

GitHub parses every workflow on a new head, including manual-only workflows and
jobs disabled by a repository condition. A syntax or expression-context error can
therefore produce a failed run with **zero jobs**. A rerun of the same head does
not repair the YAML. Workflow validation is a tooling check, not a scientific
replay, ledger reconciliation, certificate verification, or theorem approval.

## One-time tool installation and everyday checks

Python 3.10+ and Git are needed. The standard-library installer downloads
[actionlint 1.7.12](https://github.com/rhysd/actionlint/releases/tag/v1.7.12)
from its official release and verifies the platform-specific SHA-256 pinned in
`.github/actionlint.json` before writing the executable. Linux and macOS on
amd64/arm64 are supported. It only writes to the directory explicitly supplied;
it does not change shell profiles, Git settings, or existing hooks.

```sh
python3 scripts/install_actionlint.py --directory "$HOME/.local/bin"
export PATH="$HOME/.local/bin:$PATH"
# Includes uncommitted changes, and every .yml/.yaml workflow, not just a diff.
python3 -B scripts/check_workflows.py
# Validate the actual committed tree before publication.
python3 -B scripts/check_workflows.py --ref HEAD
python3 -B -m unittest discover -s tests -p test_workflow_validation.py -v
```

An existing exact-version binary can be supplied with `--actionlint /path/to/actionlint`.
For other operating systems, install the same version using its official
[installation guide](https://github.com/rhysd/actionlint/blob/v1.7.12/docs/install.md).
A missing/wrong-version linter, missing workflow directory, Git read failure,
invalid YAML, invalid expression or disallowed expression context fails closed.
The validator copies workflow and local-action metadata into an isolated temporary
project. Worktree mode uses tracked and non-ignored untracked source files; it does
not import ignored build/dependency trees or Git/worktree administrative files.
Symlink source paths are rejected explicitly rather than silently bypassing
local-action metadata checks. For local JS/Docker actions, empty placeholders preserve regular-resource
file existence without reading or executing their contents. Reusable-workflow
inputs and local-action metadata are checked in this project context. It never
runs workflow commands, changes the checked-out branch, or loads repository-specific
ignore rules. Exact-object reads disable Git replacement refs, so a local object
replacement cannot disguise the tree that will actually be published. Native actionlint checks remain enabled without error suppression.
The optional external ShellCheck/Pyflakes integrations are disabled deliberately:
this gate covers workflow syntax and semantics independently of those tools.

`Workflow validation / workflow-lint` runs on every push and pull request, with
no branch, path, or repository-name filters, and can also be dispatched manually.
It validates the complete checked-out Git tree and runs real-linter regression
tests, including the original P112 job-level `env.TMPDIR: ${{ runner.temp }}`
failure. A push run identifies the exact published head; a pull-request run may
validate GitHub's synthetic merge commit, so record which one was tested.
No branch protection, repository permissions, or existing replay triggers are
changed by this tooling. Maintainers may separately decide whether to require
this check. Always confirm that a real job ran at the expected head; an absent,
skipped, pending, or zero-job run is not a lint pass.

## Opt-in pre-push guard

No hook is activated automatically. First inspect any existing hook configuration
and hook file; do not overwrite an existing hook or bypass it. A configured
`core.hooksPath` replaces Git's default hook directory for **all** hook types.
If an existing hook framework is in use, add the validator to its existing
pre-push chain while preserving the same four-column stdin for every consumer.

```sh
git config --show-origin --get core.hooksPath || true
git rev-parse --git-path hooks/pre-push
```

For a repository without existing hooks, opt in with an **absolute path** to a
stable tooling checkout containing `.githooks`, `scripts/check_workflows.py`, and
`.github/actionlint.json`. Keep that tooling checkout at a reviewed commit and
make the pinned actionlint binary available on PATH. For example:

```sh
# Run in the repository whose pushes should be checked.
tooling=/absolute/path/to/reviewed-tooling-checkout
git config --local core.hooksPath "$tooling/.githooks"
```

This is an explicit user choice, not a command run by the installer or CI. It
changes only that repository's configuration (shared by its worktrees), not
other repositories or global Git configuration. Keep the tooling checkout
separate when research branches predate the guard; a relative `.githooks` path
would disappear when switching to such a branch. Do not remove or move the
tooling checkout while it is configured. To remove this opt-in, inspect the
current value first, then unset that value and restore any previous setup.

The hook validates the local object IDs Git actually intends to push, even when
a different branch is checked out or a worktree has an uncommitted fix. It checks
all non-deletion updates, including new branches and annotated commit tags, and
fails the push if any fail. Deletion-only pushes have no new tree to lint. Tags
pointing to non-commit objects are rejected. Hooks can be bypassed by `--no-verify`
and are not invoked by GitHub/API writes; those publication paths must run the
same validator against the exact prepared tree and then verify remote CI.

## New research branches from an intentionally frozen base

Refresh and read canonical guidance using the
[agent checkpoint procedure](research/agent-procedure.md#looping-agents-refresh-upstream-at-each-research-checkpoint).
Do not reset, rebase, or edit another worker's in-progress branch. Create a new
isolated worktree, pin the scientific starting commit, and record any separate
tooling overlay in the PR. Fetching a corrected fork main does not automatically
correct a new branch created from an older upstream commit.

```sh
git fetch --no-tags https://github.com/CrocSwap/integer-mult-bounds.git main
research_base=$(git rev-parse FETCH_HEAD)
# Read the fetched guidance and compare priorities before choosing the base.
git show "$research_base:AGENTS.md"
git show "$research_base:autoresearch.md"
git show "$research_base:CONTRIBUTING.md"
# Use a new branch name and an unused worktree path.
git worktree add -b audit/your-scoped-task ../your-scoped-task "$research_base"
cd ../your-scoped-task
python3 "$tooling/scripts/check_workflows.py" --repo "$PWD" --ref HEAD
```

As of 2026-10-19, canonical main `20ecf2891d80b5acf18d980b2164fb44ea7faa90`
still contains the invalid P112 job-level runner context. The workflow-only
correction is reviewed commit `89031d21f01f8b676f87c76443f5afc41457a17c`,
merged in [fork PR #8](https://github.com/Clear1nsight/integer-mult-bounds/pull/8)
and proposed in [upstream PR #3885](https://github.com/CrocSwap/integer-mult-bounds/pull/3885).
It writes `TMPDIR` from `$RUNNER_TEMP` into `$GITHUB_ENV` in the first step.
Check the current tree and PR state; do not assume this temporary overlay remains
necessary forever. When it is missing and CI changes are within the branch's
approved scope, inspect and apply that **single** reviewed commit:

```sh
p112_fix=89031d21f01f8b676f87c76443f5afc41457a17c
git fetch --no-tags https://github.com/Clear1nsight/integer-mult-bounds.git "$p112_fix"
git show --stat "$p112_fix"
git show "$p112_fix" -- .github/workflows/p112-common-slice.yml
git cherry-pick -x "$p112_fix"
python3 "$tooling/scripts/check_workflows.py" --repo "$PWD" --ref HEAD
```

If the correction is already present, skip it. If it conflicts, stop and review
the actual difference; do not overwrite the workflow wholesale or repin research
inputs. The same rule applies to adopting this safeguard: cherry-pick only the
reviewed guard commit(s) if the branch's scope permits tooling changes. Otherwise
run the validator from the separate tooling checkout and report the blocker for
a separate tooling PR. In particular, branches explicitly excluding CI changes
must remain untouched until their owners approve a scope change.

Keep `research_base`, selected-result pins, `upstream/`, frozen source archives,
original intake history, historical receipts and contributor credits unchanged.
A tooling overlay is a separate commit, not an update to scientific evidence.
Before pushing, validate the final committed tree. After pushing, verify the
remote SHA and successful nonzero-job workflow-lint run for that SHA. Report
workflow lint separately from any scientific replay, including failed or unrun
replay stages. Do not alter evidence or relax mathematical checks to make CI green.

## Updating the linter

Review the official release notes and asset digests; update the version and all
supported checksums together. Run the full workflow lint and guard regression
suite, including negative controls, before publishing the update. A green check
at an older head is not verification of the update.

Prepared with OpenAI assistance. Apache-2.0.
178 changes: 178 additions & 0 deletions scripts/check_workflows.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,178 @@
#!/usr/bin/env python3
"""Lint every workflow in a worktree or exact Git commit, without running it.

--pre-push consumes Git's four-column pre-push input and validates every
non-deleted local object, including non-checked-out branches and annotated tags.
The validator can live in a separate tooling checkout from the repository tested.
"""
import argparse
import json
from pathlib import Path
import re
import shutil
import subprocess
import sys
import tempfile

MANIFEST = Path(__file__).resolve().parents[1] / ".github/actionlint.json"
WORKFLOWS = Path(".github/workflows")


class ValidationError(Exception):
pass


def git(repo: Path, *args: str) -> bytes:
result = subprocess.run(["git", "--no-replace-objects", "-C", str(repo), *args], capture_output=True)
if result.returncode:
raise ValidationError(result.stderr.decode(errors="replace").strip())
return result.stdout


def push_refs(lines) -> list[str]:
refs = []
for line in lines:
fields = line.split()
if len(fields) != 4:
raise ValidationError("malformed pre-push input: expected four fields")
_, local_oid, _, remote_oid = fields
if any(not re.fullmatch(r"(?:[0-9a-f]{40}|[0-9a-f]{64})", oid)
for oid in (local_oid, remote_oid)):
raise ValidationError("malformed pre-push object ID")
if set(local_oid) == {"0"}: # deletion, no new workflow tree to check
continue
if local_oid not in refs:
refs.append(local_oid)
return refs


def is_workflow(path: Path) -> bool:
return path.parent == WORKFLOWS and path.suffix in (".yml", ".yaml")


def is_metadata(path: Path) -> bool:
return path.name in ("action.yml", "action.yaml")


def snapshot(repo: Path, ref: str | None) -> tuple[str, dict[str, bytes], set[str]]:
files = {}
regular_paths = set()
if ref is None:
# Git's source list excludes .git, ignored build/dependency trees, and
# linked-worktree administrative files. Staged and untracked source
# paths are included; ordinary worktree deletions stay deleted.
paths = git(repo, "ls-files", "-z", "--cached", "--others", "--exclude-standard")
for raw_path in set(paths.split(b"\0")):
if not raw_path:
continue
path = Path(raw_path.decode())
source = repo / path
if any((repo / Path(*path.parts[:n])).is_symlink()
for n in range(1, len(path.parts) + 1)):
raise ValidationError(f"symlink source paths are unsupported: {path}")
if not source.exists():
continue
regular = source.is_file()
if regular:
regular_paths.add(path.as_posix())
if not (is_workflow(path) or is_metadata(path)):
continue
if not regular:
raise ValidationError(f"workflow/action metadata must be a regular file: {path}")
files[path.as_posix()] = source.read_bytes()
label = "working tree (includes uncommitted workflow edits)"
else:
oid = git(repo, "rev-parse", "--verify", "--end-of-options",
f"{ref}^{{commit}}").decode().strip()
label = oid
entries = git(repo, "ls-tree", "-rz", "--full-tree", oid)
for entry in entries.split(b"\0"):
if not entry:
continue
metadata, raw_path = entry.split(b"\t", 1)
mode, kind, blob = metadata.decode().split()
path = Path(raw_path.decode())
if path.is_absolute() or ".." in path.parts or ".git" in path.parts:
raise ValidationError(f"unsafe Git tree path: {path}")
if mode == "120000":
raise ValidationError(f"symlink source paths are unsupported: {path}")
regular = mode in ("100644", "100755") and kind == "blob"
if regular:
regular_paths.add(path.as_posix())
if not (is_workflow(path) or is_metadata(path)):
continue
if not regular:
raise ValidationError(f"workflow/action metadata must be a regular file: {path}")
files[path.as_posix()] = git(repo, "cat-file", "blob", blob)
if not any(is_workflow(Path(path)) for path in files):
raise ValidationError(f"no workflow files found in {label}; refusing an empty pass")
# Local JS/Docker action metadata may reference resources outside its own
# directory. Preserve regular-file existence, without reading/executing the
# resources, only when there is local-action metadata to check.
resources = regular_paths if any(is_metadata(Path(p)) for p in files) else set()
return label, files, resources


def actionlint_binary(command: str) -> str:
binary = shutil.which(command)
if not binary:
raise ValidationError("actionlint is missing; run scripts/install_actionlint.py "
"--directory /your/tool/directory and add it to PATH")
binary = str(Path(binary).resolve())
expected = json.loads(MANIFEST.read_text())["version"]
version = subprocess.run([binary, "-version"], capture_output=True, text=True)
if version.returncode or version.stdout.splitlines()[:1] != [expected]:
raise ValidationError(f"actionlint {expected} required; got {version.stdout.strip()!r}")
return binary


def lint(binary: str, label: str, files: dict[str, bytes], resources: set[str]) -> int:
with tempfile.TemporaryDirectory(prefix="workflow-lint-") as tmp:
root = Path(tmp)
# Without a project marker, actionlint silently omits local reusable
# workflow and local-action metadata checks.
(root / ".git").mkdir()
# Never import ignore rules, even as empty resource placeholders.
config_paths = {".github/actionlint.yaml", ".github/actionlint.yml"}
for name in sorted(resources | files.keys()):
if name in config_paths:
continue
destination = root / name
destination.parent.mkdir(parents=True, exist_ok=True)
destination.write_bytes(files.get(name, b""))
workflows = sorted(name for name in files if is_workflow(Path(name)))
# Native workflow/context/expression/local-metadata checks stay enabled.
# External shell/Python linters are excluded to avoid PATH-dependent
# results. Empty resource placeholders supply existence, not code.
command = [binary, "-no-color", "-shellcheck=", "-pyflakes=", *workflows]
print(f"Checking {len(workflows)} workflows at {label}", flush=True)
result = subprocess.run(command, cwd=root)
return result.returncode


def main(argv=None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--repo", type=Path, default=Path.cwd())
parser.add_argument("--actionlint", default="actionlint")
mode = parser.add_mutually_exclusive_group()
mode.add_argument("--ref", help="validate exactly this committed tree, not the checkout")
mode.add_argument("--pre-push", action="store_true")
args = parser.parse_args(argv)
try:
refs = push_refs(sys.stdin) if args.pre_push else [args.ref]
if not refs:
print("No non-deleted refs to validate")
return 0
binary = actionlint_binary(args.actionlint)
failed = False
for ref in refs:
label, files, resources = snapshot(args.repo, ref)
failed = bool(lint(binary, label, files, resources)) or failed
return 1 if failed else 0
except (ValidationError, OSError, ValueError) as error:
print(f"Workflow validation failed: {error}", file=sys.stderr)
return 1


if __name__ == "__main__":
raise SystemExit(main())
Loading