Skip to content

Commit 3865043

Browse files
johnscaclaude
andauthored
feat: add the validate-min-terraform action (ENG-8760) (#1)
[ENG-8760](https://stacklet.atlassian.net/browse/ENG-8760) ### what Adds `validate-min-terraform`, a composite action that reads a module's Terraform floor out of `required_version` and runs `terraform validate` at that release, with the parse in its own script and a test suite over it. Lays out the repository for the tools that follow it: one directory per tool, each carrying an `action.yml` and a `README.md`, alongside a lint workflow, a prek config, a justfile and a README recording the layout. ### why Five of the AWS and Azure onboarding module repositories need this check, and the version of it that ENG-8760 wrote carries enough shell to be worth one copy rather than five. Those repositories validate at the release pinned in `.tool-versions`, which sits near the top of the supported range, so nothing covered the bottom of it. That is how `required_version = ">= 1"` survived in a module that no release below 1.8 can parse. The action derives the floor instead of taking it as an input, so editing the constraint moves what CI tests in the same commit. It runs terraform-docs to parse the configuration rather than matching a regex against one file. The regex approach is what the published actions do, and it is why they misread a second `.tf`, a `.tf.json`, or a constraint split across lines, the last of which yields an empty string that semver then reads as "any version". The terraform-docs image is pinned by digest, not by tag. The project publishes no immutable releases, so a tag and the checksum published beside a release can both change underneath us, while a digest is content addressed and cannot. The release the action installs comes out of the configuration under test, so on a pull request the branch decides which Terraform runs in CI. `minimum-floor` puts a bound under that, defaulting to 1.14.0. HashiCorp publishes no support window for the community CLI, and its support policy routes a version to a regime by release date, which puts 1.14 in the band still receiving fixes and security patches. That reading is a judgement call, recorded in the README alongside the number, and the default wants revisiting whenever a new minor lands. Callers can raise it to their own support floor. Two things the clamp does not solve, and the action cannot. A pull request still chooses which providers `init` installs, and a provider plugin is a binary that `validate` runs to read its schema. So the job must hold no secrets and no write permission, which is the shape the usage example and all five callers have. The same is already true of the `terraform_validate` pre-commit hook these repos run today, so this is not new exposure, but it is worth stating where someone will read it. The parse lives in `read_floor.py` rather than inline in the action. Inline, nothing linted it: the pre-commit actionlint hook only reads `.github/workflows/`, and a composite `action.yml` is not a workflow. In a script a linter reads it and a test can run it without a runner. It is Python rather than shell because the work is parsing and comparing. One regex per term replaces a case statement, a `tr`, and two greps that all had to agree, and the version comparison is a tuple rather than a `sort -V` idiom. Splitting the derivation from the terraform-docs call is what makes the constraint cases unit tests: fourteen of the sixteen no longer start a container. Callers pin by commit SHA, because this repository publishes no release tags and a floating ref would put five repositories' CI under whatever lands here next. If the repository grows enough tools to want versioned releases, that is the point to add tags and switch the callers over. ### testing - [x] Ran the floor derivation against a module carrying three constraints at once, one in a `.tf.json` and one split across lines. It reports the highest lower bound, which is the one Terraform's own `init` enforces. - [x] Ran it against all five onboarding modules. Each resolves to the floor its `required_version` states. - [x] Checked the operator handling across a matrix of constraints. `>= 1.14.0`, `>= 1.14.0, < 2.0.0` and `~> 1.14` resolve, while `!=`, a bare version and `>` are rejected with the term that could not be read, and an upper bound on its own reports that the constraint states no lower bound. Review on the module PRs raised the first of those. - [x] Checked that a container which fails to run prints docker's own error, rather than surfacing further down as an empty parse. Review raised this one too. - [x] `just test` covers every constraint form the parser accepts or rejects, a constraint split across lines where that is the deciding bound, the three-constraint fixture, a `.terraform-docs.yml` that hides the requirements section, a module with no constraint at all, a floor under the clamp, the step output written to `GITHUB_OUTPUT`, and a container that cannot run. CI runs the same command. The suite takes under two seconds, since most cases call the parser directly and start no container. - [x] Review caught four defects, all now covered by that suite. A single-component bound such as `>= 1` was rejected outright, which is the constraint this ticket started from. A config hiding the requirements section made a module that declares a constraint look like one that declares none. The multiline fixture could not fail, because a higher bound in the sibling `.tf.json` decided the result either way. The clamp defaulted to a release nobody has patched in years. - [x] `just lint` clean, and zizmor re-run under a token to confirm its five online audits schedule rather than skip. The prek release that job installs now comes from `.tool-versions` rather than being resolved at run time, so CI and a local run agree on it. zizmor audits `action.yml` as well as the workflow. - [ ] Confirm the action runs end to end once this merges and the five module PRs point at it. Their `uses:` pin names the commit on this branch, so it resolves as soon as the branch exists on GitHub, but it wants repointing at the commit on `main` after this merges. ### docs `README.md` lists the tools and says how to add one. The action carries its own `README.md` with usage, inputs, outputs and the one case it cannot catch: `terraform validate` against a module directory treats every root variable as unknown and so never evaluates `validation` blocks, which leaves a floor that is only wrong inside one still passing. This repository is not described in `cloud-infra`, so its settings and required checks are unmanaged. Worth adding there separately. 🤖 Generated with [Claude Code](https://claude.com/claude-code) [ENG-8760]: https://stacklet.atlassian.net/browse/ENG-8760?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1 parent 8530322 commit 3865043

19 files changed

Lines changed: 585 additions & 1 deletion

File tree

‎.github/CODEOWNERS‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
* @stacklet/engineering

‎.github/workflows/ci.yml‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
pull_request:
8+
branches:
9+
- main
10+
11+
permissions: {}
12+
13+
jobs:
14+
lint:
15+
runs-on: ubuntu-latest
16+
permissions:
17+
contents: read
18+
steps:
19+
- name: Checkout
20+
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
21+
with:
22+
persist-credentials: false
23+
24+
- name: Set tool versions
25+
uses: wistia/parse-tool-versions@32f568a4ffd4bfa7720ebf93f171597d1ebc979a # v2.1.1
26+
with:
27+
postfix: _TOOL_VERSION
28+
29+
- uses: j178/prek-action@4e14d07f9231acabce116ccfca13b13dd9755ece # v3.0.0
30+
with:
31+
# Without a pin this installs whatever prek released most recently,
32+
# so a hostile release reaches CI with no change here. The version
33+
# comes from .tool-versions, so CI and a local run agree on it.
34+
prek-version: ${{ env.PREK_TOOL_VERSION }}
35+
env:
36+
# zizmor's five online audits skip silently without a token, and its
37+
# summary line reads the same either way, so CI would pass while the
38+
# supply-chain audits never ran.
39+
GH_TOKEN: ${{ github.token }}
40+
41+
test:
42+
runs-on: ubuntu-latest
43+
permissions:
44+
contents: read
45+
steps:
46+
- name: Checkout
47+
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
48+
with:
49+
persist-credentials: false
50+
51+
- name: Run the tests
52+
run: python3 -m unittest discover -s validate-min-terraform -p 'test_*.py'

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
__pycache__/

‎.pre-commit-config.yaml‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
repos:
2+
- repo: https://github.com/rhysd/actionlint
3+
rev: 393031adb9afb225ee52ae2ccd7a5af5525e03e8 # frozen: v1.7.11
4+
hooks:
5+
- id: actionlint
6+
7+
- repo: https://github.com/zizmorcore/zizmor-pre-commit
8+
rev: ea2eb407b4cbce87cf0d502f36578950494f5ac9 # frozen: v1.23.1
9+
hooks:
10+
- id: zizmor
11+
12+
- repo: https://github.com/astral-sh/ruff-pre-commit
13+
rev: f12be1ebaa5351c1fc76472de98db2c3446c8253 # frozen: v0.16.10
14+
hooks:
15+
- id: ruff-check
16+
- id: ruff-format
17+
18+
- repo: https://github.com/codespell-project/codespell
19+
rev: 2ccb47ff45ad361a21071a7eedda4c37e6ae8c5a # frozen: v2.4.2
20+
hooks:
21+
- id: codespell

‎.tool-versions‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
just 1.58.0
2+
prek 0.5.5
3+
uv 0.12.23

‎README.md‎

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,24 @@
11
# github-tooling
2-
Reusable GitHub Actions and workflows shared across Stacklet repositories
2+
3+
Reusable GitHub Actions and workflows shared across Stacklet repositories.
4+
5+
Each tool lives in its own top-level directory with an `action.yml` and a `README.md`. Reference one by path:
6+
7+
```yaml
8+
- uses: stacklet/github-tooling/<tool>@<commit-sha>
9+
```
10+
11+
Pin by commit SHA. This repository publishes no release tags, so a SHA is the only reference that cannot move.
12+
13+
## Tools
14+
15+
| Tool | Purpose |
16+
| --- | --- |
17+
| [validate-min-terraform](validate-min-terraform/) | Runs `terraform validate` at the floor a module's `required_version` declares, and fails when that floor is below a configured minimum. |
18+
19+
## Adding a tool
20+
21+
1. Create a directory named for the tool.
22+
2. Add an `action.yml` and a `README.md` covering usage, inputs, and outputs.
23+
3. Keep any non-trivial logic in its own script beside the `action.yml`, so that a linter reads it and a test can run it without a runner. Prefer Python over shell for anything that parses or compares.
24+
4. Add a row to the table above.

‎justfile‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# lint files
2+
lint:
3+
prek run --all-files
4+
5+
# run the tests
6+
test:
7+
python3 -m unittest discover -s validate-min-terraform -p 'test_*.py'

‎validate-min-terraform/README.md‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
# validate-min-terraform
2+
3+
Runs `terraform validate` against a Terraform module at the oldest release its
4+
`required_version` allows, so the bottom of the declared range gets tested
5+
rather than assumed.
6+
7+
## Usage
8+
9+
```yaml
10+
jobs:
11+
validate-min-terraform:
12+
runs-on: ubuntu-latest
13+
permissions:
14+
contents: read
15+
steps:
16+
- name: Checkout
17+
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
18+
with:
19+
persist-credentials: false
20+
21+
- uses: stacklet/github-tooling/validate-min-terraform@<commit-sha>
22+
```
23+
24+
> [!WARNING]
25+
> `terraform init` installs the providers the module names, and `validate` runs
26+
> each provider binary to read its schema. On a pull request that configuration
27+
> is untrusted, so give the job `contents: read`, and no secrets.
28+
29+
## Inputs
30+
31+
| Name | Default | Description |
32+
| --- | --- | --- |
33+
| `working-directory` | `.` | Directory holding the module. One directory per call, with no descent into subdirectories. |
34+
| `minimum-floor` | `1.14.0` | Refuse to run below this release. The default is the oldest release HashiCorp still ships fixes for. |
35+
| `terraform-docs-image` | a digest-pinned terraform-docs 0.20.0 | Image used to parse the configuration. |
36+
37+
## Outputs
38+
39+
| Name | Description |
40+
| --- | --- |
41+
| `version` | The Terraform release the module declares as its floor. |
42+
43+
## Requirements
44+
45+
The runner needs Docker and `python3`. `ubuntu-latest` has both, and the macOS
46+
runners have no Docker.
47+
48+
## Version constraints
49+
50+
`required_version` must state a lower bound with `>=`, `~>` or `=`, or name an
51+
exact version such as `1.14.0`. Upper bounds are allowed and ignored, as is
52+
`!=` unless it excludes the floor itself. The action rejects any constraint it
53+
cannot resolve to a definite release, and fails when the floor it resolves
54+
falls below `minimum-floor`.
55+
56+
Every `required_version` in the directory counts, including one in a `.tf.json`
57+
or split across lines. The floor is the highest lower bound among them, which
58+
is what Terraform itself applies.
59+
60+
## Known limitation
61+
62+
> [!NOTE]
63+
> `terraform validate` treats every root variable as unknown, so it never
64+
> evaluates `validation` blocks. A floor that is wrong only inside a variable
65+
> validation still passes here.
66+
67+
## Tests
68+
69+
`just test` runs `test_read_floor.py`, and CI runs the same command.

‎validate-min-terraform/action.yml‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
name: Validate the declared Terraform floor
2+
description: >-
3+
Reads the Terraform floor out of a module's required_version and runs
4+
terraform validate at that release.
5+
6+
inputs:
7+
working-directory:
8+
description: Directory holding the Terraform module to validate.
9+
required: false
10+
default: .
11+
minimum-floor:
12+
description: >-
13+
Refuse to run below this release. The floor comes out of the
14+
configuration under test, so on a pull request the branch decides which
15+
Terraform CI installs.
16+
required: false
17+
# HashiCorp routes a version to a support regime by release date, and 1.14
18+
# falls in the band still receiving fixes and security patches, so this is
19+
# the oldest release with a published claim to them. It wants revisiting
20+
# when a new minor lands. See the README.
21+
default: 1.14.0
22+
terraform-docs-image:
23+
description: >-
24+
Image used to parse the configuration. Pin by digest, not by tag.
25+
required: false
26+
# terraform-docs publishes no immutable releases, which leaves both a tag
27+
# and a release checksum able to change underneath us. A digest is content
28+
# addressed, so it cannot. This one is terraform-docs 0.20.0, the last
29+
# release published as a multi-arch index. Later tags are per-architecture,
30+
# so a digest pin would have to choose one.
31+
default: quay.io/terraform-docs/terraform-docs@sha256:37329e2dc2518e7f719a986a3954b10771c3fe000f50f83fd4d98d489df2eae2
32+
33+
outputs:
34+
version:
35+
description: The Terraform release the module declares as its floor.
36+
value: ${{ steps.floor.outputs.version }}
37+
38+
runs:
39+
using: composite
40+
steps:
41+
# The floor is read out of required_version rather than passed in, so that
42+
# editing the constraint moves what CI tests in the same commit.
43+
# terraform-docs parses the configuration instead of matching a regex
44+
# against one file, so it reports every required_version in the directory,
45+
# including one in a .tf.json or split across lines.
46+
- name: Read the declared Terraform floor
47+
id: floor
48+
shell: bash
49+
working-directory: ${{ inputs.working-directory }}
50+
env:
51+
TERRAFORM_DOCS_IMAGE: ${{ inputs.terraform-docs-image }}
52+
MINIMUM_FLOOR: ${{ inputs.minimum-floor }}
53+
# The script writes the step output itself, so this step carries no shell
54+
# logic. A composite step has to name a shell even so.
55+
run: '"$GITHUB_ACTION_PATH/read_floor.py"'
56+
57+
- name: Setup Terraform
58+
uses: hashicorp/setup-terraform@dfe3c3f87815947d99a8997f908cb6525fc44e9e # v4.0.1
59+
with:
60+
terraform_version: ${{ steps.floor.outputs.version }}
61+
62+
- name: Validate at the declared floor
63+
shell: bash
64+
working-directory: ${{ inputs.working-directory }}
65+
run: |
66+
terraform init -backend=false -input=false
67+
terraform validate

0 commit comments

Comments
 (0)