Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 14 additions & 6 deletions .github/workflows/catalog-release.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
name: catalog-release

# Publishes a catalog release: a GitHub Release tagged v<version> carrying
# ave-records.json, ave-records.manifest.json and SHA256SUMS, with a build
# provenance attestation for the two data files. Run by hand from
# ave-records.json, ave-records.manifest.json, ave-taxonomy.json (the SARIF
# taxonomy of exactly those records) and SHA256SUMS, with a build provenance
# attestation for the three data files. Run by hand from
# Actions > catalog-release; it builds a commit already on main and never
# pushes to the repository. Policy: docs/specs/scaling-and-governance.md,
# section 6.
Expand Down Expand Up @@ -126,19 +127,26 @@ jobs:
--manifest-out release/ave-records.manifest.json
jq --arg release "v$VERSION" '{release: $release} + .' release/ave-records.manifest.json > manifest.tmp
mv manifest.tmp release/ave-records.manifest.json
(cd release && sha256sum ave-records.json ave-records.manifest.json > SHA256SUMS)
python3 scripts/build_taxonomy.py \
--records release/ave-records.json \
--release "v$VERSION" \
--release-date "$(jq -r '.generated_at[:10]' release/ave-records.manifest.json)" \
--download-uri "https://github.com/$GITHUB_REPOSITORY/releases/download/v$VERSION/ave-taxonomy.json" \
--out release/ave-taxonomy.json
(cd release && sha256sum ave-records.json ave-records.manifest.json ave-taxonomy.json > SHA256SUMS)
cat release/ave-records.manifest.json release/SHA256SUMS

# the tag does not exist yet, so source_commit is checked against this run's commit
- name: Verify the release files
run: python scripts/check_release_assets.py release --tag "v$VERSION" --commit "$(git rev-parse HEAD)"
run: python scripts/check_release_assets.py release --tag "v$VERSION" --commit "$(git rev-parse HEAD)" --repo "$GITHUB_REPOSITORY"

- name: Attest build provenance for both data files
- name: Attest build provenance for the three data files
uses: actions/attest-build-provenance@4d101475d8b20a2381f78447822ac1eab6504dd8 # v4.2.2
with:
subject-path: |
release/ave-records.json
release/ave-records.manifest.json
release/ave-taxonomy.json

- name: Create the release
env:
Expand All @@ -157,4 +165,4 @@ jobs:
--title "AVE v$VERSION" \
--notes-file release-notes.md \
"${draft_flag[@]}" \
release/ave-records.json release/ave-records.manifest.json release/SHA256SUMS
release/ave-records.json release/ave-records.manifest.json release/ave-taxonomy.json release/SHA256SUMS
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,13 @@ Format: [Semantic Versioning](https://semver.org). Schema versions and record se
## [Unreleased]

### SARIF
- **The taxonomy as a verified catalog-release asset** (#442). The `catalog-release` workflow builds
`ave-taxonomy.json` with `scripts/build_taxonomy.py` from the release's own `ave-records.json`, stamped with the
release's version, the manifest's date and its own download URL, lists it in `SHA256SUMS` and attests it with the
other data files. `scripts/check_release_assets.py` regenerates it from the release's records and demands a byte
match, naming the field or taxon that differs first, and takes a new `--repo owner/name` (default `aveproject/ave`)
so a fork's release is verified against the fork. Releases from v1.3.0 must carry the taxonomy; v1.0.0 to v1.2.0
verify as before. That makes the order mechanical: this has to merge before v1.3.0 is cut.
- **The AVE taxonomy as a canonical file, form C and taxonomy checks** (#420). `dist/ave-taxonomy-latest.json` is the AVE
taxonomy as a SARIF 2.1.0 `toolComponent` with a frozen `guid` (`5110b285-34d7-5334-94db-0308b24cfb49`, the UUIDv5 of
`https://aveproject.org/taxonomy/AVE`) and one taxon per record, including records whose status is not `active`.
Expand Down
27 changes: 24 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -512,12 +512,16 @@ watching and the API.
A catalog release is a [GitHub Release](https://github.com/aveproject/ave/releases)
tagged `vX.Y.Z`: a fixed copy of the record set that does not change after it is
published. Each one carries three files and a build provenance attestation for
the first two:
the first two; from v1.3.0 it carries a fourth, the SARIF taxonomy, which the
attestation covers too:

- `ave-records.json`: every published record.
- `ave-records.manifest.json`: the release tag, `schema_version`, `record_count`,
the SHA-256 of `ave-records.json` and `source_commit`, the commit it was built from.
- `SHA256SUMS`: checksums of both files.
- `ave-taxonomy.json` (from v1.3.0): the [SARIF taxonomy](docs/specs/ave-in-sarif.md)
of exactly these records, stamped with the release's own version, date and
download link, ready to embed in a tool's SARIF output.
- `SHA256SUMS`: checksums of the data files (three of them from v1.3.0).

Pin a tag, then check what you downloaded:

Expand All @@ -535,7 +539,24 @@ gh attestation verify ave-records.manifest.json --repo aveproject/ave
```

`check_release_assets.py` confirms the checksums, that the manifest matches the
records file, and that `source_commit` is the commit the tag points at.
records file, and that `source_commit` is the commit the tag points at. From
v1.3.0 it also regenerates `ave-taxonomy.json` from the records file and requires
a byte for byte match, and refuses a release without it.

From v1.3.0 on, add the taxonomy to the same steps. `--repo` is the repository the
release is published under, so verify a fork's release with `--repo owner/name`:

```bash
TAG=v1.3.0
curl -fsSLO "https://github.com/aveproject/ave/releases/download/$TAG/ave-taxonomy.json"
sha256sum -c SHA256SUMS # now covers three files
python ave-src/scripts/check_release_assets.py . --tag "$TAG" --git-dir ave-src --repo aveproject/ave
gh attestation verify ave-taxonomy.json --repo aveproject/ave
```

The taxonomy has stable URLs: `https://github.com/aveproject/ave/releases/download/vX.Y.Z/ave-taxonomy.json`
for a release, and `https://github.com/aveproject/ave/releases/latest/download/ave-taxonomy.json`
for the newest.

Three kinds of record file, and which is which:

Expand Down
21 changes: 18 additions & 3 deletions docs/specs/ave-implementer-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,13 +35,15 @@ comes first.
## Pin and verify a release

A catalog release is a [GitHub Release](https://github.com/aveproject/ave/releases)
tagged `vX.Y.Z`. It never changes after it is published, and carries three files:
tagged `vX.Y.Z`. It never changes after it is published, and carries three files,
and from v1.3.0 a fourth:

- `ave-records.json`: every published record, whatever its status, except drafts.
- `ave-records.manifest.json`: what the release is (below).
- `SHA256SUMS`: checksums of both files.
- `ave-taxonomy.json` (from v1.3.0): the SARIF taxonomy of exactly those records.
- `SHA256SUMS`: checksums of the data files.

The first two also carry a build provenance attestation, signed by the workflow
The data files also carry a build provenance attestation, signed by the workflow
that built them. `v1.2.0` is the first release with these files.

Download a release and verify it. This needs network access, `curl`, `git`,
Expand Down Expand Up @@ -78,6 +80,19 @@ What each check proves:
`aveproject/ave`. This is the check that does not trust where you downloaded
from. A file that differs by a single byte has no attestation and fails.

From v1.3.0 the release also carries `ave-taxonomy.json`, the AVE taxonomy for the SARIF you
emit ([AVE-in-SARIF](ave-in-sarif.md)), already stamped with the release's version, date and
download link. Download it with the other files; `sha256sum -c SHA256SUMS` then covers three
files, `check_release_assets.py` regenerates the taxonomy from the release's records and refuses a
release whose copy differs by a byte (pass `--repo owner/name` to verify a fork's release), and
the attestation check takes it too:

```bash
curl -fsSL -o "ave-$TAG/ave-taxonomy.json" "https://github.com/aveproject/ave/releases/download/$TAG/ave-taxonomy.json"
gh attestation verify "ave-$TAG/ave-taxonomy.json" --repo aveproject/ave \
--signer-workflow aveproject/ave/.github/workflows/catalog-release.yml
```

The manifest of `v1.2.0`:

```json
Expand Down
2 changes: 1 addition & 1 deletion docs/specs/ave-in-sarif.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,7 @@ Add one taxon per AVE record that appears in the scan results. Scanners may incl
python3 scripts/build_taxonomy.py --only AVE-2026-00001,AVE-2026-00075 --out -
```

The release-stamped form, for a scanner pinned to a catalog release, takes that release's records file and its facts: `python3 scripts/build_taxonomy.py --records ave-records.json --release v1.2.0 --release-date 2026-10-29 --download-uri <the release's ave-records.json> --out -`. The file is not yet a catalog-release asset. A taxon's `id` and `name` are the record's `ave_id` and `title` exactly, and the checker compares them with the catalog (`taxon_unknown`, `taxon_name_differs`).
The release-stamped form, for a scanner pinned to a catalog release, takes that release's records file and its facts: `python3 scripts/build_taxonomy.py --records ave-records.json --release v1.2.0 --release-date 2026-10-29 --download-uri <the release's ave-records.json> --out -`. From v1.3.0 every catalog release carries the same file as an asset, `ave-taxonomy.json`: the pinned copy a tool should embed, already stamped with the release's `version`, `semanticVersion`, `releaseDateUtc` and `downloadUri` (`https://github.com/aveproject/ave/releases/download/vX.Y.Z/ave-taxonomy.json`, or `https://github.com/aveproject/ave/releases/latest/download/ave-taxonomy.json` for the newest), listed in the release's `SHA256SUMS`, attested, and checked byte for byte against the release's own records by `scripts/check_release_assets.py`. `dist/ave-taxonomy-latest.json` is the moving copy, and `latest` has no `version`. A taxon's `id` and `name` are the record's `ave_id` and `title` exactly, and the checker compares them with the catalog (`taxon_unknown`, `taxon_name_differs`).

---

Expand Down
22 changes: 20 additions & 2 deletions docs/specs/scaling-and-governance.md
Original file line number Diff line number Diff line change
Expand Up @@ -293,15 +293,33 @@ The two are numbered separately.
- `ave-records.manifest.json`: `release`, `schema_version`, `record_count`,
`generated_at`, `source`, `sha256` (of `ave-records.json`) and `source_commit`,
the tagged commit.
- `SHA256SUMS`: both files, in `sha256sum` format.
- A build provenance attestation for both files, checked with
- `ave-taxonomy.json` (from v1.3.0): the SARIF taxonomy (see
`docs/specs/ave-in-sarif.md`) of exactly this release's records, built by
`scripts/build_taxonomy.py` from `ave-records.json`, with the release's own
version, the manifest's `generated_at` date and its own download URL.
- `SHA256SUMS`: the data files, in `sha256sum` format (all three from v1.3.0).
- A build provenance attestation for each data file, checked with
`gh attestation verify <file> --repo aveproject/ave`.

Asset names carry no version; the tag does. `scripts/check_release_assets.py`
checks the files against each other, against the tag and against the tagged commit.
The release workflow runs it before publishing, and an implementer can run it after
downloading (README, "Releases and pinning").

**The taxonomy asset.** The verifier regenerates `ave-taxonomy.json` from the
release's own `ave-records.json` and demands a byte match, so no field, taxon or
character can differ from what the generator produces; a failure names the field
or taxon first. Releases from v1.3.0 must carry it and a release without it fails
verification; v1.0.0 to v1.2.0 shipped before it existed and verify without it.
That makes the order mechanical: a release cut by a workflow that does not build
the taxonomy fails its own verification, so the change that builds it has to merge
before v1.3.0 is cut. Its `downloadUri` must point at the repository it is
published under (`--repo owner/name`, default `aveproject/ave`), so a fork's
release is verified against the fork. Stable URLs:
`https://github.com/aveproject/ave/releases/download/vX.Y.Z/ave-taxonomy.json`
and `https://github.com/aveproject/ave/releases/latest/download/ave-taxonomy.json`.
`dist/ave-taxonomy-latest.json` is the moving copy and has no `version`.

**Cadence:** a release when records have changed, at most monthly unless a
correction is urgent. This is the maintainer's policy, and the maintainer can change
it.
Expand Down
Loading
Loading