Add a release skill, and put every release command in RELEASING.md by ryanwelcher · Pull Request #710 · WordPress/contributor-toolkit

ryanwelcher · 2026-10-23T14:14:07Z

Why

RELEASING.md described each step but left most of the commands to the person: how to check the macOS artifact, how to find the merged commit, which tag the release notes start from, how to tell the docs deploy actually ran. That makes a release slow to do by hand and easy for an agent to get wrong.

What changes

RELEASING.md gets the commands. Start-state checks and the release/X.Y.Z branch name, the bump diff check, the macOS artifact checks (mount the .dmg, list the app.asar root, read npm's version and CFBundleShortVersionString), re-checking after a branch update, confirming the merged commit matches the checked build, the base tag and commit count for the notes, --latest for a stable release, checking the docs deploy job succeeded, the milestone commands, and a new section 7 that checks the result.
A release skill in .claude/skills/release/ (/release X.Y.Z in Claude Code) and .agents/skills/release/. It is a thin wrapper over RELEASING.md, like self-review is over the review standard. It adds five stops where it waits for a yes (branch, push and PR, a branch update, gh release create, docs and milestone), and merges only when the person asks, guarded by the checked head.
AGENTS.md points at RELEASING.md and the skill under "Where things are".
What the 2.0.0 release turned up (c5ef531): the bump diff allows the build.electronLanguages lines that i18n:download writes since Trim the packaged app by about 350 MB: unused PHP builds, renderer libraries and Chromium locales #707; section 2 checks the macOS signature, notarization and staple, and the Windows installer's Authenticode signer; Buildkite's commit is confirmed by the person, because the build reports nothing to the pull request; the merge uses --match-head-commit; and section 7 checks the docs site's Download button from a new tab, because the button keeps the release it read for the tab's session.
Third review round (43fdd1b): only X.Y.Z and X.Y.Z-beta.N are accepted, so an rc is never published with --latest, and the stable notes base excludes every pre-release tag (--exclude '*-*'); section 3 checks the branch contains trunk before the merge and says to stop without tagging if the merged commit differs; a beta skips the docs deploy, as docs.yml and CONTRIBUTING.md intend; and section 2 notes that the pipeline asks for commit statuses that never reach the pull request.
Fourth review round (72c3f69): the skill runs the check after the merge whichever way the bump was merged, not only when it merged it; a mismatch there says how to recover (check the trunk build of the merged commit with section 2, then tag); and the milestone check lists open pull requests as well as issues.
Semantic Versioning (140a866): a paragraph at the top of RELEASING.md says versions follow semver and what major, minor and patch mean for an app with no public API.

Not in it: downloading artifacts from Buildkite. The person downloads them; the skill checks them.

How to test this

Platforms: macOS for the artifact checks (they use hdiutil and plutil); anything else works on either.

Starting state: this branch checked out, npm ci done, and a macOS build in dist/ (npm run dist), or a .dmg downloaded from any Buildkite build.

From the repository root, run the section 1, item 1 checks in RELEASING.md with X.Y.Z = 2.1.0. The two rev-parse lines match, and the other two commands print nothing.
Run the section 2 block against the .dmg. It prints signed; accepted with source=Notarized Developer ID and origin naming Automattic (an unsigned local build fails these three); The validate action worked!; /node_modules, /package.json, /src; a version starting with 11.; then the build's version. Then run the Windows signature command against a Buildkite .exe. It prints a subject line naming O=Automattic Inc..
In Claude Code, run /release 2.1.0 and stop it at the first question. It runs the start checks and asks before creating release/2.1.0.

What must not have happened:

After step 2, git status shows no change to package.json. An earlier draft used asar extract-file, which writes into the current directory.
The temporary mount point is gone after the block finishes.

Risks and limitations

Review outcome: four rounds, 24 findings, all fixed. CodeRabbit did not run (automatic review is off), so the pre-PR review is the review of record.
The skill has not driven a full release yet. 2.0.0 was released by hand from trunk's RELEASING.md, and what it turned up is folded in here (c5ef531).
The contents checks look inside the macOS build only. The Windows file is also checked for its signature; the Linux file is checked by name.

Procedure in RELEASING.md, behaviour in the skill. The first draft put the commands in the skill. The review flagged that a person releasing by hand would never see them and that the two would drift, so they moved to RELEASING.md, and the skill keeps only what is agent-specific: the stops, leaving the merge to the person, and where the notes draft lives.
No Buildkite API path. Downloading through the API needs a token and the pipeline slug; asking the person to download into a named directory works today with nothing to configure.

Review outcome (required — see AGENTS.md)

Four rounds, 24 findings, all fixed.

Review: completed. Fresh Explore subagent with the diff and code-review.instructions.md. First round on head a31b25d, base 630253f (trunk, no PR yet): 9 findings (6 fix here, 3 follow-up), all fixed in 5fb565e, including asar extract-file overwriting package.json from the repo root, an unconditional milestone close that contradicted the beta rule, unfetched refs in the start checks, and procedure living only in the skill. Second round on 5fb565e: 8 findings, all fixed in fb32bc6, including the docs check reading the previous run and not failing on a failed run, notes commands placed after the command that needs the notes, ambiguous step references, and missing milestone commands. npm run lint and npm test pass (2212 pass, 2 skipped).
Since review: 5fb565e → fb32bc6: the second round's fixes only, not re-reviewed by a third pass. The new gh commands were run against this repository (milestone listing, the deploy-job query on the last docs dispatch) and the artifact commands against a local macOS build. fb32bc6 → c5ef531: the 2.0.0 findings, not reviewed. Every new command was run against the signed 2.0.0 files and printed what the text says, and the Windows command was also run against a copy with its signature entry cleared, where it fails with not signed. npm run lint passes.
Third round on c5ef531, base trunk, merge base 630253f: 4 findings (2 fix here, 2 follow-up), all fixed in 43fdd1b. One, that Buildkite posts commit statuses to the pull request, was checked against GitHub and did not hold: the 2.0.0 bump head 9df142b has only CodeRabbit's status, so the fix is a note, not the suggested gh api …/status check. The new up-to-date check and the --exclude '*-*' base were run against this repository. npm run lint passes; npm test was not run, since only Markdown changed.
Fourth round on 43fdd1b, base trunk, merge base 630253f: fresh Explore subagent with the diff and code-review.instructions.md. 3 findings (3 fix here, 0 follow-up), all fixed in 72c3f69: the skill tied the check after the merge only to the path where it merges itself, so a merge done by the person went straight to tagging; the stop on a mismatched merge left no way forward; and gh issue list --milestone misses open pull requests (gh pr list --search 'milestone:…' added, run against v2.0.0). npm run lint and npm test pass on 43fdd1b (2212 pass, 2 skipped).
Since review: 43fdd1b → 72c3f69: the fourth round's fixes only, Markdown in RELEASING.md and both SKILL.md copies, not re-reviewed. 72c3f69 → 140a866: one paragraph in RELEASING.md on Semantic Versioning, not reviewed.
CodeRabbit: not run. Its check reads pass but the message says automatic reviews are disabled; the pre-PR review above is the review of record, covering head 43fdd1b.

Screenshots or recording

Nothing on screen changed; this is documentation and an agent skill.

🤖 Generated with Claude Code

…blic step

…e release skill

…, and order the notes before the release

coderabbitai · 2026-10-23T14:14:19Z

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Source mode: Auto
Repository: WordPress/contributor-toolkit
Branch: add/release-skill
File: .coderabbit.yaml
Commit: 140a866
Layers: Repository YAML
Review profile: CHILL
Plan: Advanced
Run ID: 0f66493d-c603-48eb-8efe-9c21fe217eb0

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

## Why `v2.0.0-beta.1` shipped on 13 October. Ten commits have reached trunk since then (#667 to #707), and milestone `v2.0.0` has 8 closed issues and none open. This is a stable release: it takes `releases/latest` from `v1.2.1`. `electron-builder` names the artifacts after `package.json`, so the version has to move before the tag and artifacts exist. ## What changes - **The version.** The three root version fields in `package.json` and `package-lock.json` move from `2.0.0-beta.1` to `2.0.0`. No dependency versions change. - **Translations.** `npm run i18n:download` refreshed the seven existing catalogs and added `lo`, `sr` and `tr`. None was removed, and no right-to-left locale was held back. - **Chromium locales.** Since #707 the same script keeps `build.electronLanguages` in line with the shipped catalogs, so it added `sr` and `tr` there. `lo` has no Chromium locale of its own. | Locale | Translated | Strings | | --- | --- | --- | | cs | 94% | 1037 | | en-ca | 95% | 1050 | | en-gb | 100% | 1100 | | gl | 96% | 1066 | | hu | 95% | 1050 | | lo | 95% | 1050 | | nl | 100% | 1100 | | pl | 99% | 1090 | | sr | 100% | 1100 | | tr | 100% | 1100 | ## How to test this **Platforms:** any for steps 1 to 3. Step 4 needs macOS for `Info.plist` and Windows for the `.exe`. Steps 4 and 5 use the Buildkite build for the current head. **Starting state:** this branch checked out with `node_modules` installed. Run every command from the repository root. 1. `npm run lint` is clean and `npm test` passes. 2. `node -e "console.log(require('./package.json').version)"` prints `2.0.0`. 3. `python3 -c "import json;d=json.load(open('package-lock.json'));print(d['version'], d['packages']['']['version'])"` prints `2.0.0 2.0.0`. 4. On the Buildkite build for the current head, the artifacts are named `wordpress-contributor-toolkit-2.0.0-*`. The `app.asar` root holds exactly `node_modules`, `package.json` and `src`, with npm 11 under `node_modules/npm` and no `.codesigning`. On macOS, `CFBundleShortVersionString` reads `2.0.0`. 5. In the installed app, set the language to Turkish in the settings and relaunch. The interface is in Turkish. **What must not have happened:** no dependency version may change. `git diff trunk -- . ':!src/languages'` shows five changed lines: the three root version fields and the two new `electronLanguages` entries. ## Risks and limitations None known. RELEASING.md still says to expect exactly three changed lines outside `src/languages/`. Since #707 a release that adds a locale with a Chromium locale changes `electronLanguages` too. #710 rewrites RELEASING.md and can update that line. ## Related Milestone `v2.0.0`. Release notes follow on the release. --- <details> <summary>Review outcome (required — see AGENTS.md)</summary> 0 [fix here] · 0 [follow-up]. Automatic checks only, as in #681: this change has no logic to review. - **Review:** completed, deterministic layer, in session; head `9df142b` / base `630253f` (trunk). `npm run lint` passes, `npm test` passes (2214 tests, 0 failures, 2 skipped), both files are valid JSON, and the diff outside `src/languages/` is the three root version fields plus the two `electronLanguages` entries. Outcome: nothing to fix. - **Since review:** none. </details> 🤖 Generated with [Claude Code](https://claude.com/claude-code)

- Allow the build.electronLanguages lines in the bump diff, which i18n:download rewrites since #707. - Check the signatures in section 2: codesign, spctl and stapler on macOS, and the Authenticode certificate of the Windows installer. - Say that Buildkite reports nothing to the pull request, so the person confirms the build's commit. - Merge with --match-head-commit, and let the skill merge when the person asks. - Check the docs site's Download button from a new tab, since it keeps the release it read for the tab's session.

…docs deploys

Copilot

🟡 Changes recommended

The Windows signature check does not verify artifact integrity, and publication instructions need repository-selection and path-quoting corrections.

3 open issues

What changed in this PR

Centralizes the Contributor Toolkit’s release procedure in RELEASING.md and adds agent wrappers that require confirmation before public release actions.

Changes:

Adds commands for artifact checks, guarded merging, publication, and post-release verification.
Defines stable-versus-beta handling for release notes, docs deployment, and milestones.
Adds matching release skills and links them from AGENTS.md.

Review outcome: 3 [fix here] · 0 [follow-up]. GitHub Copilot reviewed the supplied four-file diff; reported head 43fdd1b, merge base 630253f. Lint and tests were not run.

File	Description
`RELEASING.md`	Expands release commands and verification steps.
`AGENTS.md`	Links the canonical release procedure and skills.
`.claude/skills/release/SKILL.md`	Adds the release wrapper with confirmation checkpoints.
`.agents/skills/release/SKILL.md`	Provides the equivalent open-standard skill wrapper.

_{Review effort: Balanced}

+The Windows signature can be checked on Windows, with `Get-AuthenticodeSignature <dir>\wordpress-contributor-toolkit-X.Y.Z-win-x64.exe` in PowerShell (`Status` is `Valid` and the signer names Automattic Inc.), or on a Mac with `openssl`, which reads the certificates out of the installer's signature:
+
+```bash
+node -e 'const fs=require("fs");const b=fs.readFileSync(process.argv[1]);const o=b.readUInt32LE(0x3c)+24;const d=o+(b.readUInt16LE(o)===0x20b?112:96)+32;const at=b.readUInt32LE(d),n=b.readUInt32LE(d+4);if(!n)throw new Error("not signed");process.stdout.write(b.subarray(at+8,at+n))' "<dir>/wordpress-contributor-toolkit-X.Y.Z-win-x64.exe" | openssl pkcs7 -inform DER -print_certs -noout | grep subject
+```
+
+It prints a `subject` line naming `O=Automattic Inc.`, and fails with `not signed` when the installer has no signature.


+Then create the tag and the release together, on the bump commit. Every file is named by its path, so the command can run from anywhere:

 ```bash
 gh release create vX.Y.Z \


+  --notes-file <path>/notes.md \
+  <dir>/wordpress-contributor-toolkit-X.Y.Z-win-x64.exe \
+  <dir>/wordpress-contributor-toolkit-X.Y.Z-mac-arm64.dmg \
+  <dir>/wordpress-contributor-toolkit-X.Y.Z-linux-x86_64.AppImage


…ver from a mismatch, and list open pull requests in the milestone

ryanwelcher added 3 commits October 23, 2026 10:05

Add a release skill that walks RELEASING.md with stops before each pu…

a31b25d

…blic step

Move the release commands into RELEASING.md and point AGENTS.md at th…

5fb565e

…e release skill

Put the start checks, milestone and docs-run commands in RELEASING.md…

fb32bc6

…, and order the notes before the release

ryanwelcher mentioned this pull request Oct 23, 2026

Bump version to 2.0.0 #711

Merged

ryanwelcher added 2 commits October 23, 2026 12:05

Guard the release steps against rc versions, a stale branch and beta …

43fdd1b

…docs deploys

ryanwelcher requested a balanced review from Copilot October 23, 2026 20:04

Copilot started reviewing on behalf of ryanwelcher October 23, 2026 20:05

View session

Copilot AI reviewed Oct 23, 2026

View reviewed changes

Run the post-merge check however the bump was merged, say how to reco…

72c3f69

…ver from a mismatch, and list open pull requests in the milestone

ryanwelcher added this to the v2.0.1 milestone Oct 23, 2026

Say what major, minor and patch mean for this app in RELEASING.md

140a866

ryanwelcher self-assigned this Oct 23, 2026

ryanwelcher added the enhancement New feature or request label Oct 23, 2026

Navigation Menu

Repository navigation

Add a release skill, and put every release command in RELEASING.md - #710

Add a release skill, and put every release command in RELEASING.md#710
ryanwelcher wants to merge 7 commits into
trunkfrom
add/release-skill

ryanwelcher commented Oct 23, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Oct 23, 2026 •

edited

Loading

Review skipped

Uh oh!

Copilot AI left a comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

ryanwelcher commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What changes

How to test this

Risks and limitations

Related

Uh oh!

coderabbitai Bot commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ryanwelcher commented Oct 23, 2026 •

edited

Loading

coderabbitai Bot commented Oct 23, 2026 •

edited

Loading