Skip to content
Merged
1 change: 1 addition & 0 deletions .devflow/features/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@
- **mds-lint** — crates/mds-core/src/lint, crates/mds-cli/src, crates/mds-wasm/src, crates/mds-napi/src, crates/mds-python/src, packages/mds/src — Use when adding or modifying lint rules, extending the --fix pipeline, changing the JSON wire format, wiring lint into a binding layer, debugging unexpected exit codes and reverify gate refusals, or working on the ESC/bidi/newline injection defences. Keywords: mds lint, LintDiagnostic, fix_removals, fix_edits, TextEdit, FixLineSpan, diag_to_edits, LintResult, LintConfig, to_canonical_json, fix tier, reverify gate, FixOutcome, PartiallyFixed, apply_fixes_incremental, preview_fixes, PreviewOutcome, set_diag_display_path, AnalysisContext, ElseifBranch, end_offset, DefineFact, assertKnownKeys, CheckOptions, unreachable-branch, unused-variable, duplicate-import, empty-block, legacy-interpolation, is_output_neutral, all_output_neutral, Tier A Tier B Tier C, structural-standalone, compile-clean, is_standalone, sanitize_control_chars, sanitize_control_chars_wire, named_source_for_render, neutralize_source_for_render, SanitizedReport, SanitizedNode, MAX_AUX_DEPTH, EscapeMode, HUMAN WIRE, eprint_warning, safe_path, safe_inline, safe_file_display, preview_text_for, print_discipline, reverify_failure_reason, LintDirCtx, config_cache, dedup_contained_or_identical, EXIT 0 1 2 3, render_error_sanitized, eprint_error, display_sanitized, MdsError::display_sanitized, ESC-injection, CWE-150, CWE-117, bidi, Trojan-Source, CVE-2021-42574, U+061C, U+202E, U+FEFF, U+2028, U+2029, PF-014, PF-005, construction-time sanitization, per-field rule, Cow, #176, ADR-008, ResultSink, from_rules_checked, relative_display, write_bytes, PF-020, #309, emit-ordering.
- **source-map-security** — crates/mds-core/src, crates/mds-cli/src, packages/mds/src — Use when working with Source Map v3 generation, sources[] path relativization, the relativize_source choke-point, FileSystem::source_root(), CompileOptions.source_map_base, cross-surface source-map parity tests, or the Windows verbatim UNC path fix. Keywords: source map, sources[], relativize_source, source_map_base, source_root, path containment, basename fallback, PF-005, ADR-005, SEC-3, Windows verbatim UNC, path_to_unified, compute_source_map_base, apply_source_map_file_label, CF-SM2, V-SM1, differential test, two-level anchoring, map-relative, root-relative.
- **mds-js** — packages/mds/src, packages/mds/__test__ — Use when modifying the JS/TS public API surface, adding backend methods, changing option types, debugging basePath rejection behaviour, changing result types, updating the backend contract, working on WASM/native backend validation, or debugging why a backend result is rejected. Keywords: compileFile, compile, check, checkFile, lint, lintFile, lintVirtual, CompileResult, MarkdownResult, MessagesResult, CheckResult, LintResult, LintDiagnostic, LintFileOptions, CompileFileOptions, FileOptions, assertResultShape, validateBackendMethods, METHOD_KEYS, forwardOpts, assertKnownKeys, getBasePathError, BASEPATH_REJECTORS, BASE_METHODS, NODE_METHODS, WASM_EXPORTS, discriminated union, kind, mds::invalid_backend_result, mds::invalid_options, basePath, synchronous throw, native.ts, wasm.ts, contract.ts, types.ts, node.ts, browser.ts, options.ts.
- **release-pipeline** — .github/workflows, .github/actions, scripts, scripts/__test__ — Use when modifying release.yml, adding CI jobs, updating TIER_B_EXPECTED_SKIPPED, adjusting the pull_request surface trigger, debugging a publish failure, running the pre-merge verifier, or reasoning about the publish job ordering. Keywords: release, release.yml, verify-pr-checks, TIER_B_EXPECTED_SKIPPED, RELEASE_SURFACE, rehearse-publish-python, tag-push, TestPyPI, publish-crates, publish-npm, publish-python, github-release, version-gate, stage-and-verify-napi, ADR-013, PF-040.
361 changes: 361 additions & 0 deletions .devflow/features/release-pipeline/KNOWLEDGE.md

Large diffs are not rendered by default.

23 changes: 20 additions & 3 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -85,9 +85,8 @@ jobs:
echo "::error::NPM_TOKEN is empty or unset — npm publish would fail AFTER cargo publish (irreversible). Set the NPM_TOKEN repo secret. (security-08)"
exit 1
fi
# Guard: cargo token must be non-empty. A missing token fails before the
# irreversible crates.io publish, but catching it here avoids burning the
# expensive 7-target cross-compile matrix.
# Guard: cargo token non-empty. That is the strongest check crates.io allows for an
# API token (no read-only endpoint accepts a scoped token; #345, RELEASING.md "credential probe").
if [ -z "$CARGO_REG_TOKEN" ]; then
echo "::error::CARGO_REGISTRY_TOKEN is empty or unset — cargo publish would fail. Set the CARGO_REGISTRY_TOKEN repo secret. (security-08)"
exit 1
Expand Down Expand Up @@ -350,7 +349,18 @@ jobs:
- uses: dtolnay/rust-toolchain@stable
with:
targets: ${{ matrix.settings.target }}
# Per-leg cache key (PF-041, #352). The automatic key is job id + runner
# os/arch + a hash of the rustc HOST triple, env and lockfiles; the cross
# target is not part of it, so without `key:` all four ubuntu legs and both
# macOS legs restore ONE blob (run 34065573775: every Linux leg restored
# v0-rust-build-napi-Linux-x64-...). Keying on the target keeps each leg's
# target/<triple>/ artifacts and host-built build scripts to itself. The
# host is already encoded (runner.os/arch + rustc host hash), so only the
# target is added. Intentional; must survive rust-cache version bumps --
# spec S20 in scripts/__test__/release-auth-probe.spec.mjs pins it.
- uses: Swatinem/rust-cache@v2
with:
key: ${{ matrix.settings.target }}
- name: Install zig
if: matrix.settings.use-zig
uses: mlugg/setup-zig@v2
Expand Down Expand Up @@ -516,6 +526,11 @@ jobs:
if: ${{ !matrix.sdist && runner.os != 'Linux' }}
with:
targets: ${{ matrix.target }}
# Per-leg cache key: target + manylinux flavour so a containerised Linux
# leg (glibc 2.17 manylinux / musl) never restores host-built build scripts
# or proc-macro .so files written by another leg; legs without manylinux
# render a trailing dash (harmless, validated warm in #347). Intentional;
# spec S20 in scripts/__test__/release-auth-probe.spec.mjs pins it.
- uses: Swatinem/rust-cache@v2
if: ${{ !matrix.sdist }}
with:
Expand Down Expand Up @@ -984,6 +999,7 @@ jobs:
with:
ref: ${{ github.ref }}
- uses: dtolnay/rust-toolchain@stable
# Single-leg job: the automatic job-id key is already unique; no per-leg key (#352).
- uses: Swatinem/rust-cache@v2
- name: Verify tag matches workspace version
run: |
Expand Down Expand Up @@ -1079,6 +1095,7 @@ jobs:
# update here to keep both in sync.
- uses: dtolnay/rust-toolchain@01ba1edad32c6f80dbcce879d3e0fa5a00b2a84e # 1.96.0
with: { targets: wasm32-unknown-unknown }
# Single-leg job: the automatic job-id key is already unique; no per-leg key (#352).
- uses: Swatinem/rust-cache@v2
- uses: ./.github/actions/setup-wasm
- run: npm ci
Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- npm dependency sweep: relaxed the three phantom floor pins to caret ranges — fast-uri 3.1.5 → ^3.1.6 (oldest release patching GHSA-5jgf-p345-68v8, GHSA-fph4-wmhf-6fwf, GHSA-f65p-4m7j-42xc, GHSA-jqff-g426-hqxp), nanoid 3.3.18 → ^3.3.18, js-yaml 4.3.1 → ^4.3.1 (#336); @napi-rs/cli ^3.0.0 → ^3.8.6 (lock 3.7.0 → 3.8.6); vite lock 8.1.5 → 8.2.2; Dependabot `ignore` rules for semver-major bumps of the three phantom pins. Supersedes Dependabot #315 #332 #346 #362 #355 #357 #279.
- GitHub Actions sweep: actions/checkout v6 → v7 (16 call sites: 9 ci.yml + 7 release.yml), actions/setup-node v6 → v7 (6 sites), actions/setup-python v5 → v7 (5 sites, ci.yml only; action runtime node20 → node24), PyO3/maturin-action pin normalized from the v1.51.0 annotated-tag object (`3e2bdf6`) to the commit it points to (`e83996d1`), same version (PF-040); Dependabot `ignore` for typescript semver-major version updates pending the TS 7 migration (#364). Supersedes Dependabot #111, #189, #241, #356; replaces #169.
- Release-surface PR gate: `release.yml` now triggers on `pull_request` events touching `.github/workflows/release.yml`, `.github/actions/**`, `crates/mds-napi/**`, `crates/mds-python/**`, and `scripts/verify-napi-names.mjs`; adds an unguarded `rehearse-publish-python` job (PF-039) that rehearses `publish-python` without uploading — pin shape (both the annotated-tag-object SHA a892a5a6 and the commit SHA dc37677b are rejected as positive controls), GHCR manifest (404 + `MANIFEST_UNKNOWN` body required), bounded `docker pull` (3-attempt loop), and `twine check` from the pinned image with `--network none`, each with a positive control (PF-013, PF-040) — plus a `publish-testpypi` opt-in leg (dispatch-guarded); the rehearsal never invokes `pypa/gh-action-pypi-publish` (the action has no dry-run mode) and is denied `id-token`, both pinned by specs S14/S15; credential and OIDC probes run on `pull_request` events and fail closed on fork/Dependabot PRs (no secrets, no `id-token: write`) with an actionable error; `verify-pr-checks.mjs` requires `Version gate`, `Stage + verify platform packages` and `Rehearse PyPI publish (no upload)` on release-surface PRs and fails closed when the changed-file list cannot be read (#342, #350).
- `verify-pr-checks.mjs` suite keying (D-PR8): the skipped-publish allowance and D-PR7 context attribution are now keyed on the check-run's `check_suite.id` mapping to a `release.yml` workflow run (one bounded `GET /actions/runs?head_sha=` call); the verifier exits 2 when it cannot enumerate the head's workflow runs; 2026-09 branch-protection and check fixtures added (`checks-pr366-e02bcf2.json`, `runs-pr366-e02bcf2.json`, `protection-main-2026-09.json`) (#341).
- `build-napi` per-leg rust-cache key: adds `key: ${{ matrix.settings.target }}` to the `Swatinem/rust-cache` step so each cross-compile leg's target artifacts stay isolated (PF-041; without the key all four ubuntu legs and both macOS legs restored one shared blob, confirmed live in run 34065573775); `build-python`'s existing `key: matrix.target-matrix.manylinux` (#347) unchanged; spec S20 in `release-auth-probe.spec.mjs` pins both and fails `Version gate` if a key is dropped; spec S3 extended to pin the `-z` CARGO_REG_TOKEN guard in executable code; #345 verified that crates.io `GET /api/v1/me` is `AuthCheck::only_cookie()` (HTTP 403 for any API token) and the only token-accepting read route rejects scoped tokens — non-empty guard is the strongest check available, durable fix tracked in #368; #345 closed won't-fix-as-filed (#345 #352).

## [0.4.2] — 2026-09-03

Expand Down
74 changes: 54 additions & 20 deletions RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,10 @@ These are **not** automated and must be done before the first release:
Provenance requires the `id-token: write` permission (already set on the
publish job) plus publishing from GitHub Actions.
3. **Add the `CARGO_REGISTRY_TOKEN` repo secret** with publish rights to
`mds-core` and `mds-cli` on crates.io.
`mds-core` and `mds-cli` on crates.io. The workflow cannot probe this token
(crates.io has no read-only endpoint that accepts a scoped token — see the
credential probe under Pre-flight); verify it in the crates.io UI before
tagging (PF-023).
4. **Enable GitHub private vulnerability reporting** (Settings → Code security →
Private vulnerability reporting) so the SECURITY.md flow works.
5. **Configure PyPI trusted publisher** for `markdown-script` at
Expand Down Expand Up @@ -158,23 +161,44 @@ gh workflow run release.yml # workflow_dispatch — builds the 7-target
# step. Publishes NOTHING.
```

The dry-run workflow runs `version-gate` in full, which now includes the
**credential probe** (security-08): it calls `npm whoami` against the live
registry to verify the `NPM_TOKEN` is valid, guards `CARGO_REGISTRY_TOKEN`
for non-empty, and probes the PyPI trusted publisher via the OIDC mint-token
exchange. A revoked token, absent secret, or misconfigured trusted publisher
therefore fails the dry run — all before any irreversible crates.io release.

Both probes **run on every event, including `pull_request`**. On fork and
Dependabot PRs — which receive no repository secrets and no `id-token:
write` — the probes fail closed with an actionable error: maintainers must
supersede with a first-party branch PR or dispatch `gh workflow run
release.yml --ref <branch>`. No PR run can reach a publish in any case,
so the fail-closed behaviour is informational, not a merge blocker by itself.

**Note:** `npm whoami` verifies authentication, not publish rights to the
`@mdscript` scope. A read-only or wrongly-scoped token passes the probe but
fails at publish time.
The dry-run workflow runs `version-gate` in full, which includes the
**credential probe** (security-08). What each registry check proves:

- **npm** (`npm whoami`): proves the token is accepted by the npm registry
(authentication). Does NOT prove publish rights to the `@mdscript` scope —
a read-only or wrongly-scoped token passes this check but fails at publish time.
- **PyPI**: the OIDC mint-token exchange (`pypi.org/_/oidc/mint-token`) proves the
trusted-publisher record matches the workflow. The minted token expires unused —
the probe is free and safe.
- **crates.io**: non-empty guard only. This is the strongest check the crates.io
API allows for an API token: `GET /api/v1/me` is `AuthCheck::only_cookie()`
(`src/controllers/user/me.rs:38-41`) and returns HTTP 403 for any token, scoped
or unscoped (`src/auth.rs:136-144`). The only token-accepting read endpoint,
`GET /api/v1/me/tokens/{id}` (`src/controllers/token.rs:269-282`), accepts
legacy unscoped tokens only — a scoped token (the least-privilege kind a publish
secret should be) is rejected there with HTTP 403 "this token does not have the
required permissions". A well-formed but
revoked/deleted token gets HTTP 403 "authentication failed" (`src/auth.rs:297-303`);
a malformed token gets HTTP 401 "The given API token does not match the format
used by crates.io" (`src/auth.rs:295`, `InsecurelyGeneratedTokenRevoked`).
Every crates.io API request must also carry a `User-Agent` header — without one
the `require_user_agent` middleware (`src/middleware/require_user_agent.rs:35-47`)
returns HTTP 403 with a plaintext body before authentication runs, so a manual
probe without that header fails for an unrelated reason. In
practice this means a revoked crates.io token is first detected at the first
`cargo publish` (fail-before-write, after the build matrix has been paid for).
The v0.4.0 release experienced exactly this (run 33569514359 attempt 1 failed
at `Publish mds-core` with HTTP 403, nothing published, `gh run rerun --failed`
completed it; see PF-023). A durable fix — Trusted Publishing for crates.io —
is tracked in #368; #345 is closed as won't-fix-as-filed with this finding.

All three checks **run on every event, including `pull_request`**. On fork and
Dependabot PRs — which receive no repository secrets and no `id-token: write` —
they fail closed with an actionable error: maintainers must supersede with a
first-party branch PR or dispatch `gh workflow run release.yml --ref <branch>`.
On release-surface PRs a fail-closed `Version gate` **blocks the merge** because
D-PR7 requires it (by design, not advisory) — supersede with a first-party branch
PR or dispatch by hand.

Even though release-surface PRs now trigger `release.yml` automatically, a
manual `gh workflow run release.yml --ref <branch>` is still required in four
Expand Down Expand Up @@ -245,7 +269,12 @@ Five jobs are expected-skipped on a standard `workflow_dispatch` dry run and
are listed in `TIER_B_EXPECTED_SKIPPED` in `scripts/verify-pr-checks.mjs`:
`Publish to crates.io`, `Publish to npm`, `Publish to PyPI`, `GitHub Release`,
and `Publish to TestPyPI (rehearsal)`. The same five are skipped on a
release-surface PR run.
release-surface PR run. The skipped tolerance applies only to check-runs whose
`check_suite.id` maps (via `GET /actions/runs?head_sha=`) to a `release.yml`
check suite (any event); the three D-PR7 contexts (`Version gate`,
`Stage + verify platform packages`, `Rehearse PyPI publish (no upload)`) are
attributed by the same suite. The verifier exits 2 when it cannot enumerate the
head's workflow runs (D-PR8, #341).

Confirm the **A3 name-gate** step (`scripts/verify-napi-names.mjs`) passes in that
run. **This is a hard checkpoint** — if the generated platform package names or
Expand All @@ -272,7 +301,11 @@ That path list lives in **two** places that must stay identical: the
`scripts/verify-pr-checks.mjs`. Spec S10 compares them as sets — a filter the
verifier does not know about would let a release-surface PR pass as a silent
no-run (ADR-013 amendment). The verifier also fails closed (exit 2) if it
cannot enumerate the PR's changed files at all.
cannot enumerate the PR's changed files at all. The skipped-publish allowance
and D-PR7 context attribution are both keyed on the check-run's `release.yml`
check suite (D-PR8, #341 — any release.yml event counts, including `pull_request`
and `workflow_dispatch`); the verifier exits 2 when it cannot enumerate the
head's workflow runs.

## Release

Expand Down Expand Up @@ -352,3 +385,4 @@ The `release.yml` workflow runs, in order:
- wasm-opt = ["-Oz", "--enable-bulk-memory", "--enable-sign-ext", ...] is enabled in crates/mds-wasm/Cargo.toml; CI installs wasm-pack and Binaryen v129 via the composite action at .github/actions/setup-wasm/ (version pins live there). Local builds do not need system Binaryen — wasm-pack auto-downloads wasm-opt (v117) on first use; install Binaryen v129+ (brew install binaryen / apt install binaryen) only for offline builds, to override a stale wasm-opt on PATH, or to reproduce CI's exact release optimizer.
- Platform packages are generated in CI only — they cannot be validated with a local npm pack; use the dry-run workflow instead.
- Due to its temp-file-then-rename implementation, atomic_write_file does not preserve hard links, ACLs, extended attributes (xattrs), or owner/group metadata of the original file.
- `Swatinem/rust-cache` computes its key as `v0-rust[-<key>]-<job>-<runner.os>-<runner.arch>-<envhash>-<lockhash>`; the env hash covers `rustc -vV` (the HOST triple) plus `CARGO*`/`CC*`/`CFLAGS` env. The cross TARGET is not in it, so without `key:` all four ubuntu legs in `build-napi` and both macOS legs share one blob. Both matrix jobs carry per-leg keys: `build-napi` uses `key: ${{ matrix.settings.target }}` (#352, PF-041); `build-python` uses `key: ${{ matrix.target }}-${{ matrix.manylinux }}` (#347) so a containerised Linux leg never restores host-built build scripts or proc-macro `.so` files written by another leg. `publish-crates` and `publish-npm` are single-leg and use the automatic key. Spec S20 in `scripts/__test__/release-auth-probe.spec.mjs` pins this — dropping `with: key:` from a matrix job's rust-cache step causes S20 to fail `Version gate`. Validation: two runs in the SAME cache scope — first run shows `No cache found.`; second shows `Restored from cache key "v0-rust-<target>-build-napi-…" full match: true.` for every leg with the readelf gates green. `pull_request` caches live under `refs/pull/N/merge` and are invisible to a branch dispatch, so warm evidence comes from a second dispatch on the same branch (or a PR-run rerun), never from a dispatch that follows a PR run.
1 change: 1 addition & 0 deletions scripts/__test__/fixtures/checks-pr366-e02bcf2.json

Large diffs are not rendered by default.

Loading
Loading