Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/flake-source-build.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@systemfsoftware/claude-code-comment-checker": patch
---

Build the CLI binary directly from source in the Nix flake instead of fetching release assets, ensuring environments using Nix or direnv run the matching version with full command support.
28 changes: 28 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,31 @@ jobs:
uses: ./.github/workflows/mutation.yml
permissions:
contents: read

nix:
name: nix flake check
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: DeterminateSystems/nix-installer-action@e50d5f73bfe71c2dd0aa4218de8f4afa59f8f81d # v16
with:
extra-conf: |
access-tokens = github.com=${{ github.token }}
- name: Flake evaluates
run: nix flake check
- name: Attribute resolves
run: nix eval .#packages.x86_64-linux.comment-checker-bwrap
- name: Build the source-built binary
# nix flake check and nix eval only evaluate; the derivation must be
# built or a broken src filter / compile failure ships green (the
# eval-only gate is precisely how a dropped Cargo.toml slipped past).
run: nix build --no-link .#packages.x86_64-linux.comment-checker-bwrap
- name: Smoke the built binary version
# Smoke the unwrapped binary: the bwrap wrapper needs --unshare-net,
# which GitHub Actions containers do not permit.
run: |
v="$(sed -n 's/^version = "\([^"]*\)"/\1/p' Cargo.toml | head -1)"
nix shell .#comment-checker -c comment-checker --version | grep -qF "$v" \
|| { echo "nix-built binary version mismatch (expected $v)" >&2; exit 1; }
6 changes: 0 additions & 6 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,6 @@ jobs:
permissions:
contents: write
id-token: write
pull-requests: write
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
Expand Down Expand Up @@ -166,8 +165,3 @@ jobs:
run: ./scripts/tools/create-github-release.ts
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Sync flake.nix release asset hashes
run: ./scripts/tools/sync-flake-hashes.ts
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ No eager-read mandates: document reads are situational, triggered by the work, n
- `README.md` — when working in a directory you have not worked in before.
- `ARCHITECTURE.md` — when the task changes a module boundary or data flow.
- Product/requirements docs — when a decision depends on product intent.
- `docs/solutions/` — when implementing or debugging in an area with documented past solutions.
- `CONCEPTS.md` — when orienting to shared domain vocabulary and definitions.

## Working Rules

Expand Down
175 changes: 175 additions & 0 deletions docs/plans/2026-08-30-0433-feat-flake-source-build-plan.md

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -2,62 +2,52 @@
title: flake.nix release-asset hashes froze at the v0.1.5 era, so nix builds served the 0.1.0 binary under the 0.3.2 name
date: 2026-08-29
category: integration-issues
module: flake.nix fetchurl hash block + release pipeline (release-version.ts, check-versions.ts, sync-flake-hashes.ts)
module: flake.nix source build (rustPlatform.buildRustPackage) + CI nix gate; version surface (release-version.ts, check-versions.ts)
problem_type: integration_issue
component: distribution
root_cause: config_error
resolution_type: config_change
resolution_type: design_change
severity: high
symptoms:
- "`nix build .#comment-checker` reports store path 0yxkwm3r...-comment-checker-0.3.2 and `--version` prints 0.1.0"
- "The v0.3.2 GitHub release asset itself is byte-correct; only the flake's declared hash is wrong"
- "A warm store builds the stale derivation in ~0.35s from cache — the intended 0.3.2 bytes are never fetched"
- "Issue #81 reported the released binary is 'the 0.1.0 binary'"
tags: [flake, nix, fixed-output-derivation, fetchurl, sri, release-assets, hash-drift, check-versions, effect]
tags: [flake, nix, fixed-output-derivation, fetchurl, release-assets, hash-drift, source-build]
---

# Nix FOD hash drift served a stale binary under a current version name

## Problem

`nix build .#comment-checker` produced a binary that reported version 0.1.0 even though the flake declared 0.3.2. The GitHub release assets for v0.3.2 were correct; the `flake.nix` `fetchurl` SRI hash block was not. The four hashes were set once in commit `403d2d079` (the v0.1.5 era) and never updated by the release pipeline, which bumps only the `version =` string.
`nix build .#comment-checker` produced a binary that reported version 0.1.0 even though the flake declared 0.3.2. The GitHub release assets for v0.3.2 were correct; the `flake.nix` hash block was not. Consumers of `github:systemfsoftware/comment-checker` silently ran a binary from six releases ago that lacked `--strip`.

## Symptoms

- Store path `0yxkwm3r09fdbvr2pnhpbajiw01dp3i4-comment-checker-0.3.2` serves a 6100-byte-larger 0.1.0-era binary.
- The same build on a warm store completes from cache in ~0.35s (fixed-output derivation cache hit) — no network fetch of the published asset.
- Live asset probes are green (`--version` -> 0.3.2, `--help` lists `--strip`, `--strip` exits 2 with findings); only the declared hashes are stale.
- Consumers of `github:systemfsoftware/comment-checker` silently ran a binary from six releases ago that lacked `--strip`.

## What Didn't Work
## Root cause

- **Bumping the version string as a sync mechanism.** `release-version.ts` rewrote `version =` everywhere, but Nix fixed-output derivations cache by name + declared hash. The `fetchurl` FOD stores by the URL basename (`comment-checker-<triple>`, no version); the version appears only in the derivation output name. A version bump alone can never invalidate or refresh the cached bytes.
- **Re-uploading or re-releasing.** The correct binary was already published; a new tag would churn the npm/cargo/flake version surface for zero behavioral gain.
- **Relying on the rc-only smoke gate.** `run-binary-smoke.ts` asserted exit codes (clean 0, flagged 2) but never the version string — the 0.1.0-era binaries passed it exactly because it never checked what they were.
A fixed-output derivation (fetchurl) caches by URL basename + declared output hash. The released version appears only in the derivation output name, so bumping the `version =` string never touches which bytes the cache serves. The declared hashes were set once, by hand, and nothing recomputed them against the live release; the release pipeline bumps only the version surface. A smoke gate that asserted exit codes — not the binary's own identity — passed the stale binaries precisely because it never checked what they were.

## Solution
## Remedy

Four changes on branch gh-81 (pending PR):
The flake builds the binary from the repo's own source with `rustPlatform.buildRustPackage`, using the toolchain pinned in `rust-toolchain.toml` via `makeRustPlatform`. The derivation copies `Cargo.toml`, `Cargo.lock`, `crates/`, and `.cargo/` from the current tree (`cleanSourceWith`), so it always carries the current code; `doCheck = false` keeps the crate's tests in CI, not in the derivation.

1. **Refresh the flake hash block to the live v0.3.2 SRIs** (`flake.nix`): the four `sha256-...` values now equal the bytes at `releases/download/v0.3.2/comment-checker-<triple>` (measured this session from the live assets).
2. **Publish-phase auto-sync** (`scripts/tools/sync-flake-hashes.ts` + `release.yml`): after `create-github-release.ts` extracts the binaries, the step hashes them, rewrites the flake block, round-trip re-reads the file (CHK1), and opens an auto-merged `fix/flake-hashes-v<X>` PR. A retried publish reuses the open PR (edit body) instead of failing on a duplicate create. The publish job gains `pull-requests: write`.
3. **Recomputing gate** (`scripts/lib/version-files.ts` + `check-versions.ts`): for the tag the flake declares, downloads each asset via `gh release download` (2 retries with fail-closed outcome), recomputes the SRI, and fails with `FlakeHashMismatch` naming the stale triple. Absent tags skip only when the declared version is newer than every published release, else `FlakeTagMissing`; an unreachable remote is `FlakeRemoteUnreachable` — never a silent skip. All services (`ChildProcessSpawner`, `Crypto`, `FileSystem`) are Effect services yielded in-effect via `DenoServices.layer`.
4. **Version assertion in the smoke gate** (`scripts/tools/run-binary-smoke.ts`): parses `--version` output and requires it to equal the workspace `Cargo.toml` version, with a strict semver guard.
There is no fetch of the released binary and no binary hash to go stale. The only `fetchurl` left is the tree-sitter-language-pack parser-sources bundle, delivered to the sandboxed build as a hash-pinned dependency (`TSLP_SOURCE_BUNDLE_URL=file://...`): nix verifies that hash, drift fails the build loudly, and the bundle version must track the crate version in Cargo.lock — a dependency pin, not an identity mechanism.

Shared helpers live in `scripts/lib/shared.ts`: `sriFromSha256` (SRI encoding) and `unixTargetTriples` (the flake block's four keys, derived from `targets.json` excluding win32) — one source of truth consumed by both sync and gate.

## Why This Works

Nix keys a fixed-output derivation on name + declared output hash. When the declared hash matches the published bytes, the FOD fetches the correct asset; when it doesn't, warm stores replay whatever bytes the stale hash previously certified. The fix makes the declared hash *be* the live asset hash (U1), makes publishing write the hashes automatically (U2), makes CI recompute and compare them from the live release (U3), and makes the smoke gate assert the binary's own identity (U4). The gate never trusts `version =` alone — it recomputes from fetched bytes (CHK1 discipline: a self-reported field certifies nothing).
The `version = "0.3.2"` binding stays so the version-sync surface keeps npm, cargo, and the flake in lockstep. CI runs a nix gate that evaluates the flake, builds the `comment-checker-bwrap` derivation, and asserts the built binary's `--version` equals the workspace `Cargo.toml` version.

## Prevention

- Never let release automation bump only the `version` string inside a flake: the hash block is an output of the release, not a constant. Sync it in the publish phase and verify with a recomputing gate.
- Any gate that stands in for a real invariant (here: "the nix build serves the version it claims") must exercise the same assets and inputs the real consumer uses — the gate downloads the same `comment-checker-<triple>` assets for the same `v<version>` tag the flake fetches.
- A smoke gate for a released binary asserts identity, not just exit codes — assert `--version` against a single source of truth (the workspace manifest).
- Remote-unreachable and absent-tag states must fail loudly, not skip: a gate that silently skips on its own infrastructure failing is a permanent blind spot in exactly the broken-pipeline state it exists to catch.
- Do not hand-pin a released binary's hash in the flake. When the flake's repo is the release source, build from source — there is no hash to sync.
- A smoke gate asserts identity, not just exit codes: the built binary's `--version` must equal the workspace manifest, the single source of truth.
- A gate that guards a distribution surface must build what it guards. Evaluation-only checks (`nix flake check`, `nix eval`) pass a flake whose derivation cannot build.
- Hash-pinned dependency bundles are safe where nix verifies the hash and a mismatch is a hard build error. The failure mode to avoid is an unverified hash that silently serves old bytes.

## Related Issues

- Fixes https://github.com/systemfsoftware/comment-checker/issues/81
- docs/solutions/architecture-patterns/rust-cli-npm-distribution.md — release automation and the version-surface discipline this flake hash block participates in.
- Related: https://github.com/systemfsoftware/comment-checker/issues/81
- docs/solutions/architecture-patterns/rust-cli-npm-distribution.md — release automation and the version-surface discipline.
53 changes: 32 additions & 21 deletions flake.nix
Original file line number Diff line number Diff line change
Expand Up @@ -22,31 +22,42 @@
};
in f pkgs);
version = "0.3.2";
# Source build: no fetchurl of the released *binary*, so no binary hash
# to go stale (that fixed-output caching was the #81 failure).
#
# The toolchain pin is the repo's own: rust-toolchain.toml, from rust-overlay.
mkCommentChecker = pkgs:
let
target = {
"x86_64-linux" = "x86_64-unknown-linux-gnu";
"aarch64-linux" = "aarch64-unknown-linux-gnu";
"x86_64-darwin" = "x86_64-apple-darwin";
"aarch64-darwin" = "aarch64-apple-darwin";
}.${pkgs.system} or (throw "unsupported system ${pkgs.system}");
hash = {
"x86_64-unknown-linux-gnu" = "sha256-ZWq3FcS8ILUCVNrC8u4jHCltywXNNuZAluF8IOK5BnI=";
"aarch64-unknown-linux-gnu" = "sha256-NxMnMpTcXQ/A1D2cZRbXDfYJEL+bqqJQphwWM3YZ7+M=";
"x86_64-apple-darwin" = "sha256-8RGZ/X6hFJe0sEHnD0OCJ7BF1iQX6DrGt0qAwYs78ho=";
"aarch64-apple-darwin" = "sha256-R62Dc6QVr64K5N/HPIDkfG6hXx9JUy/o43451wzpbNw=";
}.${target};
src = pkgs.fetchurl {
url = "https://github.com/systemfsoftware/comment-checker/releases/download/v${version}/comment-checker-${target}";
inherit hash;
toolchain = pkgs.rust-bin.fromRustupToolchainFile ./rust-toolchain.toml;
rustPlatform = pkgs.makeRustPlatform { cargo = toolchain; rustc = toolchain; };
# tree-sitter-language-pack's build.rs downloads a parser-sources
# tarball at compile time; the nix sandbox has no network, so the
# bundle rides in as a hash-pinned fetchurl (like Cargo.lock —
# drift fails the build loudly). Keep the URL version equal to the
# tree-sitter-language-pack version in Cargo.lock, and the sha256
# equal to this tarball's real hash. TSLP_SOURCE_BUNDLE_URL also
# accepts file://, which is how the sandboxed build reads it.
tslpParserSources = pkgs.fetchurl {
url = "https://github.com/xberg-io/tree-sitter-language-pack/releases/download/v1.14.3/parser-sources-1.14.3.tar.zst";
sha256 = "f13bcf0be1b9a99c45ead43671ae311346c2a10807c3fdc7c76407445ea7f2c7";
};
in pkgs.stdenv.mkDerivation {
in rustPlatform.buildRustPackage {
pname = "comment-checker";
inherit version src;
dontUnpack = true;
installPhase = ''
install -Dm755 $src $out/bin/comment-checker
'';
inherit version;
src = nixpkgs.lib.cleanSourceWith {
src = ./.;
filter = path: type:
(type == "directory") ||
(builtins.elem (baseNameOf path) [ "Cargo.toml" "Cargo.lock" ]) ||
(builtins.match ".*/.cargo/.*" path != null) ||
(builtins.match ".*/crates/.*" path != null);
};
cargoLock.lockFile = ./Cargo.lock;
# The repo's quality gates (cargo test, mutation) run in CI, not in
# this derivation; doCheck defaults to true in buildRustPackage and
# would run the whole suite inside the nix sandbox.
doCheck = false;
TSLP_SOURCE_BUNDLE_URL = "file://${tslpParserSources}";
meta = with pkgs.lib; {
description = "Claude Code PostToolUse hook that flags unnecessary comments";
homepage = "https://github.com/systemfsoftware/comment-checker";
Expand Down
7 changes: 0 additions & 7 deletions scripts/lib/shared.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,3 @@ export interface LauncherManifest {
repository: { type: string; url: string }
optionalDependencies?: Record<string, string>
}
/** SRI encoding of a SHA-256 digest (Nix `fetchurl` hash format). */
export const sriFromSha256 = (digest: Uint8Array): string =>
`sha256-${btoa(String.fromCharCode(...digest))}`

/** Flake hash-block keys: every target except the win32 row in targets.json. */
export const unixTargetTriples = (targets: Target[]): string[] =>
targets.filter((t) => t.os !== 'win32').map((t) => t.target)
Loading
Loading