v0.33.0 (in progress): SDKs, payload identity, post-deposit corrections, and a legible rotation - #137
Draft
avrabe wants to merge 24 commits into
Draft
v0.33.0 (in progress): SDKs, payload identity, post-deposit corrections, and a legible rotation#137avrabe wants to merge 24 commits into
avrabe wants to merge 24 commits into
Conversation
REQ-SDK-001 is verified and the CONSUMER half genuinely works — `varve export-sdk` unpacks a signed tree, relocates it the way Yocto's own relocate_sdk.py does, and refuses a destination whose path is longer than the interpreter field can hold. What does not exist is any way to PUT one in. The producer implements three layouts, all of which mine ONE BINARY out of an archive and discard the rest, which is the opposite of what an SDK is. So `sdk` was never "a name only". It is a half. Four requirements, each measured against a real upstream on 2026-09-06 rather than assumed: REQ-SDKDEPOSIT-001 — a layout where the archive IS the payload. Stored exactly as upstream published it, because relocation is an export concern and a producer that unpacked and re-packed would break the digest upstream's own sums cover. No architecture check on a tree: an SDK holds executables for several architectures and checking its first ELF proves nothing. REQ-ARCHIVEFMT-001 — .tar.xz and .tar.bz2. One missing branch in one match statement is the whole blocker for two separate things: wasmtime ships .tar.xz, so the bytecodealliance realm can carry wasm-tools and wit-bindgen but would silently omit the runtime; and all 140 Zephyr SDK toolchains are .tar.xz. The shell that actually deposits is worse — it hardcodes `tar xzf`. The refusal for genuinely unknown formats stays, and a self-extracting installer is explicitly NOT to be executed: running a vendor's installer to find out what it contains defeats the property the deposit exists to establish. REQ-SDKTARGET-001 — a payload may name a TARGET, not only the host it runs on. varve files payloads by the machine that runs them, which is right for a compiler you invoke and wrong for a cross-toolchain, identified by a pair. sdk-ng ships 140 toolchain_gnu_<host>_<target> assets; a realm wants a few targets across four hosts. Without this the four arm-zephyr-eabi assets get four hand-invented names and nothing connects them. REQ-UPSTREAMSUMS-001 — the ladder's three rungs have never had to distinguish "unsigned upstream sums" from "nothing", because every pulseengine payload clears rung 1. The SDKs land in three different places: sdk-ng publishes a sha256.sum and no signature; wasi-sdk publishes nothing at all and has no build provenance either (verified: attestation exits 1); wasm-tools and wasmtime have no sums but DO carry provenance that verifies. Calling Zephyr's manifest "unverified" discards a real property — it catches the truncated 14GB download, which is the failure that transfer actually has. Calling it a signature would be a lie, since the same host serves the manifest and the bytes. So it gets its own name, and is refused wherever a realm demands signed provenance. REQ-REUSEBLOB-001 moves here too, still blocked on the clause-6 vs artifact-of-record decision it surfaced.
Six months was the first number written into the policy and it was wrong in the direction that matters. The rolling line ships MONTHLY, so a layer is superseded roughly a month after it is issued, and nobody has undertaken to publish advisories or fixes for a superseded rolling layer. Six months promised five months of attention no one had agreed to give. That is the same class of untruth this requirement was written to remove. The original defect was a layer saying "no stated support window" while the docs promised one. Stating a window longer than the commitment behind it is the same mistake pointing the other way, and it is worse in one respect: a consumer can act on it. "No window" makes someone ask; "supported until March" makes them stop asking. `qualified` keeps 24 months. That is the channel an assessor is pointed at and the one whose contents are deliberately frozen, so a long horizon there is a commitment somebody can actually keep. The month arithmetic already handled the hard cases and the tests move with the policy rather than being deleted: 31 Dec + 1 = 31 Jan (a 31-day target month keeps its day), 31 Mar + 1 = 30 Apr (a shorter one clamps), 31 Jan + 1 = 28 or 29 Feb depending on the Gregorian rule, checked at 2024, 2026, 2100 and 2000. LIVE CONSEQUENCE, not yet fixed: layer 2026.09.1 was published an hour ago carrying support-until 2027-03-06. That promise is now overstated by five months. It is correctable without touching the layer — the window lives in the line-status, which is a DSSE-signed OCI referrer designed to be reissued after deposit — but there is no workflow that issues a status update; deposit-layer is the only thing that signs one. Tracked separately.
Checking an assumption before building on it found that the assumption was false, and the false one was mine, asserted an hour earlier in varve#131. I said a support window is correctable without touching the layer, because REQ-KP-001 describes the line-status as an OCI referrer "added after deposit without changing the layer digest". The registry path does not implement that. `line_status_for_tag` resolves the artifact manifest for THE LAYER'S OWN TAG and pulls the blob whose role is line-status, so the status lives inside the layer's manifest. Reissuing it re-pushes that manifest under a new digest — the exact republish REQ-IMMUTABLE-001 was written to refuse, arrived at by a well-intentioned route. `grep -c referrers registry.rs` is 0. So the gap is not a missing workflow. The MECHANISM is missing, and it is missing for all three things that are the same act: a yank, an advisory, and a correction are each a statement about a layer after it is published, and none of them can currently reach a registry consumer. The fix is already in the codebase, one document over. `fetch_line_index` reads from a SEPARATE tag — line-index-<line> — independent of any layer, so reissuing touches no layer digest. That is the shape line-status needs. Except nothing publishes a line-index either. No line-index-* tag exists in the registry and deposit-layer.yml never creates one, while REQ-INDEXAUTH-001 is verified with a tested fetch path and the realm does not set signed-index, so the absence fails open and nobody notices. That is the third instance of one pattern in this release. support-until shipped verified in v0.5.0 and was never set. min-counter had a consumer and no producer. The line-index has a fetch path and no publisher. Each is a complete, tested, verified capability that no artifact carries — and each reads, from inside the code, exactly like a capability that works. REQ-KP-001 and REQ-INDEXAUTH-001 both carry a field recording this, so neither can be read as discharging what it describes. Deliberately NOT built: the issue-line-status workflow I proposed and the user approved. As described it would republish a layer manifest to fix a date, which is what the previous release exists to prevent. Approval was for an outcome, not for a mechanism that turned out to be the thing we just outlawed. 2026.09.1 continues to overstate its window at 2027-03-06 against a one-month policy. That is the honest position: there is currently no way to correct it short of depositing a new layer, and depositing a layer to fix a date is the waste the referrer design was supposed to avoid.
…tion it depends on
One missing branch in one match statement was the whole blocker for two separate pieces of work. bytecodealliance/wasmtime ships .tar.xz, so a second realm would have carried wasm-tools and wit-bindgen and silently omitted the runtime — the layer would have looked complete. All 140 toolchain_gnu_<host>_<target> assets of the Zephyr SDK are .tar.xz, so the sdk kind had nothing it could actually fetch. kiln's VxWorks SDK is .tar.bz2. Both halves, because the shell is what actually deposits today and it hardcoded `tar xzf` — a .tar.xz would have failed there with a tar error rather than varve's refusal, which is a worse way to learn the same thing. The compression comes from the asset NAME, never from letting `tar xf` sniff it. An archive whose contents disagree with its name is a case worth failing on rather than accommodating, and that is the same reason choose_binary refuses ambiguity instead of picking. Self-extracting installers stay refused, and the refusal now SAYS so rather than leaving it to be inferred from a missing branch. A Yocto SDK most often ships as a .sh, and running a vendor's installer to discover what it contains defeats the property the deposit exists to establish. Someone will want to add that branch; the error message argues against it in advance. Verified against a live upstream rather than a fixture: wasmtime v48.0.1-aarch64-macos.tar.xz downloads, unpacks with the emitted flags, and the binary inside runs and reports wasmtime 48.0.1. 25 mutants, zero survivors.
… can open it `sdk` was never "a name only" — REQ-SDK-001 is verified and `varve export-sdk` genuinely unpacks, relocates and refuses a destination that will not fit. It was a HALF: nothing could put an sdk in. The three producer layouts all mine one binary out of an archive and discard the rest, which is the opposite of what an SDK is. The tree is now stored exactly as upstream published it. Not unpacked and re-packed, because that breaks the digest upstream's own sums cover; relocation belongs on the consumer's machine where the destination is known. No architecture check, because an SDK holds executables for several architectures and checking its first ELF would refuse a correct cross-toolchain for containing a cross-compiler. The staged filename keeps the archive extension, because that is how anything downstream tells gzip from xz instead of guessing. Building it found the half I would have shipped past: `export-sdk` decoded GZIP ONLY. Every wasmtime archive and all 140 Zephyr SDK toolchains are .tar.xz, so a deposited Zephyr SDK would have been unopenable by the only command that exists to open it — another capability nobody could use, which is the pattern this release keeps turning up. Fixed with lzma-rs: pure Rust, MIT, and `cargo tree` shows zero `-sys` crates, so no C build dependency enters the crate every consumer links. The maintainer asked for pure Rust after trouble with C and C++ elsewhere, and the trade is favourable anyway — the bytes are digest-verified before a decoder ever sees them, so decoder maturity buys less here than cross-compilation does. bzip2 is now NAMED rather than left to fail. Without that, a .tar.bz2 payload falls through to the tar parser and reports "not a readable tar archive", which is true and useless: the download is fine and the decoder is missing. Decided by MAGIC here and by NAME in the producer, deliberately and not by accident. The producer picks a tool to run on bytes nobody has verified, so a name disagreeing with content is a reason to stop. The consumer holds bytes already checked against a signed digest, so what they ARE is the only question left. Clause 5 uses varve's own reader, so the producer proves the payload can be opened by exactly the code the consumer will open it with, rather than by a second implementation that might disagree. The failure it catches is concrete: a 14 GB download that turns out to be an HTML error page hashes and signs perfectly well. Proven against the real thing, not a fixture: the live 90 MB toolchain_gnu_macos-aarch64_arm-zephyr-eabi.tar.xz decodes through `read_members` to 5514 members, 48 of them under bin/. The xz tests build their input with the system `xz` rather than an encoder of my own, so they cannot agree with a mistake I made twice. Mutation testing found two: the shape check matched only by prefix, so a `contains` naming a FILE was never exercised and an absent file could be satisfied by a sibling; and `archive_ext`'s slice arithmetic was unasserted. 135 mutants across the three modules, zero survivors. sdkexport.rs joins the required gate.
`varve docs sdk`. Findable by the person who needs it, which is whoever is
about to hand varve a vendor SDK and discover that a .sh installer is refused.
The Yocto answer is a one-line change on the producing side rather than a
design fight: populate_sdk emits a self-extracting .sh by default, and
SDK_ARCHIVE_TYPE = "tar.xz"
makes it write a plain tarball under tmp/deploy/sdk/ instead. tar.xz rather
than tar.bz2 because varve decodes xz and does not decode bzip2 — stated in the
format table rather than left to be discovered at deposit time.
Also records why the .sh refusal is not negotiable: running a vendor's
installer to find out what it contains means executing unreviewed vendor code
on the machine that signs the layer, BEFORE anyone knows what the payload is.
"Run it and see" and "state what these bytes are" cannot both be the first
step. The paragraph exists so the next person to want that branch has to argue
with it first.
Documents what actually vouches for each of the three SDKs, because they differ
and the difference matters: Zephyr publishes an unsigned sha256.sum, WASI
publishes nothing at all, and neither is called a signature.
Writing a documented example is a test, and this one failed twice before it passed, both times for real reasons. First, the repo-name guard: `name = "zephyr-sdk"` from `zephyrproject-rtos/sdk-ng` is a RepoNameMismatch, because the shell assembler derives a tool's identity from its repository basename and a disagreeing name is discarded. The expressible form is `name = "sdk-ng"` with `binary = "zephyr-sdk"` — the same shape varve-producer uses in this repo — so the doc shows that, with a comment saying why, rather than showing something that does not translate. Second, and worse: `varve layer-spec` translated `layout = "sdk"` into a plain TARBALL_TOOLS entry. The env encoding has no field for a layout, so an sdk entry arrives at the shell assembler as an ordinary tarball tool and the tree is mined for a binary — silently, because every other field survives the trip. The Rust assembler reads layout directly and is unaffected. So a realm carrying SDKs must deposit with `varve-producer deposit --manifest layer.toml`, and deposit-layer.yml still uses the env encoding. That is now a blockquote in the doc rather than a footnote, and a field on REQ-SDKDEPOSIT-001. The Yocto answer is one line on the producing side: populate_sdk emits a self-extracting .sh by default, and SDK_ARCHIVE_TYPE = "tar.xz" makes it write a plain tarball instead. tar.xz not tar.bz2, because varve decodes xz and does not decode bzip2. `varve docs sdk`.
…d get an answer it cannot misread jess reported four measured failures of the shell contract in one repo in one day (varve#130), and two of them are failures this repo made in the same week. The sharpest is not arguable. A gate shelled out to objcopy, which on a runner without ARM binutils printed a format error and wrote nothing; the value became the empty string, `[ "" -gt N ]` ERRORS AND EVALUATES FALSE, both range tests fell through, and the script printed "ok" and exited 0 — a green verdict on a file it never parsed. No amount of care in that script fixes it. Only a type does. The other three: an error naming the subcommand when the fault was which binary; a pipeline's exit code quoted as a tool's verdict; and `set -e` aborting at the very command whose failure was being measured. I did the last two myself this week, so I am reading the list as evidence rather than as argument. Much of the answer already exists and nobody could find it. varve-core is published — seven versions on crates.io — and `resolve` already returns a properly differentiated error, each variant carrying what to do next. That a consumer wrote four failure reports without discovering it is the first finding, and clause 6 exists because an undiscoverable typed API loses to a discoverable shell one every time. What is genuinely missing is the one that matters most: `resolve` DOES NOT VERIFY. It reads the store and trusts that install verified on the way in, so "present but does not verify" is not among the answers a consumer can receive. The obvious patch — "call verify first" — would reintroduce the shell contract wearing types: a two-call protocol with an implicit ordering, where forgetting the first call fails OPEN. Clause 3 forbids that shape rather than the symptom. Clause 2 is the requester's, and it is the one to keep: verification failure must never be foldable into absence. "I could not find it" and "I found it and it did not verify" are opposite facts, and a consumer that treats the second as the first fails open. This repo has now found that same conflation three times in its own code this week — in the attestation prober, in the registry classifier, and in the error handler of the check written to prevent it. v0.33.0 now carries five open requirements. If this one does not land, it moves by an explicit bump, not by going quiet.
…one payload
Decision on varve#130: with-device stays in jess. The maintainer's argument is
better than mine — jess is the integrator archive and a tool arbitrating a
physical bench IS integration work, so it sits with the work it serves.
Revisit if it grows past bench arbitration.
But the maintainer also made the point that turns this from a placement
question into a defect: we cannot ensure outside repos avoid a common release
tag with per-tool sub-versions, and jess is only the first we noticed. Testing
that produced a structural finding.
ONE REPOSITORY MAY CONTRIBUTE AT MOST ONE PAYLOAD. A tool's `name` must equal
its repository basename, and two entries may not share a name. Together those
pin identity to the repo and allow the repo once:
[[tool]] name = "sdk-ng" binary = "zephyr-sdk-arm" …arm-zephyr-eabi.tar.xz
[[tool]] name = "sdk-ng" binary = "zephyr-sdk-riscv" …riscv64-zephyr-elf.tar.xz
error: two tool entries are both named "sdk-ng"
sdk-ng publishes 140 host×target toolchains. A layer wanting two targets cannot
say so, and no target field fixes that — which makes this the root cause under
REQ-SDKTARGET-001 rather than a sibling of it. Recorded there as such.
So moving with-device would have fixed nothing. It would have left the same
wall standing for Zephyr and WASI, and I would have walked into it within the
week while building the sdk work I had just scoped. My recommendation framed a
general pattern as one repo's mistake.
Five current sources already deviate from repo-derived identity: sigil ships
wsc, kiln ships kilnd, jess ships with-device at a version its tag does not
carry, rivet's vsix avoids the collision only by living in a different manifest
section, and sdk-ng would ship many.
Clause 4 keeps what the guard was protecting. RepoNameMismatch exists because a
tool whose name disagreed with its repo was once silently deposited under the
basename — a consumer asking for it found nothing in a layer that deposited and
verified. That property survives; the check moves to where the deposited name
is decided instead of being deleted.
The fifth entry field already made this correction for VERSIONS — a payload's
version is its own, not its repository's tag. This is the same correction for
NAMES. They were always one idea and I implemented half of it.
…om the registry, keep the artifact whole Clause 6 as first written was unsatisfiable, and I wrote it. It promised that a re-deposit fetches NO payload bytes, while the deposit layout contains every payload blob and is uploaded as the artifact of record. No download means no bytes means an incomplete artifact. Both could not hold. Three ways out. Let a carry-forward layout reference blobs it does not contain, weakening the artifact of record that REQ-ARCHIVE-002 already calls inadequate. Keep the promise and abandon the artifact. Or narrow the promise. Narrowing it. A carried-forward payload is now fetched from the DESTINATION REGISTRY — which clause 4 already requires to hold it before reuse is permitted, so the check that makes reuse safe and the source that makes it possible turn out to be the same check. The layout stays complete and REQ-ARCHIVE-002 is untouched. The saving is smaller than I first claimed and the requirement now says so: not zero bytes, but one host instead of four upstream CDNs, no upstream rate limits, and no re-verification of an unchanged upstream release. 421 MiB still moves; it moves from somewhere closer, already verified, that we control. Promising a number nobody can deliver is the failure this requirement family keeps finding in other people's documents, and clause 6 was mine. Two clauses added that the first framing did not have, because moving the source moves the risks: Clause 2 — the digest used to fetch is the one the CURRENT proof states, never the one the previous layer recorded. Fetching by the old digest would carry a republished upstream forward unnoticed, which is precisely the substitution this requirement exists to catch, reintroduced by the optimisation meant to respect it. Clause 4 — a blob absent from the destination falls back to upstream rather than aborting. Clause 4's presence check can race a garbage collection, and a deposit that fails because a cache was pruned is worse than a slower one. And clause 3: registry bytes are verified against the digest exactly as an upstream download is. The destination is a source like any other and gets no more trust for being ours.
One repository could contribute at most one payload. A tool's `name` had to
equal its repository basename, and two entries could not share a name, so
identity was pinned to the repo and the repo could appear once. That refused
zephyrproject-rtos/sdk-ng's 140 host×target toolchains outright, and would have
refused any monorepo upstream — and we do not get to ask upstreams to
restructure their releases.
Three separate conflations, all of them the same mistake:
NAME. `stage_one` took one `binary_name` and used it for the executable INSIDE
the archive AND for the payload's deposited name, so a payload could only ever
be deposited under the name of the file found in its own tarball. Now two
names, bundled as `Names { deposited, binary }` — which also answered clippy's
too-many-arguments, so the type earns its place twice.
REPOSITORY. The guard forcing name == basename existed because the SHELL
assembler takes a tool's identity from the repository basename and has no field
for a deposited name. That is the encoding's limit, not the manifest's, so the
check moved to the translation boundary rather than being deleted: it fires
when a name really would be discarded, and not otherwise. The property it
protects is intact — a payload deposited under a name nobody asked for is still
refused.
VERSION. Found by the agent porting pulseengine-layers, and it would have been
a silent regression: `ManifestTool.version` served as both the release TAG to
fetch and the version signed into the layer. For a hub repo those differ —
jess tags v0.7.2 and ships with-device 0.2.2 — so layer.toml could say one or
the other, sign a version the binary contradicts or name a tag that does not
exist. The env encoding could already express this via its fifth positional
field; layer.toml could not, so porting a realm to layer.toml LOST the payload.
An optional `release` field fixes it, and the agent left the entry out of its
PR rather than sign a false version, which was the right call.
A branch that mutation testing had proved DEAD came back to life. The entry
head was shortened to the bare name whenever the owner was the default, with a
comment recording that `repo_name == t.name` held by the check above so testing
it here was a condition no input could vary — cargo-mutants had flipped it and
killed nothing. True, and true only relative to an invariant I then removed:
with the shortcut left in, `with-device` from `pulseengine/jess` emitted
`with-device:v0.7.2:…` and the shell would have fetched `pulseengine/with-device`,
a repository that does not exist. Caught by reading the emitted string rather
than the exit code. The comment now records both the original reasoning and why
it stopped holding.
Also: `layout = "sdk"` now REFUSES to translate into the env encoding rather
than arriving at the shell as an ordinary tarball entry to be mined for a
binary. The docs test that requires every documented layer.toml to translate
accepts that one refusal, because it is the documented behaviour — any other
translation failure is still a broken example.
35 mutants on layerspec, zero survivors. The survivor found on the way was
`binary.is_none() || asset.is_none()` — both my cases omitted both fields, so
`&&` behaved identically; the case that actually corrupts is exactly one
missing, where the payload version lands in the TEMPLATE slot and the assembler
looks for an asset named "0.2.2".
…ost convention This code was written for v0.33.0 and then committed to the wrong branch. It was swept into the root-rotation commit by a `git add -A`, where it sat as 350 lines of producer work inside a diff whose message said "Only its public half appears here". Moving it back; the rotation is now #136, eight files, all key. Two things, both found by trying to deposit a real Zephyr SDK. `upstream-sums` — a fourth ingest rung, between `build-provenance` and `unverified`. zephyrproject-rtos publishes `sha256.sum` beside its assets and signs nothing. Transcribing it establishes exactly one fact: the bytes are the bytes that list names. For a 90 MB toolchain over a CDN that is the failure which actually happens, so it is worth having. It establishes NOTHING about who produced them, because the same host serves the list and the bytes, and the rung's name has to keep saying so — which is why this is a named rung and not a quiet promotion to `cosign-sums`. The realm states the asset name rather than varve guessing it: a file that is not the digest manifest, parsed as one, vouches for nothing while looking like it does. `%H` — the host tag written os-first. There is no single upstream convention. bytecodealliance writes `aarch64-macos`, which `%U` already produced; zephyrproject-rtos/sdk-ng writes `macos-aarch64`. Same machine, opposite order, and a template using the wrong one matches nothing. Adding the token was not enough, and the way it failed is the point. `is_per_platform` did not know `%H`, so a template naming a machine was read as portable, expanded once with no platform, and the payload reported ABSENT. No error, no unknown-token complaint — just a correct-looking "not found" for an asset that was right there. The new test walks every machine-naming token rather than checking %H alone, because the next one added will fail the same way. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
REQ-DOCS-002's baseline records a 2026-08-14 review that caught a docs topic teaching a fabricated root under a heading promising every file was literal. The record said "The real published root is 4e771dc6…" — true when written, false since v0.32.1, and written in a tense that does not say which. Not rewriting the history: the finding and the value it names are what happened, and a verification baseline that gets edited to match the present is worth less than one that does not. Only the tense is fixed, with a pointer forward. Worth the pointer because the two findings are the same finding, eighteen days apart. That review pinned the DOCUMENTED copy of the root to rolling.pub. It did not ask whether any other copy existed. varve-realms.toml — the copy consumers actually resolve, and the one that outranks the environment — went on uncompared until the rotation forced someone to look at both. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
…ing touched A yank, an advisory and a support-window correction are one act — saying something about a layer AFTER it is published — and until now varve could not perform it. The document type existed, signed, countered, with known-problems and yank markers, and `varve status` read it. What was missing was any way for a corrected one to REACH a registry consumer. The baseline is a blob inside a LAYER's artifact manifest. Reissuing it means re-pushing that manifest with a different blob digest, which changes the manifest digest — the republish REQ-IMMUTABLE-001 refuses, reached by a well-intentioned route. Yanking a layer must not require mutating it. So a status document is now publishable under the line's OWN tag, `line-status-<line>`, exactly as the signed index goes under `line-index-<line>`. That precedent was already in the codebase and is right; nothing had followed it for status. The tag cannot be parsed as a LayerId, so `layers_of_line` excludes it by construction and an honest registry is never made to look like it serves a layer the signed index never named. VERIFY FIRST, RANK SECOND — the order is the security property, not an implementation detail. Both documents are untrusted bytes from the party this evidence exists to constrain. Ranking by counter before checking signatures would hand the choice to whoever serves the tag: write a large counter, win the comparison, and a forged document displaces a real one — or, failing verification afterwards, denies the consumer the good baseline it already had. Four tests hold the corners: a correction overtakes an older baseline; a STALE tag document cannot walk a consumer backwards past its baseline (clause 2 says prefer the newer, not prefer the tag, so a registry cannot suppress a yank by serving an old one); an unverifiable tag document is discarded rather than fatal; a validly-signed document for another line is refused by the same guard the baseline path uses, not a second rule that could drift. The baseline is untouched and stays exactly as useful — offline install and `varve install --from` behave as before. Clause 4 is one dispatchable workflow, because the three acts are one document and three workflows would drift. It refuses a counter that cannot supersede what is published — consumers keep the newer, so a low counter would be discarded silently from the operator's point of view — and it establishes absence from `oras repo tags` rather than from error prose, because inferring absence from a "404" in a message is how the publish-check was once fooled by a repository path. It then reads the document back through the tag a consumer resolves and verifies it against the realm's PUBLISHED root, not the key that just signed it, which would only prove the signer agrees with itself. Exit 3 is accepted there: a yank that did NOT make `status` exit 3 would be the bug. Two corrections made while building this, both of the same shape — a document naming something that is not there: - Clause 3 cited `LineStatus::refuse_regression`. No such function exists; `refuse_regression` is a method on LineIndex. The behaviour is real, in `StatusCache::update`, and the clause now names it. A spec naming a function that is not there is how an implementer writes the second rule the clause forbids. - `cmd-attach-status.md` told readers newer documents arrive via "a re-pushed registry baseline" — the very thing REQ-IMMUTABLE-001 refuses. It now documents the tag, with the consumer's two-document rule spelled out. Clause 5 (publish the realm's line-index) is deliberately NOT done; it turns on where the index's layer list comes from, which decides whether the index can detect the omission it exists to detect. Recorded on the requirement. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
…mits to REQ-POSTDEPOSIT-001 clause 5 asks for the line-index to actually be published. Building the publisher surfaced a prior question that decides whether the check can fire even once it exists: where the index's list of layers comes from. The cheap answer is vacuous, and worth writing down precisely because it looks correct. Derive the list from the registry's own tag listing at deposit time and there is no committed file, no maintenance, no conflict when two deposits race — and no detection. The index exists to catch a registry HIDING a layer; if the list comes from that registry, a hiding registry omits the layer from the tag listing and therefore from the index, the two agree perfectly, and varve reports nothing wrong. It would pass its own tests and publish real signatures. An artifact that carries a guarantee it cannot provide is worse than no artifact, because everyone downstream believes it. So the realm repository commits the index document and each deposit appends the layer it just pushed. The list becomes an independent statement about what the realm contains, which is what omission is measured against. Accepted cost: a reviewed file the depositor maintains, and a merge conflict if two deposits race, which the per-layer concurrency group already makes rare. It lands in pulseengine-layers rather than varve because deposits are moving there; building it here would be building it in the place it is leaving. varve supplies sign-index, attach-index and the documented manifest shape. Filed as pulseengine-layers#6, blocked on #5. Stated so it is not discovered later: until that ships, no line-index tag exists, so no varve realm may declare signed-index = true. A realm that declares it with no index published fails closed on every install. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
…e a stranger
Measured the day after living it. When the rolling root rotated in v0.32.1, a
consumer who took the new varve-realms.toml without moving their pin got exactly
this and nothing else:
error: manifest signature verification failed: … No valid signatures
and their installed layers simultaneously stopped showing a realm name in
`varve list`, appearing under a bare fingerprint, because the store partitions
by root and no realm named that partition any more. Two symptoms, neither naming
the cause — and the cause is the one thing a consumer cannot deduce, because
nothing distinguished "signed by a root this realm retired last week" from
"signed by a stranger". varve already held the information to tell those apart
and did not use it.
A realm may now declare the roots it has RETIRED. varve's own realms file
declares the one it just retired, so this is not a capability waiting for
someone to carry it.
WHAT THIS IS NOT. varve still has no key rotation. Nothing signs "this new root
replaces the old one", and no consumer would check such a statement if you
produced it; `docs threat-model` and `docs root-ceremony` still say so and are
still true. A succession statement signed by the retiring root is the real
mechanism, and it was rejected here on evidence rather than effort: it requires
still HOLDING the old key, and the entire reason v0.32.1 happened is that the
old key was a write-only CI secret nobody could read. A mechanism that only
works for the rotations you planned would not have helped the one that occurred.
It changes the MESSAGE, never the VERDICT, and three tests hold that line
because a diagnostic that quietly widened trust would be far worse than the
confusing error it replaces:
- A retired root never verifies anything. The layer is rejected exactly as
before — confirmed end to end: exit 1, nothing installed.
- An unknown signer is NOT excused. A stranger's signature still gets the plain
error; otherwise the diagnostic would tell an operator a rotation happened
while they were being attacked.
- Listing the realm's live trust-root as retired is refused at parse. No
legitimate use, and precisely the slip a half-finished rotation makes —
update one field, paste the same value into the other.
`varve list` now names a partition a retired root left behind, dated, because
"my tools vanished" is the symptom noticed before any error message.
The docs gate earned its keep again. The config-reference example used an
elided `4e771dc6...`, and `every_documented_file_example_parses_with_the_real_parser`
refused it. The tempting fix is to pad the placeholder until the parser is
happy, which is how a shape check certifies a false fact — the exact failure
REQ-DOCS-002 exists for. The retired root is public, so the example now carries
the true value.
The lesson worth keeping, written into root-ceremony: ROTATE WHILE YOU STILL
HOLD THE KEY. The last useful act of a retiring root is to sign something saying
it is retiring, and that is only possible while you can still use it. A root you
cannot use is a root you have already partly lost.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
The trust-critical gate covers realm.rs, and the retired-root work put three survivors there. Fixing them found a real defect beside them. realm.rs, both survivors of the same shape: `RetiredRoot::fingerprint` could be replaced by `String::new()` or a constant with nothing noticing, because the test only asserted the retired fingerprint DIFFERED from the live one — true of any constant. It now asserts the fingerprint equals the one that root's partition was actually written under, computed by resolving a realm where that key IS the live root. That is the property that matters: the value has to name a directory already on disk, or `varve list` looks for one that does not exist. realm.rs, the third: `||` -> `&&` in the retired-key validation survived because the only malformed input tested was "not-a-key", which fails length AND alphabet at once and so cannot tell the operators apart. Now each half is exercised alone — 64 characters of non-hex, and hex of the wrong length, including one char short. linestatus.rs: `>` -> `>=` when choosing between the baseline and the tag document. Not cosmetic. Iteration order is [baseline, published], so `>=` lets a document served under the tag displace a baseline of the SAME counter — and the interesting version of that is a registry serving a same-counter document with the yank removed, suppressing it. Strict greater-than keeps the copy bound to the immutable layer, which is the conservative choice when a signer has issued two documents at one counter. The first attempt at that test was VACUOUS and passed under the mutant. The `status()` fixture already yanks 2026.07.0, so the "quiet" document built from it still carried the yank and the assertion held whichever document won. The fixture's own contents were the trap — see the standing lesson about checking an oracle's fixture before trusting what it proves. Now the yank is cleared explicitly, a precondition asserts the baseline really does yank, and the test was confirmed RED against the mutant and green without it. NOT fixed, and worth stating rather than leaving to be discovered: linestatus.rs carries 18 further survivors and is NOT in the trust-critical mutation gate, so none of them are enforced. Most are io::ErrorKind guards of little consequence, but the file also holds yank and advisory logic — a stop-the-build signal — and the gate's own comment says the list is meant to grow. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
…g checks Marking for the next release what v0.33.0 surfaced but should not fix in a rush. REQ-MUTATE-002 is verified and rightly so: it named five modules and they were added. Its TITLE — "covers the trust decisions, not a subset of them" — claims more than its body delivered, and the gap is now measurable. The gate lists 16 varve-core files by hand; the crate has 39. Outside it: keys.rs, deposit.rs, store.rs, manifest.rs, registry.rs, lineindex.rs, linestatus.rs, attest.rs, attestcarry.rs, referrers.rs, update.rs. Key handling, layer signing, digest computation, what a registry serves, omission detection, yank state. Not theoretical. Adding the two-document preference to linestatus.rs this release put a survivor there that permitted YANK SUPPRESSION — greater-or-equal instead of greater-than lets a same-counter document served under the tag displace a baseline carrying the yank. It is killed. It was found because someone chose to run cargo mutants by hand on a file the gate does not cover, and choosing to look is not a control. The same file holds five ErrorKind::NotFound guards replaceable by `true`, collapsing "this file cannot be read" into "this file is absent" — the EXACT defect class REQ-MUTATE-002 found and fixed in reverify.rs and exportstamp.rs. Same defect, different file, still live, outside the gate that exists because of it. The deeper problem is the one worth the requirement: coverage is a hand-maintained list and nothing checks it for completeness, so a file that gains a trust decision does not get noticed. linestatus.rs gained one this very release. It will drift again, and the next person to notice will also notice by accident. Deliberately NOT fixed now. Adding those files means clearing 18 survivors first, and doing that while v0.33.0 still owes two requirements would be widening scope mid-release. Clause 5 exists because the tempting shortcut — adding files by relaxing the zero-survivor rule — moves the number and not the risk. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
…its own varve's docs gate is real and works — 34 subcommands documented, 22 with worked examples, and it has twice refused a FALSE example rather than a missing one. It covers one binary. varve-producer is separately built, separately released and separately RUN. REQ-PRODUCERGET-001 exists because consumers had to improvise a way to obtain it; pulseengine-layers now downloads and executes it in CI. It has six subcommands, no `docs` subcommand, and zero documented topics. So REQ-DOCS-001's invariant — a new subcommand cannot ship undocumented — is true of varve and false of varve-producer, and nothing anywhere says so. Third instance of one pattern in a single release, which is why this is a requirement and not a fix. A gate is built, it is real, it works, and its scope is a hand-drawn boundary nothing checks: the mutation gate covers 16 of 39 varve-core files (REQ-MUTATE-003); the docs gate covers one of two shipped binaries; four capabilities shipped complete, tested, and carried by no artifact. Each time the gate is believed for more than it checks. Where it would already have paid: the producer's own help text says of its asset template language that it "has silently dropped a tool from a published layer". That language is explained nowhere but --help and the source — and it gained %H this release precisely because %U encoded the wrong convention for one upstream and matched nothing. The ingest ladder is in the same position: four rungs, one added this release, documented in doc comments. The audience is other repositories' CI jobs, and the ten-persona audit already established that a reader needing the Rust source is itself the finding. Clause 5 is the one worth arguing with: documenting the producer inside `varve docs` does NOT discharge this. A CI job holding the producer may not hold varve, and sending someone to a different binary for the manual is the friction this exists to remove. Recorded but deliberately not claimed here: only publish-check offers structured output and no `--format json` is advertised in top-level help, against the org CLI convention. Separate requirement. `--version` and exit-2-on-unknown-flag already conform. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
Replaces REQ-DOCSEXPORT-001, which I scoped wrongly and which is removed rather than left as a second, misleading requirement in the same area. I had it as "publish varve's own documentation as HTML for a website". That is the case least worth solving: a tool that embeds its docs already answers for itself, at the version installed, offline — `varve docs` and `rivet docs` do exactly that. The gap is the documentation NOTHING CAN BE ASKED FOR. Two real cases. varve carries `crate` payloads, and a crate's documentation is its rustdoc: HTML, generated at build time, reachable nowhere offline, because docs.rs is a network service and a pinned air-gapped build cannot consult it. And the toolchain's architecture documentation lives on pulseengine.eu, belongs to no binary, and is therefore versioned by nothing — a reader cannot tell which layer the page in front of them describes. Measured: no pulseengine release publishes a documentation asset at all. rivet ships binaries, an SBOM, a vsix and a compliance report; meld, synth, witness and loom ship none. The documentation obtainable for the exact versions layer 2026.09.1 pins is, today, nothing. ONE kind with a SUBTYPE, not a kind per flavour. rustdoc, architecture, manual and whatever follows are identical to the store, the manifest and the signature, differing only in what a reader does with them. A kind per flavour would change the signed manifest's schema every time someone has a new sort of document. Opening it is part of the requirement, not a convenience bolted on. An HTML tree needs serving, because rustdoc's cross-links and search do not work from file:// in every browser and a consumer should not have to know that. A PDF needs opening. Documentation a reader must locate by hand inside a content-addressed store is documentation nobody reads, so varve does the guiding. And it must be addressable by a machine: an agent working with a pinned toolchain should be able to obtain the docs for exactly the versions in play without scraping human output. Same argument as `--format json` everywhere else, applied to the thing an agent most needs and currently cannot get. This commit is the requirement only — one file. While scoping the version I had wrong I briefly added pulldown-cmark to render markdown; it was removed in the working tree before anything was committed, so there is nothing here to back out and the dependency count is unchanged at five. Noting it because the reasoning is worth keeping: varve renders no markdown under this design either. rustdoc arrives as HTML and a PDF arrives as a PDF, so the renderer would have been a dependency bought for a problem the layer does not have. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
…er serve
varve does not render the documentation it carries and does not serve it. For a
pinned layer it reports WHERE each document is and prints the exact command to
open it — the system handler for a PDF, a one-line local server for an HTML
tree — in a form a human can paste and an agent can execute.
The previous wording said varve shall OPEN what it carries "in a single
command", which reads as varve doing the opening. That would have made a
documentation viewer the first exception to a stated design principle: NO SERVER
OF OUR OWN, in the README, and repeated verbatim in the help for `varve
deposit` ("varve runs no server and pushes nothing, by design"). A tool whose
value is partly that it has no network listener should not acquire one to show
somebody a rustdoc page.
It would also have bought a rendering dependency for a supply-chain tool that
currently has five direct dependencies — the same purchase I nearly made an hour
ago for the version of this requirement that was wrong.
Serving is one command the consumer runs with a runtime they already have.
Knowing WHICH command, and WHERE to run it, is the whole of the help that is
actually needed — and it is help varve is uniquely able to give, because it is
the only thing that knows which layer is pinned and where its bytes landed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
…t proved it REQ-POSTDEPOSIT-001 records the maintainer accepting layer 2026.09.1's six-month support window against a one-month rolling policy, on a stated condition: fix forward, because "the next rolling layer carries a correct one-month window from deposit". Dispatching that next deposit shows it does not. The one-month policy lives on plan/v0.33.0, unreleased. layer.toml pins the builder at varve v0.32.1, whose support.rs still returns six months for rolling. So 2026.09.2 computed "supported until 2027-03-07" and would have carried the same overstatement into another immutable layer — correctable only through the line-status tag added by REQ-POSTDEPOSIT-001 itself, which is on the same unreleased branch. It was not published only because the registry push failed on a missing package permission. That is luck, not a control, and it is worth writing down as luck. The general shape is the part worth keeping. A fix-forward plan is a promise about a FUTURE release, and it becomes false the moment the fix sits on a branch that has not shipped while the thing it fixes runs from one that has. Nothing checked that the deposit's builder carried the policy the decision assumed — the decision and the pin were never compared, and there is no reason anyone would have thought to compare them. Sequence corrected: v0.33.0 releases first, layer.toml moves to it, and only then is 2026.09.2 deposited. Depositing now would spend the layer id on the wrong window. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft — opened to get CI on the work, not because the release is ready. CI runs only on PRs and pushes to
main, so these 18 commits have so far been verified only locally: fmt, clippy, 17 test suites andrivet validate. The mutation gate, the fuzz targets, MSRV, the systest matrix and the non-macOS platforms have not seen any of it.Scope: 6 of 8 done
.tar.xz/.tar.bz2in both halvesThe two that carry the most risk
REQ-POSTDEPOSIT-001 — a yank, an advisory and a support-window correction are one act, and varve could not perform any of them. The baseline lives inside a layer's manifest, so correcting it would re-push that manifest with a new digest: the republish REQ-IMMUTABLE-001 refuses. Corrections now go under
line-status-<line>, mirroring the existingline-index-<line>precedent. The load-bearing rule is verify first, rank second — ranking two documents by counter before checking signatures would let whoever serves the tag pick the winner by writing a large number.REQ-ROTATE-002 — measured the day after the v0.32.1 rotation. A consumer who took the new realms file without moving their pin got
No valid signatures, indistinguishable from a forgery, plus their layers losing their realm name invarve list. A realm may now declareretired-roots; varve's own realms file declares the one it just retired. It changes the message, never the verdict — a stranger's signature still gets the plain error, and listing the live root as retired is refused at parse.This is not key rotation and the docs still say so. Succession signed by the retiring root was rejected on evidence: it needs the old key, and the whole reason v0.32.1 happened is that ours was unreadable.
Deferred, with the reasoning recorded
DD-027 — clause 5's index publisher goes to
pulseengine-layers(issue #6). Deriving the index's layer list from the registry's own tag listing would be vacuous: a hiding registry omits the layer from both, they agree, and nothing is reported. Consequence: no varve realm may setsigned-index = trueuntil that ships, or installs fail closed.Conflicts to resolve before this leaves draft
#111 and #113 both touch this ground and are overtaken by events — see the review comment.
🤖 Generated with Claude Code
https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu