From 100ef3b9c25cf2b887b08518042519b60cba1a9f Mon Sep 17 00:00:00 2001 From: James Ross Date: Fri, 2 Oct 2026 14:09:01 -0700 Subject: [PATCH 1/3] Docs: reconcile current durable surfaces with main (#130) --- CHANGELOG.md | 2 ++ README.md | 21 ++++++++++----- docs/formats/segment-store-v2/README.md | 22 ++++++++++++--- docs/formats/segment-store-v2/closure.md | 6 +---- docs/formats/segment-store-v2/requirements.md | 2 +- .../segment-store-v2/retention-publication.md | 6 ++--- .../current-durable-surfaces.md | 27 +++++++++++++++++++ src/lib.rs | 17 ++++++++++-- 8 files changed, 82 insertions(+), 21 deletions(-) create mode 100644 docs/testing-evidence/current-durable-surfaces.md diff --git a/CHANGELOG.md b/CHANGELOG.md index c23e7c27..00b1a69b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,8 @@ after its public API and format compatibility policies are established. ## [Unreleased] +- Crate and version-two documentation now distinguish merged retention recovery and fenced snapshots from pending durable reads, verification, ingestion, GC, compaction, and general candidate-catalog retained-closure admission (#130). + - Retention recovery execution errors report the exact failed boundary, original typed cause, known namespace effects and uncertain effect/durability; retries freshly observe the store. Observed stage identity remains binding across reopening, and cleanup preserves verified pool evidence rather than promising the removed pathname survives (#99). - Retention recovery now preserves incomplete stages and requires explicit disposition before any recovery mutation or publication retry; automatic incomplete-stage disposal is deferred by maintainer decision (#99). diff --git a/README.md b/README.md index 5a470d1f..1f2fcdf4 100644 --- a/README.md +++ b/README.md @@ -75,13 +75,19 @@ Keep is required to refuse all three, before mutating anything. at 68 before/during/after coordinates (`KEEP-CRASH-053`–`073`), preserving every version-1 byte. Broader hostile restart combinations remain in #111. -## What it does not do yet - -Version-2 retention recovery, fenced reader snapshots and model-based transitions are implemented in this branch but still require the correctness corrections and independent acceptance review tracked in PR #99. +Version-2 complete-stage retention recovery, fenced retention snapshots and model-based transitions are implemented on main through [PR #99](https://github.com/flyingrobots/keep/pull/99). The retention process-death sequence checks the recovered head generation and exact selected-root bytes before retry; its [evidence receipt](docs/testing-evidence/retention-crash-reader-oracle.md) bounds that claim to the declared initial-publication crash coordinates. -Incomplete retention stages are preserved and block publication pending explicit disposition; automatic disposal is deferred. The [landing ledger](docs/testing-evidence/retention-landing.md) tracks execution-failure reporting and final acceptance under the [approved recovery contract](docs/formats/segment-store-v2/retention-recovery.md). +Recovery failures retain their typed cause and report known effects separately from uncertain effects or durability under the [approved recovery contract](docs/formats/segment-store-v2/retention-recovery.md). + +Writer authority coordinates cooperating writers in a managed namespace; it does not isolate arbitrary concurrent out-of-band filesystem mutation. + +## What it does not do yet + +Incomplete retention stages are preserved and block publication pending explicit disposition; automatic disposal remains deferred in [#155](https://github.com/flyingrobots/keep/issues/155). + +Fenced retention snapshots provide bound retention evidence and selected-root verification; a durable authenticated blob-to-writer or exact-range read API remains in [#109](https://github.com/flyingrobots/keep/issues/109). Complete orphans remain recovery-protected until explicit disposition lands with garbage collection (#21). @@ -91,9 +97,10 @@ A version-1 store stays admitted until its owner migrates it. | Gap | Tracked | | --- | --- | -| Retention recovery correctness remediation and broader migration corruption coverage | [PR #99](https://github.com/flyingrobots/keep/pull/99), [#111](https://github.com/flyingrobots/keep/issues/111) | -| Fenced reader correctness remediation and independent acceptance | [PR #99](https://github.com/flyingrobots/keep/pull/99) | -| Precise verification reports at explicit depths | [#20](https://github.com/flyingrobots/keep/issues/20) | +| Broader migration corruption and compatibility evidence | [#111](https://github.com/flyingrobots/keep/issues/111), [#112](https://github.com/flyingrobots/keep/issues/112) | +| Reader-fence process-death evidence | [#113](https://github.com/flyingrobots/keep/issues/113) | +| Candidate-catalog preservation of every retained closure | [#125](https://github.com/flyingrobots/keep/issues/125) | +| Precise durable verification reports at explicit depths | [#114](https://github.com/flyingrobots/keep/issues/114), parent [#20](https://github.com/flyingrobots/keep/issues/20) | | Garbage collection and identity-preserving compaction | [#21](https://github.com/flyingrobots/keep/issues/21) | | Bounded production ingestion through the durable store | [#82](https://github.com/flyingrobots/keep/issues/82) | | Encrypted representations | [#86](https://github.com/flyingrobots/keep/issues/86) | diff --git a/docs/formats/segment-store-v2/README.md b/docs/formats/segment-store-v2/README.md index ddad4eef..7b7b8196 100644 --- a/docs/formats/segment-store-v2/README.md +++ b/docs/formats/segment-store-v2/README.md @@ -6,7 +6,11 @@ catalog, and publication-head byte while adding explicit retention state, reader fences, migration evidence, and reserved GC and recovery-disposition namespaces. -ADR-0009 owns the cross-cutting retention and liveness decision. These pages own its durable representation. The one-way migration, version-two reopen, forward retention publication, partial-prefix migration recovery, and the 68-case migration process-death matrix are implemented with executable evidence. Retention recovery and reader fencing are implemented in this branch; correctness remediation and independent acceptance remain tracked in PR #99. Collection remains planned in #21. The [requirements ledger](requirements.md) records requirements and their evidence status. A version-1 store remains admitted until its owner migrates it. +ADR-0009 owns the cross-cutting retention and liveness decision. These pages own its durable representation. + +The one-way migration, version-two reopen, forward retention publication, partial-prefix migration recovery, complete-stage retention recovery, and fenced retention snapshots are implemented on main through [PR #99](https://github.com/flyingrobots/keep/pull/99). + +The [requirements ledger](requirements.md) records evidence and remaining obligations; the laws below are normative requirements, not a claim that every corresponding runtime surface exists. ## Core laws @@ -89,9 +93,21 @@ head and the catalog it selects, and refuses superseded candidates, retained stages, replaced protocol directories, and every namespace or capacity violation before mutation, each as a typed `RetentionCurrentStateRefusal`. -Retention publication recovery, fenced reader snapshots and model-based transition evidence are implemented in this branch. +Retention publication recovery, fenced retention snapshots and model-based transition evidence are implemented on main. + +The [landing ledger](../../testing-evidence/retention-landing.md) records the accepted scope and evidence for merged PR #99. + +Incomplete retention stages are preserved pending explicit disposition; automatic disposal remains deferred in [#155](https://github.com/flyingrobots/keep/issues/155). + +Execution failures preserve the typed cause and report known effects separately from uncertain effects or durability, as specified by the [retention recovery contract](retention-recovery.md). + +Writer authority coordinates cooperating writers in a managed namespace; it does not isolate arbitrary out-of-band filesystem mutation or make pathname unlink conditional on inode identity. + +`FilesystemRetentionSnapshot` binds a catalog, retention head and manifest under a shared reader fence and verifies selected roots on demand; it does not expose the durable authenticated blob and range read API tracked in [#109](https://github.com/flyingrobots/keep/issues/109). + +General version-two catalog publication still needs the candidate-catalog retained-closure admission gate tracked in [#125](https://github.com/flyingrobots/keep/issues/125); verifying a new retention root against the current catalog is a different operation. -Their bounded landing and independent acceptance remain tracked in PR #99 and the [landing ledger](../../testing-evidence/retention-landing.md). Incomplete retention stages are preserved pending explicit disposition; automatic disposal is deferred. Execution-failure reporting remains part of the [retention recovery contract](retention-recovery.md). +Durable verification reports ([#114](https://github.com/flyingrobots/keep/issues/114)), production durable ingestion ([#82](https://github.com/flyingrobots/keep/issues/82)), garbage collection and compaction ([#21](https://github.com/flyingrobots/keep/issues/21)) are absent from main; implementations prepared in unmerged PR #107 are not delivered APIs here. The `KEEP-CRASH-036..052` initial-publication process-death sequence includes independent recovered-reader checks; its [evidence receipt](../../testing-evidence/retention-crash-reader-oracle.md) records the assertions, calibration, and scope. diff --git a/docs/formats/segment-store-v2/closure.md b/docs/formats/segment-store-v2/closure.md index 3e0e79ba..5e4c1912 100644 --- a/docs/formats/segment-store-v2/closure.md +++ b/docs/formats/segment-store-v2/closure.md @@ -1,10 +1,6 @@ # Closure Verification -- Status: Normative version-2 protocol; storage-independent verifier - implemented; publication binds this store's catalog head and the catalog it - selects to the verified closure; member re-verification under filesystem - authority is planned in issue - [#19](https://github.com/flyingrobots/keep/issues/19) +- Status: Normative version-2 protocol; the storage-independent verifier and live member re-verification under filesystem retention authority are implemented. General candidate-catalog retained-closure admission remains in [#125](https://github.com/flyingrobots/keep/issues/125). - Format coordinate: `keep.segment-store/v2` - Requirement: [`KEEP-RETENTION-005`](requirements.md#retention-transitions) - Decision record: diff --git a/docs/formats/segment-store-v2/requirements.md b/docs/formats/segment-store-v2/requirements.md index b58eaf19..0fba248c 100644 --- a/docs/formats/segment-store-v2/requirements.md +++ b/docs/formats/segment-store-v2/requirements.md @@ -15,7 +15,7 @@ case is not evidence. | `KEEP-RETENTION-004` | Retain and release compare expected and observed generations and publish exact successors only | unforgeable readiness and preflight proofs in `tests/retention_transition.rs` and `tests/retention_preflight.rs`; exact successor preparation and complete receipt evidence in `tests/retention_publication_preparation.rs` and `tests/retention_publication_execution.rs`; writer-locked initial filesystem publication in `filesystem_retention_storage_tests`; observed-head successor publication, exact predecessor binding, and absent-head refusal in `filesystem_retention_successor_tests`; the store's catalog head must name the closure's catalog generation and digest before any forward write in `filesystem_retention_catalog_tests`; a head whose predecessor disagrees with its manifest refuses in `filesystem_retention_current_tests`; a successor reopens and decodes the manifest-selected predecessor root and refuses an absent or changed one in `filesystem_retention_expectation_tests` | Implemented | | `KEEP-RETENTION-005` | Closure derivation is deterministic, bounded, cycle-safe, fail-closed, and verifies complete blob reconstruction | exact accounting, reconstruction, adversarial-catalog, and exhaustive model laws in `tests/retention_closure.rs`; corrupt members refuse through the inherited segment-record admission laws and seeded `segment_format` fuzz target routed by `closure-corruption.md` | Implemented | | `KEEP-RETENTION-006` | Publication follows the exact ordered durability protocol, including new namespace-directory admission and retention of fixed-stage evidence until head commit, and returns only after cleanup synchronization | typed vocabulary and blocking port in `tests/retention_publication_phase.rs` and `tests/retention_publication_storage.rs`; ordered execution, conditional namespace sync, and all 17 exact storage-fault boundaries in `tests/retention_publication_execution.rs`; production 17-phase forward filesystem execution, exclusive staging, byte-equal inode-substitution refusal, and retained-stage recovery refusal in `filesystem_retention_storage_tests`; orphan namespace directories count against the 4,096 ceiling and refuse a new namespace before any stage is written in `filesystem_retention_capacity_tests`; crash injection remains | In progress in #19 | -| `KEEP-RETENTION-007` | Complete-stage recovery preserves canonical history; incomplete stages require disposition before mutation | Complete publication-prefix recovery and reader-state laws remain; incomplete direct/publication recovery and process-death laws now require typed refusal and preserved bytes. See [landing ledger](../../testing-evidence/retention-landing.md). | Bounded landing in #99; automatic incomplete-stage disposition explicitly deferred | +| `KEEP-RETENTION-007` | Complete-stage recovery preserves canonical history; incomplete stages require disposition before mutation | Complete publication-prefix recovery and reader-state laws remain; incomplete direct/publication recovery and process-death laws require typed refusal and preserved bytes. See the accepted scope and evidence in the [landing ledger](../../testing-evidence/retention-landing.md). | Implemented under the bounded scope merged in #99; automatic incomplete-stage disposition deferred to #155 | | `KEEP-RETENTION-008` | Readers double-collect catalog and retention heads and bind one complete catalog, manifest, and root-generation view under a `ReaderFence` | `ReaderFence` holds a shared kernel lock on a verified zero-length `reader.lock`; `collect_retention_view` accepts a view only when both head coordinates agree before and after loading and refuses an exhausted attempt limit (`retention_view_collector_tests`); `FilesystemRetentionSnapshot` binds the catalog snapshot, retention head, and manifest under the fence and verifies each selected root on demand while the fence is held, refusing a substituted root and a replaced fence, and two readers share the fence while an exclusive lock waits (`filesystem_retention_snapshot_tests`) | Implemented | | `KEEP-RETENTION-009` | Exact already-committed retry is idempotent only while its successor remains current | byte-identical planning in `tests/retention_transition.rs`; authority-revalidated zero-mutation retry receipt in `tests/retention_publication_execution.rs`; exact already-committed filesystem retry with a byte-identical retention witness in `filesystem_retention_storage_tests`; superseded-candidate filesystem refusal with zero mutation in `filesystem_retention_successor_tests`; committed retry reopens the head-selected manifest entry and root pool bytes, refusing absent, changed, or corrupt evidence in `filesystem_retention_current_tests`; every refusal is a typed `RetentionCurrentStateRefusal` source, with superseded, committed-root-absent, committed-root-changed, and head-absent-with-artifacts pinned by downcast | Implemented | | `KEEP-RETENTION-010` | Model operation sequences agree with a deterministic namespace-to-anchor-set map and never admit caller identity, paths, clocks, or application policy | every three-operation sequence over initial publications of two namespaces, a successor, a byte-identical retry, and a stale initial (125 sequences, each in a fresh migrated store) agrees with a deterministic namespace-to-(generation, anchor-set) map plus liveness after every step, observed through the fenced reader view, in `retention_model_tests`; `tests/retention_core_architecture_contract.rs` refuses any clock, path, environment, or identity token in the retention core | Implemented | diff --git a/docs/formats/segment-store-v2/retention-publication.md b/docs/formats/segment-store-v2/retention-publication.md index d432c0fd..e696fa2c 100644 --- a/docs/formats/segment-store-v2/retention-publication.md +++ b/docs/formats/segment-store-v2/retention-publication.md @@ -20,9 +20,9 @@ the current manifest and derives exact canonical successors. ordered durability phases, and returns the complete receipt only after cleanup; exact already-committed retry revalidates authority and performs no mutation. -Version-2 catalog publication holds the same writer authority and proves every -current retained closure against its candidate catalog before replacing the -catalog `HEAD`. +Version-2 catalog publication must hold the same writer authority and prove every current retained closure against its candidate catalog before replacing the catalog `HEAD`. + +That general candidate-catalog gate remains unimplemented and is tracked in [#125](https://github.com/flyingrobots/keep/issues/125). The implemented retention publication and complete-stage retention recovery paths reverify a root's closure against the current catalog; this does not establish that an arbitrary successor catalog preserves every retained root. ## Generation transition diff --git a/docs/testing-evidence/current-durable-surfaces.md b/docs/testing-evidence/current-durable-surfaces.md new file mode 100644 index 00000000..ab5a7a86 --- /dev/null +++ b/docs/testing-evidence/current-durable-surfaces.md @@ -0,0 +1,27 @@ +# Current durable surface documentation (#130) + +Change kind: documentation correction; no runtime, public signature, identifier, codec, format byte, or test expectation changes. + +## Source boundary + +This reconciliation uses main integration `6051abb25a9fd33ae7ee0de5614514b709a4d82a`, which merged PR #99. Prepared implementation on `feature/roadmap-m4-tasks` and unmerged audit PRs do not establish delivery on this baseline. The source links below locate each claim; this is a reviewed documentation record, not a source-string assertion masquerading as runtime evidence. + +| Claim | Owning source or evidence | Documentation disposition | +| --- | --- | --- | +| Migration can resume admitted residue under writer authority | [Migration recovery](../../src/adapters/store_migration/filesystem_migration_recovery.rs), [requirements](../formats/segment-store-v2/requirements.md#migration) | Preserve the implemented migration claim and the separate #111/#112 coverage gaps. | +| Retention recovery executes complete-stage plans and preserves incomplete stages | [Filesystem recovery](../../src/adapters/retention/filesystem_retention_recovery.rs), [planner](../../src/adapters/retention/recovery_planner.rs), [accepted landing evidence](retention-landing.md) | Remove obsolete absence and pending-#99 claims; keep incomplete-stage disposition deferred to #155. | +| Execution failure is distinct from pre-effect refusal | [Execution](../../src/adapters/retention/recovery_execution.rs), [progress](../../src/adapters/retention/retention_storage_progress.rs), [contract](../formats/segment-store-v2/retention-recovery.md) | Preserve typed causes, known/uncertain effects, and durability distinctions. | +| Retention snapshots bind a fenced view and admit selected roots | [Snapshot](../../src/adapters/retention/filesystem_retention_snapshot.rs), [collector](../../src/adapters/retention/retention_view_collector.rs) | Describe the delivered surface without promising the absent durable authenticated read API (#109). | +| Forward retention publication and complete-root recovery reverify live closure | [Closure admission](../../src/adapters/retention/filesystem_retention_closure_admission.rs), [forward verification](../../src/adapters/retention/filesystem_retention_catalog.rs) | Remove the stale planned member-reverification claim. | +| General candidate-catalog admission must preserve every retained root | [Catalog preflight](../../src/adapters/filesystem_catalog_current.rs), [catalog storage](../../src/adapters/filesystem_catalog_storage.rs), [publication requirement](../formats/segment-store-v2/retention-publication.md#closure-admission) | Keep the normative requirement; explicitly identify its implementation gap in #125. Current-root verification against the current catalog does not satisfy this different requirement. | +| Durable reads, reports, ingestion, GC, and compaction are not delivered on this baseline | [Crate public surface](../../src/lib.rs), [GC ledger](../formats/segment-store-v2/requirements.md#garbage-collection-reservation) | Track #109, #114, #82 and #21; do not import PR #107's prepared implementation claims. | + +The managed-namespace concurrency contract remains the [accepted decision](../adr/retention-bounded-recovery-landing.md): cooperating writers under Keep authority, without a guarantee against arbitrary concurrent raw namespace mutation. + +## Validation and limitations + +The oracle is correspondence between public documentation and the source at the named integration, reviewed at each owning boundary above. Documentation integrity/refusal/link checks, Markdown lint, rustdoc generation and doctests validate document structure, references and compiling examples. Formatting and Clippy remain required. Their success does not prove storage behavior. + +No runtime assertion was introduced or materially changed, so RED-on-parent, runtime mutation calibration, fault injection, generated-case reduction and new test resource ceilings are not applicable to this documentation-only change under the [enforcement profile](../testing/enforcement.md). Existing runtime evidence remains attributed to its original source and scope; this PR does not rerun or expand it. The final PR receipt records commands, source SHA and actual check results. + +No acceptance criterion is reduced. No original ROADMAP task is newly checked. The #130 request for a source-based documentation regression is satisfied by this explicit claim/source reconciliation and existing documentation checks; a permanent test searching for prose or implementation tokens would assert document structure, not Keep runtime behavior. diff --git a/src/lib.rs b/src/lib.rs index 1127d095..fa21b2b5 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -40,8 +40,21 @@ //! namespace transitions while retaining version-1 immutable bytes. //! Partial-prefix recovery now plans and resumes lawful migration residue, //! returning typed refusals and an ordered execution receipt. Filesystem -//! retention publication is available; retention restart recovery, immutable -//! reader snapshots, and garbage collection remain absent. +//! retention publication and complete-stage restart recovery are available. +//! +//! Incomplete retention stages require explicit disposition before recovery +//! effects; automatic disposal remains deferred. Execution failures preserve +//! their typed cause and distinguish known effects from uncertain effects or +//! durability. Writer authority coordinates cooperating writers in a managed +//! namespace; it does not isolate arbitrary out-of-band filesystem mutation. +//! +//! [`FilesystemRetentionSnapshot`] holds a shared reader fence while binding +//! one catalog, retention head, and manifest view, and verifies selected roots +//! on demand. This is retention evidence, not a durable BlobId-to-writer read +//! API. Durable authenticated reads, durable verification reports, production +//! ingestion, garbage collection, and compaction remain unimplemented here. +//! General version-2 catalog publication also still needs a retained-closure +//! gate; retention publication's live closure verification does not supply it. #[cfg(test)] extern crate self as keep; From 35f8ba09c38640e31045ff2b4f8fe3eff7d0d876 Mon Sep 17 00:00:00 2001 From: James Ross Date: Sat, 3 Oct 2026 16:28:57 -0700 Subject: [PATCH 2/3] docs: reconcile linked #109 and #114 delivery records (#130) --- docs/audits/114-durable-verification-scope.md | 6 +++++- docs/invariants/authenticated-reconstruction/README.md | 2 +- .../invariants/authenticated-reconstruction/requirements.md | 6 +++--- docs/invariants/verification/README.md | 2 +- docs/invariants/verification/requirements.md | 2 +- docs/testing-evidence/durable-authenticated-reads.md | 4 +++- docs/testing-evidence/durable-verification.md | 4 +++- 7 files changed, 17 insertions(+), 9 deletions(-) diff --git a/docs/audits/114-durable-verification-scope.md b/docs/audits/114-durable-verification-scope.md index 3d2fc0f4..419734cd 100644 --- a/docs/audits/114-durable-verification-scope.md +++ b/docs/audits/114-durable-verification-scope.md @@ -1,6 +1,10 @@ # Durable verification landing scope -Status: implementation candidate for [#114](https://github.com/flyingrobots/keep/issues/114), under verification parent [#20](https://github.com/flyingrobots/keep/issues/20). +Status: implemented on main for [#114](https://github.com/flyingrobots/keep/issues/114), under verification parent [#20](https://github.com/flyingrobots/keep/issues/20). + +Delivered in [PR #165](https://github.com/flyingrobots/keep/pull/165), merged as `2efc131e8466b458088eaf5de0a5981e636d8f85`. [Independent review](https://github.com/flyingrobots/keep/pull/165#issuecomment-5974504658), [Code Lawyer closure](https://github.com/flyingrobots/keep/pull/165#issuecomment-5974510981) and all four [candidate checks](https://github.com/flyingrobots/keep/actions/runs/37160522753) cover exact head `1f3991f86fa66783d88b9ac8dbb79ecd0d9a9554`; the signed merge preserves that tree. + +The sections below preserve chronological implementation and review records. Earlier pending gates and open-finding tables describe their named intermediate heads; final closure above supersedes those statuses without expanding their historical evidence claims. This ledger reconciles the requested verification outcome with the code available at the branch baseline; current closure dispositions and their evidence are recorded below. diff --git a/docs/invariants/authenticated-reconstruction/README.md b/docs/invariants/authenticated-reconstruction/README.md index 2d9e8e29..7030352f 100644 --- a/docs/invariants/authenticated-reconstruction/README.md +++ b/docs/invariants/authenticated-reconstruction/README.md @@ -1,6 +1,6 @@ # Authenticated Reconstruction Contract -**Status:** Normative for every Keep operation that claims authenticated reconstruction. The public non-durable `ReferenceStore` implements the complete-object and exact-range forms. Linux `DurableStore` and `DurableSnapshot` compose those read cores with fenced version-two catalog and retained-root admission; final #109 acceptance remains recorded in the [evidence ledger](../../testing-evidence/durable-authenticated-reads.md). +**Status:** Normative for every Keep operation that claims authenticated reconstruction. The public non-durable `ReferenceStore` implements the complete-object and exact-range forms. Linux `DurableStore` and `DurableSnapshot` compose those read cores with fenced version-two catalog and retained-root admission; delivery through merged [PR #164](https://github.com/flyingrobots/keep/pull/164) and its #109 acceptance evidence are recorded in the [evidence ledger](../../testing-evidence/durable-authenticated-reads.md). The [rationale](rationale.md) records the governed decisions and rejected alternatives. The [requirement ledger](requirements.md) maps each law to its diff --git a/docs/invariants/authenticated-reconstruction/requirements.md b/docs/invariants/authenticated-reconstruction/requirements.md index ee08a7a6..3be1f9f6 100644 --- a/docs/invariants/authenticated-reconstruction/requirements.md +++ b/docs/invariants/authenticated-reconstruction/requirements.md @@ -14,7 +14,7 @@ gap, not implementation evidence. | `KEEP-RECONSTRUCT-006` | Authenticated success, evidenced content refusal, and operational failure remain distinct outcomes; operational failure supports no content conclusion. | Typed outcome classification | Public API integration tests and contract inspection | Implemented typed outcomes; persisted refusal receipts remain separate | `tests/streaming_cas/refusal_laws.rs`, `tests/range_read_failures.rs`, `tests/golden_file_worldline/durable_refusal_laws.rs`, `tests/golden_file_worldline/durable_writer_failures.rs` | | `KEEP-RECONSTRUCT-007` | Whole-object and range receipts bind target, exact layout, proof scope, and exact emitted coordinates without granting retention or application authority. | Receipt type inspection | Public API contract tests | Implemented | `src/authenticated_read/reconstruction_receipt.rs`, `src/authenticated_read/range_read_receipt.rs`, `tests/range_read_contract.rs` | | `KEEP-RECONSTRUCT-008` | Automatic layout choice is deterministic; an exact requested layout never falls back. | Canonically ordered layout set | Unit and public API integration tests | Implemented | `src/reference/store_tests.rs`, `tests/streaming_cas/reconstruction_laws.rs` | -| `KEEP-RECONSTRUCT-009` | A durable read pins one immutable view and prevents required evidence from being garbage-collected, deleted, or invalidated through completion. | Pinned-generation and retained-closure model | Recovery, concurrency, corruption, and crash-injection tests | Implemented for cooperating managed-store operations; final #109 acceptance pending | `src/adapters/retention/durable_view_law_tests.rs`, `tests/golden_file_worldline/durable_assertions.rs`; [scope and evidence](../../testing-evidence/durable-authenticated-reads.md) | -| `KEEP-RECONSTRUCT-010` | Durable reconstruction names its view and returns either authenticated success, evidenced refusal, or operational failure without hidden whole-blob allocation. | Durable consumer conformance model | Public API integration, memory, recovery, and crash-injection tests | Implemented; final #109 acceptance pending | `tests/golden_file_worldline/durable_read_memory.rs`, `tests/golden_file_worldline/durable_corruption_laws.rs`, `tests/golden_file_worldline/durable_output_laws.rs`; [scope and evidence](../../testing-evidence/durable-authenticated-reads.md) | +| `KEEP-RECONSTRUCT-009` | A durable read pins one immutable view and prevents required evidence from being garbage-collected, deleted, or invalidated through completion. | Pinned-generation and retained-closure model | Recovery, concurrency, corruption, and crash-injection tests | Implemented for cooperating managed-store operations; delivered through merged #164 | `src/adapters/retention/durable_view_law_tests.rs`, `tests/golden_file_worldline/durable_assertions.rs`; [scope and evidence](../../testing-evidence/durable-authenticated-reads.md) | +| `KEEP-RECONSTRUCT-010` | Durable reconstruction names its view and returns either authenticated success, evidenced refusal, or operational failure without hidden whole-blob allocation. | Durable consumer conformance model | Public API integration, memory, recovery, and crash-injection tests | Implemented; delivered through merged #164 | `tests/golden_file_worldline/durable_read_memory.rs`, `tests/golden_file_worldline/durable_corruption_laws.rs`, `tests/golden_file_worldline/durable_output_laws.rs`; [scope and evidence](../../testing-evidence/durable-authenticated-reads.md) | -The durable implementation status describes the current candidate, not a merged release or completed independent review. Snapshot admission materializes catalog-selected segment bytes under the caller's aggregate segment-byte limit; catalog bytes have a separate format maximum, while decoded indexes and retention records allocate additionally under format and record-count limits. The segment limit is not a total snapshot-memory cap; the memory witness measures additional reconstruction allocation. Managed-namespace cooperation is required for the fence guarantee. The #109 ledger distinguishes fresh reopen from process death and kernel exclusion from absent production GC. +The durable implementation is delivered on main through [PR #164](https://github.com/flyingrobots/keep/pull/164), with exact-head independent review and validation recorded in the linked evidence; mainline delivery is not a packaged-release claim. Snapshot admission materializes catalog-selected segment bytes under the caller's aggregate segment-byte limit; catalog bytes have a separate format maximum, while decoded indexes and retention records allocate additionally under format and record-count limits. The segment limit is not a total snapshot-memory cap; the memory witness measures additional reconstruction allocation. Managed-namespace cooperation is required for the fence guarantee. The #109 ledger distinguishes fresh reopen from process death and kernel exclusion from absent production GC. diff --git a/docs/invariants/verification/README.md b/docs/invariants/verification/README.md index 27db61b3..7fcd8b13 100644 --- a/docs/invariants/verification/README.md +++ b/docs/invariants/verification/README.md @@ -1,6 +1,6 @@ # Verification reports -Status: durable verification candidate under [#114](https://github.com/flyingrobots/keep/issues/114); final acceptance is tracked in the [closure ledger](../../audits/114-durable-verification-scope.md). +Status: implemented on main through [PR #165](https://github.com/flyingrobots/keep/pull/165) for [#114](https://github.com/flyingrobots/keep/issues/114); final acceptance and historical scope are recorded in the [closure ledger](../../audits/114-durable-verification-scope.md). ## Contract diff --git a/docs/invariants/verification/requirements.md b/docs/invariants/verification/requirements.md index c927c507..276720f9 100644 --- a/docs/invariants/verification/requirements.md +++ b/docs/invariants/verification/requirements.md @@ -4,4 +4,4 @@ The [original task](../../audits/114-durable-verification-scope.md) remains auth | ID | Requirement | Status | Evidence and remaining work | | --- | --- | --- | --- | -| `KEEP-VERIFY-006` | Durable verification at explicit subject-specific achieved depths, with precise refusals, immutable reports and bounded costs. | Implemented on the PR branch | The candidate implements subject-specific catalog, segment, record, blob and retained-namespace reports, typed durable ingress outcomes and bounded conflict evidence; focused runtime/calibration and catalog-ceiling evidence are recorded. Exact-head validation, independent acceptance and mainline integration are recorded on [PR #165](https://github.com/flyingrobots/keep/pull/165); implementation does not imply merged delivery. See the [closure ledger](../../audits/114-durable-verification-scope.md) and [execution evidence](../../testing-evidence/durable-verification.md). | +| `KEEP-VERIFY-006` | Durable verification at explicit subject-specific achieved depths, with precise refusals, immutable reports and bounded costs. | Implemented on main through #165 | The implementation provides subject-specific catalog, segment, record, blob and retained-namespace reports, typed durable ingress outcomes and bounded conflict evidence; focused runtime/calibration and catalog-ceiling evidence are recorded. Exact-head validation, independent acceptance and mainline integration are recorded on [PR #165](https://github.com/flyingrobots/keep/pull/165); the signed merge preserves the independently reviewed candidate tree. See the [closure ledger](../../audits/114-durable-verification-scope.md) and [execution evidence](../../testing-evidence/durable-verification.md). | diff --git a/docs/testing-evidence/durable-authenticated-reads.md b/docs/testing-evidence/durable-authenticated-reads.md index b8bb17db..4f2fbc9b 100644 --- a/docs/testing-evidence/durable-authenticated-reads.md +++ b/docs/testing-evidence/durable-authenticated-reads.md @@ -1,6 +1,8 @@ # Durable authenticated reads (#109) -Change kind: new read API with a shared-core extraction. The baseline is main `6051abb25a9fd33ae7ee0de5614514b709a4d82a`. No write protocol or on-disk format is changed; #125's candidate-catalog publication gate is not added or bypassed. This is an in-progress implementation ledger, not acceptance of #109. +Change kind: new read API with a shared-core extraction. The baseline is main `6051abb25a9fd33ae7ee0de5614514b709a4d82a`. No write protocol or on-disk format is changed; #125's candidate-catalog publication gate is not added or bypassed. The sections below preserve chronological implementation and review evidence; earlier pending gates describe their historical heads and are superseded by this final landing receipt. + +Delivered in [PR #164](https://github.com/flyingrobots/keep/pull/164), merged as `1079551bc6b331eb9847823e7d22b22ea4c47b62`. [Independent review](https://github.com/flyingrobots/keep/pull/164#issuecomment-5974246110), [Code Lawyer closure](https://github.com/flyingrobots/keep/pull/164#issuecomment-5974261856), all four [candidate checks](https://github.com/flyingrobots/keep/actions/runs/37158962636) and [post-merge checks](https://github.com/flyingrobots/keep/actions/runs/37159373928) pass; reviewed head `9d19e2e0c3184efd5bc08c1f8cc12edd15421a93` and signed merge have the same tree. ## Review after draft readiness diff --git a/docs/testing-evidence/durable-verification.md b/docs/testing-evidence/durable-verification.md index cc662653..15355e58 100644 --- a/docs/testing-evidence/durable-verification.md +++ b/docs/testing-evidence/durable-verification.md @@ -1,6 +1,8 @@ # Durable verification evidence -Status: the scoped implementation and post-readiness review corrections for [#114](https://github.com/flyingrobots/keep/issues/114) are implemented on the PR branch. Final exact-head independent review and required checks remain acceptance gates in the [scope ledger](../audits/114-durable-verification-scope.md); mainline delivery is not claimed. The sections below preserve chronological slice evidence, including superseded intermediate limitations. +Status: Delivered in [PR #165](https://github.com/flyingrobots/keep/pull/165), merged as `2efc131e8466b458088eaf5de0a5981e636d8f85`. [Independent review](https://github.com/flyingrobots/keep/pull/165#issuecomment-5974504658), [Code Lawyer closure](https://github.com/flyingrobots/keep/pull/165#issuecomment-5974510981) and all four [candidate checks](https://github.com/flyingrobots/keep/actions/runs/37160522753) cover exact head `1f3991f86fa66783d88b9ac8dbb79ecd0d9a9554`; the signed merge preserves that tree. + +The sections below preserve chronological slice and intermediate review evidence. Earlier pending acceptance statements describe those historical heads and are superseded by the final landing receipt above; no broader runtime or power-loss guarantee is inferred. ## Catalog report slice From 26c32d05038c7a3a38eda7cb2d259177013f5bf8 Mon Sep 17 00:00:00 2001 From: James Ross Date: Sat, 3 Oct 2026 16:33:21 -0700 Subject: [PATCH 3/3] docs: record merged migration evidence in current ledger (#130) --- docs/formats/segment-store-v2/requirements.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/formats/segment-store-v2/requirements.md b/docs/formats/segment-store-v2/requirements.md index 2aae90e5..09057b01 100644 --- a/docs/formats/segment-store-v2/requirements.md +++ b/docs/formats/segment-store-v2/requirements.md @@ -32,7 +32,7 @@ case is not evidence. | `KEEP-MIGRATION-002` | Format marker, intent, and receipt have complete fixed byte tables, named domains, bounds, checksums, deterministic store identity, and exact initial-state digests | exact admission in `tests/store_format_marker.rs`, `tests/store_migration_intent.rs`, and `tests/store_migration_receipt.rs`; canonical construction in `tests/store_migration_intent_encoding.rs` and `tests/store_migration_receipt_encoding.rs`; seeded `migration_format` fuzz target | Implemented | | `KEEP-MIGRATION-003` | Migration revalidates version-1 head, catalog, pools, root identity (all three coordinates within the migrating process; device and file across restart), and writer authority before mutation | bounded canonical pool inventory in `tests/store_migration_inventory.rs`; writer-locked filesystem pool admission in `filesystem_inventory_*_tests`; exact authority observation and drift refusal in `filesystem_migration_authority_tests`; verification-first execution in `tests/store_migration_execution.rs`; fresh filesystem integration and post-publication drift refusal in `filesystem_migration_storage_tests`; a version-one store still holding a retained stage refuses before the intent is observed in `filesystem_migration_storage_tests` | Implemented | | `KEEP-MIGRATION-004` | Every partial migration prefix continues idempotently under writer authority, comparing only the restart-stable root coordinates (device and file) against the persisted intent | restart-stable coordinate laws in `filesystem_version_two_admission_tests` and complete reopen laws in `filesystem_migration_remount_tests`; `tests/store_migration_recovery.rs` and `tests/store_migration_recovery_order.rs` freeze planner and ordering laws; `src/adapters/store_migration/filesystem_migration_recovery_tests.rs` covers every forward prefix; `src/adapters/store_migration/filesystem_migration_recovery_truncation_tests.rs` covers every strict fixed-stage truncation | Implemented in #108 | -| `KEEP-MIGRATION-005` | Unknown, out-of-order, substituted, corrupt, conflicting, or changed evidence is unrecoverable ambiguity | Forward-execution laws remain. Filesystem restart record, pair, ordering, root, pool and namespace laws assert exact existing refusal boundaries and preserve complete names, device/inode identities and bytes; see [restart matrix](../../testing-evidence/migration-restart-matrix.md) for scenarios, calibration, diagnostic limits and validation ownership. | Implemented in #111 candidate, including reserved complete-state namespace refusal; final review and integration pending | +| `KEEP-MIGRATION-005` | Unknown, out-of-order, substituted, corrupt, conflicting, or changed evidence is unrecoverable ambiguity | Forward-execution laws remain. Filesystem restart record, pair, ordering, root, pool and namespace laws assert exact existing refusal boundaries and preserve complete names, device/inode identities and bytes; see [restart matrix](../../testing-evidence/migration-restart-matrix.md) for scenarios, calibration, diagnostic limits and validation ownership. | Evidence delivered in merged PR #161 (#111), including complete-state namespace refusal; see the restart matrix for bounded coverage and separate diagnostic obligations | | `KEEP-MIGRATION-006` | Migration never rewrites or deletes admitted version-1 immutable bytes | exact segment, catalog, and head witnesses in `src/adapters/store_migration/filesystem_migration_storage_tests.rs`, `src/adapters/store_migration/filesystem_migration_recovery_tests.rs`, and `src/adapters/store_migration/filesystem_migration_recovery_truncation_tests.rs`; subprocess restart witnesses in `cargo xtask durability-crash-matrix --sequence migration` | Implemented in #108 | | `KEEP-MIGRATION-007` | Process death around every intent stage, canonical link, namespace prefix, marker stage, receipt stage, cleanup, and synchronization boundary reaches a documented lawful state | ordered phases and capabilities in `tests/store_migration_phase.rs` and `tests/store_migration_storage.rs`; exact phase-failure execution in `tests/store_migration_execution.rs`; production 21-phase forward execution in `filesystem_migration_storage_tests`; `cargo xtask durability-crash-matrix --sequence migration` runs 68 production subprocess cases at `KEEP-CRASH-053..=073`, debug and release | Implemented in #108 | | `KEEP-MIGRATION-008` | Version-1 admission refuses every version-2 or partial-migration artifact after migration begins | Every forward-prefix compatibility law preserves v1 HEAD/catalog/segment bytes and refuses v1 authority after migration effects; public decoder laws pin unsupported versions and every mandatory flag bit; bounded seeded migration parser/recovery-planner fuzzing explores valid and malformed transitions. Existing jointly bound version-two admission and root-identity laws remain. See [compatibility evidence](../../testing-evidence/migration-compatibility-fuzz.md) for coverage, calibration and limits. | Implemented |