Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ after its public API and format compatibility policies are established.

## [Unreleased]

- Current durable-surface documentation distinguishes delivered recovery, fenced authenticated reads and explicit verification from pending ingestion, GC, compaction and general candidate-catalog retained-closure admission (#130).

- Retention verification refuses observed file or symlink substitutions of a selected namespace directory as typed corruption, preserving the original selected root evidence (#114).

- Verification preserves typed canonical-namespace and no-follow entry-kind contradictions as corruption while leaving inconclusive observation failures operational (#114).
Expand Down
24 changes: 16 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,15 +73,25 @@ Keep is required to refuse all three, before mutating anything.
is discarded and rebuilt from freshly verified current intent. Its crash
matrix kills real writer processes
at 68 before/during/after coordinates (`KEEP-CRASH-053`–`073`), preserving
every version-1 byte. Broader hostile restart combinations remain in #111.
every version-1 byte. The [restart matrix](docs/testing-evidence/migration-restart-matrix.md) records additional hostile-prefix evidence and its limits.

## What it does not do yet

Version-2 retention recovery, fenced reader snapshots and model-based transitions are implemented in this branch but still require the correctness corrections and independent acceptance review tracked in PR #99.
Version-2 complete-stage retention recovery, fenced retention snapshots and model-based transitions are implemented on main through [PR #99](https://github.com/flyingrobots/keep/pull/99).

The retention process-death sequence checks the recovered head generation and exact selected-root bytes before retry; its [evidence receipt](docs/testing-evidence/retention-crash-reader-oracle.md) bounds that claim to the declared initial-publication crash coordinates.

Incomplete retention stages are preserved and block publication pending explicit disposition; automatic disposal is deferred. The [landing ledger](docs/testing-evidence/retention-landing.md) tracks execution-failure reporting and final acceptance under the [approved recovery contract](docs/formats/segment-store-v2/retention-recovery.md).
Recovery failures retain their typed cause and report known effects separately from uncertain effects or durability under the [approved recovery contract](docs/formats/segment-store-v2/retention-recovery.md).

Writer authority coordinates cooperating writers in a managed namespace; it does not isolate arbitrary concurrent out-of-band filesystem mutation.

`DurableStore` supplies fenced authenticated whole-blob reconstruction and exact-range reads, delivered through [PR #164](https://github.com/flyingrobots/keep/pull/164); its [read contract](docs/invariants/authenticated-reconstruction/README.md) distinguishes complete-blob and range evidence and separate allocation limits.

[Explicit verification reports](docs/invariants/verification/README.md) name the subject, requested depth and evidence actually established, delivered through [PR #165](https://github.com/flyingrobots/keep/pull/165). Unsupported depths refuse; a report grants no live retention authority.

[Reader-fence process-death evidence](docs/testing-evidence/reader-fence-process.md) verifies the lock lifecycle and preserved fence bytes; it is not physical power-loss evidence.

## What it does not do yet

Incomplete retention stages are preserved and block publication pending explicit disposition; automatic disposal remains deferred in [#155](https://github.com/flyingrobots/keep/issues/155).

Complete orphans remain recovery-protected until explicit disposition lands with garbage collection (#21).

Expand All @@ -91,9 +101,7 @@ A version-1 store stays admitted until its owner migrates it.

| Gap | Tracked |
| --- | --- |
| Retention recovery correctness remediation and broader migration corruption coverage | [PR #99](https://github.com/flyingrobots/keep/pull/99), [#111](https://github.com/flyingrobots/keep/issues/111) |
| Fenced reader correctness remediation and independent acceptance | [PR #99](https://github.com/flyingrobots/keep/pull/99) |
| Precise verification reports at explicit depths | [#20](https://github.com/flyingrobots/keep/issues/20) |
| Candidate-catalog preservation of every retained closure | [#125](https://github.com/flyingrobots/keep/issues/125) |
| Garbage collection and identity-preserving compaction | [#21](https://github.com/flyingrobots/keep/issues/21) |
| Bounded production ingestion through the durable store | [#82](https://github.com/flyingrobots/keep/issues/82) |
| Encrypted representations | [#86](https://github.com/flyingrobots/keep/issues/86) |
Expand Down
6 changes: 5 additions & 1 deletion docs/audits/114-durable-verification-scope.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
# Durable verification landing scope

Status: implementation candidate for [#114](https://github.com/flyingrobots/keep/issues/114), under verification parent [#20](https://github.com/flyingrobots/keep/issues/20).
Status: implemented on main for [#114](https://github.com/flyingrobots/keep/issues/114), under verification parent [#20](https://github.com/flyingrobots/keep/issues/20).

Delivered in [PR #165](https://github.com/flyingrobots/keep/pull/165), merged as `2efc131e8466b458088eaf5de0a5981e636d8f85`. [Independent review](https://github.com/flyingrobots/keep/pull/165#issuecomment-5974504658), [Code Lawyer closure](https://github.com/flyingrobots/keep/pull/165#issuecomment-5974510981) and all four [candidate checks](https://github.com/flyingrobots/keep/actions/runs/37160522753) cover exact head `1f3991f86fa66783d88b9ac8dbb79ecd0d9a9554`; the signed merge preserves that tree.

The sections below preserve chronological implementation and review records. Earlier pending gates and open-finding tables describe their named intermediate heads; final closure above supersedes those statuses without expanding their historical evidence claims.

This ledger reconciles the requested verification outcome with the code available at the branch baseline; current closure dispositions and their evidence are recorded below.

Expand Down
40 changes: 28 additions & 12 deletions docs/formats/segment-store-v2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,11 @@ catalog, and publication-head byte while adding explicit retention state,
reader fences, migration evidence, and reserved GC and recovery-disposition
namespaces.

ADR-0009 owns the cross-cutting retention and liveness decision. These pages own its durable representation. The one-way migration, version-two reopen, forward retention publication, partial-prefix migration recovery, and the 68-case migration process-death matrix are implemented with executable evidence. Retention recovery and reader fencing are implemented in this branch; correctness remediation and independent acceptance remain tracked in PR #99. Collection remains planned in #21. The [requirements ledger](requirements.md) records requirements and their evidence status. A version-1 store remains admitted until its owner migrates it.
ADR-0009 owns the cross-cutting retention and liveness decision. These pages own its durable representation.

The one-way migration, version-two reopen, forward retention publication, partial-prefix migration recovery, complete-stage retention recovery, and fenced retention snapshots are implemented on main through [PR #99](https://github.com/flyingrobots/keep/pull/99).

The [requirements ledger](requirements.md) records evidence and remaining obligations; the laws below are normative requirements, not a claim that every corresponding runtime surface exists.

## Core laws

Expand Down Expand Up @@ -89,19 +93,31 @@ head and the catalog it selects, and refuses superseded candidates, retained
stages, replaced protocol directories, and every namespace or capacity
violation before mutation, each as a typed `RetentionCurrentStateRefusal`.

Retention publication recovery, fenced reader snapshots and model-based transition evidence are implemented in this branch.
Retention publication recovery, fenced retention snapshots and model-based transition evidence are implemented on main.

The [landing ledger](../../testing-evidence/retention-landing.md) records the accepted scope and evidence for merged PR #99.

Incomplete retention stages are preserved pending explicit disposition; automatic disposal remains deferred in [#155](https://github.com/flyingrobots/keep/issues/155).

Execution failures preserve the typed cause and report known effects separately from uncertain effects or durability, as specified by the [retention recovery contract](retention-recovery.md).

Writer authority coordinates cooperating writers in a managed namespace; it does not isolate arbitrary out-of-band filesystem mutation or make pathname unlink conditional on inode identity.

Their bounded landing and independent acceptance remain tracked in PR #99 and the [landing ledger](../../testing-evidence/retention-landing.md). Incomplete retention stages are preserved pending explicit disposition; automatic disposal is deferred. Execution-failure reporting remains part of the [retention recovery contract](retention-recovery.md).
`FilesystemRetentionSnapshot` binds a catalog, retention head and manifest under a shared reader fence and verifies selected roots on demand. `DurableStore` composes that view with authenticated whole-blob and exact-range reads, delivered in [PR #164](https://github.com/flyingrobots/keep/pull/164); the [read contract](../../invariants/authenticated-reconstruction/README.md) records proof and allocation limits.

General version-two catalog publication still needs the candidate-catalog retained-closure admission gate tracked in [#125](https://github.com/flyingrobots/keep/issues/125); verifying a new retention root against the current catalog is a different operation.

[Durable verification reports](../../invariants/verification/README.md) are delivered in [PR #165](https://github.com/flyingrobots/keep/pull/165), with explicit subject/depth evidence and typed non-success outcomes.

Production durable ingestion ([#82](https://github.com/flyingrobots/keep/issues/82)), garbage collection and compaction ([#21](https://github.com/flyingrobots/keep/issues/21)) remain absent from main; those prepared portions of unmerged PR #107 are not delivered APIs here.

The `KEEP-CRASH-036..052` initial-publication process-death sequence includes independent recovered-reader checks; its [evidence receipt](../../testing-evidence/retention-crash-reader-oracle.md) records the assertions, calibration, and scope.

Partial-prefix migration recovery and the 68-case `KEEP-CRASH-053..073`
process-death matrix are implemented. Broader migration restart corruption
and compatibility coverage remain in #111 and #112; issue #21 owns garbage
collection. Reopen compares only the restart-stable root coordinates, device
and inode, against the intent; see
[root identity across restart](recovery.md#root-identity-across-restart). A
version-1 store
remains admitted until its owner migrates it, and the
[requirements ledger](requirements.md) is the authority on which requirements
are proven.
process-death matrix are implemented.

The [migration restart matrix](../../testing-evidence/migration-restart-matrix.md) and [compatibility and fuzz evidence](../../testing-evidence/migration-compatibility-fuzz.md) record the additional merged #111/#112 evidence and its remaining limits.
Comment thread
flyingrobots marked this conversation as resolved.

Reopen compares restart-stable root coordinates, device and inode, against the intent; see [root identity across restart](recovery.md#root-identity-across-restart).

A version-1 store remains admitted until its owner migrates it. The [requirements ledger](requirements.md) records implementation and evidence gaps; planned cases are not proof.
6 changes: 1 addition & 5 deletions docs/formats/segment-store-v2/closure.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,6 @@
# Closure Verification

- Status: Normative version-2 protocol; storage-independent verifier
implemented; publication binds this store's catalog head and the catalog it
selects to the verified closure; member re-verification under filesystem
authority is planned in issue
[#19](https://github.com/flyingrobots/keep/issues/19)
- Status: Normative version-2 protocol; the storage-independent verifier and live member re-verification under filesystem retention authority are implemented. General candidate-catalog retained-closure admission remains in [#125](https://github.com/flyingrobots/keep/issues/125).
- Format coordinate: `keep.segment-store/v2`
- Requirement: [`KEEP-RETENTION-005`](requirements.md#retention-transitions)
- Decision record:
Expand Down
Loading
Loading