Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ after its public API and format compatibility policies are established.

## [Unreleased]

- Incomplete version-one segment seals now refuse observed fixed-framing corruption before recovery assessment can authorize discard, preserving precise seal diagnostics (#171).

- Retention recovery execution errors report the exact failed boundary, original typed cause, known namespace effects and uncertain effect/durability; retries freshly observe the store. Observed stage identity remains binding across reopening, and cleanup preserves verified pool evidence rather than promising the removed pathname survives (#99).

- Retention recovery now preserves incomplete stages and requires explicit disposition before any recovery mutation or publication retry; automatic incomplete-stage disposal is deferred by maintainer decision (#99).
Expand Down
2 changes: 1 addition & 1 deletion docs/formats/segment-store-v1/recovery.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ only regular files, and refuses entry replacement or length drift after
reading. `classify_recovery_segment_stage` classifies complete caller-supplied
stage bytes as a validated reusable prefix, a complete admitted segment, or an
exact truncation only when every available segment- or record-header framing
byte remains canonical. It preserves proven partial-framing and
byte remains canonical. Recognized incomplete seals also validate available fixed version, flags, seal length, algorithm and reserved fields before returning truncation; failures preserve their precise seal cause through `RecoverySegmentStageError::Seal`. This does not prove future completion feasibility for every remaining coordinate. It preserves proven partial-framing and
complete-looking corruption as typed refusals. Catalog- and
next-head-stage classifiers apply the same available-fixed-framing rule before
distinguishing exact truncation from complete canonical bytes. Every
Expand Down
2 changes: 1 addition & 1 deletion docs/formats/segment-store-v1/requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ device fault evidence.
| `KEEP-RECOVERY-007` | Name classification requires the four initialized root entries, admits only fixed protocol names and canonical pool coordinates in their owning namespaces, refuses simultaneous fixed recovery stages before artifact reads, and moves a refused raw name without duplicating its allocation | Canonical-name matrix and allocation counter | `tests/recovery_name_classification.rs`, `tests/recovery_name_classification_memory.rs` | Implemented in #17 |
| `KEEP-RECOVERY-008` | Stage evidence is fingerprinted through a zero-allocation bounded streaming reader under the named recovery domain; metadata and observed bytes cannot exceed the name-selected protocol maximum, and failures retain exact stage and offset | Independent framing oracle, adversarial reader matrix, and allocation counter | `tests/recovery_stage_fingerprint.rs`, `tests/recovery_stage_fingerprint_memory.rs` | Implemented in #17 |
| `KEEP-RECOVERY-009` | Filesystem stage observation uses the pinned inventory capability, never follows a fixed-stage link, admits only regular files, and refuses entry replacement or length drift after bounded fingerprinting | Capability-relative replacement fixtures | `src/adapters/filesystem_recovery_stage_tests.rs` | Implemented in #17 |
| `KEEP-RECOVERY-010` | Whole-byte segment-stage classification distinguishes a validated reusable prefix, a complete admitted immutable segment, and exact header, record, or seal truncation only while every available fixed-framing byte remains canonical; proven partial-framing corruption, complete-looking corruption, duplicates, and resource-limit excess remain typed refusals | Exhaustive available-framing-byte, canonical prefix, and corruption matrix | `tests/recovery_segment_classification.rs`, `tests/recovery_segment_classification/*.rs` | Implemented in #17 |
| `KEEP-RECOVERY-010` | Whole-byte segment-stage classification distinguishes a validated reusable prefix, a complete admitted immutable segment, and exact header, record, or seal truncation only while every available fixed-framing byte remains canonical; proven partial-framing corruption, complete-looking corruption, duplicates, and resource-limit excess remain typed refusals | Exhaustive available-framing-byte, canonical prefix, and corruption matrix | `tests/recovery_segment_classification.rs`, `tests/recovery_segment_classification/*.rs`, `tests/recovery_partial_seal.rs`, `tests/recovery_partial_seal/*.rs` | Implemented in #17; partial-seal framing correction in #171 ([evidence](../../testing-evidence/partial-seal-corruption.md)) |
| `KEEP-RECOVERY-011` | Whole-byte catalog and next-head stage classification distinguishes exact fixed-header, declared-body, and fixed-width truncation from complete canonical bytes only while every available fixed-framing byte remains canonical; proven partial-framing corruption, complete-looking corruption, and oversize remain typed format or metadata refusals | Exhaustive available-framing-byte, canonical publication-artifact truncation, and corruption matrix | `tests/recovery_publication_stage_classification.rs`, `tests/recovery_publication_stage_classification/*.rs` | Implemented in #17 |
| `KEEP-RECOVERY-012` | Read-only semantic assessment admits materialized stage bytes only when the canonical-name stage, exact observed length, and `KEEP:RECOVERY:STAGE\0` fingerprint equal prior evidence, then dispatches through the name-selected segment, catalog, or next-head classifier | Evidence-binding mutation matrix and canonical stage assessments | `tests/recovery_stage_assessment.rs`, `tests/recovery_stage_assessment/*.rs` | Implemented in #17 |
| `KEEP-RECOVERY-013` | Explicit discard plans only from an exact truncation assessment, retains the observation evidence and typed truncation reason, refuses changed evidence without mutation, synchronizes the name-selected parent after exact removal or admitted absence, and returns a receipt only after synchronization | Truncation-planning, evidence-drift, operation-order, and retry matrix | `tests/recovery_stage_discard.rs`, `tests/recovery_stage_discard/*.rs` | Implemented in #17 |
Expand Down
51 changes: 51 additions & 0 deletions docs/testing-evidence/partial-seal-corruption.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Partial segment seal corruption

Change kind: bug fix for [#171](https://github.com/flyingrobots/keep/issues/171), under the T-13.2 audit. Owner: `@flyingrobots`; the author supplies execution evidence. The contract is KEEP-RECOVERY-010: demonstrated available fixed-framing corruption must refuse classification and assessment before a discard plan can be constructed.

## Parent RED and correction

Main `6051abb25a9fd33ae7ee0de5614514b709a4d82a` accepts a canonical one-zero segment truncated after seal version 2 as lawful truncation. The [parent RED receipt](partial-seal-corruption/parent-red.txt) records both public classifier and fingerprint-bound assessment regressions failing after successful compilation. The tests are committed separately at `1b6da984ece52e94777fe1b52dcf1bb6b82c3960`; run `cargo test --locked --test recovery_partial_seal` at that revision to reproduce the refusal gap.

The classifier now validates the available fixed fields of a recognized incomplete seal before returning `Truncated`. It preserves the exact `SegmentSealError` under `RecoverySegmentStageError::Seal`; assessment preserves that source. This is read-only slice classification, not a filesystem deletion experiment. The borrowed bytes are immutable by API construction; an equality assertion on the same untouched caller buffer would add no runtime evidence.

The finite sweep mutates each version, flags, embedded length, reserved and algorithm byte with a specified independent expected error, then visits every recognized incomplete seal length. A mutation beyond the observed end must remain a typed truncation; an observed contradiction must yield the exact specified refusal. The inputs and order are deterministic and finite, with offset/observed-length replay coordinates on failure. This is a fixed-framing family sweep, not arbitrary-byte coverage or proof of full future completion feasibility.

## Execution and current limits

[Focused and adjacent recovery GREEN](partial-seal-corruption/recovery-green.txt) covers the new public laws and existing classification, assessment and discard laws in debug/release. [Framing validation](partial-seal-corruption/framing-green.txt) includes formatting, source structure, all-feature workspace/all-target Clippy and the new laws in debug/release. Commands execute in copied Linux Docker trees with Rust 1.96.0 and the committed lockfile; the laws are small, in-memory and deterministic. Per-test resource ceilings and denied-egress enforcement remain repository gaps, not claimed implemented controls.

Receipt normalization replaces container source/target prefixes with `<isolated-build>` and removes trailing spaces and redundant trailing blank lines; assertion diagnostics and outcomes are preserved. These receipts cover the implementation slice, not final acceptance. Stable-candidate full validation and independent exact-head review remain required before this PR is ready. No physical power-loss, actual unlink, allocation benchmark or complete recovery audit claim is made.

## Permanent counterexample and parser exploration

The regression is reduced to the empty-segment header, seal magic and unsupported version 2 in `tests/fixtures/recovery/unsupported-partial-seal-version.hex`. Removing the record and unused seal suffix preserves both public failures on main, as shown by the [reduced parent runtime RED](partial-seal-corruption/reduced-parent-runtime-red.txt). Earlier compiler/setup failures are excluded from this runtime receipt. The ordinary tests and deterministic fuzz-seed preparation both consume the retained input.

The existing registered `segment_format` target adds selector 5 for the production recovery classifier. Its independent byte-table oracle requires every available fixed seal byte in a returned seal truncation to be canonical. The [fuzz parent RED](partial-seal-corruption/fuzz-parent-red.txt) records the version-byte assertion failing against main's classifier; this is a semantic oracle failure, not merely a parser crash. [Fuzz GREEN](partial-seal-corruption/fuzz-green.txt) records successful single-input replay followed by a seeded bounded campaign on the corrected implementation. Fuzz target Clippy and the instrumented target build also pass.

Replay inside a copied Docker checkout with the pinned `nightly-2026-07-24` toolchain and `cargo-fuzz 0.13.2`:

```sh
cargo xtask prepare-fuzz-corpus
cargo +nightly-2026-07-24 fuzz run segment_format fuzz/corpus/segment_format/recovery-unsupported-partial-seal-version -- -runs=1 -timeout=5 -rss_limit_mb=1024
cargo +nightly-2026-07-24 fuzz run segment_format -- -seed=17101 -max_total_time=15 -timeout=5 -rss_limit_mb=1024 -max_len=1048576
```

The seed and commands were recorded before launch. The campaign uses cargo-fuzz's instrumented release profile and address sanitizer, with libFuzzer timeout/RSS/input bounds; it does not establish exhaustive input coverage, an allocation benchmark, or isolation of every ambient dependency. The baseline replay used unchanged main production with only the new oracle/input copied in; the ordinary parent regression remains separately committed and reproducible. Future fuzz findings retain minimized reproducers through the repository's existing corpus workflow.

## Direct assertion calibration and seed-count retirement

The original parent RED establishes missing refusal, not the later exact diagnostic/source assertions. Review required additional direct calibration. These production mutations compile and reach their named runtime checks; [restored debug/release and Clippy](partial-seal-corruption/calibration-green.txt) pass after restoring source and invalidating timestamps.

| Broken contract | Replay patch | Observed failure |
| --- | --- | --- |
| Diagnostic reports a false observed value | [wrong diagnostic](partial-seal-corruption/wrong-diagnostic.patch) | [Exact expected/observed error comparison](partial-seal-corruption/wrong-diagnostic-red.txt) |
| Error loses the original seal source | [dropped source](partial-seal-corruption/dropped-source.patch) | [Required typed cause becomes absent](partial-seal-corruption/dropped-source-red.txt) |
| Canonical incomplete seal reports the wrong observed length | [wrong coordinate](partial-seal-corruption/wrong-coordinate.patch) | [Unobserved mutation must retain exact truncation](partial-seal-corruption/wrong-coordinate-red.txt) |

Apply one patch to a clean copied candidate, touch its changed Rust file, run `cargo test --locked --test recovery_partial_seal`, reverse only that patch, touch the restored file and rerun debug/release. The expected failures are assertion failures after compilation, not exit status alone. Patches do not belong in a committed production tree.

The first full candidate run also encountered `seed_preparation_materializes_the_complete_deterministic_set`: it failed because adding recovery inputs changed its frozen cardinality. Those counts neither asserted Keep behavior nor established that any seed reached its parser. They are retired under the no-protected-contract deletion criterion. The replacement feeds the actual materialized recovery input to Keep's classifier and requires the precise seal refusal; repeated preparation still must preserve the complete emitted bytes. This is tool-to-runtime replay evidence, distinct from the small direct classifier law and the fuzz campaign.

The emitted-input witness is calibrated with a [wrong selector](partial-seal-corruption/materialized-selector.patch), [missing named input](partial-seal-corruption/materialized-absent.patch), and the same wrong-diagnostic production patch above. The respective runtime RED receipts are [selector](partial-seal-corruption/materialized-selector-red.txt), [absent input](partial-seal-corruption/materialized-absent-red.txt), and [diagnostic](partial-seal-corruption/materialized-diagnostic-red.txt). Run `cargo test --locked --package xtask --bin xtask seed_preparation_preserves_a_replayable_recovery_counterexample` with each mutation separately. Earlier setup attempts selecting zero tests are excluded; the admitted receipts execute and fail the named law.

After all three emitted-input mutations are removed, [restored materialization GREEN](partial-seal-corruption/materialized-green.txt) records the named law executing in debug/release, followed by workspace Clippy and source-structure validation. The remaining broad acceptance chain is rerun on the resulting stable candidate; the earlier full run's seed-count failure is not hidden or treated as a product regression.
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
Compiling keep v0.0.0 (<isolated-build>)
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.94s
Running tests/recovery_partial_seal.rs (<isolated-build>/debug/deps/recovery_partial_seal-50cc4959e316d0ba)

running 3 tests
test unsupported_partial_seal_version_refuses_classification ... ok
test unsupported_partial_seal_version_refuses_discard_assessment ... ok
test framing_laws::available_fixed_seal_contradictions_refuse_every_later_short_boundary ... ok

test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s

Compiling keep v0.0.0 (<isolated-build>)
Finished `release` profile [optimized] target(s) in 2.37s
Running tests/recovery_partial_seal.rs (<isolated-build>/release/deps/recovery_partial_seal-6b9a4d5315d5b74a)

running 3 tests
test unsupported_partial_seal_version_refuses_classification ... ok
test unsupported_partial_seal_version_refuses_discard_assessment ... ok
test framing_laws::available_fixed_seal_contradictions_refuse_every_later_short_boundary ... ok

test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Checking keep v0.0.0 (<isolated-build>)
Checking xtask v0.0.0 (<isolated-build>/xtask)
Checking keep-benchmark v0.0.0 (<isolated-build>/benchmark)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 2.65s
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
Compiling keep v0.0.0 (<isolated-build>)
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.91s
Running tests/recovery_partial_seal.rs (<isolated-build>/debug/deps/recovery_partial_seal-50cc4959e316d0ba)

running 3 tests
test unsupported_partial_seal_version_refuses_discard_assessment ... FAILED
test unsupported_partial_seal_version_refuses_classification ... FAILED
test framing_laws::available_fixed_seal_contradictions_refuse_every_later_short_boundary ... FAILED

failures:

---- unsupported_partial_seal_version_refuses_discard_assessment stdout ----

thread 'unsupported_partial_seal_version_refuses_discard_assessment' (1794097) panicked at tests/recovery_partial_seal.rs:45:5:
assertion `left == right` failed
left: None
right: Some(UnsupportedVersion { expected: 1, observed: 2 })

---- unsupported_partial_seal_version_refuses_classification stdout ----

thread 'unsupported_partial_seal_version_refuses_classification' (1794096) panicked at tests/recovery_partial_seal.rs:23:5:
assertion `left == right` failed
left: None
right: Some(UnsupportedVersion { expected: 1, observed: 2 })
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- framing_laws::available_fixed_seal_contradictions_refuse_every_later_short_boundary stdout ----

thread 'framing_laws::available_fixed_seal_contradictions_refuse_every_later_short_boundary' (1794095) panicked at tests/recovery_partial_seal/framing_laws.rs:41:17:
assertion `left == right` failed: offset=16, observed=17
left: None
right: Some(UnsupportedVersion { expected: 1, observed: 257 })


failures:
framing_laws::available_fixed_seal_contradictions_refuse_every_later_short_boundary
unsupported_partial_seal_version_refuses_classification
unsupported_partial_seal_version_refuses_discard_assessment

test result: FAILED. 0 passed; 3 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

error: test failed, to rerun pass `--test recovery_partial_seal`
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
--- a/src/adapters/recovery/recovery_segment_stage_error.rs
+++ b/src/adapters/recovery/recovery_segment_stage_error.rs
@@ -75,7 +75,7 @@
match self {
Self::Metadata { source } => Some(source),
Self::Header { source } => Some(source),
- Self::Seal { source } => Some(source),
+ Self::Seal { .. } => None,
Self::Record { source } | Self::Complete { source } => Some(source),
Self::AddressSpace { .. } => None,
}
25 changes: 25 additions & 0 deletions docs/testing-evidence/partial-seal-corruption/framing-green.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
Checking keep v0.0.0 (<isolated-build>)
Checking xtask v0.0.0 (<isolated-build>/xtask)
Checking keep-benchmark v0.0.0 (<isolated-build>/benchmark)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.88s
Compiling keep v0.0.0 (<isolated-build>)
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.32s
Running tests/recovery_partial_seal.rs (<isolated-build>/debug/deps/recovery_partial_seal-50cc4959e316d0ba)

running 3 tests
test unsupported_partial_seal_version_refuses_classification ... ok
test unsupported_partial_seal_version_refuses_discard_assessment ... ok
test framing_laws::available_fixed_seal_contradictions_refuse_every_later_short_boundary ... ok

test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s

Compiling keep v0.0.0 (<isolated-build>)
Finished `release` profile [optimized] target(s) in 0.23s
Running tests/recovery_partial_seal.rs (<isolated-build>/release/deps/recovery_partial_seal-6b9a4d5315d5b74a)

running 3 tests
test unsupported_partial_seal_version_refuses_classification ... ok
test unsupported_partial_seal_version_refuses_discard_assessment ... ok
test framing_laws::available_fixed_seal_contradictions_refuse_every_later_short_boundary ... ok

test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
Loading
Loading