diff --git a/CHANGELOG.md b/CHANGELOG.md index 4167bfa0..728160e8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,24 @@ after its public API and format compatibility policies are established. ## [Unreleased] +- Durable read diagnostics render their boundary once while preserving each original typed source for error-chain reporters (#109). + +- Caller-supplied malformed durable layout records now preserve the reconstruction or range input-decode boundary instead of reporting committed-layout corruption (#109). + +- Corrected durable-read allocation documentation: the caller's aggregate byte limit caps catalog-selected segment bytes; catalog bytes and decoded metadata have separate bounds and allocate additionally (#109). + +- Clarified the root-directory synchronization and precise admission-time I/O failure performed when opening a durable reader snapshot; the existing platform policy and runtime behavior are unchanged (#109). + +- Reference and durable reads now share an inward authentication core and immutable chunk port, preserving public receipts, precise errors and codec admission at the adapter boundary (#109). + +- Retention and durable snapshot readers now enforce the existing version-two filesystem profile through the same opened directory capability, rejecting unsupported filesystems without acquiring writer authority (#109). + +- `DurableStore::open` now fixes an absolute locator and returns a typed result, preventing later working-directory changes from retargeting a handle; locator failures preserve their original I/O source (#109). + +- Selected retention-root reads now refuse a canonical root belonging to another namespace, preserving typed expected/observed namespace digests before durable snapshot admission or output (#109). + +- Added an in-progress fenced durable read API with view-bound reconstruction and range receipts; full read-law and Worldline acceptance remains tracked in #109. + - Strengthened writer-lock replacement evidence with a private after-lock scheduling checkpoint, exact refusal and preserved file bytes; production lock ordering and public behavior are unchanged (#169). - Migration compatibility laws preserve version-one bytes and reject version-one authority at every forward prefix; public version/flag refusal tests and bounded seeded recovery-planner fuzzing extend transition evidence (#112). diff --git a/README.md b/README.md index 5a470d1f..dbe311ab 100644 --- a/README.md +++ b/README.md @@ -189,6 +189,32 @@ assert_eq!(output, b"exact bytes, or nothing"); # Ok::<(), Box>(()) ``` +On Linux with the admitted ext4 profile, read an existing migrated version-two store whose retention roots anchor the requested blob. The following example is also compiled in the `DurableStore` API documentation. Its 16 MiB limit applies to aggregate catalog-selected segment bytes; catalog bytes and decoded metadata allocate separately. + +```rust +#[cfg(target_os = "linux")] +fn copy_retained_blob( + root: &std::path::Path, + target: keep::BlobId, + output: &mut impl std::io::Write, +) -> Result> { + use keep::{ + CatalogRestartByteLimit, CatalogRestartPolicy, DurableStore, LayoutEntryLimit, + ReaderAttemptLimit, SegmentReadPolicy, SegmentRecordLimit, + }; + // Limit catalog-selected segment bytes; catalog and metadata allocate separately. + let policy = CatalogRestartPolicy::new( + SegmentReadPolicy::new(SegmentRecordLimit::MAXIMUM, LayoutEntryLimit::MAXIMUM), + CatalogRestartByteLimit::new(16_777_216)?, + ); + let store = DurableStore::open(root, policy, ReaderAttemptLimit::DEFAULT)?; + let snapshot = store.snapshot()?; + Ok(snapshot.reconstruct(target, output)?) +} +``` + +Snapshot admission materializes catalog bytes under the catalog format bounds and selected segment bytes under the supplied aggregate segment budget. Decoded indexes and retention records allocate additionally under separate format and record-count limits; the segment budget is not a total memory cap. Reads stream to the caller without an additional whole-blob buffer; a failed write may leave an untrusted prefix. Retain an explicit snapshot to make several reads against one fenced view. The receipt records that view but grants no retention after the snapshot is dropped. This read API performs no publication, repair or collection; see the [durable-read evidence and remaining acceptance work](docs/testing-evidence/durable-authenticated-reads.md). + Run the full gate suite the way CI does: ```bash diff --git a/docs/invariants/authenticated-reconstruction/README.md b/docs/invariants/authenticated-reconstruction/README.md index 4c3dea09..2d9e8e29 100644 --- a/docs/invariants/authenticated-reconstruction/README.md +++ b/docs/invariants/authenticated-reconstruction/README.md @@ -1,9 +1,6 @@ # Authenticated Reconstruction Contract -**Status:** Normative for every Keep operation that claims authenticated -reconstruction. The public non-durable `ReferenceStore` implements the -complete-object and exact-range forms. A consolidated durable logical-read -surface is not yet implemented. +**Status:** Normative for every Keep operation that claims authenticated reconstruction. The public non-durable `ReferenceStore` implements the complete-object and exact-range forms. Linux `DurableStore` and `DurableSnapshot` compose those read cores with fenced version-two catalog and retained-root admission; final #109 acceptance remains recorded in the [evidence ledger](../../testing-evidence/durable-authenticated-reads.md). The [rationale](rationale.md) records the governed decisions and rejected alternatives. The [requirement ledger](requirements.md) maps each law to its @@ -208,7 +205,7 @@ range and explicitly carry the narrower range proof posture. ## Durable reconstruction requirement -A future operation claiming durable logical reconstruction must additionally: +An operation claiming durable logical reconstruction must additionally: - bind reads to one admitted immutable snapshot or catalog generation; - prevent required supporting evidence from being garbage-collected, deleted, @@ -220,12 +217,17 @@ A future operation claiming durable logical reconstruction must additionally: - separate evidenced refusal from operational failure; - preserve the output-visibility rule above. -The current durable segment, catalog, publication, and recovery surfaces do -not yet form this consolidated high-level `BlobId`-to-writer contract. -Retention publication now records verified closures as generation-checked -roots, but nothing collects or fences yet, so no current surface protects or -releases the evidence closure this operation requires. These lower-level surfaces must not be -described as an implemented durable logical reconstruction API. +`DurableStore` pins a fresh `DurableSnapshot` per convenience read. An explicit snapshot holds the shared reader fence, verifies manifest-selected retained closures, and preserves its catalog and liveness coordinates while retention successors publish. Blob reads select a retained anchor; exact-layout reads may use an unretained catalogued layout. Both authenticate exact immutable bytes before output and return the reference receipt together with catalog generation/digest and the selected retention head when present. + +`DurableStore::open` returns a result and fixes an absolute locator at construction; changing the process working directory later cannot select a different store through that handle. Failure to resolve the locator preserves the original I/O cause through `DurableStoreError::Locator`, before store admission or output. + +Snapshot admission requires the existing writable, case-sensitive local Linux ext4 profile across the root and present version-two protocol directories, retaining the admitted root capability through collection without acquiring writer authority. Valid migration records on an unsupported filesystem do not admit a readable snapshot. + +Every selected retention root must match its manifest entry's namespace as well as its root generation and digest; a canonical foreign-namespace root refuses with preserved expected and observed namespace coordinates before a durable snapshot or caller output is exposed. + +Admission materializes bounded catalog and segment bytes and verifies broader stored evidence than the logical range core. Each read re-admits those owned bytes and decoded indexes; the core's single authentication pass is not an end-to-end single-hash or minimal physical-I/O guarantee. Reads add no whole-blob output buffer. The caller's aggregate byte budget limits catalog-selected segment bytes only. Catalog bytes have separate format bounds, and decoded indexes and retention records allocate additionally under their own format and record-count limits. Retained-root traversal uses the persisted per-root limits. + +The fence coordinates cooperating Keep operations in a managed namespace. Arbitrary concurrent out-of-band mutation is outside that isolation guarantee; exact-byte, no-follow, identity and corruption protections still apply. Actual GC remains absent, so collector evidence demonstrates kernel fence exclusion rather than end-to-end collection. A receipt grants no retention beyond the snapshot lifetime and does not make caller output transactional. ## Current public evidence @@ -241,12 +243,12 @@ Evidence anchors: - [exact logical byte identity](../../adr/0001-exact-logical-byte-identity.md) - [identity and physical-storage separation](../../adr/0002-separate-identity-from-physical-storage.md) - [`ReferenceStore` contract tests](../../../tests/reference_store_contract.rs) -- [reconstruction implementation](../../../src/reference/reconstruction.rs) -- [range-read implementation](../../../src/reference/range_read.rs) +- [shared reconstruction core](../../../src/authenticated_read/reconstruction.rs) +- [shared range-read core](../../../src/authenticated_read/range_read_execution.rs) - [whole-object refusal laws](../../../tests/streaming_cas/refusal_laws.rs) - [range-read refusal laws](../../../tests/range_read_failures.rs) -- [reconstruction receipt](../../../src/reference/reconstruction_receipt.rs) -- [range-read receipt](../../../src/reference/range_read_receipt.rs) +- [reconstruction receipt](../../../src/authenticated_read/reconstruction_receipt.rs) +- [range-read receipt](../../../src/authenticated_read/range_read_receipt.rs) ## Consumer rule diff --git a/docs/invariants/authenticated-reconstruction/rationale.md b/docs/invariants/authenticated-reconstruction/rationale.md index 225a257d..cb69b8d2 100644 --- a/docs/invariants/authenticated-reconstruction/rationale.md +++ b/docs/invariants/authenticated-reconstruction/rationale.md @@ -20,10 +20,7 @@ successful receipt authenticates the complete emitted sequence. Failure may leave an untrusted prefix, so consumers that require atomic visibility must quarantine output and publish it transactionally after receipt validation. -Current `ReferenceStore` behavior is the executable oracle for non-durable -complete-object and range forms. It is not evidence that Keep has a durable -logical reconstruction API. A future durable form must pin one immutable view, -retain its complete supporting evidence, and bind the view into its result. +Current `ReferenceStore` behavior is the executable oracle for non-durable complete-object and range forms. The Linux durable adapter has its own public filesystem witnesses and composes the same immutable read cores with catalog, retained-closure and reader-fence admission. It pins the supporting view and binds its coordinates into successful receipts; reference tests alone are not evidence for those filesystem obligations. ## Governed surfaces @@ -34,7 +31,7 @@ This decision governs: - receipt and refusal meaning; - output visibility after failure; - layout selection and committed layout-to-target binding; -- the future durable read aperture and evidence-retention obligation; and +- the durable read aperture and evidence-retention obligation; and - the public integration boundary available to consumers. It does not govern Echo semantics, causal authority, application retry law, or @@ -86,6 +83,5 @@ is retained under the same contract. - Caller-supplied range layouts must resolve through an admitted store view. - A failure after output began returns no success receipt; accepted bytes remain untrusted. -- Durable integration remains blocked on a pinned-view consumer capability and - evidence retention. +- Durable reads hold a shared reader fence across output; production GC is still absent, and the fence does not isolate arbitrary raw namespace mutation. - Application-specific meaning remains outside Keep core. diff --git a/docs/invariants/authenticated-reconstruction/requirements.md b/docs/invariants/authenticated-reconstruction/requirements.md index 89148258..ee08a7a6 100644 --- a/docs/invariants/authenticated-reconstruction/requirements.md +++ b/docs/invariants/authenticated-reconstruction/requirements.md @@ -11,8 +11,10 @@ gap, not implementation evidence. | `KEEP-RECONSTRUCT-003` | A range receipt proves only requested bytes from authenticated overlapping chunks; it proves neither the complete blob nor any storage-profile boundary. | Minimal-overlap range model | Unit, property, and public API integration tests | Implemented | `src/reference/range_read_tests.rs`, `tests/range_read.rs`, `tests/range_read_properties.rs` | | `KEEP-RECONSTRUCT-004` | A range receipt names only a layout-to-target binding admitted by the selected store view. | Forged same-length target-layout fixture | Public API corruption test | Implemented | `tests/range_read_entrypoints.rs` | | `KEEP-RECONSTRUCT-005` | Success authenticates the complete emitted sequence; failure returns no success receipt and reports the exact accepted prefix, which remains untrusted. | Deterministic prefix-then-fail writer | Public API failure tests | Implemented | `tests/streaming_cas/refusal_laws.rs`, `tests/range_read_failures.rs` | -| `KEEP-RECONSTRUCT-006` | Authenticated success, evidenced content refusal, and operational failure remain distinct outcomes; operational failure supports no content conclusion. | Typed outcome classification | Public API integration tests and contract inspection | Implemented for `ReferenceStore`; durable refusal receipts planned | `tests/streaming_cas/refusal_laws.rs`, `tests/range_read_failures.rs`; [Keep #22](https://github.com/flyingrobots/keep/issues/22) | -| `KEEP-RECONSTRUCT-007` | Whole-object and range receipts bind target, exact layout, proof scope, and exact emitted coordinates without granting retention or application authority. | Receipt type inspection | Public API contract tests | Implemented | `src/reference/reconstruction_receipt.rs`, `src/reference/range_read_receipt.rs`, `tests/range_read_contract.rs` | +| `KEEP-RECONSTRUCT-006` | Authenticated success, evidenced content refusal, and operational failure remain distinct outcomes; operational failure supports no content conclusion. | Typed outcome classification | Public API integration tests and contract inspection | Implemented typed outcomes; persisted refusal receipts remain separate | `tests/streaming_cas/refusal_laws.rs`, `tests/range_read_failures.rs`, `tests/golden_file_worldline/durable_refusal_laws.rs`, `tests/golden_file_worldline/durable_writer_failures.rs` | +| `KEEP-RECONSTRUCT-007` | Whole-object and range receipts bind target, exact layout, proof scope, and exact emitted coordinates without granting retention or application authority. | Receipt type inspection | Public API contract tests | Implemented | `src/authenticated_read/reconstruction_receipt.rs`, `src/authenticated_read/range_read_receipt.rs`, `tests/range_read_contract.rs` | | `KEEP-RECONSTRUCT-008` | Automatic layout choice is deterministic; an exact requested layout never falls back. | Canonically ordered layout set | Unit and public API integration tests | Implemented | `src/reference/store_tests.rs`, `tests/streaming_cas/reconstruction_laws.rs` | -| `KEEP-RECONSTRUCT-009` | A durable read pins one immutable view and prevents required evidence from being garbage-collected, deleted, or invalidated through completion. | Pinned-generation and retained-closure model | Recovery, concurrency, corruption, and crash-injection tests | Planned gap | [Keep #22](https://github.com/flyingrobots/keep/issues/22), [Keep #23](https://github.com/flyingrobots/keep/issues/23) | -| `KEEP-RECONSTRUCT-010` | Durable reconstruction names its view and returns either authenticated success, evidenced refusal, or operational failure without hidden whole-blob allocation. | Durable consumer conformance model | Public API integration, memory, recovery, and crash-injection tests | Planned gap | [Keep #22](https://github.com/flyingrobots/keep/issues/22), [Keep #23](https://github.com/flyingrobots/keep/issues/23) | +| `KEEP-RECONSTRUCT-009` | A durable read pins one immutable view and prevents required evidence from being garbage-collected, deleted, or invalidated through completion. | Pinned-generation and retained-closure model | Recovery, concurrency, corruption, and crash-injection tests | Implemented for cooperating managed-store operations; final #109 acceptance pending | `src/adapters/retention/durable_view_law_tests.rs`, `tests/golden_file_worldline/durable_assertions.rs`; [scope and evidence](../../testing-evidence/durable-authenticated-reads.md) | +| `KEEP-RECONSTRUCT-010` | Durable reconstruction names its view and returns either authenticated success, evidenced refusal, or operational failure without hidden whole-blob allocation. | Durable consumer conformance model | Public API integration, memory, recovery, and crash-injection tests | Implemented; final #109 acceptance pending | `tests/golden_file_worldline/durable_read_memory.rs`, `tests/golden_file_worldline/durable_corruption_laws.rs`, `tests/golden_file_worldline/durable_output_laws.rs`; [scope and evidence](../../testing-evidence/durable-authenticated-reads.md) | + +The durable implementation status describes the current candidate, not a merged release or completed independent review. Snapshot admission materializes catalog-selected segment bytes under the caller's aggregate segment-byte limit; catalog bytes have a separate format maximum, while decoded indexes and retention records allocate additionally under format and record-count limits. The segment limit is not a total snapshot-memory cap; the memory witness measures additional reconstruction allocation. Managed-namespace cooperation is required for the fence guarantee. The #109 ledger distinguishes fresh reopen from process death and kernel exclusion from absent production GC. diff --git a/docs/testing-evidence/durable-authenticated-reads.md b/docs/testing-evidence/durable-authenticated-reads.md new file mode 100644 index 00000000..b8bb17db --- /dev/null +++ b/docs/testing-evidence/durable-authenticated-reads.md @@ -0,0 +1,320 @@ +# Durable authenticated reads (#109) + +Change kind: new read API with a shared-core extraction. The baseline is main `6051abb25a9fd33ae7ee0de5614514b709a4d82a`. No write protocol or on-disk format is changed; #125's candidate-catalog publication gate is not added or bypassed. This is an in-progress implementation ledger, not acceptance of #109. + +## Review after draft readiness + +The independent approval and successful checks on `186ab8a00796101d26640084f05960624d76f77d` predate the subsequent hosted Codex and CodeRabbit findings; they do not establish readiness of a revised candidate. + +| Review obligation | Disposition and evidence | Exit condition | +| --- | --- | --- | +| CodeRabbit: selected root namespace binding (`discussion_r4170510757`). | Confirmed runtime defect on `186ab8a`: both direct selected-root read and durable snapshot admission accepted a canonical foreign-namespace root. The fix at `FilesystemRetentionSnapshot::retained_root` retains existing digest/generation checks and reports typed expected/observed namespace coordinates. | Focused debug/release laws, source-preservation calibration, final-head full validation and independent delta review. | +| Codex: inward ownership of shared authentication (`discussion_r4170495607`). | The `f1312d6` baseline imported policy and the port through `reference`. The revised source moves authentication, immutable chunk lookup, semantic failures and receipts to `authenticated_read`; shared codec-bearing public errors and lossless outward mapping live under `adapters/authenticated_read`. | Move the shared behavior to an inward semantic boundary without changing authenticated output/refusal behavior; preserve public API compatibility and generated parity evidence. | +| Codex: relative store locator stability (`discussion_r4170495611`). | Confirmed RED on `28f1720`; the constructor now fixes an absolute locator and preserves resolution failure as `DurableStoreError::Locator`. Public isolated-process laws cover two valid stores and a deleted current directory. | Focused runtime and source-preservation evidence below; final-head full validation and independent delta review remain required. | +| Codex: reader production-platform admission (`discussion_r4170495617`). | Confirmed RED on `082c515`; direct and durable snapshot readers admitted a valid migrated tmpfs namespace. The shared reader loader now uses the existing version-two platform admission and retains its returned directory capability. | Focused tmpfs refusal and ext4 reader-availability evidence below; final-head full validation and independent delta review remain required. | + +### Namespace regression + +Change kind: bug fix; the public fixture publishes real content, migrates, and retains it on the admitted ext4 profile, then deliberately installs a checksummed manifest selecting the original root under another namespace outside publication. + +No raw mutation occurs concurrently with the read under test. + +`a_selected_root_from_another_namespace_refuses_direct_read` and `a_foreign_retained_namespace_refuses_durable_output` were observed RED against production head `186ab8a` after successful compilation, reporting respectively `foreign namespace root was returned by the public reader` and `foreign namespace root admitted a durable snapshot`. + +The final laws assert `FilesystemRetentionSnapshotError::Root`, `InvalidData`, and the exact `RetentionSelectedRootRefusal::Namespace` expected/observed digests; the convenience-read law also checks the caller's output sentinel remains unchanged. + +The tests are medium, real-filesystem public-path evidence; no process-death, power-loss, or arbitrary concurrent namespace-isolation guarantee is inferred. + +The new diagnostic is preserved inside the existing root error boundary, so unrelated checksum, generation, digest, and operational causes retain their existing paths. + +Raw artifacts retain `namespace-red-on-186ab8a.log`, `namespace-green.log`, and `namespace-structure.log`; a replay initially included the new diagnostic type against the old library and failed compilation (`namespace-red-replay.log`), which is excluded from behavioral RED evidence. + +The corrected parent-compatible regression was replayed RED in `namespace-red-replay-corrected.log` and committed separately as `0a43198` before the fix; that commit can reproduce the original acceptance defect without the new diagnostic type. + +The fixed public laws pass in debug and release; the full Golden File Worldline integration binary, existing retention snapshot laws, doctests, all-target/all-feature Clippy, formatting, and structure checks also pass in the copied Docker candidate. + +Swapping only the new diagnostic's expected and observed namespace digests in a separate source/target directory made both final public laws fail their exact-coordinate assertion (`namespace-coordinate-red.log`); the unchanged candidate remained green (`namespace-relevant-validation.log`). + +Final full-chain checks and independent review remain pending while the remaining review obligations are resolved. + +### Relative locator regression + +Change kind: bug fix with a fallible-constructor adjustment to the new, unmerged API; `DurableStore::open` now returns `Result` and fixes an absolute locator before returning a handle. + +Resolving a relative path queries the current directory once, while store admission remains deferred to snapshot creation. + +This prevents working-directory changes from retargeting a handle; it does not pin the filesystem pathname against unsupported out-of-band replacement. + +The medium public regression creates two production-profile stores with different retained blobs in an isolated child, opens `.` in the first, changes to the second, and checks original retention and reconstructed bytes through the same handle. + +The unfixed code at `28f172043c6a3e360478137231c9c51e0e4ad725` failed `the same handle must retain its original store after cwd changes` after successful compilation (`locator-red-on-28f1720.log`); the standalone RED test is commit `4f37597`. + +A second isolated child deletes its current directory and checks the constructor's exact `Locator` / `NotFound` / `ENOENT` outcome and preserved `io::Error` source. + +Each child is selected by the harness with an exact test name, runs alone, and has a 20-second execution ceiling enforced by `timeout`; the ceiling is a hang guard, not a performance promise. + +There are no sleeps or schedule sampling, and cwd mutation never occurs in the parallel parent test process. + +Focused debug/release laws, the full Worldline binary, existing durable-read unit laws, doctests, Clippy, formatting, and structure validation passed in the copied Linux arm64 Docker candidate (`locator-green-corrected.log`). + +The original `locator-green.log` retains a Clippy type-complexity failure in the child runner; a local result alias corrected it without changing a product expectation. + +All existing constructor call sites and the compiled public example now propagate the constructor error; no existing output, refusal, or receipt expectations were weakened. + +Separate copied-source/target mutants replaced the original locator cause with `Other` and removed its `Error::source` link; the deleted-cwd law failed respectively at the exact-cause and source-chain assertions after successful compilation (`locator-mutants/cause/red.log` and `locator-mutants/source/red.log`). + +### Reader platform regression + +Change kind: bug fix; `FilesystemRetentionSnapshot::load` now calls the existing version-two filesystem-profile admission before namespace, migration identity, fence, and coordinate collection. + +The returned directory capability is reused throughout admission; no second ambient-path open or writer lock is introduced. + +Both `a_canonical_tmpfs_store_refuses_the_public_reader_profile` and `a_canonical_tmpfs_store_refuses_durable_snapshot_admission` were observed RED on `082c5155a244416cef94ea5217f15ca19a344383`, after successful compilation and construction of valid migration records, with successful snapshot admission being the wrong runtime outcome (`reader-platform-red-on-082c515.log`). + +The standalone RED commit is `397164e`. + +These are medium Linux filesystem laws: the negative fixture verifies `/dev/shm` is tmpfs, uses test-only writer admission to construct an internally consistent version-two store, and then calls the public production reader methods. + +The bypass is limited to adversarial fixture construction; the methods under test do not use a bypass. + +The exact expected refusal is `FilesystemRetentionSnapshotError::Admission` with `Unsupported`, preserved under `DurableStoreError::Snapshot` for the durable entry point. + +`reader_platform_admission_does_not_acquire_writer_authority` holds the actual writer lock while the production-profile reader admits the frozen catalog, demonstrating compatible reader availability rather than only rejection on an unsupported filesystem. + +Focused debug/release laws, existing snapshot laws, full Worldline integration, doctests, all-target/all-feature Clippy, formatting, and source structure pass in the copied Linux arm64 Docker candidate (`reader-platform-green.log`). + +The existing profile checks filesystem type, writable state, casefold flags, and device/mount agreement of present protocol directories; this change reuses those checks rather than introducing a second platform policy. + +These receipts exercise actual tmpfs rejection and ext4 acceptance, not new filesystem-specific crash, power-loss, or concurrent raw namespace guarantees. + +A separate copied-source/target mutant acquired writer authority inside reader admission; the reader-availability law then failed with the actual typed writer `Busy` cause (`reader-writer-lock-calibration-red.log`), while the unchanged candidate passed again (`reader-platform-post-calibration-green.log`). + +### Shared authentication ownership + +Change kind: structural refactoring with unchanged public behavior, using `f1312d6f60c10421ad32aec66a926a0bb5535456` as the before-state. + +The before-state placed both production callers behind `reference::{ChunkSource, read_admitted, reconstruct_admitted}`; the revised callers depend on `authenticated_read`, whose transitive semantic dependencies contain no storage adapter or codec error. + +The reference adapter retains lookup and decoding; durable ingress retains its catalog, retained-root, platform and fence checks. + +Core failures preserve original causes and coordinates, and exhaustive outward mappings preserve every existing public variant without adding an error-source layer. + +The two existing source-inspection test files only update relocated input paths; their assertions are unchanged and are static evidence, not product evidence. + +No runtime RED is claimed for ownership alone, and no harness-count or source-string regression is added for it. + +The existing generated range/source-slice laws, private corruption laws, public writer/refusal laws and Worldline cases supply behavioral verification; their expected bytes, receipts and error coordinates remain unchanged. + +The first local check attempt used a login shell that omitted Cargo from its path (`core-extraction-check.log`); the corrected Docker shell compiled successfully (`core-extraction-check-corrected.log`). + +The first focused run stopped at Clippy's crate-visibility and large-error lints (`core-extraction-focused.log`); explicit scoped expectations preserve crate-private boundaries and allocation-free typed failures rather than changing the public contract. + +Full validation and independent review of the resulting candidate remain required before readiness. + +## Delivered candidate behavior + +`DurableStore` pins a fresh `DurableSnapshot` per convenience call. Snapshot admission keeps the existing shared reader fence, catalog, retention head and manifest view and verifies every selected retained closure against the catalog. Blob lookup scans selected roots one at a time and chooses the lowest retained layout identity; exact-layout reads use the catalog directly. Reconstruction and ranges use the shared immutable domain cores and return receipts with the view coordinates only after emission succeeds. + +The allocation, blocking and failure contract is in the public API documentation and [rationale](../../src/adapters/durable/rationale.md). Selected segment bytes are materialized under the caller's aggregate segment-byte limit. Catalog bytes are bounded separately by `CatalogLength::MAXIMUM`; decoded indexes and retention records allocate additionally under format and record-count limits. The segment limit is not a total snapshot-memory cap, and the API does not claim lazy segment reads or constant total memory. It adds no whole-blob output buffer or aggregate anchor index. + +## Closure ledger + +| Obligation | Current evidence | Required exit condition | +| --- | --- | --- | +| Golden bytes and exact read coordinates | `durable_read_law_tests` reconstructs the independent one-zero corpus and compares catalog and manifest digests against frozen bytes. Public Worldline tests reconstruct all frozen identities after reopening. | Parity map below is populated; independent review must assess the stated scope and calibration gaps. | +| Stable retained view across publication | `durable_view_law_tests` releases the retained root while an older snapshot remains alive; the old reader emits its original byte and head generation, and a fresh view sees release. | Current supported retention-publication path is exercised. A production v2 catalog writer is absent on the baseline; review must not confuse that absent path with demonstrated catalog-successor execution. | +| Collector exclusion | The public snapshot holds the actual kernel shared fence; a deterministic exclusive try-lock refuses until drop. | Preserve this claim as fence evidence; actual GC execution remains absent on main under #21. | +| Operational failures versus evidenced absence | Removing the selected segment yields exact `OpenSegment` / `NotFound`; public whole/range reads of a catalogued layout missing its chunk yield exact logical `ChunkMissing` with untouched output. Prefix failures preserve layout, accepted bytes and `PermissionDenied`; corrupt input layouts preserve exact checksum coordinates. | Public writer, absence, physical whole/range corruption and false-profile witnesses are present; require final-head validation and review. | +| Reference-core equivalence | Existing reference algorithms are generalized only over a crate-private immutable source; their single-pass authentication tests are retained. | Reference and durable generated domains are exercised. Core authentication-pass evidence remains distinct from repeated durable admission; final validation and independent review remain. | +| Golden File Worldline | `tests/golden_file_worldline/durable_assertions.rs` publishes the corpus through production v1 authority, migrates it, retains it, reopens and checks frozen identities and exact bytes/ranges. `durable_range_properties.rs` adds exhaustive short intervals and multichunk boundary sweeps. | Retain precise scope: reopening after writer handles close, not a process-death test. Final-head validation remains required. | +| Documentation and final acceptance | API costs and design rationale are documented. | Requirements and compiled Linux example are updated for the candidate; final checks and exact-head independent review remain open. | + +## Evidence limits + +The initial unit filesystem laws use owned test directories and the existing repository migration fixture with platform admission bypassed. The newer Linux public Worldline laws use production platform admission, publication, migration and retention in owned ext4 scratch. No sleep or probabilistic race is used. The collector test uses a kernel try-lock in one process, not a full GC integration or process-death claim. + +New laws are medium-size because they own filesystem state. Per-test resource enforcement and suite SLOs remain gaps described in the repository [enforcement profile](../testing/enforcement.md); no compliance waiver or new resource ceiling is claimed here. Deletion criteria and oracles appear beside the laws. + +Initial debug/release golden-read checks and all-feature Clippy passed; expanded view laws passed in debug. Initial authoring/compilation and Clippy failures are retained in the local evidence logs and are not product RED evidence. Final source SHAs, complete check results and actual mutation observations belong in the final PR receipt. This new API did not exist on the parent; parent compilation failure would not prove a runtime regression. + +## First implementation receipt + +Implementation commit `d6c38a1` passed formatting, source structure, both workspace Clippy feature profiles, complete all-feature workspace debug/release tests and doctests, and rustdoc generation on pinned Rust 1.96.0 in a copied Docker source tree. These runs include the existing generated range-to-source-slice properties and reference single-hash-pass laws. Subsequent documentation and visibility narrowing require their own final-head checks; this receipt does not transfer to future semantic changes. + +Three isolated source/build mutations were observed RED on that implementation: substituting `[1]` during emission after verification made whole-read, exact-layout and nonempty-range assertions report expected `[0]` versus observed `[1]`; advancing the receipt's catalog generation made its coordinate assertion report expected 1 versus observed 2; replacing shared fence acquisition with unlock made the collector-exclusion assertion report expected `WouldBlock` versus observed successful acquisition. These are runtime calibration, not parent-bug reproductions or an assertion that every new check has already been calibrated. + +At that first receipt, generated durable laws, Worldline reopen/range witnesses, missing-member/output failures and independent digest expectations remained open; the subsequent expansion below addresses those items. The issue stays open and its PR remains a draft until the full closure ledger is satisfied. + +## Worldline and layout-ingress expansion + +Commit `ab518b2` added production-admitted Worldline reopen/range reads, bounded range sweeps, exact logical-member absence, output-prefix failures and independent digest coordinates. It passed complete all-feature workspace debug/release tests, both Clippy profiles, formatting and source structure. This evidence does not transfer to later semantic changes. + +Isolated mutations made the new whole/range prefix assertions report a false zero accepted count instead of five, and made Worldline and range source-slice assertions detect an altered emitted byte after verification. Both were observed RED at the intended runtime assertions. The mutation that changes emitted bytes correctly leaves refusal-only laws green; those protect different claims. + +The next candidate adds the four caller-supplied layout entry points so reference whole-blob and catalogued-range binding laws have durable equivalents. Exact whole-blob mismatch and corrupt-layout checksum coordinates are asserted before output. These entry points and the public memory law pass focused debug/release tests and all-feature Clippy; final source-bound validation remains pending. + +`durable_read_memory` measures incremental allocation during reconstruction of the frozen 1 MiB Worldline source into a nonallocating sink. It requires peak incremental allocation below one whole blob, after explicit snapshot admission; it does not claim that snapshot construction avoids materializing selected segments or prove a universal resident-memory ceiling. + +The short-range sweep enumerates its finite domain in increasing length, making the first failing interval minimal within that domain. The multichunk sweep uses fixed documented boundary coordinates over the frozen source. Neither uses ambient randomness or claims exhaustive coverage of arbitrary large layouts. Replay is the named public test on the recorded source; any discovered counterexample must be retained before extending the domain. + +Memory calibration inserted a deliberately unnecessary 1,048,576-byte allocation into reconstruction in an isolated source/build copy. The new public memory assertion went RED with an observed peak of 1,048,576 bytes against its strictly smaller-than-blob requirement; the unmodified candidate passed. This validates that concrete additional-buffer detector, not the entire process memory budget. + +## Physical corruption and range output boundaries + +Candidate `e0a540c1e0913097329b209c3ab06be40e017e56` passed the complete local validation chain and all four required hosted checks: Rust quality gates, documentation and workflow integrity, runtime fuzz smoke and dependency policy. Later test additions require their own checks; this receipt does not transfer to them. + +The next test slice corrupts an actual selected segment payload in a production-admitted ext4 store. Its independent checksum preimage follows the normative v1 framing; reconstruction must preserve the typed record-index, offset and expected/observed checksum refusal before touching caller output. This is physical filesystem corruption evidence, not simulated port failure or process-death evidence. + +Additional range-output laws verify exact bytes through short/interrupted writes and exact zero-progress refusal before any accepted output. The complete public Worldline integration target passes in debug and release with these additions, and all-feature Clippy passes. The local receipts are `physical-corruption.log` and `corruption-output-validation.log`; no new production behavior is introduced by this test slice. + +Calibration changed the zero-progress writer's reported accepted count from zero to one in an isolated source copy. The new range assertion was observed RED on the exact `WriteZero` outcome with `ByteLength(1)` (`range-zero-mutation-red.log`), rather than a setup or compilation failure. The candidate source retains zero and is rechecked separately. The broader calibration ledger remains open; this witness is specific to accepted-byte accounting. + +An initial post-mutation candidate run reused mutant build output because both copies shared a Cargo target directory. That run is invalid candidate evidence and is retained as `corruption-final-slice-corrected.log`; the earlier mistyped xtask command is retained separately. The corrected protocol gives the mutant its own target directory and invalidates the candidate's compiled source before rerunning the full public integration target in debug/release, formatting, source structure and Clippy. Only `range-zero-isolated-mutation-red.log` and `corruption-final-candidate.log` are admissible for that corrected RED/GREEN pair. + +## Reference read-law reconciliation + +This map covers runtime read claims, not source-string assertions in `range_read_contract.rs` or write/staging laws. Durable twins may preserve a stronger pre-output admission boundary; that difference must be named instead of describing every reference error as identical. + +| Reference claim / source | Durable evidence | Remaining difference or action | +| --- | --- | --- | +| Exact reconstructed bytes, empty blobs, frozen identities (`streaming_cas/reconstruction_laws`, Worldline) | `durable_assertions`, `durable_read_law_tests` | Complete frozen corpus exercised after writer handles close and store reopen. | +| Short/interrupted writes, zero progress, accepted prefix (`reconstruction_laws`, `refusal_laws`, `range_read`, `range_read_failures`) | `durable_output_laws` | Whole and range outputs covered; calibration receipts are above. | +| Immediate output error and impossible returned write count (`refusal_laws`, `range_read_failures`) | `durable_writer_failures` | Exact layout, accepted prefix, cause or supplied/observed count checked. New validation recorded separately. | +| Whole-blob identity mismatch (`reconstruction_laws`) | `durable_layout_laws` | Caller-supplied layout reconstructs from catalogued chunks and refuses the wrong target before output. | +| False chunk-profile boundaries (`refusal_laws`) | `durable_layout_laws` | Frozen false-boundary record against production-published constituent chunks; exact expected/observed boundary and untouched output. | +| Absent blob/layout, bounds (`range_read`) | `durable_refusal_laws`, unretained-blob unit law | Exact identities and unchanged caller output; debug/release validation passes. | +| Malformed canonical layout (`refusal_laws`, `range_read`) | `durable_layout_laws` | Whole and range ingress preserve exact checksum coordinates. | +| Committed target binding / semantic and record ingress (`range_read_entrypoints`) | `durable_layout_laws` | Both durable ingress paths are exercised. | +| All short intervals / generated multichunk ranges (`range_read_properties`) | `durable_range_properties` | Short finite domain exhaustive; the durable suite also runs the reference's fixed 786,432-byte patterned source and affine coordinate domain, alongside the Worldline boundary grid. | +| Missing selected chunk (`reconstruction_laws`, reference private range laws) | `durable_refusal_laws` | Catalogued unretained layout exercises evidenced absence; retained missing closure refuses earlier at snapshot admission. | +| Corrupt selected chunk (reference private reconstruction/range laws) | `durable_corruption_laws` | Physical payload corruption refuses during segment admission before whole or range output, preserving exact record/checksum coordinates. | +| Only overlapping chunks read (reference private range law) | `a_durable_range_needs_no_nonoverlapping_chunk_records` | An unretained catalogued layout with only its interior chunk present serves an exact interior range; whole reconstruction refuses the missing first chunk. This establishes logical member independence, not minimal physical I/O during snapshot admission. | +| Exactly one chunk hash per reconstruction/selected range (reference private instrumentation) | Existing shared-core reference tests retained | The normative single-hash paragraph explicitly describes ReferenceStore. Durable reads reuse that immutable core pass, while snapshot/catalog admission independently verifies persisted evidence. No durable end-to-end single-hash promise is made; independent review must assess this scope reconciliation. | + +The issue itself states that collection exclusion is vacuous until #21 lands. Current evidence strengthens that floor with real shared/exclusive kernel fence exclusion and a live snapshot surviving retention publication. It does not claim an executable GC or unsupported version-two catalog publisher. The final requirement reconciliation must preserve these distinctions. + +The next parity slice adds the false-profile-boundary twin using the frozen mutation record and production-published constituent chunks. It preserves the exact expected 262,143-byte first boundary versus the observed 262,144-byte boundary before output. Immediate writer errors, impossible write counts, absent range blobs and absent reconstruction layouts also have direct public durable witnesses. The complete public integration target passes debug/release, Clippy and source structure (`read-parity.log`). + +Hosted validation of `cc189ff` caught a formatting-only error in the module declaration order: the earlier copy-back omitted the formatted `suite.rs`. Its Rust quality gate failed; the other required jobs passed. The author corrected the module list and records this as a validation-transfer mistake, not a runtime failure or a green final-head receipt. Subsequent validation checks the copied source against the committed files. + +The writer-parity calibration uses an isolated source and target directory. It substitutes an impossible maximum of zero and wraps the immediate I/O cause as `Other`; the four whole/range assertions fail on those specific wrong public outcomes (`writer-parity-mutation-red.log`). The original source passed the same assertions in debug/release. This is assertion calibration for the new API, not a parent regression: the durable API does not exist on main. + +## Range-domain and overlap closure + +The range continuation adds a direct physical-corruption range witness, the reference suite's fixed patterned multichunk source/coordinate domain, and a selected-member-only catalog. The last case deliberately cannot reconstruct its whole layout: the absent first chunk produces an exact `ChunkMissing` with untouched output, while the interior range succeeds with exact source bytes and receipt coordinates. The comparison is one proof-scope behavior, not a count of harness cases. Publication and migration use production capabilities; no retained root falsely claims closure over the intentionally incomplete layout. + +The complete Worldline target passes debug/release, all-feature Clippy and source structure (`range-overlap-parity.log`). Earlier generator-only and physical-corruption receipts are retained separately. The finite affine coordinate domain is replayable from source but does not claim a general shrinking framework; the exhaustive short-domain run remains ordered by increasing length. This narrows the remaining acceptance work without claiming arbitrary-input exhaustiveness. + +An isolated source/build mutation made range verification start with all layout entries instead of the selected entries. The overlap law went RED at the read boundary with an exact missing first-chunk refusal (`range-overlap-mutation-red.log`); the original candidate succeeds on the interior range. This calibrates logical overlap independence without using internal lookup counters as its oracle. + +## Acceptance documentation candidate + +The Linux README example mirrors a compiled `DurableStore` doctest and names an explicit 16 MiB aggregate catalog-selected segment budget, with catalog bytes and decoded metadata allocated separately. Normative documents now identify the available API, typed outcomes, view coordinates, managed-namespace fence scope and materialization costs. Requirement rows 009 and 010 identify the candidate implementation and explicitly retain final #109 acceptance as pending; they do not certify an unreviewed release. + +Review queue inspection found no submitted review bodies or inline threads; the sole top-level comment reports that CodeRabbit skipped this draft. All retrieved connections were exhausted. Independent review must cover the whole diff, the raw receipts and the scope distinctions above; no absence of comments is treated as approval. + +## Independent review and retained-closure correction + +Independent Codex review of `dd42dcbede0316fc64487283b45a78e7a4a0cac6` returned REQUEST CHANGES for evidence gaps, with no demonstrated production defect. The [full findings and checklist](https://github.com/flyingrobots/keep/pull/164#issuecomment-5962524242) were posted before remediation. Agy exhausted its quota without a verdict; it supplies no approval. All required hosted checks passed on the reviewed head, which does not transfer to subsequent changes. + +The first finding is addressed by `an_unsatisfied_retained_closure_refuses_snapshot_admission_before_output`. A production-admitted ext4 store contains a canonical layout but lacks its chunk. The fixture deliberately installs checksummed root/manifest/head evidence claiming that incomplete closure, outside publication; this is adversarial persisted state, not an assertion that production preflight permits it. Snapshot admission and the convenience reconstruction both preserve the exact namespace and `MissingMember::Chunk` coordinates. Caller output and selected root/manifest/head bytes remain unchanged. + +Focused debug/release, all-feature Clippy and source structure pass (`closure-refusal-corrected.log`). The initial Clippy tuple-complexity failure is preserved separately as authoring evidence. An isolated source and target bypasses closure verification only when a manifest is present, leaving all earlier physical/canonical admission intact; the new test was observed RED with “incomplete retained closure admitted a snapshot” (`closure-admission-mutation-red.log`). This directly challenges the new owning boundary rather than only the shared closure verifier. The remaining review finding requires the consolidated refusal-calibration map and its missing observations before re-review. + +## Refusal calibration requested by independent review + +The remaining explicitly identified refusal claims were challenged in isolated source/build copies of `4d801e491eebc580866d6fa1c18d1365035fa3a9`. These mutations change production behavior or returned diagnostics, not test expectations. `refusal-calibration-green.log` records the unmodified layout laws passing in debug/release and the retained-closure law passing again. Each mutant compiles and reaches the named runtime check; build failures are not counted. + +| Protected claim / assertion | Deliberate violation | Observed RED receipt | +| --- | --- | --- | +| Wrong whole-blob claims refuse before output | Skip complete-object verification for nonempty layouts | `proof-and-semantic-binding-red.log`: `false blob claim succeeded` | +| Content-correct false profile boundaries refuse | Same skipped complete-object verification pass | `proof-and-semantic-binding-red.log`: `false profile boundaries reconstructed` | +| Semantic range ingress requires exact catalogued binding | Execute the range core directly on the supplied layout | `proof-and-semantic-binding-red.log`: `uncatalogued layout succeeded` | +| Canonical record range ingress requires exact binding independently | Bypass binding only in record ingress; semantic ingress remains unchanged | `record-binding-red.log`: first semantic refusal passes, then `uncatalogued record succeeded` | +| Whole-record checksum refusal preserves expected/observed coordinates | Swap decoder checksum coordinates | `proof-and-semantic-binding-red.log`: whole-record checksum assertion fails | +| Range-record checksum refusal preserves coordinates independently | Swap checksum coordinates only at range ingress | `range-checksum-red.log`: whole-record assertion passes, then range-record checksum assertion fails | +| Snapshot admission proves selected retained closure | Skip closure verification only for a present manifest | `closure-admission-mutation-red.log`: `incomplete retained closure admitted a snapshot` | + +The grouped proof mutation falsifies two distinct promised outcomes: complete identity and storage-profile admission. The record-only mutations deliberately leave the earlier semantic/whole assertions intact so their failures cannot mask the later entrypoint assertions. The successful ingress-equivalence law remains green in all three layout mutation copies, demonstrating that ordinary successful reads alone would not detect these omissions. + +The earlier calibration receipts cover the other established claim families: emitted bytes and generated source-slice oracles (`emission-mutation-red.log`, `worldline-emission-mutation-red.log`); view generation (`coordinate-mutation-red.log`); actual collector exclusion (`fence-mutation-red.log`); accepted-prefix accounting (`output-prefix-mutation-red.log`, `range-zero-isolated-mutation-red.log`); immediate writer cause and maximum write count (`writer-parity-mutation-red.log`); additional whole-blob allocation (`read-memory-mutation-red.log`); and logical overlap independence (`range-overlap-mutation-red.log`). Historical witnesses retain their recorded source coordinates; the final independent review must assess whether the combined mapping satisfies the binding standard. This table does not turn unexecuted individual diagnostic-field mutations into evidence. + +## Shared-core extraction validation + +The copied Docker candidate passes all-target/all-feature Clippy and the unchanged reference private laws, generated range properties, streaming CAS suite and complete Golden File Worldline binary in both debug and release (`core-extraction-focused-corrected.log`). + +These generated tests retain their independent source-slice and frozen-corpus oracles; agreement does not prove untested input spaces or new filesystem concurrency guarantees. + +Markdown validation passes after removing one extra blank line (`core-markdown-corrected.log`); the original formatting failure remains in `core-markdown.log`. + +The local dependency-policy attempt could not start because this container has no `cargo-deny` installation (`core-dependency-validation.log`); dependency policy must be verified by the final-head hosted job rather than counted as a local pass. + +The stable-candidate full validation command sequence is retained in `core-final-validation.sh`, with output in `core-final-validation.log`; its completion and the final pushed SHA are recorded in the PR review activity rather than anticipated here. + +## Independent review of the extracted core + +The independent Codex reviewer assessed exact head `c0bd6fb960ec864d5e6c6de71fbab3dbdb6f6160` under the agy-review protocol and confirmed the four hosted findings were addressed. + +Its remaining P2 finding was a public documentation mismatch: store reads claimed no synchronization, while reader platform admission invokes root-directory `sync_all` before fencing. + +The full feedback and checklist were posted before correction at [the independent review](https://github.com/flyingrobots/keep/pull/164#issuecomment-5963396072). + +The documentation correction discloses the blocking directory synchronization and original I/O cause under `Admission`, while preserving the distinction from publication, caller-output flushing and a content-durability promise. + +Change kind for that correction: documentation-only; the oracle is the unchanged production call chain `DurableStore::snapshot` → `DurableSnapshot::open` → `FilesystemRetentionSnapshot::load` → `open_version_two` → `admit_linux_profile` → `sync_all`, not an artificial runtime regression. + +The initial full local chain passed its corpus, debug/release crash, structure, formatting, feature and Clippy steps, then stopped at an existing xtask test that clones the source repository because the Docker validation copy had no commit to clone. + +Creating a local validation-copy commit corrected that environment without changing the source tree: both the pushed `c0bd6fb` and the committed copy have tree `f3bc4733a444af62c945805dcb3eb9c1335fc23a`. + +The affected exact law then passed, followed by complete debug/release workspace tests, doctests, documentation, MSRV and fuzz-target check/Clippy (`core-final-validation-corrected.log`, exit zero); the original failure remains in `core-final-validation.log` and is not counted as a product regression or silently retried away. + +Current-head hosted checks and independent review outcomes are recorded in the [PR activity](https://github.com/flyingrobots/keep/pull/164); this committed implementation evidence does not substitute for those exact-head gates. + +## CodeRabbit fixture follow-up + +CodeRabbit's review of `0a19dde68b2e6bf0fd46610b753f7d5974df7c83` raised a live-status wording concern and two test-fixture isolation concerns; it introduced no new production-read finding. + +Change kind: test-infrastructure correction and documentation clarification; no production statement, API, format, assertion expectation or storage policy changes. + +| Review obligation | Disposition and evidence | Exit condition | +| --- | --- | --- | +| Avoid a stale pending-review sentence (`discussion_r4170862136`). | The committed document now points to current PR activity for exact-head status instead of anticipating approval of its own commit. Historical evidence remains pinned. | Current-head review and checks are recorded on the PR. | +| Avoid tmpfs collisions after PID reuse (`discussion_r4170862139`). | Scratch creation atomically tries bounded deterministic suffixes, skips existing names and never removes a name it did not create. No clock, ambient randomness or dependency is added. | Existing platform laws pass with the initial scratch name deliberately occupied; occupied sentinel files remain unchanged. | +| Restore cwd on failing isolated locator paths (`discussion_r4170862148`). | A scoped guard attempts restoration on error/unwind, while normal-path restoration remains checked. A child marker alone no longer enables in-process execution: its law name and complete single-law command arguments must match. | Existing debug/release locator laws pass even with an inherited stale child marker; independent inspection confirms guarded cleanup on early exits. | + +The tmpfs creation loop admits at most 1,024 candidate names before returning an explicit setup error; this is a finite setup-work cap, not a measured latency guarantee. + +A controlled subprocess occupied both the old PID-only path and the revised first-suffix path before executing the existing direct-reader law with that same PID. + +The old fixture failed with `AlreadyExists` before reaching its product assertion (`tmpfs-collision-before.log`); the revised fixture reached and passed the unchanged typed platform-refusal assertion (`tmpfs-collision-after.log`), leaving both occupied witnesses intact (`tmpfs-collision-preserved.log`). + +That before-state is fixture-setup failure evidence, not a product RED or additional proof of Keep's runtime semantics. + +The cwd guard's early-exit coverage is source inspection; normal-path runtime evidence still comes from the existing isolated public locator laws, not a new test of helper choreography. + +Focused platform and locator laws pass in debug/release and Clippy passes with warnings denied (`fixture-followup-validation-corrected.log`); an earlier edit-script mismatch left the source unchanged, so the preceding `fixture-followup-validation.log` is not evidence for the revised fixtures. + +No test was deleted, and no product expectation was weakened to obtain these passes. + +## Landing allocation-contract correction + +The fresh landing review found that the README example and snapshot API documentation incorrectly included catalog bytes in the caller's aggregate byte budget. The unchanged loader reads the catalog under `CatalogLength::MAXIMUM` before applying `CatalogRestartPolicy::retained_segment_bytes` to selected segments; decoded indexes and retention records have separate bounds and allocate additionally. + +Change kind: documentation-only. The corrected README, API, normative requirements and current evidence distinguish these allocations without changing runtime admission or inventing a total-memory cap. Earlier runtime receipts remain historical evidence; their wording does not establish an aggregate cap covering the catalog. The source oracle is `catalog_restart_loader.rs:67–80`, `catalog_restart_segments.rs:75–85`, and `filesystem_catalog_snapshot.rs:61–96`; compiled examples and documentation checks passed for tree `59a921a04283046c67a1e74ffabefdf6c91512ec`, and independent delta review approved `8794c9e` before the later CodeRabbit findings reopened acceptance. All four hosted jobs subsequently passed for that same head in run `37158167907`. These completed receipts do not transfer approval to a successor; current review and check status is recorded on the PR. + +## Landing caller-input error boundary + +Change kind: bug fix with correction of an existing erroneous oracle. CodeRabbit's late review of `8794c9ec6a349fabbfaef72f11dcfb47eaae2869` identified that caller-supplied malformed layout records and corrupt committed layouts shared the committed-record error variant. The reference adapter and public variant documentation require distinct boundaries. + +The prior combined checksum law is replaced by separate whole-record and range-record laws; neither exact checksum-coordinate assertion nor output-preservation obligation is removed. Each independently fails on the unfixed production source at `8794c9e` with `DurableReadError::LayoutDecode` (`164-caller-decode-red.log`). Both now expect their operation's nested input-decode error; committed catalog decoding keeps the original variant. The copied-Docker `durable_layout_laws` suite passes in debug and release (`164-caller-decode-green.log`). No serialization, stored bytes, successful read behavior or admission policy changes. Final successor review and hosted checks remain separate PR gates. + +## Landing diagnostic rendering and review reconciliation + +Change kind: bug fix. Public durable store/read wrappers previously printed their source and exposed that same source again through `Error::source`. Controlled public reconstruction, range and absent-store failures reach the repeated-message assertions on unfixed `8794c9e` (`164-diagnostic-red.log`). With only the outer wrapper corrected, both inner wrapper assertions independently fail (`164-diagnostic-inner-red.log`); no earlier assertion masks those checks. Fixed contextual messages preserve the original typed source chain. This claim covers the new durable wrappers, not every older inner error's rendering convention. + +The three diagnostic laws pass in copied-Docker debug and release. The first focused Clippy pass then rejected an unquoted `Error::source` doc identifier, after the runtime laws passed; the comment was corrected without changing their oracle. Full candidate validation and independent successor review are separate final gates. + +CodeRabbit's terminology clarification is applied across current public allocation claims: the budget covers catalog-selected segment bytes, including bytes backing unretained layouts, and excludes catalog/metadata allocations. The runtime policy field name is unchanged. The earlier completed documentation review is recorded with its actual coordinates above; it is not represented as current successor approval. + +The range-corruption law now names unexpected range success in its failure message. The requested fixture extraction is declined: these two small explicit laws independently display normative v1 framing and their checksum oracle. Testing Standards Rule 18 prefers clarity over abstraction or forced duplication; no current framing inconsistency or weakened assertion was demonstrated. Both existing runtime laws remain registered and unchanged in expectation. diff --git a/src/adapters/authenticated_read/mod.rs b/src/adapters/authenticated_read/mod.rs new file mode 100644 index 00000000..b87c8b8b --- /dev/null +++ b/src/adapters/authenticated_read/mod.rs @@ -0,0 +1,17 @@ +//! This module owns the public read error boundary shared by storage adapters. +//! +//! It combines codec/lookup refusals with lossless translation of semantic +//! core failures. Authentication policy remains in the inward read core. + +mod profile_error_mapping; +mod range_failure_mapping; +mod range_read_error; +mod range_read_error_display; +mod range_read_error_mapping; +mod reconstruction_error; +mod reconstruction_error_display; +mod reconstruction_error_mapping; +mod reconstruction_failure_mapping; + +pub use range_read_error::RangeReadError; +pub use reconstruction_error::ReconstructionError; diff --git a/src/adapters/authenticated_read/profile_error_mapping.rs b/src/adapters/authenticated_read/profile_error_mapping.rs new file mode 100644 index 00000000..62fcacc9 --- /dev/null +++ b/src/adapters/authenticated_read/profile_error_mapping.rs @@ -0,0 +1,29 @@ +//! This module owns lossless outward mapping of storage-profile failures. + +use super::ReconstructionError; +use crate::LayoutId; +use crate::profile::StorageProfileVerificationError; + +pub(super) const fn profile_error( + layout: LayoutId, + error: StorageProfileVerificationError, +) -> ReconstructionError { + match error { + StorageProfileVerificationError::Unsupported { profile } => { + ReconstructionError::ProfileVerifierUnavailable { layout, profile } + } + StorageProfileVerificationError::Chunking { source } => { + ReconstructionError::ProfileChunking { layout, source } + } + StorageProfileVerificationError::BoundaryMismatch { + index, + expected, + observed, + } => ReconstructionError::ProfileBoundaryMismatch { + layout, + index, + expected, + observed, + }, + } +} diff --git a/src/adapters/authenticated_read/range_failure_mapping.rs b/src/adapters/authenticated_read/range_failure_mapping.rs new file mode 100644 index 00000000..b34d83ed --- /dev/null +++ b/src/adapters/authenticated_read/range_failure_mapping.rs @@ -0,0 +1,44 @@ +//! This module owns lossless translation of semantic range failures. + +use super::RangeReadError; +use super::range_read_error_mapping::{range_chunk_error, range_output_error}; +use crate::authenticated_read::RangeReadFailure; + +impl From for RangeReadError { + fn from(failure: RangeReadFailure) -> Self { + match failure { + RangeReadFailure::RangePlan(source) => Self::RangePlan(source), + RangeReadFailure::Chunk(source) => range_chunk_error(source), + RangeReadFailure::PlanEntriesUnavailable { + first, + end, + available, + } => Self::PlanEntriesUnavailable { + first, + end, + available, + }, + RangeReadFailure::ChunkSliceUnavailable { + layout, + index, + requested, + chunk, + } => Self::ChunkSliceUnavailable { + layout, + index, + requested, + chunk, + }, + RangeReadFailure::Output { layout, source } => range_output_error(layout, source), + RangeReadFailure::WrittenLengthMismatch { + layout, + expected, + observed, + } => Self::WrittenLengthMismatch { + layout, + expected, + observed, + }, + } + } +} diff --git a/src/reference/range_read_error.rs b/src/adapters/authenticated_read/range_read_error.rs similarity index 100% rename from src/reference/range_read_error.rs rename to src/adapters/authenticated_read/range_read_error.rs diff --git a/src/reference/range_read_error_display.rs b/src/adapters/authenticated_read/range_read_error_display.rs similarity index 100% rename from src/reference/range_read_error_display.rs rename to src/adapters/authenticated_read/range_read_error_display.rs diff --git a/src/reference/range_read_error_mapping.rs b/src/adapters/authenticated_read/range_read_error_mapping.rs similarity index 95% rename from src/reference/range_read_error_mapping.rs rename to src/adapters/authenticated_read/range_read_error_mapping.rs index 3b62f410..3eea7e43 100644 --- a/src/reference/range_read_error_mapping.rs +++ b/src/adapters/authenticated_read/range_read_error_mapping.rs @@ -3,8 +3,8 @@ use crate::{ByteLength, LayoutId}; use super::RangeReadError; -use super::chunk_verification::ChunkVerificationError; -use super::output_write::OutputWriteError; +use crate::authenticated_read::ChunkVerificationError; +use crate::authenticated_read::OutputWriteError; pub(super) const fn range_chunk_error(error: ChunkVerificationError) -> RangeReadError { match error { diff --git a/src/reference/reconstruction_error.rs b/src/adapters/authenticated_read/reconstruction_error.rs similarity index 100% rename from src/reference/reconstruction_error.rs rename to src/adapters/authenticated_read/reconstruction_error.rs diff --git a/src/reference/reconstruction_error_display.rs b/src/adapters/authenticated_read/reconstruction_error_display.rs similarity index 100% rename from src/reference/reconstruction_error_display.rs rename to src/adapters/authenticated_read/reconstruction_error_display.rs diff --git a/src/adapters/authenticated_read/reconstruction_error_mapping.rs b/src/adapters/authenticated_read/reconstruction_error_mapping.rs new file mode 100644 index 00000000..c806a58e --- /dev/null +++ b/src/adapters/authenticated_read/reconstruction_error_mapping.rs @@ -0,0 +1,81 @@ +//! This module owns lossless outward mapping of chunk and output failures. + +use super::ReconstructionError; +use crate::authenticated_read::{ChunkVerificationError, OutputWriteError}; +use crate::{BlobLength, LayoutId}; + +pub(super) const fn reconstruction_chunk_error( + error: ChunkVerificationError, +) -> ReconstructionError { + match error { + ChunkVerificationError::Missing { + layout, + index, + requested, + } => ReconstructionError::ChunkMissing { + layout, + index, + requested, + }, + ChunkVerificationError::Hash { + layout, + index, + expected, + source, + } => ReconstructionError::ChunkHash { + layout, + index, + expected, + source, + }, + ChunkVerificationError::IdentityMismatch { + layout, + index, + expected, + observed, + } => ReconstructionError::ChunkIdentityMismatch { + layout, + index, + expected, + observed, + }, + } +} + +pub(super) fn reconstruction_output_error( + layout: LayoutId, + error: OutputWriteError, +) -> ReconstructionError { + match error { + OutputWriteError::WriteZero { bytes_written } => ReconstructionError::WriteZero { + layout, + bytes_written: BlobLength::new(bytes_written), + }, + OutputWriteError::InvalidWriteCount { + maximum, + observed, + bytes_written, + } => ReconstructionError::InvalidWriteCount { + layout, + maximum, + observed, + bytes_written: BlobLength::new(bytes_written), + }, + OutputWriteError::Write { + bytes_written, + source, + } => ReconstructionError::Write { + layout, + bytes_written: BlobLength::new(bytes_written), + source, + }, + OutputWriteError::LengthOverflow { + bytes_written, + incoming, + } => ReconstructionError::WrittenLengthOverflow { + layout, + bytes_written: BlobLength::new(bytes_written), + incoming, + }, + } +} diff --git a/src/adapters/authenticated_read/reconstruction_failure_mapping.rs b/src/adapters/authenticated_read/reconstruction_failure_mapping.rs new file mode 100644 index 00000000..bc1f1710 --- /dev/null +++ b/src/adapters/authenticated_read/reconstruction_failure_mapping.rs @@ -0,0 +1,39 @@ +//! This module owns lossless translation of semantic reconstruction failures. + +use super::ReconstructionError; +use super::profile_error_mapping::profile_error; +use super::reconstruction_error_mapping::{ + reconstruction_chunk_error, reconstruction_output_error, +}; +use crate::authenticated_read::ReconstructionFailure; + +impl From for ReconstructionError { + fn from(failure: ReconstructionFailure) -> Self { + match failure { + ReconstructionFailure::Chunk(source) => reconstruction_chunk_error(source), + ReconstructionFailure::BlobHash(source) => Self::BlobHash(source), + ReconstructionFailure::BlobIdentityMismatch { + layout, + expected, + observed, + } => Self::BlobIdentityMismatch { + layout, + expected, + observed, + }, + ReconstructionFailure::Profile { layout, source } => profile_error(layout, source), + ReconstructionFailure::Output { layout, source } => { + reconstruction_output_error(layout, source) + } + ReconstructionFailure::WrittenLengthMismatch { + layout, + expected, + observed, + } => Self::WrittenLengthMismatch { + layout, + expected, + observed, + }, + } + } +} diff --git a/src/adapters/durable/error.rs b/src/adapters/durable/error.rs new file mode 100644 index 00000000..596bfeb1 --- /dev/null +++ b/src/adapters/durable/error.rs @@ -0,0 +1,133 @@ +//! This boundary module owns typed durable-read failures, keeping evidenced +//! refusal apart from operational failure as the reconstruction contract +//! requires. + +use std::error::Error; +use std::fmt; +use std::io; + +use crate::adapters::{CatalogRestartError, FilesystemRetentionSnapshotError}; +use crate::{ + BlobId, LayoutDecodeError, LayoutId, RangeReadError, ReconstructionError, + RetentionClosureVerificationError, RetentionNamespaceDigest, RetentionRootDecodeError, +}; + +/// Why a durable store or snapshot could not be opened. +#[derive(Debug)] +pub enum DurableStoreError { + /// The store locator could not be resolved at handle construction. + Locator { + /// Original path-resolution failure, preserved without stringification. + source: io::Error, + }, + /// The root did not admit as a version-two store, the fence could not + /// be taken, or one consistent view could not be collected. + Snapshot(Box), + /// The pinned catalog could not be re-admitted. + Catalog(Box), + /// The manifest-selected root was unexpectedly absent. + RootMissing { + /// The namespace selected by the manifest. + namespace: RetentionNamespaceDigest, + }, + /// A selected root failed canonical admission. + RootDecode { + /// The namespace whose root refused. + namespace: RetentionNamespaceDigest, + /// The exact refusal. + source: RetentionRootDecodeError, + }, + /// A retained closure could not be proven against the pinned catalog. + Closure { + /// The namespace whose closure refused. + namespace: RetentionNamespaceDigest, + /// The exact closure refusal. + source: Box, + }, +} + +impl fmt::Display for DurableStoreError { + fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + Self::Locator { .. } => { + formatter.write_str("the durable store locator could not be resolved") + } + Self::Snapshot(_) => formatter.write_str("the durable view could not be pinned"), + Self::Catalog(_) => formatter.write_str("the pinned catalog refused"), + Self::RootMissing { .. } => formatter.write_str("the selected root is absent"), + Self::RootDecode { .. } => formatter.write_str("a retained root refused"), + Self::Closure { .. } => formatter.write_str("a retained closure refused"), + } + } +} + +impl Error for DurableStoreError { + fn source(&self) -> Option<&(dyn Error + 'static)> { + match self { + Self::Locator { source } => Some(source), + Self::Snapshot(source) => Some(source.as_ref()), + Self::Catalog(source) => Some(source.as_ref()), + Self::RootMissing { .. } => None, + Self::RootDecode { source, .. } => Some(source), + Self::Closure { source, .. } => Some(source.as_ref()), + } + } +} + +/// Why one durable read did not return a receipt. +/// +/// Source variants retain the exact admission, corruption or I/O boundary; +/// they must not all be interpreted as content corruption. Missing identities +/// are evidenced against the admitted view. Reconstruction and range errors +/// preserve output failures separately from content refusal. +#[derive(Debug)] +pub enum DurableReadError { + /// Retained anchor evidence could not be read or admitted. + Retention(Box), + /// The pinned catalog could not be re-admitted for this read. + View(Box), + /// No retained root anchors the blob in this view. + BlobMissing { + /// The requested blob. + requested: BlobId, + }, + /// The catalog names no layout record under the identity. + LayoutMissing { + /// The requested layout. + requested: LayoutId, + }, + /// The committed layout record does not decode as the identity that + /// names it. + LayoutDecode(LayoutDecodeError), + /// The reconstruction core refused or its output failed. + Reconstruction(Box), + /// The range core refused or its output failed. + RangeRead(Box), +} + +impl fmt::Display for DurableReadError { + fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + Self::Retention(_) => formatter.write_str("the retained anchor evidence refused"), + Self::View(_) => formatter.write_str("the pinned view could not be re-admitted"), + Self::BlobMissing { .. } => formatter.write_str("no retained root anchors the blob"), + Self::LayoutMissing { .. } => formatter.write_str("the catalog names no such layout"), + Self::LayoutDecode(_) => formatter.write_str("committed layout refused"), + Self::Reconstruction(_) => formatter.write_str("durable reconstruction failed"), + Self::RangeRead(_) => formatter.write_str("durable range read failed"), + } + } +} + +impl Error for DurableReadError { + fn source(&self) -> Option<&(dyn Error + 'static)> { + match self { + Self::Retention(source) => Some(source.as_ref()), + Self::View(source) => Some(source.as_ref()), + Self::LayoutDecode(source) => Some(source), + Self::Reconstruction(source) => Some(source.as_ref()), + Self::RangeRead(source) => Some(source.as_ref()), + Self::BlobMissing { .. } | Self::LayoutMissing { .. } => None, + } + } +} diff --git a/src/adapters/durable/layout_reads.rs b/src/adapters/durable/layout_reads.rs new file mode 100644 index 00000000..42669cd3 --- /dev/null +++ b/src/adapters/durable/layout_reads.rs @@ -0,0 +1,127 @@ +//! This module owns caller-supplied layout ingress for fenced durable reads. + +use std::io::Write; + +use super::snapshot::CatalogChunks; +use super::{ + DurableRangeReadReceipt, DurableReadError, DurableReconstructionReceipt, DurableSnapshot, +}; +use crate::authenticated_read::reconstruct_admitted; +use crate::{AdmittedLayout, ByteRange, LayoutDecodePolicy, RangeReadError, ReconstructionError}; + +impl DurableSnapshot { + /// Reconstructs through a caller-supplied admitted semantic layout. + /// + /// The layout need not be catalogued, but every chunk it names must exist + /// in this immutable catalog. The core authenticates the complete blob and + /// profile before output. Canonical identity calculation materializes one + /// bounded layout record; catalog re-admission has the costs documented on + /// [`DurableSnapshot`]. No whole-blob buffer or durability effect is added. + /// + /// # Errors + /// + /// Returns the precise layout-encoding, catalog, reconstruction or output + /// failure, retaining the original source and accepted-prefix accounting. + pub fn reconstruct_admitted_layout( + &self, + layout: &AdmittedLayout, + output: &mut W, + ) -> Result + where + W: Write + ?Sized, + { + let identity = layout + .encode_record() + .map_err(|source| { + DurableReadError::Reconstruction(Box::new(ReconstructionError::LayoutEncoding( + source, + ))) + })? + .id(); + let catalog = self.catalog()?; + reconstruct_admitted(&CatalogChunks::new(&catalog), identity, layout, output) + .map(|receipt| DurableReconstructionReceipt::new(receipt, self.view())) + .map_err(|source| DurableReadError::Reconstruction(Box::new(source.into()))) + } + + /// Decodes a bounded canonical layout record and reconstructs its exact blob. + /// + /// Decoding allocates bounded entry metadata under `policy` before content + /// lookup or emission. The allocation, verification and blocking contract + /// then follows [`Self::reconstruct_admitted_layout`]. + /// + /// # Errors + /// + /// Returns exact decoder refusal for invalid or policy-exceeding input, + /// otherwise the errors of [`Self::reconstruct_admitted_layout`]. + pub fn reconstruct_record( + &self, + encoded: &[u8], + policy: LayoutDecodePolicy, + output: &mut W, + ) -> Result + where + W: Write + ?Sized, + { + let layout = AdmittedLayout::decode_record(encoded, policy).map_err(|source| { + DurableReadError::Reconstruction(Box::new(ReconstructionError::LayoutDecode(source))) + })?; + self.reconstruct_admitted_layout(&layout, output) + } + + /// Resolves a supplied layout identity to an exact catalogued range. + /// + /// The supplied layout only determines its canonical identity. That exact + /// layout must exist in the catalog; range planning uses the committed + /// record rather than caller-supplied target claims. Identity calculation + /// allocates one bounded canonical record; remaining costs follow + /// [`Self::read_layout_range`]. No complete blob buffer is created. + /// + /// # Errors + /// + /// Returns exact encoding or missing-layout refusal, otherwise the errors + /// of [`Self::read_layout_range`], without substituting another layout. + pub fn read_admitted_layout_range( + &self, + layout: &AdmittedLayout, + requested: ByteRange, + output: &mut W, + ) -> Result + where + W: Write + ?Sized, + { + let identity = layout + .encode_record() + .map_err(|source| { + DurableReadError::RangeRead(Box::new(RangeReadError::LayoutEncoding(source))) + })? + .id(); + self.read_layout_range(identity, requested, output) + } + + /// Decodes a canonical layout record and resolves its exact catalogued range. + /// + /// Decoding admits bounded metadata under `policy`; then + /// [`Self::read_admitted_layout_range`] establishes catalog membership and + /// performs verification and blocking output with the same allocation costs. + /// + /// # Errors + /// + /// Returns exact decoder refusal before catalog lookup or output, + /// otherwise the errors of [`Self::read_admitted_layout_range`]. + pub fn read_record_range( + &self, + encoded: &[u8], + policy: LayoutDecodePolicy, + requested: ByteRange, + output: &mut W, + ) -> Result + where + W: Write + ?Sized, + { + let layout = AdmittedLayout::decode_record(encoded, policy).map_err(|source| { + DurableReadError::RangeRead(Box::new(RangeReadError::LayoutDecode(source))) + })?; + self.read_admitted_layout_range(&layout, requested, output) + } +} diff --git a/src/adapters/durable/mod.rs b/src/adapters/durable/mod.rs new file mode 100644 index 00000000..2e4d166c --- /dev/null +++ b/src/adapters/durable/mod.rs @@ -0,0 +1,15 @@ +//! This module owns authenticated reads through an admitted fenced durable view. + +mod error; +mod layout_reads; +mod receipt; +mod retained_anchors; +mod snapshot; +mod store; +mod view; + +pub use error::{DurableReadError, DurableStoreError}; +pub use receipt::{DurableRangeReadReceipt, DurableReconstructionReceipt}; +pub use snapshot::DurableSnapshot; +pub use store::{DurableOutcome, DurableStore}; +pub use view::DurableView; diff --git a/src/adapters/durable/rationale.md b/src/adapters/durable/rationale.md new file mode 100644 index 00000000..7b0022a5 --- /dev/null +++ b/src/adapters/durable/rationale.md @@ -0,0 +1,25 @@ +# Fenced durable authenticated reads + +The read adapter composes the existing immutable catalog snapshot, shared reader fence, retention root admission and authenticated reconstruction cores. It performs no publication, deletion or repair. An opened store handle is a locator; only snapshot admission establishes a readable view. + +Handle construction resolves a relative locator to an absolute path once and preserves any resolution failure as a typed locator error. Resolving on every read was rejected because an unrelated working-directory change could silently select another valid store. This does not canonicalize or pin a directory capability, and every snapshot still admits the named store. The new constructor is fallible; resolving a relative path may query the current directory but does not open or verify the store. + +A durable snapshot keeps the fence and selected catalog alive for every borrowed read. Snapshot admission verifies every manifest-selected retained closure against that catalog. Blob lookup chooses the lowest canonical retained layout identity; exact-layout reads may name any layout in the admitted catalog, including an unretained one, while the fence protects the view. + +The existing catalog loader materializes selected segment bytes under the caller's aggregate segment-byte policy. It separately materializes catalog bytes under `CatalogLength::MAXIMUM`, with additional decoded indexes bounded by format and record-count limits. The segment budget does not cap total snapshot memory. This cost is explicit in the API documentation; it is not a lazy segment reader. Reconstruction adds no whole-blob buffer. Selected roots are loaded one at a time rather than accumulating a second index of all anchors. The format bounds each root and manifest, and each root supplies traversal counters. This bounds memory without inventing another on-disk limit. + +The [domain read core](../../authenticated_read/rationale.md) owns the shared verification and emission policy; both storage adapters depend inward on it. Codec-bearing public errors are assembled at the shared adapter boundary without changing their variants or sources. + +The shared crate-private chunk source must return immutable bytes throughout verification and emission. The reference map and durable catalog satisfy this requirement through owned immutable storage. Both cores preserve the verify-before-output contract and single hash pass; no public mutable or callback-provided source is admitted through this internal boundary. + +Refusal and operational failure remain distinct typed sources. Missing logical content is evidenced against an admitted view; inability to open a physical segment is a catalog I/O failure. Output failures preserve the exact accepted prefix through the existing reconstruction/range errors. A receipt is constructed only after successful emission and includes the complete admitted retention head and catalog coordinates. + +The writer lock and reader fence coordinate cooperating Keep operations in a managed namespace. They do not isolate arbitrary concurrent raw filesystem mutation. Existing exact-byte, identity, namespace and corruption checks remain in force, including selected-root re-admission during lookup. + +Readers reuse version-two platform admission before consuming migration records and retain the admitted directory capability through collection. Consistent record bytes alone cannot establish the filesystem semantics assumed by the shared fence and immutable pools. Reusing writer authority for this check was rejected because readers must coexist with a writer; the platform check itself acquires no writer lock. Unsupported filesystems now refuse through the existing admission error boundary. + +Platform admission performs a blocking root-directory synchronization before acquiring the shared fence. A synchronization failure preserves its original I/O cause under reader `Admission`, nested under the durable snapshot error. This probe does not publish content, flush caller output or grant a new content-durability guarantee. An existing snapshot read does not repeat this platform probe; a convenience read opens a new snapshot and therefore pays it again. + +Selected-root admission binds the canonical root's namespace to the selecting manifest entry in addition to its digest and generation. A canonical root with a valid complete closure still refuses if another namespace selects it; the existing root error retains typed expected and observed namespace digests. This check belongs to the shared root-read boundary so initial admission and subsequent anchor lookup enforce the same rule. + +The delivery checklist and remaining evidence obligations are recorded in [the #109 evidence ledger](../../../docs/testing-evidence/durable-authenticated-reads.md). This rationale does not assert that the issue's acceptance checks are already complete. diff --git a/src/adapters/durable/receipt.rs b/src/adapters/durable/receipt.rs new file mode 100644 index 00000000..7442ca06 --- /dev/null +++ b/src/adapters/durable/receipt.rs @@ -0,0 +1,55 @@ +//! This boundary module owns durable read receipts: the reference receipt +//! plus the view it was established against. + +use super::DurableView; +use crate::{RangeReadReceipt, ReconstructionReceipt}; + +/// One authenticated complete reconstruction against one durable view. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +#[must_use = "the receipt records the authenticated identity, length, and view"] +pub struct DurableReconstructionReceipt { + receipt: ReconstructionReceipt, + view: DurableView, +} + +impl DurableReconstructionReceipt { + pub(super) const fn new(receipt: ReconstructionReceipt, view: DurableView) -> Self { + Self { receipt, view } + } + + /// The target, exact layout, and emitted length authenticated. + pub const fn receipt(self) -> ReconstructionReceipt { + self.receipt + } + + /// The view the reconstruction was established against. + #[must_use] + pub const fn view(self) -> DurableView { + self.view + } +} + +/// One authenticated exact range read against one durable view. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +#[must_use = "the receipt records the authenticated range, layout, and view"] +pub struct DurableRangeReadReceipt { + receipt: RangeReadReceipt, + view: DurableView, +} + +impl DurableRangeReadReceipt { + pub(super) const fn new(receipt: RangeReadReceipt, view: DurableView) -> Self { + Self { receipt, view } + } + + /// The target, exact layout, requested range, and emitted length. + pub const fn receipt(self) -> RangeReadReceipt { + self.receipt + } + + /// The view the range was established against. + #[must_use] + pub const fn view(self) -> DurableView { + self.view + } +} diff --git a/src/adapters/durable/retained_anchors.rs b/src/adapters/durable/retained_anchors.rs new file mode 100644 index 00000000..78ebbfac --- /dev/null +++ b/src/adapters/durable/retained_anchors.rs @@ -0,0 +1,64 @@ +//! This module owns bounded selected-root admission and canonical anchor lookup. + +use super::DurableStoreError; +use crate::adapters::{ + AdmittedRetentionRoot, FilesystemRetentionSnapshot, verify_retention_closure, +}; +use crate::{BlobId, LayoutId, RetentionNamespaceDigest}; + +pub(super) fn verify(view: &FilesystemRetentionSnapshot) -> Result<(), DurableStoreError> { + let Some(manifest) = view.manifest() else { + return Ok(()); + }; + let catalog = view + .catalog() + .snapshot() + .map_err(|source| DurableStoreError::Catalog(Box::new(source)))?; + for entry in manifest.entries() { + let namespace = entry.namespace(); + let bytes = root_bytes(view, namespace)?; + let root = AdmittedRetentionRoot::decode(&bytes) + .map_err(|source| DurableStoreError::RootDecode { namespace, source })?; + let _verified = verify_retention_closure(root.root(), &catalog).map_err(|source| { + DurableStoreError::Closure { + namespace, + source: Box::new(source), + } + })?; + } + Ok(()) +} + +pub(super) fn first_layout( + view: &FilesystemRetentionSnapshot, + target: BlobId, +) -> Result, DurableStoreError> { + let Some(manifest) = view.manifest() else { + return Ok(None); + }; + let mut selected: Option = None; + for entry in manifest.entries() { + let namespace = entry.namespace(); + let bytes = root_bytes(view, namespace)?; + let root = AdmittedRetentionRoot::decode(&bytes) + .map_err(|source| DurableStoreError::RootDecode { namespace, source })?; + for anchor in root.root().anchors() { + if anchor.blob_id() == target { + selected = Some(selected.map_or_else( + || anchor.layout_id(), + |previous| previous.min(anchor.layout_id()), + )); + } + } + } + Ok(selected) +} + +fn root_bytes( + view: &FilesystemRetentionSnapshot, + namespace: RetentionNamespaceDigest, +) -> Result, DurableStoreError> { + view.retained_root(namespace) + .map_err(|source| DurableStoreError::Snapshot(Box::new(source)))? + .ok_or(DurableStoreError::RootMissing { namespace }) +} diff --git a/src/adapters/durable/snapshot.rs b/src/adapters/durable/snapshot.rs new file mode 100644 index 00000000..dca8c1e3 --- /dev/null +++ b/src/adapters/durable/snapshot.rs @@ -0,0 +1,244 @@ +//! This module owns one pinned durable view and the reads it answers. + +use std::io::Write; +use std::path::Path; + +use super::{ + DurableRangeReadReceipt, DurableReadError, DurableReconstructionReceipt, DurableStoreError, + DurableView, +}; +use crate::adapters::{ + AdmittedSegmentRecord, CatalogRestartPolicy, CatalogSnapshot, FilesystemRetentionSnapshot, + LayoutDecodePolicy, ReaderAttemptLimit, SegmentRecordIdentity, +}; +use crate::authenticated_read::{ChunkSource, read_admitted, reconstruct_admitted}; +use crate::{AdmittedLayout, BlobId, ByteRange, ChunkId, LayoutId}; + +/// One consistent, fenced view of a version-two store. +/// +/// The snapshot owns the shared reader fence for its lifetime, so no +/// collector can retire a segment it may read; every read borrows the +/// snapshot, so dropping the view mid-read is impossible. Blobs resolve +/// through the retained roots' anchors; layouts and chunks resolve through +/// the pinned catalog and are authenticated by the domain read cores +/// before a byte is emitted. +/// +/// Opening materializes selected segment bytes within the aggregate segment-byte +/// limit in `CatalogRestartPolicy`. Catalog bytes are bounded separately by +/// [`crate::CatalogLength::MAXIMUM`]; decoded indexes allocate additionally under +/// format and record-count limits. The segment-byte limit is not a total +/// snapshot-memory cap, and this is not a lazy segment reader. +/// Retained roots are decoded and their closures verified +/// one at a time under their stored traversal limits. Reads decode one bounded +/// layout and stream authenticated chunks to the caller without assembling an +/// additional whole-blob buffer. Caller-owned output may allocate separately. +/// Each read also re-admits the already materialized catalog and segment bytes, +/// allocating bounded decoded indexes; even a short range pays that cost. +/// +/// Blob lookup performs synchronous reads over the manifest-selected roots, +/// retaining at most one root's bytes and decoded anchors at a time. Its cost +/// is linear in those roots and anchors. A lookup rechecks canonical identity; +/// substituted or unreadable evidence fails rather than becoming absence. +/// The shared fence remains held across caller output callbacks so collection +/// cannot invalidate the read. No writer lock is acquired by these reads; +/// callers must not make output wait for an exclusive collector fence that +/// their own live snapshot prevents from being acquired. +#[must_use] +pub struct DurableSnapshot { + view: FilesystemRetentionSnapshot, + coordinates: DurableView, + policy: CatalogRestartPolicy, +} + +/// The pinned catalog as a chunk source: every chunk record's exact payload. +pub(super) struct CatalogChunks<'snapshot, 'head, 'catalog, 'records> { + catalog: &'snapshot CatalogSnapshot<'head, 'catalog, 'records>, +} + +impl<'snapshot, 'head, 'catalog, 'records> CatalogChunks<'snapshot, 'head, 'catalog, 'records> { + pub(super) const fn new( + catalog: &'snapshot CatalogSnapshot<'head, 'catalog, 'records>, + ) -> Self { + Self { catalog } + } +} + +impl ChunkSource for CatalogChunks<'_, '_, '_, '_> { + fn chunk(&self, identity: ChunkId) -> Option<&[u8]> { + self.catalog + .record(SegmentRecordIdentity::Chunk(identity)) + .map(AdmittedSegmentRecord::payload) + } +} + +impl DurableSnapshot { + /// Admits `store_root` as a version-two store, acquires the shared + /// reader fence, double-collects one consistent view within `limit` + /// attempts, and verifies every retained root's closure. + /// + /// The reader enforces the existing local writable, case-sensitive Linux + /// ext4 profile without acquiring writer authority. The admitted directory + /// capability is retained through view collection. Platform admission + /// synchronizes the root directory before fencing; a synchronization + /// failure is preserved under the snapshot's admission error. + /// + /// The aggregate byte limit in `policy` covers catalog-selected segment bytes only. + /// Catalog bytes use [`crate::CatalogLength::MAXIMUM`] independently; decoded + /// indexes and retention records allocate additionally under their format + /// and record-count limits. Closure work is bounded per root by its persisted limits. + /// This is blocking filesystem I/O and CPU verification, not publication; + /// it performs no namespace writes and establishes no new durability. + /// + /// # Errors + /// + /// Returns [`DurableStoreError`] at the exact admission, fence, + /// collection, or retained-root refusal. + pub fn open( + store_root: &Path, + policy: CatalogRestartPolicy, + limit: ReaderAttemptLimit, + ) -> Result { + let view = FilesystemRetentionSnapshot::load(store_root, policy, limit) + .map_err(|source| DurableStoreError::Snapshot(Box::new(source)))?; + let coordinates = DurableView::new( + view.catalog().generation(), + view.catalog().catalog_digest(), + view.retention_head().copied(), + ); + super::retained_anchors::verify(&view)?; + Ok(Self { + view, + coordinates, + policy, + }) + } + + /// The exact view this snapshot pinned. + #[must_use] + pub const fn view(&self) -> DurableView { + self.coordinates + } + + /// Whether some retained root anchors `target` in this view. + /// + /// Scans the pinned manifest and admits one selected root at a time. + /// + /// # Errors + /// + /// Returns the exact selected-root read or admission failure. + pub fn contains_blob(&self, target: BlobId) -> Result { + super::retained_anchors::first_layout(&self.view, target).map(|layout| layout.is_some()) + } + + /// Reconstructs `target` through its canonically first retained anchor. + /// + /// # Errors + /// + /// Returns [`DurableReadError`] when no root anchors the blob, the view + /// cannot be re-admitted, the layout refuses, or the reconstruction + /// core refuses or its output fails. + pub fn reconstruct( + &self, + target: BlobId, + output: &mut W, + ) -> Result + where + W: Write + ?Sized, + { + let layout_id = self.first_layout_id(target)?; + self.reconstruct_layout(layout_id, output) + } + + /// Reconstructs the exact committed layout `layout_id`, never another. + /// + /// # Errors + /// + /// As [`Self::reconstruct`], with `LayoutMissing` when the catalog names + /// no such layout record. + pub fn reconstruct_layout( + &self, + layout_id: LayoutId, + output: &mut W, + ) -> Result + where + W: Write + ?Sized, + { + let catalog = self.catalog()?; + let layout = self.layout(&catalog, layout_id)?; + let chunks = CatalogChunks::new(&catalog); + reconstruct_admitted(&chunks, layout_id, &layout, output) + .map(|receipt| DurableReconstructionReceipt::new(receipt, self.coordinates)) + .map_err(|source| DurableReadError::Reconstruction(Box::new(source.into()))) + } + + /// Reads exactly `requested` of `target` through its first retained + /// anchor. The range receipt covers only overlapping chunks; snapshot + /// and catalog admission also verify the surrounding stored evidence. + /// + /// # Errors + /// + /// As [`Self::reconstruct`], with the range core's refusals. + pub fn read_range( + &self, + target: BlobId, + requested: ByteRange, + output: &mut W, + ) -> Result + where + W: Write + ?Sized, + { + let layout_id = self.first_layout_id(target)?; + self.read_layout_range(layout_id, requested, output) + } + + /// Reads exactly `requested` through the exact committed layout. + /// + /// # Errors + /// + /// As [`Self::reconstruct_layout`], with the range core's refusals. + pub fn read_layout_range( + &self, + layout_id: LayoutId, + requested: ByteRange, + output: &mut W, + ) -> Result + where + W: Write + ?Sized, + { + let catalog = self.catalog()?; + let layout = self.layout(&catalog, layout_id)?; + let chunks = CatalogChunks::new(&catalog); + read_admitted(&chunks, layout_id, &layout, requested, output) + .map(|receipt| DurableRangeReadReceipt::new(receipt, self.coordinates)) + .map_err(|source| DurableReadError::RangeRead(Box::new(source.into()))) + } + + fn first_layout_id(&self, target: BlobId) -> Result { + super::retained_anchors::first_layout(&self.view, target) + .map_err(|source| DurableReadError::Retention(Box::new(source)))? + .ok_or(DurableReadError::BlobMissing { requested: target }) + } + + pub(super) fn catalog(&self) -> Result, DurableReadError> { + self.view + .catalog() + .snapshot() + .map_err(|source| DurableReadError::View(Box::new(source))) + } + + fn layout( + &self, + catalog: &CatalogSnapshot<'_, '_, '_>, + layout_id: LayoutId, + ) -> Result { + let record = catalog + .record(SegmentRecordIdentity::Layout(layout_id)) + .ok_or(DurableReadError::LayoutMissing { + requested: layout_id, + })?; + let policy = LayoutDecodePolicy::new(self.policy.segment_read().layout_entry_limit()) + .with_expected_id(layout_id); + AdmittedLayout::decode_record(record.payload(), policy) + .map_err(DurableReadError::LayoutDecode) + } +} diff --git a/src/adapters/durable/store.rs b/src/adapters/durable/store.rs new file mode 100644 index 00000000..98950acc --- /dev/null +++ b/src/adapters/durable/store.rs @@ -0,0 +1,205 @@ +//! This boundary module owns the durable store handle: a version-two root +//! that pins snapshots on demand. + +use std::io::Write; +use std::path::{Path, PathBuf}; + +use super::{ + DurableRangeReadReceipt, DurableReadError, DurableReconstructionReceipt, DurableSnapshot, + DurableStoreError, +}; +use crate::adapters::{CatalogRestartPolicy, ReaderAttemptLimit}; +use crate::{BlobId, ByteRange, LayoutId}; + +/// One migrated version-two store to read from. +/// +/// The handle holds no fence and no view; every read pins a fresh +/// [`DurableSnapshot`] unless the caller pins one with [`Self::snapshot`] +/// and reads through it. `open` resolves a relative locator against the current +/// directory once and takes no store authority; admission happens when a +/// snapshot is pinned. +/// Every convenience read pays the complete snapshot admission and allocation +/// cost described by [`DurableSnapshot`]; callers doing repeated reads should +/// retain an explicit snapshot. Snapshot admission performs a blocking +/// root-directory synchronization as part of the platform check and may +/// fail at that admission boundary. These methods do not publish content or +/// flush caller output; a read receipt grants no new content-durability claim. +/// +/// # Example +/// +/// On Linux, read an already migrated version-two store with a retained blob. +/// The caller owns output visibility: a failed write can leave an untrusted +/// prefix. The receipt names the view; it does not extend retention after the +/// snapshot is dropped. This example limits aggregate catalog-selected segment bytes; +/// catalog bytes and decoded metadata allocate separately. +/// +/// ```no_run +/// #[cfg(target_os = "linux")] +/// fn copy_retained_blob( +/// root: &std::path::Path, +/// target: keep::BlobId, +/// output: &mut impl std::io::Write, +/// ) -> Result> { +/// use keep::{ +/// CatalogRestartByteLimit, CatalogRestartPolicy, DurableStore, LayoutEntryLimit, +/// ReaderAttemptLimit, SegmentReadPolicy, SegmentRecordLimit, +/// }; +/// // Limit catalog-selected segment bytes; catalog and metadata allocate separately. +/// let policy = CatalogRestartPolicy::new( +/// SegmentReadPolicy::new(SegmentRecordLimit::MAXIMUM, LayoutEntryLimit::MAXIMUM), +/// CatalogRestartByteLimit::new(16_777_216)?, +/// ); +/// let store = DurableStore::open(root, policy, ReaderAttemptLimit::DEFAULT)?; +/// let snapshot = store.snapshot()?; +/// Ok(snapshot.reconstruct(target, output)?) +/// } +/// ``` +#[must_use] +#[derive(Debug)] +pub struct DurableStore { + root: PathBuf, + policy: CatalogRestartPolicy, + limit: ReaderAttemptLimit, +} + +impl DurableStore { + /// Names the store at `root`, reading under `policy` and collecting a + /// consistent view within `limit` attempts. + /// + /// Allocates an absolute locator and queries the current directory for a + /// relative path. It does not open the store or prove that it exists. + /// Later working-directory changes cannot retarget this handle. The path + /// remains a locator, not a content identity or a pinned directory handle; + /// snapshot admission still checks the named store on every call. + /// + /// # Errors + /// + /// Returns [`DurableStoreError::Locator`] with the original I/O cause when + /// the absolute locator cannot be established, including a deleted current + /// directory for a relative path. + pub fn open( + root: &Path, + policy: CatalogRestartPolicy, + limit: ReaderAttemptLimit, + ) -> Result { + let root = + std::path::absolute(root).map_err(|source| DurableStoreError::Locator { source })?; + Ok(Self { + root, + policy, + limit, + }) + } + + /// The absolute store locator fixed at handle construction. + #[must_use] + pub fn root(&self) -> &Path { + &self.root + } + + /// Pins one consistent, fenced view. + /// + /// # Errors + /// + /// Returns [`DurableStoreError`] at the exact admission, fence, + /// collection, or retained-root refusal. + pub fn snapshot(&self) -> Result { + DurableSnapshot::open(&self.root, self.policy, self.limit) + } + + /// Whether some retained root anchors `target` in a fresh view. + /// + /// # Errors + /// + /// As [`Self::snapshot`]. + pub fn contains_blob(&self, target: BlobId) -> Result { + self.snapshot()?.contains_blob(target) + } + + /// Reconstructs `target` against a fresh view. + /// + /// # Errors + /// + /// Returns snapshot admission through [`DurableOutcome::Store`] or the + /// exact read failure through [`DurableOutcome::Read`], preserving sources. + pub fn reconstruct( + &self, + target: BlobId, + output: &mut W, + ) -> Result + where + W: Write + ?Sized, + { + let snapshot = self.snapshot().map_err(DurableOutcome::Store)?; + snapshot + .reconstruct(target, output) + .map_err(DurableOutcome::Read) + } + + /// Reconstructs the exact committed layout against a fresh view. + /// + /// # Errors + /// + /// As [`Self::reconstruct`]. + pub fn reconstruct_layout( + &self, + layout_id: LayoutId, + output: &mut W, + ) -> Result + where + W: Write + ?Sized, + { + let snapshot = self.snapshot().map_err(DurableOutcome::Store)?; + snapshot + .reconstruct_layout(layout_id, output) + .map_err(DurableOutcome::Read) + } + + /// Reads exactly `requested` of `target` against a fresh view. + /// + /// # Errors + /// + /// As [`Self::reconstruct`]. + pub fn read_range( + &self, + target: BlobId, + requested: ByteRange, + output: &mut W, + ) -> Result + where + W: Write + ?Sized, + { + let snapshot = self.snapshot().map_err(DurableOutcome::Store)?; + snapshot + .read_range(target, requested, output) + .map_err(DurableOutcome::Read) + } +} + +/// Why a store-level read returned no receipt: the view could not be +/// pinned, or the pinned view refused. +#[derive(Debug)] +pub enum DurableOutcome { + /// The snapshot could not be pinned. + Store(DurableStoreError), + /// The pinned view refused the read. + Read(DurableReadError), +} + +impl std::fmt::Display for DurableOutcome { + fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Self::Store(_) => formatter.write_str("durable store admission failed"), + Self::Read(_) => formatter.write_str("durable read failed"), + } + } +} + +impl std::error::Error for DurableOutcome { + fn source(&self) -> Option<&(dyn std::error::Error + 'static)> { + match self { + Self::Store(source) => Some(source), + Self::Read(source) => Some(source), + } + } +} diff --git a/src/adapters/durable/view.rs b/src/adapters/durable/view.rs new file mode 100644 index 00000000..add2ccc4 --- /dev/null +++ b/src/adapters/durable/view.rs @@ -0,0 +1,42 @@ +//! This module owns the exact catalog and retention coordinates of a read. + +use crate::{CatalogDigest, CatalogGeneration, RetentionHead}; + +/// Immutable coordinates admitted under one shared reader fence. +/// +/// An absent retention head means no retention generation has been published. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub struct DurableView { + catalog_generation: CatalogGeneration, + catalog_digest: CatalogDigest, + retention: Option, +} + +impl DurableView { + pub(super) const fn new( + catalog_generation: CatalogGeneration, + catalog_digest: CatalogDigest, + retention: Option, + ) -> Self { + Self { + catalog_generation, + catalog_digest, + retention, + } + } + + /// The admitted catalog generation. + pub const fn catalog_generation(self) -> CatalogGeneration { + self.catalog_generation + } + + /// The exact catalog digest selected by the admitted head. + pub const fn catalog_digest(self) -> CatalogDigest { + self.catalog_digest + } + + /// The complete retention head, including liveness generation and manifest digest. + pub const fn retention(self) -> Option { + self.retention + } +} diff --git a/src/adapters/mod.rs b/src/adapters/mod.rs index db478fee..64d9c0c3 100644 --- a/src/adapters/mod.rs +++ b/src/adapters/mod.rs @@ -7,6 +7,13 @@ //! policy. mod admitted_catalog; +mod authenticated_read; +pub use authenticated_read::{RangeReadError, ReconstructionError}; +mod durable; +pub use durable::{ + DurableOutcome, DurableRangeReadReceipt, DurableReadError, DurableReconstructionReceipt, + DurableSnapshot, DurableStore, DurableStoreError, DurableView, +}; mod admitted_recovery_stage_bytes; mod admitted_segment; mod admitted_segment_record; diff --git a/src/adapters/retention.rs b/src/adapters/retention.rs index 9a7d92dc..180ca1c3 100644 --- a/src/adapters/retention.rs +++ b/src/adapters/retention.rs @@ -14,6 +14,10 @@ mod closure_member; mod closure_profile_error; mod closure_verifier; #[cfg(test)] +mod durable_read_law_tests; +#[cfg(test)] +mod durable_view_law_tests; +#[cfg(test)] mod filesystem_recovery_admission_tests; #[cfg(test)] mod filesystem_retention_anchor_order_prefix_tests; @@ -164,6 +168,7 @@ mod root_field_decoder; mod root_header_decoder; mod root_integrity; mod root_semantic_header; +mod selected_root_refusal; mod successor_manifest; mod transition_disposition; mod transition_error; @@ -177,6 +182,8 @@ mod verified_closure; mod filesystem_retention_forward_error_tests; mod reader_attempt_limit; mod reader_fence; +#[cfg(all(test, target_os = "linux"))] +mod reader_platform_law_tests; mod recovery_evidence; mod recovery_execution; #[cfg(test)] @@ -267,6 +274,7 @@ pub use retention_view_collector::{ }; pub use root_decode_error::RetentionRootDecodeError; pub use root_encode_error::RetentionRootEncodeError; +pub use selected_root_refusal::RetentionSelectedRootRefusal; pub use transition_disposition::RetentionTransitionDisposition; pub use transition_error::RetentionTransitionError; pub use transition_planner::plan_retention_transition; diff --git a/src/adapters/retention/durable_read_law_tests.rs b/src/adapters/retention/durable_read_law_tests.rs new file mode 100644 index 00000000..fb33db05 --- /dev/null +++ b/src/adapters/retention/durable_read_law_tests.rs @@ -0,0 +1,142 @@ +//! This module owns durable public read laws against the independent one-zero corpus. +//! +//! Size: medium (owned filesystem scratch, no network or sleeps). +//! Oracle: the frozen one-zero bundle and root identify the single byte [0]. +//! Delete when durable reads are removed or stronger public laws subsume these outcomes. + +use std::error::Error; + +use super::filesystem_retention_test_fixture::{ + HEAD_HEX, ROOT_HEX, fixture, initial_preparation, open_authority, +}; +use crate::{ + AdmittedRetentionRoot, ByteLength, ByteOffset, ByteRange, CatalogRestartByteLimit, + CatalogRestartPolicy, DurableReadError, DurableStore, LayoutEntryLimit, ReaderAttemptLimit, + SegmentReadPolicy, SegmentRecordLimit, execute_retention_publication, +}; + +fn store(root: &std::path::Path) -> Result> { + Ok(DurableStore::open( + root, + CatalogRestartPolicy::new( + SegmentReadPolicy::new(SegmentRecordLimit::MAXIMUM, LayoutEntryLimit::MAXIMUM), + CatalogRestartByteLimit::new(1_048_576)?, + ), + ReaderAttemptLimit::DEFAULT, + )?) +} + +#[test] +fn durable_reconstruction_returns_golden_bytes_with_exact_view_coordinates() +-> Result<(), Box> { + let (sandbox, mut authority) = open_authority("durable-golden-reconstruction")?; + let bytes = fixture(ROOT_HEX)?; + let preparation = initial_preparation(&bytes)?; + let _published = execute_retention_publication(&mut authority, &preparation)?; + drop(authority); + let root = AdmittedRetentionRoot::decode(&bytes)?; + let anchor = root + .root() + .anchors() + .first() + .ok_or("golden anchor absent")?; + let snapshot = store(sandbox.path())?.snapshot()?; + let mut output = Vec::new(); + let receipt = snapshot.reconstruct(anchor.blob_id(), &mut output)?; + assert_eq!( + output, + [0], + "reconstruction must emit the exact golden byte" + ); + assert_eq!(receipt.receipt().target(), anchor.blob_id()); + assert_eq!(receipt.receipt().layout_id(), anchor.layout_id()); + assert_eq!(receipt.receipt().bytes_written().get(), 1); + assert_eq!(receipt.view().catalog_generation().get(), 1); + assert_eq!( + receipt.view().catalog_digest().as_bytes().as_slice(), + fixture("0b7cad1b6de663d34beacbc214db7497f2e36ab6b08dfbd5febbc8d06a418811\n")?, + "catalog coordinate must match the independently frozen bundle" + ); + let golden_head = fixture(HEAD_HEX)?; + assert_eq!( + receipt + .view() + .retention() + .ok_or("retention coordinates absent")? + .manifest_digest() + .as_bytes() + .as_slice(), + golden_head + .get(40..72) + .ok_or("golden manifest digest absent")?, + "manifest coordinate must match the normative head field in the golden record" + ); + assert_eq!( + receipt + .view() + .retention() + .ok_or("retention coordinates absent")? + .generation(), + preparation.liveness_generation() + ); + assert_eq!(receipt.view(), snapshot.view()); + Ok(()) +} + +#[test] +fn durable_ranges_emit_only_the_requested_golden_interval() -> Result<(), Box> { + let (sandbox, mut authority) = open_authority("durable-golden-ranges")?; + let bytes = fixture(ROOT_HEX)?; + let preparation = initial_preparation(&bytes)?; + let _published = execute_retention_publication(&mut authority, &preparation)?; + drop(authority); + let root = AdmittedRetentionRoot::decode(&bytes)?; + let anchor = root + .root() + .anchors() + .first() + .ok_or("golden anchor absent")?; + let snapshot = store(sandbox.path())?.snapshot()?; + for (offset, length, expected) in [(0, 0, &[][..]), (0, 1, &[0][..]), (1, 0, &[][..])] { + let requested = ByteRange::new(ByteOffset::new(offset), ByteLength::new(length))?; + let mut output = Vec::new(); + let receipt = snapshot.read_range(anchor.blob_id(), requested, &mut output)?; + assert_eq!( + output, expected, + "range {requested:?} must emit its exact interval" + ); + assert_eq!(receipt.receipt().requested(), requested); + assert_eq!(receipt.receipt().bytes_written().get(), length); + assert_eq!(receipt.view(), snapshot.view()); + } + Ok(()) +} + +#[test] +fn unretained_blob_refuses_without_output_but_exact_catalog_layout_remains_readable() +-> Result<(), Box> { + let (sandbox, authority) = open_authority("durable-unretained-blob")?; + drop(authority); + let bytes = fixture(ROOT_HEX)?; + let root = AdmittedRetentionRoot::decode(&bytes)?; + let anchor = root + .root() + .anchors() + .first() + .ok_or("golden anchor absent")?; + let snapshot = store(sandbox.path())?.snapshot()?; + assert!(!snapshot.contains_blob(anchor.blob_id())?); + let mut output = Vec::new(); + let failure = snapshot + .reconstruct(anchor.blob_id(), &mut output) + .err() + .ok_or("unretained blob was read")?; + assert!( + matches!(failure, DurableReadError::BlobMissing { requested } if requested == anchor.blob_id()) + ); + assert!(output.is_empty(), "refusal must not emit content"); + let receipt = snapshot.reconstruct_layout(anchor.layout_id(), &mut output)?; + assert_eq!(output, [0]); + assert_eq!(receipt.view().retention(), None); + Ok(()) +} diff --git a/src/adapters/retention/durable_view_law_tests.rs b/src/adapters/retention/durable_view_law_tests.rs new file mode 100644 index 00000000..6af789a6 --- /dev/null +++ b/src/adapters/retention/durable_view_law_tests.rs @@ -0,0 +1,130 @@ +//! This module owns pinned-read lifetime and exact durable admission failure laws. +//! +//! Size: medium; owned filesystem scratch, deterministic publication order and kernel try-lock. +//! Oracle: published generations, the independent one-zero corpus, and typed I/O boundaries. +//! Delete when durable snapshots are removed or stronger public laws subsume these outcomes. + +use std::error::Error; +use std::fs; + +use rustix::fs::{FlockOperation, flock}; +use rustix::io::Errno; + +use super::filesystem_retention_test_fixture::{ + ROOT_HEX, SEGMENT_NAME, fixture, initial_preparation, open_authority, successor_preparation, +}; +use crate::{ + AdmittedRetentionManifest, AdmittedRetentionRoot, CanonicalRetentionRoot, + CatalogRestartByteLimit, CatalogRestartError, CatalogRestartPhase, CatalogRestartPolicy, + DurableSnapshot, DurableStoreError, FilesystemRetentionPublicationAuthority, + FilesystemRetentionSnapshotError, FilesystemVersionTwoAdmission, LayoutEntryLimit, + ReaderAttemptLimit, RetentionPolicy, RetentionRoot, SegmentReadPolicy, SegmentRecordLimit, + execute_retention_publication, +}; + +fn snapshot(path: &std::path::Path) -> Result> { + Ok(DurableSnapshot::open( + path, + CatalogRestartPolicy::new( + SegmentReadPolicy::new(SegmentRecordLimit::MAXIMUM, LayoutEntryLimit::MAXIMUM), + CatalogRestartByteLimit::new(1_048_576)?, + ), + ReaderAttemptLimit::DEFAULT, + )?) +} + +#[test] +fn a_pinned_read_keeps_its_retained_view_after_release_publication() -> Result<(), Box> { + let (sandbox, mut authority) = open_authority("durable-pinned-release")?; + let bytes = fixture(ROOT_HEX)?; + let preparation = initial_preparation(&bytes)?; + let _published = execute_retention_publication(&mut authority, &preparation)?; + drop(authority); + let current = AdmittedRetentionRoot::decode(&bytes)?; + let anchor = current + .root() + .anchors() + .first() + .ok_or("golden anchor absent")?; + let pinned = snapshot(sandbox.path())?; + let admission = + FilesystemVersionTwoAdmission::reopen_unchecked_for_repository_tasks(sandbox.path())?; + let mut authority = FilesystemRetentionPublicationAuthority::open(admission)?; + let state = authority.observe_current()?.ok_or("retention absent")?; + let manifest = AdmittedRetentionManifest::decode(state.manifest_bytes())?; + let successor = RetentionRoot::new( + current.root().namespace().clone(), + current.root().generation().successor()?, + RetentionPolicy::new(current.root().profile(), current.root().limits()), + Some(current.digest()), + Vec::new(), + )?; + let encoded = CanonicalRetentionRoot::from_root(&successor)?; + let release = successor_preparation(¤t, &manifest, encoded.encoded())?; + let _released = execute_retention_publication(&mut authority, &release)?; + drop(authority); + let mut output = Vec::new(); + let receipt = pinned.reconstruct(anchor.blob_id(), &mut output)?; + assert_eq!( + output, + [0], + "old view must still emit its exact retained bytes" + ); + assert_eq!( + receipt + .view() + .retention() + .ok_or("old head absent")? + .generation() + .get(), + 1 + ); + let fresh = snapshot(sandbox.path())?; + assert!( + !fresh.contains_blob(anchor.blob_id())?, + "fresh view must observe release" + ); + assert_eq!( + fresh + .view() + .retention() + .ok_or("new head absent")? + .generation() + .get(), + 2 + ); + Ok(()) +} + +#[test] +fn durable_snapshot_holds_the_collector_fence_until_drop() -> Result<(), Box> { + let (sandbox, authority) = open_authority("durable-collector-fence")?; + drop(authority); + let pinned = snapshot(sandbox.path())?; + let collector = fs::File::open(sandbox.path().join("reader.lock"))?; + assert_eq!( + flock(&collector, FlockOperation::NonBlockingLockExclusive), + Err(Errno::WOULDBLOCK) + ); + drop(pinned); + flock(&collector, FlockOperation::NonBlockingLockExclusive)?; + Ok(()) +} + +#[test] +fn an_unreadable_segment_preserves_the_operational_open_failure() -> Result<(), Box> { + let (sandbox, authority) = open_authority("durable-missing-segment")?; + drop(authority); + fs::remove_file(sandbox.path().join("segments").join(SEGMENT_NAME))?; + let failure = snapshot(sandbox.path()) + .err() + .ok_or("missing segment was admitted")?; + assert!( + matches!(failure.downcast_ref::(), Some(DurableStoreError::Snapshot(source)) + if matches!(source.as_ref(), FilesystemRetentionSnapshotError::Catalog { source } + if matches!(source, CatalogRestartError::Io { phase: CatalogRestartPhase::OpenSegment, source } + if source.kind() == std::io::ErrorKind::NotFound))), + "missing physical evidence must preserve OpenSegment/NotFound: {failure:?}" + ); + Ok(()) +} diff --git a/src/adapters/retention/filesystem_retention_snapshot.rs b/src/adapters/retention/filesystem_retention_snapshot.rs index 751aebce..a47b6cab 100644 --- a/src/adapters/retention/filesystem_retention_snapshot.rs +++ b/src/adapters/retention/filesystem_retention_snapshot.rs @@ -10,16 +10,16 @@ use super::filesystem_retention_current::{self, ObservedRetentionState}; use super::filesystem_retention_pool_name as pool_name; use super::{ AdmittedRetentionRoot, FilesystemRetentionSnapshotError as Error, ReaderAttemptLimit, - ReaderFence, RetentionViewCoordinates, RetentionViewSource, collect_retention_view, - root_header_decoder, + ReaderFence, RetentionSelectedRootRefusal, RetentionViewCoordinates, RetentionViewSource, + collect_retention_view, root_header_decoder, }; use crate::adapters::filesystem_exact_record::{self as exact_record, ExactRecordError}; use crate::adapters::filesystem_platform_profile::root_identity; use crate::adapters::filesystem_version_two_admission::require_root_identity; use crate::adapters::{ CatalogRestartError, CatalogRestartPolicy, ChecksummedPublicationHead, - FilesystemCatalogSnapshot, filesystem_initialization_namespace, filesystem_version_two_records, - publication_head_decoder, + FilesystemCatalogSnapshot, filesystem_initialization_namespace, filesystem_platform_profile, + filesystem_version_two_records, publication_head_decoder, }; use crate::{RetentionHead, RetentionManifest, RetentionNamespaceDigest}; @@ -105,13 +105,19 @@ impl RetentionViewSource for Source { } impl FilesystemRetentionSnapshot { - /// Admits the root as version two, acquires the reader fence, and + /// Admits the production filesystem profile and version-two root, acquires the reader fence, and /// double-collects one consistent view within `limit` attempts. /// Admission requires the opened directory's restart-stable device and /// inode to match the jointly admitted migration records before fencing. /// - /// The call takes no writer authority and mutates nothing. It may block + /// The call takes no writer authority and performs no namespace writes. + /// Platform admission synchronizes the opened root directory; failure is + /// returned as `Admission` with the original I/O cause. It may also block /// while collection holds the fence exclusively. + /// Platform admission requires the existing local writable, case-sensitive + /// Linux ext4 profile across every present version-two protocol directory. + /// The same opened root capability is retained through namespace, migration + /// identity, fence, and coordinate admission; the ambient path is not reopened. /// /// # Errors /// @@ -124,7 +130,7 @@ impl FilesystemRetentionSnapshot { policy: CatalogRestartPolicy, limit: ReaderAttemptLimit, ) -> Result { - let root = Dir::open_ambient_dir(store_root, cap_std::ambient_authority()) + let root = filesystem_platform_profile::open_version_two(store_root) .map_err(|source| Error::Admission { source })?; filesystem_initialization_namespace::admit_version_two(&root) .map_err(|source| Error::Admission { source })?; @@ -185,12 +191,14 @@ impl FilesystemRetentionSnapshot { /// Returns `None` when the manifest names no root for the namespace. The /// pool entry is read without following links, bounded by the root /// format's maximum length, decoded, and required to carry exactly the - /// generation and digest the manifest names. + /// namespace, generation, and digest the manifest names. /// /// # Errors /// /// Returns [`FilesystemRetentionSnapshotError::Root`](super::FilesystemRetentionSnapshotError::Root) /// when the entry is absent, unreadable, or not the selected root. + /// A namespace contradiction preserves [`RetentionSelectedRootRefusal`] + /// inside that error's I/O source, including expected and observed digests. pub fn retained_root( &self, namespace: RetentionNamespaceDigest, @@ -246,6 +254,18 @@ impl FilesystemRetentionSnapshot { source: invalid("selected root does not decode to the manifest's selection"), }); } + let observed = root.root().namespace().digest(); + if observed != namespace { + return Err(Error::Root { + source: io::Error::new( + io::ErrorKind::InvalidData, + RetentionSelectedRootRefusal::Namespace { + expected: namespace, + observed, + }, + ), + }); + } Ok(Some(bytes.into_boxed_slice())) } } diff --git a/src/adapters/retention/filesystem_retention_snapshot_error.rs b/src/adapters/retention/filesystem_retention_snapshot_error.rs index daa6a634..507af461 100644 --- a/src/adapters/retention/filesystem_retention_snapshot_error.rs +++ b/src/adapters/retention/filesystem_retention_snapshot_error.rs @@ -11,9 +11,9 @@ use crate::adapters::CatalogRestartError; #[derive(Debug)] #[non_exhaustive] pub enum FilesystemRetentionSnapshotError { - /// The root is not an exactly admitted version-two store. + /// The platform profile or exact version-two root admission failed. Admission { - /// The exact namespace or record refusal. + /// The exact platform I/O, namespace, or record failure. source: io::Error, }, /// The reader fence could not be acquired. diff --git a/src/adapters/retention/reader_platform_law_tests.rs b/src/adapters/retention/reader_platform_law_tests.rs new file mode 100644 index 00000000..3f31c1c7 --- /dev/null +++ b/src/adapters/retention/reader_platform_law_tests.rs @@ -0,0 +1,136 @@ +//! Reader platform admission at the public snapshot boundaries. +//! +//! Size: medium. Oracle: production readers require the existing writable, +//! case-sensitive local ext4 profile, independently of valid migration bytes. +//! The negative fixture deliberately uses test-only writer admission on tmpfs. +//! Delete when the supported reader profile changes or stronger public laws subsume it. + +use std::error::Error; +use std::fs; +use std::io; +use std::path::PathBuf; + +use super::filesystem_retention_test_fixture::{ + CATALOG_NAME, SEGMENT_NAME, fixture, migrated_store, +}; +use crate::{ + CatalogRestartByteLimit, CatalogRestartPolicy, DurableSnapshot, DurableStoreError, + FilesystemPlatformAdmission, FilesystemRetentionSnapshot, FilesystemRetentionSnapshotError, + FilesystemStoreMigrationAuthority, FilesystemWriterLock, ReaderAttemptLimit, SegmentReadPolicy, + execute_store_migration, +}; + +#[test] +fn a_canonical_tmpfs_store_refuses_the_public_reader_profile() -> Result<(), Box> { + let store = TmpfsStore::create("direct")?; + let refusal = + FilesystemRetentionSnapshot::load(&store.0, policy()?, ReaderAttemptLimit::DEFAULT) + .err() + .ok_or("valid migration bytes admitted an unsupported reader filesystem")?; + assert!( + matches!(&refusal, FilesystemRetentionSnapshotError::Admission { source } + if source.kind() == io::ErrorKind::Unsupported), + "unsupported filesystem must refuse at admission: {refusal:?}" + ); + Ok(()) +} + +#[test] +fn a_canonical_tmpfs_store_refuses_durable_snapshot_admission() -> Result<(), Box> { + let store = TmpfsStore::create("durable")?; + let refusal = DurableSnapshot::open(&store.0, policy()?, ReaderAttemptLimit::DEFAULT) + .err() + .ok_or("durable snapshot accepted an unsupported filesystem")?; + assert!( + matches!(&refusal, DurableStoreError::Snapshot(source) + if matches!(source.as_ref(), FilesystemRetentionSnapshotError::Admission { source } + if source.kind() == io::ErrorKind::Unsupported)), + "durable admission must preserve the unsupported platform boundary: {refusal:?}" + ); + Ok(()) +} + +#[test] +fn reader_platform_admission_does_not_acquire_writer_authority() -> Result<(), Box> { + let store = migrated_store("reader-platform-existing-writer")?; + let writer = FilesystemWriterLock::try_acquire(store.path())?; + + let snapshot = + FilesystemRetentionSnapshot::load(store.path(), policy()?, ReaderAttemptLimit::DEFAULT)?; + + assert_eq!( + snapshot.catalog().generation().get(), + 1, + "reader must admit its catalog while another authority holds the writer lock" + ); + drop(writer); + Ok(()) +} + +struct TmpfsStore(PathBuf); + +impl TmpfsStore { + // Atomically claim only a name we created; stale or concurrent fixtures + // remain untouched. The finite retry budget bounds test setup work. + fn reserve(name: &str) -> io::Result { + for attempt in 0_u16..1_024 { + let path = PathBuf::from("/dev/shm").join(format!( + "keep-reader-profile-{name}-{}-{attempt}", + std::process::id() + )); + match fs::create_dir(&path) { + Ok(()) => return Ok(Self(path)), + Err(source) if source.kind() == io::ErrorKind::AlreadyExists => {} + Err(source) => return Err(source), + } + } + Err(io::Error::new( + io::ErrorKind::AlreadyExists, + "tmpfs fixture names exhausted", + )) + } + + fn create(name: &str) -> Result> { + let filesystem = rustix::fs::fstatfs(&fs::File::open("/dev/shm")?)?; + assert_eq!( + filesystem.f_type, 0x0102_1994, + "negative profile requires Linux tmpfs" + ); + let store = Self::reserve(name)?; + let admission = FilesystemPlatformAdmission::initialize_unchecked_for_tests(&store.0)?; + for (name, encoded) in [ + ( + format!("segments/{SEGMENT_NAME}"), + include_str!("../../../conformance/segment-store/v1/one-zero-bundle-segment.hex"), + ), + ( + format!("catalogs/{CATALOG_NAME}"), + include_str!("../../../conformance/segment-store/v1/one-zero-bundle-catalog.hex"), + ), + ( + "HEAD".into(), + include_str!("../../../conformance/segment-store/v1/one-zero-bundle-head.hex"), + ), + ] { + fs::write(store.0.join(name), fixture(encoded)?)?; + } + let mut authority = + FilesystemStoreMigrationAuthority::open(admission, SegmentReadPolicy::MAXIMUM)?; + let intent = authority.observe_intent()?; + let _receipt = execute_store_migration(&mut authority, &intent)?; + Ok(store) + } +} + +impl Drop for TmpfsStore { + fn drop(&mut self) { + let _ = fs::remove_dir_all(&self.0); + } +} + +fn policy() -> Result> { + Ok(CatalogRestartPolicy::new( + SegmentReadPolicy::MAXIMUM, + CatalogRestartByteLimit::new(1_048_576)?, + )) +} diff --git a/src/adapters/retention/selected_root_refusal.rs b/src/adapters/retention/selected_root_refusal.rs new file mode 100644 index 00000000..cac54997 --- /dev/null +++ b/src/adapters/retention/selected_root_refusal.rs @@ -0,0 +1,34 @@ +//! This module owns contradictions between a selected root and its namespace. + +use std::error::Error; +use std::fmt; + +use crate::RetentionNamespaceDigest; + +/// A canonically decoded root contradicts the manifest selection that named it. +/// +/// The public reader preserves this cause inside its root-boundary I/O error. +/// This refusal proves a contradiction; it is not an operational read failure. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +#[non_exhaustive] +pub enum RetentionSelectedRootRefusal { + /// The root belongs to a different namespace than the manifest entry. + Namespace { + /// Namespace selected by the manifest and requested by the reader. + expected: RetentionNamespaceDigest, + /// Namespace authenticated by the decoded canonical root. + observed: RetentionNamespaceDigest, + }, +} + +impl fmt::Display for RetentionSelectedRootRefusal { + fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + Self::Namespace { .. } => { + formatter.write_str("selected root belongs to a different retention namespace") + } + } + } +} + +impl Error for RetentionSelectedRootRefusal {} diff --git a/src/authenticated_read/chunk_verification.rs b/src/authenticated_read/chunk_verification.rs new file mode 100644 index 00000000..d459c686 --- /dev/null +++ b/src/authenticated_read/chunk_verification.rs @@ -0,0 +1,90 @@ +//! This module owns exact chunk authentication through an immutable semantic source. + +use crate::{ChunkHashError, ChunkId, LayoutEntry, LayoutId}; + +/// One admitted view's exact chunk lookup: the reference store's in-memory +/// map, or a durable snapshot's fenced catalog. Every read core hashes what +/// the source returns before trusting it. +/// Implementations must expose immutable bytes for the entire read operation: +/// emission deliberately reuses the verification pass without rehashing. +pub(crate) trait ChunkSource { + /// The exact bytes stored under `identity`, if the view holds them. + fn chunk(&self, identity: ChunkId) -> Option<&[u8]>; + + /// Records that `identity` was hashed, for laws over the hashing pass. + fn note_chunk_hash(&self, identity: ChunkId) { + let _ = identity; + } +} + +pub(super) fn verified_chunk( + store: &S, + layout_id: LayoutId, + index: usize, + entry: LayoutEntry, +) -> Result<&[u8], ChunkVerificationError> { + let expected = entry.chunk_id(); + let bytes = ChunkSource::chunk(store, expected).ok_or(ChunkVerificationError::Missing { + layout: layout_id, + index, + requested: expected, + })?; + store.note_chunk_hash(expected); + let observed = ChunkId::hash_bytes(bytes).map_err(|source| ChunkVerificationError::Hash { + layout: layout_id, + index, + expected, + source, + })?; + if observed != expected { + return Err(ChunkVerificationError::IdentityMismatch { + layout: layout_id, + index, + expected, + observed, + }); + } + Ok(bytes) +} + +#[derive(Clone, Copy, Debug)] +pub(crate) enum ChunkVerificationError { + Missing { + layout: LayoutId, + index: usize, + requested: ChunkId, + }, + Hash { + layout: LayoutId, + index: usize, + expected: ChunkId, + source: ChunkHashError, + }, + IdentityMismatch { + layout: LayoutId, + index: usize, + expected: ChunkId, + observed: ChunkId, + }, +} + +/// Fetches an already-authenticated immutable chunk for emission. +/// +/// The verification pass hashed every selected chunk before the first byte +/// was written, and the in-memory view cannot change under `&self`, so the +/// emission pass looks the chunk up by identity and hashes nothing. A chunk +/// that vanished between the passes is impossible here; the arm exists so a +/// durable adapter that reuses this shape cannot forget it. +pub(super) fn emitted_chunk( + store: &S, + layout_id: LayoutId, + index: usize, + entry: LayoutEntry, +) -> Result<&[u8], ChunkVerificationError> { + let expected = entry.chunk_id(); + ChunkSource::chunk(store, expected).ok_or(ChunkVerificationError::Missing { + layout: layout_id, + index, + requested: expected, + }) +} diff --git a/src/authenticated_read/mod.rs b/src/authenticated_read/mod.rs new file mode 100644 index 00000000..b6a81dd4 --- /dev/null +++ b/src/authenticated_read/mod.rs @@ -0,0 +1,33 @@ +//! This module owns authentication and exact emission of admitted logical reads. +//! +//! Adapters supply immutable chunk bytes and admitted layouts. This core owns +//! verification policy, output accounting and proof receipts; it never decodes +//! records, opens storage, resolves publication, or imports adapter errors. + +#![expect( + clippy::redundant_pub_crate, + reason = "semantic ports and failures remain crate-private if the module is later exposed" +)] +#![expect( + clippy::result_large_err, + reason = "retain exact identity coordinates and original causes without heap allocation" +)] + +mod chunk_verification; +mod output_write; +mod profile_verification; +mod range_read_execution; +mod range_read_failure; +mod range_read_receipt; +mod reconstruction; +mod reconstruction_failure; +mod reconstruction_receipt; + +pub(crate) use chunk_verification::{ChunkSource, ChunkVerificationError}; +pub(crate) use output_write::OutputWriteError; +pub(crate) use range_read_execution::read_admitted; +pub(crate) use range_read_failure::RangeReadFailure; +pub use range_read_receipt::RangeReadReceipt; +pub(crate) use reconstruction::reconstruct_admitted; +pub(crate) use reconstruction_failure::ReconstructionFailure; +pub use reconstruction_receipt::ReconstructionReceipt; diff --git a/src/reference/output_write.rs b/src/authenticated_read/output_write.rs similarity index 98% rename from src/reference/output_write.rs rename to src/authenticated_read/output_write.rs index 328b8cf6..f1d97a07 100644 --- a/src/reference/output_write.rs +++ b/src/authenticated_read/output_write.rs @@ -56,7 +56,7 @@ fn checked_written(written: u64, incoming: usize) -> Result { + layout: LayoutId, + verifier: StorageProfileVerifier<'a>, +} + +impl<'a> ProfileVerifier<'a> { + pub(super) fn new( + layout: LayoutId, + admitted: &'a AdmittedLayout, + ) -> Result { + let verifier = StorageProfileVerifier::new(admitted).map_err(|error| { + ReconstructionFailure::Profile { + layout, + source: error, + } + })?; + Ok(Self { layout, verifier }) + } + + pub(super) fn feed(&mut self, bytes: &[u8]) -> Result<(), ReconstructionFailure> { + self.verifier + .feed(bytes) + .map_err(|error| ReconstructionFailure::Profile { + layout: self.layout, + source: error, + }) + } + + pub(super) fn finish(self) -> Result<(), ReconstructionFailure> { + self.verifier + .finish() + .map_err(|error| ReconstructionFailure::Profile { + layout: self.layout, + source: error, + }) + } +} diff --git a/src/reference/range_read_execution.rs b/src/authenticated_read/range_read_execution.rs similarity index 64% rename from src/reference/range_read_execution.rs rename to src/authenticated_read/range_read_execution.rs index feceb2b2..9398806a 100644 --- a/src/reference/range_read_execution.rs +++ b/src/authenticated_read/range_read_execution.rs @@ -2,33 +2,33 @@ use std::io::Write; -use crate::{ - AdmittedLayout, ByteLength, ByteRange, ChunkId, LayoutEntry, LayoutId, RangePlan, - ReferenceStore, -}; +use crate::{AdmittedLayout, ByteLength, ByteRange, ChunkId, LayoutEntry, LayoutId, RangePlan}; -use super::chunk_verification::{emitted_chunk, verified_chunk}; +use super::chunk_verification::{ChunkSource, emitted_chunk, verified_chunk}; use super::output_write::write_all; -use super::range_read_error_mapping::{range_chunk_error, range_output_error}; -use super::{RangeReadError, RangeReadReceipt}; +use super::{RangeReadFailure, RangeReadReceipt}; -pub(super) fn read_admitted( - store: &ReferenceStore, +/// Authenticates only the chunks overlapping `requested` against `store`, +/// then emits exactly the requested bytes: the one range core every view +/// shares. +pub(crate) fn read_admitted( + store: &S, layout_id: LayoutId, layout: &AdmittedLayout, requested: ByteRange, output: &mut W, -) -> Result +) -> Result where + S: ChunkSource + ?Sized, W: Write + ?Sized, { let plan = layout .plan_range(requested) - .map_err(RangeReadError::RangePlan)?; + .map_err(RangeReadFailure::RangePlan)?; verify_selected(store, layout_id, layout, plan)?; let written = emit_selected(store, layout_id, layout, plan, output)?; if written != requested.length() { - return Err(RangeReadError::WrittenLengthMismatch { + return Err(RangeReadFailure::WrittenLengthMismatch { layout: layout_id, expected: requested.length(), observed: written, @@ -42,36 +42,41 @@ where )) } -fn verify_selected( - store: &ReferenceStore, +fn verify_selected( + store: &S, layout_id: LayoutId, layout: &AdmittedLayout, plan: RangePlan, -) -> Result<(), RangeReadError> { +) -> Result<(), RangeReadFailure> { let (first, entries) = selected_entries(layout, plan)?; for (index, entry) in (first..plan.end_entry()).zip(entries.iter().copied()) { - let _bytes = verified_chunk(store, layout_id, index, entry).map_err(range_chunk_error)?; + let _bytes = + verified_chunk(store, layout_id, index, entry).map_err(RangeReadFailure::Chunk)?; } Ok(()) } -fn emit_selected( - store: &ReferenceStore, +fn emit_selected( + store: &S, layout_id: LayoutId, layout: &AdmittedLayout, plan: RangePlan, output: &mut W, -) -> Result +) -> Result where + S: ChunkSource + ?Sized, W: Write + ?Sized, { let (first, entries) = selected_entries(layout, plan)?; let mut written = 0_u64; for (index, entry) in (first..plan.end_entry()).zip(entries.iter().copied()) { - let bytes = emitted_chunk(store, layout_id, index, entry).map_err(range_chunk_error)?; + let bytes = + emitted_chunk(store, layout_id, index, entry).map_err(RangeReadFailure::Chunk)?; let selected = selected_chunk_slice(layout_id, index, entry, plan.requested(), bytes)?; - write_all(output, selected, &mut written) - .map_err(|error| range_output_error(layout_id, error))?; + write_all(output, selected, &mut written).map_err(|error| RangeReadFailure::Output { + layout: layout_id, + source: error, + })?; } Ok(ByteLength::new(written)) } @@ -79,18 +84,16 @@ where fn selected_entries( layout: &AdmittedLayout, plan: RangePlan, -) -> Result<(usize, &[LayoutEntry]), RangeReadError> { +) -> Result<(usize, &[LayoutEntry]), RangeReadFailure> { let first = plan.first_entry().map_or(0, std::convert::identity); let end = plan.end_entry(); - let entries = - layout - .entries() - .get(first..end) - .ok_or_else(|| RangeReadError::PlanEntriesUnavailable { - first, - end, - available: layout.entries().len(), - })?; + let entries = layout.entries().get(first..end).ok_or_else(|| { + RangeReadFailure::PlanEntriesUnavailable { + first, + end, + available: layout.entries().len(), + } + })?; Ok((first, entries)) } @@ -100,7 +103,7 @@ fn selected_chunk_slice( entry: LayoutEntry, requested: ByteRange, bytes: &[u8], -) -> Result<&[u8], RangeReadError> { +) -> Result<&[u8], RangeReadFailure> { let chunk = entry.chunk_id(); let entry_start = entry.offset().get(); let entry_end = entry_start @@ -128,15 +131,11 @@ const fn slice_unavailable( index: usize, requested: ByteRange, chunk: ChunkId, -) -> RangeReadError { - RangeReadError::ChunkSliceUnavailable { +) -> RangeReadFailure { + RangeReadFailure::ChunkSliceUnavailable { layout, index, requested, chunk, } } - -#[cfg(test)] -#[path = "range_read_tests.rs"] -mod tests; diff --git a/src/authenticated_read/range_read_failure.rs b/src/authenticated_read/range_read_failure.rs new file mode 100644 index 00000000..f4377249 --- /dev/null +++ b/src/authenticated_read/range_read_failure.rs @@ -0,0 +1,30 @@ +//! This module owns semantic exact-range authentication and emission failures. + +use super::{ChunkVerificationError, OutputWriteError}; +use crate::{ByteLength, ByteRange, ChunkId, LayoutId, RangePlanError}; + +#[derive(Debug)] +pub(crate) enum RangeReadFailure { + RangePlan(RangePlanError), + Chunk(ChunkVerificationError), + PlanEntriesUnavailable { + first: usize, + end: usize, + available: usize, + }, + ChunkSliceUnavailable { + layout: LayoutId, + index: usize, + requested: ByteRange, + chunk: ChunkId, + }, + Output { + layout: LayoutId, + source: OutputWriteError, + }, + WrittenLengthMismatch { + layout: LayoutId, + expected: ByteLength, + observed: ByteLength, + }, +} diff --git a/src/reference/range_read_receipt.rs b/src/authenticated_read/range_read_receipt.rs similarity index 100% rename from src/reference/range_read_receipt.rs rename to src/authenticated_read/range_read_receipt.rs diff --git a/src/authenticated_read/rationale.md b/src/authenticated_read/rationale.md new file mode 100644 index 00000000..cc528c6c --- /dev/null +++ b/src/authenticated_read/rationale.md @@ -0,0 +1,13 @@ +# Shared authenticated read policy + +The domain read core owns chunk authentication, complete-blob and storage-profile verification, selected-range verification, exact synchronous emission, and success receipts. It accepts an admitted layout and a crate-private immutable chunk source implemented independently by the reference map and durable catalog. + +Lookup policy, canonical codec ingress, retained-root selection, filesystem admission and fences belong to the adapters. The core imports neither adapter implementations nor their codec-bearing public errors. A facade that re-exported reference-adapter code would retain the wrong ownership and was rejected. + +Core failures carry semantic identities, coordinates and original causes. The shared outward read-error boundary maps them to the existing public variants without stringification, additional wrapping of I/O sources, or changed accepted-prefix accounting. Codec and missing-layout/blob variants remain at that outward boundary because the core receives an already admitted layout. + +Nested internal failure enums retain those coordinates on the stack rather than allocating a box to satisfy a size lint. The scoped lint expectation records this deliberate choice; it introduces no heap allocation or new public error variant. + +Chunk bytes must remain immutable for the entire verification and emission operation. The reference map and pinned catalog supply that guarantee through borrowed immutable storage; an arbitrary mutable callback source is not exposed publicly. A successful whole-blob receipt proves identity and profile verification, while a range receipt proves only the requested bytes from authenticated overlapping chunks. + +The reference adapter retains its public lookup and codec entry points and its private corruption fixtures. Existing runtime tests and generated source-slice oracles retain their expectations; relocating static inspection inputs is not evidence of runtime correctness. This change does not alter formats, synchronization, recovery or supported filesystem concurrency. diff --git a/src/authenticated_read/reconstruction.rs b/src/authenticated_read/reconstruction.rs new file mode 100644 index 00000000..5d0fb39b --- /dev/null +++ b/src/authenticated_read/reconstruction.rs @@ -0,0 +1,99 @@ +//! This module owns whole-blob authentication before synchronous emission. + +use super::chunk_verification::{ChunkSource, emitted_chunk, verified_chunk}; +use super::output_write::write_all; +use super::profile_verification::ProfileVerifier; +use super::{ReconstructionFailure, ReconstructionReceipt}; +use crate::{AdmittedLayout, BlobHasher, BlobLength, LayoutId}; +use std::io::Write; + +/// Authenticates the complete blob `layout` names against `store`, then +/// emits it: the one reconstruction core every view shares. +pub(crate) fn reconstruct_admitted( + store: &S, + layout_id: LayoutId, + layout: &AdmittedLayout, + output: &mut W, +) -> Result +where + S: ChunkSource + ?Sized, + W: Write + ?Sized, +{ + verify_complete_blob(store, layout_id, layout)?; + let written = emit_authenticated(store, layout_id, layout, output)?; + let expected = layout.target().logical_length(); + if written != expected { + return Err(ReconstructionFailure::WrittenLengthMismatch { + layout: layout_id, + expected, + observed: written, + }); + } + Ok(ReconstructionReceipt::new( + layout.target(), + layout_id, + written, + )) +} + +fn verify_complete_blob( + store: &S, + layout_id: LayoutId, + layout: &AdmittedLayout, +) -> Result<(), ReconstructionFailure> { + let mut hasher = BlobHasher::new(); + let mut profile = ProfileVerifier::new(layout_id, layout)?; + for (index, entry) in layout.entries().iter().copied().enumerate() { + let bytes = + verified_chunk(store, layout_id, index, entry).map_err(ReconstructionFailure::Chunk)?; + profile.feed(bytes)?; + hasher + .update(bytes) + .map_err(ReconstructionFailure::BlobHash)?; + } + profile.finish()?; + let observed = hasher.finish(); + let expected = layout.target(); + if observed != expected { + return Err(ReconstructionFailure::BlobIdentityMismatch { + layout: layout_id, + expected, + observed, + }); + } + Ok(()) +} + +fn emit_authenticated( + store: &S, + layout_id: LayoutId, + layout: &AdmittedLayout, + output: &mut W, +) -> Result +where + S: ChunkSource + ?Sized, + W: Write + ?Sized, +{ + let mut written = 0_u64; + for (index, entry) in layout.entries().iter().copied().enumerate() { + let bytes = + emitted_chunk(store, layout_id, index, entry).map_err(ReconstructionFailure::Chunk)?; + write_chunk(output, layout_id, bytes, &mut written)?; + } + Ok(BlobLength::new(written)) +} + +fn write_chunk( + output: &mut W, + layout_id: LayoutId, + bytes: &[u8], + written: &mut u64, +) -> Result<(), ReconstructionFailure> +where + W: Write + ?Sized, +{ + write_all(output, bytes, written).map_err(|error| ReconstructionFailure::Output { + layout: layout_id, + source: error, + }) +} diff --git a/src/authenticated_read/reconstruction_failure.rs b/src/authenticated_read/reconstruction_failure.rs new file mode 100644 index 00000000..23f2529a --- /dev/null +++ b/src/authenticated_read/reconstruction_failure.rs @@ -0,0 +1,29 @@ +//! This module owns semantic whole-blob authentication and emission failures. + +use super::{ChunkVerificationError, OutputWriteError}; +use crate::profile::StorageProfileVerificationError; +use crate::{BlobHashError, BlobId, BlobLength, LayoutId}; + +#[derive(Debug)] +pub(crate) enum ReconstructionFailure { + Chunk(ChunkVerificationError), + BlobHash(BlobHashError), + BlobIdentityMismatch { + layout: LayoutId, + expected: BlobId, + observed: BlobId, + }, + Profile { + layout: LayoutId, + source: StorageProfileVerificationError, + }, + Output { + layout: LayoutId, + source: OutputWriteError, + }, + WrittenLengthMismatch { + layout: LayoutId, + expected: BlobLength, + observed: BlobLength, + }, +} diff --git a/src/reference/reconstruction_receipt.rs b/src/authenticated_read/reconstruction_receipt.rs similarity index 100% rename from src/reference/reconstruction_receipt.rs rename to src/authenticated_read/reconstruction_receipt.rs diff --git a/src/lib.rs b/src/lib.rs index 7c1be6a7..d6ff05b3 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -40,13 +40,17 @@ //! namespace transitions while retaining version-1 immutable bytes. //! Partial-prefix recovery now plans and resumes lawful migration residue, //! returning typed refusals and an ordered execution receipt. Filesystem -//! retention publication is available; retention restart recovery, immutable -//! reader snapshots, and garbage collection remain absent. +//! retention publication, bounded restart recovery and fenced snapshots are +//! available. [`DurableStore`] composes a fenced snapshot with authenticated +//! reconstruction and exact-range reads; its snapshot allocation policy is +//! explicit. Garbage collection remains absent. Complete durable read-law and +//! Worldline acceptance remains tracked in issue #109. #[cfg(test)] extern crate self as keep; mod adapters; +mod authenticated_read; mod blob; mod catalog; mod chunk; @@ -55,6 +59,11 @@ mod profile; mod reference; mod retention; +pub use adapters::{ + DurableOutcome, DurableRangeReadReceipt, DurableReadError, DurableReconstructionReceipt, + DurableSnapshot, DurableStore, DurableStoreError, DurableView, +}; + pub use adapters::{ AdmittedCatalog, AdmittedRecoveryStageBytes, AdmittedSegment, AdmittedSegmentRecord, AdmittedStoreFormatMarker, AdmittedStoreMigrationIntent, AdmittedStoreMigrationReceipt, @@ -148,14 +157,15 @@ pub use adapters::{ RetentionRecoveryOutcome, RetentionRecoveryPlan, RetentionRecoveryReceipt, RetentionRecoveryRefusal, RetentionRecoveryStep, RetentionRecoveryStorage, RetentionRootDecodeError, RetentionRootEncodeError, RetentionRootStageAssessment, - RetentionStageAssessment, RetentionStageAssessments, RetentionStorageBoundary, - RetentionStorageError, RetentionStorageProgress, RetentionTransitionDisposition, - RetentionTransitionError, RetentionTransitionPreflight, RetentionTransitionPreflightError, - RetentionTransitionReadiness, RetentionViewCoordinates, RetentionViewError, - RetentionViewSource, VerifiedRetentionClosure, assess_head_stage, assess_manifest_stage, - assess_root_stage, collect_retention_view, execute_retention_publication, - execute_retention_recovery, plan_retention_recovery, plan_retention_transition, - preflight_retention_transition, prepare_retention_publication, verify_retention_closure, + RetentionSelectedRootRefusal, RetentionStageAssessment, RetentionStageAssessments, + RetentionStorageBoundary, RetentionStorageError, RetentionStorageProgress, + RetentionTransitionDisposition, RetentionTransitionError, RetentionTransitionPreflight, + RetentionTransitionPreflightError, RetentionTransitionReadiness, RetentionViewCoordinates, + RetentionViewError, RetentionViewSource, VerifiedRetentionClosure, assess_head_stage, + assess_manifest_stage, assess_root_stage, collect_retention_view, + execute_retention_publication, execute_retention_recovery, plan_retention_recovery, + plan_retention_transition, preflight_retention_transition, prepare_retention_publication, + verify_retention_closure, }; pub use adapters::{ FilesystemMigrationRecoveryRefusal, FilesystemMigrationResidueKind, MIGRATION_NAMESPACE_PREFIX, @@ -184,10 +194,11 @@ pub use layout::{ AdmittedLayout, LayoutEntry, LayoutEntryLimit, LayoutEntryLimitError, LayoutId, LayoutIdMismatch, LayoutRecordLength, LayoutValidationError, RangePlan, RangePlanError, }; -pub use profile::{RegisteredStorageProfile, StorageProfileAdmissionError, StorageProfileId}; +pub use profile::{ + ProfileBoundary, RegisteredStorageProfile, StorageProfileAdmissionError, StorageProfileId, +}; pub use reference::{ - IngestionAllocation, IngestionError, ProfileBoundary, PublishError, PublishedBlob, - RangeReadError, RangeReadReceipt, ReconstructionError, ReconstructionReceipt, ReferenceStore, + IngestionAllocation, IngestionError, PublishError, PublishedBlob, ReferenceStore, ReferenceStoreCapacity, StagedBlob, }; pub use retention::{ @@ -201,3 +212,6 @@ pub use retention::{ RetentionProfileAdmissionError, RetentionRoot, RetentionRootDigest, RetentionRootError, RootGeneration, RootGenerationError, }; + +pub use adapters::{RangeReadError, ReconstructionError}; +pub use authenticated_read::{RangeReadReceipt, ReconstructionReceipt}; diff --git a/src/reference/chunk_source.rs b/src/reference/chunk_source.rs new file mode 100644 index 00000000..1b5ab136 --- /dev/null +++ b/src/reference/chunk_source.rs @@ -0,0 +1,17 @@ +//! This module owns the reference adapter implementation of immutable chunk lookup. + +use crate::ChunkId; +use crate::authenticated_read::ChunkSource; + +impl ChunkSource for crate::ReferenceStore { + fn chunk(&self, identity: ChunkId) -> Option<&[u8]> { + self.chunk(identity) + } + + fn note_chunk_hash(&self, identity: ChunkId) { + #[cfg(test)] + self.observed_chunk_hashes.borrow_mut().push(identity); + #[cfg(not(test))] + let _ = identity; + } +} diff --git a/src/reference/chunk_verification.rs b/src/reference/chunk_verification.rs deleted file mode 100644 index b1d80697..00000000 --- a/src/reference/chunk_verification.rs +++ /dev/null @@ -1,80 +0,0 @@ -//! Exact reference-store chunk lookup and authentication. - -use crate::{ChunkHashError, ChunkId, LayoutEntry, LayoutId, ReferenceStore}; - -pub(super) fn verified_chunk( - store: &ReferenceStore, - layout_id: LayoutId, - index: usize, - entry: LayoutEntry, -) -> Result<&[u8], ChunkVerificationError> { - let expected = entry.chunk_id(); - let bytes = store - .chunk(expected) - .ok_or(ChunkVerificationError::Missing { - layout: layout_id, - index, - requested: expected, - })?; - #[cfg(test)] - store.observed_chunk_hashes.borrow_mut().push(expected); - let observed = ChunkId::hash_bytes(bytes).map_err(|source| ChunkVerificationError::Hash { - layout: layout_id, - index, - expected, - source, - })?; - if observed != expected { - return Err(ChunkVerificationError::IdentityMismatch { - layout: layout_id, - index, - expected, - observed, - }); - } - Ok(bytes) -} - -#[derive(Clone, Copy, Debug)] -pub(super) enum ChunkVerificationError { - Missing { - layout: LayoutId, - index: usize, - requested: ChunkId, - }, - Hash { - layout: LayoutId, - index: usize, - expected: ChunkId, - source: ChunkHashError, - }, - IdentityMismatch { - layout: LayoutId, - index: usize, - expected: ChunkId, - observed: ChunkId, - }, -} - -/// Fetches an already-authenticated immutable chunk for emission. -/// -/// The verification pass hashed every selected chunk before the first byte -/// was written, and the in-memory view cannot change under `&self`, so the -/// emission pass looks the chunk up by identity and hashes nothing. The -/// checked lookup preserves precise failure reporting without relying on -/// unchecked access. This capability is private to the immutable adapter. -pub(super) fn emitted_chunk( - store: &ReferenceStore, - layout_id: LayoutId, - index: usize, - entry: LayoutEntry, -) -> Result<&[u8], ChunkVerificationError> { - let expected = entry.chunk_id(); - store - .chunk(expected) - .ok_or(ChunkVerificationError::Missing { - layout: layout_id, - index, - requested: expected, - }) -} diff --git a/src/reference/mod.rs b/src/reference/mod.rs index d612492a..b7c15bc2 100644 --- a/src/reference/mod.rs +++ b/src/reference/mod.rs @@ -5,35 +5,22 @@ //! makes a retention, crash-recovery, or durability claim. mod capacity; +mod chunk_source; mod chunk_staging; -mod chunk_verification; mod ingestion; mod ingestion_error; -mod output_write; -mod profile_verification; mod publish_error; mod published_blob; mod range_read; -mod range_read_error; -mod range_read_error_display; -mod range_read_error_mapping; -mod range_read_execution; -mod range_read_receipt; mod reconstruction; -mod reconstruction_error; -mod reconstruction_error_display; -mod reconstruction_receipt; mod staged_blob; mod store; -pub use crate::profile::ProfileBoundary; pub use capacity::ReferenceStoreCapacity; pub use ingestion_error::{IngestionAllocation, IngestionError}; pub use publish_error::PublishError; pub use published_blob::PublishedBlob; -pub use range_read_error::RangeReadError; -pub use range_read_receipt::RangeReadReceipt; -pub use reconstruction_error::ReconstructionError; -pub use reconstruction_receipt::ReconstructionReceipt; pub use staged_blob::StagedBlob; pub use store::ReferenceStore; + +use crate::{RangeReadError, RangeReadReceipt, ReconstructionError, ReconstructionReceipt}; diff --git a/src/reference/profile_verification.rs b/src/reference/profile_verification.rs deleted file mode 100644 index e14e061c..00000000 --- a/src/reference/profile_verification.rs +++ /dev/null @@ -1,58 +0,0 @@ -//! Reference-adapter mapping for domain-owned storage-profile replay. - -use crate::profile::{StorageProfileVerificationError, StorageProfileVerifier}; -use crate::{AdmittedLayout, LayoutId}; - -use super::ReconstructionError; - -pub(super) struct ProfileVerifier<'a> { - layout: LayoutId, - verifier: StorageProfileVerifier<'a>, -} - -impl<'a> ProfileVerifier<'a> { - pub(super) fn new( - layout: LayoutId, - admitted: &'a AdmittedLayout, - ) -> Result { - let verifier = - StorageProfileVerifier::new(admitted).map_err(|error| map_error(layout, error))?; - Ok(Self { layout, verifier }) - } - - pub(super) fn feed(&mut self, bytes: &[u8]) -> Result<(), ReconstructionError> { - self.verifier - .feed(bytes) - .map_err(|error| map_error(self.layout, error)) - } - - pub(super) fn finish(self) -> Result<(), ReconstructionError> { - self.verifier - .finish() - .map_err(|error| map_error(self.layout, error)) - } -} - -const fn map_error( - layout: LayoutId, - error: StorageProfileVerificationError, -) -> ReconstructionError { - match error { - StorageProfileVerificationError::Unsupported { profile } => { - ReconstructionError::ProfileVerifierUnavailable { layout, profile } - } - StorageProfileVerificationError::Chunking { source } => { - ReconstructionError::ProfileChunking { layout, source } - } - StorageProfileVerificationError::BoundaryMismatch { - index, - expected, - observed, - } => ReconstructionError::ProfileBoundaryMismatch { - layout, - index, - expected, - observed, - }, - } -} diff --git a/src/reference/range_read.rs b/src/reference/range_read.rs index aa57c9a1..60cee453 100644 --- a/src/reference/range_read.rs +++ b/src/reference/range_read.rs @@ -4,8 +4,8 @@ use std::io::Write; use crate::{AdmittedLayout, BlobId, ByteRange, LayoutDecodePolicy, LayoutId, ReferenceStore}; -use super::range_read_execution::read_admitted; use super::{RangeReadError, RangeReadReceipt}; +use crate::authenticated_read::read_admitted; impl ReferenceStore { /// Reads one exact logical byte range from a committed blob. @@ -93,7 +93,7 @@ impl ReferenceStore { .ok_or(RangeReadError::LayoutMissing { requested: layout_id, })?; - read_admitted(self, layout_id, layout, requested, output) + read_admitted(self, layout_id, layout, requested, output).map_err(RangeReadError::from) } /// Resolves a caller-supplied admitted layout to one committed range. @@ -157,3 +157,7 @@ impl ReferenceStore { self.read_admitted_layout_range(&layout, requested, output) } } + +#[cfg(test)] +#[path = "range_read_tests.rs"] +mod tests; diff --git a/src/reference/reconstruction.rs b/src/reference/reconstruction.rs index a2b7d86e..149792d9 100644 --- a/src/reference/reconstruction.rs +++ b/src/reference/reconstruction.rs @@ -2,14 +2,10 @@ use std::io::Write; -use crate::{ - AdmittedLayout, BlobHasher, BlobId, BlobLength, LayoutDecodePolicy, LayoutId, ReferenceStore, -}; +use crate::{AdmittedLayout, BlobId, LayoutDecodePolicy, LayoutId, ReferenceStore}; -use super::chunk_verification::{ChunkVerificationError, emitted_chunk, verified_chunk}; -use super::output_write::{OutputWriteError, write_all}; -use super::profile_verification::ProfileVerifier; use super::{ReconstructionError, ReconstructionReceipt}; +use crate::authenticated_read::reconstruct_admitted; impl ReferenceStore { /// Reconstructs the exact bytes named by `target`. @@ -69,7 +65,7 @@ impl ReferenceStore { .ok_or(ReconstructionError::LayoutMissing { requested: layout_id, })?; - reconstruct_admitted(self, layout_id, layout, output) + reconstruct_admitted(self, layout_id, layout, output).map_err(ReconstructionError::from) } /// Reconstructs through a caller-supplied admitted semantic layout. @@ -96,7 +92,7 @@ impl ReferenceStore { .encode_record() .map_err(ReconstructionError::LayoutEncoding)? .id(); - reconstruct_admitted(self, layout_id, layout, output) + reconstruct_admitted(self, layout_id, layout, output).map_err(ReconstructionError::from) } /// Decodes and reconstructs one exact canonical layout record. @@ -127,161 +123,6 @@ impl ReferenceStore { } } -fn reconstruct_admitted( - store: &ReferenceStore, - layout_id: LayoutId, - layout: &AdmittedLayout, - output: &mut W, -) -> Result -where - W: Write + ?Sized, -{ - verify_complete_blob(store, layout_id, layout)?; - let written = emit_authenticated(store, layout_id, layout, output)?; - let expected = layout.target().logical_length(); - if written != expected { - return Err(ReconstructionError::WrittenLengthMismatch { - layout: layout_id, - expected, - observed: written, - }); - } - Ok(ReconstructionReceipt::new( - layout.target(), - layout_id, - written, - )) -} - -fn verify_complete_blob( - store: &ReferenceStore, - layout_id: LayoutId, - layout: &AdmittedLayout, -) -> Result<(), ReconstructionError> { - let mut hasher = BlobHasher::new(); - let mut profile = ProfileVerifier::new(layout_id, layout)?; - for (index, entry) in layout.entries().iter().copied().enumerate() { - let bytes = - verified_chunk(store, layout_id, index, entry).map_err(reconstruction_chunk_error)?; - profile.feed(bytes)?; - hasher - .update(bytes) - .map_err(ReconstructionError::BlobHash)?; - } - profile.finish()?; - let observed = hasher.finish(); - let expected = layout.target(); - if observed != expected { - return Err(ReconstructionError::BlobIdentityMismatch { - layout: layout_id, - expected, - observed, - }); - } - Ok(()) -} - -fn emit_authenticated( - store: &ReferenceStore, - layout_id: LayoutId, - layout: &AdmittedLayout, - output: &mut W, -) -> Result -where - W: Write + ?Sized, -{ - let mut written = 0_u64; - for (index, entry) in layout.entries().iter().copied().enumerate() { - let bytes = - emitted_chunk(store, layout_id, index, entry).map_err(reconstruction_chunk_error)?; - write_chunk(output, layout_id, bytes, &mut written)?; - } - Ok(BlobLength::new(written)) -} - -fn write_chunk( - output: &mut W, - layout_id: LayoutId, - bytes: &[u8], - written: &mut u64, -) -> Result<(), ReconstructionError> -where - W: Write + ?Sized, -{ - write_all(output, bytes, written).map_err(|error| reconstruction_output_error(layout_id, error)) -} - -const fn reconstruction_chunk_error(error: ChunkVerificationError) -> ReconstructionError { - match error { - ChunkVerificationError::Missing { - layout, - index, - requested, - } => ReconstructionError::ChunkMissing { - layout, - index, - requested, - }, - ChunkVerificationError::Hash { - layout, - index, - expected, - source, - } => ReconstructionError::ChunkHash { - layout, - index, - expected, - source, - }, - ChunkVerificationError::IdentityMismatch { - layout, - index, - expected, - observed, - } => ReconstructionError::ChunkIdentityMismatch { - layout, - index, - expected, - observed, - }, - } -} - -fn reconstruction_output_error(layout: LayoutId, error: OutputWriteError) -> ReconstructionError { - match error { - OutputWriteError::WriteZero { bytes_written } => ReconstructionError::WriteZero { - layout, - bytes_written: BlobLength::new(bytes_written), - }, - OutputWriteError::InvalidWriteCount { - maximum, - observed, - bytes_written, - } => ReconstructionError::InvalidWriteCount { - layout, - maximum, - observed, - bytes_written: BlobLength::new(bytes_written), - }, - OutputWriteError::Write { - bytes_written, - source, - } => ReconstructionError::Write { - layout, - bytes_written: BlobLength::new(bytes_written), - source, - }, - OutputWriteError::LengthOverflow { - bytes_written, - incoming, - } => ReconstructionError::WrittenLengthOverflow { - layout, - bytes_written: BlobLength::new(bytes_written), - incoming, - }, - } -} - #[cfg(test)] #[path = "reconstruction_tests.rs"] mod tests; diff --git a/tests/golden_file_worldline.rs b/tests/golden_file_worldline.rs index 8b8c79e1..6f374afe 100644 --- a/tests/golden_file_worldline.rs +++ b/tests/golden_file_worldline.rs @@ -4,3 +4,7 @@ pub(crate) mod support; #[path = "golden_file_worldline/suite.rs"] mod suite; + +#[cfg(target_os = "linux")] +#[path = "layout_mutations/support.rs"] +pub(crate) mod layout_mutation_support; diff --git a/tests/golden_file_worldline/durable_assertions.rs b/tests/golden_file_worldline/durable_assertions.rs new file mode 100644 index 00000000..c9f1fafe --- /dev/null +++ b/tests/golden_file_worldline/durable_assertions.rs @@ -0,0 +1,96 @@ +//! Durable Worldline reads through public production admission and publication. +//! +//! Size: medium; owned admitted ext4 storage, no ambient network or sleeps. +//! Oracle: Worldline's independently frozen identities and original input bytes. +//! Delete when durable reads are removed or stronger corpus witnesses subsume these laws. + +use std::error::Error; + +use keep::{ByteLength, ByteOffset, ByteRange, DurableStore, ReaderAttemptLimit}; + +use super::durable_fixture::{build, policy}; +use super::identity_corpus::{IdentityCase, identity_cases}; + +#[test] +fn reopened_durable_views_reconstruct_every_worldline_identity() -> Result<(), Box> { + let cases = identity_cases()?; + let bytes = cases + .iter() + .map(IdentityCase::bytes) + .collect::, _>>()?; + let sources = bytes.iter().map(Vec::as_slice).collect::>(); + let sandbox = build("durable-worldline-reopen", &sources)?; + for (case, expected) in cases.iter().zip(&bytes) { + let store = DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?; + let target = case.expected_id()?; + let mut output = Vec::new(); + let receipt = store.reconstruct(target, &mut output)?; + assert_eq!( + &output, expected, + "reopened durable bytes for {}", + case.name + ); + assert_eq!( + receipt.receipt().target(), + target, + "frozen identity for {}", + case.name + ); + assert_eq!( + receipt.receipt().bytes_written().get(), + u64::try_from(expected.len())? + ); + assert!( + store.contains_blob(target)?, + "published Worldline identity must remain discoverable" + ); + } + sandbox.remove()?; + Ok(()) +} + +#[test] +fn durable_worldline_ranges_match_source_slices_after_reopen() -> Result<(), Box> { + let cases = identity_cases()?; + let bytes = cases + .iter() + .map(IdentityCase::bytes) + .collect::, _>>()?; + let sources = bytes.iter().map(Vec::as_slice).collect::>(); + let sandbox = build("durable-worldline-ranges", &sources)?; + for (case, expected) in cases.iter().zip(&bytes) { + let store = DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?; + let snapshot = store.snapshot()?; + let target = case.expected_id()?; + let length = u64::try_from(expected.len())?; + for (start, end) in [ + (0, 0), + (0, length), + (length, length), + ( + length.checked_div(3).ok_or("zero divisor")?, + length + .checked_sub(length.checked_div(3).ok_or("zero divisor")?) + .ok_or("inverted interval")?, + ), + ] { + let requested = ByteRange::new( + ByteOffset::new(start), + ByteLength::new(end.checked_sub(start).ok_or("inverted interval")?), + )?; + let mut output = Vec::new(); + let receipt = snapshot.read_range(target, requested, &mut output)?; + assert_eq!( + Some(output.as_slice()), + expected.get(usize::try_from(start)?..usize::try_from(end)?), + "range {requested:?} for {}", + case.name + ); + assert_eq!(receipt.receipt().target(), target); + assert_eq!(receipt.receipt().requested(), requested); + assert_eq!(receipt.view(), snapshot.view()); + } + } + sandbox.remove()?; + Ok(()) +} diff --git a/tests/golden_file_worldline/durable_closure_refusal.rs b/tests/golden_file_worldline/durable_closure_refusal.rs new file mode 100644 index 00000000..8bdc4a9c --- /dev/null +++ b/tests/golden_file_worldline/durable_closure_refusal.rs @@ -0,0 +1,145 @@ +//! Selected retained closure admission precedes every usable durable view. +//! +//! Size: medium. Oracle: every retained anchor requires its complete named closure. +//! The adversarial fixture is checksummed but semantically incomplete, installed +//! outside publication deliberately; production preflight must never publish it. +//! Delete only if durable retained closure admission is removed or subsumed. + +use std::error::Error; +use std::fmt::Write as _; +use std::fs; +use std::path::{Path, PathBuf}; + +use super::durable_fixture::{build_missing_chunk, identify, policy}; +use keep::{ + CanonicalRetentionHead, CanonicalRetentionManifest, CanonicalRetentionRoot, DurableOutcome, + DurableStore, DurableStoreError, LivenessGeneration, ReaderAttemptLimit, + RegisteredRetentionProfile, RetentionAnchor, RetentionClosureLimits, + RetentionClosureVerificationError, RetentionHead, RetentionManifest, RetentionManifestEntry, + RetentionManifestLength, RetentionNamespace, RetentionNamespaceDigest, RetentionPolicy, + RetentionRoot, RootGeneration, SegmentRecordIdentity, +}; + +struct InstalledClaim { + namespace: RetentionNamespaceDigest, + evidence: Vec, +} + +#[test] +fn an_unsatisfied_retained_closure_refuses_snapshot_admission_before_output() +-> Result<(), Box> { + let bytes = b"retained root claims a missing chunk"; + let sandbox = build_missing_chunk("durable-incomplete-retained-closure", bytes)?; + let named = identify(bytes)?; + let missing = named.spans.first().ok_or("chunk absent")?.id(); + let InstalledClaim { + namespace, + evidence, + } = install_claim( + sandbox.path(), + RetentionAnchor::new(named.target, named.record.id()), + )?; + let before = evidence + .iter() + .map(fs::read) + .collect::, _>>()?; + let store = DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?; + let failure = store + .snapshot() + .err() + .ok_or("incomplete retained closure admitted a snapshot")?; + assert!( + matches!(&failure, DurableStoreError::Closure { namespace: actual, source } + if *actual == namespace && matches!(source.as_ref(), + RetentionClosureVerificationError::MissingMember { identity } + if *identity == SegmentRecordIdentity::Chunk(missing))), + "snapshot must refuse exact retained closure member: {failure:?}" + ); + let mut output = vec![0xAB]; + let failure = store + .reconstruct(named.target, &mut output) + .err() + .ok_or("incomplete retained closure reconstructed")?; + assert!( + matches!(&failure, DurableOutcome::Store(DurableStoreError::Closure { namespace: actual, source }) + if *actual == namespace && matches!(source.as_ref(), + RetentionClosureVerificationError::MissingMember { identity } + if *identity == SegmentRecordIdentity::Chunk(missing))), + "convenience read must preserve snapshot refusal: {failure:?}" + ); + assert_eq!( + output, + [0xAB], + "admission refusal must not touch caller output" + ); + assert_eq!( + evidence + .iter() + .map(fs::read) + .collect::, _>>()?, + before, + "admission refusal must preserve selected root, manifest and head bytes" + ); + Ok(()) +} + +fn install_claim(path: &Path, anchor: RetentionAnchor) -> Result> { + let namespace = RetentionNamespace::try_from(&b"incomplete"[..])?; + let digest = namespace.digest(); + let root = RetentionRoot::new( + namespace, + RootGeneration::INITIAL, + RetentionPolicy::new( + RegisteredRetentionProfile::SINGLE_CANONICAL_WITNESS_V1, + RetentionClosureLimits::new(4096, 8, 16_777_216, 67_108_864)?, + ), + None, + vec![anchor], + )?; + let root_bytes = CanonicalRetentionRoot::from_root(&root)?; + let manifest = RetentionManifest::new( + LivenessGeneration::INITIAL, + None, + vec![RetentionManifestEntry::new( + digest, + RootGeneration::INITIAL, + root_bytes.digest(), + )], + )?; + let manifest_bytes = CanonicalRetentionManifest::from_manifest(&manifest)?; + let head = RetentionHead::new( + LivenessGeneration::INITIAL, + RetentionManifestLength::new(u64::try_from(manifest_bytes.encoded().len())?)?, + manifest_bytes.digest(), + None, + )?; + let directory = path.join("retention/roots").join(hex(digest.as_bytes())?); + fs::create_dir(&directory)?; + let root_path = directory.join(format!( + "0000000000000001-{}.root", + hex(root_bytes.digest().as_bytes())? + )); + let manifest_path = path.join("retention/manifests").join(format!( + "0000000000000001-{}.manifest", + hex(manifest_bytes.digest().as_bytes())? + )); + let head_path = path.join("retention/HEAD"); + fs::write(&root_path, root_bytes.encoded())?; + fs::write(&manifest_path, manifest_bytes.encoded())?; + fs::write( + &head_path, + CanonicalRetentionHead::from_head(&head).encoded(), + )?; + Ok(InstalledClaim { + namespace: digest, + evidence: vec![root_path, manifest_path, head_path], + }) +} + +fn hex(bytes: &[u8]) -> Result { + let mut result = String::new(); + for byte in bytes { + write!(result, "{byte:02x}")?; + } + Ok(result) +} diff --git a/tests/golden_file_worldline/durable_corruption_laws.rs b/tests/golden_file_worldline/durable_corruption_laws.rs new file mode 100644 index 00000000..e617d798 --- /dev/null +++ b/tests/golden_file_worldline/durable_corruption_laws.rs @@ -0,0 +1,122 @@ +//! Durable payload corruption is refused before any output. +//! +//! Size: medium. Oracle: the normative v1 record framing and independently +//! assembled checksum preimage, plus caller output preservation. +//! Delete when durable segment reads disappear or stronger corruption laws subsume this case. + +use std::error::Error; +use std::fs; + +use super::durable_fixture::{build, identify, policy}; +use keep::{ + CatalogRestartError, DurableOutcome, DurableStore, DurableStoreError, + FilesystemRetentionSnapshotError, ReaderAttemptLimit, SegmentReadError, + SegmentRecordDecodeError, +}; + +#[test] +fn a_corrupt_selected_chunk_preserves_exact_record_refusal_before_output() +-> Result<(), Box> { + let source = b"one physical chunk"; + let sandbox = build("durable-corrupt-chunk", &[source])?; + let target = identify(source)?.target; + let segment_path = fs::read_dir(sandbox.path().join("segments"))? + .next() + .ok_or("segment absent")?? + .path(); + let mut bytes = fs::read(&segment_path)?; + // The v1 grammar fixes the segment header at 64 bytes and record header at 112. + let payload = 64_usize.checked_add(112).ok_or("payload offset overflow")?; + let checksum_offset = payload + .checked_add(source.len()) + .ok_or("checksum offset overflow")?; + let checksum_end = checksum_offset + .checked_add(32) + .ok_or("checksum end overflow")?; + let observed: [u8; 32] = bytes + .get(checksum_offset..checksum_end) + .ok_or("record checksum absent")? + .try_into()?; + *bytes.get_mut(payload).ok_or("payload absent")? ^= 1; + let covered = bytes + .get(64..checksum_offset) + .ok_or("record preimage absent")?; + let expected = record_checksum(covered)?; + fs::write(segment_path, bytes)?; + let store = DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?; + let mut output = vec![0xAB]; + let failure = store + .reconstruct(target, &mut output) + .err() + .ok_or("corrupt chunk reconstructed")?; + assert!( + matches!(&failure, DurableOutcome::Store(DurableStoreError::Snapshot(snapshot)) + if matches!(snapshot.as_ref(), FilesystemRetentionSnapshotError::Catalog { source: CatalogRestartError::Segment { source, .. } } + if matches!(source.as_ref(), SegmentReadError::RecordDecode { record_index: 0, offset: 64, + source: SegmentRecordDecodeError::ChecksumMismatch { expected: actual_expected, observed: actual_observed } } + if actual_expected.as_bytes() == &expected && actual_observed.as_bytes() == &observed))), + "payload corruption must retain exact record and checksum coordinates: {failure:?}" + ); + assert_eq!(output, [0xAB]); + Ok(()) +} + +#[test] +fn a_corrupt_selected_chunk_refuses_a_range_before_output() -> Result<(), Box> { + let source = b"one physical chunk"; + let sandbox = build("durable-corrupt-range", &[source])?; + let target = identify(source)?.target; + let segment_path = fs::read_dir(sandbox.path().join("segments"))? + .next() + .ok_or("segment absent")?? + .path(); + let mut bytes = fs::read(&segment_path)?; + // The v1 grammar fixes the segment header at 64 bytes and record header at 112. + let payload = 64_usize.checked_add(112).ok_or("payload offset overflow")?; + let checksum_offset = payload + .checked_add(source.len()) + .ok_or("checksum offset overflow")?; + let checksum_end = checksum_offset + .checked_add(32) + .ok_or("checksum end overflow")?; + let observed: [u8; 32] = bytes + .get(checksum_offset..checksum_end) + .ok_or("record checksum absent")? + .try_into()?; + *bytes.get_mut(payload).ok_or("payload absent")? ^= 1; + let covered = bytes + .get(64..checksum_offset) + .ok_or("record preimage absent")?; + let expected = record_checksum(covered)?; + fs::write(segment_path, bytes)?; + let store = DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?; + let mut output = vec![0xAB]; + let failure = store + .read_range( + target, + keep::ByteRange::new(keep::ByteOffset::new(1), keep::ByteLength::new(3))?, + &mut output, + ) + .err() + .ok_or("corrupt chunk produced a range")?; + assert!( + matches!(&failure, DurableOutcome::Store(DurableStoreError::Snapshot(snapshot)) + if matches!(snapshot.as_ref(), FilesystemRetentionSnapshotError::Catalog { source: CatalogRestartError::Segment { source, .. } } + if matches!(source.as_ref(), SegmentReadError::RecordDecode { record_index: 0, offset: 64, + source: SegmentRecordDecodeError::ChecksumMismatch { expected: actual_expected, observed: actual_observed } } + if actual_expected.as_bytes() == &expected && actual_observed.as_bytes() == &observed))), + "payload corruption must retain exact record and checksum coordinates: {failure:?}" + ); + assert_eq!(output, [0xAB]); + Ok(()) +} + +fn record_checksum(covered: &[u8]) -> Result<[u8; 32], Box> { + let mut oracle = blake3::Hasher::new(); + oracle.update(b"KEEP:SEG:RECORD:SUM\0"); + oracle.update(&1_u16.to_be_bytes()); + oracle.update(&[1]); + oracle.update(covered); + oracle.update(&u64::try_from(covered.len())?.to_be_bytes()); + Ok(*oracle.finalize().as_bytes()) +} diff --git a/tests/golden_file_worldline/durable_diagnostic_laws.rs b/tests/golden_file_worldline/durable_diagnostic_laws.rs new file mode 100644 index 00000000..2b96f585 --- /dev/null +++ b/tests/golden_file_worldline/durable_diagnostic_laws.rs @@ -0,0 +1,76 @@ +//! Durable failures render each boundary without repeating the preserved cause. +//! +//! Size: medium. Oracle: each boundary describes itself; `Error::source` retains +//! the original typed cause for reporters to render separately. +//! Delete when durable read APIs disappear or stronger diagnostic evidence subsumes these laws. + +use super::durable_fixture::{build, identify, policy}; +use crate::support::ZeroWriter; +use keep::{ByteLength, ByteOffset, ByteRange, DurableStore, ReaderAttemptLimit}; +use std::error::Error; + +#[test] +fn durable_whole_failure_renders_each_boundary_once() -> Result<(), Box> { + let bytes = b"diagnostic whole output"; + let sandbox = build("durable-whole-diagnostic", &[bytes])?; + let store = DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?; + let failure = store + .reconstruct(identify(bytes)?.target, &mut ZeroWriter) + .err() + .ok_or("zero writer succeeded")?; + let read = failure.source().ok_or("read cause absent")?; + let core = read.source().ok_or("core cause absent")?; + assert!( + !failure.to_string().contains(&read.to_string()), + "store boundary repeats read cause: {failure}" + ); + assert!( + !read.to_string().contains(&core.to_string()), + "read boundary repeats core cause: {read}" + ); + Ok(()) +} + +#[test] +fn durable_range_failure_renders_each_boundary_once() -> Result<(), Box> { + let bytes = b"diagnostic range output"; + let sandbox = build("durable-range-diagnostic", &[bytes])?; + let store = DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?; + let range = ByteRange::new(ByteOffset::new(0), ByteLength::new(1))?; + let failure = store + .read_range(identify(bytes)?.target, range, &mut ZeroWriter) + .err() + .ok_or("zero range writer succeeded")?; + let read = failure.source().ok_or("read cause absent")?; + let core = read.source().ok_or("core cause absent")?; + assert!( + !failure.to_string().contains(&read.to_string()), + "store boundary repeats range cause: {failure}" + ); + assert!( + !read.to_string().contains(&core.to_string()), + "range boundary repeats core cause: {read}" + ); + Ok(()) +} + +#[test] +fn durable_admission_failure_renders_its_boundary_once() -> Result<(), Box> { + let bytes = b"diagnostic admission"; + let sandbox = build("durable-admission-diagnostic", &[bytes])?; + let store = DurableStore::open( + &sandbox.path().join("absent"), + policy()?, + ReaderAttemptLimit::DEFAULT, + )?; + let failure = store + .reconstruct(identify(bytes)?.target, &mut Vec::new()) + .err() + .ok_or("absent store admitted")?; + let cause = failure.source().ok_or("admission cause absent")?; + assert!( + !failure.to_string().contains(&cause.to_string()), + "store boundary repeats admission cause: {failure}" + ); + Ok(()) +} diff --git a/tests/golden_file_worldline/durable_fixture.rs b/tests/golden_file_worldline/durable_fixture.rs new file mode 100644 index 00000000..265250db --- /dev/null +++ b/tests/golden_file_worldline/durable_fixture.rs @@ -0,0 +1,203 @@ +//! This module owns real version-one publication followed by migration and retention. + +use std::collections::BTreeSet; +use std::error::Error; +use std::fs; + +use keep::{ + AdmittedLayout, AdmittedRetentionRoot, AdmittedSegment, AdmittedSegmentRecord, BlobId, + CanonicalCatalog, CanonicalLayoutRecord, CanonicalRetentionRoot, CatalogGeneration, + CatalogPublicationExpectation, CatalogRestartByteLimit, CatalogRestartPolicy, ChunkSpan, + FastCdc, FilesystemCatalogPublisher, FilesystemCatalogSnapshot, FilesystemPlatformAdmission, + FilesystemRetentionPublicationAuthority, FilesystemStoreMigrationAuthority, + FilesystemVersionTwoAdmission, LayoutEntryLimit, RegisteredRetentionProfile, + RegisteredStorageProfile, RetentionAnchor, RetentionClosureLimits, + RetentionGenerationExpectation, RetentionNamespace, RetentionPolicy, RetentionRoot, + RootGeneration, SegmentReadPolicy, SegmentRecordLimit, StagedSegment, + execute_retention_publication, execute_store_migration, preflight_retention_transition, + prepare_retention_publication, publish_catalog_generation, +}; + +use super::durable_sandbox::TestDirectory; + +type TestResult = Result>; + +#[derive(Clone, Copy, Eq, PartialEq)] +enum Records { + Complete, + LayoutsOnly, + OnlyChunk(keep::ChunkId), +} + +pub(super) struct Identified { + pub(super) spans: Vec, + pub(super) record: CanonicalLayoutRecord, + pub(super) target: BlobId, +} + +pub(super) fn identify(bytes: &[u8]) -> TestResult { + let target = BlobId::hash_bytes(bytes)?; + let mut detector = FastCdc::new(); + let mut spans = Vec::new(); + detector.feed(bytes, |span| spans.push(span))?; + if let Some(span) = detector.finish()? { + spans.push(span); + } + let layout = AdmittedLayout::from_spans( + target, + RegisteredStorageProfile::FAST_CDC_64K_V1, + spans.clone(), + LayoutEntryLimit::MAXIMUM, + )?; + Ok(Identified { + spans, + record: layout.encode_record()?, + target, + }) +} + +pub(super) fn policy() -> TestResult { + Ok(CatalogRestartPolicy::new( + SegmentReadPolicy::new(SegmentRecordLimit::MAXIMUM, LayoutEntryLimit::MAXIMUM), + CatalogRestartByteLimit::new(16_777_216)?, + )) +} + +pub(super) fn build(name: &str, contents: &[&[u8]]) -> TestResult { + let sandbox = TestDirectory::create(name)?; + let identified = contents + .iter() + .map(|bytes| identify(bytes)) + .collect::>>()?; + publish_catalog(&sandbox, contents, &identified, Records::Complete)?; + migrate(&sandbox)?; + publish_retention(&sandbox, &identified)?; + Ok(sandbox) +} + +pub(super) fn build_missing_chunk(name: &str, content: &[u8]) -> TestResult { + let sandbox = TestDirectory::create(name)?; + let identified = [identify(content)?]; + publish_catalog(&sandbox, &[content], &identified, Records::LayoutsOnly)?; + migrate(&sandbox)?; + Ok(sandbox) +} + +pub(super) fn build_selected_chunk( + name: &str, + content: &[u8], + index: usize, +) -> TestResult { + let sandbox = TestDirectory::create(name)?; + let identified = [identify(content)?]; + let selected = identified + .first() + .ok_or("identity absent")? + .spans + .get(index) + .ok_or("selected chunk absent")? + .id(); + publish_catalog( + &sandbox, + &[content], + &identified, + Records::OnlyChunk(selected), + )?; + migrate(&sandbox)?; + Ok(sandbox) +} + +fn migrate(sandbox: &TestDirectory) -> TestResult<()> { + let admission = FilesystemPlatformAdmission::reopen(sandbox.path())?; + let mut migration = FilesystemStoreMigrationAuthority::open( + admission, + SegmentReadPolicy::new(SegmentRecordLimit::MAXIMUM, LayoutEntryLimit::MAXIMUM), + )?; + let intent = migration.observe_intent()?; + let _receipt = execute_store_migration(&mut migration, &intent)?; + drop(migration); + Ok(()) +} + +fn publish_catalog( + sandbox: &TestDirectory, + contents: &[&[u8]], + identified: &[Identified], + records: Records, +) -> TestResult<()> { + let admission = FilesystemPlatformAdmission::initialize(sandbox.path())?; + let mut publisher = FilesystemCatalogPublisher::open(admission, policy()?)?; + let mut stage = StagedSegment::begin( + publisher.create_segment_stage()?, + SegmentRecordLimit::MAXIMUM, + )?; + let mut chunks = BTreeSet::new(); + let mut layouts = BTreeSet::new(); + for (bytes, identity) in contents.iter().zip(identified) { + for span in &identity.spans { + if matches!(records, Records::Complete) || records == Records::OnlyChunk(span.id()) { + if !chunks.insert(span.id()) { + continue; + } + let start = usize::try_from(span.offset().get())?; + let end = usize::try_from(span.end().get())?; + stage = stage.append(AdmittedSegmentRecord::for_chunk( + bytes.get(start..end).ok_or("chunk outside source")?, + )?)?; + } + } + if layouts.insert(identity.record.id()) { + stage = stage.append(AdmittedSegmentRecord::for_layout(&identity.record)?)?; + } + } + let sealed = stage.seal()?; + let bytes = fs::read(sandbox.path().join("staging/current.seg"))?; + let segments = [AdmittedSegment::decode( + &bytes, + SegmentReadPolicy::new(SegmentRecordLimit::MAXIMUM, LayoutEntryLimit::MAXIMUM), + )?]; + let selected = publisher.select_segment(sealed, segments.first().ok_or("segment absent")?)?; + let catalog = CanonicalCatalog::from_segments(CatalogGeneration::new(1)?, None, &segments)?; + let _receipt = publish_catalog_generation( + &mut publisher, + CatalogPublicationExpectation::uninitialized(), + selected, + &catalog, + &segments, + )?; + Ok(()) +} + +fn publish_retention(sandbox: &TestDirectory, identified: &[Identified]) -> TestResult<()> { + let anchors = identified + .iter() + .map(|entry| RetentionAnchor::new(entry.target, entry.record.id())) + .collect::>() + .into_iter() + .collect(); + let root = RetentionRoot::new( + RetentionNamespace::try_from(&b"worldline"[..])?, + RootGeneration::INITIAL, + RetentionPolicy::new( + RegisteredRetentionProfile::SINGLE_CANONICAL_WITNESS_V1, + RetentionClosureLimits::new(4096, 8, 16_777_216, 67_108_864)?, + ), + None, + anchors, + )?; + let canonical = CanonicalRetentionRoot::from_root(&root)?; + let candidate = AdmittedRetentionRoot::decode(canonical.encoded())?; + let current = FilesystemCatalogSnapshot::load(sandbox.path(), policy()?)?; + let snapshot = current.snapshot()?; + let preflight = preflight_retention_transition( + RetentionGenerationExpectation::Absent, + None, + candidate, + &snapshot, + )?; + let prepared = prepare_retention_publication(preflight, None)?; + let admission = FilesystemVersionTwoAdmission::reopen(sandbox.path())?; + let mut authority = FilesystemRetentionPublicationAuthority::open(admission)?; + let _receipt = execute_retention_publication(&mut authority, &prepared)?; + Ok(()) +} diff --git a/tests/golden_file_worldline/durable_layout_laws.rs b/tests/golden_file_worldline/durable_layout_laws.rs new file mode 100644 index 00000000..03c1e923 --- /dev/null +++ b/tests/golden_file_worldline/durable_layout_laws.rs @@ -0,0 +1,213 @@ +//! Caller-supplied layouts preserve the same durable admission laws as reference reads. +//! +//! Size: medium. Oracle: source bytes and explicitly different same-length blob identities. +//! Delete when layout ingress is removed or stronger boundary evidence subsumes these laws. + +use std::error::Error; + +use super::durable_fixture::{build, identify, policy}; +use keep::{ + AdmittedLayout, BlobId, ByteLength, ByteOffset, ByteRange, DurableReadError, DurableStore, + LayoutDecodePolicy, LayoutEntryLimit, RangeReadError, ReaderAttemptLimit, ReconstructionError, + RegisteredStorageProfile, +}; + +#[test] +fn durable_layout_ingress_routes_return_identical_authenticated_bytes() -> Result<(), Box> +{ + let bytes = b"one catalogued layout through every ingress"; + let sandbox = build("durable-layout-ingress", &[bytes])?; + let identified = identify(bytes)?; + let decode = LayoutDecodePolicy::new(LayoutEntryLimit::MAXIMUM); + let layout = AdmittedLayout::decode_record(identified.record.bytes(), decode)?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let mut admitted_output = Vec::new(); + let admitted = snapshot.reconstruct_admitted_layout(&layout, &mut admitted_output)?; + let mut record_output = Vec::new(); + let record = + snapshot.reconstruct_record(identified.record.bytes(), decode, &mut record_output)?; + assert_eq!(admitted_output, bytes); + assert_eq!(record_output, bytes); + assert_eq!(admitted, record); + let requested = ByteRange::new(ByteOffset::new(4), ByteLength::new(10))?; + let mut admitted_range = Vec::new(); + let first = snapshot.read_admitted_layout_range(&layout, requested, &mut admitted_range)?; + let mut record_range = Vec::new(); + let second = snapshot.read_record_range( + identified.record.bytes(), + decode, + requested, + &mut record_range, + )?; + assert_eq!(Some(admitted_range.as_slice()), bytes.get(4..14)); + assert_eq!(Some(record_range.as_slice()), bytes.get(4..14)); + assert_eq!(first, second); + Ok(()) +} + +#[test] +fn a_wrong_whole_blob_claim_refuses_before_any_durable_output() -> Result<(), Box> { + let bytes = b"aaaa"; + let sandbox = build("durable-wrong-blob", &[bytes])?; + let identified = identify(bytes)?; + let wrong = BlobId::hash_bytes(b"bbbb")?; + let layout = AdmittedLayout::from_spans( + wrong, + RegisteredStorageProfile::FAST_CDC_64K_V1, + identified.spans, + LayoutEntryLimit::MAXIMUM, + )?; + let layout_id = layout.encode_record()?.id(); + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let mut output = vec![0xAB]; + let failure = snapshot + .reconstruct_admitted_layout(&layout, &mut output) + .err() + .ok_or("false blob claim succeeded")?; + assert!( + matches!(&failure, DurableReadError::Reconstruction(source) + if matches!(source.as_ref(), ReconstructionError::BlobIdentityMismatch { layout, expected, observed } + if *layout == layout_id && *expected == wrong && *observed == identified.target)), + "exact whole-blob refusal: {failure:?}" + ); + assert_eq!(output, [0xAB]); + Ok(()) +} + +#[test] +fn supplied_range_layouts_cannot_replace_the_catalogued_target_binding() +-> Result<(), Box> { + let bytes = b"aaaa"; + let sandbox = build("durable-range-target-binding", &[bytes])?; + let identified = identify(bytes)?; + let wrong = BlobId::hash_bytes(b"bbbb")?; + let layout = AdmittedLayout::from_spans( + wrong, + RegisteredStorageProfile::FAST_CDC_64K_V1, + identified.spans, + LayoutEntryLimit::MAXIMUM, + )?; + let record = layout.encode_record()?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let requested = ByteRange::new(ByteOffset::new(0), ByteLength::new(1))?; + let mut output = vec![0xAB]; + let failure = snapshot + .read_admitted_layout_range(&layout, requested, &mut output) + .err() + .ok_or("uncatalogued layout succeeded")?; + assert!( + matches!(failure, DurableReadError::LayoutMissing { requested } if requested == record.id()) + ); + let failure = snapshot + .read_record_range( + record.bytes(), + LayoutDecodePolicy::new(LayoutEntryLimit::MAXIMUM), + requested, + &mut output, + ) + .err() + .ok_or("uncatalogued record succeeded")?; + assert!( + matches!(failure, DurableReadError::LayoutMissing { requested } if requested == record.id()) + ); + assert_eq!(output, [0xAB]); + Ok(()) +} + +#[test] +fn supplied_corrupt_whole_record_reports_input_decode_before_output() -> Result<(), Box> +{ + let bytes = b"a canonical layout whose checksum will be damaged"; + let sandbox = build("durable-layout-checksum", &[bytes])?; + let identified = identify(bytes)?; + let mut encoded = identified.record.bytes().to_vec(); + let offset = encoded.len().checked_sub(32).ok_or("checksum absent")?; + let expected: [u8; 32] = encoded.get(offset..).ok_or("checksum absent")?.try_into()?; + *encoded.last_mut().ok_or("record empty")? ^= 1; + let observed: [u8; 32] = encoded.get(offset..).ok_or("checksum absent")?.try_into()?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let decode = LayoutDecodePolicy::new(LayoutEntryLimit::MAXIMUM); + let mut output = vec![0xAB]; + + let failure = snapshot + .reconstruct_record(&encoded, decode, &mut output) + .err() + .ok_or("corrupt input reconstructed")?; + assert!( + matches!(&failure, DurableReadError::Reconstruction(error) + if matches!(error.as_ref(), ReconstructionError::LayoutDecode(keep::LayoutDecodeError::ChecksumMismatch { expected: e, observed: o }) + if *e == expected && *o == observed)), + "caller whole-record decode boundary: {failure:?}" + ); + assert_eq!(output, [0xAB]); + Ok(()) +} + +#[test] +fn supplied_corrupt_range_record_reports_input_decode_before_output() -> Result<(), Box> +{ + let bytes = b"a canonical layout whose checksum will be damaged"; + let sandbox = build("durable-range-layout-checksum", &[bytes])?; + let identified = identify(bytes)?; + let mut encoded = identified.record.bytes().to_vec(); + let offset = encoded.len().checked_sub(32).ok_or("checksum absent")?; + let expected: [u8; 32] = encoded.get(offset..).ok_or("checksum absent")?.try_into()?; + *encoded.last_mut().ok_or("record empty")? ^= 1; + let observed: [u8; 32] = encoded.get(offset..).ok_or("checksum absent")?.try_into()?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let decode = LayoutDecodePolicy::new(LayoutEntryLimit::MAXIMUM); + let mut output = vec![0xAB]; + + let range = ByteRange::new(ByteOffset::new(0), ByteLength::new(1))?; + let failure = snapshot + .read_record_range(&encoded, decode, range, &mut output) + .err() + .ok_or("corrupt input range succeeded")?; + assert!( + matches!(&failure, DurableReadError::RangeRead(error) + if matches!(error.as_ref(), RangeReadError::LayoutDecode(keep::LayoutDecodeError::ChecksumMismatch { expected: e, observed: o }) + if *e == expected && *o == observed)), + "caller range-record decode boundary: {failure:?}" + ); + assert_eq!(output, [0xAB]); + Ok(()) +} + +#[test] +fn a_content_correct_durable_layout_refuses_false_profile_boundaries() -> Result<(), Box> +{ + let mutation = crate::layout_mutation_support::mutation_cases()? + .into_iter() + .find(|candidate| candidate.case() == "profile-boundary-mismatch") + .ok_or("profile mismatch fixture absent")?; + let encoded = mutation.mutated_record()?; + let first = vec![0_u8; 262_143]; + let last = [0_u8; 2]; + let sandbox = build("durable-false-profile", &[&first, &last])?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let mut output = vec![0xAB]; + let failure = snapshot + .reconstruct_record( + &encoded, + LayoutDecodePolicy::new(LayoutEntryLimit::MAXIMUM), + &mut output, + ) + .err() + .ok_or("false profile boundaries reconstructed")?; + assert!( + matches!(&failure, DurableReadError::Reconstruction(error) + if matches!(error.as_ref(), ReconstructionError::ProfileBoundaryMismatch { + index: 0, expected: Some(expected), observed: Some(observed), .. } + if expected.offset().get() == 0 && expected.length().get() == 262_143 + && observed.offset().get() == 0 && observed.length().get() == 262_144)), + "false profile must preserve exact boundary coordinates: {failure:?}" + ); + assert_eq!(output, [0xAB]); + Ok(()) +} diff --git a/tests/golden_file_worldline/durable_locator_laws.rs b/tests/golden_file_worldline/durable_locator_laws.rs new file mode 100644 index 00000000..f435f326 --- /dev/null +++ b/tests/golden_file_worldline/durable_locator_laws.rs @@ -0,0 +1,139 @@ +//! A durable handle's relative locator is bound when the handle is created. +//! +//! Size: medium; one isolated child owns working-directory changes and real +//! production-profile stores. Oracle: changing cwd cannot change a handle's +//! selected store. The child has a 20-second execution ceiling, not a latency +//! assertion. Delete if relative locators are removed or stronger laws subsume it. + +use std::error::Error; +use std::io; +use std::path::{Path, PathBuf}; +use std::process::Command; + +use keep::{DurableStore, DurableStoreError, ReaderAttemptLimit}; + +use super::durable_fixture::{build, identify, policy}; +use super::durable_sandbox::TestDirectory; + +type TestResult = Result<(), Box>; + +const CHILD: &str = "KEEP_DURABLE_LOCATOR_CHILD"; +const LAW: &str = "suite::durable_locator_laws::relative_store_handles_keep_their_initial_store_after_a_directory_change"; + +#[test] +fn relative_store_handles_keep_their_initial_store_after_a_directory_change() +-> Result<(), Box> { + run_isolated(LAW, change_directory_after_open) +} + +#[test] +fn an_unresolvable_relative_locator_preserves_its_io_cause() -> Result<(), Box> { + run_isolated( + "suite::durable_locator_laws::an_unresolvable_relative_locator_preserves_its_io_cause", + deleted_current_directory, + ) +} + +fn run_isolated(law: &str, operation: fn() -> TestResult) -> Result<(), Box> { + let child_arguments = ["--exact", law, "--nocapture", "--test-threads=1"]; + if std::env::var_os(CHILD).as_deref() == Some(std::ffi::OsStr::new(law)) + && std::env::args_os() + .skip(1) + .eq(child_arguments.map(std::ffi::OsStr::new)) + { + return operation(); + } + let output = Command::new("timeout") + .arg("20s") + .arg(std::env::current_exe()?) + .args(child_arguments) + .env(CHILD, law) + .output()?; + assert!( + output.status.success(), + "isolated locator law failed: {:?}\n{}\n{}", + output.status.code(), + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); + Ok(()) +} + +fn deleted_current_directory() -> Result<(), Box> { + let sandbox = TestDirectory::create("durable-locator-deleted-cwd")?; + let restore = WorkingDirectory(std::env::current_dir()?); + std::env::set_current_dir(sandbox.path())?; + std::fs::remove_dir(sandbox.path())?; + + let result = DurableStore::open(Path::new("."), policy()?, ReaderAttemptLimit::DEFAULT); + + restore.restore()?; + let refusal = result + .err() + .ok_or("an unresolvable relative locator was accepted")?; + assert!( + matches!(&refusal, DurableStoreError::Locator { source } + if source.kind() == io::ErrorKind::NotFound + && source.raw_os_error() == Some(rustix::io::Errno::NOENT.raw_os_error())), + "locator failure must retain its exact NotFound cause: {refusal:?}" + ); + assert_eq!( + refusal + .source() + .and_then(|source| source.downcast_ref::()) + .and_then(io::Error::raw_os_error), + Some(rustix::io::Errno::NOENT.raw_os_error()), + "the locator's original I/O cause must remain available through Error::source" + ); + Ok(()) +} + +fn change_directory_after_open() -> Result<(), Box> { + let first = build("durable-locator-first", &[b"first store"])?; + let second = build("durable-locator-second", &[b"second store"])?; + let first_blob = identify(b"first store")?.target; + let second_blob = identify(b"second store")?.target; + let restore = WorkingDirectory(std::env::current_dir()?); + std::env::set_current_dir(first.path())?; + let store = DurableStore::open(Path::new("."), policy()?, ReaderAttemptLimit::DEFAULT)?; + assert!( + store.contains_blob(first_blob)?, + "initial store must contain its own blob" + ); + + std::env::set_current_dir(second.path())?; + + assert!( + store.contains_blob(first_blob)?, + "the same handle must retain its original store after cwd changes" + ); + assert!( + !store.contains_blob(second_blob)?, + "the other store must not supply retention through the original handle" + ); + let mut output = Vec::new(); + let _receipt = store.reconstruct(first_blob, &mut output)?; + assert_eq!( + output, b"first store", + "relative locator must still read the original store's bytes" + ); + restore.restore()?; + Ok(()) +} + +// The normal path checks restoration; Drop also attempts it on errors/unwind. +// This guard does not make cwd mutation safe in a parallel shared process; +// run_isolated admits only the exact single-law child invocation above. +struct WorkingDirectory(PathBuf); + +impl WorkingDirectory { + fn restore(&self) -> io::Result<()> { + std::env::set_current_dir(&self.0) + } +} + +impl Drop for WorkingDirectory { + fn drop(&mut self) { + let _ = self.restore(); + } +} diff --git a/tests/golden_file_worldline/durable_namespace_laws.rs b/tests/golden_file_worldline/durable_namespace_laws.rs new file mode 100644 index 00000000..4e530aa9 --- /dev/null +++ b/tests/golden_file_worldline/durable_namespace_laws.rs @@ -0,0 +1,150 @@ +//! Selected retention roots must belong to the namespace selecting them. +//! +//! Size: medium (owned production-profile filesystem). Oracle: a manifest +//! entry binds namespace, root generation, and root digest jointly. A canonical +//! foreign root is contradictory evidence, even when its closure is complete. +//! Delete only if namespace selection is removed or stronger public laws subsume it. + +use std::error::Error; +use std::fmt::Write as _; +use std::fs; +use std::io::ErrorKind; +use std::path::Path; + +use keep::{ + CanonicalRetentionHead, CanonicalRetentionManifest, DurableOutcome, DurableStore, + DurableStoreError, FilesystemRetentionSnapshot, FilesystemRetentionSnapshotError, + ReaderAttemptLimit, RetentionHead, RetentionManifest, RetentionManifestEntry, + RetentionManifestLength, RetentionNamespace, RetentionNamespaceDigest, + RetentionSelectedRootRefusal, +}; + +use super::durable_fixture::{build, identify, policy}; + +#[test] +fn a_selected_root_from_another_namespace_refuses_direct_read() -> Result<(), Box> { + let sandbox = build("durable-direct-foreign-root", &[b"namespace-bound"])?; + let namespace = install_foreign_selection(sandbox.path())?; + let snapshot = + FilesystemRetentionSnapshot::load(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?; + + let refusal = snapshot + .retained_root(namespace) + .err() + .ok_or("foreign namespace root was returned by the public reader")?; + + assert_root_refusal(&refusal, namespace)?; + Ok(()) +} + +#[test] +fn a_foreign_retained_namespace_refuses_durable_output() -> Result<(), Box> { + let bytes = b"namespace-bound"; + let sandbox = build("durable-foreign-root-output", &[bytes])?; + let namespace = install_foreign_selection(sandbox.path())?; + let store = DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?; + let refusal = store + .snapshot() + .err() + .ok_or("foreign namespace root admitted a durable snapshot")?; + let DurableStoreError::Snapshot(source) = refusal else { + return Err(format!("unexpected snapshot refusal: {refusal:?}").into()); + }; + assert_root_refusal(&source, namespace)?; + let mut output = vec![0xAB]; + let refusal = store + .reconstruct(identify(bytes)?.target, &mut output) + .err() + .ok_or("foreign namespace root produced durable output")?; + let DurableOutcome::Store(DurableStoreError::Snapshot(source)) = refusal else { + return Err(format!("unexpected read refusal: {refusal:?}").into()); + }; + assert_root_refusal(&source, namespace)?; + assert_eq!( + output, + [0xAB], + "namespace refusal must precede caller output" + ); + Ok(()) +} + +fn assert_root_refusal( + refusal: &FilesystemRetentionSnapshotError, + namespace: RetentionNamespaceDigest, +) -> Result<(), Box> { + let original = RetentionNamespace::try_from(&b"worldline"[..])?.digest(); + assert!( + matches!(refusal, FilesystemRetentionSnapshotError::Root { source } + if source.kind() == ErrorKind::InvalidData + && source.get_ref().and_then(|cause| cause.downcast_ref::()) + == Some(&RetentionSelectedRootRefusal::Namespace {expected: namespace, observed: original})), + "a namespace contradiction must preserve its exact root refusal and coordinates: {refusal:?}" + ); + Ok(()) +} + +fn install_foreign_selection(path: &Path) -> Result> { + let view = FilesystemRetentionSnapshot::load(path, policy()?, ReaderAttemptLimit::DEFAULT)?; + let selected = view + .manifest() + .ok_or("manifest absent")? + .entries() + .first() + .copied() + .ok_or("root entry absent")?; + let original = view + .retained_root(selected.namespace())? + .ok_or("root absent")?; + let head = *view.retention_head().ok_or("retention head absent")?; + drop(view); + let namespace = RetentionNamespace::try_from(&b"different-namespace"[..])?.digest(); + let manifest = RetentionManifest::new( + head.generation(), + None, + vec![RetentionManifestEntry::new( + namespace, + selected.root_generation(), + selected.root_digest(), + )], + )?; + let manifest = CanonicalRetentionManifest::from_manifest(&manifest)?; + let head = RetentionHead::new( + head.generation(), + RetentionManifestLength::new(u64::try_from(manifest.encoded().len())?)?, + manifest.digest(), + None, + )?; + let directory = path + .join("retention/roots") + .join(hex(namespace.as_bytes())?); + fs::create_dir(&directory)?; + fs::write( + directory.join(format!( + "{:016x}-{}.root", + selected.root_generation().get(), + hex(selected.root_digest().as_bytes())? + )), + original, + )?; + fs::write( + path.join("retention/manifests").join(format!( + "{:016x}-{}.manifest", + head.generation().get(), + hex(manifest.digest().as_bytes())? + )), + manifest.encoded(), + )?; + fs::write( + path.join("retention/HEAD"), + CanonicalRetentionHead::from_head(&head).encoded(), + )?; + Ok(namespace) +} + +fn hex(bytes: &[u8]) -> Result { + let mut result = String::new(); + for byte in bytes { + write!(result, "{byte:02x}")?; + } + Ok(result) +} diff --git a/tests/golden_file_worldline/durable_output_laws.rs b/tests/golden_file_worldline/durable_output_laws.rs new file mode 100644 index 00000000..a9141dc0 --- /dev/null +++ b/tests/golden_file_worldline/durable_output_laws.rs @@ -0,0 +1,136 @@ +//! Public durable output failure and short-write laws. +//! +//! Size: medium; controlled filesystem and writer capabilities, no network or sleeps. +//! Oracle: caller-supplied bytes, specified accepted-prefix accounting and writer refusal. +//! Delete when these APIs disappear or stronger boundary evidence subsumes these laws. + +use std::error::Error; +use std::io::ErrorKind; + +use super::durable_fixture::{build, identify, policy}; +use crate::support::{PartitionWriter, PrefixThenFailWriter, ZeroWriter}; +use keep::{ + ByteLength, ByteOffset, ByteRange, DurableReadError, DurableStore, RangeReadError, + ReaderAttemptLimit, ReconstructionError, +}; + +#[test] +fn durable_reconstruction_completes_short_and_interrupted_writes() -> Result<(), Box> { + let bytes = b"authenticated durable bytes through short writes"; + let sandbox = build("durable-short-writes", &[bytes])?; + let identified = identify(bytes)?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let mut output = PartitionWriter::new(&[1, 7, 3])?; + let receipt = snapshot.reconstruct(identified.target, &mut output)?; + assert_eq!(output.bytes(), bytes); + assert_eq!( + receipt.receipt().bytes_written().get(), + u64::try_from(bytes.len())? + ); + Ok(()) +} + +#[test] +fn durable_reconstruction_preserves_a_failed_writers_accepted_prefix() -> Result<(), Box> +{ + let bytes = b"a failed output has no successful receipt"; + let sandbox = build("durable-output-prefix", &[bytes])?; + let identified = identify(bytes)?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let mut output = PrefixThenFailWriter::new(5)?; + let failure = snapshot + .reconstruct(identified.target, &mut output) + .err() + .ok_or("failed writer received a receipt")?; + assert!( + matches!(&failure, DurableReadError::Reconstruction(source) + if matches!(source.as_ref(), ReconstructionError::Write { layout, bytes_written, source } + if *layout == identified.record.id() && bytes_written.get() == 5 + && source.kind() == ErrorKind::PermissionDenied)), + "exact prefix and cause: {failure:?}" + ); + assert_eq!(output.bytes(), bytes.get(..5).ok_or("prefix absent")?); + Ok(()) +} + +#[test] +fn durable_range_preserves_a_failed_writers_accepted_prefix() -> Result<(), Box> { + let bytes = b"a range failure preserves its exact output prefix"; + let sandbox = build("durable-range-prefix", &[bytes])?; + let identified = identify(bytes)?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let requested = ByteRange::new(ByteOffset::new(3), ByteLength::new(13))?; + let mut output = PrefixThenFailWriter::new(5)?; + let failure = snapshot + .read_range(identified.target, requested, &mut output) + .err() + .ok_or("failed range writer received a receipt")?; + assert!( + matches!(&failure, DurableReadError::RangeRead(source) + if matches!(source.as_ref(), RangeReadError::Write { layout, bytes_written, source } + if *layout == identified.record.id() && bytes_written.get() == 5 + && source.kind() == ErrorKind::PermissionDenied)), + "exact prefix and cause: {failure:?}" + ); + assert_eq!(output.bytes(), bytes.get(3..8).ok_or("prefix absent")?); + Ok(()) +} + +#[test] +fn a_zero_progress_durable_writer_refuses_at_the_output_boundary() -> Result<(), Box> { + let bytes = b"nonempty output must make progress"; + let sandbox = build("durable-output-zero", &[bytes])?; + let identified = identify(bytes)?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let failure = snapshot + .reconstruct(identified.target, &mut ZeroWriter) + .err() + .ok_or("zero writer succeeded")?; + assert!( + matches!(&failure, DurableReadError::Reconstruction(source) + if matches!(source.as_ref(), ReconstructionError::WriteZero { layout, bytes_written } + if *layout == identified.record.id() && bytes_written.is_empty())), + "exact zero-progress refusal: {failure:?}" + ); + Ok(()) +} + +#[test] +fn durable_ranges_complete_short_and_interrupted_writes() -> Result<(), Box> { + let bytes = b"authenticated durable range through short writes"; + let sandbox = build("durable-range-short", &[bytes])?; + let identified = identify(bytes)?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let requested = ByteRange::new(ByteOffset::new(3), ByteLength::new(13))?; + let mut output = PartitionWriter::new(&[1, 7, 3])?; + let receipt = snapshot.read_range(identified.target, requested, &mut output)?; + assert_eq!(output.bytes(), bytes.get(3..16).ok_or("range absent")?); + assert_eq!(receipt.receipt().bytes_written(), requested.length()); + Ok(()) +} + +#[test] +fn a_zero_progress_durable_range_writer_refuses_before_acceptance() -> Result<(), Box> { + let bytes = b"nonempty range must make progress"; + let sandbox = build("durable-range-zero", &[bytes])?; + let identified = identify(bytes)?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let requested = ByteRange::new(ByteOffset::new(3), ByteLength::new(13))?; + let failure = snapshot + .read_range(identified.target, requested, &mut ZeroWriter) + .err() + .ok_or("zero range writer succeeded")?; + assert!( + matches!(&failure, DurableReadError::RangeRead(source) + if matches!(source.as_ref(), RangeReadError::WriteZero { layout, bytes_written } + if *layout == identified.record.id() && bytes_written.is_empty())), + "exact zero-progress range refusal: {failure:?}" + ); + Ok(()) +} diff --git a/tests/golden_file_worldline/durable_range_properties.rs b/tests/golden_file_worldline/durable_range_properties.rs new file mode 100644 index 00000000..6126e276 --- /dev/null +++ b/tests/golden_file_worldline/durable_range_properties.rs @@ -0,0 +1,180 @@ +//! Bounded generated durable range laws with an independent source-slice oracle. +//! +//! Size: medium; owned ext4 stores and fixed deterministic input domains. +//! No random seed: coordinates are enumerated in checked-in order. Short ranges +//! run in increasing length, so the first failure is minimal in that domain. +//! Delete when range reads disappear or stronger source-slice evidence subsumes these laws. + +use std::error::Error; + +use super::durable_fixture::{build, policy}; +use super::identity_corpus::find_case; +use keep::{ + BlobId, ByteLength, ByteOffset, ByteRange, DurableSnapshot, DurableStore, ReaderAttemptLimit, +}; + +#[test] +fn every_short_durable_range_equals_its_source_slice() -> Result<(), Box> { + let source = (0_u8..64).collect::>(); + let sandbox = build("durable-all-short-ranges", &[&source])?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let target = BlobId::hash_bytes(&source)?; + let size = u64::try_from(source.len())?; + for length in 0..=size { + for start in 0..=size + .checked_sub(length) + .ok_or("range length exceeds source")? + { + let range = ByteRange::new(ByteOffset::new(start), ByteLength::new(length))?; + assert_slice(&snapshot, target, &source, range)?; + } + } + Ok(()) +} + +#[test] +fn multichunk_durable_boundary_ranges_equal_the_frozen_worldline_source() +-> Result<(), Box> { + let case = find_case("large-ramp")?; + let source = case.bytes()?; + let sandbox = build("durable-multichunk-boundaries", &[&source])?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let size = u64::try_from(source.len())?; + let positions = [ + 0, + 1, + 65_535, + 65_536, + 65_537, + 262_143, + 262_144, + size.checked_sub(1).ok_or("large corpus empty")?, + size, + ]; + for (index, start) in positions.iter().copied().enumerate() { + for end in positions + .get(index..) + .ok_or("position suffix absent")? + .iter() + .copied() + { + let length = end.checked_sub(start).ok_or("inverted range")?; + let range = ByteRange::new(ByteOffset::new(start), ByteLength::new(length))?; + assert_slice(&snapshot, case.expected_id()?, &source, range)?; + } + } + Ok(()) +} + +fn assert_slice( + snapshot: &DurableSnapshot, + target: BlobId, + source: &[u8], + requested: ByteRange, +) -> Result<(), Box> { + let mut output = Vec::new(); + let receipt = snapshot.read_range(target, requested, &mut output)?; + let expected = source + .get(usize::try_from(requested.offset().get())?..usize::try_from(requested.end().get())?) + .ok_or("range outside source")?; + assert_eq!(output, expected, "source-slice oracle for {requested:?}"); + assert_eq!(receipt.receipt().requested(), requested); + assert_eq!(receipt.receipt().bytes_written(), requested.length()); + Ok(()) +} + +#[test] +fn generated_multichunk_durable_ranges_equal_the_reference_domain() -> Result<(), Box> { + // This is the same finite input domain as range_read_properties.rs, with + // independent source slices as the oracle rather than another read engine. + let mut source = Vec::new(); + for index in 0_usize..786_432 { + let value = index + .checked_mul(17) + .and_then(|scaled| scaled.checked_add(29)) + .and_then(|shifted| shifted.checked_rem(251)) + .ok_or("source arithmetic overflow")?; + source.push(u8::try_from(value)?); + } + let sandbox = build("durable-reference-range-domain", &[&source])?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let target = BlobId::hash_bytes(&source)?; + let coordinate_count = u64::try_from(source.len())? + .checked_add(1) + .ok_or("coordinate overflow")?; + for case in 0_u64..128 { + let first = range_coordinate(case, 104_729, 17, coordinate_count)?; + let second = range_coordinate(case, 130_363, 101, coordinate_count)?; + let start = first.min(second); + let length = first + .max(second) + .checked_sub(start) + .ok_or("inverted range")?; + let requested = ByteRange::new(ByteOffset::new(start), ByteLength::new(length))?; + assert_slice(&snapshot, target, &source, requested)?; + } + Ok(()) +} + +fn range_coordinate( + case: u64, + multiplier: u64, + increment: u64, + count: u64, +) -> Result> { + case.checked_mul(multiplier) + .and_then(|product| product.checked_add(increment)) + .and_then(|expanded| expanded.checked_rem(count)) + .ok_or_else(|| "coordinate arithmetic refused".into()) +} + +#[test] +fn a_durable_range_needs_no_nonoverlapping_chunk_records() -> Result<(), Box> { + let source = [ + vec![0_u8; 262_144], + vec![1_u8; 262_144], + vec![2_u8; 262_144], + ] + .concat(); + let identified = super::durable_fixture::identify(&source)?; + let selected = identified.spans.get(1).ok_or("interior chunk absent")?; + let sandbox = super::durable_fixture::build_selected_chunk("durable-only-overlap", &source, 1)?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let start = selected + .offset() + .get() + .checked_add(1) + .ok_or("offset overflow")?; + let requested = ByteRange::new(ByteOffset::new(start), ByteLength::new(1))?; + let mut output = Vec::new(); + let receipt = snapshot.read_layout_range(identified.record.id(), requested, &mut output)?; + assert_eq!( + output, + source + .get( + usize::try_from(start)? + ..usize::try_from(start.checked_add(1).ok_or("end overflow")?)? + ) + .ok_or("slice absent")? + ); + assert_eq!(receipt.receipt().requested(), requested); + assert_eq!(receipt.receipt().bytes_written(), requested.length()); + let mut whole_output = vec![0xAB]; + let failure = snapshot + .reconstruct_layout(identified.record.id(), &mut whole_output) + .err() + .ok_or("missing nonoverlapping chunk reconstructed")?; + let first = identified.spans.first().ok_or("first chunk absent")?.id(); + assert!( + matches!(&failure, keep::DurableReadError::Reconstruction(error) + if matches!(error.as_ref(), keep::ReconstructionError::ChunkMissing { layout, index: 0, requested } + if *layout == identified.record.id() && *requested == first)), + "whole reconstruction must expose missing evidence outside the successful range: {failure:?}" + ); + assert_eq!(whole_output, [0xAB]); + Ok(()) +} diff --git a/tests/golden_file_worldline/durable_read_memory.rs b/tests/golden_file_worldline/durable_read_memory.rs new file mode 100644 index 00000000..2bcbc706 --- /dev/null +++ b/tests/golden_file_worldline/durable_read_memory.rs @@ -0,0 +1,38 @@ +//! Incremental read allocation after explicit durable snapshot materialization. +//! +//! Size: medium. Oracle: reading the 1 MiB Worldline source must not allocate a +//! second whole-blob buffer; the already-admitted snapshot is outside this scope. +//! Delete when the no-additional-whole-blob-buffer contract changes or stronger +//! generated allocation evidence subsumes this concrete witness. + +use std::error::Error; +use std::io; + +use super::durable_fixture::{build, policy}; +use super::identity_corpus::find_case; +use keep::{DurableStore, ReaderAttemptLimit}; + +#[test] +fn durable_reconstruction_does_not_allocate_an_additional_whole_blob() -> Result<(), Box> +{ + let case = find_case("large-ramp")?; + let bytes = case.bytes()?; + let sandbox = build("durable-read-allocation", &[&bytes])?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let target = case.expected_id()?; + let mut output = io::sink(); + let mut result = None; + let allocation = allocation_counter::measure(|| { + result = Some(snapshot.reconstruct(target, &mut output)); + }); + let receipt = result.ok_or("read was not executed")??; + let length = u64::try_from(bytes.len())?; + assert_eq!(receipt.receipt().bytes_written().get(), length); + assert!( + allocation.bytes_max < length, + "incremental read allocation {} must be below the {length}-byte blob; snapshot materialization is separately bounded by policy", + allocation.bytes_max + ); + Ok(()) +} diff --git a/tests/golden_file_worldline/durable_refusal_laws.rs b/tests/golden_file_worldline/durable_refusal_laws.rs new file mode 100644 index 00000000..98518b0c --- /dev/null +++ b/tests/golden_file_worldline/durable_refusal_laws.rs @@ -0,0 +1,135 @@ +//! Evidenced logical absence through a completely admitted durable catalog. +//! +//! Size: medium; production publication and migration in an owned ext4 namespace. +//! Oracle: a published layout names the exact missing chunk; no output is permitted. +//! Delete when durable exact-layout reads are removed or stronger refusal evidence subsumes these laws. + +use std::error::Error; + +use super::durable_fixture::{build, build_missing_chunk, identify, policy}; +use keep::{ + ByteLength, ByteOffset, ByteRange, DurableReadError, DurableStore, RangePlanError, + RangeReadError, ReaderAttemptLimit, ReconstructionError, +}; + +#[test] +fn a_durable_layout_naming_an_absent_chunk_refuses_before_reconstruction_output() +-> Result<(), Box> { + let bytes = b"layout evidence exists but its chunk does not"; + let sandbox = build_missing_chunk("durable-missing-logical-chunk", bytes)?; + let expected = identify(bytes)?; + let chunk = expected + .spans + .first() + .ok_or("nonempty input has no chunk")? + .id(); + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let mut output = vec![0xAB]; + let failure = snapshot + .reconstruct_layout(expected.record.id(), &mut output) + .err() + .ok_or("missing chunk reconstructed")?; + assert!( + matches!(&failure, DurableReadError::Reconstruction(source) + if matches!(source.as_ref(), ReconstructionError::ChunkMissing { layout, index, requested } + if *layout == expected.record.id() && *index == 0 && *requested == chunk)), + "missing logical member must be evidenced with exact coordinates: {failure:?}" + ); + assert_eq!( + output, + [0xAB], + "refusal must preserve the caller's existing output" + ); + Ok(()) +} + +#[test] +fn a_durable_range_naming_an_absent_chunk_refuses_before_output() -> Result<(), Box> { + let bytes = b"range layout evidence without its selected chunk"; + let sandbox = build_missing_chunk("durable-range-missing-chunk", bytes)?; + let expected = identify(bytes)?; + let chunk = expected + .spans + .first() + .ok_or("nonempty input has no chunk")? + .id(); + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let requested = ByteRange::new(ByteOffset::new(1), ByteLength::new(3))?; + let mut output = vec![0xAB]; + let failure = snapshot + .read_layout_range(expected.record.id(), requested, &mut output) + .err() + .ok_or("missing chunk range succeeded")?; + assert!( + matches!(&failure, DurableReadError::RangeRead(source) + if matches!(source.as_ref(), RangeReadError::ChunkMissing { layout, index, requested } + if *layout == expected.record.id() && *index == 0 && *requested == chunk)), + "missing logical range member must preserve exact coordinates: {failure:?}" + ); + assert_eq!(output, [0xAB]); + Ok(()) +} + +#[test] +fn an_out_of_bounds_durable_range_refuses_the_exact_requested_coordinates() +-> Result<(), Box> { + let bytes = b"one"; + let sandbox = build("durable-range-outside", &[bytes])?; + let expected = identify(bytes)?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let requested = ByteRange::new(ByteOffset::new(2), ByteLength::new(2))?; + let mut output = vec![0xAB]; + let failure = snapshot + .read_range(expected.target, requested, &mut output) + .err() + .ok_or("out of bounds range succeeded")?; + assert!( + matches!(&failure, DurableReadError::RangeRead(source) + if matches!(source.as_ref(), RangeReadError::RangePlan(RangePlanError::OutOfBounds { requested: actual, target_length }) + if *actual == requested && *target_length == expected.target.logical_length())), + "exact range bounds: {failure:?}" + ); + assert_eq!(output, [0xAB]); + Ok(()) +} + +#[test] +fn an_absent_durable_blob_refuses_range_output_with_its_exact_identity() +-> Result<(), Box> { + let sandbox = build("durable-absent-blob", &[b"present"])?; + let absent = identify(b"absent")?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let requested = ByteRange::new(ByteOffset::new(0), ByteLength::new(1))?; + let mut output = vec![0xAB]; + let failure = snapshot + .read_range(absent.target, requested, &mut output) + .err() + .ok_or("absent blob produced a range")?; + assert!( + matches!(failure, DurableReadError::BlobMissing { requested } if requested == absent.target) + ); + assert_eq!(output, [0xAB]); + Ok(()) +} + +#[test] +fn an_absent_durable_layout_refuses_before_reconstruction_output() -> Result<(), Box> { + let sandbox = build("durable-absent-layout", &[b"present"])?; + let absent = identify(b"absent")?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let mut output = vec![0xAB]; + let failure = snapshot + .reconstruct_layout(absent.record.id(), &mut output) + .err() + .ok_or("absent layout reconstructed")?; + assert!( + matches!(failure, DurableReadError::LayoutMissing { requested } if requested == absent.record.id()) + ); + assert_eq!(output, [0xAB]); + Ok(()) +} diff --git a/tests/golden_file_worldline/durable_writer_failures.rs b/tests/golden_file_worldline/durable_writer_failures.rs new file mode 100644 index 00000000..5737a4cb --- /dev/null +++ b/tests/golden_file_worldline/durable_writer_failures.rs @@ -0,0 +1,103 @@ +//! Exact durable output capability failures. +//! +//! Size: medium. Oracle: the Write contract and exact accepted-byte accounting. +//! Delete when durable reads disappear or stronger output laws subsume these cases. + +use super::durable_fixture::{build, identify, policy}; +use crate::support::{FailingWriter, LyingWriter}; +use keep::{ + ByteLength, ByteOffset, ByteRange, DurableReadError, DurableStore, RangeReadError, + ReaderAttemptLimit, ReconstructionError, +}; +use std::error::Error; +use std::io::ErrorKind; + +#[test] +fn durable_reconstruction_preserves_immediate_output_failure() -> Result<(), Box> { + let bytes = b"exact output capability failure"; + let sandbox = build("durable-reconstruction-failing", &[bytes])?; + let identified = identify(bytes)?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let failure = snapshot + .reconstruct(identified.target, &mut FailingWriter) + .err() + .ok_or("broken writer received a successful receipt")?; + assert!( + matches!(&failure, DurableReadError::Reconstruction(error) + if matches!(error.as_ref(), ReconstructionError::Write { layout, bytes_written, source } + if *layout == identified.record.id() && bytes_written.is_empty() + && source.kind() == ErrorKind::PermissionDenied)), + "exact output failure: {failure:?}" + ); + Ok(()) +} + +#[test] +fn durable_reconstruction_rejects_impossible_write_counts() -> Result<(), Box> { + let bytes = b"exact output capability failure"; + let sandbox = build("durable-reconstruction-lying", &[bytes])?; + let identified = identify(bytes)?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let failure = snapshot + .reconstruct(identified.target, &mut LyingWriter) + .err() + .ok_or("broken writer received a successful receipt")?; + let expected_maximum = bytes.len(); + let expected_observed = expected_maximum.checked_add(1).ok_or("count overflow")?; + assert!( + matches!(&failure, DurableReadError::Reconstruction(error) + if matches!(error.as_ref(), ReconstructionError::InvalidWriteCount { layout, maximum, observed, bytes_written } + if *layout == identified.record.id() && bytes_written.is_empty() + && *maximum == expected_maximum && *observed == expected_observed)), + "exact output failure: {failure:?}" + ); + Ok(()) +} + +#[test] +fn durable_range_preserves_immediate_output_failure() -> Result<(), Box> { + let bytes = b"exact output capability failure"; + let sandbox = build("durable-range-failing", &[bytes])?; + let identified = identify(bytes)?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let requested = ByteRange::new(ByteOffset::new(3), ByteLength::new(11))?; + let failure = snapshot + .read_range(identified.target, requested, &mut FailingWriter) + .err() + .ok_or("broken writer received a successful receipt")?; + assert!( + matches!(&failure, DurableReadError::RangeRead(error) + if matches!(error.as_ref(), RangeReadError::Write { layout, bytes_written, source } + if *layout == identified.record.id() && bytes_written.is_empty() + && source.kind() == ErrorKind::PermissionDenied)), + "exact output failure: {failure:?}" + ); + Ok(()) +} + +#[test] +fn durable_range_rejects_impossible_write_counts() -> Result<(), Box> { + let bytes = b"exact output capability failure"; + let sandbox = build("durable-range-lying", &[bytes])?; + let identified = identify(bytes)?; + let snapshot = + DurableStore::open(sandbox.path(), policy()?, ReaderAttemptLimit::DEFAULT)?.snapshot()?; + let requested = ByteRange::new(ByteOffset::new(3), ByteLength::new(11))?; + let failure = snapshot + .read_range(identified.target, requested, &mut LyingWriter) + .err() + .ok_or("broken writer received a successful receipt")?; + let expected_maximum = 11_usize; + let expected_observed = expected_maximum.checked_add(1).ok_or("count overflow")?; + assert!( + matches!(&failure, DurableReadError::RangeRead(error) + if matches!(error.as_ref(), RangeReadError::InvalidWriteCount { layout, maximum, observed, bytes_written } + if *layout == identified.record.id() && bytes_written.is_empty() + && *maximum == expected_maximum && *observed == expected_observed)), + "exact output failure: {failure:?}" + ); + Ok(()) +} diff --git a/tests/golden_file_worldline/suite.rs b/tests/golden_file_worldline/suite.rs index ca61ecaa..7fffef31 100644 --- a/tests/golden_file_worldline/suite.rs +++ b/tests/golden_file_worldline/suite.rs @@ -15,6 +15,43 @@ mod scenario_corpus; #[path = "storage_assertions.rs"] mod storage_assertions; +#[cfg(target_os = "linux")] +#[path = "durable_assertions.rs"] +mod durable_assertions; +#[cfg(target_os = "linux")] +#[path = "durable_corruption_laws.rs"] +mod durable_corruption_laws; +#[cfg(target_os = "linux")] +#[path = "durable_diagnostic_laws.rs"] +mod durable_diagnostic_laws; +#[cfg(target_os = "linux")] +#[path = "durable_fixture.rs"] +mod durable_fixture; +#[cfg(target_os = "linux")] +#[path = "durable_layout_laws.rs"] +mod durable_layout_laws; +#[cfg(target_os = "linux")] +#[path = "durable_locator_laws.rs"] +mod durable_locator_laws; +#[cfg(target_os = "linux")] +#[path = "durable_namespace_laws.rs"] +mod durable_namespace_laws; +#[cfg(target_os = "linux")] +#[path = "durable_output_laws.rs"] +mod durable_output_laws; +#[cfg(target_os = "linux")] +#[path = "durable_range_properties.rs"] +mod durable_range_properties; +#[cfg(target_os = "linux")] +#[path = "durable_read_memory.rs"] +mod durable_read_memory; +#[cfg(target_os = "linux")] +#[path = "durable_refusal_laws.rs"] +mod durable_refusal_laws; +#[cfg(target_os = "linux")] +#[path = "../segment_filesystem_stage/sandbox.rs"] +mod durable_sandbox; + use std::error::Error; use std::io::ErrorKind; @@ -216,3 +253,11 @@ fn reader_failures_preserve_precise_boundaries_and_sources() -> TestResult { )); Ok(()) } + +#[cfg(target_os = "linux")] +#[path = "durable_writer_failures.rs"] +mod durable_writer_failures; + +#[cfg(target_os = "linux")] +#[path = "durable_closure_refusal.rs"] +mod durable_closure_refusal; diff --git a/tests/range_read_contract.rs b/tests/range_read_contract.rs index e0009bf9..29fd291e 100644 --- a/tests/range_read_contract.rs +++ b/tests/range_read_contract.rs @@ -9,7 +9,7 @@ const ARCHITECTURE_RATIONALE: &str = const FORMAT_SPEC: &str = include_str!("../docs/formats/flat-chunk-layout-v1/README.md"); const FORMAT_RATIONALE: &str = include_str!("../docs/formats/flat-chunk-layout-v1/rationale.md"); const RANGE_READ_API: &str = include_str!("../src/reference/range_read.rs"); -const RANGE_READ_RECEIPT: &str = include_str!("../src/reference/range_read_receipt.rs"); +const RANGE_READ_RECEIPT: &str = include_str!("../src/authenticated_read/range_read_receipt.rs"); const RANGE_PLAN_TEST_ENTRYPOINT: &str = include_str!("range_plan.rs"); const RANGE_READ_TEST_ENTRYPOINT: &str = include_str!("range_read.rs"); const RANGE_READ_ENTRYPOINTS_TEST: &str = include_str!("range_read_entrypoints.rs"); diff --git a/tests/reference_store_contract.rs b/tests/reference_store_contract.rs index 0873fcdf..bca9dc2d 100644 --- a/tests/reference_store_contract.rs +++ b/tests/reference_store_contract.rs @@ -10,11 +10,13 @@ const ARCHITECTURE: &str = include_str!("../docs/architecture/reference-store/RE const ARCHITECTURE_RATIONALE: &str = include_str!("../docs/architecture/reference-store/rationale.md"); const RECONSTRUCTION_ERROR_DISPLAY: &str = - include_str!("../src/reference/reconstruction_error_display.rs"); -const RECONSTRUCTION_ERROR: &str = include_str!("../src/reference/reconstruction_error.rs"); + include_str!("../src/adapters/authenticated_read/reconstruction_error_display.rs"); +const RECONSTRUCTION_ERROR: &str = + include_str!("../src/adapters/authenticated_read/reconstruction_error.rs"); const REFERENCE_INGESTION: &str = include_str!("../src/reference/ingestion.rs"); const PUBLISHED_BLOB: &str = include_str!("../src/reference/published_blob.rs"); -const RECONSTRUCTION_RECEIPT: &str = include_str!("../src/reference/reconstruction_receipt.rs"); +const RECONSTRUCTION_RECEIPT: &str = + include_str!("../src/authenticated_read/reconstruction_receipt.rs"); const STAGED_BLOB_TESTS: &str = include_str!("../src/reference/staged_blob_tests.rs"); const GOLDEN_TEST_ENTRYPOINT: &str = include_str!("golden_file_worldline.rs"); const STREAMING_TEST_ENTRYPOINT: &str = include_str!("streaming_cas.rs");