Skip to content

P105: offline SwissPPNFTL checkpoint encoder and GC transaction planner - #93

Open
shaynelegault wants to merge 16 commits into
OrchardLinux:mainfrom
shaynelegault:p105/08-ftl-reclamation
Open

shaynelegault wants to merge 16 commits into
OrchardLinux:mainfrom
shaynelegault:p105/08-ftl-reclamation

Conversation

@shaynelegault

Copy link
Copy Markdown
Contributor

The bounded append writer eventually exhausts its blank blocks. This draft provides the checkpoint encoding and transaction planning needed to develop reclamation: an owned snapshot of the whole recovered map, native context/range serialization, and checks for relocation, checkpoint readback and cold-recovery receipts.

This stacks on the bounded append-writer draft at c63274165870417f01cc3e1264067cef71d9c0b2. The current kernel source is shaynelegault/linux:p105/08-ftl-reclamation-integration at c504eae2a214915dcf5d2c4856c52e7a6ac79d2d, with source commits c8fb01811bc801be48ff7e8863dbd74c587c921e and c504eae2a214915dcf5d2c4856c52e7a6ac79d2d. Orchard carries the two source commits as MTD patches 0009 and 0010, scoped to apple-armv7, plus chip evidence and pending P105 state. Applying those exports to the append-writer source reproduces the kernel branch exactly. The selected patch counts are 33 for ARMv7 and 450 for each ARM64 platform; the ARM64 selections contain neither topic08 patch.

Snapshots cover all 15,625,016 logical keys, including the 16 keys beyond the exported disk. The encoder preserves opaque accounting payloads, updates the decoded context-block list and emits bounded subtype-4 range fragments with native metadata. The planner checks memory/capacity bounds, reserves footer space, rejects references into inactive/footer space and compares the complete recovered map. Its supplied blank/reserved/lane receipts require authoritative VFL/context ownership proof before future destructive I/O.

Kernel media GC remains disconnected. These helpers register no erase operation or GC worker. Opaque accounting still needs native field updates before a checkpoint can be installed. The current loader rejects an incomplete newest SFTL context and requires contiguous replay; native previous-checkpoint selection and interrupted-save replay remain open. Native VFL rotation also requires both its initial checkpoint retaining old table references and its final checkpoint after copying them. Physical erase effects, SLC aliasing and retirement/replacement recovery remain unqualified. P105 storage capability and installation gates stay unchanged.

Validation:

  • Orchard contribution suite: 663 tests passed with one expected skip; catalog/matrix checks, 18 scoped-selection tests and the focused export test passed.
  • Exact integrated core/fixture C under host ASan/UBSan: 27 writer-enabled cases and 14 read-only cases passed. Eight checkpoint/planner cases passed separately, covering opaque bytes, torn/stale receipts, source sequence gaps and allocation bounds.
  • Twenty-four C metadata records matched the pinned 13G36 native formatter executed in synthetic RAM. Two hundred nine encoded maps round-tripped through the Python range decoder.
  • Strict code-content checks and non-patch whitespace checks passed. Export checks retain Orchard's required Assisted-by: LLM / no-signoff policy and preserve source commit messages; unified-diff context whitespace is checked through patch application rather than as source text.
  • Exact-source core, frontend, checkpoint/planner, both fixture and H2FMI provider objects compiled against the current ARMv7 tree with read-only and experimental write configs. W=1 and warnings-as-errors passed with the existing vmstat.h enum-arithmetic warning retained as a warning.
  • Actual ARM64 QEMU Linux KUnit passed 35/35 tests in the pinned pristine v6.12 overlay using the exact integrated C sources. The overlay supplies a validation-only H2FMI_WRITE Kconfig dependency symbol for synthetic program callbacks; it exercises no H2FMI hardware backend or controller timing.

Native format work follows IOFlashStorage-730/13G36 and the pinned Cascadia NAND research. Public files contain synthetic fixtures and format descriptions. No revision from this draft has been qualified on physical NAND.

Select ARM/v7 or ARM64 tool recipes explicitly and keep their outputs in
separate caches. Pin Debian's ARMHF BusyBox package and verify each finished
tool's ELF architecture and static linkage. Native builds require a host
whose architecture matches the selected target.

Assisted-by: LLM
Use board-owned handoff data to compile the ARM32 staging loader. Preserve
Cascadia's cache/MMU handoff and require executable code at the upload
address. A checked configuration places the supplied external initramfs
in reserved RAM before entering Linux through its edited DTB.

Bound the zImage decompressor workspace and preserve the kernel bytes.
Add synthetic layout tests and a Linux fixture that exercises the actual
loader C copy and rejection paths. Source provenance is recorded in NOTICE.
The external-initramfs handoff still requires physical qualification.

Assisted-by: LLM
Resolve one root architecture's per-platform loader and target tools through
Orchard's producer contract. Preserve explicit board selection for Cascadia
and stage ARM EABI5 hard-float Python dependency closures. Platform layers
use the declared kernel image and validate the requested root ISA.

Route RAM and recovery assembly through the declared payload format, bind
Cascadia DTBs to their selected board, and keep Pongo inputs on their own
boot path. Existing ARM64 default arguments remain supported. Document the
producer contract and exercise cache separation and architecture refusal.

Assisted-by: LLM
Carry a descriptor-selected host USB identity policy through the checked Cascadia bootargs, bind each upload to its selected hardware and a fresh attempt, and revalidate that provenance in RAM. ARM64 recovery keeps its existing ADT identity path.

Stage declared USB dependencies and defer manual storage modules until the host has serial and authenticated SSH. Reject manual drivers built into the kernel or selected as USB dependencies before assembly.

Validated with 666 unit tests (one Linux-only skip) and independent cross-layer review. Physical P105 qualification remains pending.

Assisted-by: LLM
ARMv7 and ARM64 use different kernel bases while driver patches belong in
shared subsystem directories. An adjacent patch scope lets those directories
carry both platforms without applying a patch to the wrong base.

Add schema-1 .patch.toml scopes and scoped-only series layers. Export writes
scopes and missing layers with --platform, preserving existing patch numbers.
Builds, conversion and regeneration use the same selector, and published
source keys include selected scope metadata. Refuse malformed scopes and
duplicate selection before build or export work starts.

Existing patch bytes, ordered platform selections and published source keys
are unchanged. Cover scope validation, export planning and deterministic
kernel/m1n1 regeneration with host tests.

Assisted-by: LLM
Record the measured geometry, native address views and the proposed
kernel read path. Keep the chip evidence and device capability comments
consistent with the pending exact-revision qualification.

All 34 catalog tests pass; regenerating the matrix keeps capability rows.

Assisted-by: LLM
Export the current ARMv7 H2FMI read source through kernel-tree/export.py.
Keep the existing import intact and scope the new MTD layer to apple-armv7.
The generated mail patch reproduces source fe8c2f696 exactly.

The owner approved draft exports before physical qualification. The new
source has focused ARMv7 and sanitizer results; exact-build device checks
remain pending. Public logical storage capability stays gated.

Assisted-by: LLM
Document the default-off H2FMI program primitive and its host checks.
Keep logical storage_write missing while exact-build programming and
translation-layer persistence remain unqualified. Record chip evidence
and the shared reference-profile regression requirement.

All 34 catalog tests pass. Matrix regeneration leaves capability rows
unchanged; the P105 device and A5 chip schemas validate.

Assisted-by: LLM
Export source cef7165b0 after the geometry/read dependency, preserving the
scoped MTD series. Require complete pages, blank destinations and exact
synchronous readback. The controller supplies no erase operation.

The generated patch reproduces the source commit exactly. Programming
stays disabled in the recovery candidate, and exact-revision physical
qualification remains pending under the approved draft export order.

Assisted-by: LLM
Document native VFL/SwissPPNFTL recovery, saved-capture crosschecks,
KUnit results and the latest finite provider lifetime. Keep both logical
storage capabilities missing until exact-revision device qualification.
Record the host and ARM64 test limits separately from hardware evidence.

All 34 catalog tests pass. Matrix regeneration leaves capability rows
unchanged; the P105 device and A5 chip schemas validate.

Assisted-by: LLM
Export the canonical PPN provider, native map reader and finite lifetime
policy from current ARMv7 source. Preserve the preceding geometry and
programming topics in series order, with every MTD patch scoped to ARMv7.

The official export reproduces e7d118292 after its read prerequisite.
Saved-capture crosschecks, real KUnit and ARMv7 object builds pass. The
owner approved draft export before exact device qualification, which
remains pending. Logical storage writes stay gated.

Assisted-by: LLM
Resolve the Apple ARMv7 config from kernel source commit
0921ec24248913d2a99ad1a262b2ab1e66f062e3 through kernel-tree/export.py.
CONFIG_MTD_APPLE_SFTL=m lets recovery load the read-only frontend after
its USB recovery link is ready. Keep CONFIG_MTD_APPLE_H2FMI_WRITE=n so
this config exposes no raw page-program path.

LLVM 19 olddefconfig and savedefconfig passed. The 23 focused kernel,
platform and export tests passed. Physical qualification remains pending.

Assisted-by: LLM
Describe the kernel append, summary replay and logical write policy
following native-map recovery. Keep storage_write missing and retain the
installation gate while the exact kernel revision awaits physical write
qualification. Record separate A5 append evidence and the outstanding
recovery, persistence and reclamation work.

The source passes both current ARMv7 object variants and 26 real Linux
KUnit cases. Host checks cover request updates and bounded provider
forwarding. Document their scope together with default-off policy and
NOIO request allocation. Kernel patch export remains a separate step.

Regenerate the support matrix and run all 34 catalog tests with the
checkout's absolute PYTHONPATH; both storage states stay missing.

Assisted-by: LLM
Export the optional PPN program callback and append/replay implementation
with its NOIO request safeguard. Preserve source authorship and trailers
from Linux commits ab137f3be, cb20cc76a and c63274165. The official exporter
proves that these three patches reproduce the source head from dependency
161217fd32. No platform config changes are required.

Scope all three MTD patches to apple-armv7. Its selected patch count grows
from 28 to 31; apple-4k and apple-16k each remain at 450. All 34 catalog
checks and four kernel-tree/export tests pass. The exact source also
passed both ARMv7 object variants and 26 Linux KUnit cases.

Keep P105 storage capabilities missing while the exact writer awaits
physical qualification. Update bring-up links to the exported patches.

Assisted-by: LLM
Record the normalized kernel source, software checks and native format work for the checkpoint encoder and GC transaction planner. Preserve missing storage capability and installation gates while native accounting and interrupted-save recovery remain unqualified.

Regenerated and checked the matrix; all 34 catalog tests ran successfully with one expected skip. The exact normalized source passed both ARMv7 object configurations and 35 Linux KUnit tests with synthetic media.

Assisted-by: LLM
Export the normalized source on the bounded append writer, preserving its
source authors and trailers. The two MTD patches select apple-armv7 and
reproduce the source tree when applied to the append-writer base.

The encoder and planner operate on snapshots and supplied receipts.
Physical GC remains disconnected while native accounting and interrupted
checkpoint recovery require further work.

Validation includes both current ARMv7 object variants, 35 Linux KUnit
cases, host sanitizer fixtures and native metadata comparisons. Platform
selection remains unchanged for both ARM64 configurations.

Assisted-by: LLM
@shaynelegault
shaynelegault force-pushed the p105/08-ftl-reclamation branch from e1c4d55 to 9e0fd8e Compare November 6, 2026 23:43
@shaynelegault
shaynelegault marked this pull request as ready for review November 7, 2026 05:08
henrikklee added a commit that referenced this pull request Nov 7, 2026
Add the append writes of #91 to kernel/patches/armv7/mtd: the iOS disk
of the 64 GB iPad mini 1 becomes writable. Writing is part of the driver;
the default-off MTD_APPLE_SFTL_WRITE option and the write_enable module
parameter are gone. The disk stays read-only only when the replayed
journal does not end at the allocation floor. Without reclamation (#93),
writes fail once the blank blocks are used up. The patch keeps Shayne
Legault as author and notes the changes and his original commit IDs.
The bring-up notes and the board README follow the driver as carried.

Assisted-by: LLM
henrikklee added a commit to shaynelegault/hardware that referenced this pull request Nov 7, 2026
Add the append writes of OrchardLinux#91 to kernel/patches/armv7/mtd: the iOS disk
of the 64 GB iPad mini 1 becomes writable. Writing is part of the driver;
the default-off MTD_APPLE_SFTL_WRITE option and the write_enable module
parameter are gone. The disk stays read-only only when the replayed
journal does not end at the allocation floor. Without reclamation (OrchardLinux#93),
writes fail once the blank blocks are used up. The patch keeps Shayne
Legault as author and notes the changes and his original commit IDs.
The bring-up notes and the board README follow the driver as carried.

Assisted-by: LLM
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant