build: align Orchard producers and the ARMv7 recovery base

BUILD.md

@@ -75,7 +75,19 @@ signature or trust service.
 
 Use fresh output paths; producers refuse overwriting. The official Alpine OCI digest is pinned in `system/Containerfile`; APK repositories retain signature checking. The base records package versions in `/etc/alpine-packages.lock`. Its archive hash and all selected hardware hashes are recorded separately from the install contract. The APK mirror is not an immutable snapshot: future base rebuilds may select newer signed package revisions and must get new receipts.
 
-The producer composes Alpine-owned policy into one public gzip root for every platform (no kernel or modules: orchard's platform layer brings those at install, and orchard writes the boot entry), seals a recovery per platform to the root's receipt, adds orchard's layer per platform, and emits `release.toml` plus the index that pins it. Public build receipts omit local machine paths. No private firmware, SSH keys, ADT or machine IDs are embedded.
+The producer composes Alpine-owned policy into one public root archive, converts it with `orchard.image.root_image`, and takes platform layers, loaders and install tools from `orchard.image.platforms_index`. It emits `release.toml` plus the index that pins it. Public build receipts omit local machine paths. No private firmware, SSH keys, ADT or machine IDs are embedded.
+
+Builds using `--orchard-inputs` validate every selected platform against the
+ARM64 release root before root assembly. The standalone signed ARMv7 recovery
+base remains available through `armv7/build-base.py`.
+
+LwVM records require a static mapper for the selected ISA and a coherent rescue
+tool pair. This build passes the validated input document to
+`orchard.image.platforms_index --inputs`; Orchard's installed-root companion
+consumes the selected mapper and rescue paths when creating the layer. A local
+integration fixture checks the selected ARM64 board, loader and tool bytes
+through the real index and validates the resulting manifest. Physical P105
+qualification remains part of the ARMv7 device work.
 
 ## Native isolated checks
 

armv7/Containerfile

@@ -5,7 +5,21 @@ COPY --from=armkeys /etc/apk/keys /armroot/etc/apk/keys
 COPY --from=armkeys /etc/apk/keys /etc/apk/keys
 RUN apk --root /armroot --arch armv7 --initdb --no-scripts \
     --keys-dir /etc/apk/keys --repositories-file /etc/apk/repositories \
-    add busybox musl python3 dropbear nfs-utils \
+    add busybox musl python3 dropbear nfs-utils util-linux e2fsprogs \
     && mkdir -p /armroot/bin \
     && cp /etc/apk/repositories /armroot/etc/apk/repositories \
-    && cp -a /etc/apk/keys /armroot/etc/apk/keys
+    && cp -a /etc/apk/keys /armroot/etc/apk/keys \
+    && for directory in bin sbin lib; do \
+        mkdir -p "/armroot/usr/$directory"; \
+        cp -a "/armroot/$directory/." "/armroot/usr/$directory/"; \
+        rm -rf "/armroot/$directory"; \
+        ln -s "usr/$directory" "/armroot/$directory"; \
+       done \
+    && test -f /armroot/usr/bin/python3 \
+    && test -f /armroot/usr/sbin/blkid \
+    && test -f /armroot/usr/sbin/sfdisk \
+    && test -f /armroot/usr/sbin/mke2fs \
+    && test -f /armroot/etc/mke2fs.conf \
+    && test "$(readlink /armroot/lib)" = usr/lib \
+    && test "$(readlink /armroot/bin)" = usr/bin \
+    && test "$(readlink /armroot/sbin)" = usr/sbin

armv7/README.md

@@ -5,15 +5,19 @@ Wi-Fi) use an ARMv7 recovery environment. Build its Alpine package base in an
 ARM64 Podman environment:
 
 ```sh
-python3 armv7/build-base.py --output /absolute/new/alpine-base.tar
+python3 armv7/build-base.py --output /absolute/new/alpine-base.tar.gz
 ```
 
 Pass `--no-cache` to refresh the package layer. Each build uses its returned
 image ID, leaving existing Podman image tags untouched.
 
 The recipe uses pinned container images and the official ARMv7 Alpine signing
 keys. It extracts signed ARMv7 packages without running their scripts. A JSON
-receipt beside the archive records the target architecture, image identity,
-recipe hash and archive hash. The base does not contain owner firmware or
+receipt at `alpine-base.tar.gz.json` records Orchard's target architecture
+(`arm`), image identity, recipe hash and archive hash. The gzip tar uses the
+merged `/usr` layout expected by Orchard's recovery producer and includes
+native `blkid`, `sfdisk`, `mke2fs` and its configuration. Python's musl loader
+and libraries live under `/usr/lib` so the producer can stage their closure.
+The base does not contain owner firmware or
 calibration. A separate private recovery composer and the companion hardware
 and CLI contributions are required for a tethered boot.

armv7/build-base.py

@@ -1,10 +1,12 @@
 #!/usr/bin/env python3
 """Export signed Alpine ARMv7 packages for the 32-bit (apple-armv7) RAM recovery."""
 import argparse
+import gzip
 import hashlib
 import json
 from pathlib import Path
 import re
+import shutil
 import subprocess
 import tempfile
 
@@ -15,7 +17,7 @@ def main():
     parser.add_argument('--no-cache', action='store_true', help='refresh the package build without cached layers')
     args = parser.parse_args()
     output = args.output.absolute()
-    receipt = output.with_suffix('.json')
+    receipt = output.with_suffix(output.suffix + '.json')
     if output.exists() or output.is_symlink() or receipt.exists() or receipt.is_symlink():
         parser.error('Choose a fresh base output')
     output.parent.mkdir(parents=True, exist_ok=True)
@@ -32,11 +34,17 @@ def main():
         observed = subprocess.check_output(['podman', 'image', 'inspect', image, '--format', '{{.Id}}'], text=True).strip()
         if observed.removeprefix('sha256:') != image.removeprefix('sha256:'):
             parser.error('Podman image inspection differs from the build output')
-        with output.open('xb') as stream:
+        archive = Path(temporary)/'base.tar'
+        with archive.open('xb') as stream:
             subprocess.run(['podman', 'run', '--rm', '--network', 'none', image,
                             'tar', '-C', '/armroot', '-cf', '-', '.'], stdout=stream, check=True)
+        with archive.open('rb') as source, output.open('xb') as stream:
+            with gzip.GzipFile(fileobj=stream, filename='', mode='wb', mtime=0,
+                               compresslevel=1) as compressed:
+                shutil.copyfileobj(source, compressed)
         with receipt.open('x') as stream:
-            stream.write(json.dumps({'schema': 1, 'architecture': 'armv7',
+            stream.write(json.dumps({'schema': 1, 'architecture': 'arm',
+                'archive': 'tar+gzip', 'layout': 'merged-usr',
                 'signed_apk': True, 'image': image, 'bytes': output.stat().st_size,
                 'sha256': hashlib.sha256(output.read_bytes()).hexdigest(),
                 'recipe_sha256': hashlib.sha256(recipe.read_bytes()).hexdigest()}, indent=2)+'\n')

build.py

@@ -177,7 +177,9 @@ def manifests(output, version, root, offered, orchard):
     document += table('install_tools', offered['install_tools'])
     for name, layer in sorted(offered['platforms'].items()):
         document += f'\n[platforms.{name}]\nboards = {json.dumps(layer["boards"])}\n'
-        document += table(f'platforms.{name}.layer', {key: value for key, value in layer.items() if key != 'boards'})
+        document += table(f'platforms.{name}.layer', {key: layer[key] for key in
+            ('file', 'url', 'bytes', 'sha256', 'compression', 'expanded_bytes', 'members', 'kernel_release')
+            if key in layer})
     path = output / 'release.toml'
     path.write_text(document)
     (output / 'index.toml').write_text('schema = 2\nkind = "orchard.index"\ndistro = "minimal-linux"\n'
@@ -201,6 +203,12 @@ def main():
     components, inputs = verify_inputs(args.inputs)
     resolved = json.loads(args.orchard_inputs.read_text()) if args.orchard_inputs else None
     revision = resolved['orchard'] if resolved else None
+    if resolved is not None:
+        try:
+            from producer_inputs import components as producer_components
+            producer_components(resolved)
+        except ValueError as error:
+            parser.error(str(error))
     args.output.parent.mkdir(parents=True, exist_ok=True)
     with tempfile.TemporaryDirectory(prefix='.minimal-build-', dir=args.output.parent) as temp:
         output = Path(temp) / 'release'

producer_inputs.py

@@ -0,0 +1,100 @@
+"""Validate Orchard producer records against the shared Alpine release root."""
+from pathlib import Path
+import struct
+
+TOOLS = ('busybox', 'dropbear', 'dropbearkey', 'wpa_supplicant', 'wpa_cli', 'iw')
+
+
+def lwvm_helper(row):
+    """Validate an optional mapper before starting the shared ARM64 root build."""
+    root_format = row.get('root_format', 'gpt')
+    if root_format not in ('gpt', 'lwvm'):
+        raise ValueError('Unsupported platform root format')
+    value = row.get('tools', {}).get('lwvm')
+    if root_format == 'gpt':
+        if value is not None:
+            raise ValueError('A GPT platform cannot supply an LwVM mapper')
+        return None
+    if not isinstance(value, str) or not value or not Path(value).is_absolute():
+        raise ValueError('An LwVM platform needs an absolute mapper path')
+    path = Path(value)
+    if path.is_symlink() or not path.is_file():
+        raise ValueError('The LwVM mapper must be a regular file')
+    data = path.read_bytes()
+    if len(data) < 64 or data[:7] != b'\x7fELF\x02\x01\x01' or struct.unpack_from('<H', data, 18)[0] != 183:
+        raise ValueError('The LwVM mapper must match the ARM64 release root')
+    if struct.unpack_from('<H', data, 16)[0] not in (2, 3):
+        raise ValueError('The LwVM mapper must be an ELF executable')
+    offset = struct.unpack_from('<Q', data, 32)[0]
+    size, count = struct.unpack_from('<HH', data, 54)
+    if not count or size < 56 or offset < 64 or offset + size * count > len(data):
+        raise ValueError('The LwVM mapper has invalid ELF program headers')
+    entry = struct.unpack_from('<Q', data, 24)[0]
+    loadable = False
+    executable_entry = False
+    for index in range(count):
+        kind, flags, start, virtual, _, length, memory, _ = struct.unpack_from('<II6Q', data, offset + index * size)
+        if start + length > len(data):
+            raise ValueError('The LwVM mapper has a truncated ELF segment')
+        if kind == 1:
+            if memory < length:
+                raise ValueError('The LwVM mapper has an invalid loadable ELF segment')
+            loadable |= length > 0
+            executable_entry |= bool(flags & 1 and virtual <= entry < virtual + length)
+        if kind == 3:
+            raise ValueError('The LwVM mapper must be static')
+        if kind == 2:
+            if length % 16:
+                raise ValueError('The LwVM mapper has an invalid dynamic table')
+            for position in range(start, start + length, 16):
+                tag, _ = struct.unpack_from('<QQ', data, position)
+                if tag == 0:
+                    break
+                if tag == 1:
+                    raise ValueError('The LwVM mapper must be static')
+            else:
+                raise ValueError('The LwVM mapper has an unterminated dynamic table')
+    if not loadable:
+        raise ValueError('The LwVM mapper has no loadable ELF segment')
+    if not executable_entry:
+        raise ValueError('The LwVM mapper entry is outside executable file bytes')
+    return str(path)
+
+
+def validate_backend(row):
+    """Check a backend's mapper and rescue pair before handing inputs to Orchard."""
+    helper = lwvm_helper(row)
+    if helper is None:
+        return
+    tools = row['tools']
+    parents = {Path(tools[name]).parent for name in ('dropbear', 'dropbearkey')}
+    if len(parents) != 1:
+        raise ValueError('The LwVM rescue tools must share a directory')
+
+
+def components(document):
+    """Return one coherent m1n1/tool set for the existing ARM64 release."""
+    if document.get('architecture', 'arm64') != 'arm64':
+        raise ValueError('The release needs an ARM64 base; build ARMv7 recovery bases with armv7/build-base.py')
+    platforms = document.get('platforms', {})
+    if not platforms:
+        raise ValueError('Orchard inputs name no platforms')
+    shared = document.get('shared', {})
+    records = []
+    for name, row in platforms.items():
+        if row.get('architecture', 'arm64') != 'arm64' or row.get('image', 'Image.gz') != 'Image.gz':
+            raise ValueError('Platform ' + name + ' does not match the ARM64 release root')
+        loader = row.get('loader', {'kind': 'm1n1', 'file': shared.get('m1n1')})
+        if loader.get('kind') != 'm1n1' or not loader.get('file'):
+            raise ValueError('Platform ' + name + ' needs an m1n1 loader')
+        tools = row.get('tools', shared.get('tools', {}))
+        if any(not isinstance(tools.get(key), str) or not tools[key] for key in TOOLS):
+            raise ValueError('Platform ' + name + ' has incomplete static tools')
+        validate_backend(row)
+        records.append((str(Path(loader['file'])), {key: str(Path(tools[key])) for key in TOOLS}))
+    if any(record != records[0] for record in records[1:]):
+        raise ValueError('Platforms in one release need the same loader and static tools')
+    if not isinstance(shared.get('pongo'), str) or not shared['pongo']:
+        raise ValueError('The ARM64 release needs its Pongo input')
+    loader, tools = records[0]
+    return {'m1n1': loader, 'pongo': shared['pongo'], 'tools': tools}

tests/test_armv7_base.py

@@ -1,5 +1,6 @@
 """Host-only checks for the signed ARMv7 recovery base exporter."""
 import hashlib
+import gzip
 import importlib.util
 import json
 from pathlib import Path
@@ -17,7 +18,7 @@
 class Armv7Base(unittest.TestCase):
     def test_export_receipt_binds_image_recipe_and_stream(self):
         with tempfile.TemporaryDirectory() as temporary:
-            output = Path(temporary) / 'base.tar'
+            output = Path(temporary) / 'base.tar.gz'
             image = 'sha256:' + 'a' * 64
             commands = []
             def run(command, **kwargs):
@@ -30,9 +31,13 @@ def run(command, **kwargs):
                  patch.object(base.subprocess, 'run', side_effect=run), \
                  patch.object(base.subprocess, 'check_output', return_value=image + '\n'):
                 base.main()
-            receipt = json.loads(output.with_suffix('.json').read_text())
+            receipt = json.loads(output.with_suffix('.gz.json').read_text())
             self.assertEqual(receipt['image'], image)
-            self.assertEqual(receipt['architecture'], 'armv7')
+            self.assertEqual(receipt['architecture'], 'arm')
+            self.assertEqual(receipt['archive'], 'tar+gzip')
+            self.assertEqual(receipt['layout'], 'merged-usr')
+            self.assertEqual(gzip.decompress(output.read_bytes()), b'fixture base')
+            self.assertEqual(output.read_bytes()[4:8], b'\0\0\0\0')
             self.assertTrue(receipt['signed_apk'])
             self.assertEqual(receipt['sha256'], hashlib.sha256(output.read_bytes()).hexdigest())
             self.assertEqual(receipt['recipe_sha256'], hashlib.sha256((PROJECT / 'armv7/Containerfile').read_bytes()).hexdigest())
@@ -63,7 +68,7 @@ def run(command, **kwargs):
             self.assertNotIn('-t', commands[0])
             self.assertNotIn('--tag', commands[0])
             self.assertEqual(inspect.call_args.args[0][3], image)
-            self.assertEqual(json.loads(output.with_suffix('.json').read_text())['image'], image)
+            self.assertEqual(json.loads(output.with_suffix('.tar.json').read_text())['image'], image)
 
     def test_mismatched_inspection_is_rejected_before_export(self):
         with tempfile.TemporaryDirectory() as temporary:
@@ -78,12 +83,12 @@ def build(command, **_kwargs):
                     base.main()
             self.assertEqual(run.call_count, 1)
             self.assertFalse(output.exists())
-            self.assertFalse(output.with_suffix('.json').exists())
+            self.assertFalse(output.with_suffix('.tar.json').exists())
 
     def test_existing_output_or_receipt_stops_before_build(self):
         with tempfile.TemporaryDirectory() as temporary:
             output = Path(temporary) / 'base.tar'
-            for present in (output, output.with_suffix('.json')):
+            for present in (output, output.with_suffix('.tar.json')):
                 with self.subTest(present=present):
                     present.write_bytes(b'existing')
                     with patch('sys.argv', ['build-base', '--output', str(output)]), \
@@ -93,7 +98,7 @@ def test_existing_output_or_receipt_stops_before_build(self):
                     run.assert_not_called()
                     self.assertEqual(present.read_bytes(), b'existing')
                     present.unlink()
-            receipt = output.with_suffix('.json')
+            receipt = output.with_suffix('.tar.json')
             receipt.symlink_to(Path(temporary) / 'missing')
             with patch('sys.argv', ['build-base', '--output', str(output)]), \
                  patch.object(base.subprocess, 'run') as run:
@@ -110,6 +115,11 @@ def test_recipe_pins_images_and_uses_verified_armv7_packages(self):
         self.assertIn('--arch armv7', recipe)
         self.assertIn('--no-scripts', recipe)
         self.assertIn('/etc/apk/keys', recipe)
+        self.assertIn('util-linux e2fsprogs', recipe)
+        self.assertIn('for directory in bin sbin lib', recipe)
+        for name in ('blkid', 'sfdisk', 'mke2fs'):
+            self.assertIn('test -f /armroot/usr/sbin/' + name, recipe)
+        self.assertIn('test -f /armroot/etc/mke2fs.conf', recipe)
 
 
 if __name__ == '__main__':