Skip to content

[Architecture]: the digest owner recognizes an envelope, 94 sites still build it by hand #5336

Description

@sakurahello1

Why this is a task with an anchor, not a new idea

tests/architecture/test_content_digest_single_owner.py owns the question "what does a stored SHA-256 look like". Its module docstring states, in its last paragraph, that one half of that decision is deliberately not owned:

Only whole-value shapes are owned. ... and producers that concatenate "sha256:" by hand are the other half of the decision and are deliberately unchanged.

That sentence is the gap. The owner (loopx/control_plane/content_digest.py, generated from its TypeScript value owner) exports only the two recognition patterns. There is no production counterpart, so every writer builds the envelope itself.

Measured gap, on 649826221

  • Hand-built envelope prefixes in loopx/: 94 sites across 81 files (Python f"sha256:{...}" and "sha256:" + ...hexdigest() forms; consumer-side removeprefix / startswith sites are excluded), plus 26 sites in 17 TypeScript files.
  • ~30 of them are private _digest / _canonical_digest / _sha256 helpers with the same body restated per module, e.g. capabilities/periodic_report/{cadence_journal,audience,machine_defaults,bindings}.py, capabilities/change_quality/shadow.py, capabilities/issue_fix/candidate_evidence.py, capabilities/material_lifecycle/intake.py, extensions/openviking_periodic_report/provider.py.
  • The guard's four layers cannot see this class at all: layer 1 scans for a restated shape (^sha256:[0-9a-f]{64}$), and a concatenation of a prefix with a hash call states no shape, so it is invisible by design. That is why "producers unchanged" was recorded rather than scanned.

Consequence today: an envelope written by one module and read by another is checked by the owner's pattern at the reader and by nothing at the writer. A writer that drops the prefix, doubles it, or uppercases the hex fails at read time in whichever surface happens to validate — the same split that the guard already proved for recognition, on the side it left open.

Exit

One producer owner for the envelope, and one directory slice migrated onto it:

  1. A single home for "build an enveloped digest", which asserts its own output against ENVELOPED_SHA256_PATTERN so production and recognition cannot drift apart.
  2. A fifth guard layer in test_content_digest_single_owner.py: a module outside that owner may not construct the prefix by hand, judged by folded value rather than spelling (concatenation, f-string, %-format, and a same-file constant), with an allowlist of the not-yet-migrated sites in the style of DEFERRED_WHOLE_VALUE_SITES — recorded, not hidden.
  3. loopx/capabilities/periodic_report migrated: all 9 of its non-census sites (cadence_journal, bindings, archive, audience, workspace, runtime_producer, machine_defaults, incremental, adapters). Each is an envelope over hashlib.sha256(<bytes>).hexdigest(); their canonicalization differs (two use ensure_ascii=True, one hashes a string rather than canonical JSON), so the shared part is exactly the envelope. pending_intent.py, post_writeback_hook.py and request_action.py are excluded here because they are census hosts whose import lines are pinned by row number in the project-registry I/O manifest; they need the manifest regenerated in place, which is its own change.
  4. Mutation evidence that the new layer is not decorative: a hand-built prefix in a non-owner module fails the scan, and the migrated helpers' digests are byte-identical before and after (the invariant that makes this safe to land).
  5. The one-line addition of the new module to the existing guard's CONSUMER_MODULES, since layer 2 pins every module that imports the owner.

Where the builder goes, and why not in the generated leaf

The obvious home is the value owner itself. It cannot take one today, for two reasons that are both in the repository already:

  • loopx/control_plane/content_digest.py is generated. scripts/generate_semantic_bindings.py parses the TypeScript owner and refuses anything except two flagless literal pattern exports ("These two flagless literal patterns are a deliberately narrow cross-runtime contract. Reject extra syntax rather than execute TS or guess its meaning"), and it also restricts the patterns to one shared regex subset. A hashing envelope is not a pattern.
  • test_the_owner_is_a_leaf_and_exports_only_the_two_shapes asserts the owner's public namespace is exactly {re, BARE_SHA256_PATTERN, ENVELOPED_SHA256_PATTERN}.

Widening the cross-runtime generator for a construct that is not a pattern, and relaxing a leaf assertion the same PR adds, is a bigger ask than this half deserves -- and hashing bytes is Python-side work anyway. So the builder is a hand-written sibling, loopx/control_plane/digest_envelope.py, which borrows the owner's own ENVELOPED_SHA256_PATTERN object and validates every value it returns against it. Production and recognition still cannot drift, because the writer checks its output with the reader's pattern rather than a restatement of it.

If you would rather have the builder inside the generated leaf, the migration is one import line per file to move back; say so and I will do that instead.

Out of scope

The other 68 Python sites and all 26 TypeScript ones stay allowlisted until a face-by-face migration, the same staging the shape half used. Files touched by open PRs #3313, #4915, #5302 and #5324 are excluded from any slice while those are in flight.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions