diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md new file mode 100644 index 000000000..eda8268eb --- /dev/null +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -0,0 +1,267 @@ +# Design Spec: Client-Cycle Edge Cookie Providers and the Resolve Endpoint + +**Status:** **Deferred — informative draft, not part of the epic's +normative set.** No production adapter has the required CAS-class +primitive (providers spec §7 matrix), the feature has no concrete +consumer, and successive reviews keep finding open protocol questions +(§7). It re-enters the epic only through its own dedicated issue, with +this document as the starting bar — findings against this spec do not +block ratification of the core specs. +**Author:** Engineering +**Issue references:** none yet (this spec exists to force one; #778 does not +cover this feature) +**Related specs:** `2026-07-30-pluggable-providers-design.md` +**Last updated:** 2026-07-31 + +> **Context.** PR #838 shipped, undeclared and unspec'd, a second provider +> _type_: a "client-cycle" EC provider whose identifier is established by a +> browser POST to a new public endpoint (`POST /_ts/api/v1/ec/resolve`), +> plus a demo provider (`client-fixed`) and a JS bundle. Review found the +> endpoint accepted cross-origin identity-setting posts with no origin +> check, minted cookies with no identity-graph row (violating an invariant +> the organic path enforces explicitly), was registered on only one of four +> adapters, and could never round-trip because the core did not recognize +> non-HMAC identifiers. None of that is an argument the feature is a bad +> idea — vendor identity systems with a browser leg (e.g. signed-envelope +> schemes) are a real integration target. It is an argument that the feature +> needs a threat model before an implementation. This spec is that threat +> model and the bar an implementation must clear. + +--- + +## 1. Overview + +A **client-cycle** EC provider establishes the identifier via a browser +round trip: server-injected first-party JS obtains or derives a value in the +page (typically a signed envelope from a vendor identity system), posts it to +a Trusted Server endpoint, and the endpoint — after provider-specific +verification — sets the first-party `ts-ec` cookie. + +This differs from server-side providers in one security-critical way: **the +identifier is attacker-influenceable input**, not server-derived evidence. +Everything in this spec follows from that. + +## 2. Threat model + +| Threat | Vector | Consequence if unmitigated | +| -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Cross-site identity fixation** | `text/plain` POST is a CORS-simple request: any page on the web can `fetch(resolveUrl, {method: "POST", credentials: "include", body: payload})` with no preflight | An attacker pins a chosen identity onto a victim's first-party cookie jar — login-CSRF for the ad-identity layer; the victim's activity accretes to an attacker-controlled ID | +| **Replay** | A captured valid payload (from the attacker's own session or a leak) replayed against another browser | Same as fixation, without needing to mint payloads | +| **Phantom identity** | Endpoint sets the cookie without an identity-graph row | Later requests carry an EC that the KV graph has never seen; downstream sync and withdrawal logic operate on an identity that half-exists (the organic generation path explicitly refuses to write a cookie when the graph write fails, for exactly this reason) | +| **Un-tombstoneable identity** | Core does not recognize the provider's identifier shape | Withdrawal cannot expire or tombstone the identity — a compliance failure, not just a bug | +| **Amplification** | The page script cannot observe an HttpOnly cookie, so it cannot know the cookie is already set | A POST on every page view of every session (PR #838's JS gated on reading a cookie its own server marked HttpOnly, making the guard permanently false) | + +## 3. Requirements on the endpoint + +`POST /_ts/api/v1/ec/resolve` (final path TBD) MUST: + +1. **Reject cross-site requests with an exact origin check.** The request + is authorized only by **exact membership of the `Origin` header value in + the publisher's origin allowlist** — configuration that does not exist + yet and must be defined by this feature (§7, question 5) — or by a + **session-bound CSRF token**. `Sec-Fetch-Site` is **defense-in-depth + only, never an authorizing alternative**: it carries no origin value to + compare against an allowlist, and `same-site` admits every sibling + subdomain — one compromised or attacker-registered subdomain would be + enough to set identity. Requests with no `Origin` and no valid token are + rejected. +2. **Verify the payload cryptographically per provider — including against + replay.** `resolve_from_client` belongs to the client-resolve acquisition + surface that was **removed from the normative provider contract with + this feature's deferral** (providers spec §4) — it, the acquisition + enum, and the reservation schemas live only in this informative + draft, to be renormalized (against the current trait and the + delimiter-free key grammar, which superseded the `resv/…` sketch + here) when the feature gets its issue. It + accepts only payloads + that are signed by an expected party, **audience-bound** to this + publisher, and **expiring**. Audience binding and expiry alone do not + mitigate replay — a captured token installs in another browser for the + whole validity window. **Session binding is required, with no ownerless + escape hatch** (a server-issued nonce the payload must embed): + one-time consumption alone limits multiplicity but proves nothing + about _which_ browser redeems first — a captured bearer payload can + simply win the race — so it is defense-in-depth, not the mitigation. + An earlier draft allowed a "first-presenter, product-accepted" + ownerless mode; it is **removed**: risk acceptance does not make a + security invariant true, and the promised orphan cleanup was + unimplementable anyway — the server cannot observe `Set-Cookie` + acceptance, so it cannot distinguish a lost-response orphan from a + successful-but-dormant identity. A scheme that cannot embed the + session nonce cannot ship. "Single-use where the scheme allows" is + not a mitigation. +3. **Preserve the identity-graph invariant.** The cookie is set only after + the corresponding graph row is written, mirroring the organic path. Graph + unavailable → no cookie, same as organic generation. +4. **Round-trip through the lifecycle contract.** The identifier set here + must be parseable, graph-keyed, and tombstonable by the selected provider + (providers spec §3). The conformance suite runs against every + client-cycle provider. +5. **Exist on every adapter — where parity means identical behavior, + including identical refusal.** Route registration goes through shared + route wiring. On adapters whose capability matrix rows are green + (today only the dev adapter has the required CAS class — providers + spec §7), the parity suite asserts identical endpoint behavior; on + adapters without them, parity means **identical startup rejection of + the client-cycle selection** — not a proxied 404 (PR #838's failure + mode: Fastly-only registration let the Axum dev server proxy the POST + to the publisher origin), and not a silently absent route. +6. **Be uncacheable and permission-gated on the provider's full + declaration.** `Cache-Control: no-store`; before any cookie is set, + the endpoint enforces the selected provider's complete + `required_permissions()` — not a hard-coded `store-on-device` check; a + client-resolve provider declaring more than P1 gets all of it + enforced, exactly as organic minting does (providers spec §5). +7. **Bound every input — exact values, testable boundaries.** Request + body: at most **65,536 bytes** (inclusive; byte 65,537 → `413`), + enforced by a bounded read independent of `Content-Length` — a + missing, false, or chunked length must not bypass it. `Content-Type` + allowlist: `text/plain` and `application/json`, matched on the media + type alone — case-insensitively, ignoring parameters, so the browser's + default `text/plain;charset=UTF-8` passes; duplicate `Content-Type` + headers → `400`; anything else → `415`. + The resulting identifier: at most **256 bytes**, cookie-safe alphabet + (providers spec §3 global bounds); violation → `400`. Reservation key + (payload unique id): at most **128 bytes**. Status codes are part of + the contract: `400` malformed payload/identifier, `403` origin/token + rejection, `409` different-identity or revoked-family conflict (§3.8), + `413` body, `415` content type. Tests exercise each boundary at its + exact edge, including the missing/false/chunked-length cases. +8. **Define behavior against an existing identity — no silent + replacement.** When the request already carries a recognized EC: + resolving to the **same** identity is an idempotent no-op (cookie + refreshed, same response); resolving to a **different** identity is + **rejected** — replacing a live identity via an unauthenticated POST is + identity takeover, and any legitimate re-identification flow (account + link, vendor migration) is an explicit linking design this spec does + not authorize (own open question, §7). +9. **Make replay consumption and graph persistence one idempotent + sequence — without letting idempotency reinstall the identity + elsewhere.** Neither naive order works: consume-the-nonce-first makes a + subsequent graph failure unretryable (the token is spent, the identity + never existed); graph-first lets the losers of a replay race leave + residual rows. Required shape: consumption is an **atomic single-key + reservation** (CAS — an adapter capability the composition root checks, + providers spec §7) keyed by the payload's unique id, with explicit + states: `pending` → `committed` | `failed`, each carrying an **owner + hash** (the session binding) and a **monotonic lease epoch**. Takeover + of an expired `pending` lease increments the epoch, and every state + transition is a fenced CAS on (state, epoch) — a stale owner resuming + after its lease expired cannot commit over the takeover's work, because + its epoch no longer matches. `failed` is retryable: the same owner may + supersede it with a fresh `pending` at a higher epoch. The graph write + happens under the reservation and is deterministic under its key, so + any retry converges on the same row; reservations are retained at + least through the token's expiry. Reservation keys are **namespaced** + per the providers spec §6.3 grammar + (`resv////`), so + payloads cannot collide across publishers, providers, or versions. + Ownership conflicts are terminal per state: a non-owner hitting + `pending` gets `409` (retry only after lease expiry); a non-owner + hitting `committed`/`failed` gets the no-cookie terminal response; an + owner hitting `failed` may supersede it (higher epoch); cleanup + deletes reservations after retention, never before token expiry. + + **A duplicate must never receive the cookie unless it proves the + original session binding.** "Duplicates observe the recorded outcome" + cannot mean replaying `Set-Cookie` — that would hand a captured token's + identity to a second browser, recreating the fixation §2 exists to + prevent. A requester matching the reservation's owner hash **does** + have the `Set-Cookie` re-emitted — which is precisely how a legitimate + browser whose original response was lost recovers on retry, so a + committed graph row never strands as an orphan for the intended + browser; anyone else gets a terminal response with no cookie. The **same-identity no-op of §3.8 first checks the family revocation + record**, and "revocation wins" is enforced by an explicit + linearization point, not by read strength alone — a linearizable read + followed by a separate commit still loses the race (read "not + revoked" → withdrawal commits → resolve emits a cookie for a revoked + family). The family record carries an **epoch** (providers spec + §6.3), bumped by every revocation-state change; the resolve reads + epoch _e_ at the start, and its cookie-emitting commit is a **CAS in + the strong class conditioned on the family epoch still being _e_**. + A withdrawal landing between read and commit bumps the epoch, the + commit fails, and the resolve is rejected — the CAS is the + linearization point. A resolve against an already-revoked family is + rejected, never refreshed. Tests cover crash-between-steps, lease + takeover with a stale-epoch commit attempt, two concurrent requests + with the same payload, a duplicate from a second client receiving no + cookie, owner-hash recovery receiving the cookie, and + resolve-vs-revocation races. + +## 4. Requirements on the page script + +**The page leg is permission-gated before it executes.** The browser +module obtains or derives a vendor identity — vendor contact, stable +identifier in hand — so injecting it whenever the provider is merely +_selected_ would run identity code for a visitor who denied everything, +with only the later POST refused. The module is injected/activated only +when the request's resolved permissions already satisfy the provider's +complete `required_permissions()`, and the page leg is a listed row in +the permission spec's §7 enforcement inventory (deferred alongside this +feature). Injection-time gating alone has a TOCTOU gap: consent can be +withdrawn between document delivery and asynchronous vendor contact, or +the document can be restored from BFCache long after its permissions +were resolved — so the module must additionally perform a **live +CMP/permission check immediately before vendor contact** and abort on +permission change or BFCache restoration; endpoint rejection is too late +to undo browser-side identity derivation or vendor egress. + +- The re-post guard must not depend on reading an HttpOnly cookie. Either + the server injects a "resolved" marker the script _can_ read (a + non-identity companion cookie or an injected page variable), or the + endpoint is cheap-idempotent and rate-limited per session; the design must + state which and test it. +- The JS module ships through the standard integration bundle mechanism, + loaded only when a client-cycle provider is the selected EC provider. Note + the consequence: bundle content becomes a function of EC configuration, + which interacts with the bundle's content-hash/SRI pinning and caching — + the mechanism today keys off the integration registry, not EC provider + selection (open question, §7). +- Any constant shared between Rust and TS (endpoint path, marker name) is + asserted equal by a test, not "kept in sync by hand". + +## 5. Demo providers + +A demonstration provider (fixed identifier, no verification) fails §3.2 by +design and therefore MUST NOT be selectable in a production build: gate it +behind a cargo feature or `#[cfg(test)]` so the settings validator does not +accept its key in release artifacts. PR #838's `client-fixed` was selectable +in any production config, giving every visitor the same identity, with a doc +sentence as the only guardrail. + +## 6. Testing + +- Endpoint: origin-rejection (including `Sec-Fetch-Site`-only requests, + which must fail), expired/replayed/foreign-audience payload rejection, + graph-unavailable refusal, permission-gate refusal, and the §3.7 body / + content-type / identifier limits at their exact boundaries — each as an + integration test, not only unit tests. +- Browser round trip (JS → POST → Set-Cookie → next request recognized) in + the integration suite; PR #838 shipped the JS with in-process unit tests + only, including one asserting a state (reading the HttpOnly cookie) that + cannot occur in a real browser. +- Parity: all four adapters. + +## 7. Open questions — to be settled in the feature's issue before any code + +0. The commit path spans keys (reservation, identity row, family-epoch + CAS): the cross-key atomicity or saga/compensation design is + **undefined** — the single-key CAS steps are specified, their + composition is not. + +1. Which concrete vendor scheme is the first real consumer, and does its + envelope format satisfy §3.2 (audience binding, expiry)? If no concrete + consumer exists, the feature waits — the demo provider is not a + consumer. +2. Does the resolve flow need consent-state echo in its response (so the + page can react), and if so what is the minimal disclosure? +3. Rate limiting / abuse posture at the edge for an unauthenticated POST. +4. Whether the endpoint should be versioned separately from the identify + API family it sits beside. +5. The shape of the publisher **origin allowlist** that §3.1 validates + against — no such configuration exists today (`publisher.origin_url` is + a single upstream and the cookie domain is a cookie scope, not an origin + set), so it is new settings surface this feature must define. +6. How JS module selection keyed off EC provider configuration coexists + with content-hashed/SRI-pinned bundles (§4) — per-config hashes, cache + keying, and the config-push story for them. diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md new file mode 100644 index 000000000..97dd5d484 --- /dev/null +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -0,0 +1,324 @@ +# Design Spec: Integration Response-Header Hook + +**Status:** Draft +**Author:** Engineering +**Issue references:** #782 +**Related specs:** `2026-07-30-pluggable-providers-design.md` +**Last updated:** 2026-07-31 + +> **Context.** Issue #782 already specifies this feature well; its done-when +> is the contract. PR #838 shipped the trait and registry wiring with **no +> adapter call site** — `apply_response_headers` had zero production +> callers, so the feature existed only in its own unit test. This short spec +> restates the contract plus the two details the issue left open (ordering +> and collision policy), and adds the rule that prevents a repeat: the hook +> lands with a consumer or not at all. + +--- + +## 1. Overview + +Integrations can today rewrite request-path behavior (proxies, attribute +rewriters, head injectors) but cannot mutate **response** headers. The hook +adds that: an integration registers a response-header mutator via its +`IntegrationRegistration` builder, and every adapter applies all registered +mutators to the outbound response for HTML document responses it processed. + +## 2. Contract + +- `IntegrationRegistration::builder(ID).with_response_mutator(...)` registers + a mutator; `IntegrationRegistry::apply_response_headers(...)` applies all + registered mutators in registration order. +- **The mutator API is structured operations, not header-map access.** A + mutator returns (or is handed a recorder for) typed operations — + `append(name, value)` and `replace(name, value)` (v1 is headers-only; + the cookie operation arrives with the deferred cookie surface, §3) — + which **core validates and applies**, + attributing each to its integration id. PR #838's shape handed the + integration an unrestricted `&mut HeaderMap`, which makes §3's collision + policy unenforceable by construction: core cannot validate or attribute + writes it never sees. An API that cannot express a violation beats one + that promises to catch it. +- **Every adapter calls the apply point** on its outbound-response path for + processed documents. The call site lives in shared response-finalization + code where one exists; where adapters finalize independently, each adapter + gains the call and a test proving it. +- **Ordering is three stages, and the last one is inviolable:** core + response-header handling (EC Set-Cookie emission, EC header clearing, + privacy headers) → integration operations → **final cache/privacy + invariant enforcement**, which no integration operation can override. + Running the hook dead-last would be wrong: current `main` deliberately + runs cookie-cache protection _after_ arbitrary header changes, stripping + surrogate caching and forcing private/no-store on any response that sets + a cookie — a hook applied after that recheck could combine an appended + `Set-Cookie` with a replaced public `Cache-Control` into a + **shared-cacheable cookie response**. The invariant pass therefore runs + after all mutations, unconditionally — and it enforces more than the + cookie rule. Core **snapshots the complete pre-hook cache restriction + state** — whether the restriction came from core's own classification + (processed auction HTML is marked private even when no cookie is + emitted; today's final helper returns early without `Set-Cookie`) **or + from the origin** (an origin-supplied `private, no-store` that core + merely passed through) — and the post-hook response may only be + **equal or stronger** on the privacy axis: integrations can tighten + caching, never loosen it, regardless of which header they replaced. + "Equal or stronger" is a defined merge over **independent sticky + directives, not a totally ordered lattice** — `no-cache` and `private` + are orthogonal constraints (RFC 9111: `no-cache` permits shared + storage subject to revalidation; `private` forbids shared storage), so + "replace `private` with the stronger `no-cache`" would make a + personalized response shared-storable. The merge: each of `no-store`, + `no-cache`, `private`, `must-revalidate`, `proxy-revalidate`, + `must-understand`, and `no-transform` is **sticky** — present in the snapshot or the + mutation ⇒ present in the final response, independently; `public` is + dropped whenever any restriction is present; `max-age`/`s-maxage` may + only shrink relative to the snapshot; `stale-while-revalidate`/ + `stale-if-error` may appear only if the snapshot had them **and their + durations may only shrink** (present-at-1s must not become + present-at-1y); CDN-specific cache fields are **reserved outright, by enumerated + name in the field registry** — `Surrogate-Control`, + `CDN-Cache-Control`, `Cloudflare-CDN-Cache-Control`, and + `Edge-Control`, each individually tested ("host equivalents" was not + a matching rule four adapters would implement identically) — + merging them per-directive on unrestricted responses was a hole (an + unrestricted `CDN-Cache-Control: max-age=60` could become a year), and + they are additionally stripped from any restricted response; and the + final `Vary` is the **union of the complete snapshot `Vary` set** — + origin-supplied members included, not only core-required ones — and + the mutation. Middle-stage placement also keeps + the earlier property: an integration mutation is not silently stripped + by ordinary core handling — only by the invariant pass, which logs the + downgrade it applies. + +## 3. Collision policy + +- Mutators may not touch **reserved surface**, which is defined at two + granularities because `Set-Cookie` is multi-valued: (a) reserved header + _names_ — HTTP framing and hop-by-hop headers (`Content-Length`, + `Transfer-Encoding`, `Connection`, `Trailer`, `Upgrade`, `TE`, + `Keep-Alive`), **freshness metadata** (`Age`, `Date`, `Expires` — replacing `Age: 59` + with `Age: 0` on a cached `max-age=60` response, or pushing `Expires` + into the future, extends downstream freshness in exactly the way the + monotonic merge forbids for `max-age`, so these are reserved outright + rather than merged), **representation headers coupled to body bytes + the hook cannot see** (`Content-Encoding`, `Content-Range`, + `Content-Type`, `ETag`, `Last-Modified`, `Accept-Ranges`, and digest + headers — + relabeling uncompressed bytes as Brotli, or advertising a validator or + digest for bytes the hook never saw, corrupts responses or poisons + caches), the `x-ts-*` namespace, and the + consent/privacy headers core emits; (b) reserved cookie _names_ within `Set-Cookie` — `ts-ec`, + `ts-eids`, and the other `ts-*` cookies core owns. **Cookie operations are deferred out of the v1 hook — headers only.** + The write-side gate alone ("persistent cookies require P1") was shown + insufficient: it never modeled reading, using, forwarding, or + withdrawing the cookie — a P1-granted-then-withdrawn integration + cookie would keep arriving on every request with nothing required to + expire, hide, or stop egressing it, and an advertising-identifier + cookie needs P4 the contract never expressed. Rather than ship + "inside the permission model" as a claim the model does not back, + `append_set_cookie` and the typed cookie builder are **deferred** to a + follow-up spec whose entry bar is: declared per-cookie required + permissions, a typed authorized request-side view, stripping from + unauthorized integration/proxy inputs, mandatory expiry on destructive + P1 withdrawal, and startup-unique (name, domain, path) ownership. + Integration IDs are **startup-unique, enforced**: registry + construction rejects a duplicate ID (current code silently coalesces, + which corrupts attribution and budgets), with a duplicate-ID test in + the done-when. Until then the + operation set is headers-only, and `Set-Cookie` is fully reserved. + reserved cookie name — in v1 that is every cookie name, since + `Set-Cookie` is fully reserved (§3 deferral). Violations are rejected + at the operation layer (§2) and + logged at `warn` with the integration id. The reserved lists are single + constants next to the definitions they protect, not duplicated in the + hook. +- For non-reserved headers, the mutator API distinguishes **append** from + **replace** explicitly; append/replace legality comes from a **core-owned field registry**, + not adapter judgment — and the v1 registry admits **inert fields + only**: "headers-only" is not automatically permission-neutral, since + `Link` (preload/prefetch), `Reporting-Endpoints`/NEL, CSP report + directives, and `Refresh` cause browser-initiated vendor contact on + requests that granted nothing. Fields with active egress side effects + are **rejected in v1**; a follow-up may admit them behind declared + required permissions gated at mutation time. Within the inert set, + each field is classified append-legal (genuinely list-valued), + replace-only (true singletons — e.g. `Content-Location`, `Retry-After`; + an earlier draft miscited `Content-Language`, which is list-valued), + or rejected; **unknown extension fields are rejected entirely in v1** + (neither append nor replace — their side-effect class is unknowable) (`Set-Cookie` is fully reserved in v1 — neither append nor replace). Replacing a + header the origin set is a deliberate act, visible in the mutator's code. +- Later registrations see earlier mutations (order = registration order, + which is deterministic). +- **Operation-layer hygiene:** generic `append`/`replace` reject the + `Set-Cookie` header name outright — cookies go only through + the deferred cookie builder (when it exists), so cookie validation + cannot be bypassed by spelling + the header name in a generic op. Per-integration limits bound total + operations (≤ 32), added headers (≤ 16), and added bytes (≤ 8 KiB), and + a **cumulative final-response budget** (≤ 128 headers / ≤ 32 KiB total, + counting `name: value` plus separators, within any lower adapter + ceiling) bounds the sum across integrations — enforced in registration + order, so which operations are rejected when a budget trips is + deterministic. Each mutator receives an **immutable, redacted snapshot of the + response head** (status and headers as of its turn, prior integrations' + accepted operations applied) as its read context; it never holds a + mutable reference (§2). Redaction is a security boundary, not + tidiness: the hook runs after core queues the EC `Set-Cookie`, so an + unredacted view would hand a mutator the raw EC to copy into + `X-Vendor-Identity` or its own cookie — walking around + `AuthorizedIdentity` entirely. The snapshot therefore + **excludes every `Set-Cookie` value and every reserved identity, + consent, and privacy header value** (names may be listed as present; + values are withheld). + Operations arrive as **attributed batches bound to a registration + ID** — one batch per integration per response, ordered by + registration, with the security channel's batch (§4a) ordered **after** + ordinary response mutators — one global order, core finalization → + ordinary mutators → security effects → invariant pass — so the + security layer's precedence over publisher-facing mutations holds + without a second ordering claim; the current flat effects vector satisfies neither + attribution nor budgets and is restructured accordingly. Validation + and budgeting are **atomic per batch**: a batch that exceeds its + budget is rejected whole (logged, attributed), never partially + applied — item-by-item rejection could apply a security 302's + `Set-Cookie` while dropping its `Location`. The response itself is + never rejected. A mutator that returns an error is skipped in full — its + operations are all-or-nothing — and the response proceeds without it. + **Panics are forbidden and fatal, not recoverable**: the primary target + (`wasm32-wasip1`) builds with `panic = "abort"`, so there is no unwind + boundary to catch at — a spec that promised panic recovery would be + unimplementable there. Mutators are infallible-by-construction or + return `Result`; a panic is a bug that takes the instance down, same as + anywhere else in the request path. + +## 3a. Response eligibility — normative + +Which responses the hook runs on, enumerated so two implementations cannot +diverge silently: + +| Response | Hook runs? | +| ------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Processed HTML document (rewritten by TS) | Yes | +| Streamed processed document | Yes — operations apply to the header block before first byte | +| Pass-through proxy response (not processed) | No — TS is a transparent proxy for it | +| Served-from-cache processed document | Yes, applied at serve time (mutations are not cached) | +| Redirect (3xx) | No | +| Error responses TS itself generates (4xx/5xx) | No | +| `304 Not Modified` for a processed representation | **304-safe metadata pass**: the hook's header mutations for the corresponding processed 200 are re-applied (a 304 updates stored `Cache-Control`/`Vary` — excluding it while running on HEAD contradicted the cache-metadata rationale); where mutations cannot be reproduced, respond 200 instead | +| `HEAD` of a processed document | **Yes — header parity with GET is mandatory** (a cache may refresh stored GET metadata from HEAD, and divergent CSP/privacy metadata between the two is a leak) | +| Informational `1xx`, `204`, `205`, `206` | No — enumerated so adapters do not infer independently | + +This deliberately narrows #782's general "outbound response" phrasing to +processed documents (§6). + +## 4a. The security channel — normative closed boundary + +The security channel (today: DataDome) is not a general exception; every +degree of freedom is closed: + +- **Typed security-cookie operation with a concrete lifecycle, not + header strings.** The channel emits cookies only through a typed + operation, and the registration is not a placeholder — for DataDome + it pins: cookie name exactly `datadome`; scope the publisher apex, + path `/`; mandatory `Secure` and `SameSite=Lax`; lifetime at most + DataDome's documented maximum (thirteen months ceiling); size ≤ 4 KiB; + a violating operation is rejected whole (the batch rule). Every + `ts-*` name is rejected. **Read is owner-only** — the cookie is + visible to the security channel and stripped from every other + integration's request view; vendor egress goes only to DataDome + endpoints; deletion is always possible; and whether TS's own + destructive withdrawal also expires it is exactly the open half of + **sign-off item 23** — the carve-out is _pending ratification_, not + ratified, and the permission inventory's cookie deferral stands until + it closes. No other request filter inherits the cookie capability. +- **Request-header pointers are a positive, enumerated allowlist.** + "Documented enrichment headers" is not enforceable; the registration + enumerates the exact names — for DataDome today that is + **`X-DataDome-ClientID` and the documented `X-DataDome-*` enrichment + set, listed one by one** — resolving what was a contradiction: + ClientID propagation is required by the existing DataDome contract + and test, and its identity-class nature is precisely why it applies + only to an **owner-scoped publisher-upstream overlay**, never the + shared request that later integrations read, with its vendor egress + ratified under sign-off 23. Everything else — authentication, + `Cookie`, `Forwarded`/`X-Forwarded-*`, other identity, consent, and + routing-authority fields — is rejected by name and by class: a + compromised endpoint must not replace origin credentials, inject + `ts-ec`, or spoof client location. +- **Representation rules are decision-scoped.** A _Respond_ decision + (challenge/deny) **owns its body** and may set representation headers + (`Content-Type`, encoding, validators) for it — the hook's + representation reservation exists because ordinary mutators do not own + the body, and this one does. A _Continue_ decision may not touch + representation metadata of publisher bytes. +- **One global order:** core finalization → ordinary mutators → + security effects → **final cache/privacy invariant pass, + unconditionally last**. Security precedence over publisher-facing + mutations comes from its position, not a "wins" rule; nothing outranks + the invariant pass, or a challenge could combine `Set-Cookie` with + public caching. The older DataDome spec's "applies last, after + finalization" wording is **superseded by this order** — updating that + document is a done-when item, since as written it would place DataDome + after the invariant pass and reopen the public-cache-plus-cookie bug. +- The channel adopts the shared layers: structured attributed batches + (§3, atomic per batch — a 302 must never lose `Location` to a budget + while keeping its cookie), reserved header names, budgets, and the + invariant pass — with one sequencing rule fail-open depends on: the + complete challenge batch is **validated and budgeted before the + Respond decision commits**, so a rejection converts to Continue while + the publisher route is still available; discovering the rejection + after Respond has short-circuited routing would leave nothing to fail + open _to_. + +## 4. Done-when (from #782, sharpened) + +1. Trait + builder + registry application, each public item documented. +2. **The pre-existing `RequestFilterEffects.response_headers` channel + remains a distinct, core-owned security channel — §4a defines its + closed boundary.** Folding it into this hook would break its one real + consumer: DataDome sets headers **and cookies** on 200, 301/302, 401, + 403, and 429 responses — response classes (§3a) this hook never runs + on, with cookie emission v1 reserves. +3. **At least one real consumer ships in the same PR** — an existing + integration registering a mutator for a real need (or, failing a real + need, the feature waits; scaffolding with only self-referential tests is + dead code and will be removed). +4. Every adapter applies mutations on its outbound path, with a per-adapter + route test asserting an integration-set header appears in the response. +5. A parity-suite case asserts identical mutation behavior across adapters. +6. Reserved-surface, append/replace, operation-limit, and erroring-mutator + semantics covered by unit tests. +7. **Every row of the §3a eligibility matrix has a test** — streaming, + cache-hit, pass-through, redirect, error, and 304 each proven to run or + not run the hook — not merely one positive header test per adapter. +8. Cache/privacy invariant tests, one per restriction source and shape: + a **core-owned** cookie already queued before the hook + an + integration's public `Cache-Control` replacement → private/no-store, + surrogate stripped; **core-private cookieless** processed HTML + + public replacement → restriction preserved; **origin-private cookieless** processed HTML that retained the + origin's cache restrictions + public replacement → restriction + preserved (pass-through responses never run the hook, §3a); a cache-hit serve re-applying mutations without + weakening the stored classification; a `Vary` mutation neither + dropping core-required values nor bypassing the snapshot; each CDN + directive (`Surrogate-Control`, `CDN-Cache-Control`, host equivalents) + individually stripped; and a rejected `Content-Encoding` mutation. + +## 5. Size and sequencing + +This is a modest feature plus tests with zero coupling to the provider +architecture or, in its v1 headers-only form (§3), to the permission +model. It lands whenever its first real consumer is identified (§4.2); +cookie operations arrive only with their own follow-up spec (§3) and its +permission-model coupling. If no consumer +materializes, it does not land; being unblocked is not a reason to ship +scaffolding. + +## 6. Divergences from issue #782 + +This spec supersedes #782 on the following points; the issue is updated to +reference this spec when the PR merges: + +| #782 says | This spec says | Why | +| -------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | +| Mutations apply to the outbound response generally | Eligibility is the explicit §3a matrix, centered on processed documents | Pass-through/error/304 mutation has different semantics per adapter; enumerating beats implying | +| Ship the trait + registry + adapter application | Additionally: structured operations API (§2), reserved surface (§3), and a real consumer in the same PR (§4.2) | PR #838 shipped the trait with zero call sites; an unrestricted `&mut HeaderMap` cannot enforce any collision policy | diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md new file mode 100644 index 000000000..f049b38c6 --- /dev/null +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -0,0 +1,943 @@ +# Design Spec: Jurisdiction Permission Model + +**Status:** Draft +**Author:** Engineering +**Issue references:** #779 +**Related specs:** `2026-07-30-pluggable-providers-design.md`, +`2026-07-30-provider-migration-rollout-design.md` +**Last updated:** 2026-07-31 + +> **Context.** PR #838 proposed a permission model whose review surfaced two +> classes of defect this spec exists to prevent in the next pass: (1) silent +> behavioral inversions of consent-signal precedence — most seriously, a +> present TCF string short-circuiting GPC/GPP/US-Privacy opt-outs — and +> (2) fail-open jurisdiction resolution when geolocation is disabled. The +> precedence rules (§4) and the failure-mode matrix (§6) are the two +> documents whose absence allowed those defects to hide in a 67-file diff. +> They are normative: an implementation whose behavior differs from these +> tables is wrong, whatever its tests say. This spec also reverses one +> PR #838 structural decision: policy lives in `trusted-server.toml`, not in +> a build-time-embedded YAML file (§3.1). + +--- + +## 1. Overview + +The permission model replaces the hard-wired jurisdiction gate +(`allows_ec_creation` and its companions) with a single resolved +**permission set** per request. Every data decision Trusted Server itself +makes — EC provider execution, EC creation and withdrawal, and EID +transmission into the bidstream — reads that set (§7). + +The set is resolved from three inputs: + +1. **Jurisdiction** — the country/region the request resolves to (§5). +2. **Policy** — a declarative map from jurisdiction to a baseline + acquisition rule per permission (§3). +3. **Signals** — the request's privacy signals: TCF, GPP, GPC, US Privacy + (§4). + +These are the initial sources. #777/#779 also envision publisher +interaction and external services as permission sources; that +source-interface is **explicitly deferred**, not silently dropped — §10 +records the divergence, and adding a source later means adding a grant- or +opt-out-class input to §4's taxonomy, not a new resolution algorithm. + +Scope: the model governs decisions Trusted Server makes. Downstream RTB +partners receive the full, unmodified regulatory context and make their own +compliance decisions. + +## 2. Vocabulary: enforced permissions only + +Permissions are named by IAB TCF Europe purpose identifiers, used strictly as +technical identifiers (no CMP or TCF policy is implemented by naming them). + +**Rule: a purpose appears in the model only when it has both a signal mapping +and an enforcement point.** PR #838 shipped 11 purposes of which 9 were +inert — computed into the bitset and consumed by nothing but a startup log — +while the policy file invited operators to set flags (e.g. +`market-research: denied`) that changed nothing. A policy vocabulary that +overstates what is enforced is a compliance hazard, not forward +compatibility. + +The initial vocabulary is therefore exactly: + +| Identifier | TCF purpose | Enforcement points | +| ------------------------- | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `store-on-device` | 1 | EC provider execution; EC creation; input to the §4.2 withdrawal decision (revocation itself is never permission-gated, §7) | +| `select-personalised-ads` | 4 | All bidstream and partner identity egress — raw EC in `user.id`, derived request IDs, page bids, EIDs, identify, pull/batch sync (jointly with `store-on-device`; the full path table is §7) | + +(The identifier strings are the IAB names verbatim, including their original +spelling.) The extension procedure — add the signal mapping, add the +enforcement point, add the policy vocabulary entry, in one change — is +documented alongside the policy schema. Policy validation **rejects** a rule +that references an identifier outside the current vocabulary, so a policy can +never promise more than the code enforces. + +## 3. Policy + +### 3.1 Location: `[permissions]` in `trusted-server.toml` + +Policy is operator-owned runtime configuration, expressed as a +`[permissions]` section of `trusted-server.toml`, flowing through the same +pipeline as every other setting (`ts config push` publishes it as part of the +config blob envelope; instances pick it up like any config change). + +This deliberately reverses PR #838, which embedded a `permissions.yaml` at +build time via `include_str!`. That design was rejected because: + +- a policy edit — the operation the whole model exists to make easy — + required recompiling and redeploying the binary, cutting against the + runtime config-store pipeline the project has standardized on; +- it introduced a second configuration language and a second validation + path next to the TOML settings machinery that already exists; +- the `include_str!` reached two directory levels above the crate root, + breaking crate packaging; +- validation ran lazily at first use behind a `OnceLock` + `expect`, so a + bad edit that escaped unit tests became a 500 on every request. + +Auditability is preserved where it actually lives: the source-controlled +`trusted-server.example.toml` ships the complete recommended policy table +(the reviewable reference artifact), and the operator's own config history — +git for the file, config-store versions for pushes — is the change log. + +**Compiled-in fallback:** when a config has no `[permissions]` section, a +minimal compiled-in policy applies in which **every permission is +`requires_signal` for every jurisdiction**, with **`regime = "gdpr"`** so +auction dispatch (§7) is defined and maximally protective too. (This is +the most protective posture that still admits consent — `denied` would be +stricter but would make a signal-carrying deployment inoperable by +default; the distinction is stated, not glossed.) Absence of policy is +always safe; there is no fail-open default. + +### 3.2 Format + +Named **groups** (baselines) and **rules** mapping a country (`FR`) or +country/state pair (`"US/CA"`) to a group, with optional per-permission +overrides. Each permission resolves to an **acquisition rule**: + +- `granted` — set without any signal, +- `requires_signal` — set only when a signal grants it (opt-in), +- `denied` — never set, even when a signal grants it. + +```toml +# Illustrative schema example — NOT the shipped policy. The shipped +# example (trusted-server.example.toml) pairs these groups with the most +# protective rules.default; the permissive default below demonstrates the +# reserved key. +[permissions.groups.gdpr-eu] +regime = "gdpr" +default = "requires_signal" + +# Opt-out regime, expressed as requires_signal: explicit non-opt-out +# values are grant-class signals (§4), so signal-carrying traffic is +# granted while no-signal traffic stays blocked — matching today's US +# behavior, which `granted` cannot express. +[permissions.groups.us-opt-out] +regime = "us-privacy" +default = "requires_signal" + +[permissions.groups.non-regulated] +regime = "none" +default = "granted" + +[permissions.rules] +FR = "gdpr-eu" +# US privacy gating applies per configured privacy state, matching today's +# state-list behavior; country-level US traffic (a Wyoming request, or one +# whose geo provider yields no region) stays non-regulated. One US/ +# rule per configured privacy state: +"US/CA" = "us-opt-out" +US = "non-regulated" +# Overrides name explicit acquisition rules — no +/- sigil syntax; TOML +# expresses the target state directly. +"US/CO" = { group = "us-opt-out", overrides = { select-personalised-ads = "requires_signal" } } +# Reserved key: countries that resolve but match no rule. Required whenever +# the [permissions] section is present. Distinct from [geo] default_country, +# which handles requests that resolve no country at all (§5.4). +default = "non-regulated" +``` + +A group's `default` covers unlisted permissions; a group may also name +permissions explicitly. Overrides map identifier → acquisition rule, so any +target state (including `requires_signal`) is expressible — PR #838's +`+`/`-` sigil scheme could not express "requires a signal", the most common +real-world override. + +Each group carries a required **`regime`** class (`gdpr`, `us-privacy`, or +`none`). This is the explicit legal-classification channel: consumers that +need a jurisdiction _class_ — above all server-side auction dispatch — read +`regime`, never infer a class from purpose flags. Inference is lossy +(Purpose 1 and Purpose 4 may legitimately carry different rules, and a +non-GDPR operator may choose an opt-in Purpose 4) and would smuggle legal +meaning back into identifiers this spec declares purely technical (§2). + +### 3.3 Validation — at config acceptance, not request time + +Policy is validated where every other setting is: at `ts config push` (a bad +policy is rejected before publication) and at settings construction on +startup (a bad stored config produces the startup-error state, never a +per-request failure). + +Validation rejects: + +- unknown fields anywhere (`deny_unknown_fields` on every deserialized + struct — PR #838's untagged rule type silently swallowed a misspelled + override key, dropping the operator's override with no diagnostic); +- rule keys whose country part is not in the embedded **assigned** ISO + 3166-1 alpha-2 list (not merely `[A-Z]{2}` — an unassigned code is + almost certainly a typo silently diverting a country to the fallback); + the region part must be an assigned ISO 3166-2 subdivision of that country (not merely a shape check — `US/ZZ` would parse but can never match a request), unless the selected geo provider declares its own region vocabulary **together with a canonical mapping to ISO subdivisions** — §4.5 applicability and policy rule keys operate on canonical `US/CA`-form keys, so a provider emitting anything else must declare the translation, validated at startup. The `US/CA` slash form is the + house rule-key format corresponding to ISO 3166-2 `US-CA`; +- references to permissions outside the enforced vocabulary (§2); +- references to undefined groups, and groups missing the `regime` class; +- groups that neither list every permission nor provide `default`; +- a present `[permissions]` section without a `rules.default` entry (§5.4 + depends on it existing — its absence must be a validation error, not a + runtime surprise); +- an empty `[permissions]` section (ambiguous intent: an operator who wants + the compiled-in fallback omits the section entirely); +- duplicate rule keys under case-insensitive comparison (`FR` and `fr`); +- a `[geo] default_country` whose country part is not an assigned ISO + code; it accepts either a country (`FR`) or a country/region key whose + region part is validated as an assigned subdivision exactly like rule + keys (`US/ZZ` is rejected here too) + (`US/CA`) — PR #838 supported region defaults, and a no-geo, + single-state deployment must be able to select its state rule. It is + canonicalized to uppercase, and startup logs which rule (or + `rules.default`) it resolves to. + +### 3.4 One source of jurisdiction truth + +Today, `detect_jurisdiction` — driven by the runtime lists +`consent.gdpr.applies_in` and `consent.us_states.privacy_states` — is the sole +jurisdiction source for **both** the auction consent gate and the EC gate. +The permission model replaces the EC side; if the auction gate keeps reading +the old lists while EC reads policy rules, the two will drift (adding a +country to one has no effect on the other, and an operator has no signal +that they disagree). + +Requirement: the auction gate's jurisdiction class derives from the same +resolved policy, reading the rule's explicit **`regime`** class (§3.2) — a +country is GDPR-class when its rule resolves to a `regime = "gdpr"` group. +The class is never inferred from purpose flags. Where the legacy lists must +survive an interim period, a CI test asserts consistency between each list +and the policy's regime classes, with deliberate divergences recorded as +explicit, commented exceptions in the test — never silent. Both legacy +lists are in scope, not only the GDPR one — and the US check is +region-shaped: **every configured `consent.us_states.privacy_states` entry must +have a matching `US/` rule**, and the country-level `US` rule must +resolve non-regulated (today applies privacy gating only to the configured +states), or the divergence is an explicit commented exception. An adapter +whose geo provider cannot resolve regions degrades **intentionally and +declaredly**: regionless US traffic hits the country rule — non-regulated, +today's behavior for non-privacy-state traffic; an operator preferring +protective country-wide gating writes `US = "us-opt-out"` as their own +declared choice. + +### 3.5 Shipped-table coverage + +A CI test asserts every member of the GDPR country list resolves to a +GDPR-class baseline in the example policy. This closes a defect class +nothing in PR #838's validation covered: a mistyped country key (`DL:` for +`DK:`) parses cleanly, starts cleanly, and silently drops a member state to +the fallback rule. Countries intentionally unlisted are governed by the +`rules.default` entry (§3.2); the example policy documents that fallback +inline, and ships it as the most protective baseline. + +## 4. Signal precedence — normative + +Signals are classified into three classes — a two-class model (TCF grant / +opt-out) cannot reproduce today's US behavior, where no-signal traffic is +blocked but an **explicit non-opt-out** value grants: + +- **Opt-out signals**, in two subclasses assigned by the §4.5 mapping: + **destructive** opt-outs (GPC; sale opt-outs; USP opt-out) revoke and + trigger withdrawal; **non-destructive** opt-outs (sharing, + targeted-advertising) revoke the permissions they map to but never + destroy the stored identity — a targeted-ads choice must not tombstone. + Both subclasses are honored **globally**, not only in the jurisdictions whose law defines + them — a deliberate, more-protective simplification: scoping a browser's + explicit opt-out to a geolocation guess would honor it for some visitors + and ignore it for others based on IP evidence. (For jurisdictions outside + US states this is a declared behavior change; migration spec §2 records + it.) +- **Grant signals** (affirmative permission): a decodable TCF record + consenting to the purpose; an **explicit GPP non-opt-out value** (e.g. + `sale_opt_out = false`); a **US Privacy string present and not opting + out** — including the "not applicable" flag, which today's tests pin as + allowing. Grant signals are what let a `requires_signal` US rule + preserve today's "no signal → block, explicit non-opt-out → allow" + behavior, which neither `granted` nor a TCF-only grant class could + express. **Which grant evidence a rule accepts is regime- and + permission-scoped** — grant signals are NOT interchangeable across + regimes: + + | Regime of the resolved rule | Evidence accepted as a grant for a `requires_signal` permission | + | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | `gdpr` | **Only** a TCF record consenting to that specific purpose | + | `us-privacy` | TCF consent for the purpose, or GPP/USP evidence **per the §4.5 field mapping** — a field grants only the permissions it maps to | + | `none` | TCF consent; GPP/USP grants only where §4.5 applicability yields one (the national section applies under `us-privacy` regimes, so in practice `none` grants via TCF — moot in the shipped policy, whose `none` baseline is `granted`) | + + Without this scoping, a US-style `sale_opt_out = false` would satisfy a + French `requires_signal` rule — no TCF, both purposes granted, EC minted, + partner egress authorized — contradicting the GDPR preservation row of + the migration matrix. Auction dispatch blocking separately would not + help; identity use would already be authorized. Opt-out signals and + refusals remain regime-agnostic (global), as before: scoping applies + only to what can _grant_, never to what can _revoke_. + +- **Refusals**: a decodable TCF record refusing the purpose. A refusal is + neither a grant nor an opt-out — it blocks acquisition (precedence 3) + and withdraws only per §4.2. + +**Precedence, highest first:** + +1. Policy `denied` — never set, regardless of any signal. +2. **Opt-out signal — always revokes**, regardless of any consent record + present. A GPC header revokes `store-on-device` and + `select-personalised-ads` even when an accompanying TCF string consents to + them. _(This is the rule PR #838 inverted: its resolution returned from + inside the TCF branch before ever reaching the opt-out check, so a + consenting CMP string made the browser's GPC signal a no-op — a + CCPA-facing regression. The pre-existing tests pinning this rule — + `ec_blocked_us_state_gpc_overrides_tcf` and companions — are reinstated + against the new API, not deleted.)_ +3. Consent record refusal — a TCF record present and refusing the purpose + revokes it. This applies in **every** jurisdiction, including + `granted`-baseline ones: an expressed refusal always beats a policy + default. Note this is a declared, more-protective divergence from the + pre-epic gate, which ignored consent records entirely outside regulated + jurisdictions — the migration spec's matrix (row 6) records it. Refusal + revokes new grants only; whether it also destroys existing identity is + governed strictly by §4.2. +4. Grant signal — a grant-class signal **accepted by the resolved rule's + regime for that permission** (table above) grants it (subject to 1–3: a + coexisting TCF refusal beats a non-TCF grant signal, matching today's + US-state ordering where a present TCF record decides before GPP/USP + values are consulted). +5. Malformed-present, no valid record of that family (§4.4) — **blocks + the baseline grant**: the permission is unset even under `granted`. + Never withdraws. +6. No signal — the policy baseline decides: `granted` sets it, + `requires_signal` leaves it unset. + +Normalization (§4.4) reduces each record family to exactly one of six +states — **valid-grant, valid-refusal, opt-out, malformed-present, +expired, absent** — and the precedence above plus the §4.1 matrix are +defined over those states, so no input state is unmapped. + +### 4.1 Decision matrix + +For each enforced permission, with baseline _B_ ∈ {granted, +requires_signal, denied}: + +| Opt-out present | TCF refusal present | Accepted grant present (regime-scoped) | Malformed-present | Result | +| --------------- | ------------------- | -------------------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| yes | — | — | — | **unset** (and withdrawal semantics apply, §4.2) | +| no | yes | — | — | unset (withdrawal per §4.2, trigger 2) | +| no | no | yes | — | set, unless B = denied | +| no | no | no | yes | **unset** (precedence 5 — blocks baseline grant) | +| no | no | no | no | set iff B = granted **and no suppression entry stands** — an active suppression (§4.3) beats the baseline, so one malformed request denies later no-signal requests under `granted` until newer valid grant evidence clears it; a policy-baseline grant alone does **not** clear non-user suppression (sign-off 24 covers this consequence) | + +### 4.2 Withdrawal vs. absence + +Withdrawal (destructive: expire the EC cookie, write revocation tombstones) +and non-grant (the permission is simply unset) are distinct outcomes, never +conflated. "Baseline" below always means the **resolved acquisition rule for +`store-on-device` in the request's jurisdiction, after overrides** — never a +group label, since a group can mix rules across permissions. + +The triggers, exhaustively — nothing else withdraws: + +1. **A destructive opt-out signal (per §4.5's destructive column: GPC, + sale opt-outs, USP opt-out) withdraws in every jurisdiction, whatever + the baseline.** Non-destructive opt-outs (sharing, + targeted-advertising) never trigger this — they revoke acquisition + only. (For US states this preserves today's behavior; elsewhere it is + the declared change of §4's global-opt-out rule.) +2. **A TCF record refusing `store-on-device` withdraws iff the baseline + is `requires_signal` or `denied` — and only when the refusal is + carried by the live request.** A persisted-KV consent record + participates in acquisition only and **never triggers withdrawal**: + without this, a refusal stored years ago under a `granted` policy + would destructively fire on the first signal-less request after the + policy tightens to `denied` — a policy edit tombstoning by proxy, + which trigger 3 forbids, and the counterexample to §5.5's + mixed-revision safety claim. Where the baseline is `granted`, + refusal blocks _new_ grants but never tombstones: tombstones are + irreversible, and PR #838 wrote them for visitors in unregulated + jurisdictions whose global CMP emitted a purpose-refusing string — + permanent identity loss under a regime the deployment never opted into. + (The `denied` arm exists so trigger 3 is coherent: after a policy + tightens to `denied`, an affirmative refusal must still be able to + withdraw a pre-existing identity.) +3. **A policy edit is not a user signal.** Tightening a baseline to + `denied` stops new identity but does not itself tombstone identities + minted before the change; cleaning those up is an operational action + (migration spec §6). An affirmative user signal (trigger 1, or trigger 2 + under the now-`denied` baseline) still withdraws them — with a test + pinning exactly this sequence: existing EC → policy tightens to + `denied` → refusal arrives → tombstone. +4. **Absence of signal never destroys identity.** A visitor who has not yet + made a choice is never stripped of an existing identity. + +Withdrawal checking follows §4 precedence: an opt-out signal triggers +withdrawal even when a consenting TCF record is present. +`ec_storage_withdrawn` (or its successor) gets direct unit coverage for +every trigger above; in PR #838 the headline "withdrawal expires identity" +behavior had no unit test at all. + +### 4.3 Withdrawal durability + +Withdrawal spans multiple KV writes and a cookie expiry; the contract for +partial failure is explicit (PR #838 expired the cookie first and +logged-and-swallowed tombstone-write failures, which can leave a live graph +identity with no browser handle pointing at it). The design centers on one +record that is simultaneously the durable intent, the discovery mechanism, +and the fail-closed marker: + +- **The family revocation record is written first.** Every identity carries + a stable **family ID**: minted rows store it, and — the case that makes + or breaks the protocol — **rows that lack the field derive it + deterministically** as a function of (record kind, provider namespace, + canonical graph key), e.g. `fam:v0:hmac:`. Determinism is the + point: a withdrawal arriving on the **first post-upgrade request** — a + GPC-carrying visitor whose v1 row has no family field and has never been + backfilled — computes the same family ID that every future reader of + that row computes, so the revocation record is discoverable even if the + writer crashes before ever touching the member row. A **random** ID + would recreate the exact partial-withdrawal orphan this design exists to + eliminate. Revocation writes one record keyed by the family ID; that + single write is the withdrawal. Per-member tombstones are cleanup that + follows, idempotent and retried. +- **Every consumer checks the family record, not per-member tombstones.** + A reader arriving through any still-live member row finds the family ID + in the row and the revocation record under it — partial revocation is + discoverable from every member, and the record survives member-tombstone + replacement (which today discards the original row's identity and + metadata, making sibling discovery impossible). +- **Negative authority has its own permission-exempt record, with a + complete transition contract.** A live refusal or opt-out must clear + prior positive provenance, but the row write that would do it requires + `store-on-device` — which the refusal just unset — and identity rows + may be eventually consistent. The **suppression record** + (`s`-class key per family, providers spec §6.3) resolves this. Its + contract: + + **Creation is cause-aware and mostly read-free.** A live resolution + whose outcome for a permission is unset writes suppression when the + cause is a **signal state** — refusal, non-destructive opt-out, + malformed-present — **unconditionally**, with no row read: conditioning + on observing positive provenance through an eventually consistent row + loses the race where a stale replica hides a just-committed grant. The + one cause that inherently needs prior state — applicable **absence** + clearing a previously positive permission — uses a narrow + **permission-exempt suppression-decision read** exposing only the + family ID and authority metadata (an undeclared exempt read was the + alternative, and skipping it leaves stale S2S authority). + **Policy-only tightening writes nothing**: a policy edit is not a user + signal (§4.2 trigger 3), and a signal-less request after + granted→denied must not create sticky user suppression that a policy + rollback cannot undo. + + **Writes are CAS-fenced; evidence recency decides semantics.** The + record requires linearizable per-key CAS (providers spec §7): CAS + serialization prevents lost updates, but arrival order does **not** + decide outcomes — each per-permission entry stores its cause, source + class, and authoritative evidence timestamp (first-seen normalization + for timestamp-less sources), and an incoming transition applies only + when its evidence timestamp is **newer than or equal to** the stored + entry's; ties resolve to the more restrictive state. So a delayed + grant with `LastUpdated = 100` never clears a suppression whose + refusal carried `200`, while a genuine re-consent at `300` does. The + transition table (causes without an intrinsic timestamp — malformed + records decode no `LastUpdated`, absence has no source — use their + **observation timestamp**, server receipt on the shared clock basis + within the skew window; cross-source comparison uses the authoritative + timestamp where one exists, else the observation timestamp, ties + restrictive), by stored cause: **opt-out from a timestamp-less + source** — cleared only by a grant with an authoritative timestamp + newer than the suppression's observation (sticky opt-out, sign-off + 16); **TCF refusal** — cleared by any regime-accepted grant with newer + authoritative evidence; **malformed-present / absence** — cleared by + any regime-accepted valid grant with newer evidence, including a + timestamp-less grant whose first-seen is newer (these causes are not + user opt-outs, so stickiness does not apply — without this rule, one + truncated request would permanently deny a GPP-only user). Policy + changes never clear user-signal suppressions. + + **Anti-replay for timestamps.** A future-dated record is rejected as + malformed beyond the skew window; within it, the record's digest is + stored with its **first normalized timestamp, which re-presentation + never advances** — otherwise a future-dated TCF string replayed after + an opt-out would keep re-normalizing to "now" and clear it. Equality is + **source-specific**: for GPP/USP the digest is the **canonical + per-permission semantic result** of §4.5 aggregation alone — two + encodings (or `N` vs explicit N/A) with the same meaning are the same + evidence and keep the original first-seen, so alternating equivalent + values cannot renew authority; for TCF the digest is the semantic + result **plus the authoritative `LastUpdated`** — a genuine CMP + renewal with unchanged purposes carries a newer `LastUpdated` and + legitimately refreshes authority age, which a semantics-only digest + would wrongly ignore. + + **Boundary, retention, ordering — without deadlocking recovery.** + Suppression is checked by **every `AuthorizedIdentity` constructor** + (both scopes), by pull sync's live path, and by every S2S recompute. + That gate plus clear-after-provenance would deadlock re-consent — + fresh P1 provenance cannot be written while P1 suppression blocks + `GraphOps`, and clearing first is forbidden — so recovery has its own + narrow write path: **`AuthorityRefresh`**, permission-exempt but + strictly scoped to committing provenance from the _current live + resolution_ (nothing else: no partner writes, no egress, no reads + beyond the row being refreshed) while suppression remains effective; + the clear then references that provenance's revision. Revisions are an + **application-level monotonic counter written with the row** — never + backend generation markers, which (per Fastly's own contract) only + detect change and carry no order — and S2S honors a clear only when it + can read that revision or newer; clearing first would expose the + _older_ positive snapshot through an eventual read. + + **The strong record carries positive-authority state too.** The + per-family record doubles as the **authority-state record**: alongside + negative entries it stores a per-permission positive-authority summary + (revision, evidence timestamp), CAS-updated by every provenance write. + The **absence decision reads this strong summary, never the eventual + identity row** — deciding "no prior authority" from an eventual + not-found loses the race where a just-committed grant is invisible on + a stale replica. A suppression/authority read failure **fails closed** + like a revocation read failure; retention must outlive the positive + authority it masks (providers spec durability/retention capability). + + _older_ positive snapshot through an eventual read. **Write failure + fails closed for the live request**, and the S2S residual is unbounded + for a never-returning visitor (sign-off 11), with fault tests for + suppress-vs-clear races, repeated-value sequences, and the + stale-provenance-read case. + +- **The cookie expires only after the family record commits.** +- **If the family-record write itself fails, nothing durable exists** — + the cookie stays and the durable client-side signal (GPC, CMP-stored + TCF) retries the whole withdrawal on the next request. Mitigations: + while graph **writes are degraded** (health signal), S2S partner egress + and sync updates fail closed on that instance (providers spec §6.2); + the failure is logged at `error` with a metric feeding the operational + repair path. The residual that remains — a single failed write on an + otherwise healthy graph, for a visitor who **never returns** — is + **unbounded**, not "bounded by return latency": return latency has no + bound for a non-returning visitor, and the per-instance breaker does + not reach other instances. Accepting this residual instead of building + a durable external retry queue is **product sign-off item 11** + (migration spec §8), not a footnote. +- **Consistency and retention are backend contracts with a single + normative home**: the providers spec consistency matrix (§7). It — not + this spec — states the requirement, and it requires a **strongly + consistent (read-after-write) primitive** for revocation records; no + bounded-lag alternative exists (an earlier draft here permitted one, + which contradicted the matrix — an adapter with a two-second lag would + have passed one spec and failed the other). A **failed family-record + read fails closed** for egress (revoked-unknown ≠ live), and revocation + records are retained beyond the maximum of cookie lifetime, row TTL, + rewrite grace, and downstream retry horizon — today's 24-hour tombstone + TTL is far below this bar and does not carry over. +- Fault-injection tests cover: family-record write fails → cookie + untouched, S2S behavior per degraded mode, retry completes; member + tombstone N fails after the family record → identity already revoked for + every reader, cleanup retries; the same-signal retry path end to end; + **first post-upgrade request is a withdrawal** (v1 row, no family field, + derived ID; crash between family record and row write; reader of the + untouched v1 row still sees the revocation). + +### 4.4 Signal normalization — normative matrix + +§4's precedence operates on normalized inputs: one effective consent +record and one effective opt-out state per request. The normalization +layer is where today's real-world mess lives, and PR #838 collapsed it +silently. These are the outcomes — decided here, not delegated to the +implementation; each row marked **changed** also appears in the migration +matrix. The pipeline order is itself normative: **(1) syntax validation +per source, (2) expiry per source — expired sources drop to absent +_before_ conflict resolution, (3) conflict resolution over the remaining +valid sources.** Current runtime resolves conflicts first and can select +an expired record before clearing both sources; expiry-first is a +**declared change** (migration matrix) that removes that path: + +| Input state | Effective record / outcome | Status | +| ---------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| Standalone TCF and GPP-embedded TCF disagree, mode `restrictive` | **Whole-record selection comparing the P1 ∧ P4 conjunction only** — today's algorithm, preserved (an earlier draft's lexicographic (P1, P4) tuple would have changed split-purpose outcomes): if exactly one record's conjunction is false, `restrictive` selects it; **equal conjunctions — including split-purpose disagreements — keep the standalone record**, as current code does | Preserved — pinned against current tests | +| Same, mode `permissive` | Same conjunction comparison, selecting the record whose conjunction is true; equal conjunctions keep the standalone record | Preserved — same pinning | +| Same, mode `newest` | Whole-record selection by **`LastUpdated`** subject to the existing freshness threshold; a tie, an incomparable pair, or timestamps inside the threshold fall back to the `restrictive` rule above (itself deterministic) | Preserved — same pinning | +| Expired consent record | Treated as **absent entirely** — grants nothing, refuses nothing, withdraws nothing; the baseline applies. Under a `granted` baseline that means the grant stands: an expired refusal is not current evidence and must not revoke indefinitely | Preserved | +| One valid record + a second malformed record of the same family | The **valid record governs**; the malformed one is ignored with a `warn` log. Fail-closed-on-malformed (below) applies only when no valid record of that family exists | Decided here | +| One valid record + one **expired** record of the same family | The valid record governs — the expired one dropped at pipeline step 2, before conflict resolution ever saw it | **Changed (declared)** — current runtime resolves the conflict first and can select the expired record | +| Persisted-KV consent record, live record present | **Live wins**, always; the stored record is never consulted | Preserved | +| Persisted-KV consent record, no live record | Substitutes as the effective record **iff within the same TTL as a live record**, then flows through the full normalization pipeline (syntax, expiry, conflict) like any live record; staler → absent. This narrow read is exempt from the graph-read permission gate (§7) — determining `store-on-device` cannot itself require `store-on-device` | **Changed (declared)**: current code returns immediately after the KV load, bypassing expiry and conflict normalization | +| Proxy/mirror mode | **Minimal opt-out extraction still runs; full semantic decoding is skipped.** Because opt-outs are globally authoritative (§4), proxy mode must not suppress them: the §4.5-mapped opt-out fields (GPP US sections) and the US Privacy string are decoded — nothing else — alongside syntax validation, so a valid SaleOptOut or USP opt-out revokes and withdraws exactly as outside proxy mode. No grants are ever derived from records in proxy mode; a present record otherwise blocks grants (fail-closed); absent → baseline. GPC needs no decoding | **Changed (declared)**: today proxy mode skips decoding entirely — fail-open under permissive baselines and, worse, opt-out-blind | +| GPP / US Privacy fields | Per the normative field mapping of §4.5 — fields are not interchangeable signals; **explicit N/A is grant-class (not-opted-out), absent grants nothing** — one meaning, everywhere | Decided here (§4.5) | +| Malformed-but-present record, no valid record of that family | **Blocks grants** (fail-closed acquisition — it does not degrade to "absent", which under a `granted` baseline would turn garbage into a grant, the fail-open path in both #838 and the first draft of this spec). Never triggers withdrawal — destruction requires an affirmative, decodable signal (§4.2) | Changed (declared) | + +### 4.5 US signal field mapping — normative + +GPP and US Privacy fields map to specific permissions with specific +effects; they are never interchangeable, a field's absence or N/A value +behaves per its table row — **explicit _Not Applicable_ is grant-class +(not-opted-out), preserving current USP tests and GPP `NotApplicable` +handling; only a genuinely absent field contributes nothing** (this is +the single normative statement; an earlier "N/A contributes nothing" +rule is dead, and the P4-authorizing consequence is sign-off item 17) — +and only the fields marked destructive trigger +withdrawal. Section IDs and versions are those of the IAB GPP +specification current at implementation time; adding a section or field is +a change to this table. + +| Source · field | Value | `store-on-device` (P1) | `select-personalised-ads` (P4) | Destructive withdrawal? | +| -------------------------------------------- | --------------------------- | -------------------------------------- | -------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | +| GPP US section · `SaleOptOut` | opted out | opt-out | opt-out | **Yes** (preserves today) | +| GPP US section · `SaleOptOut` | not opted out | grant | grant | — | +| GPP US section · `SharingOptOut` | opted out | — | opt-out | No | +| GPP US section · `SharingOptOut` | not opted out | — | grant | — | +| GPP US section · `TargetedAdvertisingOptOut` | opted out | — | opt-out | **No** — a targeted-advertising choice must never destroy the stored identity | +| GPP US section · `TargetedAdvertisingOptOut` | not opted out | — | grant | — | +| US Privacy · `opt_out_sale` | `Y` | opt-out | opt-out | **Yes** (preserves today) | +| US Privacy · present, `N` or N/A | — | grant | grant | — (today's tests pin N/A as allowing; USP carries no distinct targeted-advertising field, so it never maps to one) | +| Any field | explicitly _Not Applicable_ | as the field's not-opted-out row above | as the field's not-opted-out row above | — | +| Any field | absent | — | — | — | + +**N/A vs absent (restating the single rule):** explicit _Not +Applicable_ = grant-class; absent = nothing; a non-applicable section's +fields grant nothing (their opt-outs still count, per step 2). + +**Applicability and aggregation — ordered algorithm:** + +1. **Section map (normative, pinned here — not "whatever GPP is + current"), matching the official IAB registry in full:** section 6 ↔ + the **US Privacy string carried as a GPP section** — it maps to the USP + rows of the field table in full, opt-outs _and_ grant-class values, + under the same applicability rules as the national section; `US` national ↔ 7 (usnat); + the state sections — `US/CA` ↔ 8, `US/VA` ↔ 9, `US/CO` ↔ 10, + `US/UT` ↔ 11, `US/CT` ↔ 12, `US/FL` ↔ 13, `US/MT` ↔ 14, `US/OR` ↔ 15, + `US/TX` ↔ 16, `US/DE` ↔ 17, `US/IA` ↔ 18, `US/NE` ↔ 19, `US/NH` ↔ 20, + `US/NJ` ↔ 21, `US/TN` ↔ 22, `US/MN` ↔ 23, **`US/MD` ↔ 24, + `US/IN` ↔ 25, `US/KY` ↔ 26, `US/RI` ↔ 27** (an earlier draft wrongly + claimed MD/IN/KY/RI had no section). A truncated map silently loses + opt-outs — a Texas (16) or Maryland (24) sale opt-out must not vanish. + The implementation PR cross-checks this list against both the current + decoder's section set and the official registry, and the accepted version per + section is **pinned to a registry snapshot vendored into this + repository** — a checked-in file enumerating, per mapped section, the + accepted version(s), taken from the IAB registry at ratification (a + date is not an immutable identifier, and "enumerated by the + implementation PR" was two-implementations-diverge territory; the + vendored file is the single reproducible authority, and updating it + is a reviewed spec change); a + mapped section carrying a version outside the pinned revision is + treated as malformed-present (blocks grants, never withdraws — §4.4), + not as absent. Adding a section or version is a + change to this map. +2. **Applicability gates grants only — never opt-outs.** A mapped + **opt-out** field (either subclass) is honored from **any** section on + **any** request, whatever the regime — this is §4's global-opt-out + rule, and filtering it by jurisdiction would make a French visitor's + `usnat SaleOptOut` simultaneously mandatory (§4) and ignored (here). + For **grants**: the national section is applicable to any + `us-privacy`-regime request; a state section is applicable iff it maps + to the resolved `US/`; foreign-state sections and all sections + on non-`us-privacy` requests grant nothing. Regionless US traffic: + national section only. A configured privacy state with no + state-specific section uses the national section alone. +3. **State-over-national, per field — for grants only:** where an + applicable state section carries a field, its value governs that + field's **grant** derivation; the national section fills only fields + the state section lacks. This precedence **never suppresses an + opt-out**: a national-section opt-out stands even where the state + section's same field says not-opted-out — step 2's global rule wins, + or a state string could erase a globally authoritative national + opt-out. +4. **Aggregate across what remains applicable:** an opt-out (of either + subclass) in any applicable field beats a grant from another — + restrictive aggregation. + +`SharingOptOut` and `TargetedAdvertisingOptOut` are new enforcement +inputs — current code consults only the sale field — and are declared as +such in the migration matrix. + +## 5. Jurisdiction resolution + +### 5.1 Order + +Geo resolution runs **before** permission resolution — jurisdiction is an +input to the permission set, which is why geo providers cannot themselves be +gated on it (providers spec §5). A selected geo provider resolves country +and optional region; rules match `country/region` first, then `country`, +case-insensitively. + +### 5.2 Lookup failure + +Provider selected, lookup resolves nothing for a request → the configured +`[geo] default_country` rules apply (per #779). An adapter whose geo +implementation can never resolve anything must not accept the selection at +all — that is the capability check of providers spec §6, and it prevents a +"selected but always empty" provider from silently converting every request +to §5.3 semantics without §5.3's guard. + +Declared residual: when the default country's baseline is permissive, a +per-request lookup failure is a per-request grant to traffic of unknown +origin — this path is not fail-closed, and the spec does not pretend it is. +The lookup-failure rate is exported as a metric and logged, so an elevated +rate (a degraded geo backend silently converting traffic to the default) is +observable rather than invisible. + +### 5.3 No geo provider selected + +Every request resolves to `default_country` — jurisdiction becomes a static +constant, which is only honest when the operator can genuinely assert +single-jurisdiction traffic. It is not only `granted` baselines that make +this dangerous: with a `requires_signal` baseline, a page-global CMP that +emits a consenting TCF string grants permissions for every mis-attributed +visitor just as effectively. + +Constraint: **startup fails** when no geo provider is selected and any +**jurisdiction consumer** is enabled, unless the operator sets an explicit +acknowledgment (`[geo] assume_single_jurisdiction = true`). Jurisdiction +consumers are enumerated, not implied: an EC provider is selected, +server-side auction dispatch is gated on `regime` (§7), or any raw-EC / +EID egress path is active. An EC-provider-only exemption would be too +narrow — a stateless deployment still dispatches auctions off the policy's +regime class, and no geo + a permissive static jurisdiction misclassifies +EU traffic for that decision just as it would for identity. Only a +deployment with **no** jurisdiction-sensitive behavior is exempt. Without +this guard, the natural migration config (`default_country = "US"`, geo +unset) silently grants `store-on-device` and EID transmission to every EU +visitor — the highest-severity finding of the PR #838 review. The startup +log always prints the effective baseline and whether geo is live. + +### 5.4 Defaults, two distinct fallbacks + +`[geo] default_country` is required; startup fails without it (per #779). +It covers requests that resolve **no country at all**. Countries that +resolve but match no rule fall to the policy's `rules.default` entry +(§3.2). The two fallbacks are deliberately separate: "we could not place +this request" and "we placed it somewhere we have no rule for" are +different states, and pre-epic behavior treated them differently (fail +closed vs. non-regulated) — collapsing them is what made PR #838's +migration story unresolvable (migration spec §2, rows 5 and 7). + +### 5.5 Policy revision activation + +A policy edit propagates through the config store, so a fleet briefly +mixes revisions. The contract: instances stamp every resolution and every +provenance write with the policy revision they used (already required by +§7); the mixing window is bounded by config propagation and observable via +the config-version metric; and mixed-revision irreversibility is bounded and **accepted, not +denied** (sign-off 19): destructive withdrawal triggers are user +signals, never policy (§4.2 trigger 3) — the one revision-sensitive destructive case +(trigger 2 under a now-`denied` baseline) requires an affirmative user +refusal at the evaluating instance, which is safe under either revision. +S2S recomputation always evaluates against the instance's current +revision and records it. One divergence is explicitly accepted rather +than fenced: during convergence, a live refusal under a +`granted`-revision instance suppresses while the same refusal under a +tightened-revision instance destroys (trigger 2) — the destructive +outcome is the target revision's intended behavior arriving early on +part of the fleet, coordinated activation fencing is not worth its +machinery, and the acceptance is sign-off item 19. Rolling a policy +revision back restores acquisition rules but **cannot resurrect +tombstoned identities**; the migration guide says so where operators +will read it. + +## 6. Failure-mode matrix — normative + +| Condition | Resolution behavior | +| ---------------------------------------------------- | --------------------------------------------------------------------------------------------- | +| Geo lookup fails at request time (provider selected) | `default_country` baseline | +| No geo provider configured | `default_country` baseline, guarded by §5.3 | +| Country resolved, no matching rule | Policy `rules.default` | +| Region resolved, no region rule | Country rule | +| No `[permissions]` section | Compiled-in fallback: everything `requires_signal`, `regime = "gdpr"` | +| S2S sync request (no user signals) | Authorized by stored provenance re-validated against current policy (§7) | +| Malformed policy | Rejected at config push / startup (§3.3) — never per request | +| No `default_country` | Startup failure | +| Undecodable TCF/GPP record (present but malformed) | Blocks grants (fail-closed acquisition, §4.4); never withdraws; opt-out signals still honored | +| Signals contradict (opt-out + consent) | Opt-out wins (§4) | + +The intended posture is fail-closed, with its two exceptions stated rather +than glossed: geo lookup failure resolves to the configured default (§5.2's +declared, metered residual — permissive defaults make this path fail-open), +and the §5.3 static-jurisdiction configuration exists only behind an +explicit operator acknowledgment. Every other ambiguous state resolves to +the configured baseline or more restrictive. + +## 7. Enforcement points + +Consumers of the resolved set in this epic: + +1. **EC provider execution** (providers spec §5) — the provider's declared + `required_permissions()` must all be set for minting and identity use. + This gate applies to EC providers only. **Geo** is ungated because + gating it is circular — jurisdiction is an input to permission + resolution. **Device** is ungated by a different, deliberate decision + (it is _not_ a resolution input): its security-classification role must + run for traffic that has granted nothing, and operator selection is the + recorded authorization — providers spec §5 states the decision and its + boundary. If a future vocabulary adds a purpose covering geolocation or + fingerprinting, gating those providers will require a two-phase + resolution specified then, not improvised. +2. **EC lifecycle** — creation requires `store-on-device`; withdrawal per + §4.2. Recognition, canonicalization, and revocation of an existing + identifier are **never** permission-gated — they must run precisely when + permissions are withdrawn (providers spec §5). +3. **Every raw-EC egress and identity operation** — the concrete + inventory, normative per path (one test per row; a denylist check + proves no ungated egress exists): + + | Path | Required permissions | Notes | + | ---------------------------------------------------------------- | ---------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | OpenRTB `user.id` | `store-on-device` ∧ `select-personalised-ads` | Raw EC is identity in the bidstream — gated exactly as EIDs. PR #838 gated only EIDs, leaving `user.id` reachable with Purpose 4 refused | + | EC-derived auction request IDs | both purposes | Derived values are identity | + | Page-bids path | both purposes | | + | Bidstream EIDs | both purposes | The one gate PR #838 had | + | Proxy / click / Testlight forwarding of the EC cookie or headers | both purposes | **New hardening, declared change** — these paths extract the raw cookie/header without today's jurisdiction gate (migration spec §2 row 11b) | + | Identify endpoint (partner-facing) | both purposes | Partner identity exchange, not a first-party lookup — decided here | + | Pull sync (browser-request-scoped partner exchange) | both purposes, from the **live** request resolution | Pull sync is created from a browser request and checks the live `EcContext` today — it keeps using the live P1 ∧ P4 decision plus the family revocation state (§4.3); stored provenance is never a substitute for available live evidence | + | Batch sync (context-free S2S partner exchange) | both purposes, from **stored provenance** | The only truly signal-less path; authority rules below. Today's handler only authenticates and checks row state, so this gate is **declared hardening** (migration spec §2) | + | Request-scoped graph reads/writes (non-revocation) | `store-on-device` | | + | Revocation paths (tombstones, withdrawal reads) | **exempt** | Must work when permissions are unset | + | Stored consent-state lookup (§4.4) | **exempt**, narrowly scoped | Determining `store-on-device` cannot require `store-on-device` | + | Integration persistent response cookies | `store-on-device` (+ P4 where the cookie is an advertising identifier) | **Deferred with the hook's cookie surface** — the write-side gate alone was insufficient (read/use/forward/withdrawal unmodeled), so cookie operations ship only with the full model; this row and the client-cycle **page leg** (module injection gated on the provider's full declaration) join the inventory when their features do, and the §5.3 no-geo guard's consumer list grows with them | + | Suppression-record writes (§4.3) | **exempt** | Clearing authority is protective, like revocation | + | Authority-state / suppression-decision read (§4.3) | **exempt**, narrowly scoped | Returns only family ID, per-permission authority summary, and suppression entries — no identity values, no partner data; a test proves nothing else escapes | + | `AuthorityRefresh` provenance write (§4.3) | **exempt**, strictly scoped | Commits current-live-resolution provenance only; enables suppression recovery without reopening `GraphOps` | + + With **no EC provider configured**, identity use fails closed: a cookie + value present on the request never egresses anywhere — never vacuously + allowed (#838's `ec_allowed` was `is_none_or`, vacuously true with no + provider). + + **S2S authority (batch sync).** A context-free server-to-server request + carries no user signals, geo, or `EcContext`. Its authority is the + identity's **stored provenance**: per-permission, time-bounded evidence + written at mint and replaced on later live requests — grant basis + (which signal class granted, per permission), the evidence's + **authoritative timestamp and `valid_until`** (per evidence class), + resolved jurisdiction, and policy revision — **not** provider/version, + which lives only in the immutable mint tag, or a post-rotation visit + would restamp a v1 identity as v2 (providers + spec §6.1). Freshness is a **per-evidence-class contract**, because not + every source carries a timestamp: + + | Evidence class | Authoritative timestamp | Age reset | Max age | + | ------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | ------------------------- | + | TCF consent | The record's `LastUpdated` | Only a record with a **newer** `LastUpdated` | Existing TCF expiry TTL | + | GPP / USP values (no intrinsic timestamp) | **First-seen**: when TS first observed this exact normalized value (a **per-permission equality digest computed over only the applicable, aggregated §4.5 fields for that permission** — never the whole GPP record, or a CMP touching an unrelated notice field would mint a new digest and reset first-seen forever) | Re-presenting an identical digest **keeps the original first-seen**; a different value is new evidence with a new first-seen | Consent TTL (same as TCF) | + | Policy-baseline grant (`granted` rule, no signal) | The policy revision that granted | Re-derived on every recompute against the current revision — policy is not user evidence and does not age; it changes | n/a | + + Timestamps are compared with bounded clock-skew tolerance; + beyond-window future-dated records are **rejected as malformed**, and + within the window a record's first normalized timestamp is pinned to + its digest and never advanced by re-presentation (§4.3's anti-replay + rule — clamping every presentation to "now" would make a future-dated + string perpetually fresh). And every live + resolution **atomically replaces the complete per-permission + snapshot**, never merges — a refusal, opt-out, malformed or absent + state in the fresh resolution clears prior positive authority for its + scope, so an old P4 grant cannot survive a later P4 refusal. A sync request performs a **full recompute of both + permissions** from that stored evidence against the _current_ policy: + it fails closed when the stored jurisdiction's rule is now `denied`, + when a `granted` baseline tightened to `requires_signal` and the stored + evidence contains no accepted grant for that permission, when the + stored evidence has **expired**, or when the regime no longer accepts + the stored grant's source class (§4's regime-scoped table). Any of + these → no update, row flagged for the operational cleanup of §4.2 + trigger 3. Sync never mints authority of its own. + + **Legacy (pre-epic) rows** carry none of these fields. They are treated + as reserved `hmac-v0` provenance with **no stored grant evidence**, so + they **fail closed for partner egress and batch updates** until a live + browser request lazily backfills provenance from a fresh resolution. + Failing open here would grandfather every pre-epic identity past the + permission model indefinitely. + +4. **Server-side auction dispatch** — gated on the policy `regime` class, + normatively: + + | Regime | Dispatch rule | Preserves | + | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------- | + | `gdpr` | Dispatch only with a decodable, unexpired TCF record consenting to Purpose 1. Malformed, expired, or absent record → **no bid request leaves** (no-bid response). | Today's GDPR/unknown arm | + | `us-privacy` | Dispatch proceeds in every signal state, including opt-out — the opt-out strips identity (rows above) but the contextual auction runs. | Today's US-state arm | + | `none` | Dispatch proceeds. | Today's non-regulated arm | + | **Any regime, TCF-sourced effective record** — a raw TC string on the request, a GPP section-2 hint (both detected **before decoding**), or a persisted-KV fallback record of TCF origin (§4.4) | The `gdpr` row applies: dispatch requires the _effective_ record to be decodable, unexpired, and consenting to Purpose 1. A **malformed or expired** raw signal therefore blocks dispatch — today a malformed raw TCF blocks, and gating this arm on decodability would have silently relaxed that. A US or non-regulated request carrying a Purpose 1 refusal is likewise blocked. | Today's raw-signal arm — **must not regress** | + + The **compiled-in fallback policy has `regime = "gdpr"`** (§3.1) — the + no-policy posture must be the most protective for dispatch too, and a + regime-less fallback would leave dispatch undefined. When dispatch is + blocked, nothing leaves for that request: no PBS/APS call, no UA/IP/geo + forwarding to bidders. When dispatch proceeds, what the request may + carry is governed row-by-row by the egress inventory; the full + regulatory context (consent strings) is always forwarded so downstream + partners make their own decisions (§1). + +The client-cycle resolve endpoint (own spec, currently on hold) would be a +further consumer if and when it proceeds. + +## 8. Testing strategy + +- **The decision matrix is the test plan.** Every row of §4.1 × each + baseline, every trigger of §4.2, and every row of §6, as table-driven + tests. The ~24-case matrix deleted by PR #838 (net −18 tests in the + consent module, replaced by happy-path cases only) is restored in + equivalent form against the new API; signal-precedence conflicts + (opt-out + consenting TCF) are mandatory cases, not optional ones. +- The §4.4 normalization matrix as table-driven tests, including every + configured conflict mode and the malformed-record rows. +- The §7 raw-EC egress inventory: one test per inventoried egress proving + the gate, plus a denylist-style check that no ungated egress exists. +- The §7 auction-dispatch matrix: every regime × signal state + (consent, opt-out, malformed, expired, absent), including the + no-policy fallback regime, asserting both the dispatch decision and + that a blocked dispatch emits no outbound request. +- The §7 S2S authority path: **every denial reason individually** — + denied rule, tightened baseline without acceptable stored evidence, + expired evidence, regime-rejected grant source — plus the exempt + consent-state lookup, stale-evidence re-presentation (age must not + reset), and legacy-row fail-closed-then-backfill. +- The full cross-product **regime × permission × evidence source** from + §4's acceptance table and §4.5's field mapping, including multi-section + aggregation conflicts and the applicability algorithm's foreign-section + and regionless rows. +- Provenance snapshot-replacement transitions per permission: prior grant + → refusal, → opt-out, → malformed, → absent — plus a mid-replacement + fault proving the surviving state is the complete old **or** complete + new snapshot, never a merged mixture. +- Legacy-row withdrawal end to end (§4.3's derived family ID). +- §4.3 fault-injection cases. +- Policy validation tests for every §3.3 rejection, exercised through both + acceptance paths (push-time and startup). +- Shipped-table coverage test (§3.5) and jurisdiction-consistency test + (§3.4) covering both legacy lists. +- One end-to-end integration scenario per posture: opt-in jurisdiction with + and without consent, opt-out jurisdiction with GPC (including GPC + a + consenting TCF string), the no-geo/default-country path, and the + no-policy compiled fallback. + +## 9. Out of scope + +- Additional purposes (extension procedure in §2). +- A build-time-embedded policy file (PR #838's approach) — rejected for the + reasons in §3.1, not deferred. +- Per-signal jurisdiction scoping (honoring GPC only where a law defines + it): rejected in favor of the global rule in §4; revisiting it is a + policy-model change requiring its own review. + +## 10. Divergences from issue #779 + +This spec supersedes #779 on the following points; the issue is updated to +reference this spec when the PR merges, so there is one acceptance contract, +not two: + +| #779 says | This spec says | Why | +| -------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- | +| Unmatched countries fall to `default_country` | Unmatched-but-resolved countries fall to the policy's `rules.default`; `default_country` covers only unresolved requests | The two states had different pre-epic behavior; collapsing them made migration unresolvable (§5.4) | +| The full TCF purpose vocabulary is modeled | Only enforced purposes appear (§2) | Nine inert purposes in a policy file are a compliance hazard, not forward compatibility | +| Policy is an embedded file | Policy is `[permissions]` in `trusted-server.toml` (§3.1) | Runtime config-store pipeline; validation at push time | +| Permission sources are open-ended (#777: publisher interaction, external services may grant) | Sources are jurisdiction, policy, and the §4 signal taxonomy; a pluggable source interface is **deferred** (§1) | Shipping an interface with no second source repeats the inert-surface mistake; the extension path (a new §4 signal class) is defined instead | diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md new file mode 100644 index 000000000..f8ebc0ddf --- /dev/null +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -0,0 +1,758 @@ +# Design Spec: Pluggable Edge Cookie, Device, and Geo Providers + +**Status:** Draft +**Author:** Engineering +**Issue references:** #777, #778, #780, #781 +**Related specs:** `2026-07-30-permission-model-design.md`, +`2026-07-30-provider-migration-rollout-design.md`, +`2026-07-30-client-cycle-ec-resolve-design.md` +**Last updated:** 2026-07-31 + +> **Context.** PR #838 proposed a first implementation of this epic in a single +> change. Review of that PR surfaced design gaps this spec exists to close +> before a second implementation pass: an identity abstraction that owned +> minting but not recognition, per-adapter divergence in provider selection, +> silent misconfiguration modes, and speculative trait surface with no +> production caller. This spec is the authoritative statement of what the +> provider architecture must do; where it contradicts PR #838, this spec wins. + +--- + +## 1. Overview and goals + +Trusted Server makes three per-request data decisions that are currently +hard-wired: whether to create or keep an Edge Cookie (EC) identity, how to +classify the requesting device, and whether to resolve geolocation. Each +becomes a **provider**: a selectable component chosen in operator +configuration, with a deliberately neutral default. + +Goals: + +- A deployment picks an implementation per concern (including none) without a + code change to Trusted Server core. +- Defaults are neutral: with no configuration, no EC is created, device + classification uses only the User-Agent, and no geolocation is performed. A + default deployment makes no third-party or host-specific call. +- An **EC provider declares** the permissions its data use requires (see the + permission model spec); **core enforces** that declaration. A provider + cannot authorize itself. (Geo and device providers are governed + differently, for two different reasons spelled out in §5.) +- All adapters (Fastly, Axum, Cloudflare, Spin) behave identically for + identical configuration, or fail loudly at startup where a host cannot + satisfy the selected provider. + +Non-goals: + +- No vendor provider ships in this epic beyond the host-platform + implementations named below. +- The client-cycle (browser round-trip) provider type is **out of scope** + here; it has its own spec and must clear that spec's requirements first. + +## 2. Provider taxonomy + +| Concern | Trait | Built-in default | Opt-in host implementation | +| ----------- | -------------------- | --------------------------- | ----------------------------------------------------------------- | +| EC identity | `EdgeCookieProvider` | none (stateless) | `hmac` (in core; HMAC over client IP, preserves today's identity) | +| Device | `DeviceProvider` | `builtin` (User-Agent only) | `fastly` (JA4 / HTTP-2 fingerprints) | +| Geo | `GeoProvider` | none (no location) | `platform` (host geo lookup) | + +Selection keys are strings in operator configuration: + +```toml +[ec] +provider = "hmac" + +[ec.providers.hmac] +passphrase = "example-passphrase" + +[device] +provider = "builtin" + +[geo] +provider = "platform" +``` + +**Deliberately not carried over from PR #838:** the `host-signals` EC +provider (identity from HMAC over JA4/HTTP-2 TLS fingerprints plus client +IP). Minting _identity_ from TLS fingerprints is a different privacy +proposition from device _classification_ (#780) and was specified by no +issue; if wanted, it returns with its own spec and its own vocabulary +discussion. A config selecting `provider = "host-signals"` is rejected at +startup like any unknown key (migration spec §4). + +## 3. The identity lifecycle contract + +This is the section PR #838 lacked, and the source of its most structural +defect: the trait abstracted **minting** an identifier but left +**recognition** (`is_valid_ec_id`), **hashing** (`ec_hash`), and **KV key +normalization** hard-coded to the built-in HMAC shape. Any provider whose +identifiers do not match `{64hex}.{6alnum}` minted cookies that the very next +request discarded, and whose identities could never be tombstoned on +withdrawal. + +An `EdgeCookieProvider` owns the **complete lifecycle** of the identifiers it +mints. Every lifecycle operation core performs on an EC value MUST be routed +through the selected provider: + +| Lifecycle operation | Where core uses it today | Contract | +| ---------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Mint** | EC generation on first eligible request | Provider returns the identifier (and only core writes the cookie). | +| **Parse / canonicalize** | Reading `ts-ec` back from the request; deciding `ec_was_present`; batch-sync ingestion | Provider parses a cookie value into its **canonical** identifier, or rejects it. Canonicalization and **equivalence are provider-declared, never imposed globally**: each provider ships equivalence fixtures naming exactly which variants are the same identity — case sensitivity is provider-specific (signed/base64-style envelopes are case-sensitive; even the built-in HMAC id is case-insensitive only in its hex prefix, with a case-preserved suffix). Declared-equivalent values parse to the same canonical identifier (satisfying #778). A value the selected provider does not recognize is treated as absent (but see §6.1 legacy readers). | +| **Canonical graph key** | KV identity-graph row reads/writes | The provider supplies a canonical key **suffix**; **core constructs the physical key** per the §6.3 key grammar (legacy-HMAC verbatim keys excepted), so cross-provider and cross-record-kind isolation is enforced by construction rather than promised by provider code. Suffixes are stable, KV-safe (length and character-set limits), and collision-free within the provider's space. Two equivalent envelopes of one identity map to one key — verbatim cookie bytes as the key would fork graph rows on canonicalization differences and discard today's batch-sync canonicalization. | +| **Cluster prefix** (optional capability) | IP-cluster sizing (`cluster_trust_threshold`, implemented as a **KV prefix listing**), pull-sync dedupe, log redaction | A provider declaring cluster support returns a prefix that is a **literal byte prefix of the canonical graph key** — the cluster count lists keys by prefix, so an independently derived hash that is not an actual key prefix silently reports the wrong cluster size. The prefix deliberately collides across identifiers minted from the same client evidence. A provider without the capability declares so, and cluster-dependent gating follows a configured degradation policy (treat cluster size as unknown, with the KV-write decision that implies made explicit in config) instead of counting garbage. | +| **Tombstone** | Withdrawal: expiring the cookie and writing revocation markers | The identifiers eligible for tombstoning are exactly those the provider parses — never a shape-gated subset. | + +**Invariant:** for every provider `P` and every identifier `id` minted by `P`, +`P.parse` round-trips `id` — including every variant `P`'s declared +equivalence fixtures name, all of which canonicalize to the same identifier +and graph key; where `P` declares cluster support, `cluster_prefix(id)` is a +literal prefix of `graph_key(id)` and is shared by identifiers minted from +the same client evidence; and a withdrawal request carrying `id` tombstones +it. A conformance test suite MUST assert this round-trip for every shipped +provider — driven by each provider's equivalence fixtures, plus +cross-provider key-namespace and KV length/charset cases — and the suite +MUST be written so a future provider crate can run it against its own +implementation. Conformance tests inject deterministic entropy; +probabilistic assertions ("two random suffixes differ") are not accepted. + +Three global rules sit above every provider: + +- **Identifier bounds.** A minted identifier obeys a global cookie-safe + alphabet (valid cookie-octets: no separators, whitespace, or control + characters; normatively `[A-Za-z0-9._~-]`) and a global maximum of + **256 bytes** — stated here, in the normative contract, so dependent + documents reference one number instead of assuming their own — for the identifier itself, not + only the graph key — enforced by core at mint and at parse, so no + provider can emit a value the cookie layer or logs cannot carry. +- **Namespaces are declarative and core-proven.** Disjointness of two + opaque `parse` functions is not provable, so every provider declares a + **static namespace descriptor** in a core-owned declarative form — a set + of literal prefixes and/or fixed-shape grammars (alphabet + length + segments), never arbitrary parser logic. Core proves pairwise + disjointness of all configured descriptors at startup (§6.1), and the + conformance suite asserts each provider's `parse` accepts **only** + values matching its declared descriptor — so the declaration, not the + parser, is the authority the overlap check rests on. +- **Namespace reservation.** The legacy HMAC grammar `{64hex}.{6alnum}` is + formally **reserved as the `hmac` provider's namespace descriptor**. `hmac`'s graph + key is the identifier verbatim and its cluster prefix is the 64-hex + prefix, so every pre-epic row stays reachable and every prefix listing + intact (migration spec §3) — and **no other provider may mint + identifiers or produce graph keys matching that grammar**, which is what + makes verbatim-compatibility and provider-namespacing coexist. + Conformance fixtures include an existing pre-epic row (reachability) and + a prefix-listing case. For `hmac`, the equivalence fixtures pin: + uppercase/lowercase hex-prefix variants are equivalent; suffix case is + preserved and significant. +- **Cluster size means live identity rows.** Prefix counting lists + identity-row keys; family, suppression, reservation, and transaction + records live in other namespaces and never inflate a count. Member + tombstones share the identity key but carry the short cleanup TTL, so + their inflation is transient and biases conservative (an over-count + trips the trust threshold toward denial, never toward extra writes); + the listing filters by the value's `kind`/liveness within the existing + list limit where the backend returns values, and the residual + over-count where it cannot is declared. `cluster_trust_threshold` is validated against the backend's listing + cap at startup — a threshold of 200 against a 100-key listing cap + would make every capped count look trusted; the count must page or + saturate at threshold + 1. A computed cluster size is + **not persisted beyond its inputs' lifetime**: today's code stores the + calculated `cluster_size` in the row and reuses it for the row's full + TTL, which would freeze a tombstone-inflated count for up to a year — + stored values carry a short validity (within the tombstone-TTL + horizon) or are recomputed on use. Aliases are reserved-future + (§6.1) and excluded by `kind` when they exist. +- **No-cluster behavior is still defined.** A provider without cluster + support deduplicates pull-sync by canonical graph key and redacts logs + with a fixed-length hash of the graph key; `cluster_fallback` (§6.1) + governs only the trust/write decision, not these. + +## 4. Trait surface: minimalism rule + +Every trait method MUST have at least one production (non-test) caller in the +same PR that introduces it. Speculative surface observed in PR #838 that MUST +NOT ship without a caller: + +- `keys_equal` (no production caller; existed to serve a unit test — its + legitimate purpose, #778's equivalent-envelope comparison, is satisfied + structurally by §3's canonicalizing `parse` instead: equivalents + canonicalize to the same identifier, so no comparison method is needed), +- `GeneratedEdgeCookie::response_headers` (empty in all built-ins, plumbed + through three layers), +- `IdentityInput.permissions` / `IdentityInput.consent` (ignored by all + built-ins), +- `DeviceProvider::required_permissions` / `GeoProvider::required_permissions` + — dropped entirely, not deferred: §5 explains why these two kinds cannot + be permission-gated at all. + +If a future feature needs one of these, it arrives with that feature. + +The minimal `EdgeCookieProvider` surface implied by §3 is: + +```rust +pub trait EdgeCookieProvider { + /// Stable configuration key ("hmac"). + fn id(&self) -> &'static str; + /// Permissions this provider's data use requires. Enforced by core for + /// minting and identity use — never for parse/tombstone (§5). + fn required_permissions(&self) -> PermissionSet; + /// Parse and canonicalize a cookie value into this provider's + /// identifier; None when unrecognized. Identifies the provider + /// NAMESPACE only — never a configuration version (§6.1: versions + /// resolve from row provenance). Values the provider's declared + /// equivalence fixtures name as equivalent canonicalize identically. + fn parse(&self, value: &str) -> Option; + /// Canonical graph-key SUFFIX (bounded length, KV-safe). Core — not + /// the provider — constructs the physical key (§6.3 key grammar), so + /// cross-provider and cross-record-kind isolation is structural. + /// Sole exception: hmac keys (every version) are the identifier + /// verbatim — the reserved legacy grammar. + fn graph_key_suffix(&self, id: &EcId) -> GraphKeySuffix; + /// Cluster capability: a literal byte prefix of the physical graph + /// key, shared across identifiers minted from the same client + /// evidence. None when the provider lacks IP-cluster semantics (§3). + fn cluster_prefix(&self, id: &EcId) -> Option; + /// Mint an identifier from request evidence. The one acquisition + /// operation of the epic (server mint); failure means no identity + /// this request (§6.2). Returns the identifier WITH the active + /// configuration version — the immutable mint tag needs it and core + /// cannot reach into provider-specific configuration to learn it. + fn generate(&self, input: &IdentityInput<'_>) + -> Result>; + // GeneratedIdentity { id: EcId, mint_version: ProviderVersion } + /// Cryptographic verification of a parsed identifier against request + /// evidence. Recognition (`parse`) is not authentication; rowless + /// handling (§5) requires this. Returns the matched configuration + /// version — provenance needs it and a bool cannot carry it — or + /// None when nothing verifies. + fn verify(&self, id: &EcId, input: &IdentityInput<'_>) -> Option; +} +// VerifiedIdentity { version: ProviderVersion /* … */ } +``` + +The acquisition-mode enum (`ServerMint` / `ClientResolve`), the +`ClientResolveContext` contract, replay-reservation schemas, and the +reservation capability rows are **not part of this normative surface** — +they live in the deferred client-cycle document and return with that +feature, per this spec's own minimalism rule (§4): the epic's only +acquisition mode is server mint, expressed directly as `generate`. + +(Names indicative; the shape is normative. `required_permissions` joins the +trait at step 5 of §11, together with its enforcement point.) + +## 5. Permission enforcement is core's job — for EC providers + +The gate is on **minting and identity use, never on the lifecycle +operations that withdrawal depends on**. Before minting through an EC +provider or using an identity (raw-EC egress, permission model spec §7), +core resolves the request's permission set and refuses when the provider's +`required_permissions()` are not all set. **Parse, canonicalization, graph +lookup for revocation, and tombstoning always run**, permissions or not — a +blanket execution gate would refuse to run the provider in exactly the +state an opt-out produces, making the withdrawal it demands impossible. A +spy-provider test pins the split: with `store-on-device` unset, `generate` +is never called while a withdrawal request still parses the cookie and +writes tombstones. + +**A generated identity is not active until its graph row commits.** No +cookie write, no egress, no auction use may observe a minted identifier +before its graph row (with provenance, §6.1) has committed — PR #838 let a +generated EC reach an auction before finalization refused the cookie, +producing an identity that existed for one request and nowhere else. The +normative order is: gate → `generate` → graph-row commit → cookie +scheduled → eligible for egress. "Cookie scheduled" means queued onto the +final response — `Set-Cookie` is physically emitted after first-request +processing, so egress eligibility begins at **graph commit**, not at +header emission; the identity exists durably from that moment. A +graph-commit failure means the mint never happened: no cookie, no egress, +error logged, the next request retries. + +**Pre-existing rowless cookies are expired and re-minted — never +adopted.** An earlier adoption design failed on an authentication limit: +HMAC verification can authenticate only the 64-hex prefix; the 6-char +suffix is independent randomness, so a client holding `H.aaaaaa` can +present `H.aaaaab`, `H.aaaaac`, … — every variant prefix-verifies, and +an adopt path would mint a **separate durable row and family per +variant**. Therefore: + +- **"Rowless" requires an authoritative not-found, and only in + migration mode.** Identity-row visibility may be eventual, so a plain + not-found proves nothing — a just-minted row invisible on a stale + replica would classify its own cookie as rowless and expire/re-mint + it, forking the identity. The rowless path therefore activates only + when the deployment-metadata **graphless-migration flag** is set (set + by the §4.2 readiness step for deployments that actually ran + graphless; permanently-graphed deployments never classify anything + rowless), and the existence check uses the backend's strongest read. + Outside migration mode, or on any read error, the state is + **indeterminate**: no identity use, no mint, no cookie expiry — + "treated as absent" was the wrong contract, since absence feeds the + fresh-mint path. +- A verified rowless cookie (`verify → VerifiedIdentity`, carrying the + matched version) is **expired and replaced by a fresh mint through the + ordinary graph-backed path** when permissions allow; continuity is + deliberately not preserved (migration matrix row 13, sign-off 21). An + unverifiable cookie (including the declared roaming false-negative) is + simply expired. +- **Rowless withdrawal writes nothing** — there is no server-side state + to revoke: no row, no partner mappings, no S2S surface. The cookie is + expired, and that is the entire withdrawal. (An earlier prefix-derived + family record was over-engineering with two defects: unauthenticated + suffix variants could mint records, and — because the HMAC prefix is + per-IP — one visitor's withdrawal would have revoked every identity + behind the same IP. Family records exist only for row-backed + identities, derived from the full graph key, one derivation + everywhere.) + +**Egress is typed, not policed.** The inventory-and-denylist test +(permission model spec §7) is a backstop, but conventions do not survive +new code — the ungated proxy/click/Testlight paths happened precisely +because raw EC values circulate as ordinary strings. Core therefore +introduces a **scope-parameterized `AuthorizedIdentity`**, +constructible only by core, only after the checks _for that exact scope_: +`AuthorizedIdentity` after parse + `store-on-device` + +family-revocation **and suppression** checks; +`AuthorizedIdentity` additionally after +`select-personalised-ads` — suppression is part of both constructors, not +a separate prose obligation on S2S callers. Outbound serializers (ORTB builder, page +bids, sync, identify, forwarding) accept `AuthorizedIdentity` +and nothing weaker — an unparameterized wrapper would let a P1-only +identity flow into an ORTB request. A future bypass then +requires deliberately reconstructing the raw string — visible in review — +rather than passing along what was already in hand. The same boundary +applies **request-side, as a concrete API transition, not an +assertion**: the current filter/proxy inputs expose the raw request +(cookies included), so a filter can read `ts-ec`, copy it into +`X-Vendor-Identity`, and return it through response effects — response +snapshot redaction cannot undo that. The contract: integration-facing +request access moves to a typed **`RedactedRequestView`** whose stripped +set is enumerated — the `ts-ec` cookie and every `ts-*` cookie, `x-ts-*` +identity/consent headers, and the EIDs header — with identity reachable +only through a scoped `AuthorizedIdentity` parameter; the raw-request +filter/proxy interfaces are migrated in the **same PR** as the typed +egress boundary (they are the same boundary), and the tests are +enumerated: a denied/withdrawn request through a filter, a proxy, and a +forwarding path, each asserting no identity value is readable or +emittable. + +The gate applies to EC providers **only**. Geo and device are ungated for +two _different_ reasons, stated separately because only one of them is +structural: + +- **Geo: circularity.** The permission set is resolved _from_ jurisdiction, + which is resolved _by_ the geo provider. Gating geo on the resolved set + is unsatisfiable. +- **Device: a decision, not a circularity.** Device classification is not + an input to permission resolution (the inputs are jurisdiction, policy, + and signals), so ordering geo → resolution → device → EC and gating + device is perfectly implementable. This spec deliberately does not: + the shipped device providers process technical request metadata (UA, + JA4/HTTP-2 fingerprints) for **security classification** — the bot gate + protecting KV-backed identity writes — which must run precisely for + traffic that has granted nothing. The authorization for that processing + is the operator's explicit `[device] provider` selection — a statement + about the **opt-in host-fingerprint provider**; the `builtin` UA-only + default processes nothing beyond the User-Agent every request already + carries and needs no such authorization — and this spec records that as + the decision, with its privacy implication stated: a + device provider whose data use goes beyond security classification (for + example feeding fingerprints into targeting or identity) is **not + authorized by selection alone** and requires a vocabulary extension plus + a gate before it may ship. This bites immediately, not hypothetically: + today's graph rows persist the JA4 class, an HTTP/2 fingerprint hash, + and buyer-facing quality metadata — persistence and scoring that exceed + security classification. The epic therefore **stops writing + fingerprint-derived buyer-facing fields into new rows** (a declared + change, migration spec §2); the boolean security classification outcome + may be persisted. Re-adding them is the vocabulary-extension route. + The field-level graph contract itself is normative in this spec — + §6.3 — not deferred to the implementation. + +PR #838 declared `required_permissions` on all three traits but consulted +it only for the EC provider; the geo and device declarations were +decorative — worse than absent, because they read as a gate and are not +one. This spec resolves that by **not having** the method on those traits +(§4), with the two rationales above in place of the pretense. + +## 6. Selection, validation, and failure modes + +All validation happens at **settings construction** — a misconfiguration is a +startup error, never a request-time error and never a silent behavior change. + +| Configuration state | Behavior | +| ---------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `provider` names an unknown key | Startup error listing valid keys. | +| `provider` set, its `[ec.providers.]` block missing | Startup error. | +| `[ec.providers.]` block present, `provider` unset | **Startup error.** (In PR #838 this silently ran stateless — the half-migrated config becomes a production identity outage detected by revenue drop. Rejecting it is the fix.) An operator who genuinely wants stateless deletes the block. | +| `provider` set to an implementation the running adapter cannot satisfy (e.g. a provider requiring host TLS fingerprints on an adapter that has none) | Startup error at adapter wiring time. Adapters declare their host capabilities to the composition root; the root checks the selected provider's needs against them **once**, at startup — not per request. | +| A `[ec.providers.]` block referenced by neither `provider`, `legacy_providers`, nor a `versions`/`mint_version` chain | **Startup error.** An unreferenced block is almost always a dropped `legacy_providers` entry — accepted silently, it strands every identity that provider minted: unresolvable and, worse, non-withdrawable. | +| No `provider`, no providers block | Valid: the neutral default for that concern. | +| `rewrite_legacy` present at all (deferred out of the epic, §6.1) | **Startup error** — unknown key; transparent re-mint returns only with its own spec. | +| `provider = "none"` (explicit stateless) | Valid, and the only way to combine statelessness with `legacy_providers`: minting stops, legacy readers keep existing identities resolvable and **withdrawable** (§6.1). Without this state, `hmac` → stateless would strand every live row in revoke-proof limbo. | +| A minting provider (or any `legacy_providers`) configured, but no identity-graph store configured or openable | **Startup error.** The lifecycle contract assumes graph persistence (§5); discovering its absence at first mint would be a request-time config failure, which this table exists to forbid. | + +Unknown fields inside every provider config block are rejected +(`deny_unknown_fields` on all new settings structs — the pre-existing `Ec` +struct already has it, but PR #838 shipped `EcProviders`, `DeviceConfig`, +and `GeoConfig` without it, so a typo like `providr` was silently ignored). + +### 6.1 Provider switching: active writer, legacy readers + +Switching `[ec] provider` must not strand the identities the previous +provider minted: with only the selected provider recognizing cookies, an +`hmac` → vendor switch turns every existing cookie into "absent", orphans +its graph row, and — worst — makes a later opt-out unable to tombstone it. +The contract: + +- `[ec] provider` names the **active writer**: the only provider that + mints. +- `[ec] legacy_providers = ["hmac"]` (optional list) names **legacy + readers**: providers consulted, in order, for parse, graph lookup, and + tombstoning when the active writer does not recognize a value. Legacy + readers never mint. Each listed key must have its `[ec.providers.]` + block, validated like the active one (§6 table). +- **Parse order and ambiguity.** The active writer parses first; the first + match wins. Overlapping recognition is not resolved at request time but + **forbidden at startup**: the declared namespace descriptors (§3) of the + active writer and every legacy reader must be pairwise disjoint — a + check core can actually perform, because descriptors are declarative; + configuring a pair whose descriptors intersect is a validation error. +- **The recognizing provider governs.** A legacy-owned identity is gated + by the **legacy provider's** `required_permissions()` for identity use — + the provider that minted under a declared data-use contract is the one + whose contract applies. +- **Provenance is provider- and version-tagged, with a defined rotation + schema.** Every graph row carries the minting provider id, its + configuration version, and the per-permission grant evidence (grant + basis, evidence timestamp, resolved jurisdiction, policy revision) that + the S2S sync authority recomputes from (permission model spec §7). + Same-provider key/passphrase rotation is configuration, not a provider + switch: a provider block may hold multiple `versions` entries + (`[ec.providers.hmac.versions.v2] passphrase = …`) with + `mint_version = "v2"` selecting the writer. **`parse` cannot identify a + version** — every HMAC version shares one grammar and `parse` returns no + version — so the mint version lives in **immutable row provenance** + (rows without a tag are `hmac-v0`), and cryptographic verification + consults configured versions newest-first only where provenance is + unavailable (a cookie with no reachable row). Removing a version entry + is a retirement subject to the same evidence rules as retiring a legacy + reader (migration spec §6). +- **`rewrite_legacy` is deferred out of the epic.** Transparent re-mint + under the active writer required primitives no production adapter has + (row-store CAS with read-your-writes plus a linearizable transaction + class — §7 matrix), and successive reviews kept surfacing open protocol + problems: retention lineage (a rewritten 364-day-old row either + rejuvenates the identity or leaves a year-long cookie pointing at an + expiring row — a lineage expiry must be pinned across canonical row, + alias, family record, and emitted cookie), alias visibility under + eventual stores, chain stranding after repeated migrations, and + cluster-count inflation by alias keys. Those are recorded here as the + entry bar for a future `rewrite_legacy` spec. Within the epic, provider + switching is served by **legacy readers alone**: old identities keep + resolving and stay withdrawable; they are never transparently + re-minted. The `rewrite_legacy` key is rejected at startup as unknown, + and the alias record class exists in the key grammar (§6.3) only as + reserved-for-future — nothing in the epic writes one. +- Retiring a legacy reader is the explicit end of those identities: + the migration guide documents the cleanup procedure (migration spec + §6). **Provenance backfill is not retirement evidence** — a backfilled + row still lives under the legacy cookie namespace and still needs that + provider's parser; only a quiet period spanning the full cookie/row + lifetime justifies removal. +- Tests: switch active provider → request with old cookie → identity + still resolves and a withdrawal tombstones it; old cookie with no + matching legacy reader → treated as absent and **never egresses**; + `provider = "none"` + legacy reader → no mints, withdrawal still + works. (Rewrite-specific tests left with the rewrite deferral.) + +Cluster degradation config (referenced from §3): when the active writer +lacks the cluster capability, `[ec] cluster_fallback = "allow" | "deny"` +decides whether KV-backed writes gated on cluster trust proceed; there is +no implicit default — the operator chooses. + +### 6.2 Runtime failure matrix — normative + +Startup validation (§6) covers configuration; this covers what happens +when a healthy configuration meets an unhealthy runtime. Every row logs at +`error` with a metric; none is silent: + +| Failure | Behavior | +| ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | +| `generate` returns an error | No identity this request; request proceeds stateless; no cookie written | +| Graph-row commit fails at mint | Mint never happened (§5): no cookie, no egress; next request retries | +| Graph read fails on an existing identity | Identity unusable this request (fail closed for egress); cookie untouched | +| Cluster prefix listing fails | Treated as cluster-size-unknown → `cluster_fallback` policy applies | +| Tombstone write fails | Permission model spec §4.3: family retries, readers fail closed on partial families | +| Geo provider returns invalid output (unparseable country) at runtime | Treated as lookup failure → `default_country` (permission model spec §5.2), counted in the lookup-failure metric | +| Device provider signals unavailable at runtime (e.g. no JA4 on a request) | Classification degrades per the provider's declared fallback, never silently upgrades `looks_like_browser` | + +The **degraded-graph health signal** referenced above and by the +withdrawal contract is a defined state machine, not a vibe: it is +**per-instance and in-memory** (no shared propagation, no stored health +record whose own read could fail), entered when graph-write failures cross +a sliding-window threshold (N failures within window W), and exited with +hysteresis after M consecutive successes. While degraded: S2S partner +egress and sync updates fail closed; organic requests continue stateless. +The thresholds ship as constants with the implementation and are printed +in the startup log. + +Its protection is therefore **local-only, and the spec says so**: a +backend-wide outage degrades every instance through its own observations +within one window, but an instance-local family-write failure leaves +other instances — which have no record to find, and healthy backends of +their own — serving S2S egress until the browser's durable signal retries +successfully. That residual is **unbounded for a never-returning visitor** +(sign-off item 11 — the permission and migration specs state this and +this spec must not undercut them), is counted (failed family writes are +a first-class metric), and is accepted +in place of a deployment-wide shared fail-closed channel, whose own +availability and freshness would be a harder problem than the one it +solves. + +### 6.3 Storage contract — normative + +**Physical key grammar.** Core constructs every key; providers supply only +the bounded suffix: + +| Record class | Key | Notes | +| -------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Identity row (non-hmac) | `id//` | Suffix from `graph_key_suffix`, ≤ 128 bytes, KV-safe alphabet. **No version segment** — the mint version lives in the row envelope; a versioned key would be circular (core would need the version to read the row that states the version) | +| Identity row (hmac, **every** version) | The identifier verbatim (`{64hex}.{6alnum}`) | Reserved grammar for the whole provider, not just v0 — keeping all HMAC versions on the verbatim scheme is what keeps the 64-hex cluster prefix a literal key prefix for every HMAC row. (Passphrase rotation still changes a given IP's HMAC, so clusters split across rotation until old identities expire — inherent to rotation, declared, not a key-scheme artifact) | +| Alias | **The source identity key itself** — the alias is a value written _at_ the replaced row's address, distinguished by a `kind` discriminator in the JSON envelope | A separate `alias/…` address could never work: lookups by the old cookie hit the source key, and a single-key CAS cannot install a record at a different address | +| Family revocation | `fam/` | Family ID from mint or the deterministic legacy derivation (permission spec §4.3) | +| Suppression (negative authority) | `sup/` | Per-permission suppression entries + timestamps; permission-exempt writes; consulted by every S2S recompute and partner-egress check (permission spec §4.3) | +| Rewrite transaction | `rwx/` | One in-flight rewrite per family | +| Replay reservation _(informative — deferred with client-cycle; not normative surface)_ | `resv/…` | Deferred client-cycle draft | + +Grammars are pairwise non-intersecting by their literal prefixes (plus +the reserved hmac grammar), and every record value carries a `kind` +discriminator alongside its schema version — so a reader always knows +what it fetched, including where two classes deliberately share an +address (row vs. alias). The `/` shown in key sketches is **notation, +not the wire byte** — and the wire form is **one portable grammar, not +per-adapter delimiters** (per-adapter delimiters would give the same +logical identity different physical keys on different adapters, breaking +migration, shared storage, and parity; and Fastly's prefix queries +reject both `/` and `:`, so no delimiter character is safely portable). +Physical keys are **delimiter-free with fixed-width segments**: a +1-character class tag — `i` row, `r` family revocation, `s` suppression, +`x` transaction, every tag chosen **outside the hex alphabet** so no +generated key can begin with 64 hex characters, which is what makes +disjointness from the legacy `{64hex}.{6alnum}` grammar _provable_ +rather than asserted (an earlier `f` tag was itself a hex digit) — then +a **4-character provider code from a checked-in, append-only, +never-reused registry file** (allocation is a reviewed commit; +codes are immutable and never recycled, including for retired +providers), then the suffix. Segment boundaries are positional, so no +segment can contain or escape a delimiter, prefix queries are plain +string prefixes on every backend, and a grammar-disjointness test covers +every class against the legacy grammar. (hmac verbatim keys remain the +reserved exception, with the 64-hex cluster prefix at position zero.) + +**Wire schemas** (JSON, like identity rows; every class carries a schema +version): the **alias record** (reserved-future, with rewrite) holds target key, created-at, retirement +deadline, and fencing epoch; the **family revocation record** holds the +family ID, revoked-at, triggering signal class (§4.5 destructive column), +and a **family epoch** bumped on every revocation-state change (the +client-cycle commit CAS is conditioned on it) — deliberately no identity +data, so it can outlive its members; the **authority-state (suppression) record** holds, per permission: +state (`suppressed`/`cleared`), cause, source class, authoritative or +observation evidence timestamp, the **application-level provenance +revision** a clear references, and the positive-authority summary +(revision + evidence timestamp) — plus the record-level CAS version +counter and schema version; unknown-field and range validation apply +like every class (strong class, permission-exempt writes per the +permission spec's inventory; revisions are app-level counters because +backend generation markers detect change without ordering); +the **rewrite transaction** holds source key, target key, copy point, +state, and epoch; the **reservation** holds state, owner hash, lease +epoch, outcome, and created-at (client-cycle spec). Field validation and +TTLs: aliases live to their retirement deadline; family records to the +§7 retention rule (beyond every member, cookie, rewrite, and retry +lifetime); transactions to completion plus an audit window; reservations +at least through token expiry. + +#### Graph row contract + +The per-field contract for identity rows, covering today's v1 fields and +the fields this epic adds. Serialization is JSON with the existing `v` +schema-version discriminator; from release N+1 onward (migration spec §4), +readers round-trip unknown keys **semantically** (values preserved through +read-modify-write; byte-identical output is not required and not +achievable through a structured serializer). + +| Field | Purpose | Source | Gating permission (egress) | TTL / refresh | Rewrite | On revocation | +| ------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------- | ----------------------------------------- | ---------------------------------------------------------------------------------- | ------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------- | +| key (v1: identifier verbatim; v2: core-constructed, §4) | Row identity | Provider/core | — | Row TTL (1 y today) | New canonical row; old key becomes alias | Family record governs; member tombstone as cleanup | +| `v` | Schema discriminator | Core | — | — | Written at current version | Retained | +| `created` / **`expires_at`** | Row age and the **absolute retention deadline, pinned at mint** — every update writes with the _remaining_ lifetime, never a fresh full TTL (today's full-TTL rewrite lets a frequently visited identity live forever; refreshable derived state must never rejuvenate the identity) | Core | P1 (first-party ops) | Never extended | Preserved (no rejuvenation) | Retained in tombstone | +| `consent.tcf` / `consent.gpp` | Raw signal snapshot for audit; superseded as authority by provenance | Request | Never egressed to partners | Replaced on live resolution (§7 snapshot rule, permission spec) | Fresh live values | Scrubbed | +| `consent.ok` / `consent.updated` | v1 liveness flag — **superseded by the family revocation record**; written during member cleanup for v1-reader benefit through the transition | Core | — | — | Fresh | `ok = false` written as cleanup; the family record is authoritative (v1's 24 h tombstone TTL does not bound revocation) | +| New: **immutable mint tag** (`mint_provider`, `mint_version`) | Credential retirement and audit — write-once at mint (legacy backfill may populate a missing tag once); **never part of the replaceable snapshot**, or a v1 identity revisited after rotation would be restamped v2 | Mint (or one-time backfill) | — | Immutable | — | Retained | +| New: per-permission provenance (grant basis, authoritative timestamp, `valid_until`, jurisdiction, policy revision, provider/version) | S2S authority | Live resolution only | Read by S2S recompute | `valid_until` per evidence class; replaced atomically, never merged | **Fresh live resolution** — never copied | Scrubbed | +| New: `family_id` | Revocation discovery | Core at mint (derived for legacy, permission spec §4.3) | — | Immutable | Shared across linked rows | Is the revocation key | +| `geo.country` / `geo.region` | Jurisdiction snapshot | Geo provider at mint | P1 | Written at mint | Fresh | Scrubbed | +| `geo.asn` / `geo.dma` | Cluster disambiguation / market signal | Platform at mint | P1; DMA additionally P4 for bidstream use | Written at mint | Fresh | Scrubbed | +| `pub_properties` (origin/seen domains) | Creation context | Core at mint | P1 | Write-once | Preserved | Scrubbed | +| `device.*` (JA4 class, H2 hash, quality metadata) | **Discontinued for new rows** (§5): fingerprint-derived, buyer-facing — beyond security-classification authorization. v1 rows retain them read-only; they are never egressed post-epic and are dropped at rewrite | Fastly device provider | None grants egress | Write-once (v1) | **Dropped** | Scrubbed | +| New: security classification outcome (boolean) | Bot-gate result | Device provider | — (never egressed) | Written at mint | Fresh | Scrubbed | +| `network.*` (immutable evidence: ASN etc.) | Cluster disambiguation | Platform at mint | P1 | Write-once | Fresh | Scrubbed | +| Derived cluster state (`cluster_size`, computed-at) | Trust gating | Computed | — | **Refreshable, short validity; generation-CAS update; never touches `expires_at`** | Recomputed | Scrubbed | +| `ids` (partner → UID map) | Partner identity graph | Pixel/pull/batch sync | P1 ∧ P4 (partner egress) | Per-mapping timestamps; bounded count/length | Copied **with original timestamps/expiry** | Scrubbed | +| New: alias record kind | Rewrite indirection (§6.1) | Core | — | Retirement deadline | Is the mechanism | Family-revoked like any member | + +## 7. Composition root and adapter parity + +Provider construction happens in exactly one place per concern +(`build_ec_provider`, `build_device_provider`, `build_geo_provider`), called +by **every** adapter. No adapter may wire a concrete implementation directly: +in PR #838 the Cloudflare adapter installed its host geo unconditionally, +so identical configuration produced different jurisdictions on different +adapters — which the permission model then turned into different privacy +outcomes. + +Requirements: + +- **Adapters declare capabilities against an explicit matrix — with + consistency semantics, not just feature bits.** The capability set: + identity-graph persistence, atomic single-key reservation, KV prefix + listing (cluster support), platform geo, device host evidence + (JA4/HTTP-2), and legacy-rewrite support. Persistence capabilities carry + **per-record-class consistency requirements**, because "has KV" says + nothing about whether revocation is observable: + + | Record class | Required semantics | + | ---------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | Replay reservations (client-cycle) | **Linearizable CAS with fencing** (ownership epoch). Eventually-consistent KV cannot provide this — on Cloudflare that means a Durable-Object-class primitive, not Workers KV; an adapter without it fails startup for client-cycle selection | + | Family revocation records | **Globally observable strong consistency** — every instance's read observes a committed revocation, not merely the writing session's own writes (writer-scoped read-your-writes is insufficient for a fleet). Cloudflare Workers KV is **not eligible** — "60 seconds or more" is an expectation, not a bound; on Cloudflare this record class needs a Durable-Object-class primitive. An adapter without an eligible primitive fails startup for identity features. A **failed or erroring revocation-record read fails closed** for egress | + | Family suppression records | **Linearizable per-key CAS** — read-after-write alone cannot provide read-modify-write monotonicity: two writers both read, and an older clear overwrites a newer suppress | + | Identity-row mutation | **Generation CAS** (conditional write on row generation) with reread/recompute on conflict — rows are heavily mutable (snapshots replaced, partner IDs merged, derived state refreshed), so unordered last-writer-wins loses newer evidence and mappings; _visibility_ may stay eventual, unordered _mutation_ may not. Fastly KV offers generation-marker conditional writes; Workers KV's documented concurrent last-write-wins is ineligible for mutation-bearing rows | + | Row creation | **Atomic create-if-absent** (fresh mints), same primitive family | + | Deployment metadata (schema floor) | **Write-once/CAS**, outside ordinary config storage (migration spec §4) | + | Rewrite transactions | **Linearizable fenced CAS required** (same primitive class as reservations) | + | Alias installs (reserved) | Row-store **per-key CAS with read-your-writes** — recorded for the future `rewrite_legacy` spec; nothing in the epic writes an alias | + | Identity rows | Eventual **visibility** acceptable _after_ a generation-CAS mutation commits (see Identity-row mutation above) — the earlier "rows are accretive" claim is deleted: rows replace snapshots, merge partner IDs, and refresh derived state, and unordered last-writer-wins loses newer evidence | + + Every record class additionally declares **durability and maximum + retention**: a store passing the consistency check but capping TTLs + below the computed revocation/suppression horizon (e.g. a 30-day + maximum against one-year rows) would let identities become usable + again when their revocation expires — startup proves the configured + store meets each class's computed horizon, and persistence across + restart is part of the declaration. Each adapter's declaration is part + of its wiring, drives the §6 capability-mismatch startup error, and every §6.2 runtime-failure row + gets fault-injection coverage on every adapter declaring the + corresponding capability. The **concrete per-adapter values** — the + actual matrix, not the abstract capability list — as known today; a + cell marked _verify_ must be established before the depending feature + is selectable on that adapter, and the filled matrix is normative: + + Cells distinguish **platform availability** (the host offers a + primitive) from **wired** (Trusted Server integrates it) — conflating + them is how a "yes" cell hides an unusable feature. Feature eligibility + requires wired, not merely available: + + | Capability | Fastly | Axum (dev) | Cloudflare | Spin | + | ----------------------------------------------------------- | -------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | + | Graph persistence (eventual OK) | KV Store: available + wired | **Not wired** — the current adapter installs `UnavailableKvStore`; an in-process store is dev-feasible but does not exist yet | Workers KV: available + wired (eventually consistent) | Key-value: **available, not wired** — Spin config is embedded and EC KV routes are unwired today | + | Prefix listing (cluster) | Yes (used today) | **Unavailable** (no store wired — in-process feasibility is a note, not a cell) | Yes (eventual) | _verify_ | + | Strongly consistent revocation reads | _verify_ against Fastly KV semantics | **Unavailable** (no store wired) | **Workers KV: no** — needs Durable Objects, not currently wired | _verify_ | + | Linearizable fenced CAS _(informative — deferred features)_ | **Not currently available** | **Unavailable** (no store wired) | Durable Objects: possible, not wired | **No** | + | Platform geo | Yes — country + region | No | **Yes — country only, no region** (`cf-ipcountry`): regionless US traffic degrades per the permission spec's declared rule, which directly changes state-level US outcomes | No | + | Device host evidence (JA4/H2) | Yes | No | No | No | + | Suppression / authority-state CAS | Generation-marker conditional write: available, **wiring to verify** | **Unavailable** (no store wired) | Workers KV: **ineligible** (last-write-wins); Durable Objects: feasible, not wired | **Unavailable** | + | Identity-row generation-CAS mutation | Same primitive as above | **Unavailable** | Workers KV: **ineligible**; DO: feasible, not wired | **Unavailable** | + | Row create-if-absent | Generation-marker create: available, **wiring to verify** | **Unavailable** | Workers KV: **ineligible**; DO: feasible, not wired | **Unavailable** | + | Deployment metadata (write-once/CAS) | **Not wired** — needs a primitive distinct from the config store | **Unavailable** | DO: feasible, not wired | **Unavailable** | + | Durability / max-retention proof | KV durable; TTL ceilings **to verify** against computed horizons | **Unavailable** | Workers KV TTLs: to verify; DO storage: feasible | **Unavailable** | + +- Each adapter's runtime-services setup routes through the shared builders. +- Providers are constructed **once** per application instance and stored in + app state; PR #838 rebuilt the provider (cloning the secret into a fresh + `Box`) up to three times per request. +- The cross-adapter parity suite gains cases asserting: (a) the selected + provider is honored on every adapter, (b) the neutral default performs no + host call on every adapter, and (c) a capability-unsatisfiable selection + fails startup on the adapters that cannot satisfy it. + +## 8. Crate layout and CI + +Provider crates live flat under `crates/` following the existing naming +convention: `crates/trusted-server-geo-fastly`, +`crates/trusted-server-device-fastly`. (PR #838 introduced a nested +`crates/geo/fastly` layout that broke the directory–package correspondence +every other member follows.) No placeholder directories: a `crates/…/README.md` +with no crate ships when the first crate does. + +Every new crate is added to the `.cargo/config.toml` aliases +(`check-fastly`, `clippy-fastly`, `test-fastly`, `build-fastly`) in the same +PR that adds the crate, and to the CI gate list in `CLAUDE.md`. PR #838's new +crates compiled only transitively and were never linted with `-D warnings` +nor had a single test. + +## 9. Behavior preservation notes + +Two defaults chosen for neutrality change effective behavior on existing +Fastly deployments; both are called out in the migration spec and must be +prominent in release notes: + +- **Bot gate.** The pre-provider EC bot gate required JA4 _and_ platform + class. With `device.provider = "builtin"` the gate degrades to User-Agent + heuristics. Restoring the stronger gate requires `[device] provider = +"fastly"`; the migration guide lists this as a behavior-preserving step for + Fastly deployments. +- **Geo.** With no geo provider, jurisdiction resolution falls to the + configured default country. The permission model spec (§5.3) constrains + this combination so it cannot silently grant permissions to mis-attributed + traffic, and §11 below sequences the default flip so the constraint exists + before the flip does. + +## 10. Testing strategy + +- Provider conformance suite (§3 invariant, deterministic entropy) run + against every shipped provider. +- EC minting-gate tests (§5), including the spy-provider case: permission + unset → `generate` never called, withdrawal still tombstones. +- Legacy-reader tests (§6.1): provider switch → old cookie resolves and + withdraws; unmatched old cookie never egresses. +- Settings validation tests for every row of the §6 table, including the + block-without-selector rejection and the `legacy_providers` rules. +- Parity suite additions of §7. +- Unit tests inside each provider crate; crates with no native-target tests + still get clippy coverage via the alias wiring of §8. + +## 11. Implementation order + +1. Traits + lifecycle contract + conformance suite, `hmac` provider + passing it (behavior-identical to today; see migration spec §3 for the + ID-stability vectors). +2. Settings selection + validation table. +3. Composition root + all four adapters wired through it, parity cases. +4. Device and geo provider selection. **The geo neutral default does not + flip in this step**: under the current jurisdiction gate, absent geo + resolves to `Unknown`, which fails closed — flipping the default here + would zero EC issuance for every deployment that had not yet opted into + `[geo] provider = "platform"`. Until step 5, the Fastly adapter's geo + selection defaults to `platform` (today's always-on behavior); the + selector exists, only its default is held back. +5. The permission model PR: flips the geo default to none **in the same + change** that introduces the `default_country` fallback and the §5.3 + acknowledgment guard, and adds the EC permission-enforcement point of + §5; `required_permissions()` appears on the EC trait in this step, not + before (per the §4 minimalism rule). + +Steps 1–4 are independently reviewable, behavior-preserving, and do not +depend on the permission model: the EC gate keeps its current jurisdiction +logic until the permission model PR replaces it. + +## 12. Divergences from issue #778 + +This spec supersedes #778 on the following points; the issue is updated to +reference this spec when the PR merges, so implementation has one +acceptance contract: + +| #778 says | This spec says | Why | +| ------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | +| Identifier comparison is a provider operation (`keys_equal`) | Comparison is structural: `parse` canonicalizes, so equivalent envelopes become the same identifier and graph key (§3) | Satisfies the same requirement with no comparison method to leave uncalled | +| A provider can return response headers | Dropped (§4) | Empty in every built-in in PR #838, plumbed through three layers with no consumer; returns with the first feature that needs it | +| One built-in provider (HMAC) preserving today's behavior | Same, plus explicit legacy-reader semantics for later switches (§6.1) | Switching was unspecified in #778 and stranded identities in the #838 shape | diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md new file mode 100644 index 000000000..eb0ffe8b5 --- /dev/null +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -0,0 +1,470 @@ +# Design Spec: Provider and Permission Model — Migration and Rollout + +**Status:** Draft +**Author:** Engineering +**Issue references:** #777–#781 (epic) +**Related specs:** `2026-07-30-pluggable-providers-design.md`, +`2026-07-30-permission-model-design.md` +**Last updated:** 2026-07-31 + +> **Context.** The provider/permission epic is a breaking change to a live +> identity system. PR #838's review showed that the riskiest part of such a +> change is not the new code but the transition: silent misconfiguration +> modes, undeclared behavior changes discovered by deleted tests, and no +> written statement of which pre-change behaviors were guaranteed to +> survive. This spec is that statement. Any implementation PR in the epic +> must reconcile its diff against §2's matrix and list every deliberate +> divergence in its description. + +--- + +## 1. Scope + +Covers the transition of existing deployments from the hard-wired EC / +device / geo behavior to the provider architecture and permission model. +Applies to every implementation PR in the epic, and to the operator-facing +migration guide that ships with the last of them. + +## 2. Behavior-preservation matrix + +For each decision the system makes today, the target behavior after the epic, +and whether that is a preservation or a declared change. **Silent changes are +defects.** PR #838 changed six of these without declaring any; each was +discoverable only because a deleted test had pinned the old behavior. + +| # | Decision (today) | After epic | Status | +| --- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| 1 | EU/GDPR request, no TCF consent → no EC | Same (opt-in baseline) | Preserved | +| 2 | US-state request, GPC/GPP/USP opt-out → no EC, existing EC expired + tombstoned, **even when a consenting TCF string is present** | Same (precedence §4 of permission spec) | Preserved — **must not regress** | +| 3 | US-state request, no signals at all → no EC (fail-closed) | Preserved by the recipe: the US rule is `requires_signal`, and the extended grant-signal class (permission spec §4) is what makes that possible — `granted` would allow no-signal traffic; a TCF-only grant class could not grant from GPP/USP values | Preserved under the recipe | +| 3a | US-state request, explicit GPP `sale_opt_out = false` or a US Privacy string that is present and not opting out (including "not applicable") → EC allowed | Same: these are grant-class signals satisfying `requires_signal` (permission spec §4) | Preserved — **must not regress** | +| 3b | US-state request, TCF record present and refusing Purpose 1 (no US opt-out signal) → no EC | Same: refusal beats coexisting non-TCF grant signals (permission spec §4, precedence 3–4) | Preserved | +| 3c | Consent-record conflict modes (restrictive/permissive/newest), expiry, KV fallback, proxy mode | Each row of the normalization matrix (permission spec §4.4) is individually marked preserved or changed there; changed rows: malformed-present now blocks acquisition | Per §4.4 matrix | +| 3d | Valid + expired consent records: conflict resolution runs first and can select the expired record | Expired sources drop **before** conflict resolution (permission spec §4.4 pipeline) | **Changed (declared)** | +| 3e | Only the GPP sale field (and USP) is consulted; `SharingOptOut` / `TargetedAdvertisingOptOut` are ignored | All three fields enforced per the §4.5 mapping (targeted-advertising affects P4 only, never destructive) | **New enforcement, declared** — opt-out effects are more protective; the same fields' not-opted-out values can also **newly grant P4**, which is not (both effects classified in permission spec §4.5) | +| 3f | Non-privacy-state US traffic (e.g. Wyoming) is non-regulated → EC allowed | Same: policy enumerates `US/` rules for configured privacy states; country-level `US` resolves non-regulated (permission spec §3.4) | Preserved — **must not regress** (a country-wide `US = "us-opt-out"` rule would deny all of it) | +| 3g | Graph rows persist JA4 class, H2 fingerprint hash, and buyer-facing quality metadata | Discontinued for new rows; v1 values retained read-only, never egressed, dropped at rewrite (providers spec §6.3) | **Changed (declared)** — more protective | +| 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB, with citation and sign-off | Declared change (if made) | +| 5 | No country resolvable (geo unavailable) → no EC (fail-closed) | `default_country` baseline, constrained by permission spec §5.3 so the fail-open combination cannot occur silently | Declared change, guarded | +| 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, existing identity never tombstoned | Refusal now blocks _new_ grants everywhere (permission spec §4, precedence 3) — a declared, more-protective change. Existing identity is still **never tombstoned** where the baseline is `granted` (permission spec §4.2) | Split: creation is a declared change; no-tombstone is preserved | +| 7 | Country resolved but not in any regulation list ("non-regulated") → EC created, EIDs pass through | Governed by the policy's `rules.default` entry (permission spec §5.4). The §5 recipe sets it to a `granted` baseline to preserve today's behavior; the protective example policy instead requires a signal worldwide — a declared operator choice between the two | Preserved under the recipe; declared change under the protective default | +| 8 | Opt-out signal (GPC/GPP/USP) **outside** US states → ignored today | Revokes and withdraws globally (permission spec §4 and §4.2 trigger 1) — including tombstoning, which is irreversible | Declared change, more protective, **irreversible** — see §6.4 | +| 9 | Fastly bot gate requires JA4 + platform class before KV-backed EC writes | Only with `[device] provider = "fastly"`; the `builtin` default is UA-only | Declared change with a documented restore step (§5) | +| 10 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"`. The neutral geo default flips **only** in the permission model PR, together with the §5.3 guard — never in an intermediate step where absent geo would fail closed and zero EC issuance (providers spec §11) | Declared change, sequenced, with a documented restore step (§5) | +| 11a | Raw EC egress on paths gated by the jurisdiction gate today (OpenRTB `user.id`, derived request IDs, page bids, EIDs, identify, pull sync — pull checks the live `EcContext` today) | Gated by the egress inventory (permission spec §7): bidstream and partner egress require both purposes, revocation exempt — at least as strict as today for every path | Preserved (strengthened); **must not regress** — PR #838 gated only EIDs and left `user.id` reachable | +| 11b | Proxy / click / Testlight forwarding extract the raw EC cookie/header **without** today's jurisdiction gate | Gated by the egress inventory (both purposes) | **New privacy hardening, declared change** — not preservation | +| 11c | Batch sync today only authenticates the S2S caller and checks live/tombstoned row state — it is **not** jurisdiction-gated | Gated by stored-provenance recompute (permission spec §7); legacy rows fail closed until backfilled | **New privacy hardening, declared change** — not preservation | +| 12 | EC generation succeeds without a configured identity-graph store | A minting provider requires an openable graph store at startup (providers spec §5, §6); pre-N+1 readiness step provisions it | **Breaking, declared** — graphless deployments must provision storage before upgrading | +| 13 | Cookies minted by graphless deployments have no graph row | Recognized rowless cookies are **expired and re-minted** through the ordinary graph-backed path (providers spec §5) — never adopted, since prefix-only verification cannot authenticate suffix variants; identity continuity is deliberately lost; withdrawal works without any row via the prefix-derived family ID | **Declared** — pre-existing identities restart rather than carry over | + +Rows 3, 4, and 7 are policy decisions, not code decisions: they belong in +the `[permissions]` policy review, made explicitly by maintainers — not +implied by an implementation. + +## 3. Identity stability guarantee + +Today's EC identifier is `{64-hex}.{6-char}` where the 64-hex part is +deterministic — `HMAC-SHA256(passphrase, normalized_ip)` — and the 6-char +suffix is **random per mint** (an existing test asserts two mints differ). +Full identifiers are therefore not reproducible by design, and no test may +pretend otherwise. What stability means, precisely, for a deployment that +selects `provider = "hmac"` and carries its passphrase over verbatim: + +- **The deterministic prefix is bit-identical.** Pinned known-answer + vectors: fixed passphrase + IP → exact expected 64-hex prefix, committed + so any divergence fails CI rather than rotating the production identity + base. +- **Existing cookies stay parseable.** Fixture `ts-ec` values minted by + the pre-epic code pass the provider's `parse`, and their graph rows + (keyed by the identifier verbatim) remain reachable — no row is orphaned. +- **The hash prefix keeps its semantics.** `ec_hash` remains the 64-hex + prefix, preserving both its stability and its deliberate collision across + identifiers minted from the same IP — the property IP-cluster trust + counting depends on (providers spec §3). +- **Cookie name, attributes, and max-age are unchanged** (the domain + remains config-derived, as today). + +## 4. Configuration migration + +Old shape: + +```toml +[ec] +passphrase = "example-passphrase" +``` + +New shape: + +```toml +[ec] +provider = "hmac" + +[ec.providers.hmac] +passphrase = "example-passphrase" +``` + +Requirements: + +1. **The transition has a dual-read release; loud rejection comes one + release later.** Today's binary _requires_ `[ec] passphrase` and — via + `deny_unknown_fields` — _rejects_ `[ec] provider` and + `[ec.providers.*]`; a binary that rejects the old shape outright would + mean **no config both binaries accept**, and a config-store fleet + cannot flip config and binaries atomically. Sequence: + - **Release N+1 (dual-read):** accepts the old shape (mapping + `[ec] passphrase` to the `hmac` provider internally, logging a + deprecation warning per startup) _and_ the new shape. Ordering is + **strictly reader-first, never "either order"**: current binaries + reject the new shape (and reject the new `[permissions]` / `[device]` + / `[geo]` additions as unknown fields), so the config may flip only + after **fleet convergence on N+1 is confirmed** — binaries first, + convergence gate, then `ts config push`. A config mixing old and new + fields (`[ec] passphrase` alongside `[ec] provider`) is **rejected** + by N+1, not reconciled. **N+1 is a full semantic reader and + enforcer for every N+2 record kind — not a field preserver.** + Preserving unknown JSON does not chase aliases, consult family + revocations, honor suppression records, or fail closed on + provenance; an N+1 that merely preserved would, after rollback, + treat revoked identities as live (aliases are reserved-future with + the rewrite deferral, providers spec §6.1). N+1 must also **write** + the safety-critical record kinds — family revocation and + suppression — not only read them: a withdrawal arriving on a + rolled-back N+1 fleet must still revoke. + + **N+1's identity-write behavior is v1, explicitly** — this resolves + what was an impossible trilemma (write rows without provenance, + violating active-after-commit; write provenance, violating the + N+2-only writer boundary; or stop minting, an undeclared outage): + N+1 **keeps minting v1 rows with today's semantics**, and the new + active-after-commit/provenance contract activates **with the N+2 + writer**, not before. Likewise the permission model itself: + **old-shape config on N+1 runs the pre-epic consent gate + unchanged** — dual-read means dual-behavior — so the compiled + protective fallback cannot flip behavior mid-convergence before the + operator pushes the new-shape policy; the new model engages only + with new-shape config. The interim is declared as sign-off item 20 — with one + boundary that does **not** wait for N+2: once new-shape config is + active, **context-free partner egress (batch sync) on N+1 fails + closed for rows without provenance**, exactly as the permission + spec's legacy rule requires. Otherwise N+1 would mint a P1-only v1 + row under the new model and then release it through today's + row-state-only batch check — the fail-closed rule cannot activate + later than the model it protects. Live-request paths keep v1 + semantics until N+2. + + Rollback tests therefore run the family-revocation and suppression + paths — read **and write** — plus v1-minting behavior, on N+1 + against N+2-written data. **Rollback is binaries-first too, in the other direction** — + N+2 → N+1 binaries roll back keeping the new config (N+1 reads it + fully; reverting config first would hand the old shape to N+2 + binaries that reject it) — **with one structural rule that makes it possible at all**: + providers are compiled into the composition root — there is no + dynamic provider ABI — so an N+1 binary can only read what it + shipped with. Therefore **every provider selectable in release R + must ship compiled-in (dormant: registered, parseable, + configurable, not selectable as writer) in R−1**; adopting a + genuinely new provider gets its own reader-first rollout, exactly + like the epic itself. With that rule, **schema rollback and provider rollback are + distinct sequences**: schema rollback is binaries-first (above); + **provider rollback is config-first** — a fleet whose config + _selects_ the new provider as writer cannot roll binaries first, + because the older binary rejects that active writer even while + containing its dormant code. The order: switch the current fleet's + writer back to the older provider (retaining the new one in + `legacy_providers`, satisfiable because N+1 physically contains the + code), converge, then roll binaries. + N+1 additionally **rejects writer selections whose provenance it + cannot yet encode** — new-writer adoption waits for N+2, so no row + is minted that N+2 would misclassify. Every new + config section introduced by the epic follows this same + compatibility rule, not only `[ec]`. + + - **Release N+2:** rejects `[ec] passphrase` at startup with a message + naming the new location — not a generic unknown-field error + (implementation note: producing the actionable message means keeping + a deprecated `passphrase` field whose presence triggers the custom + error). + +2. **Revocation-eligible storage is a per-adapter gate, and ungated + adapters migrate stateless.** Identity features require the adapter's + strong-consistency rows in the capability matrix (providers spec §7) + to be green: today that means Fastly must _verify_ its KV read + semantics, Cloudflare must wire a Durable-Object-class primitive, and + Spin must wire storage at all. Until an adapter passes the gate, its + migration fixture is **explicitly stateless** (`provider = "none"`, + no `[permissions]`-gated identity features) — calling an HMAC fixture + "valid" on an adapter that must reject identity features at startup + would make the required fixtures self-contradictory. Whether ungated + adapters go stateless or block the release is product sign-off + item 12. +3. **Graph-store readiness precedes everything.** Today the graph store + is optional and EC generation succeeds without one; the epic's + no-active-until-commit invariant (providers spec §5) makes it + mandatory wherever a minting provider is configured — so a currently + valid graphless HMAC deployment would **startup-fail on N+1's + dual-read mapping** without a preparatory step. The migration + therefore begins with a **pre-N+1 readiness step**: provision and + verify an openable graph store (and confirm the adapter's capability + row supports the features in use, providers spec §7) _before_ rolling + N+1. This is a **declared breaking change** for graphless deployments + (matrix row 12), not a side effect discovered at boot. +4. **The graph schema change is expand-contract, in lockstep with the + binary sequence.** New rows carry fields v1 rows never had — provider/ + version, per-permission grant evidence, policy revision, family ID, + rewrite links — and two failure modes must be engineered away: a naive + schema-version bump makes old readers fail closed on new rows, and an + old worker that reads, modifies, and reserializes a row **silently + drops** fields it does not model. The sequence shares the config + release names: **N+1 is the reader/preserver release** — it understands + the new fields and preserves unknown keys **semantically** through + read-modify-write (values round-trip; byte-identical JSON is neither + required nor achievable through a structured serializer — and a + genuinely pre-N+1 worker cannot preserve at all, which is exactly why + the floor exists); after the **fleet-convergence gate**, **N+2 + activates the writer** and begins emitting the new fields. **The rollback floor is crossed at N+2 writer activation itself** — an + observable deploy event, recorded in the **deployment-metadata + primitive** (providers spec §7 capability row; the existing + config-store interface exposes ordinary put/delete and cannot express + a monotonic floor) with a specified protocol, not an assertion: the + marker lives in a dedicated namespace outside rollbackable config; + the **first N+2 instance to activate creates/advances it via + create-or-CAS** (the creation race resolves to one winner), **reads + it back, and only then enables new-format writes**; every binary + reads the floor at startup and a binary below the floor **fails + startup**; an unreadable floor fails closed (writer stays disabled). + Floor-in-rollbackable-config would let "restore the previous config + version" erase the marker after new-format rows exist — exactly the + state it guards — and "any new-format row exists" is a fact no + operator can disprove. Below-floor rollback is + prohibited from that marker on; a pre-floor binary would silently + strip the new fields from every row it touches. + Rows carry the existing `v` schema discriminator; backfill is lazy via + live requests (the same pass that backfills legacy provenance, + permission spec §7) — and, critically, **withdrawal never depends on + backfill**: the family ID for an untouched v1 row is derived + deterministically (permission spec §4.3), so a first-post-upgrade + GPC request withdraws correctly with zero migrated state. Mixed-version tests with stated expected results: + N+1-reader/old-row → full function; old-reader/new-row → v1 semantics, + new fields untouched if read-only, preserved semantically if + read-modify-write on N+1, **test-proven lost on pre-N+1** (documenting + why the floor is a floor); N+2-reader/N+1-written-row → full function. +5. **Half-migrated fails loud.** A `[ec.providers.hmac]` block with no + `provider = "hmac"` selector is a startup error (providers spec §6). In + PR #838 this configuration — the exact state an operator following the + docs reaches if they miss one line — validated green and silently minted + zero ECs. +6. **PR #838-era keys fail loud.** `provider = "host-signals"` (shipped by + PR #838, deliberately not carried into this epic — providers spec §2) + and `provider = "client-fixed"` are unknown keys and rejected like any + other, so a config written against the PR #838 example cannot silently + select a provider that no longer exists. +7. **Provider switches go through legacy readers.** Changing + `[ec] provider` on a deployment with live identities requires listing + the outgoing provider in `[ec] legacy_providers` (providers spec §6.1) + so existing cookies keep resolving and stay withdrawable; the guide + documents the switch sequence and the retirement/cleanup step that ends + it. +8. **The example config ships the migrated happy path**, uncommented: + `provider = "hmac"` with its block, `[geo] default_country`, and (for + Fastly) the behavior-preserving `[device] provider = "fastly"` and + `[geo] provider = "platform"` lines present with a comment stating what + removing them changes. PR #838's example shipped the passphrase block + uncommented with the selector commented out — steering operators directly + into the silent-stateless state. +9. Every misconfiguration in the providers spec §6 table fails at + **startup**. Request-time failure for a configuration error is a defect. +10. Validation is split into two named layers, because "the same + validation at push and startup" is not implementable: **structural + validation** (schema, types, `[permissions]` policy — permission + spec §3.3) runs at `ts config push` and again at startup; + **deployment validation** (adapter capabilities, store bindings, + store openability — a structurally valid selection can still be one + an adapter must reject) runs at startup, where those facts exist. + Push may additionally pre-check deployment facts when given a + **machine-readable adapter capability profile** (the providers §7 + matrix, serialized), but startup remains the authority. + +## 5. Minimal-divergence migration recipe (operator-facing) + +"Keep exactly today's behavior" is not fully achievable, and the recipe's +name says so. The unavoidable divergences, enumerated (each also a matrix +row): global opt-out honoring (row 8); refusal blocking new grants +everywhere (row 6); newly enforced GPP sharing/targeted fields, which can +also **grant** P4 where nothing granted before (row 3e); the FR +unresolved-geo fallback, where valid TCF consent can grant while today's +unresolved-geo path always denies (row 5); malformed-present blocking +acquisition (§4.4); proxy-mode opt-out extraction; and the batch-sync +provenance gate (row 11c). Everything else the recipe preserves. + +The migration guide (a new `docs/guide/` page, linked from the release notes) +gives one copy-pasteable recipe per adapter for the minimal-divergence +posture — **branching on capability eligibility**: adapters passing the +revocation-storage gate get the HMAC + graph fixture below; ungated +adapters get the explicitly stateless fixture of §4.2, and no universal +HMAC requirement contradicts that: + +The recipe is a **complete, valid TOML fixture per adapter, committed to +the repository** (e.g. `docs/guide/fixtures/migration-preserving-fastly.toml` +and siblings) and included in the guide verbatim — never described as a +textual delta against the example file. Per-adapter because a single +fixture cannot be: `[device] provider = "fastly"` is Fastly-only, and +`[geo] provider = "platform"` varies by host — Cloudflare **does** +support platform geo but resolves **country only, no region** (per the +providers spec adapter matrix), which changes state-level US privacy +outcomes and engages the declared regionless degradation; Axum and Spin +have no platform geo and reject the selection (providers spec §6). Each +adapter's fixture carries the selections valid for it, and each is +CI-validated against its adapter. (An earlier draft said "copy the example table, +then set `[permissions.rules] default`" — but the copied table already +declares `[permissions.rules]`, and reopening a TOML table is a parse +error; a prose delta cannot be validated, a committed fixture can.) The +fixture contains, in one document: + +- `[ec] provider = "hmac"` with its passphrase block, **and the + identity-graph store configuration** — selecting a minting provider + without an openable graph store is a startup error (providers spec §6), + so a fixture omitting it would not start; +- `[device] provider = "fastly"` (Fastly deployments: preserves the JA4 + bot gate); +- `[geo] provider = "platform"` and `default_country = "FR"` (per-request + jurisdiction detection preserved; the FR default is a **protective + opt-in fallback**, not fail-closed — valid TCF consent still grants, + where today's unresolved-geo path always denies); +- (Fastly fixture; other adapters substitute their valid selections) + the full `gdpr-eu` / `gdpr-uk` / `us-opt-out` groups and country rules + from the example policy (US as `requires_signal` with the grant-signal + class — §2 rows 3–3b), plus the `non-regulated` group with + `rules.default = "non-regulated"` (row 7). Operators who prefer the + protective worldwide default use the example file itself instead. + +A partial policy is a trap the first draft of this spec fell into: a +`[permissions]` section containing **only** the permissive +default — with no GDPR/US rules — sends _every_ jurisdiction, France +included, to the permissive fallback, because `default_country` selects a +rule like any other country and finds none. Each committed fixture is +therefore always complete, and CI pins it: **the fixture file itself** is +loaded and run through the complete §4.1/§4.2 decision matrix of the +permission spec, asserting per-jurisdiction outcomes match the pre-epic +gate for every preservation row of §2. + +The guide separately documents the neutral configuration and what it does +_not_ do, states explicitly that `default_country` alone does not replace +geo lookup, why the no-geo combination requires the explicit acknowledgment +flag (permission spec §5.3), and that no recipe preserves row 8 of §2 — the +global honoring of opt-out signals is unconditional. + +## 6. Rollout sequence and observability + +1. Implementation PRs land in the epic's order (providers spec §11: + providers first with the geo default held at today's behavior, the + permission model PR flipping it together with its guard); each PR is + reviewable against §2 in isolation and states which rows it touches. +2. Before/after deploy, operators watch **EC issuance rate** and EID + attachment rate; the migration guide names these as the canary metrics, + because the failure mode of a bad migration is a silent drop to zero (or a + silent grant to everyone), not an error rate. The full metric set, each + with a stated healthy range: geo lookup-failure/fallback rate (permission + spec §5.2), raw-egress denials by path, tombstone family retries, + legacy-reader hit rate, and cluster-fallback engagements. Two of these carry thresholds, not just ranges: + legacy-reader hits at zero for a **quiet period no shorter than the + maximum cookie/row lifetime plus rollout skew** is the **only + retirement-readiness** bar for a + legacy provider ("trending to ~zero" is not evidence; a yearly visitor + is not churn), (rewrite-based backfill and its metrics left with the rewrite + deferral). The telemetry set also includes: graph read/commit failures, + stored-provenance denials, schema-migration failures, and + replay-reservation recoveries. **Each rollout-gate metric ships with a + threshold, an evaluation window, and a named action** (pause rollout / + roll back / block retirement) in the migration guide — a metric with a + "healthy range" but no action is dashboard decoration; the two already + specified (legacy-reader quiet period, rewrite failures) are the + pattern the rest follow. +3. Startup logs always print: selected provider per concern, whether geo is + live, the effective default baseline, and the count of granted-without- + signal permissions. One line, greppable, stable format. +4. **The batch-sync coverage dip is a gated rollout stage, not a + notification.** Provenance-coverage thresholds are normative gate + criteria: the guide defines a target coverage level and evaluation + window; recovery stalling below threshold for the window triggers the + **pause action** — investigate backfill (traffic mix, dormant rows), + never disable the gate; and staging is explicit: provenance + **writing** begins the moment N+2 activates, enforcement is already + in force (there is no fail-open stage), so the only stageable knob is + partner communication and the cleanup cadence for rows that never + recover. + Because legacy rows fail closed for batch updates until backfilled + (permission spec §7), batch-sync acceptance drops toward zero at + cutover and recovers along the live-traffic backfill curve. The + **provenance-coverage metric** (share of active rows carrying + provenance) is the gate signal; operators notify batch-sync partners + of the transient rejection rate. There is no fail-open shortcut — the + alternative (grandfathering pre-epic identities past the permission + model) is rejected in the permission spec. +5. Rollback is config-only where possible: reverting to the previous + config version restores the previous behavior on the previous binary. The + irreversible artifacts are enumerated — not "one": **family + revocation records and member tombstones** (no recovery; that is + their purpose), the **schema-floor marker** (write-once by design; + no administrative clear), and **sticky timestamp-less suppression** + (administrative clear procedure documented in the guide, requiring + recorded operator intent). Withdrawal tombstones — which is why the + withdrawal triggers (permission spec §4.2) are exhaustive, why partial + withdrawal failure has an explicit tombstones-first, browser-retries + contract (permission spec §4.3), and why §2 rows 6 and 8 call out + tombstoning explicitly. Two operational procedures are documented in the + guide, not automated: cleanup of identities minted before a policy + tightening (permission spec §4.2 trigger 3), and retirement of a legacy + reader after a provider switch (providers spec §6.1), which is the + deliberate end of the identities only that reader can resolve. + +## 7. Documentation deliverables + +- Migration guide page (§5), linked from `CHANGELOG.md` and the release + notes. +- `configuration.md` documents **every** valid `provider` value for all + three concerns, the full `[permissions]` schema, and environment-variable + overrides only if they actually work in production builds (in PR #838 the + documented `TRUSTED_SERVER__EC__PROVIDER` override existed only under + `#[cfg(test)]`). +- The permission model page states the §4 precedence rules of the permission + spec verbatim — operator docs and normative spec must not diverge on + precedence, and prose like "signals are mapped as a grant or a revoke" + without stating which wins is insufficient. + +## 8. Product decisions requiring explicit sign-off + +These are decisions this spec set makes that #838 had not already made (or +made differently). **Implementation is blocked while any row is `open`**; +each row needs an owner, a status, and a link to its decision record. +Decision records live as files under +`docs/superpowers/specs/decisions/` (one per row, `NN-title.md`, +recording the decision, the deciders, and the date) — the table links +them as rows close; an unratified row reverts to open, not to silently +implemented. + +| # | Decision | Where | Owner | Status | +| --- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | --------------------- | ------------------------------------------- | +| 1 | Opt-outs honored globally; destructive ones irreversibly withdraw outside the defining jurisdiction | permission §4, §4.2 | maintainers + legal | open | +| 2 | Sale opt-outs (GPP, USP) control both P1 and P4 and destroy the identity | permission §4.5 | maintainers + legal | open | +| 3 | Sharing / targeted-advertising fields: opt-outs remove P4 and retain the stored identity; the same fields' not-opted-out values **newly grant P4** | permission §4.5 | maintainers + legal | open | +| 4 | US contextual auctions continue during opt-out, identity removed | permission §7 | maintainers + product | open | +| 5 | Regionless US traffic treated as non-regulated unless the operator opts into country-wide gating | permission §3.4 | maintainers + legal | open | +| 6 | Full consent strings continue downstream; raw consent snapshots retained in rows for audit | providers §6.3 | maintainers + legal | open | +| 7 | Legacy batch-sync traffic rejected until live-browser provenance backfill | this spec §6.4; permission §7 | maintainers + product | open | +| 8 | Proxy / click / Testlight forwarding newly gated by P1 ∧ P4 | §2 row 11b | maintainers | open | +| 9 | Integration cookie operations — deferred out of the v1 hook with the full read/use/withdraw model as entry bar | hook §3 | maintainers | superseded by descope (ratify the deferral) | +| 10 | Session-cookie exemption question | hook §3 | maintainers + legal | deferred with item 9 | +| 11 | A single failed destructive-revocation **or suppression** write may leave S2S identity use live **indefinitely** for a never-returning visitor (no durable external retry queue) | permission §4.3 | maintainers + legal | open | +| 12 | Adapters without revocation-eligible storage migrate **stateless** rather than blocking the release | §4.2 of this spec | maintainers + product | open | +| 13 | Batch-sync acceptance dropping toward zero at cutover, with dormant identities having no automatic recovery path | §6.4 of this spec | maintainers + product | open | +| 15 | **Epic descope**: client-cycle spec demoted to deferred-informative; `rewrite_legacy` cut to a recorded deferral; hook ships headers-only | client spec status; providers §6.1; hook §3 | maintainers + product | open | +| 16 | Sticky opt-out for timestamp-less sources: a GPP/USP-only re-consent does not restore authority without a timestamped grant | permission §4.3 | maintainers + legal | open | +| 17 | Explicit N/A values are grant-class and can newly authorize personalized advertising | permission §4.5 | maintainers + legal | open | +| 18 | Permissive `default_country` remains in effect during prolonged geo-provider failure (metered residual) | permission §5.2 | maintainers + legal | open | +| 19 | Mixed policy revisions during rollout can produce irreversible destructive outcomes on part of the fleet | permission §5.5 | maintainers + product | open | +| 20 | N+1 interim: v1 minting semantics and pre-epic gating persist under old-shape config until N+2/new-shape | migration §4.4 | maintainers + product | open | +| 21 | Rowless legacy cookies are expired and re-minted **without continuity** (prefix-only verification cannot authenticate the suffix; adoption would let suffix variants mint unbounded rows) | providers §5 | maintainers + product | open | +| 22 | Device fingerprint (JA4/H2) processing authorized by operator selection, with the boolean classification persisted — collection purpose, retention, downstream visibility, and the vocabulary-extension boundary | providers §5 | maintainers + legal | open | +| 23 | DataDome security exemption: tag injection, cookie/ClientID read, vendor egress, and cross-integration visibility operate outside the permission model as a ratified security-purpose carve-out with owned cookie names, scope, and withdrawal semantics | hook §4a; permission §7 | maintainers + legal | open | +| 24 | Malformed/absence-caused suppression clears on any newer valid grant (non-sticky; opt-out stickiness applies only to opt-out causes) — and an active suppression **overrides a `granted` baseline**: one malformed request denies later no-signal requests until valid evidence clears it, and a policy-baseline grant alone never clears | permission §4.3, §4.1 | maintainers + legal | open | +| 14 | Policy tightening never reuses stored refusals destructively — a fresh, live post-change refusal is required (the spec decides this; ratify it) | permission §4.2 trigger 2 | maintainers + legal | open | diff --git a/docs/superpowers/specs/decisions/README.md b/docs/superpowers/specs/decisions/README.md new file mode 100644 index 000000000..fd3d0e1e2 --- /dev/null +++ b/docs/superpowers/specs/decisions/README.md @@ -0,0 +1,6 @@ +# PR #986 product-decision records + +One file per open row of the migration spec §8 sign-off table +(`NN-title.md`), recording the decision, the deciders, and the date. The +table links each record as its row closes; a row without a record here is +open, and implementation is blocked while any row is open. diff --git a/docs/superpowers/specs/pr986-review-ledger.md b/docs/superpowers/specs/pr986-review-ledger.md new file mode 100644 index 000000000..e58443d90 --- /dev/null +++ b/docs/superpowers/specs/pr986-review-ledger.md @@ -0,0 +1,212 @@ +# PR #986 review-finding ledger + +Disposition of every review finding against the provider/permission spec +set, by round. Statuses: **fixed** (commit noted) · **reapplied** (fix was +lost to a failed edit batch and re-landed — the round-4 script loss is +called out where it happened) · **partial → refixed** (a later round showed +the fix incomplete; both commits noted) · **superseded** (descope or a +later design change removed the surface) · **deferred** (moves with a +deferred feature; recorded as its entry bar) · **open** (sign-off table, +migration spec §8). + +Commits: R1 `a35f2ca78` · R2 `9886091e5` · R3 `5c8c2e893` · R4 `2b4d776b6` +· R5 `de70ca931` · R6 `c8b4b849e` · R7 (this commit). + +## Round 1 — adversarial self-review (22 findings) + +All 22 fixed in R1, three later shown partial and refixed: geo/device +gating circularity (refixed R3 — device half was wrong again), identity +stability vectors (refixed R3 — random suffix), §5.3 citations (fixed R1). +Policy moved YAML → TOML in R1 (maintainer decision). No open remnants. + +## Round 2 — first maintainer review (15 blocking + 1 + 4) + +| Finding | Status | +| -------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- | +| B1 raw-EC egress ungated | fixed R2; egress table concrete R3; typed R5; scoped types R6 | +| B2 recipe grants everywhere | fixed R2; fixture-not-delta R4; per-adapter R5; minimal-divergence R6 | +| B3 blanket gate blocks withdrawal | fixed R2 (split gate, spy test) | +| B4 provider switch strands identities | fixed R2 (legacy readers); rewrite portion superseded R7 (descope) | +| B5 graph-key/prefix incomplete | fixed R2; literal-prefix R3; namespace R3; core-constructed R5→R6; delimiter R7 | +| B6 device gating not circular | fixed R2; reasoning corrected R3; qualifier R5 (reapplied R6 after batch loss) | +| B7 withdrawal trigger contradiction | fixed R2 (requires_signal ∨ denied) | +| B8 withdrawal storage failure | fixed R2; family record R3; idempotent families R4; unbounded residual honesty R7 (sign-off 11) | +| B9 signal normalization missing | fixed R2 (subjects); outcomes R4; preserved semantics R5; state machine R6 | +| B10 auction class inference lossy | fixed R2 (regime); dispatch matrix R4; raw arm R5; persisted-TCF arm R7 | +| B11 no-geo guard too narrow | fixed R2 (all jurisdiction consumers); cookie consumer deferred R7 | +| B12 validation incomplete | fixed R2; rules.default/dupes R3; region assigned R5 | +| B13 Sec-Fetch-Site insufficient | fixed R2 (exact Origin / CSRF); deferred with client spec R7 | +| B14 replay unmitigated | fixed R2; session binding required R4; ownerless mode removed R7 | +| B15 unbounded inputs | fixed R2; exact limits R5; media-type matching R6 — all deferred with client spec R7 | +| ❓ issue contradictions | fixed R2 (divergence tables per spec) | +| Hook &mut HeaderMap + framing headers | fixed R2 (structured ops, reserved list) | +| 4 non-blocking (geo residual, eligibility matrix, cluster capability, deterministic entropy) | all fixed R2 | + +## Round 3 — second maintainer review (15 blocking + hook + gaps) + +| Finding | Status | +| --------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- | +| Permission algebra can't preserve US | fixed R3 (grant class); regime-scoped R4; field-scoped R5→R6 | +| Auction dispatch undefined | fixed R3 (regime matrix + fallback regime) | +| Normalization delegated | fixed R3→R5 (in-spec outcomes, preserved semantics R6) | +| Egress inventory incomplete/wrong | fixed R3 (path table, 11a/11b split); pull/batch split R5 | +| Batch sync no authority source | fixed R3 (stored provenance); full recompute R5; live-refusal rule R7 | +| Hook ordering vs cache protection | fixed R3 (invariant pass); snapshot monotonic R5; sticky axes R6; must-understand R7 | +| Universal case equivalence | fixed R3 (provider-declared fixtures) | +| Namespacing vs HMAC stability | fixed R3 (reserved grammar); all-versions-verbatim R7 | +| Legacy-reader gaps | fixed R3→R5 (namespaces, governing permissions, provenance, provider=none); rewrite parts superseded R7 | +| Graph prerequisites/runtime failures | fixed R3 (startup requirement, runtime matrix, active-after-commit); adoption path R7 | +| Withdrawal atomicity | fixed R3 (idempotent families); family record R4; suppression R6→R7 | +| No dual-compatible config | fixed R3 (dual-read N+1); ordering corrected R5; both-direction rollback R6→R7 | +| Source-agnostic sources dropped | fixed R3 (explicit deferral, divergence row) | +| Client resolve replacement/replay | fixed R3→R5; deferred with client spec R7 | +| Device contract / region default | fixed R3 (stale wording, US/CA default) | +| Hook API/eligibility | fixed R3 (ops API, eligibility matrix) | +| Completeness gaps (EcId bounds, non-cluster dedupe, mutator limits, per-row tests, runtime behavior, metrics, fail-closed labels) | all fixed R3→R5 | + +## Round 4 — architecture review (1 P0, 12 P1 groups, 9 P2, 3 P3) + +| Finding | Status | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | +| P0 legacy family-ID protocol hole | fixed R4 (deterministic derivation) | +| US GPP/USP not permission-scoped | fixed R4 (§4.5 table); regime scope R5; global aggregation R7 | +| Malformed state machine | fixed R4; six-state machine + matrix column R6 | +| TCF conflict nondeterminism | fixed R4; **wrongly "preserved" — refixed R6** (conjunction algorithm) | +| Raw-TCF arm excludes malformed | fixed R4 (raw-presence trigger); persisted fallback R7 | +| Stale authority renewal | fixed R4 (valid_until, snapshot replace); evidence classes R6; digest scope R6 | +| Degraded mode cross-instance | fixed R4 (state machine); **local-only honesty R7** (sign-off 11) | +| Workers KV "bounded" lag | **partial R4 → refixed R6/R7** (strong primitive required; single normative home) | +| Physical keys/schemas contradictory | fixed R4→R6; alias-at-source-key + versionless keys R7 | +| HMAC version via parse | fixed R4 (**lost in failed batch, reapplied R6**; provenance-resolved) | +| AuthorizedIdentity unscoped | fixed R6 (GraphOps/PartnerEgress); request-side redaction R7 | +| Client contract missing | fixed R5 (Acquisition modes); ClientResolveContext R7; deferred R7 | +| Revocation-wins impossible | fixed R6 (family epoch CAS); cross-key atomicity recorded open, deferred R7 | +| Rewrite storage/transactionality | fixed R5→R6; **superseded R7 — rewrite_legacy cut from epic** | +| Release/rollback inconsistent | fixed R5 (dual-read); reader-first R5; preconditions R6; floor marker + N+1 duties R7 | +| Graphless deployments | fixed R6 (readiness step); adoption path R7 (matrix row 13) | +| No concrete adapter matrix | fixed R5→R6; availability-vs-wiring split + Axum honesty R7 | +| Hook cookies bypass model | fixed R5 (write gate); **shown insufficient → cookie ops deferred R7** | +| Cache lattice invalid | fixed R5; **ordered lattice wrong → sticky axes R6**; full Vary R6 | +| P2/P3 (mint ordering, health machine, client limits, fixtures graph config, thresholds, N/A, KV pipeline, transitions, protective labels, FR wording, device qualifier, illustrative label) | all fixed R4–R6 (device qualifier reapplied R6 after batch loss) | + +## Round 5 — package re-review + +All 15 P1 and 4 P2 dispositioned above where they refined earlier rows; +net-new: sections map (fixed R5; **registry-corrected R7**: +6, +24–27), +live-vs-stored refusal (fixed R7), suppression record (fixed R6; +completeness R7), fixture invalidity (stateless fixtures R7). + +## Round 6 — re-review at de70ca9 + +All 18 P1 and 4 P2 fixed in R6 except where R7 shows partials (tracked in +the R7 table below). Sign-off table with owners/status introduced R6. + +## Round 7 — current + +| Finding | Status | +| ------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Client page leg pre-gate | fixed (page-leg gating; deferred with client spec) | +| Cookie read/use/withdraw unmodeled | **cookie ops deferred out of v1 hook** (entry bar recorded; sign-off 9/10 updated) | +| Ownerless mode reintroduces fixation | fixed — ownerless mode removed outright | +| Graphless cookie adoption | fixed (adoption transaction, matrix row 13) | +| Rewrite retention lineage | superseded — rewrite_legacy cut; finding recorded as entry bar | +| Unreferenced provider blocks | fixed (startup error) | +| Fastly prefix-query delimiter | R7's per-adapter `:` delimiter was itself Fastly-rejected and non-portable → **refixed R8**: delimiter-free fixed-width grammar (class tag + registry provider code) | +| P2 alias/tombstone cluster counting | fixed (liveness/kind filtering; aliases reserved-future) | +| P2 push-vs-deploy validation | fixed (two named layers, capability profile) | +| P2 rollback floor unobservable | fixed (floor = writer activation, durable marker) | +| P2 cookie ownership uniqueness | integration-ID uniqueness kept; cookie ownership deferred with cookie ops | +| Still-open: GPP 6/24–27 | fixed (registry-complete map) | +| Still-open: state-over-national opt-out erasure | fixed (grants-only precedence) | +| Still-open: suppression completeness | partial R7 (ordering claimed without CAS; cause-list coverage; timestamp-less unrealizable) → **refixed R8** (CAS + version counter, delta coverage, sticky opt-out) | +| Still-open: family-epoch cross-key CAS | recorded as client-spec open question 0; deferred | +| Still-open: eventual rows vs alias guarantees | superseded (rewrite cut) | +| Still-open: 4-hop stranding | superseded (rewrite cut) | +| Still-open: N+1 enforce vs N+2 write boundary | fixed (N+1 writes safety-critical records) | +| Still-open: N+2-only legacy reader on N+1 | fixed (accepted in legacy position) | +| Still-open: no-geo guard cookie consumers | deferred with cookie ops (inventory row updated) | +| Still-open: persisted TCF in raw arm | fixed (TCF-sourced effective record triggers arm) | +| Still-open: request-side raw identity | asserted R7 without API/tests → **specified R8** (`RedactedRequestView`, enumerated strip set, same-PR migration, denied/withdrawn tests) | +| Still-open: RequestFilterEffects.response_headers | R7 fold-in would have **broken DataDome** (302/401/403/429 + cookies) → **refixed R8**: distinct core-owned security channel sharing validation + invariant layers | +| Still-open: must-understand | fixed (sticky set extended) | +| Still-open: Axum persistence overstated | partial R7 (still marked wired; head installs `UnavailableKvStore`) → **refixed R8** (not wired) | + +## Round 8 — re-review at 09e54e96 + +| Finding | Status | +| ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| P1.1 suppression monotonicity unprovidable | R8 partial (CAS ordered arrival, not evidence recency; creation still row-observation-dependent) → **refixed R9/R10**: evidence-recency transition table, cause-aware read-free creation, exempt decision read, boundary/retention/fenced clearing | +| P1.2 malformed/absent leave stale authority | fixed (suppression on every positive→unset delta, cause-agnostic) | +| P1.3 suppression-write failure "transient" | fixed (unbounded residual, shares sign-off 11, fault test) | +| P1.4 N/A double meaning | fixed (single rule: explicit N/A = grant-class, absent = nothing; sign-off 17) | +| P1.5 adoption = syntax-as-authentication | R8 partial (prefix verify cannot authenticate suffixes — adoption itself was unsound) → **refixed R9/R10**: adoption removed; rowless cookies expire-and-re-mint; prefix-derived family collapses suffix variants; `verify → VerifiedIdentity{version}` | +| P1.6 N+1 impossible write behavior | fixed (N+1 mints v1 with today's semantics; old-shape config runs pre-epic gate; new contracts activate at N+2/new-shape — sign-off 20) | +| P1.7 N+2-only provider readable by N+1 | fixed (providers ship compiled-in dormant one release early; reader-first per provider) | +| P1.8 cookie deferral contradictions | R8 partial (append-only-non-reserved remnant survived) → **swept R9/R10** | +| P1.9 DataDome fold-in breakage | R8 partial (channel kept but boundary open: untyped cookies, header pointers, ordering conflict) → **refixed R9/R10**: §4a closed boundary — typed owned-name cookie op, direction-scoped allowlists, decision-scoped representation, invariant-last ordering, atomic batches; sign-off 23 | +| P1.10 freshness metadata weakening | fixed (`Age`/`Date`/`Expires` reserved, rationale in-spec) | +| P1.11 client-cycle in normative core | fixed (Acquisition enum/ClientResolve/reservations removed from trait surface; `verify` added; deferred doc holds the rest) | +| P1.12 redaction unspecified | fixed (see corrected R7 row above) | +| P2.1 rewrite residue | fixed (tests, runtime row, metrics/retirement swept; alias schema marked reserved) | +| P2.2 delimiter portability | fixed (see corrected R7 row above) | +| P2.3 Axum matrix | fixed (see corrected R7 row above) | +| P2.4 adoption rejuvenation | fixed (migration-cutoff-bounded TTL; sign-off 21) | +| P2.5 stored cluster overcount | fixed (no persistence beyond inputs' lifetime) | +| P2.6 GPP applicability leftovers | fixed (MD/IN/KY/RI sentence removed; section-6 grants defined; regime-`none` row reconciled with applicability) | +| P2.7 mixed-revision divergence | accepted explicitly (sign-off 19) | +| P2.8 floor marker rollbackable | fixed (write-once/CAS deployment metadata) | +| P2.9 sign-off gaps | fixed (rows 16–21 added; rows 3 and 11 amended) | +| P2.10 duplicate integration IDs | fixed (startup rejection + test; current silent coalescing named) | +| P3 GPP version pinning | R8 partial (still implementation-enumerated) → **refixed R9/R10**: pinned to a named registry revision in the normative spec | +| P3 geo region vocabulary | fixed (ISO output or declared canonical mapping) | +| P3 stale fragments + ledger overstatement | fixed (this ledger corrected; hook remnants swept) | + +## Rounds 9–10 — dual review at ff1e113e + +R9 (10 P1, 13 P2, 1 P3) and the same-head re-audit R10 (9 P1, 9 P2, 1 P3) +are dispositioned together; R10's "previously open" list = R9's P1s, +tracked once. + +| Finding | Status | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- | +| R9-1 trait cannot mint (`generate` lost in an R8 edit) | fixed — restored with failure semantics | +| R9-2 / R10-open rowless HMAC unauthenticatable suffixes | fixed — expire-and-re-mint, prefix-derived family, `VerifiedIdentity{version}` | +| R9-3 missing capability rows (suppression CAS, create-if-absent, floor metadata) + global visibility | R9/R10 partial (abstract rows only — the concrete matrix had no cells) → **refixed R11**: concrete per-adapter cells for all five primitives | +| R9-4 CAS orders arrival not recency | fixed — evidence-recency transition table with per-cause clearing | +| R9-5 creation condition fails both directions | fixed — cause-aware read-free creation; policy-only tightening writes nothing; absence uses the exempt decision read | +| R9-6 suppression outside the authorization boundary | fixed — both constructors, fail-closed reads, retention rule, provenance-generation-fenced clearing | +| R9-7 N+1 new-shape S2S unsafe | fixed — context-free partner egress fails closed on provenance-less rows once new-shape config is active | +| R9-8 schema-floor protocol unspecified | fixed — dedicated primitive, create-or-CAS, read-back-then-enable, startup enforcement, fail-closed | +| R9-9 security-cookie exception reopens cookie surface | fixed — §4a typed owned-name operation, ts-\* rejected, sign-off 23 | +| R9-10 DataDome representation/ordering conflicts | fixed — decision-scoped representation; one global order, invariant last | +| R9-P2 batch (residual wording; deferred residue markers; registry + non-hex tags; cluster cap; retirement evidence; matrix/fixture branching; cutoff removed with adoption; irreversible artifacts enumerated; mixed-policy absolute removed; version pin; effect attribution; Set-Cookie remnants; network split) | all fixed | +| R9-P3 / R10-P2.8 eligibility rows (HEAD, 1xx, 204, 205, 206) | fixed — HEAD mirrors GET; others enumerated No | +| R10-1 future-dated TCF replay | fixed — beyond-skew rejected as malformed; digest-pinned first normalization | +| R10-2 mutable rows vs eventual/accretive claim | R9/R10 partial (accretive claim survived in the matrix) → **refixed R11** (claim deleted; eventual visibility only after generation-CAS) | +| R10-3 cluster refresh extends retention | fixed — absolute `expires_at`, remaining-lifetime writes | +| R10-4 mint version inside replaceable snapshot | R9/R10 partial (mutable snapshot still listed provider/version) → **refixed R11** (removed from snapshot and from permission §7's field list) | +| R10-5 suppression needs forbidden read | fixed — read-free signal causes; narrow exempt decision read for absence | +| R10-6 durability/retention not validated | fixed — per-class durability + max-retention capabilities, startup horizon proof | +| R10-7 CDN-header weakening | fixed — CDN cache fields reserved outright; stale-\* durations shrink-only | +| R10-8 DataDome request-header injection | fixed — direction-scoped allowlist, scoped upstream overlay | +| R10-9 DataDome identifier outside permission model | ratification — sign-off 23 | +| R10-P2 batch (semantic digests; malformed/absence clearing + sign-off 24; atomic security batches; field registry; 256-byte bound in normative spec; device sign-off 22; provider-vs-schema rollback; TOCTOU live-check) | all fixed | + +## Round 11 — review at 43422b5f + +| Finding | Status | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| P1 suppression recovery deadlock | fixed — `AuthorityRefresh` scoped write path; clear references app-level provenance revision | +| P1 eventual not-found → rowless misclassification | fixed — rowless only under the graphless-migration deployment flag with strongest-read existence check; otherwise indeterminate (no use, no mint, no expiry); rowless withdrawal writes nothing (prefix-family removed — it also would have revoked whole IPs) | +| P1 absence suppression misses fresh grants | fixed — authority-state record carries a strong positive-authority summary; absence decision never reads the eventual row | +| P1 concrete matrix omits mandatory capabilities | fixed — see corrected R9-3 row above | +| P1 mutable vs accretive | fixed — see corrected R10-2 row above | +| P1 suppression wire schema + generation ordering | fixed — full field schema; app-level monotonic revisions (Fastly markers detect change, no order) | +| P1 `generate` cannot carry mint version | fixed — `GeneratedIdentity { id, mint_version }` | +| P1 ClientID required and prohibited | fixed — positively enumerated allowlist including `X-DataDome-ClientID`, owner-scoped upstream overlay, egress under sign-off 23 | +| P1 cookie exception without lifecycle | fixed — concrete registration (name `datadome`, scope, attributes, 13-month ceiling, 4 KiB, owner-only read, deletion always); "ratified" corrected to _pending ratification_ | +| P1 ordering contradiction | fixed — one order: core → mutators → security → invariant; older DataDome doc marked superseded, update in done-when | +| P1 headers-only not permission-neutral | fixed — v1 field registry admits inert fields only; active-egress fields rejected; unknown fields fully rejected | +| P2 batch (inventory rows for the exempt read and AuthorityRefresh; TCF-vs-GPP source-specific digests; observation timestamps; granted-baseline suppression row in §4.1 + sign-off 24 expanded; snapshot field cleanup; indeterminate read errors; Axum cells unavailable; rewrite/backfill retirement alternative removed; GPP vendored snapshot; 304-safe metadata pass; Respond-first validation; CDN names enumerated; deferred residue bracketed) | all fixed | +| P3 batch (verbatim comment covers every hmac version; `Content-Language` example corrected; stale fragments swept; this ledger corrected) | fixed | +| Ratification note | decisions directory created (`docs/superpowers/specs/decisions/`), table text points to it; item 23 wording no longer claims ratification |