Repair selected output dates from retained raw data

HANDOFF.md

@@ -2,36 +2,37 @@
 
 ## Finished
 
-- Added a read-only comparison of an unsaved graph with its exact saved revision, including downstream outputs and write effects.
-- Added readable explanations of recorded run outcomes and eligible next actions; unavailable measurements remain unknown.
-- Exposed both operations through existing Control authentication and regenerated their public contracts.
+- Merged read-only graph change preview and recorded run explanations through protected PR #547.
+- Implemented selected-date output repair from retained raw data with single-task Cloud Run and BigQuery.
+- Reused durable run identity, cancellation, history, replay, lease fencing, and conditional cleanup.
+- Added preview/start contracts, publication measurements, and operator documentation with explicit support limits.
 
 ## Try It
 
-With Control running and a saved graph selected, POST candidate graph JSON to `/v1/projects/{project}/graphs/{graph}/change-preview` with its saved `If-Match` ETag. GET `/v1/runs/{run_id}/explanation` for a configured lifecycle's run. See `docs/control-contracts.md` for semantics and client compatibility.
+Use the API examples in `docs/control-contracts.md` and `docs/output-repair.md`. Preview a saved graph's UTC date interval with its `If-Match` ETag, then start `/repairs` with the same revision and an `Idempotency-Key`. A repair-capable runtime and existing partitioned outputs are required. Published clients do not yet expose this flow.
 
 ## Checks
 
-- 87 focused comparison, explanation, HTTP, OIDC, and contract tests passed.
-- Canonical strict typing passed across 513 files.
-- Ruff lint/format, generated Control contract drift, and diff checks passed.
-- Protected CI and exact-main verification remain pending for this implementation step. No live cloud workload was run.
+- Full local run: 2,429 passed, 56 skipped; all 28 loopback-server failures passed on retry with socket permission. The additional explanation test passed in its 17-test file.
+- Focused Control/runtime/executor/deployment selection: 538 passed, 21 skipped.
+- Strict typing passed across 516 files; Ruff lint/format, generated contracts, and diff checks passed.
+- PR #547 protected CI passed and merged as `099fb52`; exact-main CI is running. Repair protected CI and live provider qualification remain pending.
 
 ## Decisions
 
-- Keep static graph comparison separate from infrastructure deployment preview; no new persistence or provider reads.
-- Explain existing run evidence deterministically, with no new AI service or fabricated diagnosis.
-- Deliver real date-scoped output repair and the guided interface as subsequent steps of the same active objective.
+- Preview stays read-only; explanations use collected evidence and retain unknown measurements.
+- Repair leaves source extraction progress untouched and replaces only the selected dates, atomically per output.
+- Deliver one guided workflow in Druff; additional providers and broader graph execution are separate work.
 
 ## Remaining
 
-- Implement bounded output-partition repair from retained raw data, preserving normal extraction progress and unrelated output rows.
-- Integrate connect, configure, preview, run, and outcome in Druff; its repository exception is awaiting the user's answer.
-- Publish and consume a verified immutable producer artifact for the paired Druff client; existing published clients use their original bundle.
-- Complete protected PR, merge, and exact-main CI for each runtime change.
+- Complete the repair protected PR, merge, and exact-main CI.
+- Complete the bounded live repair check after the existing AWS billing session is restored; current aggregate cash exposure is unverified.
+- Integrate Druff's guided flow once the requested repository exception is answered.
+- Publish and consume the matching immutable contract/runtime artifacts; the complete guided journey is not delivered yet.
 
 ## Review First
 
-- `src/dander/control/change_preview.py` for graph comparison and write explanations.
-- `src/dander/control/run_explanation.py` for evidence-based outcome handling.
-- `docs/control-contracts.md` for the API and publication boundaries.
+- `src/dander/providers/bigquery/graph.py` and `tests/pipeline/test_date_repair.py` for scoped publication.
+- `src/dander/control/cloud_run_execution_backend.py` for execution binding and cleanup.
+- `docs/output-repair.md` for practical behavior and limitations.

docs/control-contracts.md

@@ -188,12 +188,15 @@ Run explanations only show counters when the runtime result summary has been col
 execution can succeed before those measurements arrive; missing measurements are not zero rows.
 Explanations distinguish skipped work, pending cancellation, failed execution, and confirmed
 results without guessing an error cause. Suggested actions honor the current run controls; the
-explanation does not perform them. Replay is another execution, not rollback or date-range repair.
+explanation does not perform them. Ordinary replay is another full execution, not rollback or
+date-range repair; replaying a repair run retains its recorded date window.
 Neither operation requires provider-specific browser logic or a new service dependency.
 
 Editors can request change previews; viewers can read run explanations when a lifecycle is wired.
 Regenerate consumers from the immutable producer artifact after its release. The guided Druff
-surface and date-scoped repair are separate follow-on work, not delivered by these two routes.
+surface is separate work, not delivered by these two routes. Current source also exposes
+`graph.repair-preview` and `run.repair` for eligible execution plans; see
+[selected output date repair](output-repair.md) for the request, supported scope, and evidence limits.
 
 ## Hosted Control service
 

docs/decisions.md

@@ -1,5 +1,18 @@
 # Engineering Decisions
 
+## 2026-10-01 — Repair output dates from retained raw data
+
+- Keep a repair window in the existing durable run trigger, preserving cancellation, replay,
+  history, and request identity. Preview reads only the saved graph and current execution plans.
+  The initial runtime is single-task Cloud Run with BigQuery; other backends reject repair.
+- Skip extraction and its saved progress. Compile current retained raw data, then replace only
+  the selected UTC dates in each existing output under the normal lease fence. Reject keys
+  crossing the interval boundary; publication is atomic per output, not across all outputs.
+- Bind the exact graph hash and dates to the native execution. A fixed runtime argument rejects
+  older images before work; conditional cleanup restores the Job configuration only while the
+  repair still owns it. Live provider qualification and the paired Druff client remain separate
+  delivery requirements, not claims implied by local tests.
+
 ## 2026-10-01 — Explain pipeline changes and runs through the existing Control boundary
 
 - Compare an unsaved graph with the exact saved revision in a pure, provider-neutral operation.

docs/output-repair.md

@@ -0,0 +1,61 @@
+# Repair selected output dates
+
+Current source implements date-scoped output repair for direct Cloud Run execution with a
+BigQuery warehouse. It rebuilds selected dates from the raw data **already stored** in the
+warehouse. It does not fetch source records, advance ingestion progress, or reconstruct a past
+source snapshot. The published RC19/RC20 client/runtime pair does not include this operation.
+
+## Preview, then start
+
+1. Read the saved graph and retain its `ETag`.
+2. POST the date window to `/v1/projects/{project}/graphs/{graph}/repair-preview` with that
+   `If-Match` value. Review the outputs, UTC date boundary, available execution environments,
+   and limitations. This preview does not read warehouse rows or start a workload.
+3. POST the same window to `/v1/projects/{project}/graphs/{graph}/repairs` with `If-Match` and
+   a new `Idempotency-Key`. When the preview offers more than one environment, select one through
+   `?environment=...`. Reuse the same key and body if a response is lost.
+4. Poll the returned run through `/v1/runs/{run_id}` and `/v1/runs/{run_id}/explanation`.
+   The status retains `repair_window`. Existing cancellation, history, and replay controls apply;
+   replay retains the window but uses the raw data available when it runs.
+
+For September 1 and 2, send:
+
+```json
+{"start_date": "2026-09-01", "end_date": "2026-09-03"}
+```
+
+The start date is included and the end date is excluded. For `TIMESTAMP` fields, dates are
+interpreted in UTC. An editor can preview; starting a repair requires the operator role.
+
+## Eligible outputs and write behavior
+
+Every configured output must declare a `DATE` or `TIMESTAMP` partition field and a scalar business
+key. Output tables must already exist. The first slice retains the graph's existing replace
+configuration, but repair publication replaces only the selected dates. It requires the normal
+BigQuery lease and fencing; sandbox execution and other warehouse/execution backends reject repair.
+Outputs may not overlap each other or their raw input tables.
+
+The runtime stages the current graph result before publishing. It checks selected keys and rejects
+a key that moves across the date boundary in either direction, including an undated counterpart.
+That failure leaves the affected output unchanged; choose a wider range or do a full rebuild.
+Rows outside the interval remain untouched. Empty selected results remove only selected dates.
+
+Each output publishes in one fenced transaction. If a later output fails, earlier successful outputs
+remain; retrying converges on the current raw data. Publication counts appear in run telemetry and
+the explanation. The existing ingestion counters stay zero because extraction was skipped.
+Computing the result can scan more data than the dates being replaced; the preview cannot quote
+row counts or cost without warehouse work.
+
+## Runtime compatibility and verification
+
+Control binds the exact dates and graph hash to the deterministic Cloud Run execution. The fixed
+repair contract argument makes an old runtime reject the request before pipeline work, rather
+than silently executing a full replacement. Adoption checks the exact execution payload. A
+different ordinary execution ignores a well-formed inherited selection; Control also restores
+the base Job settings after terminal repair while they still belong to that repair.
+
+Local checks cover date boundaries, unchanged surrounding rows, collision rejection, transaction
+rollback, lease fencing, no source/watermark access, replay identity, and Cloud Run adoption and
+cleanup. The row checks execute translated predicates against SQLite; they do not establish live
+BigQuery or Cloud Run qualification. A bounded synthetic live check and the paired Druff interface
+remain required before claiming the complete guided workflow is delivered.

docs/support-status.md

@@ -1,6 +1,6 @@
 # Current implementation and support status
 
-Updated September 30, 2026. This page is the current overview; dated decisions and acceptance
+Updated October 1, 2026. This page is the current overview; dated decisions and acceptance
 records preserve what happened on their named revisions. Update this page when a capability is
 integrated or a named release/profile changes status.
 
@@ -27,6 +27,8 @@ the final paused/no-drift state. These are dated observations, not a continuous
 | Control execution through Fargate, Cloud Run, or Dataproc | Implemented with typed PostgreSQL or S3 run storage | Experimental; one Control process per run-store schema. AWS compatibility flags remain available. |
 | Distributed Spark graphs | Bounded linear, keyed-join, and two-transform shapes implemented; retained August 27–28 runs include exact Fargate/Dataproc output parity | Experimental; the [execution history](control-execution-history.md) identifies each tested revision and bounded shape. |
 | Druff hosted authoring | Hosted Control connections, OIDC/PKCE, persisted graphs, and run lifecycle controls implemented | Experimental; retained browser checks used synthetic OIDC. They do not establish production identity-provider or HA qualification. |
+| Graph change preview and run explanation | Read-only comparison against the saved revision and readable recorded outcomes in current Control source | Requires a newly generated client from the matching immutable artifact; published Druff clients do not expose these operations. |
+| Date-scoped output repair | Current source implements retained-raw repair with single-task Cloud Run and BigQuery; local transaction, lifecycle, and transport tests | Experimental and not live-qualified or published. See [output repair](output-repair.md) for existing-table, date-boundary, and per-output transaction limits. |
 | PostgreSQL, Snowflake, Redshift warehouse adapters | Local conformance and named bounded live evidence | Experimental; the runtime compatibility matrix determines allowed state/warehouse pairs. |
 | Azure, OCI, Kubernetes profiles | Named lifecycle/qualification records | No general support promotion; consult each exact profile's evidence. |
 | PostgreSQL Control graphs/runs/schedules | Database conformance plus one real Cloud Run graph with crash recovery, cancellation, output, and history checks | Experimental; one local Control process, no supported hosted or multiple-replica topology. |