diff --git a/Dockerfile b/Dockerfile index bc4f26e..863e9d7 100644 --- a/Dockerfile +++ b/Dockerfile @@ -29,15 +29,16 @@ ARG TARGETARCH RUN XCADDY_SETCAP=0 CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH \ xcaddy build v2.11.4 \ --output /usr/bin/caddy \ + --replace golang.org/x/crypto=golang.org/x/crypto@v0.55.0 \ --replace golang.org/x/net=golang.org/x/net@v0.56.0 \ --replace golang.org/x/text=golang.org/x/text@v0.39.0 \ - --replace google.golang.org/grpc=google.golang.org/grpc@v1.82.1 + --replace google.golang.org/grpc=google.golang.org/grpc@v1.83.2 FROM scratch AS runner ARG SOURCE_REVISION=unrecorded ARG SOURCE_CREATED=1970-01-01T00:00:00Z -ARG DRUFF_VERSION=0.1.0 +ARG DRUFF_VERSION=0.2.0-rc.1 LABEL org.opencontainers.image.title="Druff" LABEL org.opencontainers.image.description="Static, provider-neutral Dander control-plane interface" diff --git a/HANDOFF.md b/HANDOFF.md index 86dfd58..4895948 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -2,34 +2,40 @@ ## Finished -- Merged current-Dander compatibility in PR #23 and passed exact-main CI at `34acec00dfc9aafe93a62b1e3ed2b1b807a66487`. -- Exercised Druff's production clients against protected Dander RC32 at `33801213ee8c1bab78b8847a4b86bd9b4b8d1c63`. -- Built and retained exact active and rollback multi-platform OCI artifacts with SBOM and provenance attestations. -- Verified both artifacts are source-free, non-root, read-only, route-complete, and clean under the protected Trivy policy. +- Added a hosted Choose → Configure → Review → Run → Results journey with one primary action and existing technical controls under advanced details. +- Bound unsaved change previews to the draft, graph address and revision; running requires the reviewed content hash to match the saved graph. +- Added recorded outcome explanations, missing-measurement handling and explicit retained-raw date repair with date/revision invalidation and safe retry keys. +- Preserved loopback editing, OIDC/capability gates, graph conflicts, logs, cancellation and replay; corrected the existing font variable and toolbar overflow. +- Documented the experimental repair boundary and added a read-only production-client observation test for an already-completed native repair. ## Try It -Resolve the retained artifacts with `oras resolve --oci-layout /Users/harrison/.codex/artifacts/druff/2026-08-24-release/druff-active-34acec00dfc9.oci.tar:active` or the matching `druff-rollback-55f1565f3cb8.oci.tar:rollback`. For Control API acceptance, start current Dander and run `DANDER_CONTROL_URL=http://127.0.0.1:8770 DANDER_EXPECTED_VERSION=0.9.0rc32 pnpm test:current-dander`. +In hosted mode, browse a graph, edit its source/output configuration, choose **Preview changes**, then **Save reviewed changes** and **Run reviewed graph**. Expand **Repair selected output dates** for an eligible saved graph. Existing operator configuration still supplies connections and deployed environments. + +For real HTTP contract checks, start an ephemeral Dander Control API and use `DANDER_CONTROL_URL=... DANDER_EXPECTED_VERSION=... pnpm test:current-dander`. For an accepted native repair, use the explicit project, graph and run ID variables documented in README with `pnpm test:repair-observation`. ## Checks -- PR #23 and exact-main run `32787930579` passed Frontend quality, Source-free container, and Secret scan. -- Current-Dander acceptance, contract/artifact checks, lint, types, format, 669 unit tests, and 11 Playwright journeys passed. -- Both exact artifacts passed reproducible-layer, promotion, archive, amd64/arm64 runtime/export, and Trivy 0.70.0 HIGH/CRITICAL plus secret checks with zero findings. +- Passed 685 unit/component tests, nine artifact checks, TypeScript, ESLint, Prettier and static build against the verified published RC33 wheel; independent contract regeneration reports no drift. +- Passed all 11 Playwright journeys, including reviewed save/run, date invalidation, repair submission and existing conflict/retry behavior. Browser OIDC/API responses are synthetic. +- Passed real HTTP acceptance against published RC34 Control: exact hashes, canvas round trip, unsaved preview/save, stale rejection and truthful unsupported execution. Final RC34 regeneration/drift, 29 contract tests, 18 journey/API tests, TypeScript and focused lint passed; its bundle is unchanged. +- Visually inspected the synthetic browser screenshot; font and horizontal overflow defects are corrected. A focused test also confirms late explanations cannot replace newer run status. +- PR #25 initially found three fixed high Caddy dependency findings. Pinned x/crypto 0.55.0 and gRPC 1.83.2; CI `36870826744` passed both architecture scans, reproducibility, runtime checks and frontend/secret checks. ## Decisions -- Active is merge commit `34acec00dfc9`; rollback is green PR head `55f1565f3cb8`; both share tree `ce829147ef3b`. -- Retain the public RC19 contract pin because protected RC32's bundle is byte-identical. -- Keep the release artifacts as local OCI archives; public registry publication remains a separate external action. +- Dander remains the only semantic/execution authority; no provider client or new auth infrastructure was added. +- Missing result documents display Unknown instead of interpreting default counters as measured zeros. +- Date repair rebuilds output partitions from retained raw data; it is explicitly experimental and does not recover historical source records. ## Remaining -- No Druff implementation or release-readiness gap remains. -- Publish or deploy the retained artifact only when a destination is explicitly approved. +- Candidate version is `0.2.0-rc.1`, paired with published RC34 Control and the unchanged contract digest in README. This does not promote the RC20 public beta. +- Observe the separately authorized native repair once its accepted run ID is available; no provider workload was launched by Druff tests. +- Complete protected PR #25, exact-main checks and candidate image publication with the final RC34 pin. ## Review First -- `tickets/DRUFF-31-current-dander-release-acceptance.md` -- `integration/current-dander-control.test.ts` -- `/Users/harrison/.codex/artifacts/druff/2026-08-24-release/records` +- `src/features/hosted-control/useGraphChangePreview.ts` and `DateRepairPanel.tsx` +- `src/features/hosted-control/control-api.ts` and `integration/` +- `tickets/DRUFF-32-guided-pipeline-journey.md` diff --git a/README.md b/README.md index 0ea8677..c39cf35 100644 --- a/README.md +++ b/README.md @@ -4,13 +4,15 @@ Front-end companion to **[Dander](https://github.com/harrisonoconnorhover/dander editor for Dander's canonical `PipelineGraph` files (drag/drop nodes, wire connections, configure sources/transforms/writes). -Druff opens and saves one graph file through Dander's localhost API. Dander owns parsing, -validation, conflict detection, and atomic filesystem writes. When the operator binds that graph -to one manifest pipeline, Druff can also validate it, manually run its already-deployed Cloud Run -job, and show compact execution/run-ledger status. A version-1 `dander.yaml` can still be imported -as a detached, one-way visualization. When Dander explicitly enables it with complete operator -inputs, Druff can also request a source-free candidate and display its non-applyable Terraform -plan; Druff never writes the manifest or applies infrastructure. +Druff connects to Dander's hosted Control API or its single-file localhost API. Hosted users sign +in, choose a graph, configure it on the canvas, review unsaved changes, save, run and inspect the +recorded outcome. Dander owns parsing, validation, revision conflicts, storage and execution. + +In localhost mode, an operator can bind the open graph to one manifest pipeline to validate it, +run its already-deployed Cloud Run job and inspect the run ledger. A version-1 `dander.yaml` can +still be imported as a detached, one-way visualization. When Dander explicitly enables it with +complete operator inputs, Druff can also request a source-free candidate and display its +non-applyable Terraform plan; Druff never writes the manifest or applies infrastructure. When that manifest pins installed connector plugins, Druff discovers their presentation-safe descriptors from Dander and adds them to the palette dynamically. The first dynamic connector is @@ -31,7 +33,7 @@ read-only instead of being rewritten. See `CLAUDE.md` and `steering/00-project-overview.md` for the full picture (why this exists, the module map, decision log). The accepted documentation-only Druff 1.0 architecture checkpoint is in `steering/03-control-plane-roadmap.md`. Druff's DTOs now come from Dander's published contract -bundle; hosted graph storage, identity, and run APIs remain later phases. +bundle; hosted graph storage, identity, and run controls are implemented by the static client. ## Stack @@ -57,11 +59,13 @@ pnpm install pnpm dev # http://localhost:3000 ``` -Contract output is generated only from the pinned public -`dander-platform==0.9.0rc19` wheel on PyPI. The generator verifies the wheel, manifest, every file, -and the whole bundle before writing output; it never reads a sibling Dander checkout. Current -protected Dander RC32 retains the exact same `io.dander.control.contracts/v1` bundle bytes, so the -published pin remains the authority instead of being relabeled from an unpublished checkout. +Contract output is generated only from the pinned +[`dander-platform==0.9.0rc34` GitHub integration-candidate wheel](https://github.com/harrisonoconnorhover/dander/releases/download/v0.9.0rc34/dander_platform-0.9.0rc34-py3-none-any.whl), +SHA-256 `d37c1d91c15cca9ce3d080af6dec384f7cd9cbc521dea253e10dc44772d35bad`. +The generator verifies the wheel, manifest, every file, and the whole bundle before writing output; +it never reads a sibling Dander checkout. This client requires the matching RC34 bundle. The +GitHub integration candidate does not promote Dander's public PyPI RC20 beta or qualify additional +provider profiles. ```bash pnpm contracts:check # re-generate in a temporary directory and fail on committed drift @@ -72,12 +76,12 @@ Release acceptance can additionally exercise Druff's production graph and operat against a locally running current Dander Control API: ```bash -# In the protected Dander checkout: -uv run dander control serve --ephemeral --project demo-project --port 8770 +# In an environment installed from the pinned RC34 wheel above (not the RC20 PyPI beta): +dander control serve --ephemeral --project demo-project --port 8770 # In Druff: DANDER_CONTROL_URL=http://127.0.0.1:8770 \ -DANDER_EXPECTED_VERSION=0.9.0rc32 \ +DANDER_EXPECTED_VERSION=0.9.0rc34 \ pnpm test:current-dander ``` @@ -133,6 +137,45 @@ cancel, and replay APIs. Dander remains authoritative for authorization and sema discards stale or mismatched responses and never receives provider credentials or native payloads. When `/bootstrap.json` is absent, the existing local-loopback workspace remains unchanged. +The hosted workspace guides one saved graph through **Choose → Configure → Review → Run → Results**: + +1. Browse a project and open a graph, or create a graph from the current canvas draft. Configure + source and output nodes in the existing inspector. Connections and deployed environments still + belong to Dander's operator configuration. +2. Choose **Preview changes** to compare the unsaved draft with its saved revision. The review + lists changed nodes and connections, downstream outputs, and the write behavior of every + output a full run will write. This comparison does not query rows or estimate a bill. +3. Choose **Save reviewed changes**, then **Run reviewed graph**. Edits invalidate the review; + Dander checks the saved revision again before accepting execution. Saving alone does not run. +4. Read the recorded outcome and suggested next action. Missing result measurements stay + **Unknown**, including server-default counters before a result document exists. Advanced + validation, deployment preview, logs, cancellation and replay remain available. +5. Expand **Repair selected output dates** to preview a saved graph's half-open UTC date window + (first date included, end date excluded). A compatible environment can rebuild those output + partitions from retained raw warehouse data. This does not fetch historical source records or + move normal ingestion progress. Dander determines eligibility and publishes the selected + ranges atomically; unsupported graphs/environments return an explicit explanation. + +Date repair is an experimental producer capability until its stated qualification gates are met. +A generated client and its Dander deployment must use the same published contract bundle. Browser +journey tests use synthetic HTTP/OIDC fixtures; they do not establish live provider success. + +To inspect an already-completed operator-owned native repair through Druff's real HTTP client, +without starting another workload: + +```bash +DANDER_CONTROL_URL=http://127.0.0.1:8876 \ +DANDER_REPAIR_PROJECT=repair-proof-rc34 \ +DANDER_REPAIR_GRAPH=repair-corrections \ +DANDER_REPAIR_RUN_ID=REPLACE_WITH_ACCEPTED_RUN_ID \ +pnpm test:repair-observation +``` + +This observation verifies the exact contract, saved graph, repair preview, status and explanation +for the September 10–12 synthetic fixture, including zero extracted rows, two output rows written +and four output rows affected. +The operator's separate provider evidence must establish warehouse results, watermarks and cleanup. + To enable the narrow operational controls for one graph that is already deployed, start Dander with its matching manifest pipeline and GCP project: diff --git a/e2e/hosted-graph-management.spec.ts b/e2e/hosted-graph-management.spec.ts index b99433c..2181a4a 100644 --- a/e2e/hosted-graph-management.spec.ts +++ b/e2e/hosted-graph-management.spec.ts @@ -1,9 +1,12 @@ import { expect, test, type Page, type Route } from "@playwright/test"; import { execFileSync } from "node:child_process"; +import { createHash } from "node:crypto"; import { createReadStream, existsSync, mkdtempSync, readFileSync, rmSync, statSync } from "node:fs"; import { createServer, type Server } from "node:https"; import { tmpdir } from "node:os"; import { extname, join, resolve } from "node:path"; +import type { RunStatusResponse, RunExplanationResponse } from "../src/lib/dander-contracts"; +import repairPreviewFixture from "../src/generated/dander-contracts/bundle/fixtures/graph-repair-preview.json"; import staticSecurityHeaders from "../artifact/static-security-headers.json"; import { DANDER_CONTRACT_BUNDLE_ID, @@ -38,6 +41,7 @@ type StoredRun = { runId: string; state: "queued" | "running" | "succeeded" | "canceling" | "canceled"; polls: number; + repairWindow?: { start_date: string; end_date: string }; }; type DurableControlState = { @@ -61,6 +65,8 @@ function graphDocument(name: string, nodeName: string) { id: "source", type: "source", name: nodeName, + config: { connector: null, endpoint: null }, + fields: [], visual: { position: { x: 20, y: 40 }, color: "green", icon: "database" }, }, ], @@ -79,12 +85,24 @@ function nextRecord( graph, document, revision: `"revision-${durable.revision}"`, - contentSha: durable.revision.toString(16).padStart(64, "0"), + contentSha: graphHash(document), createdAt, updatedAt: `2026-08-14T10:00:${String(durable.revision).padStart(2, "0")}Z`, }; } +function graphHash(document: unknown): string { + return createHash("sha256") + .update( + JSON.stringify(document, (_key, value: unknown) => { + if (value && typeof value === "object" && !Array.isArray(value)) + return Object.fromEntries(Object.entries(value).sort(([a], [b]) => a.localeCompare(b))); + return value; + }), + ) + .digest("hex"); +} + function resource(record: StoredGraph) { return { project: PROJECT, @@ -100,7 +118,7 @@ function errorEnvelope(code: string, message: string) { return { error: { code, message, correlation_id: "e2e-correlation", details: [] } }; } -function runStatus(run: StoredRun) { +function runStatus(run: StoredRun): RunStatusResponse { const terminal = run.state === "succeeded" || run.state === "canceled"; return { run_id: run.runId, @@ -120,6 +138,9 @@ function runStatus(run: StoredRun) { can_cancel: run.state === "queued" || run.state === "running", can_replay: terminal, logs_available: run.state === "running" || terminal, + result_schema: + run.state === "succeeded" ? "io.dander.control.execution-result-summary/v1" : null, + repair_window: run.repairWindow ?? null, }; } @@ -291,6 +312,10 @@ function createControlService(durable: DurableControlState) { "graph.edit", "graph.delete", "graph.validate", + "graph.change-preview", + "graph.repair-preview", + "run.repair", + "run.explain", "deployment.preview", "run.start", "run.read", @@ -414,7 +439,7 @@ function createControlService(durable: DurableControlState) { } const operationMatch = - /^\/v1\/projects\/demo-project\/graphs\/([^/]+)\/(validate|deployment-preview)$/.exec( + /^\/v1\/projects\/demo-project\/graphs\/([^/]+)\/(validate|deployment-preview|change-preview|repair-preview)$/.exec( url.pathname, ); if (operationMatch && request.method() === "POST") { @@ -429,6 +454,35 @@ function createControlService(durable: DurableControlState) { ); return; } + if (operation === "repair-preview") { + await json(route, 200, { + ...repairPreviewFixture, + graph_content_sha256: current.contentSha, + window: JSON.parse(request.postData() ?? "{}"), + }); + return; + } + if (operation === "change-preview") { + const candidate = JSON.parse(request.postData() ?? "{}"); + if (durable.runCounter === 0) expect(candidate).toEqual(current.document); + await json(route, 200, { + baseline_content_sha256: current.contentSha, + candidate_content_sha256: graphHash(candidate), + graph_properties_changed: [], + node_changes: [], + connection_changes: [], + affected_outputs: [], + outputs: [], + run_output_ids: [], + estimates: { + rows_written: null, + cost_usd: null, + explanation: "This synthetic preview does not inspect warehouse rows.", + }, + limitations: [], + }); + return; + } if (operation === "validate") { await json(route, 200, { valid: false, @@ -464,7 +518,7 @@ function createControlService(durable: DurableControlState) { return; } - const startRunMatch = /^\/v1\/projects\/demo-project\/graphs\/([^/]+)\/runs$/.exec( + const startRunMatch = /^\/v1\/projects\/demo-project\/graphs\/([^/]+)\/(runs|repairs)$/.exec( url.pathname, ); if (startRunMatch && request.method() === "POST") { @@ -489,6 +543,9 @@ function createControlService(durable: DurableControlState) { runId: `run-synthetic-${durable.runCounter}`, state: "queued", polls: 0, + ...(startRunMatch[2] === "repairs" + ? { repairWindow: JSON.parse(request.postData() ?? "{}") } + : {}), }; durable.runs.set(run.runId, run); const status = runStatus(run); @@ -512,6 +569,41 @@ function createControlService(durable: DurableControlState) { return; } + const explanationMatch = /^\/v1\/runs\/([^/]+)\/explanation$/.exec(url.pathname); + if (explanationMatch && request.method() === "GET") { + const run = durable.runs.get(decodeURIComponent(explanationMatch[1]!))!; + const nextAction: RunExplanationResponse["next_action"] = + run.state === "succeeded" + ? { + kind: "inspect_outputs", + label: "Review the pipeline outputs", + reason: "Confirm that the recorded output matches the result you expected.", + } + : run.state === "canceled" + ? { + kind: "inspect_outputs", + label: "Inspect any partial outputs", + reason: "Cancellation does not establish that prior writes were undone.", + } + : { + kind: "refresh", + label: "Refresh run status", + reason: + "Wait for confirmed status and result measurements before starting another run.", + }; + await json(route, 200, { + run_id: run.runId, + state: run.state, + results_available: run.state === "succeeded", + summary: `Run ${run.runId}: ${run.state}`, + details: [], + caveats: ["Synthetic browser fixture; no provider execution."], + available_actions: [nextAction.kind], + next_action: nextAction, + } satisfies RunExplanationResponse); + return; + } + const logMatch = /^\/v1\/runs\/([^/]+)\/logs$/.exec(url.pathname); if (logMatch && request.method() === "GET") { const runId = decodeURIComponent(logMatch[1]!); @@ -761,7 +853,7 @@ test("manages hosted graphs safely across conflicts, ambiguous retries, and serv throw new Error(`Hosted bootstrap did not settle:\n${browserEvents.join("\n")}`); }); await signIn(page, browserEvents); - await page.getByRole("button", { name: "Browse hosted graphs" }).click(); + await page.getByRole("button", { name: "Browse hosted graphs" }).first().click(); await expect(page.getByText("alpha-graph", { exact: true })).toBeVisible(); await expect(page.getByText("beta-graph", { exact: true })).toHaveCount(0); await page.getByRole("button", { name: "Load more" }).click(); @@ -772,6 +864,9 @@ test("manages hosted graphs safely across conflicts, ambiguous retries, and serv .getByRole("button", { name: "Open" }) .click(); + await expect(page.getByRole("button", { name: "Preview changes", exact: true })).toBeVisible(); + await page.screenshot({ path: test.info().outputPath("guided-default.png"), fullPage: true }); + await page.getByText("Advanced validation, deployment and run controls", { exact: true }).click(); await page.getByText("Inspect catalogs and capabilities").click(); await expect(page.getByText(/Dander 0\.9\.0rc19 · API v1/)).toBeVisible(); await expect(page.getByText(/Installed connectors:.*Records API/)).toBeVisible(); @@ -809,15 +904,16 @@ test("manages hosted graphs safely across conflicts, ambiguous retries, and serv await page.getByRole("button", { name: "Reload hosted version" }).click(); await expect(page.locator(".react-flow__node", { hasText: "Server edit" })).toBeVisible(); - await page.getByRole("button", { name: "Start run" }).click(); - const normalizedRun = page.getByLabel("Normalized run status"); + await page.getByRole("button", { name: "Preview changes", exact: true }).click(); + await page.getByRole("button", { name: "Run reviewed graph" }).click(); + const normalizedRun = page.getByLabel("Run outcome"); await expect(normalizedRun).toContainText("Run run-synthetic-1: queued"); await expect(normalizedRun).toContainText("Run run-synthetic-1: running", { timeout: 5_000, }); await page.getByRole("button", { name: "Load bounded logs" }).click(); await expect(page.getByText("Synthetic run is progressing safely.")).toBeVisible(); - await page.getByRole("button", { name: "Cancel run" }).click(); + await page.getByRole("button", { name: "Cancel run", exact: true }).first().click(); await expect( page.getByText("Dander acknowledged cancellation; status polling continues."), ).toBeVisible(); @@ -830,12 +926,43 @@ test("manages hosted graphs safely across conflicts, ambiguous retries, and serv timeout: 7_000, }); + await page.getByText("Graph files and settings", { exact: true }).click(); + const serverNodesBeforeSave = durable.records.get("alpha-graph")?.document.nodes; const source = page.locator(".react-flow__node", { hasText: "Server edit" }); await source.click(); await page.getByLabel("Name", { exact: true }).fill("Client edit"); - await page.getByRole("button", { name: "Save hosted graph" }).click(); + await page.getByRole("button", { name: "Preview changes", exact: true }).click(); + await expect(page.getByText(/Rows and cost: unknown/).first()).toBeVisible(); + expect(durable.records.get("alpha-graph")?.document.nodes).toEqual(serverNodesBeforeSave); + await page.getByRole("button", { name: "Save reviewed changes", exact: true }).click(); + await expect(page.getByRole("button", { name: "Run reviewed graph", exact: true })).toBeEnabled(); await expect(page.getByText("Saved", { exact: true })).toBeVisible(); + await page.getByText("Repair selected output dates", { exact: true }).click(); + await page.getByLabel(/First date/).fill("2026-09-01"); + await page.getByLabel(/End date/).fill("2026-09-03"); + await page.getByRole("button", { name: "Preview date repair", exact: true }).click(); + await expect(page.getByRole("button", { name: "Start reviewed repair" })).toBeEnabled(); + await page.getByLabel(/End date/).fill("2026-09-04"); + await expect(page.getByRole("button", { name: "Start reviewed repair" })).toHaveCount(0); + await page.getByRole("button", { name: "Preview date repair", exact: true }).click(); + await expect(page.getByRole("button", { name: "Start reviewed repair" })).toBeEnabled(); + await page.getByRole("button", { name: "Start reviewed repair" }).scrollIntoViewIfNeeded(); + expect(await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth)).toBe( + true, + ); + await page.screenshot({ + path: test.info().outputPath("guided-repair-review.png"), + fullPage: true, + }); + await page.getByRole("button", { name: "Start reviewed repair" }).click(); + await expect(normalizedRun).toContainText("2026-09-01 through 2026-09-04"); + expect(durable.runs.get("run-synthetic-3")?.repairWindow).toEqual({ + start_date: "2026-09-01", + end_date: "2026-09-04", + }); + await expect(normalizedRun).toContainText("Run run-synthetic-3: succeeded", { timeout: 7_000 }); + const serverRecord = durable.records.get("alpha-graph")!; durable.records.set( "alpha-graph", @@ -864,19 +991,23 @@ test("manages hosted graphs safely across conflicts, ambiguous retries, and serv await expect(page.locator(".react-flow__node", { hasText: "Server edit" })).toBeVisible(); durable.loseNextCreate = true; - await page.getByRole("button", { name: "Browse hosted graphs" }).click(); + await page.getByRole("button", { name: "Browse hosted graphs" }).first().click(); await page.getByLabel("Create from current local draft").fill("restart-graph"); await page.getByRole("button", { name: "Create" }).click(); - await expect(page.getByText(/could not reach/i)).toBeVisible(); + await expect( + page + .getByRole("region", { name: "Pipeline journey", exact: true }) + .getByText(/could not reach/i), + ).toBeVisible(); restartService(); - await page.getByRole("button", { name: "Browse hosted graphs" }).click(); + await page.getByRole("button", { name: "Browse hosted graphs" }).first().click(); await page.getByRole("button", { name: "Create" }).click(); await expect(page.getByText("Saved", { exact: true })).toBeVisible(); restartService(); await page.reload(); await signIn(page, browserEvents); - await page.getByRole("button", { name: "Browse hosted graphs" }).click(); + await page.getByRole("button", { name: "Browse hosted graphs" }).first().click(); await page.getByRole("button", { name: "Load more" }).click(); await page .getByRole("listitem") @@ -884,6 +1015,7 @@ test("manages hosted graphs safely across conflicts, ambiguous retries, and serv .getByRole("button", { name: "Open" }) .click(); await expect(page.locator(".react-flow__node", { hasText: "Server edit" })).toBeVisible(); + await page.getByText("Graph files and settings", { exact: true }).click(); await expect(page.getByLabel("Hosted graph identity")).toContainText( "demo-project/restart-graph", ); diff --git a/integration/current-dander-control.test.ts b/integration/current-dander-control.test.ts index 869e745..48c1eae 100644 --- a/integration/current-dander-control.test.ts +++ b/integration/current-dander-control.test.ts @@ -13,6 +13,7 @@ import { type GraphAddress, type GraphDocument, } from "@/lib/persistence/graph-persistence"; +import { canvasToGraph, graphToCanvas } from "@/lib/pipeline-graph"; import { EXAMPLE_GRAPH } from "@/lib/pipeline-graph/__fixtures__/example-graph"; const configuredUrl = process.env.DANDER_CONTROL_URL; @@ -79,18 +80,42 @@ describe("current protected Dander Control API", () => { graph: EXAMPLE_GRAPH, }); + const canvas = graphToCanvas(opened.graph); + const unchangedDraft = canvasToGraph( + canvas.nodes, + canvas.edges, + opened.graph.name, + opened.graph.trigger, + ); + const unchangedReview = await control.previewChanges(address, opened.revision, unchangedDraft); + expect(unchangedReview.candidate_content_sha256).toBe(opened.contentSha256); + const validation = await control.validate(address, opened.revision); expect(validation).toMatchObject({ graph_name: EXAMPLE_GRAPH.name, content_sha256: opened.contentSha256, }); - current = await persistence.save( - { ...EXAMPLE_GRAPH, name: "current_dander_acceptance_updated" }, - opened.revision, - address, - ); + const candidate = { ...EXAMPLE_GRAPH, name: "current_dander_acceptance_updated" }; + const changes = await control.previewChanges(address, opened.revision, candidate); + expect(changes.baseline_content_sha256).toBe(opened.contentSha256); + expect(changes.candidate_content_sha256).not.toBe(opened.contentSha256); + expect(changes.graph_properties_changed).toContain("name"); + expect(changes.estimates?.rows_written).toBeNull(); + expect(changes.estimates?.cost_usd).toBeNull(); + // Preview must not save even though it accepts the unsaved candidate. + await expect(persistence.load(address)).resolves.toMatchObject({ + revision: opened.revision, + contentSha256: opened.contentSha256, + graph: opened.graph, + }); + + current = await persistence.save(candidate, opened.revision, address); expect(current.revision).not.toBe(opened.revision); + expect(current.contentSha256).toBe(changes.candidate_content_sha256); + await expect(control.previewChanges(address, opened.revision, candidate)).rejects.toMatchObject( + { conflict: true }, + ); const previewFailure = await control .preview(address, current.revision) @@ -104,6 +129,15 @@ describe("current protected Dander Control API", () => { expect(runFailure).toBeInstanceOf(HostedControlOperationError); expect(runFailure).toMatchObject({ unsupported: true }); + // This server intentionally has no execution plan or provider backend. Do not turn + // a read-only contract proof into an accidental live workload or pretend repair ran. + await expect( + control.previewRepair(address, current.revision, { + start_date: "2026-09-01", + end_date: "2026-09-03", + }), + ).rejects.toMatchObject({ unsupported: true }); + await persistence.delete(address, current.revision); current = null; await expect(persistence.load(address)).rejects.toThrow(/does not exist/i); diff --git a/integration/native-repair-observation.test.ts b/integration/native-repair-observation.test.ts new file mode 100644 index 0000000..856545d --- /dev/null +++ b/integration/native-repair-observation.test.ts @@ -0,0 +1,57 @@ +import { expect, it } from "vitest"; +import { HostedControlApiClient } from "@/features/hosted-control/control-api"; +import type { HostedControlFetch } from "@/features/hosted-control/authorized-fetch"; +import { + DANDER_CONTRACT_BUNDLE_ID, + DANDER_CONTRACT_BUNDLE_SHA256, +} from "@/generated/dander-contracts/metadata"; +import { HostedGraphPersistence } from "@/lib/persistence/graph-persistence"; + +const url = process.env.DANDER_CONTROL_URL; +const project = process.env.DANDER_REPAIR_PROJECT; +const graph = process.env.DANDER_REPAIR_GRAPH; +const runId = process.env.DANDER_REPAIR_RUN_ID; +if (!url || !project || !graph || !runId) { + throw new Error( + "Set DANDER_CONTROL_URL, DANDER_REPAIR_PROJECT, DANDER_REPAIR_GRAPH and DANDER_REPAIR_RUN_ID to observe an already-completed synthetic repair.", + ); +} +const base = new URL(url); +if (!/^https?:$/.test(base.protocol) || base.username || base.password) { + throw new Error("DANDER_CONTROL_URL must be credential-free HTTP(S)."); +} +const request: HostedControlFetch = async (target, init) => { + const endpoint = new URL(target.toString(), base); + if (endpoint.origin !== base.origin) throw new Error("Refused a cross-origin request."); + return fetch(endpoint, { ...init, redirect: "error" }); +}; +const control = new HostedControlApiClient(request); +const persistence = new HostedGraphPersistence(request); + +it("reads the RC34 synthetic repair through the production Druff clients without starting another workload", async () => { + const capabilities = await control.capabilities(); + expect(capabilities.contract).toEqual({ + id: DANDER_CONTRACT_BUNDLE_ID, + sha256: DANDER_CONTRACT_BUNDLE_SHA256, + }); + const address = { project, graph }; + const saved = await persistence.load(address); + const run = await control.getRun(runId); + expect(run.state).toBe("succeeded"); + expect(run.repair_window).toEqual({ start_date: "2026-09-10", end_date: "2026-09-12" }); + expect(run.result_schema).toBeTruthy(); + expect(run.extracted).toBe(0); + expect(run.telemetry).toMatchObject({ rows_written: 2, rows_affected: 4 }); + const preview = await control.previewRepair(address, saved.revision, run.repair_window!); + expect(preview.graph_content_sha256).toBe(saved.contentSha256); + expect(preview.window).toEqual(run.repair_window); + expect(preview.outputs.length).toBeGreaterThan(0); + const explanation = await control.explainRun(runId); + expect(explanation).toMatchObject({ run_id: runId, state: "succeeded", results_available: true }); + expect(explanation.summary).toBeTruthy(); + // Repair preview is read-only: no second launch, graph save, source extraction or watermark write. + await expect(persistence.load(address)).resolves.toMatchObject({ + revision: saved.revision, + contentSha256: saved.contentSha256, + }); +}); diff --git a/package.json b/package.json index 4fdbfa6..7678dc6 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "druff", - "version": "0.1.0", + "version": "0.2.0-rc.1", "private": true, "scripts": { "dev": "node scripts/copy-monaco-assets.mjs && next dev", @@ -16,7 +16,8 @@ "test:current-dander": "vitest run --config vitest.current-dander.config.ts", "test:artifact": "node --test scripts/*.test.mjs", "test:watch": "vitest", - "test:e2e": "pnpm build && playwright test" + "test:e2e": "pnpm build && playwright test", + "test:repair-observation": "vitest run --config vitest.native-repair.config.ts" }, "dependencies": { "@monaco-editor/react": "^4.7.0", diff --git a/scripts/generate-dander-contracts.mjs b/scripts/generate-dander-contracts.mjs index c9e87fd..5286db8 100644 --- a/scripts/generate-dander-contracts.mjs +++ b/scripts/generate-dander-contracts.mjs @@ -12,11 +12,13 @@ import { compile } from "json-schema-to-typescript"; import { format, resolveConfig } from "prettier"; const DANDER_PACKAGE = "dander-platform"; -const DANDER_VERSION = "0.9.0rc19"; -const WHEEL_FILENAME = "dander_platform-0.9.0rc19-py3-none-any.whl"; -const WHEEL_SHA256 = "8f1336786e46471a2048d6250008ad176ff3b62d047020872659304c7d2db552"; +const DANDER_VERSION = "0.9.0rc34"; +const WHEEL_FILENAME = "dander_platform-0.9.0rc34-py3-none-any.whl"; +const WHEEL_URL = + "https://github.com/harrisonoconnorhover/dander/releases/download/v0.9.0rc34/" + WHEEL_FILENAME; +const WHEEL_SHA256 = "d37c1d91c15cca9ce3d080af6dec384f7cd9cbc521dea253e10dc44772d35bad"; const BUNDLE_ID = "io.dander.control.contracts/v1"; -const BUNDLE_SHA256 = "695791dfda6058d68453d9e146146d5cdda1439d86c40a7ec249cb4e14a12be3"; +const BUNDLE_SHA256 = "a28316b7e5158e0520fe1c24d59885083714f47b67aa396892fa9742060fb279"; const BUNDLE_PATH = join("dander", "control", "contracts", "v1"); const repoRoot = resolve(dirname(fileURLToPath(import.meta.url)), ".."); @@ -29,8 +31,11 @@ const roots = [ ["connector-catalog", "ConnectorCatalogResponse", "validateConnectorCatalog"], ["control-bootstrap", "ControlBootstrapDescriptor", "validateControlBootstrap"], ["deployment-preview", "DeploymentPreviewResponse", "validateDeploymentPreview"], + ["graph-change-preview", "GraphChangePreviewResponse", "validateGraphChangePreview"], ["graph-create", "GraphCreateRequest", "validateGraphCreate"], ["graph-page", "GraphPageResponse", "validateGraphPage"], + ["graph-repair-preview", "GraphRepairPreviewResponse", "validateGraphRepairPreview"], + ["graph-repair-window", "GraphRepairWindow", "validateGraphRepairWindow"], ["graph-resource", "GraphResourceResponse", "validateGraphResource"], ["graph-validation", "GraphValidationResponse", "validateGraphValidation"], ["log-page", "LogPageResponse", "validateLogPage"], @@ -39,6 +44,7 @@ const roots = [ ["pipeline-graph", "PipelineGraphDocument", "validatePipelineGraph"], ["plugin-catalog", "PluginCatalogResponse", "validatePluginCatalog"], ["project-list", "ProjectListResponse", "validateProjectList"], + ["run-explanation", "RunExplanationResponse", "validateRunExplanation"], ["run-page", "RunPageResponse", "validateRunPage"], ["run-request", "RunRequest", "validateRunRequest"], ["run-status", "RunStatusResponse", "validateRunStatus"], @@ -71,23 +77,7 @@ function fail(message) { } async function fetchPublishedWheel(destination) { - const metadataUrl = `https://pypi.org/pypi/${DANDER_PACKAGE}/${DANDER_VERSION}/json`; - const metadataResponse = await fetch(metadataUrl); - if (!metadataResponse.ok) { - fail( - `PyPI metadata request failed (${metadataResponse.status} ${metadataResponse.statusText})`, - ); - } - const metadata = await metadataResponse.json(); - const wheel = metadata.urls?.find( - (file) => file.filename === WHEEL_FILENAME && file.packagetype === "bdist_wheel", - ); - if (!wheel) fail(`PyPI release does not contain ${WHEEL_FILENAME}`); - if (wheel.digests?.sha256 !== WHEEL_SHA256) { - fail(`PyPI metadata reports an unexpected wheel digest for ${WHEEL_FILENAME}`); - } - - const wheelResponse = await fetch(wheel.url); + const wheelResponse = await fetch(WHEEL_URL); if (!wheelResponse.ok) { fail(`wheel download failed (${wheelResponse.status} ${wheelResponse.statusText})`); } @@ -149,7 +139,7 @@ async function verifyBundle(bundleDirectory) { actualFiles.size !== expectedFiles.size || [...actualFiles].some((path) => !expectedFiles.has(path)) ) { - fail("published bundle files differ from the signed manifest inventory"); + fail("published bundle files differ from the verified manifest inventory"); } return manifest; } @@ -171,6 +161,7 @@ async function generateTypes(bundleDirectory, outputDirectory) { const bannerComment = [ "/**", ` * Generated from ${DANDER_PACKAGE}==${DANDER_VERSION} (${WHEEL_FILENAME}).`, + ` * Published wheel: ${WHEEL_URL}`, ` * Wheel SHA256: ${WHEEL_SHA256}`, ` * Contract bundle: ${BUNDLE_ID} (${BUNDLE_SHA256})`, " * Do not edit by hand; run `pnpm contracts:generate`.", @@ -266,6 +257,7 @@ async function generateMetadata(outputDirectory) { export const DANDER_CONTRACT_PACKAGE = ${JSON.stringify(DANDER_PACKAGE)} as const; export const DANDER_CONTRACT_PACKAGE_VERSION = ${JSON.stringify(DANDER_VERSION)} as const; export const DANDER_CONTRACT_WHEEL = ${JSON.stringify(WHEEL_FILENAME)} as const; + export const DANDER_CONTRACT_WHEEL_URL = ${JSON.stringify(WHEEL_URL)} as const; export const DANDER_CONTRACT_WHEEL_SHA256 = ${JSON.stringify(WHEEL_SHA256)} as const; export const DANDER_CONTRACT_BUNDLE_ID = ${JSON.stringify(BUNDLE_ID)} as const; export const DANDER_CONTRACT_BUNDLE_SHA256 = ${JSON.stringify(BUNDLE_SHA256)} as const; diff --git a/src/app/globals.css b/src/app/globals.css index 6a25b1b..0375448 100644 --- a/src/app/globals.css +++ b/src/app/globals.css @@ -7,7 +7,7 @@ @theme inline { --color-background: var(--background); --color-foreground: var(--foreground); - --font-sans: var(--font-sans); + --font-sans: var(--font-geist-sans); --font-mono: var(--font-geist-mono); --font-heading: var(--font-sans); --color-sidebar-ring: var(--sidebar-ring); diff --git a/src/features/graph-io/GraphEditor.tsx b/src/features/graph-io/GraphEditor.tsx index db971e1..96d0aaf 100644 --- a/src/features/graph-io/GraphEditor.tsx +++ b/src/features/graph-io/GraphEditor.tsx @@ -9,8 +9,7 @@ import { useGraphOperations } from "@/features/graph-operations/useGraphOperatio import { PipelineCanvas } from "@/features/pipeline-canvas/PipelineCanvas"; import { useHostedControl } from "@/features/hosted-control/HostedControlProvider"; import { HostedControlApiClient } from "@/features/hosted-control/control-api"; -import { HostedRunControlsBar } from "@/features/hosted-control/HostedRunControlsBar"; -import { HostedValidationPreviewBar } from "@/features/hosted-control/HostedValidationPreviewBar"; +import { PipelineJourney } from "@/features/hosted-control/PipelineJourney"; import { useHostedRunControls } from "@/features/hosted-control/useHostedRunControls"; import { useHostedValidationPreview } from "@/features/hosted-control/useHostedValidationPreview"; import { HostedConnectorDiscovery } from "@/features/connector-library/discovery"; @@ -67,28 +66,37 @@ export function GraphEditor() { return (
Experimental output repair
++ Rebuild dates in the selected graph from raw data already stored in the warehouse. This + does not re-fetch source records or restore a historical snapshot. +
+ {persistence.status !== "clean" ? ( +Save the graph before previewing a date repair.
+ ) : null} + {!canPreview ?Date repair is unavailable for this connection or role.
: null} +The end date must be later than the first date.
+ ) : null} + + {current?.error ? ( ++ {current.error} +
+ ) : null} + {current?.conflict ? ( + + ) : null} + {preview ? ( +{preview.summary}
+Rows and cost: unknown. Normal source progress is unchanged.
+No compatible execution environment is available.
+ ) : null} + ++ {preview.node_changes.length} node changes · {preview.connection_changes.length} connection + changes · {preview.affected_outputs.length} affected outputs +
++ A full run writes all {preview.run_output_ids.length} configured outputs. Saving alone does + not change warehouse data. +
+ {preview.outputs.length > 0 ? ( +{output.effect}
+ {output.after?.writer?.destination ? ( ++ Destination:{" "} + {[ + output.after.writer.destination.project, + output.after.writer.destination.dataset, + output.after.writer.destination.table, + ] + .filter(Boolean) + .join(".")} +
+ ) : null} +No warehouse outputs are configured.
+ )} ++ Rows and cost: unknown. {preview.estimates?.explanation} +
+