From 06ad849acd07e441a6f16c2003ff42b430de76ba Mon Sep 17 00:00:00 2001 From: Harrison O'Connor-Hoover Date: Thu, 1 Oct 2026 10:51:18 -0400 Subject: [PATCH] Document verified guided workflow candidate and repair qualification --- CHANGELOG.md | 5 +- HANDOFF.md | 44 +++-- README.md | 1 + docs/connector-plugins.md | 7 +- docs/control-contracts.md | 18 +- docs/guided-workflow.md | 155 ++++++++++++++++++ docs/output-repair.md | 21 ++- docs/release-audit.md | 17 ++ docs/salesforce.md | 5 +- docs/servicenow.md | 2 +- docs/support-status.md | 17 +- ...-286-guided-change-and-run-explanations.md | 9 +- .../DANDER-287-date-scoped-output-repair.md | 20 ++- 13 files changed, 264 insertions(+), 57 deletions(-) create mode 100644 docs/guided-workflow.md diff --git a/CHANGELOG.md b/CHANGELOG.md index d8ac99f8..27f6b55d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,8 +18,9 @@ capabilities enter through the next minor release. ### Compatibility -- The Control contract bundle and native repair protocol are unchanged. This Control candidate - can recover the accepted RC33 runtime execution without submitting its data work again. +- The Control contract bundle and native repair protocol are unchanged; RC34 Control uses the + verified RC33 worker. The original proof was cleaned up on its deadline. A distinct synthetic + fixture qualified the correction without repeating that accepted data operation. - Public PyPI remains RC20. The fixed GitHub wheel and named runtime/client artifacts form an experimental integration; broader qualification and support gates remain open. diff --git a/HANDOFF.md b/HANDOFF.md index 246e4dfa..91b74b75 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -2,41 +2,39 @@ ## Finished -- Merged read-only graph change preview and recorded run explanations through [PR #547](https://github.com/harrisonoconnorhover/dander/pull/547). -- Merged selected-date output repair from retained raw data with single-task Cloud Run and BigQuery through [PR #548](https://github.com/harrisonoconnorhover/dander/pull/548). -- Reused durable run identity, cancellation, history, replay, lease fencing, and conditional cleanup. -- Added preview/start contracts, publication measurements, and operator documentation with explicit support limits. -- Published and verified Salesforce `0.3.2` and ServiceNow `0.2.3`; merged the matching catalog, starter, and documentation through PR #550. +- Published immutable RC34 Control with graph previews, recorded explanations, and selected-date repair; verified the paired RC33 worker image. +- Published Salesforce `0.3.2` and ServiceNow `0.2.3` with corrected compatibility and current catalog/starter pins. +- Published Druff `0.2.0-rc.1` after protected PR #25 and exact-main checks, generated from the exact published RC34 wheel. Its guided configure → preview → save/run → understand/recover flow is ready for the documented hosted setup. +- Completed a distinct native BigQuery/Cloud Run repair and actual Druff client observation; cleaned both temporary proof environments. +- Updated the setup guide, connector documentation, support boundaries, and delivery records. ## Try It -Use the API examples in `docs/control-contracts.md` and `docs/output-repair.md`. Preview a saved graph's UTC date interval with its `If-Match` ETag, then start `/repairs` with the same revision and an `Idempotency-Key`. A repair-capable runtime and existing partitioned outputs are required. Published clients do not yet expose this flow. +Follow the [guided workflow guide](docs/guided-workflow.md) for the exact RC34 Control wheel, RC33 worker, and hosted Druff setup. Use the configured OIDC deployment and matching bootstrap descriptor. Preview changes, save the reviewed revision, then run it. Repair's end date is excluded. ## Checks -- Repair protected CI: **2,515 tests passed**, including PostgreSQL; one existing test-client deprecation warning. -- Strict typing passed across 516 files; Ruff, generated contracts, distribution installation, infrastructure validation, and container/secret scans passed. -- Exact-main CI passed for preview/explanation at `099fb52` and for repair at [`79aa1cb`](https://github.com/harrisonoconnorhover/dander/actions/runs/36849004513). -- No live repair workload or paid qualification ran. -- RC33 preparation: 4 release metadata tests, Ruff lint/format, metadata consistency, and contract drift checks passed; lockfile dependencies stayed fixed. -- CI exposed 5 historical Redshift fixture failures after the version bump. Fixtures now explicitly bind RC32; all 32 focused benchmark/metadata tests pass. Candidate CI must rerun. -- PR #550 passed combined protected CI. A follow-up explicitly includes installed prereleases when checking compatibility; the catalog suite also runs against the declared minimum `packaging` version in CI. +- Backend exact-main [CI 36874195042](https://github.com/harrisonoconnorhover/dander/actions/runs/36874195042) passed: 2,528 tests, 516 strictly typed files, Ruff, contracts, distribution, infrastructure, container and secret checks. One existing test-client deprecation warning remains. +- Public wheel bytes and GitHub attestations verified; fresh installation checked all 45 contract files. Published connector packages passed 93 Salesforce and 23 ServiceNow tests against RC34, plus earlier supported-version checks. +- RC33 worker startup/shutdown, provider imports, vulnerability scan, and anonymous digest pull passed. Local secret-pattern matches were verified Oracle SDK examples. +- Druff passed 685 unit tests, nine artifact tests, 11 browser journeys, RC34 HTTP acceptance, and the real native-run client observation. Browser OIDC/API fixtures were synthetic. +- Druff exact-main [CI 36877515144](https://github.com/harrisonoconnorhover/druff/actions/runs/36877515144) passed, including both architecture scans and build reproducibility. +- Published Druff image bytes matched the verified OCI index through anonymous registry access; both architectures passed source-free, non-root, read-only runtime and scan checks. Existing active/rollback aliases were preserved. +- Native repair recorded two rows written, four affected, and zero ingestion rows; surrounding/raw data and watermark unchanged, one execution, original Job restored. Cloud cleanup passed before its deadline; local databases, processes, watchdogs, and task credentials were removed. ## Decisions -- Preview stays read-only; explanations use collected evidence and retain unknown measurements. -- Repair leaves source extraction progress untouched and replaces only the selected dates, atomically per output. -- Deliver one guided workflow in Druff; additional providers and broader graph execution are separate work. +- Measurements come from recorded results; missing values stay Unknown. +- Repair uses currently retained raw data and commits per output. +- Deliver an experimental matched candidate; public PyPI remains RC20. ## Remaining -- Complete current aggregate billing reconciliation and bounded live repair; AWS sign-in now works. -- Finish Druff's guided flow. The user approved its repository work and protected delivery. -- Publish and consume the verified RC33 GitHub integration candidate after protected merge and exact-main CI; public PyPI RC20 stays unchanged. -- Complete the protected prerelease comparison fix and final RC33 combined package check. Both connector releases are published and their downloaded wheels passed the existing 93 Salesforce and 23 ServiceNow tests. +- Production identity-provider, scale, additional-provider, and broader release gates remain separate. +- Keep the USD 1 proof reservation within the USD 21 October aggregate while billing settles. ## Review First -- `src/dander/providers/bigquery/graph.py` and `tests/pipeline/test_date_repair.py` for scoped publication. -- `src/dander/control/cloud_run_execution_backend.py` for execution binding and cleanup. -- `docs/output-repair.md` for practical behavior and limitations. +- [Exact artifacts and guided setup](docs/guided-workflow.md) +- [Repair behavior and limits](docs/output-repair.md) +- [Druff PR #25](https://github.com/harrisonoconnorhover/druff/pull/25) diff --git a/README.md b/README.md index fdad6feb..a198cd6a 100644 --- a/README.md +++ b/README.md @@ -47,6 +47,7 @@ workflow artifact is reviewed. Operator-facing documentation: - [Current implementation and support status](docs/support-status.md) +- [Guided workflow integration candidate](docs/guided-workflow.md) - [Control startup profiles and PostgreSQL storage](docs/control-profiles.md) - [Hosted Greenhouse quickstart](https://github.com/harrisonoconnorhover/dander/blob/main/docs/getting-started.md) - [Experimental AWS-native Fargate runbook](https://github.com/harrisonoconnorhover/dander/blob/main/docs/aws-native-profile.md) diff --git a/docs/connector-plugins.md b/docs/connector-plugins.md index aa20e953..6ed198c2 100644 --- a/docs/connector-plugins.md +++ b/docs/connector-plugins.md @@ -165,15 +165,16 @@ CLI results include an exact public package pin, its compatible Dander range, su and provider-validation status. Catalog API metadata also includes documentation links. The catalog ships with Dander; PyPI remains the package store and source of the actual distribution. -The RC33 catalog pins Salesforce `0.3.2` (`dander-platform>=0.6.0rc1,<0.10`) and ServiceNow -`0.2.3` (`dander-platform>=0.5.0,<0.10`). Both accept the public Dander RC20 beta and RC33 +The RC34 catalog pins Salesforce `0.3.2` (`dander-platform>=0.6.0rc1,<0.10`) and ServiceNow +`0.2.3` (`dander-platform>=0.5.0,<0.10`). Both accept the public Dander RC20 beta and RC34 integration candidate. Older Dander installations retain their bundled catalog; use these exact pins in the project's `plugins` configuration, then run `dander plugins install` and `dander validate` before rebuilding the runtime image. Installing a package alone does not activate it or replace a deployed image. Compatibility checks cover clean package installations and each connector's existing behavior -suite. They do not add a new live Salesforce or ServiceNow acceptance result. +suite, including all 116 tests against the published RC34 wheel. They do not add a new live +Salesforce or ServiceNow acceptance result. When `dander graph serve` opens a project, `GET /v1/plugin-catalog` exposes the same non-secret catalog and marks only validated, manifest-declared plugins as installed. `GET /v1/connectors` diff --git a/docs/control-contracts.md b/docs/control-contracts.md index 5576b3fc..2ed6f3fc 100644 --- a/docs/control-contracts.md +++ b/docs/control-contracts.md @@ -1,14 +1,14 @@ # Dander Control contract bundle -Published client baseline: Dander `0.9.0rc19`. The current source bundle also contains unreleased -additions described below; their presence in this checkout does not publish a new client artifact. - -RC33 prepares those additions for a GitHub integration prerelease. Its bundle SHA-256 is +The [RC34 GitHub integration prerelease](https://github.com/harrisonoconnorhover/dander/releases/tag/v0.9.0rc34) +publishes the current contract additions. Its bundle SHA-256 is `a28316b7e5158e0520fe1c24d59885083714f47b67aa396892fa9742060fb279`. -Consumers must wait for the verified release wheel and pin its separate SHA-256 as well as this -bundle digest. This candidate does not change the public PyPI beta or qualify a runtime profile. +The wheel SHA-256 is `d37c1d91c15cca9ce3d080af6dec384f7cd9cbc521dea253e10dc44772d35bad`. +Both digests are pinned by the paired Druff candidate. See [guided workflow](guided-workflow.md) +for the exact source, installation, and verification record. Public PyPI remains RC20; the +historical RC19 client baseline and its original publication evidence are preserved below. -Dander is the authority for data crossing the future Control API boundary. The deterministic +Dander is the authority for data crossing the Control API boundary. The deterministic `io.dander.control.contracts/v1` bundle lives in `src/dander/control/contracts/v1` and is included in the `dander-platform` wheel and source distribution. Druff must generate its client from a separately approved, immutable Dander release artifact; it must not copy these files from a source @@ -198,8 +198,8 @@ date-range repair; replaying a repair run retains its recorded date window. Neither operation requires provider-specific browser logic or a new service dependency. Editors can request change previews; viewers can read run explanations when a lifecycle is wired. -Regenerate consumers from the immutable producer artifact after its release. The guided Druff -surface is separate work, not delivered by these two routes. Current source also exposes +The [paired Druff candidate](guided-workflow.md) consumes these routes through contracts generated +from the exact published RC34 wheel. That bundle also exposes `graph.repair-preview` and `run.repair` for eligible execution plans; see [selected output date repair](output-repair.md) for the request, supported scope, and evidence limits. diff --git a/docs/guided-workflow.md b/docs/guided-workflow.md new file mode 100644 index 00000000..ca03feac --- /dev/null +++ b/docs/guided-workflow.md @@ -0,0 +1,155 @@ +# Guided workflow integration candidate + +The October 1 candidate adds a single Druff journey for configuring a graph, reviewing its +changes, running the reviewed saved revision, and understanding the recorded result. Selected-date +repair is available for eligible BigQuery outputs executed by a single Cloud Run task. + +This is an experimental integration candidate. Public PyPI Dander remains `0.9.0rc20`; existing +production identity-provider, multi-replica, provider-scale, and broader release gates remain open. + +## Install the exact backend + +The immutable [RC34 release](https://github.com/harrisonoconnorhover/dander/releases/tag/v0.9.0rc34) +contains the wheel, source archive, and `SHA256SUMS`. Install the wheel into a separate environment +with the extras needed by the PostgreSQL/GCP profile: + +```bash +uv venv --python 3.12 .venv-rc34 +uv pip install --python .venv-rc34/bin/python \ + 'dander-platform[postgres,bigquery,gcp] @ https://github.com/harrisonoconnorhover/dander/releases/download/v0.9.0rc34/dander_platform-0.9.0rc34-py3-none-any.whl#sha256=d37c1d91c15cca9ce3d080af6dec384f7cd9cbc521dea253e10dc44772d35bad' +.venv-rc34/bin/dander --version +``` + +Use the prerequisites and flags in [Control profile preparation](control-profiles.md) to register +an already-deployed graph. For this artifact-pinned installation, extract the source archive from +the same RC34 release and invoke its preparation script with the installed wheel's interpreter: + +```bash +.venv-rc34/bin/python /path/to/rc34-source/examples/control/prepare_cloud_run.py \ + --config /path/to/project/dander.yaml \ + --pipeline greenhouse_jobs_graph \ + --gcp-project-id your-existing-project \ + --image 'us-central1-docker.pkg.dev/your-existing-project/dander/dander@sha256:YOUR_DIGEST' \ + --output-dir /path/to/private/control \ + --schema control_example +.venv-rc34/bin/dander control serve --profile /path/to/private/control/control.yaml +``` + +Use these commands in place of the linked source-checkout `uv sync` and `uv run` commands, so +Control keeps using the verified wheel. Set `DANDER_CONTROL_DATABASE_URL` through your existing +secret mechanism first. Credentials, deployed jobs, and the dedicated database still belong to +that profile. The guided interface does not provision infrastructure or discover secrets. + +RC34 is not on PyPI. Its generated project Dockerfile still expects a PyPI package; use the named +candidate runtime image for this integration instead of building that default Dockerfile. + +Use RC34 for the Control process. The paired Linux/AMD64 **worker** remains RC33 because the +correction changes only Control reconciliation, not the native execution contract. Do not use the +worker image to start Control; its bundled Control command still contains the RC33 defect. + +The public worker image is: + +```text +ghcr.io/harrisonoconnorhover/dander@sha256:89bd242e55d6e5ca37394ce81187f7d036e26b670c22f0ed5805a3c1df9447dc +``` + +Use this digest as the base for your project's configuration image, then deploy that image through +the existing project workflow. The Control preparation example consumes the already-deployed job's +exact configuration and image; it does not build or deploy a job. +The readable tag is `0.9.0rc33-amd64`; pin the digest for execution. The image passed native startup, +shutdown, provider-import and repair-contract checks. Its public manifest matches the local +verified build, and an anonymous pull by digest reported `dander 0.9.0rc33`. +No fixed HIGH/CRITICAL vulnerabilities were found. A separate local secret-pattern +scan matched only documented examples in the Oracle SDK; the repository secret scan passed. + +## Open the guided interface + +Druff `0.2.0-rc.1` comes from protected source +`eabceb21d7b1b38fcbccdf34f07772e89ea52f6a`. +Its [exact-main CI](https://github.com/harrisonoconnorhover/druff/actions/runs/36877515144) +passed frontend checks, contract drift, browser journeys, both architecture scans, runtime +checks, and build reproducibility. + +The published multi-architecture client image is: + +```text +ghcr.io/harrisonoconnorhover/druff@sha256:6aade12c399a67c604577e4a022af6dff22db7764626dd9a7a4f5cea1a2ceac1 +``` + +The readable tag is `0.2.0-rc.1`. The verified OCI bytes were copied without rebuilding, and +anonymous registry access confirmed that exact index. Both AMD64 and ARM64 images passed +source-free, non-root, read-only runtime, route/header, vulnerability, and secret checks. +Existing `active` and `rollback` aliases were preserved; this is a separately selected candidate. + +This journey is in Druff's **hosted mode**. Serve the candidate with the same-origin +`/bootstrap.json` generated from the matching RC34 +[hosted OIDC deployment input](control-contracts.md#hosted-oidc-boundary). Regenerate that public +descriptor from the existing deployment input when upgrading; an older contract digest will be +rejected. The descriptor contains connection settings, not credentials. + +Without that descriptor, Druff opens its existing loopback/offline editor. Starting the local +Control profile alone does not activate the hosted journey. Use the existing configured OIDC +deployment and its login flow; this candidate does not create an identity provider. +The [Druff hosted-interface instructions](https://github.com/harrisonoconnorhover/druff/blob/eabceb21d7b1b38fcbccdf34f07772e89ea52f6a/README.md#hosted-interface) +describe serving the static image and descriptor together. + +## Review and run + +1. Sign in to the configured hosted Druff interface and open a saved graph. +2. Configure its source, transformations, and outputs, then choose **Preview changes**. +3. Review the change and affected-output summary. Unknown rows and cost remain unknown. +4. Choose **Save reviewed changes**, then **Run reviewed graph**. Further edits or a conflicting + saved revision require a new preview. +5. Read the recorded result and suggested available actions. Technical controls, logs, cancellation, + and replay remain available in the existing details. + +For an eligible saved graph, **Repair selected output dates** previews the date interval before +submission. It rebuilds from currently retained raw data, preserves surrounding output dates, and +leaves source ingestion progress unchanged. Read [output repair](output-repair.md) for UTC +boundaries, existing-output requirements, key-collision rejection, and per-output transactions. + +## Fixed producer identity + +- Protected source: `6964d63f4e9e8eab15ce63e4a1c65efb5268c2d0`. +- [Exact-main CI](https://github.com/harrisonoconnorhover/dander/actions/runs/36874195042): passed. +- Wheel SHA-256: `d37c1d91c15cca9ce3d080af6dec384f7cd9cbc521dea253e10dc44772d35bad`. +- Source archive SHA-256: `3f5f812e391775149c611d868540f452efbfe42070702542eb4985800083783e`. +- Control bundle SHA-256: `a28316b7e5158e0520fe1c24d59885083714f47b67aa396892fa9742060fb279`. + +The public download matched the tested wheel, GitHub release attestations verified both archives, +and a fresh installed environment verified all 45 packaged contract files and both curated pins: +Salesforce `0.3.2` and ServiceNow `0.2.3`. Their combined 116 tests passed against the published +wheel. These connector checks do not claim a new live source run. + +RC34 corrects a defect found during the first native repair check: Cloud Run resolved the +configured image index to its AMD64 image, and RC33 Control rejected that identity during +terminal reconciliation even though the data operation succeeded. RC34 verifies the relationship +through Artifact Registry metadata before accepting the resolved image. The Control principal +needs `artifactregistry.dockerImages.get` on the existing repository; see +[profile permissions](control-profiles.md). The fixed producer passed 2,528 tests and strict +typing across 516 files on its exact merged source revision. + +## Native repair verification + +On October 1, the published RC34 Control wheel and the RC33 worker completed the synthetic run +`run-ec3ecadaeb48d3cca4ccf537`. Its window included September 10 and 11 and excluded September 12. +The task corrected an existing row, inserted another retained row, and removed obsolete selected +data. Control recorded two output rows written, four affected, zero ingestion rows, two measured +operations, and zero retries. Surrounding output rows, raw data, and ingestion progress were +unchanged. A separate direct transaction rejected a key crossing the date boundary without +changing the output. + +Repeating the request returned the same run, with exactly one native execution. Control restored +the original Job arguments and environment and reported success with results at 14:33:41 UTC. +Druff's actual production client read that run and its explanation successfully at 14:34:17 UTC. +Browser journey tests used synthetic OIDC/API fixtures; this client observation does not turn +those fixtures into production identity-provider evidence. + +The temporary Job, dataset, image index, and both image children were verified absent at +14:36:32 UTC, before the original 14:54:49 cleanup deadline. Both proof environments' local +Control processes, PostgreSQL containers, watchdogs, and task credential files were also removed. +The USD 1 qualification reservation remains held inside the USD 21 aggregate October reservation +while provider billing settles; it is a conservative reservation, not an observed final charge. + +This is a small, named synthetic repair proof. It does not establish production source accuracy, +large-table cost, other execution backends, or atomic publication across multiple outputs. diff --git a/docs/output-repair.md b/docs/output-repair.md index 53ee9501..8cdaba06 100644 --- a/docs/output-repair.md +++ b/docs/output-repair.md @@ -1,8 +1,8 @@ # Repair selected output dates -Current source implements date-scoped output repair for direct Cloud Run execution with a -BigQuery warehouse. It rebuilds selected dates from the raw data **already stored** in the -warehouse. It does not fetch source records, advance ingestion progress, or reconstruct a past +The [guided workflow candidate](guided-workflow.md) includes date-scoped output repair for direct +Cloud Run execution with a BigQuery warehouse. It rebuilds selected dates from the raw data +**already stored** in the warehouse. It does not fetch source records, advance ingestion progress, or reconstruct a past source snapshot. The published RC19/RC20 client/runtime pair does not include this operation. ## Preview, then start @@ -56,6 +56,15 @@ the base Job settings after terminal repair while they still belong to that repa Local checks cover date boundaries, unchanged surrounding rows, collision rejection, transaction rollback, lease fencing, no source/watermark access, replay identity, and Cloud Run adoption and -cleanup. The row checks execute translated predicates against SQLite; they do not establish live -BigQuery or Cloud Run qualification. A bounded synthetic live check and the paired Druff interface -remain required before claiming the complete guided workflow is delivered. +cleanup. The original local row checks execute translated predicates against SQLite. + +On October 1, the published RC34 Control wheel and RC33 worker completed one distinct synthetic +Cloud Run/BigQuery repair for September 10 through September 11. Control recorded two rows written, +four rows affected, and zero ingestion rows. The surrounding output rows, retained raw data, and +watermark stayed unchanged; temporary stages were absent, the original Job settings were restored, +and repeated submission returned the same run with one native execution. A separate direct +transaction check rejected a crossing key without changing the output. Exact artifacts and the +paired client evidence are recorded in [guided workflow](guided-workflow.md). + +This proves the named small synthetic shape. It does not establish production source correctness, +large-table cost, other warehouse backends, or transactions spanning multiple outputs. diff --git a/docs/release-audit.md b/docs/release-audit.md index bfbb18f6..0ddacf45 100644 --- a/docs/release-audit.md +++ b/docs/release-audit.md @@ -27,6 +27,23 @@ cloud path remains outside the live proof. ## Current release and deployment record +- The October 1 [RC34 integration candidate](guided-workflow.md) is published as an immutable + GitHub prerelease from protected source `6964d63f4e9e8eab15ce63e4a1c65efb5268c2d0`. Exact-main + CI passed; public wheel bytes and GitHub release attestations matched the verified distributions. + Its fresh installed environment verified all 45 contract files. The released Salesforce `0.3.2` + and ServiceNow `0.2.3` packages passed their combined 116 tests with this published wheel. + RC34 is not published to PyPI and does not close the broader Phase 8 release gates. +- RC33's first native repair completed its data work but exposed a Control reconciliation bug + when Cloud Run resolved an OCI index to its AMD64 image. Protected PR #552 corrected that + relationship check in RC34; its exact-main CI passed 2,528 tests and 516 strictly typed files. + The verified RC33 worker remains the paired runtime because its execution protocol did not + change. Use the RC34 wheel, rather than that worker image, to start Control. +- Druff `0.2.0-rc.1` was published from protected source + `eabceb21d7b1b38fcbccdf34f07772e89ea52f6a` after exact-main CI passed. Its verified OCI index + `sha256:6aade12c399a67c604577e4a022af6dff22db7764626dd9a7a4f5cea1a2ceac1` was copied without + rebuilding and verified anonymously; active/rollback aliases were preserved. The distinct + RC34 Control repair and actual Druff client observation passed, followed by complete bounded + cleanup. [Guided workflow](guided-workflow.md) records the exact pairing and proof limits. - Public Dander beta: `0.9.0rc20`; public Salesforce connector: `0.3.2`; public ServiceNow connector: `0.2.3`. The October 1 connector releases widen package compatibility through Dander 0.9 with clean-install behavior checks; the historical live evidence below keeps its recorded versions. diff --git a/docs/salesforce.md b/docs/salesforce.md index 287bed6e..22fb1005 100644 --- a/docs/salesforce.md +++ b/docs/salesforce.md @@ -53,8 +53,11 @@ add its governed projection to the relevant model. Undeclared fields fail before ## Project configuration +Projects generated by an older Dander release may include an older example pin. Set +`plugins.salesforce.version` to `0.3.2` explicitly before running `dander plugins install`. + With `dander-connector-salesforce 0.3.2`, projects using Dander `>=0.6.0rc1,<0.10` can use the -following exact pin, including the public RC20 beta and RC33 integration candidate: +following exact pin, including the public RC20 beta and RC34 integration candidate: ```yaml version: 1 diff --git a/docs/servicenow.md b/docs/servicenow.md index adf67176..9977cd6e 100644 --- a/docs/servicenow.md +++ b/docs/servicenow.md @@ -13,7 +13,7 @@ plugins: version: 0.2.3 ``` -This package accepts Dander `>=0.5.0,<0.10`, including the public RC20 beta and RC33 integration +This package accepts Dander `>=0.5.0,<0.10`, including the public RC20 beta and RC34 integration candidate. Run `dander plugins install` and `dander validate`, then rebuild the runtime image. See [connector discovery](connector-plugins.md#discover-curated-connectors) for the distinction between package compatibility and live provider evidence. diff --git a/docs/support-status.md b/docs/support-status.md index 7b6a7d8d..a84a63b7 100644 --- a/docs/support-status.md +++ b/docs/support-status.md @@ -9,7 +9,7 @@ integrated or a named release/profile changes status. | Revision or artifact | State | What it means | |---|---|---| | Public prerelease `0.9.0rc20` | Published August 14 | The package installed by the hosted quickstart; later main-source features are not automatically included. | -| Current source (`0.9.0rc33`) | Prepared GitHub integration candidate | Includes PostgreSQL Control, planning/Spark work, graph previews, run explanations, and scoped repair. Consumer integration requires the exact verified artifact; this does not promote the public PyPI beta. | +| GitHub integration candidate (`0.9.0rc34`) | Published immutable wheel and source archive | Includes PostgreSQL Control, planning/Spark work, graph previews, run explanations, and scoped repair. See [guided workflow](guided-workflow.md) for exact artifacts; public PyPI remains RC20. | | Retained GCP private RC22 | Operator trial closed | The August 2–September 1 observation and final seven clean days passed; all five schedules were subsequently paused. | | Local `codex/hdfs-enterprise-foundations` at `e187fdd` | Remaining enterprise implementation | PostgreSQL durability/startup has been integrated here. HDFS/YARN, Hive interfaces, semantics, locality, and enterprise packaging remain on the preserved branch. | @@ -23,13 +23,13 @@ the final paused/no-drift state. These are dated observations, not a continuous |---|---|---| | GCP Cloud Run + BigQuery pipeline | Released named profile and retained operator evidence | GCP remains the supported compatibility baseline within the documented beta limits. | | Direct single-container execution | Existing default path | Does not require hosted Control, Spark, Kubernetes, or Hadoop. | -| Curated connector packages | RC33 catalog pins Salesforce `0.3.2` and ServiceNow `0.2.3`; clean-install behavior checks cover the declared minimums, Dander `0.7.1`, and public RC20 | Compatibility metadata now accepts Dander 0.9 candidates. Existing provider evidence keeps its recorded versions; this is not a new live provider qualification. | +| Curated connector packages | Published Salesforce `0.3.2` and ServiceNow `0.2.3`; clean-install behavior checks cover the declared minimums, Dander `0.7.1`, public RC20, and published RC34 | RC34 pins both packages and explicitly evaluates installed prerelease compatibility. Existing provider evidence keeps its recorded versions; this is not a new live provider qualification. | | Hosted Control API, graph persistence, OIDC | Implemented with named local/cloud acceptance records | Experimental; each profile keeps its own identity, storage, and lifecycle evidence. | | Control execution through Fargate, Cloud Run, or Dataproc | Implemented with typed PostgreSQL or S3 run storage | Experimental; one Control process per run-store schema. AWS compatibility flags remain available. | | Distributed Spark graphs | Bounded linear, keyed-join, and two-transform shapes implemented; retained August 27–28 runs include exact Fargate/Dataproc output parity | Experimental; the [execution history](control-execution-history.md) identifies each tested revision and bounded shape. | | Druff hosted authoring | Hosted Control connections, OIDC/PKCE, persisted graphs, and run lifecycle controls implemented | Experimental; retained browser checks used synthetic OIDC. They do not establish production identity-provider or HA qualification. | -| Graph change preview and run explanation | Read-only comparison against the saved revision and readable recorded outcomes in current Control source | Requires a newly generated client from the matching immutable artifact; published Druff clients do not expose these operations. | -| Date-scoped output repair | Current source implements retained-raw repair with single-task Cloud Run and BigQuery; local transaction, lifecycle, and transport tests | Experimental and not live-qualified or published. See [output repair](output-repair.md) for existing-table, date-boundary, and per-output transaction limits. | +| Graph change preview and run explanation | Published RC34 contracts and the paired Druff candidate provide review, save/run, and recorded outcomes | Experimental integration; see [guided workflow](guided-workflow.md) for fixed artifacts and hosted-mode prerequisites. | +| Date-scoped output repair | RC34 Control plus the RC33 worker passed one synthetic single-task Cloud Run/BigQuery repair, including actual Druff client observation and cleanup | Qualified only for that bounded shape. See [output repair](output-repair.md) for existing-table, date-boundary, and per-output transaction limits; broader provider and production-scale claims remain open. | | PostgreSQL, Snowflake, Redshift warehouse adapters | Local conformance and named bounded live evidence | Experimental; the runtime compatibility matrix determines allowed state/warehouse pairs. | | Azure, OCI, Kubernetes profiles | Named lifecycle/qualification records | No general support promotion; consult each exact profile's evidence. | | PostgreSQL Control graphs/runs/schedules | Database conformance plus one real Cloud Run graph with crash recovery, cancellation, output, and history checks | Experimental; one local Control process, no supported hosted or multiple-replica topology. | @@ -48,15 +48,16 @@ The [profile preparation example](control-profiles.md) now registers an existing and writes its plan and startup files together. S3/Azure/OCI graph records and journals also share one definition; further operation consolidation is optional. -1. Reconcile current Dander costs and reservations before the next paid qualification phase. - The September 7 snapshot was close to the USD 25 monthly cash ceiling; it is not a current - balance. Account-wide billing alone does not identify which resources belong to Dander. +1. Refresh Dander costs and reservations before each subsequent paid qualification phase. + October 1 reconciliation conservatively reserved USD 21 of the USD 25 monthly cash ceiling, + including USD 1 for the bounded repair proof. Delayed billing and retained resources remain + accounted for; this reservation is not a final provider charge. 2. Fill the legacy graph's operation-level telemetry gap before relying on its byte counters for cost comparisons. The successful workflow exposed row metrics but no recorded operations. 3. Identify an existing secure Hadoop environment and its owner before live enterprise tests. Local simulations and disposable PostgreSQL checks do not qualify a secure Hadoop estate. 4. Select the next exact-candidate release gate from DANDER-207 once its environment and cost - bounds are ready. No new public release was created by this integration. + bounds are ready. The RC34 GitHub integration candidate does not promote the public PyPI beta. Once an environment is available, integrate the required HDFS/YARN slice from the preserved branch. Begin with one bounded two-source-to-Parquet workflow, then test restart, cancellation, diff --git a/tickets/DANDER-286-guided-change-and-run-explanations.md b/tickets/DANDER-286-guided-change-and-run-explanations.md index 1cc0c6a6..d70f18cf 100644 --- a/tickets/DANDER-286-guided-change-and-run-explanations.md +++ b/tickets/DANDER-286-guided-change-and-run-explanations.md @@ -1,7 +1,7 @@ --- id: DANDER-286 title: Explain unsaved graph changes and recorded run outcomes -status: in-code +status: done component: python depends_on: [] created: 2026-10-01 @@ -37,3 +37,10 @@ passed across 513 files. Ruff and generated contract drift checks passed. [PR #547](https://github.com/harrisonoconnorhover/dander/pull/547) merged as `099fb52` after protected CI passed. Its [exact-main CI](https://github.com/harrisonoconnorhover/dander/actions/runs/36847441914) also passed. The paired guided interface remains outside this backend ticket's completion claim. + +## Review Log + +October 1: PASS against the backend acceptance criteria. The immutable RC34 producer contains +the published contract bundle, and [Druff PR #25](https://github.com/harrisonoconnorhover/druff/pull/25) +merged the generated client and guided interface. The [integration record](../docs/guided-workflow.md) +identifies the exact releases and keeps the historical backend-only verification above intact. diff --git a/tickets/DANDER-287-date-scoped-output-repair.md b/tickets/DANDER-287-date-scoped-output-repair.md index 4a52e9a8..399a9b16 100644 --- a/tickets/DANDER-287-date-scoped-output-repair.md +++ b/tickets/DANDER-287-date-scoped-output-repair.md @@ -1,7 +1,7 @@ --- id: DANDER-287 title: Repair selected output dates from retained raw data -status: in-code +status: done component: python depends_on: [DANDER-286] created: 2026-10-01 @@ -21,8 +21,8 @@ bounded operation to the existing Control lifecycle and direct BigQuery graph ru - [x] Reject incompatible backends and older runtimes before pipeline work. - [x] Reconcile terminal Job cleanup without overwriting a newer deployment or launch. - [x] Explain measured publication counts separately from the zero ingestion counts. -- [ ] Verify a bounded synthetic BigQuery and Cloud Run run from an immutable current artifact. -- [ ] Publish the paired contract artifact and integrate the guided Druff controls. +- [x] Verify a bounded synthetic BigQuery and Cloud Run run from an immutable current artifact. +- [x] Publish the paired contract artifact and integrate the guided Druff controls. ## Verification and boundary @@ -41,3 +41,17 @@ Row-level tests execute the generated predicates through a narrow SQLite transla not live BigQuery proof. Current raw data is recompiled, so staging can scan outside the selected dates. Existing outputs are required; earlier outputs remain published if a later output fails. See [output repair](../docs/output-repair.md) for operator instructions and compatibility. + +## Review Log + +October 1: PASS against the remaining artifact and native acceptance criteria. The first RC33 +native run completed its data work but exposed the resolved-image reconciliation defect corrected +in protected [PR #552](https://github.com/harrisonoconnorhover/dander/pull/552). The original proof +was cleaned up on time; its accepted data operation was not repeated. + +The distinct `run-ec3ecadaeb48d3cca4ccf537` used the published RC34 Control wheel and immutable +RC33 worker. Control recorded two output rows written, four affected, and zero ingestion rows; +surrounding data, raw data, and the watermark were unchanged. One native execution, same-key +identity, restored Job settings, collision rejection, and cleanup all passed. Druff's production +client observed the completed run and explanation; its guided controls merged through PR #25. +See the [integration record](../docs/guided-workflow.md) for artifact identities, times, and scope.