Reconcile beta-readiness work without release metadata

.dockerignore

@@ -14,6 +14,8 @@
 !graphs/**
 !models/
 !models/**
+!examples/
+!examples/**
 !infra/
 !infra/**
 infra/.DS_Store

CHANGELOG.md

@@ -6,6 +6,17 @@ capabilities enter through the next minor release.
 
 ## Unreleased
 
+### Added
+
+- Add sanitized failure codes and summaries to local and hosted run history, and reconcile stale
+  non-terminal runs after an expired lease is safely reacquired.
+- Give every BigQuery run-scoped staging table a 24-hour expiration at creation while retaining
+  immediate cleanup for handled completion and failure paths.
+- Add plan-first stage-zero and platform commands, source-free immutable image publication, a
+  read-only permission preflight, and documented least-privilege upgrade and rollback workflows.
+- Expand the Salesforce project contract to four independently watermarked CRM endpoints, four
+  governed staging models, and an Opportunity fact with Account and historical owner dimensions.
+
 ## 0.5.1 — 2026-08-05 (alpha)
 
 ### Fixed

Dockerfile

@@ -14,6 +14,7 @@ COPY src ./src
 COPY connectors ./connectors
 COPY graphs ./graphs
 COPY models ./models
+COPY examples ./examples
 COPY infra ./infra
 RUN uv sync --frozen --no-dev --no-editable
 

HANDOFF.md

@@ -2,42 +2,40 @@
 
 ## Finished
 
-- Isolated the work on `codex/thin-init-cli`; the original `main` checkout remains untouched.
-- Replaced the 232-line `dander init` orchestration body with a typed options object and delegate.
-- Split manifest resolution, safety decisions, stage-zero sequencing, image publication, platform
-  configuration, and Terraform composition into `init_command.py`.
-- Preserved the command signature, output, bootstrap monkeypatch points, and private helper aliases.
-- Kept runtime behavior, package metadata, Terraform, and deployed resources unchanged.
+- Reconciled Josh's current `main` with the complete functional beta-readiness tip.
+- Preserved Josh's three unique commits, capability contracts, tickets, and secret-scan review.
+- Added sanitized run failures, interrupted-run reconciliation, and supported staging expiration.
+- Added plan-first operations, permission preflight, upgrade guidance, and rollback guidance.
+- Added the four-endpoint Salesforce CRM example and five governed models.
 
 ## Try It
 
-```bash
-uv run dander init --help
-uv run pytest tests/cli/test_init_cli.py
-```
+Run `uv run dander validate`, then inspect the combined optional capabilities with
+`uv run dander connector inspect PIPELINE`. The Salesforce example remains under
+`examples/salesforce/` and can be validated source-free from an installed wheel.
 
 ## Checks
 
-- Focused init suite passed: `9 passed`; full suite passed: `748 passed`.
-- Repository-wide Ruff, formatting, and strict mypy passed.
-- `dander init --help` matched `main` byte-for-byte.
-- Dependency audit, platform Terraform, stage-zero Terraform, and generated Terraform passed.
-- Wheel/sdist inspection and external installs passed; the container built and its runtime contract
-  passed.
+- Combined full suite passed: `770 passed`.
+- Ruff formatting/lint and strict mypy passed.
+- Platform and stage-zero Terraform validation passed with backends disabled.
+- Wheel/sdist inspection, external install, source-free generation, and validation passed.
+- The combined Linux container image built successfully.
+- Josh's five GitHub CI jobs passed on the cross-fork draft PR.
 
 ## Decisions
 
-- Keep Typer option declarations in `main.py`; the new module never imports `main.py`.
-- Inject existing bootstrap symbols from `main.py` to retain current test and development seams.
-- Add no duplicate tests because the existing init suite already covers the extracted wiring.
+- Merge exact functional tip `7a1378f`; exclude release-metadata commit `7703567`.
+- Preserve both sides of the decision log and all of Josh's capability implementation.
+- Keep package, lockfile, scaffold, and CI version metadata at `0.5.1`.
 
 ## Remaining
 
-- Protected GitHub CI must repeat Linux package/container checks and security scans.
-- Merge only after review; no deployment, release, Terraform apply, or GCP mutation is in scope.
+- Josh must review the cross-fork draft PR; it must not merge automatically.
+- Release publication, live acceptance, retained-project changes, and schedules remain untouched.
 
 ## Review First
 
-- `src/dander/cli/init_command.py`
-- `src/dander/cli/main.py`
-- `tests/cli/test_init_cli.py`
+- `docs/decisions.md`
+- `src/dander/ingestion/capabilities.py`
+- `src/dander/writer/bigquery.py`

README.md

@@ -217,11 +217,12 @@ credential or employee row is stored in this repository.
 
 ### Enterprise authentication templates
 
-`connectors/salesforce_jwt.example.yaml` is a complete read-only Accounts QueryAll slice: OAuth2
-JWT bearer authentication, Salesforce's distinct authorization-server audience, a short assertion,
-server-filtered SOQL, bounded streaming Bulk API 2.0 result pages, a declared raw schema, and
-soft-delete visibility. Hosted replays use an inclusive `SystemModstamp` watermark and idempotent
-SCD1 publication. Dander 0.5 supports the independently installed first-party
+`connectors/salesforce_jwt.example.yaml` is a complete read-only CRM slice for Accounts, Contacts,
+Opportunities, and Users: OAuth2 JWT authentication, bounded streaming Bulk API 2.0 result pages,
+declared raw schemas, and independent inclusive `SystemModstamp` watermarks. QueryAll retains
+Account, Contact, and Opportunity tombstones; User deactivation is represented by `IsActive`.
+Contact Email and Phone are personal data enabled by default. Dander supports the independently
+installed first-party
 [`dander-connector-salesforce`](https://github.com/harrisonoconnorhover/dander-connector-salesforce)
 plugin; an exact manifest pin takes precedence over the deprecated built-in fallback. See
 [`docs/salesforce.md`](docs/salesforce.md).
@@ -321,11 +322,12 @@ New users may instead use the [$300/90-day Free Trial](https://docs.cloud.google
 While the account remains a Free Trial account, Google says usage is not charged to the payment
 method; manually upgrading makes overages beyond remaining credit and free allowances billable.
 
-`dander init` owns the complete two-stage bootstrap. It creates a hardened/versioned state bucket,
-adopts it into Terraform, creates the administrative identity and Artifact Registry, builds and
-pushes the current runtime, then applies datasets, per-pipeline IAM/jobs/schedules/secrets, and the
-safety policy from `dander.yaml`. Newly generated projects use the ordinary hosted path without the
-optional managed cost guard:
+`dander init` and `dander init --apply` remain compatible shortcuts. The documented installation
+path separates each mutation from its saved Terraform plan: create the state bucket once, run
+`init-admin-plan`, review and run `init-admin-apply`, publish an immutable source-free image with
+`image-publish`, then run `init-platform-plan`, review, and run `init-platform-apply`. See the
+[hosted quickstart](docs/getting-started.md) for copyable commands. Newly generated projects use the
+ordinary hosted path without the optional managed cost guard:
 
 ```yaml
 platform:
@@ -350,45 +352,37 @@ override flags take precedence only when explicitly supplied. The cost guard def
 when `require_guarded_free_tier` is true and disabled when it is false; explicit cost-guard flags can
 override that default when the combination is valid.
 
-```bash
-uv run dander init \
-  --project my-gcp-project \
-  --failure-alert-email operator@example.com \
-  --github-repository owner/repository \
-  --apply
-```
-
-The state bucket defaults to `<project>-dander-state`; the active `gcloud` user becomes the
-approved administrator; operator-only stage-zero artifacts default to `~/.dander/<project>` and
-remain outside the repository. Terraform never receives secret values. Add each named value after
-bootstrap with `gcloud secrets versions add`, then execute a paused pipeline manually before
-enabling its schedule. `--failure-alert-email` is an operator input rather than a manifest field,
-so personal addresses stay out of public repositories; repeat it on later reconciliations to retain
-the email channel and per-pipeline Cloud Run failure policies. Plan-only mode remains available for
-established environments but requires an existing backend, bootstrap identity, and immutable
-`--container-image`.
-
-The granular `init-admin-*` and `init-platform-*` commands remain available for operators who need
-to review/apply each identity boundary separately. The normal path is the single command above.
-The approved administrative identity is deliberately separate from runtime identities; only it
-can provision project resources. Guarded installations additionally use it to delegate each
-runtime's read-only billing visibility.
+The standard installer does not need Project Owner. The one-time stage-zero bundle is Service Usage
+Admin, Storage Admin, Artifact Registry Administrator, Service Account Admin, and Project IAM
+Admin. A cloud administrator can perform that step and grant the operator Service Account Token
+Creator only on Dander's bootstrap account; image publication and later platform plans then use
+that account through impersonation. GitHub WIF additionally requires Workload Identity Pool Admin.
+Terraform never receives secret values. Add each named value after bootstrap with
+`gcloud secrets versions add`, then execute a paused pipeline manually before enabling its schedule.
+`--failure-alert-email` is an operator input rather than a manifest field, so personal addresses
+stay out of public repositories; repeat it on later reconciliations to retain the email channel and
+per-pipeline Cloud Run failure policies.
+
+Every plan command prints its saved-plan location and exact apply command. Every apply command
+applies only that saved plan after confirmation. The approved administrative identity is separate
+from runtime identities; only it can provision project resources. Guarded installations
+additionally use it to delegate each runtime's read-only billing visibility.
 The default unguarded path does not request billing-account IAM or grant runtime billing/Pub/Sub
 guard permissions. Dander does not manage, limit, or prevent cloud spending in that configuration.
 To opt into the managed guard, set `require_guarded_free_tier: true` and pass
 `--billing-account ABCDEF-123456-ABCDEF`; the caller then needs the additional billing-account
 permissions required for the reviewed IAM and budget plan.
 
-For an established environment, the equivalent explicit plan is:
+For an established environment, produce the complete manifest-aware plan with:
 
 ```bash
-uv run dander init \
+uv run dander init-platform-plan \
   --project my-gcp-project \
   --state-bucket my-existing-tfstate-bucket \
   --bootstrap-service-account dander-bootstrap@my-gcp-project.iam.gserviceaccount.com \
   --container-image us-central1-docker.pkg.dev/my-gcp-project/dander/dander@sha256:DIGEST \
   --config dander.yaml \
-  --github-repository owner/repository
+  --failure-alert-email operator@example.com
 ```
 
 The image must use an immutable SHA-256 digest. `dander.yaml` declares every additive pipeline,

connectors/salesforce.yaml

@@ -1,4 +1,4 @@
-# Read-only Accounts connector for Dander's disposable Salesforce Developer Edition proof org.
+# Read-only CRM connector for Dander's disposable Salesforce Developer Edition proof org.
 # The External Client App consumer key and RSA private key are secret references, never literals.
 name: salesforce
 engine: salesforce_bulk2
@@ -32,8 +32,9 @@ endpoints:
     request_body:
       operation: queryAll
       query: >-
-        SELECT Id, Name, Type, Industry, AnnualRevenue, NumberOfEmployees,
-        BillingCity, BillingState, BillingCountry, CreatedDate, LastModifiedDate,
+        SELECT Id, Name, OwnerId, ParentId, Type, Industry, AnnualRevenue,
+        NumberOfEmployees, Website, Phone, BillingCity, BillingState,
+        BillingPostalCode, BillingCountry, CreatedDate, LastModifiedDate,
         SystemModstamp, IsDeleted FROM Account
       contentType: CSV
       columnDelimiter: COMMA
@@ -54,6 +55,8 @@ endpoints:
       - name: Name
         type: STRING
         mode: REQUIRED
+      - {name: OwnerId, type: STRING}
+      - {name: ParentId, type: STRING}
       - name: Type
         type: STRING
       - name: Industry
@@ -62,10 +65,13 @@ endpoints:
         type: NUMERIC
       - name: NumberOfEmployees
         type: INT64
+      - {name: Website, type: STRING}
+      - {name: Phone, type: STRING}
       - name: BillingCity
         type: STRING
       - name: BillingState
         type: STRING
+      - {name: BillingPostalCode, type: STRING}
       - name: BillingCountry
         type: STRING
       - name: CreatedDate
@@ -80,3 +86,113 @@ endpoints:
       - name: IsDeleted
         type: BOOL
         mode: REQUIRED
+
+  - name: contacts
+    path: /jobs/query
+    pagination:
+      kind: header_cursor
+      next_cursor_header: Sforce-Locator
+      cursor_param: locator
+      size_param: maxRecords
+      page_size: 10000
+      terminal_value: "null"
+    request_body:
+      operation: queryAll
+      query: >-
+        SELECT Id, AccountId, OwnerId, FirstName, LastName, Name, Email, Phone,
+        Title, Department, MailingCity, MailingState, MailingPostalCode,
+        MailingCountry, CreatedDate, LastModifiedDate, SystemModstamp, IsDeleted
+        FROM Contact
+      contentType: CSV
+      columnDelimiter: COMMA
+      lineEnding: LF
+    primary_key: [Id]
+    incremental_cursor: SystemModstamp
+    raw_schema:
+      - {name: Id, type: STRING, mode: REQUIRED}
+      - {name: AccountId, type: STRING}
+      - {name: OwnerId, type: STRING}
+      - {name: FirstName, type: STRING}
+      - {name: LastName, type: STRING, mode: REQUIRED}
+      - {name: Name, type: STRING, mode: REQUIRED}
+      - {name: Email, type: STRING}
+      - {name: Phone, type: STRING}
+      - {name: Title, type: STRING}
+      - {name: Department, type: STRING}
+      - {name: MailingCity, type: STRING}
+      - {name: MailingState, type: STRING}
+      - {name: MailingPostalCode, type: STRING}
+      - {name: MailingCountry, type: STRING}
+      - {name: CreatedDate, type: TIMESTAMP, mode: REQUIRED}
+      - {name: LastModifiedDate, type: TIMESTAMP, mode: REQUIRED}
+      - {name: SystemModstamp, type: TIMESTAMP, mode: REQUIRED}
+      - {name: IsDeleted, type: BOOL, mode: REQUIRED}
+
+  - name: opportunities
+    path: /jobs/query
+    pagination:
+      kind: header_cursor
+      next_cursor_header: Sforce-Locator
+      cursor_param: locator
+      size_param: maxRecords
+      page_size: 10000
+      terminal_value: "null"
+    request_body:
+      operation: queryAll
+      query: >-
+        SELECT Id, AccountId, OwnerId, Name, StageName, Amount, Probability,
+        CloseDate, Type, LeadSource, ForecastCategoryName, IsClosed, IsWon,
+        CreatedDate, LastModifiedDate, SystemModstamp, IsDeleted FROM Opportunity
+      contentType: CSV
+      columnDelimiter: COMMA
+      lineEnding: LF
+    primary_key: [Id]
+    incremental_cursor: SystemModstamp
+    raw_schema:
+      - {name: Id, type: STRING, mode: REQUIRED}
+      - {name: AccountId, type: STRING}
+      - {name: OwnerId, type: STRING}
+      - {name: Name, type: STRING, mode: REQUIRED}
+      - {name: StageName, type: STRING, mode: REQUIRED}
+      - {name: Amount, type: NUMERIC}
+      - {name: Probability, type: NUMERIC}
+      - {name: CloseDate, type: DATE, mode: REQUIRED}
+      - {name: Type, type: STRING}
+      - {name: LeadSource, type: STRING}
+      - {name: ForecastCategoryName, type: STRING}
+      - {name: IsClosed, type: BOOL, mode: REQUIRED}
+      - {name: IsWon, type: BOOL, mode: REQUIRED}
+      - {name: CreatedDate, type: TIMESTAMP, mode: REQUIRED}
+      - {name: LastModifiedDate, type: TIMESTAMP, mode: REQUIRED}
+      - {name: SystemModstamp, type: TIMESTAMP, mode: REQUIRED}
+      - {name: IsDeleted, type: BOOL, mode: REQUIRED}
+
+  - name: users
+    path: /jobs/query
+    pagination:
+      kind: header_cursor
+      next_cursor_header: Sforce-Locator
+      cursor_param: locator
+      size_param: maxRecords
+      page_size: 10000
+      terminal_value: "null"
+    request_body:
+      operation: query
+      query: >-
+        SELECT Id, Name, Alias, UserType, ProfileId, IsActive, CreatedDate,
+        LastModifiedDate, SystemModstamp FROM User
+      contentType: CSV
+      columnDelimiter: COMMA
+      lineEnding: LF
+    primary_key: [Id]
+    incremental_cursor: SystemModstamp
+    raw_schema:
+      - {name: Id, type: STRING, mode: REQUIRED}
+      - {name: Name, type: STRING, mode: REQUIRED}
+      - {name: Alias, type: STRING, mode: REQUIRED}
+      - {name: UserType, type: STRING, mode: REQUIRED}
+      - {name: ProfileId, type: STRING, mode: REQUIRED}
+      - {name: IsActive, type: BOOL, mode: REQUIRED}
+      - {name: CreatedDate, type: TIMESTAMP, mode: REQUIRED}
+      - {name: LastModifiedDate, type: TIMESTAMP, mode: REQUIRED}
+      - {name: SystemModstamp, type: TIMESTAMP, mode: REQUIRED}