Isolate plugin commands and refresh vulnerable runtime dependencies

Dockerfile

@@ -26,14 +26,18 @@ RUN apt-get update \
     && apt-get install --yes --no-install-recommends \
         libpq5 \
         bsdutils \
+        gzip \
         libblkid1 \
         liblastlog2-2 \
         libmount1 \
+        libpcre2-8-0 \
         libsmartcols1 \
+        libsqlite3-0 \
         libuuid1 \
         login \
         mount \
         openssl \
+        perl-base \
         util-linux \
     && rm -rf /var/lib/apt/lists/*
 RUN pip install --no-cache-dir uv==0.12.0

HANDOFF.md

@@ -2,31 +2,38 @@
 
 ## Finished
 
-- Reconciled retained August 27–28 Control/Fargate/Spark successes and corrected the obsolete DANDER-251 non-execution claim.
-- Updated five completed tickets, Druff hosted-mode documentation, and Salesforce plugin write-command boundaries.
-- Prepared Salesforce 0.3.2, ServiceNow 0.2.3, and Dander catalog/install-check patches outside this documentation change.
+- Scanned repository structure, duplication, import boundaries, error handling, and test health; recorded scope and remaining priorities in `docs/quality-review-2026-09-30.md`.
+- Isolated plugin commands so help and catalog search avoid bootstrap, deployment, and provider modules.
+- Shared the existing durable-run identity rule between S3 and PostgreSQL, with storage behavior preserved.
+- Updated four dependencies after protected CI identified 20 published advisories; the strict runtime dependency audit is now clean locally.
+- Refreshed flagged system packages in runtime/starter, Spark, and OCI Dockerfiles; the runtime OS layer and complete Spark/OCI images pass local scans.
 
 ## Try It
 
-Read `docs/control-execution-history.md`, then `docs/support-status.md`. Prepared package and catalog patches are in `/tmp/dander-curated-compat-20260930/`.
+Run `uv run dander plugins --help` and `uv run dander plugins search incident`. Read the quality review for findings and the limits of the scan.
 
 ## Checks
 
-- Documentation: relative links, retained output equality, source/image identities, and `git diff --check` passed.
-- Prepared fixes: 93 Salesforce and 23 ServiceNow tests passed on public Dander 0.7.1 and 0.9.0rc20, current 0.9.0rc32, and each package's locked environment. Package lint, format, and typing passed.
-- Prepared Dander patch: 37 focused tests, full Ruff lint/format, canonical strict typing (507 files), contract drift, and outside-checkout wheel resolution/metadata/entry-point checks passed using local candidate wheels.
+- Full suite with disposable local PostgreSQL 17: **2,419 passed**, one existing Starlette/httpx deprecation warning.
+- Focused CLI/storage/lifecycle suite: **75 passed**; final CLI test-import adjustment: **10 passed**.
+- Updated dependencies: **139 authentication/ingestion tests passed**; strict `runtime-all` dependency audit reports **no known vulnerabilities**.
+- Runtime OS layer and Spark/OCI images: linux/amd64 builds and Trivy scans passed with **zero fixable high/critical findings**. User/driver/handler checks passed. Scaffold/release/import tests: **15 passed**; OCI image/publication tests: **10 passed**.
+- Ruff lint/format, canonical strict typing (**509 files**), Control contract drift, documentation links, and diff checks passed.
+- Compared moved command bodies and identity comparisons structurally against the original code; behavior-bearing expressions match. Protected CI and merge results are available on the task's attached quality-review PR.
 
 ## Decisions
 
-- Retained live evidence qualifies only its named revisions and workloads; no new cloud runs or support promotion.
-- Do not publish catalog pins until the corresponding packages exist on PyPI.
+- Use the existing lightweight command-dispatch pattern; preserve command options, installer pins, and storage schemas.
+- Keep provider-specific SQL and recovery explicit; defer broader extraction until changes demonstrate a shared responsibility.
 
 ## Remaining
 
-- Await the requested narrow exception to AGENTS.md's single-writable-repository rule for the two existing connector repositories.
-- Then apply the prepared connector patches, complete protected CI/publication, verify PyPI, apply the Dander catalog patch, and merge with exact-main CI.
+- Large CLI/bootstrap and warehouse/deployment modules remain the next maintenance targets; see the review's specific recommendations.
+- Curated connector releases remain separately blocked on the requested exception to the single-writable-repository rule. Prepared patches remain in `/tmp/dander-curated-compat-20260930/` and the preserved connector branch.
+- Existing test-client deprecation warning remains; no live cloud qualification was rerun.
 
 ## Review First
 
-- `docs/control-execution-history.md` and corrected support boundaries.
-- `/tmp/dander-curated-compat-20260930/*-compatibility.patch` for the pending package work.
+- `docs/quality-review-2026-09-30.md` for the assessment and scope.
+- `src/dander/cli/plugins_command.py` and `src/dander/cli/entrypoint.py` for command isolation.
+- `src/dander/control/orchestration.py` for the shared run identity rule.

docs/decisions.md

@@ -1,5 +1,16 @@
 # Engineering Decisions
 
+## 2026-09-30 — Keep command loading and run identity at their owning boundaries
+
+- Plugin commands live in `cli/plugins_command.py`. The console dispatcher selects their group
+  directly, like hosted Control; project configuration is loaded only by installation so help and
+  catalog search do not depend on deployment modules. CLI options and outputs remain compatible.
+- `same_run_identity` lives beside the provider-neutral `RunStore` contract. S3 and PostgreSQL use
+  the same existing 14-field comparison; storage schemas and lifecycle transitions are unchanged.
+- Refresh the four dependencies flagged by protected CI without audit exclusions. Raise the direct
+  PyJWT minimum to 2.15.1 so package consumers also receive its fixes; keep transitive updates in
+  the lockfile and verify the full runtime requirements with the existing strict audit.
+
 ## 2026-09-08 — Recover pending PostgreSQL runs without reading completed history
 
 - PostgreSQL Control schema v4 derives `needs_reconciliation` from canonical run records and

docs/quality-review-2026-09-30.md

@@ -0,0 +1,74 @@
+# Code quality review — September 30, 2026
+
+Reviewed from `425045ff2bbd5ba8b6d6d8eec3c28fb0f59bf525`, with changes tracked in
+[DANDER-285](../tickets/DANDER-285-quality-review-command-boundaries.md).
+
+## Assessment and scope
+
+The codebase has a sound core: typed domain contracts, explicit provider boundaries, conditional
+storage updates, and substantial tests. Its main maintenance pressure is concentrated in large
+command, orchestration, and provider modules. A wholesale rewrite is not justified by this review.
+
+The inventory covered 1,409 tracked files. Python structural analysis covered the tracked Python
+sources and tests; the runtime inventory contained 270 modules and approximately 80,000 lines,
+excluding scaffold templates. Searches examined duplicate function bodies, large modules/functions,
+broad exception handlers, unfinished markers, typing suppressions, and subprocess boundaries.
+
+Manual review followed selected paths through CLI dispatch, plugin loading, Control HTTP and run
+storage, graph mutation, pipeline execution, ingestion, warehouse staging, assertions, BigQuery
+telemetry/fencing, and CI scope selection. This combines a whole-repository scan with targeted
+reading; it is not a line-by-line review of every file or new live-cloud qualification.
+
+## Changes
+
+| Finding | Change | Evidence |
+| --- | --- | --- |
+| `plugins search` loaded 13 bootstrap, 3 deployment, and 55 provider modules via the root CLI. Offline discovery depended on unrelated command code. | Move install/scaffold/search into `cli/plugins_command.py`; dispatch the plugin group directly and load project configuration only for installation. | New subprocess tests failed before the change and pass afterward. Help/search import none of those module families. Existing CLI tests retain exact pins, version-2 manifests, scaffold behavior, and catalog filtering. |
+| S3 and PostgreSQL stores duplicated the same 14-field durable-identity comparison. A future change could update only one implementation. | Put `same_run_identity` beside the `RunStore` contract and remove the duplicated implementations and redundant run-ID checks. | Both backend tests reject a changed plan revision, retain the original stored snapshot, and still accept lifecycle transitions with conditional revisions. |
+| Protected CI found 20 published dependency advisories in the existing lockfile. | Update only AnyIO 4.14.1 → 4.14.2, urllib3 2.7.0 → 2.8.0, PyJWT 2.13.0 → 2.15.1, and OAuthLib 3.3.1 → 4.0.0. Raise Dander's direct PyJWT minimum to 2.15.1. | The strict audit of the exported `runtime-all` requirements reports no known vulnerabilities; 139 authentication and ingestion tests passed with the updates. |
+| Image scans found 13 high/critical findings in the runtime's inherited Debian packages, 3 in Spark, and 16 in OCI. | Refresh `gzip`, `libpcre2-8-0`, `libsqlite3-0`, and `perl-base` in runtime/starter Dockerfiles; refresh PCRE in Spark and `expat`, `libevent`, and `libxml2` in OCI. Keep existing bases and scan rules. | The linux/amd64 runtime OS layer and complete Spark/OCI images build and scan with zero fixable high/critical findings. Patched versions, runtime users, Spark driver identity, and OCI handler imports were checked. The 15 scaffold/release/import and 10 OCI image/publication tests pass. Protected CI also checks the complete runtime image. |
+
+No new dependency, schema, provider payload, or connector package pin was introduced. The four
+dependency updates address the audit findings; they do not imply that every advisory was exploitable
+through Dander. The failing audit is retained in [the initial PR run](https://github.com/harrisonoconnorhover/dander/actions/runs/36795849992).
+
+Compatibility review used the upstream [AnyIO advisory](https://github.com/agronholm/anyio/security/advisories/GHSA-82r6-8w77-94w6),
+[urllib3 release notes](https://github.com/urllib3/urllib3/releases/tag/2.8.0),
+[PyJWT release](https://github.com/jpadilla/pyjwt/releases/tag/2.15.1), and
+[OAuthLib release notes](https://github.com/oauthlib/oauthlib/releases/tag/v4.0.0).
+OAuthLib's documented breaking changes concern server grant validation and token revocation;
+Dander receives it through the Google authentication client's dependency chain. Dander's OIDC
+decoder already supplies fresh verification options on every call.
+
+## Existing protections confirmed
+
+- Object-store graph mutation rules and generic assertion planning already have shared owners.
+- BigQuery ingestion and graph work already use the shared job-measurement collector.
+- The inspected Control mutation routes offload synchronous storage work from the async handler.
+- PostgreSQL recovery has a pending-run query; it does not rely on scanning all completed history.
+- CI scope detection includes deleted files and both sides of renames.
+- Reviewed broad exception handlers at plugin and file boundaries convert external failures or
+  clean temporary files; they do not justify a blanket exception-handling rewrite.
+
+## Remaining maintenance priorities
+
+1. **CLI and bootstrap composition:** `cli/main.py` still owns several command groups and wide
+   bootstrap signatures. When those commands next change, extract one cohesive group or pass the
+   existing typed options object through the relevant boundary. Preserve flags and defaults.
+2. **Large provider implementations:** warehouse writers and deployment renderers remain large.
+   Extract demonstrated shared semantics; keep SQL dialects, transactions, and provider-specific
+   recovery explicit. Similar-looking SQL is not enough reason to add a generic writer framework.
+3. **Support evidence:** passing tests do not close the separately tracked curated-package
+   compatibility release or prove new cloud configurations. Use [support status](support-status.md)
+   and [execution history](control-execution-history.md) for those boundaries.
+
+## Verification
+
+- Baseline: 2,416 tests passed with a disposable local PostgreSQL 17 database; Ruff lint/format,
+  canonical strict typing, and generated Control contract drift checks passed.
+- Changed-path validation: 75 plugin CLI, console dispatch, S3/PostgreSQL storage, and lifecycle
+  tests passed. The new import tests demonstrated the pre-change coupling before passing.
+- Final full-suite and protected CI results are recorded in the handoff and pull request.
+- The full 2,419-test suite and strict typing passed again after the Python dependency updates.
+- The baseline and focused runs emitted one existing Starlette/httpx test-client deprecation
+  warning. No live provider workload was submitted by this review.

infra/oci/controller/Dockerfile

@@ -14,7 +14,7 @@ RUN pip3 install --target /python --no-cache-dir --no-deps /tmp/dander_platform-
 
 FROM fnproject/python:3.12@sha256:fa298b94e95a54e6d10c54aa5691fefdb01c76c5c38b3da352a2f9dc0fbdf739
 
-RUN microdnf update -y curl libcurl python3.12 python3.12-libs sqlite-libs \
+RUN microdnf update -y curl expat libcurl libevent libxml2 python3.12 python3.12-libs sqlite-libs \
     && microdnf clean all
 
 WORKDIR /function

infra/spark/Dockerfile

@@ -16,6 +16,7 @@ RUN apt-get update \
     && apt-get install --yes --no-install-recommends \
         ca-certificates \
         libjemalloc2 \
+        libpcre2-8-0 \
         procps \
         tini \
     && groupadd --gid 1099 spark \

pyproject.toml

@@ -31,7 +31,7 @@ dependencies = [
   "pyyaml>=6.0",
   "fastapi>=0.115",                # hosted Control API transport
   "uvicorn>=0.30",                 # hosted Control API server
-  "pyjwt[crypto]>=2.13,<3",        # hosted OIDC access-token verification
+  "pyjwt[crypto]>=2.15.1,<3",      # hosted OIDC access-token verification
 ]
 
 [project.urls]

src/dander/cli/entrypoint.py

@@ -1,4 +1,4 @@
-"""Lightweight console dispatcher that keeps hosted Control startup provider-free."""
+"""Load hosted Control and plugin commands without the full deployment CLI."""
 
 from __future__ import annotations
 
@@ -25,6 +25,16 @@ def dispatch(arguments: Sequence[str]) -> None:
             hosted_app(args=args, prog_name="dander")
             return
 
+        if args[:1] == ["plugins"]:
+            import typer
+
+            from dander.cli.plugins_command import plugins_app
+
+            plugin_app = typer.Typer()
+            plugin_app.add_typer(plugins_app, name="plugins")
+            plugin_app(args=args, prog_name="dander")
+            return
+
         from dander.cli.main import app
 
         app(args=args, prog_name="dander")