Toolstack uplift: security, reliability, Clear desk UI and local signed Mac app

.github/workflows/release-checks.yml

@@ -0,0 +1,29 @@
+name: Release packaging checks (no publishing)
+on:
+  pull_request:
+  workflow_dispatch:
+permissions:
+  contents: read
+concurrency:
+  group: release-checks-${{ github.ref }}
+  cancel-in-progress: true
+jobs:
+  wheel:
+    runs-on: ubuntu-latest
+    env:
+      UV_CONCURRENT_BUILDS: '1'
+      UV_CONCURRENT_INSTALLS: '1'
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v6
+        with:
+          version: '0.12.23'
+      - run: uv python install 3.11
+      - run: uv lock --check
+      - name: Check deployment export drift
+        run: |
+          uv export --frozen --extra admin --extra vault --no-dev --no-emit-project --no-hashes -o admin/requirements-lock.txt
+          git diff --exit-code -- admin/requirements-lock.txt
+      - run: uv run --no-project --python 3.11 python tests/release/check_release.py
+      - run: uv run --no-project --python 3.11 python tests/release/test_deployment.py
+      - run: bash -n deploy/install.sh deploy/redeploy-toolstack

.gitleaksignore

@@ -0,0 +1,5 @@
+# Source-integrity SHA-256, not a credential: verification.json line 16 records
+# macapp/Sources/ToolstackKit/ApiClient.swift at baseline 06df1b15c0c8ac996ec8ce4851e674199b124393.
+# Independently matched to SHA-256 of git show BASELINE:path. Exempt only this
+# archived finding fingerprint; no path, rule or digest-wide exemption.
+9f36f44775e4b187ac1cc6ee1155beddf5184237:docs/ui-directions/iterations/01-superseded.zip!evidence/verification.json:generic-api-key:16

README.md

@@ -1,81 +1,41 @@
-<p align="center">
-  <img src="macapp/packaging/AppIcon-source.png" alt="Toolstack logo" width="150" height="150">
-</p>
-
 # Toolstack
 
-**Trust agents with action, not access.** An agent can *ask*; the broker *decides*; tools
-*execute*; secrets stay with the tools. The agent can reach the broker, and nothing else.
-
-The security comes from physical boundaries (a single ingress, loopback binding, and
-secrets resolved at the workload), not from rules on paper. The agent carries only a
-low-power token to the broker; everything sensitive lives behind it.
-
-## Components
-
-- **Broker**: the authority boundary and the only address the agent has: auth,
-  policy, request lifecycle, approval orchestration, and audit. One process, one
-  SQLite file. (Internally split into module seams; see the architecture doc.)
-- **Toolyard**: container lifecycle + per-tool secret resolution at container start.
-- **Tool template**: what a new tool needs (drop a `toolyard.toml`, run `toolyard up`).
-- **Approval adapter**: a broker module that talks to nod; pluggable via
-  [docs/approval-surface-adapter.md](docs/approval-surface-adapter.md).
-- **Agent client**: the generic `toolstack` CLI + skill (and an MCP adapter) an
-  agent uses to discover and call tools through the broker; see
-  [client/SKILL.md](client/SKILL.md).
-- **Admin web app**: the operator's control panel: run the broker, manage
-  callers/tokens/policies, author/edit tools (form → `toolyard.toml`), and watch
-  requests + audit. Local/homelab, loopback-only; see [admin/README.md](admin/README.md).
-- **Operator apps**: a cross-platform desktop shell ([desktop/](desktop/), an OS-WebKit window
-  around the admin) and a native macOS app ([macapp/](macapp/), SwiftUI over the JSON API).
-
-## You also run
-
-- **nod**: self-hosted approval surface for human-in-the-loop decisions.
-- **Secret backend**: Infisical or SOPS.
-- **Tailnet**: Tailscale Serve (or any VPN); the ingress boundary.
-
-## Run it
-
-The fastest way to bring up the whole stack (broker + admin panel) is the one-box Docker setup:
-
-```bash
-cd deploy/docker
-cp .env.example .env       # set an admin password + vault passphrase
-docker compose up
-```
-
-Open the admin panel at http://127.0.0.1:8780 and drive everything from there. For a native
-single-host install under systemd, run `sudo deploy/install.sh` (it creates the service user,
-the virtualenv, the admin password, and the unit) or follow [deploy/README.md](deploy/README.md).
-
-## The CLIs
-
-The broker, toolyard, and client are stdlib-only Python (3.11+, for `tomllib`). `pip install -e .`
-puts the commands on your PATH:
-
-```bash
-pip install -e .
-toolstack --help
-```
-
-That installs `toolstack` (agent client), `brokerctl` (operator), `toolyard` (tool runner), and
-`toolstack-mcp` (the client as an MCP stdio server); each also runs uninstalled as
-`python3 -m client.toolstack`, and so on. **Installing the CLIs does not start anything**: they
-talk to a broker you run via one of the paths above. Only the admin app carries runtime deps
-(FastAPI/uvicorn), in its own venv; see [admin/README.md](admin/README.md).
-
-## Maturity
-
-The full path runs end to end (agent → broker → human approval → tool execution, with the
-broker never holding a secret) and it's hardened for a single-host deployment: a systemd
-state directory and sandbox, login throttling, bind and SSRF guards, runtime timeouts with
-partial-failure cleanup, pinned dependencies, and database backups. It binds loopback by
-default; reach it over a tunnel that terminates TLS. A few things are deliberately deferred:
-tmpfs secret injection, just-in-time grants, and a background approval-expiry sweeper.
-
-## Docs
-
-- [docs/walkthrough.md](docs/walkthrough.md), the system end to end: how it runs, the security properties, and the evidence.
-- [docs/component-decomposition.md](docs/component-decomposition.md): architecture, broker internals, and trust boundaries.
-- [admin/README.md](admin/README.md) · [broker/README.md](broker/README.md) · [toolyard/README.md](toolyard/README.md): per-component detail.
+Brokered actions for agents: callers receive scoped action permissions, while
+authorized tools retrieve workload credentials through SPS. Web admin controls
+callers, policies and trusted tools. The broker and SPS make separate decisions.
+
+**Supported release boundary: a dedicated Toolstack VM containing only Toolstack
+and trusted tools under its OS account. Agents stay outside that host and reach
+only the broker.** No VM setup is performed by these source changes. Same-machine
+or same-account agents (including on a Mac) are local/development use only and
+do not isolate secrets. Process mode is not hostile-code isolation. Secrets
+exist as plaintext in endpoint memory and can be returned by tool responses.
+Private files and TLS do not isolate processes sharing the same unrestricted uid.
+
+- [Runtime support matrix](docs/runtime-matrix.md)
+- [Integrated security contracts and remaining release gates](docs/security-contracts.md)
+- [Deployment and offline SPS provisioning](deploy/README.md)
+- [Fake-config walkthrough](docs/walkthrough.md)
+- [SPS architecture](docs/sps.md)
+- [Local Mac signing gate](docs/mac-local-signing.md)
+- [Release notes and pending validation](docs/release-notes.md)
+
+The Linux installer and systemd deployment are untested.
+The isolated retained-wheel full-stack check on this Mac **passed**, including admin
+restart/persisted recovery and independently verified owned cleanup. This closes
+the bounded clean-install check, using fake state and fake Nod, not real Nod,
+signature or human approval proof. No GUI checks ran. See the release notes for
+its scope. Manual hands checks have not run and remain OFF pending fresh
+principal go through chief; no release acceptance is implied.
+
+Python 3.11+ is required. Build with `uv build`; the wheel includes admin, broker,
+client, toolyard, SPS/plugins and forwarder. Admin dependencies and localfile crypto
+are optional extras (`.[admin,vault]`). Full deployment dependencies are resolved in
+`uv.lock` and exported to `admin/requirements-lock.txt`; deployment scripts consume
+that export consistently as requirements and constraints.
+
+Run `uv run --no-project --python 3.11 python tests/release/check_release.py` for
+isolated packaging verification. Release workflow checks/builds only: no tags,
+publishing, GHCR, signing or release creation. Full fake-secret E2E is a separate
+integration gate. Docker/Seatbelt with SPS and mutable Linux privileged helpers
+are unsupported. Source build/sign scripts do not establish distribution readiness.

admin/README.md

@@ -39,8 +39,9 @@ a relative tools root. Then sign in and:
    command/port or image, operations, and secret *declarations*) into a directory you
    name on the server. REST tools can be authored directly or bootstrapped from a
    pasted OpenAPI/Swagger spec. **Edit** loads an existing tool back into that form; **Remove**
-   unregisters a panel-added tool (stops it and drops it from the broker's search path;
-   its files stay on disk). Start/stop tools, then restart the broker to register a new
+   unregisters an external tool directory (stops it and removes that configured source;
+   its files stay on disk), or deletes a known managed folder. Unknown ownership refuses
+   removal; another source may still discover the same tool. Start/stop tools, then restart the broker to register a new
    or changed one (the broker reads its registry at startup). You bring the tool's code
    (a process `command`/`app.py`, or a Docker image); the panel writes only the manifest.
 
@@ -59,8 +60,8 @@ by binding a public interface.
   failed attempts are audited (`admin.login_failed` with the IP + attempted username, never
   the password).
 - Secrets stay off the browser: the nod token is **write-only / masked**, and tool
-  secret *values* are never entered here (the toolyard resolves them from the on-disk
-  secrets file); the tool editor authors only secret *declarations* (name + field).
+  secret *values* are never entered here (tools procure them through SPS);
+  the tool editor authors only secret *declarations* (name + field).
 - The tool editor writes to a directory you name (operator-trusted, loopback-only); it
   only ever writes a file called `toolyard.toml`, and registers the directory in the
   run-config's `tool_dirs` so the broker discovers it.
@@ -96,3 +97,89 @@ round-trip, a **real broker** start/stop via the supervisor, the toolyard listin
 tool authoring (build/validate a `toolyard.toml` and confirm the broker registry can
 read it), and a full login → create-caller → set-policy flow through FastAPI's
 `TestClient`.
+## Current operator contracts
+
+The implemented Clear desk web UI uses 38px controls, real HTML selects and
+accessible policy labels. Failed loads disable saving; failed saves retain drafts.
+Unknown outcomes require fresh read/comparison before retry, never automatic effect
+replay. Policy adapters preserve advanced/unavailable grants; the broker's underlying
+full-replacement policy semantics are unchanged.
+
+### Catalog, removal and authoring
+
+Public `GET /v1/health` returns `status: ok` and optionally a random opaque
+`catalog_observation_id` only: no fingerprint, tool names, paths or secret blocks.
+Authenticated `GET /api/catalog` returns `{status: verified, observation_id: <opaque hex>,
+tool_ids: [...]}` from the same fingerprint-verified Registry snapshot, including
+zero-operation tools, or `{status: unknown}`. An empty list is a valid global
+observation but verifies no target. Missing, legacy or malformed membership means
+target unknown. Each row/target needs membership in that same receipt;
+`/api/tools` discovery is independent and is not registration proof.
+
+Files added, tool Start, and broker registration are separate explicit stages.
+Adding never starts a tool or grants authority automatically. Registration proves
+neither process liveness nor health. `DELETE /api/tools/{id}` returns
+`{removed: id, deleted: false}` for external unregister/files retained and
+`deleted: true` for managed-folder deletion. Missing deletion metadata leaves the
+filesystem outcome unknown; other duplicate sources may still discover the tool.
+
+OpenAPI `auth_headers` are caller-envelope hints, not stored automatic injection.
+Preserve `allowed_headers` and secret declarations; callers supply
+`{{secret:NAME}}` placeholders in request headers. Never enter actual secret values
+in the operator UI.
+
+At source 855da6e (broker a9319b9 / release059dd13), see [integrated security contracts](../docs/security-contracts.md)
+for disclosure and remaining gates. Optional operation approval_fields is preserved
+absent versus []; both visibly omit ALL arguments. Pointers explicitly disclose
+selected subtrees, not universally safe free text. No UI disclosure controls were
+added. The shared validator now rejects '*' anywhere; '/' is a valid empty key,
+empty root pointer invalid. Refresh needs ordinary grants; token revocation needs 12–64 lowercase hex.
+Policy changes invalidate pending approvals; manifest generation is rechecked at
+both immediate/deferred claims (the executing claim is the cancellation commit point).
+Schema3 receipts distinguish received/pending_approval/executing/unknown/result_expired,
+with idempotency_key/reused/payloads_pruned and wait-timeout broker_status fields;
+consumers must adopt them and conflict409 rather than label everything pending.
+Matching keys return old receipts even after generation changes, never new effects.
+Historical output requires a live current caller/exact presented token and current
+allow OR review grant; review is historical read authority, not a new approval.
+Deny/removal/revoked token withholds output, not bytes already returned.
+
+Manual prune-payloads/backup/reconcile-offline/restore-new commands are documented
+in [security contracts](../docs/security-contracts.md#manual-retention-and-offline-recovery).
+Stop ALL broker/admin/CLI writers before confirming offline reconcile/restore;
+ordinary Store opening never relabels active work. NEW-destination restore revokes
+ALL tokens, clears ALL policies and rotates UUID, never overwrites source or replays.
+Keep restored authority revoked until external/post-snapshot history is reconciled;
+never blindly replay queues. Seven-day pruning preserves tombstones, not secure
+erasure. Native canonical fixture/status adoption is implemented, not full web parity proof.
+
+Password reset rotates the session generation atomically. Running admin instances
+reject old browser cookies and bearer tokens on their next request, without a
+restart. An already-authorized in-flight action is not cancelled by reset.
+
+Configuration audit records changed schema field names, never values, tokens,
+paths or destinations. An intent audit must succeed before saving. If completion
+auditing fails after saving, the operation fails with an explicit already-saved
+error; verify before retrying. Authenticated host administrators can delete the
+local audit (HTML also requires session-bound CSRF). Deletion leaves a new
+`audit_cleared` marker where storage succeeds, but is not secure erasure or
+tamper-proof evidence. A marker-write failure cannot undo the deletion.
+
+Blocking tool import/update, tool lifecycle and broker process control use one
+off-loop worker with immediate busy rejection, not an unbounded job queue.
+Cancelling a request does not free admission until its actual work ends.
+No automatic replay or cancellation of already-running external actions is implied.
+
+Native/API contract fixtures live in `admin/tests/fixtures/native_api.json`.
+Backend identity uses `plugin`, not `name`; unknown secret inspection is
+`status: UNKNOWN` with `provisioned: null`. Liveness is distinct from recorded
+running intent and from health; no health probe means health is unknown, not
+healthy. Failed liveness inspection is nullable/unknown. Native strict decoders
+adopt these contracts. Old API secret-value provisioning returns 410 and
+does not parse or store values; provisioning is host-side only.
+
+Lifecycle mutations require the integrated runner `state_lock` across the
+complete load/mutate/save operation. Missing lock support fails closed. A newly
+started runtime is stopped if its record cannot be published; failed cleanup
+returns explicit failure and retains a runtime reference for recovery. State
+and channel-view publication are not a single filesystem transaction.

admin/__main__.py

@@ -29,7 +29,7 @@ def _set_password(args) -> None:
     settings.write_password_hash(auth.hash_password(pw))
     settings.rotate_session_secret()  # a password change logs out existing sessions
     print(f"admin password set ({settings.password_hash_file()}); existing sessions invalidated "
-          "(restart the admin to apply).")
+          "on their next request.")
 
 
 def _serve(args) -> None: