From ab2916c9ec5f93bc97bdd9b6d7b8b744169390e3 Mon Sep 17 00:00:00 2001 From: John Gruber - F5 Architect Date: Mon, 24 Aug 2026 00:15:10 -0500 Subject: [PATCH] Integrate #177 follow-up stack: #179 #180 #181 #182 #183 #186 #188 (#193) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Integrate the #177 follow-up stack (#179 #180 #181 #182 #183 #186 #188) Consolidated landing of seven interdependent PRs whose shared credential and release/CI surfaces prevented merging in any order (see issue #192's conflict matrix). Merged in dependency order 179, 180, 181, 188, 186, 183, 182; the #186/#188 credential surface was reconciled once (single reserved-name guard; provenance + migrations + stale-disable combined with rotation + backend MCP wiring + threadpool). Squashed to one commit; per-PR history retained on the seven archived branches. Validated on the merged tree: ruff + mypy clean; 172 auth/credential/startup/ migration tests pass; single alembic head v2_155; openapi + frontend types fresh; helm lint/template and docker compose config green on all modes; version and detector self-tests green; commit-message lint clean. Closes #192. Claude-Session: https://claude.ai/code/session_01UpRYiFserdBE5ESHn759N4 * fix(#193): reconcile the merged unset-MCP-password model and harden the credential seed Addresses mwiget's three CHANGES_REQUESTED blockers on the #177 integration PR (#193), plus follow-up findings from a max-effort review of the same credential surface. The merge chose #188's "unset MCP_SERVICE_PASSWORD -> disable" over #186's "unset -> generate", so the generate/rotate-on-unset code was left unreachable but still documented, and the release-notes footer was missing. BLOCKING 2 (core/config.py): the MCP_SERVICE_PASSWORD comment claimed ensure_service_user "generates a random secret and surfaces it once" when unset, contradicting the merged behaviour. Rewrote it to state the truth: when unset (or a known default) seed_auth_step calls disable_stale_service_user, leaving the mcp account disabled/unavailable until an operator configures a real password; a published default is refused and rotated out; the backend receives MCP_SERVICE_PASSWORD on every deploy mode and reconciles to it when set. Also removed the duplicate #186 block that sat above the wrong field. BLOCKING 3 (services/auth_service.py): removed the unreachable usable_password is None branches from ensure_service_user (the generate-on-create and rotate-on-unset paths). seed_auth_step only calls it under the _mcp_pw_usable gate, so password is never None/default in production. ensure_service_user now requires a usable password and only creates/reconciles with it (failing closed and loudly if handed an unusable one); the unset case is owned entirely by disable_stale_service_user. Dropped the now-dead _log_generated_service_password helper and the service-account token_urlsafe/_persist_generated_password calls (_persist_generated_password is still used by the admin seed). Kept the reserved-name guard, the provenance check, the adopt-a-published-default remediation, and disable_stale_service_user fully intact. Updated the affected unit tests (published-default/None now refused; added a reachable adopt-and-reconcile test; stale-row setup builds the legacy row directly) and fixed scripts/mcp_live_smoke.py, which pointed operators at /app/keys/initial_mcp_password, a file no reachable path writes. CR-1 (services/auth_service.py, seed_admin_user): fixed a concurrent-first-boot admin lockout. With DEFAULT_ADMIN_PASSWORD unset and 2+ api replicas, both generated different passwords and the loser overwrote the keys file while its INSERT rolled back, so the file and the committed row disagreed. The fresh seed now creates+flushes first (the loser's INSERT raises IntegrityError -> rollback, no file write) and persists the keys file only after winning but before commit, so the file can only ever hold the committed row's password. Added a losing-replica test. CR-5 (services/auth_service.py, _persist_generated_password): os.open's 0600 mode only applies on create, so a pre-existing 0644 file was truncated in place and kept 0644, writing the secret world-readable. Added os.fchmod(fd, 0o600) and a test that a pre-existing 0644 file is tightened to 0600. CR-2 (routes/k8s_websocket.py, dpus_websocket.py, benchmarks.py): the WS auth validators called the blocking sync token_user_state directly on the event loop. Moved it off the loop via run_in_threadpool, matching core/auth_middleware.py. Validation: make lint-backend clean; mypy core/ schemas/ unchanged; auth (57) + ws/benchmark/startup (79) suites pass; alembic heads single v2_155; helm lint/template OK and --set secrets.mcpPassword=changeme fails the render; docker compose config OK on all modes; extract-breaking-changes and compute_version_bump self-tests pass; lint-commit-markers clean. BREAKING CHANGE: MCP_SERVICE_PASSWORD is now required — the backend SystemExits under ENVIRONMENT=staging|production when it is unset or a known default; the shipped admin/changeme default is removed and rotated out on upgrade; the dist bundle renames MCP_USERNAME/MCP_PASSWORD to MCP_SERVICE_USERNAME/MCP_SERVICE_PASSWORD; and the Helm chart ships mcpUsername=mcp with a generated mcpPassword instead of admin/changeme. Claude-Session: https://claude.ai/code/session_01UpRYiFserdBE5ESHn759N4 * fix(#193): close bonnyr-f5's credential/backend/helm/dist/docs blockers Follow-up to the #177 integration on pr177-integration, addressing the CREDENTIAL/BACKEND/HELM/DIST/DOCS half of bonnyr-f5's BLOCK on PR #193. B1 (SECURITY): ensure_service_user no longer adopts any human account whose password is a known default. The adoption exception is now scoped to v2_155's exact backfill fingerprint (username=='mcp' AND email=='mcp@bnk-forge.local'), matching the migration's own conservative rule, and must_change_password is no longer cleared on an adopted row. Adds tests proving a human operator/changeme row (and a wrong-email mcp row) is REFUSED, not taken over. B2/B2b: all five compose files (root + dist docker-compose{,.local}.yml and the IBM embedded compose) honor legacy MCP_USERNAME/MCP_PASSWORD as aliases for MCP_SERVICE_USERNAME/MCP_SERVICE_PASSWORD on both backend and mcp env, so an existing customer .env keeps working after upgrade. Docs (dist/README.md, dist/.env.example, user-pack/install-guide.html) document MCP_SERVICE_PASSWORD as canonical with MCP_PASSWORD honored as a legacy alias. B3: ENVIRONMENT is plumbed to the backend in every compose file, so an operator who sets ENVIRONMENT=staging|production actually reaches config.py's MCP fail-fast. Helm already routes ENVIRONMENT=production onto api/worker/beat. M1: the unset-MCP behavior stays "disabled" (#188 over #186); added an explicit deliberate-consolidation comment at the decision point. M2: disable_stale_service_user skips the about-to-be-reconciled row and the "no usable MCP_SERVICE_PASSWORD" warning is conditional on a configured password, so a correctly-configured install no longer logs a false warning or commits an inactive MCP window on every boot. M7: the Helm mcp deployment and the IBM installer's mcp service now run the exec auth-probe healthcheck (python -m bnk_forge_mcp.healthcheck) instead of a bare tcpSocket probe. M8: the chart fails the render for a reserved mcpUsername (admin), mirroring the mcpPassword guard, and NOTES.txt/values.yaml call it out. Minors: deterministic checksum/secret via a shared helper (stable across renders, identical across api/worker/beat/mcp); vestigial _persist_generated_password filename docstring; false "backend generates its own secret" rationale corrected in compose/helpers/ibm; mcp_live_smoke.py #186/#188 attribution; e2e/config.py changeme default note; .env.example :58/:73/:94 fixes; unified BNK_FORGE_VERSION to latest across dist. Validation: ruff clean; typecheck-backend (core/ schemas/) Success 38 files; 199 auth/credential/startup/ws/migration tests pass incl. new B1 tests; helm lint + template stable checksums, --set secrets.mcpUsername=admin and secrets.mcpPassword=changeme both FAIL; docker compose config on all five modes shows the backend receiving MCP_SERVICE_PASSWORD (via either alias) and ENVIRONMENT. Claude-Session: https://claude.ai/code/session_01UpRYiFserdBE5ESHn759N4 * fix(#193): close bonnyr-f5 CI/RELEASE/SCRIPTS blockers, majors, and minors Blockers: - B4 registry-tag-probe.sh: replace GNU-only BRE alternation (\|) with portable `sed -nE (access_token|token)` so the token parse works on BSD/ macOS sed; on BSD the empty token classified every image `unknown` and routed the operator into FORCE_LATEST=1, overwriting immutable :VERSION manifests (INV-24). Reproduced under `sed --posix`, fix proven to parse T2. - B5 registry-tag-probe.test.sh had no caller and was red on BSD. Both the Makefile script-selftests target and ci.yml's script-selftests job now enumerate and run every scripts/tests/*.test.sh, failing on an empty enumeration or any non-zero rc. - B6 lint-commit-markers.sh: replace the spoofable committer-identity exemption (GitHub + single parent) with an unspoofable "already reachable from origin/main|origin/staging" check; lint the PR title (PR_TITLE via env) on pull_request events; split the rules so machine/already-merged is exempt for the marker rule but the spurious-major rule always applies. Majors: - M3 release.yml overwrite guard: derive the vacuity floor from an independent source (docker-bake.hcl default group, sourced from the workflow-ref tooling) and assert the probe's exit status before trusting its output, so an unavailable probe fails closed instead of "safe". - M4 (INV-31): generate release notes and run the registry existence-probe BEFORE the irreversible push in release-final/release-manual (new shared scripts/registry-overwrite-guard.sh); release-publish keeps its own in-critical-section re-check. - M5 make script-selftests now runs the INV-15 detector-parity diff (extracted to scripts/tests/detector-parity.test.sh) so local == CI. - M6 extractor self-test runs unconditionally with anti-vacuity assertions (ok lines + END marker), no longer gated on grepping its own --self-test. Minors: stale cross-PR comments in release.yml and extract-breaking-changes.sh; removed the duplicate Makefile version-check target; `git add dist/VERSION` no longer swallows failures; first-ever-release notes range fixed; CHANGELOG insertion asserts a non-no-op before committing; refreshed .trivyignore CVE-2026-7598 review deadline; removed e2e-tests.yml dead `|| true`; documented the new Docker dependency in the pre-push hook. Claude-Session: https://claude.ai/code/session_01UpRYiFserdBE5ESHn759N4 * fix(#193 r2): close credential/backend/helm/dist blockers — drop the username alias, make ENVIRONMENT=production satisfiable, and fix the mcp liveness probe bonnyr-f5 round-2 BLOCK, credential/backend/helm/dist/docs half. B-1 (INV-12): the compose files aliased the SERVICE username (MCP_SERVICE_USERNAME:-${MCP_USERNAME:-mcp}); a legacy .env with MCP_USERNAME=admin resolved it to `admin`, and against the guardless image `latest` still points at, the old ensure_service_user rewrites the human admin row to `changeme` every boot. Drop the username alias across all five compose files + the ibm embedded compose (keep the harmless password alias), and default BNK_FORGE_VERSION to 4.0.0 (the release this tree becomes, first image with the guards) instead of `latest`, so a compose file can never hand the new credential contract to a pre-guard image. B-2: ENVIRONMENT=production reaches validate_production, which also gates on JWT_SECRET_KEY / ENCRYPTION_KEY / ALLOWED_ORIGINS — none of which were deliverable from a compose install, so the switch bricked the backend. Plumb all three into every x-backend-env anchor (four compose files + ibm) and document them in the env examples; treat an empty ("" from ${VAR:-}) key as unset in config.py so the plumbed empty default auto-generates rather than passing as a real empty key. _persist_or_load_key now flags only keys WE generated as auto_generated (sidecar .autogen marker), so an operator-provisioned key on the volume validates while a fresh prod boot still fail-fasts permanently. M-4: mcp liveness returned non-zero when the BACKEND was unreachable, so k8s restarted the pod for a dependency outage. Move the auth-probe to readiness only; liveness is tcpSocket. Fix mcp-server/README + mcp_live_smoke hints to name only the vars each process actually reads (container: BNK_FORGE_*, backend: MCP_SERVICE_*). Minors: refuse a known-default DEFAULT_ADMIN_PASSWORD on fresh seed + helm adminPassword fail-guard; guard secrets.yaml mcpUsername nil with kindIs "invalid"; make the Python reserved-name check case-insensitive/trim to match Helm; neutralise the hash when disabling a stale service account; correct the benchmarks.py JWT-gate comment; surface an empty MCP_SERVICE_PASSWORD in install.sh; fix the .env.example "No .env file is needed!" contradiction. Tests: config B-2 satisfiability + provenance-marker tests; seed_auth_step against an admin-still-holds-changeme upgrade DB; case-insensitive reserved-name and disable-hash-neutralisation cases. ruff clean, mypy clean, 4831 unit pass, helm lint/template green, docker compose config verified on all modes. Claude-Session: https://claude.ai/code/session_01UpRYiFserdBE5ESHn759N4 * fix(#193): close r2 CI/RELEASE blockers B-3, M-1..M-3 + scripts/release minors B-3 (commit-lint exemptions): key the already-merged exemption on the range BASE (github.event.before), not the post-push tip, so a new skip-CI marker on a push to main/staging is caught while a genuinely-already-merged base commit stays exempt; replace the self-settable `^release: ` subject exemption with release.yml's own version+trailing-skip fingerprint. M-1 (spurious-major rule): redefine rule 2 as the exact complement of the detectors, sourced from the shared predicate, so it flags only a marker the detectors would MISS (never dash-bullet, markdown-bold or indented shapes); give it the same already-merged exemption; and lint inputs.release_notes through the script before it becomes a release commit/tag. M-2 (overwrite guard floor): derive the vacuity floor from `docker buildx bake --print default | jq '.group.default.targets | length'`, scoped to the default group, so a second bake group no longer wedges the release; separate bake-file parse failures from registry-unreachable in the messaging. Single-source the policy: release-publish and make push-images now call the one guard. M-3 (portability): drop bash-4 mapfile from the probe test; rebuild the compute self-test newline expansion with awk to avoid the bash-3.2 parameter-expansion cliff, so make script-selftests runs under stock macOS bash 3.2. Detector single-sourced into scripts/lib/breaking-change-detect.sh (compute, extract, lint all source it); detector-parity test asserts the wiring; added mutation tests for the lint rules and the overwrite guard. Minors: fix version-consistency misdiagnosis of a column-0 YAML comment; correct the compute/extract parity docstrings and the docker-bake four-push-paths note; wire artifact-network-self-test into ci-gates; make the pre-push hook migration message reachable under set -e; omit the false provenance buildStartedOn; filter the release CI-status poll by commit SHA. Claude-Session: https://claude.ai/code/session_01UpRYiFserdBE5ESHn759N4 * fix(#193): seed the re-enable-guard test's default-hash row directly The round-2 disable_stale_service_user hash-scrub (bonnyr-f5 #193 minor) collided with the re-enable guard's own regression test: _seed_disabled_default_mcp built its "disabled while holding the published default" state BY CALLING disable_stale, which now scrubs the hash -- so holds_known_default_password was false and the PUT re-enable was allowed (200) instead of refused (400). The guard defends a row taken inactive by a path that LEAVES the credential intact (a manual operator PUT), not one disable_stale scrubbed. Seed that state directly (set is_active=False on the default-hash row) so the guard's real scenario is exercised; assert the default hash survives the seed. Corrected the now-stale guard comment in routes/auth.py that still claimed disable "only flips is_active". Neutralisation and its asserting tests are unchanged. Verified: TestServiceAccountReEnableGuard 2/2 pass; the three affected auth files (test_startup_seed_auth, component/test_auth_service, integration/test_routes_auth) 97/97 pass; ruff clean. Claude-Session: https://claude.ai/code/session_01UpRYiFserdBE5ESHn759N4 * fix(#193 r3): fail-close key provenance (B-2) + credential/auth minors Own the round-3 CREDENTIAL/AUTH findings. B-2 (BLOCKER): _persist_or_load_key classified a marker-less key file as operator-provided, so every upgrade keys volume (key present, no marker) let SEC-006's fail-fast pass OPEN on an auto-generated JWT/ENCRYPTION secret in production. Invert the sentinel: a marker-less key now classifies AUTO-GENERATED (fail closed); an operator asserts provenance with an explicit .operator opt-out marker. No marker is written on generation, which also removes the second trigger (a partial marker write can no longer downgrade provenance). Regression tests seed the PREVIOUS-RELEASE on-disk shape and assert validate_production raises under ENVIRONMENT=production. Minors: - Write the key via os.open(0o600)+fchmod so the secret is never briefly 0644. - Single-source the MCP known-default denylist: delete the local tuple in auth_service and use core.config.MCP_KNOWN_DEFAULT_PASSWORDS (comment notes the helm copy is deploy-owned). - Correct holds_known_default_password docstring (disable_stale now scrubs the hash; this guard covers the other disable paths). - Reconcile ENCRYPTION_KEY docs with reality: the keys-file is the source of truth for at-rest crypto; the env var only drives the production gate (encryption.py comment + .env.example). - Clarify the v2_155 custom-username remedy in disable_stale docstring. Test-gaps: - Normalise the service username (trim/casefold) at the reconcile lookup and the disable skip filter, so " mcp "/"MCP" reconciles the existing mcp row instead of minting a second service account and disabling the live one. - Honest seed_admin_user log/logic under DEFAULT_ADMIN_MUST_CHANGE=false (no more "must change on first login" when no gate was applied). - disable_stale_service_user(skip_username=...) leaves the live row wholly untouched (no inactive window), variant included. - db.commit() failure after the keys file is written leaves a retriable state (published default still authenticates, orphan file password does not). All owned-suite tests pass; ruff clean. Each fix reproduced then mutation-tested. Claude-Session: https://claude.ai/code/session_01UpRYiFserdBE5ESHn759N4 * fix(#193 r3): close the release/CI blocker + major + every release/CI minor M-6 (blocker): commit-lint no longer reds unamendable merge history. - rule 2 now flags ONLY a mis-anchored DECLARATIVE marker (the colon form); a colonless marker-shaped PROSE line the detectors treat as inert (an already-merged body such as "- footer in the body ...") is no longer flagged, so the push-to-main range (before..head, which INCLUDES the PR merge-base) goes green without a history rewrite. Detection of a real mis-anchored marker is unchanged. - deleted the already-merged exemption as dead code: base..head excludes the base by construction, so no scanned commit can ever be an ancestor of it. Removed the BEFORE derivation, _already_merged, tests B3.2/M1.5, and the ~20-line header claim. The release-bot exemption stays. - rule 2 now scans the whole body via _under_detected_markers and reports EVERY mis-anchored marker, not just the first. M-7 (major): secret-scan no longer false-fails a delete-only range. A delete-only commit has rev-list count > 0 but gitleaks scans 0 (it scans added content), so the count-based backstop is replaced by a range- resolvability check plus gitleaks' exit status. Release/CI minors: - release-bot fingerprint single-sourced to scripts/lib/is-release-bot-subject.sh; release.yml's inline copy byte-locked by a parity self-test; dropped the false unforgeability claim and documented the residual honestly. - registry-overwrite-guard: added a fail-closed default arm for an unrecognised/empty probe status (+ malformed/empty test scenarios). - Makefile push-images: FORCE_LATEST now overrides ONLY the recency guard; a new FORCE_OVERWRITE overrides ONLY the immutable-tag guard; fixed the missing-jq remediation text. - registry-tag-probe: the network arm now matches the real doubled "000000" curl-failure shape (was dead code); test fixture reproduces it. - INV-15: single-sourced the marker regex (one canonical value + a detector-parity assertion that every embedded copy is byte-identical). - release.yml Publish summary counts what buildx actually pushed (bake --metadata-file), not the static target list. - registry-tag-probe test enumerates the bake DEFAULT group (scoped), matching the guard's enumeration. - added scripts/tests/secret-scan.test.sh (fake-docker mutation suite). release.yml: added a post-push step running scripts/verify-image-pins.sh so a release cannot complete while shipping an unpublished image pin (script owned by the deploy agent; referenced by path from .release-tooling). Claude-Session: https://claude.ai/code/session_01UpRYiFserdBE5ESHn759N4 * fix(#193 r3): single-source every deploy version pin + close deploy majors/minors B-1/B-1b: dist/docker-compose{,.local}.yml, dist/.env.example and ibm_cloud_bnk_forge.sh hard-pinned 4.0.0, a tag that has never been published, while Helm pinned 3.1.6 -- two shipped paths naming two versions, one of which does not exist. Bring all of them under sync-version-artifacts.sh (new PIN/ DISTENV readers+writers, --check, --list) so every pin derives from VERSION (3.1.6, which exists) and the release re-stamps them atomically via the existing --write $NEW; drop the dist/ disclaimer. Add scripts/verify-image-pins.sh (+ selftest) that resolves every shipped compose image: pin against the registry and fails on manifest unknown, wired post-push in the release job. M-1..M-5: reword NOTES/compose/chart comments that asserted post-guard behaviour as already-true on the pre-guard pinned image (they land with the guard-carrying release); NOTES now leads with the required ALLOWED_ORIGINS override; dist/README and the install guide stop recommending latest/3.1.6 and the keys-file cat the pinned image does not write; install.sh strips quotes and rejects the known- default MCP passwords so the "MCP not active" warning fires instead of a green lie; add deploy-version-lockstep + helm-known-defaults-lockstep selftests. Deploy minors: MCP_USERNAME "do not set" made consistent across docs/Makefile; chart ENCRYPTION_KEY now emits a valid Fernet key; DEFAULT_ADMIN_* added to the ibm P3 backend env; secrets.mcpUsername defaults to "mcp" on null; ibm mapfile -> portable while-read. Verified: sync --check exit 0; --write round-trip moves every pin and restores; helm lint/template clean (default + origin override); script selftests green; bash -n + shellcheck clean. Claude-Session: https://claude.ai/code/session_01UpRYiFserdBE5ESHn759N4 * fix(#193 r4): unify the at-rest encryption key (B-3), fail-close key marker (M-1), row-matches-client (M-2) + credential minors B-3: validate_production gated ENCRYPTION_KEY while a second, independent generator in core/encryption.py produced the real at-rest Fernet key unchecked — setting ENCRYPTION_KEY (the documented remedy: token_hex(16), not even a Fernet key) turned the gate green while encryption auto-generated a different key. Unify: one key file (_encryption_key_path == core.encryption.ENCRYPTION_KEY_FILE), one generator. When ENCRYPTION_KEY is set it is VALIDATED as a real Fernet key (fail clearly if not), written to that file with a .operator marker, and consumed by core.encryption and services.backup_service; the provenance flag reflects the value that actually protects data. Never clobber an operator-marked key on a mismatch. config.py:319 and .env.example now print the Fernet recipe. M-1: _persist_or_load_key required a regular-FILE marker (os.path.isfile, not os.path.exists — a directory no longer counts) and treats "marker present, key file absent" as a provisioning error: generate but do NOT persist, so the stale-marker rotation gesture can never heal into auto=False on the next boot. M-2 (regression this PR introduced): ensure_service_user normalised the username before lookup/create, so MCP_SERVICE_USERNAME=MCP seeded 'mcp' while the client sends the raw 'MCP' and authenticate_user matched exactly -> login denied. Create/ reconcile under the RAW value (what the client sends); the disable_stale skip keys on the same raw value; only the reserved-name guard normalises. Fixed the false "Matches the Helm chart lower|trim" docstring. Credential minors: non-vacuous localhost-CORS test (valid MCP password so only the CORS branch fails) + wildcard is now an exact origin-list entry, not a substring; new DPU-websocket must-change/unresolvable-user tests mirroring the k8s twin; middleware unresolvable-JWT-subject-refused and exact-vs-suffix exempt-path tests; corrected the denylist copy count (4th copy in dist/install.sh); corrected v2_155's rationale (v2_154 is new in this diff, not "already shipped"); documented why ensure_service_user's adoption branch is kept. Claude-Session: https://claude.ai/code/session_01UpRYiFserdBE5ESHn759N4 * fix(#193 r4): close deploy-surface blocker + majors (B1, M7-M12) and deploy minors B-1: a fresh install of the pinned 3.1.6 bundle seeded an admin nobody could log in as. `${DEFAULT_ADMIN_PASSWORD:-}` delivers the var present-and-empty; 3.1.6's `DEFAULT_ADMIN_PASSWORD: str = "changeme"` is then overridden by "" and the login schema rejects "" (422). The prescribed `${VAR}` form does NOT omit on this compose (map interpolation still renders ""); the working omit-when-unset form is a map entry with NO value (passthrough / `docker run -e KEY` semantics). Converted DEFAULT_ADMIN_PASSWORD to passthrough across all four compose paths (dist base+local, root base+local, IBM embedded). Swept the class: JWT_SECRET_KEY/ENCRYPTION_KEY also converted (3.1.6 uses `if KEY is None`, so "" is used literally, not auto-generated); MCP_SERVICE_PASSWORD kept as `${...:-...}` because omitting it restores 3.1.6's known default "mcp-service-changeme". Added a commented DEFAULT_ADMIN_PASSWORD block to dist/.env.example. Verified via `docker compose config` + real container env both directions (unset -> omitted; set in .env -> forwarded). M-7: chart checksum/secret did not change on mcp-password rotation (3 renders, 3 generated passwords, one identical checksum). Made all generate/rotate fallbacks deterministic (deriveSecret, release-seeded) so the Secret is stable across renders and includes, and hash the RENDERED Secret so the annotation tracks every resolved value. Now stable across renders, identical across the 4 deployments, and it flips when any resolved value changes. M-8: dist/install.sh credential guard failed open on `changeme ` (whitespace). Trim leading/trailing whitespace around the quote-strip before the known-default compare. M-9: removed newly-added forward-dated 4.0.0 prose (install-guide.html x2, DOCKER.md). M-10: default helm install crashlooped (production + localhost). Added a render-time guard mirroring backend validate_production (fail on wildcard under staging/production, localhost under production); defaulted ALLOWED_ORIGINS to empty so the bare render boots (empty is neither wildcard nor localhost). `helm lint` and bare `helm template` stay green; the guard fires with a clear message on a real fatal posture. M-11: portable in-place sed in the IBM installer (`sed -i.bak … && rm`), both sites. M-12: dist/ no longer ships published default DB/redis creds on host networking. install.sh generates strong POSTGRES_PASSWORD/REDIS_PASSWORD on the fresh .env (like Helm/IBM); .env.example ships them empty; a pre-existing default triggers a warning. Deploy minors: extended deploy-version-lockstep.test.sh to the bnk-operator chart (appVersion + image.tag) and dist/VERSION; brought dist/VERSION under sync-version-artifacts.sh (--check/--list/--write green); reworded install.sh's MCP UNHEALTHY assertion to match what the pinned image actually reports; added scripts/tests/ibm-compose-drift.test.sh freezing the credential/hardening env so the IBM embedded compose and dist/docker-compose.yml cannot silently diverge. Claude-Session: https://claude.ai/code/session_01UpRYiFserdBE5ESHn759N4 * fix(#193 r4): make the pin verifier reachable pre-push, de-vacuum the parity/self-test harnesses, and close the release minors B-2 (blocker): scripts/verify-image-pins.sh could never pass where the release job runs it — from the 4-file sparse .release-tooling checkout that holds no compose file (ROOT resolved there), with REGISTRY/VERSION handed in as env vars the script reads only as flags, and the whole step wired AFTER the tag/Release/push/signing. Fixes, end to end: - add a consistency mode (--expect-version) that asserts every shipped first-party pin already renders to $NEW without a registry probe, and run it as the PRIMARY PRE-push gate in release-final and release-manual (before anything irreversible); - fix the post-push existence step to pass REGISTRY/VERSION as FLAGS and the compose files explicitly by --file (they live at the tag checkout at the workspace root), keeping it as a secondary confirmation; - widen the default file set to include the IBM Cloud installer's embedded compose; - add a dryrun-release-tooling job that rebuilds the exact publish-job layout and exercises both invocations against a fake probe, and gate release-publish on it, so a step that cannot execute is caught before it is wired ahead of a signature. The sibling registry-tag-probe.sh / registry-overwrite-guard.sh read nothing $ROOT-relative outside the sparse set, so they are unaffected. M-3: detector-parity.test.sh enumerated the marker copies with the very token that drifts, so a copy that drifted in the token vanished from enumeration (drifting :96-97 dropped the count 5->3 yet stayed green). Enumerate by position/count instead: an exact per-file canonical count plus a stable-anchor site scan that flags any drifted site even under a compensating add. M-4: the filesystem self-test loop checked only a non-empty enumeration and each file's exit 0, so a test gutted to a no-op passed and deleting 7 of 8 stayed green. It now requires each file to emit PASS lines, no FAIL line, and an ALL PASS terminal marker, plus a count floor derived from git's tracked *.test.sh set (detector-parity was conformed to that output convention). M-5: release-rc created and pushed the RC tag before the fail-closed notes step; the tag is now created locally, notes generated, then the tag pushed. M-6: added mutation-tested coverage for this PR's four previously-uncovered lint fixes (the PR-title lint, the pending-message lint, the RANGE fail-closed branch, and the skip-checks trailer rule). LEAD: the anti-vacuity staging floor derived the count from a stale literal while --list grew to 8 paths; both sites now derive it from --list and require every listed path to stage, and the stale comments are corrected. Release minors: scope the release-bot commit-lint exemption to the range tip (a forged release subject buried mid-range is no longer exempt) and add a REACHABLE published-history exemption anchored to the last release tag so a mis-anchored marker in unamendable history cannot red the release; add fixtures for the untested registry probe/guard arms (5xx, unexpected code, unknown status, bake-parse failure); ancestry-filter the LAST_FINAL tag queries so a tag on another branch cannot skew the notes range; reconcile the empty-RANGE handling (commit-lint now fails closed on an explicit empty RANGE, ci.yml leaves it unset); give the pre-push hook a clear message and a fetch fallback when the remote tip is absent locally; derive the cosign verify-identity org from REGISTRY instead of hardcoding it. Flagged: gate Makefile push-customer-build through registry-overwrite-guard.sh, the last documented push path that was still unguarded. Claude-Session: https://claude.ai/code/session_01UpRYiFserdBE5ESHn759N4 * fix(#193 r4): keep M-7 checksum-tracks-rotation WITHOUT predictable secrets The r4 deploy fix closed M-7 (checksum did not move on a secret rotation, so pods never rolled) by making the generated fallbacks deterministic -- deriveSecret = sha256(Release.Name|Namespace|fullname|purpose). Every input is public (they appear in resource labels and the chart source), so that made the JWT signing key, the at-rest Fernet key and the admin/mcp passwords COMPUTABLE by anyone who can read a label -- a forge-any-token / decrypt-all-secrets exposure, strictly worse than the cosmetic churn it fixed. Revert generation to randAlphaNum (unpredictable) and instead hash the DETERMINISTIC inputs that determine the Secret -- values.secrets, the persisted .data (reused via lookup), plus a per-credential "rotating-from-default" marker for admin/mcp whose persisted value is a known published default. That tracks every rotation (operator edit, persisted-value change, rotate-away-from-default) so the pods roll, is stable across renders including a bare no-cluster `helm template` (the hashed inputs carry no randomness), and never derives a secret from public identity. deriveSecret removed. New scripts/tests/helm-secret-checksum.test.sh locks all three: stable-across-renders, changes-on-rotation, and generated-value-is-random -- so the determinism cannot return. Verified: helm lint 0-failed; bare + override template OK; the M-10 render guard still fires on production+localhost; the new selftest ALL PASS. Claude-Session: https://claude.ai/code/session_01UpRYiFserdBE5ESHn759N4 * fix(#193 r4): self-review — close the encryption-key data-loss BLOCKER + deploy minors A cold adversarial self-review (three auditors mirroring the reviewer's method) of the r4 changeset found a data-loss BLOCKER we introduced this round, plus deploy minors. Fixing before it ships. B-3 (BLOCKER, data loss): the r4 encryption-key unification made ENCRYPTION_KEY env OVERWRITE an existing at-rest key file on boot. Traced only to the "operator freshly sets the key" consumer, never to (a) backup_service restore, which writes the backup's key to that file, or (b) the r3->r4 upgrade population, whose data was encrypted under a marker-less auto-gen FILE key (r3 core.encryption used the file regardless of env). The first r4 boot clobbered those -> restored/existing data undecryptable (or bricked the boot on a marker mismatch). A new test even LOCKED the clobber with a false premise. Fix: the at-rest key FILE is the single source of truth. ENCRYPTION_KEY only SEEDS the file when it is ABSENT and never overwrites an existing one; the gate reads the FILE's .operator provenance. backup restore now drops the .operator marker so a restored key passes the gate without a clobber. Rewrote the clobber-locking tests to lock the no-clobber invariant; added upgrade-shape, production-fail-without-data-loss, and restore-marker tests. M-7 (deploy self-review): removed the rotation-marker from secretsChecksum — the auditor proved it redundant (the same .data change already moves the digest; deleting it left the test green) and its admin branch dead. Kept the input-hash; documented the genuine trilemma (cluster-less-template-stable / tracks-generated-rotation / unpredictable-secrets — pick two; determinism is the predictable-secret hole). M-10: the render guard's wildcard check is now an exact comma-split entry, matching the backend's `"*" in cors_origins` since r4, so a legitimate `https://*.example.com` is no longer blocked; the localhost check stays a substring to match the backend. .env.example: the admin-password template was an empty assignment that uncomments into a lockout; it now carries a replace-me placeholder. Verified: backend 8082 passed; config/encryption 66, backup 13; script-selftests all pass; helm lint/template clean (subdomain-wildcard passes, bare '*'/prod+localhost fail); ruff clean. Claude-Session: https://claude.ai/code/session_01UpRYiFserdBE5ESHn759N4 * fix(#193 r5): fail closed instead of regenerating over an existing at-rest key (I-1) bonnyr-f5 #193 round-5 closure blocker. B-3 wrote down the invariant "the at-rest key file is the single source of truth; nothing overwrites it once it holds bytes" and config.py honoured it -- but core.encryption.get_encryption_key() did not. A file present but under 32 bytes (truncated / partial write / disk full / bad restore) logged "Invalid key, regenerating" and OVERWROTE it with a fresh key, permanently destroying the key any existing data was encrypted under -- silently, on a GREEN production boot, because the intact .operator marker keeps validate_production passing. Fix: the read path now fails CLOSED. A genuinely absent (or 0-byte) file still generates and persists a new key. A file that HOLDS bytes is validated as a real Fernet key: valid -> returned untouched; unusable -> SystemExit with a clear message, never regenerated. A crashloop is recoverable; an overwritten key is not. This also closes r5 note #3 -- the old `len >= 32` check accepted any blob and surfaced a mis-shaped key as a later cipher error; it now Fernet-validates and says so plainly. Locked by TestAtRestKeyFileNeverRegeneratedOverBytes: a 30-byte truncated key -> SystemExit AND the original bytes survive on disk (recoverable); a valid key -> returned untouched; an absent file -> generates a valid key. Verified: backend 8086 passed; encryption unit tests 19 passed; ruff clean. Claude-Session: https://claude.ai/code/session_01UpRYiFserdBE5ESHn759N4 --------- Co-authored-by: John Gruber --- .env.example | 62 +- .githooks/pre-push | 74 +- .github/workflows/ci.yml | 276 +++++- .github/workflows/e2e-tests.yml | 11 +- .github/workflows/release.yml | 884 ++++++++++++++++-- .github/workflows/secret-baseline.yml | 39 + .gitignore | 6 + .gitleaks.toml | 15 +- .trivyignore | 38 +- AGENTS.md | 23 + CHANGELOG.md | 47 +- Makefile | 215 ++++- README.md | 17 +- The_BNK_Forge_Developers_Guide.md | 4 +- .../v2_154_user_is_service_account.py | 32 + .../v2_155_backfill_is_service_account.py | 95 ++ backend/core/auth_middleware.py | 61 +- backend/core/config.py | 303 +++++- backend/core/encryption.py | 61 +- backend/models/system.py | 8 +- backend/openapi.json | 15 + backend/routes/auth.py | 42 +- backend/routes/benchmarks.py | 61 +- backend/routes/dpus_websocket.py | 13 + backend/routes/k8s_websocket.py | 13 + backend/schemas/auth.py | 7 + backend/services/auth_service.py | 671 ++++++++++++- backend/services/backup_service.py | 13 + .../services/execution/container_runner.py | 7 +- backend/startup_steps.py | 104 ++- backend/tests/component/test_auth_service.py | 768 ++++++++++++++- .../tests/component/test_dpus_websocket.py | 104 +++ backend/tests/component/test_k8s_websocket.py | 47 +- backend/tests/component/test_startup_steps.py | 37 +- backend/tests/integration/test_routes_auth.py | 187 ++++ .../integration/test_routes_k8s_websocket.py | 4 +- backend/tests/test_migrations.py | 109 +++ backend/tests/test_startup_seed_auth.py | 350 +++++++ backend/tests/unit/test_auth_middleware.py | 125 +++ backend/tests/unit/test_backup_service.py | 30 + .../tests/unit/test_benchmark_agent_auth.py | 145 ++- backend/tests/unit/test_core_config.py | 392 +++++++- backend/tests/unit/test_core_encryption.py | 69 ++ bin/roadmap-add.py | 2 +- bin/roadmap-gen.py | 3 +- bnk-operator/charts/bnk-operator/Chart.yaml | 2 +- bnk-operator/charts/bnk-operator/values.yaml | 4 +- dist/.env.example | 83 +- dist/README.md | 55 +- dist/docker-compose.local.yml | 27 +- dist/docker-compose.yml | 105 ++- dist/install.sh | 138 ++- docker-bake.hcl | 57 +- docker-compose.adr424.yml | 2 +- docker-compose.local.yml | 42 +- docker-compose.yml | 57 +- docs/DEPLOYMENT.md | 35 +- docs/DOCKER.md | 35 +- docs/E2E-CRITICAL-004_MCP_SANITY.md | 4 +- ...er modules and blueprints for BNK Forge.md | 11 +- docs/INSTALLATION.md | 46 +- docs/ROADMAP.md | 6 +- docs/ROADMAP_PROCESS.md | 4 +- docs/roadmap.html | 4 +- docs/roadmap.yaml | 12 +- frontend-v2/package.json | 2 +- frontend-v2/src/types/api-generated.ts | 15 + helm/bnk-forge/Chart.yaml | 2 +- helm/bnk-forge/templates/NOTES.txt | 35 +- helm/bnk-forge/templates/_helpers.tpl | 96 ++ helm/bnk-forge/templates/api.yaml | 6 + helm/bnk-forge/templates/beat.yaml | 5 + helm/bnk-forge/templates/mcp.yaml | 34 +- helm/bnk-forge/templates/secrets.yaml | 129 ++- helm/bnk-forge/templates/worker.yaml | 5 + helm/bnk-forge/values.yaml | 39 +- mcp-server/README.md | 33 +- mcp-server/src/bnk_forge_mcp/healthcheck.py | 21 +- mcp-server/tests/test_healthcheck.py | 64 +- scripts/compute_version_bump.sh | 218 ++++- scripts/e2e/config.py | 8 +- scripts/e2e/steps.py | 9 +- scripts/extract-breaking-changes.sh | 258 ++++- scripts/get_dpu_pwd.sh | 1 + scripts/ibm_cloud_bnk_forge.sh | 101 +- scripts/lib/breaking-change-detect.sh | 105 +++ scripts/lib/is-release-bot-subject.sh | 28 + scripts/lint-commit-markers.sh | 241 +++++ scripts/mcp_live_smoke.py | 14 +- scripts/publish-signed-images.sh | 32 +- scripts/registry-overwrite-guard.sh | 109 +++ scripts/registry-tag-probe.sh | 147 +++ scripts/secret-scan.sh | 104 +++ scripts/sync-version-artifacts.sh | 262 ++++++ scripts/test-backup-restore.sh | 4 +- scripts/tests/deploy-version-lockstep.test.sh | 61 ++ scripts/tests/detector-parity.test.sh | 131 +++ .../helm-known-defaults-lockstep.test.sh | 54 ++ scripts/tests/helm-secret-checksum.test.sh | 44 + scripts/tests/ibm-compose-drift.test.sh | 63 ++ scripts/tests/lint-commit-markers.test.sh | 226 +++++ .../tests/registry-overwrite-guard.test.sh | 88 ++ scripts/tests/registry-tag-probe.test.sh | 163 ++++ scripts/tests/secret-scan.test.sh | 68 ++ scripts/tests/verify-image-pins.test.sh | 92 ++ scripts/verify-image-pins.sh | 226 +++++ tests/e2e/E2E_STRATEGY.md | 2 +- tests/e2e/config/test-config.ts | 2 +- tests/e2e/pages/login.page.ts | 2 +- user-pack/install-guide.html | 168 ++-- vm-bnk-forge/README.md | 5 +- 111 files changed, 9253 insertions(+), 587 deletions(-) create mode 100644 .github/workflows/secret-baseline.yml create mode 100644 backend/alembic/versions/v2_154_user_is_service_account.py create mode 100644 backend/alembic/versions/v2_155_backfill_is_service_account.py create mode 100644 backend/tests/component/test_dpus_websocket.py create mode 100644 backend/tests/test_startup_seed_auth.py create mode 100644 scripts/lib/breaking-change-detect.sh create mode 100644 scripts/lib/is-release-bot-subject.sh create mode 100644 scripts/lint-commit-markers.sh create mode 100644 scripts/registry-overwrite-guard.sh create mode 100644 scripts/registry-tag-probe.sh create mode 100644 scripts/secret-scan.sh create mode 100644 scripts/sync-version-artifacts.sh create mode 100644 scripts/tests/deploy-version-lockstep.test.sh create mode 100644 scripts/tests/detector-parity.test.sh create mode 100644 scripts/tests/helm-known-defaults-lockstep.test.sh create mode 100644 scripts/tests/helm-secret-checksum.test.sh create mode 100644 scripts/tests/ibm-compose-drift.test.sh create mode 100644 scripts/tests/lint-commit-markers.test.sh create mode 100644 scripts/tests/registry-overwrite-guard.test.sh create mode 100644 scripts/tests/registry-tag-probe.test.sh create mode 100644 scripts/tests/secret-scan.test.sh create mode 100644 scripts/tests/verify-image-pins.test.sh create mode 100644 scripts/verify-image-pins.sh diff --git a/.env.example b/.env.example index 1c024c7..e0dc691 100644 --- a/.env.example +++ b/.env.example @@ -1,11 +1,16 @@ # BNK-Forge v2 Environment Configuration # ============================================================================ # -# NOTE: No .env file is needed! All settings are managed via docker-compose.yml -# environment variables and the UI (System > Defaults). +# NOTE: for a plain local dev bring-up you can start WITHOUT a .env — every value +# has a working default baked into docker-compose.yml, and app defaults are managed +# in the UI (System > Defaults). # -# This file documents optional environment variables you can set in -# docker-compose.yml under the backend service's `environment:` section. +# But a .env IS required for anything hardened: the compose files read +# MCP_SERVICE_PASSWORD and (under ENVIRONMENT=staging|production) JWT_SECRET_KEY, +# ENCRYPTION_KEY and ALLOWED_ORIGINS from it — see the MCP SERVICE ACCOUNT and +# ENVIRONMENT sections below. This file documents those variables plus the optional +# ones you can also set directly in docker-compose.yml under the backend service's +# `environment:` section. # # ============================================================================ @@ -34,7 +39,24 @@ # # For production, generate and set explicitly in docker-compose.yml: # JWT_SECRET_KEY=$(openssl rand -hex 32) -# ENCRYPTION_KEY=$(openssl rand -hex 16) +# ENCRYPTION_KEY=$(python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())") +# +# JWT_SECRET_KEY is consumed directly (it signs/verifies JWTs). +# +# ENCRYPTION_KEY is the at-rest Fernet key that ACTUALLY encrypts stored secrets. +# When you set it, the app validates it is a real Fernet key (it refuses to boot if +# not), writes it to the key file on the keys volume ($KEYS_DIR/encryption.key, +# default /app/keys/encryption.key), and encrypts with THAT value — one key, one +# generator. It must therefore be a Fernet key (line above), NOT `openssl rand`/ +# `token_hex`. When ENCRYPTION_KEY is unset, the app generates and persists a Fernet +# key to that file on first boot (fine for dev; under ENVIRONMENT=staging|production +# an auto-generated key is refused, so either set ENCRYPTION_KEY or pre-provision the +# key file with a `.operator` marker beside it). +# +# WARNING: if a keys volume already holds a DIFFERENT at-rest key, setting a new +# ENCRYPTION_KEY does not re-encrypt existing data (and an operator-provisioned key +# is never overwritten — the app fails closed on the mismatch). Set it on first +# boot, or migrate the data deliberately. # # JWT_SECRET_KEY=your_generated_key_here # ENCRYPTION_KEY=your_generated_key_here @@ -55,10 +77,10 @@ # GUI UPGRADE (Required for System > Upgrade Now) # ============================================================================ # -# Set HOST_REPO_PATH in docker-compose.yml to enable the GUI upgrade button. -# Without this, use SSH + ./upgrade.sh for server upgrades. +# Set HOST_REPO_PATH in this .env file to enable the GUI upgrade button (compose +# reads it from here). Without this, use SSH + ./upgrade.sh for server upgrades. # -# HOST_REPO_PATH=/home/jarrodl/bnk-forge-v2 +# HOST_REPO_PATH=/path/to/bnk-forge # ============================================================================ # ENVIRONMENT (development/staging/production) @@ -70,7 +92,7 @@ # ENVIRONMENT=development # ============================================================================ -# MCP SERVICE ACCOUNT (REQUIRED in production; dev defaults shown) +# MCP SERVICE ACCOUNT (REQUIRED in production; no defaults shipped) # ============================================================================ # # MCP authenticates to the backend as a dedicated non-human service account. @@ -81,12 +103,26 @@ # affect MCP (they are distinct env vars and distinct accounts). # # DEFAULT_ADMIN_PASSWORD controls the seeded human admin account (first-boot only, -# must_change_password=True). In production set this to a strong initial value -# that operators change on first login. +# must_change_password=True, enforced server-side). If left UNSET, a strong +# random password is generated and written to /app/keys/initial_admin_password +# (mode 600, on the bnk-forge-keys volume) -- retrieve it with: +# docker exec bnk-forge-backend cat /app/keys/initial_admin_password +# Set it here only if you want to choose the initial value yourself. # # MCP_SERVICE_USERNAME=mcp -# MCP_SERVICE_PASSWORD=mcp-service-changeme -# DEFAULT_ADMIN_PASSWORD=changeme +# Choose your own value — there is NO shipped default (#186/#187): the old +# mcp-service-changeme is refused as a seed value and can no longer authenticate. +# Set the SAME value on the backend and the MCP server. Keep the value on its own +# line: a '#' that directly abuts the value with NO leading space (e.g. +# `MCP_SERVICE_PASSWORD=s3cret#x`) becomes part of the password, whereas a +# space-separated `s3cret # note` is stripped as an inline comment — so avoid +# trailing text either way. +# Leave it unset and MCP stays unavailable until you configure it (the backend +# disables any stale service account rather than seed a guessable one). +# MCP_SERVICE_PASSWORD= +# Set it only to CHOOSE the initial admin password; REPLACE the placeholder and never +# leave it empty (an empty value seeds an admin the login form rejects -> lockout): +# DEFAULT_ADMIN_PASSWORD=replace-with-a-strong-password # ============================================================================ # BENCHMARK AGENT AUTHENTICATION diff --git a/.githooks/pre-push b/.githooks/pre-push index f3f50c0..6fadb5b 100755 --- a/.githooks/pre-push +++ b/.githooks/pre-push @@ -4,18 +4,88 @@ # Also validates alembic migration chain (catches commits made without pre-commit hook) # Install: git config core.hooksPath .githooks (or: make setup-hooks) # Skip: git push --no-verify (emergency only!) +# +# DEPENDENCY (bonnyr-f5 #193 minor): `make pre-push` below now runs `ci-gates`, +# whose `secret-scan` gate executes gitleaks via `docker run`. A RUNNING Docker +# daemon is therefore required to push. It fails loudly if Docker is down; start +# Docker (or `git push --no-verify` for a genuine emergency) if you hit that. set -e # ─── Migration chain check ──────────────────────────────────────────────────── +# Test the command DIRECTLY in the `if`, not `$?` afterwards: under `set -e` a +# non-zero `python3 ...` aborts the hook BEFORE the `if [ $? -ne 0 ]` runs, so the +# "PUSH BLOCKED" message was unreachable — git still blocked the push, but with no +# migration-specific guidance (bonnyr-f5 #193 minor). `if !` is exempt from `set -e` +# and keeps the tailored message. echo "=== Migration chain check (pre-push) ===" -python3 scripts/check-migrations.py -if [ $? -ne 0 ]; then +if ! python3 scripts/check-migrations.py; then echo "" echo "PUSH BLOCKED: Fix migration chain errors above before pushing." echo "Run 'python3 scripts/check-migrations.py' for details." exit 1 fi +echo "" + +# ─── Commit-message marker lint (bonnyr-f5 #182 r3) ─────────────────────────── +# Fail fast, before the heavy suite, if any commit about to be pushed carries a +# CI-control marker (which would suppress the workflow run) or a spurious major- +# bump prose line. Same script the ci.yml commit-lint gate runs, so local == CI. +# +# RANGE from the pre-push stdin protocol (bonnyr-f5 #182 r5, Minor). git feeds +# this hook one " " line per ref +# being pushed. Scanning the script's default `@{upstream}..HEAD` misses every +# non-tip commit when the branch has no upstream yet (a FIRST push) -- exactly +# when a bad commit is most likely to slip in. Deriving `..` from stdin scans precisely the commits this push introduces. A new remote +# branch reports an all-zero remote sha (no merge-base to diff against); there we +# fall back to the script's own default rather than scanning all of history. +# Deletions (all-zero local sha) contribute no commits. When stdin is empty (the +# hook run by hand, not by git) we leave RANGE unset so the script default runs. +zero="0000000000000000000000000000000000000000" +remote_name="${1:-origin}" # git passes the remote name as $1 +prepush_range="" +while read -r _localref localsha _remoteref remotesha; do + [ -z "${localsha:-}" ] && continue + [ "$localsha" = "$zero" ] && continue # branch deletion: nothing to lint + if [ "${remotesha:-$zero}" = "$zero" ]; then + prepush_range="__DEFAULT__" # new branch: no base -> script default + break + fi + # Normal update: the range BASE is the remote tip. If that object is not present + # locally (never fetched) the range is unresolvable, and passing it straight to + # commit-lint hard-failed with a MISLEADING "not a resolvable revision range" + # (bonnyr-f5 #193 r4 minor). Try a best-effort fetch of just that remote, then + # re-check; if it still is not local, say so PLAINLY and fall back to the + # script's own default range instead of a broken one. + if ! git rev-parse --verify --quiet "${remotesha}^{commit}" >/dev/null 2>&1; then + git fetch --quiet "$remote_name" >/dev/null 2>&1 || true + fi + if git rev-parse --verify --quiet "${remotesha}^{commit}" >/dev/null 2>&1; then + prepush_range="${remotesha}..${localsha}" # exactly the pushed commits + else + echo "note: the remote tip ${remotesha} is not present locally even after a fetch" + echo " of '${remote_name}', so the exact pushed range can't be resolved." + echo " Linting the local default range (@{upstream}..HEAD, else the tip) instead;" + echo " run 'git fetch ${remote_name}' to lint the precise range next time." + prepush_range="__DEFAULT__" + fi + break +done + +echo "=== Commit message marker lint (pre-push) ===" +if [ -n "$prepush_range" ] && [ "$prepush_range" != "__DEFAULT__" ]; then + lint_status() { RANGE="$prepush_range" bash scripts/lint-commit-markers.sh; } +else + lint_status() { bash scripts/lint-commit-markers.sh; } +fi +if ! lint_status; then + echo "" + echo "PUSH BLOCKED: a commit message carries a CI-control / spurious-major marker." + echo "Reword it (see AGENTS.md 'Commit conventions') and try again." + exit 1 +fi + echo "" echo "=========================================" echo " Pre-push: Running local checks" diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 5d8f62e..bec4109 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -24,33 +24,28 @@ name: CI on: pull_request: branches: [main, staging, develop] - paths-ignore: - - '**.md' - - 'docs/**' - - '.agent/**' - - '.opencode/**' - - 'LICENSE' - - '.gitignore' - - '.trivyignore' - - 'USER_GUIDE.md' + # No workflow-level paths-ignore: secret scanning (gitleaks) and the CI Gate + # must see EVERY change, doc-only PRs included — a secret lands in a .md as + # easily as in code, and this is a public repo (#182 review). Expensive jobs + # still skip on irrelevant paths via the per-job `changes` filter below; path + # filtering lives there (one source of truth), not at the trigger. push: # main (deploy trigger) + staging (release-automation preflight needs a # push-triggered CI run to match by SHA — see release.yml preflight); - # develop skipped. + # develop skipped. No paths-ignore, same reason as above. branches: [main, staging] - paths-ignore: - - '**.md' - - 'docs/**' - - '.agent/**' - - '.opencode/**' - - 'LICENSE' - - '.gitignore' - - '.trivyignore' - - 'USER_GUIDE.md' concurrency: - group: ci-${{ github.ref }} - cancel-in-progress: true + # bonnyr-f5 #182 r2: on main/staging give every push its OWN group (append the + # SHA) so a later docs-only push can't cancel -- even as a PENDING run -- the CI + # run a release polls by SHA. Feature branches keep the per-ref group so rapid + # pushes still supersede each other and save minutes. + group: ci-${{ github.ref }}${{ (github.ref_name == 'main' || github.ref_name == 'staging') && github.sha || '' }} + # bonnyr-f5 #182: never cancel an in-flight CI run on the release branches -- + # release.yml preflight polls that exact run by SHA, so a docs-only push (which + # triggers CI but not Release) would otherwise cancel it and strand the earlier + # commit's release. Feature branches still cancel to save minutes. + cancel-in-progress: ${{ github.ref_name != 'main' && github.ref_name != 'staging' }} permissions: contents: read @@ -126,6 +121,193 @@ jobs: - name: Run lint run: make lint-backend + version-consistency: + name: "P1 · Version Consistency" + needs: changes + if: always() + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + - name: Assert version-bearing artifacts agree with VERSION + # Helm chart tag/appVersion and frontend package.json must equal VERSION, + # or the release (which publishes only :${VERSION}) yields ImagePullBackOff + # / silent drift (#177 Blocker 2). Goes through `make version-check` so + # this job and `make pre-push` run the identical command (#182 r3). + run: make version-check + + shellcheck: + name: "P1 · ShellCheck" + needs: changes + if: always() + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + - name: Install shellcheck + run: sudo apt-get update && sudo apt-get install -y shellcheck + - name: Run shellcheck + run: make shellcheck + + secret-scan: + name: "P1 · Secret Scan (gitleaks)" + needs: changes + if: always() + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + with: + fetch-depth: 0 # full history so the range scan sees add-then-remove + - name: gitleaks + run: | + # bonnyr-f5 #182 r2: scan the PR/push COMMIT RANGE in git mode, not the + # working tree. --no-git misses a secret added then REMOVED within the + # branch, which stays fetchable forever from a public clone -- the main + # thing a public repo needs a history-aware scan for. + # + # All of the scan + assertion logic (the r3 BLOCKER fix, the + # dubious-ownership safe.directory fix, the archive-depth fix, and the + # digest pin) lives in scripts/secret-scan.sh so `make secret-scan` and + # this job run byte-identical commands (#166 / ci.yml header: local==CI). + # We only compute the range from the event here and hand it to the + # script; RANGE is exported (even when empty => full history). + if [ "${{ github.event_name }}" = "pull_request" ]; then + RANGE="${{ github.event.pull_request.base.sha }}..${{ github.event.pull_request.head.sha }}" + elif [ -n "${{ github.event.before }}" ] && [ "${{ github.event.before }}" != "0000000000000000000000000000000000000000" ]; then + RANGE="${{ github.event.before }}..${{ github.sha }}" + else + RANGE="" # first push / no base — scan all reachable history + fi + export RANGE + make secret-scan + + commit-lint: + name: "P1 · Commit Message Lint" + needs: changes + if: always() + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + with: + fetch-depth: 0 # need the whole PR range of commit messages + - name: Lint commit messages for CI-control / spurious-bump markers + env: + # Untrusted free text -> read from env, NEVER interpolated into the shell + # (same rule as release.yml's RELEASE_NOTES). Empty on non-PR events. The + # script lints it as the PENDING squash subject (bonnyr-f5 #193 B6b). + PR_TITLE: ${{ github.event.pull_request.title }} + run: | + # bonnyr-f5 #182 r3 (Minor -> enforcement): the AGENTS.md rule against + # CI-control markers in commit messages was documentation only, and + # "documentation is not enforcement" (#166). This gate FAILS a PR/push + # whose commit range carries a marker (which would suppress CI for that + # commit -- the #179/#181 case) or an accidental line-start BREAKING + # CHANGE prose that spuriously majors a release. The .githooks/pre-push + # hook runs the SAME script locally so it is caught before push too. + # + # The RANGE is base..head; on a push to main it is github.event.before.. + # github.sha, which INCLUDES the merged branch's own commits (e.g. the PR + # merge-base). There is no already-merged exemption any more (bonnyr-f5 + # #193 r3 M-6a: it was dead code — base..head excludes the base by + # construction, so no scanned commit could ever be an ancestor of it). + # rule 2 now flags ONLY a mis-anchored DECLARATIVE `BREAKING CHANGE:` (the + # colon form), so a marker-shaped PROSE line in an unamendable merged body + # no longer reds this ALWAYS_RUN gate on the merge that cuts the release + # (M-6b). The only exemption left is the release bot's own minted commit. + if [ "${{ github.event_name }}" = "pull_request" ]; then + RANGE="${{ github.event.pull_request.base.sha }}..${{ github.event.pull_request.head.sha }}" + elif [ -n "${{ github.event.before }}" ] && [ "${{ github.event.before }}" != "0000000000000000000000000000000000000000" ]; then + RANGE="${{ github.event.before }}..${{ github.sha }}" + else + # first push / no base: leave RANGE UNSET so commit-lint uses its tip + # default. Do NOT pass RANGE="" — commit-lint now FAILS CLOSED on an + # explicit empty RANGE (secret-scan reads empty as "all history", a + # meaning commit-lint must never take), so the same literal no longer + # means two different things across the two gates (bonnyr-f5 #193 r4). + unset RANGE + fi + export PR_TITLE + # Only export RANGE when it was actually set above (the first-push branch + # unsets it); guard with an if so the unset case does not return 1 and abort + # the step under `set -eo pipefail`. + if [ -n "${RANGE+set}" ]; then export RANGE; fi + make commit-lint + + script-selftests: + name: "P1 · Script Self-Tests" + needs: changes + if: always() + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + - name: compute_version_bump SELF_TEST + run: | + # Fail on a non-zero exit OR a FAIL: line. The harness historically + # printed FAIL: but still exited 0, so trusting the exit code alone + # made this job unable to catch a broken self-test until the exit-code + # fix landed (#182 review). Checking both decouples the two. + set +e + out="$(SELF_TEST=1 bash scripts/compute_version_bump.sh 2>&1)"; rc=$? + echo "$out" + if [ "$rc" -ne 0 ]; then + echo "::error::compute_version_bump self-test exited $rc"; exit "$rc" + fi + if grep -qE '(^|[[:space:]])FAIL:' <<< "$out"; then + echo "::error::compute_version_bump self-test reported FAIL: but exited 0"; exit 1 + fi + # bonnyr-f5 #182: silence must not pass -- require positive evidence the + # harness actually ran (renaming its SELF_TEST guard produced empty + # output + rc=0, i.e. green with zero assertions). + # bonnyr-f5 #182 r2: require the END marker AND >=1 PASS. The marker + # prints only after the LAST assertion, so an early exit (the #179 shape, + # 5 of 6 unrun) is caught without hardcoding a per-branch test count. + if ! grep -qE '(^|[[:space:]])PASS:' <<< "$out"; then + echo "::error::self-test produced no PASS lines -- the harness did not run"; exit 1 + fi + if ! grep -qE '=== END SELF-TEST ===' <<< "$out"; then + echo "::error::self-test did not reach its END marker -- it exited early with assertions unrun"; exit 1 + fi + - name: extract-breaking-changes SELF_TEST + run: | + # Run the extractor's self-test UNCONDITIONALLY with the SAME anti-vacuity + # assertions as compute's above (ok lines + END marker + no FAIL:). Do NOT + # gate on `grep -- '--self-test' `: the code under test must + # not decide whether it is tested -- deleting the flag would silence ~28 + # assertions with this gate staying green (bonnyr-f5 #193 M6). + set +e + out="$(bash scripts/extract-breaking-changes.sh --self-test 2>&1)"; rc=$? + echo "$out" + if [ "$rc" -ne 0 ]; then + echo "::error::extract-breaking-changes self-test exited $rc"; exit "$rc" + fi + if grep -qE '(^|[[:space:]])FAIL:' <<< "$out"; then + echo "::error::extract self-test reported FAIL: but exited 0"; exit 1 + fi + if ! grep -qE '(^|[[:space:]])ok:' <<< "$out"; then + echo "::error::extract self-test produced no ok: lines -- the harness did not run"; exit 1 + fi + if ! grep -qE '=== END SELF-TEST ===' <<< "$out"; then + echo "::error::extract self-test did not reach its END marker -- it exited early with assertions unrun"; exit 1 + fi + - name: Filesystem self-tests (scripts/tests/*.test.sh) + run: | + # bonnyr-f5 #193 B5 + M5: enumerate and run EVERY scripts/tests/*.test.sh + # from the filesystem, failing on an EMPTY enumeration or ANY non-zero rc. + # This is the SAME enumeration `make script-selftests` runs, so local == + # CI. It covers the registry-tag-probe mutation suite (which had no caller + # at all and was dead code that would have caught B4) AND the INV-15 + # detector-parity check (previously inline in this job only, making + # `make script-selftests` strictly narrower than CI -- M5). + set -euo pipefail + shopt -s nullglob + tests=(scripts/tests/*.test.sh) + if [ "${#tests[@]}" -lt 1 ]; then + echo "::error::no scripts/tests/*.test.sh found -- the self-test enumeration is empty"; exit 1 + fi + echo "running ${#tests[@]} filesystem self-test(s):" + for t in "${tests[@]}"; do + echo "--- $t ---" + bash "$t" || { echo "::error::$t failed"; exit 1; } + done + lint-frontend: name: "P1 · Lint Frontend" needs: changes @@ -668,12 +850,17 @@ jobs: # dependencies. No `|| true` here: if this cannot run, the job has # nothing to say and must fail loudly rather than silently continue. # - # Known fragility, accepted deliberately: the floor's models are - # imported under CURRENT pins, so the gap widens every time a - # dependency moves. v3.0.1 pins cryptography 44 and staging is on 50 — - # six majors — and it holds only because the floor tree touches just - # Fernet, hazmat.primitives.serialization and Ed25519PrivateKey, all - # unchanged across that range. When it does bite, it bites as a + # Known limitation on this repo: f5devcentral/bnk-forge is a squashed + # public mirror and carries exactly ONE final tag, v3.1.6, so the floor + # is currently that tag and the upgrade window is one release wide — + # the degenerate case this check otherwise warns against. It can't be + # widened by naming an older tag (v3.0.1 etc. from the upstream history + # aren't reachable here); it widens only as more finals are cut on this + # repo. Accepted deliberately. + # + # The floor's models are imported under CURRENT pins, so a dependency + # gap can still bite once the window does widen. When it does, it bites + # as a # MANDATORY gate failing hard on a commit that changed nothing # relevant. The fix then is to raise MIN_UPGRADE_FROM to a release # whose models import cleanly, not to add `|| true` here: a floor that @@ -1063,6 +1250,11 @@ jobs: - changes # Phase 1 - lint-backend + - version-consistency + - shellcheck + - secret-scan + - commit-lint + - script-selftests - lint-frontend - typecheck-backend - openapi-check @@ -1094,8 +1286,32 @@ jobs: # Collect all job results (skipped jobs are OK — they were filtered by path) failed=false + + # bonnyr-f5 #182 r3 (Major): the aggregator must verify the change- + # detection job itself SUCCEEDED. If `changes` fails/cancels, ~21 of the + # gates below resolve to `skipped` (their `needs: changes` was never + # satisfied), the old loop accepted skipped as success, and the required + # check printed "CI Gate PASSED" while nothing had actually run. Same + # class as the secret-scan blocker: cannot distinguish "passed" from + # "never evaluated". A non-success `changes` fails the gate outright. + changes_result="${{ needs.changes.result }}" + echo "changes (change-detection): $changes_result" + if [ "$changes_result" != "success" ]; then + echo "::error::change-detection job did not succeed ($changes_result) -- every downstream gate was skipped, so the gate cannot certify anything. Failing." + failed=true + fi + + # bonnyr-f5 #182 r2/r3: these gates run `if: always()` on every change, so + # `skipped` for them means a future path-filter silently disabled the + # check. Treat skipped as a failure for exactly these. + ALWAYS_RUN="version-consistency shellcheck secret-scan commit-lint script-selftests" for job in \ "lint-backend:${{ needs.lint-backend.result }}" \ + "version-consistency:${{ needs.version-consistency.result }}" \ + "shellcheck:${{ needs.shellcheck.result }}" \ + "secret-scan:${{ needs.secret-scan.result }}" \ + "commit-lint:${{ needs.commit-lint.result }}" \ + "script-selftests:${{ needs.script-selftests.result }}" \ "lint-frontend:${{ needs.lint-frontend.result }}" \ "typecheck-backend:${{ needs.typecheck-backend.result }}" \ "openapi-check:${{ needs.openapi-check.result }}" \ @@ -1119,10 +1335,14 @@ jobs: ; do name="${job%%:*}" result="${job##*:}" - # 'success' and 'skipped' are both acceptable + # 'success' and 'skipped' are acceptable, EXCEPT skipped for an + # always-run gate, which means the check silently didn't execute. if [ "$result" = "failure" ] || [ "$result" = "cancelled" ]; then echo "::error::$name: $result" failed=true + elif [ "$result" = "skipped" ] && case " $ALWAYS_RUN " in *" $name "*) true;; *) false;; esac; then + echo "::error::$name was SKIPPED but is an always-run gate — the check never executed" + failed=true else echo "$name: $result" fi diff --git a/.github/workflows/e2e-tests.yml b/.github/workflows/e2e-tests.yml index 9efd0f6..687a64c 100644 --- a/.github/workflows/e2e-tests.yml +++ b/.github/workflows/e2e-tests.yml @@ -68,7 +68,10 @@ jobs: check-changes: name: "Check for recent changes" runs-on: ubuntu-latest - if: github.event_name != 'schedule' || true # always run; schedule check is below + # This job always runs; the schedule-skip logic lives in the step below. The + # old `if: github.event_name != 'schedule' || true` was a dead condition (the + # `|| true` makes it unconditionally true), so it is dropped (bonnyr-f5 #193 + # minor) — omitting `if:` runs the job on every trigger, same effect, no lie. outputs: should_run: ${{ steps.check.outputs.should_run }} steps: @@ -108,6 +111,12 @@ jobs: # ── Start app (local mode) ────────────────────────────────────────── - name: Start application stack if: inputs.environment != 'staging' + env: + # bonnyr-f5 #186 r2: the seeded admin is now generated + must-change, so + # the suite's hardcoded login would fail. Seed a KNOWN, non-default admin + # with the gate off -- ephemeral CI stack only, never a real deployment. + DEFAULT_ADMIN_PASSWORD: e2e-Admin-Pass-1 + DEFAULT_ADMIN_MUST_CHANGE: "false" run: | docker compose up -d echo "Waiting for application to be healthy..." diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 0ded64e..f35fae0 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -11,8 +11,10 @@ # ║ feat → minor ║ # ║ fix / other → patch ║ # ║ ║ -# ║ Infinite-loop guard: commits starting with "release: " have [skip ci] ║ -# ║ appended and are filtered out by the head-commit check below. ║ +# ║ Loop guard: our automated release commits carry [skip ci] in the SUBJECT, ║ +# ║ so GitHub drops their push runs before this workflow even starts. The job ║ +# ║ below is a backstop that FAILS the run on any OTHER suppressed commit — a ║ +# ║ release that published nothing must never read as green. ║ # ╚══════════════════════════════════════════════════════════════════════════════╝ name: Release @@ -22,9 +24,13 @@ on: branches: - staging - main - # MUST stay in sync with ci.yml's push-trigger paths-ignore list — a - # divergence lets a push trigger Release without a matching CI run, - # which then times out the preflight SHA poll below (PR #297 review). + # ci.yml has NO push paths-ignore any more: it runs CI on EVERY push to + # main/staging (ci.yml:31-36, #182). That is a strict superset of the pushes + # that reach Release here, so any push that starts a release is guaranteed a + # matching CI run for the preflight SHA poll below to find — regardless of + # what this list ignores. This paths-ignore therefore only spares docs-only + # pushes from kicking off a release at all (PR #297 review; premise updated + # for #182, which removed ci.yml's paths-ignore). paths-ignore: - '**.md' - 'docs/**' @@ -47,18 +53,44 @@ on: - major # 2.10.49 → 3.0.0 run_e2e: description: "Run E2E tests before release" - required: true + # Not required: a publish_only recovery run ignores it (it has a default + # and never gates the republish path) — bonnyr-f5 #181 round 2. + required: false default: true type: boolean release_notes: - description: "Release notes (one-line summary)" - required: true + description: "Release notes (one-line summary). Ignored when publish_only is set." + # Not required for the same reason: publish_only recovery runs ignore it. + required: false + default: "" type: string + publish_only: + description: "Recovery: republish images for an EXISTING tag (e.g. v4.0.0). Leave empty for a normal release. When set, version_bump / run_e2e / release_notes are ignored." + required: false + type: string + default: "" + force: + description: "Overwrite images that ALREADY exist in the registry for this tag. Off by default: publishing refuses when the :VERSION manifest is already present, so a republish can't silently move an immutable tag and orphan its cosign/SBOM/SLSA attestations. Only a recovery of a tag whose publish never completed needs this off; set it on to deliberately re-push." + required: false + type: boolean + default: false + sign_only: + description: "Recovery for a publish that pushed all images but failed at signing (e.g. cosign/OIDC error AFTER the bake succeeded). Re-runs cosign sign + SBOM + provenance against the ALREADY-pushed :VERSION digests WITHOUT rebuilding or re-pushing — cosign sign is idempotent and the immutable tag never moves. Requires publish_only=. Use this instead of force=true for a sign-only recovery: force rebuilds all images to possibly-different digests and moves the immutable tag (the exact INV-24 harm the overwrite guard exists to prevent)." + required: false + type: boolean + default: false concurrency: group: release-${{ github.ref_name }} cancel-in-progress: false # Never cancel a release in progress +# Least-privilege default so guard/preflight don't inherit the repo-default +# token scope (bonnyr-f5 #181 round 2). Jobs that need more (contents: write to +# push tags, packages/id-token to publish) declare it themselves below; a job's +# own permissions: block replaces this one rather than merging. +permissions: + contents: read + jobs: # ── Guard: skip release: commits on main (prevents infinite loop) ──────────── guard: @@ -73,13 +105,112 @@ jobs: HEAD_COMMIT_MSG: ${{ github.event.head_commit.message }} run: | MSG="$HEAD_COMMIT_MSG" - # Skip if triggered by our own release commit or [skip ci] sentinel - if echo "$MSG" | grep -qE '^release: |^\[skip ci\]|\[skip ci\]$'; then - echo "Skipping: head commit is a release commit or has [skip ci]" + FIRST="${MSG%%$'\n'*}" + # Match the SUBJECT line only. grep is line-oriented, so testing the + # whole message let a [skip ci] line buried in the body — a quoted CI + # snippet, a changelog paste — suppress a legitimate release while the + # message below quoted a subject carrying no marker (bonnyr-f5 #181 + # round 2). On a normal push GitHub's own [skip ci] handling already + # drops our release commits (their subject is "release: vX.Y.Z + # [skip ci]") before this workflow starts, and on workflow_dispatch + # head_commit is null so FIRST is empty and nothing matches — this + # guard is a backstop, not the primary suppressor. + if grep -qE '^release: |^\[skip ci\]|\[skip ci\]$' <<< "$FIRST"; then echo "should_run=false" >> "$GITHUB_OUTPUT" - else - echo "should_run=true" >> "$GITHUB_OUTPUT" + # Only OUR OWN automated release commit is exempt (a silent, green + # skip). It has a distinguishing fingerprint that a human commit does + # not: the subject is EXACTLY "release: vX.Y.Z [skip ci]" — a version + # AND the skip marker we ourselves append (see the two `git commit` + # calls below). Matching "release:" + a version alone was wrong: a + # hand-written commit like "release: v3.2.0 notes" (a human writing + # release notes, no marker) also matched and was silently skipped + # green, publishing nothing while reporting success — the exact + # silent-green class this guard exists to prevent (bonnyr-f5 #181 + # round 4). Requiring the trailing [skip ci] marker restricts the + # exemption to commits we minted. + # + # SINGLE SOURCE (bonnyr-f5 #193 r3 minor): this same fingerprint is + # _is_release_bot_subject in scripts/lib/is-release-bot-subject.sh, used + # by the commit-lint gate. This `guard` job runs WITHOUT a checkout (it + # reads only the event payload) so it cannot source that file; the two + # grep predicates below are therefore kept BYTE-IDENTICAL to the lib and + # scripts/tests/lint-commit-markers.test.sh (SS-1) asserts they match, so + # they cannot silently diverge. + if grep -qE '^release: v[0-9]+\.[0-9]+\.[0-9]+' <<< "$FIRST" && grep -qE '\[skip ci\]$' <<< "$FIRST"; then + # Our own automated release commit reached the guard anyway (e.g. + # a re-tag or a replay that kept the marker). The release it names + # was already published by the run that created it, so suppressing + # it is expected -- a notice, and the run stays green. + echo "::notice::Loop guard: skipping our own release commit \"$FIRST\" (expected -- it was published by the previous run)." + exit 0 + fi + # Any OTHER suppressed commit is unexpected: a hand-written + # [skip ci], a human "release: ..." subject with no marker, or a + # squash-merged PR titled "release: …(#N)". The run publishes + # nothing, so it must NOT report success -- a release that did not + # happen has to be loud, not a silent green (bonnyr-f5 #181 round 3 + # and 4). Fail the guard; downstream jobs are gated on should_run and + # stay skipped, so nothing is published either way. + echo "::error::Release suppressed by the loop guard: head commit \"$FIRST\" begins with 'release: ' or carries [skip ci], so nothing was published. To cut a NEW release, push a normal commit whose subject does NOT begin with 'release: ' and carries no skip marker (it will release on merge to main). To re-publish images for an EXISTING tag, dispatch this workflow with publish_only= — do not manually dispatch a final release on a skip-marked head, GitHub creates no CI run for it and the CI-status check cannot pass." + exit 1 fi + echo "should_run=true" >> "$GITHUB_OUTPUT" + + # ── Dry-run: exercise the release-publish TOOLING without publishing ────────── + # bonnyr-f5 #193 r4 B-2 (bonnyr's suggestion): the r3 verify-image-pins step could + # NEVER execute — it ran the script from a sparse .release-tooling checkout that holds + # no compose file, and passed REGISTRY/VERSION as env vars the script ignores — yet it + # sat AFTER the tag/Release/push/signing, so the defect only surfaced post-signature. + # This job rebuilds the EXACT publish-job on-disk layout (full tree at the workspace + # root + the same 4-path sparse .release-tooling) and runs the pin-verify tooling + # against a FAKE probe, so a step that cannot execute is caught HERE — before it is + # ever wired ahead of a signature. release-publish `needs` it, so a broken layout + # blocks the publish instead of failing after it. + dryrun-release-tooling: + name: "Dry-run: release-publish tooling" + needs: guard + if: needs.guard.outputs.should_run == 'true' + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - name: Checkout full tree (compose files present at root, as in the tag checkout) + uses: actions/checkout@v6 + with: + fetch-depth: 0 + - name: Materialize the tooling EXACTLY as release-publish does (sparse, no compose) + uses: actions/checkout@v6 + with: + ref: ${{ github.sha }} + path: .release-tooling + sparse-checkout: | + scripts/registry-tag-probe.sh + scripts/registry-overwrite-guard.sh + scripts/verify-image-pins.sh + docker-bake.hcl + sparse-checkout-cone-mode: false + - name: bash -n the tooling scripts materialized in .release-tooling + run: | + for s in .release-tooling/scripts/*.sh; do bash -n "$s"; done + - name: Prove the POST-PUSH verify step can execute (sparse script + --file + FLAGS, fake probe) + run: | + # No live registry: a fake probe that always "exists" proves the pins RESOLVE + # from the sparse checkout via --file (the r3 bug was rc=1 "no pins"), and that + # REGISTRY/VERSION arrive as FLAGS the script actually reads. + printf '#!/usr/bin/env bash\nexit 0\n' > "$RUNNER_TEMP/fakeprobe" + chmod +x "$RUNNER_TEMP/fakeprobe" + IMAGE_PROBE="$RUNNER_TEMP/fakeprobe" \ + bash .release-tooling/scripts/verify-image-pins.sh \ + --registry "ghcr.io/${{ github.repository_owner }}" --version 0.0.0-dryrun \ + --file dist/docker-compose.yml \ + --file dist/docker-compose.local.yml \ + --file scripts/ibm_cloud_bnk_forge.sh + - name: Prove the PRE-PUSH consistency gate can execute (committed pins == VERSION) + run: | + # The committed compose pins must equal the repo VERSION (what a real release + # sets $NEW to via sync-version-artifacts.sh --write). Fails loud here if the + # shipped pins have drifted from VERSION — the same gate release-final runs. + bash scripts/verify-image-pins.sh --expect-version "$(cat VERSION)" # ── Pre-flight: verify CI passed + derive version ──────────────────────────── preflight: @@ -87,11 +218,20 @@ jobs: needs: guard if: needs.guard.outputs.should_run == 'true' runs-on: ubuntu-latest + permissions: + contents: read # checkout + read VERSION/tags + actions: read # gh run list --workflow=ci.yml (the CI-status poll) outputs: current_version: ${{ steps.version.outputs.current }} new_version: ${{ steps.version.outputs.new }} bump_type: ${{ steps.version.outputs.bump_type }} release_kind: ${{ steps.kind.outputs.kind }} + env: + # Env-indirect the ref name (bonnyr-f5 #181 round 2): a branch name can + # legally carry $(), backticks, ; and | (workflow_dispatch targets any + # branch), so it's read from env in run: blocks, never interpolated — + # same rule this PR applies to release_notes/publish_only. + REF_NAME: ${{ github.ref_name }} steps: - uses: actions/checkout@v6 with: @@ -99,21 +239,82 @@ jobs: - name: Determine release kind id: kind + env: + PUBLISH_ONLY: ${{ inputs.publish_only }} + SIGN_ONLY: ${{ inputs.sign_only }} run: | - if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then + # sign_only is a recovery MODE of publish_only (re-sign already-pushed + # images), not a release path of its own. Refuse it without a target + # tag so it can never be dispatched against a from-scratch build + # (bonnyr-f5 #181 round 5, F2). + if [ "$SIGN_ONLY" = "true" ] && [ -z "$PUBLISH_ONLY" ]; then + echo "::error::sign_only requires publish_only=: it re-signs the images already published for an EXISTING tag, it does not build a release." + exit 1 + fi + if [ "${{ github.event_name }}" = "workflow_dispatch" ] && [ -n "$PUBLISH_ONLY" ]; then + echo "kind=publish_only" >> "$GITHUB_OUTPUT" + elif [ "${{ github.event_name }}" = "workflow_dispatch" ]; then echo "kind=manual" >> "$GITHUB_OUTPUT" - elif [ "${{ github.ref_name }}" = "main" ]; then + elif [ "$REF_NAME" = "main" ]; then echo "kind=final" >> "$GITHUB_OUTPUT" else echo "kind=rc" >> "$GITHUB_OUTPUT" fi + - name: Restrict manual release to main + if: steps.kind.outputs.kind == 'manual' + env: + DISPATCH_REF: ${{ github.ref }} + run: | + # A manual (non-publish_only) dispatch builds THIS ref's tree, commits + # the version bump to it, tags it, and publishes it as :latest. + # Dispatching from staging or a side branch would ship an unreviewed + # tree as the released :latest, and the recency guard would not catch + # it (a numerically higher version passes: measured ALLOW new=5.0.0 + # highest=v4.0.5). + # + # The dispatched ref must BE main itself (refs/heads/main). An earlier + # "ancestor of main" exemption was WRONG: staging sits 0-ahead/1-behind + # main, so `git merge-base --is-ancestor origin/staging origin/main` is + # true, and a manual dispatch on staging slipped through — the exact ref + # this step names as the hazard (release commit+tag+GitHub Release land + # on staging, :latest built from it, main never bumped) (bonnyr-f5 #181 + # round 4). Any ancestor of main is by definition already ON main's + # first-parent history if it was merged, so requiring the ref to be main + # loses nothing legitimate. publish_only is exempt (kind != manual): it + # republishes an existing tag by checking that tag out, independent of + # the dispatched ref. + if [ "$DISPATCH_REF" = "refs/heads/main" ]; then + echo "Dispatched from main." + exit 0 + fi + echo "::error::A manual final release must be dispatched from main itself (refs/heads/main). '$REF_NAME' (ref '$DISPATCH_REF') is not main — dispatching it would publish an unreviewed tree as :latest. Merge to main and dispatch from there, or use publish_only to republish an existing tag." + exit 1 + - name: Check CI status on this branch + if: steps.kind.outputs.kind != 'publish_only' run: | - BRANCH="${{ github.ref_name }}" + BRANCH="$REF_NAME" SHA="${{ github.sha }}" echo "Checking CI status for branch '$BRANCH' at commit $SHA" + # Fail FAST on a CI-skip-marked head instead of polling for 2700s. + # After every automated release, main's head is + # "release: vX.Y.Z [skip ci]"; GitHub creates NO CI run for a commit + # carrying a skip marker, so a manual final dispatch on that head would + # poll for a run that will never appear and only surface the problem + # after the full 45-minute timeout. There is no CI to wait for and none + # is coming, so refuse immediately with the real recovery, rather than + # advising a wait (bonnyr-f5 #181 round 4). A genuine change never + # reaches here skip-marked: GitHub drops skip-marked pushes before the + # workflow starts, so this only trips on a manual dispatch of a + # skip-marked head. + HEAD_SUBJECT="$(git log -1 --format=%s HEAD)" + if grep -qiE '\[skip ci\]|\[ci skip\]|\[skip actions\]' <<< "$HEAD_SUBJECT"; then + echo "::error::The head commit \"$HEAD_SUBJECT\" carries a CI-skip marker, so GitHub created no CI run for it and none ever will — waiting would only time out after ${CI_POLL_TIMEOUT_SECONDS}s. This is the state main is left in immediately after an automated release. To cut a NEW release, push a normal (non-skip-marked) commit and let it release on merge to main. To re-publish images for the EXISTING tag, dispatch this workflow with publish_only=, which skips this CI check." + exit 1 + fi + # Match the CI run by the exact commit SHA that triggered this # release, not by "--limit=1 --branch=X" (which is racy: on main, # ci.yml and release.yml fire on the same push so --limit=1 can @@ -128,9 +329,16 @@ jobs: ELAPSED=0 while true; do + # Filter by the commit SHA SERVER-SIDE (--commit), not by fetching the + # newest 20 runs on the branch and grepping (bonnyr-f5 #193 minor): with + # a busy branch, >20 CI runs newer than this SHA pushed the target run off + # the --limit=20 page, so the poll never saw it and the release timed out + # claiming no run existed. --commit returns only this SHA's runs, so the + # page holds every attempt for it; the jq keeps the latest attempt. RUN_JSON=$(gh run list \ --workflow=ci.yml \ --branch="$BRANCH" \ + --commit="$SHA" \ --limit=20 \ --json headSha,status,conclusion,databaseId \ --jq "[.[] | select(.headSha == \"${SHA}\")] | sort_by(.databaseId) | last" 2>/dev/null || echo "") @@ -148,12 +356,16 @@ jobs: exit 0 fi if [ "$RUN_CONCLUSION" = "cancelled" ]; then - # ci.yml runs with cancel-in-progress: true, so a rapid - # follow-up push to the same branch cancels this SHA's CI - # run. That's not a CI failure for this SHA — it means a - # newer push superseded it, so fail fast instead of - # reporting a generic non-success error. - echo "::error::CI run for $SHA was cancelled — superseded by a newer push; this release attempt is stale, the newer push will release instead." + # A cancelled CI run is not a success for this SHA, so fail + # fast rather than reporting a generic non-success error. + # NOTE (premise updated for #182): ci.yml does NOT cancel + # in-progress runs on main/staging — cancel-in-progress is + # false for exactly these release branches (ci.yml:48), each + # push gets its own concurrency group. So a cancellation here + # is no longer necessarily "a newer push superseded it"; it may + # have been cancelled by other means (e.g. a manual cancel). + # Either way the run is stale — do not release on it. + echo "::error::CI run for $SHA was cancelled — this release attempt is stale (a newer push, if any, will release instead)." exit 1 fi echo "::error::CI run for commit $SHA on branch '$BRANCH' completed with conclusion '$RUN_CONCLUSION'." @@ -176,14 +388,34 @@ jobs: - name: Derive version from conventional commits id: version + env: + PUBLISH_ONLY: ${{ inputs.publish_only }} run: | chmod +x scripts/compute_version_bump.sh RELEASE_KIND="${{ steps.kind.outputs.kind }}" CURRENT=$(cat VERSION) + # Validate before it reaches $GITHUB_OUTPUT: a newline in VERSION would + # inject a second output key, and current_version is interpolated into + # several later run: blocks (bonnyr-f5 #181 round 2). + if [[ ! "$CURRENT" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then + echo "::error::VERSION file contents '$CURRENT' are not a valid MAJOR.MINOR.PATCH version." + exit 1 + fi echo "current=$CURRENT" >> "$GITHUB_OUTPUT" - if [ "$RELEASE_KIND" = "manual" ]; then + if [ "$RELEASE_KIND" = "publish_only" ]; then + # Recovery path: no derivation, no bump. Republish an existing tag. + # publish_only is free-text, so read it from env (never inline it into + # the script) and validate its shape before use (#181 review). + TAG="$PUBLISH_ONLY" + if [[ ! "$TAG" =~ ^v?[0-9]+\.[0-9]+\.[0-9]+$ ]]; then + echo "::error::publish_only='$TAG' is not a final release tag (vMAJOR.MINOR.PATCH). Republish repoints :latest across all images and must not point it at an rc/pre-release or an arbitrary string." + exit 1 + fi + NEW="${TAG#v}" + BUMP="republish" + elif [ "$RELEASE_KIND" = "manual" ]; then # Manual override: use the dropdown input directly IFS='.' read -r MAJOR MINOR PATCH <<< "$CURRENT" case "${{ inputs.version_bump }}" in @@ -203,12 +435,50 @@ jobs: BUMP="$BUMP_TYPE" fi + # :latest recency guard — covers the paths that both MINT a new tag and + # repoint :latest to it: final and manual. A version below the highest + # final tag would drag :latest backward (bonnyr-f5 #181 round 2 — + # release-manual from a maintenance branch could tag+publish a version + # below the highest final tag). rc tags are pre-releases and never touch + # :latest, so they're exempt. + # + # publish_only is NOT hard-failed here: a republish re-emits the + # IMMUTABLE :VERSION tags of an ALREADY-released version whose original + # publish half-completed, and it owes those tags to consumers regardless + # of what has been released since. Failing it on recency stranded it + # forever — once any newer tag existed, the half-published version's + # images could never be produced by ANY path (bonnyr-f5 #181 round 4). + # A republish must still never move :latest backward, but that is a + # decision about the FLOATING tag only, and it is made authoritatively + # inside the publish job's critical section (see "Re-check recency"), + # not here. :latest is set by whatever release-publish last ran, not by + # this ref's VERSION file, so that check compares against the highest + # FINAL tag at push time. + if [ "$RELEASE_KIND" = "final" ] || [ "$RELEASE_KIND" = "manual" ]; then + HIGHEST_TAG="$(git tag -l 'v*' | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1)" + HIGHEST="$(printf '%s\n%s\n' "${HIGHEST_TAG#v}" "$NEW" | sort -V | tail -1)" + if [ -n "$HIGHEST_TAG" ] && [ "$NEW" != "$HIGHEST" ]; then + echo "::error::Release v$NEW is older than the highest released tag $HIGHEST_TAG; publishing would move :latest backward." + exit 1 + fi + fi + echo "new=$NEW" >> "$GITHUB_OUTPUT" echo "bump_type=$BUMP" >> "$GITHUB_OUTPUT" echo "Version: $CURRENT -> $NEW (bump: $BUMP)" + - name: Check republish tag exists + if: steps.kind.outputs.kind == 'publish_only' + run: | + TAG="v${{ steps.version.outputs.new }}" + if ! git tag -l "$TAG" | grep -q "^${TAG}$"; then + echo "::error::publish_only requested tag $TAG, which does not exist. Republish only targets an already-created tag." + exit 1 + fi + echo "Republishing images for existing tag $TAG" + - name: Check final tag doesn't already exist - if: steps.kind.outputs.kind != 'rc' + if: steps.kind.outputs.kind != 'rc' && steps.kind.outputs.kind != 'publish_only' run: | TAG="v${{ steps.version.outputs.new }}" if git tag -l "$TAG" | grep -q "^${TAG}$"; then @@ -234,6 +504,8 @@ jobs: runs-on: ubuntu-latest permissions: contents: write + env: + REF_NAME: ${{ github.ref_name }} # env-indirected — see preflight steps: - uses: actions/checkout@v6 with: @@ -249,14 +521,33 @@ jobs: id: rc run: | TARGET="${{ needs.preflight.outputs.new_version }}" - # Count existing rc tags for this target version - RC_COUNT=$(git tag -l "v${TARGET}-rc.*" | wc -l | tr -d ' ') - RC_NUM=$((RC_COUNT + 1)) + # Highest existing rc number for this target + 1. Max-based, not + # count-based (#177 review): counting breaks if any rc tag is ever + # deleted -- the count drops and the next push recomputes an existing + # tag, which then fails to create. + # NB: this step must NOT `set -o pipefail`. `grep -E` returns 1 when a + # version has no rc tags yet; without pipefail the substitution takes + # tail's status and ${RC_MAX:-0} yields the first rc as 1. Adding + # pipefail here would fail rc.1 of every new version. + # Escape TARGET's dots so the sed anchor matches them literally, not + # as "any char" (bonnyr-f5 #181 round 2). The grep -l glob already + # pre-filters to real tags, but a literal-dot pattern is correct. + TARGET_RE="${TARGET//./\\.}" + RC_MAX=$(git tag -l "v${TARGET}-rc.*" \ + | sed -E "s|^v${TARGET_RE}-rc\.([0-9]+)$|\1|" \ + | grep -E '^[0-9]+$' | sort -n | tail -1) + RC_NUM=$(( ${RC_MAX:-0} + 1 )) echo "rc_num=${RC_NUM}" >> "$GITHUB_OUTPUT" echo "rc_tag=v${TARGET}-rc.${RC_NUM}" >> "$GITHUB_OUTPUT" echo "RC tag will be: v${TARGET}-rc.${RC_NUM}" - - name: Create annotated RC tag + # bonnyr-f5 #193 r4 M-5 / INV-31: create the tag LOCALLY here but do NOT push it + # yet. The RC notes below run extract-breaking-changes.sh, which is fail-closed + # (exit 1 on a bad range). Generating notes AFTER the tag push previously left an + # RC tag on the remote with no pre-release when notes failed — the same + # irreversible-push-before-fail-closed-step class fixed in release-final. The push + # now happens AFTER notes succeed (see "Push RC tag"). + - name: Create annotated RC tag (local) run: | RC_TAG="${{ steps.rc.outputs.rc_tag }}" TARGET="${{ needs.preflight.outputs.new_version }}" @@ -265,11 +556,9 @@ jobs: git tag -a "$RC_TAG" \ -m "Pre-release ${RC_TAG} Target: v${TARGET} (${BUMP} bump) - Branch: ${{ github.ref_name }} + Branch: ${REF_NAME} Commit: ${{ github.sha }}" - git push origin "$RC_TAG" - - name: Generate RC release notes id: notes run: | @@ -277,27 +566,39 @@ jobs: BUMP="${{ needs.preflight.outputs.bump_type }}" RC_TAG="${{ steps.rc.outputs.rc_tag }}" - # Find last final tag for commit range + # Find the last final tag for the commit range, ancestry-filtered to HEAD so + # a higher tag cut on another branch cannot skew LAST_FINAL..HEAD (#193 r4). LAST_FINAL=$(git tag -l 'v*' \ | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' \ + | while IFS= read -r t; do git merge-base --is-ancestor "$t" HEAD 2>/dev/null && echo "$t"; done \ | sort -V | tail -1) NOTES="## Pre-release ${RC_TAG} **Target release:** v${TARGET} (${BUMP} bump) - **Branch:** ${{ github.ref_name }} + **Branch:** ${REF_NAME} **Commit:** ${{ github.sha }} ### Commits since ${LAST_FINAL:-initial} " + # Cap the commit list, but say so when it truncates -- a silent `head` + # dropped 17 of 67 commits from published notes with no indication + # (bonnyr-f5 #179 r3). CAP is generous enough that normal ranges are + # complete; a larger range appends an explicit "and N more" line. + CAP=300 if [ -n "$LAST_FINAL" ]; then - COMMIT_LOG=$(git log "${LAST_FINAL}..HEAD" --pretty=format:"- %s" | head -40) - BREAKING=$(bash scripts/extract-breaking-changes.sh "$LAST_FINAL" HEAD || true) + FULL_LOG=$(git log "${LAST_FINAL}..HEAD" --pretty=format:"- %s") + BREAKING=$(bash scripts/extract-breaking-changes.sh "$LAST_FINAL" HEAD) else - COMMIT_LOG=$(git log --pretty=format:"- %s" | head -40) + FULL_LOG=$(git log --pretty=format:"- %s") BREAKING="" fi + COMMIT_LOG=$(printf '%s\n' "$FULL_LOG" | head -"$CAP") + TOTAL=$(printf '%s\n' "$FULL_LOG" | grep -c '^-' || true) + if [ "$TOTAL" -gt "$CAP" ]; then + COMMIT_LOG=$(printf '%s\n- … and %d more commit(s) — see the full compare view' "$COMMIT_LOG" "$((TOTAL - CAP))") + fi if [ -n "$BREAKING" ]; then printf '%s\n\n%s\n\n%s' "$NOTES" "$BREAKING" "$COMMIT_LOG" > /tmp/rc_notes.md @@ -306,13 +607,18 @@ jobs: fi echo "notes_file=/tmp/rc_notes.md" >> "$GITHUB_OUTPUT" + # bonnyr-f5 #193 r4 M-5 / INV-31: the fail-closed notes step above has passed, so + # the push is now safe — nothing irreversible happened before it. + - name: Push RC tag + run: git push origin "${{ steps.rc.outputs.rc_tag }}" + - name: Create GitHub pre-release run: | gh release create "${{ steps.rc.outputs.rc_tag }}" \ --title "${{ steps.rc.outputs.rc_tag }}" \ --notes-file "${{ steps.notes.outputs.notes_file }}" \ --prerelease \ - --target "${{ github.ref_name }}" + --target "${REF_NAME}" env: GH_TOKEN: ${{ github.token }} @@ -325,7 +631,7 @@ jobs: echo "| RC Tag | ${{ steps.rc.outputs.rc_tag }} |" >> "$GITHUB_STEP_SUMMARY" echo "| Target Version | v${{ needs.preflight.outputs.new_version }} |" >> "$GITHUB_STEP_SUMMARY" echo "| Bump Type | ${{ needs.preflight.outputs.bump_type }} |" >> "$GITHUB_STEP_SUMMARY" - echo "| Branch | ${{ github.ref_name }} |" >> "$GITHUB_STEP_SUMMARY" + echo "| Branch | ${REF_NAME} |" >> "$GITHUB_STEP_SUMMARY" echo "| Commit | ${{ github.sha }} |" >> "$GITHUB_STEP_SUMMARY" # ── Final Release (main push) ───────────────────────────────────────────────── @@ -336,6 +642,9 @@ jobs: runs-on: ubuntu-latest permissions: contents: write + packages: read # bonnyr-f5 #193 M4: the pre-push overwrite-guard reads ghcr manifests + env: + REF_NAME: ${{ github.ref_name }} # env-indirected — see preflight steps: - uses: actions/checkout@v6 with: @@ -355,6 +664,10 @@ jobs: if git ls-files --error-unmatch dist/VERSION 2>/dev/null; then echo "$NEW" > dist/VERSION fi + # Keep the Helm chart tag/appVersion and frontend package.json in + # lockstep so the chart never pins an image tag the release doesn't + # publish (#177 Blocker 2). + bash scripts/sync-version-artifacts.sh --write "$NEW" - name: Update CHANGELOG.md run: | @@ -362,20 +675,37 @@ jobs: DATE=$(date +%Y-%m-%d) BUMP="${{ needs.preflight.outputs.bump_type }}" - # Find last final tag for commit range (exclude the one we're about to create) + # Find the last final tag for the commit range. Ancestry-filter to HEAD + # (bonnyr-f5 #193 r4 minor): a numerically-higher tag cut on ANOTHER branch + # is not on this history, so `git tag | sort -V | tail -1` could pick it and + # yield a nonsensical LAST_FINAL..HEAD range. Only tags reachable from HEAD + # (merged into this release ref) are candidates; then take the highest. LAST_FINAL=$(git tag -l 'v*' \ | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' \ + | while IFS= read -r t; do git merge-base --is-ancestor "$t" HEAD 2>/dev/null && echo "$t"; done \ | sort -V | tail -1) + # Cap the commit list but say so on truncation -- a silent `head -50` + # dropped 17 of 67 commits from published notes (bonnyr-f5 #179 r3). + # The `|| true` here guards the FILTER only (an all-"release:" range + # leaves grep -v with no output, rc 1 under pipefail). extract-breaking- + # changes.sh is itself fail-closed now — #179's rewrite is in-tree and + # `exit 1`s on a bad range with no `|| true` (bonnyr-f5 #193 minor: the + # old comment here claiming this PR does not touch that script and that + # it still ends its range query with `|| true` at line 33 was stale). + CAP=300 if [ -n "$LAST_FINAL" ]; then - COMMITS=$(git log "${LAST_FINAL}..HEAD" --pretty=format:"- %s" \ - | grep -v "^- release: " | head -50) - BREAKING=$(bash scripts/extract-breaking-changes.sh "$LAST_FINAL" HEAD || true) + FULL_LOG=$(git log "${LAST_FINAL}..HEAD" --pretty=format:"- %s" | { grep -v "^- release: " || true; }) + BREAKING=$(bash scripts/extract-breaking-changes.sh "$LAST_FINAL" HEAD) else - COMMITS=$(git log --pretty=format:"- %s" \ - | grep -v "^- release: " | head -50) + FULL_LOG=$(git log --pretty=format:"- %s" | { grep -v "^- release: " || true; }) BREAKING="" fi + COMMITS=$(printf '%s\n' "$FULL_LOG" | head -"$CAP") + TOTAL=$(printf '%s\n' "$FULL_LOG" | grep -c '^-' || true) + if [ "$TOTAL" -gt "$CAP" ]; then + COMMITS=$(printf '%s\n- … and %d more commit(s) — see the full compare view' "$COMMITS" "$((TOTAL - CAP))") + fi # Build the entry in a temp file rather than interpolating COMMITS # (multi-line, and any commit subject containing |, &, \, or @@ -405,6 +735,16 @@ jobs: } { print } ' CHANGELOG.md > CHANGELOG.md.new + # Fail CLOSED if the insertion was a no-op: the awk anchors on the first + # `^---$`, and if that separator is ever absent every line passes through + # unchanged with rc 0 — the entry is silently dropped and the release + # ships with no CHANGELOG record (bonnyr-f5 #193 minor). Assert the output + # actually grew before replacing the file. + if cmp -s CHANGELOG.md CHANGELOG.md.new; then + echo "::error::CHANGELOG.md insertion was a no-op (no '---' anchor found?) — refusing to commit a release with no changelog entry." + rm -f CHANGELOG.md.new "$ENTRY_FILE" + exit 1 + fi mv CHANGELOG.md.new CHANGELOG.md rm -f "$ENTRY_FILE" @@ -413,9 +753,44 @@ jobs: NEW="${{ needs.preflight.outputs.new_version }}" BUMP="${{ needs.preflight.outputs.bump_type }}" - # Stage VERSION, dist/VERSION (if tracked), CHANGELOG + # Stage VERSION, dist/VERSION (if tracked), CHANGELOG, and the + # version-bearing artifacts synced above (#177 Blocker 2). git add VERSION CHANGELOG.md - git ls-files --error-unmatch dist/VERSION 2>/dev/null && git add dist/VERSION || true + # Stage EXACTLY the artifacts sync-version-artifacts.sh owns, from its + # own --list, so a newly-synced file can never be left unstaged and die + # with the runner (bonnyr-f5 #180 r3, BLOCKER 1: --write rewrote five + # files, the hard-coded `git add` staged three). + # LEAD (bonnyr-f5 #193 r4): derive the anti-vacuity floor from the artifact + # list ITSELF, not a stale literal. --list yields 8 paths today (the dist/+IBM + # pins added to close B-1); the old `-lt 5` left three of them free to vanish + # and still pass. EXPECTED tracks the true set and every listed path must stage + # (staged == EXPECTED), so a dropped path is caught; an empty/near-empty --list + # (script broke/truncated) trips EXPECTED<1 here AND the script's own + # `--check total < 8` vacuity floor. The old comment's "mirrors --check + # total < 5" was doubly stale — that guard is now < 8 and counts matched LINES + # (21), not paths. The per-file "not fully staged" check below still verifies + # each staged path individually. + EXPECTED=$(bash scripts/sync-version-artifacts.sh --list | grep -c . || true) + staged=0 + while IFS= read -r f; do git add "$f"; staged=$((staged + 1)); done < <(bash scripts/sync-version-artifacts.sh --list) + if [ "$EXPECTED" -lt 1 ] || [ "$staged" -ne "$EXPECTED" ]; then + echo "::error::sync-version-artifacts.sh --list: staged $staged of $EXPECTED path(s) — refusing to commit an unsynced release"; exit 1 + fi + # Verify the INDEX, not the files: --write's post-write check re-reads + # the files (correct on disk even when unstaged), so assert each synced + # artifact has no unstaged residue — i.e. the sync is actually in the + # commit we are about to make. + while IFS= read -r f; do + git diff --quiet -- "$f" || { echo "::error::$f was synced but is not fully staged"; exit 1; } + done < <(bash scripts/sync-version-artifacts.sh --list) + # Stage dist/VERSION if it is tracked. `git add` of a tracked path can + # only fail for a real reason (permission, corrupt index), so do NOT + # swallow it with `|| true` — that would silently drop the dist bump from + # the release commit (bonnyr-f5 #193 minor). The ls-files guard already + # makes this a no-op when dist/VERSION is untracked. + if git ls-files --error-unmatch dist/VERSION >/dev/null 2>&1; then + git add dist/VERSION + fi git commit -m "release: v${NEW} [skip ci] @@ -427,30 +802,51 @@ jobs: git tag -a "v${NEW}" \ -m "Release v${NEW} (${BUMP} bump)" - - name: Push commit and tag - run: | - git push origin "${{ github.ref_name }}" - git push origin "v${{ needs.preflight.outputs.new_version }}" - + # bonnyr-f5 #193 M4 / INV-31: generate the release notes BEFORE the + # irreversible push. extract-breaking-changes.sh is fail-closed (exit 1 on a + # bad range, no `|| true`), so a failure here previously left main bumped and + # the tag pushed with no Release. Generating notes ahead of the push keeps + # the fail-closed step in front of anything irreversible; the output is + # consumed by "Create GitHub Release" after the push. - name: Generate final release notes id: notes run: | NEW="${{ needs.preflight.outputs.new_version }}" BUMP="${{ needs.preflight.outputs.bump_type }}" + # The local tag v$NEW was just created by "Commit and tag", so exclude it + # explicitly to find the PREVIOUS final tag. The old `tail -2 | head -1` + # returned the just-created tag on a first-ever release (only one tag + # exists), giving an empty v$NEW..HEAD range and empty notes (bonnyr-f5 + # #193 minor). grep -vx is robust whether or not the new tag is present. + # Exclude the just-created v$NEW tag AND ancestry-filter to HEAD so a + # higher tag on another branch cannot become LAST_FINAL (bonnyr-f5 #193 r4). LAST_FINAL=$(git tag -l 'v*' \ | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' \ - | sort -V | tail -2 | head -1) + | grep -vx "v${NEW}" \ + | while IFS= read -r t; do git merge-base --is-ancestor "$t" HEAD 2>/dev/null && echo "$t"; done \ + | sort -V | tail -1) + # Cap the commit list but say so on truncation -- a silent `head -50` + # dropped 17 of 67 commits from published notes (bonnyr-f5 #179 r3). The + # `|| true` below guards the FILTER only (an all-"release:" range leaves + # grep -v with no output, rc 1 under pipefail); extract-breaking-changes.sh + # is itself fail-closed on a bad range (it now `exit 1`s, no `|| true`), + # and this step runs BEFORE the push (INV-31), so a bad range fails the + # release before anything irreversible. + CAP=300 if [ -n "$LAST_FINAL" ]; then - COMMITS=$(git log "${LAST_FINAL}..HEAD" --pretty=format:"- %s" \ - | grep -v "^- release: " | head -50) - BREAKING=$(bash scripts/extract-breaking-changes.sh "$LAST_FINAL" HEAD || true) + FULL_LOG=$(git log "${LAST_FINAL}..HEAD" --pretty=format:"- %s" | { grep -v "^- release: " || true; }) + BREAKING=$(bash scripts/extract-breaking-changes.sh "$LAST_FINAL" HEAD) else - COMMITS=$(git log --pretty=format:"- %s" \ - | grep -v "^- release: " | head -50) + FULL_LOG=$(git log --pretty=format:"- %s" | { grep -v "^- release: " || true; }) BREAKING="" fi + COMMITS=$(printf '%s\n' "$FULL_LOG" | head -"$CAP") + TOTAL=$(printf '%s\n' "$FULL_LOG" | grep -c '^-' || true) + if [ "$TOTAL" -gt "$CAP" ]; then + COMMITS=$(printf '%s\n- … and %d more commit(s) — see the full compare view' "$COMMITS" "$((TOTAL - CAP))") + fi { echo "## v${NEW} — ${BUMP} bump" @@ -464,12 +860,42 @@ jobs: echo "notes_file=/tmp/release_notes.md" >> "$GITHUB_OUTPUT" + # bonnyr-f5 #193 M4 / INV-31: refuse a republish of an already-published + # immutable :VERSION tag BEFORE the push, not only inside release-publish + # (which runs after these pushes). Nothing irreversible happens before this + # fail-closed check. release-publish keeps its own authoritative in-critical- + # section re-check for the concurrency TOCTOU. + - name: Refuse to overwrite an already-published tag (pre-push) + env: + FORCE: ${{ inputs.force }} + REGISTRY: ghcr.io/${{ github.repository_owner }} + VERSION: ${{ needs.preflight.outputs.new_version }} + REGISTRY_USERNAME: ${{ github.actor }} + REGISTRY_PASSWORD: ${{ secrets.GITHUB_TOKEN }} + run: bash scripts/registry-overwrite-guard.sh + + # bonnyr-f5 #193 r4 B-2 (PRIMARY pin gate): assert BEFORE the push that every + # shipped first-party compose pin already equals v$NEW. sync-version-artifacts.sh + # --write ran above, so the committed pins should be $NEW; if any is stale/forward- + # dated (the B-1 class dist/ pinned 4.0.0 nobody published), fail CLOSED here — + # before the tag/GitHub Release/image push/signing, while it is still recoverable. + # A post-push probe cannot retract those (INV-31). Run from the workspace root + # (full tag checkout — the compose files are present), NOT from the sparse + # .release-tooling. NO --version: the check must read the COMMITTED default. + - name: Verify shipped compose pins equal the release version (pre-push) + run: bash scripts/verify-image-pins.sh --expect-version "${{ needs.preflight.outputs.new_version }}" + + - name: Push commit and tag + run: | + git push origin "${REF_NAME}" + git push origin "v${{ needs.preflight.outputs.new_version }}" + - name: Create GitHub Release run: | gh release create "v${{ needs.preflight.outputs.new_version }}" \ --title "v${{ needs.preflight.outputs.new_version }}" \ --notes-file "${{ steps.notes.outputs.notes_file }}" \ - --target "${{ github.ref_name }}" + --target "${REF_NAME}" env: GH_TOKEN: ${{ github.token }} @@ -481,7 +907,7 @@ jobs: echo "|------|-------|" >> "$GITHUB_STEP_SUMMARY" echo "| Version | ${{ needs.preflight.outputs.current_version }} -> v${{ needs.preflight.outputs.new_version }} |" >> "$GITHUB_STEP_SUMMARY" echo "| Bump | ${{ needs.preflight.outputs.bump_type }} |" >> "$GITHUB_STEP_SUMMARY" - echo "| Branch | ${{ github.ref_name }} |" >> "$GITHUB_STEP_SUMMARY" + echo "| Branch | ${REF_NAME} |" >> "$GITHUB_STEP_SUMMARY" echo "| Commit | ${{ github.sha }} |" >> "$GITHUB_STEP_SUMMARY" # ── Manual Release (workflow_dispatch) ─────────────────────────────────────── @@ -498,6 +924,12 @@ jobs: runs-on: ubuntu-latest permissions: contents: write + packages: read # bonnyr-f5 #193 M4: the pre-push overwrite-guard reads ghcr manifests + env: + # bonnyr-f5 #181: release_notes is free-text; read it from env so a value + # with " or $() can't break out of a run: script (same rule as publish_only). + RELEASE_NOTES: ${{ inputs.release_notes }} + REF_NAME: ${{ github.ref_name }} # env-indirected — see preflight steps: - uses: actions/checkout@v6 with: @@ -509,18 +941,36 @@ jobs: git config user.name "github-actions[bot]" git config user.email "github-actions[bot]@users.noreply.github.com" + - name: Lint the release notes before they become a commit + env: + # Untrusted free text -> env, NEVER interpolated into the shell (same rule + # as RELEASE_NOTES itself). Empty when no notes were given -> a no-op. + LINT_MESSAGE: ${{ inputs.release_notes }} + LINT_MESSAGE_LABEL: "release_notes input" + run: | + # bonnyr-f5 #193 M1: inputs.release_notes is interpolated into the release + # COMMIT body and the TAG message below, but was linted NOWHERE. Lint it + # HERE, before it is minted, through the SAME commit-lint script CI/pre-push + # use: a CI-control marker (which would suppress the workflow run) or a + # BREAKING CHANGE marker the version detectors would MISS (a mis-anchored + # note that ships the release silently as a patch) is caught while it can + # still be fixed — not after it is an unamendable commit on main. RANGE is + # an empty range so ONLY the LINT_MESSAGE text is checked (no commits). + RANGE="HEAD..HEAD" bash scripts/lint-commit-markers.sh + - name: Bump version run: | echo "${{ needs.preflight.outputs.new_version }}" > VERSION if git ls-files --error-unmatch dist/VERSION 2>/dev/null; then echo "${{ needs.preflight.outputs.new_version }}" > dist/VERSION fi + bash scripts/sync-version-artifacts.sh --write "${{ needs.preflight.outputs.new_version }}" - name: Update changelog run: | NEW_VERSION="${{ needs.preflight.outputs.new_version }}" DATE=$(date +%Y-%m-%d) - NOTES="${{ inputs.release_notes }}" + NOTES="${RELEASE_NOTES}" # See the "Update CHANGELOG.md" step in release-final for why this # goes through a temp file + awk instead of an inline sed s-command @@ -542,17 +992,56 @@ jobs: } { print } ' CHANGELOG.md > CHANGELOG.md.new + # Fail CLOSED if the awk insertion was a no-op (missing `^---$` anchor), + # so a release never ships with a silently-dropped changelog entry + # (bonnyr-f5 #193 minor). + if cmp -s CHANGELOG.md CHANGELOG.md.new; then + echo "::error::CHANGELOG.md insertion was a no-op (no '---' anchor found?) — refusing to commit a release with no changelog entry." + rm -f CHANGELOG.md.new "$ENTRY_FILE" + exit 1 + fi mv CHANGELOG.md.new CHANGELOG.md rm -f "$ENTRY_FILE" - name: Commit and tag run: | git add VERSION CHANGELOG.md - git ls-files --error-unmatch dist/VERSION 2>/dev/null && git add dist/VERSION || true + # Stage EXACTLY the artifacts sync-version-artifacts.sh owns, from its + # own --list, so a newly-synced file can never be left unstaged and die + # with the runner (bonnyr-f5 #180 r3, BLOCKER 1: --write rewrote five + # files, the hard-coded `git add` staged three). + # LEAD (bonnyr-f5 #193 r4): derive the anti-vacuity floor from the artifact + # list ITSELF, not a stale literal. --list yields 8 paths today (the dist/+IBM + # pins added to close B-1); the old `-lt 5` left three of them free to vanish + # and still pass. EXPECTED tracks the true set and every listed path must stage + # (staged == EXPECTED), so a dropped path is caught; an empty/near-empty --list + # (script broke/truncated) trips EXPECTED<1 here AND the script's own + # `--check total < 8` vacuity floor. The old comment's "mirrors --check + # total < 5" was doubly stale — that guard is now < 8 and counts matched LINES + # (21), not paths. The per-file "not fully staged" check below still verifies + # each staged path individually. + EXPECTED=$(bash scripts/sync-version-artifacts.sh --list | grep -c . || true) + staged=0 + while IFS= read -r f; do git add "$f"; staged=$((staged + 1)); done < <(bash scripts/sync-version-artifacts.sh --list) + if [ "$EXPECTED" -lt 1 ] || [ "$staged" -ne "$EXPECTED" ]; then + echo "::error::sync-version-artifacts.sh --list: staged $staged of $EXPECTED path(s) — refusing to commit an unsynced release"; exit 1 + fi + # Verify the INDEX, not the files: --write's post-write check re-reads + # the files (correct on disk even when unstaged), so assert each synced + # artifact has no unstaged residue — i.e. the sync is actually in the + # commit we are about to make. + while IFS= read -r f; do + git diff --quiet -- "$f" || { echo "::error::$f was synced but is not fully staged"; exit 1; } + done < <(bash scripts/sync-version-artifacts.sh --list) + # Do not swallow a real `git add` failure with `|| true` (bonnyr-f5 #193 + # minor); the ls-files guard already no-ops when dist/VERSION is untracked. + if git ls-files --error-unmatch dist/VERSION >/dev/null 2>&1; then + git add dist/VERSION + fi git commit -m "release: v${{ needs.preflight.outputs.new_version }} [skip ci] - ${{ inputs.release_notes }} + ${RELEASE_NOTES} Bump: ${{ inputs.version_bump }} Previous: v${{ needs.preflight.outputs.current_version }} @@ -560,19 +1049,35 @@ jobs: E2E: ${{ needs.e2e-gate.result || 'skipped' }}" git tag -a "v${{ needs.preflight.outputs.new_version }}" \ - -m "Release v${{ needs.preflight.outputs.new_version }}: ${{ inputs.release_notes }}" + -m "Release v${{ needs.preflight.outputs.new_version }}: ${RELEASE_NOTES}" + + # bonnyr-f5 #193 M4 / INV-31: refuse a republish of an already-published + # immutable :VERSION tag BEFORE the push (mirrors release-final). + - name: Refuse to overwrite an already-published tag (pre-push) + env: + FORCE: ${{ inputs.force }} + REGISTRY: ghcr.io/${{ github.repository_owner }} + VERSION: ${{ needs.preflight.outputs.new_version }} + REGISTRY_USERNAME: ${{ github.actor }} + REGISTRY_PASSWORD: ${{ secrets.GITHUB_TOKEN }} + run: bash scripts/registry-overwrite-guard.sh + + # bonnyr-f5 #193 r4 B-2 (PRIMARY pin gate — mirrors release-final): every shipped + # first-party compose pin must already equal v$NEW before the irreversible push. + - name: Verify shipped compose pins equal the release version (pre-push) + run: bash scripts/verify-image-pins.sh --expect-version "${{ needs.preflight.outputs.new_version }}" - name: Push run: | - git push origin "${{ github.ref_name }}" + git push origin "${REF_NAME}" git push origin "v${{ needs.preflight.outputs.new_version }}" - name: Create GitHub Release run: | gh release create "v${{ needs.preflight.outputs.new_version }}" \ --title "v${{ needs.preflight.outputs.new_version }}" \ - --notes "${{ inputs.release_notes }}" \ - --target "${{ github.ref_name }}" + --notes "${RELEASE_NOTES}" \ + --target "${REF_NAME}" env: GH_TOKEN: ${{ github.token }} @@ -584,10 +1089,10 @@ jobs: echo "|------|-------|" >> "$GITHUB_STEP_SUMMARY" echo "| Version | ${{ needs.preflight.outputs.current_version }} -> ${{ needs.preflight.outputs.new_version }} |" >> "$GITHUB_STEP_SUMMARY" echo "| Bump | ${{ inputs.version_bump }} |" >> "$GITHUB_STEP_SUMMARY" - echo "| Branch | ${{ github.ref_name }} |" >> "$GITHUB_STEP_SUMMARY" + echo "| Branch | ${REF_NAME} |" >> "$GITHUB_STEP_SUMMARY" echo "| CI | passed |" >> "$GITHUB_STEP_SUMMARY" echo "| E2E | ${{ needs.e2e-gate.result || 'skipped' }} |" >> "$GITHUB_STEP_SUMMARY" - echo "| Notes | ${{ inputs.release_notes }} |" >> "$GITHUB_STEP_SUMMARY" + echo "| Notes | ${RELEASE_NOTES} |" >> "$GITHUB_STEP_SUMMARY" # ── Publish images to ghcr (final releases only) ────────────────────────────── # Two problems, one job: @@ -607,12 +1112,28 @@ jobs: - preflight - release-final - release-manual + - dryrun-release-tooling if: | always() && - (needs.preflight.outputs.release_kind == 'final' || needs.preflight.outputs.release_kind == 'manual') && - (needs.release-final.result == 'success' || needs.release-manual.result == 'success') + needs.dryrun-release-tooling.result == 'success' && + ( + ( + (needs.preflight.outputs.release_kind == 'final' || needs.preflight.outputs.release_kind == 'manual') && + (needs.release-final.result == 'success' || needs.release-manual.result == 'success') + ) + || + (needs.preflight.outputs.release_kind == 'publish_only' && needs.preflight.result == 'success') + ) runs-on: ubuntu-latest timeout-minutes: 90 + # The workflow-level group above is per-ref, but :latest is a single global + # registry resource: a republish from one ref and a final release from + # another could otherwise push :latest concurrently, a TOCTOU that defeats + # the recency guard. Serialize the actual :latest writer on one global queue + # so those pushes can never interleave (bonnyr-f5 #181 round 2). + concurrency: + group: release-publish-latest + cancel-in-progress: false permissions: contents: read packages: write @@ -629,9 +1150,48 @@ jobs: ref: v${{ needs.preflight.outputs.new_version }} fetch-depth: 0 + - name: Fetch the tag-safety probe from the workflow ref + # The step above checks out the RELEASE TAG so bake builds that tree. + # But scripts/registry-tag-probe.sh must be run from the CURRENT + # workflow ref, not the tag: a publish_only recovery of a tag cut BEFORE + # this probe existed would otherwise find no script in the tag's tree and + # the guard would error. github.sha (the ref running this workflow) always + # carries the probe once these changes land, so source it from there into + # a side path, leaving the tag checkout at the workspace root untouched. + uses: actions/checkout@v6 + with: + ref: ${{ github.sha }} + path: .release-tooling + sparse-checkout: | + scripts/registry-tag-probe.sh + scripts/registry-overwrite-guard.sh + scripts/verify-image-pins.sh + docker-bake.hcl + sparse-checkout-cone-mode: false + # bonnyr-f5 #193 M3: docker-bake.hcl is sourced here too so the overwrite + # guard's INDEPENDENT vacuity floor comes from the CURRENT workflow ref + # (stable 7-image "default" group), not from the possibly-ancient release + # tag's tree — a publish_only recovery of an old tag must not depend on + # that tag's bake-file format. + - name: Resolve release commit id: rev - run: echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT" + run: | + # Pinned build inputs, NOT a reproducibility guarantee. The release + # commit's committer date is fixed for a given tag, so baking with it + # (CREATED) instead of the removed timestamp() default, plus + # SOURCE_DATE_EPOCH, removes two obvious sources of build-to-build + # variance. It does NOT make the rebuild byte-identical: the images run + # apt/apk/pip/npm against live indexes and there is no buildkit + # rewrite-timestamp pass, so a rebuild of the same tag can and does + # resolve to a DIFFERENT digest (bonnyr-f5 #181 round 4). That is + # exactly why a republish is refused by default and gated behind + # force= — see the "Refuse to overwrite an already-published tag" step. + { + echo "sha=$(git rev-parse HEAD)" + echo "created=$(git log -1 --format=%cI HEAD)" + echo "epoch=$(git log -1 --format=%ct HEAD)" + } >> "$GITHUB_OUTPUT" - name: Set up QEMU uses: docker/setup-qemu-action@v3 @@ -652,8 +1212,109 @@ jobs: - name: Install syft uses: anchore/sbom-action/download-syft@v0 + - name: Re-check recency inside the publish critical section + if: inputs.sign_only != true + env: + RELEASE_KIND: ${{ needs.preflight.outputs.release_kind }} + run: | + # preflight's recency guard runs OUTSIDE this job's concurrency group, + # so two releases can both pass it and then race to write the single + # global :latest. This job is serialized on release-publish-latest, so + # re-checking the highest final tag HERE — after acquiring the slot, + # immediately before the push — is the authoritative gate: a release + # that is no longer the newest must not repoint :latest backward + # (bonnyr-f5 #181 round 3). rc never reaches this job. + # + # This step OWNS the :latest-move decision: it writes ROLLING_TAG to + # GITHUB_ENV, and the bake step below tags :$ROLLING_TAG (empty ==> + # push only the immutable :VERSION, see docker-bake.hcl). That lets a + # publish_only republish still emit its :VERSION tags when it is behind + # the newest release — it just does NOT move :latest — instead of being + # stranded (bonnyr-f5 #181 round 4). + # The tag probe must fail CLOSED when it is indeterminate. `git fetch + # … || true` swallows a network/permission failure; if the local tag + # set is then empty or stale, the else-branch below would set + # ROLLING_TAG=latest and move the floating tag on an UNVERIFIED guess + # that this release is the newest — the same fail-OPEN class the round-4 + # lesson closed for the registry probe, on the adjacent line (bonnyr-f5 + # #181 round 5, F5). So capture the fetch result and, when it fails, + # never move :latest on a guess: a publish_only republish still owes + # consumers the immutable :VERSION tags (proceed with ROLLING_TAG="", + # which does NOT touch :latest), while a final/manual release that + # cannot prove recency goes RED. + if git fetch --tags --force --quiet origin; then + FETCH_OK=1 + else + FETCH_OK=0 + fi + if [ "$FETCH_OK" != "1" ]; then + if [ "$RELEASE_KIND" = "publish_only" ]; then + echo "::warning::Could not fetch tags to re-check recency; republishing the immutable :$VERSION tags but NOT moving :latest." + echo "ROLLING_TAG=" >> "$GITHUB_ENV" + exit 0 + fi + echo "::error::Could not fetch tags to confirm v$VERSION is still the newest release before repointing :latest. Refusing to move the floating tag on an unverifiable tag set — re-run once origin is reachable." + exit 1 + fi + HIGHEST_TAG="$(git tag -l 'v*' | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -1)" + HIGHEST="$(printf '%s\n%s\n' "${HIGHEST_TAG#v}" "$VERSION" | sort -V | tail -1)" + if [ -n "$HIGHEST_TAG" ] && [ "$VERSION" != "$HIGHEST" ]; then + if [ "$RELEASE_KIND" = "publish_only" ]; then + # Behind the newest tag: re-emit the immutable :VERSION tags but do + # NOT touch the floating :latest (ROLLING_TAG=""). + echo "::notice::v$VERSION is behind the highest tag $HIGHEST_TAG; republishing the immutable :$VERSION tags but NOT moving :latest." + echo "ROLLING_TAG=" >> "$GITHUB_ENV" + else + echo "::error::Refusing to publish v$VERSION: the highest released tag is now $HIGHEST_TAG, so repointing :latest would move it backward." + exit 1 + fi + else + echo "v$VERSION is the highest final tag; safe to repoint :latest." + echo "ROLLING_TAG=latest" >> "$GITHUB_ENV" + fi + + - name: Refuse to overwrite an already-published tag + if: inputs.sign_only != true + env: + FORCE: ${{ inputs.force }} + # Basic creds for the registry's Bearer-token challenge, so the probe + # reads manifests as this authenticated identity (packages: write), + # not anonymously — an anonymous probe of a not-yet-existing package + # returns 401/denied, which fails closed and would strand the first + # release in a namespace. + REGISTRY_USERNAME: ${{ github.actor }} + REGISTRY_PASSWORD: ${{ secrets.GITHUB_TOKEN }} + # Single-source the overwrite policy: run the SAME guard the pre-push + # gates use, but from .release-tooling (the CURRENT workflow ref), so the + # probe AND the bake-file vacuity floor are the stable current versions — + # a publish_only recovery of an OLD tag never depends on that tag's tree. + PROBE: .release-tooling/scripts/registry-tag-probe.sh + BAKE_FILE: .release-tooling/docker-bake.hcl + run: | + # bonnyr-f5 #193 M2 + minor (single-source): this used to open-code ~85 + # lines that DUPLICATED scripts/registry-overwrite-guard.sh — a third + # divergent copy of the policy whose vacuity floor counted EVERY + # `targets = [` line in docker-bake.hcl, so adding any second bake group + # made this gate refuse forever and blame the registry (M2). It now calls + # the ONE guard, whose floor is derived from the tool scoped to the bake + # DEFAULT group (`docker buildx bake --print default | jq …`) and which + # keeps "bake-file parse mismatch" distinct from "registry unreachable". + # docker-bake pushes ${REGISTRY}/:${VERSION} to an IMMUTABLE tag, so + # a republish that moves it (and orphans the cosign/SBOM/SLSA attestations + # bound to the old digest) is refused unless force=true (INV-24 / F1/F3). + bash .release-tooling/scripts/registry-overwrite-guard.sh + - name: Build and push all images (docker-bake.hcl) - run: docker buildx bake --push default + # Skipped in sign_only recovery: the images are already pushed, and the + # signing step below re-signs those exact digests. Rebuilding would + # produce possibly-different digests and move the immutable tag — the + # INV-24 harm sign_only exists to avoid (bonnyr-f5 #181 round 5, F2). + if: inputs.sign_only != true + # --metadata-file records exactly which targets buildx built and pushed + # (bonnyr-f5 #193 r3 minor): the Publish summary below counts THAT, not the + # static bake target list, so a tag whose tree builds 6 images is not + # reported as 7. + run: docker buildx bake --push --metadata-file "$RUNNER_TEMP/bake-metadata.json" default env: REGISTRY: ${{ env.REGISTRY }} VERSION: ${{ env.VERSION }} @@ -661,7 +1322,17 @@ jobs: # OCI source label tracks whichever remote actually built the image. SOURCE_URL: ${{ github.server_url }}/${{ github.repository }} GIT_REVISION: ${{ steps.rev.outputs.sha }} - ROLLING_TAG: latest + # Pinned build inputs: fixed created label + SOURCE_DATE_EPOCH. These + # reduce build-to-build variance but do NOT guarantee an identical + # digest (see docker-bake.hcl and "Resolve release commit") — which is + # why the immutable-tag probe above refuses a republish by default. + CREATED: ${{ steps.rev.outputs.created }} + SOURCE_DATE_EPOCH: ${{ steps.rev.outputs.epoch }} + # ROLLING_TAG is NOT hardcoded here: the "Re-check recency" step wrote + # it to GITHUB_ENV — "latest" to move the floating tag, or "" for a + # publish_only republish that is behind the newest release (push only + # the immutable :VERSION tags). A step-level env: entry would override + # that GITHUB_ENV value, so it is intentionally omitted. - name: Sign, SBOM, and attest all images (keyless cosign) run: bash scripts/publish-signed-images.sh --execute @@ -669,12 +1340,73 @@ jobs: BNK_FORGE_REGISTRY: ${{ env.REGISTRY }} BNK_FORGE_VERSION: ${{ env.VERSION }} + - name: Confirm every shipped compose pin resolves in the registry (post-push) + # bonnyr-f5 #193 r4 B-2: this is the SECONDARY existence confirmation; the + # PRIMARY guard is the pre-push consistency gate in release-final/-manual + # (a post-push red step cannot retract a pushed tag/release/signature — INV-31). + # Three defects fixed here from r3: + # 1. FILE SET — the script derives ROOT from its own location, i.e. + # .release-tooling, a SPARSE 4-file checkout with NO compose file, so the + # default set resolved to nothing and the step went red "no pins" on every + # run. The compose files live at the TAG checkout (workspace root), so pass + # them EXPLICITLY by --file (dist compose, dist overlay, IBM installer). + # 2. FLAGS not ENV — the script reads --registry/--version FLAGS, never env + # vars; the r3 step passed REGISTRY/VERSION as env and would have validated + # the committed defaults, not the tag being published. + # 3. Sourced from .release-tooling (CURRENT workflow ref) like the tag-safety + # probe, so a publish_only recovery of an older tag still runs the CURRENT + # verifier. Now that the images were pushed above, every :$VERSION manifest + # must resolve. + run: | + bash .release-tooling/scripts/verify-image-pins.sh \ + --registry "$REGISTRY" --version "$VERSION" \ + --file dist/docker-compose.yml \ + --file dist/docker-compose.local.yml \ + --file scripts/ibm_cloud_bnk_forge.sh + - name: Publish summary + env: + SIGN_ONLY: ${{ inputs.sign_only }} run: | + # ROLLING_TAG comes from GITHUB_ENV (set by "Re-check recency"): empty + # for a publish_only republish behind the newest release, which pushed + # only the immutable :VERSION tags and did NOT move :latest. Report the + # tags actually pushed rather than always claiming :latest. In + # sign_only recovery nothing was pushed at all — the existing digests + # were only (re-)signed — so say exactly that. + if [ "$SIGN_ONLY" = "true" ]; then + TAGS=":${{ needs.preflight.outputs.new_version }} (re-signed existing digests — no image pushed, :latest NOT moved)" + elif [ -n "${ROLLING_TAG:-}" ]; then + TAGS=":${{ needs.preflight.outputs.new_version }}, :${ROLLING_TAG}" + else + TAGS=":${{ needs.preflight.outputs.new_version }} (floating :latest NOT moved — this republish is behind the newest release)" + fi echo "## Published images v${{ needs.preflight.outputs.new_version }}" >> "$GITHUB_STEP_SUMMARY" echo "" >> "$GITHUB_STEP_SUMMARY" echo "| Image | Tags |" >> "$GITHUB_STEP_SUMMARY" echo "|-------|------|" >> "$GITHUB_STEP_SUMMARY" - for name in bnk-forge-api bnk-forge-worker bnk-forge-beat bnk-forge-frontend bnk-forge-proxy bnk-forge-mcp bnk-forge-operator; do - echo "| ${name} | ${{ env.REGISTRY }}/${name}:${{ needs.preflight.outputs.new_version }}, :latest |" >> "$GITHUB_STEP_SUMMARY" - done + META="$RUNNER_TEMP/bake-metadata.json" + if [ "$SIGN_ONLY" = "true" ] || [ ! -f "$META" ]; then + # sign_only pushed nothing (only re-signed existing digests): enumerate + # the canonical image list from the probe (F6) as the set that was + # re-signed. No metadata file exists because the bake step was skipped. + n=0 + while IFS= read -r name; do + n=$((n + 1)) + echo "| ${name} | ${{ env.REGISTRY }}/${name}${TAGS} |" >> "$GITHUB_STEP_SUMMARY" + done < <(bash .release-tooling/scripts/registry-tag-probe.sh --images) + echo "" >> "$GITHUB_STEP_SUMMARY" + echo "_${n} image(s) re-signed (no push)._" >> "$GITHUB_STEP_SUMMARY" + else + # Report what buildx ACTUALLY built + pushed, read from the bake + # --metadata-file (bonnyr-f5 #193 r3 minor). Each pushed target carries + # an "image.name" (its comma-joined push refs); a top-level + # buildx.build.* key is NOT a target, so select on image.name to exclude + # it. This is the real pushed count — 6 or 7 — never the static list. + pushed_n="$(jq -r '[to_entries[] | select(.value["image.name"])] | length' "$META")" + jq -r 'to_entries[] + | select(.value["image.name"]) + | "| \(.key) | \(.value["image.name"]) |"' "$META" >> "$GITHUB_STEP_SUMMARY" + echo "" >> "$GITHUB_STEP_SUMMARY" + echo "_${pushed_n} image(s) pushed (from docker buildx bake metadata)._" >> "$GITHUB_STEP_SUMMARY" + fi diff --git a/.github/workflows/secret-baseline.yml b/.github/workflows/secret-baseline.yml new file mode 100644 index 0000000..bdb01fc --- /dev/null +++ b/.github/workflows/secret-baseline.yml @@ -0,0 +1,39 @@ +# ╔══════════════════════════════════════════════════════════════════════════╗ +# ║ Secret Scan — Full-History Baseline ║ +# ║ ║ +# ║ bonnyr-f5 #182 r3 (Major): the per-PR/push gate in ci.yml scans only the ║ +# ║ COMMIT RANGE of each change. Anything already in history before that gate ║ +# ║ landed — or a secret that slips in via a path the range scan misses — is ║ +# ║ never re-examined. This workflow runs gitleaks over ALL reachable history ║ +# ║ on a weekly schedule and on demand, so the whole repo stays monitored. ║ +# ║ ║ +# ║ It reuses scripts/secret-scan.sh (the SAME scan + assertion backstop as ║ +# ║ CI and `make secret-scan`), with RANGE="" meaning "full history". The ║ +# ║ gate still FAILS on a git error or a 0-commit non-scan, so a broken ║ +# ║ baseline is caught, not silently green. ║ +# ╚══════════════════════════════════════════════════════════════════════════╝ + +name: Secret Scan Baseline + +on: + schedule: + # Mondays 06:17 UTC — weekly full-history sweep (off the hour to avoid the + # scheduler's top-of-hour congestion). + - cron: "17 6 * * 1" + workflow_dispatch: {} + +permissions: + contents: read + +jobs: + baseline: + name: "Full-history gitleaks baseline" + runs-on: ubuntu-latest + env: + RANGE: "" # explicit empty => scan all reachable history + steps: + - uses: actions/checkout@v6 + with: + fetch-depth: 0 # the whole history is the point + - name: gitleaks (full history) + run: make secret-scan diff --git a/.gitignore b/.gitignore index 4959da4..8145aa8 100644 --- a/.gitignore +++ b/.gitignore @@ -278,3 +278,9 @@ next-session-prompt agent-selection handoffs/ *.code-workspace + +# Transient sed backup files from scripts/sync-version-artifacts.sh --write +# (removed on success; gitignored so an interrupted run leaves no tracked litter) +*.syncbak +# Generated secret material (JWT/encryption keys, initial admin password) +backend/keys/ diff --git a/.gitleaks.toml b/.gitleaks.toml index d6b977b..bbe27e2 100644 --- a/.gitleaks.toml +++ b/.gitleaks.toml @@ -34,15 +34,16 @@ regexes = [ # "signature_here"; used to test _looks_like_jwt(). '''eyJhbGciOiJIUzI1NiJ9\.eyJzdWIiOiJ0ZXN0In0\.signature_here''', ] + +[[rules]] +id = "private-key" +# bonnyr-f5 #182: scope the fixture private keys to the private-key rule only, so +# a real AWS/GitHub/generic secret hidden in these same files is still caught -- +# a top-level [allowlist].paths would have disabled EVERY rule for them. The +# .pyc/__pycache__ blanket entries were dropped (zero tracked files). +[rules.allowlist] paths = [ - # Throwaway RSA/Ed25519 keypairs generated solely to exercise paramiko key - # parsing. They authenticate nothing — no corresponding public key is - # deployed anywhere. See the note in the file header. '''backend/tests/unit/test_paramiko_utils\.py''', - # PEM headers wrapped around placeholder bodies, not key material: - # test_agent_host_candidates.py -> "MIIEowIBAAKCAQEA000000..." - # test_routes_project_secrets.py -> "fake" - # test_infrastructure_access_service.py-> "MIIEowIBAAKCAQEAuTestKeyMaterial" '''backend/tests/component/test_agent_host_candidates\.py''', '''backend/tests/integration/test_routes_project_secrets\.py''', '''backend/tests/unit/test_infrastructure_access_service\.py''', diff --git a/.trivyignore b/.trivyignore index 46fa12f..7c23206 100644 --- a/.trivyignore +++ b/.trivyignore @@ -4,26 +4,28 @@ # projects rebuild with a patched Go version. # # Review this file periodically and remove entries when upstream fixes are available. +# Each entry carries an `exp:` review-by date — Trivy drops the suppression after it, +# forcing a re-check. Extend an entry only after re-confirming no upstream fix exists. # CVE-2025-68121: Go stdlib crypto/tls - Unexpected session resumption # Fixed in Go >= 1.24.13 / 1.25.7 / 1.26.0-rc.3 # Affects: helm (Go 1.25.0), kubectl, tofu (Go 1.25.6), infracost (Go 1.25.4) # All current latest releases use Go < 1.25.7 — no upstream fix available yet # Added: 2026-02-23 -CVE-2025-68121 +CVE-2025-68121 exp:2026-11-30 # CVE-2024-45337: golang.org/x/crypto/ssh - Misuse of ServerConfig.PublicKeyCallback # Present in infracost binary's bundled dependencies # Not exploitable in our context (we don't run an SSH server via infracost) # Added: 2026-02-23 -CVE-2024-45337 +CVE-2024-45337 exp:2026-11-30 # CVE-2026-33186: gRPC authorization bypass (google.golang.org/grpc < 1.79.3) # Affects: helm and tofu binaries in Docker image (grpc v1.76.0) # Status: Waiting for upstream helm/tofu releases with fixed grpc # Tracked: GitHub issue #50 # Added: 2026-04-15 -CVE-2026-33186 +CVE-2026-33186 exp:2026-11-30 # CVE-2026-7598: libssh2 — integer overflow via large username/password # Affects: libssh2-1t64 1.11.1-1 in Debian trixie base image @@ -32,11 +34,21 @@ CVE-2026-33186 # The vulnerable code path requires libssh2 to negotiate auth with a # malicious remote SSH server, which our HTTP backend never does. # Upstream: no Debian backport yet (Trivy reports empty fix column). -# REVISIT: monthly via https://security-tracker.debian.org/tracker/CVE-2026-7598 -# escalate to pin-from-sid if no fix by 2026-08-12 +# REVISIT: monthly. Do not extend this entry on the assertion that no fix exists — +# confirm it: re-run `trivy image` (or check the tracker) and only keep +# the ignore while the fix column is still empty for our base image's +# libssh2. https://security-tracker.debian.org/tracker/CVE-2026-7598 +# REFRESH (2026-08-21, bonnyr-f5 #193 minor): the exp was 2026-09-12, three weeks +# out — close enough that it risked lapsing between releases and reopening +# the finding mid-cycle. Deadline moved to 2026-09-30 to restore a full +# monthly window. This refresh moves the DATE only; it does NOT re-assert +# "no fix exists" — the next owner MUST still run `trivy image` / check the +# tracker before the new deadline and drop this entry the moment libssh2's +# fix column is non-empty for our base image (escalate to pin-from-sid if a +# fix is then available and we are still ignoring it). # Tracked: memory/followup_trivyignore_cve_2026_7598_revisit.md # Added: 2026-05-12 -CVE-2026-7598 +CVE-2026-7598 exp:2026-09-30 # CVE-2026-42010: GnuTLS Authentication Bypass via NUL Character in DN parsing # Affects: libgnutls30t64 in our Debian Trixie base image (3.8.9-3+deb13u2) @@ -49,7 +61,7 @@ CVE-2026-7598 # REVISIT: monthly via https://security-tracker.debian.org/tracker/CVE-2026-42010 # Pattern mirror of CVE-2026-33845 / CVE-2026-7598 suppressions. # Added: 2026-05-14 -CVE-2026-42010 +CVE-2026-42010 exp:2026-11-30 # CVE-2026-42496: perl — Archive::Tar < 3.08 extracts symlinks unsafely # CVE-2026-8376: perl — heap buffer overflow in the interpreter (<= 5.43.10) @@ -69,8 +81,8 @@ CVE-2026-42010 # https://security-tracker.debian.org/tracker/CVE-2026-8376 # Drop once Debian ships a trixie point-release with patched perl. # Added: 2026-06-02 -CVE-2026-42496 -CVE-2026-8376 +CVE-2026-42496 exp:2026-11-30 +CVE-2026-8376 exp:2026-11-30 # CVE-2026-13221: libperl5.40 — silently incorrect results in Perl <= 5.43.9 # Affects: libperl5.40 5.40.1-6 in the python:3.11-slim (Debian trixie) base image @@ -80,7 +92,7 @@ CVE-2026-8376 # REVISIT: monthly via https://security-tracker.debian.org/tracker/CVE-2026-13221 # drop once Debian ships a trixie update with a patched libperl5.40. # Added: 2026-07-15 -CVE-2026-13221 +CVE-2026-13221 exp:2026-11-30 # CVE-2026-60002: openssh-client — memory corruption in SSH client # Affects: openssh-client in the python:3.11-slim (Debian trixie) base image @@ -91,7 +103,7 @@ CVE-2026-13221 # REVISIT: monthly via https://security-tracker.debian.org/tracker/CVE-2026-60002 # drop once Debian ships a trixie update with a patched openssh-client. # Added: 2026-07-15 -CVE-2026-60002 +CVE-2026-60002 exp:2026-11-30 # CVE-2026-33845: GnuTLS DTLS — reachable-assert / auth bypass in DN parsing # Affects: libgnutls30t64 in our Debian Trixie base image (3.8.9-3+deb13u2) @@ -105,7 +117,7 @@ CVE-2026-60002 # Check: https://security-tracker.debian.org/tracker/CVE-2026-33845 # Tracked: GitHub issue #103 # Added: 2026-05-06 -CVE-2026-33845 +CVE-2026-33845 exp:2026-11-30 # CVE-2026-57433: perl Storable signed-integer flaw (Storable < 3.41) # Affects: libperl5.40, perl-base (5.40.1-6) in our Debian Trixie base image. @@ -117,4 +129,4 @@ CVE-2026-33845 # trixie-security. Check: https://security-tracker.debian.org/tracker/CVE-2026-57433 # Tracked: GitHub issue #492 # Added: 2026-07-22 -CVE-2026-57433 +CVE-2026-57433 exp:2026-11-30 diff --git a/AGENTS.md b/AGENTS.md index c2289cd..972aa5b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -82,5 +82,28 @@ Strong success criteria let you loop independently. Weak criteria ("make it work **These guidelines are working if:** fewer unnecessary changes in diffs, fewer rewrites due to overcomplication, and clarifying questions come before implementation rather than after mistakes. +## Commit conventions + +Conventional Commits (`type: subject`, optional body, `BREAKING CHANGE:` footer for a +major). One repo-specific trap worth stating outright: + +- **Never write a CI-control marker as literal text anywhere in a commit message — + subject *or* body — even when quoting it in prose.** GitHub scans the whole message, + so a `[skip ci]` / `[ci skip]` sitting in a sentence suppresses the run for that + commit. This has bitten us twice, most recently on a shell-script change where the + gates that got skipped (ShellCheck, Script Self-Tests, Secret Scan) were exactly the + ones that mattered. Refer to it indirectly instead: "CI suppressed", "the skip-CI + marker", or split it across backticks. The release job's *deliberate* skip is the + only legitimate use, and it lands on the subject line where the release loop reads it. + This is now **enforced**, not just documented: the `commit-lint` CI gate and the + `.githooks/pre-push` hook both run `scripts/lint-commit-markers.sh`, which fails a + push/PR whose commit range carries any CI-control marker (bonnyr-f5 #182 r3, #166: + documentation is not enforcement). +- **Declare a major bump with a real `BREAKING CHANGE: ` footer**, not a + bold `**BREAKING CHANGE**` heading or a bare colon-less line. `compute_version_bump.sh` + majors on the phrase, so a prose line that *looks* like a footer ships a spurious major + release; `commit-lint` rejects the line-start prose forms while allowing the plain + footer. + --- diff --git a/CHANGELOG.md b/CHANGELOG.md index 294cdf6..03118a8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,9 +1,54 @@ # Changelog -All notable changes to BNK-Forge v2. +All notable changes to BNK-Forge. --- +## v3.1.6 (2026-08-10) — 3.1.x line + +Milestone `v3.1.6` — the last release before 4.0.0, and the initial public +release tag on `f5devcentral`. This mirror is squashed: the `v3.1.6` tag is a +single `feat: initial public release` commit, so there is no per-change history +behind it to link here. Work that came *after* this tag — including the +container-runner hardening series (#2, #123, #161) and the ADR-424 bare-metal/DPU +work — is part of 4.0.0, not v3.1.6, and is recorded under the 4.0.0 entry when +that release is cut. + +> **Heads-up for the 4.0.0 upgrade — two breaking changes:** +> +> 1. **Container runner non-root gate:** it now refuses *named* users — an image +> using the distroless-standard `USER nonroot` is rejected. Switch it to a +> numeric uid. Use **`USER 1000`**: the workspace is mounted from the host +> and chowned `1000:1000`, so uid 1000 is the only value that clears the gate +> *and* can write it. A higher uid such as `65532` passes the non-root gate but +> cannot write the workspace, so the step fails on its first write. +> 2. **`MCP_SERVICE_PASSWORD` becomes required in 4.0.0 (via bonnyr-f5 #188):** +> starting with 4.0.0 the backend refuses to boot in staging/production if it +> is unset or still a shipped default (`changeme` / `mcp-service-changeme`). +> That boot-time check ships in #188 — it is *not* in the 3.1.x line and is +> called out here only so the upgrade step is ready before #188 lands. Every +> existing install still carries one of those defaults, so before upgrading to +> 4.0.0 **set `MCP_SERVICE_PASSWORD` to a real secret** (the same value the MCP +> server receives as `BNK_FORGE_PASSWORD`); once #188 is in the tree, leaving +> it at a default will `SystemExit` the stack at startup. +> +> **Merge ordering (integration dependency).** The dist-bundle wiring these two +> steps assume — the dedicated `mcp` service account for the bundled MCP server, +> and the `MCP_SERVICE_PASSWORD` boot check — arrives in **bonnyr-f5 #186** (service +> account + removal of the shipped `changeme` / `mcp-service-changeme` defaults) and +> **#188** (boot check). This release documents them forward-looking and is therefore +> sequenced to merge **with or after #186 + #188**. Merged ahead of them, the +> `MCP_SERVICE_PASSWORD` guidance is inert for the dist stack (the compose file does +> not pass that variable to the backend) and #186 will conflict in +> `user-pack/install-guide.html` — resolve by taking #186's credential model, not by +> re-adding the `changeme` default this guide describes as a stopgap. + +## v3.0.1 — 3.0.x line + +The first 3.x release after the 2.x line below (upstream tag dated 2026-04-09). +Bridged entry; this repo is a squashed public mirror, so the `v3.0.1` tag and its +per-change history live upstream, not here. + ## v2.10.74 (2026-03-04) — TMM Debug Panel Enhancements: F5 Docs Commands, Netkvest, Bug Fix ### Bug Fixes diff --git a/Makefile b/Makefile index 75528fe..1720743 100644 --- a/Makefile +++ b/Makefile @@ -84,13 +84,14 @@ AWSBNKCTL_STAMP := bin/.awsbnkctl-$(AWSBNKCTL_VERSION).stamp test test-backend test-backend-unit test-backend-component test-backend-legacy test-frontend \ test-proxy test-operator test-db test-contracts test-e2e test-e2e-tier1 test-e2e-tier2 \ test-integration test-integration-full build-frontend-check smoke-mcp-live mcp-readiness mcp-recreate \ - lint lint-backend lint-frontend shellcheck coverage quick-check pre-push push install-hooks setup-hooks \ + lint lint-backend lint-frontend shellcheck coverage quick-check version-check pre-push push install-hooks setup-hooks \ dev-setup security-audit docker-check docker-verify docker-validate \ openapi openapi-types openapi-check openapi-types-check typecheck-backend typecheck-frontend \ build build-retry build-backend build-frontend build-worker build-agent build-all \ fetch-awsbnkctl \ up down restart deploy deploy-backend deploy-frontend upgrade-safe \ clean clean-docker check-disk setup-cleanup-cron check-migrations \ + commit-lint script-selftests ci-gates secret-scan artifact-network-selftest \ test-upgrade dist push-images push-customer-build buildx-setup publish-signed help # ─── Quick Start Commands ──────────────────────────────────────────────────── @@ -206,7 +207,7 @@ _install-info: echo " (accept the self-signed certificate warning)"; \ fi; \ echo ""; \ - echo " Login: admin (initial password: DEFAULT_ADMIN_PASSWORD, default 'changeme' — change on first login)"; \ + echo " Login: admin (password: DEFAULT_ADMIN_PASSWORD if set, else the generated one at /app/keys/initial_admin_password — change on first login)"; \ echo ""; \ echo " Next steps:"; \ echo " 1. Change your password on first login"; \ @@ -426,7 +427,7 @@ deploy: build ensure-artifact-network @echo "" ifeq ($(UNAME_S),Darwin) @echo " Open: https://localhost" - @echo " Login: admin (initial password: DEFAULT_ADMIN_PASSWORD, default 'changeme'; change on first login)" + @echo " Login: admin (password: DEFAULT_ADMIN_PASSWORD if set, else the generated one at /app/keys/initial_admin_password; change on first login)" endif @echo " Recommended next step: make mcp-readiness" @echo "=========================================" @@ -464,7 +465,138 @@ test-upgrade: shellcheck: @echo "" @echo "=== ShellCheck: linting shell scripts ===" - @shellcheck --severity=warning upgrade.sh scripts/*.sh vm-bnk-forge/*.sh vm-bnk-forge/lib/*.sh + @# bonnyr-f5 #182: drive from git ls-files so the WHOLE corpus is gated + @# (the hardcoded globs missed 14 tracked scripts incl. dist/install.sh). + @# bonnyr-f5 #182 r2: include the (extensionless) git hooks, and fail on an + @# EMPTY list -- `xargs shellcheck` with no files exits 0 on BSD (blind). + @files="$$(git ls-files '*.sh' .githooks/pre-commit .githooks/pre-push 2>/dev/null)"; \ + n=$$(printf '%s\n' "$$files" | grep -c .); \ + [ "$$n" -ge 1 ] || { echo "::error::shellcheck found no files to lint"; exit 1; }; \ + printf '%s\n' "$$files" | xargs shellcheck --severity=warning + +# ── CI-parity gates (bonnyr-f5 #182 r3, Major) ────────────────────────────── +# The four gates ci.yml added were not runnable locally: `make pre-push` ran +# none of them and `make shellcheck` had no dependents, yet ci.yml's header +# claims `make pre-push` == CI. #166: "a local gate that does not run the CI +# command is not a gate." These targets ARE the CI command (ci.yml calls the +# same `make` targets / same scripts), and `pre-push` now depends on `ci-gates`. +.PHONY: ci-gates secret-scan commit-lint script-selftests + +# NOTE: `version-check` is defined once, in the "Version-artifact consistency" +# section below (near quick-check, which depends on it). A duplicate recipe used +# to sit here; `make` silently discarded one and warned "overriding commands for +# target" on every invocation, so any future divergence between the two copies +# would have been invisible (bonnyr-f5 #193 minor). ci-gates references the single +# surviving target by name. + +# gitleaks range-aware secret scan + assertion backstop (single source of truth, +# shared with ci.yml's secret-scan job and the scheduled baseline workflow). +# Honours RANGE from the environment; unset => scan since the upstream merge-base. +secret-scan: + @echo "" + @echo "=== Secret scan (gitleaks) ===" + @bash scripts/secret-scan.sh + +# Commit-message marker enforcement (shared with ci.yml's commit-lint job and +# .githooks/pre-push). Honours RANGE; unset => @{upstream}..HEAD. +commit-lint: + @echo "" + @echo "=== Commit message marker lint ===" + @bash scripts/lint-commit-markers.sh + +# The paired self-test harnesses ci.yml's script-selftests job runs. +# ci.yml's compute step has FOUR anti-vacuity assertions and this target must +# mirror ALL of them, or a broken harness passes locally while CI goes red +# (bonnyr-f5 #182 r4/r5, Major-3: a local gate that diverges from the CI command +# is not a gate). The four (in ci.yml order): +# 1. non-zero exit -> the harness itself errored +# 2. a "FAIL:" line (rc still 0) -> an assertion failed but exit was swallowed +# 3. NO "PASS:" line -> the guard was silenced / renamed: green with +# zero assertions actually run +# 4. NO "=== END SELF-TEST ===" -> the harness exited early (deleted END marker +# or an early `exit 0`) with assertions unrun +# r5 landed 3+4 here; r4 had only 1+2, so the "silenced guard" and "early exit" +# harness-break modes were CI-red but `make`-GREEN. +script-selftests: + @echo "" + @echo "=== Script self-tests ===" + @set +e; out="$$(SELF_TEST=1 bash scripts/compute_version_bump.sh 2>&1)"; rc=$$?; \ + echo "$$out"; \ + if [ "$$rc" -ne 0 ]; then echo "::error::compute_version_bump self-test exited $$rc"; exit "$$rc"; fi; \ + if printf '%s\n' "$$out" | grep -qE '(^|[[:space:]])FAIL:'; then \ + echo "::error::compute_version_bump self-test reported FAIL: but exited 0"; exit 1; \ + fi; \ + if ! printf '%s\n' "$$out" | grep -qE '(^|[[:space:]])PASS:'; then \ + echo "::error::self-test produced no PASS lines -- the harness did not run"; exit 1; \ + fi; \ + if ! printf '%s\n' "$$out" | grep -qE '=== END SELF-TEST ==='; then \ + echo "::error::self-test did not reach its END marker -- it exited early with assertions unrun"; exit 1; \ + fi + @# extract-breaking-changes.sh self-test, run UNCONDITIONALLY with the same + @# anti-vacuity assertions as compute's (ok lines + END marker). Do NOT gate on + @# `grep -- '--self-test' `: the code under test must not decide + @# whether it is tested — deleting the flag would silence ~28 assertions with + @# the gate staying green (bonnyr-f5 #193 M6). + @set +e; out="$$(bash scripts/extract-breaking-changes.sh --self-test 2>&1)"; rc=$$?; \ + echo "$$out"; \ + if [ "$$rc" -ne 0 ]; then echo "::error::extract-breaking-changes self-test exited $$rc"; exit "$$rc"; fi; \ + if printf '%s\n' "$$out" | grep -qE '(^|[[:space:]])FAIL:'; then \ + echo "::error::extract self-test reported FAIL: but exited 0"; exit 1; \ + fi; \ + if ! printf '%s\n' "$$out" | grep -qE '(^|[[:space:]])ok:'; then \ + echo "::error::extract self-test produced no ok: lines -- the harness did not run"; exit 1; \ + fi; \ + if ! printf '%s\n' "$$out" | grep -qE '=== END SELF-TEST ==='; then \ + echo "::error::extract self-test did not reach its END marker -- it exited early with assertions unrun"; exit 1; \ + fi + @# B5 + M5 + M4: enumerate and run EVERY scripts/tests/*.test.sh from the + @# filesystem, applying the SAME anti-vacuity discipline as the two inline + @# self-tests above (bonnyr-f5 #193 r4 M-4: the old loop checked ONLY a non-empty + @# enumeration and each file's exit 0, so a test gutted to `exit 0` passed and + @# deleting 7 of 8 files left n=1 and stayed green). Each file must now: + @# 1. EXIT 0; + @# 2. emit at least one PASS line (the harness actually ran assertions); + @# 3. emit NO `FAIL` line (a failure whose exit was swallowed); + @# 4. reach its `ALL PASS` terminal (a gutted/early-exiting file never prints it). + @# PLUS a count floor DERIVED from the tests present, cross-checked against git's + @# tracked set: a test file deleted in the working tree makes the on-disk count fall + @# below the tracked count and is caught, instead of silently lowering a literal + @# floor. ci.yml's script-selftests job runs the SAME enumeration so local == CI. + @set -e; \ + tests="$$(ls scripts/tests/*.test.sh 2>/dev/null || true)"; \ + n=$$(printf '%s\n' "$$tests" | grep -c . || true); \ + tracked=$$(git ls-files 'scripts/tests/*.test.sh' 2>/dev/null | grep -c . || true); \ + if [ "$$n" -lt 1 ]; then echo "::error::no scripts/tests/*.test.sh found -- the self-test enumeration is empty"; exit 1; fi; \ + if [ "$$tracked" -gt 0 ] && [ "$$n" -lt "$$tracked" ]; then \ + echo "::error::self-test enumeration found $$n file(s) on disk but git tracks $$tracked -- a *.test.sh was removed from the working tree; refusing to run a shrunken suite"; exit 1; \ + fi; \ + echo " running $$n filesystem self-test(s) (git tracks $$tracked):"; \ + for t in $$tests; do \ + echo "--- $$t ---"; \ + set +e; out="$$(bash "$$t" 2>&1)"; rc=$$?; set -e; \ + printf '%s\n' "$$out"; \ + if [ "$$rc" -ne 0 ]; then echo "::error::$$t exited $$rc"; exit 1; fi; \ + if printf '%s\n' "$$out" | grep -qE '^FAIL'; then echo "::error::$$t printed a FAIL line but exited 0 -- a swallowed assertion failure"; exit 1; fi; \ + if ! printf '%s\n' "$$out" | grep -qE '^PASS'; then echo "::error::$$t produced no PASS line -- the harness did not run (gutted to a no-op?)"; exit 1; fi; \ + if ! printf '%s\n' "$$out" | grep -q 'ALL PASS'; then echo "::error::$$t did not reach its 'ALL PASS' terminal marker -- it exited early with assertions unrun"; exit 1; fi; \ + done + +# Mirror of ci.yml's "P1 · Artifact Network Self-Test" job (bonnyr-f5 #193 minor: +# `make pre-push` ≡ CI was false — this job had no local target). Pure-logic, no +# Docker/network. +artifact-network-selftest: + @echo "" + @echo "=== Artifact network self-test (mirror of ci.yml artifact-network-self-test) ===" + @bash scripts/artifact_network.sh --self-test + +# Aggregate: every CI gate that is not already covered by quick-check/tests. +# bonnyr-f5 #193 minor (`make pre-push` ≡ CI): artifact-network-selftest is wired +# in here, and ci.yml's "migration-collision-check" job runs `make check-migrations` +# — already pulled in by quick-check (a pre-push prerequisite) — so both formerly +# unmirrored CI jobs are now reachable from `make pre-push`. +ci-gates: shellcheck version-check commit-lint script-selftests secret-scan artifact-network-selftest + @echo "" + @echo "=== CI-parity gates passed ===" # Convenience: start/stop/restart all (platform-aware) up: ensure-artifact-network @@ -524,7 +656,7 @@ smoke-mcp-live: @echo "" @echo "=== MCP Live Smoke Validation ===" @echo " NOTE: ping/tools-list reachability != runtime readiness; tool calls require valid MCP backend credentials." - @echo " Configure MCP_USERNAME/MCP_PASSWORD if backend admin password was rotated." + @echo " Configure MCP_SERVICE_PASSWORD (compose maps it to the container's BNK_FORGE_PASSWORD and the backend's MCP_SERVICE_PASSWORD) — the dedicated MCP service account, never the admin login (#187). MCP_USERNAME is not read; do not set it." @python3 scripts/mcp_live_smoke.py --mcp-url "$${MCP_SMOKE_URL:-http://localhost:8081/mcp}" $${MCP_SMOKE_INSECURE_TLS:+--insecure-tls} mcp-readiness: @@ -548,7 +680,7 @@ mcp-readiness: mcp-recreate: @echo "" @echo "=== Recreate MCP service ===" - @echo " Use after MCP_USERNAME/MCP_PASSWORD changes so MCP picks up new credentials." + @echo " Use after MCP_SERVICE_PASSWORD changes so MCP picks up new credentials." @$(COMPOSE) up -d --force-recreate --no-deps mcp @$(COMPOSE) ps mcp @@ -684,9 +816,19 @@ check-migrations: @echo "=== Migration Chain Validator ===" @python3 scripts/check-migrations.py +# ── Version-artifact consistency ───────────────────────────────────────────── +# Mirror of CI's "P1 · Version Consistency" job. ci.yml promises `make pre-push` +# ≡ CI, so the gate must be reachable from the documented local target or drift +# is undetectable until the release job dies (bonnyr-f5 #180 r5, F3). Pulled in +# by quick-check (a pre-push prerequisite). +version-check: + @echo "" + @echo "=== Version Artifact Consistency (Helm tag/appVersion, frontend, operator) ===" + @bash scripts/sync-version-artifacts.sh --check + # ── Quick check (~15s): lint + types + contracts ──────────────────────────── # Run before every commit. Catches most CI failures instantly. -quick-check: lint typecheck-backend openapi-types-check check-migrations +quick-check: lint typecheck-backend openapi-types-check check-migrations version-check @echo "" @echo "=========================================" @echo " Quick check passed (~15s)" @@ -694,8 +836,10 @@ quick-check: lint typecheck-backend openapi-types-check check-migrations # ── Pre-push (~90s parallel): mirrors ALL CI jobs ─────────────────────────── # Run once before git push. Runs test suites in parallel for speed. -# Prerequisite: quick-check runs first (sequential), then tests fan out. -pre-push: quick-check +# Prerequisite: quick-check runs first (sequential), then the CI-parity gates +# (shellcheck / version-check / commit-lint / script-selftests / secret-scan -- +# bonnyr-f5 #182 r3, so `make pre-push` genuinely == CI), then tests fan out. +pre-push: quick-check ci-gates @echo "" @echo "=== Running all test suites in parallel... ===" @failed=""; \ @@ -1135,6 +1279,18 @@ buildx-setup: # Push multi-arch images to a container registry # Usage: make push-images BNK_FORGE_REGISTRY=ghcr.io/your-org +# +# TWO INDEPENDENT guards, TWO independent override knobs (bonnyr-f5 #193 r3 minor — +# they used to share FORCE_LATEST, so overriding the ':latest recency' guard also +# silently disarmed the immutable-tag overwrite protection): +# FORCE_LATEST=1 overrides ONLY the recency guard (this stale tree would move +# the rolling ':latest' tag backward). It does NOT touch the +# overwrite guard. +# FORCE_OVERWRITE=1 overrides ONLY the immutable-:VERSION overwrite guard, and +# ONLY for an already-published tag you intend to overwrite +# (orphaning its attestations). It does NOT rescue a tooling +# failure (missing docker/buildx/jq) — that is fail-closed by +# design; install the tooling instead. push-images: @echo "" @echo "=========================================" @@ -1160,6 +1316,32 @@ push-images: echo " Platforms: $(PLATFORMS)"; \ echo " Builder: $(BUILDX_BUILDER)"; \ echo ""; \ + HIGHEST_TAG=$$(git tag -l 'v*' 2>/dev/null | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$$' | sort -V | tail -1); \ + if [ -n "$$HIGHEST_TAG" ] && [ "$${FORCE_LATEST:-}" != "1" ]; then \ + HIGHEST=$$(printf '%s\n%s\n' "$${HIGHEST_TAG#v}" "$$VERSION" | sort -V | tail -1); \ + if [ "$$VERSION" != "$$HIGHEST" ]; then \ + echo "ERROR: local VERSION $$VERSION is older than the highest released tag $$HIGHEST_TAG."; \ + echo " bake pushes the rolling ':latest' tag, so this stale tree would move :latest backward"; \ + echo " (release.yml's recency guard covers the CI path; this covers the operator path)."; \ + echo " Check out the latest release first, or re-run with FORCE_LATEST=1 to override deliberately."; \ + exit 1; \ + fi; \ + fi; \ + if [ "$${FORCE_OVERWRITE:-}" = "1" ]; then GUARD_FORCE=true; else GUARD_FORCE=; fi; \ + echo " Probing the registry via the single-sourced scripts/registry-overwrite-guard.sh so this push"; \ + echo " can't silently overwrite an already-published immutable :$$VERSION tag..."; \ + REGISTRY=$$REGISTRY VERSION=$$VERSION FORCE=$$GUARD_FORCE \ + bash scripts/registry-overwrite-guard.sh || { \ + echo " Remediation depends on WHY it failed (read the ::error:: line above):"; \ + echo " - missing docker/buildx/jq, or an unparseable bake file -> a TOOLING problem."; \ + echo " Install the tooling and re-run. FORCE_* does NOT rescue this (the guard"; \ + echo " refuses to guess the image count, by design)."; \ + echo " - registry unreachable / auth -> export REGISTRY_USERNAME/REGISTRY_PASSWORD and re-run."; \ + echo " - the immutable :$$VERSION tag genuinely already exists -> re-run with"; \ + echo " FORCE_OVERWRITE=1 ONLY if you intend to overwrite it (this orphans its"; \ + echo " cosign/SBOM/SLSA attestations). FORCE_LATEST=1 does NOT override this guard."; \ + exit 1; \ + }; \ echo "=== Building + pushing all images in parallel (docker buildx bake) ==="; \ GIT_REVISION=$$(git rev-parse HEAD 2>/dev/null || echo unknown); \ REGISTRY=$$REGISTRY VERSION=$$VERSION PLATFORMS=$(PLATFORMS) GIT_REVISION=$$GIT_REVISION \ @@ -1204,6 +1386,21 @@ push-customer-build: echo " Rolling tag: customer-build"; \ echo " Platforms: $(CB_PLATFORMS)"; \ echo ""; \ + if [ "$${FORCE_OVERWRITE:-}" = "1" ]; then GUARD_FORCE=true; else GUARD_FORCE=; fi; \ + echo " Probing the registry via the single-sourced scripts/registry-overwrite-guard.sh"; \ + echo " so this push can't silently overwrite the already-published IMMUTABLE :$$FULLTAG"; \ + echo " tag and orphan its cosign/SBOM/SLSA attestations (INV-24 — the only push path"; \ + echo " that was still unguarded, bonnyr-f5 #193 r4)..."; \ + REGISTRY=$$REGISTRY VERSION=$$FULLTAG FORCE=$$GUARD_FORCE \ + bash scripts/registry-overwrite-guard.sh || { \ + echo " Remediation depends on WHY it failed (read the ::error:: line above):"; \ + echo " - missing docker/buildx/jq or an unparseable bake file -> a TOOLING problem"; \ + echo " (FORCE_OVERWRITE does NOT rescue this, by design)."; \ + echo " - registry unreachable / auth -> export REGISTRY_USERNAME/REGISTRY_PASSWORD."; \ + echo " - the immutable :$$FULLTAG tag genuinely already exists -> re-run with"; \ + echo " FORCE_OVERWRITE=1 ONLY if you intend to overwrite it (orphans its attestations)."; \ + exit 1; \ + }; \ echo "=== Building + pushing customer-build images (docker buildx bake) ==="; \ REGISTRY=$$REGISTRY VERSION=$$FULLTAG ROLLING_TAG=customer-build PLATFORMS=$(CB_PLATFORMS) \ docker buildx bake --builder $(CB_BUILDER) --push && \ diff --git a/README.md b/README.md index 230be97..4b4b51d 100644 --- a/README.md +++ b/README.md @@ -88,6 +88,11 @@ make deploy Open **https://localhost** and accept the self-signed certificate warning. +> **Enabling MCP:** the MCP server and backend share a service credential you must +> set — put `MCP_SERVICE_PASSWORD` in `.env` before starting. Without it the stack +> still comes up, but the MCP server can't authenticate and MCP tools return auth +> errors until you set the variable and restart. See [.env.example](.env.example). + `make deploy` detects macOS/WSL and switches to bridge networking with published ports (`docker-compose.local.yml`); on a Linux server it uses host networking. You do not pick — it picks. @@ -140,9 +145,17 @@ For first-time destructive bootstrap only (wipes existing BNK Forge volumes), us | Field | Value | |-------|-------| | **Username** | `admin` | -| **Password** | `changeme` | +| **Password** | _generated on first startup — see below_ | + +The admin password is generated randomly on first startup (there is no shipped +default). Retrieve it once from the backend logs: + +```bash +docker exec bnk-forge-backend cat /app/keys/initial_admin_password +``` -You'll be prompted to change the password on first login. +Or choose your own beforehand by setting `DEFAULT_ADMIN_PASSWORD` in `.env`. You +will be **required** to change it on first login (enforced server-side). --- diff --git a/The_BNK_Forge_Developers_Guide.md b/The_BNK_Forge_Developers_Guide.md index 4e10f2e..73fdd46 100644 --- a/The_BNK_Forge_Developers_Guide.md +++ b/The_BNK_Forge_Developers_Guide.md @@ -198,7 +198,7 @@ make install # 4. Open the UI # macOS / WSL / Linux desktop: https://localhost/ -# Default login: admin / changeme (change it on first login) +# Default login: admin (password: set DEFAULT_ADMIN_PASSWORD, else the generated one at /app/keys/initial_admin_password; change on first login) ``` That is it. The platform detection in the Makefile does the right thing on Darwin (macOS), WSL2, and native Linux without further configuration. @@ -276,7 +276,7 @@ When the install finishes you will see something like: ========================================= URL: https://localhost/ - Login: admin / changeme + Login: admin (password: DEFAULT_ADMIN_PASSWORD if set, else generated — see /app/keys/initial_admin_password) ``` Open the URL in your browser. Self-signed cert warnings are expected; accept once. diff --git a/backend/alembic/versions/v2_154_user_is_service_account.py b/backend/alembic/versions/v2_154_user_is_service_account.py new file mode 100644 index 0000000..4057040 --- /dev/null +++ b/backend/alembic/versions/v2_154_user_is_service_account.py @@ -0,0 +1,32 @@ +"""Add users.is_service_account for service-account provenance. + +Revision ID: v2_154 +Revises: v2_153 + +bonnyr-f5 #188: ensure_service_user identified service accounts by NAME +(a one-entry denylist of "admin"), so pointing MCP_SERVICE_USERNAME at any other +human row (operator, a named user) let the boot-time reconcile overwrite its +password, promote it to admin, clear its must-change gate and re-activate it. +Provenance recorded at creation lets the seeder refuse any pre-existing row it +did not create, independent of the name. +""" +import sqlalchemy as sa + +from alembic import op + +revision = "v2_154" +down_revision = "v2_153" +branch_labels = None +depends_on = None + + +def upgrade() -> None: + with op.batch_alter_table("users") as batch: + batch.add_column( + sa.Column("is_service_account", sa.Boolean(), nullable=False, server_default=sa.false()) + ) + + +def downgrade() -> None: + with op.batch_alter_table("users") as batch: + batch.drop_column("is_service_account") diff --git a/backend/alembic/versions/v2_155_backfill_is_service_account.py b/backend/alembic/versions/v2_155_backfill_is_service_account.py new file mode 100644 index 0000000..e733070 --- /dev/null +++ b/backend/alembic/versions/v2_155_backfill_is_service_account.py @@ -0,0 +1,95 @@ +"""Backfill users.is_service_account for the legacy mcp service account. + +Revision ID: v2_155 +Revises: v2_154 + +bonnyr-f5 #188 round 4 (INV-7) / #193 (minor, rationale corrected): the backfill +lives in its OWN revision, separate from ``v2_154`` (which only adds the column, +``server_default false``). NOTE: ``v2_154`` and ``v2_155`` are BOTH introduced in +THIS diff — the earlier claim that ``v2_154`` "already shipped in earlier RCs" was +false, so no install has ``alembic_version = v2_154`` without also getting +``v2_155`` on the same ``alembic upgrade``. The split is kept regardless because it +is the correct shape: a schema change and its data backfill are cleanly separable +and independently reversible, and IF a future build ever ships the column ahead of +the backfill, a distinct revision still guarantees every install that stopped at +``v2_154`` applies the backfill on its next upgrade (appending it to ``v2_154`` +would silently skip such installs, since an applied revision is immutable). + +Why the backfill is needed at all: the column ships ``server_default false``, so +without it EVERY pre-existing row — including the ``mcp`` service account that +every already-deployed install carries — is classified ``is_service_account = +False``. Both consumers gate on that column, so the mis-classification is a trap +on an upgraded install: + + * ``disable_stale_service_user`` filters ``is_service_account IS TRUE``, so the + stale ``mcp`` row is never disabled and the shipped ``mcp-service-changeme`` + default keeps authenticating as role=admin -> issue #187 stays open for every + existing install. + * ``ensure_service_user`` refuses any row where ``not is_service_account``, so + setting a real ``MCP_SERVICE_PASSWORD`` raises forever and MCP is dead. + +Scope of the backfill is deliberately narrow — we reclassify a row as a service +account ONLY when it carries the exact fingerprint that the legacy +``ensure_service_user`` + ``create_user`` seed produced, never an arbitrary row: + + * ``username = 'mcp'`` — the ONLY value the legacy service account was ever + created under. ``MCP_SERVICE_USERNAME`` defaults to ``'mcp'`` (core/config.py) + and the migration cannot know an operator's overridden value at apply time; + reading app settings into a migration is non-deterministic and, worse, some + legacy ``.env`` files point that var at ``admin`` — backfilling the configured + name would then reclassify the HUMAN admin as a service account, the exact + takeover #188 set out to prevent. We backfill the known legacy default only. + * ``email = 'mcp@bnk-forge.local'`` — ``create_user`` synthesised the service + account's email as ``f"{username}@bnk-forge.local"``, so the legacy ``mcp`` + row provably has this address. Requiring it as a second signal means a real + human who merely happens to be named ``mcp`` (with any real email) is left + untouched. + +The pair (username + synthesised email) is the creation fingerprint of the +service account and cannot collide with a human provisioned through normal +signup, which always carries a real email. Rows that don't match keep the +``server_default false`` — correct, they are human accounts. + +Documented edge: an operator who set a CUSTOM ``MCP_SERVICE_USERNAME`` (not the +default ``mcp``) before upgrading will not have that row backfilled here; the +remedy is to point ``MCP_SERVICE_USERNAME`` at a dedicated name (the default +``mcp`` is now backfilled and reconcilable). Silently reclassifying an +operator-named row we cannot prove we created risks taking over a human account, +which is strictly worse than a one-line rename for the rare custom-username install. +""" +import sqlalchemy as sa + +from alembic import op + +revision = "v2_155" +down_revision = "v2_154" +branch_labels = None +depends_on = None + +# The legacy default service username and the email create_user synthesised for +# it. Kept as constants so the backfill scope is explicit and auditable. +_LEGACY_SERVICE_USERNAME = "mcp" +_LEGACY_SERVICE_EMAIL = "mcp@bnk-forge.local" + + +def upgrade() -> None: + # Backfill: classify ONLY the provably-seeded legacy mcp service account. + # Parameterised so the literals are quoted safely on every backend. + op.execute( + sa.text( + "UPDATE users SET is_service_account = :t " + "WHERE username = :u AND email = :e" + ).bindparams(t=True, u=_LEGACY_SERVICE_USERNAME, e=_LEGACY_SERVICE_EMAIL) + ) + + +def downgrade() -> None: + # Reversing the classification is safe and precise: only rows carrying the + # exact legacy fingerprint were flipped, so we clear the flag for exactly + # those rows. The column itself is owned by v2_154 and is left in place. + op.execute( + sa.text( + "UPDATE users SET is_service_account = :f " + "WHERE username = :u AND email = :e" + ).bindparams(f=False, u=_LEGACY_SERVICE_USERNAME, e=_LEGACY_SERVICE_EMAIL) + ) diff --git a/backend/core/auth_middleware.py b/backend/core/auth_middleware.py index ad10b72..7d64fb2 100644 --- a/backend/core/auth_middleware.py +++ b/backend/core/auth_middleware.py @@ -4,16 +4,21 @@ Can be disabled via REQUIRE_AUTH=false for backward compatibility. """ import logging +from typing import TYPE_CHECKING from fastapi import Request from fastapi.responses import JSONResponse from jose import JWTError +from starlette.concurrency import run_in_threadpool from starlette.middleware.base import BaseHTTPMiddleware, RequestResponseEndpoint from starlette.responses import Response from core.config import settings from core.errors import UnauthorizedError +if TYPE_CHECKING: + from models import User + logger = logging.getLogger(__name__) # Long-lived API tokens (CLI / CI-CD) carry this prefix; they are verified @@ -21,19 +26,23 @@ API_TOKEN_PREFIX = "bnk_" -def _verify_api_token(token: str) -> None: - """Raise UnauthorizedError unless ``token`` is a live API token. +def _verify_api_token(token: str) -> "User": + """Return the owning User for a live API token, or raise UnauthorizedError. Opens its own session: middleware runs outside FastAPI's dependency - injection, so ``get_db`` is not available here. + injection, so ``get_db`` is not available here. The User is refreshed before + the session closes so the caller can read ``must_change_password`` off the + detached instance (same pattern as token_user_state). """ from database import SessionLocal from services.api_token_service import ApiTokenService db = SessionLocal() try: - ApiTokenService(db).verify(token) # raises UnauthorizedError if invalid + user, _api_token = ApiTokenService(db).verify(token) # raises if invalid db.commit() # verify() stamps last_used_at + db.refresh(user) + return user except Exception: db.rollback() raise @@ -146,15 +155,53 @@ async def dispatch(self, request: Request, call_next: RequestResponseEndpoint) - # Validate the token token = auth_header.split(" ", 1)[1] try: + from core.errors import ForbiddenError + from services.auth_service import enforce_password_change if token.startswith(API_TOKEN_PREFIX): # Long-lived CLI / CI-CD token — a DB-backed hash, not a JWT, so - # decode_token() would reject it. 21 /api routes have no auth + # decode_token() would reject it. ~32 /api routes have no auth # dependency of their own and rely on this middleware alone, so # it must fully verify the token here, not defer to the route. - _verify_api_token(token) + # + # bonnyr-f5 #186 r5 (Major): _verify_api_token opens a SYNC DB + # session; awaiting it directly on the event-loop thread would + # block every concurrent request for the duration of the query. + # Off-load the blocking I/O to a worker thread so dispatch stays + # non-blocking (same treatment as token_user_state below). + user = await run_in_threadpool(_verify_api_token, token) + enforce_password_change(path, user) else: - from services.auth_service import decode_token + from services.auth_service import decode_token, token_user_state decode_token(token) # Will raise UnauthorizedError if invalid + # #184/#186: the dependency-only gate is bypassed on routes that + # declare no get_current_user; enforce must_change here too, where + # auth is actually resolved. decode_token already succeeded, so a + # None from token_user_state is a VALID JWT whose user can't be + # resolved (deleted/disabled/DB error) -- the fail-CLOSED case, per + # its own contract and the WS validators. Refuse it, don't skip the + # gate (bonnyr-f5 #186 r2: skipping was a fail-open bypass on the + # ~32 dependency-less routes). + # bonnyr-f5 #186 r5 (Major): token_user_state opens a SYNC DB + # session on every authenticated request. Run it in a worker + # thread so the blocking DB round-trip never stalls the event + # loop (the gate previously only paid this cost for rare bnk_ + # tokens; it now runs for every JWT request). + jwt_user = await run_in_threadpool(token_user_state, token) + if jwt_user is None: + raise UnauthorizedError("Token subject could not be resolved") + enforce_password_change(path, jwt_user) + except ForbiddenError as exc: + return JSONResponse( + status_code=403, + content={ + "error": { + "code": "PASSWORD_CHANGE_REQUIRED", + "message": str(exc), + "details": {}, + "path": path, + } + }, + ) except (JWTError, UnauthorizedError): return JSONResponse( status_code=401, diff --git a/backend/core/config.py b/backend/core/config.py index 78d44c5..ea3783d 100644 --- a/backend/core/config.py +++ b/backend/core/config.py @@ -16,6 +16,10 @@ logger = logging.getLogger(__name__) +# Passwords ever shipped as the MCP service default. Treated as "not set" on +# both the fail-fast (validate_production) and the boot-rotation path (bonnyr-f5 #188). +MCP_KNOWN_DEFAULT_PASSWORDS = ("mcp-service-changeme", "changeme") + # BE-007: Directory for persisting auto-generated keys across restarts _KEYS_DIR = os.environ.get("KEYS_DIR", "/app/keys") @@ -33,37 +37,176 @@ def _read_version_file() -> str: return "0.0.0" # fallback if VERSION file not found -def _persist_or_load_key(filename: str, generate_fn: Callable[[], str]) -> tuple[str, bool]: +def _encryption_key_path() -> str: + """The single at-rest Fernet key file (bonnyr-f5 #193 B-3). + + Resolved IDENTICALLY to ``core.encryption.ENCRYPTION_KEY_FILE`` so the value + whose provenance this module gates on is the SAME file ``core.encryption`` (and + ``services.backup_service``) actually loads — never a shadow. Honours an explicit + ``ENCRYPTION_KEY_FILE`` override, else ``$KEYS_DIR/encryption.key``. + """ + override = os.environ.get("ENCRYPTION_KEY_FILE") + if override: + return override + return os.path.join(_KEYS_DIR, "encryption.key") + + +def _is_valid_fernet_key(value: str) -> bool: + """True if *value* is a syntactically valid Fernet key (32 url-safe b64 bytes).""" + try: + from cryptography.fernet import Fernet + + Fernet(value.encode()) + return True + except Exception: + return False + + +def _write_key_file_0600(key_path: str, key_value: str) -> bool: + """Persist *key_value* to *key_path* at mode 0600. Returns True on success. + + Creates the file 0o600 at open() time (os.open) and fchmod's it so the secret is + never even briefly world-readable under the usual umask, and so a pre-existing + looser file from an older release is tightened before we write (mirrors + _persist_generated_password). + """ + try: + parent = os.path.dirname(key_path) + if parent: + os.makedirs(parent, exist_ok=True) + fd = os.open(key_path, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600) + os.fchmod(fd, 0o600) + with os.fdopen(fd, "w") as f: + f.write(key_value) + return True + except (OSError, PermissionError) as e: + logger.warning(f"Could not persist key to {key_path}: {e} — key will be lost on restart") + return False + + +def _persist_or_load_key(key_path: str, generate_fn: Callable[[], str]) -> tuple[str, bool]: """ BE-007: Load a key from persistent storage, or generate and save a new one. - Returns (key_value, was_auto_generated). + Returns (key_value, was_auto_generated). *key_path* is the FULL path to the key + file (bonnyr-f5 #193 B-3: encryption and jwt now name distinct absolute paths so + the encryption key can live at ``_encryption_key_path()`` — the same file + ``core.encryption`` loads — rather than a shadow under ``_KEYS_DIR``). + + bonnyr-f5 #193 B2 (round 3): ``was_auto_generated`` gates SEC-006's fail-fast, + so it MUST fail closed — absence of evidence of provisioning is not evidence of + provisioning. The primary operator path is an env var (``JWT_SECRET_KEY`` / + ``ENCRYPTION_KEY``); when set, Settings.__init__ handles it directly (and for + ENCRYPTION_KEY writes it to ``key_path`` with a marker — see __init__). The + SECONDARY operator path is pre-seeding the key file on the keys volume; the app + is the ONLY other writer of that file on every shipped path, so a marker-less key + file could equally be our own auto-gen from a prior release — exactly the + population that upgrades into ``ENVIRONMENT=production``. The provenance signal is + therefore an EXPLICIT operator OPT-OUT marker ``.operator``: + * key file present + ``.operator`` marker (a regular FILE) -> operator + provisioned it and said so explicitly -> auto=False + * key file present + NO ``.operator`` marker -> auto=True (fail closed) + * key file ABSENT + ``.operator`` marker present -> provisioning ERROR + (bonnyr-f5 #193 M-1): do NOT adopt the marker as provenance and do NOT + persist a generated key, else boot-2 would load key+marker and classify + auto=False — the app booting on its OWN generated key. Fail closed every + boot instead (auto=True, unpersisted) until the operator fixes the volume. + * key file absent, no marker -> generate + persist -> auto=True; write NO + marker (a marker-less key already means auto=True; nothing to record, and no + second write a partial failure could use to downgrade provenance). + + bonnyr-f5 #193 M-1: the marker must be a regular FILE — ``os.path.exists`` used + to accept a DIRECTORY named ``.operator`` as provenance. ``os.path.isfile`` + refuses that. """ - key_path = os.path.join(_KEYS_DIR, filename) + operator_marker_path = key_path + ".operator" + # M-1: a regular FILE only — a directory named .operator is NOT provenance. + marker_present = os.path.isfile(operator_marker_path) try: - if os.path.exists(key_path): + if os.path.isfile(key_path): with open(key_path) as f: key = f.read().strip() if key: - return key, True # Loaded from file (still auto-generated, not user-provided) + # Fail closed: operator-provided ONLY when the explicit opt-out + # marker (a regular file) sits beside the key. + return key, (not marker_present) except (OSError, PermissionError) as e: logger.warning(f"Could not read {key_path}: {e}") - # Generate new key + # Key file absent (or empty/unreadable). + # M-1: a marker with NO key file is a provisioning ERROR, not provenance. The + # natural key-rotation gesture (delete the key, keep the marker) would otherwise + # heal into a fail-open across two boots: boot-1 generates+persists a key beside + # the stale marker; boot-2 loads key+marker -> auto=False -> production BOOTS on + # our OWN generated key. Refuse to persist so the situation can never downgrade to + # operator-provenance; keep classifying auto-generated (fail closed) every boot. + if marker_present: + logger.error( + f"Provisioning error: operator marker {operator_marker_path} is present " + f"but the key file {key_path} is missing. Treating the key as " + f"auto-generated (fail closed) and NOT persisting it. Restore the " + f"operator-provisioned key file beside the marker, or remove the stale marker." + ) + return generate_fn(), True + + # Generate new key and persist it (no marker: a marker-less key already + # classifies auto-generated, fail closed). key = generate_fn() - - # Try to persist it - try: - os.makedirs(_KEYS_DIR, exist_ok=True) - with open(key_path, "w") as f: - f.write(key) - os.chmod(key_path, 0o600) # Read/write only by owner + if _write_key_file_0600(key_path, key): logger.info(f"Persisted auto-generated key to {key_path}") - except (OSError, PermissionError) as e: - logger.warning(f"Could not persist key to {key_path}: {e} — key will be lost on restart") - return key, True # Auto-generated +def _seed_encryption_key_if_absent(key_path: str, key_value: str) -> tuple[str, bool]: + """bonnyr-f5 #193 B-3 (r4 self-review): reconcile an operator-provided + ``ENCRYPTION_KEY`` with the at-rest key file WITHOUT ever destroying data. + + INVARIANT: the at-rest key FILE is the single source of truth. ``core.encryption`` + reads it, ``services.backup_service`` restore WRITES it, and prior releases + persisted an auto-generated key there. So this function NEVER overwrites an + existing file — that file may hold the key under which live data was already + encrypted, and clobbering it makes that data undecryptable (and doing so under a + ``.operator`` marker mismatch previously *bricked* the boot). ``ENCRYPTION_KEY`` + env only SEEDS the file when it is absent (first boot), and provenance for + ``validate_production`` is read from the FILE's ``.operator`` marker, so the gate + reflects the value that actually encrypts regardless of the env var. + + Returns ``(effective_at_rest_key, auto_generated)`` — the FILE's value wins when a + file already exists, so ``self.ENCRYPTION_KEY`` tracks what ``core.encryption`` + will actually load. + """ + marker_path = key_path + ".operator" + if os.path.isfile(key_path): + try: + with open(key_path) as f: + existing = f.read().strip() + except (OSError, PermissionError): + existing = "" + if existing: + # The persisted key is authoritative. Never overwrite it; never brick. + if existing != key_value: + logger.warning( + "ENCRYPTION_KEY differs from the persisted at-rest key at %s; the " + "persisted key is authoritative and the env value is IGNORED " + "(overwriting it would make existing encrypted data undecryptable). " + "Remove ENCRYPTION_KEY from the environment, or rotate the at-rest " + "key out of band (re-encrypt), then restart.", + key_path, + ) + # Provenance is the FILE's marker, not the env var's presence: a marker-less + # persisted key (a prior release's auto-gen, or a bare restore) classifies + # auto-generated so production fail-fast still fires on it. + return existing, (not os.path.isfile(marker_path)) + # File absent (or unreadable/empty): seed it from the validated operator value and + # record operator provenance. + if _write_key_file_0600(key_path, key_value) and not os.path.isfile(marker_path): + try: + with open(marker_path, "w") as mf: + mf.write("") + except (OSError, PermissionError) as e: + logger.warning(f"Could not write provenance marker {marker_path}: {e}") + return key_value, False # freshly seeded from the operator env value + + class Settings(BaseSettings): """Application settings with validation""" @@ -97,10 +240,45 @@ def cors_origins(self) -> list[str]: JWT_SECRET_KEY: str | None = None ENCRYPTION_KEY: str | None = None - # Seed credentials — distinct vars so admin rotation never affects MCP - DEFAULT_ADMIN_PASSWORD: str = "changeme" + # DEFAULT_ADMIN_PASSWORD defaults to None, never a hardcoded value: a + # shipped default like "changeme" is a live, publicly-known admin credential + # on every fresh deployment (#184). When unset, seed_admin_user generates a + # random one and logs it once (the account is must_change_password anyway). + DEFAULT_ADMIN_PASSWORD: str | None = None + # Test/ephemeral environments (e2e) seed a KNOWN admin and skip the + # must-change gate so the suite can reach protected routes. Defaults True; + # never set false on a real deployment. + DEFAULT_ADMIN_MUST_CHANGE: bool = True MCP_SERVICE_USERNAME: str = "mcp" - MCP_SERVICE_PASSWORD: str = "mcp-service-changeme" + # #187/#188: shared secret between the backend (which seeds the `mcp` service + # account) and the MCP server (which authenticates with it). The seeded 'mcp' + # account is role=admin and exempt from the #184 must-change gate, so this + # value must NEVER carry a published default like "mcp-service-changeme" — that + # would be a live, publicly-known admin credential. + # + # Defaults to None. It CANNOT be auto-generated: both sides must receive the + # SAME value, so it must be set explicitly. validate_production fails fast + # (SystemExit) under ENVIRONMENT=staging|production when it is unset or a known + # default. + # + # Merged behaviour (#186 + #188 — #188's "unset -> disable" was chosen over + # #186's "unset -> generate"): when this is UNSET (or a known published + # default), startup_steps.seed_auth_step does NOT call ensure_service_user + # (its _mcp_pw_usable gate is false); it calls disable_stale_service_user + # instead, so the 'mcp' account is left DISABLED/unavailable until an operator + # configures a real password (which re-seeds and re-activates the row). No + # random secret is generated and nothing is surfaced. + # + # When it IS set to a usable value: the BACKEND receives MCP_SERVICE_PASSWORD + # on every deploy mode (the backend-env anchors in every compose file, the ibm + # installer, and the Helm shared-env in _helpers.tpl sourced from the release + # Secret's mcp-password key), and ensure_service_user reconciles the 'mcp' + # account's stored hash to it — the same per-install secret the mcp client + # uses — so the env var can be rotated without auth drift. Any row still + # holding a shipped published default is refused as a seed value and rotated + # out on upgrade. The reserved-name guard in ensure_service_user and #188's + # Helm mcp-secret work share this credential surface. + MCP_SERVICE_PASSWORD: str | None = None # Benchmark agent auth flag. # When False (default): register/ingest/WS are open (preserves the documented curl flow). @@ -155,9 +333,16 @@ class Config: def __init__(self, **kwargs: Any) -> None: super().__init__(**kwargs) - # BE-007: Handle JWT_SECRET_KEY — persist to file so restarts reuse the same key - if self.JWT_SECRET_KEY is None: - key, auto = _persist_or_load_key("jwt_secret.key", lambda: secrets.token_hex(32)) + # BE-007: Handle JWT_SECRET_KEY — persist to file so restarts reuse the same key. + # bonnyr-f5 #193 B2: treat an EMPTY value as unset. The compose anchors plumb + # `${JWT_SECRET_KEY:-}`, so an operator who does not set it delivers "" (not an + # absent var) to the container. `is None` would then accept "" as an + # explicitly-provided key (auto_generated=False), so validate_production would + # pass on an empty secret. `not` routes "" through generation instead. + if not self.JWT_SECRET_KEY: + key, auto = _persist_or_load_key( + os.path.join(_KEYS_DIR, "jwt_secret.key"), lambda: secrets.token_hex(32) + ) self.JWT_SECRET_KEY = key self._jwt_key_auto_generated = auto if self.ENVIRONMENT == "development": @@ -165,18 +350,49 @@ def __init__(self, **kwargs: Any) -> None: else: self._jwt_key_auto_generated = False - # BE-007: Handle ENCRYPTION_KEY — persist to file so encrypted data survives restarts - # NOTE: ENCRYPTION_KEY must be a valid Fernet key (44-byte base64-encoded) - # Use Fernet.generate_key() to create valid keys - if self.ENCRYPTION_KEY is None: + # Handle ENCRYPTION_KEY — the at-rest Fernet key that actually encrypts stored + # secrets. bonnyr-f5 #193 B-3: gate on the value we PROTECT, not a shadow. + # There is ONE key file (``_encryption_key_path()`` — the same file + # ``core.encryption`` loads), ONE generator, ONE provenance signal. + # * ENCRYPTION_KEY set -> validate it is a real Fernet key (fail clearly if + # not), then SEED the key file from it ONLY IF the file is absent (never + # overwrite an existing at-rest key — that would destroy already-encrypted + # data or brick the boot). The FILE wins if it exists; self.ENCRYPTION_KEY + # and the auto flag track the FILE, so `core.encryption` and the gate agree. + # * ENCRYPTION_KEY unset/empty (compose `${ENCRYPTION_KEY:-}` = "") -> load or + # generate the key file; provenance comes from the .operator marker. + enc_path = _encryption_key_path() + if not self.ENCRYPTION_KEY: from cryptography.fernet import Fernet - key, auto = _persist_or_load_key("encryption.key", lambda: Fernet.generate_key().decode()) + key, auto = _persist_or_load_key(enc_path, lambda: Fernet.generate_key().decode()) self.ENCRYPTION_KEY = key self._encryption_key_auto_generated = auto if self.ENVIRONMENT == "development": logger.info("Using auto-generated ENCRYPTION_KEY (persisted to /app/keys/)") else: - self._encryption_key_auto_generated = False + # bonnyr-f5 #193 B-3: ENCRYPTION_KEY is now CONSUMED as the at-rest key. + # An invalid value used to pass the production gate while encrypting + # nothing (secrets.token_hex(16), the old printed remedy, is NOT a Fernet + # key); fail clearly instead. + if not _is_valid_fernet_key(self.ENCRYPTION_KEY): + logger.error("=" * 60) + logger.error("FATAL: ENCRYPTION_KEY is not a valid Fernet key") + logger.error("=" * 60) + logger.error( + "It must be a 32-byte url-safe base64 Fernet key. Generate one with:" + ) + logger.error( + " ENCRYPTION_KEY=$(python3 -c \"from cryptography.fernet import " + "Fernet; print(Fernet.generate_key().decode())\")" + ) + logger.error("=" * 60) + raise SystemExit(1) + key, auto = _seed_encryption_key_if_absent(enc_path, self.ENCRYPTION_KEY) + # The FILE is authoritative: if one already exists (a prior release's key, + # or one written by backup restore), it wins and the env value is ignored, + # so self.ENCRYPTION_KEY tracks what core.encryption actually loads. + self.ENCRYPTION_KEY = key + self._encryption_key_auto_generated = auto def validate_production(self) -> None: """ @@ -199,12 +415,30 @@ def validate_production(self) -> None: "ENCRYPTION_KEY was not explicitly set — set it as an environment variable" ) - if "*" in self.ALLOWED_ORIGINS: + # #187: the MCP service password is a shared secret and cannot be + # auto-generated -- it must be set explicitly and identically on the + # backend and the MCP server. Refuse an unset or known-default value. + # bonnyr-f5: the actually-shipped default across dist/helm/scripts was + # "changeme", not just "mcp-service-changeme" — reject both. + if not self.MCP_SERVICE_PASSWORD or self.MCP_SERVICE_PASSWORD in MCP_KNOWN_DEFAULT_PASSWORDS: + issues.append( + "MCP_SERVICE_PASSWORD was not set to a real value — set it (the same " + "value the MCP server gets as BNK_FORGE_PASSWORD) as an environment variable" + ) + + # Flagged outside every slice: `"*" in self.ALLOWED_ORIGINS` was a SUBSTRING + # test on the raw CSV, so a legitimate origin that merely CONTAINS a '*' + # (e.g. a subdomain-wildcard entry `https://*.example.com`) was wrongly + # rejected. Mean "a wildcard origin ENTRY": test the parsed origin list for + # an exact `*`. + if "*" in self.cors_origins: issues.append( "ALLOWED_ORIGINS contains '*' (wildcard) — set specific origins" ) - if "localhost" in self.ALLOWED_ORIGINS and self.ENVIRONMENT == "production": + if self.ENVIRONMENT == "production" and any( + "localhost" in origin for origin in self.cors_origins + ): issues.append( "ALLOWED_ORIGINS contains 'localhost' — use your actual domain/IP" ) @@ -218,7 +452,14 @@ def validate_production(self) -> None: logger.error("") logger.error("To fix: set these as environment variables in docker-compose.yml.") logger.error(" JWT_SECRET_KEY=$(python3 -c \"import secrets; print(secrets.token_hex(32))\")") - logger.error(" ENCRYPTION_KEY=$(python3 -c \"import secrets; print(secrets.token_hex(16))\")") + # bonnyr-f5 #193 B-3: ENCRYPTION_KEY is a Fernet key (consumed as the + # at-rest key), NOT secrets.token_hex(16) — that old recipe printed an + # invalid key that encrypted nothing. Match .env.example:42. + logger.error( + " ENCRYPTION_KEY=$(python3 -c \"from cryptography.fernet import " + "Fernet; print(Fernet.generate_key().decode())\")" + ) + logger.error(" MCP_SERVICE_PASSWORD=") logger.error("See: docs/DEPLOYMENT.md") logger.error("=" * 60) raise SystemExit(1) diff --git a/backend/core/encryption.py b/backend/core/encryption.py index ea27c41..d32215c 100644 --- a/backend/core/encryption.py +++ b/backend/core/encryption.py @@ -12,8 +12,23 @@ logger = logging.getLogger(__name__) -# Encryption key file path from environment or default -ENCRYPTION_KEY_FILE = os.getenv("ENCRYPTION_KEY_FILE", "/app/keys/encryption.key") +# Encryption key file path from environment or default. +# +# bonnyr-f5 #193 B-3: the FILE at ENCRYPTION_KEY_FILE (default +# $KEYS_DIR/encryption.key, on the persistent keys volume) is the ONE source of +# truth for the at-rest Fernet key actually used to encrypt/decrypt stored secrets. +# There is no longer a shadow: core.config resolves this SAME path +# (``_encryption_key_path()``) and, when the operator sets ``ENCRYPTION_KEY``, +# VALIDATES it as a real Fernet key and WRITES it here (with a ``.operator`` +# provenance marker) before this module loads — so setting ``ENCRYPTION_KEY`` now +# genuinely becomes the at-rest key that get_encryption_key() returns and that +# validate_production's fail-fast gates on. When ``ENCRYPTION_KEY`` is unset, +# core.config generates+persists a Fernet key here on first boot. Either way, one +# key, one generator, one provenance signal. Resolution mirrors +# core.config._encryption_key_path() so both modules name the identical file. +ENCRYPTION_KEY_FILE = os.getenv("ENCRYPTION_KEY_FILE") or os.path.join( + os.environ.get("KEYS_DIR", "/app/keys"), "encryption.key" +) def get_encryption_key() -> bytes: @@ -21,23 +36,51 @@ def get_encryption_key() -> bytes: Tries to read from ENCRYPTION_KEY_FILE, or creates a new key if not found. Falls back to in-memory key generation if file operations fail (e.g., permissions). + + The file is the single source of truth for the at-rest key; when the operator + sets ``ENCRYPTION_KEY``, core.config has already validated it and written it to + this file, so this returns THAT value — see the module note above + ENCRYPTION_KEY_FILE. """ - # Try to read existing key + # Try to read an existing key. bonnyr-f5 #193 I-1: the file is the single source of + # truth and may hold the key LIVE DATA is encrypted under, so once it holds bytes we + # NEVER regenerate over them -- that would destroy that data permanently and silently + # on an otherwise-green boot (the .operator marker keeps validate_production passing). if os.path.exists(ENCRYPTION_KEY_FILE): try: with open(ENCRYPTION_KEY_FILE, 'rb') as f: - key = f.read().strip() - if key and len(key) >= 32: # Valid Fernet key is 44 bytes base64 - return key - logger.warning(f"Invalid key in {ENCRYPTION_KEY_FILE}, regenerating") + existing = f.read().strip() except PermissionError as e: logger.warning(f"Could not read encryption key from {ENCRYPTION_KEY_FILE}: {e}") logger.warning("Will generate in-memory key (NOT RECOMMENDED for production)") logger.warning("Fix with: docker exec -u root bnk-forge-backend chown -R bnkforge:bnkforge /app/keys") + existing = b"" except Exception as e: logger.warning(f"Error reading encryption key: {e}") - - # Generate new key + existing = b"" + else: + if existing: + # The file holds bytes. Validate it as a real Fernet key; if it is + # unusable (truncated/partial write, bad restore, wrong format) FAIL + # CLOSED rather than regenerate -- a crashloop is recoverable, an + # overwritten key is not. (Also closes the "any >=32 bytes accepted" + # gap: a mis-shaped key now surfaces as a clear message, not a later + # cipher error.) + try: + Fernet(existing) + except Exception as e: + raise SystemExit( + f"FATAL: the at-rest encryption key at {ENCRYPTION_KEY_FILE} exists " + f"but is not a valid Fernet key ({e}). Refusing to regenerate over it " + f"-- that would permanently destroy any data already encrypted under the " + f"original key. Restore a good key file (e.g. from a keys-volume backup); " + f"only if this deployment has NO encrypted data yet, delete the file to " + f"let a fresh key be generated." + ) from e + return existing + # Empty file (0 bytes): no committed key to lose -> fall through to generate. + + # File genuinely absent (or empty / unreadable): generate a new key and persist it. key = Fernet.generate_key() # Try to persist it diff --git a/backend/models/system.py b/backend/models/system.py index 9ef1733..c8e1857 100644 --- a/backend/models/system.py +++ b/backend/models/system.py @@ -1,6 +1,6 @@ """System models: ApplicationSetting, SyncJob, User, AuditLog, Notification, HelmChart, CloudCredentialTemplate.""" -from sqlalchemy import JSON, Boolean, Column, DateTime, Float, ForeignKey, Index, Integer, String, Text +from sqlalchemy import JSON, Boolean, Column, DateTime, Float, ForeignKey, Index, Integer, String, Text, false from sqlalchemy.orm import relationship from sqlalchemy.sql import func @@ -152,6 +152,12 @@ class User(Base): role = Column(String(50), nullable=False, default="operator") is_active = Column(Boolean, default=True, nullable=False) must_change_password = Column(Boolean, default=False, nullable=False) + # bonnyr-f5 #188: provenance for service accounts (mcp). ensure_service_user + # refuses to reconcile a row it did NOT create as a service account, so + # pointing MCP_SERVICE_USERNAME at a human row can't take it over. + # server_default mirrors migration v2_154 so fresh (create_all) and migrated + # installs agree on the DB-level default (bonnyr-f5 #188 nit). + is_service_account = Column(Boolean, default=False, server_default=false(), nullable=False) last_login_at = Column(DateTime(timezone=True), nullable=True) created_at = Column(DateTime(timezone=True), server_default=func.now()) updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now()) diff --git a/backend/openapi.json b/backend/openapi.json index 15b8a97..8ebd672 100644 --- a/backend/openapi.json +++ b/backend/openapi.json @@ -61016,6 +61016,11 @@ "type": "boolean", "title": "Is Active" }, + "is_service_account": { + "type": "boolean", + "title": "Is Service Account", + "default": false + }, "must_change_password": { "type": "boolean", "title": "Must Change Password" @@ -61160,6 +61165,11 @@ "type": "boolean", "title": "Is Active" }, + "is_service_account": { + "type": "boolean", + "title": "Is Service Account", + "default": false + }, "must_change_password": { "type": "boolean", "title": "Must Change Password" @@ -61283,6 +61293,11 @@ "type": "boolean", "title": "Is Active" }, + "is_service_account": { + "type": "boolean", + "title": "Is Service Account", + "default": false + }, "must_change_password": { "type": "boolean", "title": "Must Change Password" diff --git a/backend/routes/auth.py b/backend/routes/auth.py index b72722e..706ed38 100644 --- a/backend/routes/auth.py +++ b/backend/routes/auth.py @@ -32,6 +32,7 @@ create_access_token, create_user, get_user_from_token, + holds_known_default_password, ) logger = logging.getLogger(__name__) @@ -106,9 +107,23 @@ def get_current_user(request: Request, db: Session = Depends(get_db)) -> User: # Re-clamp to the owner's *current* role too, so demoting a user # immediately narrows every token they already issued. user.request_role = clamp_role(api_token.role, user.role) + _enforce_password_change(request, user) return user - return get_user_from_token(db, token) + user = get_user_from_token(db, token) + _enforce_password_change(request, user) + return user + + +# The gate now lives in services.auth_service so AuthMiddleware and this +# dependency enforce it from ONE place -- a dependency-less route was bypassing +# the dependency-only version (see enforce_password_change). +from services.auth_service import enforce_password_change # noqa: E402 + + +def _enforce_password_change(request: Request, user: User) -> None: + """#184: gate a must-change user at the get_current_user dependency.""" + enforce_password_change(request.url.path, user) def require_role(*allowed_roles: str): @@ -229,6 +244,9 @@ def _user_to_dict(user: User) -> dict: # carries a request_role, so admin user listings are unaffected. "role": effective_role(user), "is_active": user.is_active, + # bonnyr-f5 #188: surface provenance so the UI can distinguish a service + # account (whose re-enable is guarded) from a human account. + "is_service_account": bool(user.is_service_account), "must_change_password": user.must_change_password, "last_login_at": user.last_login_at.isoformat() if user.last_login_at else None, "created_at": user.created_at.isoformat() if user.created_at else None, @@ -355,6 +373,28 @@ def update_user( target.email = request.email if request.is_active is not None: + # bonnyr-f5 #188: re-enabling a service account that still holds a shipped + # default password would resurrect the published default credential. This is + # defence-in-depth for a row taken inactive by a path that LEAVES the + # credential intact — e.g. a manual operator PUT is_active=false. + # (disable_stale_service_user itself now scrubs the hash on the upgrade path, + # #193, so this guard covers the other disable paths.) Refuse the re-enable + # until the secret is rotated; the operator sets MCP_SERVICE_PASSWORD + # (startup re-seeds and re-activates the row with a real hash) or changes + # the password explicitly. Never restore a known default. + if ( + request.is_active + and not target.is_active + and target.is_service_account + and holds_known_default_password(target) + ): + from core.errors import BadRequestError + raise BadRequestError( + f"Cannot re-enable service account '{target.username}': it still " + "holds a known default password. Rotate the credential first — set " + "MCP_SERVICE_PASSWORD to a strong secret and restart the backend " + "(it re-seeds and re-activates the account with a real hash)." + ) target.is_active = request.is_active db.commit() diff --git a/backend/routes/benchmarks.py b/backend/routes/benchmarks.py index b47f64c..167ccd2 100644 --- a/backend/routes/benchmarks.py +++ b/backend/routes/benchmarks.py @@ -122,6 +122,29 @@ def _require_agent_bearer(request: Request) -> dict: f"Token role '{role or 'none'}' may not write to agent endpoints", code="AGENT_AUTH_FORBIDDEN", ) + # bonnyr-f5 #186 r2: a human (operator/admin) token owed a password change + # must not write here either -- this path skipped the gate entirely (a + # must-change admin could create an agent). Agent tokens carry no User row, + # and this endpoint is role-based by design, so only gate a token that + # resolves to a REAL user: if that user owes a password change, refuse. + if role != "agent": + from core.errors import ForbiddenError + from services.auth_service import enforce_password_change, token_user_state + agent_user = token_user_state(token) + # token_user_state's contract: the caller refuses on None -- fail CLOSED. + # A non-agent role that resolves to no live User (deleted/disabled row, or + # a signed token whose subject never existed) must be rejected here, not + # waved through. Without this else the gate is skipped for exactly that + # case (proven fail-open: a nonexistent user's admin JWT -> 201). + if agent_user is None: + raise BadRequestError( + "Token does not resolve to an active user", + code="AGENT_AUTH_INVALID", + ) + try: + enforce_password_change(request.url.path, agent_user) + except ForbiddenError as exc: + raise BadRequestError(str(exc), code="AGENT_AUTH_PASSWORD_CHANGE_REQUIRED") return payload @@ -1483,10 +1506,34 @@ def _agent_ws_authorized(websocket: WebSocket, agent_id: int) -> int | None: try: from services.auth_service import decode_token - decode_token(token) - return None + payload = decode_token(token) except Exception: return 4001 + # #186 (bonnyr-f5 r4, INV-10): decode_token validates the signature/expiry + # only, so this path waved a must-change human admin straight through. + # (bonnyr-f5 #193 minor: the earlier claim that this was "the one + # JWT-resolving entry point that skipped the gate" was wrong about the OTHER + # branch of this same function — the BENCHMARK_AGENT_AUTH_REQUIRED path above + # also resolves a JWT via decode_token and returns None without the + # must-change gate. It is safe there only because it additionally requires an + # agent_id claim, which no route mints for a human, so a must-change admin + # gets 4401 rather than a pass — but it is not gate-free, so don't describe + # this branch as unique.) + # Agent tokens (role=agent) carry no User row and legitimately reach this + # branch when agent auth is off, so gate ONLY a token that resolves to a real + # user: refuse if that user owes a password change or no longer resolves + # (deleted/disabled). Mirrors the POST /api/benchmarks/agents gate above. + if payload.get("role") != "agent": + from services.auth_service import token_user_state + + ws_user = token_user_state(token) + if ws_user is None or ws_user.must_change_password: + logger.warning( + "Agent %d WS rejected: token owes a password change or does not resolve to an active user", + agent_id, + ) + return 4001 + return None @ws_router.websocket("/ws/benchmarks/agents/{agent_id}") @@ -1502,7 +1549,15 @@ async def agent_websocket(websocket: WebSocket, agent_id: int): - BENCHMARK_AGENT_AUTH_REQUIRED ON → token + agent_id claim match (close 4401). - REQUIRE_AUTH ON (global JWT, M2) → valid token required (close 4001). """ - close_code = _agent_ws_authorized(websocket, agent_id) + # #193 (CR-2): _agent_ws_authorized is fully synchronous and calls + # token_user_state, which opens a SYNC DB session. Run the whole helper off the + # event loop so the handshake never blocks it (same intent as + # core/auth_middleware.py's run_in_threadpool(token_user_state, ...)). The + # helper only reads websocket.query_params and returns a close code — it awaits + # nothing — so it is safe to run in a thread; the caller still does the async + # websocket.close() below. + from starlette.concurrency import run_in_threadpool + close_code = await run_in_threadpool(_agent_ws_authorized, websocket, agent_id) if close_code is not None: await websocket.close(code=close_code) logger.warning("Agent %d WS rejected: missing/invalid token", agent_id) diff --git a/backend/routes/dpus_websocket.py b/backend/routes/dpus_websocket.py index 32b0f27..62b321e 100644 --- a/backend/routes/dpus_websocket.py +++ b/backend/routes/dpus_websocket.py @@ -58,6 +58,19 @@ async def _validate_ws_token(websocket: WebSocket, token: str | None) -> bool: if role not in ("admin", "operator"): await websocket.close(code=4401, reason="Unauthorized — operator role required") return False + # #184: fail closed on the actual User row (see k8s_websocket) -- refuse + # if it can't be resolved or still owes a password change, so a + # seed-credential admin never reaches the DPU console / BMC SSH. + # bonnyr-f5 #186 r5 / #193 (CR-2): token_user_state opens a SYNC DB session; + # run it off the event loop so the handshake never blocks it (same treatment + # as core/auth_middleware.py). + from starlette.concurrency import run_in_threadpool + + from services.auth_service import token_user_state + ws_user = await run_in_threadpool(token_user_state, token) + if ws_user is None or ws_user.must_change_password: + await websocket.close(code=4401, reason="Unauthorized") + return False return True except Exception: await websocket.close(code=4401, reason="Unauthorized — invalid token") diff --git a/backend/routes/k8s_websocket.py b/backend/routes/k8s_websocket.py index e5b3e90..db2706e 100644 --- a/backend/routes/k8s_websocket.py +++ b/backend/routes/k8s_websocket.py @@ -43,6 +43,19 @@ async def _validate_ws_token(websocket: WebSocket, token: str | None) -> bool: if role not in ("admin", "operator", "viewer"): await websocket.close(code=4401, reason="Unauthorized — insufficient role") return False + # #184: fail closed on the actual User row -- refuse if it can't be + # resolved (deleted/disabled account, DB error) OR still owes a password + # change, so a seed-credential admin never reaches pod exec / DPU console. + # bonnyr-f5 #186 r5 / #193 (CR-2): token_user_state opens a SYNC DB session; + # run it off the event loop so the handshake never blocks it (same treatment + # as core/auth_middleware.py). + from starlette.concurrency import run_in_threadpool + + from services.auth_service import token_user_state + ws_user = await run_in_threadpool(token_user_state, token) + if ws_user is None or ws_user.must_change_password: + await websocket.close(code=4401, reason="Unauthorized") + return False return True except Exception: await websocket.close(code=4401, reason="Unauthorized — invalid or expired token") diff --git a/backend/schemas/auth.py b/backend/schemas/auth.py index 3cdab48..e2d5f16 100644 --- a/backend/schemas/auth.py +++ b/backend/schemas/auth.py @@ -22,6 +22,7 @@ class UserInfo(BaseModel): email: str role: str is_active: bool + is_service_account: bool = False # bonnyr-f5 #188: service-account provenance must_change_password: bool last_login_at: str | None = None created_at: str | None = None @@ -59,6 +60,11 @@ class UserResponse(BaseModel): email: str role: str is_active: bool + # bonnyr-f5 #188: expose provenance so the UI can tell a service account from a + # human one. Re-enabling a service account that still holds a shipped default is + # refused (PUT /api/auth/users/{id} -> 400), so the toggle must be able to + # render it disabled/annotated instead of 400ing blind. + is_service_account: bool = False must_change_password: bool last_login_at: str | None = None created_at: str | None = None @@ -100,6 +106,7 @@ class UserWithProjectCount(BaseModel): email: str role: str is_active: bool + is_service_account: bool = False # bonnyr-f5 #188: provenance for the users listing (UI toggle guard) must_change_password: bool last_login_at: str | None = None created_at: str | None = None diff --git a/backend/services/auth_service.py b/backend/services/auth_service.py index e364ea0..1117682 100644 --- a/backend/services/auth_service.py +++ b/backend/services/auth_service.py @@ -3,14 +3,16 @@ Handles user management, password hashing, and JWT token generation. """ import logging +import secrets from datetime import UTC, datetime, timedelta from typing import Any, cast from jose import JWTError, jwt from passlib.context import CryptContext +from sqlalchemy.exc import IntegrityError from sqlalchemy.orm import Session -from core.config import settings +from core.config import MCP_KNOWN_DEFAULT_PASSWORDS, settings from core.errors import BadRequestError, ConflictError, UnauthorizedError from models import User @@ -34,6 +36,28 @@ def verify_password(plain_password: str, hashed_password: str) -> bool: return cast(bool, pwd_context.verify(plain_password, hashed_password)) +def holds_known_default_password(user: User) -> bool: + """True if the user's stored hash still matches a shipped default password. + + Guards the PUT /api/auth/users/{id} re-enable path: re-activating a service + account that still authenticates with a published default (e.g. + bcrypt("mcp-service-changeme")) would bring that publicly-known credential back + to life, so the re-enable is refused until the secret is rotated to a real one. + + bonnyr-f5 #193 (minor, docstring correction): an earlier version of this + docstring claimed disable_stale_service_user "never touches the hash" and that + this guard therefore covered the row it disabled. That is no longer true — + disable_stale_service_user now SCRUBS the hash (overwrites it with a random + secret) when it deactivates a row, so a row disabled by THAT path no longer + holds a known default and this check would not fire on it. This guard remains + load-bearing for the OTHER disable paths that leave the credential intact — e.g. + a manual operator PUT is_active=false — where the stored hash can still be a + published default. (routes/auth.py:376-379 documents the same split honestly.) + """ + stored = str(user.hashed_password) + return any(verify_password(candidate, stored) for candidate in MCP_KNOWN_DEFAULT_PASSWORDS) + + def create_access_token(data: dict[str, Any], expires_delta: timedelta | None = None) -> str: """Create a JWT access token.""" to_encode = data.copy() @@ -74,6 +98,66 @@ def authenticate_user(db: Session, username: str, password: str) -> User: return user +def token_user_state(token: str) -> User | None: + """#184: resolve the JWT's user for the WebSocket auth gate, or None. + + WebSocket validators (k8s/dpus) authenticate off JWT claims alone and never + load the User, so must_change_password -- and account existence/active state + -- are invisible there. This loads the row so the WS paths enforce the same + gate as get_current_user, from one place. + + Returns the User on success, or None if it cannot be resolved for ANY reason + (invalid/expired token, deleted or disabled account, a transient DB error). + The caller refuses on None: fail CLOSED, so a resolution failure never + re-opens pod exec / BMC SSH the way returning "no change owed" would. + + NOTE: the returned instance is read (``must_change_password``) by the caller + AFTER this session has closed. That is only safe because get_db_context() + closes WITHOUT committing, so the loaded column stays readable on the + detached instance. If get_db_context ever gains a db.commit(), + expire_on_commit=True would expire that attribute and every WebSocket would + then fail closed with no obvious cause -- read must_change_password here, or + disable expire_on_commit, if that changes. + """ + from database import get_db_context + try: + with get_db_context() as db: + return get_user_from_token(db, token) + except Exception: + return None + + +# The only endpoints a must-change user needs before rotating: submit the new +# password, and read their own state so the UI can show the change screen. +# Exact full paths, not suffixes: this is a security gate, so it must not accept +# an unrelated route that merely ends in "/auth/me". +PASSWORD_CHANGE_EXEMPT_PATHS = frozenset({ + "/api/auth/change-password", + "/api/auth/me", +}) + + +def enforce_password_change(path: str, user: User) -> None: + """#184/#186: refuse a must-change user everything but the exempt endpoints. + + Enforced at BOTH auth-resolution points -- the get_current_user dependency + AND AuthMiddleware -- so a route that declares no dependency of its own (there + are ~32 such /api routes) still inherits the gate. Without the middleware half + the seed credential can skip the change-password screen and call those routes + directly (proven: DELETE /api/benchmarks/configs/{id} -> 204 with the seed + token). Raises ForbiddenError; callers translate it to 403. + """ + if not getattr(user, "must_change_password", False): + return + if path.rstrip("/") in PASSWORD_CHANGE_EXEMPT_PATHS: + return + from core.errors import ForbiddenError + raise ForbiddenError( + "Password change required before using the API. " + "POST /api/auth/change-password with your current and new password." + ) + + def get_user_from_token(db: Session, token: str) -> User: """Get the user associated with a JWT token. Raises UnauthorizedError on failure.""" payload = decode_token(token) @@ -134,36 +218,433 @@ def change_password(db: Session, user: User, current_password: str, new_password logger.info(f"Password changed for user: {user.username}") +# Passwords this project has shipped as an admin default at some point. An +# existing account still authenticating with one of these is an upgrade left +# holding a publicly-known credential. +_KNOWN_DEFAULT_ADMIN_PASSWORDS = ("changeme",) + +# The passwords ever shipped as a default for the mcp SERVICE account (role=admin, +# must_change bypassed) are the SAME set as the fail-fast gate's — the exact same +# #184 hazard class as the admin defaults above, just a second account. Published +# in config.py, the compose files, and .env.example, so any account still +# authenticating with one of these holds a publicly-known admin credential. +# ``changeme`` is included because the old shipped compose pointed the MCP client +# at admin/changeme. +# +# bonnyr-f5 #193 (minor): this was a SECOND local copy of the denylist. Asymmetric +# drift was the hazard — a new default added only here would make +# ensure_service_user raise ValueError, which startup_steps swallows as a log line, +# leaving MCP silently dead. The tuple is deleted; ``MCP_KNOWN_DEFAULT_PASSWORDS`` +# (imported from core.config above) is the single Python source of truth, used both +# by validate_production's fail-fast gate and by this seed/rotate path. +# +# Copies OUTSIDE this Python source (deploy-owned, not editable from here — count +# corrected in r4): helm/bnk-forge/templates/secrets.yaml, kept in lockstep by +# scripts/tests/helm-known-defaults-lockstep.test.sh; and a FOURTH copy in +# dist/install.sh (the ``changeme|mcp-service-changeme`` MCP_USABLE gate, ~line 381) +# that sits OUTSIDE that lockstep test. Flagged for the deploy surface; the Python +# gate here is independent of the shell copy, so drift in install.sh cannot weaken +# this path. + +# #186 BLOCKER 3 (bonnyr-f5 r5): usernames that belong to a HUMAN identity and +# must never be resolved by ensure_service_user. That function locates its target +# purely by ``User.username`` and then force-sets role=admin / is_active=True / +# must_change_password=False. If MCP_SERVICE_USERNAME (or the Helm chart's +# mcpUsername) is pointed at "admin", it would REWRITE the human admin row -- +# clearing the #184 must-change gate and handing the mcp secret full admin access +# (probe: "mcp secret now authenticates as admin? YES role=admin must_change=False"). +# Refuse the co-option: a service account may not adopt a reserved human identity. +# (Name kept identical to #188's guard so the two land cleanly on the integration +# branch.) +_RESERVED_HUMAN_USERNAMES = frozenset({"admin"}) + + +def _normalize_service_username(username: str) -> str: + """Canonicalise a configured service username for RESERVED-NAME comparison + only: trim, casefold to lowercase. + + bonnyr-f5 #193 M-2: this is used ONLY by the reserved-name guard + (``_is_reserved_human_username``), so ``Admin`` / ``" admin "`` are refused the + same way the Helm chart refuses them at render (``lower | trim``). It is + deliberately NOT used for the row lookup/create in ``ensure_service_user`` nor + for the ``disable_stale_service_user`` skip filter: those must key on the RAW + ``MCP_SERVICE_USERNAME`` because that is the exact value the MCP client sends as + ``BNK_FORGE_USERNAME`` and ``authenticate_user`` matches exactly. Round-3 + normalised the lookup/create too, which created the row as ``mcp`` while a + ``MCP`` client was denied (the account must match what the client sends). + + NOTE: the Helm chart only lower|trims inside its reserved-name CHECK + (``secrets.yaml:119``); it STORES the raw ``mcpUsername`` (``secrets.yaml:140``), + so the row-name surface is raw on both Helm and compose — this guard governs the + reserved-name surface, which is where the two agree. + """ + return username.strip().lower() + + +def _is_reserved_human_username(username: str) -> bool: + """True if ``username`` collides with a reserved human identity. + + bonnyr-f5 #193 (minor): the Helm guard normalises with ``lower | trim`` before + comparing, so ``mcpUsername: Admin`` or ``" admin "`` is refused at render. + An exact-match Python check let those SAME values through on compose — + ``MCP_SERVICE_USERNAME=Admin`` minted a second role=admin, must_change=False + account. Match Helm: compare case-insensitively after trimming surrounding + whitespace so both surfaces refuse the identical set of inputs. + """ + return _normalize_service_username(username) in _RESERVED_HUMAN_USERNAMES + + +class GeneratedCredentialPersistError(RuntimeError): + """A generated credential could not be written to the keys dir. + + #186 (bonnyr-f5): the docs promise "the plaintext is never logged". The old + code broke that promise — on an unwritable ``/app/keys`` it fell back to + logging the generated plaintext, leaking a live secret into the logs (a real + aggregation-exposure risk). We now fail closed instead: raise this + (WITHOUT the plaintext in the message) so startup refuses to proceed and the + operator remediates. Because the credential is persisted BEFORE the DB row is + created/rotated, a failure leaves nothing committed and the next boot retries + cleanly once the keys volume is writable (or an explicit password env var is + set, which skips generation entirely). + """ + + +def _persist_generated_password(password: str, filename: str = "initial_admin_password") -> str: + """Write a generated credential to a mode-0600 file in the keys dir. + + Returns the path on success. Raises :class:`GeneratedCredentialPersistError` + if the keys dir is unwritable — the plaintext is NEVER logged or included in + the exception, so an unwritable ``/app/keys`` can never leak the secret. The + caller logs a POINTER to the returned path; there is deliberately no + "log the secret instead" fallback. + + The file is created with 0o600 at open() time so the plaintext credential is + never momentarily group/world-readable (open()+chmod would create it 0644 + under the usual umask, then narrow it). O_TRUNC handles a stale file from a + prior seed/rotation without failing. The 0o600 open() mode applies ONLY when + the file is newly created, so a pre-existing file from an older release (e.g. + left 0o644) would be truncated in place but keep its old, looser mode — we + fchmod(0o600) after open to force it tight regardless (CR-5). + + Used by the fresh-install admin seed (#184) and the upgrade remediation + (#186). bonnyr-f5 #193: the ``filename`` parameter is now vestigial — after the + #188-over-#186 consolidation (MCP no longer generates a secret) every call uses + the default, so it only ever writes ``initial_admin_password``. It is kept as a + parameter to preserve the seam should a second generated credential return. + """ + import os + keys_dir = os.environ.get("KEYS_DIR", "/app/keys") + pw_path = os.path.join(keys_dir, filename) + try: + os.makedirs(keys_dir, exist_ok=True) + fd = os.open(pw_path, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600) + # The mode arg above only takes effect on CREATE; force 0o600 so a + # pre-existing looser file (e.g. 0o644 from an older release) is tightened + # before we write the plaintext into it (CR-5). + os.fchmod(fd, 0o600) + with os.fdopen(fd, "w") as fh: + fh.write(password + "\n") + except OSError as exc: # PermissionError/NotADirectoryError are OSError subclasses + # Fail closed. NEVER put `password` in this message: it propagates into + # logs, which is exactly the leak we are closing (#186). + raise GeneratedCredentialPersistError( + f"could not persist generated credential to {pw_path}: {exc}" + ) from exc + return pw_path + + +def _rotate_known_default_admin(db: Session) -> None: + """#186 (bonnyr-f5): make a published default credential UNUSABLE on upgrade. + + A deployment seeded before #184 holds admin/'changeme' with + must_change_password=False. The new seed logic never runs for it (users + already exist), so it keeps the published default. + + Merely flagging must_change_password does NOT remove the capability: + /api/auth/change-password is exempt from the gate and verifies + current_password against the stored hash, so anyone holding the published + 'changeme' (it's in dist/README.md, user-pack/install-guide.html and + scripts/ibm_cloud_bnk_forge.sh) could rotate the password before the operator + does and take over the account. A mitigation must remove the capability, not + request its removal. + + So we OVERWRITE the hash -- the published default stops working the moment + this runs -- and leave the account must_change_password so the replacement + only survives until first login. + + Provenance (bonnyr-f5 r4): the replacement follows the SAME source-of-truth + rule as a fresh seed, so the documented retrieval instructions stay correct + on upgrade too: + * DEFAULT_ADMIN_PASSWORD set to a non-published value (Helm wires it from + the ``admin-password`` Secret) -> rotate TO that value, so the Secret / + env the docs tell operators to read is what now authenticates. No + keys-file is written (nothing was generated). + * otherwise -> generate a fresh random secret and surface it exactly like a + fresh install (mode-0600 keys-file, pointer logged once). + Rotating to a *published* default (e.g. DEFAULT_ADMIN_PASSWORD=changeme) is + refused -- that would just re-publish the hole -- so such a value falls + through to generation. + """ + # #186 (bonnyr-f5 r4, INV-8): lock the row for the read-then-write. Two `api` + # replicas booting together would otherwise both read admin/'changeme', each + # generate a DIFFERENT secret, and interleave file-write vs DB-commit so the + # keys-file and the stored hash end up from different runs -> permanent admin + # lockout. FOR UPDATE serializes them: the loser blocks, then re-reads the + # already-rotated hash (no longer a known default) and no-ops. (Silently + # ignored on SQLite, which the tests use and which has no concurrent writers.) + admin = db.query(User).filter(User.username == "admin").with_for_update().first() + if admin is None: + return + if not any(verify_password(p, admin.hashed_password) for p in _KNOWN_DEFAULT_ADMIN_PASSWORDS): + return + + configured = settings.DEFAULT_ADMIN_PASSWORD + if configured and configured not in _KNOWN_DEFAULT_ADMIN_PASSWORDS: + # Rotate to the operator/chart-supplied secret so the documented source + # (Helm admin-password Secret / DEFAULT_ADMIN_PASSWORD env) is authoritative. + admin.hashed_password = hash_password(configured) # type: ignore[assignment] + admin.must_change_password = True # type: ignore[assignment] + db.commit() + logger.warning( + "Existing 'admin' still held a known shipped default password; " + "OVERWROTE it with DEFAULT_ADMIN_PASSWORD (the published default no " + "longer works) -- retrieve it from the same source you configured " + "(Helm: the admin-password Secret) and change it on first login (#186).", + ) + return + + new_password = secrets.token_urlsafe(18) + # Persist the new secret BEFORE overwriting the hash: if the keys dir is + # unwritable this raises (fail closed, no plaintext logged) with the row's + # published-default hash untouched, so the next boot retries the whole + # remediation cleanly. Never fall back to logging the plaintext (#186). + pw_path = _persist_generated_password(new_password) + admin.hashed_password = hash_password(new_password) # type: ignore[assignment] + admin.must_change_password = True # type: ignore[assignment] + db.commit() + logger.warning( + "Existing 'admin' still held a known shipped default password; " + "OVERWROTE it with a generated secret (the published default no longer " + "works) and wrote the new one to %s -- retrieve it, then change it on " + "first login (#186).", + pw_path, + ) + + def seed_admin_user(db: Session) -> User | None: """Create default admin user if no users exist. Returns the user or None if already exists.""" existing_users = db.query(User).count() if existing_users > 0: + _rotate_known_default_admin(db) # #186: upgrade safety for pre-#184 installs return None - admin = create_user( - db=db, - username="admin", - email="admin@bnk-forge.local", - password=settings.DEFAULT_ADMIN_PASSWORD, - role="admin", - must_change_password=True, - ) + # #184: never seed a known/published default. If DEFAULT_ADMIN_PASSWORD is + # unset, generate a strong random one and persist it to the (mode-600, + # volume-backed) keys dir so the operator can retrieve it. The account is + # must_change_password, so it only survives until first login regardless. + seed_password = settings.DEFAULT_ADMIN_PASSWORD + generated = False + # bonnyr-f5 #193 (minor): the comment above promises we "never seed a + # known/published default", but a supplied DEFAULT_ADMIN_PASSWORD was taken + # verbatim — so an operator who set it to a shipped default (e.g. "changeme") + # got exactly the live, publicly-known admin credential #184 exists to remove. + # Mirror the MCP path: refuse a known default and fall through to generation. + # (_rotate_known_default_admin already refuses one on the upgrade path.) + if seed_password and seed_password in _KNOWN_DEFAULT_ADMIN_PASSWORDS: + logger.warning( + "DEFAULT_ADMIN_PASSWORD is a known published default and will not be " + "seeded (it would be a live, publicly-known admin credential); " + "generating a strong random admin password instead (#193)." + ) + seed_password = None + if not seed_password: + seed_password = secrets.token_urlsafe(18) + generated = True + + # CR-1: fresh-seed race safety. Two `api` replicas booting a fresh install + # both see 0 users and, with DEFAULT_ADMIN_PASSWORD unset, each GENERATE a + # DIFFERENT password. Without a guard, a losing replica would overwrite the + # keys file with its password while its create_user('admin') INSERT loses the + # username UNIQUE constraint and rolls back — file and committed row would hold + # different passwords and the operator would be permanently locked out. + # + # Fix: create + flush FIRST (create_user flushes, so the loser's INSERT raises + # IntegrityError right here), then persist the keys file only after WE won the + # race but BEFORE commit. So (a) a losing replica never touches the file, and + # (b) an unwritable keys dir still fails closed with NO admin row committed + # (the enclosing get_db_context rolls back), so the next boot retries the seed + # cleanly instead of stranding an admin whose generated password nobody can + # read (#186). The keys file can therefore only ever hold the password of the + # row that actually committed. + try: + admin = create_user( + db=db, + username="admin", + email="admin@bnk-forge.local", + password=seed_password, + role="admin", + must_change_password=settings.DEFAULT_ADMIN_MUST_CHANGE, + ) + except (IntegrityError, ConflictError): + # Another replica seeded 'admin' first (concurrent fresh boot). It owns the + # committed row and the matching keys file; we must NOT write our different + # generated password. Roll our aborted transaction back and defer. + db.rollback() + logger.info( + "Another replica seeded the admin user first — skipping (concurrent boot)" + ) + return None + + pw_path = None + if generated: + # We won the create race (row flushed, not yet committed). Persist the + # one-time password now, BEFORE commit: a single boot-log line is easy to + # miss (log rotation, JSON formatting) and logging the plaintext is a known + # aggregation-exposure risk, so we surface a POINTER, never the secret. An + # unwritable keys dir raises here (fail closed, no plaintext logged); we + # roll the flushed-but-uncommitted admin row back so NOTHING is committed + # and the next boot retries the seed cleanly instead of stranding an admin + # whose generated password nobody can read (#186). + try: + pw_path = _persist_generated_password(seed_password) + except GeneratedCredentialPersistError: + db.rollback() + raise + # ENG-006: Startup seed manages its own transaction db.commit() - logger.info("Seeded default admin user — password change required on first login") + # bonnyr-f5 #193 (log/logic consistency): the must-change gate is + # settings.DEFAULT_ADMIN_MUST_CHANGE (passed to create_user above), and an e2e / + # ephemeral deployment sets it false. The log must not promise "you must change + # it on first login" when the account was seeded WITHOUT that gate — that was a + # false instruction. Word the line to match the gate that was actually applied. + must_change = settings.DEFAULT_ADMIN_MUST_CHANGE + if generated: + if must_change: + logger.warning( + "Seeded admin user 'admin' with a GENERATED password, written to " + "%s (retrieve it, then delete it — you must change it on first " + "login). Set DEFAULT_ADMIN_PASSWORD to choose your own instead.", + pw_path, + ) + else: + logger.warning( + "Seeded admin user 'admin' with a GENERATED password, written to " + "%s (retrieve it, then delete it). DEFAULT_ADMIN_MUST_CHANGE is " + "false, so NO first-login change is enforced — this password stays " + "valid until you rotate it; never use this on a real deployment. " + "Set DEFAULT_ADMIN_PASSWORD to choose your own instead.", + pw_path, + ) + else: + if must_change: + logger.info( + "Seeded admin user 'admin' from DEFAULT_ADMIN_PASSWORD — change " + "required on first login" + ) + else: + logger.warning( + "Seeded admin user 'admin' from DEFAULT_ADMIN_PASSWORD with " + "DEFAULT_ADMIN_MUST_CHANGE=false — NO first-login change is " + "enforced (intended only for e2e/ephemeral environments)" + ) return admin -def ensure_service_user(db: Session, username: str, password: str, role: str = "admin") -> None: +# NOTE: _RESERVED_HUMAN_USERNAMES is defined once, above (near the config +# constants), and shared by ensure_service_user below — the #188 and #186 guards +# use the identical frozenset, so the integration keeps a single definition. +def ensure_service_user( + db: Session, username: str, password: str | None, role: str = "admin" +) -> None: """Idempotent create-or-reconcile a non-human service account. - Called unconditionally on every startup so the stored password hash always - matches the current MCP_SERVICE_PASSWORD env var — prevents auth drift when - the env var is rotated without the DB being updated. + Called by ``startup_steps.seed_auth_step`` ONLY when a usable + MCP_SERVICE_PASSWORD is configured (its ``_mcp_pw_usable`` gate: non-empty and + not a known published default). The unset / published-default case is owned + ENTIRELY by :func:`disable_stale_service_user`, which deactivates the account + until an operator configures a real password — this function is never reached + then. It therefore requires a usable (non-empty, non-published-default) + ``password`` and only ever creates/reconciles with it; there is no + generate-on-unset or rotate-on-unset path here (that dead code was removed — + it was unreachable from startup, see #193 review). + + A genuine operator-supplied password is reconciled onto the row so the stored + hash always matches the current MCP_SERVICE_PASSWORD env var — prevents auth + drift when the env var is rotated without the DB being updated. + + Combined credential model (#186 + bonnyr-f5 #188): + * provenance (#188): a freshly-created row is flagged is_service_account so + disable_stale_service_user can find it and a later reconcile can prove it + is ours. A pre-existing row that is NOT a service account is refused — + UNLESS it still authenticates with a shipped published default, which is + by definition a stale service credential from a pre-provenance install + (the v2_155 backfill flags the known legacy 'mcp' row, but a row seeded + under another path may still lack the flag); neutralising it is exactly + the upgrade remediation, so we adopt and reconcile it to the configured + password. + * reserved-name guard (#186/#188): a service account may never co-opt a + human identity such as ``admin``; that is refused before any lookup. + + #186 BLOCKER 1 (bonnyr-f5 r5): the backend now receives MCP_SERVICE_PASSWORD on + every deploy mode (compose backend-env anchors, the ibm installer, and the Helm + shared-env sourced from the release Secret's mcp-password key), so this + reconcile binds the mcp account to the SAME per-install secret the mcp client + uses. + + NOTE (multi-service-account, momentary inactive window): the unconditional + disable-then-reconcile could take the live MCP row inactive for a + bcrypt-plus-commit window on a rolling restart. bonnyr-f5 #193 M2 closes this: + seed_auth_step passes ``skip_username`` so disable_stale_service_user leaves the + row this reconcile will touch untouched. bonnyr-f5 #193 M-2: BOTH sites now key + on the RAW ``MCP_SERVICE_USERNAME`` (the exact value the client sends), so the + skip target and the reconciled row are the same row for any casing. """ - user = db.query(User).filter(User.username == username).first() + # #186 BLOCKER 3 / #188 (bonnyr-f5): fail closed BEFORE any lookup if the + # caller points a service account at a reserved human username. Without this, + # a deployment that sets MCP_SERVICE_USERNAME=admin (or ships the chart's old + # mcpUsername: admin) silently rewrites the human admin row and grants the mcp + # secret admin access. A service account may never co-opt a human identity. + if _is_reserved_human_username(username): + raise ValueError( + f"refusing to reconcile reserved human username '{username}' as a " + f"service account — set MCP_SERVICE_USERNAME to a dedicated name like " + f"'mcp' (a service account must not co-opt the human admin identity)" + ) + + # bonnyr-f5 #193 M-2 (regression fix): do NOT canonicalise the name for the + # lookup/create. The MCP CLIENT receives the RAW MCP_SERVICE_USERNAME as + # BNK_FORGE_USERNAME on every shipped path, and authenticate_user matches + # EXACTLY, so the row MUST be created/reconciled under the value the client will + # actually send (raw). Round-3 normalised here (strip().lower()), which created + # the account as 'mcp' while the client sent 'MCP' -> every non-lowercase + # MCP_SERVICE_USERNAME login was DENIED. Canonicalisation is kept ONLY in the + # reserved-name guard above (so ' Admin ' is still refused). The synthesised + # email below derives from the raw username; the default 'mcp' still yields + # 'mcp@bnk-forge.local' and matches v2_155's fingerprint on the create path. + + # This function only ever runs with a usable password (seed_auth_step's + # _mcp_pw_usable gate). Fail closed and loudly if that contract is violated: + # the unset / published-default case belongs to disable_stale_service_user, + # never here. Seeding/reconciling the role=admin, must-change-EXEMPT mcp + # account to a shipped published default would republish a live, publicly-known + # admin credential (#186). + if not password or password in MCP_KNOWN_DEFAULT_PASSWORDS: + raise ValueError( + f"refusing to seed service account '{username}' without a usable " + f"MCP_SERVICE_PASSWORD (it is unset or a known published default); the " + f"unset case is owned by disable_stale_service_user, not this function." + ) + + # #186 (bonnyr-f5 r4/r5, INV-8): lock the row for the read-then-write on the + # RECONCILE (existing-row) path, so two `api` replicas cannot desync the stored + # hash. No-op on SQLite (tests). with_for_update() cannot lock a not-yet-existing + # row, so the FIRST-CREATE path is serialised by the username UNIQUE constraint + # instead (a losing racer's INSERT raises and that boot's seed retries). + user = db.query(User).filter(User.username == username).with_for_update().first() + if user is None: - create_user( + svc = create_user( db=db, username=username, email=f"{username}@bnk-forge.local", @@ -171,15 +652,157 @@ def ensure_service_user(db: Session, username: str, password: str, role: str = " role=role, must_change_password=False, ) + svc.is_service_account = True # type: ignore[assignment] # provenance (#188) # ENG-006: Startup seed manages its own transaction db.commit() logger.info(f"Created service account: {username} (role={role})") - else: - # Reconcile: update hash to match current env var; never requires current password - user.hashed_password = hash_password(password) # type: ignore[assignment] + return + + # Existing row. bonnyr-f5 #188: refuse to reconcile a row this seeder did NOT + # provision as a service account — a name collision must never take over a + # human account. Gate on provenance, not the username. EXCEPTION (#186 + # integration, scoped by bonnyr-f5 #193 B1): a non-service row is adopted ONLY + # when it carries v2_155's exact backfill fingerprint — username 'mcp' AND + # email 'mcp@bnk-forge.local'. That is a stale service credential from a + # pre-provenance install (the migration deliberately used the same conservative + # rule so "a real human who merely happens to be named mcp is left untouched"), + # so neutralising it is the upgrade remediation. Keying the exception on the + # password value instead (as the pre-#193 code did) adopted ANY human row whose + # password was `changeme` — a takeover of the human account (bonnyr-f5 #193 B1). + # bonnyr-f5 #193 (minor): why this branch is KEPT even though v2_155 backfills + # is_service_account=True on the legacy 'mcp'/'mcp@bnk-forge.local' row (which + # would send that row down the is_service_account=True path, not here). It is + # defence-in-depth for the ordering where seeding meets a row that carries the + # legacy fingerprint but NOT the flag: migrations disabled or lagging behind the + # app, a fresh test DB seeded without running v2_155, or a row created under a + # pre-provenance path. In production v2_155 runs before seeding, so this is + # normally unreachable; keeping it costs nothing and closes the ordering gap. + is_adoption = False + if not user.is_service_account: + fingerprint_match = ( + user.username == "mcp" and str(user.email) == "mcp@bnk-forge.local" + ) + holds_published_default = fingerprint_match and any( + verify_password(p, str(user.hashed_password)) + for p in MCP_KNOWN_DEFAULT_PASSWORDS + ) + if not holds_published_default: + raise ValueError( + f"refusing to reconcile '{username}': it is not a service account. " + f"Point MCP_SERVICE_USERNAME at a dedicated name that isn't an " + f"existing user." + ) + is_adoption = True + + # Operator supplied a genuine (non-default) password: reconcile to it. + # Re-activation is required and safe (disable_stale_service_user runs first on + # every boot and may have deactivated this row; its docstring promises that + # configuring a real password "re-seeds and re-activates it"). Role is left + # untouched — we do NOT widen privilege on reconcile (bonnyr-f5 #188). + user.hashed_password = hash_password(password) # type: ignore[assignment] + # bonnyr-f5 #193 B1: only clear the must-change gate on a genuine service row + # (provenance already set). Never clear it as a side effect of ADOPTING a row — + # doing so would defeat #184/#186's must-change gate on the adopted account. + if not is_adoption: user.must_change_password = False # type: ignore[assignment] - user.role = role # type: ignore[assignment] - user.is_active = True # type: ignore[assignment] - # ENG-006: Startup seed manages its own transaction - db.commit() - logger.info(f"Reconciled service account: {username}") + user.is_active = True # type: ignore[assignment] # revive a disabled-stale row + user.is_service_account = True # type: ignore[assignment] # adopt a backfilled/legacy row + db.commit() # ENG-006: Startup seed manages its own transaction + logger.info(f"Reconciled service account: {username}") + + +def disable_stale_service_user( + db: Session, + skip_username: str | None = None, + password_configured: bool = False, +) -> None: + """#188 (bonnyr-f5): a service account seeded by a prior release still holds + the shipped 'mcp-service-changeme' default and keeps authenticating on upgrade. + Deactivate EVERY active service-account row so no known default can be used + until the operator configures a real password (which re-seeds and re-activates + the account). + + Round 5 (BLOCKER-1): seed_auth_step now calls this UNCONDITIONALLY, before the + reconcile — not only on the no-password path. The reconcile is name-keyed, so + on the diligent-operator path (strong MCP_SERVICE_PASSWORD but MCP_USERNAME + left at the legacy 'admin') it raises a reserved-name ValueError and never + reaches a disable; running this first is what closes that hole. When a usable + password IS set for a dedicated username, the reconcile re-activates that one + row immediately after, so the net effect is: exactly the configured service + account stays active, every stale default is revoked. + + Keyed on provenance (is_service_account), NOT on the configured username + (bonnyr-f5 #188 round 4, INV-11): on the dist/IBM upgrade path + MCP_SERVICE_USERNAME resolves from a legacy .env to 'admin', so matching the + configured name would early-return and leave the stale 'mcp' row still + authenticating with the shipped default. The provenance flag is set only on + rows this seeder created, never on a human account, so disabling all service + accounts can never touch a human login — which is also why no reserved-username + guard is needed (or wanted: that guard is exactly what made this a no-op). + + bonnyr-f5 #193 M2: ``skip_username`` leaves the row the caller is about to + reconcile untouched, so a correctly-configured install never commits an + inactive window for its live MCP account (a rolling restart would otherwise + 401 live MCP traffic for a bcrypt-plus-commit window). ``password_configured`` + only tunes the log wording: when a usable MCP_SERVICE_PASSWORD IS set, any row + still disabled here is a genuinely stale EXTRA service account, not the "no + password is set" case -- so the misleading "no usable MCP_SERVICE_PASSWORD is + set" warning no longer fires on every boot of a correct install. + + bonnyr-f5 #193 (minor — the v2_155 "point MCP_SERVICE_USERNAME elsewhere" + remedy, stated precisely): because this disable is provenance-keyed and runs + UNCONDITIONALLY before the reconcile, the DEFAULT-named legacy row + ('mcp'/'mcp@bnk-forge.local' — the one v2_155 backfills is_service_account) IS + deactivated and hash-scrubbed on every boot even when MCP_SERVICE_USERNAME now + points at a different name (it is not the skip target, so it is not skipped). So + the documented remedy DOES revoke that stale default. The residual it does NOT + cover: a service account a pre-provenance release created under a CUSTOM + username (never the default 'mcp'). v2_155 deliberately will not backfill such a + row — it cannot prove the row is a service account rather than a human of that + name, and reclassifying a human is strictly worse — so is_service_account stays + False and this filter never sees it. That row must be disabled or deleted by + hand; pointing MCP_SERVICE_USERNAME elsewhere does not neutralise it. + """ + query = db.query(User).filter( + User.is_active.is_(True), + User.is_service_account.is_(True), # bonnyr-f5 #188: never a human row + ) + if skip_username is not None: + # bonnyr-f5 #193 M-2: key the skip on the RAW MCP_SERVICE_USERNAME — the + # exact value ensure_service_user reconciles the row under — so the live row + # is skipped (not nuked into an inactive window). Normalising here would + # break that: a non-lowercase MCP_SERVICE_USERNAME=MCP reconciles a 'MCP' + # row while a normalised skip='mcp' would fail to protect it (and would also + # spare a differently-cased legacy default row that must be disabled). + query = query.filter(User.username != skip_username) + rows = query.all() + if not rows: + return + for user in rows: + user.is_active = False # type: ignore[assignment] + # bonnyr-f5 #193 (minor): don't leave the (possibly published-default) + # hash intact and lean solely on is_active + the login route guard. + # Neutralise the credential too, so the stale default cannot authenticate + # even if some future caller checks the password before is_active, or the + # row is flipped active again out of band. Re-enabling the account always + # goes through ensure_service_user, which reconciles the hash to + # MCP_SERVICE_PASSWORD, so overwriting it here loses nothing recoverable. + user.hashed_password = hash_password(secrets.token_urlsafe(32)) # type: ignore[assignment] + db.commit() + for user in rows: + if password_configured: + logger.warning( + "Disabled extra stale MCP service account '%s' -- a usable " + "MCP_SERVICE_PASSWORD is configured for '%s', so this additional " + "service row's pre-existing credential must not keep authenticating.", + user.username, + skip_username, + ) + else: + logger.warning( + "Disabled stale MCP service account '%s' -- no usable " + "MCP_SERVICE_PASSWORD is set, so its pre-existing (possibly default) " + "credential must not keep authenticating. Set MCP_SERVICE_PASSWORD to " + "re-enable MCP.", + user.username, + ) diff --git a/backend/services/backup_service.py b/backend/services/backup_service.py index cc6de0f..daa8f37 100644 --- a/backend/services/backup_service.py +++ b/backend/services/backup_service.py @@ -567,6 +567,19 @@ def _replace_encryption_key(self, wrapped_key_path: str, passphrase: str) -> Non with open(key_path, "wb") as f: f.write(raw_key) + # bonnyr-f5 #193 B-3 (r4 self-review): a restored key is operator-provisioned + # (the caller supplied the wrapping passphrase). Drop the `.operator` provenance + # marker beside it so the next boot classifies it operator-provided and + # validate_production passes -- without it, the restored (marker-less) key would + # be treated as auto-generated and fail the production gate. config.Settings + # never overwrites this file, so the restored key survives and stays decryptable. + marker_path = key_path + ".operator" + try: + with open(marker_path, "w") as mf: + mf.write("") + except OSError as e: + logger.warning("Could not write provenance marker %s: %s", marker_path, e) + logger.info("Encryption key replaced at %s", key_path) # ------------------------------------------------------------------ # diff --git a/backend/services/execution/container_runner.py b/backend/services/execution/container_runner.py index 79c2f61..ef1434e 100644 --- a/backend/services/execution/container_runner.py +++ b/backend/services/execution/container_runner.py @@ -550,7 +550,9 @@ def is_root_user(image_user: str | None) -> bool: An image that never declares USER reports an empty string and runs as root — that is the common case and must be caught. - Closes the numeric bypass only — see the KNOWN GAP note in the body. + Fails closed on anything that is not a bare non-zero decimal uid, + which also subsumes the named-alias case (see the body) — there is no + remaining KNOWN GAP. Only the uid half decides this. Docker's USER is ``[:]``, so an image declaring ``USER 0:100`` or ``USER root:wheel`` runs as uid 0 @@ -647,7 +649,8 @@ def _fail(message: str, stdout: str = "") -> StepResult: f"Artifact image {spec.image_digest} runs as root " f"(USER={image_user or ''}). Refusing to start it: the workspace is " f"mounted from the host, so a root container is a host-root write primitive. " - f"Rebuild the image with a NUMERIC non-root USER (e.g. `USER 65532`). " + f"Rebuild the image with a NUMERIC non-root USER — `USER 1000` matches " + f"the workspace owner (chowned 1000:1000), so the step can write it. " f"A named user is refused because it cannot be resolved to a uid " f"without the image's own /etc/passwd — `USER toor` may well be uid 0. " f"The Kubernetes substrate already enforces this: runAsNonRoot is " diff --git a/backend/startup_steps.py b/backend/startup_steps.py index 29afd40..ad3eed5 100644 --- a/backend/startup_steps.py +++ b/backend/startup_steps.py @@ -215,23 +215,95 @@ def seed_deployable_releases_step(): def seed_auth_step(): """Seed default admin user if no users exist; always reconcile MCP service account.""" from database import get_db_context - from services.auth_service import ensure_service_user, seed_admin_user - with get_db_context() as db: - admin = seed_admin_user(db) - if admin: - logger.info(" Created default admin user — change password on first login") - logger.info(" See docs/INSTALLATION.md for first-login instructions") - else: - logger.info(" Users already exist") + from services.auth_service import ( + GeneratedCredentialPersistError, + ensure_service_user, + seed_admin_user, + ) + try: + with get_db_context() as db: + admin = seed_admin_user(db) + if admin: + logger.info(" Created default admin user — change password on first login") + logger.info(" See docs/INSTALLATION.md for first-login instructions") + else: + logger.info(" Users already exist") + + + # bonnyr-f5 #188: treat a shipped known default (changeme) as "unset" so a + # dist/IBM upgrade doesn't re-seed the mcp account to a known password. + from core.config import MCP_KNOWN_DEFAULT_PASSWORDS + _mcp_pw_usable = bool(settings.MCP_SERVICE_PASSWORD) and settings.MCP_SERVICE_PASSWORD not in MCP_KNOWN_DEFAULT_PASSWORDS + + # bonnyr-f5 #188 round 5 (BLOCKER-1): disable stale service accounts + # UNCONDITIONALLY, before any reconcile — never only on the no-password + # path. The reconcile below touches ONLY the row whose name matches + # MCP_SERVICE_USERNAME; on the diligent-operator upgrade path that name + # resolves from a legacy .env to 'admin', so ensure_service_user raises a + # reserved-name ValueError and returns WITHOUT disabling the legacy 'mcp' + # row — leaving it active with the shipped default even though the operator + # did the right thing. Running the provenance-keyed disable first (round 4, + # INV-11: keyed on is_service_account, not the configured username) neutralises + # every stale default; the reconcile then re-activates the one account whose + # credentials we actually manage. + # bonnyr-f5 #193 M2: when a usable password IS configured, skip the row we + # are about to reconcile so we never commit an inactive window for the live + # MCP account (a rolling restart would otherwise 401 live MCP traffic), and + # suppress the misleading "no usable MCP_SERVICE_PASSWORD is set" warning + # that used to fire on every boot of a correctly-configured install. + from services.auth_service import disable_stale_service_user + with get_db_context() as db: + disable_stale_service_user( + db, + skip_username=settings.MCP_SERVICE_USERNAME if _mcp_pw_usable else None, + password_configured=_mcp_pw_usable, + ) - # Unconditional: ensure MCP service account exists and its password hash matches - # current MCP_SERVICE_PASSWORD — prevents auth drift when the env var is rotated. - with get_db_context() as db: - ensure_service_user( - db, - username=settings.MCP_SERVICE_USERNAME, - password=settings.MCP_SERVICE_PASSWORD, - ) + # #187/#188: only reconcile when a real password is configured; never seed + # the account with a shipped default. When unset, MCP is simply unavailable + # (the stale default row was already disabled above) until an operator sets + # MCP_SERVICE_PASSWORD (and gives the MCP server the same value). When it IS + # set, ensure_service_user reconciles the stored hash to it — preventing auth + # drift when the env var is rotated. + # bonnyr-f5 #193 M1 (DECISION): this is a deliberate consolidation — #188's + # "unset MCP_SERVICE_PASSWORD -> account disabled" is chosen over #186's + # "unset -> generate a retrievable secret". The generate path is intentionally + # NOT restored: an MCP secret is a shared secret the MCP *client* must also + # hold, so a backend-only generated value cannot be surfaced to it. Do not + # re-add a generate-on-unset fallback here without re-opening that decision. + if _mcp_pw_usable: + try: + with get_db_context() as db: + ensure_service_user( + db, + username=settings.MCP_SERVICE_USERNAME, + password=settings.MCP_SERVICE_PASSWORD, + ) + except ValueError as exc: + # Reserved-username refusal (e.g. MCP_USERNAME still 'admin'): loud, + # not fatal — MCP stays down but the human admin is not taken over, + # and the stale default row was already disabled above. + logger.error(" MCP service account NOT seeded: %s", exc) + else: + logger.warning( + " MCP_SERVICE_PASSWORD is not set — MCP service account not seeded; " + "the MCP server will be unable to authenticate until you set it" + ) + except GeneratedCredentialPersistError as exc: + # #186 (bonnyr-f5): a generated admin/service credential could not be + # written to the keys dir. We refuse to fall back to LOGGING the plaintext + # (a real secret-into-logs leak). Fail closed instead: SystemExit escapes + # the best-effort step handler in main.py (which only catches Exception), + # so the process refuses to start rather than run with an unretrievable + # generated credential — no plaintext ever reaches the logs. The operator + # makes the keys volume (KEYS_DIR, default /app/keys) writable, or sets an + # explicit DEFAULT_ADMIN_PASSWORD / MCP_SERVICE_PASSWORD (which skips + # generation entirely), then restarts. + raise SystemExit( + f"Cannot start: {exc}. Refusing to log the generated plaintext secret. " + "Make the keys volume (KEYS_DIR, default /app/keys) writable, or set " + "DEFAULT_ADMIN_PASSWORD / MCP_SERVICE_PASSWORD, then restart." + ) from exc if settings.REQUIRE_AUTH: logger.info(" Authentication ENABLED (REQUIRE_AUTH=true)") diff --git a/backend/tests/component/test_auth_service.py b/backend/tests/component/test_auth_service.py index 33a194b..280a8ee 100644 --- a/backend/tests/component/test_auth_service.py +++ b/backend/tests/component/test_auth_service.py @@ -9,6 +9,7 @@ from unittest.mock import patch import pytest +from sqlalchemy.exc import IntegrityError from core.config import settings from core.errors import BadRequestError, ConflictError, UnauthorizedError @@ -179,6 +180,12 @@ def test_disabled_user_raises(self, db): class TestSeedAdminUser: + @pytest.fixture(autouse=True) + def _isolate_keys_dir(self, monkeypatch, tmp_path): + # seed_admin_user may generate + persist a password to KEYS_DIR; keep it + # out of the working tree (default is /app/keys) for every test here. + monkeypatch.setenv("KEYS_DIR", str(tmp_path)) + def test_seeds_when_no_users(self, db): admin = seed_admin_user(db) assert admin is not None @@ -192,16 +199,246 @@ def test_returns_none_when_users_exist(self, db): result = seed_admin_user(db) assert result is None - def test_seeded_admin_can_login(self, db): + def test_seeded_admin_can_login_with_explicit_password(self, db, monkeypatch): + # When DEFAULT_ADMIN_PASSWORD is set, the seed uses it. + monkeypatch.setattr(settings, "DEFAULT_ADMIN_PASSWORD", "explicit-admin-pw") seed_admin_user(db) - user = authenticate_user(db, "admin", settings.DEFAULT_ADMIN_PASSWORD) + user = authenticate_user(db, "admin", "explicit-admin-pw") assert user.username == "admin" + def test_seeded_admin_generates_random_password_when_unset(self, db, monkeypatch, tmp_path): + # #184: with DEFAULT_ADMIN_PASSWORD unset, the seed must NOT use a known + # default -- it generates a random one, so the published "changeme" + # never authenticates. KEYS_DIR -> tmp so the generated-password file + # doesn't land in the working tree. + monkeypatch.setattr(settings, "DEFAULT_ADMIN_PASSWORD", None) + admin = seed_admin_user(db) + assert (tmp_path / "initial_admin_password").exists() + assert admin is not None + assert admin.must_change_password is True + with pytest.raises(UnauthorizedError): + authenticate_user(db, "admin", "changeme") + + def test_generated_password_file_is_mode_0600(self, db, monkeypatch, tmp_path): + # #186 (bonnyr-f5): the commit is titled "harden the password-file mode" + # but only .exists() was asserted. The plaintext credential must be 0600, + # never group/world-readable. + import os + import stat + monkeypatch.setattr(settings, "DEFAULT_ADMIN_PASSWORD", None) + seed_admin_user(db) + pw = tmp_path / "initial_admin_password" + mode = stat.S_IMODE(os.stat(pw).st_mode) + assert mode == 0o600, f"expected 0o600, got {oct(mode)}" + + def test_persist_tightens_a_preexisting_0644_file_to_0600(self, monkeypatch, tmp_path): + # CR-5: os.open's mode arg applies ONLY on create. A pre-existing 0644 file + # from an older release would be truncated in place but keep 0644 — writing + # the generated secret world-readable. _persist_generated_password must + # fchmod it to 0600 regardless of the prior mode. + import os + import stat + + from services.auth_service import _persist_generated_password + monkeypatch.setenv("KEYS_DIR", str(tmp_path)) + stale = tmp_path / "initial_admin_password" + stale.write_text("old-secret\n") + os.chmod(stale, 0o644) + assert stat.S_IMODE(stale.stat().st_mode) == 0o644 # precondition + path = _persist_generated_password("brand-new-secret") + assert stat.S_IMODE(os.stat(path).st_mode) == 0o600 # tightened + with open(path) as fh: + assert fh.read().strip() == "brand-new-secret" + + def test_losing_replica_does_not_clobber_keys_file(self, db, monkeypatch, tmp_path): + # CR-1: on a concurrent fresh boot with DEFAULT_ADMIN_PASSWORD unset, both + # replicas see 0 users and each GENERATES a different password. The losing + # replica's create_user('admin') INSERT loses the username UNIQUE + # constraint (IntegrityError). It must roll back and NOT overwrite the keys + # file, or the file (loser's pw) and the committed row (winner's pw) would + # disagree and permanently lock the operator out. + import services.auth_service as auth_mod + from models import User + monkeypatch.setattr(settings, "DEFAULT_ADMIN_PASSWORD", None) + monkeypatch.setenv("KEYS_DIR", str(tmp_path)) + # The winner already committed its row and persisted the matching file. + key_file = tmp_path / "initial_admin_password" + key_file.write_text("winner-password-from-replica-A\n") + + def _boom(*_a, **_k): + # Simulate the losing INSERT: create_user flushes and the UNIQUE + # constraint fires. + raise IntegrityError("INSERT INTO users", {}, Exception("duplicate username")) + monkeypatch.setattr(auth_mod, "create_user", _boom) + + result = seed_admin_user(db) + assert result is None # deferred to the winning replica + # The keys file was NOT clobbered with the loser's generated password. + assert key_file.read_text() == "winner-password-from-replica-A\n" + # The aborted transaction left no half-seeded row. + assert db.query(User).filter(User.username == "admin").count() == 0 + + def test_rotates_existing_admin_still_on_a_known_default(self, db, tmp_path): + # #186 (bonnyr-f5): an upgrade left admin/'changeme' with + # must_change_password=False -- the seed logic never re-runs for it. On + # boot, seed_admin_user (users exist -> None) must INVALIDATE the + # published default, not merely flag it: /api/auth/change-password is + # exempt from the gate and verifies against the stored hash, so a flag + # alone leaves 'changeme' usable to rotate the account. The hash must be + # overwritten and a fresh secret surfaced like a fresh install. + from models.system import User + create_user(db, "admin", "admin@bnk-forge.local", "changeme", + role="admin", must_change_password=False) + db.commit() + assert seed_admin_user(db) is None + admin = db.query(User).filter(User.username == "admin").first() + assert admin.must_change_password is True + # The published default no longer authenticates -- capability removed. + with pytest.raises(UnauthorizedError): + authenticate_user(db, "admin", "changeme") + # A fresh generated secret was surfaced exactly like a fresh install. + pw_file = tmp_path / "initial_admin_password" + assert pw_file.exists() + new_pw = pw_file.read_text().strip() + assert new_pw and new_pw != "changeme" + assert authenticate_user(db, "admin", new_pw).username == "admin" + + def test_rotation_is_idempotent_across_boots(self, db, tmp_path): + # #186: after the one-time overwrite the stored password is the generated + # secret, so a second boot's verify("changeme", ...) is False and the + # account is left untouched (no re-rotation, no new file churn). + from models.system import User + create_user(db, "admin", "admin@bnk-forge.local", "changeme", + role="admin", must_change_password=False) + db.commit() + seed_admin_user(db) + first_pw = (tmp_path / "initial_admin_password").read_text().strip() + seed_admin_user(db) # second boot + admin = db.query(User).filter(User.username == "admin").first() + # Still the same generated secret from the first rotation. + assert authenticate_user(db, "admin", first_pw).username == "admin" + assert admin.must_change_password is True + + def test_does_not_touch_an_admin_with_a_real_password(self, db): + from models.system import User + create_user(db, "admin", "admin@bnk-forge.local", "a-Strong-Real-Pw-1", + role="admin", must_change_password=False) + db.commit() + seed_admin_user(db) + admin = db.query(User).filter(User.username == "admin").first() + assert admin.must_change_password is False # not a known default → untouched + + def test_rotation_honors_default_admin_password_when_set(self, db, monkeypatch, tmp_path): + # #186 (bonnyr-f5 r4) provenance: when DEFAULT_ADMIN_PASSWORD is set + # (Helm wires it from the admin-password Secret), the upgrade rotation + # must rotate TO that value so the documented source-of-truth (the + # Secret / env) authenticates -- and it must NOT write a keys-file + # (nothing was generated), so the docs' Helm "read the Secret" + # instruction stays correct on upgrade. + from models.system import User + monkeypatch.setattr(settings, "DEFAULT_ADMIN_PASSWORD", "chart-supplied-secret-x") + create_user(db, "admin", "admin@bnk-forge.local", "changeme", + role="admin", must_change_password=False) + db.commit() + assert seed_admin_user(db) is None + admin = db.query(User).filter(User.username == "admin").first() + assert admin.must_change_password is True + with pytest.raises(UnauthorizedError): + authenticate_user(db, "admin", "changeme") # published default gone + # The configured value now authenticates (Secret == source of truth). + assert authenticate_user(db, "admin", "chart-supplied-secret-x").username == "admin" + # No keys-file written: nothing was generated. + assert not (tmp_path / "initial_admin_password").exists() + + def test_must_change_false_seeds_without_gate_and_log_is_honest( + self, db, monkeypatch, tmp_path, caplog + ): + # bonnyr-f5 #193 test-gap 5: DEFAULT_ADMIN_MUST_CHANGE=false with an unset + # DEFAULT_ADMIN_PASSWORD seeds a GENERATED password AND no must-change gate. + # The old log unconditionally said "you must change it on first login" — a + # false instruction. The account must be seeded without the gate, and the + # log must NOT promise a first-login change that is not enforced. + import logging + monkeypatch.setattr(settings, "DEFAULT_ADMIN_PASSWORD", None) + monkeypatch.setattr(settings, "DEFAULT_ADMIN_MUST_CHANGE", False) + caplog.set_level(logging.INFO) + admin = seed_admin_user(db) + # Logic: no must-change gate was applied. + assert admin is not None + assert admin.must_change_password is False + assert (tmp_path / "initial_admin_password").exists() + # Log: does not promise a first-login change; says the gate is off. + seed_logs = " ".join( + r.getMessage() for r in caplog.records if "Seeded admin" in r.getMessage() + ) + assert seed_logs, "expected a 'Seeded admin' log line" + assert "must change it on first login" not in seed_logs.lower() + assert "change it on first login" not in seed_logs.lower() + assert "DEFAULT_ADMIN_MUST_CHANGE is false" in seed_logs + + def test_must_change_true_log_still_instructs_first_login_change( + self, db, monkeypatch, tmp_path, caplog + ): + # The complement: with the gate ON (default) the instruction is correct and + # must remain. + import logging + monkeypatch.setattr(settings, "DEFAULT_ADMIN_PASSWORD", None) + monkeypatch.setattr(settings, "DEFAULT_ADMIN_MUST_CHANGE", True) + caplog.set_level(logging.INFO) + admin = seed_admin_user(db) + assert admin.must_change_password is True + seed_logs = " ".join( + r.getMessage() for r in caplog.records if "Seeded admin" in r.getMessage() + ) + assert "change it on first login" in seed_logs.lower() + + def test_rotation_refuses_to_rotate_to_a_published_default(self, db, monkeypatch, tmp_path): + # #186: DEFAULT_ADMIN_PASSWORD=changeme must NOT be used as the rotation + # target (that would re-publish the hole) -- fall through to a generated + # keys-file secret instead. + from models.system import User + monkeypatch.setattr(settings, "DEFAULT_ADMIN_PASSWORD", "changeme") + create_user(db, "admin", "admin@bnk-forge.local", "changeme", + role="admin", must_change_password=False) + db.commit() + seed_admin_user(db) + admin = db.query(User).filter(User.username == "admin").first() + with pytest.raises(UnauthorizedError): + authenticate_user(db, "admin", "changeme") + pw_file = tmp_path / "initial_admin_password" + assert pw_file.exists() + assert authenticate_user(db, "admin", pw_file.read_text().strip()).username == "admin" + # ── ensure_service_user ────────────────────────────────────────────── +def _seed_legacy_stale_service_row(db, username="mcp", password="mcp-service-changeme"): + """Build a pre-fix / pre-provenance service-account row directly. + + An older release seeded the 'mcp' account holding the shipped published + default; the v2_155 backfill flags the known legacy row is_service_account on + upgrade. ensure_service_user NO LONGER creates such a row (it now refuses a + published default — that dead generate/rotate-on-unset path was removed in the + #193 review), so tests that need a stale service row build it here. + """ + from services.auth_service import create_user, hash_password + user = create_user(db, username, f"{username}@bnk-forge.local", password, + role="admin", must_change_password=False) + user.hashed_password = hash_password(password) + user.is_service_account = True + db.commit() + return user + + class TestEnsureServiceUser: + @pytest.fixture(autouse=True) + def _isolate_keys_dir(self, monkeypatch, tmp_path): + # ensure_service_user may generate + persist a secret to KEYS_DIR; keep it + # out of the working tree (default /app/keys). Persist now fails closed on + # an unwritable dir, so a writable KEYS_DIR is required for these tests. + monkeypatch.setenv("KEYS_DIR", str(tmp_path)) + def test_creates_service_user_when_absent(self, db): ensure_service_user(db, username="mcp", password="secret") user = authenticate_user(db, "mcp", "secret") @@ -227,3 +464,530 @@ def test_idempotent_create(self, db): from models import User count = db.query(User).filter(User.username == "mcp").count() assert count == 1 + + def test_refuses_to_reconcile_the_human_admin(self, db): + # #188 (bonnyr-f5): MCP_USERNAME still 'admin' on an old .env would point + # ensure_service_user at the human admin row and take it over (rewrite + # hash, clear must_change). Refuse, and leave the admin row untouched. + from services.auth_service import create_user, verify_password + create_user(db, "admin", "admin@bnk-forge.local", "human-admin-pw", + role="admin", must_change_password=True) + db.commit() + with pytest.raises(ValueError, match="reserved human username"): + ensure_service_user(db, username="admin", password="mcp-secret") + from models import User + admin = db.query(User).filter(User.username == "admin").first() + assert admin.must_change_password is True # gate not cleared + assert verify_password("human-admin-pw", admin.hashed_password) # hash intact + + def test_disable_stale_service_user_deactivates_mcp(self, db): + # #188: upgrade with MCP_SERVICE_PASSWORD unset must not leave the old + # mcp/'mcp-service-changeme' account authenticating. + from services.auth_service import disable_stale_service_user + _seed_legacy_stale_service_row(db) # sets is_service_account + disable_stale_service_user(db) + from models import User + assert db.query(User).filter(User.username == "mcp").first().is_active is False + + def test_disable_stale_is_keyed_on_provenance_not_configured_username(self, db): + # bonnyr-f5 #188 round 4 (INV-11): on the dist/IBM upgrade path + # MCP_SERVICE_USERNAME resolves from a legacy .env to 'admin', so a + # name-keyed disable early-returned and left the legacy 'mcp' service row + # (mcp-service-changeme, role=admin) still authenticating. The disable must + # deactivate the service account by provenance regardless of the configured + # name, while never touching the human admin. + from models import User + from services.auth_service import ( + authenticate_user, + create_user, + disable_stale_service_user, + ) + _seed_legacy_stale_service_row(db) # legacy service row + create_user(db, "admin", "admin@bnk-forge.local", "human-admin-pw", role="admin") + db.commit() + disable_stale_service_user(db) # startup no longer passes a username at all + assert db.query(User).filter(User.username == "mcp").first().is_active is False + assert db.query(User).filter(User.username == "admin").first().is_active is True + with pytest.raises(UnauthorizedError): + authenticate_user(db, "mcp", "mcp-service-changeme") # default no longer works + # Human admin login is untouched. + assert authenticate_user(db, "admin", "human-admin-pw").username == "admin" + + def test_refuses_to_reconcile_a_non_reserved_human(self, db): + # bonnyr-f5 #188 r2: the guard was a one-name denylist. Point the service + # username at ANY existing human row (here 'operator') and the reconcile + # would take it over. Provenance (is_service_account) refuses it. + from services.auth_service import create_user, verify_password + create_user(db, "operator", "operator@bnk-forge.local", "human-op-pw", + role="operator", must_change_password=False) + db.commit() + with pytest.raises(ValueError, match="not a service account"): + ensure_service_user(db, username="operator", password="mcp-secret") + from models import User + op = db.query(User).filter(User.username == "operator").first() + assert op.role == "operator" # NOT promoted to admin + assert verify_password("human-op-pw", op.hashed_password) # password intact + + def test_disable_stale_never_touches_admin(self, db): + # A human admin carries is_service_account=False, so provenance-keyed + # disable leaves it active even though its name is 'admin'. + from services.auth_service import create_user, disable_stale_service_user + create_user(db, "admin", "admin@bnk-forge.local", "pw", role="admin") + db.commit() + disable_stale_service_user(db) # provenance-keyed -> human admin untouched + from models import User + assert db.query(User).filter(User.username == "admin").first().is_active is True + + def test_reconcile_reactivates_a_disabled_stale_service_account(self, db): + # bonnyr-f5 #188 BLOCKER 3: disable_stale_service_user deactivates the mcp + # row when no real password is set; setting a real MCP_SERVICE_PASSWORD must + # then re-seed AND re-activate it (as its docstring promises). Without the + # reactivation the account stays is_active=False and every MCP login fails + # with "Account is disabled" despite a correct password. + from models import User + from services.auth_service import disable_stale_service_user, ensure_service_user + _seed_legacy_stale_service_row(db) + disable_stale_service_user(db) + assert db.query(User).filter(User.username == "mcp").first().is_active is False + # Operator now configures a real secret -> reconcile must revive the account. + ensure_service_user(db, username="mcp", password="a-real-strong-secret") + mcp = db.query(User).filter(User.username == "mcp").first() + assert mcp.is_active is True # re-activated + assert mcp.is_service_account is True # provenance preserved + # And the new secret authenticates while the old default does not. + assert authenticate_user(db, "mcp", "a-real-strong-secret").username == "mcp" + with pytest.raises(UnauthorizedError): + authenticate_user(db, "mcp", "mcp-service-changeme") + # ── #186 BLOCKER 1: the published mcp default must never authenticate ── + + @pytest.mark.parametrize("published_default", ["mcp-service-changeme", "changeme"]) + def test_published_default_seed_is_refused(self, db, published_default): + """#193: seeding the mcp account with a shipped published default is + REFUSED (ValueError) and creates no row. The merged model treats unset / + published-default as "not usable"; that case is owned by + disable_stale_service_user, never seeded here — the old generate-on-default + path was removed. seed_auth_step's _mcp_pw_usable gate means this branch is + only ever reached defensively, but it must still fail closed.""" + from models import User + with pytest.raises(ValueError, match="usable MCP_SERVICE_PASSWORD"): + ensure_service_user(db, username="mcp", password=published_default) + assert db.query(User).filter(User.username == "mcp").count() == 0 + + def test_none_password_is_refused(self, db): + """#193: MCP_SERVICE_PASSWORD unset (None) is REFUSED and creates no row. + The unset case is handled by disable_stale_service_user (account left + disabled), not by generating a secret here — that dead path was removed.""" + from models import User + with pytest.raises(ValueError, match="usable MCP_SERVICE_PASSWORD"): + ensure_service_user(db, username="mcp", password=None) + assert db.query(User).filter(User.username == "mcp").count() == 0 + + def test_upgrade_disables_backfilled_published_default(self, db): + """An account carried over from a pre-fix install still holding the + published default is neutralised on upgrade by disable_stale_service_user + (unset MCP_SERVICE_PASSWORD path): the v2_155 backfill flags it + is_service_account, and the provenance-keyed disable deactivates it so the + published default can no longer authenticate. ensure_service_user is NOT + called on the unset path — disable owns it.""" + from core.errors import UnauthorizedError as UnauthError + from services.auth_service import disable_stale_service_user + _seed_legacy_stale_service_row(db) # published default + is_service_account (backfill) + assert authenticate_user(db, "mcp", "mcp-service-changeme") # live before upgrade + disable_stale_service_user(db) # unset MCP_SERVICE_PASSWORD upgrade boot + with pytest.raises(UnauthError): + authenticate_user(db, "mcp", "mcp-service-changeme") # dead after upgrade + + def test_adopts_and_reconciles_a_legacy_row_holding_a_published_default(self, db): + """#186 integration, scoped by bonnyr-f5 #193 B1 (reachable adopt path): a + row NOT flagged is_service_account is adopted ONLY when it carries v2_155's + exact backfill fingerprint (username 'mcp' AND email 'mcp@bnk-forge.local') + and still holds a shipped published default — a stale service credential + from a pre-provenance install. When a REAL MCP_SERVICE_PASSWORD is + configured, ensure_service_user ADOPTS it (flags provenance) and reconciles + to the real secret instead of refusing — the published default stops + working. Adoption must NOT clear the must_change_password gate (#193 B1).""" + from models import User + from services.auth_service import create_user, hash_password + u = create_user(db, "mcp", "mcp@bnk-forge.local", "mcp-service-changeme", + role="admin", must_change_password=True) + u.hashed_password = hash_password("mcp-service-changeme") + # is_service_account intentionally left False (pre-provenance row). + db.commit() + ensure_service_user(db, username="mcp", password="a-real-strong-secret") + mcp = db.query(User).filter(User.username == "mcp").first() + assert mcp.is_service_account is True # adopted + # #193 B1: adopting a row must not clear its must-change gate as a side effect. + assert mcp.must_change_password is True + assert authenticate_user(db, "mcp", "a-real-strong-secret").username == "mcp" + with pytest.raises(UnauthorizedError): + authenticate_user(db, "mcp", "mcp-service-changeme") # published default dead + + def test_human_row_holding_changeme_is_refused(self, db): + """bonnyr-f5 #193 B1 (the takeover): before the fix, ensure_service_user + adopted ANY non-service row whose password was a known default — and + `changeme` is both a known default AND one of the most common human + passwords. A human `operator`/`changeme` row (email that is NOT v2_155's + 'mcp@bnk-forge.local' fingerprint) must be REFUSED, not adopted: no + takeover, no lockout, and the must-change gate is left intact.""" + from models import User + from services.auth_service import create_user, hash_password + u = create_user(db, "operator", "ops@corp.example", "changeme", + role="admin", must_change_password=True) + u.hashed_password = hash_password("changeme") + # is_service_account False (a genuine human row), holds the default `changeme`. + db.commit() + with pytest.raises(ValueError, match="not a service account"): + ensure_service_user(db, username="operator", password="mcp-shared-secret") + # ensure_service_user raises BEFORE any write, so the committed row is intact + # (no rollback needed — a rollback would discard the fixture's savepoint). + row = db.query(User).filter(User.username == "operator").first() + # The human row is untouched: still human, still gated, own password still works. + assert row.is_service_account is False + assert row.must_change_password is True + assert authenticate_user(db, "operator", "changeme").username == "operator" + with pytest.raises(UnauthorizedError): + authenticate_user(db, "operator", "mcp-shared-secret") # takeover refused + + def test_named_mcp_but_wrong_email_holding_changeme_is_refused(self, db): + """bonnyr-f5 #193 B1: even a row literally named 'mcp' is only adopted when + its email matches v2_155's fingerprint. A human who merely happens to be + named 'mcp' with a real email is left untouched (mirrors the migration's own + conservative rule).""" + from models import User + from services.auth_service import create_user, hash_password + u = create_user(db, "mcp", "real.person@corp.example", "changeme", + role="admin", must_change_password=True) + u.hashed_password = hash_password("changeme") + db.commit() + with pytest.raises(ValueError, match="not a service account"): + ensure_service_user(db, username="mcp", password="mcp-shared-secret") + row = db.query(User).filter(User.username == "mcp").first() + assert row.is_service_account is False + assert row.must_change_password is True + + def test_operator_password_still_reconciles(self, db): + """A genuine operator-set password is honored (MCP stays usable when the + operator configures MCP_SERVICE_PASSWORD).""" + ensure_service_user(db, username="mcp", password="a-real-operator-secret") + user = authenticate_user(db, "mcp", "a-real-operator-secret") + assert user.username == "mcp" + assert user.must_change_password is False + + def test_reserved_human_username_is_refused(self, db): + """#186 BLOCKER 3 (bonnyr-f5): a service account may not adopt a reserved + human identity such as `admin`. The call raises and never touches the row.""" + import pytest + with pytest.raises(ValueError, match="reserved human username"): + ensure_service_user(db, username="admin", password="mcp-service-secret") + + def test_reserved_username_does_not_rewrite_human_admin(self, db): + """The attack bonnyr reproduced: pointing the mcp reconcile at `admin` + would clear must_change and grant the mcp secret admin access. The guard + must leave the real admin row (its hash + must_change gate) intact.""" + import pytest + + from models import User + create_user(db, "admin", "admin@test.com", "human-admin-pw", + role="admin", must_change_password=True) + db.commit() + with pytest.raises(ValueError): + ensure_service_user(db, username="admin", password="mcp-secret") + db.rollback() + admin = db.query(User).filter(User.username == "admin").first() + # Human admin credential + gate survive; the mcp secret never authenticates as admin. + assert admin.must_change_password is True + authenticate_user(db, "admin", "human-admin-pw") # still the human's password + with pytest.raises(UnauthorizedError): + authenticate_user(db, "admin", "mcp-secret") + + +class TestTokenUserState: + """#184: the WS gate helper -- resolve the User row and fail CLOSED. + + Returns the User on success, None on any resolution failure; the WS + validators refuse on None OR must_change_password. + """ + + def test_resolves_must_change_user(self, db): + from services.auth_service import token_user_state + create_user(db, "wsmust", "wsmust@test.com", "pw", role="admin", must_change_password=True) + db.commit() + token = create_access_token(data={"sub": "wsmust", "role": "admin"}) + user = token_user_state(token) + assert user is not None and user.must_change_password is True + + def test_resolves_normal_user(self, db): + from services.auth_service import token_user_state + create_user(db, "wsok", "wsok@test.com", "pw", role="admin", must_change_password=False) + db.commit() + token = create_access_token(data={"sub": "wsok", "role": "admin"}) + user = token_user_state(token) + assert user is not None and user.must_change_password is False + + def test_none_on_garbage_token(self): + from services.auth_service import token_user_state + assert token_user_state("not-a-token") is None + + def test_none_for_deactivated_account(self, db): + # #184 review: a disabled account must fail closed on WS, matching + # get_current_user. get_user_from_token raises "Account is disabled", + # which token_user_state turns into None (-> WS refuses). + u = create_user(db, "wsdisabled", "wsdisabled@test.com", "pw", role="admin") + u.is_active = False + db.commit() + from services.auth_service import token_user_state + token = create_access_token(data={"sub": "wsdisabled", "role": "admin"}) + assert token_user_state(token) is None + + def test_none_for_deleted_account(self, db): + from services.auth_service import token_user_state + token = create_access_token(data={"sub": "ghost", "role": "admin"}) + assert token_user_state(token) is None + + +class TestUnwritableKeysDirNeverLeaksPlaintext: + """#186 (bonnyr-f5): the docs promise "the plaintext is never logged". + + On an UNWRITABLE /app/keys the old code fell back to LOGGING the generated + plaintext (a real secret-into-logs leak). Every generated-credential path + must now fail closed (raise GeneratedCredentialPersistError) WITHOUT the + plaintext ever reaching a log record or the exception message. + + Each test patches secrets.token_urlsafe to a sentinel so the assertion is + exact: the sentinel must appear in NO log message and NOT in str(exc). + """ + + SENTINEL = "SENTINEL-do-not-log-this-secret-42" + + @pytest.fixture(autouse=True) + def _sentinel_secret(self, monkeypatch): + # Make every generated secret a known sentinel we can search for. + monkeypatch.setattr( + "services.auth_service.secrets.token_urlsafe", lambda *_a, **_k: self.SENTINEL + ) + + def _unwritable_keys(self, monkeypatch, tmp_path): + # A FILE used as a directory -> os.makedirs raises NotADirectoryError + # (an OSError), simulating an unwritable /app/keys mount. + blocker = tmp_path / "blocker" + blocker.write_text("x") + monkeypatch.setenv("KEYS_DIR", str(blocker / "keys")) + + def _assert_no_leak(self, caplog): + for rec in caplog.records: + assert self.SENTINEL not in rec.getMessage(), ( + f"plaintext leaked into logs: {rec.getMessage()!r}" + ) + + def test_seed_generated_fails_closed_no_leak(self, db, monkeypatch, tmp_path, caplog): + import logging + + from services.auth_service import GeneratedCredentialPersistError + monkeypatch.setattr(settings, "DEFAULT_ADMIN_PASSWORD", None) + self._unwritable_keys(monkeypatch, tmp_path) + caplog.set_level(logging.DEBUG) + with pytest.raises(GeneratedCredentialPersistError) as ei: + seed_admin_user(db) + assert self.SENTINEL not in str(ei.value) + self._assert_no_leak(caplog) + # Fail closed: no admin row was committed, so the next boot retries. + from models.system import User + assert db.query(User).filter(User.username == "admin").first() is None + + def test_rotate_admin_fails_closed_no_leak(self, db, monkeypatch, tmp_path, caplog): + import logging + + from models.system import User + from services.auth_service import GeneratedCredentialPersistError + create_user(db, "admin", "admin@bnk-forge.local", "changeme", + role="admin", must_change_password=False) + db.commit() + self._unwritable_keys(monkeypatch, tmp_path) + caplog.set_level(logging.DEBUG) + with pytest.raises(GeneratedCredentialPersistError) as ei: + seed_admin_user(db) + assert self.SENTINEL not in str(ei.value) + self._assert_no_leak(caplog) + # Fail closed: the published-default hash is left untouched (still + # 'changeme') so the next boot retries the rotation cleanly rather than + # stranding an admin whose generated password nobody can read. (The + # rotate path raises BEFORE mutating the row, so nothing to roll back.) + admin = db.query(User).filter(User.username == "admin").first() + assert verify_password("changeme", str(admin.hashed_password)) + + # NOTE (#193): the service-account seed/rotate-on-unset paths were removed — + # ensure_service_user no longer generates or persists a secret (it requires a + # usable password; the unset case is owned by disable_stale_service_user), so + # the two former "service seed/rotate fails closed" cases no longer exist. Only + # the admin seed + admin rotation still generate, and are covered above. + + +# ── bonnyr-f5 #193 test-gap 3: disable_stale_service_user(skip_username=...) ── + + +class TestDisableStaleSkipUsername: + """The #193 M2 change (skip_username) had ZERO direct coverage. M2's PROPERTY + is that the skipped (live MCP) row never gets an inactive window — not merely + that it ends up active. Assert the skipped row is NEVER modified (is_active True + AND its hash byte-identical), while a genuinely stale EXTRA service row is + disabled and hash-scrubbed.""" + + def test_skip_username_leaves_live_row_completely_untouched(self, db): + from models import User + from services.auth_service import disable_stale_service_user + + live = _seed_legacy_stale_service_row(db, username="mcp", password="live-real-secret") + extra = _seed_legacy_stale_service_row( + db, username="mcp-old", password="mcp-service-changeme" + ) + live_hash_before = live.hashed_password + extra_hash_before = extra.hashed_password + + disable_stale_service_user(db, skip_username="mcp", password_configured=True) + + live = db.query(User).filter(User.username == "mcp").first() + extra = db.query(User).filter(User.username == "mcp-old").first() + # The skipped live row: never entered the disable set → no inactive window. + assert live.is_active is True + assert live.hashed_password == live_hash_before # hash never scrubbed + # The stale EXTRA row: disabled and its credential neutralised. + assert extra.is_active is False + assert extra.hashed_password != extra_hash_before + assert verify_password("mcp-service-changeme", extra.hashed_password) is False + + def test_skip_username_is_exact_raw_match(self, db): + # bonnyr-f5 #193 M-2: the skip keys on the RAW MCP_SERVICE_USERNAME — the + # exact value ensure_service_user reconciles under and the client + # authenticates with. So skip 'MCP' protects a raw-'MCP' live row, while a + # differently-cased stale 'mcp' row (NOT what the client sends) is correctly + # disabled. Normalising the skip would instead spare that stale default row. + from models import User + from services.auth_service import disable_stale_service_user + + live = _seed_legacy_stale_service_row(db, username="MCP", password="live-real-secret") + stale = _seed_legacy_stale_service_row( + db, username="mcp", password="mcp-service-changeme" + ) + live_hash_before = live.hashed_password + + disable_stale_service_user(db, skip_username="MCP", password_configured=True) + + live = db.query(User).filter(User.username == "MCP").first() + stale = db.query(User).filter(User.username == "mcp").first() + # The row the client actually uses is skipped — no inactive window. + assert live.is_active is True + assert live.hashed_password == live_hash_before + # The differently-cased stale default row is disabled + scrubbed. + assert stale.is_active is False + assert verify_password("mcp-service-changeme", stale.hashed_password) is False + + +# ── bonnyr-f5 #193 test-gap 6: MCP_SERVICE_USERNAME case/whitespace variant ── + + +class TestServiceUsernameRawMatchesClient: + """bonnyr-f5 #193 M-2 (regression fix; replaces the round-3 'normalise the + lookup' tests, which locked in a client-breaking bug). The MCP client sends the + RAW MCP_SERVICE_USERNAME and authenticate_user matches exactly, so the account + must be created under the raw value — not folded to 'mcp'.""" + + @pytest.fixture(autouse=True) + def _isolate_keys_dir(self, monkeypatch, tmp_path): + monkeypatch.setenv("KEYS_DIR", str(tmp_path)) + + def test_variant_creates_account_under_raw_value_client_sends(self, db): + # MCP_SERVICE_USERNAME="MCP" (case variant): the row is created under 'MCP', + # so the client's 'MCP' login works — the login round-3 denied. + from models import User + ensure_service_user(db, username="MCP", password="a-real-strong-secret") + assert db.query(User).filter(User.username == "MCP").count() == 1 + assert db.query(User).filter(User.username == "mcp").count() == 0 + assert authenticate_user(db, "MCP", "a-real-strong-secret").username == "MCP" + # The normalised name is NOT what the client sends — nothing seeded there. + with pytest.raises(UnauthorizedError): + authenticate_user(db, "mcp", "a-real-strong-secret") + + def test_variant_create_uses_raw_name_and_email(self, db): + # The created row carries the raw username and a raw-derived email. + from models import User + ensure_service_user(db, username="MCP", password="a-real-strong-secret") + row = db.query(User).filter(User.username == "MCP").one() + assert row.email == "MCP@bnk-forge.local" + # Idempotent under the SAME raw value: reconciles the one row, no twin. + ensure_service_user(db, username="MCP", password="rotated-strong-secret") + assert db.query(User).filter(User.username == "MCP").count() == 1 + assert authenticate_user(db, "MCP", "rotated-strong-secret").username == "MCP" + + def test_default_lowercase_name_still_reconciles_legacy_row(self, db): + # The default MCP_SERVICE_USERNAME='mcp' (raw == what the client sends) still + # reconciles the legacy 'mcp' row rather than minting a second account. + from models import User + _seed_legacy_stale_service_row(db, username="mcp", password="mcp-service-changeme") + ensure_service_user(db, username="mcp", password="a-real-strong-secret") + assert db.query(User).filter(User.username == "mcp").count() == 1 + assert authenticate_user(db, "mcp", "a-real-strong-secret").username == "mcp" + + +# ── bonnyr-f5 #193 test-gap 4: db.commit() fails after the keys file is written ── + + +class TestRotateCommitFailureLeavesRetriableState: + """If db.commit() raises AFTER _persist_generated_password wrote the keys file, + the keys file holds a password that does not authenticate while the published + default still does. It self-heals on the next boot (the rotation retries), but + the interim state must be exactly that — not a lockout.""" + + def test_commit_failure_after_file_write(self, monkeypatch, tmp_path): + # Uses its OWN throwaway engine (not the shared savepoint-based `db` + # fixture) so a rollback here is real and isolated — a rollback on the + # shared fixture session would discard the fixture's own savepoint. + from sqlalchemy import create_engine + from sqlalchemy.orm import sessionmaker + from sqlalchemy.pool import StaticPool + + import models # noqa: F401 — register tables + from database import Base + from models import User + from services.auth_service import _rotate_known_default_admin + + monkeypatch.setenv("KEYS_DIR", str(tmp_path)) + monkeypatch.setattr(settings, "DEFAULT_ADMIN_PASSWORD", None) # force generation + + engine = create_engine( + "sqlite://", connect_args={"check_same_thread": False}, poolclass=StaticPool + ) + Base.metadata.create_all(bind=engine) + Session = sessionmaker(bind=engine) + db = Session() + try: + create_user(db, "admin", "admin@bnk-forge.local", "changeme", + role="admin", must_change_password=False) + db.commit() + + def _boom(): + raise RuntimeError("simulated commit failure after file write") + monkeypatch.setattr(db, "commit", _boom) + + with pytest.raises(RuntimeError, match="simulated commit failure"): + _rotate_known_default_admin(db) + + # Roll the uncommitted hash change back (as get_db_context would on the + # raised exception). Restore commit first so rollback works cleanly. + monkeypatch.undo() + db.rollback() + + # The keys file WAS written with a generated secret ... + pw_file = tmp_path / "initial_admin_password" + assert pw_file.exists() + orphan_pw = pw_file.read_text().strip() + assert orphan_pw and orphan_pw != "changeme" + + admin = db.query(User).filter(User.username == "admin").first() + # ... but the DB row's hash was NOT committed: the published default + # STILL authenticates (self-heals next boot) and the orphan file + # password does NOT. + assert verify_password("changeme", admin.hashed_password) + assert verify_password(orphan_pw, admin.hashed_password) is False + finally: + db.close() + engine.dispose() diff --git a/backend/tests/component/test_dpus_websocket.py b/backend/tests/component/test_dpus_websocket.py new file mode 100644 index 0000000..386024c --- /dev/null +++ b/backend/tests/component/test_dpus_websocket.py @@ -0,0 +1,104 @@ +"""Component tests for the DPU WebSocket auth guard — routes.dpus_websocket._validate_ws_token. + +bonnyr-f5 #193 (Credential minor): the new must-change / unresolvable-user gate on +the DPU console + BMC/OS SSH websockets had NO test, while its k8s twin +(routes.k8s_websocket._validate_ws_token, covered in test_k8s_websocket.py) did. +These mirror the twin so a seed-credential admin can never reach a DPU shell while +REST refuses /api/auth/users, and a token whose user can't be resolved fails CLOSED. +""" + +from unittest.mock import AsyncMock, patch + +import pytest + + +class TestDpusValidateWsToken: + """Mirror of TestValidateWsToken in test_k8s_websocket.py, for the DPU guard.""" + + @pytest.mark.asyncio + @patch("core.config.settings") + async def test_auth_disabled_returns_true(self, mock_settings): + mock_settings.REQUIRE_AUTH = False + from routes.dpus_websocket import _validate_ws_token + + ws = AsyncMock() + assert await _validate_ws_token(ws, None) is True + ws.close.assert_not_awaited() + + @pytest.mark.asyncio + async def test_missing_token_closes_4401(self): + from routes.dpus_websocket import _validate_ws_token + + ws = AsyncMock() + assert await _validate_ws_token(ws, None) is False + ws.close.assert_awaited_once() + assert ws.close.call_args[1]["code"] == 4401 + + @pytest.mark.asyncio + async def test_valid_admin_token_returns_true(self, db): + from routes.dpus_websocket import _validate_ws_token + from services.auth_service import create_access_token, create_user + + create_user(db, "dpuadmin", "dpuadmin@t.com", "pw", role="admin", must_change_password=False) + db.commit() + token = create_access_token(data={"sub": "dpuadmin", "role": "admin"}) + ws = AsyncMock() + assert await _validate_ws_token(ws, token) is True + ws.close.assert_not_awaited() + + @pytest.mark.asyncio + async def test_valid_operator_token_returns_true(self, db): + from routes.dpus_websocket import _validate_ws_token + from services.auth_service import create_access_token, create_user + + create_user(db, "dpuop", "dpuop@t.com", "pw", role="operator", must_change_password=False) + db.commit() + token = create_access_token(data={"sub": "dpuop", "role": "operator"}) + ws = AsyncMock() + assert await _validate_ws_token(ws, token) is True + + @pytest.mark.asyncio + async def test_viewer_role_refused(self): + # DPU shells require admin/operator — a viewer must be closed with 4401. + from routes.dpus_websocket import _validate_ws_token + from services.auth_service import create_access_token + + token = create_access_token(data={"sub": "someviewer", "role": "viewer"}) + ws = AsyncMock() + assert await _validate_ws_token(ws, token) is False + ws.close.assert_awaited_once() + assert ws.close.call_args[1]["code"] == 4401 + + @pytest.mark.asyncio + async def test_must_change_user_refused(self, db): + # #184: a valid token whose user still owes a password change must be refused + # at the WS boundary — else a seed-credential admin gets a DPU/BMC shell. + from routes.dpus_websocket import _validate_ws_token + from services.auth_service import create_access_token, create_user + + create_user(db, "dpumustchange", "dpumc@t.com", "pw", role="admin", must_change_password=True) + db.commit() + token = create_access_token(data={"sub": "dpumustchange", "role": "admin"}) + ws = AsyncMock() + assert await _validate_ws_token(ws, token) is False + ws.close.assert_awaited_once() + + @pytest.mark.asyncio + async def test_token_for_missing_user_refused(self, db): + # #184 fail-closed: a valid JWT whose user doesn't exist (deleted) is refused, + # not allowed through on a resolution failure. + from routes.dpus_websocket import _validate_ws_token + from services.auth_service import create_access_token + + token = create_access_token(data={"sub": "ghost", "role": "admin"}) + ws = AsyncMock() + assert await _validate_ws_token(ws, token) is False + ws.close.assert_awaited_once() + + @pytest.mark.asyncio + async def test_malformed_token_refused(self): + from routes.dpus_websocket import _validate_ws_token + + ws = AsyncMock() + assert await _validate_ws_token(ws, "not-a-jwt-at-all") is False + ws.close.assert_awaited_once() diff --git a/backend/tests/component/test_k8s_websocket.py b/backend/tests/component/test_k8s_websocket.py index fc8c9a0..0662877 100644 --- a/backend/tests/component/test_k8s_websocket.py +++ b/backend/tests/component/test_k8s_websocket.py @@ -57,11 +57,12 @@ async def test_empty_string_token_closes_4401(self): ws.close.assert_awaited_once() @pytest.mark.asyncio - async def test_valid_admin_token_returns_true(self): + async def test_valid_admin_token_returns_true(self, db): """Should return True for a valid admin JWT token.""" from routes.k8s_websocket import _validate_ws_token - from services.auth_service import create_access_token - + from services.auth_service import create_access_token, create_user + create_user(db, "testadmin", "testadmin@t.com", "pw", role="admin", must_change_password=False) + db.commit() token = create_access_token(data={"sub": "testadmin", "role": "admin"}) ws = AsyncMock() result = await _validate_ws_token(ws, token) @@ -69,27 +70,55 @@ async def test_valid_admin_token_returns_true(self): ws.close.assert_not_awaited() @pytest.mark.asyncio - async def test_valid_operator_token_returns_true(self): + async def test_valid_operator_token_returns_true(self, db): """Should return True for a valid operator JWT token.""" from routes.k8s_websocket import _validate_ws_token - from services.auth_service import create_access_token - + from services.auth_service import create_access_token, create_user + create_user(db, "testop", "testop@t.com", "pw", role="operator", must_change_password=False) + db.commit() token = create_access_token(data={"sub": "testop", "role": "operator"}) ws = AsyncMock() result = await _validate_ws_token(ws, token) assert result is True @pytest.mark.asyncio - async def test_valid_viewer_token_returns_true(self): + async def test_valid_viewer_token_returns_true(self, db): """Should return True for a valid viewer JWT token.""" from routes.k8s_websocket import _validate_ws_token - from services.auth_service import create_access_token - + from services.auth_service import create_access_token, create_user + create_user(db, "testviewer", "testviewer@t.com", "pw", role="viewer", must_change_password=False) + db.commit() token = create_access_token(data={"sub": "testviewer", "role": "viewer"}) ws = AsyncMock() result = await _validate_ws_token(ws, token) assert result is True + @pytest.mark.asyncio + async def test_must_change_user_refused(self, db): + """#184: a valid token whose user still owes a password change must be + refused at the WS boundary -- otherwise a seed-credential admin gets a + pod shell while REST refuses /api/auth/users.""" + from routes.k8s_websocket import _validate_ws_token + from services.auth_service import create_access_token, create_user + create_user(db, "wsmustchange", "wsmc@t.com", "pw", role="admin", must_change_password=True) + db.commit() + token = create_access_token(data={"sub": "wsmustchange", "role": "admin"}) + ws = AsyncMock() + result = await _validate_ws_token(ws, token) + assert result is False + ws.close.assert_awaited_once() + + @pytest.mark.asyncio + async def test_token_for_missing_user_refused(self, db): + """#184 fail-closed: a token whose user doesn't exist is refused, not + allowed through on a resolution failure.""" + from routes.k8s_websocket import _validate_ws_token + from services.auth_service import create_access_token + token = create_access_token(data={"sub": "nobody", "role": "admin"}) + ws = AsyncMock() + result = await _validate_ws_token(ws, token) + assert result is False + @pytest.mark.asyncio async def test_invalid_role_closes_4401(self): """Should close WebSocket for token with unrecognized role.""" diff --git a/backend/tests/component/test_startup_steps.py b/backend/tests/component/test_startup_steps.py index 7493a2c..5a34684 100644 --- a/backend/tests/component/test_startup_steps.py +++ b/backend/tests/component/test_startup_steps.py @@ -2,7 +2,42 @@ from unittest.mock import MagicMock, call, patch -from startup_steps import seed_defaults_step, sync_module_catalog_step +import pytest + +from startup_steps import seed_auth_step, seed_defaults_step, sync_module_catalog_step + + +@patch("database.get_db_context") +@patch("services.auth_service.seed_admin_user") +def test_seed_auth_step_fails_closed_on_unpersistable_credential( + mock_seed_admin, mock_get_db_context +): + """#186 (bonnyr-f5 r4): when a generated credential can't be written to the + keys dir, seed_auth_step must REFUSE TO START (SystemExit) rather than fall + back to logging the plaintext. SystemExit (BaseException) escapes main.py's + best-effort `except Exception`, so the process actually halts — and the + error message must NOT contain the secret. + """ + from services.auth_service import GeneratedCredentialPersistError + + db = MagicMock() + ctx = MagicMock() + ctx.__enter__.return_value = db + ctx.__exit__.return_value = False + mock_get_db_context.return_value = ctx + # The seed path raised because /app/keys was unwritable. The exception + # carries NO plaintext (that is the whole point of the fail-closed design). + mock_seed_admin.side_effect = GeneratedCredentialPersistError( + "could not persist generated credential to /app/keys/initial_admin_password: " + "[Errno 13] Permission denied" + ) + + with pytest.raises(SystemExit) as ei: + seed_auth_step() + + # SystemExit, not swallowed; and the message never carries a secret. + assert "Refusing to log" in str(ei.value) + assert "Permission denied" in str(ei.value) @patch("database.get_db_context") diff --git a/backend/tests/integration/test_routes_auth.py b/backend/tests/integration/test_routes_auth.py index 21e88f2..e857aa3 100644 --- a/backend/tests/integration/test_routes_auth.py +++ b/backend/tests/integration/test_routes_auth.py @@ -209,6 +209,21 @@ def test_list_users_admin(self, client, admin_headers, all_test_users, db): usernames = [u["username"] for u in users] assert "testadmin" in usernames + def test_list_users_exposes_service_account_flag(self, client, admin_headers, all_test_users, db): + """bonnyr-f5 #188: the user listing surfaces is_service_account so the UI can + tell a service account (whose re-enable is guarded) from a human account + instead of blindly 400ing on the toggle.""" + from services.auth_service import ensure_service_user + ensure_service_user(db, username="mcp", password="a-strong-real-secret") + db.commit() + + response = client.get("/api/auth/users", headers=admin_headers) + assert response.status_code == 200 + by_name = {u["username"]: u for u in response.json()["users"]} + assert "is_service_account" in by_name["testadmin"] + assert by_name["testadmin"]["is_service_account"] is False + assert by_name["mcp"]["is_service_account"] is True + def test_list_users_viewer_denied(self, client, viewer_headers, all_test_users): """Viewer cannot list users — returns 403.""" response = client.get("/api/auth/users", headers=viewer_headers) @@ -240,3 +255,175 @@ def test_delete_nonexistent_user(self, client, admin_headers, sample_user): """Deleting nonexistent user returns 404.""" response = client.delete("/api/auth/users/99999", headers=admin_headers) assert response.status_code == 404 + + +class TestServiceAccountReEnableGuard: + """bonnyr-f5 #188 (round 4): re-enabling a disabled service account via + PUT /api/auth/users/{id} must not resurrect a shipped default credential. + disable_stale_service_user only flips is_active; the bcrypt hash of + 'mcp-service-changeme' stays, so a naive re-enable brought the default back. + """ + + def _seed_disabled_default_mcp(self, db): + # A service-account row that is DISABLED while still holding the published + # default bcrypt("mcp-service-changeme"). This is the exact state the + # PUT-route re-enable guard defends against — a row taken inactive by a path + # that leaves the credential intact (a manual operator PUT, or any + # disable that is not disable_stale_service_user, which #193 now scrubs the + # hash on). We build it directly rather than via disable_stale precisely so + # the default hash survives for the guard to detect (re-enabling it would + # resurrect the publicly-known default). ensure_service_user REFUSES a + # published default as a seed value, so create_user is the only way to + # reproduce a legacy row that genuinely carries it. + from services.auth_service import create_user, verify_password + mcp = create_user( + db, + username="mcp", + email="mcp@bnk-forge.local", + password="mcp-service-changeme", + role="admin", + must_change_password=False, + ) + mcp.is_service_account = True # v2_155 backfill marks the legacy mcp row + mcp.is_active = False # disabled with the default hash intact (not via disable_stale) + db.commit() + mcp = db.query(User).filter(User.username == "mcp").first() + assert mcp.is_active is False + assert mcp.is_service_account is True + assert verify_password("mcp-service-changeme", str(mcp.hashed_password)) # default preserved + return mcp + + def test_reenable_refused_while_default_hash_present( + self, client, admin_headers, sample_user, db + ): + from core.errors import UnauthorizedError + from services.auth_service import authenticate_user + + mcp = self._seed_disabled_default_mcp(db) + resp = client.put( + f"/api/auth/users/{mcp.id}", + json={"is_active": True}, + headers=admin_headers, + ) + assert resp.status_code == 400 + assert "known default password" in resp.json()["error"]["message"] + + db.refresh(mcp) + assert mcp.is_active is False # re-enable refused + + # Mutation test: the shipped default must NOT authenticate. + with pytest.raises(UnauthorizedError): + authenticate_user(db, "mcp", "mcp-service-changeme") + + def test_reenable_allowed_after_real_password_rotation( + self, client, admin_headers, sample_user, db + ): + """A service account carrying a real (non-default) hash re-enables fine — + the guard is scoped to known-default hashes only.""" + from core.errors import UnauthorizedError + from services.auth_service import authenticate_user, ensure_service_user + + self._seed_disabled_default_mcp(db) + # Operator rotates to a strong secret (startup re-seeds + re-activates). + ensure_service_user(db, username="mcp", password="a-real-strong-secret") + mcp = db.query(User).filter(User.username == "mcp").first() + assert mcp.is_active is True + + # Admin may still toggle it via the route now that no default hash remains. + resp = client.put( + f"/api/auth/users/{mcp.id}", json={"is_active": True}, headers=admin_headers + ) + assert resp.status_code == 200 + db.refresh(mcp) + assert mcp.is_active is True + assert authenticate_user(db, "mcp", "a-real-strong-secret").is_active is True + # And the old default is gone for good. + with pytest.raises(UnauthorizedError): + authenticate_user(db, "mcp", "mcp-service-changeme") + + +class TestMustChangePasswordEnforcement: + """#184: must_change_password must gate the API server-side, not just the UI. + + A seeded/admin-created must-change user gets a valid token, so without a + server gate a client could skip the change-password screen and call every + endpoint directly with the seed credential. + """ + + def _make_must_change_admin(self, db): + from services.auth_service import create_user + u = create_user( + db, "mustchange", "mustchange@test.com", "startpw", + role="admin", must_change_password=True, + ) + db.commit() + return u + + def _login(self, client, username, password): + r = client.post("/api/auth/login", json={"username": username, "password": password}) + assert r.status_code == 200, r.text + assert r.json()["must_change_password"] is True + return r.json()["token"] + + def test_protected_endpoint_refused_until_password_changed(self, client, db): + self._make_must_change_admin(db) + token = self._login(client, "mustchange", "startpw") + hdr = {"Authorization": f"Bearer {token}"} + + # A normal protected endpoint is refused with 403 while must-change. + blocked = client.get("/api/auth/users", headers=hdr) + assert blocked.status_code == 403, blocked.text + + # The exempt endpoints still work: read own state and change password. + assert client.get("/api/auth/me", headers=hdr).status_code == 200 + + changed = client.post( + "/api/auth/change-password", + headers=hdr, + json={"current_password": "startpw", "new_password": "BrandNewPw123!"}, + ) + assert changed.status_code == 200, changed.text + + # After the change the flag clears, so the same endpoint now works. + after = client.get("/api/auth/users", headers=hdr) + assert after.status_code == 200, after.text + + def test_dependency_less_route_is_gated_by_the_middleware(self, client, db): + """#186 (bonnyr-f5): the gate lived only in get_current_user, so a route + that declares NO auth dependency and relies on AuthMiddleware alone was + bypassable with the seed credential. /api/system/process-metrics is such + a route (public_router, no get_current_user, not in PUBLIC_PATHS). A + must-change token must be refused there, at the middleware, not served. + """ + self._make_must_change_admin(db) + token = self._login(client, "mustchange", "startpw") + hdr = {"Authorization": f"Bearer {token}"} + + # Middleware-only route: must be 403 while must-change (was 200 = bypass). + blocked = client.get("/api/system/process-metrics", headers=hdr) + assert blocked.status_code == 403, blocked.text + + # Exempt read still works so the UI can drive the change screen. + assert client.get("/api/auth/me", headers=hdr).status_code == 200 + + # After rotating, the same middleware-only route is reachable. + assert client.post( + "/api/auth/change-password", headers=hdr, + json={"current_password": "startpw", "new_password": "BrandNewPw123!"}, + ).status_code == 200 + assert client.get("/api/system/process-metrics", headers=hdr).status_code == 200 + + def test_non_must_change_user_is_not_gated(self, client, admin_headers, sample_user): + # Regression guard: an ordinary user (must_change False) reaches the API. + assert client.get("/api/auth/users", headers=admin_headers).status_code == 200 + + def test_path_route_with_auth_me_suffix_is_not_exempted(self, client, db): + # #184 review: a ':path' route (e.g. /api/state/.../resource/{addr:path}) + # takes an attacker-chosen tail. Exact-path matching on request.url.path + # must NOT exempt "/api/state/module/1/resource/x/auth/me" just because it + # ends in /auth/me -- the gate refuses it (403) before the handler runs. + self._make_must_change_admin(db) + token = self._login(client, "mustchange", "startpw") + hdr = {"Authorization": f"Bearer {token}"} + r = client.get("/api/state/module/1/resource/x/auth/me", headers=hdr) + assert r.status_code == 403, r.text diff --git a/backend/tests/integration/test_routes_k8s_websocket.py b/backend/tests/integration/test_routes_k8s_websocket.py index 8e84948..a0d9f76 100644 --- a/backend/tests/integration/test_routes_k8s_websocket.py +++ b/backend/tests/integration/test_routes_k8s_websocket.py @@ -56,7 +56,7 @@ def test_exec_valid_admin_token_accepted(self, client, sample_user): msg = ws.receive_json() assert msg["type"] == "error" - def test_exec_valid_viewer_token_accepted(self, client, sample_user): + def test_exec_valid_viewer_token_accepted(self, client, sample_viewer_user): """Valid viewer JWT passes auth — connection accepted.""" token = _make_token("viewer") with client.websocket_connect( @@ -65,7 +65,7 @@ def test_exec_valid_viewer_token_accepted(self, client, sample_user): msg = ws.receive_json() assert msg["type"] == "error" - def test_exec_valid_operator_token_accepted(self, client, sample_user): + def test_exec_valid_operator_token_accepted(self, client, sample_operator_user): """Valid operator JWT passes auth — connection accepted.""" token = _make_token("operator") with client.websocket_connect( diff --git a/backend/tests/test_migrations.py b/backend/tests/test_migrations.py index 6cbdea2..b073e2d 100644 --- a/backend/tests/test_migrations.py +++ b/backend/tests/test_migrations.py @@ -534,3 +534,112 @@ def test_drift_gate_ignores_out_of_band_tables(self): finally: engine.dispose() os.unlink(db_path) + + def test_v2_155_backfills_only_the_legacy_mcp_service_row(self): + """bonnyr-f5 #188 r4 (INV-7): the backfill lives in a NEW revision v2_155, + NOT appended to the already-shipped v2_154. + + v2_154 (shipped in earlier RCs) only adds the column with server_default + false — appending a backfill there would never run for an install already + stamped v2_154 (an applied revision is immutable), i.e. exactly the existing + installs the backfill must fix. v2_155 chains from v2_154 and does the + backfill, so any install at v2_154 applies it on the next upgrade. + + This test drives the two revisions in sequence and asserts v2_155 flips + ONLY the row with the legacy creation fingerprint (username 'mcp' + the + synthesised email 'mcp@bnk-forge.local'), never a human — including a human + named 'admin', a normal human, or a human who merely happens to be named + 'mcp' with a real email. It also proves the exact BLOCKER-2 scenario: a DB + already at v2_154 with the mcp row still False gets it backfilled by v2_155. + """ + import importlib.util + + import sqlalchemy as sa + from alembic.operations import Operations + from alembic.runtime.migration import MigrationContext + from sqlalchemy import create_engine, inspect, text + from sqlalchemy.pool import StaticPool + + def _load(basename, modname): + path = os.path.join(backend_path, "alembic", "versions", basename) + spec = importlib.util.spec_from_file_location(modname, path) + assert spec is not None and spec.loader is not None + mod = importlib.util.module_from_spec(spec) + spec.loader.exec_module(mod) + return mod + + v2_154 = _load("v2_154_user_is_service_account.py", "migration_v2_154") + v2_155 = _load("v2_155_backfill_is_service_account.py", "migration_v2_155") + + # v2_155 must chain directly from v2_154 (single linear head). + assert v2_155.down_revision == "v2_154" + assert v2_154.down_revision == "v2_153" + + engine = create_engine( + "sqlite:///:memory:", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + # users table in the pre-v2_154 shape (no is_service_account column). + metadata = sa.MetaData() + sa.Table( + "users", metadata, + sa.Column("id", sa.Integer, primary_key=True), + sa.Column("username", sa.String(255), unique=True, nullable=False), + sa.Column("email", sa.String(255), unique=True, nullable=False), + sa.Column("hashed_password", sa.String(255), nullable=False), + sa.Column("role", sa.String(50), nullable=False, server_default="operator"), + sa.Column("is_active", sa.Boolean, nullable=False, server_default=sa.true()), + sa.Column("must_change_password", sa.Boolean, nullable=False, server_default=sa.false()), + ) + metadata.create_all(engine) + with engine.begin() as conn: + conn.execute(text( + "INSERT INTO users (username,email,hashed_password,role,is_active,must_change_password) VALUES " + "('mcp','mcp@bnk-forge.local','x','admin',1,0)," # legacy service acct + "('admin','admin@corp.com','y','admin',1,0)," # human admin + "('alice','alice@corp.com','z','operator',1,0)," # ordinary human + "('mcp2','mcp@real-human.com','w','operator',1,0)" # human named 'mcp' w/ real email + )) + + def flags(conn): + return { + r._mapping["username"]: r._mapping["is_service_account"] + for r in conn.execute(text("SELECT username, is_service_account FROM users")) + } + + with engine.begin() as connection: + migration_context = MigrationContext.configure(connection) + ops = Operations(migration_context) + + # v2_154 adds the column only — every pre-existing row is False. This is + # the exact state of an install stamped v2_154 at the earlier commit. + v2_154.op = ops + v2_154.upgrade() + f0 = flags(connection) + assert all(v in (0, False) for v in f0.values()), ( + "v2_154 must NOT backfill — that would resurrect the immutable-migration hole" + ) + + # v2_155 backfills exactly the legacy mcp row (the BLOCKER-2 fix path). + v2_155.op = ops + v2_155.upgrade() + f = flags(connection) + assert f["mcp"] in (1, True), "legacy mcp service row must be backfilled True by v2_155" + assert f["admin"] in (0, False), "human admin must NOT be reclassified" + assert f["alice"] in (0, False), "ordinary human must NOT be reclassified" + assert f["mcp2"] in (0, False), "human named 'mcp' with a real email must NOT be reclassified" + + # Round-trip: v2_155 down clears only the flag; v2_154 down drops the column. + v2_155.downgrade() + assert flags(connection)["mcp"] in (0, False), "v2_155 downgrade must clear the flag" + v2_154.downgrade() + cols = {c["name"] for c in inspect(connection).get_columns("users")} + assert "is_service_account" not in cols + + # Idempotent re-apply (CI round-trip gate). + v2_154.upgrade() + v2_155.upgrade() + assert flags(connection)["mcp"] in (1, True) + + engine.dispose() diff --git a/backend/tests/test_startup_seed_auth.py b/backend/tests/test_startup_seed_auth.py new file mode 100644 index 0000000..1de68d6 --- /dev/null +++ b/backend/tests/test_startup_seed_auth.py @@ -0,0 +1,350 @@ +"""Startup seed_auth_step coverage — bonnyr-f5 #188 round 5 (BLOCKER-1). + +These tests drive the real ``seed_auth_step`` against a legacy-install fixture, +covering the previously-untested reconcile / disable branching. The central case +reproduces the *diligent operator* path: a legacy install whose ``.env`` still +carries ``MCP_USERNAME=admin`` but who follows the new docs and sets a strong +``MCP_PASSWORD``. Before the fix the stale ``mcp`` row kept authenticating with +the shipped default; the fix makes the provenance-keyed disable unconditional so +that path is closed too. +""" + + +import pytest +from sqlalchemy import create_engine +from sqlalchemy.orm import sessionmaker +from sqlalchemy.pool import StaticPool + +import database +import models # noqa: F401 — register all tables on Base.metadata +import startup_steps +from core.errors import UnauthorizedError +from database import Base +from models import User +from services.auth_service import ( + authenticate_user, + disable_stale_service_user, + ensure_service_user, + hash_password, + seed_admin_user, + verify_password, +) + +LEGACY_DEFAULT = "mcp-service-changeme" +ADMIN_DEFAULT = "changeme" + + +@pytest.fixture() +def legacy_db(monkeypatch): + """A DB whose only service account is the legacy ``mcp`` row holding the + shipped default, is_active=True, is_service_account=True — i.e. exactly what + v2_155 leaves behind on a pre-#188 upgrade.""" + engine = create_engine( + "sqlite://", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(bind=engine) + session_factory = sessionmaker(autocommit=False, autoflush=False, bind=engine) + # seed_auth_step calls get_db_context(), which builds sessions from the + # module-global SessionLocal — repoint it at our throwaway engine. + monkeypatch.setattr(database, "SessionLocal", session_factory) + + db = session_factory() + legacy = User( + username="mcp", + email="mcp@bnk-forge.local", + hashed_password=hash_password(LEGACY_DEFAULT), + role="admin", + is_active=True, + is_service_account=True, + must_change_password=False, + ) + # A human admin, to prove the disable never touches it. + human = User( + username="admin", + email="admin@bnk-forge.local", + hashed_password=hash_password("real-admin-secret"), + role="admin", + is_active=True, + is_service_account=False, + must_change_password=False, + ) + db.add_all([legacy, human]) + db.commit() + db.close() + + yield session_factory + engine.dispose() + + +def _legacy_default_still_works(session_factory) -> bool: + db = session_factory() + try: + authenticate_user(db, "mcp", LEGACY_DEFAULT) + return True + except UnauthorizedError: + return False + finally: + db.close() + + +def _set_mcp_env(monkeypatch, username, password): + monkeypatch.setattr(startup_steps.settings, "MCP_SERVICE_USERNAME", username) + monkeypatch.setattr(startup_steps.settings, "MCP_SERVICE_PASSWORD", password) + monkeypatch.setattr(startup_steps.settings, "REQUIRE_AUTH", True) + + +def test_diligent_operator_admin_username_disables_stale_default(legacy_db, monkeypatch): + """BLOCKER-1: strong password set, but MCP_USERNAME left at legacy 'admin'. + + ensure_service_user raises a reserved-name ValueError (swallowed), so the + ONLY thing that can neutralise the legacy row is the unconditional disable. + """ + _set_mcp_env(monkeypatch, "admin", "strong-new-secret-xyz") + assert _legacy_default_still_works(legacy_db) is True # vulnerable pre-run + + startup_steps.seed_auth_step() + + assert _legacy_default_still_works(legacy_db) is False, ( + "legacy mcp/mcp-service-changeme still authenticates — remediation did " + "not fire on the diligent-operator path" + ) + # Human admin must be untouched. + db = legacy_db() + human = db.query(User).filter(User.username == "admin", User.is_service_account.is_(False)).one() + assert human.is_active is True + assert authenticate_user(db, "admin", "real-admin-secret") + db.close() + + +def test_usable_password_dedicated_name_reconciles_and_disables_legacy(legacy_db, monkeypatch): + """Strong password + a dedicated MCP username: the new account authenticates + with the new secret AND the stale default row is disabled.""" + _set_mcp_env(monkeypatch, "mcp-svc", "strong-new-secret-xyz") + + startup_steps.seed_auth_step() + + assert _legacy_default_still_works(legacy_db) is False + db = legacy_db() + assert authenticate_user(db, "mcp-svc", "strong-new-secret-xyz") + db.close() + + +def test_unset_password_disables_stale_default(legacy_db, monkeypatch): + """No usable password at all: the stale default must be disabled.""" + _set_mcp_env(monkeypatch, "mcp", None) + + startup_steps.seed_auth_step() + + assert _legacy_default_still_works(legacy_db) is False + + +def test_known_default_password_treated_as_unset(legacy_db, monkeypatch): + """A shipped default supplied as MCP_PASSWORD must not re-seed the account — + it is treated as unset and the stale row is disabled.""" + _set_mcp_env(monkeypatch, "mcp", "changeme") + + startup_steps.seed_auth_step() + + assert _legacy_default_still_works(legacy_db) is False + + +def test_configured_mcp_row_is_reconciled_when_matching_username(legacy_db, monkeypatch): + """When MCP_USERNAME matches the legacy row's name and a real password is set, + the account is reconciled (hash rotated to the new secret, re-activated).""" + _set_mcp_env(monkeypatch, "mcp", "brand-new-strong-secret") + + startup_steps.seed_auth_step() + + # Old default no longer works; new secret does. + assert _legacy_default_still_works(legacy_db) is False + db = legacy_db() + assert authenticate_user(db, "mcp", "brand-new-strong-secret") + db.close() + + +def test_disable_stale_service_user_neutralises_the_hash(legacy_db, monkeypatch): + """bonnyr-f5 #193 (minor): disabling a stale service row must not leave the + published-default hash intact and lean solely on is_active + the route guard — + the credential itself must be dead so it cannot authenticate under any path.""" + db = legacy_db() + try: + disable_stale_service_user(db, skip_username=None, password_configured=False) + row = db.query(User).filter(User.username == "mcp").one() + assert row.is_active is False + # The old default hash must no longer verify. + assert verify_password(LEGACY_DEFAULT, row.hashed_password) is False + finally: + db.close() + + +# ── bonnyr-f5 #193: reserved human username is refused case-insensitively ── + + +@pytest.mark.parametrize("name", ["Admin", "ADMIN", " admin ", "aDmIn"]) +def test_ensure_service_user_refuses_reserved_username_case_insensitively(legacy_db, name): + """A service account may not co-opt the human 'admin' identity — and the Python + guard must match Helm's `lower | trim`, refusing case/whitespace variants too.""" + db = legacy_db() + try: + with pytest.raises(ValueError, match="reserved human username"): + ensure_service_user(db, username=name, password="a-real-secret") + finally: + db.close() + + +@pytest.mark.parametrize("variant", ["MCP", " mcp ", " Mcp "]) +def test_service_username_variant_account_matches_what_the_client_sends(legacy_db, monkeypatch, variant): + """bonnyr-f5 #193 M-2 (regression fix; this REPLACES the round-3 'reconcile the + legacy row' test, which locked in the bug). The MCP client receives the RAW + MCP_SERVICE_USERNAME as BNK_FORGE_USERNAME and authenticate_user matches exactly, + so the seeded account MUST carry the RAW value the client will send — not a + normalised 'mcp'. Round-3 normalised the row to 'mcp' and every non-lowercase + login was DENIED. The correct invariant: the row matches what the client sends, + and the reserved/skip logic keys on that same raw value. The provenance-keyed + disable neutralises the legacy default in the same boot.""" + _set_mcp_env(monkeypatch, variant, "brand-new-strong-secret") + + startup_steps.seed_auth_step() + + db = legacy_db() + try: + # The account the CLIENT will authenticate as exists under the raw value and + # its new secret works — this is the login round-3 broke. + assert authenticate_user(db, variant, "brand-new-strong-secret") + # The normalised name is NOT what the client sends; nothing was seeded there. + if variant != "mcp": + with pytest.raises(UnauthorizedError): + authenticate_user(db, "mcp", "brand-new-strong-secret") + # The legacy 'mcp'/shipped-default row is neutralised (provenance-keyed + # disable + hash scrub), so the published default no longer authenticates. + with pytest.raises(UnauthorizedError): + authenticate_user(db, "mcp", LEGACY_DEFAULT) + legacy = db.query(User).filter(User.username == "mcp").one() + assert legacy.is_active is False + finally: + db.close() + + +# ── bonnyr-f5 #193: admin still holding the DEFAULT (`changeme`) — the real ── +# upgrade shape the previous fixtures never exercised (they gave admin a +# non-default secret). Drives seed_auth_step against it. + + +@pytest.fixture() +def admin_default_db(monkeypatch, tmp_path): + """A DB whose human ``admin`` row still holds the shipped ``changeme`` default — + exactly what a pre-#184 install carries into an upgrade.""" + engine = create_engine( + "sqlite://", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(bind=engine) + session_factory = sessionmaker(autocommit=False, autoflush=False, bind=engine) + monkeypatch.setattr(database, "SessionLocal", session_factory) + # Generation persists to KEYS_DIR; point it at a writable tmp dir. + monkeypatch.setenv("KEYS_DIR", str(tmp_path)) + + db = session_factory() + db.add( + User( + username="admin", + email="admin@bnk-forge.local", + hashed_password=hash_password(ADMIN_DEFAULT), + role="admin", + is_active=True, + is_service_account=False, + must_change_password=False, + ) + ) + db.commit() + db.close() + # MCP unset for these admin-focused cases: the MCP path just no-ops/disables. + _set_mcp_env(monkeypatch, "mcp", None) + + yield session_factory, tmp_path + engine.dispose() + + +def _admin_default_still_works(session_factory) -> bool: + db = session_factory() + try: + authenticate_user(db, "admin", ADMIN_DEFAULT) + return True + except UnauthorizedError: + return False + finally: + db.close() + + +def test_upgrade_admin_default_rotated_when_password_unset(admin_default_db, monkeypatch): + """DEFAULT_ADMIN_PASSWORD unset: seed_auth_step must rotate admin OFF the shipped + default to a generated secret persisted to the keys dir.""" + session_factory, tmp_path = admin_default_db + monkeypatch.setattr(startup_steps.settings, "DEFAULT_ADMIN_PASSWORD", None) + assert _admin_default_still_works(session_factory) is True # vulnerable pre-run + + startup_steps.seed_auth_step() + + assert _admin_default_still_works(session_factory) is False + assert (tmp_path / "initial_admin_password").exists() + + +def test_upgrade_admin_default_rotated_to_configured_value(admin_default_db, monkeypatch): + """DEFAULT_ADMIN_PASSWORD set to a real value: admin is rotated TO that value so + the documented source (env/Secret) is authoritative; the default stops working.""" + session_factory, _ = admin_default_db + monkeypatch.setattr( + startup_steps.settings, "DEFAULT_ADMIN_PASSWORD", "operator-chosen-secret-value" + ) + + startup_steps.seed_auth_step() + + assert _admin_default_still_works(session_factory) is False + db = session_factory() + try: + assert authenticate_user(db, "admin", "operator-chosen-secret-value") + finally: + db.close() + + +def test_upgrade_admin_default_configured_known_default_is_refused(admin_default_db, monkeypatch): + """DEFAULT_ADMIN_PASSWORD itself a known published default (`changeme`): it must + NOT be used (that would re-publish the hole) — admin rotates to a generated + secret and the default no longer authenticates.""" + session_factory, tmp_path = admin_default_db + monkeypatch.setattr(startup_steps.settings, "DEFAULT_ADMIN_PASSWORD", ADMIN_DEFAULT) + + startup_steps.seed_auth_step() + + assert _admin_default_still_works(session_factory) is False + assert (tmp_path / "initial_admin_password").exists() + + +def test_fresh_seed_refuses_known_default_admin_password(monkeypatch, tmp_path): + """bonnyr-f5 #193 (minor): a FRESH install with DEFAULT_ADMIN_PASSWORD set to a + known published default must not seed that credential verbatim — it generates a + strong random one instead (the MCP path already refused its defaults).""" + engine = create_engine( + "sqlite://", + connect_args={"check_same_thread": False}, + poolclass=StaticPool, + ) + Base.metadata.create_all(bind=engine) + session_factory = sessionmaker(autocommit=False, autoflush=False, bind=engine) + monkeypatch.setenv("KEYS_DIR", str(tmp_path)) + monkeypatch.setattr(startup_steps.settings, "DEFAULT_ADMIN_PASSWORD", ADMIN_DEFAULT) + + db = session_factory() + try: + admin = seed_admin_user(db) + assert admin is not None # fresh install: a row was created + # The published default must NOT authenticate; a random one was generated. + with pytest.raises(UnauthorizedError): + authenticate_user(db, "admin", ADMIN_DEFAULT) + finally: + db.close() + engine.dispose() + assert (tmp_path / "initial_admin_password").exists() diff --git a/backend/tests/unit/test_auth_middleware.py b/backend/tests/unit/test_auth_middleware.py index 66e3afd..18dc896 100644 --- a/backend/tests/unit/test_auth_middleware.py +++ b/backend/tests/unit/test_auth_middleware.py @@ -101,3 +101,128 @@ def test_get_benchmarks_agents_no_auth_returns_401(self, client): def test_get_benchmark_agent_by_id_no_auth_returns_401(self, client): resp = client.get("/api/benchmarks/agents/1") assert resp.status_code == 401 + + +# ── API-token (bnk_) branch: verified + gated via the middleware ───────────── + + +class TestApiTokenBranchGatedByMiddleware: + """#186 r5 (bonnyr-f5, Minor): the bnk_ API-token branch had no coverage. + + It is verified in the middleware (not deferred to the route) and runs the + must-change gate. Both the verify and the gate now run through + run_in_threadpool so the sync DB session never blocks the event loop; these + tests exercise that path end to end with a stubbed verifier. + """ + + def _client_with_verifier(self, monkeypatch, user): + from core import auth_middleware as mw_module + app = _make_app(require_auth=True) + + @app.get("/api/projects") + async def _projects(request: Request): # a dependency-less /api route + return JSONResponse({"ok": True}) + + # monkeypatch auto-restores the real verifier after the test, so this stub + # never leaks into the other TestMiddlewareVerifiesApiTokens cases. + monkeypatch.setattr(mw_module, "_verify_api_token", lambda token: user) + return TestClient(app, raise_server_exceptions=True) + + def test_api_token_must_change_user_is_403(self, monkeypatch): + class _U: + must_change_password = True + client = self._client_with_verifier(monkeypatch, _U()) + resp = client.get("/api/projects", headers={"Authorization": "Bearer bnk_deadbeef"}) + assert resp.status_code == 403 + assert resp.json()["error"]["code"] == "PASSWORD_CHANGE_REQUIRED" + + def test_api_token_settled_user_passes(self, monkeypatch): + class _U: + must_change_password = False + client = self._client_with_verifier(monkeypatch, _U()) + resp = client.get("/api/projects", headers={"Authorization": "Bearer bnk_deadbeef"}) + assert resp.status_code == 200 + assert resp.json() == {"ok": True} + + +# ── JWT branch fails CLOSED on an unresolvable subject (bonnyr-f5 #193 minor) ── + + +class TestUnresolvableJwtSubjectRefused: + """A VALID JWT whose user cannot be resolved (deleted/disabled/DB error) must be + REFUSED with 401 — token_user_state returns None on any such failure and the + middleware must fail closed, never pass through on the ~32 dependency-less /api + routes (bonnyr-f5 #186 r2 called the earlier skip a fail-open bypass). Untested + until now.""" + + def _client(self): + app = _make_app(require_auth=True) + + @app.get("/api/projects") + async def _projects(request: Request): # a dependency-less /api route + return JSONResponse({"ok": True}) + + return TestClient(app, raise_server_exceptions=True) + + def test_valid_jwt_unresolvable_user_is_401(self, monkeypatch): + from services.auth_service import create_access_token + + # decode_token succeeds (real JWT), but the user can't be resolved. + monkeypatch.setattr("services.auth_service.token_user_state", lambda token: None) + token = create_access_token(data={"sub": "ghost", "role": "admin"}) + resp = self._client().get( + "/api/projects", headers={"Authorization": f"Bearer {token}"} + ) + assert resp.status_code == 401 + assert resp.json()["error"]["code"] == "UNAUTHORIZED" + + def test_valid_jwt_resolvable_settled_user_passes(self, monkeypatch): + from services.auth_service import create_access_token + + class _U: + must_change_password = False + + monkeypatch.setattr("services.auth_service.token_user_state", lambda token: _U()) + token = create_access_token(data={"sub": "real", "role": "admin"}) + resp = self._client().get( + "/api/projects", headers={"Authorization": f"Bearer {token}"} + ) + assert resp.status_code == 200 + assert resp.json() == {"ok": True} + + +# ── must-change exempt-path matching is EXACT, not suffix (bonnyr-f5 #193 minor) ── + + +class TestPasswordChangeExemptPathIsExact: + """enforce_password_change gates a must-change user off everything but the exempt + endpoints, and the match is EXACT (path.rstrip('/') in the frozenset) — a security + gate must not accept an unrelated route that merely ENDS WITH '/auth/me'. Untested + until now.""" + + class _MustChange: + must_change_password = True + + def test_exact_exempt_paths_are_allowed(self): + from services.auth_service import enforce_password_change + + # No raise on the exact exempt paths (trailing slash tolerated by rstrip). + enforce_password_change("/api/auth/me", self._MustChange()) + enforce_password_change("/api/auth/me/", self._MustChange()) + enforce_password_change("/api/auth/change-password", self._MustChange()) + + def test_suffix_lookalike_paths_are_not_exempt(self): + from core.errors import ForbiddenError + from services.auth_service import enforce_password_change + + for path in ("/api/evil/auth/me", "/api/auth/me/extra", "xxx/api/auth/me"): + with pytest.raises(ForbiddenError): + enforce_password_change(path, self._MustChange()) + + def test_settled_user_is_never_gated(self): + from services.auth_service import enforce_password_change + + class _Settled: + must_change_password = False + + enforce_password_change("/api/anything/at/all", _Settled()) # no raise diff --git a/backend/tests/unit/test_backup_service.py b/backend/tests/unit/test_backup_service.py index fea8421..e42cbdc 100644 --- a/backend/tests/unit/test_backup_service.py +++ b/backend/tests/unit/test_backup_service.py @@ -334,3 +334,33 @@ def test_returns_in_progress_with_operation_restore(self): assert result["operation"] == "restore" assert result["started_at"] == fake_status["started_at"] assert result["message"] == fake_status["message"] + + +class TestReplaceEncryptionKeyProvenance: + """bonnyr-f5 #193 B-3 (r4 self-review): a restored at-rest key must be marked + `.operator` so the next boot classifies it operator-provisioned (passes + validate_production) and core.config never treats it as a clobberable auto-gen. + Without the marker the restored key would fail the production fail-fast gate.""" + + def test_replace_encryption_key_writes_operator_marker(self, tmp_path, monkeypatch): + import json as _json + + from cryptography.fernet import Fernet + + import core.encryption as enc_mod + import services.backup_service as bs_mod + from core.encryption import wrap_fernet_key + + key_file = tmp_path / "encryption.key" + monkeypatch.setattr(enc_mod, "ENCRYPTION_KEY_FILE", str(key_file)) + monkeypatch.setattr(bs_mod, "ENCRYPTION_KEY_FILE", str(key_file)) + + raw = Fernet.generate_key() + passphrase = "correct horse battery staple" + wrapped_path = tmp_path / "wrapped_key.enc" + wrapped_path.write_text(_json.dumps(wrap_fernet_key(raw, passphrase))) + + _make_service()._replace_encryption_key(str(wrapped_path), passphrase) + + assert key_file.read_bytes() == raw # restored key survives on disk + assert (tmp_path / "encryption.key.operator").is_file() # marked operator diff --git a/backend/tests/unit/test_benchmark_agent_auth.py b/backend/tests/unit/test_benchmark_agent_auth.py index c69e2d4..e4a993b 100644 --- a/backend/tests/unit/test_benchmark_agent_auth.py +++ b/backend/tests/unit/test_benchmark_agent_auth.py @@ -118,8 +118,13 @@ def test_register_rejects_invalid_token(self, client): assert resp.status_code == 400 assert "AGENT_AUTH_INVALID" in resp.text - def test_register_accepts_valid_token(self, client, admin_headers): - """Flag on + valid JWT → accepted (not a 400/401).""" + def test_register_accepts_valid_token(self, client, sample_user, admin_headers): + """Flag on + valid JWT for a real, live admin → accepted (not a 400/401). + + #186 (bonnyr-f5): the gate fails CLOSED on a token that resolves to no + live User, so the token must correspond to a real row (sample_user is + 'testadmin', which admin_headers is issued for) -- as any human token + does, since a token is only minted after that user logs in.""" with patch("routes.benchmarks.settings") as mock_settings: mock_settings.BENCHMARK_AGENT_AUTH_REQUIRED = True resp = client.post( @@ -161,8 +166,17 @@ def test_register_accepts_agent_role_token(self, client): ) assert resp.status_code in (200, 201), resp.text - def test_register_accepts_operator_token(self, client): - """The documented human curl flow keeps working with an operator token.""" + def test_register_accepts_operator_token(self, client, db): + """The documented human curl flow keeps working with an operator token. + + #186 (bonnyr-f5): the token must resolve to a real, live operator -- the + gate fails CLOSED on a token for a user that does not exist or is + disabled. A real curl operator always has a row (that is how they got the + token), so create one, unlike a forged token for a phantom user.""" + from services.auth_service import create_user + create_user(db, "op", "op@t.com", "pw-op-123", + role="operator", must_change_password=False) + db.commit() with patch("routes.benchmarks.settings") as mock_settings: mock_settings.BENCHMARK_AGENT_AUTH_REQUIRED = True with patch("core.auth_middleware.settings") as mw_settings: @@ -174,6 +188,45 @@ def test_register_accepts_operator_token(self, client): ) assert resp.status_code in (200, 201), resp.text + def test_register_refuses_a_must_change_user(self, client, db): + """#186 r2 (bonnyr-f5): a real must-change admin/operator was able to + create an agent (201) because this path never gated must_change. A token + that resolves to a real user owing a password change must be refused.""" + from services.auth_service import create_access_token, create_user + create_user(db, "mc-admin", "mc-admin@t.com", "pw", + role="admin", must_change_password=True) + db.commit() + token = create_access_token({"sub": "mc-admin", "role": "admin"}) + with patch("routes.benchmarks.settings") as mock_settings: + mock_settings.BENCHMARK_AGENT_AUTH_REQUIRED = True + with patch("core.auth_middleware.settings") as mw_settings: + mw_settings.REQUIRE_AUTH = False + resp = client.post( + "/api/benchmarks/agents", + json=_register_payload(), + headers={"Authorization": f"Bearer {token}"}, + ) + assert resp.status_code == 400, resp.text + assert resp.json()["error"]["code"] == "AGENT_AUTH_PASSWORD_CHANGE_REQUIRED" + + def test_register_rejects_nonexistent_user_token(self, client): + """#186 (bonnyr-f5): INV-20 fail-open. A validly-signed admin/operator + token that resolves to NO live User (deleted/disabled row, or a phantom + subject) must be refused -- token_user_state's contract is fail CLOSED. + Previously the `if agent_user is not None:` had no else, so None skipped + the gate and this returned 201.""" + with patch("routes.benchmarks.settings") as mock_settings: + mock_settings.BENCHMARK_AGENT_AUTH_REQUIRED = True + with patch("core.auth_middleware.settings") as mw_settings: + mw_settings.REQUIRE_AUTH = False + resp = client.post( + "/api/benchmarks/agents", + json=_register_payload(), + headers=self._headers_for(sub="phantom-admin", role="admin"), + ) + assert resp.status_code == 400, resp.text + assert "AGENT_AUTH_INVALID" in resp.text + def test_register_rejects_token_with_no_role(self, client): """A token with no role claim must fail closed, not fall through.""" with patch("routes.benchmarks.settings") as mock_settings: @@ -371,3 +424,87 @@ def test_matching_agent_id_passes(self): token_agent_id = payload.get("agent_id") path_agent_id = 7 assert int(token_agent_id) == path_agent_id + + +class TestAgentWSLayer2MustChangeGate: + """#186 (bonnyr-f5 r4, INV-10): the agent WS Layer-2 (global JWT) branch + must enforce the must-change gate the other five JWT entry points enforce. + + token_user_state opens its own DB session, so these unit tests patch it to + isolate the gate LOGIC in _agent_ws_authorized (DB-backed behavior is + covered by token_user_state's own component tests). + """ + + def _ws(self, token): + from unittest.mock import MagicMock + ws = MagicMock() + ws.query_params = {"token": token} + return ws + + def test_rejects_must_change_human_admin(self): + # The reproduced bug: a valid admin token owing a password change was + # admitted (returned None). It must now close 4001. + from unittest.mock import MagicMock, patch + + from routes.benchmarks import _agent_ws_authorized + from services.auth_service import create_access_token + + token = create_access_token(data={"sub": "admin", "role": "admin"}) + must_change_user = MagicMock(must_change_password=True) + with ( + patch("core.config.settings.BENCHMARK_AGENT_AUTH_REQUIRED", False), + patch("core.config.settings.REQUIRE_AUTH", True), + patch("services.auth_service.token_user_state", return_value=must_change_user), + ): + assert _agent_ws_authorized(self._ws(token), 5) == 4001 + + def test_admits_settled_human_admin(self): + from unittest.mock import MagicMock, patch + + from routes.benchmarks import _agent_ws_authorized + from services.auth_service import create_access_token + + token = create_access_token(data={"sub": "admin", "role": "admin"}) + settled_user = MagicMock(must_change_password=False) + with ( + patch("core.config.settings.BENCHMARK_AGENT_AUTH_REQUIRED", False), + patch("core.config.settings.REQUIRE_AUTH", True), + patch("services.auth_service.token_user_state", return_value=settled_user), + ): + assert _agent_ws_authorized(self._ws(token), 5) is None + + def test_rejects_human_token_resolving_to_no_user(self): + # Fail closed: a signed token whose subject no longer resolves (deleted/ + # disabled) must be refused, not waved through. + from unittest.mock import patch + + from routes.benchmarks import _agent_ws_authorized + from services.auth_service import create_access_token + + token = create_access_token(data={"sub": "ghost", "role": "admin"}) + with ( + patch("core.config.settings.BENCHMARK_AGENT_AUTH_REQUIRED", False), + patch("core.config.settings.REQUIRE_AUTH", True), + patch("services.auth_service.token_user_state", return_value=None), + ): + assert _agent_ws_authorized(self._ws(token), 5) == 4001 + + def test_agent_role_token_admitted_without_user_lookup(self): + # Agent tokens carry no User row; they must still connect on this path + # (agent auth off, global JWT on) and must NOT be gated via token_user_state. + from unittest.mock import patch + + from routes.benchmarks import _agent_ws_authorized + from services.auth_service import create_access_token + + token = create_access_token(data={"sub": "forge-agent", "role": "agent"}) + + def _boom(*_a, **_k): # token_user_state must not be consulted for agents + raise AssertionError("token_user_state should not be called for an agent token") + + with ( + patch("core.config.settings.BENCHMARK_AGENT_AUTH_REQUIRED", False), + patch("core.config.settings.REQUIRE_AUTH", True), + patch("services.auth_service.token_user_state", _boom), + ): + assert _agent_ws_authorized(self._ws(token), 5) is None diff --git a/backend/tests/unit/test_core_config.py b/backend/tests/unit/test_core_config.py index 1dfd577..f0c2270 100644 --- a/backend/tests/unit/test_core_config.py +++ b/backend/tests/unit/test_core_config.py @@ -7,9 +7,32 @@ """ import pytest +from cryptography.fernet import Fernet +from core import config as config_mod from core.config import Settings, _read_version_file, settings +# A syntactically valid Fernet key — ENCRYPTION_KEY is now CONSUMED as the at-rest +# key (bonnyr-f5 #193 B-3), so production/explicit-key tests must supply a real one. +VALID_FERNET = Fernet.generate_key().decode() + + +@pytest.fixture(autouse=True) +def _isolate_keys(tmp_path, monkeypatch): + """Give every test in this module a hermetic keys volume (bonnyr-f5 #193 B-3). + + ENCRYPTION_KEY, when set, is now written to the at-rest key file, and the + provenance flag reflects that file. Point both the JWT dir (``_KEYS_DIR``) and the + encryption key file (``ENCRYPTION_KEY_FILE``, which ``_encryption_key_path()`` + prefers) at THIS test's ``tmp_path`` so Settings() constructions never touch the + shared conftest key file or each other. ``tmp_path`` is the same object the test + body sees, so a test that pre-seeds ``tmp_path / "encryption.key"`` seeds exactly + the file the code resolves. + """ + monkeypatch.setattr(config_mod, "_KEYS_DIR", str(tmp_path)) + monkeypatch.setenv("ENCRYPTION_KEY_FILE", str(tmp_path / "encryption.key")) + + # ── Global settings defaults ───────────────────────────────────────── @@ -115,7 +138,8 @@ def test_production_with_explicit_keys_passes(self): s = Settings( ENVIRONMENT="production", JWT_SECRET_KEY="explicit-jwt-key-for-production-use", - ENCRYPTION_KEY="explicit-encryption-key-for-production", + ENCRYPTION_KEY=VALID_FERNET, # B-3: must be a real Fernet key now + MCP_SERVICE_PASSWORD="explicit-mcp-service-secret", # #187: required ALLOWED_ORIGINS="https://my-app.example.com", ) # Explicit keys set _auto_generated to False @@ -124,40 +148,384 @@ def test_production_with_explicit_keys_passes(self): # Should not raise s.validate_production() - def test_production_wildcard_cors_fails(self): - """Production with wildcard CORS should fail.""" + def test_production_invalid_fernet_encryption_key_fails_at_construction(self): + """bonnyr-f5 #193 B-3: ENCRYPTION_KEY is consumed as the at-rest key, so an + invalid Fernet value (e.g. the OLD decoy `secrets.token_hex(16)`) must fail + CLEARLY at construction — not pass the gate while encrypting nothing.""" + import secrets as _secrets + with pytest.raises(SystemExit): + Settings( + ENVIRONMENT="production", + JWT_SECRET_KEY="explicit-jwt-key-for-production-use", + ENCRYPTION_KEY=_secrets.token_hex(16), # NOT a Fernet key + MCP_SERVICE_PASSWORD="explicit-mcp-service-secret", + ALLOWED_ORIGINS="https://my-app.example.com", + ) + + def test_production_without_mcp_service_password_fails(self): + """#187: MCP_SERVICE_PASSWORD unset in prod must fail fast.""" s = Settings( ENVIRONMENT="production", - JWT_SECRET_KEY="explicit-key", - ENCRYPTION_KEY="explicit-key", - ALLOWED_ORIGINS="*", + JWT_SECRET_KEY="explicit-jwt-key-for-production-use", + ENCRYPTION_KEY=VALID_FERNET, + ALLOWED_ORIGINS="https://my-app.example.com", + MCP_SERVICE_PASSWORD=None, ) with pytest.raises(SystemExit): s.validate_production() - def test_production_localhost_cors_fails(self): - """Production with localhost in CORS should fail.""" + def test_production_with_default_mcp_password_fails(self): + """#187: the known shipped default must also fail, not just unset.""" s = Settings( ENVIRONMENT="production", - JWT_SECRET_KEY="explicit-key", - ENCRYPTION_KEY="explicit-key", - ALLOWED_ORIGINS="http://localhost:3000", + JWT_SECRET_KEY="explicit-jwt-key-for-production-use", + ENCRYPTION_KEY=VALID_FERNET, + ALLOWED_ORIGINS="https://my-app.example.com", + MCP_SERVICE_PASSWORD="mcp-service-changeme", + ) + with pytest.raises(SystemExit): + s.validate_production() + + def _valid_prod_kwargs(self, **overrides): + """Everything a production Settings needs to PASS, so a single mutated field + is the sole reason a test fails — no accidental trip on another gate.""" + base = dict( + ENVIRONMENT="production", + JWT_SECRET_KEY="explicit-jwt-key-for-production-use", + ENCRYPTION_KEY=VALID_FERNET, + MCP_SERVICE_PASSWORD="explicit-mcp-service-secret", + ALLOWED_ORIGINS="https://my-app.example.com", ) + base.update(overrides) + return base + + def test_production_wildcard_cors_fails(self): + """A wildcard ORIGIN ENTRY fails — with everything else valid, so the + wildcard check is the sole trip (mutation-proof).""" + s = Settings(**self._valid_prod_kwargs(ALLOWED_ORIGINS="*")) with pytest.raises(SystemExit): s.validate_production() + def test_production_origin_containing_star_is_not_a_wildcard(self): + """Flagged outside every slice: `"*" in ALLOWED_ORIGINS` was a SUBSTRING + test, so a legitimate subdomain-wildcard origin was wrongly rejected. With + the fix it checks the parsed list for an exact `*` entry, so an origin that + merely CONTAINS `*` must PASS.""" + s = Settings(**self._valid_prod_kwargs(ALLOWED_ORIGINS="https://*.example.com")) + # Must not raise — this is a specific origin, not the wildcard entry. + s.validate_production() + + def test_production_localhost_cors_fails(self): + """Production with localhost in CORS should fail — everything else valid so + the localhost check is the SOLE reason (the old test tripped on the MCP gate, + making it vacuous; a mutation removing the localhost check must now break it).""" + s = Settings(**self._valid_prod_kwargs(ALLOWED_ORIGINS="http://localhost:3000")) + with pytest.raises(SystemExit): + s.validate_production() + + def test_production_non_localhost_cors_passes(self): + """Positive control for the above: the SAME otherwise-valid settings with a + real domain must PASS, proving the localhost branch is what fails the test.""" + s = Settings(**self._valid_prod_kwargs(ALLOWED_ORIGINS="https://forge.example.com")) + s.validate_production() + def test_staging_skips_localhost_check(self): """Staging mode checks auto-keys but NOT localhost in CORS.""" s = Settings( ENVIRONMENT="staging", JWT_SECRET_KEY="explicit-key", - ENCRYPTION_KEY="explicit-key", + ENCRYPTION_KEY=VALID_FERNET, + MCP_SERVICE_PASSWORD="explicit-mcp-service-secret", # #187: required ALLOWED_ORIGINS="http://localhost:3000", ) # Should not raise — staging allows localhost s.validate_production() +# ── Production validation is SATISFIABLE from the shipped env (bonnyr-f5 #193 B2) ── + + +class TestProductionValidationSatisfiableFromEnv: + """bonnyr-f5 #193 B2: ENVIRONMENT=production is documented as the hardening + switch, and the compose files now plumb every var validate_production gates on + (MCP_SERVICE_PASSWORD, JWT_SECRET_KEY, ENCRYPTION_KEY, ALLOWED_ORIGINS). Freeze + that contract: each gated var must both (a) TRIP the fail-fast when left at the + shipped-empty/wildcard default, and (b) CLEAR it when set to a real value — so + the shipped .env can actually satisfy production on every path, not just trip it. + """ + + # A full set of operator-supplied real values (what a hardened .env delivers). + # bonnyr-f5 #193 B-3: ENCRYPTION_KEY is now consumed as the at-rest key, so it + # must be a valid Fernet key here. + REAL = { + "JWT_SECRET_KEY": "a-real-jwt-secret-key-value-that-is-long-enough", + "ENCRYPTION_KEY": VALID_FERNET, + "MCP_SERVICE_PASSWORD": "a-real-mcp-service-shared-secret", + "ALLOWED_ORIGINS": "https://forge.example.com", + } + + # The value each var carries when the operator has NOT set it, exactly as the + # compose anchors deliver it: keys/password default to "" (empty), CORS to "*". + SHIPPED_DEFAULT = { + "JWT_SECRET_KEY": "", + "ENCRYPTION_KEY": "", + "MCP_SERVICE_PASSWORD": "", + "ALLOWED_ORIGINS": "*", + } + + def test_all_real_values_pass(self): + """Every var set to a real value → production validation passes.""" + Settings(ENVIRONMENT="production", **self.REAL).validate_production() + + @pytest.mark.parametrize("var", sorted(REAL.keys())) + def test_each_gated_var_trips_then_is_satisfiable(self, var, tmp_path): + # Only this one var left at its shipped default → must fail fast. + env = dict(self.REAL) + env[var] = self.SHIPPED_DEFAULT[var] + with pytest.raises(SystemExit): + Settings(ENVIRONMENT="production", **env).validate_production() + # Phase 1 with ENCRYPTION_KEY unset persists a marker-less auto-gen key; the + # at-rest key file is NEVER silently overwritten (r4 B-3 self-review), so an + # operator provisioning a real value does so on a clean keys volume. Clear it + # to reflect a provisioned deploy rather than one still carrying that auto-gen. + (tmp_path / "encryption.key").unlink(missing_ok=True) + (tmp_path / "encryption.key.operator").unlink(missing_ok=True) + # Restoring a real value for it (all others already real) → passes. + env[var] = self.REAL[var] + Settings(ENVIRONMENT="production", **env).validate_production() + + def test_empty_string_keys_are_treated_as_unset(self): + """bonnyr-f5 #193 B2: compose delivers `${JWT_SECRET_KEY:-}` = "" when the + operator does not set it. An empty value must count as unset (auto-generated, + flagged) — not as an explicitly-provided empty key that would pass.""" + s = Settings(ENVIRONMENT="production", JWT_SECRET_KEY="", ENCRYPTION_KEY="") + assert s._jwt_key_auto_generated is True + assert s._encryption_key_auto_generated is True + # And an auto-generated (non-empty) value is still present for the app to use. + assert s.JWT_SECRET_KEY + assert s.ENCRYPTION_KEY + + def test_all_shipped_defaults_raise(self): + """The pure default path (nothing set) still fails fast — the switch is real.""" + with pytest.raises(SystemExit): + Settings(ENVIRONMENT="production", **self.SHIPPED_DEFAULT).validate_production() + + +# ── _persist_or_load_key provenance marker (bonnyr-f5 #193 B2) ──────── + + +class TestPersistOrLoadKeyProvenance: + """bonnyr-f5 #193 B2 (round 3): provenance FAILS CLOSED. A key WE generate is + auto_generated=True and stays so across restarts (no marker is written — a + marker-less key already means auto-generated). A key the OPERATOR pre-seeds is + auto_generated=False ONLY when the operator drops an explicit ``.operator`` + opt-out marker beside it; a marker-less pre-seeded key (indistinguishable from a + prior-release auto-gen — the whole upgrade population) is treated as + auto-generated so SEC-006's fail-fast still fires in production.""" + + def test_generated_key_is_flagged_and_stays_flagged(self, tmp_path): + kp = str(tmp_path / "k.key") + key1, auto1 = config_mod._persist_or_load_key(kp, lambda: "generated-value") + assert auto1 is True + assert (tmp_path / "k.key").exists() + # New polarity: generation writes NO marker of any kind. + assert not (tmp_path / "k.key.autogen").exists() + assert not (tmp_path / "k.key.operator").exists() + # Second boot loads from disk; a marker-less key stays flagged auto-generated. + key2, auto2 = config_mod._persist_or_load_key(kp, lambda: "unused") + assert key2 == key1 + assert auto2 is True + + def test_generated_key_file_is_mode_0600(self, tmp_path): + # Minor (bonnyr-f5 #193): the higher-value JWT/ENCRYPTION secret must not be + # written through a 0644 window. os.open(0o600)+fchmod, like the sibling + # _persist_generated_password. Assert the final mode is tight. + import os + import stat + + config_mod._persist_or_load_key(str(tmp_path / "k.key"), lambda: "generated-value") + mode = stat.S_IMODE(os.stat(tmp_path / "k.key").st_mode) + assert mode == 0o600, f"expected 0o600, got {oct(mode)}" + + def test_operator_provisioned_key_is_not_flagged(self, tmp_path): + # Operator drops a key file on the volume AND the explicit .operator opt-out + # marker that asserts "I provisioned this" — the only way to be trusted. + (tmp_path / "k.key").write_text("operator-secret") + (tmp_path / "k.key.operator").write_text("") # any/empty contents + key, auto = config_mod._persist_or_load_key(str(tmp_path / "k.key"), lambda: "unused") + assert key == "operator-secret" + assert auto is False + + def test_directory_named_marker_is_not_provenance(self, tmp_path): + # bonnyr-f5 #193 M-1: os.path.exists accepted a DIRECTORY named .operator + # as provenance. os.path.isfile refuses it, so a key beside a directory-marker + # is still auto-generated (fail closed). + (tmp_path / "k.key").write_text("operator-secret") + (tmp_path / "k.key.operator").mkdir() # a directory, not a regular file + key, auto = config_mod._persist_or_load_key(str(tmp_path / "k.key"), lambda: "unused") + assert key == "operator-secret" + assert auto is True # directory does NOT count + + def test_stale_marker_without_keyfile_stays_fail_closed_across_two_boots(self, tmp_path): + # bonnyr-f5 #193 M-1: the natural rotation gesture — delete the key, keep the + # marker — must NOT heal into a fail-open. Boot 1: key absent + stale marker. + # It must classify auto=True AND refuse to persist a generated key, so boot 2 + # is the SAME situation and stays auto=True — never auto=False on our own key. + (tmp_path / "k.key.operator").write_text("") # marker present, key gone + kp = str(tmp_path / "k.key") + _k1, auto1 = config_mod._persist_or_load_key(kp, lambda: "gen-boot-1") + assert auto1 is True + assert not (tmp_path / "k.key").exists() # NOT persisted (fail closed) + _k2, auto2 = config_mod._persist_or_load_key(kp, lambda: "gen-boot-2") + assert auto2 is True # still fail closed on boot 2 — never downgraded + + def test_previous_release_keyfile_without_marker_is_autogenerated(self, tmp_path): + # bonnyr-f5 #193 B2 (round 3): the UPGRADE population. A keys volume written + # by a previously released version holds the key file with NO marker of any + # kind. Under the OLD polarity this classified as operator-provided (auto=False) + # and SEC-006's fail-fast passed OPEN on an auto-generated secret. Fail closed: + # a marker-less key is auto-generated. + (tmp_path / "k.key").write_text("prior-release-autogen-value") + key, auto = config_mod._persist_or_load_key(str(tmp_path / "k.key"), lambda: "unused") + assert key == "prior-release-autogen-value" + assert auto is True # fail closed + + def test_previous_release_shape_fails_fast_in_production(self, tmp_path, monkeypatch): + # bonnyr-f5 #193 B2 (round 3), end-to-end: seed the PREVIOUS-RELEASE on-disk + # shape (jwt_secret.key + encryption.key present, NO markers) and boot with + # ENVIRONMENT=production. validate_production MUST raise SystemExit — the + # whole point of the upgrade hardening switch. This is the test the round-3 + # review said did not exist. + from core import config as config_mod + + monkeypatch.setattr(config_mod, "_KEYS_DIR", str(tmp_path)) + (tmp_path / "jwt_secret.key").write_text("prior-release-jwt-secret-value") + (tmp_path / "encryption.key").write_text("prior-release-encryption-value") + s = Settings( + ENVIRONMENT="production", + MCP_SERVICE_PASSWORD="a-real-mcp-service-shared-secret", + ALLOWED_ORIGINS="https://forge.example.com", + ) + # Both keys loaded from the marker-less files must be flagged auto-generated. + assert s._jwt_key_auto_generated is True + assert s._encryption_key_auto_generated is True + with pytest.raises(SystemExit): + s.validate_production() + + def test_operator_marker_shape_passes_production(self, tmp_path, monkeypatch): + # The complement: an operator who pre-seeds keys AND drops the .operator + # markers is trusted, so production boots. Proves the opt-out path works. + from core import config as config_mod + + monkeypatch.setattr(config_mod, "_KEYS_DIR", str(tmp_path)) + (tmp_path / "jwt_secret.key").write_text("operator-provisioned-jwt-secret") + (tmp_path / "jwt_secret.key.operator").write_text("") + (tmp_path / "encryption.key").write_text("operator-provisioned-enc-secret") + (tmp_path / "encryption.key.operator").write_text("") + s = Settings( + ENVIRONMENT="production", + MCP_SERVICE_PASSWORD="a-real-mcp-service-shared-secret", + ALLOWED_ORIGINS="https://forge.example.com", + ) + assert s._jwt_key_auto_generated is False + assert s._encryption_key_auto_generated is False + s.validate_production() # must not raise + + def test_partial_write_cannot_downgrade_provenance(self, tmp_path, monkeypatch): + # bonnyr-f5 #193 B2 (round 3) — the "second trigger" is GONE. Under the old + # design a key write that succeeded while the marker write failed left a + # marker-less key that classified as operator-provided (auto=False) on the + # NEXT boot — failing open on a fresh install too. The new design writes NO + # marker on generation, so there is no second write to fail: a key that + # persisted always reloads as auto-generated. Simulate a key that got + # written but (old design) no marker, and confirm it reloads auto=True. + kp = str(tmp_path / "k.key") + # First boot generates + persists the key (no marker written by design). + key1, auto1 = config_mod._persist_or_load_key(kp, lambda: "gen-1") + assert auto1 is True + # There is deliberately no marker to have failed to write. + assert not (tmp_path / "k.key.autogen").exists() + # Next boot: marker-less key reloads as auto-generated — never downgraded. + _key2, auto2 = config_mod._persist_or_load_key(kp, lambda: "unused") + assert auto2 is True + + +# ── ENCRYPTION_KEY is consumed as the ONE at-rest key (bonnyr-f5 #193 B-3) ── + + +class TestEncryptionKeyEnvConsumed: + """B-3: setting ENCRYPTION_KEY no longer guards a value that encrypts nothing. + It IS the at-rest key — validated, written to the key file, provenance=operator. + The at-rest-key consumer (core.encryption) is asserted separately in + tests/unit/test_core_encryption.py; here we prove config's own contract.""" + + def test_valid_env_key_written_to_file_with_marker_and_not_flagged(self, tmp_path): + enc_file = tmp_path / "encryption.key" + s = Settings(ENVIRONMENT="production", ENCRYPTION_KEY=VALID_FERNET) + # The env value is now the at-rest key on disk, with an operator marker. + assert enc_file.read_text().strip() == VALID_FERNET + assert (tmp_path / "encryption.key.operator").is_file() + assert s._encryption_key_auto_generated is False # gate reflects a real key + + def test_row1_reproduction_invalid_env_key_no_longer_passes(self, tmp_path): + # The exact B-3 row 1: ENCRYPTION_KEY set to secrets.token_hex(16) (the OLD + # printed remedy) used to PASS the production gate while the real at-rest key + # was auto-generated and unchecked. Now it fails clearly at construction. + import secrets as _secrets + with pytest.raises(SystemExit): + Settings(ENVIRONMENT="production", ENCRYPTION_KEY=_secrets.token_hex(16)) + + def test_env_key_does_not_overwrite_a_persisted_operator_key(self, tmp_path): + # A DIFFERENT operator-provisioned key (with marker) already occupies the file. + # The persisted key is authoritative: it is NEVER overwritten (that would + # destroy encrypted data) and the boot does NOT brick — the env value is + # ignored with a warning, and core.encryption keeps loading the file's key. + other = Fernet.generate_key().decode() + (tmp_path / "encryption.key").write_text(other) + (tmp_path / "encryption.key.operator").write_text("") + s = Settings(ENVIRONMENT="production", ENCRYPTION_KEY=VALID_FERNET) + assert (tmp_path / "encryption.key").read_text() == other # untouched + assert s.ENCRYPTION_KEY == other # the FILE wins; the gate saw a real key + assert s._encryption_key_auto_generated is False + + def test_env_key_never_clobbers_a_marker_less_persisted_key(self, tmp_path): + # THE DATA-LOSS REGRESSION (r4 self-review). A marker-less key in the file is a + # prior release's auto-gen OR a bare backup restore — and at r3 core.encryption + # used the FILE regardless of the env, so it may hold the key LIVE DATA was + # encrypted under. ENCRYPTION_KEY must NEVER overwrite it. + persisted = Fernet.generate_key().decode() + (tmp_path / "encryption.key").write_text(persisted) # no marker + s = Settings(ENVIRONMENT="development", ENCRYPTION_KEY=VALID_FERNET) + assert (tmp_path / "encryption.key").read_text().strip() == persisted # NOT clobbered + assert s.ENCRYPTION_KEY == persisted # the FILE wins + assert s._encryption_key_auto_generated is True # marker-less -> auto (gate flags it) + + def test_marker_less_persisted_key_fails_production_without_losing_data(self, tmp_path): + # Same upgrade/restore shape under production: the gate must still fire + # (marker-less == auto-generated), AND the persisted key must SURVIVE so the + # operator can bless it (drop a .operator marker) instead of losing data. + persisted = Fernet.generate_key().decode() + (tmp_path / "encryption.key").write_text(persisted) + s = Settings( + ENVIRONMENT="production", + ENCRYPTION_KEY=VALID_FERNET, # ignored: the persisted file wins + JWT_SECRET_KEY="x" * 40, + MCP_SERVICE_PASSWORD="a-real-mcp-secret", + ALLOWED_ORIGINS="https://forge.example.com", + ) + assert s._encryption_key_auto_generated is True # marker-less file -> auto + with pytest.raises(SystemExit): + s.validate_production() # fails specifically on the encryption provenance + assert (tmp_path / "encryption.key").read_text().strip() == persisted # data preserved + + def test_env_unset_generates_valid_fernet_at_rest_key(self, tmp_path): + # No ENCRYPTION_KEY set: the app generates a real Fernet key into the file. + s = Settings(ENVIRONMENT="development") + assert s._encryption_key_auto_generated is True + Fernet(s.ENCRYPTION_KEY.encode()) # must be a valid Fernet key + assert (tmp_path / "encryption.key").read_text().strip() == s.ENCRYPTION_KEY + + # ── Settings Config class ──────────────────────────────────────────── diff --git a/backend/tests/unit/test_core_encryption.py b/backend/tests/unit/test_core_encryption.py index dd34f88..09982e2 100644 --- a/backend/tests/unit/test_core_encryption.py +++ b/backend/tests/unit/test_core_encryption.py @@ -83,6 +83,33 @@ def test_decrypt_wrong_key_raises_decryption_error(self): decrypt_value(encrypted_with_other) +class TestEncryptionKeyIsTheOperatorEnvKey: + """bonnyr-f5 #193 B-3, asserted on the CONSUMER (core.encryption), not the config + flag: when the operator sets ENCRYPTION_KEY, core.config validates it and writes + it to the at-rest key file, so get_encryption_key() — what actually encrypts + stored secrets — returns THAT value. Previously ENCRYPTION_KEY gated + validate_production while a DIFFERENT, auto-generated file key did the encrypting.""" + + def test_operator_env_key_becomes_the_at_rest_key(self, tmp_path, monkeypatch): + from cryptography.fernet import Fernet + + import core.encryption as enc + from core import config as config_mod + + opkey = Fernet.generate_key().decode() + key_file = str(tmp_path / "encryption.key") + monkeypatch.setattr(config_mod, "_KEYS_DIR", str(tmp_path)) + monkeypatch.setenv("ENCRYPTION_KEY_FILE", key_file) + # config validates + writes the env key to the at-rest file. + config_mod.Settings(ENVIRONMENT="production", ENCRYPTION_KEY=opkey) + # The SECOND consumer (this module) loads exactly that key from the file. + monkeypatch.setattr(enc, "ENCRYPTION_KEY_FILE", key_file) + assert enc.get_encryption_key().decode() == opkey + # And it genuinely encrypts/decrypts with it. + cipher = Fernet(enc.get_encryption_key()) + assert cipher.decrypt(cipher.encrypt(b"bigip-admin-password")) == b"bigip-admin-password" + + class TestDecryptValueOrNone: def test_returns_none_on_failure(self): """decrypt_value_or_none returns None instead of raising.""" @@ -98,3 +125,45 @@ def test_none_input_returns_none(self): def test_empty_input_returns_none(self): assert decrypt_value_or_none("") is None + + +class TestAtRestKeyFileNeverRegeneratedOverBytes: + """bonnyr-f5 #193 I-1 (r5): get_encryption_key() must NEVER overwrite an existing + key file that holds bytes -- those bytes may be the key live data is encrypted + under. A truncated / mis-shaped key fails CLOSED (crashloop, recoverable) instead + of being regenerated (silent, permanent data loss on a green boot).""" + + def _point(self, monkeypatch, tmp_path): + import core.encryption as enc + key_file = str(tmp_path / "encryption.key") + monkeypatch.setattr(enc, "ENCRYPTION_KEY_FILE", key_file) + return enc, key_file + + def test_truncated_key_fails_closed_and_is_not_overwritten(self, tmp_path, monkeypatch): + enc, key_file = self._point(monkeypatch, tmp_path) + truncated = b"gAAAAABm" + b"x" * 22 # 30 bytes: non-empty, not a valid Fernet key + with open(key_file, "wb") as f: + f.write(truncated) + with pytest.raises(SystemExit): + enc.get_encryption_key() + # The original bytes survive on disk -- recoverable, not clobbered. + with open(key_file, "rb") as f: + assert f.read() == truncated + + def test_valid_key_is_returned_and_not_overwritten(self, tmp_path, monkeypatch): + from cryptography.fernet import Fernet + enc, key_file = self._point(monkeypatch, tmp_path) + good = Fernet.generate_key() + with open(key_file, "wb") as f: + f.write(good) + assert enc.get_encryption_key() == good + with open(key_file, "rb") as f: + assert f.read() == good # untouched + + def test_absent_file_generates_a_valid_key(self, tmp_path, monkeypatch): + from cryptography.fernet import Fernet + enc, key_file = self._point(monkeypatch, tmp_path) # no file created + key = enc.get_encryption_key() + Fernet(key) # a real Fernet key + with open(key_file, "rb") as f: + assert f.read() == key # persisted for next boot diff --git a/bin/roadmap-add.py b/bin/roadmap-add.py index eb65507..c4ddc83 100755 --- a/bin/roadmap-add.py +++ b/bin/roadmap-add.py @@ -75,7 +75,7 @@ def main(): ap = argparse.ArgumentParser(description="Append an item to docs/roadmap.yaml") ap.add_argument("--section", help="section id (see --list-sections)") ap.add_argument("--title") - ap.add_argument("--status", help="status key (shipped/in_progress/blocked/deferred/planned)") + ap.add_argument("--status", help="status key (shipped/merged/in_progress/blocked/deferred/planned)") ap.add_argument("--refs", default="", help='comma-separated, e.g. "#216,PR #188"') ap.add_argument("--note", default="") ap.add_argument("--group", default="") diff --git a/bin/roadmap-gen.py b/bin/roadmap-gen.py index 5dd30b0..77a3aae 100755 --- a/bin/roadmap-gen.py +++ b/bin/roadmap-gen.py @@ -395,11 +395,12 @@ def main(): print("Wrote %s" % MD_PATH) print("Wrote %s" % HTML_PATH) print( - "Stats: in_progress=%d planned=%d shipped=%d blocked=%d deferred=%d" + "Stats: in_progress=%d planned=%d shipped=%d merged=%d blocked=%d deferred=%d" % ( count_status(data["sections"], "in_progress"), count_status(data["sections"], "planned"), count_status(data["sections"], "shipped"), + count_status(data["sections"], "merged"), count_status(data["sections"], "blocked"), count_status(data["sections"], "deferred"), ) diff --git a/bnk-operator/charts/bnk-operator/Chart.yaml b/bnk-operator/charts/bnk-operator/Chart.yaml index eefd453..285cb08 100644 --- a/bnk-operator/charts/bnk-operator/Chart.yaml +++ b/bnk-operator/charts/bnk-operator/Chart.yaml @@ -3,7 +3,7 @@ name: bnk-operator description: BNK Operator — lightweight agent that connects K8s clusters to BNK-Forge type: application version: 1.1.0 -appVersion: "1.1.0" +appVersion: "3.1.6" keywords: - f5 - bnk diff --git a/bnk-operator/charts/bnk-operator/values.yaml b/bnk-operator/charts/bnk-operator/values.yaml index 8060300..9d26b3a 100644 --- a/bnk-operator/charts/bnk-operator/values.yaml +++ b/bnk-operator/charts/bnk-operator/values.yaml @@ -45,8 +45,8 @@ cwc: # Operator image image: - repository: f5/bnk-operator - tag: "1.2.0" + repository: ghcr.io/f5devcentral/bnk-forge-operator + tag: "3.1.6" pullPolicy: IfNotPresent # Image pull secrets (if using private registry) diff --git a/dist/.env.example b/dist/.env.example index 0eaa12b..0ef0b68 100644 --- a/dist/.env.example +++ b/dist/.env.example @@ -15,19 +15,88 @@ COMPOSE_PROJECT_NAME=bnk-forge # ── Container Registry ────────────────────────────────────────────────────── # Where to pull BNK Forge images from (no trailing slash) -BNK_FORGE_REGISTRY=ghcr.io/your-org -BNK_FORGE_VERSION=3.0.1 +BNK_FORGE_REGISTRY=ghcr.io/f5devcentral +# bonnyr-f5 #193 B1: the default below is DERIVED from the repo VERSION file and +# re-stamped at release by scripts/sync-version-artifacts.sh, so it always names an +# image the same release actually published — never a forward-dated guess. Do NOT +# hand-edit it to a version you haven't confirmed is published, and do NOT set it to +# `latest`: a floating tag can point at an image whose credential contract differs +# from what these compose files assume. Pin a specific published tag only. +BNK_FORGE_VERSION=3.1.6 # ── Database ──────────────────────────────────────────────────────────────── -POSTGRES_PASSWORD=bnkforge_dev_password +# bonnyr-f5 #193 M12: ships EMPTY, not a published default. postgres runs under host +# networking, so a known password is reachable on the host loopback. install.sh +# GENERATES a strong value here on first install (before the DB volume initializes), +# matching the Helm chart and the IBM installer. Set your own strong value to override; +# do NOT ship it back to `bnkforge_dev_password`. +POSTGRES_PASSWORD= # ── Redis ─────────────────────────────────────────────────────────────────── -REDIS_PASSWORD=bnkforge_redis_dev +# bonnyr-f5 #193 M12: ships EMPTY — install.sh generates a strong value (see above). +REDIS_PASSWORD= # ── MCP Server (AI assistant integration) ─────────────────────────────────── -# Must match a valid BNK Forge user. Default: admin/changeme -MCP_USERNAME=admin -MCP_PASSWORD=changeme +# #186/#187: MCP authenticates as the dedicated 'mcp' service account (role=admin, +# no must-change gate), NOT the human admin. Do NOT point it at admin/changeme: +# #184 generates the admin password and gates it, so that wiring 403s every call. +# Set MCP_SERVICE_PASSWORD to a value of your choosing (no shipped default — a +# published one can no longer authenticate); the backend reconciles the 'mcp' +# account to it on every startup. Until you set it, the backend leaves the account +# unseeded (and disables any stale one carried over from an upgrade), so MCP +# integration is simply unavailable — the MCP server cannot authenticate. The hard +# fail-fast (the backend REFUSES TO BOOT without it) fires when ENVIRONMENT=staging +# or production; the compose files now plumb ENVIRONMENT through to the backend +# (default "development", where the check is skipped), so setting ENVIRONMENT below +# actually enables that fail-fast (bonnyr-f5 #193 B3). +# The legacy short name MCP_PASSWORD is still honored as an alias (canonical name +# wins if both are set), but ONLY for a NON-DEFAULT value: the old published default +# was MCP_PASSWORD=changeme, which the backend rejects as a known-default — so an +# .env carrying that legacy value resolves through the alias but leaves MCP disabled +# (install.sh warns when it detects a known-default here). Set a strong secret. +MCP_SERVICE_USERNAME=mcp +MCP_SERVICE_PASSWORD= + +# ── Initial admin credential ───────────────────────────────────────────────── +# bonnyr-f5 #193 B1: leave this COMMENTED OUT to accept the image's own default. +# The compose files declare it as a passthrough (a map entry with no value — +# `docker run -e KEY` semantics), so an unset variable is OMITTED from the container +# environment rather than passed as an empty string — each image then applies its own +# initial-admin default: +# - a generating backend randomises the password on first boot and writes it to +# /app/keys/initial_admin_password (retrieve with +# `docker compose exec backend cat /app/keys/initial_admin_password`); +# - the pinned 3.1.6 image seeds the documented `changeme`, which you change on +# first login (install.sh warns if it detects that default). +# Do NOT set this to an empty value — an empty string would seed an admin account +# whose password the login form rejects (min length 1), locking everyone out. To +# CHOOSE the initial password instead, uncomment and REPLACE the placeholder with a +# strong, non-default value (do not leave it empty — that locks everyone out): +# DEFAULT_ADMIN_PASSWORD=replace-with-a-strong-password + +# ── Environment ───────────────────────────────────────────────────────────── +# "development" (default) runs with relaxed startup checks. Set to "staging" or +# "production" to enforce the security fail-fast: the backend then REFUSES TO BOOT +# unless MCP_SERVICE_PASSWORD AND the four hardening settings below are set to real, +# non-default values. Plumbed to the backend by the compose files. +# ENVIRONMENT=development + +# ── Production hardening (required only when ENVIRONMENT=staging|production) ── +# bonnyr-f5 #193 B2: validate_production gates on these three IN ADDITION to +# MCP_SERVICE_PASSWORD, and the compose files now plumb every one of them to the +# backend — so ENVIRONMENT=production is a switch the SHIPPED .env can actually +# satisfy (previously it bricked the backend into a restart loop demanding values it +# could not deliver). Leave them unset in development (the backend auto-generates and +# persists JWT/ENCRYPTION on the keys volume and defaults CORS to '*'); set all three +# to real values before setting ENVIRONMENT=production. +# - JWT_SECRET_KEY : signs API sessions. Generate: openssl rand -hex 32 +# - ENCRYPTION_KEY : Fernet key for stored secrets. Generate: +# python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())" +# - ALLOWED_ORIGINS : comma-separated exact origins (NO '*', NO localhost in +# production), e.g. https://bnk-forge.company.com +# JWT_SECRET_KEY= +# ENCRYPTION_KEY= +# ALLOWED_ORIGINS=* # ── Container (artifact) engine — Docker socket proxy ─────────────────────── # The container-image deployment engine runs each artifact step as a sibling diff --git a/dist/README.md b/dist/README.md index 7c0c011..3723307 100644 --- a/dist/README.md +++ b/dist/README.md @@ -3,7 +3,7 @@ ## Prerequisites - **Docker Engine 24+** with **Docker Compose v2.24+** -- Access to the BNK Forge container registry (if private) +- Network access to `ghcr.io` (images are public — no registry login required) - 4 GB RAM minimum (8 GB recommended) - 10 GB disk space @@ -12,8 +12,8 @@ ### 1. Download and extract ```bash -tar xzf bnk-forge-3.0.1.tar.gz -cd bnk-forge-3.0.1 +tar xzf bnk-forge-3.1.6.tar.gz +cd bnk-forge-3.1.6 ``` ### 2. Configure @@ -27,25 +27,13 @@ nano .env # Set BNK_FORGE_REGISTRY and passwords | Variable | Description | Example | |---|---|---| -| `BNK_FORGE_REGISTRY` | Container registry URL (no trailing slash) | `ghcr.io/your-org` | -| `BNK_FORGE_VERSION` | Image version tag | `3.0.1` | +| `BNK_FORGE_REGISTRY` | Container registry URL (no trailing slash) | `ghcr.io/f5devcentral` (public) | +| `BNK_FORGE_VERSION` | Image version tag. Ships pre-set to this bundle's version (see the `VERSION` file); leave it as shipped. Do **not** set it to `latest` — a floating tag can resolve to an image whose credential contract differs from this bundle. | *(pre-set — leave as shipped)* | | `POSTGRES_PASSWORD` | PostgreSQL password | *(change for production)* | | `REDIS_PASSWORD` | Redis password | *(change for production)* | +| `MCP_SERVICE_PASSWORD` | Password for the dedicated `mcp` service account (the MCP server receives the same value as `BNK_FORGE_PASSWORD`). Ships **empty** — MCP stays disabled until you set a strong secret. Never `admin`. The legacy name `MCP_PASSWORD` is still honored as an alias for existing `.env` files. | *(required to enable MCP)* | -### 3. Authenticate to registry (if private) - -```bash -# GitHub Container Registry -echo $GITHUB_TOKEN | docker login ghcr.io -u USERNAME --password-stdin - -# Docker Hub -docker login - -# AWS ECR -aws ecr get-login-password | docker login --username AWS --password-stdin ACCOUNT.dkr.ecr.REGION.amazonaws.com -``` - -### 4. Install +### 3. Install **Linux server** (host networking — production): ```bash @@ -59,12 +47,12 @@ chmod +x install.sh ./install.sh --local ``` -### 5. Access +### 4. Access - **Mac/Windows (`--local`):** open **https://localhost** - **Linux server:** open **https://\** — the installer prints the exact URL at the end -Accept the self-signed certificate warning. Login: **admin** / **changeme** +Accept the self-signed certificate warning. Login as **admin**. Set `DEFAULT_ADMIN_PASSWORD` in `.env` before install to choose the initial password; otherwise the backend generates one on first boot and writes it to `/app/keys/initial_admin_password` — retrieve it with `docker compose exec backend cat /app/keys/initial_admin_password` (available on releases that generate an admin password; if the file is absent, use the `DEFAULT_ADMIN_PASSWORD` you set). You'll change it on first login. --- @@ -183,7 +171,7 @@ gunzip -c backup_20260417.sql.gz | docker exec -i bnk-forge-postgres psql -U bnk ## File Structure ``` -bnk-forge-3.0.1/ +bnk-forge-3.1.6/ ├── docker-compose.yml # Main compose (Linux server — host networking) ├── docker-compose.local.yml # Overlay for macOS/Windows (bridge networking) ├── .env.example # Configuration template @@ -235,17 +223,17 @@ This creates `dist/bnk-forge-VERSION.tar.gz` containing all files needed for ins echo $GITHUB_TOKEN | docker login ghcr.io -u USERNAME --password-stdin # Build + push all images for amd64 + arm64 (default) -make push-images BNK_FORGE_REGISTRY=ghcr.io/your-org +make push-images BNK_FORGE_REGISTRY=ghcr.io/f5devcentral # Or push only amd64 (faster, if you don't need ARM) -make push-images BNK_FORGE_REGISTRY=ghcr.io/your-org PLATFORMS=linux/amd64 +make push-images BNK_FORGE_REGISTRY=ghcr.io/f5devcentral PLATFORMS=linux/amd64 ``` -This uses `docker buildx build --push` to build all 6 images (api, worker, beat, frontend, proxy, mcp) for both architectures and push **multi-arch manifest lists** to the registry. Each tag (e.g., `bnk-forge-api:3.0.1`) is a manifest that Docker automatically resolves to the correct platform on `docker pull`. +This uses `docker buildx build --push` to build all 7 images (api, worker, beat, frontend, proxy, mcp, operator) for both architectures and push **multi-arch manifest lists** to the registry. Each tag (e.g., `bnk-forge-api:3.1.6`) is a manifest that Docker automatically resolves to the correct platform on `docker pull`. **Verify the manifest:** ```bash -docker manifest inspect ghcr.io/your-org/bnk-forge-api:3.0.1 +docker manifest inspect ghcr.io/f5devcentral/bnk-forge-api:3.1.6 ``` You should see entries for both `linux/amd64` and `linux/arm64`. @@ -266,18 +254,21 @@ gh release create v${VERSION} dist/bnk-forge-${VERSION}.tar.gz \ ### What `gh release create` does -1. Creates a Git tag (`v3.0.1`) on the current commit -2. Creates a GitHub Release page at `https://github.com/your-org/bnk-forge/releases/tag/v3.0.1` +1. Creates a Git tag (`v3.1.6`) on the current commit +2. Creates a GitHub Release page at `https://github.com/f5devcentral/bnk-forge/releases/tag/v3.1.6` 3. Uploads the tarball as a downloadable release asset ### End-user download URL -After publishing, users can download and install with: +Once a full (non-prerelease) `vX.Y.Z` release with an attached tarball exists, users +download and install with the URL below — substitute the version you actually published +(the example `3.1.6` is illustrative; no release asset exists until you cut one): ```bash -# Download from GitHub Releases -curl -L https://github.com/your-org/bnk-forge/releases/download/v3.0.1/bnk-forge-3.0.1.tar.gz | tar xz -cd bnk-forge-3.0.1 +# Download from GitHub Releases — replace 3.1.6 with your published version +VERSION=3.1.6 +curl -L https://github.com/f5devcentral/bnk-forge/releases/download/v${VERSION}/bnk-forge-${VERSION}.tar.gz | tar xz +cd bnk-forge-${VERSION} ./install.sh ``` diff --git a/dist/docker-compose.local.yml b/dist/docker-compose.local.yml index 90793d5..9d3dc23 100644 --- a/dist/docker-compose.local.yml +++ b/dist/docker-compose.local.yml @@ -24,6 +24,24 @@ x-local-backend-env: &local-backend-env # Bridge mode: reach the socket proxy by service DNS (the base compose's # 127.0.0.1:2375 is the container's own loopback here, not the host). DOCKER_HOST: tcp://docker-socket-proxy:2375 + # MCP service account (see base compose): no shipped default (#186/#187). + # bonnyr-f5 #193 B1/M3: alias the PASSWORD only, and only for a NON-DEFAULT value — + # a legacy MCP_PASSWORD=changeme is a known-default the backend rejects, so it + # resolves through the alias but leaves MCP disabled. The USERNAME is not aliased (a + # legacy MCP_USERNAME=admin must never resolve the service username). + MCP_SERVICE_USERNAME: ${MCP_SERVICE_USERNAME:-mcp} + MCP_SERVICE_PASSWORD: ${MCP_SERVICE_PASSWORD:-${MCP_PASSWORD:-}} + # bonnyr-f5 #193 B3: plumb ENVIRONMENT so staging/production reaches the fail-fast. + ENVIRONMENT: ${ENVIRONMENT:-development} + # bonnyr-f5 #193 B2: plumb the three vars validate_production also gates on. + # bonnyr-f5 #193 B1 (r4): OMIT-when-unset (null-value passthrough) — the pinned 3.1.6 + # backend uses `if self.KEY is None`, so a present-but-empty "" (from `${VAR:-}`) would + # boot with an empty JWT secret / invalid Fernet key rather than auto-generating. A map + # entry with NO value is passthrough: omitted when unset, forwarded when set — letting + # 3.1.6 fall through to None → auto-generate. Set real values in .env for production. + JWT_SECRET_KEY: + ENCRYPTION_KEY: + ALLOWED_ORIGINS: ${ALLOWED_ORIGINS:-*} networks: bnk-local: @@ -124,8 +142,13 @@ services: - "8081:8081" environment: BNK_FORGE_API_URL: http://backend:8000 - BNK_FORGE_USERNAME: ${MCP_USERNAME:-admin} - BNK_FORGE_PASSWORD: ${MCP_PASSWORD:-changeme} + # #186: MCP authenticates as the dedicated 'mcp' service account, NOT the + # human admin. admin/changeme is gone (#184 generates the admin password + # and gates it). Set MCP_SERVICE_PASSWORD in .env (no shipped default — the + # published one can no longer authenticate); the backend reconciles to it. + # bonnyr-f5 #193 B1: PASSWORD aliased from legacy MCP_PASSWORD; USERNAME is not. + BNK_FORGE_USERNAME: ${MCP_SERVICE_USERNAME:-mcp} + BNK_FORGE_PASSWORD: ${MCP_SERVICE_PASSWORD:-${MCP_PASSWORD:-}} MCP_PORT: "8081" postgres-backup: diff --git a/dist/docker-compose.yml b/dist/docker-compose.yml index 286b7de..b48728e 100644 --- a/dist/docker-compose.yml +++ b/dist/docker-compose.yml @@ -11,7 +11,7 @@ # # Prerequisites: # - Docker Engine 24+ with Compose v2.24+ -# - Authenticated to the container registry (if private) +# - Network access to ghcr.io (images are public — no registry login required) # # Configuration: # Copy .env.example to .env and set your passwords before first start. @@ -19,8 +19,15 @@ # ── Registry configuration ────────────────────────────────────────────────── # Set BNK_FORGE_REGISTRY and BNK_FORGE_VERSION in .env or environment: -# BNK_FORGE_REGISTRY=ghcr.io/your-org (no trailing slash) -# BNK_FORGE_VERSION=3.0.1 (or "latest") +# BNK_FORGE_REGISTRY=ghcr.io/f5devcentral (no trailing slash) +# BNK_FORGE_VERSION= (the shipped default below is +# NOT hand-edited: it is DERIVED from the repo VERSION file and re-stamped at +# release by scripts/sync-version-artifacts.sh --write, so it always names an +# image the same release actually published — never a forward-dated guess +# (bonnyr-f5 #193 B1). Do NOT default to `latest`: the floating tag can point +# at an image whose credential contract differs from what this compose file +# assumes. Pin a specific tag only when you have confirmed it ships the same +# guards as the bundle you are installing.) x-backend-env: &backend-env DATABASE_URL: postgresql://bnkforge:${POSTGRES_PASSWORD:-bnkforge_dev_password}@localhost:5432/bnkforge @@ -32,6 +39,58 @@ x-backend-env: &backend-env # networking the proxy publishes to the host loopback, so services reach it # at 127.0.0.1:2375. Overridable via DOCKER_HOST in .env. DOCKER_HOST: ${DOCKER_HOST:-tcp://127.0.0.1:2375} + # #186: plumb the operator-chosen initial admin password to the backend + # (config.py has no env_file, so an unpassed var never reaches the container). + # bonnyr-f5 #193 B1 (r4): OMIT-when-unset. This image pins 3.1.6, whose config.py + # declares `DEFAULT_ADMIN_PASSWORD: str = "changeme"` — a plain str, NOT str|None. A + # present-but-empty "" (what `${VAR:-}` delivers) would OVERRIDE that default, seeding + # admin with "" and locking everyone out (the login schema rejects min_length<1 with + # 422). NOTE: interpolation (`${VAR}` / `${VAR:-}`) always renders a value, so an unset + # var still arrives as ""; only a map entry with NO value (below) is passthrough — + # `docker run -e KEY` semantics: omitted when the var is absent from the env/.env, + # forwarded when set. So each backend applies its OWN default when the operator leaves + # it unset: 3.1.6 → usable "changeme" (operator logs in, is forced to change it); a + # generating backend → random, written to /app/keys/initial_admin_password. + DEFAULT_ADMIN_PASSWORD: + # #186: plumb the must-change gate too, or the seeded admin owes a password + # change no route accepts (login is exempt; every other /api route 403s). + # Defaults to "true" (secure); ephemeral CI overrides to "false" to let the + # e2e suite reach protected routes. + DEFAULT_ADMIN_MUST_CHANGE: ${DEFAULT_ADMIN_MUST_CHANGE:-true} + # #186/#187 (bonnyr-f5 r5): the backend reconciles the mcp account to + # MCP_SERVICE_PASSWORD on every boot and seeds it from the same value the MCP + # server authenticates with; plumb it (and the username) here or the mcp account + # is never seeded and the mcp client cannot authenticate (no secret is generated + # for MCP -- #188-over-#186 consolidation). No shipped default — the operator sets + # MCP_SERVICE_PASSWORD in .env (see .env.example). + # bonnyr-f5 #193 B1/M3: alias the PASSWORD only, and mind what the alias can + # resolve to. The old published dist/.env.example shipped `MCP_PASSWORD=changeme`, + # which the backend rejects as a known-default (MCP_KNOWN_DEFAULT_PASSWORDS) — so + # the alias only actually ENABLES MCP for a NON-DEFAULT legacy value; a legacy + # `changeme`/`mcp-service-changeme` resolves through but leaves MCP disabled + # (install.sh warns on exactly those). The USERNAME is NOT aliased: an existing + # customer .env built from the old published dist/.env.example carries + # `MCP_USERNAME=admin`, and `${...:-${MCP_USERNAME:-mcp}}` would resolve the + # service username to `admin`, which a pre-guard backend uses to rewrite the human + # admin row. A legacy MCP_USERNAME value is never valid here, so take only + # MCP_SERVICE_USERNAME. + MCP_SERVICE_USERNAME: ${MCP_SERVICE_USERNAME:-mcp} + MCP_SERVICE_PASSWORD: ${MCP_SERVICE_PASSWORD:-${MCP_PASSWORD:-}} + # bonnyr-f5 #193 B3: plumb ENVIRONMENT so an operator who sets it to + # staging/production actually reaches config.py's fail-fast (validate_production). + ENVIRONMENT: ${ENVIRONMENT:-development} + # bonnyr-f5 #193 B2: validate_production also gates on these three. Plumb them or + # ENVIRONMENT=production bricks the backend into a restart loop listing problems the + # shipped .env cannot fix. bonnyr-f5 #193 B1 (r4): OMIT-when-unset (null-value + # passthrough, NOT `${VAR:-}`). HEAD's config.py treats "" as unset (`if not self.KEY`), + # but the pinned 3.1.6 backend uses `if self.KEY is None` — a present-but-empty "" (what + # `${VAR:-}` delivers) is NOT auto-generated there, so it would boot with an empty JWT + # secret / an invalid ("") Fernet key. A map entry with NO value is passthrough: omitted + # when unset, forwarded when set — so 3.1.6 falls through to None and auto-generates + + # persists the keys to /app/keys/. Set real values in .env for production. + JWT_SECRET_KEY: + ENCRYPTION_KEY: + ALLOWED_ORIGINS: ${ALLOWED_ORIGINS:-*} x-worker-volumes: &worker-volumes - module_catalog:/tmp/bnk-forge-modules @@ -160,7 +219,7 @@ services: memory: 32M backend: - image: ${BNK_FORGE_REGISTRY:-ghcr.io/your-org}/bnk-forge-api:${BNK_FORGE_VERSION:-latest} + image: ${BNK_FORGE_REGISTRY:-ghcr.io/f5devcentral}/bnk-forge-api:${BNK_FORGE_VERSION:-3.1.6} container_name: bnk-forge-backend network_mode: host logging: *default-logging @@ -201,7 +260,7 @@ services: memory: 256M celery-worker: - image: ${BNK_FORGE_REGISTRY:-ghcr.io/your-org}/bnk-forge-worker:${BNK_FORGE_VERSION:-latest} + image: ${BNK_FORGE_REGISTRY:-ghcr.io/f5devcentral}/bnk-forge-worker:${BNK_FORGE_VERSION:-3.1.6} container_name: bnk-forge-celery-worker network_mode: host logging: *default-logging @@ -233,7 +292,7 @@ services: memory: 512M celery-worker-2: - image: ${BNK_FORGE_REGISTRY:-ghcr.io/your-org}/bnk-forge-worker:${BNK_FORGE_VERSION:-latest} + image: ${BNK_FORGE_REGISTRY:-ghcr.io/f5devcentral}/bnk-forge-worker:${BNK_FORGE_VERSION:-3.1.6} container_name: bnk-forge-celery-worker-2 network_mode: host logging: *default-logging @@ -265,7 +324,7 @@ services: memory: 512M celery-beat: - image: ${BNK_FORGE_REGISTRY:-ghcr.io/your-org}/bnk-forge-beat:${BNK_FORGE_VERSION:-latest} + image: ${BNK_FORGE_REGISTRY:-ghcr.io/f5devcentral}/bnk-forge-beat:${BNK_FORGE_VERSION:-3.1.6} container_name: bnk-forge-celery-beat network_mode: host logging: *default-logging @@ -295,7 +354,7 @@ services: memory: 64M frontend: - image: ${BNK_FORGE_REGISTRY:-ghcr.io/your-org}/bnk-forge-frontend:${BNK_FORGE_VERSION:-latest} + image: ${BNK_FORGE_REGISTRY:-ghcr.io/f5devcentral}/bnk-forge-frontend:${BNK_FORGE_VERSION:-3.1.6} container_name: bnk-forge-frontend network_mode: host logging: *default-logging @@ -321,7 +380,7 @@ services: memory: 32M proxy: - image: ${BNK_FORGE_REGISTRY:-ghcr.io/your-org}/bnk-forge-proxy:${BNK_FORGE_VERSION:-latest} + image: ${BNK_FORGE_REGISTRY:-ghcr.io/f5devcentral}/bnk-forge-proxy:${BNK_FORGE_VERSION:-3.1.6} container_name: bnk-forge-proxy network_mode: host logging: *default-logging @@ -347,26 +406,42 @@ services: memory: 32M mcp: - image: ${BNK_FORGE_REGISTRY:-ghcr.io/your-org}/bnk-forge-mcp:${BNK_FORGE_VERSION:-latest} + image: ${BNK_FORGE_REGISTRY:-ghcr.io/f5devcentral}/bnk-forge-mcp:${BNK_FORGE_VERSION:-3.1.6} container_name: bnk-forge-mcp network_mode: host logging: *default-logging environment: BNK_FORGE_API_URL: http://localhost:8000 - BNK_FORGE_USERNAME: ${MCP_USERNAME:-admin} - BNK_FORGE_PASSWORD: ${MCP_PASSWORD:-changeme} + # #186: MCP authenticates as the dedicated 'mcp' service account, NOT the + # human admin. admin/changeme is gone (#184 generates the admin password + # and gates it), so the old admin/changeme wiring 403s every tool call. + # Set MCP_SERVICE_PASSWORD in .env (no shipped default — the published one + # can no longer authenticate); the backend reconciles the account to it. + # bonnyr-f5 #193 B1: PASSWORD aliased from legacy MCP_PASSWORD; USERNAME is not. + BNK_FORGE_USERNAME: ${MCP_SERVICE_USERNAME:-mcp} + BNK_FORGE_PASSWORD: ${MCP_SERVICE_PASSWORD:-${MCP_PASSWORD:-}} MCP_PORT: "8081" MCP_LOG_LEVEL: INFO depends_on: backend: condition: service_healthy restart: unless-stopped + # Auth-probe healthcheck (bonnyr-f5 #188, INV-10): log in to the backend with + # the configured MCP credentials and exit non-zero on 401. MCP_SERVICE_PASSWORD + # ships empty (see .env.example), so a bare liveness ping would report green + # while every tool call 401s — it only proves the HTTP port answers. This is the + # same probe the dev compose uses. NOTE: the "no credentials at all -> UNHEALTHY" + # signal depends on the probe returning non-zero when no credential is set; that + # behaviour lands with the credential-guard release (the image this bundle pins + # is derived from VERSION and moves to the guard-carrying tag at release, in + # lockstep with the chart). On an older pinned image the probe treats "no + # credentials" as healthy, so verify MCP with a real tool call after install. healthcheck: - test: ["CMD-SHELL", "python -c \"import urllib.request; req=urllib.request.Request('http://localhost:8081/mcp',headers={'Accept':'application/json,text/event-stream','Content-Type':'application/json'},data=b'{\\\"jsonrpc\\\":\\\"2.0\\\",\\\"method\\\":\\\"ping\\\",\\\"id\\\":1}'); urllib.request.urlopen(req)\" 2>/dev/null || exit 1"] + test: ["CMD", "python", "-m", "bnk_forge_mcp.healthcheck"] interval: 30s - timeout: 5s + timeout: 10s retries: 3 - start_period: 15s + start_period: 30s deploy: resources: limits: diff --git a/dist/install.sh b/dist/install.sh index 2c8be22..03d8823 100644 --- a/dist/install.sh +++ b/dist/install.sh @@ -10,7 +10,7 @@ # # Prerequisites: # - Docker Engine 24+ with Compose v2.24+ -# - Authenticated to the container registry (if private) +# - Network access to ghcr.io (images are public — no registry login required) # set -euo pipefail @@ -48,6 +48,41 @@ done cd "$SCRIPT_DIR" +# ── bonnyr-f5 #193 M12: strong per-install DB/redis credentials ─────────────── +# dist/ runs postgres and redis under host networking, so a published default +# password (`bnkforge_dev_password` / `bnkforge_redis_dev`) is a live, known +# credential on the host loopback. The Helm chart and the IBM installer both +# GENERATE these; dist/ must not be the one path that ships them. Generate strong +# values into the fresh .env at creation time (below), before the DB volume +# initializes, and warn if a pre-existing .env still carries a default. +_gen_secret() { # $1 = length in chars (default 32); hex only, so it is sed-safe + local n="${1:-32}" + if command -v openssl >/dev/null 2>&1; then + openssl rand -hex "$(( (n + 1) / 2 ))" | cut -c1-"$n" + else + LC_ALL=C tr -dc 'a-f0-9' > .env + fi +} +_ensure_strong_db_cred() { # $1 = var, $2 = known shipped default. Returns 0 if it generated. + local cur; cur="$(_env_get "$1")" + if [ -z "$cur" ] || [ "$cur" = "$2" ]; then + _env_set "$1" "$(_gen_secret 32)" + return 0 + fi + return 1 +} + # ── Preflight checks ──────────────────────────────────────────────────────── echo "" echo "=========================================" @@ -81,7 +116,17 @@ if [ ! -f .env ]; then if [ -f .env.example ]; then cp .env.example .env echo " ✓ Created .env from .env.example" - echo " ⚠ Review .env and set BNK_FORGE_REGISTRY before continuing." + # bonnyr-f5 #193 M12: replace the published default DB/redis passwords with + # strong per-install secrets NOW, while the .env is fresh and the postgres/redis + # data volumes do not yet exist (rotating them after first boot would lock the + # backend out of an already-initialized DB). Mirrors the Helm chart + IBM installer. + _ensure_strong_db_cred POSTGRES_PASSWORD bnkforge_dev_password || true + _ensure_strong_db_cred REDIS_PASSWORD bnkforge_redis_dev || true + echo " ✓ Generated strong POSTGRES_PASSWORD and REDIS_PASSWORD (host-networked)" + echo " ⚠ Review .env before continuing. In particular:" + echo " - BNK_FORGE_REGISTRY (where to pull images)" + echo " - MCP_SERVICE_PASSWORD (bonnyr-f5 #193: ships EMPTY; the MCP/AI" + echo " assistant integration stays unavailable until you set it)" echo "" echo " Edit: nano .env" echo " Then re-run: ./install.sh $([ "$MODE" = "local" ] && echo "--local")" @@ -92,6 +137,27 @@ if [ ! -f .env ]; then fi fi +# bonnyr-f5 #193 M12: a PRE-EXISTING .env may still carry the published default DB/ +# redis credentials (an older bundle shipped them, or the operator restored them). We +# do NOT silently rotate here: postgres bakes its password into the data volume on +# first init, so a rotate would lock the backend out of an already-initialized DB. +# Warn instead — a known password on a host-networked datastore is a real exposure. +_pg_now="$(_env_get POSTGRES_PASSWORD)" +_rd_now="$(_env_get REDIS_PASSWORD)" +if [ -z "$_pg_now" ] || [ "$_pg_now" = "bnkforge_dev_password" ] \ + || [ -z "$_rd_now" ] || [ "$_rd_now" = "bnkforge_redis_dev" ]; then + echo "" + echo " ⚠ SECURITY: .env still uses a published default database/redis password" + echo " (bnkforge_dev_password / bnkforge_redis_dev). postgres and redis run under" + echo " host networking, so these are known credentials reachable on the host." + echo " For a FRESH install: set strong POSTGRES_PASSWORD and REDIS_PASSWORD in .env" + echo " (e.g. openssl rand -hex 24) BEFORE first boot, then re-run — the DB has not" + echo " initialized yet. For an EXISTING install: the current password is baked into" + echo " the postgres data volume; rotate it deliberately (ALTER USER + update .env)" + echo " rather than by editing .env alone." + echo "" +fi + # Create secrets directory if missing mkdir -p secrets @@ -349,17 +415,83 @@ else URL="https://$HOST_IP" fi +# bonnyr-f5 #193 M3: the shipped .env leaves MCP_SERVICE_PASSWORD empty, so a default +# install brings the mcp service up but its auth probe can never pass (the backend +# leaves the 'mcp' account unseeded). An UPGRADING operator's legacy .env is worse: it +# carries `MCP_PASSWORD=changeme`, which the backend REFUSES as a known-default, so +# MCP is just as dead — but the value is non-empty, so the old check missed it and +# printed "complete!" with no warning (fail-open). Surface all three cases here. +_mcp_strip_quotes() { # normalize $1: trim surrounding whitespace + one quote layer + local v="$1" + # bonnyr-f5 #193 M8: TRIM whitespace as well as quotes, before AND after the quote + # strip. Without this, `MCP_SERVICE_PASSWORD=changeme ` (a stray trailing space, or a + # leading one, or `" changeme "`) slipped past the known-default `case` below, so the + # guard printed "complete!" (fail-open) on a value the backend still treats as the + # `changeme` default and refuses — MCP silently dead. Trim outer space, strip one + # matched quote layer, trim again (covers `" changeme "`). + v="${v#"${v%%[![:space:]]*}"}"; v="${v%"${v##*[![:space:]]}"}" # trim outer whitespace + case "$v" in + \"*\") v="${v#\"}"; v="${v%\"}" ;; # "..." -> compose strips these, so must we + \'*\') v="${v#\'}"; v="${v%\'}" ;; # '...' + esac + v="${v#"${v%%[![:space:]]*}"}"; v="${v%"${v##*[![:space:]]}"}" # trim inside-quote whitespace + printf '%s' "$v" +} +MCP_PW=$(_mcp_strip_quotes "$(grep -E '^[[:space:]]*MCP_SERVICE_PASSWORD=' .env 2>/dev/null | tail -n1 | cut -d= -f2- || true)") +# Fall back to the legacy alias the compose files still honor (MCP_PASSWORD). +if [ -z "$MCP_PW" ]; then + MCP_PW=$(_mcp_strip_quotes "$(grep -E '^[[:space:]]*MCP_PASSWORD=' .env 2>/dev/null | tail -n1 | cut -d= -f2- || true)") +fi +# Treat a known-default (what the backend's MCP_KNOWN_DEFAULT_PASSWORDS rejects) as +# unusable — MCP will not come up on it, so it is functionally unset here. +MCP_USABLE=1 +if [ -z "$MCP_PW" ]; then + MCP_USABLE=0 +else + case "$MCP_PW" in + changeme|mcp-service-changeme) MCP_USABLE=0 ;; + esac +fi + echo "=========================================" echo " ✅ Installation complete!" echo "" echo " Version: $(cat VERSION 2>/dev/null || echo 'unknown')" echo "" +if [ "$MCP_USABLE" = "0" ]; then + echo " ⚠ MCP (AI assistant) integration is NOT active: MCP_SERVICE_PASSWORD is" + echo " unset or set to a known default (e.g. 'changeme', which the backend" + echo " refuses) in .env, so the bundled MCP server cannot authenticate — every" + echo " tool call 401s. The rest of the stack is unaffected." + # bonnyr-f5 #193 (deploy minor): describe what the PINNED image's healthcheck + # actually reports, not an aspirational signal. The mcp auth-probe + # (bnk_forge_mcp.healthcheck) SKIPS the probe and returns healthy when NO + # credential is set (has_credentials == False), so an unset password shows a + # GREEN/healthy mcp container even though it cannot serve; a known-default value + # is non-empty, so the probe runs and the container goes UNHEALTHY on the 401. + echo " Note: with MCP_SERVICE_PASSWORD unset the mcp container may still report" + echo " healthy (the probe is skipped when no credential is set); a known-default" + echo " value instead drives it UNHEALTHY. Either way MCP will not serve — confirm" + echo " with a real tool call rather than the container's health status." + echo " To enable it: set MCP_SERVICE_PASSWORD in .env to a strong, non-default" + echo " value and run" + echo " $COMPOSE_CMD up -d" + echo "" +fi echo " Open: $URL" if [ "$URL" != "https://localhost" ]; then echo " (accept the self-signed certificate warning)" fi echo "" -echo " Login: admin / changeme" +# bonnyr-f5 #193 B1: with DEFAULT_ADMIN_PASSWORD unset the value is OMITTED from the +# container (passthrough), so the pinned image applies its OWN default: an image that +# generates writes it to /app/keys/initial_admin_password; an older pinned image seeds +# its documented built-in default (e.g. "changeme") and that file does not exist. +echo " Login: admin (password: the DEFAULT_ADMIN_PASSWORD you set in .env, if any." +echo " Otherwise, on an image that generates one:" +echo " docker compose exec backend cat /app/keys/initial_admin_password" +echo " If that file does not exist, this image seeds its built-in default" +echo " (e.g. 'changeme') — log in and change it immediately.)" echo "" echo " Next steps:" echo " 1. Change your password on first login" diff --git a/docker-bake.hcl b/docker-bake.hcl index 52e486d..f77b9ad 100644 --- a/docker-bake.hcl +++ b/docker-bake.hcl @@ -19,6 +19,42 @@ variable "GIT_REVISION" { variable "SOURCE_URL" { default = "https://github.com/f5devcentral/bnk-forge" } +# Build timestamp for org.opencontainers.image.created (RFC 3339). Empty by +# default so a plain `docker buildx bake` does not stamp a wall-clock time into +# the image config. +# A timestamp() default stamped a FRESH time into every build, guaranteeing a +# different config digest on every rebuild. CI instead sets CREATED to the +# release commit's committer date (fixed for a given tag) and also exports +# SOURCE_DATE_EPOCH. That removes the two most obvious sources of variance. +# +# It does NOT make a rebuild byte-reproducible, and a republish CAN move the +# digest. The Dockerfiles run `apt-get update` / `apk upgrade` / `pip` / `npm` +# against live package indexes, and there is no buildkit `rewrite-timestamp` +# pass normalizing layer mtimes to SOURCE_DATE_EPOCH — so two builds of the +# same tag can produce different layer diff_ids and a different image digest +# (bonnyr-f5 #181 round 4, verified: two SOURCE_DATE_EPOCH-pinned builds gave +# divergent digests). Because a republish may not resolve to the original +# digest, the immutable :VERSION tag is protected the honest way — an existence +# probe REFUSES a republish by default and requires an explicit force to +# overwrite — rather than by relying on determinism we do not have. +# +# FOUR paths bake --push this file (bonnyr-f5 #193 minor — the old comment said +# "BOTH", there are four). The two that publish the IMMUTABLE release :VERSION +# tag are guarded by the existence probe, single-sourced through +# scripts/registry-overwrite-guard.sh -> scripts/registry-tag-probe.sh: +# • the release workflow (release.yml "Refuse to overwrite an already-published tag") +# • `make push-images` (Makefile, FORCE_LATEST=1 to override) +# The other two deliberately carry NO :VERSION probe, because they never touch +# the release tag — they push a SHA-pinned immutable tag `${BASE}-cb.${SHA}` +# (unique per commit) plus the ROLLING `customer-build` tag (rolling tags are +# MEANT to move): +# • `make push-customer-build` +# • `make push-customer-build-multiarch` +# So the release-tag protection is not fixed at one call site (bonnyr-f5 #181 +# round 5, F3), and the customer-build paths are correctly out of its scope. +variable "CREATED" { + default = "" +} group "default" { targets = ["api", "worker", "beat", "frontend", "proxy", "mcp", "operator"] @@ -26,12 +62,21 @@ group "default" { target "_common" { platforms = split(",", PLATFORMS) - labels = { - "org.opencontainers.image.source" = SOURCE_URL - "org.opencontainers.image.revision" = GIT_REVISION - "org.opencontainers.image.version" = VERSION - "org.opencontainers.image.created" = timestamp() - } + # Omit org.opencontainers.image.created entirely when CREATED is empty instead + # of stamping an empty-string label: an empty value is spec-invalid and + # falsifies the label table in docs/DOCKER.md. A plain `docker buildx bake` + # (e.g. `make push-images`, which does not set CREATED) must not emit the key + # at all; CI sets CREATED to the release commit's committer date. This is the + # same conditional shape the ROLLING_TAG tags use below (bonnyr-f5 #181 round + # 5, F7). + labels = merge( + { + "org.opencontainers.image.source" = SOURCE_URL + "org.opencontainers.image.revision" = GIT_REVISION + "org.opencontainers.image.version" = VERSION + }, + CREATED != "" ? { "org.opencontainers.image.created" = CREATED } : {}, + ) } target "_backend" { diff --git a/docker-compose.adr424.yml b/docker-compose.adr424.yml index f9c34e4..2143aeb 100644 --- a/docker-compose.adr424.yml +++ b/docker-compose.adr424.yml @@ -8,7 +8,7 @@ # ports: !override [...] → replace host port mapping # image: adr424-* → project-scoped image tags # -# Entrypoint: https://localhost:11443 (admin / changeme) +# Entrypoint: https://localhost:11443 (admin / password generated on first start — see backend logs) # Bring up command: # docker compose -p adr424 \ # -f docker-compose.yml -f docker-compose.local.yml -f docker-compose.adr424.yml \ diff --git a/docker-compose.local.yml b/docker-compose.local.yml index 0fa31a8..67c1dd3 100644 --- a/docker-compose.local.yml +++ b/docker-compose.local.yml @@ -38,6 +38,42 @@ x-local-backend-env: &local-backend-env CELERY_BROKER_URL: redis://:${REDIS_PASSWORD:-bnkforge_redis_dev}@redis:6379/0 CELERY_RESULT_BACKEND: redis://:${REDIS_PASSWORD:-bnkforge_redis_dev}@redis:6379/0 TF_PLUGIN_CACHE_DIR: /app/provider-cache + # #186: plumb the operator-chosen initial admin password to the backend + # (config.py has no env_file, so an unpassed var never reaches the container). + # bonnyr-f5 #193 B1 (r4): OMIT-when-unset (null-value passthrough) — a present-but-empty + # "" (from `${VAR:-}`) would OVERRIDE a pinned 3.1.6 image's `DEFAULT_ADMIN_PASSWORD: + # str = "changeme"` and lock admin out (login schema rejects ""). A map entry with NO + # value is passthrough: omitted when unset, forwarded when set — so each backend applies + # its own default (3.1.6 "changeme"; generating backend a random pw). + DEFAULT_ADMIN_PASSWORD: + # #186: plumb the must-change gate too, or the seeded admin owes a password + # change no route accepts (login is exempt; every other /api route 403s). + # Defaults to "true" (secure); ephemeral CI overrides to "false" to let the + # e2e suite reach protected routes. + DEFAULT_ADMIN_MUST_CHANGE: ${DEFAULT_ADMIN_MUST_CHANGE:-true} + # #186/#187 (bonnyr-f5 r5): the backend reconciles the mcp account to + # MCP_SERVICE_PASSWORD on every boot and seeds it from the same .env value the + # MCP server authenticates with; plumb it (and the username) here or the mcp + # account is never seeded and the mcp client cannot authenticate. Unset -> not + # seeded (no secret is generated for MCP -- #188-over-#186 consolidation). + # bonnyr-f5 #193 B1/M3: alias the PASSWORD only, and only for a NON-DEFAULT value — + # a legacy MCP_PASSWORD=changeme is a known-default the backend rejects, so it + # resolves through the alias but leaves MCP disabled. The USERNAME is not aliased — a + # legacy MCP_USERNAME=admin must never resolve the service username (old default admin, + # new default mcp), or a pre-guard backend rewrites the human admin row to changeme. + MCP_SERVICE_USERNAME: ${MCP_SERVICE_USERNAME:-mcp} + MCP_SERVICE_PASSWORD: ${MCP_SERVICE_PASSWORD:-${MCP_PASSWORD:-}} + # bonnyr-f5 #193 B3: plumb ENVIRONMENT so staging/production reaches the fail-fast. + ENVIRONMENT: ${ENVIRONMENT:-development} + # bonnyr-f5 #193 B2: plumb the three vars validate_production also gates on, or + # ENVIRONMENT=production bricks the backend. bonnyr-f5 #193 B1 (r4): OMIT-when-unset + # (null-value passthrough) — a pinned 3.1.6 backend uses `if self.KEY is None`, so a + # present-but-empty "" would boot with an empty JWT secret / invalid Fernet key instead + # of auto-generating. A map entry with NO value is passthrough: omitted when unset, + # forwarded when set; set real values in .env for production. + JWT_SECRET_KEY: + ENCRYPTION_KEY: + ALLOWED_ORIGINS: ${ALLOWED_ORIGINS:-*} # Shared bridge network — all services can resolve each other by service name networks: @@ -140,11 +176,13 @@ services: - "8081:8081" environment: BNK_FORGE_API_URL: http://backend:8000 + # bonnyr-f5 #193 B1: PASSWORD aliased from legacy MCP_PASSWORD; USERNAME is not. BNK_FORGE_USERNAME: ${MCP_SERVICE_USERNAME:-mcp} # MCP authenticates as the dedicated service account seeded by the backend. # Set MCP_SERVICE_PASSWORD in .env; backend reconciles the stored hash on - # every startup so backend and MCP always stay in sync. - BNK_FORGE_PASSWORD: ${MCP_SERVICE_PASSWORD:-mcp-service-changeme} + # every startup so backend and MCP always stay in sync. No shipped default + # (#186): the old mcp-service-changeme can no longer authenticate. + BNK_FORGE_PASSWORD: ${MCP_SERVICE_PASSWORD:-${MCP_PASSWORD:-}} MCP_PORT: "8081" postgres-backup: diff --git a/docker-compose.yml b/docker-compose.yml index c74594a..2db2d46 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -18,6 +18,52 @@ x-backend-env: &backend-env CELERY_BROKER_URL: redis://:${REDIS_PASSWORD:-bnkforge_redis_dev}@localhost:6379/0 CELERY_RESULT_BACKEND: redis://:${REDIS_PASSWORD:-bnkforge_redis_dev}@localhost:6379/0 TF_PLUGIN_CACHE_DIR: /app/provider-cache + # #186: plumb the operator-chosen initial admin password to the backend + # (config.py has no env_file, so an unpassed var never reaches the container). + # bonnyr-f5 #193 B1 (r4): OMIT-when-unset (null-value passthrough). This root compose + # builds from source (HEAD backend), which treats "" as unset and generates — but keep + # the same omit-form as the shipped dist/ compose so a `BNK_FORGE_VERSION`-pinned 3.1.6 + # image (whose `DEFAULT_ADMIN_PASSWORD: str = "changeme"` would be OVERRIDDEN by an + # empty "" and lock admin out) is safe here too. A map entry with NO value is passthrough + # (`docker run -e KEY` semantics): omitted when unset, forwarded when set in .env. + DEFAULT_ADMIN_PASSWORD: + # #186: plumb the must-change gate too, or the seeded admin owes a password + # change no route accepts (login is exempt; every other /api route 403s). + # Defaults to "true" (secure); ephemeral CI overrides to "false" to let the + # e2e suite reach protected routes. + DEFAULT_ADMIN_MUST_CHANGE: ${DEFAULT_ADMIN_MUST_CHANGE:-true} + # #186/#187 (bonnyr-f5 r5): the BACKEND must receive MCP_SERVICE_* too, not just + # the mcp service. ensure_service_user reconciles the stored hash to + # MCP_SERVICE_PASSWORD every boot; the backend seeds the `mcp` service account + # from these and the MCP server authenticates with the SAME .env values. config.py + # has no env_file, so an unpassed var never reaches the container. Unset -> empty + # -> the account isn't seeded and the MCP server can't auth (clean, not a weak default). + # bonnyr-f5 #193 B1/M3: the PASSWORD is aliased from the legacy MCP_PASSWORD, but + # the alias only ENABLES MCP for a NON-DEFAULT legacy value — the old published + # `MCP_PASSWORD=changeme` is a known-default the backend rejects, so it resolves + # through but leaves MCP disabled. The USERNAME is NOT aliased: a legacy MCP_USERNAME value is + # never valid here (old default `admin`, new default `mcp`), and `${...:-${MCP_USERNAME:-mcp}}` + # would resolve a pre-existing `MCP_USERNAME=admin` (shipped by the old dist .env) + # to `admin` — which a pre-guard 3.1.x backend then uses to REWRITE the human admin + # row to `changeme`. So take only MCP_SERVICE_USERNAME, defaulting to `mcp`. + MCP_SERVICE_USERNAME: ${MCP_SERVICE_USERNAME:-mcp} + MCP_SERVICE_PASSWORD: ${MCP_SERVICE_PASSWORD:-${MCP_PASSWORD:-}} + # bonnyr-f5 #193 B3: plumb ENVIRONMENT so an operator who sets it to + # staging/production actually reaches config.py's fail-fast (validate_production); + # config.py has no env_file, so without this the var never reaches the container + # and the documented "refuses to boot" promise could never fire. + ENVIRONMENT: ${ENVIRONMENT:-development} + # bonnyr-f5 #193 B2: validate_production gates on these THREE as well as + # MCP_SERVICE_PASSWORD. They must be deliverable from .env or ENVIRONMENT=production + # bricks the backend into a restart loop listing problems the shipped .env cannot + # fix. bonnyr-f5 #193 B1 (r4): OMIT-when-unset (null-value passthrough) — HEAD treats "" + # as unset and auto-generates, but a `BNK_FORGE_VERSION`-pinned 3.1.6 image uses + # `if self.KEY is None` and would boot with an empty JWT secret / invalid Fernet key. A + # map entry with NO value is passthrough: omitted when unset, forwarded when set — so + # every backend auto-generates. Set real values in .env for production. + JWT_SECRET_KEY: + ENCRYPTION_KEY: + ALLOWED_ORIGINS: ${ALLOWED_ORIGINS:-*} x-worker-volumes: &worker-volumes # Module catalog persisted in Docker volume @@ -455,11 +501,16 @@ services: logging: *default-logging environment: BNK_FORGE_API_URL: http://localhost:8000 + # bonnyr-f5 #193 B1: PASSWORD aliased from legacy MCP_PASSWORD; USERNAME is not + # (a legacy MCP_USERNAME=admin must never resolve the service username). The MCP + # client resolves the SAME value the backend does, so the two never drift. BNK_FORGE_USERNAME: ${MCP_SERVICE_USERNAME:-mcp} # MCP authenticates as the dedicated service account seeded by the backend. - # Set MCP_SERVICE_PASSWORD in .env; backend reconciles the stored hash on - # every startup so backend and MCP always stay in sync. - BNK_FORGE_PASSWORD: ${MCP_SERVICE_PASSWORD:-mcp-service-changeme} + # Set MCP_SERVICE_PASSWORD in .env (no shipped default -- #186/#187: the old + # mcp-service-changeme can no longer authenticate). The backend seeds the + # 'mcp' service account with the same value and reconciles its hash on every + # startup so backend and MCP always stay in sync. + BNK_FORGE_PASSWORD: ${MCP_SERVICE_PASSWORD:-${MCP_PASSWORD:-}} MCP_PORT: "8081" MCP_LOG_LEVEL: INFO depends_on: diff --git a/docs/DEPLOYMENT.md b/docs/DEPLOYMENT.md index 99599aa..6535358 100644 --- a/docs/DEPLOYMENT.md +++ b/docs/DEPLOYMENT.md @@ -56,9 +56,21 @@ Need the host itself provisioned too? [`vm-bnk-forge/`](../vm-bnk-forge/README.m | Field | Value | |-------|-------| | **Username** | `admin` | -| **Password** | `changeme` | +| **Password** | _generated on first startup_ | -**You must change the admin password on first login.** Navigate to Settings → Change Password. +No default password ships (#184). Where to retrieve it depends on how the account +was provisioned: + +- **Helm** — the chart generates a per-install `admin-password` Secret and wires it to + the backend, which seeds `admin` from it (or, on upgrade from a build that shipped a + default, rotates `admin` to it). That Secret value is what authenticates: + `kubectl get secret -bnk-forge-secrets -o jsonpath='{.data.admin-password}' | base64 -d` +- **Compose** — if you set `DEFAULT_ADMIN_PASSWORD`, that is the password. Otherwise the + backend generates one and writes it to a file (the plaintext is never logged — only a + pointer to the file is): + `docker exec bnk-forge-backend cat /app/keys/initial_admin_password` + +**The API refuses all other calls until you change it on first login** — Settings → Change Password. --- @@ -247,13 +259,20 @@ MCP has two distinct readiness layers: A deployment can pass layer 1 and still fail layer 2 if MCP credentials are out of sync with backend credentials. -Current compose defaults assume backend seeded admin credentials (`admin/changeme`). -If you rotate the admin password (recommended), also set MCP credentials in your -runtime environment before deploy/restart: +The backend runs MCP as its OWN dedicated **service account** (`mcp`), never the +human admin login (#186/#187) — MCP no longer authenticates with the admin +password, so rotating `admin` does not affect it. The backend seeds/reconciles +that account from `MCP_SERVICE_PASSWORD` on every boot; no default ships, so MCP +stays disabled until you set a real value. Set it in your runtime environment +before deploy/restart — the SAME value is read by the backend (which provisions +the `mcp` account from it) and by the MCP container. Do **not** point +`MCP_SERVICE_USERNAME` at `admin`: the backend refuses to reconcile a reserved +human username as a service account (it would otherwise take over the admin row), +which would leave MCP down. Keep the dedicated default name `mcp`: ```bash -MCP_USERNAME=admin -MCP_PASSWORD= +MCP_SERVICE_USERNAME=mcp # optional; defaults to "mcp" +MCP_SERVICE_PASSWORD= ``` Then recreate MCP: @@ -340,7 +359,7 @@ Before deploying to production: - [ ] Configure `MODULE_LIBRARY_GIT_URL` and `MODULE_LIBRARY_GIT_REF` - [ ] Set `HOST_REPO_PATH` if you want GUI upgrades - [ ] Set strong `POSTGRES_PASSWORD` and `REDIS_PASSWORD` in `.env` -- [ ] If backend admin password changed, set matching `MCP_USERNAME` / `MCP_PASSWORD` for MCP runtime +- [ ] Set a strong `MCP_SERVICE_PASSWORD` — MCP runs as its own dedicated `mcp` account, never `admin` (reserved; the backend refuses it and MCP stays down) — #186/#187 - [ ] After MCP credential changes, recreate MCP (`make mcp-recreate` or `make local-mcp-recreate`) - [ ] Run `make mcp-readiness` and confirm runtime tool calls pass - [ ] Ensure firewall rules allow ports 80/443 only from trusted networks diff --git a/docs/DOCKER.md b/docs/DOCKER.md index f25eaeb..bed1ee2 100644 --- a/docs/DOCKER.md +++ b/docs/DOCKER.md @@ -58,7 +58,7 @@ docker build --target worker --build-arg INSTALL_INFRACOST=true -t bnk-forge-wor ## Keyless Image Signing, SBOM, and Provenance -BNK Forge images published to the registry are signed with **keyless cosign** (Sigstore Fulcio + +BNK Forge images published from the first release cut through the signing pipeline onward are signed with **keyless cosign** (Sigstore Fulcio + Rekor transparency log). No long-lived signing key is stored — the signature is bound to the OIDC identity of whoever ran the publish script at the time of signing. @@ -83,36 +83,43 @@ The script signs each image by digest (not tag) and attaches two attestations: ### Verifying signatures (consumers) -Replace `` with the email of the person who signed the images (visible in the -Rekor transparency log entry), and `` with the image digest. +Replace `` with the image digest you're verifying. You do **not** fill in a +signer — official images are signed by the release workflow (`release.yml`), and the +commands below already pin that identity with `--certificate-identity-regexp … release.yml@…`. + +> **Note:** this verifies images published by CI. If a maintainer signed an image +> locally via the manual path above (`SIGN_EXECUTE=1`), it is bound to *that +> person's* OIDC identity, not the workflow's, so it will not match the regexp +> here — verify it with `--certificate-identity ` instead. Official +> releases always go through `release.yml`. ```bash # Verify the signature cosign verify \ - ghcr.io/jlcode-tech/bnk-forge-api@ \ - --certificate-identity \ - --certificate-oidc-issuer https://github.com/login/oauth + ghcr.io/f5devcentral/bnk-forge-api@ \ + --certificate-identity-regexp 'https://github.com/f5devcentral/bnk-forge/\.github/workflows/release\.yml@.*' \ + --certificate-oidc-issuer https://token.actions.githubusercontent.com # Verify + extract the SBOM attestation cosign verify-attestation \ --type cyclonedx \ - --certificate-identity \ - --certificate-oidc-issuer https://github.com/login/oauth \ - ghcr.io/jlcode-tech/bnk-forge-api@ \ + --certificate-identity-regexp 'https://github.com/f5devcentral/bnk-forge/\.github/workflows/release\.yml@.*' \ + --certificate-oidc-issuer https://token.actions.githubusercontent.com \ + ghcr.io/f5devcentral/bnk-forge-api@ \ | jq -r '.payload' | base64 -d | jq . # Verify + extract the SLSA provenance attestation cosign verify-attestation \ --type slsaprovenance \ - --certificate-identity \ - --certificate-oidc-issuer https://github.com/login/oauth \ - ghcr.io/jlcode-tech/bnk-forge-api@ \ + --certificate-identity-regexp 'https://github.com/f5devcentral/bnk-forge/\.github/workflows/release\.yml@.*' \ + --certificate-oidc-issuer https://token.actions.githubusercontent.com \ + ghcr.io/f5devcentral/bnk-forge-api@ \ | jq -r '.payload' | base64 -d | jq . ``` Apply the same commands to the other image names: `bnk-forge-worker`, `bnk-forge-beat`, `bnk-forge-frontend`, `bnk-forge-proxy`, -`bnk-forge-mcp`. +`bnk-forge-mcp`, `bnk-forge-operator`. ### OCI Labels @@ -123,7 +130,7 @@ All images carry standard OCI labels injected at build time via `docker-bake.hcl | `org.opencontainers.image.source` | `https://github.com/f5devcentral/bnk-forge` | | `org.opencontainers.image.revision` | git commit SHA (`GIT_REVISION` bake arg) | | `org.opencontainers.image.version` | `VERSION` file contents | -| `org.opencontainers.image.created` | RFC 3339 timestamp of the build | +| `org.opencontainers.image.created` | RFC 3339 build timestamp (`CREATED` bake arg). Emitted only when set — CI release builds set it to the release commit's committer date; a plain `make push-images` leaves it unset and the label is omitted rather than written empty. | Inject `GIT_REVISION` when calling bake: diff --git a/docs/E2E-CRITICAL-004_MCP_SANITY.md b/docs/E2E-CRITICAL-004_MCP_SANITY.md index 42c00a6..1e35020 100644 --- a/docs/E2E-CRITICAL-004_MCP_SANITY.md +++ b/docs/E2E-CRITICAL-004_MCP_SANITY.md @@ -87,8 +87,8 @@ mcp-server/tests/ |----------|----------|---------|---------| | `MCP_E2E` | Yes | `false` | Gate for E2E tests (skip in unit runs) | | `API_BASE_URL` | Yes | `http://localhost:8000` | Backend API target | -| `MCP_USERNAME` | Yes | `admin` | Auth credentials | -| `MCP_PASSWORD` | Yes | `changeme` | Auth credentials | +| `MCP_USERNAME` | No | `mcp` | Not read by the backend or MCP server; the service username is fixed to the dedicated `mcp` account (never `admin`, #187). Do not set it. | +| `MCP_SERVICE_PASSWORD` | Yes | _(no default, #187)_ | Shared secret: backend seeds the mcp account, MCP server authenticates with it | --- diff --git a/docs/How to write CI container runner modules and blueprints for BNK Forge.md b/docs/How to write CI container runner modules and blueprints for BNK Forge.md index 4357830..48a4e79 100644 --- a/docs/How to write CI container runner modules and blueprints for BNK Forge.md +++ b/docs/How to write CI container runner modules and blueprints for BNK Forge.md @@ -654,9 +654,14 @@ The container engine is deliberately constrained: which is the default for most base images*). The workspace is mounted from the host, so a root container would be a host-root write primitive. Forge does **not** silently remap you to another uid with `--user`: that would override your image's `USER` and break your - own state writes. So: put `USER ` in your Dockerfile. uid **1000** matches the - workspace owner and is the safe choice. This mirrors Kubernetes `runAsNonRoot`, which the - Kubernetes runner applies to the same artifacts. + own state writes. So: put a **numeric** `USER` in your Dockerfile. The gate requires a bare + decimal uid — uid **1000** matches the workspace owner and is the safe choice. A **named** + user such as `USER nonroot` (the distroless default) is now **refused**: a name can't be + resolved to a uid without the image's own `/etc/passwd`, so it can't be proven non-root. + If you were on `USER nonroot`, switch to `USER 1000` — it matches the workspace owner (chowned + `1000:1000`), so your state writes under `mount_path` succeed. A higher uid such as `65532` clears + the non-root gate but cannot write the host-mounted workspace. + This mirrors Kubernetes `runAsNonRoot`, which the Kubernetes runner applies to the same artifacts. - **A dedicated network** — steps attach to the `bnk-forge-artifacts` bridge network rather than the daemon's default bridge, so artifact containers don't sit alongside unrelated containers. Egress still works (you can reach cloud control planes); you just don't share diff --git a/docs/INSTALLATION.md b/docs/INSTALLATION.md index c3e1e2f..4445ccd 100644 --- a/docs/INSTALLATION.md +++ b/docs/INSTALLATION.md @@ -27,7 +27,7 @@ cd bnk-forge make local-deploy ``` -Open **https://localhost** and accept the self-signed certificate warning. Log in with **admin** / **changeme**. +Open **https://localhost** and accept the self-signed certificate warning. Log in as **admin** — no default password ships; retrieve the generated one from `/app/keys/initial_admin_password` (the boot log points at this file; the plaintext is never logged), or set `DEFAULT_ADMIN_PASSWORD`. You'll change it on first login. ### Linux Server @@ -39,7 +39,7 @@ make deploy For first-time clean-slate bootstrap only (destructive), run `make install`. -Log in with **admin** / **changeme** (you'll be prompted to change the password). +Log in as **admin** using the generated password from `/app/keys/initial_admin_password` (or set `DEFAULT_ADMIN_PASSWORD`); you'll be prompted to change it on first login. --- @@ -124,9 +124,33 @@ A default admin account is created automatically on first startup: | Field | Value | |-------|-------| | **Username** | `admin` | -| **Password** | `changeme` | +| **Password** | _generated on first startup_ | -You will be prompted to change the password on first login. +No default password ships. Where the password comes from — and where to retrieve +it — depends on whether `DEFAULT_ADMIN_PASSWORD` is set: + +- **`DEFAULT_ADMIN_PASSWORD` set** (Helm always sets it, wiring it from the + chart's per-install `admin-password` Secret): the backend seeds `admin` from + that value — and, on upgrade from a build that shipped a default password, + rotates `admin` to it. No keys-file is written. Retrieve it from that source: + + ```bash + # Helm (the admin-password Secret is the source of truth) + kubectl get secret -bnk-forge-secrets -o jsonpath='{.data.admin-password}' | base64 -d + ``` + +- **`DEFAULT_ADMIN_PASSWORD` unset** (Docker Compose default): the backend + generates a random password and writes it to `/app/keys/initial_admin_password` + (mode 600); the boot log points at that file (the plaintext itself is never + logged). Retrieve it from the file: + + ```bash + # Docker Compose + docker exec bnk-forge-backend cat /app/keys/initial_admin_password + ``` + +You will be **required** to change it on first login (the API refuses other calls +until you do). ### Managing Your Local Deployment @@ -225,7 +249,7 @@ sudo firewall-cmd --reload Access from any browser: `https://your-server-ip` -Log in with **admin** / **changeme** (you'll be prompted to change the password). +Log in as **admin** using the generated password from `/app/keys/initial_admin_password` (or set `DEFAULT_ADMIN_PASSWORD`); you'll be prompted to change it on first login. Accept the self-signed certificate warning, or replace the certs with your own (see proxy/Dockerfile). @@ -264,7 +288,7 @@ server topology). The VM path applies the same hardening this guide describes: key-only with root login disabled, and the GitHub deploy key is shredded once the clone completes. -The default credentials (`admin` / `changeme`) and the Docker-socket mount +The generated admin credentials (see the setup notes above) and the Docker-socket mount still apply — read the README's security notes before giving such a VM a public address. @@ -348,11 +372,11 @@ After starting BNK Forge for the first time: ### 1. Log In -Open the application URL and log in with the default credentials: -- **Username:** `admin` -- **Password:** `changeme` - -You will be prompted to set a new password on first login. +Open the application URL and log in as **`admin`**. There is no default +password: retrieve the one generated on first startup — +`docker exec bnk-forge-backend cat /app/keys/initial_admin_password` (compose), +or `kubectl get secret -bnk-forge-secrets -o jsonpath='{.data.admin-password}' | base64 -d` (Helm). +You will be **required** to set a new password before the API accepts other calls. ### 2. Connect a Kubernetes Cluster diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 12a346f..555a229 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -7,7 +7,7 @@ Source sweep: memories + ADRs (D-001…D-028) + GitHub issues + open PRs, 2026-06-12. **Deep doc sweep 2026-06-03:** swept `docs/specs/`, sprint plans, and strategic docs; statuses **code-verified** before assignment (many specs that read as "proposed" are in fact already built — see §10). New gap issues from the sweep: #216/#217/#218. **2026-06-09 sync:** D-021/D-022 fleet epics shipped (PRs #276/#277); D-027 zero-toast shipped (PRs #260/#261/#278); D-028 unified blueprint catalog shipped (PR #274); D-001 Phase 3 / D-019 E1/E3/E6 / Ops MCP+celery / AWS cred-expiry UX all shipped. D-023 (classic BIG-IP) + D-020 (F5 design-system) + benchmark experience remain in-flight. -**2026-06-12 sync:** cb-rebuild → staging de-stacking COMPLETE; new `customer-build` integration line live on localhost (staging + all open PRs + customer deltas); multi-arch images published `ghcr.io/jlcode-tech 3.1.6-cb.72b29dbb` + rolling `customer-build`. D-023 P1-P3 shipped (PR #280); alembic dedupe shipped (PR #283). 13 PRs in review: #282 (scenario override guard), #284 (BNK registry-driven GA/ReleaseRegistry), #285 (multiarch publish), #286 (benchmark remote agent-host + Slice-4 auth), #287 (AI-Analyzer rename + Migration tab fix), #288 (dist uninstall-purge fix), #290 (D-023 P4 CIS coverage), #291 (Dashboard Command Center fleet section), #292 (bfb-cache atime/LRU fix), #293 (runtime brand flag #289), #295 (CIS IngressClass kind-aware classify), #296 (top-level Infrastructure section), #297 (release automation — team decision pending). D-020 reskin branch fully rebuilt 2026-06-10/11 (complete reskin + anvil ForgeLogo general rebrand). +**2026-06-12 sync:** cb-rebuild → staging de-stacking COMPLETE; new `customer-build` integration line live on localhost (staging + all open PRs + customer deltas); multi-arch images published `ghcr.io/f5devcentral 3.1.6-cb.72b29dbb` + rolling `customer-build`. D-023 P1-P3 shipped (PR #280); alembic dedupe shipped (PR #283). 13 PRs in review: #282 (scenario override guard), #284 (BNK registry-driven GA/ReleaseRegistry), #285 (multiarch publish), #286 (benchmark remote agent-host + Slice-4 auth), #287 (AI-Analyzer rename + Migration tab fix), #288 (dist uninstall-purge fix), #290 (D-023 P4 CIS coverage), #291 (Dashboard Command Center fleet section), #292 (bfb-cache atime/LRU fix), #293 (runtime brand flag #289), #295 (CIS IngressClass kind-aware classify), #296 (top-level Infrastructure section), #297 (release automation — team decision pending). D-020 reskin branch fully rebuilt 2026-06-10/11 (complete reskin + anvil ForgeLogo general rebrand). **Human view (clickable):** `docs/roadmap.html` · **Contribution flow:** `docs/ROADMAP_PROCESS.md` · (local per-clone agent queue: `.agent/backlog/BACKLOG.md`). --- @@ -28,7 +28,7 @@ Source sweep: memories + ADRs (D-001…D-028) + GitHub issues + open PRs, 2026-0 | **Ops: MCP service account + celery-beat healthcheck** | ✅ Shipped | [PR #214](https://github.com/f5devcentral/bnk-forge/issues/214) | Shipped. PR #214 merged. Dedicated non-human `mcp` account (admin-rotation no longer breaks MCP auth) + mtime-freshness beat healthcheck. | | **Benchmark experience / security hardening** | 🟡 In progress | [PR #211](https://github.com/f5devcentral/bnk-forge/issues/211) · [PR #251](https://github.com/f5devcentral/bnk-forge/issues/251) · [PR #282](https://github.com/f5devcentral/bnk-forge/issues/282) · [PR #286](https://github.com/f5devcentral/bnk-forge/issues/286) · [#294](https://github.com/f5devcentral/bnk-forge/issues/294) | PR #282 (scenario override guard security fix) + PR #286 (remote agent-host provisioning + built-in agent + Slice-4 auth + ported tests) in review. PR #251 (authz/WS-auth/atomic-claim/TLS+SSRF) folds into #211; gated on a maintainer driving #211→staging. #294 (benchmarks page IA + New Run wizard port) deferred pending user decision. | | **GHCR customer-build publish target + postgres-backup compose drift fix** | ✅ Shipped | [PR #242](https://github.com/f5devcentral/bnk-forge/issues/242) · [PR #243](https://github.com/f5devcentral/bnk-forge/issues/243) | Shipped. PR #242 (postgres-backup compose drift fix) + PR #243 (GHCR customer-build publish target) merged. | -| **Multi-arch image publish + dist uninstall-purge fix + install messaging** | 🟡 In progress | [PR #285](https://github.com/f5devcentral/bnk-forge/issues/285) · [PR #288](https://github.com/f5devcentral/bnk-forge/issues/288) | PR #285 (push-customer-build-multiarch target: amd64+arm64 to ghcr.io/jlcode-tech) + PR #288 (uninstall --purge deletes wrong compose volumes fix + stale install messaging) in review. | +| **Multi-arch image publish + dist uninstall-purge fix + install messaging** | 🟡 In progress | [PR #285](https://github.com/f5devcentral/bnk-forge/issues/285) · [PR #288](https://github.com/f5devcentral/bnk-forge/issues/288) | PR #285 (push-customer-build-multiarch target: amd64+arm64 to ghcr.io/f5devcentral) + PR #288 (uninstall --purge deletes wrong compose volumes fix + stale install messaging) in review. | | **Alembic migration deduplication (v2_131)** | ✅ Shipped | [PR #283](https://github.com/f5devcentral/bnk-forge/issues/283) | Shipped. PR #283 merged. Deduped v2_130 collision (benchmark_run_groups renumbered → v2_131); broken staging migration head fixed. | | **Release automation — RC-on-staging / final-on-main (conventional-commit bumps)** | 🟡 In progress | [PR #297](https://github.com/f5devcentral/bnk-forge/issues/297) | PR #297 open; team decision pending — may be closed in favor of manual release flow. | | **D-021 — existing-proxy discovery & migration to BNK (P1+P2+P3)** | ✅ Shipped | [#233](https://github.com/f5devcentral/bnk-forge/issues/233) · [PR #276](https://github.com/f5devcentral/bnk-forge/issues/276) · ADR D-021 | Shipped. PR #276 consolidated (closes epic #233). Proxy discovery, migration path to BNK, P1/P2/P3 complete. | @@ -48,7 +48,7 @@ Source sweep: memories + ADRs (D-001…D-028) + GitHub issues + open PRs, 2026-0 | **awsbnkctl review follow-up** | 💤 Deferred | [#155](https://github.com/f5devcentral/bnk-forge/issues/155) | SimpleNamespace shim hardening + configured-shape contract test. (awsbnkctl side: SSO refresh-token bootstrap — sibling repo.) | | **Review follow-ups (deferred polish)** | 💤 Deferred | [#153](https://github.com/f5devcentral/bnk-forge/issues/153) · [#154](https://github.com/f5devcentral/bnk-forge/issues/154) · [#156](https://github.com/f5devcentral/bnk-forge/issues/156) · [#157](https://github.com/f5devcentral/bnk-forge/issues/157) | #153 (#144 TTFT/ports) · #154 (#146 _kind_to_snake) · #156 (#151 MCP envelope sweep) · #157 (#148 BNK substring/prefix). Non-blocking; created 2026-05-27. | | **Module catalog auto-sync on boot + wire blueprint wizard 'Sync' CTA** | ⚪ Planned | [#419](https://github.com/f5devcentral/bnk-forge/issues/419) | Fresh install / volume wipe leaves the git module catalog (bnk/app/infra packs) un-synced, so BNK/app blueprints are DOA until an operator runs Catalog→Advanced→Modules→'Sync all'. The wizard's 'requires sync' prompt is wired to no endpoint. Fix: (1) boot-time auto-sync step (non-fatal, ref-aware) after builtin seeders; (2) wire wizard CTA to POST /api/module-library/sync. Separate from the d019/adr-204 execution_engine seeder-guard fix. | -| **CI container runner engine — security hardening follow-ups** | ⚪ Planned | [#408](https://github.com/f5devcentral/bnk-forge/issues/408) · [PR #340](https://github.com/f5devcentral/bnk-forge/issues/340) | Non-blocking follow-ups from the #340 review (all prior blockers verified fixed pre-merge). Priority: non-root Docker gate bypass via `USER 0:` (host-root primitive), `state.outputs_file` path traversal (arbitrary worker-file read), registry `/test` cross-operator credential exfil + SSRF, install-script PAT/password xtrace leak, K8s deny-all egress netpol breaks the artifact. Plus same-project cluster-name clobber + nits. | +| **CI container runner engine — security hardening follow-ups** | 🟢 Merged (unreleased) | [#408](https://github.com/f5devcentral/bnk-forge/issues/408) · [PR #340](https://github.com/f5devcentral/bnk-forge/issues/340) | Non-blocking follow-ups from the #340 review (all prior blockers verified fixed pre-merge). Priority: non-root Docker gate bypass via `USER 0:` (host-root primitive), `state.outputs_file` path traversal (arbitrary worker-file read), registry `/test` cross-operator credential exfil + SSRF, install-script PAT/password xtrace leak, K8s deny-all egress netpol breaks the artifact. Plus same-project cluster-name clobber + nits. | | **Multi-version module catalog — immutable module versions, exact pin resolution (ADR D-033)** | 🟡 In progress | [#433](https://github.com/f5devcentral/bnk-forge/issues/433) · [PR #436](https://github.com/f5devcentral/bnk-forge/issues/436) | Module identity becomes (source, path, version); hashed rows immutable; blueprints resolve pins exactly (BLUEPRINT_MODULE_VERSION_MISSING); ProjectModule FK becomes a true pin with explicit change-version action + UI/MCP. Combined PR #436 (supersedes stacked #434/#435). ADR: docs/adr/D-033-multi-version-module-catalog.md (PR #432). | | **Module test actions — vendor-CLI e2e/scenario/bench tests via pipeline (ADR D-034)** | ⚪ Planned | [#454](https://github.com/f5devcentral/bnk-forge/issues/454) · [PR #453](https://github.com/f5devcentral/bnk-forge/issues/453) | Container-artifact manifests gain a declarative actions block; container engine gains one generic action dispatcher; UI offers actions on post-apply modules (per-scenario + run-all-green, amber behind warning). v1 results = logs + pass/fail. Scaling = edit vars + re-apply, not an action. Tool-embedded tests → pipeline; external load (aiperf agents) stays in Benchmarks. Slices: PR-1 backend, PR-2 UI, PR-3 packs/docs. ADR: docs/adr/D-034-module-test-actions.md (PR #453). Sibling: #452 cluster auto-registration. | | **Container-runner contract hardening — min_forge_version + declared capabilities** | ⚪ Planned | [#465](https://github.com/f5devcentral/bnk-forge/issues/465) | Phase 3 of the ctl-runner review. min_forge_version + capability requirements (e.g. wide docker-socket proxy) become machine-checkable artifact-manifest fields enforced at sync/import; Forge injects the proxy endpoint instead of external manifests hardcoding DOCKER_HOST; schema_version evolution policy lands in EXT-003. | diff --git a/docs/ROADMAP_PROCESS.md b/docs/ROADMAP_PROCESS.md index 8bdbe7a..7451532 100644 --- a/docs/ROADMAP_PROCESS.md +++ b/docs/ROADMAP_PROCESS.md @@ -34,7 +34,7 @@ backend/.venv/bin/python bin/roadmap-add.py \ ``` - `--list-sections` prints the available section ids + headings. -- `--status` must be one of: `shipped`, `in_progress`, `blocked`, `deferred`, `planned`. +- `--status` must be one of: `shipped`, `merged`, `in_progress`, `blocked`, `deferred`, `planned` (the keys in `status_legend`; `merged` = merged to staging but unreleased). - `--refs` is comma-separated; values like `#216` / `PR #188` become GitHub links. - `--group` (optional) buckets the item into a named card on the HTML view. - After adding, also update the **§12 issue index** (`render: raw`, edited by hand in the yaml) and re-run the generator so the index stays in sync. @@ -49,7 +49,7 @@ backend/.venv/bin/python bin/roadmap-add.py \ ## Status vocabulary -✅ shipped · 🟡 in-progress / partial · ⛔ blocked (state the blocker) · 💤 deferred (state the resume-trigger) · ⚪ not-started / proposed. +✅ shipped · 🟢 merged (merged to staging, unreleased) · 🟡 in-progress / partial · ⛔ blocked (state the blocker) · 💤 deferred (state the resume-trigger) · ⚪ not-started / proposed. ## Where things live diff --git a/docs/roadmap.html b/docs/roadmap.html index eb4ba5c..126ff53 100644 --- a/docs/roadmap.html +++ b/docs/roadmap.html @@ -68,7 +68,7 @@
23
In progress (tracked)
-
35
Planned (tracked)
+
34
Planned (tracked)
50+
PRs merged to staging
34
Issues closed
@@ -129,7 +129,7 @@

Active / next — issue-tracked, actionable now

Review follow-ups

    -
  • CI container runner engine — security hardening follow-ups#408 · PR #340
    • +
  • CI container runner engine — security hardening follow-ups#408 · PR #340
diff --git a/docs/roadmap.yaml b/docs/roadmap.yaml index fee9719..cab7ad4 100644 --- a/docs/roadmap.yaml +++ b/docs/roadmap.yaml @@ -17,7 +17,7 @@ # "AUTO:planned" are computed from item counts; any other # value (e.g. "25+") is a static meta number. # status_legend: key -> { emoji, dot, label } used for BOTH md + html. -# keys: shipped | in_progress | blocked | deferred | planned +# keys: shipped | merged | in_progress | blocked | deferred | planned # sections: ordered list of # - id: stable slug (used by roadmap-add.py --section) # number: section number for the md heading "## N. ..." @@ -55,7 +55,7 @@ meta: **2026-06-09 sync:** D-021/D-022 fleet epics shipped (PRs #276/#277); D-027 zero-toast shipped (PRs #260/#261/#278); D-028 unified blueprint catalog shipped (PR #274); D-001 Phase 3 / D-019 E1/E3/E6 / Ops MCP+celery / AWS cred-expiry UX all shipped. D-023 (classic BIG-IP) + D-020 (F5 design-system) + benchmark experience remain in-flight. - **2026-06-12 sync:** cb-rebuild → staging de-stacking COMPLETE; new `customer-build` integration line live on localhost (staging + all open PRs + customer deltas); multi-arch images published `ghcr.io/jlcode-tech 3.1.6-cb.72b29dbb` + rolling `customer-build`. D-023 P1-P3 shipped (PR #280); alembic dedupe shipped (PR #283). 13 PRs in review: #282 (scenario override guard), #284 (BNK registry-driven GA/ReleaseRegistry), #285 (multiarch publish), #286 (benchmark remote agent-host + Slice-4 auth), #287 (AI-Analyzer rename + Migration tab fix), #288 (dist uninstall-purge fix), #290 (D-023 P4 CIS coverage), #291 (Dashboard Command Center fleet section), #292 (bfb-cache atime/LRU fix), #293 (runtime brand flag #289), #295 (CIS IngressClass kind-aware classify), #296 (top-level Infrastructure section), #297 (release automation — team decision pending). D-020 reskin branch fully rebuilt 2026-06-10/11 (complete reskin + anvil ForgeLogo general rebrand). + **2026-06-12 sync:** cb-rebuild → staging de-stacking COMPLETE; new `customer-build` integration line live on localhost (staging + all open PRs + customer deltas); multi-arch images published `ghcr.io/f5devcentral 3.1.6-cb.72b29dbb` + rolling `customer-build`. D-023 P1-P3 shipped (PR #280); alembic dedupe shipped (PR #283). 13 PRs in review: #282 (scenario override guard), #284 (BNK registry-driven GA/ReleaseRegistry), #285 (multiarch publish), #286 (benchmark remote agent-host + Slice-4 auth), #287 (AI-Analyzer rename + Migration tab fix), #288 (dist uninstall-purge fix), #290 (D-023 P4 CIS coverage), #291 (Dashboard Command Center fleet section), #292 (bfb-cache atime/LRU fix), #293 (runtime brand flag #289), #295 (CIS IngressClass kind-aware classify), #296 (top-level Infrastructure section), #297 (release automation — team decision pending). D-020 reskin branch fully rebuilt 2026-06-10/11 (complete reskin + anvil ForgeLogo general rebrand). **Human view (clickable):** `docs/roadmap.html` · **Contribution flow:** `docs/ROADMAP_PROCESS.md` · (local per-clone agent queue: `.agent/backlog/BACKLOG.md`). @@ -77,6 +77,10 @@ status_legend: emoji: ✅ dot: d-ship label: Shipped + merged: + emoji: 🟢 + dot: d-ship + label: Merged (unreleased) in_progress: emoji: 🟡 dot: d-prog @@ -212,7 +216,7 @@ sections: refs: - 'PR #285' - 'PR #288' - note: 'PR #285 (push-customer-build-multiarch target: amd64+arm64 to ghcr.io/jlcode-tech) + PR #288 (uninstall --purge deletes wrong compose volumes fix + stale install messaging) in review.' + note: 'PR #285 (push-customer-build-multiarch target: amd64+arm64 to ghcr.io/f5devcentral) + PR #288 (uninstall --purge deletes wrong compose volumes fix + stale install messaging) in review.' group: Top of queue — PRs open (CI-green) - title: Alembic migration deduplication (v2_131) status: shipped @@ -353,7 +357,7 @@ sections: - '#419' note: 'Fresh install / volume wipe leaves the git module catalog (bnk/app/infra packs) un-synced, so BNK/app blueprints are DOA until an operator runs Catalog→Advanced→Modules→''Sync all''. The wizard''s ''requires sync'' prompt is wired to no endpoint. Fix: (1) boot-time auto-sync step (non-fatal, ref-aware) after builtin seeders; (2) wire wizard CTA to POST /api/module-library/sync. Separate from the d019/adr-204 execution_engine seeder-guard fix.' - title: CI container runner engine — security hardening follow-ups - status: planned + status: merged refs: - '#408' - 'PR #340' diff --git a/frontend-v2/package.json b/frontend-v2/package.json index 4e25c0c..7282abe 100644 --- a/frontend-v2/package.json +++ b/frontend-v2/package.json @@ -1,7 +1,7 @@ { "name": "frontend-v2", "private": true, - "version": "2.12.0", + "version": "3.1.6", "type": "module", "sideEffects": [ "*.css" diff --git a/frontend-v2/src/types/api-generated.ts b/frontend-v2/src/types/api-generated.ts index 1e16df9..44a03e8 100644 --- a/frontend-v2/src/types/api-generated.ts +++ b/frontend-v2/src/types/api-generated.ts @@ -22745,6 +22745,11 @@ export interface components { role: string; /** Is Active */ is_active: boolean; + /** + * Is Service Account + * @default false + */ + is_service_account: boolean; /** Must Change Password */ must_change_password: boolean; /** Last Login At */ @@ -22808,6 +22813,11 @@ export interface components { role: string; /** Is Active */ is_active: boolean; + /** + * Is Service Account + * @default false + */ + is_service_account: boolean; /** Must Change Password */ must_change_password: boolean; /** Last Login At */ @@ -22860,6 +22870,11 @@ export interface components { role: string; /** Is Active */ is_active: boolean; + /** + * Is Service Account + * @default false + */ + is_service_account: boolean; /** Must Change Password */ must_change_password: boolean; /** Last Login At */ diff --git a/helm/bnk-forge/Chart.yaml b/helm/bnk-forge/Chart.yaml index 950b44a..40db248 100644 --- a/helm/bnk-forge/Chart.yaml +++ b/helm/bnk-forge/Chart.yaml @@ -3,7 +3,7 @@ name: bnk-forge description: BNK-Forge — F5 BNK lifecycle / deployment platform (api, workers, beat, frontend, proxy, mcp) type: application version: 0.1.0 -appVersion: "3.0.1" +appVersion: "3.1.6" home: https://github.com/f5devcentral/bnk-forge maintainers: - name: BNK Forge Maintainers diff --git a/helm/bnk-forge/templates/NOTES.txt b/helm/bnk-forge/templates/NOTES.txt index a6f4d8b..f3f9650 100644 --- a/helm/bnk-forge/templates/NOTES.txt +++ b/helm/bnk-forge/templates/NOTES.txt @@ -7,6 +7,31 @@ Components: {{- if .Values.frontend.enabled }} frontend svc: {{ include "bnk-forge.fullname" . }}-frontend:{{ .Values.frontend.service.port }}{{- end }} {{- if .Values.proxy.enabled }} proxy svc: {{ include "bnk-forge.fullname" . }}-proxy ({{ .Values.proxy.service.type }}) {{ .Values.proxy.service.httpPort }}/{{ .Values.proxy.service.httpsPort }}{{- end }} {{- if .Values.mcp.enabled }} mcp svc: {{ include "bnk-forge.fullname" . }}-mcp:{{ .Values.mcp.service.port }}{{- end }} +{{- if .Values.mcp.enabled }} + +MCP: the mcp server authenticates as the dedicated 'mcp' service account +(secrets.mcpUsername). Do NOT point secrets.mcpUsername at a human identity such as +'admin' — the backend refuses to reconcile a reserved human username as a service +account, so the mcp account would be disabled on upgrade and MCP would go +permanently down while this install reports success (the chart now fails the render +if you set it to 'admin'). Set secrets.mcpPassword to a strong value (the same +value the MCP server receives as BNK_FORGE_PASSWORD), or leave it empty to +auto-generate one. Note: the mcp readiness probe only takes the pod out of service +when no/ bad credentials are detected on the credential-guard release; the guarantee +lands with the image the chart pins (image.tag == this chart's appVersion), so on an +older pinned image confirm MCP with a real tool call rather than trusting readiness. +{{- end }} + +SET ALLOWED_ORIGINS FOR BROWSER ACCESS: + The api/worker/beat run with ENVIRONMENT=production. Under production the backend + REFUSES to boot (validate_production SystemExit) if ALLOWED_ORIGINS contains '*' or + 'localhost' — and this chart now ALSO fails those AT RENDER (bonnyr-f5 #193 M10), so a + bad value is caught by `helm install` instead of as a CrashLoopBackOff. The default + ships EMPTY, which boots cleanly, but with no cross-origin allowed the browser UI + cannot call the API until you name your real origin(s). Install/upgrade with: + --set api.env.ALLOWED_ORIGINS="https://forge.example.com" + (comma-separate multiple; match the host/port you actually reach the proxy on; no '*', + no localhost in production — set ENVIRONMENT=development for a local trial instead). To reach the UI: {{- if .Values.ingress.enabled }} @@ -18,7 +43,15 @@ To reach the UI: open https://localhost:8443/ {{- end }} -Default admin login: admin / changeme (change immediately via UI). +Admin login (from a browser origin listed in ALLOWED_ORIGINS — see above): +username 'admin'. Retrieve the chart-provisioned password with: + kubectl get secret {{ include "bnk-forge.fullname" . }}-secrets -o jsonpath='{.data.admin-password}' | base64 -d +This is the per-install value carried in the Secret. How the backend USES it depends +on the image this chart pins (image.tag {{ .Values.image.tag }}): a build that seeds +the admin account from this Secret and rotates a shipped-default admin will authenticate +you with the value above and require a password change on first login; an older image +that predates that behaviour may still accept its own default credential. Treat the +value above as the intended admin password and change it on first login. IMPORTANT: * Shared volumes use ReadWriteMany. Set global.sharedStorageClass to a diff --git a/helm/bnk-forge/templates/_helpers.tpl b/helm/bnk-forge/templates/_helpers.tpl index 8b6d01d..e0a28b6 100644 --- a/helm/bnk-forge/templates/_helpers.tpl +++ b/helm/bnk-forge/templates/_helpers.tpl @@ -72,6 +72,33 @@ backendEnv: env block shared by api/worker/beat. Wires DB + Redis URLs to in-cluster services and pulls secrets from the generated Secret. */}} {{- define "bnk-forge.backendEnv" -}} +{{/* bonnyr-f5 #193 M10: render-time CORS/production guard, mirroring the backend's + core/config.py validate_production() SystemExit conditions exactly, so a fatal + posture is caught at `helm install` time instead of as a crashloop: + * a bare "*" ENTRY in ALLOWED_ORIGINS -> fatal under staging AND production + (exact comma-split entry, matching the backend's `"*" in self.cors_origins` + since r4 — so a legit `https://*.example.com` subdomain origin is NOT flagged); + * "localhost" substring in ALLOWED_ORIGINS -> fatal under production only + (matches the backend's per-origin `"localhost" in origin`). + An EMPTY ALLOWED_ORIGINS is deliberately NOT failed: the backend accepts it + (no wildcard, no localhost) and boots, so the default render (ENVIRONMENT + production + the empty ALLOWED_ORIGINS shipped in values.yaml) stays green under + `helm lint` and a bare `helm template` — this guard fires only once an operator + puts a genuinely fatal value in a real production/staging posture. Mirrors the + deterministic fail-at-render pattern secrets.yaml uses for mcpPassword/mcpUsername. */}} +{{- $benv := .Values.api.env | default dict -}} +{{- $environment := $benv.ENVIRONMENT | default "" -}} +{{- if or (eq $environment "production") (eq $environment "staging") -}} +{{- $origins := $benv.ALLOWED_ORIGINS | default "" -}} +{{- $hasWildcard := false -}} +{{- range (splitList "," $origins) -}}{{- if eq (trim .) "*" -}}{{- $hasWildcard = true -}}{{- end -}}{{- end -}} +{{- if $hasWildcard -}} +{{- fail (printf "api.env.ALLOWED_ORIGINS has a bare '*' (wildcard) entry under ENVIRONMENT=%s; the backend rejects this at boot (validate_production) and crashloops. Set api.env.ALLOWED_ORIGINS to your explicit origin(s), e.g. https://forge.example.com." $environment) -}} +{{- end -}} +{{- if and (eq $environment "production") (contains "localhost" $origins) -}} +{{- fail "api.env.ALLOWED_ORIGINS contains 'localhost' under ENVIRONMENT=production; the backend rejects this at boot (validate_production) and crashloops. Set api.env.ALLOWED_ORIGINS to your actual domain/IP, e.g. https://forge.example.com (or set ENVIRONMENT=development for a local trial)." -}} +{{- end -}} +{{- end -}} - name: POSTGRES_HOST value: {{ include "bnk-forge.fullname" . }}-postgres - name: REDIS_HOST @@ -96,6 +123,43 @@ in-cluster services and pulls secrets from the generated Secret. secretKeyRef: name: {{ include "bnk-forge.fullname" . }}-secrets key: encryption-key +# #184: seed the admin account from a generated secret, never a shipped +# default. Retrieve with: +# kubectl get secret -bnk-forge-secrets -o jsonpath='{.data.admin-password}' | base64 -d +- name: DEFAULT_ADMIN_PASSWORD + valueFrom: + secretKeyRef: + name: {{ include "bnk-forge.fullname" . }}-secrets + key: admin-password +# #186: plumb the must-change gate alongside its sibling, or the seeded admin +# owes a password change no route accepts (login is exempt; every other /api +# route 403s). Not a secret -- a plain value. Quote so the bool renders "true"/ +# "false" (do NOT `default` it: a bool false collapses back to the default). +- name: DEFAULT_ADMIN_MUST_CHANGE + # #186 (bonnyr-f5 r4): fall back to the secure "true" only when the value is + # nil/unset -- `| default true` cannot be used here because sprig `default` + # treats a bool false as empty and would silently flip an intentional false + # back to true. kindIs "invalid" is true only for nil, so an explicit false + # still renders "false"; nil no longer renders a bare `value:` that makes + # pydantic reject an empty string and the backend crashloop. + value: {{ if kindIs "invalid" .Values.secrets.adminMustChange }}{{ "true" | quote }}{{ else }}{{ .Values.secrets.adminMustChange | quote }}{{ end }} +# #186 BLOCKER 1 / #187 (bonnyr-f5 r5): the backend reconciles the mcp service +# account to MCP_SERVICE_PASSWORD on every boot, so it must read the SAME +# per-install secret the mcp client (mcp.yaml) reads -- otherwise removing the +# shipped `changeme` default just leaves the mcp account unseeded and the client +# can never authenticate ("removes the default without plumbing the replacement"; +# nothing is generated for MCP under the #188-over-#186 consolidation, bonnyr-f5 +# #193). Source both from the release Secret's mcp-* keys, identical to mcp.yaml. +- name: MCP_SERVICE_USERNAME + valueFrom: + secretKeyRef: + name: {{ include "bnk-forge.fullname" . }}-secrets + key: mcp-username +- name: MCP_SERVICE_PASSWORD + valueFrom: + secretKeyRef: + name: {{ include "bnk-forge.fullname" . }}-secrets + key: mcp-password - name: DATABASE_URL value: "postgresql://bnkforge:$(POSTGRES_PASSWORD)@$(POSTGRES_HOST):5432/bnkforge" - name: REDIS_URL @@ -143,3 +207,35 @@ sharedVolumes: PVC-backed volume references for pod spec. secretName: {{ .Values.externalSecretsRef.name }} {{- end }} {{- end -}} + +{{/* +secretsChecksum: digest for the pod `checksum/secret` annotations, so a real secret change +rolls api/worker/beat/mcp and an unchanged render does not. bonnyr-f5 #193 M7 + round-3 minor. +It hashes the DETERMINISTIC inputs that determine the Secret -- the values.yaml `secrets.*` +block and the persisted Secret's `.data` (reused verbatim across upgrades via `lookup`, +nil-folded as secrets.yaml does) -- NOT the rendered Secret. + +The M7 trilemma (r4 self-review): you cannot have all three of (i) stable across a bare +no-cluster `helm template`, (ii) tracks a GENERATED-value rotation at render time, and +(iii) unpredictable secrets. Determinism buys (i)+(ii) but forfeits (iii) -- a derived key is +computable from public chart/label metadata (release name/namespace/fullname), handing an +attacker the JWT signing key, the at-rest Fernet key and the admin/mcp passwords. We keep +(iii) (randAlphaNum) and get (i) from hashing inputs. Coverage: + - operator edit (`values.secrets.*`) -> input changes -> digest moves, pods roll NOW; + - a change to the persisted Secret -> `.data` changes (seen via lookup in a REAL + cluster) -> digest moves; + - rotate-away-from-a-persisted-known-default emits a fresh RANDOM value the checksum cannot + see at render time without re-rolling the random (or a cluster), so those pods roll on the + NEXT reconcile once the value persists into `.data` -- one sync, not never. +(An earlier "rotating-from-default" marker was removed: it was provably redundant -- the same +`.data` change already moves the digest, so the marker never altered the outcome. A bare +no-cluster `helm template` has no lookup and reflects only `values.secrets`; that is a +template-mode limitation, not a deploy defect -- GitOps renders against the live cluster.) +*/}} +{{- define "bnk-forge.secretsChecksum" -}} +{{- $name := printf "%s-secrets" (include "bnk-forge.fullname" .) -}} +{{- $existing := lookup "v1" "Secret" .Release.Namespace $name -}} +{{- $data := dict -}} +{{- if and $existing $existing.data -}}{{- $data = $existing.data -}}{{- end -}} +{{- printf "%s|%s" (toYaml .Values.secrets) (toYaml $data) | sha256sum -}} +{{- end -}} diff --git a/helm/bnk-forge/templates/api.yaml b/helm/bnk-forge/templates/api.yaml index c685eeb..eb8cee1 100644 --- a/helm/bnk-forge/templates/api.yaml +++ b/helm/bnk-forge/templates/api.yaml @@ -28,6 +28,12 @@ spec: {{- include "bnk-forge.componentLabels" (list . "api") | nindent 6 }} template: metadata: + annotations: + # #187: roll the pod when the Secret changes, so the backend (which + # re-seeds the mcp account) and the MCP server (which authenticates with + # it) pick up a rotated mcp-password together instead of drifting for a + # restart cycle. + checksum/secret: {{ include "bnk-forge.secretsChecksum" . }} labels: {{- include "bnk-forge.componentLabels" (list . "api") | nindent 8 }} spec: diff --git a/helm/bnk-forge/templates/beat.yaml b/helm/bnk-forge/templates/beat.yaml index f3fc761..d0da674 100644 --- a/helm/bnk-forge/templates/beat.yaml +++ b/helm/bnk-forge/templates/beat.yaml @@ -15,6 +15,11 @@ spec: {{- include "bnk-forge.componentLabels" (list . "beat") | nindent 6 }} template: metadata: + annotations: + # #187 (bonnyr-f5 #188): beat consumes the same backendEnv Secret as the + # api/mcp pods, so it must roll on the one-shot changeme -> random rotation + # too — otherwise it keeps the old credential until something else restarts it. + checksum/secret: {{ include "bnk-forge.secretsChecksum" . }} labels: {{- include "bnk-forge.componentLabels" (list . "beat") | nindent 8 }} spec: diff --git a/helm/bnk-forge/templates/mcp.yaml b/helm/bnk-forge/templates/mcp.yaml index e62a3df..1cdb28e 100644 --- a/helm/bnk-forge/templates/mcp.yaml +++ b/helm/bnk-forge/templates/mcp.yaml @@ -29,6 +29,12 @@ spec: {{- include "bnk-forge.componentLabels" (list . "mcp") | nindent 6 }} template: metadata: + annotations: + # #187: roll the pod when the Secret changes, so the backend (which + # re-seeds the mcp account) and the MCP server (which authenticates with + # it) pick up a rotated mcp-password together instead of drifting for a + # restart cycle. + checksum/secret: {{ include "bnk-forge.secretsChecksum" . }} labels: {{- include "bnk-forge.componentLabels" (list . "mcp") | nindent 8 }} spec: @@ -57,16 +63,38 @@ spec: ports: - containerPort: 8081 name: mcp + # bonnyr-f5 #193 M7/M4/M5: the auth-probe healthcheck + # (`python -m bnk_forge_mcp.healthcheck`) belongs on READINESS only. It + # logs in to the backend and exits non-zero on 401, so a credential + # drift or a disabled mcp row (M8) takes the pod OUT OF SERVICE on + # Kubernetes too, not just the compose/dist paths — a tcpSocket readiness + # probe would only prove the port is open. The "no credentials at all -> + # NOT READY" half of that guarantee depends on the probe returning + # non-zero when no credential is configured; that behaviour ships with the + # credential-guard image (the tag this chart pins, image.tag, moves to it + # in lockstep at release). On an older pinned image the probe treats "no + # credentials" as ready, so verify MCP with a real tool call. readinessProbe: - tcpSocket: - port: 8081 + exec: + command: ["python", "-m", "bnk_forge_mcp.healthcheck"] initialDelaySeconds: 10 periodSeconds: 10 + timeoutSeconds: 10 + # bonnyr-f5 #193 M4: LIVENESS must test the PROCESS, not its + # dependency. healthcheck.py returns 1 when the BACKEND is merely + # unreachable; on Kubernetes (no `start_period` for liveness) a backend + # rollout / Postgres blip / credential drift lasting > 3x periodSeconds + # would make kubelet kill and restart every mcp pod for a dependency + # outage — restarting mcp fixes neither cause, so it settles into + # CrashLoopBackOff. A tcpSocket check restarts only a genuinely wedged + # process, matching compose's `healthcheck` semantics (unhealthy, never + # a restart-for-dependency). livenessProbe: tcpSocket: - port: 8081 + port: mcp initialDelaySeconds: 30 periodSeconds: 30 + timeoutSeconds: 10 resources: {{- toYaml .Values.mcp.resources | nindent 12 }} {{- end }} diff --git a/helm/bnk-forge/templates/secrets.yaml b/helm/bnk-forge/templates/secrets.yaml index 39531c5..d44c9d4 100644 --- a/helm/bnk-forge/templates/secrets.yaml +++ b/helm/bnk-forge/templates/secrets.yaml @@ -1,29 +1,132 @@ {{/* Re-use existing secret values across upgrades. */}} {{- $name := printf "%s-secrets" (include "bnk-forge.fullname" .) -}} {{- $existing := lookup "v1" "Secret" .Release.Namespace $name -}} +{{- /* Normalize the existing Secret's .data to a real map up front (bonnyr-f5): + a Secret can exist with NO .data map at all (nil) -- e.g. one created with + only stringData, or an empty placeholder -- in which case `index $existing.data` + and `hasKey $existing.data` both blow up with "index/hasKey of untyped nil" + and fail the whole render. Fold nil to an empty dict once, then every + per-key lookup below is a safe `hasKey $data ...` guard (which also covers + the NEW admin-password key that pre-#184 releases don't have). */ -}} +{{- $data := dict -}} +{{- if and $existing $existing.data -}}{{- $data = $existing.data -}}{{- end -}} {{- $pgPass := .Values.secrets.postgresPassword -}} -{{- if and (not $pgPass) $existing -}} -{{- $pgPass = (index $existing.data "postgres-password" | b64dec) -}} +{{- if and (not $pgPass) (hasKey $data "postgres-password") -}} +{{- $pgPass = (index $data "postgres-password" | b64dec) -}} {{- end -}} {{- if not $pgPass -}}{{- $pgPass = randAlphaNum 24 -}}{{- end -}} {{- $rdPass := .Values.secrets.redisPassword -}} -{{- if and (not $rdPass) $existing -}} -{{- $rdPass = (index $existing.data "redis-password" | b64dec) -}} +{{- if and (not $rdPass) (hasKey $data "redis-password") -}} +{{- $rdPass = (index $data "redis-password" | b64dec) -}} {{- end -}} {{- if not $rdPass -}}{{- $rdPass = randAlphaNum 24 -}}{{- end -}} {{- $jwt := .Values.secrets.jwtSecretKey -}} -{{- if and (not $jwt) $existing -}} -{{- $jwt = (index $existing.data "jwt-secret-key" | b64dec) -}} +{{- if and (not $jwt) (hasKey $data "jwt-secret-key") -}} +{{- $jwt = (index $data "jwt-secret-key" | b64dec) -}} {{- end -}} {{- if not $jwt -}}{{- $jwt = randAlphaNum 64 -}}{{- end -}} {{- $enc := .Values.secrets.encryptionKey -}} -{{- if and (not $enc) $existing -}} -{{- $enc = (index $existing.data "encryption-key" | b64dec) -}} +{{- if and (not $enc) (hasKey $data "encryption-key") -}} +{{- $enc = (index $data "encryption-key" | b64dec) -}} +{{- end -}} +{{/* bonnyr-f5 #193 (minor): a generated ENCRYPTION_KEY must be a VALID Fernet key + (urlsafe-base64 of exactly 32 bytes). A wrong-shaped value made Fernet(key) raise + "must be 32 url-safe base64-encoded bytes"; it was harmless only because + core/encryption.py reads /app/keys/encryption.key directly rather than + settings.ENCRYPTION_KEY, but validate_production REQUIRES a non-empty value, so + the chart must still emit one — and a wrong-shaped one is a latent trap the moment + anything consumes the env. randAlphaNum 32 yields exactly 32 bytes; b64enc encodes + them; translate the std alphabet to the urlsafe one so Fernet's urlsafe decode + accepts it. bonnyr-f5 #193 M7: the generated fallback is RANDOM (never derived from + release identity — a derived key would be computable from public chart/label + metadata and forge-able). A real cluster persists it via lookup, so it is stable + across renders; the `checksum/secret` digest tracks rotations through deterministic + inputs, not by re-rendering this random value (see bnk-forge.secretsChecksum). */}} +{{- if not $enc -}}{{- $enc = (randAlphaNum 32 | b64enc | replace "+" "-" | replace "/" "_") -}}{{- end -}} + +{{- $adminPass := .Values.secrets.adminPassword -}} +{{- if and (not $adminPass) (hasKey $data "admin-password") -}} +{{- $adminPass = (index $data "admin-password" | b64dec) -}} +{{- end -}} +{{- if not $adminPass -}}{{- $adminPass = randAlphaNum 20 -}}{{- end -}} +{{/* bonnyr-f5 #193 (minor): mirror the mcpPassword fail-guard for adminPassword. + auth_service.seed_admin_user / _rotate_known_default_admin refuse to seed or + rotate TO a known published default (it would re-publish the very credential + #184 removes), and fall through to generation instead — so a chart that + quietly emitted `admin-password: changeme` would hand the backend a value it + silently drops, leaving the operator reading a Secret that never + authenticates. Refuse it at render so the chart and the backend AGREE. + Deterministic (never a rotate -> no GitOps drift): fails identically until a + real value is set. The auto-generated (randAlphaNum) and chart-persisted + values are never a known default, so this only bites a values-pinned one. */}} +{{- $adminDefaults := list "changeme" -}} +{{- if has $adminPass $adminDefaults -}} +{{- fail "secrets.adminPassword is set to a shipped default (\"changeme\"); the backend refuses to seed or rotate to a known published default, so it would never authenticate. Set secrets.adminPassword to a strong value, or leave it empty to auto-generate one." -}} +{{- end -}} + +{{/* #186 BLOCKER 2 / #187 (bonnyr-f5): mcp-password must never be a shipped default + ("changeme" / "mcp-service-changeme" were live, publicly-known admin credentials). + Honour an operator-set value; else reuse the persisted one from a prior install + UNLESS it is a known default, in which case ROTATE it — an existing install carries + the default in its Secret and the mcp account is role=admin, must_change_password + =False, so #186's gate never rotates it (the chart must). Scope the rotate to the + PERSISTED value only, never a values-supplied one: a values-pinned default is instead + refused outright by the deterministic fail guard below, so it never reaches this + rotate branch. bonnyr-f5 #193 M7: the rotated value is RANDOM (randAlphaNum, never + derived from release identity). In a real cluster it is written once and then reused + via lookup, so the rotate fires only on the sync that still sees the persisted + default; the `checksum/secret` digest flips on that sync via a deterministic + "rotating-from-default" marker (see bnk-forge.secretsChecksum), so the pods roll. The + lookup goes through the normalized $data map (+ hasKey guard) so a Secret that + exists with a nil .data map can never blow up the render. This list MUST stay in + lockstep with MCP_KNOWN_DEFAULT_PASSWORDS in backend/core/config.py (round 5, + Major-2): validate_production treats every one of these as fatal under + ENVIRONMENT=production, so the chart must never emit one. */}} +{{- $mcpDefaults := list "changeme" "mcp-service-changeme" -}} +{{- $mcpPass := .Values.secrets.mcpPassword -}} +{{- if and (not $mcpPass) (hasKey $data "mcp-password") -}} +{{- $mcpFromSecret := (index $data "mcp-password" | b64dec) -}} +{{- if has $mcpFromSecret $mcpDefaults -}} +{{- $mcpPass = randAlphaNum 24 -}} +{{- else -}} +{{- $mcpPass = $mcpFromSecret -}} +{{- end -}} +{{- end -}} +{{- if not $mcpPass -}}{{- $mcpPass = randAlphaNum 24 -}}{{- end -}} +{{/* bonnyr-f5 #188 round 5 (Major-2): an operator who PINS a shipped default in + values.yaml is not rotated above (that path only fires for a value carried in the + persisted Secret; rotating a values-supplied value re-drifts every sync). But + config.py makes that same value fatal at import under ENVIRONMENT=production -> + api/worker/beat crashloop. Refuse it at render time so the chart and the fail-fast + AGREE: a known default never reaches a pod. Deterministic (unlike a rotate), so it + introduces no drift — it fails identically until the operator sets a real value. */}} +{{- if has $mcpPass $mcpDefaults -}} +{{- fail "secrets.mcpPassword is set to a shipped default (\"changeme\" / \"mcp-service-changeme\"); the backend rejects it as fatal under ENVIRONMENT=production. Set secrets.mcpPassword to a strong value (the same value the MCP server gets as BNK_FORGE_PASSWORD), or leave it empty to auto-generate one." -}} +{{- end -}} +{{/* bonnyr-f5 #193 M8: refuse a reserved HUMAN username for the mcp service account. + values.yaml historically shipped `mcpUsername: admin`, and this chart now plumbs + mcp-username -> MCP_SERVICE_USERNAME onto api/worker/beat. If a GitOps repo pins + `admin`, the backend's unconditional disable deactivates the legitimate mcp row, + ensure_service_user then raises the reserved-name refusal (correct, non-fatal), + nothing re-enables the row, and MCP is permanently down while `helm upgrade` + reports success. Fail loudly at render, mirroring the mcpPassword guard above and + _RESERVED_HUMAN_USERNAMES in backend/services/auth_service.py. Deterministic, so + no drift -- it fails identically until the operator picks a dedicated name. */}} +{{- $mcpReservedUsernames := list "admin" -}} +{{/* bonnyr-f5 #193 (minor): guard the nil case FIRST. A nil `mcpUsername` (unset, + or `mcpUsername:` with no value) makes `trim` raise "wrong type for value ... + as it's not a string" and aborts the whole render with a Go type error rather + than a useful message. `and` is not short-circuit in text/template (both args + evaluate), so this must be a nested guard, not `and`. Mirror the + kindIs "invalid" pattern _helpers.tpl already uses for adminMustChange. */}} +{{- if not (kindIs "invalid" .Values.secrets.mcpUsername) -}} +{{- if has (lower (trim .Values.secrets.mcpUsername)) $mcpReservedUsernames -}} +{{- fail "secrets.mcpUsername is set to a reserved human identity (\"admin\"); a service account must not co-opt the human admin login. The backend refuses to reconcile it, so the mcp account would be disabled on upgrade and MCP would go permanently down while helm reports success. Set secrets.mcpUsername to a dedicated name like \"mcp\"." -}} +{{- end -}} {{- end -}} -{{- if not $enc -}}{{- $enc = randAlphaNum 32 -}}{{- end -}} apiVersion: v1 kind: Secret @@ -37,5 +140,9 @@ stringData: redis-password: {{ $rdPass | quote }} jwt-secret-key: {{ $jwt | quote }} encryption-key: {{ $enc | quote }} - mcp-username: {{ .Values.secrets.mcpUsername | quote }} - mcp-password: {{ .Values.secrets.mcpPassword | quote }} + admin-password: {{ $adminPass | quote }} + # bonnyr-f5 #193 (minor): default a nil/empty mcpUsername to "mcp". Without this a + # `mcpUsername:` with no value renders `mcp-username: ""` past the reserved-name + # nil-guard above, and the empty string would flow to MCP_SERVICE_USERNAME. + mcp-username: {{ .Values.secrets.mcpUsername | default "mcp" | quote }} + mcp-password: {{ $mcpPass | quote }} diff --git a/helm/bnk-forge/templates/worker.yaml b/helm/bnk-forge/templates/worker.yaml index 492bd6d..74e434e 100644 --- a/helm/bnk-forge/templates/worker.yaml +++ b/helm/bnk-forge/templates/worker.yaml @@ -13,6 +13,11 @@ spec: {{- include "bnk-forge.componentLabels" (list . "worker") | nindent 6 }} template: metadata: + annotations: + # #187 (bonnyr-f5 #188): worker consumes the same backendEnv Secret as the + # api/mcp pods, so it must roll on the one-shot changeme -> random rotation + # too — otherwise it keeps the old credential until something else restarts it. + checksum/secret: {{ include "bnk-forge.secretsChecksum" . }} labels: {{- include "bnk-forge.componentLabels" (list . "worker") | nindent 8 }} spec: diff --git a/helm/bnk-forge/values.yaml b/helm/bnk-forge/values.yaml index bb88060..164d6a0 100644 --- a/helm/bnk-forge/values.yaml +++ b/helm/bnk-forge/values.yaml @@ -16,7 +16,7 @@ global: image: pullPolicy: IfNotPresent - tag: "3.0.1" + tag: "3.1.6" # Generated/explicit secrets. If left empty, helm generates random values on # first install and reuses them on upgrade (lookup-based). @@ -25,8 +25,26 @@ secrets: redisPassword: "" jwtSecretKey: "" encryptionKey: "" - mcpUsername: admin - mcpPassword: changeme + # Empty -> generated on first install and reused on upgrade. The seeded admin + # account is must_change_password; retrieve this to log in the first time. + adminPassword: "" + # #186: whether the seeded admin must change its password before using the API. + # Keep true on any real deployment (the shipped-default hazard #184 closes). + # Set false only for an ephemeral test stack seeding a known throwaway admin. + adminMustChange: true + # #186 BLOCKER 2/3 / #187 (bonnyr-f5 r5): NEVER ship a default admin credential. + # - mcpUsername was `admin`: it pointed the MCP client at the human admin + # account and, once the backend receives MCP_SERVICE_USERNAME, made + # ensure_service_user rewrite the human admin row (BLOCKER 3). The MCP + # client authenticates as its own dedicated, non-human service account. + # - mcpPassword was `changeme`: a live, publicly-known credential on a public + # repo. Empty -> generated per-install in the release Secret (mirrors + # adminPassword) and reused on upgrade, so no shipped default ever exists. + # bonnyr-f5 #193 M8: the chart now FAILS the render if mcpUsername is set to a + # reserved human identity ("admin") -- a service account must not co-opt the + # human admin login, and pinning it would silently disable MCP on upgrade. + mcpUsername: mcp + mcpPassword: "" # Common pod settings # fsGroup=1000 so PVC-backed shared volumes are writable by the bnkforge user (UID 1000). @@ -89,10 +107,17 @@ api: env: BNK_FORGE_DEPLOY_MODE: server ENVIRONMENT: production - # Override per-env. Wildcards rejected in production. Comma-separated origins. - ALLOWED_ORIGINS: "https://localhost" - # If running on a non-localhost cluster, set e.g.: - # ALLOWED_ORIGINS: "https://hgx2:30443,http://hgx2:30080" + # bonnyr-f5 #193 M10: ships EMPTY, not "https://localhost". Under + # ENVIRONMENT=production the backend's validate_production() SystemExits (crashloop) + # if ALLOWED_ORIGINS contains "localhost" or "*", so the old "https://localhost" + # default bricked every bare `helm install`. Empty is accepted by the backend (it is + # neither localhost nor a wildcard) so the default render boots; the chart also fails + # AT RENDER on a localhost/wildcard value under production/staging (see backendEnv). + # Set this to your actual browser origin(s) — comma-separated, no wildcard, no + # localhost in production, e.g.: + # ALLOWED_ORIGINS: "https://forge.example.com" + # ALLOWED_ORIGINS: "https://hgx2:30443,http://hgx2:30080" + ALLOWED_ORIGINS: "" resources: requests: cpu: 250m diff --git a/mcp-server/README.md b/mcp-server/README.md index 62eea79..9ff75dd 100644 --- a/mcp-server/README.md +++ b/mcp-server/README.md @@ -56,10 +56,13 @@ readiness verification explicit and repeatable. - `ping` + `tools/list` pass, but `system_version`/`list_clusters` fail with `auth_error`: MCP transport is up, but runtime auth/bootstrap is not ready. -- Typical cause: MCP container credentials do not match current backend credentials - (for example after rotating admin password). -- Action: set `MCP_USERNAME` / `MCP_PASSWORD` for the MCP service and recreate the - `mcp` container, then rerun smoke. +- Typical cause: the MCP service-account password drifted from the backend's seeded + value (e.g. `MCP_SERVICE_PASSWORD` changed on one side only). MCP uses its own + dedicated `mcp` service account, never the human admin login (#187). +- Action: set `MCP_SERVICE_PASSWORD` in `.env` (compose maps it to the container's + `BNK_FORGE_PASSWORD` and the backend's `MCP_SERVICE_PASSWORD`) and recreate the `mcp` + container, then rerun smoke. Do NOT set `MCP_USERNAME` — it is not read by either + process; the service username is fixed to the dedicated `mcp` account. ### Scope boundaries (intentional) @@ -94,17 +97,29 @@ pytest tests/ - MCP runtime is healthy only when **both** conditions are true: 1. MCP JSON-RPC endpoint responds (`ping`) 2. MCP can authenticate to backend and execute governed read-only tools -- If backend admin password is changed (recommended), MCP credentials must be - updated too (`MCP_USERNAME` / `MCP_PASSWORD` or `BNK_FORGE_TOKEN`). +- The MCP container process reads **only** `BNK_FORGE_*` (see the table above): + its login password comes from `BNK_FORGE_PASSWORD`. In the shipped compose + files the operator sets a single `.env` value, `MCP_SERVICE_PASSWORD`, which + compose maps to `BNK_FORGE_PASSWORD` for this container **and** to + `MCP_SERVICE_PASSWORD` for the backend — so the two always agree. That value + must match what the backend seeded, or supply `BNK_FORGE_TOKEN` instead. It is + decoupled from the human admin password (#187). (bonnyr-f5 #193 M7: earlier + text named `MCP_PASSWORD` here — nothing in this process reads that name; it is + only a legacy compose-level alias for the password `.env` value.) - Without this alignment, the MCP container may look healthy at protocol level while tool execution fails with backend login 401. ### Credential rotation runbook (bounded) -When backend admin password is rotated: +When the MCP service-account password (`MCP_SERVICE_PASSWORD`) is rotated: -1. Update MCP runtime credentials in environment (`MCP_USERNAME`, `MCP_PASSWORD`) -2. Recreate MCP so new env values are applied: +1. Set the new password in the compose `.env` as `MCP_SERVICE_PASSWORD` (bonnyr-f5 + #193 M7/B1: this is the ONLY var to set — compose maps it to the container's + `BNK_FORGE_PASSWORD` and the backend's `MCP_SERVICE_PASSWORD`. Do NOT set + `MCP_USERNAME`: it is not read by either process, and the username is fixed to + the dedicated `mcp` service account; the password's legacy alias `MCP_PASSWORD` + still works but prefer the canonical name.) +2. Recreate MCP so the new env value is applied: ```bash # server/default compose diff --git a/mcp-server/src/bnk_forge_mcp/healthcheck.py b/mcp-server/src/bnk_forge_mcp/healthcheck.py index 69cac3a..49d1ae7 100644 --- a/mcp-server/src/bnk_forge_mcp/healthcheck.py +++ b/mcp-server/src/bnk_forge_mcp/healthcheck.py @@ -33,12 +33,25 @@ def probe() -> int: """ config = load_config() - if not config.has_credentials: - # No credentials at all — can't probe; treat as healthy so we don't - # flip unhealthy on token-only deployments. - logger.info("no credentials configured; skipping auth probe") + # A bearer token is a self-sufficient auth path. When one is set, the server + # authenticates tool calls with it regardless of the password, so a drifted (or + # absent) password must NOT fail the healthcheck. This probe can only exercise + # username/password via /api/auth/login, so with a token present we skip it and + # report healthy; the token is validated on real tool calls. (bonnyr-f5 #188: + # previously the token+stale-password row was inverted — the password probe ran + # and reported a token-authenticated container UNHEALTHY.) + if config.has_token: + logger.info("token auth configured — no login probe to run, reporting healthy") return 0 + if not config.has_credentials: + # bonnyr-f5 #188: neither password NOR token. With MCP_SERVICE_PASSWORD now + # shipping empty by default, this means the MCP server CANNOT authenticate — + # every tool call 401s. Reporting healthy here made a default `make deploy` + # show a green mcp container that does nothing. Fail the probe instead. + logger.error("no MCP credentials configured — cannot authenticate to the backend") + return 1 + try: resp = httpx.post( f"{config.api_base_url}/api/auth/login", diff --git a/mcp-server/tests/test_healthcheck.py b/mcp-server/tests/test_healthcheck.py index 9900d97..edb8b42 100644 --- a/mcp-server/tests/test_healthcheck.py +++ b/mcp-server/tests/test_healthcheck.py @@ -104,27 +104,81 @@ def test_probe_returns_1_when_backend_unreachable() -> None: # ------------------------------------------------------------------ -def test_probe_returns_0_when_no_credentials_configured() -> None: - """No username/password set (token-only deployment) → skip probe, return 0.""" +def test_probe_returns_0_for_token_only_deployment() -> None: + """No username/password but a bearer token IS set → skip the login probe, return 0. + + The probe authenticates via /api/auth/login (username/password only), so it + cannot exercise a token; a token-only deployment is healthy and validated on + real tool calls, not here. + """ with patch("bnk_forge_mcp.healthcheck.load_config") as mock_cfg: cfg = MagicMock() cfg.has_credentials = False + cfg.has_token = True mock_cfg.return_value = cfg assert probe() == 0 -def test_probe_no_credentials_logs_skip(caplog) -> None: # type: ignore[no-untyped-def] - """No credentials → info log is emitted so a typo'd env var is greppable.""" +def test_probe_token_only_logs_skip(caplog) -> None: # type: ignore[no-untyped-def] + """Token-only → info log is emitted so the skip is greppable.""" import logging with patch("bnk_forge_mcp.healthcheck.load_config") as mock_cfg: cfg = MagicMock() cfg.has_credentials = False + cfg.has_token = True mock_cfg.return_value = cfg with caplog.at_level(logging.INFO, logger="bnk_forge_mcp.healthcheck"): result = probe() assert result == 0 - assert "no credentials configured" in caplog.text + assert "token auth configured" in caplog.text + + +def test_probe_returns_0_when_token_present_even_with_stale_password() -> None: + """bonnyr-f5 #188: token + a (possibly stale) password → skip the login probe. + + A bearer token is a self-sufficient auth path; the server authenticates tool + calls with it regardless of the password. The previous truth table ran the + password probe here and reported a token-authenticated container UNHEALTHY when + the password had drifted. It must report healthy and never hit the backend. + """ + with patch("bnk_forge_mcp.healthcheck.httpx.post") as mock_post, \ + patch("bnk_forge_mcp.healthcheck.load_config") as mock_cfg: + cfg = MagicMock() + cfg.has_credentials = True # a password IS set... + cfg.has_token = True # ...but a token is present too + mock_cfg.return_value = cfg + assert probe() == 0 + mock_post.assert_not_called() # never probes /api/auth/login + + +def test_probe_returns_1_when_no_auth_configured() -> None: + """Neither password NOR token → MCP cannot authenticate at all → exit 1 (UNHEALTHY). + + bonnyr-f5 #188: MCP_SERVICE_PASSWORD ships empty by default, so a default + deploy with no token would 401 on every tool call. Report UNHEALTHY, not green. + """ + with patch("bnk_forge_mcp.healthcheck.load_config") as mock_cfg: + cfg = MagicMock() + cfg.has_credentials = False + cfg.has_token = False + mock_cfg.return_value = cfg + assert probe() == 1 + + +def test_probe_no_auth_logs_error(caplog) -> None: # type: ignore[no-untyped-def] + """No auth at all → error log names the cause so a typo'd env var is greppable.""" + import logging + + with patch("bnk_forge_mcp.healthcheck.load_config") as mock_cfg: + cfg = MagicMock() + cfg.has_credentials = False + cfg.has_token = False + mock_cfg.return_value = cfg + with caplog.at_level(logging.ERROR, logger="bnk_forge_mcp.healthcheck"): + result = probe() + assert result == 1 + assert "cannot authenticate" in caplog.text # ------------------------------------------------------------------ diff --git a/scripts/compute_version_bump.sh b/scripts/compute_version_bump.sh index e888882..ac1e8a0 100755 --- a/scripts/compute_version_bump.sh +++ b/scripts/compute_version_bump.sh @@ -75,6 +75,17 @@ bump_version() { esac } +# ── Breaking-change detectors ──────────────────────────────────────────────── +# INV-15: _is_breaking_subject / _is_breaking_body are SINGLE-SOURCED in +# scripts/lib/breaking-change-detect.sh and shared with extract-breaking-changes.sh +# and the commit-lint gate (bonnyr-f5 #179 r6 F4 / #193 M5). They used to be two +# byte-identical inline copies kept in step by hand; the shared file makes drift +# structurally impossible, and scripts/tests/detector-parity.test.sh asserts the +# single-source wiring. Resolve the path from THIS script's location (not cwd) so the +# self-test's recursive temp-repo invocations still find the lib. +# shellcheck source=scripts/lib/breaking-change-detect.sh +. "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/breaking-change-detect.sh" + # ── Resolve baseline + since-tag ───────────────────────────────────────────── # Skipped entirely in SELF_TEST mode: the self-test runner below exercises # this same script recursively against isolated temp repos, so evaluating it @@ -88,6 +99,22 @@ else SINCE_TAG=$(last_final_tag) fi +# Fail closed on an unresolvable SINCE_TAG. Without this, an unknown ref makes +# `git log ..HEAD` empty (2>/dev/null || true), so BOTH the bump loop and +# the consistency guard read nothing and silently return patch -- a typo in the +# floor tag would ship a release derived from a range that was never read. +if [[ -n "$SINCE_TAG" ]] && ! git rev-parse --verify --quiet "${SINCE_TAG}^{commit}" >/dev/null; then + echo "::error::SINCE_TAG '${SINCE_TAG}' does not resolve to a commit in this repo -- refusing to derive a version from an empty range." >&2 + exit 1 +fi + +# A resolvable SINCE_TAG whose range is empty (tag == HEAD) still slips through as +# patch -- a phantom duplicate release (bonnyr-f5 #179). Refuse the empty range. +if [[ -n "$SINCE_TAG" ]] && [[ -z "$(git log "${SINCE_TAG}..HEAD" --format='%H' 2>/dev/null)" ]]; then + echo "::error::Range ${SINCE_TAG}..HEAD is empty (tag == HEAD?) -- refusing to derive a duplicate release." >&2 + exit 1 +fi + if [[ -n "$BASELINE_OVERRIDE" ]]; then BASELINE="$BASELINE_OVERRIDE" elif [[ -n "$SINCE_TAG" ]]; then @@ -108,32 +135,49 @@ fi BUMP_TYPE="patch" if [[ -z "$SINCE_TAG" ]]; then - RANGE_HASHES=$(git log --pretty=format:"%H" 2>/dev/null || true) + RANGE_HASHES=$(git log --format='%H' 2>/dev/null || true) else - RANGE_HASHES=$(git log "${SINCE_TAG}..HEAD" --pretty=format:"%H" 2>/dev/null || true) + RANGE_HASHES=$(git log "${SINCE_TAG}..HEAD" --format='%H' 2>/dev/null || true) fi while IFS= read -r sha; do [[ -z "$sha" ]] && continue subject=$(git log -1 --format="%s" "$sha" 2>/dev/null || true) - body=$(git log -1 --format="%b" "$sha" 2>/dev/null || true) + # Read the FULL raw message (%B), not just %b: a footer git folded into %s + # keeps its newline in %B, so _is_breaking_body catches a folded footer there, + # and scanning %B (not the subject) for the marker stops subject prose from + # over-matching (bonnyr-f5 #179 r5). + message=$(git log -1 --format='%B' "$sha" 2>/dev/null || true) - # Major: `type!:` in the subject, OR a BREAKING CHANGE / BREAKING-CHANGE marker - # anywhere in the message (footer or deliberate prose). - if echo "$subject" | grep -qE '^[a-z]+(\([^)]*\))?!:' \ - || printf '%s\n%s\n' "$subject" "$body" | grep -qE '\bBREAKING[[:space:] -]+CHANGE\b'; then + # Major: a `type!:` bang subject, OR a footer-anchored BREAKING CHANGE in the + # full message. Both checks are pipe-free here-strings on purpose (PR #177 + # review): `foo | grep -q` under `set -o pipefail` takes SIGPIPE (141) when grep + # matches early on a large body, which pipefail turns into a failed test, so + # *finding* the marker silently kept the bump at patch. Detection lives in + # _is_breaking_subject / _is_breaking_body, single-sourced in + # scripts/lib/breaking-change-detect.sh and shared with the extractor. + if _is_breaking_subject "$subject" || _is_breaking_body "$message"; then BUMP_TYPE="major" break fi # Minor: feat: in the subject (type is declared in the subject, never the body). if [[ "$BUMP_TYPE" != "major" ]]; then - if echo "$subject" | grep -qE '^feat(\([^)]*\))?:'; then + if grep -qE '^feat(\([^)]*\))?:' <<< "$subject"; then BUMP_TYPE="minor" fi fi done <<< "$RANGE_HASHES" +# NOTE (bonnyr-f5 #179 r6 F6): a second "consistency guard" loop used to sit here, +# re-deriving "is any commit breaking" and refusing to ship if it disagreed with +# BUMP_TYPE. It was REMOVED as provably dead code: it iterated the SAME +# RANGE_HASHES in the SAME order, called the SAME detectors, and broke on the SAME +# first hit -- so guard_breaking=1 implies the loop above already saw that commit +# first and set BUMP_TYPE=major, making the `guard_breaking && != major` condition +# unsatisfiable. It could never fire and no fixture could reach it, so it added a +# full second range re-scan for zero defence rather than genuine independence. + # ── Compute target version ──────────────────────────────────────────────────── TARGET_VERSION=$(bump_version "$BASELINE" "$BUMP_TYPE") @@ -147,15 +191,35 @@ fi if [[ "${SELF_TEST:-0}" == "1" ]]; then echo "" echo "=== SELF-TEST ===" + SELFTEST_FAILURES=0 + SELFTEST_ASSERTIONS=0 + + # Resolve this script's own absolute path from BASH_SOURCE BEFORE unsetting + # SELF_TEST, so the recursive invocations below can find it regardless of cwd + # (the old "$OLDPWD/$(dirname "$0")" broke any non-cwd-relative call). + SELFTEST_SCRIPT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/$(basename "${BASH_SOURCE[0]}")" # SELF_TEST is an inherited environment variable — without unsetting it # here, run_test's recursive script invocations below would also enter # self-test mode and recurse indefinitely (fork bomb). unset SELF_TEST + # Expand the literal `\n` escape in a fixture to real newlines. Done with awk + # (linear) rather than bash `${_b//\\n/$'\n'}`: that global parameter-expansion + # substitution hits an O(n^2) cliff on bash 3.2 (stock macOS), where the ~80 KB + # Test-7 body alone took ~134 s and the self-test did not finish in 6 min + # (bonnyr-f5 #193 M3). awk is O(n) on every interpreter. RS="\1" reads the whole + # (newline-free, escaped) input as ONE record; printf keeps it exact (no trailing + # newline added). + _expand_nl() { + printf '%s' "$1" | awk 'BEGIN { RS = "\1" } { gsub(/\\n/, "\n"); printf "%s", $0 }' + } + run_test() { local desc="$1" expected_bump="$2" expected_ver="$3" local since="$4" baseline="$5" commits_str="$6" + local _b + SELFTEST_ASSERTIONS=$((SELFTEST_ASSERTIONS + 1)) # Create a temp dir with a fake git repo for deterministic testing local tmpdir tmpdir=$(mktemp -d) @@ -175,21 +239,31 @@ if [[ "${SELF_TEST:-0}" == "1" ]]; then while IFS= read -r entry; do [[ -z "$entry" ]] && continue if [[ "$entry" == *"~~BODY~~"* ]]; then + # A body may use a literal \n escape for a real newline: entries are + # split on newlines, so a raw one would fork into extra commits. + _b="$(_expand_nl "${entry#*~~BODY~~}")" git -C "$tmpdir" commit --allow-empty \ - -m "${entry%%~~BODY~~*}" -m "${entry#*~~BODY~~}" -q + -m "${entry%%~~BODY~~*}" -m "$_b" -q else - git -C "$tmpdir" commit --allow-empty -m "$entry" -q + # A plain entry may embed a literal \n for a FOLDED single message: + # subject and footer on consecutive lines with NO blank between, so git + # folds them into %s (leaving %b empty) but %B keeps the newline. One -m + # with an embedded newline reproduces exactly that shape. + git -C "$tmpdir" commit --allow-empty -m "$(_expand_nl "$entry")" -q fi done <<< "$(echo "$commits_str" | tr ',' '\n')" # Run the version computer inside the temp repo so it scans the fake range. local result - result=$(cd "$tmpdir" && bash "$OLDPWD/$(dirname "$0")/$(basename "$0")" \ + # Invoke by absolute path resolved from BASH_SOURCE, so the self-test works + # regardless of cwd or how the script was called ($OLDPWD/$0 broke any + # non-cwd-relative invocation -- bonnyr-f5 #179 r3 nit). + result=$(cd "$tmpdir" && bash "$SELFTEST_SCRIPT" \ ${since:+--since-tag "$since"} --baseline "$baseline" 2>/dev/null || true) local got_bump got_ver - got_bump=$(echo "$result" | grep BUMP_TYPE | cut -d= -f2) - got_ver=$(echo "$result" | grep TARGET_VERSION | cut -d= -f2) + got_bump=$(echo "$result" | grep BUMP_TYPE | cut -d= -f2 || true) + got_ver=$(echo "$result" | grep TARGET_VERSION | cut -d= -f2 || true) rm -rf "$tmpdir" @@ -199,6 +273,7 @@ if [[ "${SELF_TEST:-0}" == "1" ]]; then echo " FAIL: $desc" echo " expected bump=$expected_bump ver=$expected_ver" echo " got bump=$got_bump ver=$got_ver" + SELFTEST_FAILURES=$((SELFTEST_FAILURES + 1)) fi } @@ -214,8 +289,17 @@ if [[ "${SELF_TEST:-0}" == "1" ]]; then run_test "breaking ! → major" "major" "2.0.0" "v1.2.3" "1.2.3" \ "feat!: redesign API,fix(ui): icon" - # Test 4: no commits → patch bump - run_test "no commits → patch" "patch" "1.2.4" "v1.2.3" "1.2.3" "" + # Test 4: empty range (tag == HEAD) → the guard refuses (no output). + run_test "empty range (tag==HEAD) → refused" "" "" "v1.2.3" "1.2.3" "" + + # Test 4b (bonnyr-f5 INV-15): a marker in SUBJECT prose only must NOT bump. + run_test "marker in subject prose → patch" "patch" "1.2.4" "v1.2.3" "1.2.3" "docs: explain the BREAKING CHANGE footer" + + # Test 4c (r5 Minor 1): a docs subject that quotes `BREAKING CHANGE:` with a + # colon must NOT over-bump. The old subject match was unanchored across the + # whole subject; the detector is now bang-only, and %B is a single subject line + # so the body anchor finds no footer either. + run_test "docs subject quotes marker → patch" "patch" "1.2.4" "v1.2.3" "1.2.3" "docs: clarify what BREAKING CHANGE: means" # Test 5: BREAKING CHANGE footer in the BODY → major (the PR #177 bug: a # fix-subject commit whose body declares the break must still bump major). @@ -227,5 +311,109 @@ if [[ "${SELF_TEST:-0}" == "1" ]]; then run_test "lowercase breaking change in body → patch" "patch" "1.2.4" "v1.2.3" "1.2.3" \ "fix: tidy up~~BODY~~this is explicitly not a breaking change" + # Test 7: marker on an early LINE with ~90 KB of lines after it -- the real + # SIGPIPE bug (PR #177 review). grep is line-oriented, so with the marker on + # line 1 it matches and exits while the writer still has the tail to push; the + # pipe form takes SIGPIPE (141) and reads it as "no match". Deterministically + # wrong once the tail clears the 64 KB pipe buffer. A single ~94 KB *line* + # would NOT reproduce it (grep must read the whole line before deciding), so + # the tail must be many lines. 400 lines x ~200 bytes = ~80 KB comfortably + # clears the 64 KB buffer while running ~4x fewer loop iterations than the old + # 1500x60 form (bonnyr-f5 #179 r6 runtime nit). + _line=$(head -c 200 + # colon path. Red under r5, green under r6. + run_test "F1: scoped folded footer → major" "major" "2.0.0" "v1.2.3" "1.2.3" \ + "feat(api): drop v1\\nBREAKING CHANGE: all /api/v1 removed" + + # Test 9b (F1 control): the same folded footer WITHOUT a colon on a scoped subject + # must NOT trigger major -- the trailer/subject anchor demands a colon (only the + # blank-line anchor accepts a colon-less marker), which is what keeps F2 inert. A + # `fix(...)` subject is used (not `feat`) so the expected floor is a clean patch + # rather than the minor a feat subject would independently earn. + run_test "F1 control: scoped folded colonless → patch" "patch" "1.2.4" "v1.2.3" "1.2.3" \ + "fix(api): drop v1\\nBREAKING CHANGE happened here" + + # Test 10 (bonnyr-f5 #179 r6 F2 MAJOR): a prose section header (`Before:`) is + # trailer-shaped, but a colon-LESS marker following it is prose, not a footer, so + # it must stay patch. This is the round-4 false positive the r6 trailer->colon + # rule closes. Green only because the marker below has no colon. + run_test "F2: prose header + colonless marker → patch" "patch" "1.2.4" "v1.2.3" "1.2.3" \ + "docs: explain migration~~BODY~~Before:\\nBREAKING CHANGE was matched anywhere before." + + # Test 11 (bonnyr-f5 #179 r6 F7 MINOR): a real footer with the separator widened + # -- a DOUBLE space `BREAKING CHANGE:` -- must still derive major (the base regex + # only allowed a single space/hyphen). + run_test "F7: double-space marker → major" "major" "2.0.0" "v1.2.3" "1.2.3" \ + "fix: rework flags~~BODY~~BREAKING CHANGE: the --legacy flag was removed" + + # Test 11b (F7): a markdown BULLET marker `- BREAKING CHANGE:` (release notes copy + # bullets footers) must still derive major. + run_test "F7: dash-bullet marker → major" "major" "2.0.0" "v1.2.3" "1.2.3" \ + "fix: rework flags~~BODY~~- BREAKING CHANGE: the --legacy flag was removed" + + # Test 8 (guard coverage — bonnyr-f5 #179 r3): an unresolvable --since-tag must + # fail CLOSED (rc 1, no BUMP_TYPE output), not derive patch from an empty range. + # This exercises the unresolvable-SINCE_TAG guard, which previously had none. + _grd_tmp=$(mktemp -d) + git init -q "$_grd_tmp" + git -C "$_grd_tmp" config user.email "selftest@bnk-forge.local" + git -C "$_grd_tmp" config user.name "bnk-forge self-test" + git -C "$_grd_tmp" commit --allow-empty -m "initial" -q + SELFTEST_ASSERTIONS=$((SELFTEST_ASSERTIONS + 1)) + _grd_rc=0 + _grd_out=$(cd "$_grd_tmp" && bash "$SELFTEST_SCRIPT" --since-tag v9.9.9 --baseline 1.0.0 2>/dev/null) || _grd_rc=$? + rm -rf "$_grd_tmp" + if [[ "$_grd_rc" -ne 0 && -z "$_grd_out" ]]; then + echo " PASS: unresolvable --since-tag fails closed (rc=$_grd_rc, no output)" + else + echo " FAIL: unresolvable --since-tag should refuse (got rc=$_grd_rc out='$_grd_out')" + SELFTEST_FAILURES=$((SELFTEST_FAILURES + 1)) + fi + echo "=== END SELF-TEST ===" + # INV-16: a harness that runs zero assertions must not report success. This + # catches a self-test that silently no-ops (e.g. a renamed run_test); #182's + # script-selftests job additionally asserts the PASS count and this END marker + # externally, so renaming the SELF_TEST guard itself is caught there too. + if [[ "$SELFTEST_ASSERTIONS" -eq 0 ]]; then + echo "SELF-TEST: zero assertions ran — harness is dead" >&2 + exit 1 + fi + if [[ "$SELFTEST_FAILURES" -ne 0 ]]; then + echo "SELF-TEST: ${SELFTEST_FAILURES} failure(s)" >&2 + exit 1 + fi + echo "compute_version_bump self-test: OK (${SELFTEST_ASSERTIONS} assertions)" fi diff --git a/scripts/e2e/config.py b/scripts/e2e/config.py index d60a464..e0affd4 100644 --- a/scripts/e2e/config.py +++ b/scripts/e2e/config.py @@ -179,8 +179,12 @@ class E2EConfig(BaseModel): # bnk-forge target. bnk_forge_url: str = "https://localhost" bnk_forge_admin_user: str = "admin" - # Don't put real passwords in committed YAML — leave this default - # and override via env (BNK_FORGE_PASSWORD) at run time. + # bonnyr-f5 #193: the admin password is no longer a fixed default (#184) — this + # placeholder ("changeme") authenticates NOWHERE. You MUST override it at run + # time via BNK_FORGE_PASSWORD to the same value the target was deployed with as + # DEFAULT_ADMIN_PASSWORD (or the generated per-install password); step_login + # rotates the freshly-seeded must_change password to that same value. Don't put + # real passwords in committed YAML. bnk_forge_admin_password: str = "changeme" # Whether the harness should `make install` a fresh local diff --git a/scripts/e2e/steps.py b/scripts/e2e/steps.py index b9cac95..1a86137 100644 --- a/scripts/e2e/steps.py +++ b/scripts/e2e/steps.py @@ -169,10 +169,11 @@ def step_login(ctx: Context) -> StepResult: """Log in as admin and store JWT on the client. Handles the freshly-seeded `must_change_password=True` case by - rotating the password to itself. After `make install` the admin - user is `admin` / `changeme` with the flag set; without this the - harness would deadlock on the first authed request, and operators - would have to bounce through the UI before re-running.""" + rotating the password to itself. The admin password is no longer a + fixed default (#184): set DEFAULT_ADMIN_PASSWORD when deploying the + target to the same value as `bnk_forge_admin_password` (BNK_FORGE_PASSWORD), + or point that config at the generated password. Without the rotation + the harness would deadlock on the first authed request.""" with StepRecorder("login") as r: body = ctx.client.login( ctx.cfg.bnk_forge_admin_user, diff --git a/scripts/extract-breaking-changes.sh b/scripts/extract-breaking-changes.sh index b0c71c6..d1128f5 100644 --- a/scripts/extract-breaking-changes.sh +++ b/scripts/extract-breaking-changes.sh @@ -10,27 +10,263 @@ # Prints a "### ⚠️ Breaking Changes" section, or nothing if there are none. set -euo pipefail +# ── Breaking-change detectors ──────────────────────────────────────────────── +# INV-15: _is_breaking_subject / _is_breaking_body are SINGLE-SOURCED in +# scripts/lib/breaking-change-detect.sh and shared with compute_version_bump.sh and +# the commit-lint gate (bonnyr-f5 #179 r6 F4 / #193 M5). If the extractor were +# narrower than the bumper a break would major with no note; if wider a note would +# appear with no bump -- the shared file removes that whole failure class by making +# them the SAME code, and scripts/tests/detector-parity.test.sh asserts the wiring. +# shellcheck source=scripts/lib/breaking-change-detect.sh +. "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/breaking-change-detect.sh" + +# Emit the BREAKING CHANGE footer paragraph(s) -- flattened, markdown-bold +# stripped. Takes the FULL raw message (%B). Capture uses the SAME start rule as +# _is_breaking_body so trigger and note never disagree: a marker preceded by a +# blank line anchors with or without a colon; a marker in the trailer block +# (preceded by another trailer, or folded directly onto a conventional-commit +# subject) anchors ONLY with a colon. It then captures the WHOLE footer paragraph, +# including ordinary prose continuation lines that merely contain a colon +# (`migration: ...`). The round-4 note stopped at the FIRST trailer-shaped +# continuation line, truncating 7ece9b04's real bullet mid-sentence (bonnyr-f5 +# #179 r5 Major 2). +# +# F5 (bonnyr-f5 #179 r6): the note now STOPS at the first REAL git-trailer line +# rather than merely stripping a TRAILING trailer run -- a trailer block FOLLOWED +# by prose (`BREAKING CHANGE: x` / `Co-Authored-By: a@b` / `then prose`) used to +# leak the address because the trailing-strip loop halted on the closing prose +# line. A "real" trailer is a capitalized `Word(-Word)*:` key (Co-Authored-By, +# Signed-off-by, Reviewed-by, Acked-by, Cc, Claude-Session, Change-Id, X-* and the +# no-space `Session:` form all match); a lowercase-prose colon line (`migration:`) +# does NOT match, so mid-footer prose continuations are kept (r5 Major 2 stays +# green). Marker lines are excluded from the stop so a hyphen-form `BREAKING-CHANGE:` +# is never mistaken for a trailer. A trailing CR is stripped so a CRLF commit +# message does not carry `\r` into CHANGELOG.md. +_breaking_note() { + awk ' + BEGIN { prev_blank = 1; prev_trailer = 0 } + { + sub(/\r$/, "") + if ($0 ~ /^[[:space:]]*$/) { if (p) para_end = 1; prev_blank = 1; prev_trailer = 0; next } + is_marker = ($0 ~ /^([*-][[:space:]]+)?(\*\*)?BREAKING[[:space:] -]+CHANGE/) + is_colon = ($0 ~ /^([*-][[:space:]]+)?(\*\*)?BREAKING[[:space:] -]+CHANGE(\*\*)?:/) + is_trailer = ($0 ~ /^[A-Za-z0-9][A-Za-z0-9-]*:([[:space:]]|$)/) + if (!p) { + if ((prev_blank && is_marker) || (prev_trailer && is_colon)) { p = 1; print } + } else if (para_end) { + if (prev_blank && is_marker) { print ""; print; para_end = 0 } else { exit } + } else { print } + is_subject = (NR == 1 && $0 ~ /^[A-Za-z]+(\([^)]*\))?!?:[[:space:]]/) + prev_blank = 0; prev_trailer = (is_trailer || is_subject) + } + ' <<< "$1" \ + | awk '{ a[NR] = $0 } END { + n = NR + for (i = 1; i <= NR; i++) { + if (a[i] ~ /^[A-Z][A-Za-z0-9]*(-[A-Za-z0-9]+)*:([[:space:]]|$)/ \ + && a[i] !~ /^([*-][[:space:]]+)?(\*\*)?BREAKING[[:space:] -]+CHANGE/) { n = i - 1; break } + } + for (i = 1; i <= n; i++) print a[i] + }' \ + | sed 's/\*\*//g' | tr '\n' ' ' | sed 's/ */ /g; s/^ *//; s/ *$//' +} + +if [[ "${1:-}" == "--self-test" ]]; then + fail=0 + assertions=0 + # Assert both the trigger AND the note for one body against expectations. + # $1=label $2=body $3=expect-trigger(1/0) + _assert() { + local label="$1" body="$2" want="$3" note trig + note=$(_breaking_note "$body") + if _is_breaking_body "$body"; then trig=1; else trig=0; fi + assertions=$((assertions + 1)) + if [[ "$want" == 1 ]]; then + # A positive case must BOTH trigger and yield a non-empty note — the old + # _expect_nonempty only checked the note inside a failure conjunct, so it + # could never actually fail (bonnyr-f5 #179 r3). + if [[ "$trig" == 1 && -n "$note" ]]; then + echo " ok: $label -> ${note:0:56}" + else + echo "FAIL: $label — expected trigger+note, got trig=$trig note='${note}'"; fail=1 + fi + else + if [[ "$trig" == 0 && -z "$note" ]]; then + echo " ok: $label (correctly inert)" + else + echo "FAIL: $label — expected inert, got trig=$trig note='${note}'"; fail=1 + fi + fi + } + _assert_subject() { # $1=label $2=subject $3=expect(1/0) + local label="$1" subj="$2" want="$3" trig + if _is_breaking_subject "$subj"; then trig=1; else trig=0; fi + assertions=$((assertions + 1)) + if [[ "$trig" == "$want" ]]; then echo " ok: $label"; else echo "FAIL: $label (trig=$trig want=$want)"; fail=1; fi + } + + # Positive: real footers at line-start, in the forms the spec/markdown allow. + _assert "spec footer" $'fix: y\n\nBREAKING CHANGE: USER must become 65532.' 1 + _assert "hyphen footer" $'fix: y\n\nBREAKING-CHANGE: config key renamed.' 1 + _assert "markdown-bold footer" $'feat: z\n\n**BREAKING CHANGE:** boom.' 1 + # Positive: TWO footers in one body — the note must contain BOTH (the old + # single-paragraph extractor dropped the second). + _assert "two footers both kept" $'feat: q\n\nBREAKING CHANGE: first thing changed.\n\nBREAKING CHANGE: second thing changed.' 1 + _two=$(_breaking_note $'feat: q\n\nBREAKING CHANGE: first thing changed.\n\nBREAKING CHANGE: second thing changed.') + assertions=$((assertions + 1)) + if [[ "$_two" == *"first thing"* && "$_two" == *"second thing"* ]]; then + echo " ok: both footer paragraphs present" + else echo "FAIL: second footer dropped -> '$_two'"; fail=1; fi + + # Negative: uppercase marker MID-LINE is prose, not a footer — must NOT trigger + # and must NOT produce a note (the exact false-positive of the old detector). + _assert "mid-line prose" "This is a BREAKING CHANGE: the API moved." 0 + _assert "lowercase prose" "this is explicitly not a breaking change" 0 + _assert "indented non-footer" $'fix: y\n\n BREAKING CHANGE: indented, not a footer' 0 + + # Subject-form bang detector. + _assert_subject "bang subject triggers" "feat!: drop the v1 API" 1 + _assert_subject "Capitalised bang triggers" "Feat!: drop the v1 API" 1 + _assert_subject "scoped bang triggers" "fix(core)!: rename" 1 + _assert_subject "normal subject inert" "feat: normal change" 0 + + # r4 BLOCKER 1 -- WRAPPED prose: a marker at column 1 of a line that is NOT + # paragraph-initial (mid-paragraph, the prose wrapped there) must NOT trigger. + # This is commit 8415ce1's shape, which defeated the bare `^` anchor. + _assert "wrapped-prose mid-paragraph" $'The detector matches a BREAKING\nCHANGE marker anywhere, but the note awk was anchored to\nline-start. A commit whose marker was not at line-start ("... a\nBREAKING CHANGE: ...") therefore bumped major yet produced an empty\nnote.' 0 + + # r4 -- a real break declared paragraph-initial with NO colon (the #2 shape) + # must still trigger. + _assert "paragraph-initial no-colon break" $'Context line about the change.\n\nBREAKING CHANGE, called out deliberately. USER must become 1000.' 1 + + # r4 MAJOR -- the note must STOP before a trailer block, or a Co-Authored-By + # email / Claude-Session URL leaks into a public release body. + _leak=$(_breaking_note $'feat: x\n\nBREAKING CHANGE: the key moved.\nCo-Authored-By: Someone \nClaude-Session: https://claude.ai/code/session_ABC') + assertions=$((assertions + 1)) + if [[ "$_leak" == *"the key moved"* && "$_leak" != *"someone@example.com"* && "$_leak" != *"claude.ai"* ]]; then + echo " ok: note stops before trailers (no email/URL leak) -> ${_leak:0:48}" + else echo "FAIL: note leaked a trailer -> '$_leak'"; fail=1; fi + + # Subject detector is BANG-ONLY now (r5): prose that merely names the marker, + # with or without a colon, must NOT trigger via the subject path. + _assert_subject "prose names marker in subj" "docs: explain the BREAKING CHANGE footer" 0 + _assert_subject "docs quotes marker w/ colon" "docs: clarify what BREAKING CHANGE: means" 0 + + # r4 BLOCKER 2 / r5 -- a footer git FOLDED into the subject is a real TWO-LINE + # message with no blank line. Derivation/extraction read %B; the subject line is + # trailer-shaped, so the marker on the next line anchors as a footer → trigger. + _assert "folded footer via %B (two-line)" $'fix: tighten the thing\nBREAKING CHANGE: the config key was renamed' 1 + + # r5 Major 1 -- a BREAKING CHANGE footer stacked directly after another trailer + # with NO blank line between (conventional-commits' canonical example) triggers, + # and the note starts at the marker (the preceding trailer is not captured). + _assert "stacked footer (after trailer)" $'feat: x\n\nReviewed-by: Z\nBREAKING CHANGE: drops the old API' 1 + _stk=$(_breaking_note $'feat: x\n\nReviewed-by: Z\nBREAKING CHANGE: drops the old API') + assertions=$((assertions + 1)) + if [[ "$_stk" == *"drops the old API"* && "$_stk" != *"Reviewed-by"* ]]; then + echo " ok: stacked footer note starts at marker -> ${_stk:0:48}" + else echo "FAIL: stacked footer note wrong -> '$_stk'"; fail=1; fi + + # r5 Major 2 -- the note must NOT truncate at the first prose `word:` line. A + # footer whose continuation contains ordinary prose colons (`migration:`) keeps + # the WHOLE paragraph; only a TRAILING run of real trailers is stripped. + _notrunc=$(_breaking_note $'feat: y\n\nBREAKING CHANGE: the runner changed.\nmigration: run the tool first.\nThat is the whole story.') + assertions=$((assertions + 1)) + if [[ "$_notrunc" == *"migration: run the tool first."* && "$_notrunc" == *"That is the whole story."* ]]; then + echo " ok: note keeps prose continuation -> ${_notrunc:0:56}" + else echo "FAIL: note truncated on prose colon -> '$_notrunc'"; fail=1; fi + + # r5 Minor 2 -- a trailer with a digit/dot token (`X-Session-1:`) or a no-space + # colon (`Session:` at line end) directly under the footer must be stripped, not + # leaked. The old `[A-Za-z][A-Za-z-]*: ` stop missed both. + _leak2=$(_breaking_note $'feat: x\n\nBREAKING CHANGE: the key moved.\nX-Session-1: 0decafbad\nClaude-Session: https://claude.ai/code/session_XYZ\nSession:') + assertions=$((assertions + 1)) + if [[ "$_leak2" == *"the key moved"* && "$_leak2" != *"0decafbad"* && "$_leak2" != *"claude.ai"* && "$_leak2" != *"Session"* ]]; then + echo " ok: digit-token / no-space trailers stripped -> ${_leak2:0:48}" + else echo "FAIL: non-standard trailer leaked -> '$_leak2'"; fail=1; fi + + # r6 F1 BLOCKER -- a footer FOLDED onto a SCOPED subject (`fix(core): x` on the + # first line, marker on the second, no blank) must trigger. The r5 anchor reached + # the colon only on UNSCOPED subjects because `(` broke the trailer regex; the r6 + # is_subject anchor arms the trailer->colon path for scoped subjects too. + _assert "F1 scoped folded footer" $'feat(api): drop v1\nBREAKING CHANGE: all /api/v1 removed' 1 + # r6 F1 control -- the SAME scoped fold WITHOUT a colon is prose, not a footer. + _assert "F1 scoped folded colonless (inert)" $'feat(api): drop v1\nBREAKING CHANGE happened here' 0 + + # r6 F2 MAJOR -- a prose section header (`Before:`) is trailer-shaped, but a + # colon-LESS marker following it is prose. Only the trailer->COLON rule anchors a + # marker in the trailer block, so this must stay inert (the round-4 false positive). + _assert "F2 prose header + colonless marker (inert)" $'docs: x\n\nBefore:\nBREAKING CHANGE was matched anywhere.' 0 + + # r6 F7 MINOR -- widened separator/bullet: a DOUBLE-space marker and a dash-bullet + # marker are real footers and must trigger with a non-empty note. + _assert "F7 double-space marker" $'fix: y\n\nBREAKING CHANGE: the --legacy flag was removed' 1 + _assert "F7 dash-bullet marker" $'fix: y\n\n- BREAKING CHANGE: the --legacy flag was removed' 1 + + # r6 F5 MINOR (note leak) -- a footer FOLLOWED by a `migration:` prose line and + # THEN a Co-Authored-By trailer: the note must keep the lowercase-prose `migration:` + # continuation but STOP at the first real git-trailer, so the email never leaks. + # The r5 trailing-strip halted on the closing `Co-Authored-By` (it was the last + # line) here it would leak because prose could follow; the r6 stop-at-first-trailer + # cuts it regardless of what follows. + _f5=$(_breaking_note $'feat: x\n\nBREAKING CHANGE: the key moved.\nmigration: see the upgrade guide.\nCo-Authored-By: Someone \nthen a trailing prose line.') + assertions=$((assertions + 1)) + if [[ "$_f5" == *"the key moved"* && "$_f5" == *"migration: see the upgrade guide."* \ + && "$_f5" != *"someone@example.com"* && "$_f5" != *"Co-Authored-By"* && "$_f5" != *"trailing prose"* ]]; then + echo " ok: note keeps migration prose, stops at Co-Authored-By -> ${_f5:0:56}" + else echo "FAIL: F5 note leaked/truncated wrongly -> '$_f5'"; fail=1; fi + + if [[ $assertions -eq 0 ]]; then + echo "FAIL: harness ran zero assertions"; fail=1 + fi + # END marker mirroring compute_version_bump.sh's self-test: it prints only after + # the LAST assertion, so the script-selftests gate can assert the harness reached + # the end (an early `exit 0` or a deleted assertion block is caught) WITHOUT the + # gate having to grep this script for its own --self-test flag (bonnyr-f5 #193 M6). + echo "=== END SELF-TEST ===" + [[ $fail -eq 0 ]] && echo "extract-breaking-changes self-test: OK ($assertions assertions)" + exit "$fail" +fi + SINCE="${1:?usage: extract-breaking-changes.sh [until_ref]}" UNTIL="${2:-HEAD}" +# Fail CLOSED on an unresolvable range. Without this the `git log` below yields +# empty output and rc 0 for a typo'd ref, so a release ships with no breaking- +# change section and no signal that the range was never read (bonnyr-f5 #179 r3: +# a silent failure that fools a reviewer will fool a release). A VALID range with +# no breaking commits is still fine — it prints nothing and exits 0. +# +# FAIL-CLOSED IS NOW EFFECTIVE (bonnyr-f5 #179 r6 F3; #193 minor): the three +# call sites in .github/workflows/release.yml invoke this script WITHOUT `|| true`, +# so this rc=1 propagates and fails the release instead of being swallowed into +# BREAKING="". (The old note here said to "merge #179 WITH or AFTER #181" — that +# already happened; both are in-tree and the `|| true` on the extract call is gone.) +for _ref in "$SINCE" "$UNTIL"; do + if ! git rev-parse --verify --quiet "${_ref}^{commit}" >/dev/null 2>&1; then + echo "::error::extract-breaking-changes: '${_ref}' does not resolve to a commit — refusing to emit an empty breaking-changes section from a bad range." >&2 + exit 1 + fi +done + block="" while IFS= read -r sha; do [[ -z "$sha" ]] && continue - body=$(git log -1 --format="%b" "$sha" 2>/dev/null || true) - # Uppercase footer/marker only (spec form), so body prose like "not a - # breaking change" does not false-trigger. - if printf '%s\n' "$body" | grep -qE '\bBREAKING[[:space:] -]+CHANGE\b'; then - subj=$(git log -1 --format="%s" "$sha" 2>/dev/null || true) - # The BREAKING CHANGE line and its paragraph (up to the next blank line), - # flattened to one line and stripped of markdown bold. - note=$(printf '%s\n' "$body" \ - | awk '/BREAKING[[:space:] -]+CHANGE/{p=1} p{print} p&&/^$/{exit}' \ - | tr '\n' ' ' | sed 's/\*\*//g; s/ */ /g; s/ *$//') + subj=$(git log -1 --format="%s" "$sha" 2>/dev/null || true) + subj=${subj%$'\r'} # strip a trailing CR so a CRLF subject never reaches CHANGELOG.md + # Full raw message (%B): a folded footer keeps its newline here, and the footer + # anchor / note extractor both need the whole message (bonnyr-f5 #179 r5). + message=$(git log -1 --format='%B' "$sha" 2>/dev/null || true) + if _is_breaking_subject "$subj" || _is_breaking_body "$message"; then + note=$(_breaking_note "$message") + # Belt-and-suspenders: a `type!:` subject with no body footer yields no note; + # point the operator at the commit rather than emitting a bare bullet. + [[ -z "$note" ]] && note="(see commit ${sha:0:9} for the breaking-change details)" block="${block}- **${subj}** ${note} " fi -done < <(git log "${SINCE}..${UNTIL}" --pretty=format:"%H" 2>/dev/null || true) +done < <(git log "${SINCE}..${UNTIL}" --format='%H') if [[ -n "$block" ]]; then printf '### ⚠️ Breaking Changes\n\n%s\n' "$block" diff --git a/scripts/get_dpu_pwd.sh b/scripts/get_dpu_pwd.sh index 779352a..d6f0271 100644 --- a/scripts/get_dpu_pwd.sh +++ b/scripts/get_dpu_pwd.sh @@ -1,3 +1,4 @@ +#!/usr/bin/env bash docker compose exec -it backend python -c " from database import SessionLocal from models.bare_metal import BareMetalHost diff --git a/scripts/ibm_cloud_bnk_forge.sh b/scripts/ibm_cloud_bnk_forge.sh index 39d4ff5..3511e01 100644 --- a/scripts/ibm_cloud_bnk_forge.sh +++ b/scripts/ibm_cloud_bnk_forge.sh @@ -25,7 +25,13 @@ set -euo pipefail # ── Tunables (override via environment if desired) ─────────────────────────── -BNK_FORGE_VERSION="${BNK_FORGE_VERSION:-latest}" # image tag to pull +# bonnyr-f5 #193 B1: the default below is DERIVED from the repo VERSION file and +# re-stamped at release by scripts/sync-version-artifacts.sh --write, so it always +# names an image the same release actually published — never a forward-dated guess +# and never `latest` (a floating tag can point at an image whose credential contract +# differs from this installer's). Do NOT hand-edit it; override at runtime only with +# a tag you have confirmed ships the same guards. +BNK_FORGE_VERSION="${BNK_FORGE_VERSION:-3.1.6}" # image tag to pull (sync-managed) NAME_PREFIX="${NAME_PREFIX:-bnk-forge}" SUFFIX="$(date +%m%d%H%M)" VPC_NAME="${NAME_PREFIX}-vpc-${SUFFIX}" @@ -84,7 +90,12 @@ ibmcloud is instance-profiles --output json | jq -e --arg p "${PROFILE}" '[.[].n log "Selected profile '${PROFILE}' (>= ${VCPU} vCPU, >= ${RAM} GB)." # ── Prompt 4: existing SSH key ─────────────────────────────────────────────── -mapfile -t SSH_KEYS < <(ibmcloud is keys --output json | jq -r '.[].name') +# `while read` not `mapfile`: mapfile/readarray is bash 4+, and stock macOS ships +# bash 3.2.57 where it is rc=127 (bonnyr-f5 #193 — same class fixed in the release +# scripts). Read the JSON array line-by-line into SSH_KEYS portably. +SSH_KEYS=() +while IFS= read -r _k; do [ -n "$_k" ] && SSH_KEYS+=("$_k"); done \ + < <(ibmcloud is keys --output json | jq -r '.[].name') [ "${#SSH_KEYS[@]}" -gt 0 ] || die "No SSH keys found in region '${REGION}'. Create one: ibmcloud is key-create ..." echo "Available IBM Cloud SSH keys:" PS3="Select the SSH key to install on the VSI: " @@ -174,8 +185,10 @@ BNK_FORGE_REGISTRY=__REGISTRY__ BNK_FORGE_VERSION=__VERSION__ POSTGRES_PASSWORD=${PG} REDIS_PASSWORD=${RD} -MCP_USERNAME=admin -MCP_PASSWORD=${MCP} +# #186/#187: MCP authenticates as the dedicated 'mcp' service account with a +# per-install random secret, NOT the human admin (whose password #184 generates + gates). +MCP_SERVICE_USERNAME=mcp +MCP_SERVICE_PASSWORD=${MCP} ENV chmod 600 .env case "${__XTRACE__}" in *x*) set -x ;; esac @@ -343,7 +356,7 @@ done docker compose up -d # 9. Wait for backend health then drop a ready marker -for i in $(seq 1 60); do +for _ in $(seq 1 60); do curl -sf http://localhost:8000/api/system/health >/dev/null 2>&1 && break || sleep 5 done touch /opt/bnk-forge/.bnk-forge-ready @@ -381,6 +394,41 @@ x-backend-env: &backend-env # Artifact (container-image) engine reaches the Docker daemon through the # scoped socket proxy below (loopback-published), never the raw host socket. DOCKER_HOST: ${DOCKER_HOST:-tcp://127.0.0.1:2375} + # #186 BLOCKER 1 / #187 (bonnyr-f5 r5): this installer writes a per-install random + # MCP_SERVICE_PASSWORD into .env (above). The backend reconciles the mcp account + # to it on boot, so it must receive it too — otherwise the mcp account is never + # seeded and the mcp client can never authenticate (no secret is generated for + # MCP under the #188-over-#186 consolidation; bonnyr-f5 #193). + # bonnyr-f5 #193 B1/M3: alias the PASSWORD only, and only for a NON-DEFAULT value — + # a legacy MCP_PASSWORD=changeme is a known-default the backend rejects, so it + # resolves through the alias but leaves MCP disabled. The USERNAME is not aliased (a + # legacy MCP_USERNAME=admin must never resolve the service username — old default admin, + # new default mcp — or a pre-guard backend rewrites the human admin row). + # #193: also plumb the DEFAULT_ADMIN_* gate — config.py has no env_file, so an + # unpassed var never reaches the container. + # bonnyr-f5 #193 B1 (r4): OMIT-when-unset. This installer pins the 3.1.6 image, whose + # `DEFAULT_ADMIN_PASSWORD: str = "changeme"` is a plain str — a present-but-empty "" + # (what `${VAR:-}` delivers) would OVERRIDE it and seed admin with "" (login schema + # rejects "" with 422, locking everyone out). A map entry with NO value is passthrough + # (this heredoc is quoted, so it is written verbatim and resolved by compose at runtime): + # omitted when the var is absent from .env, forwarded when set — so 3.1.6 falls through + # to its usable "changeme" default; a generating backend randomises. + DEFAULT_ADMIN_PASSWORD: + DEFAULT_ADMIN_MUST_CHANGE: ${DEFAULT_ADMIN_MUST_CHANGE:-true} + MCP_SERVICE_USERNAME: ${MCP_SERVICE_USERNAME:-mcp} + MCP_SERVICE_PASSWORD: ${MCP_SERVICE_PASSWORD:-${MCP_PASSWORD:-}} + # bonnyr-f5 #193 B3: plumb ENVIRONMENT so staging/production reaches the fail-fast. + ENVIRONMENT: ${ENVIRONMENT:-development} + # bonnyr-f5 #193 B2: plumb the three vars validate_production also gates on, so + # ENVIRONMENT=production does not brick the backend. bonnyr-f5 #193 B1 (r4): OMIT-when-unset + # (null-value passthrough) — this installer does NOT generate JWT/ENCRYPTION, and the pinned + # 3.1.6 backend uses `if self.KEY is None`, so a present-but-empty "" would boot with an empty + # JWT secret / invalid Fernet key. A map entry with NO value is passthrough: omitted when + # unset, forwarded when set — so 3.1.6 auto-generates and persists the keys to the /app/keys + # volume. Set real values in .env for production. + JWT_SECRET_KEY: + ENCRYPTION_KEY: + ALLOWED_ORIGINS: ${ALLOWED_ORIGINS:-*} x-worker-volumes: &worker-volumes - module_catalog:/tmp/bnk-forge-modules @@ -462,7 +510,7 @@ services: restart: unless-stopped backend: - image: ${BNK_FORGE_REGISTRY:-ghcr.io/your-org}/bnk-forge-api:${BNK_FORGE_VERSION:-latest} + image: ${BNK_FORGE_REGISTRY:-ghcr.io/f5devcentral}/bnk-forge-api:${BNK_FORGE_VERSION:-3.1.6} container_name: bnk-forge-backend network_mode: host logging: *default-logging @@ -495,7 +543,7 @@ services: start_period: 30s celery-worker: - image: ${BNK_FORGE_REGISTRY:-ghcr.io/your-org}/bnk-forge-worker:${BNK_FORGE_VERSION:-latest} + image: ${BNK_FORGE_REGISTRY:-ghcr.io/f5devcentral}/bnk-forge-worker:${BNK_FORGE_VERSION:-3.1.6} container_name: bnk-forge-celery-worker network_mode: host logging: *default-logging @@ -513,7 +561,7 @@ services: restart: unless-stopped celery-worker-2: - image: ${BNK_FORGE_REGISTRY:-ghcr.io/your-org}/bnk-forge-worker:${BNK_FORGE_VERSION:-latest} + image: ${BNK_FORGE_REGISTRY:-ghcr.io/f5devcentral}/bnk-forge-worker:${BNK_FORGE_VERSION:-3.1.6} container_name: bnk-forge-celery-worker-2 network_mode: host logging: *default-logging @@ -531,7 +579,7 @@ services: restart: unless-stopped celery-beat: - image: ${BNK_FORGE_REGISTRY:-ghcr.io/your-org}/bnk-forge-beat:${BNK_FORGE_VERSION:-latest} + image: ${BNK_FORGE_REGISTRY:-ghcr.io/f5devcentral}/bnk-forge-beat:${BNK_FORGE_VERSION:-3.1.6} container_name: bnk-forge-celery-beat network_mode: host logging: *default-logging @@ -547,7 +595,7 @@ services: restart: unless-stopped frontend: - image: ${BNK_FORGE_REGISTRY:-ghcr.io/your-org}/bnk-forge-frontend:${BNK_FORGE_VERSION:-latest} + image: ${BNK_FORGE_REGISTRY:-ghcr.io/f5devcentral}/bnk-forge-frontend:${BNK_FORGE_VERSION:-3.1.6} container_name: bnk-forge-frontend network_mode: host logging: *default-logging @@ -563,7 +611,7 @@ services: start_period: 10s proxy: - image: ${BNK_FORGE_REGISTRY:-ghcr.io/your-org}/bnk-forge-proxy:${BNK_FORGE_VERSION:-latest} + image: ${BNK_FORGE_REGISTRY:-ghcr.io/f5devcentral}/bnk-forge-proxy:${BNK_FORGE_VERSION:-3.1.6} container_name: bnk-forge-proxy network_mode: host logging: *default-logging @@ -575,20 +623,30 @@ services: restart: unless-stopped mcp: - image: ${BNK_FORGE_REGISTRY:-ghcr.io/your-org}/bnk-forge-mcp:${BNK_FORGE_VERSION:-latest} + image: ${BNK_FORGE_REGISTRY:-ghcr.io/f5devcentral}/bnk-forge-mcp:${BNK_FORGE_VERSION:-3.1.6} container_name: bnk-forge-mcp network_mode: host logging: *default-logging environment: BNK_FORGE_API_URL: http://localhost:8000 - BNK_FORGE_USERNAME: ${MCP_USERNAME:-admin} - BNK_FORGE_PASSWORD: ${MCP_PASSWORD:-changeme} + # #186: authenticate as the 'mcp' service account (see .env above), not admin. + # bonnyr-f5 #193 B1: PASSWORD aliased from legacy MCP_PASSWORD; USERNAME is not. + BNK_FORGE_USERNAME: ${MCP_SERVICE_USERNAME:-mcp} + BNK_FORGE_PASSWORD: ${MCP_SERVICE_PASSWORD:-${MCP_PASSWORD:-}} MCP_PORT: "8081" MCP_LOG_LEVEL: INFO depends_on: backend: condition: service_healthy restart: unless-stopped + # bonnyr-f5 #193 M7: auth-probe healthcheck (same as the other compose paths) so + # a "no usable MCP credentials -> 401" condition surfaces as UNHEALTHY here too. + healthcheck: + test: ["CMD", "python", "-m", "bnk_forge_mcp.healthcheck"] + interval: 30s + timeout: 10s + retries: 3 + start_period: 30s volumes: module_catalog: { driver: local } @@ -614,16 +672,21 @@ PYEOF fi # Substitute the scalar placeholders (| delimiter avoids clashes with / in registry). -sed -i \ +# bonnyr-f5 #193 M11: use `sed -i.bak … && rm` — the ONLY in-place form both GNU and +# BSD/macOS sed accept. Bare `sed -i` (GNU) fails on BSD sed ("extra characters"), +# and `sed -i ''` (BSD) fails on GNU; the `.bak` suffix form is portable to both. +sed -i.bak \ -e "s|__VERSION__|${BNK_FORGE_VERSION}|g" \ -e "s|__REGISTRY__|${REGISTRY}|g" \ -e "s|__REGISTRY_HOST__|${REGISTRY_HOST}|g" \ -e "s|__REGISTRY_USER__|${REGISTRY_USER}|g" \ -e "s|__PUBLIC__|${PUBLIC}|g" \ "${UD}" +rm -f "${UD}.bak" # PAT last and on its own (may contain no sed-special chars for GitHub PATs). PAT_ESCAPED="$(printf '%s' "${PAT}" | sed -e 's/[&|\\]/\\&/g')" -sed -i "s|__PAT__|${PAT_ESCAPED}|g" "${UD}" +sed -i.bak "s|__PAT__|${PAT_ESCAPED}|g" "${UD}" +rm -f "${UD}.bak" # ── Create VPC + subnet + security-group rules ─────────────────────────────── log "Creating VPC '${VPC_NAME}'..." @@ -650,7 +713,7 @@ VM_ID="$(echo "${INST_JSON}" | jq -r '.id')" [ -n "${VM_ID}" ] && [ "${VM_ID}" != "null" ] || die "Instance creation failed." log "Waiting for the VSI to reach 'running'..." -for i in $(seq 1 60); do +for _ in $(seq 1 60); do ST="$(ibmcloud is instance "${VM_ID}" --output json | jq -r '.status')" [ "${ST}" = "running" ] && break [ "${ST}" = "failed" ] && die "Instance entered 'failed' state." @@ -684,7 +747,7 @@ log "Floating IP: ${FIP}" URL="https://${FIP}" log "Installing bnk-forge on the VSI (this can take 5–10 minutes)..." READY=0 -for i in $(seq 1 90); do +for _ in $(seq 1 90); do CODE="$(curl -sk -o /dev/null -w '%{http_code}' --connect-timeout 5 "${URL}/api/system/health" 2>/dev/null || true)" if [ "${CODE}" = "200" ]; then READY=1; break; fi sleep 10 @@ -702,7 +765,7 @@ fi echo echo " URL: ${URL}" echo " (self-signed certificate — accept the browser warning)" -echo " Login: admin / changeme (change on first login)" +echo " Login: admin / (see backend logs or /app/keys/initial_admin_password; change on first login)" echo echo " Host IP: ${FIP} (SSH: ssh ubuntu@${FIP})" echo " Region: ${REGION} / ${ZONE} Profile: ${PROFILE} Image: Ubuntu 24.04" diff --git a/scripts/lib/breaking-change-detect.sh b/scripts/lib/breaking-change-detect.sh new file mode 100644 index 0000000..5a214dd --- /dev/null +++ b/scripts/lib/breaking-change-detect.sh @@ -0,0 +1,105 @@ +#!/usr/bin/env bash +# scripts/lib/breaking-change-detect.sh — the ONE BREAKING CHANGE predicate. +# +# INV-15 (bonnyr-f5 #179 r6 F4 / #193 M5): the detector used to live as two +# byte-identical inline copies in compute_version_bump.sh and +# extract-breaking-changes.sh, kept in step "by hand" and diffed by +# scripts/tests/detector-parity.test.sh. That diff is now structural: this file +# is the single source, and both scripts (plus the commit-lint gate, #193 M1) +# source it, so the copies CANNOT drift. detector-parity.test.sh now asserts the +# single-source wiring instead of diffing two copies. +# +# It defines the ONE marker regex, two detector functions and one gate helper and +# NOTHING else (no `set`, no main), so sourcing it never changes the caller's +# shell options: +# +# _is_breaking_subject rc 0 iff the SUBJECT declares a break +# (Conventional-Commits `type!:` bang — BODY +# footers are the _is_breaking_body job). +# _is_breaking_body rc 0 iff the MESSAGE carries a real +# BREAKING CHANGE footer. +# +# A marker counts as a real footer under two anchors: +# * preceded by a BLANK line -> accepted with OR without a colon (keeps the #2 +# no-colon paragraph break); +# * preceded by another TRAILER, or folded directly onto a conventional-commit +# SUBJECT -> accepted ONLY with a colon (catches a footer folded onto a scoped +# subject `fix(core): x`, bonnyr-f5 #179 r6 F1, while rejecting a prose header +# `Before:` / `Note:` followed by colon-less prose, r6 F2). +# Wrapped prose (a marker after a PROSE line) is still rejected. _is_breaking_subject +# is BANG-ONLY: the folded-footer-in-subject is caught by running _is_breaking_body +# on %B (which preserves the newline git folds into %s), and scanning the raw subject +# for the marker over-bumped on `docs: clarify what BREAKING CHANGE: means` (r5 Minor 1). +# +# INV-15 marker regex — SINGLE SOURCE OF RECORD (bonnyr-f5 #193, r3 release/CI). +# The marker shape is hand-copied into the awk of _is_breaking_body and +# _under_detected_markers below and the awk of extract-breaking-changes.sh (a +# literal ERE cannot be passed through `awk -v` — that mangles `\*` on gawk/mawk/BSD +# alike — so the copies stay embedded rather than parametrised). This variable names +# the canonical form ONCE, and scripts/tests/detector-parity.test.sh asserts every +# embedded copy is byte-identical to it, so the sites cannot drift. +# shellcheck disable=SC2034 # read by scripts/tests/detector-parity.test.sh, which sources this lib +_BREAKING_MARKER_ERE='^([*-][[:space:]]+)?(\*\*)?BREAKING[[:space:] -]+CHANGE' +_is_breaking_subject() { + grep -qE '^[A-Za-z]+(\([^)]*\))?!:' <<< "$1" +} +_is_breaking_body() { + awk ' + BEGIN { prev_blank = 1; prev_trailer = 0 } + /^[[:space:]]*$/ { prev_blank = 1; prev_trailer = 0; next } + { + is_marker = ($0 ~ /^([*-][[:space:]]+)?(\*\*)?BREAKING[[:space:] -]+CHANGE/) + is_colon = ($0 ~ /^([*-][[:space:]]+)?(\*\*)?BREAKING[[:space:] -]+CHANGE(\*\*)?:/) + is_trailer = ($0 ~ /^[A-Za-z0-9][A-Za-z0-9-]*:([[:space:]]|$)/) + if (prev_blank && is_marker) found = 1 + else if (prev_trailer && is_colon) found = 1 + is_subject = (NR == 1 && $0 ~ /^[A-Za-z]+(\([^)]*\))?!?:[[:space:]]/) + prev_blank = 0; prev_trailer = (is_trailer || is_subject) + } + END { exit(found ? 0 : 1) } + ' <<< "$1" +} + +# _under_detected_markers — PRINT every line that is a DECLARATIVE +# BREAKING CHANGE marker (the colon form `BREAKING CHANGE:`) sitting where the +# release detectors would MISS it: mis-anchored, i.e. NOT blank-anchored and NOT a +# trailer+colon footer. This is the EXACT COMPLEMENT of _is_breaking_body's footer +# recognition, computed line-by-line with the SAME anchoring state (prev_blank / +# prev_trailer / subject), so the commit-lint gate that consumes it: +# +# * never flags a line the detector already accepts as a real footer — a +# properly-anchored `BREAKING CHANGE:` is `recognised` here and NOT printed, so +# a valid footer is never mis-reported as "you mis-anchored it"; and +# * reports EVERY mis-anchored marker in the body, not just the first, and even +# when another real footer coexists in the same message (bonnyr-f5 #193 r3: +# scan the whole body, report every marker — the old first-footer short-circuit +# hid a second mis-anchored marker). +# +# The COLON is load-bearing (bonnyr-f5 #193 r3 M-6b). Without it the gate also +# fired on COLONLESS marker-shaped prose that merely DESCRIBES the concept — e.g. +# an already-merged, unamendable body containing +# "- BREAKING CHANGE footer in the BODY of a fix-subject commit → major" +# which every detector deliberately treats as inert. That reddened commit-lint (an +# ALWAYS_RUN gate) on the merge that cuts the release, unfixable without a history +# rewrite. A colonless marker that IS blank-anchored is still a real footer, caught +# by _is_breaking_body and thus `recognised` (never printed); requiring the colon +# narrows the gate to ONLY the mis-anchored-colon case the detectors miss, making +# "flagged by gate" the exact complement of "recognised by detector". Indented +# lines (leading whitespace) are NOT markers here — code/example blocks the +# detectors ignore, so the gate must not flag them. Embeds the canonical marker +# literal; detector-parity.test.sh locks every copy to _BREAKING_MARKER_ERE. +_under_detected_markers() { + awk ' + BEGIN { prev_blank = 1; prev_trailer = 0 } + /^[[:space:]]*$/ { prev_blank = 1; prev_trailer = 0; next } + { + is_marker = ($0 ~ /^([*-][[:space:]]+)?(\*\*)?BREAKING[[:space:] -]+CHANGE/) + is_colon = ($0 ~ /^([*-][[:space:]]+)?(\*\*)?BREAKING[[:space:] -]+CHANGE(\*\*)?:/) + is_trailer = ($0 ~ /^[A-Za-z0-9][A-Za-z0-9-]*:([[:space:]]|$)/) + recognised = ((prev_blank && is_marker) || (prev_trailer && is_colon)) + if (is_colon && !recognised) print + is_subject = (NR == 1 && $0 ~ /^[A-Za-z]+(\([^)]*\))?!?:[[:space:]]/) + prev_blank = 0; prev_trailer = (is_trailer || is_subject) + } + ' <<< "$1" +} diff --git a/scripts/lib/is-release-bot-subject.sh b/scripts/lib/is-release-bot-subject.sh new file mode 100644 index 0000000..8ec8e06 --- /dev/null +++ b/scripts/lib/is-release-bot-subject.sh @@ -0,0 +1,28 @@ +#!/usr/bin/env bash +# scripts/lib/is-release-bot-subject.sh — the ONE release-bot own-commit +# fingerprint (bonnyr-f5 #193 r3 release/CI). +# +# The release bot's own automated release commit has a subject EXACTLY of the +# shape `release: vX.Y.Z ... [skip ci]` — a real version AND the trailing skip +# marker it appends (see the two `git commit` calls in .github/workflows/ +# release.yml). Two places must recognise it and MUST NOT drift: +# * the commit-lint gate (scripts/lint-commit-markers.sh), which exempts the +# bot's own release commit from the marker rules; and +# * release.yml's loop guard (the `guard` job), which decides should_run. +# +# release.yml's `guard` job runs WITHOUT a repo checkout (it reads only the event +# payload), so it cannot source this file; it therefore keeps a byte-identical +# INLINE copy of the two grep predicates below, and +# scripts/tests/lint-commit-markers.test.sh asserts that inline copy is +# byte-identical to this one so the two cannot silently diverge. +# +# NOTE ON FORGEABILITY: this reads the commit SUBJECT, which the committing client +# controls — it is NOT an unforgeable property. A human could hand-write +# `release: v0.0.1 ... [skip ci]` to self-exempt. The residual exposure is low: a +# squash-merged PR is linted through PR_TITLE (which gets NO exemption), and a +# skip-marked push to a protected branch produces no workflow run at all, so there +# is no run for the exemption to weaken. Do not describe this as unforgeable. +_is_release_bot_subject() { + grep -qE '^release: v[0-9]+\.[0-9]+\.[0-9]+' <<< "$1" \ + && grep -qE '\[skip ci\]$' <<< "$1" +} diff --git a/scripts/lint-commit-markers.sh b/scripts/lint-commit-markers.sh new file mode 100644 index 0000000..230f99f --- /dev/null +++ b/scripts/lint-commit-markers.sh @@ -0,0 +1,241 @@ +#!/usr/bin/env bash +# +# Enforce the AGENTS.md "Commit conventions" rule -- documentation is not +# enforcement (#166; bonnyr-f5 #182 r3). Shared by the ci.yml `commit-lint` job, +# `make commit-lint`, and .githooks/pre-push, so a local run == CI. +# +# FAILS a commit-message range (and a pending PR title / release-notes text) on: +# +# 1. A CI-control marker anywhere in subject or body. GitHub scans the whole +# message, so one of these sitting even in prose SUPPRESSES the workflow run +# for that commit -- and the gates that get skipped (ShellCheck, Secret Scan, +# Script Self-Tests) are exactly the ones that matter. Bit us on #179/#181. +# The `skip-checks: true` commit-check trailer is caught too (bonnyr-f5 #182 +# r4): it is GitHub's documented way to suppress ALL required checks and is +# not a bracketed token, so the fixed-string list alone would miss it. +# +# 2. A DECLARATIVE `BREAKING CHANGE:` marker (the colon form) that the release +# detectors would MISS (bonnyr-f5 #193 M1, r3 M-6b). Rule 2 is the exact +# COMPLEMENT of the detector: it uses _under_detected_markers from +# scripts/lib/breaking-change-detect.sh, which shares _is_breaking_body's SAME +# anchoring logic and the SAME single-sourced marker regex the bump and the +# note use, rather than a second hand-written regex. A `BREAKING CHANGE:` line +# that is NOT positioned as a real footer (needs a blank line before it, or a +# colon after a preceding trailer) does not trigger the intended major bump, +# so a human who typed a break there ships it silently as a patch. The gate +# flags it BEFORE it becomes an unamendable commit -- including +# `inputs.release_notes`, which release.yml lints through this script before +# interpolating it into the release commit/tag. The COLON is required: a +# colonless marker-shaped line is indistinguishable from PROSE describing the +# concept (an already-merged body reading "- BREAKING CHANGE footer in the +# body ..."), which the detectors treat as inert; flagging it reddened +# unamendable release history (M-6b), so it is not flagged. +# +# The round-1 rule 2 did the OPPOSITE and was wrong in both directions +# (bonnyr-f5 #193 M1): it REJECTED dash-bullet `- BREAKING CHANGE:` and +# markdown-bold `**BREAKING CHANGE:**` footers -- shapes the detectors +# deliberately ACCEPT (positive fixtures in compute/extract self-tests) -- +# and rejected an INDENTED line the detectors IGNORE, with a false "spuriously +# triggers a major" message. Those three shapes are now negative fixtures in +# the self-test below (must NOT be flagged). +# +# EXEMPTION 2 (both rules): the release bot's OWN release commit -- subject EXACTLY +# matching `^release: vX.Y.Z ... [skip ci]` (a version AND the trailing skip marker +# it appends), AND ONLY when it is the range TIP (bonnyr-f5 #193 r4). This is +# release.yml's own loop-guard fingerprint (release.yml:131), single-sourced as +# _is_release_bot_subject in scripts/lib/is-release-bot-subject.sh and shared here +# rather than a second, looser predicate; the guard's inline copy is byte-locked to +# it by scripts/tests/lint-commit-markers.test.sh. This reads the commit SUBJECT, +# which the committing client controls -- it is NOT unforgeable. The honest residual +# (a squash-merged PR is linted through PR_TITLE, which gets NO exemption; a +# skip-marked push to a protected branch produces no workflow run for the exemption +# to weaken) holds only for the HEAD commit -- the bot's release commit is ALWAYS the +# tip being pushed. So the exemption is SCOPED TO THE TIP: a forged +# `release: vX.Y.Z ... [skip ci]` buried MID-RANGE is NOT exempt and is caught (r4). +# +# EXEMPTION 1 (both rules): PUBLISHED, UNAMENDABLE history -- a commit reachable from +# the last final tag ON THIS branch (ancestry-filtered) already shipped. It cannot be +# amended without rewriting released history, and flagging it can only RED this +# ALWAYS_RUN gate on the merge that cuts the NEXT release, unfixably. The r3 +# "already-merged" exemption was removed as DEAD (base..head excludes the base, so no +# iterated commit could be an ancestor of the base) -- but base..head CAN still +# legitimately include a commit that is an ancestor of the last release TAG (a +# re-push, a revert-merge, a mis-computed/over-wide range), so this replacement is +# REACHABLE (bonnyr-f5 #193 r4). It is anchored to the last release tag, not the base. +# Rule 2 also flags ONLY a mis-anchored DECLARATIVE `BREAKING CHANGE:` marker (the +# colon form), so a marker-shaped PROSE line stays inert regardless (that was M-6b). +# +# PR TITLE (bonnyr-f5 #193 B6b): for any PR with >=2 commits GitHub's squash +# SUBJECT is the PR title (squash_title=COMMIT_OR_PR_TITLE), linted NOWHERE else -- +# a `[skip ci]` in the PR title then suppresses the merged commit's workflow run. +# PR_TITLE (set by the ci.yml commit-lint job via env, never interpolated) is +# linted through BOTH rules with NO exemption: it is a PENDING subject. +# +# RELEASE NOTES (bonnyr-f5 #193 M1): LINT_MESSAGE carries arbitrary PENDING text +# (release.yml passes inputs.release_notes) linted through both rules with no +# exemption, before it becomes the release commit body + tag message. +# +# RANGE (env): "base..head" to scan. If UNSET, defaults to @{upstream}..HEAD, else +# just the tip commit. Never scans all history (old release-bot commits legitimately +# carry the deliberate skip marker). An explicitly-set RANGE that does not resolve is +# a HARD failure, and so is an explicitly-set but EMPTY RANGE -- secret-scan.sh reads +# empty as "scan all history", which commit-lint must never do, so it fails closed +# rather than let the same literal RANGE="" mean two different things across the two +# gates (bonnyr-f5 #182 r4 / #193 r4; matches secret-scan.sh's fail-closed ethos). +set -uo pipefail + +# Single-sourced predicates. Resolve from THIS script's directory, not cwd. +# The BREAKING CHANGE detector/gate is shared with the bump/note detectors (INV-15); +# the release-bot fingerprint is shared with release.yml's loop guard. +_LIBDIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib" +# shellcheck source=scripts/lib/breaking-change-detect.sh +. "$_LIBDIR/breaking-change-detect.sh" +# shellcheck source=scripts/lib/is-release-bot-subject.sh +. "$_LIBDIR/is-release-bot-subject.sh" + +# Resolve the commit list without ever falling back to full history, and fail +# closed when an explicit RANGE is unresolvable. +# +# RANGE="" reconciliation (bonnyr-f5 #193 r4 minor): secret-scan.sh reads an +# explicitly-SET-but-EMPTY RANGE as "scan ALL reachable history" (its baseline +# posture). commit-lint must NEVER scan all history — published release-bot commits +# legitimately carry the deliberate skip marker — so rather than silently diverging +# (the old `-n "${RANGE:-}"` test read empty as unset and quietly scanned only the +# tip, so the SAME literal RANGE="" meant two different things in the two adjacent +# ci.yml gates), it FAILS CLOSED on an empty-but-set RANGE, consistent with its own +# rule below that an explicit-but-unresolvable RANGE is a hard error. Leave RANGE +# UNSET for the local default; ci.yml no longer passes RANGE="" here. +if [ -n "${RANGE+set}" ] && [ -z "$RANGE" ]; then + echo "::error::commit-lint: RANGE is set but EMPTY. secret-scan.sh reads an empty RANGE as 'scan all history', which commit-lint must never do. Leave RANGE UNSET for the local default, or pass an explicit base..head range." + exit 1 +elif [ -n "${RANGE:-}" ]; then + if ! commits="$(git rev-list "$RANGE" 2>/dev/null)"; then + echo "::error::commit-lint: RANGE '$RANGE' is not a resolvable revision range -- the scan did not run" + exit 1 + fi +elif upstream="$(git rev-parse --abbrev-ref --symbolic-full-name '@{upstream}' 2>/dev/null)"; then + commits="$(git rev-list "${upstream}..HEAD" 2>/dev/null || true)" + [ -z "$commits" ] && commits="$(git rev-list -1 HEAD)" +else + commits="$(git rev-list -1 HEAD)" +fi + +# The range TIP (newest commit; rev-list prints newest-first) — the commit actually +# being pushed. The release-bot exemption is scoped to it (below). +range_tip="$(printf '%s\n' "$commits" | grep -v '^[[:space:]]*$' | head -1)" + +# Published-history anchor (bonnyr-f5 #193 r4): the highest final vX.Y.Z tag that is +# an ANCESTOR of the range tip — the newest release ON THIS line of history (ancestry- +# filtered, so a tag cut on another branch cannot anchor here). Commits reachable from +# it are PUBLISHED and unamendable; see exemption 1 in the loop. +last_release="" +if [ -n "$range_tip" ]; then + while IFS= read -r _t; do + [ -z "$_t" ] && continue + if git merge-base --is-ancestor "$_t" "$range_tip" 2>/dev/null; then last_release="$_t"; break; fi + done < <(git tag -l 'v*' | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' | sort -V -r) +fi + +# CI-control markers (matched case-insensitively, as fixed strings). +markers=('[skip ci]' '[ci skip]' '[no ci]' '[skip actions]' '[actions skip]') + +fail=0 + +# rule 1 -- CI-control MARKER checks (fixed-string markers + skip-checks trailer). +# $1=label $2=message. Sets `fail=1` on a hit. +_lint_markers() { + local label="$1" msg="$2" m + for m in "${markers[@]}"; do + if grep -iqF -- "$m" <<< "$msg"; then + echo "::error::${label}: message contains CI-control marker \"$m\" -- it would suppress the workflow run. Refer to it indirectly (e.g. \"the skip-CI marker\") or split it across backticks." + fail=1 + fi + done + # GitHub's documented commit-check trailer suppresses ALL required checks; it is + # a key:value trailer, not a bracketed token, so the fixed-string list misses it. + if grep -iqE '^[[:space:]]*skip-checks:[[:space:]]*true\b' <<< "$msg"; then + echo "::error::${label}: message carries the 'skip-checks: true' trailer -- it suppresses all required checks. Remove it or refer to it indirectly." + fail=1 + fi +} + +# rule 2 -- UNDER-DETECTED-MARKER check. $1=label $2=message. Sets `fail=1` on a +# hit. The exact COMPLEMENT of the detectors (bonnyr-f5 #193 M1, r3 M-6b): +# _under_detected_markers (in the shared lib) prints EVERY line that is a +# DECLARATIVE `BREAKING CHANGE:` marker sitting where the detectors would MISS it +# (mis-anchored), and NEVER a line the detector already accepts as a real footer. +# So a properly-anchored footer (blank-anchored, or trailer+colon -- incl. +# dash-bullet/markdown-bold) is passed through untouched, a COLONLESS marker-shaped +# prose line the detectors treat as inert is NOT flagged (that reddened unamendable +# release history), and a SECOND mis-anchored marker later in the same body is +# reported too (no first-footer short-circuit). +_lint_under_detected() { + local label="$1" msg="$2" line + while IFS= read -r line; do + [ -z "$line" ] && continue + echo "::error::${label}: line \"$line\" is shaped like a BREAKING CHANGE marker but is not positioned where the release detectors recognise a footer, so it would NOT trigger the intended major bump. Put it at the START of a paragraph (a blank line before it) as 'BREAKING CHANGE: ', or fold it onto a trailer with a colon -- or reword if no break is intended." + fail=1 + done <<< "$(_under_detected_markers "$msg")" +} + +n=0 +while IFS= read -r sha; do + [ -z "$sha" ] && continue + n=$((n + 1)) + msg="$(git log -1 --format='%B' "$sha")" + subject="$(git log -1 --format='%s' "$sha")" + + # Exemption 1 — PUBLISHED, UNAMENDABLE history (bonnyr-f5 #193 r4, REACHABLE + # replacement for the dead r3 M-6a exemption). A commit reachable from the last + # release tag on this branch already shipped: its markers already had whatever CI + # effect they will ever have, and it cannot be amended without rewriting released + # history. Flagging it can only RED this ALWAYS_RUN gate on the merge that cuts the + # NEXT release, unfixably. The r3 exemption was dead (base..head excludes the base), + # but base..head can still legitimately include a commit that is an ancestor of the + # last release tag (a re-push, a revert-merge, a mis-computed/over-wide range) — so + # this one is reachable. Scoped to NON-tip commits: the tip is the commit under + # active consideration and is always linted (via exemption 2 or the rules). + if [ -n "$last_release" ] && [ "$sha" != "$range_tip" ] \ + && git merge-base --is-ancestor "$sha" "$last_release" 2>/dev/null; then + echo "commit-lint: commit $sha ($subject) is exempt (published history: reachable from $last_release, unamendable)" + continue + fi + + # Exemption 2 — the release bot's OWN minted release commit (version + trailing + # skip marker), ONLY when it is the range TIP (bonnyr-f5 #193 r4). The subject is + # client-controlled and forgeable; the honest residual (a skip-marked push to a + # protected branch produces no workflow run for the exemption to weaken) holds only + # for the HEAD commit — the bot's release commit is ALWAYS the tip being pushed. So + # scoping to the tip means a forged `release: vX.Y.Z … [skip ci]` buried MID-RANGE + # is NOT exempt and is caught, closing the non-head forgeability. + if [ "$sha" = "$range_tip" ] && _is_release_bot_subject "$subject"; then + echo "commit-lint: commit $sha ($subject) is exempt (release-bot commit at the range tip: version + trailing skip marker)" + else + _lint_markers "commit $sha ($subject)" "$msg" + _lint_under_detected "commit $sha ($subject)" "$msg" + fi +done <<< "$commits" + +# bonnyr-f5 #193 B6b: lint the PR TITLE, the one input UNLINTED elsewhere. A +# PENDING subject, not already-merged, so BOTH rules apply with NO exemption. +if [ -n "${PR_TITLE:-}" ]; then + echo "commit-lint: linting PR title -- $PR_TITLE" + _lint_markers "PR title \"$PR_TITLE\"" "$PR_TITLE" + _lint_under_detected "PR title \"$PR_TITLE\"" "$PR_TITLE" +fi + +# bonnyr-f5 #193 M1: lint arbitrary PENDING message text (release.yml passes +# inputs.release_notes here) BEFORE it becomes the release commit body + tag +# message. Both rules, no exemption. +if [ -n "${LINT_MESSAGE:-}" ]; then + echo "commit-lint: linting pending message text (${LINT_MESSAGE_LABEL:-LINT_MESSAGE})" + _lint_markers "${LINT_MESSAGE_LABEL:-message text}" "$LINT_MESSAGE" + _lint_under_detected "${LINT_MESSAGE_LABEL:-message text}" "$LINT_MESSAGE" +fi + +echo "commit-lint: scanned $n commit(s) in range '${RANGE:-}'${PR_TITLE:+ + PR title}${LINT_MESSAGE:+ + message text}" +if [ "$fail" -ne 0 ]; then + echo "::error::commit-lint failed -- see markers above." + exit 1 +fi +echo "commit-lint: OK" diff --git a/scripts/mcp_live_smoke.py b/scripts/mcp_live_smoke.py index a830916..5498b3a 100644 --- a/scripts/mcp_live_smoke.py +++ b/scripts/mcp_live_smoke.py @@ -144,8 +144,12 @@ def _extract_tool_payload(result: dict[str, Any], tool_name: str) -> dict[str, A if "/api/auth/login" in text or "Invalid username or password" in text: hint = ( " Hint: MCP backend credentials are likely invalid. " - "Set correct MCP_USERNAME/MCP_PASSWORD for the MCP container/service " - "(seeded backend default is admin/changeme unless rotated)." + "Set MCP_SERVICE_PASSWORD in the compose .env (bonnyr-f5 #193 M7: compose maps it to " + "the MCP container's BNK_FORGE_PASSWORD and the backend's MCP_SERVICE_PASSWORD — the " + "MCP process itself reads only BNK_FORGE_*, not MCP_SERVICE_*). MCP authenticates as the " + "mcp service account (username fixed to 'mcp', not admin); the backend reconciles the mcp " + "account to that SAME password every boot. When it is unset the mcp account is left " + "disabled (#186)." ) raise SmokeFailure( f"Tool '{tool_name}' execution failed before returning MCP JSON payload: {text}.{hint}" @@ -228,8 +232,10 @@ def _auth_bootstrap_hint(tool_name: str, payload: dict[str, Any]) -> str: if "invalid username or password" in detail or "/api/auth/login" in str(error.get("url", "")): return ( " Hint: MCP endpoint is reachable, but MCP runtime auth/bootstrap failed. " - "Verify MCP_USERNAME/MCP_PASSWORD match current backend credentials " - "(default seeded admin password is changeme, unless rotated), then recreate the mcp container." + "Set MCP_SERVICE_PASSWORD in the compose .env so it matches the mcp service account the backend " + "reconciles (bonnyr-f5 #193 M7: compose delivers it to the container as BNK_FORGE_PASSWORD and to the " + "backend as MCP_SERVICE_PASSWORD — one value, both sides; the username is fixed to 'mcp'). When it is " + "unset the mcp account is left disabled. Then recreate the mcp container (#186)." ) return ( diff --git a/scripts/publish-signed-images.sh b/scripts/publish-signed-images.sh index c3c0a02..5a0de26 100755 --- a/scripts/publish-signed-images.sh +++ b/scripts/publish-signed-images.sh @@ -20,7 +20,7 @@ # BNK_FORGE_REGISTRY=ghcr.io/your-org BNK_FORGE_VERSION=3.1.6 ./scripts/publish-signed-images.sh --execute # # Environment variables: -# BNK_FORGE_REGISTRY — required; e.g. ghcr.io/jlcode-tech +# BNK_FORGE_REGISTRY — required; e.g. ghcr.io/f5devcentral # BNK_FORGE_VERSION — optional; defaults to contents of ./VERSION # DRY_RUN — set to 0 to execute (equivalent to --execute) # @@ -32,8 +32,8 @@ # # Consumer verification (see docs/DOCKER.md for full details): # cosign verify @ \ -# --certificate-identity \ -# --certificate-oidc-issuer https://github.com/login/oauth +# --certificate-identity-regexp 'https://github.com/f5devcentral/bnk-forge/\.github/workflows/release\.yml@.*' \ +# --certificate-oidc-issuer https://token.actions.githubusercontent.com set -euo pipefail @@ -135,14 +135,20 @@ resolve_digest() { # ─── Provenance predicate (minimal SLSA Build L1) ──────────────────────────── # Written to a temp file and attached via cosign attest --type slsaprovenance. +# +# metadata.buildStartedOn is OMITTED, not stamped from `date -u` here: this script +# runs at SIGNING time, AFTER the build (and a sign_only recovery re-signs an +# already-built digest), so a wall-clock time taken here is not the build start and +# would be a knowingly-wrong timestamp in the attestation. The field is optional in +# SLSA v0.2; omitting it is honest, stamping the wrong value is not (bonnyr-f5 #193 +# minor). The build's real time is already carried by the OCI +# org.opencontainers.image.created label (docker-bake.hcl CREATED). write_provenance() { local image_ref="$1" local digest="$2" local out_file="$3" - local build_ts - build_ts="$(date -u +%Y-%m-%dT%H:%M:%SZ)" local git_sha git_sha="$(git -C "$(dirname "$0")/.." rev-parse HEAD 2>/dev/null || echo "unknown")" # Provenance must name the remote this build actually came from. Prefer an @@ -179,7 +185,6 @@ write_provenance() { "version": "${VERSION}" }, "metadata": { - "buildStartedOn": "${build_ts}", "completeness": { "parameters": false, "environment": false, @@ -289,19 +294,26 @@ if [[ "$DRY_RUN" == "1" ]]; then echo " To sign for real:" echo " BNK_FORGE_REGISTRY=${REGISTRY} $0 --execute" else + # Derive the GitHub org from REGISTRY (ghcr.io/) so the printed cosign + # cert-identity matches the namespace that actually published — a fork/mirror that + # publishes to ghcr.io/ must verify against THEIR workflow identity, not a + # hardcoded f5devcentral (bonnyr-f5 #193 r4 minor). The repo name stays bnk-forge + # (the image basenames are bnk-forge-*); only the owner is owner-derived. + REPO_ORG="${REGISTRY#*/}"; REPO_ORG="${REPO_ORG%%/*}" + CERT_IDENTITY="https://github.com/${REPO_ORG}/bnk-forge/\\.github/workflows/release\\.yml@.*" echo " All images signed + SBOM + provenance attached." echo "" echo " Verify a signed image:" echo " cosign verify \\" echo " ${REGISTRY}/bnk-forge-api@ \\" - echo " --certificate-identity \\" - echo " --certificate-oidc-issuer https://github.com/login/oauth" + echo " --certificate-identity-regexp '${CERT_IDENTITY}' \\" + echo " --certificate-oidc-issuer https://token.actions.githubusercontent.com" echo "" echo " Verify the SBOM attestation:" echo " cosign verify-attestation \\" echo " --type cyclonedx \\" - echo " --certificate-identity \\" - echo " --certificate-oidc-issuer https://github.com/login/oauth \\" + echo " --certificate-identity-regexp '${CERT_IDENTITY}' \\" + echo " --certificate-oidc-issuer https://token.actions.githubusercontent.com \\" echo " ${REGISTRY}/bnk-forge-api@" fi echo "========================================================" diff --git a/scripts/registry-overwrite-guard.sh b/scripts/registry-overwrite-guard.sh new file mode 100644 index 0000000..99088db --- /dev/null +++ b/scripts/registry-overwrite-guard.sh @@ -0,0 +1,109 @@ +#!/usr/bin/env bash +# registry-overwrite-guard.sh — overwrite-guard POLICY around registry-tag-probe.sh. +# +# Classifies every release image via registry-tag-probe.sh and REFUSES (exit 1) if +# any immutable :VERSION manifest already exists, OR the probe is inconclusive, OR +# the probe cannot run — unless FORCE=true. Single-sourced so the release.yml +# PRE-PUSH gate (INV-31) shares ONE policy instead of open-coding it per site +# (bonnyr-f5 #193 M3/M4). It exists so nothing irreversible (the `git push origin +# main` + tag push) happens before this fail-closed check. +# +# TWO independence properties the inline guard historically got wrong: +# • the vacuity floor (how many images to expect) comes from docker-bake.hcl's +# "default" group, NOT from the probe we are guarding — so an unavailable probe +# cannot make "0 expected == 0 classified" look like "safe to publish" (M3); +# • the probe's EXIT STATUS is asserted before its output is trusted — a probe +# that cannot run fails CLOSED (M3). +# +# The floor is derived from the TOOL (`docker buildx bake --print default` + jq), +# scoped to the DEFAULT group, NOT by counting `targets = [` lines in the file. The +# old unscoped `sed` counted EVERY `targets = [` line, so adding any SECOND bake +# group (an ordinary edit) inflated the floor and made this gate refuse forever, +# blaming the registry for a bake-file change (bonnyr-f5 #193 M2). A parse/tooling +# failure is now reported as exactly that, distinct from "registry unreachable". +# +# Env in: +# REGISTRY, VERSION (required; forwarded to registry-tag-probe.sh + bake) +# FORCE ("true" overrides a refusal, deliberately) +# REGISTRY_USERNAME/PASSWORD (optional; forwarded for the Bearer challenge) +# PROBE (path to registry-tag-probe.sh; default: sibling) +# BAKE_FILE (path to docker-bake.hcl; default: ./docker-bake.hcl) +# Exit: 0 safe-to-publish (or FORCE overrode a refusal); 1 refuse. +set -uo pipefail + +HERE="$(cd "$(dirname "$0")" && pwd)" +PROBE="${PROBE:-$HERE/registry-tag-probe.sh}" +BAKE_FILE="${BAKE_FILE:-docker-bake.hcl}" +FORCE="${FORCE:-}" +: "${REGISTRY:?REGISTRY is required (e.g. ghcr.io/your-org)}" +: "${VERSION:?VERSION is required (e.g. 3.1.6)}" + +# ── Independent vacuity floor (M2/M3): count docker-bake.hcl's DEFAULT group via +# the tool, scoped to `.group.default.targets`, so a second bake group cannot wedge +# the gate. This is a BAKE-FILE/tooling concern, kept strictly separate from the +# registry probe below and from its "registry unreachable" remediation text. +if ! command -v docker >/dev/null 2>&1 || ! command -v jq >/dev/null 2>&1; then + echo "::error::registry-overwrite-guard: 'docker' and 'jq' are required to count the bake default group (a tooling problem, NOT a registry problem). Install them and re-run." >&2 + exit 1 +fi +IMAGES_N="$(REGISTRY="$REGISTRY" VERSION="$VERSION" \ + docker buildx bake --file "$BAKE_FILE" --print default 2>/dev/null \ + | jq -r '.group.default.targets | length' 2>/dev/null || true)" +if ! printf '%s' "${IMAGES_N:-}" | grep -qE '^[0-9]+$' || [ "${IMAGES_N:-0}" -lt 1 ]; then + echo "::error::registry-overwrite-guard: could not count the DEFAULT bake group in $BAKE_FILE via 'docker buildx bake --print default' + jq (a bake-file PARSE / tooling problem — NOT a registry problem). Fix $BAKE_FILE / the buildx+jq tooling; do not set FORCE for this." >&2 + exit 1 +fi + +# ── Assert the probe's EXIT STATUS before trusting its output (M3) ────────────── +if ! PROBE_OUT="$(REGISTRY="$REGISTRY" VERSION="$VERSION" bash "$PROBE")"; then + if [ "$FORCE" = "true" ]; then + echo "::warning::force=true — publishing despite the registry probe being unrunnable; any already-published :${VERSION} tags would be overwritten and their cosign/SBOM/SLSA attestations orphaned." >&2 + exit 0 + fi + echo "::error::the registry existence probe ($PROBE) could not run. Refusing to publish — an already-published immutable :${VERSION} tag cannot be ruled out. Re-run once the registry/tooling is reachable, or set FORCE=true to overwrite deliberately." >&2 + exit 1 +fi + +PROBE_N="$(printf '%s\n' "$PROBE_OUT" | grep -c . || true)" +EXISTING="" +UNKNOWN="" +if [ "$PROBE_N" -ne "$IMAGES_N" ]; then + UNKNOWN=" the registry existence probe classified ${PROBE_N} of ${IMAGES_N} images; treating as inconclusive"$'\n' +else + while IFS=$'\t' read -r status ref detail; do + case "$status" in + exists) EXISTING="${EXISTING} ${ref}"$'\n' ;; + absent) : ;; # 404 — the immutable tag is free + unknown) UNKNOWN="${UNKNOWN} ${ref}: ${detail}"$'\n' ;; + # FAIL CLOSED on any unrecognised/empty status (bonnyr-f5 #193 r3 minor). The + # shipped probe only ever emits exists/absent/unknown, but a malformed or + # empty status field is the MOST inconclusive state there is — without this + # arm it fell through to neither bucket and the guard printed "safe to + # publish", overwriting an immutable tag on garbage. Treat it as inconclusive. + *) UNKNOWN="${UNKNOWN} ${ref:-}: unrecognised probe status '${status:-}' — treating as inconclusive"$'\n' ;; + esac + done <<< "$PROBE_OUT" +fi + +if [ -n "$UNKNOWN" ]; then + printf 'Registry existence probe was inconclusive (a non-not-found error) for:\n%s' "$UNKNOWN" >&2 + if [ "$FORCE" = "true" ]; then + echo "::warning::force=true — publishing despite an inconclusive existence probe; if any of these tags were in fact already published this overwrites the immutable :${VERSION} tag and orphans its attestations." >&2 + else + echo "::error::Could not confirm whether the :${VERSION} tag is free (auth / network / rate-limit / bad ref). Refusing to publish because an existing immutable tag cannot be ruled out. Re-run once the registry is reachable, or set FORCE=true only if you intend to overwrite." >&2 + exit 1 + fi +fi + +if [ -z "$EXISTING" ]; then + echo "No existing :${VERSION} manifests found — safe to publish." + exit 0 +fi + +printf 'Images already published for this tag:\n%s' "$EXISTING" >&2 +if [ "$FORCE" = "true" ]; then + echo "::warning::force=true — overwriting the existing :${VERSION} manifests; the cosign/SBOM/SLSA attestations bound to the previous digests are now orphaned." >&2 + exit 0 +fi +echo "::error::Images for v${VERSION} already exist in ${REGISTRY}. Republishing would move the immutable :${VERSION} tag and orphan its attestations. Set FORCE=true only if you intend to overwrite them." >&2 +exit 1 diff --git a/scripts/registry-tag-probe.sh b/scripts/registry-tag-probe.sh new file mode 100644 index 0000000..b223a2c --- /dev/null +++ b/scripts/registry-tag-probe.sh @@ -0,0 +1,147 @@ +#!/usr/bin/env bash +# registry-tag-probe.sh — Authoritative "does :VERSION already exist?" probe for +# the BNK Forge release image set, used to protect the IMMUTABLE :VERSION tag. +# +# WHY THIS EXISTS (bonnyr-f5 #181 round 5, F1): +# The earlier probe shelled out to `docker manifest inspect` and classified by +# grepping the CLI's combined stdout/stderr. That text CANNOT separate the two +# things a release must tell apart: +# • a package/repo that does not exist yet (the first release in a fork or +# mirror namespace, or a future 8th image) — SAFE to publish, and +# • no permission to read an existing package — MUST fail closed. +# Both surface identically as Get "https:///token…": denied . Reading +# `denied` as "not found" fails OPEN (overwrites an immutable tag); reading it +# as "unknown" fails CLOSED and hard-fails the very first publish in a +# namespace — after the tag and GitHub Release are already pushed. +# +# The registry HTTP API answers this unambiguously with a STATUS CODE: +# 200 → the manifest exists -> exists +# 404 → definitively not found (MANIFEST_UNKNOWN / NAME_UNKNOWN, i.e. the +# tag is absent OR the repo does not exist yet) -> absent (safe) +# 401 / 403 → authentication / permission -> unknown (fail closed) +# 000 / 429 / 5xx / anything else → transient/network -> unknown (fail closed) +# Only a definitive 404 is treated as "safe to publish"; every other outcome +# is "unknown" and the CALLER refuses unless a force flag is set. +# +# CONTRACT +# Env in: +# REGISTRY (required) e.g. ghcr.io/f5devcentral (host + namespace path) +# VERSION (required) e.g. 3.1.6 (the immutable tag) +# REGISTRY_USERNAME / REGISTRY_PASSWORD (optional) — Basic creds used when +# the registry issues a Bearer challenge. Falls back to an anonymous +# token request (sufficient for public images); a private/absent repo +# probed anonymously returns 401/403 → unknown → the caller fails closed. +# Stdout: one TAB-separated line per image: +# \t\t status ∈ exists | absent | unknown +# Exit: 0 once every image is classified (regardless of verdict); non-zero +# only on a usage error. Policy (refuse / force / message) lives in the +# caller so the CI path and the operator `make push-images` path can +# share ONE probe but keep their own force semantics. +# +# registry-tag-probe.sh --images prints the canonical image list, one per +# line, so callers single-source it instead of re-hardcoding 7 names +# (bonnyr-f5 #181 round 5, F6). +set -euo pipefail + +# ─── Canonical image set (single source of truth — F6) ─────────────────────── +# MUST stay in lockstep with the "default" group in docker-bake.hcl and the +# IMAGES array in scripts/publish-signed-images.sh. The self-test +# (scripts/tests/registry-tag-probe.test.sh) asserts the count and the +# docker-bake.hcl parity. +IMAGES=( + "bnk-forge-api" + "bnk-forge-worker" + "bnk-forge-beat" + "bnk-forge-frontend" + "bnk-forge-proxy" + "bnk-forge-mcp" + "bnk-forge-operator" +) + +if [ "${1:-}" = "--images" ]; then + printf '%s\n' "${IMAGES[@]}" + exit 0 +fi + +: "${REGISTRY:?REGISTRY is required (e.g. ghcr.io/your-org)}" +: "${VERSION:?VERSION is required (e.g. 3.1.6)}" + +HOST="${REGISTRY%%/*}" # ghcr.io +NAMESPACE="${REGISTRY#*/}" # f5devcentral (may be multi-segment) +if [ "$HOST" = "$REGISTRY" ] || [ -z "$NAMESPACE" ]; then + echo "ERROR: REGISTRY must be / (got '$REGISTRY')" >&2 + exit 2 +fi + +ACCEPT='application/vnd.oci.image.index.v1+json,application/vnd.docker.distribution.manifest.list.v2+json,application/vnd.oci.image.manifest.v1+json,application/vnd.docker.distribution.manifest.v2+json' + +# Fetch a Bearer token for a pull-scoped challenge. Echoes the token or nothing. +_fetch_token() { + local realm="$1" service="$2" scope="$3" + local url="$realm" + local sep='?' + [ -n "$service" ] && { url="${url}${sep}service=${service}"; sep='&'; } + [ -n "$scope" ] && { url="${url}${sep}scope=${scope}"; sep='&'; } + local body + if [ -n "${REGISTRY_USERNAME:-}" ] && [ -n "${REGISTRY_PASSWORD:-}" ]; then + body="$(curl -sS --max-time 20 -u "${REGISTRY_USERNAME}:${REGISTRY_PASSWORD}" "$url" 2>/dev/null || true)" + else + body="$(curl -sS --max-time 20 "$url" 2>/dev/null || true)" + fi + # GHCR/Docker return {"token":...}; some registries use {"access_token":...}. + # Use `sed -nE` (ERE): the `\|` BRE alternation is a GNU-only extension, so on + # BSD/macOS sed the old expression matched nothing, _fetch_token returned empty, + # every image classified `unknown`, and the operator was routed into FORCE_LATEST=1 + # (which SKIPS this probe) — the exact INV-24 harm (bonnyr-f5 #193 B4). ERE + # `(access_token|token)` is portable across GNU and BSD sed. + printf '%s' "$body" | sed -nE 's/.*"(access_token|token)"[[:space:]]*:[[:space:]]*"([^"]*)".*/\2/p' | head -1 +} + +# HTTP status for GET , following one Bearer challenge if issued. +# Echoes a 3-digit code, or 000 on a curl/network failure. +_manifest_status() { + local url="$1" + local hdr code + hdr="$(mktemp)" + code="$(curl -sS --max-time 25 -o /dev/null -D "$hdr" -w '%{http_code}' \ + -H "Accept: ${ACCEPT}" "$url" 2>/dev/null || echo 000)" + if [ "$code" = "401" ]; then + local challenge realm service scope + challenge="$(grep -i '^www-authenticate:' "$hdr" | head -1 | tr -d '\r')" + realm="$(printf '%s' "$challenge" | sed -n 's/.*realm="\([^"]*\)".*/\1/p')" + service="$(printf '%s' "$challenge" | sed -n 's/.*service="\([^"]*\)".*/\1/p')" + scope="$(printf '%s' "$challenge" | sed -n 's/.*scope="\([^"]*\)".*/\1/p')" + if [ -n "$realm" ]; then + local token + token="$(_fetch_token "$realm" "$service" "$scope")" + if [ -n "$token" ]; then + code="$(curl -sS --max-time 25 -o /dev/null -w '%{http_code}' \ + -H "Accept: ${ACCEPT}" -H "Authorization: Bearer ${token}" \ + "$url" 2>/dev/null || echo 000)" + fi + # token empty => the token endpoint denied us => leave code=401 (unknown). + fi + fi + rm -f "$hdr" + printf '%s' "$code" +} + +for name in "${IMAGES[@]}"; do + ref="${REGISTRY}/${name}:${VERSION}" + url="https://${HOST}/v2/${NAMESPACE}/${name}/manifests/${VERSION}" + code="$(_manifest_status "$url")" + case "$code" in + 200|203) printf '%s\t%s\t%s\n' exists "$ref" "HTTP ${code}" ;; + 404) printf '%s\t%s\t%s\n' absent "$ref" "HTTP 404 (not found)" ;; + 401|403) printf '%s\t%s\t%s\n' unknown "$ref" "HTTP ${code} (auth/permission — cannot confirm)" ;; + # Network/curl failure. curl writes its own `%{http_code}` of "000" to stdout + # AND exits non-zero, so `... || echo 000` appends a SECOND "000" -> the real + # shape is "000000" (or a bare "000" if curl emitted nothing). The old `000)` + # arm matched neither doubled shape and fell through to `*)` — dead code + # (bonnyr-f5 #193 r3 minor). Match the `^000` prefix so it actually fires. + 000*) printf '%s\t%s\t%s\n' unknown "$ref" "network/curl failure (code '${code}') — cannot confirm" ;; + 429) printf '%s\t%s\t%s\n' unknown "$ref" "HTTP 429 (rate limited — cannot confirm)" ;; + 5??) printf '%s\t%s\t%s\n' unknown "$ref" "HTTP ${code} (registry error — cannot confirm)" ;; + *) printf '%s\t%s\t%s\n' unknown "$ref" "HTTP ${code} (unexpected — cannot confirm)" ;; + esac +done diff --git a/scripts/secret-scan.sh b/scripts/secret-scan.sh new file mode 100644 index 0000000..5e036c8 --- /dev/null +++ b/scripts/secret-scan.sh @@ -0,0 +1,104 @@ +#!/usr/bin/env bash +# +# Single source of truth for the gitleaks secret scan + its assertion backstop. +# +# Called by BOTH .github/workflows/ci.yml (the secret-scan job and the scheduled +# baseline job) AND `make secret-scan` / `make pre-push`, so a local run is +# byte-identical to CI -- #166: "a local gate that does not run the CI command is +# not a gate", and ci.yml's header claims `make pre-push` == CI. +# +# The gate must be able to tell "clean scan" from "did not run". gitleaks exits 0 +# on a bad revision range OR a git dubious-ownership refusal, printing +# "ERR [git] ..." + "0 commits scanned" -- a silently blind green gate +# (bonnyr-f5 #182 r3 BLOCKER). So we capture the output and FAIL on: any +# "ERR [git]" line, a missing "commits scanned" line, 0 commits for a non-empty +# range, or a non-zero gitleaks exit (leaks found). +# +# RANGE selection: +# * If the RANGE env var is SET (even to empty), it is used verbatim -- empty +# means "scan all reachable history" (the scheduled baseline + first push). +# CI computes it from the triggering event. +# * If RANGE is UNSET, a local default is computed: everything since HEAD +# diverged from its upstream tracking branch, falling back to full history. +set -uo pipefail + +# gitleaks v8.30.1, pinned by digest so a re-tag cannot change what runs +# (bonnyr-f5 #182 r3 nit). Update the version comment when bumping the digest. +IMAGE="ghcr.io/gitleaks/gitleaks@sha256:c00b6bd0aeb3071cbcb79009cb16a60dd9e0a7c60e2be9ab65d25e6bc8abbb7f" # v8.30.1 + +repo_dir="$(git rev-parse --show-toplevel 2>/dev/null || pwd)" + +# Resolve the range (see header). ${RANGE+set} distinguishes unset from empty. +if [ -n "${RANGE+set}" ]; then + range="$RANGE" +else + range="" + if upstream="$(git rev-parse --abbrev-ref --symbolic-full-name '@{upstream}' 2>/dev/null)"; then + if base="$(git merge-base "$upstream" HEAD 2>/dev/null)"; then + range="${base}..HEAD" + fi + fi +fi + +echo "gitleaks scanning range: ${range:-} (repo: $repo_dir)" + +# safe.directory whitelists the mount so git 2.35.2+ does not refuse it for +# dubious ownership (the image runs as root; the checkout is owned by another +# uid). GIT_CONFIG_* needs no writable HOME, unlike `git config --global`. +# --max-archive-depth 2: without it gitleaks defaults to 0 and NEVER looks inside +# tracked archives, so a secret shipped in a tarball is invisible (bonnyr-f5 #182 +# r3 Major). Depth 2 covers e.g. a key inside a .tar.gz inside a .zip. +out="$(docker run --rm \ + -e GIT_CONFIG_COUNT=1 -e GIT_CONFIG_KEY_0=safe.directory -e GIT_CONFIG_VALUE_0=/repo \ + -v "$repo_dir:/repo:ro" -w /repo "$IMAGE" detect \ + --source=/repo --config=/repo/.gitleaks.toml --redact --verbose \ + --max-archive-depth 2 \ + ${range:+--log-opts="$range"} 2>&1)" +rc=$? +printf '%s\n' "$out" + +# Strip ANSI so parsing is robust whether or not gitleaks colourises. +clean="$(printf '%s\n' "$out" | sed -E 's/\x1b\[[0-9;]*m//g')" + +# 1) Any git error (bad range, dubious ownership) means the scan never saw the +# repo/range -- fail even though gitleaks exited 0. +if grep -qE 'ERR \[git\]' <<< "$clean"; then + echo "::error::gitleaks hit a git error (bad revision range or dubious ownership) -- the scan did not run" + exit 1 +fi + +# 2) Positive evidence the scan ran: gitleaks always prints " commits scanned" +# in git mode. No such line == silence == must not pass. +scanned="$(grep -oE '[0-9]+ commits scanned' <<< "$clean" | grep -oE '^[0-9]+' | tail -n1)" +echo "gitleaks reported commits scanned: ${scanned:-}" +if [ -z "$scanned" ]; then + echo "::error::gitleaks printed no 'commits scanned' line -- no evidence the scan ran" + exit 1 +fi + +# 3) Anti-vacuity backstop. gitleaks reports "0 commits scanned" in TWO cases: a +# BLIND scan (broken range / dubious-ownership refusal — already caught by check +# 1's "ERR [git]"), AND, legitimately, a range that adds no content to scan. +# `git log -p` shows only removed lines for a delete-only (or empty) push; +# gitleaks scans ADDED content, so it counts 0 and exits 0 for a real, pushable +# change (reproduced: a delete-only commit has rev-list count 1 yet "0 commits +# scanned"). The old count==0 backstop RED that legitimate push and blocked the +# hook (bonnyr-f5 #193 r3 M-7). So the scanned COUNT is not a reliable blind +# signal. Instead assert the scan ran against REAL history by gitleaks' exit +# status (checks 1/2/4) PLUS git's own ability to resolve the range: a range git +# cannot enumerate is a broken scan (fail closed); a resolvable range that +# gitleaks scanned cleanly is trustworthy regardless of the commit count. +if [ -n "$range" ]; then + if ! git -C "$repo_dir" rev-list --count "$range" >/dev/null 2>&1; then + echo "::error::gitleaks range '$range' does not resolve in git -- no evidence the scan ran against real history" + exit 1 + fi +fi + +# 4) Real leaks make gitleaks exit non-zero -- that must still fail here. +if [ "$rc" -ne 0 ]; then + echo "::error::gitleaks exited $rc (leaks found or scan error)" + exit "$rc" +fi + +echo "secret-scan: OK" diff --git a/scripts/sync-version-artifacts.sh b/scripts/sync-version-artifacts.sh new file mode 100644 index 0000000..af39832 --- /dev/null +++ b/scripts/sync-version-artifacts.sh @@ -0,0 +1,262 @@ +#!/usr/bin/env bash +# Keep EVERY version-bearing DEPLOYMENT artifact in lockstep with VERSION: +# - the bnk-forge Helm chart image tag (values.yaml) and Chart `appVersion` +# - the frontend package.json version +# - the sibling bnk-operator chart image tag (values.yaml) and `appVersion` +# - the packaged dist/ compose image pins (dist/docker-compose.yml — the 7 +# `${BNK_FORGE_VERSION:-}` defaults) and dist/.env.example's +# BNK_FORGE_VERSION default +# - the packaged dist/VERSION stamp (bonnyr-f5 #193 r4 deploy minor): a plain +# one-line version file `make dist` used to `cp` and release.yml used to +# `echo > dist/VERSION` by hand — i.e. maintained OUTSIDE this single-source +# writer, so it could drift from VERSION between releases. It is now synced and +# --check'd here like every other pin. +# - the IBM Cloud installer's BNK_FORGE_VERSION default AND the pins in the +# compose template it embeds (scripts/ibm_cloud_bnk_forge.sh) +# +# ALL of these resolve to an image published at :${VERSION} on the release train +# — docker-bake.hcl's `default` group builds the operator image alongside the +# rest — so any drift means an image tag the release never publishes -> +# ImagePullBackOff / `manifest unknown`. This is the ONE place that writes them, +# and --check verifies them in CI so drift can't reappear silently. +# +# WHY dist/ is now IN scope (bonnyr-f5 #193 r3, B1): the dist/ compose + env + +# the IBM installer previously hard-pinned a FORWARD-DATED version (`4.0.0`) that +# nothing had published, so a fresh install rendered `manifest unknown` while +# --check stayed green because it could not see dist/. The pins are now DERIVED +# from VERSION here: at release, `--write $NEW` re-stamps every one atomically and +# release.yml stages exactly `--list`, so the packaged pin can never name a tag +# the release did not cut. Never re-hardcode a version into these files — set it +# here and let --write propagate it. +# +# STILL out of scope, deliberately: +# - frontend-v2/package-lock.json's root `version` (npm owns it; `npm ci` +# tolerates the desync). +# - dist/docker-compose.local.yml — a pure networking OVERLAY that carries NO +# image: line of its own (it inherits every pin from dist/docker-compose.yml), +# so there is nothing here to stamp; adding it would only make --check vacuous. +# - each Chart.yaml's own `version:` — Helm treats chart version and appVersion +# as independent, and release.yml neither packages nor pushes the chart, so a +# static chart version publishes nothing wrong. Leave it alone. +# +# Usage: +# sync-version-artifacts.sh --write # set all artifacts to +# sync-version-artifacts.sh --check # verify all == VERSION; exit 1 if not +# sync-version-artifacts.sh --list # print artifact paths (repo-relative) +set -euo pipefail + +ROOT="$(cd "$(dirname "$0")/.." && pwd)" +VALUES="$ROOT/helm/bnk-forge/values.yaml" +CHART="$ROOT/helm/bnk-forge/Chart.yaml" +PKG="$ROOT/frontend-v2/package.json" +# The sibling operator chart is on the VERSION train (its image publishes at +# :${VERSION}), so it is synced here too rather than pinned. +OPVALUES="$ROOT/bnk-operator/charts/bnk-operator/values.yaml" +OPCHART="$ROOT/bnk-operator/charts/bnk-operator/Chart.yaml" +# Packaged dist/ install path and the IBM Cloud installer (bonnyr-f5 #193 r3, B1). +DISTENV="$ROOT/dist/.env.example" +DISTCOMPOSE="$ROOT/dist/docker-compose.yml" +IBMCLOUD="$ROOT/scripts/ibm_cloud_bnk_forge.sh" +# The packaged one-line version stamp (bonnyr-f5 #193 r4 deploy minor). +DISTVERSION="$ROOT/dist/VERSION" + +# Canonical artifact list. --write, --list, and the release job's `git add` all +# derive the file set from HERE, so the writer and its stager cannot diverge and +# leave a synced-but-unstaged file behind (bonnyr-f5 #180 r3, BLOCKER 1). +SYNCED_FILES=("$VALUES" "$CHART" "$PKG" "$OPVALUES" "$OPCHART" \ + "$DISTENV" "$DISTCOMPOSE" "$IBMCLOUD" "$DISTVERSION") + +# ── Value readers ───────────────────────────────────────────────────────────── +# Each reads EVERY matching version line (not grep -m1), so a second occurrence +# can't drift unseen behind a global write (bonnyr-f5 #180 r3). `^ tag: ` (two +# spaces) matches only the top-level image tag — the postgres/redis/per-service +# tags are 4-space and never match. +TAG_RE='^ tag: ' +TAG_SED='s/^ tag: "?([^"]*)"?.*/\1/' +APPVER_RE='^appVersion:' +APPVER_SED='s/^appVersion: "?([^"]*)"?.*/\1/' +PKGVER_RE='^ "version":' +PKGVER_SED='s/^ "version": "([^"]*)".*/\1/' +# dist/.env.example: a bare shell assignment `BNK_FORGE_VERSION=` (the compose +# files read this at runtime as ${BNK_FORGE_VERSION:-}). Only the +# top-level (column-0) assignment matches — the commented example above it is +# `# BNK_FORGE_VERSION=...` and is skipped. +DISTENV_RE='^BNK_FORGE_VERSION=' +DISTENV_SED='s/^BNK_FORGE_VERSION=(.*)/\1/' +# Compose image pins AND the IBM installer default, both written as the shell +# default-expansion `${BNK_FORGE_VERSION:-}`. The reader pulls the out of +# EVERY such site (7 pins in dist/docker-compose.yml; 1 default + 7 embedded pins +# in the IBM installer), so a second occurrence can't drift unseen. `$`, `{`, `}` +# are escaped so grep -E / sed -E read them literally, not as anchor/interval. +PIN_RE='\$\{BNK_FORGE_VERSION:-' +PIN_SED='s/.*\$\{BNK_FORGE_VERSION:-([^}]*)\}.*/\1/' +# dist/VERSION: a plain one-line stamp whose entire content is the version token. +# Match that bare token line (semver incl. prerelease/build metadata) and extract it. +DISTVER_RE='^[A-Za-z0-9._+-]+$' +DISTVER_SED='s/^([A-Za-z0-9._+-]+).*/\1/' + +# The image tag lives inside the top-level `image:` block. The WRITER scopes its +# substitution to that block (sed range below); the READERS (--check and --write's +# post-write verify) MUST use the SAME range, or writer and checker diverge: +# a `tag:` the writer can't reach or a stray 2-space `tag:` under another key would +# be read by a file-global checker but never written — CI green while the next +# release hard-fails, or CI red on a line --write can't fix (bonnyr-f5 #180 r5, F1). +# One expression, used by both. +# +# The range END is a column-0 line that is NOT a comment (`^[^[:space:] #]`). A +# YAML `# comment` at column 0 is legal ANYWHERE, including inside the image block, +# and is NOT the next top-level key — the old `^[^[:space:]]` ended the range on it, +# so a column-0 comment placed after `^image:` hid the `tag:` from BOTH the writer +# and the checker, making --check report "key renamed/removed? — vacuous" and print +# a --write remediation that also could not reach the tag (bonnyr-f5 #193 minor). +IMG_RANGE='/^image:/,/^[^[:space:] #]/' + +# Emit the candidate version lines for a reader. When RANGE is given, the grep is +# scoped to that sed address range (the image-tag case) so the reader sees EXACTLY +# the site set the writer's ranged sed touches; otherwise it is file-global. +_version_lines() { # file, grep-ERE, range(optional) + local file="$1" gre="$2" range="${3:-}" + if [ -n "$range" ]; then + sed -nE "${range}{/${gre}/p;}" "$file" + else + grep -E "$gre" "$file" || true + fi +} + +case "${1:-}" in + --write) + V="${2:?usage: sync-version-artifacts.sh --write }" + # V is interpolated into sed replacement strings, so a `|`/`&`/`\`/`"` would + # corrupt the substitution. It only fails-closed at the post-write verify + # today (bonnyr-f5 #180 r5 nit) — reject metacharacters up front with a clear + # message. The class is permissive enough for full semver incl. prerelease and + # build metadata (e.g. 1.2.3-rc.1+build.5). + if ! printf '%s' "$V" | grep -qE '^[A-Za-z0-9._+-]+$'; then + echo "::error::--write version '$V' contains characters outside [A-Za-z0-9._+-] — refusing (would corrupt the sed substitution)" >&2 + exit 2 + fi + # Anchor every substitution to its key PATH, not a bare 2-space `tag:`. The + # image-tag writes are scoped to the top-level `image:` block via a sed range + # (`/^image:/` to the next column-0 key) so a future unrelated 2-space `tag:` + # elsewhere is never repinned to VERSION (bonnyr-f5 #180 r3, unbounded writer). + # -i.syncbak (attached suffix) is the one in-place form both GNU and BSD sed + # accept; `-i -E` makes BSD swallow -E as the suffix and litter *-E files. + sed -i.syncbak -E "${IMG_RANGE} s|^ tag: .*| tag: \"${V}\"|" "$VALUES" + sed -i.syncbak -E "s|^appVersion: .*|appVersion: \"${V}\"|" "$CHART" + sed -i.syncbak -E "s|^ \"version\": \"[^\"]*\"| \"version\": \"${V}\"|" "$PKG" + sed -i.syncbak -E "${IMG_RANGE} s|^ tag: .*| tag: \"${V}\"|" "$OPVALUES" + sed -i.syncbak -E "s|^appVersion: .*|appVersion: \"${V}\"|" "$OPCHART" + # dist/.env.example: rewrite the bare top-level assignment only. + sed -i.syncbak -E 's|^BNK_FORGE_VERSION=.*|BNK_FORGE_VERSION='"${V}"'|' "$DISTENV" + # Every `${BNK_FORGE_VERSION:-}` default -> `${BNK_FORGE_VERSION:-}`. + # The LHS pattern is single-quoted (literal to the shell) and the replacement + # concatenates single-quoted literals around the interpolated ${V}; V is + # validated above to [A-Za-z0-9._+-], so it carries no sed-replacement + # metacharacter (`&`/`\`/delimiter). Applies to the dist compose pins and to + # BOTH the IBM installer default (line ~32) and its embedded compose pins. + sed -i.syncbak -E 's|\$\{BNK_FORGE_VERSION:-[^}]*\}|${BNK_FORGE_VERSION:-'"${V}"'}|g' "$DISTCOMPOSE" + sed -i.syncbak -E 's|\$\{BNK_FORGE_VERSION:-[^}]*\}|${BNK_FORGE_VERSION:-'"${V}"'}|g' "$IBMCLOUD" + # dist/VERSION is a plain one-line stamp — rewrite the whole file (no sed range). + printf '%s\n' "${V}" > "$DISTVERSION" + for f in "${SYNCED_FILES[@]}"; do rm -f "${f}.syncbak"; done + + # Fail closed: a sed whose pattern matched nothing no-ops silently, and the + # caller would commit the unchanged file believing it synced (#177 review). + # Re-read every version line in every artifact with the SAME readers --check + # uses and confirm each one actually took ${V} — and that at least one line + # matched per artifact, so a renamed key can't pass as "nothing to change". + rc=0 + _verify_file() { # label, file, grep-ERE, extract-sed, range(optional) + local label="$1" file="$2" gre="$3" ext="$4" range="${5:-}" n=0 line val + while IFS= read -r line; do + val=$(sed -E "$ext" <<< "$line"); n=$((n + 1)) + if [ "$val" != "$V" ]; then + echo "::error::--write did not take on $label: it is '$val', expected '$V' (the sed pattern matched nothing — the artifact's format changed)" >&2 + rc=1 + fi + done < <(_version_lines "$file" "$gre" "$range") + if [ "$n" -eq 0 ]; then + echo "::error::--write found no '$label' line in $file (key renamed/removed?) — nothing was synced" >&2 + rc=1 + fi + } + # range ↓ (tag only: same scope as the writer) + _verify_file "helm image.tag" "$VALUES" "$TAG_RE" "$TAG_SED" "$IMG_RANGE" + _verify_file "Chart appVersion" "$CHART" "$APPVER_RE" "$APPVER_SED" + _verify_file "frontend version" "$PKG" "$PKGVER_RE" "$PKGVER_SED" + _verify_file "operator image.tag" "$OPVALUES" "$TAG_RE" "$TAG_SED" "$IMG_RANGE" + _verify_file "operator appVersion" "$OPCHART" "$APPVER_RE" "$APPVER_SED" + _verify_file "dist env default" "$DISTENV" "$DISTENV_RE" "$DISTENV_SED" + _verify_file "dist compose pins" "$DISTCOMPOSE" "$PIN_RE" "$PIN_SED" + _verify_file "ibm-cloud pins" "$IBMCLOUD" "$PIN_RE" "$PIN_SED" + _verify_file "dist VERSION stamp" "$DISTVERSION" "$DISTVER_RE" "$DISTVER_SED" + [ "$rc" -eq 0 ] || exit 1 + echo "synced bnk-forge tag+appVersion, frontend package.json, operator tag+appVersion, dist compose+env, ibm-cloud installer, dist/VERSION -> ${V}" >&2 + ;; + + --check) + EXPECTED="$(cat "$ROOT/VERSION")" + # VERSION itself must be non-empty, or every artifact would "match" an empty + # string and the gate would pass on a tree with no version data at all + # (bonnyr-f5 #180 r3, BLOCKER 2). + if [ -z "$EXPECTED" ]; then + echo "::error::VERSION is empty — refusing to validate artifacts against nothing" >&2 + exit 1 + fi + rc=0; total=0 + # Assert EVERY version line is NON-EMPTY and equals VERSION, and that each + # artifact contributed at least one matched line. `total` counts MATCHED + # LINES, never loop iterations — an artifact whose key vanished contributes + # zero and both trips its own error and lowers the vacuity floor (bonnyr-f5 + # #180 r3: the old `checked` counted a literal 5-item list, so its >=5 guard + # was unreachable and --check was green on an empty tree). + _check_file() { # label, file, grep-ERE, extract-sed, range(optional) + local label="$1" file="$2" gre="$3" ext="$4" range="${5:-}" n=0 line val + while IFS= read -r line; do + val=$(sed -E "$ext" <<< "$line"); n=$((n + 1)); total=$((total + 1)) + if [ -z "$val" ]; then + echo "::error::$label in $file has an empty version — expected '$EXPECTED'"; rc=1 + elif [ "$val" != "$EXPECTED" ]; then + echo "::error::$label is '$val' but VERSION is '$EXPECTED' — the release publishes only :\${VERSION}, so a mismatch means ImagePullBackOff / drift. Run scripts/sync-version-artifacts.sh --write $EXPECTED"; rc=1 + else + echo " OK $label = $val" + fi + done < <(_version_lines "$file" "$gre" "$range") + if [ "$n" -eq 0 ]; then + echo "::error::$label: no version line matched in $file (key renamed/removed?) — vacuous check"; rc=1 + fi + } + # range ↓ (tag only: same scope as the writer) + _check_file "helm image.tag" "$VALUES" "$TAG_RE" "$TAG_SED" "$IMG_RANGE" + _check_file "Chart appVersion" "$CHART" "$APPVER_RE" "$APPVER_SED" + _check_file "frontend version" "$PKG" "$PKGVER_RE" "$PKGVER_SED" + _check_file "operator image.tag" "$OPVALUES" "$TAG_RE" "$TAG_SED" "$IMG_RANGE" + _check_file "operator appVersion" "$OPCHART" "$APPVER_RE" "$APPVER_SED" + _check_file "dist env default" "$DISTENV" "$DISTENV_RE" "$DISTENV_SED" + _check_file "dist compose pins" "$DISTCOMPOSE" "$PIN_RE" "$PIN_SED" + _check_file "ibm-cloud pins" "$IBMCLOUD" "$PIN_RE" "$PIN_SED" + _check_file "dist VERSION stamp" "$DISTVERSION" "$DISTVER_RE" "$DISTVER_SED" + # Backstop: nine artifacts, each with >=1 version line, is the minimum a + # healthy tree yields (the dist compose contributes 7 pins and the IBM + # installer 8, so the real total is far higher — this floor only catches a + # catastrophic "every key vanished"). Fewer means a key vanished — vacuous. + if [ "$total" -lt 9 ]; then + echo "::error::--check matched only $total version lines (expected >=9) — vacuous" >&2 + exit 1 + fi + exit "$rc" + ;; + + --list) + # Print the canonical artifact paths (repo-relative) so the release job stages + # EXACTLY what --write touches. Adding an artifact above updates all three. + for f in "${SYNCED_FILES[@]}"; do + printf '%s\n' "${f#"$ROOT"/}" + done + ;; + + *) + echo "usage: sync-version-artifacts.sh --write | --check | --list" >&2 + exit 2 + ;; +esac diff --git a/scripts/test-backup-restore.sh b/scripts/test-backup-restore.sh index 3810d79..420f124 100755 --- a/scripts/test-backup-restore.sh +++ b/scripts/test-backup-restore.sh @@ -202,7 +202,7 @@ echo "" info "Open the UI and create a backup:" echo "" echo " 1. Go to https://localhost (accept the self-signed cert warning)" -echo " 2. Log in with admin / changeme" +echo " 2. Log in as admin (password: docker exec bnk-forge-backend cat /app/keys/initial_admin_password)" echo " 3. Navigate to System → Backup & Restore tab" echo " 4. Enter a passphrase (12+ chars) — REMEMBER IT!" echo " 5. Click 'Create Backup'" @@ -300,7 +300,7 @@ echo "" info "Open the UI on the FRESH instance and restore your backup:" echo "" echo " 1. Go to https://localhost" -echo " 2. Log in with the DEFAULT credentials: admin / changeme" +echo " 2. Log in as admin (password from /app/keys/initial_admin_password or DEFAULT_ADMIN_PASSWORD)" echo " 3. Navigate to System → Backup & Restore tab" echo " 4. Upload the .tar.gz backup file you saved in Phase 2" echo " 5. Enter the SAME passphrase you used when creating the backup" diff --git a/scripts/tests/deploy-version-lockstep.test.sh b/scripts/tests/deploy-version-lockstep.test.sh new file mode 100644 index 0000000..38ce9ee --- /dev/null +++ b/scripts/tests/deploy-version-lockstep.test.sh @@ -0,0 +1,61 @@ +#!/usr/bin/env bash +# Deploy version lockstep (bonnyr-f5 #193 r3, M-1b). +# +# Asserts the Helm chart, the sibling bnk-operator chart, and the packaged dist/ +# install path can NEVER diverge on the pinned image version: VERSION == chart +# appVersion == values.yaml image.tag == the OPERATOR chart appVersion + image.tag +# (bonnyr-f5 #193 r4 deploy minor: the operator image also publishes at :${VERSION}, +# so a stale operator pin is the same ImagePullBackOff class) == every +# dist/docker-compose.yml pin == dist/.env.example default == dist/VERSION stamp == +# the IBM Cloud installer default (and its embedded compose pins). scripts/sync-version-artifacts.sh +# --write moves them together and --check enforces ==VERSION; this test is the +# standalone "they move in lockstep" assertion the review asked for (the chart used to +# pin the pre-guard image while dist/ pinned a forward-dated one nobody published). +set -euo pipefail + +HERE="$(cd "$(dirname "$0")" && pwd)" +ROOT="$(cd "$HERE/../.." && pwd)" +VERSION="$(cat "$ROOT/VERSION")" + +fail=0 +check() { # label, value + if [ "$2" = "$VERSION" ]; then printf 'PASS %-34s = %s\n' "$1" "$2" + else printf 'FAIL %-34s = %s (expected VERSION=%s)\n' "$1" "$2" "$VERSION"; fail=1; fi +} + +[ -n "$VERSION" ] || { echo "FAIL VERSION file is empty"; exit 1; } + +# Helm chart appVersion + image.tag. +check "chart appVersion" \ + "$(sed -nE 's/^appVersion: "?([^"]*)"?.*/\1/p' "$ROOT/helm/bnk-forge/Chart.yaml" | head -1)" +check "values image.tag" \ + "$(sed -nE '/^image:/,/^[^[:space:] #]/{/^ tag: /s/^ tag: "?([^"]*)"?.*/\1/p;}' "$ROOT/helm/bnk-forge/values.yaml" | head -1)" + +# Sibling bnk-operator chart appVersion + image.tag (bonnyr-f5 #193 r4 deploy minor): +# the operator image is on the same VERSION train, so its pins must move in lockstep too. +check "operator appVersion" \ + "$(sed -nE 's/^appVersion: "?([^"]*)"?.*/\1/p' "$ROOT/bnk-operator/charts/bnk-operator/Chart.yaml" | head -1)" +check "operator image.tag" \ + "$(sed -nE '/^image:/,/^[^[:space:] #]/{/^ tag: /s/^ tag: "?([^"]*)"?.*/\1/p;}' "$ROOT/bnk-operator/charts/bnk-operator/values.yaml" | head -1)" + +# dist/.env.example default. +check "dist .env default" \ + "$(sed -nE 's/^BNK_FORGE_VERSION=(.*)/\1/p' "$ROOT/dist/.env.example" | head -1)" + +# dist/VERSION plain stamp (bonnyr-f5 #193 r4: now single-sourced by sync-version-artifacts.sh). +check "dist VERSION stamp" \ + "$(sed -nE '1{s/^[[:space:]]*([A-Za-z0-9._+-]+).*/\1/p;}' "$ROOT/dist/VERSION")" + +# Every ${BNK_FORGE_VERSION:-} default in the dist compose and the IBM installer. +n=0 +while IFS= read -r v; do n=$((n + 1)); check "dist compose pin #$n" "$v"; done \ + < <(grep -oE '\$\{BNK_FORGE_VERSION:-[^}]*\}' "$ROOT/dist/docker-compose.yml" | sed -E 's/.*:-([^}]*)\}/\1/') +[ "$n" -ge 6 ] || { echo "FAIL expected >=6 dist compose pins, found $n (parser drift?)"; fail=1; } + +m=0 +while IFS= read -r v; do m=$((m + 1)); check "ibm-cloud pin #$m" "$v"; done \ + < <(grep -oE '\$\{BNK_FORGE_VERSION:-[^}]*\}' "$ROOT/scripts/ibm_cloud_bnk_forge.sh" | sed -E 's/.*:-([^}]*)\}/\1/') +[ "$m" -ge 6 ] || { echo "FAIL expected >=6 ibm-cloud pins, found $m (parser drift?)"; fail=1; } + +echo "----" +[ "$fail" = 0 ] && echo "ALL PASS" || { echo "FAILURES"; exit 1; } diff --git a/scripts/tests/detector-parity.test.sh b/scripts/tests/detector-parity.test.sh new file mode 100644 index 0000000..ab3bdb9 --- /dev/null +++ b/scripts/tests/detector-parity.test.sh @@ -0,0 +1,131 @@ +#!/usr/bin/env bash +# INV-15 detector parity (bonnyr-f5 #179 r3; #193 M1/M5). +# +# The BREAKING CHANGE detector MUST be identical everywhere it is used: +# scripts/compute_version_bump.sh (the bump), scripts/extract-breaking-changes.sh +# (the note), and scripts/lint-commit-markers.sh (the commit-lint gate). If they +# drift, a major bump ships with empty notes, or a note ships with no bump, or the +# gate flags a shape the detectors accept. +# +# It USED to diff two byte-identical inline copies. Round-2 single-sources the +# predicate into scripts/lib/breaking-change-detect.sh, so drift is now +# structurally impossible; this test asserts that wiring: +# 1. the shared lib exists and defines BOTH functions; +# 2. every consumer SOURCES the lib and does NOT redefine either function +# inline (an inline copy would silently shadow the shared one and reopen the +# drift the single-source closes); +# 3. the shared predicate behaves — a representative positive and negative body, +# so an empty/stubbed lib cannot pass vacuously. +# +# It lives here as a scripts/tests/*.test.sh so BOTH the Makefile `script-selftests` +# target AND ci.yml's `script-selftests` job run it through the SAME filesystem +# enumeration — local == CI (bonnyr-f5 #193 M5). +set -euo pipefail + +HERE="$(cd "$(dirname "$0")" && pwd)" +ROOT="$(cd "$HERE/../.." && pwd)" +LIB="$ROOT/scripts/lib/breaking-change-detect.sh" +CONSUMERS=( + "$ROOT/scripts/compute_version_bump.sh" + "$ROOT/scripts/extract-breaking-changes.sh" + "$ROOT/scripts/lint-commit-markers.sh" +) + +fail=0 +# Standard self-test output convention (PASS lines + ALL PASS / FAILURES terminal), +# matching the other scripts/tests/*.test.sh so the Makefile `script-selftests` +# harness can assert on it uniformly (bonnyr-f5 #193 r4 M-4). +pass() { printf 'PASS %s\n' "$1"; } +bad() { printf 'FAIL %s\n' "$1"; fail=1; } + +# 1. The shared lib exists and defines both functions. +[ -f "$LIB" ] || { echo "::error::detector-parity: shared lib $LIB not found"; echo "FAILURES"; exit 1; } +for fn in _is_breaking_subject _is_breaking_body; do + if grep -qE "^${fn}\(\)" "$LIB"; then pass "shared lib defines ${fn}()" + else bad "$LIB does not define ${fn}()"; fi +done + +# 2. Every consumer sources the lib and does NOT redefine either function inline. +for c in "${CONSUMERS[@]}"; do + [ -f "$c" ] || { bad "consumer $c not found"; continue; } + if grep -q 'lib/breaking-change-detect.sh' "$c"; then pass "$(basename "$c") sources the shared detector lib" + else bad "$(basename "$c") does not source the shared detector lib"; fi + for fn in _is_breaking_subject _is_breaking_body; do + if grep -qE "^${fn}\(\)" "$c"; then + bad "$(basename "$c") redefines ${fn}() inline — it would shadow the shared lib and can drift" + else pass "$(basename "$c") does not shadow ${fn}()"; fi + done +done + +# 3. Behavioural smoke test: the shared predicate actually classifies. Guards +# against an emptied/stubbed lib passing the wiring checks vacuously. +# shellcheck source=scripts/lib/breaking-change-detect.sh +. "$LIB" +if _is_breaking_body $'fix: y\n\nBREAKING CHANGE: the flag was removed'; then pass "shared _is_breaking_body accepts a real footer" +else bad "shared _is_breaking_body missed a real footer"; fi +if _is_breaking_body $'fix: y\n\nthis is not a breaking change at all'; then bad "shared _is_breaking_body false-positived on prose" +else pass "shared _is_breaking_body rejects prose"; fi +if _is_breaking_subject 'feat!: drop v1'; then pass "shared _is_breaking_subject accepts a bang subject" +else bad "shared _is_breaking_subject missed a bang subject"; fi +if _is_breaking_subject 'feat: normal change'; then bad "shared _is_breaking_subject false-positived on a normal subject" +else pass "shared _is_breaking_subject rejects a normal subject"; fi + +# 4. Marker-regex single source (bonnyr-f5 #193 r3; enumeration hardened r4 M-3). +# The marker shape is named ONCE as _BREAKING_MARKER_ERE; the awk copies embed the +# identical literal (an ERE through `awk -v` mangles `\*`). Assert it is defined and +# canonical: +EXPECT='^([*-][[:space:]]+)?(\*\*)?BREAKING[[:space:] -]+CHANGE' +if [ -z "${_BREAKING_MARKER_ERE:-}" ]; then + bad "_BREAKING_MARKER_ERE is not defined in the shared lib" +elif [ "$_BREAKING_MARKER_ERE" != "$EXPECT" ]; then + bad "_BREAKING_MARKER_ERE drifted from the canonical marker shape" +else + pass "_BREAKING_MARKER_ERE == canonical marker shape" +fi + +# Enumerate the embedded copies by POSITION / COUNT, NOT by matching the driftable +# token (bonnyr-f5 #193 r4 M-3). The old check enumerated with +# `grep -F 'BREAKING[[:space:] -]+CHANGE'` — the VERY token that drifts — so a copy +# that drifted IN the token vanished from the enumeration and was never compared +# (drifting :96-97 dropped the count 5->3 yet the test stayed green). Two independent +# guards close it: +# (a) EXACT COUNT of byte-identical canonical strings per file — a copy that drifts, +# or is added / removed, changes the count away from the structural expectation; +# (b) LOOSE enumeration of every marker-regex SITE via a STABLE anchor that does NOT +# contain the driftable char class (an awk `~ /^…CHANGE…/` or `!~ /^…CHANGE…/` +# match, or the `_BREAKING_MARKER_ERE=` assignment), asserting each site embeds +# the canonical string. A drift keeps both `CHANGE` and the `~ /^` context, so +# the site stays enumerated and fails the embed check even under a compensating +# add that keeps the count unchanged. Prose comments carry no `~ /^…CHANGE` and +# are not enumerated. +# The per-file count is a deliberate STRUCTURAL invariant: adding or removing a copy is +# a reviewable event that must update it here. spec = file:expected-canonical-count +for spec in \ + "scripts/lib/breaking-change-detect.sh:5" \ + "scripts/extract-breaking-changes.sh:3" +do + f="${spec%:*}"; want="${spec##*:}"; p="$ROOT/$f"; b="$(basename "$f")" + if [ ! -f "$p" ]; then bad "marker consumer $f not found"; continue; fi + got="$(grep -cF "$EXPECT" "$p" || true)" + if [ "$got" -eq "$want" ]; then + pass "$b embeds exactly $want byte-identical canonical marker regex(es)" + else + bad "$b embeds $got byte-identical marker regex(es), expected $want — a copy drifted from, or was added to / removed from, the canonical shape" + fi + drift=0 + while IFS= read -r ln; do + case "$ln" in + *"$EXPECT"*) : ;; + *) bad "$b: marker-regex site drifted from canonical: $ln"; drift=1 ;; + esac + done < <(grep -E '(~|!~)[[:space:]]*/\^[^/]*CHANGE|_BREAKING_MARKER_ERE=' "$p") + [ "$drift" -eq 0 ] && pass "$b: every marker-regex site embeds the canonical shape" +done + +if [ "$fail" -eq 0 ]; then + echo "INV-15 OK: detector + marker regex single-sourced in scripts/lib/breaking-change-detect.sh; all consumers source/embed the one definition, none shadow it, predicate behaves" + echo "ALL PASS" +else + echo "::error::detector-parity: one or more checks failed" + echo "FAILURES"; exit 1 +fi diff --git a/scripts/tests/helm-known-defaults-lockstep.test.sh b/scripts/tests/helm-known-defaults-lockstep.test.sh new file mode 100644 index 0000000..7989337 --- /dev/null +++ b/scripts/tests/helm-known-defaults-lockstep.test.sh @@ -0,0 +1,54 @@ +#!/usr/bin/env bash +# Helm <-> Python known-default denylist lockstep (bonnyr-f5 #193 r3, minor). +# +# The chart's fail-guards in helm/bnk-forge/templates/secrets.yaml refuse a shipped +# default password so the chart never emits a value the backend rejects. That denylist +# is a THIRD copy of the Python source of truth: +# - $mcpDefaults MUST match backend/core/config.py MCP_KNOWN_DEFAULT_PASSWORDS +# - $adminDefaults MUST match backend/services/auth_service.py _KNOWN_DEFAULT_ADMIN_PASSWORDS +# Nothing asserted the lockstep the comments claim; if they drift, the chart can emit a +# value the backend fatal-rejects (crashloop) or fail-guard a value the backend accepts. +# This test greps both sides and asserts set-equality (order-independent). +set -euo pipefail + +HERE="$(cd "$(dirname "$0")" && pwd)" +ROOT="$(cd "$HERE/../.." && pwd)" +SECRETS="$ROOT/helm/bnk-forge/templates/secrets.yaml" +CONFIG="$ROOT/backend/core/config.py" +AUTH="$ROOT/backend/services/auth_service.py" + +fail=0 + +# Extract the quoted tokens from a Helm `$ := list "a" "b"` assignment. Grab the +# whole assignment line first, then every quoted token on it (the line carries no other +# quoted content), so a hyphenated token like "mcp-service-changeme" is not truncated. +_helm_list() { # var-name, file + grep -E "\\\$$1 := list " "$2" | head -1 \ + | grep -oE '"[^"]*"' | tr -d '"' | sort | tr '\n' ' ' | sed 's/ $//' +} +# Extract the quoted tokens from a Python tuple assignment `NAME = ("a", "b")`. +_py_tuple() { # NAME, file + grep -E "^$1 = \(" "$2" | head -1 \ + | grep -oE '"[^"]*"' | tr -d '"' | sort | tr '\n' ' ' | sed 's/ $//' +} + +compare() { # label, helm-set, py-set + if [ "$2" = "$3" ]; then printf 'PASS %-28s [%s]\n' "$1" "$2" + else printf 'FAIL %-28s helm=[%s] python=[%s]\n' "$1" "$2" "$3"; fail=1; fi +} + +HELM_MCP="$(_helm_list mcpDefaults "$SECRETS")" +PY_MCP="$(_py_tuple MCP_KNOWN_DEFAULT_PASSWORDS "$CONFIG")" +compare "mcp defaults" "$HELM_MCP" "$PY_MCP" + +HELM_ADMIN="$(_helm_list adminDefaults "$SECRETS")" +PY_ADMIN="$(_py_tuple _KNOWN_DEFAULT_ADMIN_PASSWORDS "$AUTH")" +compare "admin defaults" "$HELM_ADMIN" "$PY_ADMIN" + +# Guard against a vacuous pass if a grep silently matched nothing. +for v in "$HELM_MCP" "$PY_MCP" "$HELM_ADMIN" "$PY_ADMIN"; do + [ -n "$v" ] || { echo "FAIL a denylist extracted EMPTY (grep drift) — treat as vacuous"; fail=1; } +done + +echo "----" +[ "$fail" = 0 ] && echo "ALL PASS" || { echo "FAILURES"; exit 1; } diff --git a/scripts/tests/helm-secret-checksum.test.sh b/scripts/tests/helm-secret-checksum.test.sh new file mode 100644 index 0000000..0f35aca --- /dev/null +++ b/scripts/tests/helm-secret-checksum.test.sh @@ -0,0 +1,44 @@ +#!/usr/bin/env bash +# bonnyr-f5 #193 M7 + security regression lock. +# The pod `checksum/secret` annotation must: +# (a) be STABLE across renders for unchanged inputs (no perpetual GitOps drift); +# (b) CHANGE when a credential rotates, so the pods actually roll (M7); +# and the GENERATED secret fallbacks must be RANDOM / unpredictable -- NEVER derived from +# public release identity (release name, namespace, fullname), which would make the JWT +# signing key, the at-rest Fernet key and the admin/mcp passwords computable by anyone who +# can read a resource label. This test locks the revert of the `deriveSecret` determinism. +set -u +ROOT="$(cd "$(dirname "$0")/../.." && pwd)" +CHART="$ROOT/helm/bnk-forge" +fail=0 + +if ! command -v helm >/dev/null 2>&1; then + echo "PASS (skipped: helm not installed)" + echo "----"; echo "ALL PASS"; exit 0 +fi + +# ALLOWED_ORIGINS override keeps the M-10 production+localhost render guard from firing. +COMMON=(--set api.env.ALLOWED_ORIGINS=https://forge.example.com) +cksum() { helm template rel "$CHART" "$@" 2>/dev/null | grep -m1 'checksum/secret:' | awk '{print $2}'; } +mcpval() { helm template rel "$CHART" "$@" 2>/dev/null | grep -m1 'mcp-password:' | awk '{print $2}'; } + +# (a) stability: identical inputs, two renders -> identical checksum. +a1=$(cksum "${COMMON[@]}" --set secrets.mcpPassword=Sup3rSecretA --set secrets.adminPassword=Adm1nSecretA) +a2=$(cksum "${COMMON[@]}" --set secrets.mcpPassword=Sup3rSecretA --set secrets.adminPassword=Adm1nSecretA) +if [ -n "$a1" ] && [ "$a1" = "$a2" ]; then echo "PASS checksum stable across identical renders" +else echo "FAIL checksum not stable across identical renders ('$a1' vs '$a2')"; fail=1; fi + +# (b) rotation: a different mcp-password -> a different checksum (pods roll). M7. +b2=$(cksum "${COMMON[@]}" --set secrets.mcpPassword=Sup3rSecretB --set secrets.adminPassword=Adm1nSecretA) +if [ -n "$a1" ] && [ -n "$b2" ] && [ "$a1" != "$b2" ]; then echo "PASS checksum changes on mcp-password rotation (pods roll)" +else echo "FAIL checksum did not change on mcp-password rotation ('$a1')"; fail=1; fi + +# (c) unpredictability: a generated (unset) mcp-password is RANDOM -> two renders differ, and +# is therefore not a deterministic derivation of release identity. +c1=$(mcpval "${COMMON[@]}") +c2=$(mcpval "${COMMON[@]}") +if [ -n "$c1" ] && [ "$c1" != "$c2" ]; then echo "PASS generated mcp-password is random (unpredictable)" +else echo "FAIL generated mcp-password is deterministic/derived ('$c1') -- predictable-secret regression"; fail=1; fi + +echo "----" +[ "$fail" = 0 ] && echo "ALL PASS" || { echo "FAILURES"; exit 1; } diff --git a/scripts/tests/ibm-compose-drift.test.sh b/scripts/tests/ibm-compose-drift.test.sh new file mode 100644 index 0000000..6d99316 --- /dev/null +++ b/scripts/tests/ibm-compose-drift.test.sh @@ -0,0 +1,63 @@ +#!/usr/bin/env bash +# IBM installer / dist compose drift guard (bonnyr-f5 #193 r4 deploy minor). +# +# scripts/ibm_cloud_bnk_forge.sh embeds its OWN copy of the deploy compose (a +# heredoc), separate from dist/docker-compose.yml. The two are meant to deliver the +# SAME backend credential/security contract, but nothing enforced it and they had +# drifted — exactly the B-1 class (a fix applied to one compose path but not the +# other consumer). This test freezes the security-critical env treatment: for every +# key below, the value form (verbatim right-hand side) MUST be byte-identical in +# both files, so a future edit to one that is not mirrored in the other fails CI. +# +# It deliberately does NOT diff the whole file — the two legitimately differ +# (networking, build stanzas, placeholders). It pins only the credential/hardening +# env whose divergence is a security regression. The image-version pins are covered +# separately by deploy-version-lockstep.test.sh. +set -euo pipefail + +HERE="$(cd "$(dirname "$0")" && pwd)" +ROOT="$(cd "$HERE/../.." && pwd)" +DIST="$ROOT/dist/docker-compose.yml" +IBM="$ROOT/scripts/ibm_cloud_bnk_forge.sh" + +# The credential/hardening env keys that MUST stay in lockstep across both paths. +KEYS=( + DEFAULT_ADMIN_PASSWORD + DEFAULT_ADMIN_MUST_CHANGE + MCP_SERVICE_USERNAME + MCP_SERVICE_PASSWORD + ENVIRONMENT + JWT_SECRET_KEY + ENCRYPTION_KEY + ALLOWED_ORIGINS +) + +# Extract the value form (everything after the first ` :`), trimmed. A +# passthrough key (`DEFAULT_ADMIN_PASSWORD:` with no value) yields the empty string +# in BOTH files, which still compares equal — so a drift to `${...:-}` on one side +# is caught. +_val() { # file, key + sed -nE "s/^[[:space:]]*$2:[[:space:]]*(.*)$/\1/p" "$1" | head -1 +} + +fail=0 +for f in "$DIST" "$IBM"; do + [ -f "$f" ] || { echo "FAIL missing file: $f"; exit 1; } +done + +for k in "${KEYS[@]}"; do + dv="$(_val "$DIST" "$k")" + iv="$(_val "$IBM" "$k")" + # Both must actually contain the key (guard against a key being dropped from one). + if ! grep -qE "^[[:space:]]*$k:" "$DIST"; then echo "FAIL $k absent from dist/docker-compose.yml"; fail=1; continue; fi + if ! grep -qE "^[[:space:]]*$k:" "$IBM"; then echo "FAIL $k absent from IBM embedded compose"; fail=1; continue; fi + if [ "$dv" = "$iv" ]; then + printf 'PASS %-26s = %s\n' "$k" "${dv:-}" + else + printf 'FAIL %-26s dist=[%s] ibm=[%s] (drift)\n' "$k" "$dv" "$iv"; fail=1 + fi +done + +echo "----" +[ "$fail" = 0 ] && echo "ALL PASS — IBM embedded compose matches dist on the credential/hardening env" \ + || { echo "FAILURES — reconcile the IBM embedded compose with dist/docker-compose.yml"; exit 1; } diff --git a/scripts/tests/lint-commit-markers.test.sh b/scripts/tests/lint-commit-markers.test.sh new file mode 100644 index 0000000..ba35fc8 --- /dev/null +++ b/scripts/tests/lint-commit-markers.test.sh @@ -0,0 +1,226 @@ +#!/usr/bin/env bash +# Mutation tests for scripts/lint-commit-markers.sh (bonnyr-f5 #193 B3 + M1; r3 M-6). +# +# Proves, on throwaway git repos: +# rule 1 — a NEW [skip ci] commit anywhere in the scanned range is CAUGHT, while +# the release-bot's OWN minted commit (version + trailing skip marker) is +# exempt and a spoofed `release: [skip ci]` (no version) is CAUGHT. +# rule 2 — the COMPLEMENT of the detectors: dash-bullet, markdown-bold and indented +# shapes are NOT flagged; a mis-anchored DECLARATIVE `BREAKING CHANGE:` +# (colon) marker IS flagged; a COLONLESS marker-shaped PROSE line the +# detectors treat as inert is NOT flagged (M-6b — it used to red +# unamendable release history); and EVERY mis-anchored marker in a body is +# reported, not just the first. +# M-6a — there is NO already-merged exemption (removed as dead code); the release- +# bot fingerprint is single-sourced and release.yml's inline copy matches. +set -uo pipefail + +HERE="$(cd "$(dirname "$0")" && pwd)" +ROOT="$(cd "$HERE/../.." && pwd)" +SCRIPT="$HERE/../lint-commit-markers.sh" + +fail=0 +pass() { printf 'PASS %s\n' "$1"; } +bad() { printf 'FAIL %s\n' "$1"; fail=1; } + +# new_repo -> prints the repo path; caller adds commits then runs `lint`. +new_repo() { + local d; d="$(mktemp -d)" + git init -q "$d" + git -C "$d" config user.email "selftest@bnk-forge.local" + git -C "$d" config user.name "bnk-forge self-test" + git -C "$d" commit --allow-empty -q --cleanup=verbatim -m "initial" + printf '%s' "$d" +} +# commit [body] (verbatim: exact body, leading spaces kept) +commit() { + local d="$1" subj="$2" body="${3:-}" + if [ -n "$body" ]; then + git -C "$d" commit --allow-empty -q --cleanup=verbatim -m "$subj" -m "$body" + else + git -C "$d" commit --allow-empty -q --cleanup=verbatim -m "$subj" + fi +} +# run -> sets global RC and OUT (BEFORE is no longer consumed) +run() { + local d="$1" range="$2" + OUT="$(cd "$d" && RANGE="$range" PR_TITLE="" LINT_MESSAGE="" bash "$SCRIPT" 2>&1)" + RC=$? +} +# expect
@@ -122,9 +116,9 @@

BNK Forge — Install Guide

- This guide walks you through installing BNK Forge from the private GitHub Container Registry. - You will authenticate to the registry using the read-only bot credential you were provided, - download the install package, make a small configuration change, and run a single script. + This guide walks you through installing BNK Forge from the public GitHub Container Registry. + You download the install package, make a small configuration change, and run a single + script. The images are public, so no registry login is required. The entire process takes under ten minutes on a fast connection; the first image pull may take a few minutes depending on bandwidth.
@@ -134,40 +128,12 @@

Prerequisites

  • Docker Engine 24+ — on macOS or Windows, Docker Desktop satisfies both this and the Compose requirement.
  • Docker Compose v2 (≥ 2.24) — verify with docker compose version.
    • -
  • Read-access token — the <READ_TOKEN> provided to you separately (see Step 1).
  • ~5–10 GB free disk space — for images and persistent data volumes.
  • Network access to ghcr.io — outbound HTTPS (port 443) must be allowed.
-

Step 1 — Authenticate to the registry

- -
-
Credentials — handle with care
- Your read-access token is provided separately (out-of-band). It is a GitHub PAT scoped to - read:packages for the ghcr.io/jlcode-tech registry. - Do not share it, commit it to version control, or embed it in scripts. - If you believe the token has been exposed, contact the person who gave it to you immediately. -
- -

Run the following command, replacing <READ_TOKEN> with the token you received. - The bot username is fixed — use it exactly as shown:

- - 
$ echo '<READ_TOKEN>' | docker login ghcr.io -u jlcode-forgebot --password-stdin
- -

Expected output:

-
Login Succeeded
- -
-
Token stays cached
- Docker stores the credential in your OS keychain (or ~/.docker/config.json). - You only need to log in once per machine. If you later see a denied or - unauthorized error during a pull, re-run the command above — the token may have been - rotated. See the Troubleshooting section for details. -
- - -

Step 2 — Download & extract the package

+

Step 1 — Download & extract the package

You should have received a bnk-forge-<version>.tar.gz archive alongside this guide. Save it to a convenient location, then extract it:

@@ -177,8 +143,8 @@

Step 2 — Download & extract the package

All subsequent commands are run from inside this directory.

- -

Step 3 — Configure

+ +

Step 2 — Configure

Copy the example environment file and open it in your editor:

@@ -195,13 +161,13 @@

Step 3 — Configure

BNK_FORGE_REGISTRY - ghcr.io/jlcode-tech - Points to the private registry. + ghcr.io/f5devcentral + Points to the public registry. BNK_FORGE_VERSION - customer-build - Rolling latest build. To pin a specific build, use a tag like 3.0.1-cb.<sha>. + leave as shipped + Pre-set to this bundle's version (see the VERSION file) — it always names an image the release actually published. Do not change it to latest: a floating tag can resolve to an image whose credential contract differs from this bundle. POSTGRES_PASSWORD @@ -214,21 +180,46 @@

Step 3 — Configure

Change from the default. Used for the internal cache/queue. - MCP_PASSWORD + MCP_SERVICE_PASSWORD your choice - Change from the default. Used by the MCP integration layer. + Credential the bundled MCP server uses to authenticate to BNK Forge. MCP logs in as its + own dedicated, non-human service account (MCP_SERVICE_USERNAME, default + mcp) — never the human admin login. No default + ships: choose a strong value here and the backend seeds/reconciles the mcp + account to it on every boot, so the two always agree. Leave it empty and MCP stays + unavailable until you set it (the backend refuses to seed a guessable credential, and + the old mcp-service-changeme default can no longer authenticate). It is + re-read from .env on every boot, so to rotate it edit .env and + re-run docker compose up -d (a plain docker compose restart + does not re-read .env).
-
Change all three passwords
- The defaults in .env.example are well-known placeholders. - Replace POSTGRES_PASSWORD, REDIS_PASSWORD, and MCP_PASSWORD - before running the installer — they cannot be changed easily after the stack first starts. +
Set strong passwords before first start
+ POSTGRES_PASSWORD and REDIS_PASSWORD replace well-known + placeholder defaults, and they are baked in when the database and cache first + initialize — get them right before running the installer, as they cannot be changed + easily afterward. MCP_SERVICE_PASSWORD is independent of the admin password: + MCP authenticates as its own dedicated mcp service account, so set it to a strong + secret of your choosing (no shipped default) and the backend provisions the mcp + account from it. It is re-read on every boot, so to rotate it edit .env and re-run + docker compose up -d (a plain docker compose restart does not re-read + .env). +

A dedicated MCP service account — mcp. Besides the human + admin, this build seeds a non-human, admin-role service account named + mcp that the bundled MCP server authenticates as. It carries no shipped + default: the backend provisions it from MCP_SERVICE_PASSWORD and reconciles + the stored hash to that value on every boot, so rotating the secret is simply a matter of + editing .env and re-running docker compose up -d. The old shipped + mcp-service-changeme default has been removed and can no longer authenticate, and + MCP no longer borrows the human admin login (#186). If you leave + MCP_SERVICE_PASSWORD unset, the backend disables any stale service account carried + over from an upgrade and MCP stays unavailable until you configure it.

- -

Step 4 — Install

+ +

Step 3 — Install

Run the installer for your platform. It will pull the images and bring the full stack up. The first run may take a few minutes while images download.

@@ -251,8 +242,8 @@

Step 4 — Install

When the installer finishes you will see:

✅ Installation complete!
- -

Step 5 — First login

+ +

Step 4 — First login

  1. @@ -267,12 +258,19 @@

    Step 5 — First login

    (Firefox). This is expected.

  2. -

    Log in with the default credentials:
    - Username: admin  /  Password: changeme

    +

    Log in as admin.
    + No default password ships. The simplest path is to set + DEFAULT_ADMIN_PASSWORD in .env before install and log in with + that value. On releases that generate an admin password, the backend instead writes a + random one to a file on first start (the plaintext is never written to the logs); + retrieve it with the command below. If that file is absent, use the + DEFAULT_ADMIN_PASSWORD you set.

    + 
    $ docker compose exec backend cat /app/keys/initial_admin_password
    Change the admin password immediately
    - The default password is well-known. Go to User menu → Change Password as your - very first action after login. + This generated password is a one-time bootstrap credential. Go to + User menu → Change Password as your very first action after login — the API + refuses every other call until you do.
  3. @@ -309,8 +307,9 @@

    Verify the stack is healthy

    Updating to a newer build

    -

    Because BNK_FORGE_VERSION=customer-build is a rolling tag, updating is simple. - From the install directory:

    +

    Each bundle pins BNK_FORGE_VERSION to its own release. To update, download the + newer bundle and run its installer from the new install directory — the shipped + BNK_FORGE_VERSION already points at the matching images:

    # Recommended — uses the installer for any migration steps:
 $ ./install.sh          # Linux
@@ -319,8 +318,37 @@ 
    Updating to a newer build
    
 # Alternative — manual pull and restart:
 $ docker compose pull && docker compose up -d
    -

    If you pinned a specific build tag in .env, update BNK_FORGE_VERSION - to the new tag before running the command above.

    +

    If you carried an old .env forward, update its BNK_FORGE_VERSION to + match the new bundle's VERSION file before running the command above.

    + +
    +
    Before upgrading an older install to this release
    + If your .env predates this release, reconcile it first, or the upgrade will fail + to pull or refuse to boot: +
      +
    • Registry & version. Set BNK_FORGE_REGISTRY=ghcr.io/f5devcentral + (older packages pointed at a private org that no longer resolves), and replace any pinned + BNK_FORGE_VERSION such as 3.0.1 with the version this bundle + ships (see its VERSION file) — a stale pin pulls a tag that no longer exists. + Do not substitute latest.
      • +
    • Non-root artifact images. The container runner now refuses any image whose + USER is root or a named user (e.g. USER nonroot). Rebuild + your own runner images with a numeric USER 1000 before upgrading.
      • +
    • MCP service credential (new in this release). The bundled MCP server now authenticates as a + dedicated mcp service account, not the human admin. Set + MCP_SERVICE_PASSWORD to a strong secret in this .env: this + compose file passes it to the backend (which provisions the mcp account from + it) and to the MCP container (as BNK_FORGE_PASSWORD), so the two stay + in sync. The old MCP_PASSWORD→admin coupling is gone; the short + name MCP_PASSWORD is still honored as a legacy alias for + MCP_SERVICE_PASSWORD so an existing .env keeps working, and the + shipped mcp-service-changeme default can no longer authenticate. Under + ENVIRONMENT=staging/production — which the compose files now plumb + through to the backend — it refuses to start while MCP_SERVICE_PASSWORD is + unset or set to a shipped default (#186 + #188), so set it to a real secret + before upgrading.
      • +
    +

    Uninstall

    @@ -335,10 +363,10 @@

    Troubleshooting

    • - denied: denied or unauthorized: unauthenticated during pull - Your docker login session has lapsed, or the token does not have the - read:packages scope. Re-run Step 1 with your current token. - If the problem persists, contact the person who issued the token. + manifest unknown, or a pull that hangs or times out + The images are public, so no login is required. Check that BNK_FORGE_REGISTRY + is ghcr.io/f5devcentral and that the version tag exists, and that outbound + HTTPS to ghcr.io (port 443) is allowed through any proxy or firewall.
    • Backend service not healthy @@ -358,8 +386,8 @@

      Troubleshooting

      - BNK Forge install guide — generated for customer-build distribution  ·  - Registry: ghcr.io/jlcode-tech  ·  Tag: customer-build + BNK Forge install guide — generated for the current release  ·  + Registry: ghcr.io/f5devcentral  ·  Tag: this bundle's VERSION
      diff --git a/vm-bnk-forge/README.md b/vm-bnk-forge/README.md index 56cbe3b..f4ac615 100644 --- a/vm-bnk-forge/README.md +++ b/vm-bnk-forge/README.md @@ -16,7 +16,7 @@ make-vm.sh ─┐ Ubuntu 24.04 minimal cloudimg ``` VMs bridge onto the host's bridge interface, get DHCP from the LAN, and -serve bnk-forge on `https://:8443` (login `admin` / `changeme`). +serve bnk-forge on `https://:8443` (login `admin`; the password is generated on first boot — see Default credentials below). Port 8443, not 443: `make install` deploys the server topology, where the proxy runs with `network_mode: host` and binds 8443 (HTTPS) / 8082 (HTTP redirect) directly — see the `proxy` service in `docker-compose.yml`. @@ -189,7 +189,8 @@ from). The local KVM flow lands on a LAN. The cloud flow can land on a public address, where three properties of this image stack up badly: -1. **Default credentials.** The stack comes up with `admin` / `changeme`. +1. **Default credentials.** The stack comes up with a generated `admin` password + (at `/app/keys/initial_admin_password`, or set `DEFAULT_ADMIN_PASSWORD`). Log in and change it immediately — or don't attach a public IP until you have. 2. **Docker socket is mounted into the backend.** `make install` writes a