From cd3b4a9a09b5e3b33d6fba09ea4e7106e5883e67 Mon Sep 17 00:00:00 2001 From: PatternCatalyst Date: Thu, 8 Oct 2026 21:30:06 -0400 Subject: [PATCH 01/42] chore(plans): plan published loopback NodePorts replacing tunnels --- _plans/nodeports-plan.md | 198 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 198 insertions(+) create mode 100644 _plans/nodeports-plan.md diff --git a/_plans/nodeports-plan.md b/_plans/nodeports-plan.md new file mode 100644 index 0000000..f9f40a6 --- /dev/null +++ b/_plans/nodeports-plan.md @@ -0,0 +1,198 @@ +# Plan: published loopback NodePorts replace SSH tunnels / port-forward + +Branch `fix/nodeports-no-tunnels`. `EX/` = `examples/lgtm-datamesh/`. Planned 2026-10-08 (Opus). +Move this file to `_plans/archive/` once executed (its own text would fail the forbidden-syntax gate). + +## Approach + +All 14 host-facing services are already NodePorts, so only the transport changes. The +`capstone` profile publishes them at creation: `minikube start --ports=127.0.0.1::,...`. +Host port = nodePort, and clients use `http://127.0.0.1:`. + +- `EX/demos/lib/endpoints.sh` replaces `tunnels.sh` and holds the only port list. The setup script and + the demos both read it. +- The setup script refuses any existing profile (running or stopped) whose `podman inspect` + PortBindings lack a 127.0.0.1 binding for every listed port, and points to `--replace`. +- `scripts/forbidden-syntax.sh` gates CI. + +Rejected: +- autossh, a port-forward respawn loop, `minikube tunnel` with LB: all are tunnels. +- Bare `--ports=np:np`: binds 0.0.0.0. +- Keeping the old low host ports: they collide with other local services, and the skill says host = nodePort. +- kvm2 with `minikube ip`: loses the rootless podman tuning. +- One ingress port routed by Host header: too much churn for now. +- `TP_*` aliases: the name is wrong. + +## Design decisions + +1. **Names.** Drop `TP_*` with no aliases. Keep the `NP_*` names and values. + `endpoints.sh` provides: + - `_endpoint_row `, returning "np ns svc label", for the same 14 names. + - `ENDPOINT_NAMES`. + - `CAPSTONE_NODE_PORTS`, derived from the table, plus `EXTRA_NODE_PORTS`. + - `endpoint_port`, `endpoint_url`, `node_ports_arg`. + - `published_ports`: podman inspect `.HostConfig.PortBindings`, counting only HostIp 127.0.0.1; + a missing container means an empty set. + - `nonloopback_ports`: hard fail if any port is published on another address. + - `check_published_ports`: lists what's missing and prints the `--replace` hint. + - `ensure_endpoint ...`, which checks three things: + - the port is published; + - the Service type and nodePort match; + - a TCP reach, skipped for scaled-to-zero notification. + - `wait_http`, `wake_gateway` and `GATEWAY_HOST`, carried over. + - Optionally, `registry_host_port`. +2. **One list.** `setup-capstone-profile.sh` sources `../demos/lib/endpoints.sh` and builds + `--ports="$(node_ports_arg)"`. Scan 5 of forbidden-syntax checks that chart and patch nodePorts match + the map. +3. **Rootless podman.** minikube documents `--ports` for docker and podman, and the node already + publishes 22, 2376, 5000, 8443 and 32443 on 127.0.0.1. Risks: + - the IP is dropped, or the flag is ignored; + - `minikube start` on an existing profile ignores `--ports`. + + Mitigation: inspect the bindings before and after start, and require `--replace`. Live acceptance 9 + checks it. +4. **Registry.** Leave it as is: minikube puts 5000 on a random loopback port, and it's discovered + dynamically. Pinning 5000 would collide with helm4dev. + +## NodePorts (14) + +| NodePort | Name | Service / ns | How it becomes NodePort | +|---|---|---|---| +| 30300 | grafana | grafana / observability | grafana-values.yaml | +| 30091 | prometheus | prometheus-server / observability | prometheus-values.yaml | +| 30320 | tempo | tempo / observability | tempo-values + JSON patch (index 2) in setup-observability.sh | +| 30418 | tempo-otlp | tempo / observability | same patch (index 9) | +| 30201 | kiali | kiali / istio-system | patch in setup-kiali.sh | +| 30585 | openmetadata | openmetadata / capstone | om-app-values + patch in setup-openmetadata.sh | +| 30084 | apicurio | apicurio / capstone | chart values | +| 30089 | kafka-ui | kafka-ui / capstone | chart values | +| 30080 | order | order-service / capstone | chart values | +| 30081 | gateway | keda-add-ons-http-interceptor-proxy / keda | setup-keda.sh patch, re-pinned in bootstrap | +| 30083 | notification | notification-service / capstone | chart values (scales to zero) | +| 30086 | review | review-service / capstone | chart values | +| 30087 | inventory | inventory-service / capstone | chart values | +| 30088 | ingress | istio-ingressgateway / istio-system | setup-istio.sh patch (index 1), re-pinned in bootstrap | + +These stay in-cluster: graphql-gateway (reached via the interceptor), payment, shipping, Postgres, Kafka. +helm4dev also binds 30080, 30081, 30300 and 5000, so the two profiles must never run together. A +pre-flight check makes sure every port is free. + +## Inventory / steps + +1. **Foundation.** + - `EX/demos/lib/endpoints.sh`. + - `EX/scripts/show-endpoints.sh`: a read-only table of URL, svc type/np, published, and reachable. + Exits 1 if anything is unpublished or wrong. + - 1b, in parallel: `scripts/forbidden-syntax.sh`. +2. **Machinery.** + - `setup-capstone-profile.sh`: + - source endpoints.sh; + - run the free-port pre-flight on create (`ss -Htln`); + - if `podman container exists`, run `check_published_ports` without `--replace`, so a stopped + profile is covered too; + - start with `--ports`; + - check after start; + - print the published list. + - `cluster-up.sh`: `check_published_ports`. + - `cluster-status.sh`: endpoints section. + - `teardown.sh`: drop the tunnel stop line. + - `bootstrap-capstone.sh`: `ensure_endpoint` and `NP_*`, a "show endpoints" step, a final URL block + on 127.0.0.1 NodePorts. +3. **Demos A.** order, grpc, graphql, kafka, avro, notifications, discovery, service, reviews, + add-data-product (rename `pf()` to `ep()`). The change is mechanical: + - `TP_*` to `NP_*` + - `ensure_tunnel` to `ensure_endpoint` + - `tunnel_port_for` to `endpoint_port` + - reworded step text +4. **Demos B.** openmetadata, om-lineage, observability, tracing, trace-flow, keda-http (reword the + idle-connection text), kiali, canary (`curl 127.0.0.1:30088/version`), canary-verify, + verify-grafana, walkthrough (show-endpoints; Kiali at `127.0.0.1:30201/kiali`; the stale act-1 + port-forward comment now describes the interceptor). +5. **Setup scripts and config comments.** + - Scripts: + - setup-observability; + - setup-kiali, with functional `GRAFANA_EXT_URL=http://127.0.0.1:30300`; + - setup-istio, setup-keda; + - setup-openmetadata, ingest-openmetadata; + - publish-discovery-contracts; + - scaffold-service (local Postgres container). + - Comments in the 6 chart values files, grafana-values, istio/routing.yaml, + keda/gateway-httpscaledobject.yaml, reviews_lineage.py and the 5 service config.py files. + - `services/order-service/README.md`: local Postgres on 127.0.0.1:5432. + - The 4 openshift files: contrast wording only (Routes vs published NodePorts). + - `git rm -r EX/examples/17-capstone`, a stale r0 snapshot that nothing references and that holds a + real port-forward. +6. **Delete** `EX/demos/lib/tunnels.sh` and `EX/scripts/tunnel-services.sh` (after 2–5). +7. **Docs, deck, CI.** + - Docs: + - `EX/README.md`: access model, URL table, show-endpoints, the `--replace` rule. + - `setup.html` "Accessing the tools" and `demos.html`. + - `_docs/06` line 40. + - `_docs/11`: line 10, and lines 171–176 under "Routes, not published NodePorts". + - `_docs/02`: a host-access paragraph. + - `onboarding/LESSONS-LEARNED.md`: the lessons, reframed as the reason; any remaining lines get + `forbidden-ok`. + - `_plans/decisions.md`: edit DRA-005, add **DRA-017**. + - `_plans/reconciliation.md`: reword the row, add a DRA-017 row. + - `_plans/archive/README.md`: banner. + - Deck: patch speaker notes 33 and 49 of `presentation/data-mesh-openshift/Data_Mesh_on_OpenShift.pptx` + with python-pptx (notes only), and note the post-build patch in the deck README. + - CI: + - `forbidden-syntax.sh`: scans 1–5 (wording; `TP_`/ensure_tunnel; non-loopback `--ports`; pptx + slide and notes XML; nodePort consistency). Excludes `_plans/archive/`, `*.archive.md`, + node_modules, .venv and itself. Allow marker `forbidden-ok`. + - New `.github/workflows/checks.yml` (PR and push): forbidden-syntax plus `bash -n`. + - Add the same gate to `pages.yml` before the build. +8. **Static gate.** Acceptance 1–8, then move this plan to the archive. +9. **Live re-check.** Record it in reconciliation and promote DRA-017. + +Archive rule: `_plans/archive/` and `*.archive.md` are history. Their wording isn't changed, they're +excluded from the scanner and the Jekyll build, and they're never linked as current guidance. + +## Acceptance criteria + +1. `bash scripts/forbidden-syntax.sh` prints "forbidden-syntax: OK". Negative test: a planted + port-forward line fails, and the same line with `forbidden-ok` passes. +2. A broad grep for tunnel/port-forward finds 0 lines outside the archive and `forbidden-ok` lines. +3. 0 hits for `\bTP_[A-Z]|ensure_tunnel|tunnel-services|tunnels\.sh|tunnel_port_for|_tp_resolve_ssh`. +4. tunnels.sh, tunnel-services.sh and `EX/examples/17-capstone` are gone. +5. Sourcing endpoints.sh gives 14 ports, and `node_ports_arg` has 0 items without the `127.0.0.1:` prefix. +6. Every former `ensure_tunnel` file now uses `ensure_endpoint`. +7. setup-kiali `GRAFANA_EXT_URL=http://127.0.0.1:30300`. +8. `bash -n` passes on changed scripts, and the CI checks and pages build are green. +9. After `--replace`: + - `podman port capstone` shows 14 `-> 127.0.0.1:` lines and no 0.0.0.0; + - `ss` shows them on loopback only; + - LAN-IP curl fails. +10. `EXTRA_NODE_PORTS=30999` exits 1 and names the port and `--replace`, whether the profile is + running or stopped. +11. If a port is busy, the pre-flight exits 1 before any delete. +12. `show-endpoints.sh` exits 0 with 14 rows (notification may be scaled to zero). +13. The live demo set passes, and walkthrough passes 5/5. + +## Live re-check + +1. Stop CRC. Check that datamesh and helm4dev are stopped. If less than about 30 GB is available, + quit Docker Desktop. Remove leftover ssh forwards and `/tmp/capstone-tunnel.pids`. +2. `MINIKUBE_ROOTLESS=true ./scripts/setup-capstone-profile.sh --replace`, then acceptance 9. + Stop if this fails. +3. `./scripts/bootstrap-capstone.sh`, then show-endpoints and acceptance 10. +4. Run demos: order, reviews, service inventory-service, notifications, keda-http, discovery, + observability, tracing, kiali, openmetadata, canary-verify. Curl kafka-ui. Run walkthrough. +5. `minikube stop -p capstone && ./scripts/cluster-up.sh`, then show-endpoints again. +6. Recommended full suite: canary cycle, add-data-product cycle, om-lineage, avro, graphql, grpc, + kafka, trace-flow, verify-grafana. +7. Record the results, then `minikube stop -p capstone`. + +## Risks + +- minikube with rootless podman may ignore the IP or the flag. Mitigation: the post-start hard fail. + Don't ship until it's resolved. +- An existing profile ignores new ports. Mitigation: inspect it before start and require `--replace`. +- Port collisions with helm4dev. Mitigation: the pre-flight check and one cluster at a time. +- Index-based patches can drift. Mitigation: `ensure_endpoint` checks the actual nodePort. +- Scaled-to-zero services refuse connections when idle. Mitigation: keep the wake and retry logic. +- User URLs change (3000 to 30300, and so on), and the Kiali external Grafana link has to follow. +- The pptx notes are hand-patched and drift from build-deck.js. Mitigation: the CI pptx scan. +- The broad `tunnel` rule needs `forbidden-ok` on prohibition and lesson lines. +- Memory: the profile needs 24 GB, so stop CRC and quit Docker Desktop if needed. From 9e5a0cd2cc25cc837462702f76ca3c4ff8ef6a79 Mon Sep 17 00:00:00 2001 From: PatternCatalyst Date: Thu, 8 Oct 2026 21:54:44 -0400 Subject: [PATCH 02/42] chore(plans): keep host ports; map 127.0.0.1:host:nodePort --- _plans/nodeports-plan.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/_plans/nodeports-plan.md b/_plans/nodeports-plan.md index f9f40a6..222bcd0 100644 --- a/_plans/nodeports-plan.md +++ b/_plans/nodeports-plan.md @@ -3,6 +3,21 @@ Branch `fix/nodeports-no-tunnels`. `EX/` = `examples/lgtm-datamesh/`. Planned 2026-10-08 (Opus). Move this file to `_plans/archive/` once executed (its own text would fail the forbidden-syntax gate). +## Revision 2026-10-08 (user direction) — supersedes conflicting text below + +- **Host ports do not change.** Every URL stays the same (Grafana `127.0.0.1:3000`, order `:8080`, Kiali `:20001`, …). + Publish as `--ports=127.0.0.1::` (e.g. `127.0.0.1:3000:30300`), and keep the existing + `TP_*` (host port) / `NP_*` (node port) pairs and values. Only the transport changes, so docs keep the same URLs. +- **Keep the `TP_*` variable names** to minimise churn. Their comments now read "host port published on 127.0.0.1", + and the scanner does not flag `TP_`. `ensure_tunnel` / `tunnel_port_for` still get renamed (`ensure_endpoint` / + `endpoint_port`). +- `published_ports` checks that container port `/tcp` is bound to HostIp 127.0.0.1 **and** HostPort == the mapped host port. +- **Workshops run in isolation.** Before a live run, everything else is shut down (CRC, other minikube profiles, + Docker Desktop), so there are no host-port collisions with helm4dev. Keep the free-port pre-flight as a cheap guard. +- Rootless podman minikube is proven on this host in other projects. The 127.0.0.1 binding check is a verification + step, not an expected problem. +- `EX/examples/17-capstone/`: delete vs convert is pending the user's answer. + ## Approach All 14 host-facing services are already NodePorts, so only the transport changes. The From fcd6c1f3966e1136743c32100c332410eb671788 Mon Sep 17 00:00:00 2001 From: PatternCatalyst Date: Thu, 8 Oct 2026 21:56:06 -0400 Subject: [PATCH 03/42] chore(plans): no Docker Desktop assumption; isolation rule --- _plans/nodeports-plan.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/_plans/nodeports-plan.md b/_plans/nodeports-plan.md index 222bcd0..6cd6559 100644 --- a/_plans/nodeports-plan.md +++ b/_plans/nodeports-plan.md @@ -13,7 +13,9 @@ Move this file to `_plans/archive/` once executed (its own text would fail the f `endpoint_port`). - `published_ports` checks that container port `/tcp` is bound to HostIp 127.0.0.1 **and** HostPort == the mapped host port. - **Workshops run in isolation.** Before a live run, everything else is shut down (CRC, other minikube profiles, - Docker Desktop), so there are no host-port collisions with helm4dev. Keep the free-port pre-flight as a cheap guard. + any container engine VM not used by this workshop), so there are no host-port collisions with helm4dev. +- **Host requirements are only** docker, docker compose, podman, podman compose, minikube and crc. Docs and + scripts must not assume Docker Desktop (or any other desktop app); this workshop needs only rootless podman + minikube. Keep the free-port pre-flight as a cheap guard. - Rootless podman minikube is proven on this host in other projects. The 127.0.0.1 binding check is a verification step, not an expected problem. - `EX/examples/17-capstone/`: delete vs convert is pending the user's answer. @@ -188,7 +190,7 @@ excluded from the scanner and the Jekyll build, and they're never linked as curr ## Live re-check 1. Stop CRC. Check that datamesh and helm4dev are stopped. If less than about 30 GB is available, - quit Docker Desktop. Remove leftover ssh forwards and `/tmp/capstone-tunnel.pids`. + stop any other container-engine VM. Remove leftover ssh forwards and `/tmp/capstone-tunnel.pids`. 2. `MINIKUBE_ROOTLESS=true ./scripts/setup-capstone-profile.sh --replace`, then acceptance 9. Stop if this fails. 3. `./scripts/bootstrap-capstone.sh`, then show-endpoints and acceptance 10. @@ -210,4 +212,4 @@ excluded from the scanner and the Jekyll build, and they're never linked as curr - User URLs change (3000 to 30300, and so on), and the Kiali external Grafana link has to follow. - The pptx notes are hand-patched and drift from build-deck.js. Mitigation: the CI pptx scan. - The broad `tunnel` rule needs `forbidden-ok` on prohibition and lesson lines. -- Memory: the profile needs 24 GB, so stop CRC and quit Docker Desktop if needed. +- Memory: the profile needs 24 GB, so stop CRC and every other cluster/engine VM. From e910a470c09149b73ef3efd89a3f1499fdce7836 Mon Sep 17 00:00:00 2001 From: PatternCatalyst Date: Thu, 8 Oct 2026 21:56:54 -0400 Subject: [PATCH 04/42] ci: add forbidden-syntax gate for tunnels and non-loopback ports --- .github/workflows/checks.yml | 23 ++++++++ .github/workflows/pages.yml | 3 + scripts/forbidden-syntax.sh | 108 +++++++++++++++++++++++++++++++++++ 3 files changed, 134 insertions(+) create mode 100644 .github/workflows/checks.yml create mode 100755 scripts/forbidden-syntax.sh diff --git a/.github/workflows/checks.yml b/.github/workflows/checks.yml new file mode 100644 index 0000000..bb3541d --- /dev/null +++ b/.github/workflows/checks.yml @@ -0,0 +1,23 @@ +name: checks + +on: + pull_request: + push: + +permissions: + contents: read + +jobs: + checks: + runs-on: ubuntu-latest + steps: + - name: Check out repo + uses: actions/checkout@v5 + + - name: Forbidden syntax + run: bash scripts/forbidden-syntax.sh + + - name: Shell syntax check + run: | + find examples/lgtm-datamesh/scripts examples/lgtm-datamesh/demos scripts \ + -name '*.sh' -type f -print0 | xargs -0 -n1 bash -n diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml index 5761d39..6c70745 100644 --- a/.github/workflows/pages.yml +++ b/.github/workflows/pages.yml @@ -27,6 +27,9 @@ jobs: - name: Check out repo uses: actions/checkout@v5 + - name: Forbidden syntax + run: bash scripts/forbidden-syntax.sh + - name: Set up Ruby uses: ruby/setup-ruby@v1 with: diff --git a/scripts/forbidden-syntax.sh b/scripts/forbidden-syntax.sh new file mode 100755 index 0000000..1d98f7f --- /dev/null +++ b/scripts/forbidden-syntax.sh @@ -0,0 +1,108 @@ +#!/usr/bin/env bash +# +# forbidden-syntax.sh - fail on host-access tunnels and non-loopback port publishing. +# +# Policy: host access is minikube NodePorts published on 127.0.0.1 at profile +# creation (--ports=127.0.0.1::). Tunnels break or +# disconnect, so these are forbidden in docs, examples, scripts, site pages and decks: +# SSH tunnels (ssh -L), kubectl port-forward, minikube tunnel, +# minikube service --url, any use of the word "tunnel" or the old helper +# names, and bare --ports=a:b (binds 0.0.0.0; use a 127.0.0.1: prefix). +# +# Scans (relative to ROOT_DIR, which defaults to the repo root): +# 1. case-insensitive tunnel / port-forward patterns in text files +# 2. case-sensitive legacy helper names +# 3. --ports values lacking a 127.0.0.1: prefix (shell expansions are ignored, +# they come from node_ports_arg) +# 4. the same patterns inside presentation/**/*.pptx slides and notes +# +# A line that states the prohibition (or must mention a forbidden term) carries +# the marker `forbidden-ok` (HTML comment in Markdown, trailing comment in +# shell). Excluded: _plans/archive/, *.archive.md, node_modules, __pycache__, +# .venv, _site, .git, lock files, binary files, and this script. +# Exit 1 on any hit; otherwise print "forbidden-syntax: OK". + +set -euo pipefail +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +ROOT_DIR="${ROOT_DIR:-$(cd "$HERE/.." && pwd)}" +cd "$ROOT_DIR" + +targets=() +for t in _docs onboarding examples scripts _plans _includes _layouts \ + README.md PRD.md index.html setup.html demos.html; do + [[ -e "$t" ]] && targets+=("$t") +done +for f in presentation/*/*.js presentation/*/README.md; do + [[ -f "$f" ]] && targets+=("$f") +done + +fail=0 +total=0 +report() { #