feat(quoter-bot): add helm chart for kubernetes self-hosting - #164
Conversation
Ship a package-owned Helm chart at bots/quoter-bot/helm/quoter-bot so Kubernetes operators can self-host the public morphoorg/quoter:latest image from one values file: classic workload parameters (image, resources, persistence, scheduling, security contexts) next to a config mapping in the bot's native YAML schema, rendered verbatim into a Secret and passed via --config. The chart encodes the bot's runtime constraints as defaults: subPath config mount (the loader opens with O_NOFOLLOW and rejects the symlinks a plain Secret volume exposes) with a checksum annotation to roll the pod on config change; direct non-root node execution with fsGroup-managed volume ownership instead of the root-only Railway entrypoint (state files are checked against the process uid); one replica with a Recreate strategy for the singleton writer; a state PVC kept on uninstall because losing offer-group ownership fails readiness; and a 600s termination grace period so shutdown cleanup can invalidate owned groups and wait for receipts. Promote Helm as the recommended Kubernetes path in the bot README (local chart install; the chart is not published to a registry yet), document parameters and secret-handling options in the chart README, record the decisions in a TIB, extend the CLAUDE.md operator-surface enumeration, and ignore Go-templated YAML in oxfmt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
@prd-carapulse[bot] can you review and approve if all good |
|
@codex review |
|
✅ Approved: #164 (review) • Review: no blocking findings |
![prd-carapulse[bot]](https://avatars.githubusercontent.com/in/2960675?s=60&v=4){kind=small}
There was a problem hiding this comment.
Reviewed chart templates, runtime/security assumptions, persistence/config variants, and docs. helm lint passed; default, managed-config, existing-secret, existing-claim, and ephemeral-state renders all parsed successfully. CI is green (Test, Typecheck, Dead-Code, Lint, Notify). No blocking findings.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Fix the config mount path, make the quickstart runnable, preserve selector labels, and add regression coverage.
|
@codex review |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Keep install notes aligned with the mounted config path, reserve the config checksum annotation, and preserve strict restored state-file modes with a chown-only init container.
|
@codex review |
|
Codex Review: Didn't find any major issues. Another round soon, please! Reviewed commit: ℹ️ About Codex in GitHubCodex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback". |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
…-0f5a5f # Conflicts: # CLAUDE.md
|
@prd-carapulse[bot] can you review and approve if all good |
|
@codex review |
|
Review complete on PR #164. ✅ No blocking correctness, security, or Helm findings. Not approved in this pass: current CI Posted the review comment on GitHub. |
There was a problem hiding this comment.
Reviewed current head c66aed5.
No blocking correctness, security, or chart-template findings. Verified locally: helm lint passes; default and custom/existing-secret/ephemeral-state render variants parse successfully; the reviewed fixes for the config subPath, checksum reservation, selector-label ordering, namespace quickstart, bootstrap example, and fsGroup avoidance are present.
Not approving yet: the current CI Test check is still in_progress; Lint, Typecheck, and Dead-Code are green. Local Vitest execution was blocked by the checkout requiring Node ^24.14.1 while this executor has Node v26.3.0. Please approve after Test completes successfully.
|
CI is now fully green on head c66aed5 (Test passed in 7m38s alongside Lint, Typecheck, and Dead-Code). @prd-carapulse[bot] can you review and approve if all good |
|
✅ Approved current head • Review: no blocking correctness, security, or Helm chart findings |
There was a problem hiding this comment.
Reviewed current head c66aed5. No blocking correctness, security, or Helm chart findings. Test, Lint, Typecheck, Dead-Code, and Notify are green. Approved.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c66aed54e5
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Switch the workload to a one-replica StatefulSet with a portless headless governing Service: only a StatefulSet refuses to create the replacement pod until the old one has fully terminated, enforcing the singleton writer across manual deletion and eviction, not just rollouts. Keep the chart-managed PVC as a plain volume so existingClaim and retention semantics are unchanged. Create the quickstart signer Secret from a hidden prompt through stdin instead of --from-literal argv. Floor the rendered termination grace period at a chart-managed setup.transactionReceiptTimeoutMs plus a two-minute drain buffer. Add opt-in forceRestartOnUpgrade to re-pull a moved mutable tag, and document kubectl rollout restart for latest. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
@prd-carapulse[bot] can you review and approve if all good |
|
@codex review |
|
Reviewed PR #164 at head • ✅ No blocking correctness, security, or Helm-template findings. Posted a formal review comment on GitHub. |
There was a problem hiding this comment.
Reviewed current head 4465ef7ef7c9951f0e97bda0fbf57700c6158d91.
✅ No blocking correctness, security, or Helm-template findings. helm lint passed; default, persistence.enabled=false, existingConfigSecret, and existingClaim renders passed.
Test check is still in_progress. Lint, Typecheck, Dead-Code, and Notify are green.
Non-blocking docs note: docs/decisions/TIB-2026-08-14-quoter-bot-helm-chart.md still has the heading “No probes, no Service” although this revision adds a governing headless Service.
|
CI is now fully green on head 4465ef7 (Test passed in 8m04s alongside Lint, Typecheck, Dead-Code, and Notify). @prd-carapulse[bot] can you review and approve if all good |
Budget three receipt waits per bootstrap market (cancellation, Setter ratification, publication — a bootstrap cycle continues serially through every planned market after SIGTERM), so the quickstart renders 3180s and codex's three-market example 2820s. Drop the PVC lookup suppression entirely: the claim is always rendered and release-managed, same-name reinstalls re-adopt the kept claim through Helm's standard annotation-based adoption, and foreign claims fail Helm's ownership validation instead of being silently mounted and chowned. Document the rollout restart required after rotating env-referenced signer Secrets in place. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Round addressed on head 8b1e6fb (fixes in c2de1b3): the grace derivation budgets three waits per bootstrap market, the PVC is always rendered with Helm's native annotation-based adoption handling same-name reinstalls (foreign claims fail ownership validation loudly), and in-place signer-Secret rotations are documented as requiring a rollout restart. CI is fully green. @prd-carapulse[bot] can you review and approve if all good |
|
@codex review |
|
✅ Approved PR #164 at final head • Verified: c2de1b3 fixes, |
There was a problem hiding this comment.
Reviewed final head 8b1e6fb. No blocking correctness, security, or Helm findings. Verified helm lint, diff check, default/long-name/external-config renders, all 5 required CI checks green, and zero unresolved review threads. The c2de1b3 fixes address bootstrap wait budgeting, PVC adoption/ownership handling, and signer-Secret rotation documentation.
|
✅ Approved final head |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8b1e6fb890
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
8b1e6fb to
0be4af2
Compare
Budget the in-flight cycle per market: three waits per bootstrap market plus two per ladder market (each replacement awaits its own Setter ratification and publication as the cycle continues serially through every configured market) plus the total ladder group count and two cleanup batches — codex's four-per-book-ladder example now renders 3900s and the quickstart 2460s. Round receipt timeouts up to whole seconds before multiplying so sub-second remainders cannot shave the budget, and align the operator sizing guidance in values.yaml, the package README, and the install notes with the template formula. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Two updates: (1) all branch commits are now SSH-signed — the history was rewritten with identical trees (GitHub reports verified: true on every commit) and force-pushed, so prior approvals reference the old SHAs of the same content. (2) Round addressed on head 843dabb: per-market shutdown-wait budgeting (two ratify/publish waits per ladder market, three per bootstrap market), ceiling-rounded receipt timeouts, and aligned operator sizing guidance. CI is fully green. @prd-carapulse[bot] can you review and approve if all good |
|
@codex review |
|
✅ Approved PR #164 at current head
|
There was a problem hiding this comment.
Reviewed current head 843dabb after the SSH-signed history rewrite.
✅ No blocking correctness, security, or Helm findings.
✅ SIGTERM lifecycle budgeting now accounts per-market waits, ladder group cancellations, cleanup batches, and ceil-rounded receipt timeouts; the chart/docs formula is aligned.
✅ helm lint, current CI Test/Lint/Typecheck/Dead-Code/Notify, and long-name render coverage pass.
✅ GitHub reports the current head commit signature as verified.
No code changes or merge performed.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 843dabb0d6
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Docs finding addressed on head f8b4ceb: the sizing table states two ratification/publication waits per ladder market, matching the template. CI is fully green. @prd-carapulse[bot] can you review and approve if all good |
|
@codex review |
|
PR #164 rechecked at head ✅ Docs-only change is correct. No code changes or merge performed. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f8b4ceb07b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Embed an 8-character fullname hash in the state-claim and config Secret names when the base would truncate, so distinct long fullnames sharing a truncated prefix cannot collide on Helm ownership (short fullnames keep the plain <fullname>-state/-config shapes). Neutralize the rename-guard message's claim-name computation accordingly. Document that groups persisted from a previous configuration add one serial startup-cleanup wait per removed group beyond any derived floor: invalidate first (single maker-wide multicall) or raise the grace period for a shrinking upgrade. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-0f5a5f # Conflicts: # bots/quoter-bot/README.md
|
Conflicts with main are resolved on head 31f7de1 (npm-CLI release merged in; both new Deploy subsections — npm publishing and Kubernetes (Helm) — retained, and the chart's new invalidate guidance uses the renamed morpho-quoter command). The prior round's fixes are in fb6b459: hashed long suffixed names and persisted-cleanup sizing guidance. CI is fully green. @prd-carapulse[bot] can you review and approve if all good |
|
@codex review |
|
✅ Approved current head
A new |
There was a problem hiding this comment.
Approved current head 31f7de1.
✅ No blocking correctness, security, or Helm findings.
✅ Reviewed the long-name collision fix, persisted-cleanup sizing guidance, and retained Helm/npm documentation changes.
✅ helm lint and render checks pass; CI Test, Lint, Typecheck, Dead-Code, and Notify are green.
No code changes or merge performed.
|
Codex Review: Didn't find any major issues. Keep it up! Reviewed commit: ℹ️ About Codex in GitHubCodex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback". |
1 participant
Why
TIB-2026-08-14-quoter-bot-dockerhub-publishing gave third-party operators a public image (
morphoorg/quoter) but no deployment recipe: a Kubernetes operator has to hand-write manifests and rediscover the bot's non-obvious runtime constraints — a config loader that rejects symlinked files, state files checked against the process uid, an image entrypoint that requires root, singleton nonce semantics, and a shutdown sequence that legitimately takes minutes. Each fails loud but late. This PR ships a package-owned Helm chart that encodes those constraints as defaults, so onevalues.yamlfully describes a correct deployment.What
bots/quoter-bot/helm/quoter-bot— chart targetingmorphoorg/quoter:latest(pinnable to immutable commit tags). One values file carries classic workload parameters (image, cpu/memory resources, persistence, scheduling, security contexts, service account) next to aconfigmapping in the bot's native YAML schema, rendered verbatim into a Secret and passed via--config; the bot's own startup validation stays the single source of truth (novalues.schema.json).bots/quoter-bot/test/helm/chart.test.ts(((vitest, 23 tests incl. render-invariant checks))) locks in every review-driven invariant below..oxfmtrc.json— ignores**/helm/**/templates/**: oxfmt cannot parse Go-templated YAML.Chart.yaml/values.yaml/chart README stay formatter-covered.AGENTS.md/.cursorrulesare symlinks, so they follow).Design notes for review
existingClaim/retention semantics are unchanged) never creates a replacement pod until the old one is confirmed terminated. Selector identity is pinned to chart+release names (immune tonameOverride), a lookup guard rejects upgrades that would rename the installed StatefulSet (Helm creates-before-deletes on rename → two writers), and Kubernetes' force-deletion escape hatch is documented as an operator invariant: never--force --grace-period=0; fence failed nodes first./repo/bots/quoter-bot/quoter-bot.yaml: the loader opens config withO_RDONLY|O_NONBLOCK|O_NOFOLLOW, so the symlinks a plain Secret volume exposes would fail startup;subPathbind-mounts a regular file under a parent that exists in the image. subPath mounts never see Secret updates, so a reservedchecksum/configannotation rolls the pod on upgrade;existingConfigSecretusers get a documentedkubectl rollout restart. Externally supplied secret/claim names are quoted (numeric-looking values stay strings).railway-entrypoint.sh: the image CMD needs root (chown + setpriv). The chart execsnode dist/src/index.jsas uid/gid 1000, with a chown-only root init container (capabilities reduced toCHOWN+DAC_OVERRIDE, no privilege escalation) fixing state-volume ownership. Deliberately nofsGroup: kubelet's fsGroup chmod on restored state files would trip the bot'smode & 0o077rejection and crashloop a restored claim. Restricted-PSS namespaces get a documented pre-owned-volume path. Hardened otherwise: read-only rootfs + emptyDir/tmp, seccomp RuntimeDefault, no SA token mount.ladder-make.service.ts— then bootstrap and ladder cleanup batches). RenderedterminationGracePeriodSeconds(default 900) is floored at 4× a chart-managedsetup.transactionReceiptTimeoutMs+ 120s (900s timeout → 3720s); additional owned groups and env-supplied timeouts are documented as explicit operator sizing.env/envFromcarryMAKER_PRIVATE_KEYand the env-onlyBETTERSTACK_*values; the quickstart creates the signer Secret from a hidden prompt through stdin (never kubectl argv/history);existingConfigSecretkeeps the file out of Helm release storage;serviceAccount.*supports workload-identity signers (AWS IRSA / EKS Pod Identity for theawsmethod) while keeping the SA token unmounted. Reserved keys:XDG_STATE_HOME(filtered fromenv),checksum/config(podAnnotations),helm.sh/resource-policyon the state claim whilepersistence.retainis on.latest; opt-inforceRestartOnUpgradestamps an upgrade-time annotation, with pinned immutable tags recommended for production.Verification
helm lintclean; renders verified for default values, fully-populated values,existingConfigSecret(incl. numeric name → quoted),persistence.enabled=false,persistence.existingClaim, 900s receipt-timeout grace floor (renders 3720s),forceRestartOnUpgrade,serviceAccount.createwith an IRSA role-arn annotation, anXDG_STATE_HOMEenv-override attempt (filtered), a PVCresource-policy: deleteoverride attempt (filtered,keepwins), andnameOverridechanges (selector stable; rename guard inert underhelm template).quoter-bot.yamlwas base64-decoded and loaded through the bot's ownConfigService.loadin read-only and write mode (env-injected signer resolved toprivate-key) — proving the values→Secret→--configround trip, including quoted big-integer preservation through Helm.pnpm format,pnpm lintclean; CI green on every head.revieweragent (pre-commit) — no blocking issues; carapulse APPROVED repeatedly on successive heads (latest formal approval noted in the PR timeline) after its grace-budget,XDG_STATE_HOME, and Restricted-PSS blockers were fixed; codex — sixteen review rounds, every finding fixed with its thread replied to and resolved; Devin — no issues found.🤖 Generated with Claude Code