Skip to content

feat(ladder): phase 3 - pipeline (schema, scheduled task, approval/auto modes, UI, terraform) #70

Open

Description

@cristim

Part of LeanerCloud/cloud-commitments-go#27 (commitment laddering automation). Phase 3 of 4: persistence, scheduled task, approval/auto modes, UI, Terraform. All flag-gated default-off.

PRs:

  1. Migrations 000078-000080 + stores: ladder_configs (per cloud_account x provider), ladder_runs (immutable audit: inputs snapshot, plan jsonb with rationales, run-level approval token, status CAS), ladder_tranches; purchase_executions.ladder_run_id + ri_exchange_history.ladder_run_id link columns; global_config.laddering_enabled kill-switch; PurchaseSourceLadder = "cudly-ladder"; store methods in internal/config/store_postgres_ladder.go.
  2. Scheduled task, plan-only: ladder_run task type + dispatch in internal/server/handler.go; handler_ladder.go with cadence self-gate (daily <20h / weekly <6d20h slack); persists planned runs + tranches, applies guardrails; no execution or email yet (safe dry-run milestone).
  3. Email-approval mode: pending purchase_executions + pending reshapes linked to the run; run-level token fan-out approve/cancel endpoints (3-mode auth dispatch, fail-closed); SendLadderRunSummary (SES targeted only, ErrTokenInBroadcast guard); frontend/src/ladder-deeplink.ts.
  4. Auto-approve mode: status=scheduled purchases with fire_at = now + PurchaseDelayHours riding the existing FireScheduledDelayedPurchases sweep (minimally extended for ladder-linked reshapes); pre-notification + SendLadderRunCompleted confirmation.
  5. Settings UI + config API: per-account laddering config editor in settings.ts + /api/ladder/configs CRUD.
  6. Run history/detail UI + Terraform: /api/ladder/runs list/detail (tokens stripped), run detail view with planned-vs-executed actions and rationales; EventBridge rule/target/permission behind enable_ladder_run_schedule (default false), vars threaded through environments/profiles.

Activity

  1. cristim commented on Jul 3, 2026

    @cristim
    MemberAuthor

    Scope note from the phase-1/2 reviews: the engine (#1340) sizes allocations in account-wide $/hr per layer, but executing a purchase needs concrete instrument targeting - an EC2 Instance SP purchase requires family+region, a Convertible RI requires instance type/platform/tenancy/scope (the write side LeanerCloud/cloud-commitments-cli#1349 hard-requires these in the Recommendation). The pipeline's plan-assembly step must translate each layer allocation into concrete recommendations (likely via the existing CE GetReservationPurchaseRecommendation / GetSavingsPlansPurchaseRecommendation parsers, scaled to the tranche amount via common.ScaleRecommendationCosts) before creating purchase_executions. This is the largest engine-to-pipeline seam; budget it explicitly in the ladder_run task PR.

  2. cristim commented on Jul 16, 2026

    @cristim
    MemberAuthor

    Phase-3 PR-2 (#1362, the plan-only ladder_run scheduled task) is merged to main (7adacd715). Now on main: run/tranche store layer with single-tx persistence and typed-status CAS, the ladder_run task with kill-switch + fail-closed cadence gating + fail-loud account/region resolution, plan-only executeLadderRun, and the capability factory seam. In flight toward this issue: LeanerCloud/cloud-commitments-cli#1365 (stale-baseline guard, rebasing on the new factory), LeanerCloud/cloud-commitments-cli#1366 (truncate guardrail), LeanerCloud/cloud-commitments-cli#1368 (scoped reshape cancellation), LeanerCloud/cloud-commitments-cli#1369 (EventBridge ticks). Launching next: L2 (wire real CE/SP/RI read adapters into the factory so runs produce real plans), then the execution chain (write-side wiring, email-approval, auto-approve, tranche firing, reaper, alerting) per the full-automation plan.

  3. cristim commented on Jul 16, 2026

    @cristim
    MemberAuthor

    L2 merged via LeanerCloud/cloud-commitments-cli#1378: the capability factory is now wired to real Cost Explorer / Savings Plans / RI data - scheduled ladder_run invocations produce real plans end-to-end (plan-only). Chain continues with L5 (net in-flight tranches into the gap) and L6 (write-side wiring behind a kill-switch).

  4. cristim commented on Jul 16, 2026

    @cristim
    MemberAuthor

    Phase-3 progress: the ladder write-side wiring landed via LeanerCloud/cloud-commitments-cli#1396 (merged) behind a default-off ladder_execution_enabled kill-switch (fail-closed; spend provably impossible when off). This is the L6 milestone of the pipeline; the umbrella stays open for the remaining phase-3 items (approval/auto-approve modes, settings + run-history UI, terraform EventBridge tick). L5 convergence fix (#1397) is in adversarial review.

  5. cristim commented on Jul 16, 2026

    @cristim
    MemberAuthor

    Phase-3 progress: L5 tranche-netting convergence fix landed via LeanerCloud/cloud-commitments-cli#1397 (merged). The ladder engine now holds compute at the target coverage without the drift-up under-commit. Combined with the L6 kill-switch (#1396), the plan-only + gated write-side pipeline is in place. Remaining phase-3: approval/auto-approve modes, settings + run-history UI, terraform EventBridge tick. Hardening follow-ups: LeanerCloud/cloud-commitments-go#36.

  6. cristim commented on Oct 8, 2026

    @cristim
    MemberAuthor

    Current-main audit at f2d60df86209eea85d81e9091c20832bd670a65f confirms this remains a planning-only feature across the full UI/API/task path:

    Verification: go test ./internal/server -run 'Ladder|Tranche' -count=1 passed on an exact archived live-main tree (mock boundary tests, no cloud calls or purchases). The fake write methods in handler_ladder_test.go:89-102 fail the test if a write is attempted, confirming those tests intentionally protect the plan-only milestone rather than exercise an executor.

    Before enabling an executor, preserve these distinctions in the UI/API: planned vs scheduled vs executed, provider/service support, deployment-account gate, due vs elapsed tranches, updated usage/headroom and current permissions. New independently reproduced read-side issues from this audit are shared-go #282 (residual baseline subtracts existing twice) and platform #660 (accepted 1-6 day lookbacks cannot satisfy AWS baseline's 7-point minimum). No duplicate pipeline issue was opened.

  7. cristim commented on Oct 17, 2026

    @cristim
    MemberAuthor

    Owner decision (2026-10-17): this issue moves from the ladder sessions to the platform implementation driver. #118 is taken first as a concrete fix; #70 is planned as an epic and split into slices of at most 400 lines.

  8. 67 remaining items

  9. cristim commented on Nov 16, 2026

    @cristim
    MemberAuthor

    Part of #70, landed by claude-platform-2 (two review passes, CI watched): S7d convertible RI instrument builder as c0bf222 (revoke notice states reservation spend), 6bef00d (builder + registry: per stored 1y no-upfront regional EC2 shape, share by weight, count = floor(share/unit) with an exact count*unit <= share guard, too-small shapes buy nothing and are not redistributed, any bad price fails the whole tranche) and be07020 (pre-claim pricer: 1y no-upfront convertible offering, refuses nil/other type/non-USD/upfront/other term or payment). The production pricer factory (Manager.LadderRIPricer) and RIPricer in handleLadderFire are NOT wired: the sweep is unregistered until S14, and with a nil pricer RI stays unresolved. Also eab1885 (#17 log detail). Free next: S13 (run detail), S16a/b, S1, S14 wiring (claude-platform).

  10. cristim commented on Nov 16, 2026

    @cristim
    MemberAuthor

    Part of #70, landed by claude-platform-2 (two review passes each): S13 backend half b1f656a (each run in GET /api/ladder/runs carries tranche_counts {scheduled, fired, completed, failed, canceled} and delayed_first_step = the run's earliest tranche is scheduled later than the fire grace after started_at; correlated subselects in the existing query, no N+1, read-only, no migration; the review added a missing scoping test for the delayed flag), S1 9622fd5 (ladder help text no longer promises automatic buying; Azure/GCP disabled for new configs, stored rows keep their provider). Backend follow-up noted: PUT still accepts azure/gcp configs (the fire gate refuses non-AWS, so it is an honesty gap, not a money risk). Free next: S13 UI half (run detail rendering of the counts), S16a/b (reshape), S14 production wiring incl. the RI pricer factory (claude-platform).

  11. cristim commented on Nov 16, 2026

    @cristim
    MemberAuthor

    Correction by claude-platform-2: the previous comment listed S1 as landed as 9622fd5, but that push was rejected (a concurrent landing moved main) and my landing script printed PUSHED without checking the push result. S1 is NOT on main yet; I am re-landing it now. The script now checks the push exit code and retries. All other SHAs I reported were verified to be ancestors of origin/main.

  12. cristim commented on Nov 16, 2026

    @cristim
    MemberAuthor

    Part of #70, landed by claude-platform-2 (two review passes each, CI green): S1 18cfe70 (ladder help text no longer promises automatic buying; Azure/GCP disabled for new configs, stored rows keep their provider; this replaces the S1 SHA I wrongly reported earlier), S13 UI half 80c147b (tranche counts and a Delayed first step note in the run detail), and c5f8545 (PUT /api/ladder/configs refuses creating a non-AWS config with a 400; existing rows stay editable; lookup error fails closed). Remaining in #70: S16a/b (reshape), S14 production wiring incl. the RI pricer factory (claude-platform), S15 after S14. Open follow-ups: none for this lane beyond those.

  13. cristim commented on Nov 16, 2026

    @cristim
    MemberAuthor

    Part of #70, landed by claude-platform-2 (two review passes, both split prefixes built and linted clean): S16a as 3dfcc76 (LatestLadderRunID store query + real-Postgres test + fail-closed mock) and 55892d4 (ladderReshapeConfig: caps via the library's RequireCap, LadderRunID always set so the standalone task's pendings are never cancelled, DryRun false, threshold/lookback checked; ladderReshapeEligible: typed refusal per gate incl. an allow-list of latest-run statuses planned/approved/executing/completed, which also refuses the legacy 'cancelled' spelling the first draft would have accepted). Unregistered, nothing wired. Next: S16b (runLadderReshape, the only caller passing true to wireLadderWriteSide): it MUST also (1) fix exchangeRunnerAdapter.run (ladder_write.go:62): the go ReshapeBuffer always passes Mode=auto and the adapter only downgrades via fourEyesExchangeMode, so with the real write side and four-eyes off exchanges would execute immediately even if the global RIExchangeMode is manual (Default 2 requires manual unless the global mode is auto), and (2) enforce the require_different_approver refusal like the fire gate (ladder_fire_gate.go:59). S14/S15 stay with claude-platform.

  14. cristim commented on Nov 16, 2026

    @cristim
    MemberAuthor

    Part of #70, landed by claude-platform-2 (two review passes, every prefix built/vetted/linted): S16b as f6f3dab (the ladder exchange adapter now runs manual unless the global RI exchange mode is auto AND the request is auto, then applies the four-eyes downgrade: before this, the go ReshapeBuffer always requested auto and only four-eyes downgraded it, so with the real write side an exchange would have executed immediately even in global manual mode; the adapter sends the pending-approval email after a successful run; the reshape gate refuses while RequireDifferentApprover is on, deliberately stricter than the plan's downgrade), 8ecc442 (the unregistered runLadderReshape sweep: eligibility gate, ReshapeBuffer once per config, PurchaseLayer never, at most one reshape per account+region per sweep because each reshape cancels the other ladder configs' pending exchanges (found by review), per-config errors do not stop the sweep, account ids redacted from logs) and 4536476 (AST guard: wireLadderWriteSide only from ladder_write.go, ladderCapability with a non-false flag only from the reshape sweep, so planning can never execute). Nothing registers or schedules the sweep. Remaining for the owner/claude-platform: S14 (register ladder_fire + production RI pricer factory in handleLadderFire) and S15 (register ladder_reshape: scheduler/roster/Terraform; the caller must resolve the process account via STS and the region; only one sweep at a time, there is no lock in the sweep).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions