Repository navigation
Description
Delivery bundle: C: Convergence, APIs, and gated scalability. See #395 and #368 for the four-PR stack.
Parent: #393
Roadmap: #368
Plan item: PR-22
Priority: P3
Dependencies: PR-01, PR-03, PR-07, PR-10
Kubernetes support: Rolling upstream-maintained minors; use the oldest/newest boundaries pinned in config/kubernetes-versions.env. See #368 for the current policy. Unrun jobs are not passed validation.
Problem Statement
Evidence: CruiseControlOperationReconciler stores mutable scaler state on the reconciler (controllers/cruisecontroloperation_controller.go:72) and schedules across multiple CRs for one external cluster.
Proposed Solution
Files: Cruise Control operation/task controllers, scale adapters, manager/controller options, relevant tests and HA/restart scenarios.
Interface contract: The scaler is reconcile-local. Serialize scheduling by KafkaCluster identity, not merely operation namespace/name; different operation keys for the same cluster must not submit conflicting actions. Persisted state and PR-01 recovery remain authoritative across restart/failover.
- Before enabling concurrency, define the per-cluster serialization design and how existing operation requests map to it. Do not rely solely on an in-memory lock for restart safety.
- Add two-cluster parallelism and same-cluster/different-operation-key tests; assert no scaler cross-contamination or simultaneous incompatible submissions.
- Add submit-then-crash/restart and leader-change tests using available Cruise Control task recovery. Document any residual external acknowledgment ambiguity.
- Run race tests on affected controllers/adapters. Raise concurrency only for independent clusters after the tests pass.
- Validate
go test -race ./controllers ./pkg/scale, controller envtest, Kind HA/restart tests, and PR-07 throughput/fairness measurements.
Done when: Independent clusters can progress concurrently without weakening single-cluster execution safety.
Delivery: Four consolidated PRs, approved November 16, 2026: A -> B -> C; D branches from A. Foundation is #395. The 22 PR-XX identifiers are work-item IDs, not separate PRs. Preserve every work-item acceptance criterion and native dependency. No gated experiment is automatically enabled, and no issue is complete merely because one PR references it. D must satisfy its native #375 gate, including lifecycle-domain signals assigned to B; branching from A does not waive that merge prerequisite. Write focused regression tests first; record exact commands/results and run make check plus this issue's real-cluster/measurement gates.
Alternatives Considered
Twenty-two individual PRs add unnecessary integration overhead, while one omnibus PR obscures correctness and rollout boundaries. Deliver this scope in its approved dependency-aligned bundle. Do not expand into deferred consistency, storage, or concurrency changes without their own decision gates.
Additional Context
Baseline: 95138e0d; detailed roadmap scope: #368, section PR-22. The roadmap #368 and its native sub-issues are the planning source of truth. No planning document is committed in the final PR diffs. Re-read affected source and overlapping work before implementation. Historical baseline test passes are not validation of this change.
Related historical context: #301 (closed) covers Cruise Control restart races. Preserve its resolution and avoid duplicating that narrower fix.
Activity
amuraru commentedon Nov 17, 2026
Concurrency increases are removed from active scope under #368: no demonstrated throughput bottleneck justifies new serialization/recovery machinery. Existing execution safety stays in #391; no worker increase claimed.