Repository navigation
fix(controller): honor cancellation during reconciliation - #117
Merged
Merged
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
amuraru
added a commit
that referenced
this pull request
Nov 18, 2026
Keep the health lifecycle test compatible with the reconciler-local ZooKeeper client factory introduced by #117. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
amuraru
added a commit
that referenced
this pull request
Nov 18, 2026
Update the health-test fixture to use the reconciler factory introduced by #117. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
amuraru
added a commit
that referenced
this pull request
Nov 18, 2026
…112) * fix(controller): skip unchanged ConfigMap and ServiceAccount writes Guard owned data before existing updates while preserving conflicts and subsequent StatefulSet reconciliation. Cover no-ops, changes, removals and real API-server conflict retries. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(operator): adapt health envtest to client factory Keep the health lifecycle test compatible with the reconciler-local ZooKeeper client factory introduced by #117. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change log description
Purpose of the change
Implements PR-11 under #101. This does not close the broader workstream.
Health probes remain separate in #116. No storage, write-convergence, Helm retry, or worker-concurrency changes are included.
What the code does
Threads
context.Contextthrough controller helpers and every Kubernetes request. Request logging usesctrl.LoggerFrom(ctx)instead of assigning the shared reconciler logger.Changes all
ZookeeperClientoperations exceptClose()to accept context, and replaces the shared client withNewZkClient func() zk.ZookeeperClient. Updates controller mocks, the operator entry point, YAML-exporter constructor, e2e manager constructor, and tagged integration callers. Each metadata initialization or scale-metadata scope creates and closes its own session.Adds a named
zk.OperationTimeoutof 30 seconds. DNS lookup, session readiness, and node operations honor cancellation; the controller shares one deadline across each metadata scope. The session timeout remains five seconds. A context-aware dialer retains the owned transport so cancellation can interrupt an incomplete handshake, which upstreamClose()alone does not interrupt promptly. Cleanup is idempotent and waits for the upstream session loop and cancellation callback to finish.Regression tests cover pre-canceled work without new mutations, blocked Kubernetes and ZooKeeper work, a real TCP peer that never completes session readiness, the earlier-parent and default 30-second deadline boundaries, stopping child-node creation after cancellation, and independently keyed reconciliations with isolated session ownership.
Cancellation does not roll back a mutation already accepted by a server. Upstream
Close()can add up to its one-second close-request wait while cleanup completes. The only expected overlap with #116 is the nearby reconciler constructor inmain.go; this PR changes its client factory, not manager health configuration.How to verify it
Red before production changes:
go test ./controllers ./pkg/zk -run '^(TestCanceledReconcileStopsClientWork|TestZookeeperOperationDeadline)$' -count=1failed as intended: a pre-canceled reconcile made one update, a blocked helper ignored cancellation, and a connected-but-never-ready ZooKeeper session exceeded its parent deadline.Passed locally on the final implementation:
make testincludes race detection; the changed-code linter reported 0 issues, matching CI's only-new-issues policy. The unrestrictedgolangci-lint run ./...still reports existing legacy findings outside this change.Passed manual shutdown e2e on disposable Kubernetes 1.34.3: deployed the image built from this worktree using the existing e2e manifests, and routed a scale operation to an owned TCP server that accepted connections but never completed a ZooKeeper handshake. Scaled the operator Deployment to zero during that blocked operation (
kubectl scale deployment/zookeeper-operator --replicas=0, followed bykubectl wait --for=delete pod/<operator-pod> --timeout=10s). The blocked reconciliation returnedcontext canceled, the manager loggedAll workers finished, the peer observed socket closure, and shutdown completed in 0.53 seconds. The StatefulSet remained at three replicas rather than applying the requested five. The disposable cluster was removed.The full existing Kubernetes e2e suite was not run locally; the
e2elabel requests it in GitHub Actions. No Java toolchain commands were needed.