Repository navigation
Conversation
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 39317783 | Triggered | Generic Password | 07b36c5 | pkg/adopt/milvus_test.go | View secret |
| 39317783 | Triggered | Generic Password | 07b36c5 | pkg/adopt/milvus_test.go | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secrets safely. Learn here the best practices.
- Revoke and rotate these secrets.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
58e6c83 to
74f49bc
Compare
Adopting a Distributed Milvus 2.6 that runs from a Helm chart was a manual procedure: gather the effective config, write three Secrets and a staged Milvus, diff the rendered config, then scale each role by hand. The new command group does it: - plan RELEASE: read-only. Finds the release's workloads (by app.kubernetes.io/instance and the component label, or --workload), merges the mounted config files in Milvus' load order, finds the client Service and the MilvusVersion for the running image, and checks the WAL (Woodpecker on object storage), etcd, object storage and the root password, logging in to a running proxy through a port-forward (Milvus applies a root password only on first start). --output-dir writes the objects for review. - run RELEASE: creates the Secrets, a state ConfigMap <name>-adopt and the Milvus with every role at 0 (orphan-on-delete, DoNotTerminate, external etcd with the current root path, autoOps off, write check off), removes a chart Service on a name KubeDB needs headless, and stops if the config the operator rendered differs from the running one on storage, etcd root path, WAL, security or ports. Then hands the roles over, overlap (default; one QueryNode at a time) or stop-start, recording each one; re-running resumes. - status RELEASE: where each role runs. - rollback RELEASE: deletes the Milvus with orphan-on-delete, moves each role back to the chart starting the chart's pods first, restores the client Service's selector, labels and ports, and with --cleanup deletes what KubeDB left and recreates the removed chart Services. --name covers a Milvus adopted by hand. Several workloads of one role become node groups (e.g. QueryNodes on two GPU pools). Tested end to end on a cluster running the upstream chart 5.0.19 (Milvus 2.6.15): hand-adoption rollback, plan, overlap run, state rollback, stop-start run killed and resumed; data bit-exact after each. Signed-off-by: sabnaj <sabnaj@appscode.com>
74f49bc to
07b36c5
Compare
For a chart without the upstream labels, plan and run need about fifteen flags (a --workload per workload, the secrets, the timing), repeated on every command. All four commands now take -f FILE, kind MilvusAdoption (apiVersion cli.kubedb.com/v1alpha1): release, namespace, name, version, source (selector, roleLabel, workloads with node groups, configFile), auth, restrictedSCC, handover (method and timing) and rollback (cleanup). - A flag given on the command line wins; the file fills only the flags that were not given, through the flag set, so the parsing is shared. - RELEASE may come from the file; an argument that differs is refused. The file's namespace is used unless -n is given. - Unknown keys are errors, and --yes is never read from the file. - plan --output-dir also writes adopt.yaml with what plan found, ready for run -f. Tested on the cluster: rollback, plan and run driven by the file alone. Signed-off-by: sabnaj <sabnaj@appscode.com>
Signed-off-by: sabnaj <sabnaj@appscode.com>
Adds
kubectl dba adopt milvus, which adopts a Distributed Milvus 2.6 that runs from a Helm chart (or any Deployments/StatefulSets) under KubeDB in place: same etcd and root path, same bucket, same root credentials, same client Service. It replaces the manual procedure of the adoption guide (gather the effective config, write three Secrets and a staged Milvus, diff the rendered config, scale each role by hand).plan RELEASE(read-only): finds the workloads (app.kubernetes.io/instance=RELEASE+componentlabel, or--selector/--role-label/--workload role[/group]=Kind/name), merges the mounted config files in Milvus' load order, finds the client Service and the MilvusVersion of the running image, checks the WAL (Woodpecker on object storage), etcd, object storage, and logs in to a running proxy with the root credentials through a port-forward (the API server's service proxy drops the Authorization header). Milvus applies a root password only on first start, so a changeddefaultRootPasswordis caught here.--output-dirwrites the objects for review.run RELEASE: creates the Secrets, a state ConfigMap<name>-adoptand the Milvus with every role at 0 (orphan-on-delete,DoNotTerminate, external etcd,autoOpsoff, write check off); deletes (and records) a chart Service on a name KubeDB needs headless; waits for the operator and stops if the rendered config differs from the running one on storage, etcd root path, WAL, security or ports. Then hands the roles over:--method overlap(default; KubeDB's pods first, QueryNodes one at a time) orstop-start. Each role is recorded; re-running resumes.status RELEASE: where each role runs.rollback RELEASE: deletes the Milvus with orphan-on-delete, moves each role back to the chart (chart pods first), restores the client Service's selector, labels and ports;--cleanupdeletes what KubeDB left and recreates removed chart Services.--namehandles a Milvus adopted by hand.Options in a file (
-f): every command also takes-f FILE(kindMilvusAdoption, apiVersioncli.kubedb.com/v1alpha1) with release, namespace, name, version, source (selector, roleLabel, workloads with node groups, configFile), auth, restrictedSCC, handover (method and timing) and rollback (cleanup). Flags given on the command line win; unknown keys are errors;--yesis never read from the file.plan --output-dirwritesadopt.yamlwith what it found, ready forrun -f. Tested on the cluster: rollback, plan and run driven by the file alone (T6–T8), data PASS.Several workloads of one role become node groups (e.g. QueryNodes on two GPU pools). Objects are handled unstructured, so no apimachinery bump is needed.
Tests: unit tests for config merge/extraction, carried config, rendered-config comparison, workload/group assignment, rendering, version selection, swap steps, Service record. End to end on a k3d cluster running the upstream Milvus chart 5.0.19 (Milvus 2.6.15; Proxy 3, MixCoord 2, DataNode 2, QueryNode 6, StreamingNode 2; 3-member etcd; Woodpecker on S3; auth on), with a client probe and a bit-exact data check after each phase:
--name)--workload)--cleanupKnown: with overlap, a 30-60 s stall of searches and inserts was seen at the first/last QueryNode swap while chart and KubeDB QueryNodes ran side by side (not seen when adding/removing a QueryNode on the chart alone); a rollback pauses writes ~30 s while StreamingNodes move back to the chart. Both are documented in the guide (kubedb/docs#1086).
Related: kubedb/milvus#85, kubedb/apimachinery#1947, kubedb/docs#1086.