executor/k8s: create a dedicated ServiceAccount for job pods - #951
Open
marcleblanc2 wants to merge 2 commits into
Open
marcleblanc2 wants to merge 2 commits into
marcleblanc2 wants to merge 2 commits into
Conversation
Job pods spawned by the Kubernetes executor ran as the namespace default ServiceAccount with its token automounted, sharing an identity with every other workload in the namespace. Add executor.kubernetesJob.serviceAccount values, render an sg-executor-job ServiceAccount with no RBAC bindings and automountServiceAccountToken: false, and pass the name and automount setting to the executor via KUBERNETES_JOB_SERVICE_ACCOUNT_NAME and KUBERNETES_JOB_AUTOMOUNT_SERVICE_ACCOUNT_TOKEN (sourcegraph/sourcegraph#16392). The ServiceAccount is rendered only when executor.configureRbac is true, so a second executor release in the same namespace does not try to own it. Amp-Thread-ID: https://ampcode.com/threads/T-01a0e96e-01bb-762e-a752-d2a3103593f0 Co-authored-by: Amp <amp@ampcode.com>
A pod can only reference a ServiceAccount in its own namespace, so the job ServiceAccount must live where the job pods run, not in the release namespace. Amp-Thread-ID: https://ampcode.com/threads/T-01a0e96e-01bb-762e-a752-d2a3103593f0 Co-authored-by: Amp <amp@ampcode.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Paired with sourcegraph/sourcegraph#16392
defaultServiceAccount, with its token automounted, so they share an identity with other workloads in the namespacecharts/sourcegraphdeployment: gitserver, searcher, indexed-search, blobstore, redis, the code intel workers, syntect-server, and the OTel and node-exporter podsAuthorizationPolicy) on the job pods, and hands untrusted batch change / auto-indexing code a Kubernetes API token it shouldn't haveThis change:
executor.kubernetesJob.serviceAccount.{create,name,annotations,automountToken}valuessg-executor-jobServiceAccount by default, with no RBAC bindings andautomountServiceAccountToken: falseon the ServiceAccount object itself, so even executor images that predate sourcegraph/sourcegraph#16392 stop mounting the token once they run as this SAKUBERNETES_JOB_SERVICE_ACCOUNT_NAMEandKUBERNETES_JOB_AUTOMOUNT_SERVICE_ACCOUNT_TOKENthrough the executor ConfigMapdefaultuntil the image is upgradedDesign notes:
executor.configureRbacas well asserviceAccount.create, matchingsg-executorand its Role/RoleBindingbatchesnext tocodeintel) already setsconfigureRbac: falseand will not fight over ownership of the SAexecutor.namespace(explicitmetadata.namespace), since a pod can only reference a ServiceAccount in its own namespacesg-executorRole and RoleBinding render in the Helm release namespace, so the controller can only create jobs in another namespace if RBAC there is configured out of band; that is pre-existing and not changed hereserviceAccount.name: ""opts out entirely and falls back to the namespacedefaultSA, preserving today's behaviourTest plan
helm unittest charts/sourcegraph-executor/k8sconfigureRbac: false,create: falsewith an existing name, empty name, a non-releaseexecutor.namespace, annotations, andautomountToken: truehelm lint charts/sourcegraph-executor/k8squeueNamewarninghelm template sourcegraph-executor charts/sourcegraph-executor/k8s -n sourcegraph --set executor.queueName=codeintelrenders the new ServiceAccount and the two ConfigMap keyshelm-docsregeneratedcharts/sourcegraph-executor/k8s/README.mdFollow-ups
examples/network-policyto key on the new ServiceAccountChangelog
The
sourcegraph-executor/k8schart now creates a dedicatedsg-executor-jobServiceAccount for executor job pods, with no RBAC bindings and no token mount, configurable underexecutor.kubernetesJob.serviceAccount