diff --git a/braintrust/README.md b/braintrust/README.md index 435347c..c4db5bd 100644 --- a/braintrust/README.md +++ b/braintrust/README.md @@ -18,6 +18,16 @@ The `braintrust-secrets` secret must contain the following keys: | `GCS_ACCESS_KEY_ID` | Google HMAC Access ID string | Valid S3 API Key Id (only required if `cloud` is `google` and if `enableGcsAuth` is `false`) | | `GCS_SECRET_ACCESS_KEY` | Google HMAC Secret string | Valid S3 Secret string (only required if `cloud` is `google` and if `enableGcsAuth` is `false`) | +## Optional: Loop Runtime + +Set `loopRuntime.enabled: true` to run Loop Runtime on GKE (`cloud: google`). +On AWS it runs on ECS through the Terraform module. Sandboxes run on Sandbox +Service workers in a separate GKE cluster; see +[examples/gke-loop-runtime](examples/gke-loop-runtime/README.md). The Terraform +outputs it needs come from +[terraform-google-braintrust-data-plane#28](https://github.com/braintrustdata/terraform-google-braintrust-data-plane/pull/28), +which is not merged yet. + ## Azure Key Vault Driver Integration If you're using Azure, the Azure Key Vault CSI driver is default enabled and will automatically sync secrets from Azure Key Vault into Kubernetes. This eliminates the need to manually create and manage the `braintrust-secrets` Kubernetes secret. diff --git a/braintrust/examples/gke-loop-runtime/README.md b/braintrust/examples/gke-loop-runtime/README.md new file mode 100644 index 0000000..b24af86 --- /dev/null +++ b/braintrust/examples/gke-loop-runtime/README.md @@ -0,0 +1,56 @@ +# Loop Runtime on GKE + +Loop Runtime runs in the primary GKE cluster. Its sandboxes run on Sandbox +Service workers in a separate isolated cluster. Both clusters come from the +GCP Terraform module with: + +```hcl +gke_isolated_workers = { + network_cidr = "10.64.0.0/16" # must not overlap existing ranges + discovery = { dns_name = "sandbox..internal" } +} +``` + +`<...>` values below come from `terraform output gke_isolated_workers` and +`terraform output loop_runtime_service_account`. + +Use compatible API, Brainstore, and AI Gateway images for the runtime build. +The API must report `ff_version` 82 or newer, and the Rust gateway must support +`x-bt-provider-model-map`; the chart's default API image `v2.18.0` is too old. +Deploy the gateway with `aiGateway.enabled=true` first and wait for it to become +ready before applying this example, which turns on `aiGateway.useGateway`. +The TypeScript API proxy alone cannot restore the provider model IDs sent by Codex. + +1. Create the mTLS Secrets in both clusters: + + ```sh + braintrust/examples/gke-loop-runtime/create-sandbox-tls.sh "" + ``` + +2. Install the workers. The configured Sandbox Service image prepares local SSDs before starting the worker: + + ```sh + helm --kube-context upgrade --install sandbox-service \ + https://braintrust-cf-us-east-1.s3.amazonaws.com/sandbox-service/1f4b6826c61064575b3db956b67625133257bd8a6e49e4ea86cdb55012dd53a7/sandbox-service-0.1.0.tgz \ + --namespace braintrust-sandbox --values braintrust/examples/gke-loop-runtime/sandbox-service.yaml --wait + ``` + +3. Install this chart with your GKE values plus `values.yaml` from this directory: + + ```sh + helm --kube-context upgrade --install braintrust ./braintrust --namespace braintrust \ + --values --values braintrust/examples/gke-loop-runtime/values.yaml --wait + ``` + +Workers run at a fixed count. One `c4-standard-8-lssd` worker holds about seven +1 vCPU / 2 GiB sandboxes; beyond that, new sandboxes fail with +`CapacityExhausted`. Raise `replicas` in `sandbox-service.yaml` to add workers. + +Each runtime pod claims at most two conversations and has a 256Mi in-memory cache budget. +A two-turn GKE smoke run using Git, SQL, and shell tools peaked at 329MiB; the pod retains a 2Gi limit. +Sandbox administration is disabled by default. Set `loopRuntime.enableSandboxAdmin: true` +to allow authorized sysadmins to inspect and terminate sandboxes. + +To shut down, set the worker release to `replicas: 0`, wait for the workers to +drain, then disable `loopRuntime.enabled` and uninstall the worker release. Its +pre-delete hook refuses to uninstall until workers drain. diff --git a/braintrust/examples/gke-loop-runtime/create-sandbox-tls.sh b/braintrust/examples/gke-loop-runtime/create-sandbox-tls.sh new file mode 100755 index 0000000..9b7aa47 --- /dev/null +++ b/braintrust/examples/gke-loop-runtime/create-sandbox-tls.sh @@ -0,0 +1,72 @@ +#!/bin/bash +# Creates the mTLS Secrets for Loop Runtime and Sandbox Service, matching the +# AWS module: separate server and client CAs, P-256 keys, ten-year validity. +# +# Usage: create-sandbox-tls.sh +# worker hostname: Terraform output gke_isolated_workers.discovery.hostname +set -euo pipefail + +if [ "$#" -ne 3 ]; then + echo "usage: $0 " >&2 + exit 2 +fi +HOSTNAME_SAN="$1" +WORKER_CONTEXT="$2" +RUNTIME_CONTEXT="$3" +WORKER_NAMESPACE="${WORKER_NAMESPACE:-braintrust-sandbox}" +RUNTIME_NAMESPACE="${RUNTIME_NAMESPACE:-braintrust}" +DAYS=3650 + +for context in "$WORKER_CONTEXT" "$RUNTIME_CONTEXT"; do + kubectl --context "$context" get namespaces >/dev/null +done +# sandbox-tls is written last, so a failed run can be rerun. +if kubectl --context "$WORKER_CONTEXT" -n "$WORKER_NAMESPACE" get secret sandbox-tls >/dev/null 2>&1; then + echo "sandbox-tls already exists in $WORKER_CONTEXT/$WORKER_NAMESPACE; refusing to replace it." >&2 + exit 1 +fi + +dir="$(mktemp -d)" +trap 'rm -rf "$dir"' EXIT +cd "$dir" + +for purpose in server client; do + openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out "$purpose-ca.key" 2>/dev/null + openssl req -x509 -new -sha256 -key "$purpose-ca.key" -days "$DAYS" \ + -subj "/CN=sandbox-$purpose-ca" \ + -addext "basicConstraints=critical,CA:TRUE,pathlen:0" \ + -addext "keyUsage=critical,keyCertSign,cRLSign" \ + -out "$purpose-ca.pem" + openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out "$purpose.key" 2>/dev/null + { + echo "basicConstraints=critical,CA:FALSE" + echo "keyUsage=critical,digitalSignature" + echo "authorityKeyIdentifier=keyid" + if [ "$purpose" = server ]; then + echo "extendedKeyUsage=serverAuth" + echo "subjectAltName=DNS:$HOSTNAME_SAN" + else + echo "extendedKeyUsage=clientAuth" + fi + } >"$purpose.ext" + openssl req -new -key "$purpose.key" -subj "/CN=sandbox-$purpose" | + openssl x509 -req -sha256 -days "$DAYS" -CA "$purpose-ca.pem" -CAkey "$purpose-ca.key" \ + -set_serial "0x$(openssl rand -hex 16)" -extfile "$purpose.ext" -out "$purpose.pem" 2>/dev/null +done + +apply_secret() { + kubectl --context "$1" create namespace "$2" --dry-run=client -o yaml | kubectl --context "$1" apply --server-side -f - >/dev/null + kubectl --context "$1" -n "$2" create secret generic "$3" "${@:4}" --dry-run=client -o yaml | + kubectl --context "$1" apply --server-side -f - + kubectl --context "$1" -n "$2" annotate secret "$3" kubectl.kubernetes.io/last-applied-configuration- +} + +# The worker controller and Loop Runtime present the client certificate and trust the server CA. +for target in "$RUNTIME_CONTEXT $RUNTIME_NAMESPACE" "$WORKER_CONTEXT $WORKER_NAMESPACE"; do + # shellcheck disable=SC2086 + apply_secret $target sandbox-client-tls \ + --from-file=client.pem=client.pem --from-file=client.key=client.key --from-file=server-ca.pem=server-ca.pem +done +# Workers present the server certificate and trust the client CA. +apply_secret "$WORKER_CONTEXT" "$WORKER_NAMESPACE" sandbox-tls \ + --from-file=cert.pem=server.pem --from-file=key.pem=server.key --from-file=ca_cert.pem=client-ca.pem diff --git a/braintrust/examples/gke-loop-runtime/sandbox-service.yaml b/braintrust/examples/gke-loop-runtime/sandbox-service.yaml new file mode 100644 index 0000000..5396dec --- /dev/null +++ b/braintrust/examples/gke-loop-runtime/sandbox-service.yaml @@ -0,0 +1,27 @@ +# Values for the Sandbox Service worker chart, installed in the isolated worker cluster. +# Every <...> value comes from the GCP Terraform output gke_isolated_workers. +hostname: "" +image: public.ecr.aws/braintrust/sandbox-service:38b05599cdc5e4c3a213bbf0718211b397183a7a +# Fixed worker capacity. The worker node pool autoscales, so raising this is a values change. +replicas: 1 +gitResources: true +nodePrep: + enabled: true +prewarmTemplates: + - image: public.ecr.aws/braintrust/codex-runtime@sha256:490ab364fd6a787f73957d3d46bc5e173fb418c47d53bcf7a2d3afba8b419049 + resources: + vcpu_count: 1 + memory_mib: 2048 + mounts: [] + durable_file_systems: [] + policy: + networking: + type: disabled + credentials: [] + default_workdir: /home/exo/workspace +externalDns: + enabled: true + provider: google + # One entry per argument in discovery.external_dns_extra_args. + extraArgs: + - "" diff --git a/braintrust/examples/gke-loop-runtime/values.yaml b/braintrust/examples/gke-loop-runtime/values.yaml new file mode 100644 index 0000000..b355e15 --- /dev/null +++ b/braintrust/examples/gke-loop-runtime/values.yaml @@ -0,0 +1,18 @@ +# Layer on top of your existing GKE values (for example, ../google-autopilot/values.yaml). +# The sandbox and service-account values come from the GCP Terraform module outputs. +aiGateway: + enabled: true + useGateway: true + +loopRuntime: + enabled: true + serviceAccount: + googleServiceAccount: "" + sandbox: + dnsUrl: "" + egressGateway: + address: "" + # One entry per range in gke_isolated_workers.loop_egress.source_ranges. + sourceRanges: + - "" + url: "" diff --git a/braintrust/templates/_helpers.tpl b/braintrust/templates/_helpers.tpl index c5b85ab..3da0667 100644 --- a/braintrust/templates/_helpers.tpl +++ b/braintrust/templates/_helpers.tpl @@ -146,6 +146,84 @@ Internal cluster URL for the AI Gateway service. http://{{ .Values.aiGateway.service.name | default .Values.aiGateway.name }}.{{ include "braintrust.namespace" . }}:{{ .Values.aiGateway.service.port }} {{- end -}} +{{/* +Whether Loop Runtime is enabled. Values reused from a chart release that +predates Loop Runtime have no loopRuntime map, which means disabled. +*/}} +{{- define "braintrust.loopRuntime.enabled" -}} +{{- if (.Values.loopRuntime | default dict).enabled -}}true{{- end -}} +{{- end -}} + +{{/* Validate the Loop Runtime configuration. */}} +{{- define "braintrust.loopRuntime.validate" -}} +{{- if include "braintrust.loopRuntime.enabled" . -}} +{{- range $key := list "sandbox" "image" "service" "serviceAccount" "resources" "annotations" -}} +{{- if not (kindIs "map" (get $.Values.loopRuntime $key)) -}} +{{- fail "loopRuntime defaults are missing. Values reused from a chart release that predates Loop Runtime omit them; upgrade with --reset-then-reuse-values." -}} +{{- end -}} +{{- end -}} +{{- if ne .Values.cloud "google" -}} +{{- fail "loopRuntime.enabled requires cloud: google. On AWS, Loop Runtime runs on ECS through the Terraform module." -}} +{{- end -}} +{{- $sandbox := .Values.loopRuntime.sandbox -}} +{{- $egress := $sandbox.egressGateway | default dict -}} +{{- $_ := required "objectStorage.google.brainstoreBucket is required" .Values.objectStorage.google.brainstoreBucket -}} +{{- $_ := required "objectStorage.google.apiBucket is required" .Values.objectStorage.google.apiBucket -}} +{{- $_ := required "loopRuntime.image.repository is required" .Values.loopRuntime.image.repository -}} +{{- $_ := required "loopRuntime.image.tag is required" (.Values.loopRuntime.image.tag | default "" | toString) -}} +{{- $_ := required "loopRuntime.serviceAccount.googleServiceAccount is required" .Values.loopRuntime.serviceAccount.googleServiceAccount -}} +{{- $_ := required "loopRuntime.sandbox.dnsUrl is required" $sandbox.dnsUrl -}} +{{- $_ := required "loopRuntime.sandbox.egressGateway.address is required" $egress.address -}} +{{- $_ := required "loopRuntime.sandbox.egressGateway.url is required" $egress.url -}} +{{- if not $egress.sourceRanges -}} +{{- fail "loopRuntime.sandbox.egressGateway.sourceRanges is required" -}} +{{- end -}} +{{- range $egress.sourceRanges -}} +{{- $cidr := toString . -}} +{{- $valid := regexMatch "^((0|[1-9][0-9]{0,2})\\.){3}(0|[1-9][0-9]{0,2})/(0|[1-9][0-9]?)$" $cidr -}} +{{- if $valid -}} +{{- $parts := splitList "/" $cidr -}} +{{- $prefix := int (index $parts 1) -}} +{{- $valid = le $prefix 32 -}} +{{- $address := 0 -}} +{{- range splitList "." (index $parts 0) -}} +{{- $valid = and $valid (le (int .) 255) -}} +{{- $address = add (mul $address 256) (int .) -}} +{{- end -}} +{{- if $valid -}} +{{- $blockSize := 1 -}} +{{- range until (int (sub 32 $prefix)) -}} +{{- $blockSize = mul $blockSize 2 -}} +{{- end -}} +{{- $valid = eq (mod $address $blockSize) 0 -}} +{{- end -}} +{{- end -}} +{{- if not $valid -}} +{{- fail (printf "loopRuntime.sandbox.egressGateway.sourceRanges: %q is not a CIDR range" (toString .)) -}} +{{- end -}} +{{- end -}} +{{- end -}} +{{- end -}} + +{{/* Internal cluster URL for the Loop Runtime service. */}} +{{- define "braintrust.loopRuntimeInternalUrl" -}} +http://{{ .Values.loopRuntime.service.name | default .Values.loopRuntime.name }}.{{ include "braintrust.namespace" . }}:{{ .Values.loopRuntime.service.port }} +{{- end -}} + +{{/* Product-owned Loop Runtime routes for an Istio VirtualService. */}} +{{- define "braintrust.loopRuntimeVirtualServiceRoutes" -}} +- match: + - uri: + exact: /loop/runtime + - uri: + prefix: /loop/runtime/ + route: + - destination: + host: {{ .Values.loopRuntime.service.name | default .Values.loopRuntime.name }} + port: + number: {{ .Values.loopRuntime.service.port }} +{{- end -}} + {{/* Validate API autoscaling prerequisites (GKE + AutoscalingMetric CRD). */}} diff --git a/braintrust/templates/api-configmap.yaml b/braintrust/templates/api-configmap.yaml index 41643ba..72d9217 100644 --- a/braintrust/templates/api-configmap.yaml +++ b/braintrust/templates/api-configmap.yaml @@ -15,6 +15,7 @@ {{- if and (gt (len $btqlStrictOrgIds) 0) (gt (len $btqlBestEffortOrgIds) 0) -}} {{- fail "Set either btqlAuditLogsStrictOrgIds or btqlAuditLogsBestEffortOrgIds, not both." -}} {{- end -}} +{{- include "braintrust.loopRuntime.validate" . -}} {{- range $btqlStrictOrgIds -}} {{- if not (regexMatch "^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$" .) -}} {{- fail (printf "btqlAuditLogsStrictOrgIds: %q is not a valid UUID" .) -}} @@ -113,6 +114,9 @@ data: {{- if .Values.aiGateway.useGateway }} GATEWAY_URL: {{ include "braintrust.aiGatewayInternalUrl" . | quote }} {{- end }} + {{- if include "braintrust.loopRuntime.enabled" . }} + LOOP_RUNTIME_URL: {{ include "braintrust.loopRuntimeInternalUrl" . | quote }} + {{- end }} {{- if $btqlStrictOrgIds }} BTQL_AUDIT_LOGS_STRICT_ORG_IDS: {{ $btqlStrictOrgIds | join "," | quote }} {{- end }} diff --git a/braintrust/templates/loop-runtime-configmap.yaml b/braintrust/templates/loop-runtime-configmap.yaml new file mode 100644 index 0000000..8fed6bb --- /dev/null +++ b/braintrust/templates/loop-runtime-configmap.yaml @@ -0,0 +1,68 @@ +{{- include "braintrust.loopRuntime.validate" . -}} +{{- if include "braintrust.loopRuntime.enabled" . -}} +{{- $bucket := printf "gs://%s" .Values.objectStorage.google.brainstoreBucket -}} +{{- $sandbox := .Values.loopRuntime.sandbox -}} +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ .Values.loopRuntime.name }} + namespace: {{ include "braintrust.namespace" . }} + {{- with (mergeOverwrite (deepCopy .Values.global.labels) (deepCopy .Values.loopRuntime.labels)) }} + labels: + {{- toYaml . | nindent 4 }} + {{- end }} + {{- with .Values.loopRuntime.annotations.configmap }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +# Matches the ECS Loop Runtime environment in the AWS Terraform module. +data: + BRAINTRUST_LEGACY_IDS: "true" + LOOP_RUNTIME_HOST: "0.0.0.0" + LOOP_RUNTIME_PORT: {{ .Values.loopRuntime.service.port | quote }} + LOOP_RUNTIME_DRAIN_TIMEOUT_SECONDS: "90" + LOOP_RUNTIME_REDIS_NAMESPACE: "loop-runtime:{{ .Release.Name }}" + LOOP_RUNTIME_STATE_DIR: /mnt/tmp/loop-runtime + {{- /* The runtime refuses an empty ORG_NAME; the API treats empty as every org. */}} + ORG_NAME: {{ .Values.global.orgName | default "" | toString | trim | default "*" | quote }} + BRAINTRUST_API_URL: {{ include "braintrust.apiInternalUrl" . | quote }} + LOOP_RUNTIME_AI_PROXY_URL: {{ include "braintrust.apiAiProxyInternalUrl" . | quote }} + LOOP_RUNTIME_BRAINSTORE_READER_URL: "http://{{ .Values.brainstore.fastreader.service.name | default .Values.brainstore.fastreader.name }}.{{ include "braintrust.namespace" . }}:{{ .Values.brainstore.fastreader.service.port }}" + BRAINSTORE_VERBOSE: "1" + BRAINSTORE_INDEX_URI: "{{ $bucket }}/brainstore/index" + BRAINSTORE_REALTIME_WAL_URI: "{{ $bucket }}/brainstore/wal" + BRAINSTORE_CODE_BUNDLE_URI: "gs://{{ .Values.objectStorage.google.apiBucket }}" + BRAINSTORE_CACHE_DIR: /mnt/tmp/brainstore + BRAINSTORE_OBJECT_STORE_CACHE_FILE_SIZE: 5Gi + BRAINSTORE_OBJECT_STORE_CACHE_MEMORY_LIMIT: 256Mi + LOOP_RUNTIME_MAX_CLAIMED_CONVERSATIONS: "2" + BRAINSTORE_CONTROL_PLANE_TELEMETRY: {{ .Values.global.controlPlaneTelemetry | quote }} + NO_COLOR: "1" + EXO_SANDBOX_SERVICE_DNS_URL: {{ $sandbox.dnsUrl | quote }} + EXO_SANDBOX_SERVICE_IMAGE: {{ $sandbox.image | quote }} + EXO_SANDBOX_VCPU_COUNT: {{ $sandbox.vcpuCount | quote }} + EXO_SANDBOX_MEMORY_MIB: {{ $sandbox.memoryMib | quote }} + SANDBOX_SERVICE_TLS_CERT: /etc/sandbox-tls/client.pem + SANDBOX_SERVICE_TLS_KEY: /etc/sandbox-tls/client.key + SANDBOX_SERVICE_TLS_CA: /etc/sandbox-tls/server-ca.pem + LOOP_RUNTIME_SANDBOX_FORWARD_PROXY_LISTEN: "0.0.0.0:4002" + LOOP_RUNTIME_SANDBOX_EGRESS_GATEWAY_URL: {{ $sandbox.egressGateway.url | quote }} + LOOP_RUNTIME_SANDBOX_SERVICE_EGRESS_GATEWAY_URL: {{ $sandbox.egressGateway.url | quote }} + LOOP_RUNTIME_SANDBOX_EGRESS_MODE: restricted + LOOP_RUNTIME_ENABLE_SANDBOX_ADMIN: {{ .Values.loopRuntime.enableSandboxAdmin | default false | quote }} + {{- with (.Values.global.allowedOrgIds | default "" | toString | trim) }} + ALLOWED_ORG_IDS: {{ . | quote }} + {{- end }} + {{- if eq .Values.brainstore.locksBackend "objectStorage" }} + BRAINSTORE_LOCKS_URI: "{{ $bucket }}/brainstore/locks" + {{- end }} + {{- if .Values.brainstoreWalFooterVersion }} + BRAINSTORE_WAL_FOOTER_VERSION: {{ .Values.brainstoreWalFooterVersion | quote }} + {{- end }} + {{- if .Values.skipPgForBrainstoreObjects }} + SKIP_PG_FOR_BRAINSTORE_OBJECTS: {{ .Values.skipPgForBrainstoreObjects | quote }} + {{- end }} + {{- if or .Values.brainstoreWalFooterVersion .Values.skipPgForBrainstoreObjects }} + BRAINSTORE_WAL_USE_EFFICIENT_FORMAT: "true" + {{- end }} +{{- end }} diff --git a/braintrust/templates/loop-runtime-deployment.yaml b/braintrust/templates/loop-runtime-deployment.yaml new file mode 100644 index 0000000..c10f99f --- /dev/null +++ b/braintrust/templates/loop-runtime-deployment.yaml @@ -0,0 +1,161 @@ +{{- include "braintrust.loopRuntime.validate" . -}} +{{- if include "braintrust.loopRuntime.enabled" . -}} +{{- $podLabels := mergeOverwrite (deepCopy .Values.global.labels) (deepCopy .Values.loopRuntime.labels) (deepCopy .Values.loopRuntime.podLabels) (dict "app" .Values.loopRuntime.name "gke-workload-identity/use" "true") -}} +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ .Values.loopRuntime.name }} + namespace: {{ include "braintrust.namespace" . }} + {{- with (mergeOverwrite (deepCopy .Values.global.labels) (deepCopy .Values.loopRuntime.labels)) }} + labels: + {{- toYaml . | nindent 4 }} + {{- end }} + {{- with .Values.loopRuntime.annotations.deployment }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + replicas: {{ ternary 1 .Values.loopRuntime.replicas (has .Values.loopRuntime.replicas (list nil "")) }} + selector: + matchLabels: + app: {{ .Values.loopRuntime.name }} + template: + metadata: + labels: + {{- toYaml $podLabels | nindent 8 }} + annotations: + checksum/config: {{ include (print $.Template.BasePath "/loop-runtime-configmap.yaml") . | sha256sum }} + iam.gke.io/gcp-service-account: {{ .Values.loopRuntime.serviceAccount.googleServiceAccount }} + {{- with .Values.loopRuntime.annotations.pod }} + {{- toYaml . | nindent 8 }} + {{- end }} + spec: + serviceAccountName: {{ .Values.loopRuntime.serviceAccount.name }} + # The runtime drains active turns for up to 90 seconds after SIGTERM. + terminationGracePeriodSeconds: 120 + securityContext: + runAsNonRoot: true + runAsUser: 1000 + runAsGroup: 1000 + fsGroup: 1000 + seccompProfile: + type: RuntimeDefault + {{- with .Values.loopRuntime.nodeSelector }} + nodeSelector: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.loopRuntime.tolerations }} + tolerations: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.loopRuntime.affinity }} + affinity: + {{- toYaml . | nindent 8 }} + {{- end }} + containers: + - name: loop-runtime + image: "{{ .Values.loopRuntime.image.repository }}:{{ .Values.loopRuntime.image.tag }}" + imagePullPolicy: {{ .Values.loopRuntime.image.pullPolicy }} + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + ports: + - name: {{ .Values.loopRuntime.service.portName }} + containerPort: {{ .Values.loopRuntime.service.port }} + # Git forward proxy for sandboxes. + - containerPort: 4002 + resources: + {{- toYaml .Values.loopRuntime.resources | nindent 12 }} + {{- with .Values.loopRuntime.livenessProbe }} + {{- $probe := deepCopy . -}} + {{- if and (kindIs "map" $probe.httpGet) (not (hasKey $probe.httpGet "port")) -}} + {{- $_ := set $probe.httpGet "port" $.Values.loopRuntime.service.port -}} + {{- end }} + livenessProbe: + {{- toYaml $probe | nindent 12 }} + {{- end }} + {{- with .Values.loopRuntime.readinessProbe }} + {{- $probe := deepCopy . -}} + {{- if and (kindIs "map" $probe.httpGet) (not (hasKey $probe.httpGet "port")) -}} + {{- $_ := set $probe.httpGet "port" $.Values.loopRuntime.service.port -}} + {{- end }} + readinessProbe: + {{- toYaml $probe | nindent 12 }} + {{- end }} + envFrom: + - configMapRef: + name: {{ .Values.loopRuntime.name }} + env: + - name: BRAINSTORE_METADATA_URI + valueFrom: + secretKeyRef: + name: braintrust-secrets + key: PG_URL + - name: BRAINSTORE_WAL_URI + valueFrom: + secretKeyRef: + name: braintrust-secrets + key: PG_URL + - name: BRAINSTORE_XACT_MANAGER_URI + valueFrom: + secretKeyRef: + name: braintrust-secrets + key: REDIS_URL + - name: BRAINSTORE_REDIS_URI + valueFrom: + secretKeyRef: + name: braintrust-secrets + key: REDIS_URL + - name: SERVICE_TOKEN_SECRET_KEY + valueFrom: + secretKeyRef: + name: braintrust-secrets + key: FUNCTION_SECRET_KEY + - name: FUNCTION_SECRET_KEY + valueFrom: + secretKeyRef: + name: braintrust-secrets + key: FUNCTION_SECRET_KEY + - name: BRAINSTORE_LICENSE_KEY + valueFrom: + secretKeyRef: + name: braintrust-secrets + key: BRAINSTORE_LICENSE_KEY + {{- if eq .Values.brainstore.locksBackend "redis" }} + - name: BRAINSTORE_LOCKS_URI + valueFrom: + secretKeyRef: + name: braintrust-secrets + key: REDIS_URL + {{- end }} + {{- with .Values.loopRuntime.extraEnvVars }} + {{- toYaml . | nindent 12 }} + {{- end }} + volumeMounts: + - name: state + mountPath: /mnt/tmp/loop-runtime + - name: cache + mountPath: /mnt/tmp/brainstore + - name: sandbox-tls + mountPath: /etc/sandbox-tls + readOnly: true + {{- with .Values.loopRuntime.extraContainers }} + {{- toYaml . | nindent 8 }} + {{- end }} + volumes: + - name: state + emptyDir: + sizeLimit: 2Gi + - name: cache + emptyDir: + sizeLimit: 6Gi + # Created by examples/gke-loop-runtime/create-sandbox-tls.sh. + - name: sandbox-tls + secret: + secretName: sandbox-client-tls + {{- with .Values.loopRuntime.extraVolumes }} + {{- toYaml . | nindent 8 }} + {{- end }} +{{- end }} diff --git a/braintrust/templates/loop-runtime-egress-service.yaml b/braintrust/templates/loop-runtime-egress-service.yaml new file mode 100644 index 0000000..1808663 --- /dev/null +++ b/braintrust/templates/loop-runtime-egress-service.yaml @@ -0,0 +1,28 @@ +{{- include "braintrust.loopRuntime.validate" . -}} +{{- if include "braintrust.loopRuntime.enabled" . -}} +{{- $egressGateway := .Values.loopRuntime.sandbox.egressGateway -}} +# Workers in the isolated cluster reach the Git forward proxy through this +# internal load balancer, on the address Terraform reserves. +apiVersion: v1 +kind: Service +metadata: + name: {{ .Values.loopRuntime.service.name | default .Values.loopRuntime.name }}-egress + namespace: {{ include "braintrust.namespace" . }} + {{- with (mergeOverwrite (deepCopy .Values.global.labels) (deepCopy .Values.loopRuntime.labels)) }} + labels: + {{- toYaml . | nindent 4 }} + {{- end }} + annotations: + networking.gke.io/load-balancer-type: "Internal" +spec: + type: LoadBalancer + loadBalancerIP: {{ $egressGateway.address | quote }} + loadBalancerSourceRanges: + {{- toYaml $egressGateway.sourceRanges | nindent 4 }} + selector: + app: {{ .Values.loopRuntime.name }} + ports: + - name: egress + port: 4002 + targetPort: 4002 +{{- end }} diff --git a/braintrust/templates/loop-runtime-service.yaml b/braintrust/templates/loop-runtime-service.yaml new file mode 100644 index 0000000..1174707 --- /dev/null +++ b/braintrust/templates/loop-runtime-service.yaml @@ -0,0 +1,26 @@ +{{- include "braintrust.loopRuntime.validate" . -}} +{{- if include "braintrust.loopRuntime.enabled" . -}} +{{- $resourceLabels := mergeOverwrite (deepCopy .Values.global.labels) (deepCopy .Values.loopRuntime.labels) -}} +apiVersion: v1 +kind: Service +metadata: + name: {{ .Values.loopRuntime.service.name | default .Values.loopRuntime.name }} + namespace: {{ include "braintrust.namespace" . }} + {{- with $resourceLabels }} + labels: + {{- toYaml . | nindent 4 }} + {{- end }} + {{- with .Values.loopRuntime.annotations.service }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + selector: + app: {{ .Values.loopRuntime.name }} + ports: + - name: {{ .Values.loopRuntime.service.portName }} + protocol: TCP + port: {{ .Values.loopRuntime.service.port }} + targetPort: {{ .Values.loopRuntime.service.port }} + type: {{ .Values.loopRuntime.service.type }} +{{- end }} diff --git a/braintrust/templates/loop-runtime-serviceaccount.yaml b/braintrust/templates/loop-runtime-serviceaccount.yaml new file mode 100644 index 0000000..1e65de4 --- /dev/null +++ b/braintrust/templates/loop-runtime-serviceaccount.yaml @@ -0,0 +1,17 @@ +{{- include "braintrust.loopRuntime.validate" . -}} +{{- if include "braintrust.loopRuntime.enabled" . -}} +apiVersion: v1 +kind: ServiceAccount +metadata: + name: {{ .Values.loopRuntime.serviceAccount.name }} + namespace: {{ include "braintrust.namespace" . }} + {{- with (mergeOverwrite (deepCopy .Values.global.labels) (deepCopy .Values.loopRuntime.labels)) }} + labels: + {{- toYaml . | nindent 4 }} + {{- end }} + annotations: + iam.gke.io/gcp-service-account: {{ .Values.loopRuntime.serviceAccount.googleServiceAccount }} + {{- with .Values.loopRuntime.annotations.serviceaccount }} + {{- toYaml . | nindent 4 }} + {{- end }} +{{- end }} diff --git a/braintrust/templates/virtualservice.yaml b/braintrust/templates/virtualservice.yaml index 5b4dd8d..9026db8 100644 --- a/braintrust/templates/virtualservice.yaml +++ b/braintrust/templates/virtualservice.yaml @@ -5,6 +5,7 @@ {{- if and $virtualServiceWorkloadIsolation.enabled (not .Values.api.workloadIsolation.enabled) -}} {{- fail "virtualService.workloadIsolation.enabled requires api.workloadIsolation.enabled." -}} {{- end -}} +{{- include "braintrust.loopRuntime.validate" . -}} {{- if .Values.virtualService.enabled }} apiVersion: networking.istio.io/v1 kind: VirtualService @@ -29,6 +30,9 @@ spec: {{- toYaml . | nindent 4 }} {{- end }} http: + {{- if include "braintrust.loopRuntime.enabled" . }} + {{- include "braintrust.loopRuntimeVirtualServiceRoutes" . | nindent 4 }} + {{- end }} {{- if $virtualServiceWorkloadIsolation.enabled }} {{- include "braintrust.apiWorkloadIsolationVirtualServiceRoutes" . | nindent 4 }} {{- end }} diff --git a/braintrust/tests/__fixtures__/loop-runtime-values.yaml b/braintrust/tests/__fixtures__/loop-runtime-values.yaml new file mode 100644 index 0000000..43af85a --- /dev/null +++ b/braintrust/tests/__fixtures__/loop-runtime-values.yaml @@ -0,0 +1,28 @@ +# Loop Runtime on GKE. Layer on top of base-values.yaml. +cloud: "google" + +objectStorage: + google: + brainstoreBucket: "test-brainstore-bucket" + apiBucket: "test-api-bucket" + +brainstore: + serviceAccount: + googleServiceAccount: "test@project.iam.gserviceaccount.com" + +loopRuntime: + enabled: true + image: + repository: "test/loop-runtime" + tag: "v1.0.0" + serviceAccount: + name: "braintrust-loop-runtime" + googleServiceAccount: "braintrust-loop@test-project.iam.gserviceaccount.com" + sandbox: + dnsUrl: "https://workers.sandbox.internal:9400" + egressGateway: + address: "10.0.0.50" + sourceRanges: + - "10.64.64.0/24" + - "10.64.0.0/18" + url: "http://loop-egress.sandbox.internal:4002" diff --git a/braintrust/tests/__fixtures__/pre-loop-runtime-values.yaml b/braintrust/tests/__fixtures__/pre-loop-runtime-values.yaml new file mode 100644 index 0000000..23b1551 --- /dev/null +++ b/braintrust/tests/__fixtures__/pre-loop-runtime-values.yaml @@ -0,0 +1,4 @@ +# Simulate values retained by `helm upgrade --reuse-values` from a chart release +# that predates loopRuntime. An explicit null removes the new chart default +# during Helm value coalescing. +loopRuntime: null diff --git a/braintrust/tests/__fixtures__/reuse-loop-runtime-enabled-values.yaml b/braintrust/tests/__fixtures__/reuse-loop-runtime-enabled-values.yaml new file mode 100644 index 0000000..42e8585 --- /dev/null +++ b/braintrust/tests/__fixtures__/reuse-loop-runtime-enabled-values.yaml @@ -0,0 +1,5 @@ +# Simulate `helm upgrade --reuse-values --set loopRuntime.enabled=true` from a +# chart that predates Loop Runtime. An explicit null removes the new defaults. +loopRuntime: + enabled: true + sandbox: null diff --git a/braintrust/tests/api-configmap_test.yaml b/braintrust/tests/api-configmap_test.yaml index 27de0d0..1f5fde0 100644 --- a/braintrust/tests/api-configmap_test.yaml +++ b/braintrust/tests/api-configmap_test.yaml @@ -585,3 +585,23 @@ tests: asserts: - failedTemplate: errorMessage: "btqlAuditLogsBestEffortOrgIds: \"not-a-uuid\" is not a valid UUID" + + - it: should publish the Loop Runtime URL when Loop Runtime is enabled + values: + - __fixtures__/base-values.yaml + - __fixtures__/loop-runtime-values.yaml + release: + namespace: "braintrust" + asserts: + - equal: + path: data.LOOP_RUNTIME_URL + value: http://braintrust-loop-runtime.braintrust:4001 + + - it: should omit the Loop Runtime URL when reused values have no loopRuntime map + values: + - __fixtures__/base-values.yaml + - __fixtures__/google-values.yaml + - __fixtures__/pre-loop-runtime-values.yaml + asserts: + - isNull: + path: data.LOOP_RUNTIME_URL diff --git a/braintrust/tests/labels-merge-configmaps_test.yaml b/braintrust/tests/labels-merge-configmaps_test.yaml index b4c61eb..84ef5fd 100644 --- a/braintrust/tests/labels-merge-configmaps_test.yaml +++ b/braintrust/tests/labels-merge-configmaps_test.yaml @@ -5,6 +5,7 @@ templates: - brainstore-writer-configmap.yaml - brainstore-fastreader-configmap.yaml - brainstore-automationwriter-configmap.yaml + - loop-runtime-configmap.yaml tests: # These tests verify the deepCopy fix for configmaps. # Without deepCopy, labels would bleed between components due to merge mutation. @@ -153,3 +154,33 @@ tests: path: metadata.labels["writer-specific"] - isNull: path: metadata.labels["fastreader-specific"] + + - it: should give Loop Runtime configmap only global and runtime-specific labels + values: + - __fixtures__/base-values.yaml + - __fixtures__/loop-runtime-values.yaml + set: + global.labels.shared: "global-cm" + loopRuntime.labels.loop-specific: "loop-cm" + api.labels.api-specific: "api-cm" + brainstore.reader.labels.reader-specific: "reader-cm" + brainstore.writer.labels.writer-specific: "writer-cm" + brainstore.fastreader.labels.fastreader-specific: "fastreader-cm" + release: + namespace: "braintrust" + template: loop-runtime-configmap.yaml + asserts: + - equal: + path: metadata.labels.shared + value: global-cm + - equal: + path: metadata.labels["loop-specific"] + value: loop-cm + - isNull: + path: metadata.labels["api-specific"] + - isNull: + path: metadata.labels["reader-specific"] + - isNull: + path: metadata.labels["writer-specific"] + - isNull: + path: metadata.labels["fastreader-specific"] diff --git a/braintrust/tests/labels-merge-isolation_test.yaml b/braintrust/tests/labels-merge-isolation_test.yaml index d545551..1833afc 100644 --- a/braintrust/tests/labels-merge-isolation_test.yaml +++ b/braintrust/tests/labels-merge-isolation_test.yaml @@ -10,6 +10,8 @@ templates: - brainstore-fastreader-configmap.yaml - brainstore-automationwriter-deployment.yaml - brainstore-automationwriter-configmap.yaml + - loop-runtime-deployment.yaml + - loop-runtime-configmap.yaml tests: # This test verifies that labels are isolated between components. # The bug that was fixed: Helm's merge function mutates the leftmost dict, @@ -504,3 +506,59 @@ tests: - equal: path: metadata.labels["automationwriter-only"] value: automationwriter-value + + - it: should isolate Loop Runtime deployment and pod labels + values: + - __fixtures__/base-values.yaml + - __fixtures__/loop-runtime-values.yaml + set: + global.labels.shared: "global-loop" + global.labels.environment: "production" + loopRuntime.labels.component: "loop-runtime" + loopRuntime.labels.loop-specific: "loop-only" + loopRuntime.podLabels.pod-specific: "pod-only" + api.labels.api-specific: "api-only" + brainstore.reader.labels.reader-specific: "reader-only" + brainstore.writer.labels.writer-specific: "writer-only" + brainstore.fastreader.labels.fastreader-specific: "fastreader-only" + release: + namespace: "braintrust" + template: loop-runtime-deployment.yaml + asserts: + - equal: + path: metadata.labels.shared + value: global-loop + - equal: + path: metadata.labels.environment + value: production + - equal: + path: metadata.labels.component + value: loop-runtime + - equal: + path: metadata.labels["loop-specific"] + value: loop-only + - isNull: + path: metadata.labels["api-specific"] + - isNull: + path: metadata.labels["reader-specific"] + - isNull: + path: metadata.labels["writer-specific"] + - isNull: + path: metadata.labels["fastreader-specific"] + - equal: + path: spec.template.metadata.labels.shared + value: global-loop + - equal: + path: spec.template.metadata.labels["loop-specific"] + value: loop-only + - equal: + path: spec.template.metadata.labels["pod-specific"] + value: pod-only + - isNull: + path: spec.template.metadata.labels["api-specific"] + - isNull: + path: spec.template.metadata.labels["reader-specific"] + - isNull: + path: spec.template.metadata.labels["writer-specific"] + - isNull: + path: spec.template.metadata.labels["fastreader-specific"] diff --git a/braintrust/tests/labels-merge-serviceaccounts_test.yaml b/braintrust/tests/labels-merge-serviceaccounts_test.yaml index f26efed..6b98530 100644 --- a/braintrust/tests/labels-merge-serviceaccounts_test.yaml +++ b/braintrust/tests/labels-merge-serviceaccounts_test.yaml @@ -1,6 +1,7 @@ suite: test label merge isolation for service accounts (deepCopy fix) templates: - api-serviceaccount.yaml + - loop-runtime-serviceaccount.yaml - brainstore-serviceaccount.yaml tests: # These tests verify the deepCopy fix for service accounts. @@ -45,3 +46,27 @@ tests: value: brainstore-sa - isNull: path: metadata.labels["api-specific"] + + - it: should give Loop Runtime service account only global and runtime-specific labels + values: + - __fixtures__/base-values.yaml + - __fixtures__/loop-runtime-values.yaml + set: + global.labels.shared: "global-sa" + loopRuntime.labels.loop-specific: "loop-sa" + api.labels.api-specific: "api-sa" + brainstore.labels.brainstore-specific: "brainstore-sa" + release: + namespace: "braintrust" + template: loop-runtime-serviceaccount.yaml + asserts: + - equal: + path: metadata.labels.shared + value: global-sa + - equal: + path: metadata.labels["loop-specific"] + value: loop-sa + - isNull: + path: metadata.labels["api-specific"] + - isNull: + path: metadata.labels["brainstore-specific"] diff --git a/braintrust/tests/labels-merge-services_test.yaml b/braintrust/tests/labels-merge-services_test.yaml index 5bb668c..9608d7b 100644 --- a/braintrust/tests/labels-merge-services_test.yaml +++ b/braintrust/tests/labels-merge-services_test.yaml @@ -5,6 +5,8 @@ templates: - brainstore-writer-service.yaml - brainstore-fastreader-service.yaml - brainstore-automationwriter-service.yaml + - loop-runtime-service.yaml + - loop-runtime-egress-service.yaml tests: # These tests verify the deepCopy fix for services. # Without deepCopy, labels would bleed between components due to merge mutation. @@ -164,3 +166,38 @@ tests: path: metadata.labels["writer-specific"] - isNull: path: metadata.labels["fastreader-specific"] + + - it: should give Loop Runtime services only global and runtime-specific labels + values: + - __fixtures__/base-values.yaml + - __fixtures__/loop-runtime-values.yaml + set: + global.labels.shared: global-service + global.labels.component: global + loopRuntime.labels.component: loop-runtime + api.labels.api-specific: api-service + templates: + - loop-runtime-service.yaml + - loop-runtime-egress-service.yaml + asserts: + - equal: + path: metadata.labels + value: + shared: global-service + component: loop-runtime + + - it: should keep Loop Runtime labels out of the API service + values: + - __fixtures__/base-values.yaml + - __fixtures__/loop-runtime-values.yaml + set: + global.labels.shared: global-service + loopRuntime.labels.component: loop-runtime + api.labels.api-specific: api-service + template: api-service.yaml + asserts: + - equal: + path: metadata.labels + value: + shared: global-service + api-specific: api-service diff --git a/braintrust/tests/loop-runtime-configmap_test.yaml b/braintrust/tests/loop-runtime-configmap_test.yaml new file mode 100644 index 0000000..54b8c3f --- /dev/null +++ b/braintrust/tests/loop-runtime-configmap_test.yaml @@ -0,0 +1,220 @@ +suite: test Loop Runtime ConfigMap template +templates: + - loop-runtime-configmap.yaml +tests: + - it: should not render when Loop Runtime is disabled + values: + - __fixtures__/base-values.yaml + asserts: + - hasDocuments: + count: 0 + + - it: should not render when reused values have no loopRuntime map + values: + - __fixtures__/base-values.yaml + - __fixtures__/google-values.yaml + - __fixtures__/pre-loop-runtime-values.yaml + asserts: + - hasDocuments: + count: 0 + + - it: should explain how to upgrade when reused values enable Loop Runtime without defaults + values: + - __fixtures__/base-values.yaml + - __fixtures__/google-values.yaml + - __fixtures__/reuse-loop-runtime-enabled-values.yaml + asserts: &missing-defaults + - failedTemplate: + errorMessage: "loopRuntime defaults are missing. Values reused from a chart release that predates Loop Runtime omit them; upgrade with --reset-then-reuse-values." + + - it: should render GKE storage and the Sandbox Service configuration + values: &values + - __fixtures__/base-values.yaml + - __fixtures__/loop-runtime-values.yaml + asserts: + - equal: + path: data.BRAINSTORE_INDEX_URI + value: gs://test-brainstore-bucket/brainstore/index + - equal: + path: data.BRAINSTORE_CODE_BUNDLE_URI + value: gs://test-api-bucket + - equal: + path: data.EXO_SANDBOX_SERVICE_DNS_URL + value: https://workers.sandbox.internal:9400 + - equal: + path: data.LOOP_RUNTIME_SANDBOX_SERVICE_EGRESS_GATEWAY_URL + value: http://loop-egress.sandbox.internal:4002 + - equal: + path: data.LOOP_RUNTIME_SANDBOX_EGRESS_GATEWAY_URL + value: http://loop-egress.sandbox.internal:4002 + - equal: + path: data.LOOP_RUNTIME_REDIS_NAMESPACE + value: loop-runtime:RELEASE-NAME + - equal: + path: data.ORG_NAME + value: test-org + - equal: + path: data.LOOP_RUNTIME_ENABLE_SANDBOX_ADMIN + value: "false" + - equal: + path: data.BRAINSTORE_OBJECT_STORE_CACHE_MEMORY_LIMIT + value: 256Mi + - equal: + path: data.LOOP_RUNTIME_MAX_CLAIMED_CONVERSATIONS + value: "2" + - isNull: + path: data.BRAINSTORE_LOCKS_URI + + - it: should allow sandbox administration only when opted in + values: *values + set: + loopRuntime.enableSandboxAdmin: true + asserts: + - equal: + path: data.LOOP_RUNTIME_ENABLE_SANDBOX_ADMIN + value: "true" + + - it: should keep sandbox administration disabled for retained null values + values: *values + set: + loopRuntime.enableSandboxAdmin: null + asserts: + - equal: + path: data.LOOP_RUNTIME_ENABLE_SANDBOX_ADMIN + value: "false" + + - it: should explain reused values with sandbox overrides but no image defaults + values: *values + set: + loopRuntime.image: null + asserts: *missing-defaults + + - it: should explain reused values without service defaults + values: *values + set: + loopRuntime.service: null + asserts: *missing-defaults + + - it: should explain reused values without service account defaults + values: *values + set: + loopRuntime.serviceAccount: null + asserts: *missing-defaults + + - it: should explain reused values without resource defaults + values: *values + set: + loopRuntime.resources: null + asserts: *missing-defaults + + - it: should explain reused values without annotation defaults + values: *values + set: + loopRuntime.annotations: null + asserts: *missing-defaults + + - it: should use the configured egress URL for Git authorization and sandbox traffic + values: *values + set: + loopRuntime.sandbox.egressGateway.url: http://custom-egress.internal:4002 + asserts: + - equal: + path: data.LOOP_RUNTIME_SANDBOX_EGRESS_GATEWAY_URL + value: http://custom-egress.internal:4002 + - equal: + path: data.LOOP_RUNTIME_SANDBOX_SERVICE_EGRESS_GATEWAY_URL + value: http://custom-egress.internal:4002 + + - it: should serve every org when orgName is empty + values: *values + set: + global.orgName: "" + asserts: + - equal: + path: data.ORG_NAME + value: "*" + + - it: should use GCS object storage locks when configured + values: *values + set: + brainstore.locksBackend: objectStorage + asserts: + - equal: + path: data.BRAINSTORE_LOCKS_URI + value: gs://test-brainstore-bucket/brainstore/locks + + - it: should reject Loop Runtime on AWS + values: *values + set: + cloud: aws + asserts: ¬-google + - failedTemplate: + errorMessage: "loopRuntime.enabled requires cloud: google. On AWS, Loop Runtime runs on ECS through the Terraform module." + + - it: should reject Loop Runtime on Azure + values: *values + set: + cloud: azure + asserts: *not-google + + - it: should reject Loop Runtime without a cloud + values: *values + set: + cloud: "" + asserts: *not-google + + - it: should require the Google service account + values: *values + set: + loopRuntime.serviceAccount.googleServiceAccount: "" + asserts: + - failedTemplate: + errorMessage: "loopRuntime.serviceAccount.googleServiceAccount is required" + + - it: should require the worker discovery endpoint + values: *values + set: + loopRuntime.sandbox.dnsUrl: null + asserts: + - failedTemplate: + errorMessage: "loopRuntime.sandbox.dnsUrl is required" + + - it: should require the egress address + values: *values + set: + loopRuntime.sandbox.egressGateway.address: "" + asserts: + - failedTemplate: + errorMessage: "loopRuntime.sandbox.egressGateway.address is required" + + - it: should explain a missing egress map + values: *values + set: + loopRuntime.sandbox.egressGateway: null + asserts: + - failedTemplate: + errorMessage: "loopRuntime.sandbox.egressGateway.address is required" + + - it: should require egress source ranges + values: *values + set: + loopRuntime.sandbox.egressGateway.sourceRanges: [] + asserts: + - failedTemplate: + errorMessage: "loopRuntime.sandbox.egressGateway.sourceRanges is required" + + - it: should reject a blank egress source range next to a valid one + values: *values + set: + loopRuntime.sandbox.egressGateway.sourceRanges: ["10.64.64.0/24", ""] + asserts: + - failedTemplate: + errorMessage: "loopRuntime.sandbox.egressGateway.sourceRanges: \"\" is not a CIDR range" + + - it: should reject a null image tag + values: *values + set: + loopRuntime.image.tag: null + asserts: + - failedTemplate: + errorMessage: "loopRuntime.image.tag is required" diff --git a/braintrust/tests/loop-runtime-deployment_test.yaml b/braintrust/tests/loop-runtime-deployment_test.yaml new file mode 100644 index 0000000..a101aeb --- /dev/null +++ b/braintrust/tests/loop-runtime-deployment_test.yaml @@ -0,0 +1,143 @@ +suite: test Loop Runtime Deployment template +templates: + - loop-runtime-deployment.yaml + - loop-runtime-configmap.yaml +tests: + - it: should not render when Loop Runtime is disabled + values: + - __fixtures__/base-values.yaml + asserts: + - hasDocuments: + count: 0 + + - it: should render the runtime pod with workload identity, secrets, and the client TLS Secret + template: loop-runtime-deployment.yaml + values: &values + - __fixtures__/base-values.yaml + - __fixtures__/loop-runtime-values.yaml + asserts: + - equal: + path: spec.replicas + value: 2 + - equal: + path: spec.template.metadata.labels["gke-workload-identity/use"] + value: "true" + - equal: + path: spec.template.metadata.annotations["iam.gke.io/gcp-service-account"] + value: braintrust-loop@test-project.iam.gserviceaccount.com + - equal: + path: spec.template.spec.containers[0].image + value: test/loop-runtime:v1.0.0 + - equal: + path: spec.template.spec.terminationGracePeriodSeconds + value: 120 + - equal: + path: spec.template.spec.containers[0].livenessProbe.httpGet.port + value: 4001 + - equal: + path: spec.template.spec.containers[0].readinessProbe.httpGet.port + value: 4001 + - contains: + path: spec.template.spec.containers[0].env + content: + name: FUNCTION_SECRET_KEY + valueFrom: + secretKeyRef: + name: braintrust-secrets + key: FUNCTION_SECRET_KEY + - contains: + path: spec.template.spec.volumes + content: + name: sandbox-tls + secret: + secretName: sandbox-client-tls + - contains: + path: spec.template.spec.containers[0].ports + content: + containerPort: 4002 + + - it: should follow the runtime port with a custom port name + template: loop-runtime-deployment.yaml + values: *values + set: + loopRuntime.service.port: 4888 + loopRuntime.service.portName: runtime + asserts: + - equal: + path: spec.template.spec.containers[0].livenessProbe.httpGet.port + value: 4888 + - equal: + path: spec.template.spec.containers[0].readinessProbe.httpGet.port + value: 4888 + + - it: should preserve explicit numeric and named HTTP probe ports + template: loop-runtime-deployment.yaml + values: *values + set: + loopRuntime.service.port: 4888 + loopRuntime.service.portName: runtime + loopRuntime.livenessProbe.httpGet.port: 4999 + loopRuntime.readinessProbe.httpGet.port: runtime + asserts: + - equal: + path: spec.template.spec.containers[0].livenessProbe.httpGet.port + value: 4999 + - equal: + path: spec.template.spec.containers[0].readinessProbe.httpGet.port + value: runtime + + - it: should retain disabled and non-HTTP probes + template: loop-runtime-deployment.yaml + values: *values + set: + loopRuntime.livenessProbe: null + loopRuntime.readinessProbe.httpGet: null + loopRuntime.readinessProbe.exec.command: ["true"] + asserts: + - notExists: + path: spec.template.spec.containers[0].livenessProbe + - isNull: + path: spec.template.spec.containers[0].readinessProbe.httpGet + - equal: + path: spec.template.spec.containers[0].readinessProbe.exec.command + value: ["true"] + + - it: should default null replicas to one + template: loop-runtime-deployment.yaml + values: *values + set: + loopRuntime.replicas: null + asserts: + - equal: + path: spec.replicas + value: 1 + + - it: should default empty replicas to one + template: loop-runtime-deployment.yaml + values: *values + set: + loopRuntime.replicas: "" + asserts: + - equal: + path: spec.replicas + value: 1 + + - it: should scale to zero + template: loop-runtime-deployment.yaml + values: *values + set: + loopRuntime.replicas: 0 + asserts: + - equal: + path: spec.replicas + value: 0 + + - it: should render an all-digit short SHA tag + template: loop-runtime-deployment.yaml + values: *values + set: + loopRuntime.image.tag: "1234567" + asserts: + - equal: + path: spec.template.spec.containers[0].image + value: test/loop-runtime:1234567 diff --git a/braintrust/tests/loop-runtime-egress-service_test.yaml b/braintrust/tests/loop-runtime-egress-service_test.yaml new file mode 100644 index 0000000..d419254 --- /dev/null +++ b/braintrust/tests/loop-runtime-egress-service_test.yaml @@ -0,0 +1,101 @@ +suite: test Loop Runtime egress Service template +templates: + - loop-runtime-egress-service.yaml +tests: + - it: should not render when Loop Runtime is disabled + values: + - __fixtures__/base-values.yaml + asserts: + - hasDocuments: + count: 0 + + - it: should expose the Git proxy on an internal load balancer at the reserved address + values: &values + - __fixtures__/base-values.yaml + - __fixtures__/loop-runtime-values.yaml + asserts: + - equal: + path: metadata.annotations["networking.gke.io/load-balancer-type"] + value: Internal + - equal: + path: spec.loadBalancerIP + value: 10.0.0.50 + - equal: + path: spec.loadBalancerSourceRanges + value: + - 10.64.64.0/24 + - 10.64.0.0/18 + - equal: + path: spec.ports[0].port + value: 4002 + + - it: should retain IPv4 boundary ranges + values: *values + set: + loopRuntime.sandbox.egressGateway.sourceRanges: ["0.0.0.0/0","255.255.255.255/32","10.64.64.0/24"] + asserts: + - equal: + path: spec.loadBalancerSourceRanges + value: ["0.0.0.0/0","255.255.255.255/32","10.64.64.0/24"] + + - it: should reject out of range octets + values: *values + set: + loopRuntime.sandbox.egressGateway.sourceRanges: ["256.0.0.1/24"] + asserts: &invalid-range + - failedTemplate: + errorPattern: "is not a CIDR range$" + + - it: should reject out of range prefixes + values: *values + set: + loopRuntime.sandbox.egressGateway.sourceRanges: ["10.0.0.0/33"] + asserts: *invalid-range + + - it: should reject oversized octets and prefixes + values: *values + set: + loopRuntime.sandbox.egressGateway.sourceRanges: ["999.999.999.999/999"] + asserts: *invalid-range + + - it: should reject an oversized prefix + values: *values + set: + loopRuntime.sandbox.egressGateway.sourceRanges: ["10.0.0.0/999"] + asserts: *invalid-range + + - it: should reject missing octets + values: *values + set: + loopRuntime.sandbox.egressGateway.sourceRanges: ["10.0.0/24"] + asserts: *invalid-range + + - it: should reject a padded range + values: *values + set: + loopRuntime.sandbox.egressGateway.sourceRanges: [" 10.0.0.0/24 "] + asserts: *invalid-range + + - it: should reject a null range + values: *values + set: + loopRuntime.sandbox.egressGateway.sourceRanges: [null] + asserts: *invalid-range + + - it: should reject zero-padded octets + values: *values + set: + loopRuntime.sandbox.egressGateway.sourceRanges: ["010.064.000.000/8"] + asserts: *invalid-range + + - it: should reject zero-padded prefixes + values: *values + set: + loopRuntime.sandbox.egressGateway.sourceRanges: ["10.0.0.0/08"] + asserts: *invalid-range + + - it: should reject host bits beyond the prefix + values: *values + set: + loopRuntime.sandbox.egressGateway.sourceRanges: ["10.64.64.1/24"] + asserts: *invalid-range diff --git a/braintrust/tests/loop-runtime-service_test.yaml b/braintrust/tests/loop-runtime-service_test.yaml new file mode 100644 index 0000000..4bc92b0 --- /dev/null +++ b/braintrust/tests/loop-runtime-service_test.yaml @@ -0,0 +1,25 @@ +suite: test Loop Runtime Service template +templates: + - loop-runtime-service.yaml +tests: + - it: should not render when Loop Runtime is disabled + values: + - __fixtures__/base-values.yaml + asserts: + - hasDocuments: + count: 0 + + - it: should expose the runtime port through a ClusterIP service + values: + - __fixtures__/base-values.yaml + - __fixtures__/loop-runtime-values.yaml + asserts: + - equal: + path: metadata.name + value: braintrust-loop-runtime + - equal: + path: spec.ports[0].port + value: 4001 + - equal: + path: spec.type + value: ClusterIP diff --git a/braintrust/tests/loop-runtime-serviceaccount_test.yaml b/braintrust/tests/loop-runtime-serviceaccount_test.yaml new file mode 100644 index 0000000..f84d085 --- /dev/null +++ b/braintrust/tests/loop-runtime-serviceaccount_test.yaml @@ -0,0 +1,19 @@ +suite: test Loop Runtime ServiceAccount template +templates: + - loop-runtime-serviceaccount.yaml +tests: + - it: should not render when Loop Runtime is disabled + values: + - __fixtures__/base-values.yaml + asserts: + - hasDocuments: + count: 0 + + - it: should bind the Loop Runtime Google service account + values: + - __fixtures__/base-values.yaml + - __fixtures__/loop-runtime-values.yaml + asserts: + - equal: + path: metadata.annotations["iam.gke.io/gcp-service-account"] + value: braintrust-loop@test-project.iam.gserviceaccount.com diff --git a/braintrust/tests/virtualservice_test.yaml b/braintrust/tests/virtualservice_test.yaml index 40fb1ee..b7c4037 100644 --- a/braintrust/tests/virtualservice_test.yaml +++ b/braintrust/tests/virtualservice_test.yaml @@ -336,3 +336,38 @@ tests: asserts: - failedTemplate: errorMessage: "virtualService.workloadIsolation.enabled requires virtualService.enabled." + + - it: should route Loop Runtime before the API fallback + values: + - __fixtures__/base-values.yaml + - __fixtures__/loop-runtime-values.yaml + set: + virtualService.enabled: true + asserts: + - equal: + path: spec.http[0].match[0].uri.exact + value: /loop/runtime + - equal: + path: spec.http[0].match[1].uri.prefix + value: /loop/runtime/ + - equal: + path: spec.http[0].route[0].destination.host + value: braintrust-loop-runtime + - equal: + path: spec.http[0].route[0].destination.port.number + value: 4001 + - equal: + path: spec.http[1].match[0].uri.prefix + value: / + + - it: should render without Loop Runtime routes when reused values have no loopRuntime map + values: + - __fixtures__/base-values.yaml + - __fixtures__/google-values.yaml + - __fixtures__/pre-loop-runtime-values.yaml + set: + virtualService.enabled: true + asserts: + - equal: + path: spec.http[0].match[0].uri.prefix + value: / diff --git a/braintrust/values.yaml b/braintrust/values.yaml index d91754f..4aec8cb 100644 --- a/braintrust/values.yaml +++ b/braintrust/values.yaml @@ -308,6 +308,81 @@ api: extraContainers: [] extraVolumes: [] +# Loop Runtime (optional, cloud: google only). Sandboxes run on Sandbox Service +# workers in a separate GKE cluster; see examples/gke-loop-runtime. +loopRuntime: + enabled: false + # Allow Braintrust staff sysadmins to inspect and terminate sandboxes. + enableSandboxAdmin: false + name: "braintrust-loop-runtime" + labels: {} + podLabels: {} + annotations: + configmap: {} + deployment: {} + service: {} + pod: {} + serviceaccount: {} + replicas: 2 + image: + repository: public.ecr.aws/braintrust/loop-runtime + # Tested with the Sandbox Service worker release in the GKE example. + tag: 38b05599cdc5e4c3a213bbf0718211b397183a7a + pullPolicy: IfNotPresent + service: + # Optional name for service object. If not specified (empty), loopRuntime.name is used. + name: "" + type: ClusterIP + port: 4001 + portName: http + serviceAccount: + name: "braintrust-loop-runtime" + # Terraform output loop_runtime_service_account. + googleServiceAccount: "" + resources: + requests: + cpu: "1" + memory: "2Gi" + ephemeral-storage: "9Gi" + limits: + cpu: "1" + memory: "2Gi" + ephemeral-storage: "9Gi" + livenessProbe: + httpGet: + path: /health/liveness + initialDelaySeconds: 10 + periodSeconds: 30 + timeoutSeconds: 5 + failureThreshold: 3 + successThreshold: 1 + readinessProbe: + httpGet: + path: /health/liveness + initialDelaySeconds: 10 + periodSeconds: 10 + timeoutSeconds: 5 + failureThreshold: 3 + successThreshold: 1 + # Values come from Terraform output gke_isolated_workers. + sandbox: + # discovery.endpoint + dnsUrl: "" + # loop_egress.address, loop_egress.source_ranges, and loop_egress.url + egressGateway: + address: "" + sourceRanges: [] + url: "" + image: "public.ecr.aws/braintrust/codex-runtime@sha256:490ab364fd6a787f73957d3d46bc5e173fb418c47d53bcf7a2d3afba8b419049" + vcpuCount: 1 + memoryMib: 2048 + extraEnvVars: [] + nodeSelector: {} + tolerations: [] + affinity: {} + extraContainers: [] + extraVolumes: [] + # AI Gateway configuration (optional) # Deploys the Braintrust AI gateway service for LLM proxying and completions caching. # Recommended rollout: enabled=true first, wait for gateway pods to be ready, then useGateway=true.