redis: vendor redis.conf and auto-size maxmemory from the memory limit

charts/sourcegraph/CHANGELOG.md

@@ -8,6 +8,8 @@ Use `**BREAKING**:` to denote a breaking change
 
 ## Unreleased
 
+- Added opt-in Redis config management through `redisCache.config.enabled` and `redisStore.config.enabled` (both default to `false`). Default deployments retain their image configs, custom mounts, authentication, and memory settings. When enabled, the chart mounts `/etc/redis/redis.conf` and supports `existingConfig`, `maxmemory`, `maxmemoryRatio`, `maxmemoryPolicy`, and `additionalConfig`. See [Enabling Redis config management](README.md#enabling-redis-config-management) before opting in, especially when using custom images or Secret-mounted configs.
+- With Redis config management enabled, the chart sizes `maxmemory` at 75% of `resources.limits.memory` to leave room for overhead; this reduces, but does not eliminate, OOM risk. Ratios must be greater than 0 and less than 1. Auto-sizing is skipped in local development mode or when the limit is absent or unrecognised. At the default 7Gi limit, opting in lowers the effective cap from 6GiB to 5.25GiB: `redis-cache` evicts earlier, but `redis-store` uses `noeviction`, so its write-error ceiling drops too. Set `redisStore.config.maxmemory: 6gb` to keep the old ceiling if the pod has sufficient memory.
 - Added a `network-policy` example, which limits Executor and Executor job pods to the frontend API
 - Corrected the external object storage examples to configure the shared store for frontend, worker, precise code intel, syntactic code intel, gitserver, and searcher, including credentials or workload service accounts as required.
 - Removed the unused application ports from the precise and syntactic code intel worker Deployments and Services; health checks and Prometheus metrics continue to use the debug server on port 6060.

charts/sourcegraph/README.md

@@ -7,6 +7,62 @@
 
 Visit the [Helm docs](https://docs.sourcegraph.com/admin/install/kubernetes) for guidance on using this chart.
 
+## Enabling Redis config management
+
+Redis config management is **disabled by default**. Upgrading the chart preserves
+existing image configs, custom mounts, authentication, and memory settings.
+The standard images retain their 6GiB cap, which can cause OOM kills if the pod
+memory limit is lower. Opt in for either service independently, or both:
+
+```yaml
+redisCache:
+  config:
+    enabled: true
+redisStore:
+  config:
+    enabled: true
+```
+
+For each enabled service, the chart mounts `/etc/redis/redis.conf` and sizes
+`maxmemory` to 75% of the container's memory limit. At the default 7Gi limit,
+opting in changes the cap from 6GiB to 5.25GiB. `redis-cache` evicts earlier;
+`redis-store` keeps `noeviction` and rejects writes sooner when full. Set
+`redisStore.config.maxmemory: 6gb` to retain its old cap, provided the pod has
+enough memory for Redis overhead. Auto-sizing reserves headroom but cannot
+guarantee against OOM kills, particularly during persistence operations.
+
+Before opting in:
+
+- **Custom images:** the mounted config replaces the image's config, including
+  any `requirepass`, ACL, TLS, or persistence settings. Leave
+  `redisCache.config.enabled: false` and/or `redisStore.config.enabled: false`
+  to preserve the corresponding image config. Otherwise, migrate those settings
+  explicitly; image-baked authentication is not inherited.
+- **Existing config mounts:** either leave config management disabled and
+  keep your `extraVolumes` / `extraVolumeMounts`, or remove those mounts and set
+  `config.enabled: true` and `config.existingConfig` to the name of a ConfigMap
+  with a complete `redis.conf` key. An existing ConfigMap bypasses all chart
+  sizing and directive overrides.
+- **Secrets:** both `additionalConfig` and `existingConfig` use plaintext
+  ConfigMaps. For a config containing credentials, leave config management disabled
+  and mount a Secret at `/etc/redis/redis.conf` with `extraVolumeMounts` instead.
+  Configure the clients' `connection.existingSecret` and the exporter's
+  `redisExporter.env.REDIS_PASSWORD.valueFrom.secretKeyRef` as appropriate.
+  Redis readiness alone does not verify exporter authentication; check `redis_up`.
+
+While config management is disabled, all other `config` options are ignored,
+including auto-sizing, so you must size Redis memory yourself. With management enabled,
+an explicit `config.maxmemory` takes precedence over auto-sizing, and
+`additionalConfig` is appended last. Supported memory limits are plain byte
+counts or numbers with `k`, `M`, `G`, `T`, `P`, `E`, `Ki`, `Mi`, `Gi`, `Ti`,
+`Pi`, or `Ei` suffixes. If the limit is absent or unrecognised, or
+`sourcegraph.localDevMode` is enabled, the vendored 6gb default remains unless
+overridden explicitly.
+
+Chart-managed config changes restart the Redis pod through a checksum
+annotation. Changes to an existing ConfigMap or Secret require a manual pod
+restart because `subPath` mounts do not update in running containers.
+
 ## Configuration Options
 
 Reference the below chart for all available configuration parameters.
@@ -301,6 +357,12 @@ In addition to the documented values, all services also support the following va
 | prometheus.storageAnnotations | object | `{}` | Optional annotations to add to the `prometheus` PVC |
 | prometheus.storageSize | string | `"200Gi"` | PVC Storage Request for `prometheus` data volume |
 | prometheus.storageSubPath | string | `""` | Optional subPath for the `prometheus` primary data volume mount |
+| redisCache.config.additionalConfig | string | `""` | Additional raw redis directives appended to the vendored `redis-cache` config. Notes: This is expecting a multiline string. It renders into a ConfigMap in plaintext, so do not put secrets such as `requirepass` here or in `existingConfig` (also a ConfigMap). For secrets, set `config.enabled: false` and mount a Secret using `extraVolumeMounts`. |
+| redisCache.config.enabled | bool | `false` | Opt in to a chart-managed Redis config and automatic memory sizing. Disabled by default to preserve image-baked configuration and custom `extraVolumeMounts` (including Secret mounts). When false, all other `config` options are ignored and auto-sizing is disabled. |
+| redisCache.config.existingConfig | string | `""` | Name of an existing ConfigMap for `redis-cache`. It must contain a `redis.conf` key. When set, the chart-managed ConfigMap is not rendered and this one is mounted instead, so the chart no longer sizes `maxmemory`. Mutually exclusive with `additionalConfig`. |
+| redisCache.config.maxmemory | string | `""` | Explicit redis `maxmemory` for `redis-cache` (for example `6gb`). Overrides the auto-computed value. Empty means compute it from the container memory limit. |
+| redisCache.config.maxmemoryPolicy | string | `""` | Override the redis `maxmemory-policy` for `redis-cache`. Empty keeps the vendored default (`allkeys-lru`). |
+| redisCache.config.maxmemoryRatio | float | `0.75` | Fraction of `redisCache.resources.limits.memory` used for `maxmemory` when `maxmemory` is empty. Must be greater than 0 and less than 1. Ignored when no memory limit is set or `sourcegraph.localDevMode` is enabled. An absent or unrecognised limit keeps the vendored 6gb default; use an explicit maxmemory if needed. |
 | redisCache.connection.endpoint | string | `"redis-cache:6379"` | Endpoint to use for redis-cache. Supports either host:port or IANA specification |
 | redisCache.connection.existingSecret | string | `""` | Name of existing secret to use for Redis endpoint The secret must contain the key `endpoint` and should follow IANA specification learn more from the [Helm docs](https://docs.sourcegraph.com/admin/install/kubernetes/helm#using-external-redis-instances) |
 | redisCache.containerSecurityContext | object | `{"allowPrivilegeEscalation":false,"readOnlyRootFilesystem":true,"runAsGroup":1000,"runAsUser":999}` | Security context for the `redis-cache` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) |
@@ -320,6 +382,12 @@ In addition to the documented values, all services also support the following va
 | redisExporter.image.defaultTag | string | `"6.0.0@sha256:b2ec48fc6adef31f36d525170138dec303c1c0c20c530d659f1fb7c6c54698af"` | Docker image tag for the `redis-exporter` image |
 | redisExporter.image.name | string | `"redis_exporter"` | Docker image name for the `redis-exporter` image |
 | redisExporter.resources | object | `{"limits":{"cpu":"10m","memory":"100Mi"},"requests":{"cpu":"10m","memory":"100Mi"}}` | Resource requests & limits for the `redis-exporter` sidecar container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
+| redisStore.config.additionalConfig | string | `""` | Additional raw redis directives appended to the vendored `redis-store` config. Notes: This is expecting a multiline string. It renders into a ConfigMap in plaintext, so do not put secrets such as `requirepass` here or in `existingConfig` (also a ConfigMap). For secrets, set `config.enabled: false` and mount a Secret using `extraVolumeMounts`. |
+| redisStore.config.enabled | bool | `false` | Opt in to a chart-managed Redis config and automatic memory sizing. Disabled by default to preserve image-baked configuration and custom `extraVolumeMounts` (including Secret mounts). When false, all other `config` options are ignored and auto-sizing is disabled. |
+| redisStore.config.existingConfig | string | `""` | Name of an existing ConfigMap for `redis-store`. It must contain a `redis.conf` key. When set, the chart-managed ConfigMap is not rendered and this one is mounted instead, so the chart no longer sizes `maxmemory`. Mutually exclusive with `additionalConfig`. |
+| redisStore.config.maxmemory | string | `""` | Explicit redis `maxmemory` for `redis-store` (for example `6gb`). Overrides the auto-computed value. Empty means compute it from the container memory limit. |
+| redisStore.config.maxmemoryPolicy | string | `""` | Override the redis `maxmemory-policy` for `redis-store`. Empty keeps the vendored default (`noeviction`). |
+| redisStore.config.maxmemoryRatio | float | `0.75` | Fraction of `redisStore.resources.limits.memory` used for `maxmemory` when `maxmemory` is empty. Must be greater than 0 and less than 1. Ignored when no memory limit is set or `sourcegraph.localDevMode` is enabled. An absent or unrecognised limit keeps the vendored 6gb default; use an explicit maxmemory if needed. |
 | redisStore.connection.endpoint | string | `"redis-store:6379"` | Endpoint to use for redis-store. Supports either host:port or IANA specification |
 | redisStore.connection.existingSecret | string | `""` | Name of existing secret to use for Redis endpoint The secret must contain the key `endpoint` and should follow IANA specification learn more from the [Helm docs](https://docs.sourcegraph.com/admin/install/kubernetes/helm#using-external-redis-instances) |
 | redisStore.containerSecurityContext | object | `{"allowPrivilegeEscalation":false,"readOnlyRootFilesystem":true,"runAsGroup":1000,"runAsUser":999}` | Security context for the `redis-store` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) |

charts/sourcegraph/README.md.gotmpl

@@ -7,6 +7,62 @@
 
 Visit the [Helm docs](https://docs.sourcegraph.com/admin/install/kubernetes) for guidance on using this chart.
 
+## Enabling Redis config management
+
+Redis config management is **disabled by default**. Upgrading the chart preserves
+existing image configs, custom mounts, authentication, and memory settings.
+The standard images retain their 6GiB cap, which can cause OOM kills if the pod
+memory limit is lower. Opt in for either service independently, or both:
+
+```yaml
+redisCache:
+  config:
+    enabled: true
+redisStore:
+  config:
+    enabled: true
+```
+
+For each enabled service, the chart mounts `/etc/redis/redis.conf` and sizes
+`maxmemory` to 75% of the container's memory limit. At the default 7Gi limit,
+opting in changes the cap from 6GiB to 5.25GiB. `redis-cache` evicts earlier;
+`redis-store` keeps `noeviction` and rejects writes sooner when full. Set
+`redisStore.config.maxmemory: 6gb` to retain its old cap, provided the pod has
+enough memory for Redis overhead. Auto-sizing reserves headroom but cannot
+guarantee against OOM kills, particularly during persistence operations.
+
+Before opting in:
+
+- **Custom images:** the mounted config replaces the image's config, including
+  any `requirepass`, ACL, TLS, or persistence settings. Leave
+  `redisCache.config.enabled: false` and/or `redisStore.config.enabled: false`
+  to preserve the corresponding image config. Otherwise, migrate those settings
+  explicitly; image-baked authentication is not inherited.
+- **Existing config mounts:** either leave config management disabled and
+  keep your `extraVolumes` / `extraVolumeMounts`, or remove those mounts and set
+  `config.enabled: true` and `config.existingConfig` to the name of a ConfigMap
+  with a complete `redis.conf` key. An existing ConfigMap bypasses all chart
+  sizing and directive overrides.
+- **Secrets:** both `additionalConfig` and `existingConfig` use plaintext
+  ConfigMaps. For a config containing credentials, leave config management disabled
+  and mount a Secret at `/etc/redis/redis.conf` with `extraVolumeMounts` instead.
+  Configure the clients' `connection.existingSecret` and the exporter's
+  `redisExporter.env.REDIS_PASSWORD.valueFrom.secretKeyRef` as appropriate.
+  Redis readiness alone does not verify exporter authentication; check `redis_up`.
+
+While config management is disabled, all other `config` options are ignored,
+including auto-sizing, so you must size Redis memory yourself. With management enabled,
+an explicit `config.maxmemory` takes precedence over auto-sizing, and
+`additionalConfig` is appended last. Supported memory limits are plain byte
+counts or numbers with `k`, `M`, `G`, `T`, `P`, `E`, `Ki`, `Mi`, `Gi`, `Ti`,
+`Pi`, or `Ei` suffixes. If the limit is absent or unrecognised, or
+`sourcegraph.localDevMode` is enabled, the vendored 6gb default remains unless
+overridden explicitly.
+
+Chart-managed config changes restart the Redis pod through a checksum
+annotation. Changes to an existing ConfigMap or Secret require a manual pod
+restart because `subPath` mounts do not update in running containers.
+
 ## Configuration Options
 
 Reference the below chart for all available configuration parameters.

charts/sourcegraph/files/redis-cache/redis.conf

@@ -0,0 +1,17 @@
+# Vendored from the `redis-cache` image (`/etc/redis/redis.conf`).
+# Source: sourcegraph/sourcegraph docker-images/redis-cache/redis.conf
+# Seeded from image tag 6.0.0. Keep this file in sync when the image tag changes.
+#
+# The chart mounts this file over /etc/redis/redis.conf, so the image entrypoint
+# (`redis-server /etc/redis/redis.conf`) reads it. The chart appends the computed
+# `maxmemory` and any `redisCache.config` overrides after this body.
+
+# allow access from all instances
+protected-mode no
+# limit memory usage, discard unused keys when hitting limit
+maxmemory 6gb
+maxmemory-policy allkeys-lru
+# snapshots on disk every minute
+dir /redis-data/
+appendonly no
+save 60 1

charts/sourcegraph/files/redis-store/redis.conf

@@ -0,0 +1,18 @@
+# Vendored from the `redis-store` image (`/etc/redis/redis.conf`).
+# Source: sourcegraph/sourcegraph docker-images/redis-store/redis.conf
+# Seeded from image tag 6.0.0. Keep this file in sync when the image tag changes.
+#
+# The chart mounts this file over /etc/redis/redis.conf, so the image entrypoint
+# (`redis-server /etc/redis/redis.conf`) reads it. The chart appends the computed
+# `maxmemory` and any `redisStore.config` overrides after this body.
+
+# allow access from all instances
+protected-mode no
+# limit memory usage, return error when hitting limit
+maxmemory 6gb
+maxmemory-policy noeviction
+# live commit log to disk, additionally snapshot every 5 minutes
+dir /redis-data/
+appendonly yes
+aof-use-rdb-preamble yes
+save 300 1

charts/sourcegraph/templates/_helpers.tpl

@@ -312,3 +312,66 @@ checksum/auth: {{ toJson $checksum | sha256sum }}
 {{- $checksum := append $checksum .Values.redisCache.connection -}}
 checksum/redis: {{ toJson $checksum | sha256sum }}
 {{- end -}}
+
+{{/*
+Resolve the redis `maxmemory` directive for a service.
+Usage: include "sourcegraph.redis.maxmemory" (list . "redisCache")
+
+Resolution order:
+  1. <service>.config.maxmemory, used verbatim.
+  2. floor(<service>.config.maxmemoryRatio * <service>.resources.limits.memory),
+     rendered as a plain byte count, unless localDevMode removes the limit.
+  3. Empty string, when there is no memory limit or the quantity is not
+     recognised. The caller then emits no `maxmemory` and the vendored default
+     stands.
+*/}}
+{{- define "sourcegraph.redis.maxmemory" -}}
+{{- $top := index . 0 -}}
+{{- $service := index . 1 -}}
+{{- $values := index $top.Values $service -}}
+{{- $config := $values.config | default dict -}}
+{{- if $config.maxmemory -}}
+{{- $config.maxmemory -}}
+{{- else if not $top.Values.sourcegraph.localDevMode -}}
+{{- $limit := dig "resources" "limits" "memory" "" $values | toString -}}
+{{- $number := regexReplaceAll "^([0-9]+(\\.[0-9]+)?).*$" $limit "${1}" -}}
+{{- $suffix := regexReplaceAll "^[0-9]+(\\.[0-9]+)?" $limit "" -}}
+{{- /* Kubernetes quantity suffixes: binary (1024^n) and decimal (1000^n). */ -}}
+{{- $units := dict "" 1.0 "k" 1e3 "M" 1e6 "G" 1e9 "T" 1e12 "P" 1e15 "E" 1e18 "Ki" 1024.0 "Mi" 1048576.0 "Gi" 1073741824.0 "Ti" 1099511627776.0 "Pi" 1125899906842624.0 "Ei" 1152921504606846976.0 -}}
+{{- if and (regexMatch "^[0-9]+(\\.[0-9]+)?$" $number) (hasKey $units $suffix) -}}
+{{- $ratio := 0.75 -}}
+{{- if hasKey $config "maxmemoryRatio" -}}
+{{- $ratio = float64 $config.maxmemoryRatio -}}
+{{- end -}}
+{{- if not (and (gt $ratio 0.0) (lt $ratio 1.0)) -}}
+{{- fail (printf "%s.config.maxmemoryRatio must be greater than 0 and less than 1" $service) -}}
+{{- end -}}
+{{- $bytes := floor (mulf (float64 $number) (index $units $suffix) $ratio) -}}
+{{- if gt $bytes 0.0 -}}
+{{- printf "%d" (int64 $bytes) -}}
+{{- end -}}
+{{- end -}}
+{{- end -}}
+{{- end -}}
+
+{{/*
+Fail the render when a service's extra volumes or volume mounts collide with the
+chart-managed redis config mount. Without this the collision only surfaces as an
+opaque "must be unique" rejection from the API server.
+Usage: include "sourcegraph.redis.assertNoConfClash" (list . "redisCache")
+*/}}
+{{- define "sourcegraph.redis.assertNoConfClash" -}}
+{{- $top := index . 0 -}}
+{{- $service := index . 1 -}}
+{{- $values := index $top.Values $service -}}
+{{- range ($values.extraVolumeMounts | default list) -}}
+{{- if has .mountPath (list "/etc/redis/redis.conf" "/etc/redis" "/etc/redis/") -}}
+{{- fail (printf "%s.extraVolumeMounts must not mount over /etc/redis/redis.conf; the chart now manages that file. Move your custom redis config to %s.config.existingConfig or %s.config.additionalConfig, or set %s.config.enabled=false to preserve your existing mount." $service $service $service $service) -}}
+{{- end -}}
+{{- end -}}
+{{- range (concat ($values.extraVolumes | default list) ($values.extraVolumeMounts | default list)) -}}
+{{- if eq (.name | toString) "redis-conf" -}}
+{{- fail (printf "%s must not define a volume named 'redis-conf'; the chart reserves that name for the redis config mount." $service) -}}
+{{- end -}}
+{{- end -}}
+{{- end -}}