Temporal Proxy (Pre-release)

A gRPC proxy that sits between Temporal SDK Clients, Workers, and the Temporal UI on one side and one or more upstream Temporal Services on the other. It handles Namespace translation, TLS termination, and payload encryption so applications can target a single local endpoint while the proxy fans requests out to the right upstream (a local dev Temporal Service, a self-hosted deployment, Temporal Cloud, or some mix).

Note

Pre-release: This project is under active development and evolving quickly. It is not ready for production use. Open a GitHub issue if you have questions or want to follow along.

Why

Connection details leak into application code. Every Worker and Client has to know the upstream's host, TLS material, credentials, and the exact Namespace name the upstream expects. That couples your code to an environment and makes moving between a local Temporal Service, a self-hosted deployment, and Temporal Cloud a code change.

The proxy pulls that concern out. Workers talk plaintext to a single local endpoint using a short Namespace name; the proxy owns TLS, credentials, and Namespace translation on the way out. Point a Worker at a different Namespace and it reaches a different upstream with no change to the Worker.

How

  flowchart LR
    Worker[Worker]
    Client[SDK Client]
    UI[Web UI]

    subgraph Proxy[Temporal Proxy]
        direction LR
        Gateway["Gateway<br/>routes by Namespace"]
        ProxyA["Per-upstream proxy A<br/>Namespace translation<br/>payload encryption (optional)"]
        ProxyB["Per-upstream proxy B<br/>Namespace translation<br/>payload encryption (optional)"]
        Gateway -->|in process| ProxyA
        Gateway -->|in process| ProxyB
    end

    Cloud[Temporal Cloud]
    SelfHosted[Self-hosted Temporal Service]

    Worker --> Gateway
    Client --> Gateway
    UI --> Gateway
    ProxyA --> Cloud
    ProxyB --> SelfHosted

Loading

Features

• Rule-based routing. Route requests to different upstreams by Namespace and/or request metadata, with a system upstream for Namespace-less calls and a default fallback.

• Service allowlist. Forward only the gRPC services you name, defaulting to WorkflowService and OperatorService. Server reflection is opt-in, and a service you leave out is never forwarded.

• Namespace translation. Rewrite local Namespace names to the names an upstream expects (prefix, suffix, or explicit overrides) in both requests and responses.

• TLS termination and outbound credentials. Terminate inbound TLS/mTLS and attach the upstream's own TLS and credentials (API key or mTLS), so client code carries none of it.

• SRV discovery. Set an upstream's hostPort to srv:///<record> (for example srv:///_grpc._tcp.temporal-frontend.temporal.svc.cluster.local) and the proxy balances round-robin across every backend the record lists. It re-reads the record every 30 seconds, and within 5 seconds of a backend failing. maxConnections (default 32) applies per backend, so 4 connections across 10 backends opens 40. Over TLS, every backend's certificate is verified against the record name minus its leading _service._proto labels (temporal-frontend.temporal.svc.cluster.local above), not the host each record points at; set tls.serverName when the certificates carry a different name.

• Payload encryption. Optionally seal payloads with envelope encryption on the hop to an upstream and open them on responses, so the upstream only ever sees ciphertext while local Workers keep exchanging cleartext. DEKs are wrapped by a KMS key (AWS KMS, Azure Key Vault, or GCP KMS), rotate automatically, and can be overridden per Namespace. Set encryption.failures to seal failure messages and stack traces too, as the SDK's EncodeCommonAttributes does.

  Payloads a Worker's own codec already encrypted, or that an earlier proxy sealed, can be forwarded as they are by listing their encoding under encryption.skipEncodings. Listing an encoding trusts every Worker that sets it: the proxy cannot tell ciphertext from plaintext labeled that way. Listing binary/encrypted also returns responses sealed under a KEK this proxy doesn't hold, including its own if a key was removed from config, rather than failing the call, so alert on vault_ops_total{result="unknown_key"}.

• Pluggable key management. For a backend the proxy has no built-in support for, such as an on-prem HSM or an internal key service, point it at an extension server you run and it wraps DEKs through that instead. Only key material is exchanged; payloads never reach it.

• Codec server. Serve the /encode, /decode, and /download routes the Temporal CLI and Web UI use to seal and open payloads directly, without a Worker's data converter in the loop. A codec server holds the same KMS unwrap permission the proxy itself does, so one that is reachable without authentication is a decryption oracle; see Codec server for the config and how it is locked down.

• Inbound authentication and authorization. Optional static-token or JWKS validation on the gateway; off by default. For rules neither covers, delegate the decision to an extension server you run. It is told what the call is addressing (the gRPC method, and the Namespace the proxy resolved from the request rather than from anything the caller claims), so it can decide per Namespace and per method rather than only whether the caller is who it says it is.

• Prometheus metrics. Expose request latency and counts, routing decisions, and payload sealing and opening on /metrics. The listen address and the metric prefix stamped onto every metric name are set under metrics: in the config, along with the labels every series carries: constants such as the region the proxy runs in, and request metadata carried onto the request-scoped series, so they can be sliced by a dimension only your callers know.

• Codec-transparent. Payload data passes through untouched unless encryption is configured. The proxy reads only the Namespace and the fields it translates, and needs no knowledge of your data converter or codec.

• Multiple deployment options. Ship as a Go binary, a container image, or a Helm chart.

Installation

Go

Install the proxy binary into your $GOBIN with go install:

go install github.com/temporalio/temporal-proxy/cmd/proxy@latest

@latest resolves to the newest stable release. Pin an explicit version from the releases page if you prefer:

go install github.com/temporalio/temporal-proxy/cmd/proxy@vX.Y.Z

Container image

Images are published to Docker Hub:

docker pull temporalio/temporal-proxy:latest

Helm

A chart is published to the Temporal Helm repo at https://go.temporal.io/helm-charts:

# Latest stable release
helm install temporal-proxy temporal-proxy \
  --repo https://go.temporal.io/helm-charts

# Or pin a specific proxy version (see the releases page)
helm install temporal-proxy temporal-proxy \
  --repo https://go.temporal.io/helm-charts \
  --set image.tag=vX.Y.Z

Each chart release deploys a proxy version by default; --set image.tag overrides it to pin a specific one.

Supply the proxy config under the config: key in a values file and pass it with -f:

# values.yaml
config:
  hostPort: :7233
  upstreams:
    - name: local
      hostPort: localhost:7234
      insecure: true
  routing:
    default: local

helm install temporal-proxy temporal-proxy \
  --repo https://go.temporal.io/helm-charts \
  -f values.yaml

See the chart README for the full set of options.

Get started

The Temporal Cloud example is the quickest way to see the proxy in action: a Worker and starter that carry no Cloud configuration talk plaintext to localhost:7233, and the proxy adds TLS, the API key, and the Namespace rewrite on the way to Cloud. Follow its README to run it end to end.

The KMS extension server example shows the pluggable key management path end to end: a local dev server, a key provider you run, and Workflow payloads that the Temporal Service only ever stores as ciphertext. It is built on pkg/ext, which supplies the gRPC surface, the credential check, TLS, and graceful shutdown, so writing your own extension server means implementing the key handling and little else.

The authorization example does the same for access control: an extension server maps a JWT to claims and then decides each call against them, the two steps Temporal OSS splits across its ClaimMapper and Authorizer. Four tokens show what that buys, including a Worker that cannot reach a second Namespace and an auditor that can read history but not start a Workflow.

Codec server

The proxy can serve the Temporal SDK's codec server HTTP contract, so the Temporal CLI and Web UI can seal and open payloads directly, without a Worker's data converter in the loop. It answers POST /encode, POST /decode, and POST /download, each also reachable under a Namespace path segment (/{namespace}/decode, and so on) for the CLI's --codec-endpoint templating. An unknown path answers 404; a known one called with anything but POST answers 405, which is a deliberate divergence from the Temporal SDK's own codec handler, whose equivalent case answers 404.

Config keys under codecServer::

Key	Default	Meaning
enabled	false	Turns the codec server on.
hostPort	none	Address it listens on.
insecure	false	Serve plaintext instead of TLS. Validation only allows this on a loopback bind.
tls	none	Certificate and key to terminate TLS with.
cors.origins	none	Origins a browser-based caller, such as the Cloud UI, may reach the codec server from.
cors.credentials	false	Whether a browser may send cookies and Authorization alongside an allowed origin.
auth	none	Static-token or JWKS authentication. Required, along with tls, once hostPort is not loopback.

A codec server holds the same KMS unwrap permission the proxy itself does: whoever can reach it can decrypt anything an operator's keys protect. That is why configuration requires authentication and TLS the moment hostPort is not loopback, and why a loopback bind, meant for local development where the caller is on the same host, is left unauthenticated by default rather than by oversight. Point one at anything but 127.0.0.1 or localhost only once auth and tls are configured.

auth verifies who is calling, not which Namespace they may decrypt: a token that passes it authorizes every Namespace's payloads, not just one. Scope a JWKS token narrowly at the issuer, or configure an extension-server authorizer, if a per-Namespace decision matters.

On a 4xx response from /decode, the Temporal Web UI does not surface an error: it renders whatever payload it already had rather than the plaintext, so a rejected token or a malformed request looks the same to whoever is looking at it as ciphertext the UI has not decoded yet (binary/encrypted), not an obvious failure.

Metrics

The proxy serves Prometheus metrics on /metrics. Everything under metrics: in the config controls the endpoint and how series are labeled:

Key	Default	Meaning
hostPort	:9090	Address the /metrics handler listens on.
namespace	tmprl_proxy	The metric prefix stamped onto every metric name. Unrelated to a Temporal Namespace.
labels.namespace	false	Whether series that can name a Temporal Namespace report it.
labels.fixed	none	Constant labels stamped on every series, written as a name-to-value map.
labels.metadata	none	Request metadata to report as extra labels, each written <header>:<name>.

Published series

Every name below is prefixed with the metric prefix and its subsystem, so requests_total in the server subsystem is exposed as tmprl_proxy_server_requests_total by default. Each configured metadata label is added to the series emitted while serving a request: the server and router series, and vault_ops. The KEK and DEK series do not carry them, because they are emitted off the request path where no metadata is in scope to read. Neither does vault_ops when the operation came from the codec server: an HTTP request carries no gRPC metadata, so a metadata label reports blank there even though the same series populates it for a request that passed through the gateway. A fixed label is added to every series in the table, and to nothing registered outside the proxy's own collectors, so the runtime's go_* and process_* series stay as they are.

payloads_skipped_total carries the metadata labels too (blank, as with vault_ops, for operations from the codec server, since an HTTP request carries no gRPC metadata), and vault_ops_total reports result="unknown_key" for a payload sealed under a KEK the proxy doesn't hold.

Subsystem	Metric	Type	Labels
server	requests_total	counter	method, code
server	request_duration_seconds	histogram	method
server	panics_total	counter	method
router	decisions_total	counter	upstream, outcome
router	forwarding_errors_total	counter	upstream, reason
encryption	vault_ops_total	counter	operation, result, namespace
encryption	vault_ops_duration_seconds	histogram	operation, namespace
encryption	payloads_skipped_total	counter	operation, encoding, namespace
encryption	kek_ops_total	counter	provider, operation, result
encryption	kek_ops_duration_seconds	histogram	provider, operation
encryption	dek_ops_total	counter	operation, result
encryption	dek_ops_duration_seconds	histogram	operation
encryption	dek_rotations_total	counter	reason
encryption	dek_cache_hits_total	counter	none
encryption	dek_cache_misses_total	counter	none
encryption	dek_cache_size	gauge	none
codec_server	requests_total	counter	route, code
codec_server	request_duration_seconds	histogram	route

The encryption subsystem only reports once encryption keys are configured, and codec_server only reports once the codec server is enabled. Its route label is the matched pattern (/decode, and so on), never the request path, since the Namespace-prefixed routes carry a Namespace there and labeling by it would be unbounded cardinality; there is no Namespace label on this subsystem for the same reason.

Labels and cardinality

namespace is always declared and reports an empty value when it is turned off. Prometheus treats an empty label value as the label not being there, so turning it on leaves the shape of a query that already ignores it unchanged.

namespace is the local Namespace the Client asked for, before any Namespace translation, which is the same name you write under encryption.overrides. Two proxies fronting different Temporal Services can therefore both report a Namespace called default, so a shared Prometheus needs something else to tell those series apart: a labels.fixed entry, or a label from the scrape job.

labels.namespace is off by default because the label is unbounded: the value comes from the request, so every distinct Namespace a Client names becomes another series. Turn it on when you know that set is small. method is bounded the same way, only for trusted callers, which is why the gateway should not be exposed directly to untrusted Clients.

A labels.metadata entry is unbounded for the same reason, and it multiplies every request-scoped series rather than adding to them, so name a header whose values you know. Two things to know before naming one: /metrics is served unauthenticated, so a header carrying a credential publishes it, and a name that collides with a label a series already declares fails at startup with a message naming the collision.

A labels.fixed entry costs no cardinality, since its value never varies, which is what makes it the right place for a per-installation fact like a region, a zone, or a cluster name. It is also the only shape that reaches the KEK and DEK series. The same collision rule applies, and one more: a name a labels.metadata entry already reports under is refused, because Prometheus rejects a collector whose constant labels clash with its variable ones.

Stability

These names and labels are a published contract, pinned by a test that fails on any change to them. Renames ship with a release note.

Terms

Term	Meaning
gateway	The single inbound gRPC endpoint that every SDK Client, Worker, and the UI connects to. It routes each request to an upstream by Namespace and/or request metadata, and never parses payloads.
upstream	A configured destination the proxy forwards to: a Temporal Service (local dev, self-hosted, or Temporal Cloud), or another Temporal Proxy.
system upstream	The upstream that handles Namespace-less requests, such as the SDK's GetSystemInfo call on connect.
extension server	A gRPC service you run that the proxy calls out to for a capability it has no built-in backend for: wrapping DEKs, or deciding if a call may proceed. Build one with pkg/ext.
Temporal Service	A Temporal deployment the proxy connects to: a local dev server, a self-hosted deployment, or Temporal Cloud.

Development

See .github/CONTRIBUTING.md for the dev loop. The common entry points are mise run test, mise run lint, and mise run format.

Security

See SECURITY.md for how to report vulnerabilities.

License

MIT, see LICENSE.