diff --git a/docs/stream.md b/docs/stream.md new file mode 100644 index 000000000..ca5bf1d3a --- /dev/null +++ b/docs/stream.md @@ -0,0 +1,145 @@ +# `temporal stream` + +Streams are append-only logs the Temporal Service carries next to Workflow +Executions. This fork adds a `temporal stream` command group on top of the +stream service, and a dev server that serves it. + +## Dev server + +`temporal server start-dev` registers the stream service and turns it on for +every namespace it creates. The server ships with `stream.enabled` off, so the +dev server sets it as one of its own dynamic-config defaults, the same way it +sets the CHASM and standalone-activity flags. An explicit value still wins: + +```sh +temporal server start-dev --dynamic-config-value 'stream.enabled=false' +``` + +The server behind the dev server is pinned in `go.mod` to the stream branch of +`moedash/temporal`, with the matching `moedash/api-go` pin the server needs. + +## Addressing a stream + +Two kinds of stream exist. A standalone stream has an ID of its own. An owned +stream lives inside a Workflow Execution or an Activity and is named by its +owner and a stream name. + +- `--stream-id ID`: the standalone stream `ID`. +- `--workflow-id W [--run-id R] [--name N]`: the stream `N` the workflow owns. + Without `--name` it is the workflow's default stream. +- `--workflow-id W --activity-id A [--name N]`: the stream of an activity the + workflow scheduled. +- `--activity-id A [--run-id R] [--name N]`: the stream of a standalone + activity. + +`describe`, `read` and `append` take either form. `create`, `close`, +`truncate` and `delete` act on a standalone stream only, because an owned +stream's lifecycle is its owner's. + +## Commands + +`stream create --stream-id ID [--retention D] [--max-items N] [--max-bytes N]` +: Creates a standalone stream. Retention defaults to the namespace's; records + older than it are reclaimed. `--max-items` is a rolling window on records, + `--max-bytes` a ceiling on held bytes that refuses appends until records + are reclaimed. Repeating a create with the same lifecycle answers "already + exists"; a different lifecycle is refused. + +`stream list [--query Q] [--limit N] [--page-size N]` +: Lists standalone streams from visibility. `WorkflowId` in the query is the + stream ID. Owned streams are not listed; reach them through their owner. + +`stream describe ` +: Frontier, floor, readable record count, held and appended bytes, close + state and reason, retention, caps, budget, producers and consumers. Also + the notification channel the stream notifies on each append and close, + derived from the reference alone: `stream/NAME` linked to the owner for a + workflow's or a standalone activity's stream, `stream/ACTIVITY_ID/NAME` + linked to the workflow for an activity it scheduled, and the independent + `stream/STREAM_ID` for a standalone stream. The card names the owner as + `LinkedTo workflow ID` or `LinkedTo activity ID`, with the run when one was + given. `temporal channel poll --channel C` with the same `--workflow-id` or + `--activity-id` follows it. + +`stream read [--from-offset N | --from-tail | --last N] [--follow] +[--topic T]... [--limit N]` +: Prints records with offset, kind, topic, producer, attempt, sequence and + body. Stops once caught up, or with `--follow` when the stream closes. + +`stream append --input V... [--topic T] [--producer-id P] [--attempt A] +[--sequence S] [--expected-offset N] [--finish]` +: Appends one record per `--input` or `--input-file`, or per line of stdin. + `--producer-id` with `--sequence` deduplicates a retry. `--finish` adds a + `FINISH` record after the values. + +`stream close --stream-id ID [--reason R]` +: Seals the stream. Records stay readable until retention passes. + +`stream truncate --stream-id ID --to N` +: Moves the floor to `N`. Refused below an active consumer's floor. + +`stream delete --stream-id ID [--force]` +: Deletes the stream and its records. Refused while a workflow consumes it, + unless forced. + +`--output json` prints one JSON object per record or list entry, and the +stream state proto for `describe`. + +Authorization follows the server's declaration: `truncate` and `delete` need +the admin role on the namespace, `list`, `describe` and `read` the read role, +and the rest the write role. + +## Notification channels + +`temporal channel notify|describe|poll|listener add|listener remove` reach the +channel a stream notifies, or any other channel, by `--channel` (`-c`). Without +an owner the commands use the independent channel of that name. With +`--workflow-id W` or `--activity-id A`, and an optional `--run-id`, they use +the channel linked to that Workflow Execution or standalone Activity, sent as +the request's `execution` with the matching type. The two owner flags cannot +be combined, and `--run-id` alone is refused. + +`channel describe` prints the kind and, for a linked channel, the owner line +`LinkedTo workflow W (run R)` or `LinkedTo activity A (run R)`, the run only +when the Service names one. The JSON output carries `linkedTo` as the Service +sends it. + +## Refusals + +A refusal the caller has to act on comes back from the server as a +`FailedPrecondition` whose message starts with a reason token. `create`, +`read` and `append` read the token and print what to do instead of the token, +with the server's own detail after it: + +- `STREAM_PRODUCER_CONFLICT`: the producer already appended different content + at this sequence; nothing was written. +- `STREAM_PRODUCER_STALE_SEQUENCE`: the sequence is below the producer's + latest; nothing was written. +- `STREAM_CURSOR_BELOW_FLOOR`: the read starts below the floor; those records + are gone. +- `STREAM_CLOSED`: the stream is closed; its records stay readable. +- `STREAM_POLICY_MISMATCH`: a stream with this id exists with a different + lifecycle. + +## Payload codec + +The records' bodies and metadata go through the same remote codec as every +other payload the CLI shows or sends, configured with `--codec-endpoint`, +`--codec-auth` and `--codec-header`. The gRPC interceptor that applies the +codec walks only the public API's messages, so the stream commands apply it by +hand on the records and on a close reason. + +## Examples + +```sh +temporal server start-dev --port 7533 + +temporal stream create --address localhost:7533 --stream-id scores +temporal stream append --address localhost:7533 --stream-id scores \ + --producer-id game --sequence 1 --input '{"home": 1}' --input '{"home": 2}' +temporal stream read --address localhost:7533 --stream-id scores --follow +temporal stream describe --address localhost:7533 --stream-id scores + +temporal stream read --address localhost:7533 \ + --workflow-id june-s1-abcdef12 --name scores +``` diff --git a/go.mod b/go.mod index 48cd4f572..89e641bef 100644 --- a/go.mod +++ b/go.mod @@ -1,6 +1,6 @@ module github.com/temporalio/cli -go 1.26.8 +go 1.27.0 require ( github.com/BurntSushi/toml v1.4.0 @@ -10,15 +10,15 @@ require ( github.com/fatih/color v1.19.0 github.com/google/uuid v1.6.0 github.com/mattn/go-isatty v0.0.23 - github.com/nexus-rpc/sdk-go v0.6.0 + github.com/nexus-rpc/sdk-go v0.7.0 github.com/olekukonko/tablewriter v0.0.5 github.com/spf13/cobra v1.10.2 github.com/spf13/pflag v1.0.10 github.com/stretchr/testify v1.12.1 github.com/temporalio/cli/cliext v0.0.0 github.com/temporalio/ui-server/v2 v2.54.1 - go.temporal.io/api v1.63.5 - go.temporal.io/sdk v1.47.0 + go.temporal.io/api v1.63.6-0.20260909222256-20151aa90480 + go.temporal.io/sdk v1.48.0 go.temporal.io/sdk/contrib/envconfig v1.0.2 go.temporal.io/server v1.32.0 golang.org/x/exp v0.0.0-20260611194520-c48552f49976 @@ -232,3 +232,7 @@ require ( sigs.k8s.io/structured-merge-diff/v6 v6.4.0 // indirect sigs.k8s.io/yaml v1.6.0 // indirect ) + +replace go.temporal.io/server => github.com/moedash/temporal v0.0.0-20261003023939-91a1c125ec02 + +replace go.temporal.io/api => github.com/moedash/api-go v1.63.6-0.20261003021656-71792676a9fd diff --git a/go.sum b/go.sum index 28f5a717f..8d298f0d5 100644 --- a/go.sum +++ b/go.sum @@ -334,14 +334,18 @@ github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJ github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee h1:W5t00kpgFdJifH4BDsTlE89Zl93FEloxaWZfGcifgq8= github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= +github.com/moedash/api-go v1.63.6-0.20261003021656-71792676a9fd h1:40ANlQLRwkA2/uY3o7U7v8kElCSclu55fA/RtRTyfQk= +github.com/moedash/api-go v1.63.6-0.20261003021656-71792676a9fd/go.mod h1:acM0I9WPuYg8W3Pd9jOZvEgi7mRUttUQ4+e7fowKVnM= +github.com/moedash/temporal v0.0.0-20261003023939-91a1c125ec02 h1:HOOvpfHzduzr7F3TaWM7a09/waOqjYULOgoMZ2MzPM4= +github.com/moedash/temporal v0.0.0-20261003023939-91a1c125ec02/go.mod h1:tReLM/fxuPU/oPY6leZDno9TkXXy+aO8rRMC3ufDygE= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w= github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls= github.com/nexus-rpc/nexus-proto-annotations v0.1.0 h1:2fELd+9sqUtNu6Fg//pw8YFsxOvp8vZ8hfP0nHhNI80= github.com/nexus-rpc/nexus-proto-annotations v0.1.0/go.mod h1:n3UjF1bPCW8llR8tHvbxJ+27yPWrhpo8w/Yg1IOuY0Y= -github.com/nexus-rpc/sdk-go v0.6.0 h1:QRgnP2zTbxEbiyWG/aXH8uSC5LV/Mg1fqb19jb4DBlo= -github.com/nexus-rpc/sdk-go v0.6.0/go.mod h1:FHdPfVQwRuJFZFTF0Y2GOAxCrbIBNrcPna9slkGKPYk= +github.com/nexus-rpc/sdk-go v0.7.0 h1:38NrfY5rLnZAiMMs2ZfCKI/CSDzdfJG+27iAgfA8bUI= +github.com/nexus-rpc/sdk-go v0.7.0/go.mod h1:FHdPfVQwRuJFZFTF0Y2GOAxCrbIBNrcPna9slkGKPYk= github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno= github.com/olekukonko/tablewriter v0.0.5 h1:P2Ga83D34wi1o9J6Wh1mRuqd4mF/x/lgBS7N7AbDhec= github.com/olekukonko/tablewriter v0.0.5/go.mod h1:hPp6KlRPjbx+hW8ykQs1w3UBbZlj6HuIJcUGPhkA7kY= @@ -481,16 +485,12 @@ go.opentelemetry.io/otel/trace v1.44.0 h1:jxF5CsGYCe74MCRx2X4g7WsY/VBKRqqpNvXlX/ go.opentelemetry.io/otel/trace v1.44.0/go.mod h1:oLl1jrMQAVo6v3GAggN+1VH9VIz9iUSvW53sW1Q8PIE= go.opentelemetry.io/proto/otlp v1.10.0 h1:IQRWgT5srOCYfiWnpqUYz9CVmbO8bFmKcwYxpuCSL2g= go.opentelemetry.io/proto/otlp v1.10.0/go.mod h1:/CV4QoCR/S9yaPj8utp3lvQPoqMtxXdzn7ozvvozVqk= -go.temporal.io/api v1.63.5 h1:c11+kPYHkXXL3UiShPdbMD+xtvqGsbTibUA9ypmiCa4= -go.temporal.io/api v1.63.5/go.mod h1:SrlW2JMwVlDP4nRWSNznUFqnSHd+YeMDS1BkYo63HCQ= go.temporal.io/auto-scaled-workers v0.2.0-1.32.0.158.0 h1:l+Rj0cIHMC2VB/DC+axrLLnZ2ISTgUpUw8TMiP4B9F8= go.temporal.io/auto-scaled-workers v0.2.0-1.32.0.158.0/go.mod h1:ZGNY7kCU0EZpXx02D81/jGXHbUcFcVIzzguJ/f8pvUQ= -go.temporal.io/sdk v1.47.0 h1:lZ39w1+uWSjHTL0F3mSc0t4XUnKX8CCcWxqSiLbaHnc= -go.temporal.io/sdk v1.47.0/go.mod h1:ilKs0twgP4JpP8pfhIgZumnOEyBiYn6ZO/ta//NnKMU= +go.temporal.io/sdk v1.48.0 h1:WDctKDVuh0Z8Nf7euAyqs/EwcPg1JTIIq1Fut8Tq118= +go.temporal.io/sdk v1.48.0/go.mod h1:SHv3+fLzD0GGZAwf0xNSvu8UmO1nFgG9WBSYoowApIk= go.temporal.io/sdk/contrib/envconfig v1.0.2 h1:MGHfsuPUtsf7X9M6WYn3zYJj/mWsuYHnA1uuiL0KEuE= go.temporal.io/sdk/contrib/envconfig v1.0.2/go.mod h1:MuMiH7hksps2uXnmKuAWaP9P6WbkSDy62kl64t1VJVg= -go.temporal.io/server v1.32.0 h1:JQoqsVREaGc8vcuDNfuRlySiV3tVi81XUOVC9LeL/1Y= -go.temporal.io/server v1.32.0/go.mod h1:SuxEWp1bDjSB7kHtUjyaUJBh7Qjyv5wB8lCrS0VFSvw= go.uber.org/atomic v1.5.0/go.mod h1:sABNBOSYdrvTF6hTgEIbc7YasKWGhgEQZyfxyTvoXHQ= go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE= diff --git a/internal/devserver/server.go b/internal/devserver/server.go index 09d47854f..a4876fadf 100644 --- a/internal/devserver/server.go +++ b/internal/devserver/server.go @@ -40,6 +40,8 @@ import ( uiserveroptions "github.com/temporalio/ui-server/v2/server/server_options" "go.temporal.io/api/enums/v1" "go.temporal.io/server/chasm/lib/activity" + chasmcallback "go.temporal.io/server/chasm/lib/callback" + "go.temporal.io/server/chasm/lib/stream" "go.temporal.io/server/common/authorization" "go.temporal.io/server/common/cluster" "go.temporal.io/server/common/config" @@ -250,6 +252,18 @@ func (s *StartOptions) buildServerOptions() ([]temporal.ServerOption, *slog.Leve dynConf[activity.EnableStandaloneActivityOperatorCommands.Key()] = true dynConf[dynamicconfig.FrontendEnableBatchOperationsForStandaloneActivities.Key()] = true + // The server ships streams off so a deployment opts in. The dev server is + // the opt-in: local development is what it exists for, and an explicit + // value below still wins. + dynConf[stream.EnabledSetting.Key()] = true + // The server calls no callback address until one is allowed. A local + // receiver is the usual target of a notification channel's callback on + // a dev server, and it rarely serves TLS. + dynConf[chasmcallback.AllowedAddresses.Key()] = []any{ + map[string]any{"Pattern": "127.0.0.1:*", "AllowInsecure": true}, + map[string]any{"Pattern": "localhost:*", "AllowInsecure": true}, + } + // Dynamic config if set for k, v := range s.DynamicConfigValues { dynConf[dynamicconfig.MakeKey(k)] = v diff --git a/internal/temporalcli/client.go b/internal/temporalcli/client.go index 9126a2b0f..80e70f2a1 100644 --- a/internal/temporalcli/client.go +++ b/internal/temporalcli/client.go @@ -7,6 +7,7 @@ import ( "io/fs" "os" "os/user" + "sync/atomic" "github.com/temporalio/cli/cliext" "go.temporal.io/api/common/v1" @@ -32,8 +33,30 @@ func dialClient(cctx *CommandContext, c *cliext.ClientOptions) (client.Client, e // used by the gRPC interceptor; callers can use it to decode payloads nested inside // opaque proto bytes (e.g. the request/response of a system Nexus operation). func dialClientWithCodec(cctx *CommandContext, c *cliext.ClientOptions) (client.Client, converter.PayloadCodec, error) { + cl, err := dialClientWithConn(cctx, c) + if err != nil { + return nil, nil, err + } + return cl.Client, cl.PayloadCodec, nil +} + +// dialedClient is an SDK client together with what the SDK dialed for it. +type dialedClient struct { + client.Client + // The connection the SDK dialed, carrying its TLS, credentials, headers and + // interceptors, for services the SDK client does not wrap. + Conn grpc.ClientConnInterface + // As returned by [dialClientWithCodec]. The interceptor applying it walks + // the public API's messages only, so payloads inside any other message are + // encoded and decoded by hand. + PayloadCodec converter.PayloadCodec +} + +// dialClientWithConn is like [dialClient] but also returns the connection the +// SDK dialed and the remote payload codec. +func dialClientWithConn(cctx *CommandContext, c *cliext.ClientOptions) (*dialedClient, error) { if cctx.RootCommand == nil { - return nil, nil, fmt.Errorf("root command unexpectedly missing when dialing client") + return nil, fmt.Errorf("root command unexpectedly missing when dialing client") } // Set default identity if not provided @@ -61,12 +84,12 @@ func dialClientWithCodec(cctx *CommandContext, c *cliext.ClientOptions) (client. // original setup error instead of attaching a guessed address or profile. var pathErr *fs.PathError if errors.As(err, &pathErr) { - return nil, nil, newConnectError(&connectDiagnosis{ + return nil, newConnectError(&connectDiagnosis{ Cause: causeCertFileUnreadable, Detail: pathErr.Path, }, connectMeta{}, err) } - return nil, nil, err + return nil, err } // We do not put codec on data converter here, it is applied via @@ -82,6 +105,20 @@ func dialClientWithCodec(cctx *CommandContext, c *cliext.ClientOptions) (client. clientOpts.ConnectionOptions.DialOptions = append( clientOpts.ConnectionOptions.DialOptions, grpc.WithChainUnaryInterceptor(fixedHeaderOverrideInterceptor)) + // The SDK does not hand out the connection it dials, but every interceptor + // on it is given the connection, and the dial itself makes a call. + var conn atomic.Pointer[grpc.ClientConn] + clientOpts.ConnectionOptions.DialOptions = append( + clientOpts.ConnectionOptions.DialOptions, grpc.WithChainUnaryInterceptor( + func( + ctx context.Context, + method string, req, reply any, + cc *grpc.ClientConn, invoker grpc.UnaryInvoker, opts ...grpc.CallOption, + ) error { + conn.CompareAndSwap(nil, cc) + return invoker(ctx, method, req, reply, cc, opts...) + })) + // Additional gRPC options clientOpts.ConnectionOptions.DialOptions = append( clientOpts.ConnectionOptions.DialOptions, cctx.Options.AdditionalClientGRPCDialOptions...) @@ -97,14 +134,19 @@ func dialClientWithCodec(cctx *CommandContext, c *cliext.ClientOptions) (client. cl, err := client.DialContext(dialCtx, clientOpts) if err != nil { - return nil, nil, dialConnectError(cctx, dialCtx, clientOpts, err) + return nil, dialConnectError(cctx, dialCtx, clientOpts, err) } // Since this namespace value is used by many commands after this call, // we are mutating it to be the derived one c.Namespace = clientOpts.Namespace - return cl, builder.PayloadCodec, nil + cc := conn.Load() + if cc == nil { + cl.Close() + return nil, fmt.Errorf("dialing made no call, so the connection was not captured") + } + return &dialedClient{Client: cl, Conn: cc, PayloadCodec: builder.PayloadCodec}, nil } // dialConnectError enriches a client.DialContext failure with a staged diff --git a/internal/temporalcli/commands.channel.go b/internal/temporalcli/commands.channel.go new file mode 100644 index 000000000..a4a6cf524 --- /dev/null +++ b/internal/temporalcli/commands.channel.go @@ -0,0 +1,649 @@ +package temporalcli + +import ( + "context" + "encoding/base64" + "encoding/json" + "errors" + "fmt" + "reflect" + "sort" + "strconv" + "strings" + "time" + "unicode" + "unicode/utf8" + + "github.com/fatih/color" + "github.com/google/uuid" + "github.com/temporalio/cli/internal/printer" + commonpb "go.temporal.io/api/common/v1" + enumspb "go.temporal.io/api/enums/v1" + notificationpb "go.temporal.io/api/notification/v1" + "go.temporal.io/api/serviceerror" + workflowpb "go.temporal.io/api/workflow/v1" + "go.temporal.io/api/workflowservice/v1" + "go.temporal.io/sdk/client" + "google.golang.org/protobuf/types/known/durationpb" +) + +// channelPollGrace is how long past the requested wait a poll may take before +// the CLI gives up on it, so a Service answering at its own deadline is not cut +// off. +const channelPollGrace = 10 * time.Second + +// channelTarget is the channel a command names: an independent channel by its +// name alone, or a channel linked to an execution by that owner and the name. +type channelTarget struct { + name string + execution *commonpb.Execution +} + +func (o *ChannelOptions) target() (channelTarget, error) { + if o.WorkflowId != "" && o.ActivityId != "" { + return channelTarget{}, fmt.Errorf( + "--workflow-id and --activity-id name different owners, set one of them") + } + if o.RunId != "" && o.WorkflowId == "" && o.ActivityId == "" { + return channelTarget{}, fmt.Errorf("--run-id requires --workflow-id or --activity-id") + } + t := channelTarget{name: o.Channel} + switch { + case o.WorkflowId != "": + t.execution = &commonpb.Execution{ + Type: enumspb.EXECUTION_TYPE_WORKFLOW, + BusinessId: o.WorkflowId, + RunId: o.RunId, + } + case o.ActivityId != "": + t.execution = &commonpb.Execution{ + Type: enumspb.EXECUTION_TYPE_ACTIVITY, + BusinessId: o.ActivityId, + RunId: o.RunId, + } + } + return t, nil +} + +// executionKindText is the owner's kind the way the flags name it, so output +// reads back as the flag that reaches the same owner. +func executionKindText(e *commonpb.Execution) string { + switch e.GetType() { + case enumspb.EXECUTION_TYPE_ACTIVITY: + return "activity" + case enumspb.EXECUTION_TYPE_NEXUS_OPERATION: + return "nexus operation" + default: + return "workflow" + } +} + +// executionText names an execution for a card or a message: its kind and ID, +// then the run when one was given. +func executionText(e *commonpb.Execution) string { + if e == nil { + return "" + } + text := executionKindText(e) + " " + e.GetBusinessId() + if e.GetRunId() != "" { + text += " (run " + e.GetRunId() + ")" + } + return text +} + +func (t channelTarget) String() string { + if t.execution == nil { + return fmt.Sprintf("channel %q", t.name) + } + return fmt.Sprintf("channel %q of %s %q", t.name, executionKindText(t.execution), + t.execution.GetBusinessId()) +} + +// label is the target for a line of normal output, where quotes would be noise. +func (t channelTarget) label() string { + if t.execution == nil { + return "channel " + t.name + } + return "channel " + t.name + " of " + executionKindText(t.execution) + " " + + t.execution.GetBusinessId() +} + +// plainChannelRefusal turns the Service's refusals into what to do next. Every +// other error comes back as it was. +func plainChannelRefusal(err error, t channelTarget) error { + var notFound *serviceerror.NotFound + if errors.As(err, ¬Found) { + plain := fmt.Sprintf("there is no channel %q. A channel exists once a writer "+ + "notifies it or a listener registers on it, and goes away after a while "+ + "with neither.", t.name) + if t.execution != nil { + plain = fmt.Sprintf("%s %q has no running execution to reach. A linked "+ + "channel lives only while its owner runs.", + executionKindText(t.execution), t.execution.GetBusinessId()) + } + return &streamRefusal{plain: plain, detail: notFound.Message, cause: err} + } + var exhausted *serviceerror.ResourceExhausted + if !errors.As(err, &exhausted) { + return err + } + var plain string + switch exhausted.Cause { + case enumspb.RESOURCE_EXHAUSTED_CAUSE_CONCURRENT_LIMIT: + plain = "the channel already has as many listeners as it may hold, so this one " + + "was not registered. Remove a listener it no longer needs, or use another channel." + case enumspb.RESOURCE_EXHAUSTED_CAUSE_RPS_LIMIT: + plain = "the namespace is notifying channels faster than it may, so this " + + "notification was not sent. Retry after a moment, or notify less often." + default: + plain = "the Service is over one of its limits and refused the call. Retry after " + + "a moment." + } + return &streamRefusal{plain: plain, detail: exhausted.Message, cause: err} +} + +// parseChannelMetadata reads KEY=VALUE pairs whose values are JSON, carried as +// JSON payloads the way --input carries a Signal's arguments. +func parseChannelMetadata(pairs []string) (map[string]*commonpb.Payload, error) { + if len(pairs) == 0 { + return nil, nil + } + out := make(map[string]*commonpb.Payload, len(pairs)) + for _, pair := range pairs { + key, value, ok := strings.Cut(pair, "=") + if !ok || key == "" { + return nil, fmt.Errorf("--metadata %q must be KEY=VALUE", pair) + } + if _, dup := out[key]; dup { + return nil, fmt.Errorf("--metadata key %q is given more than once", key) + } + if !json.Valid([]byte(value)) { + return nil, fmt.Errorf("--metadata value for %q is not valid JSON", key) + } + out[key] = &commonpb.Payload{ + Metadata: map[string][]byte{"encoding": []byte("json/plain")}, + Data: []byte(value), + } + } + return out, nil +} + +// positionText shows a position as the text it most often is, and as base64 +// when it holds bytes a terminal would mangle. +func positionText(p []byte) string { + if utf8.Valid(p) && strings.IndexFunc(string(p), func(r rune) bool { + return !unicode.IsPrint(r) + }) < 0 { + return string(p) + } + return "base64:" + base64.StdEncoding.EncodeToString(p) +} + +// metadataText puts a notification's metadata on one line, keys sorted so the +// same notification always reads the same. +func metadataText(md map[string]*commonpb.Payload) (string, error) { + keys := make([]string, 0, len(md)) + for k := range md { + keys = append(keys, k) + } + sort.Strings(keys) + parts := make([]string, 0, len(keys)) + for _, k := range keys { + v, err := payloadText(md[k]) + if err != nil { + return "", err + } + parts = append(parts, k+"="+v) + } + return strings.Join(parts, " "), nil +} + +type channelNotificationRow struct { + Counter int64 + Position string + Metadata string +} + +func notificationRow(n *notificationpb.Notification) (channelNotificationRow, error) { + md, err := metadataText(n.GetMetadata()) + if err != nil { + return channelNotificationRow{}, err + } + return channelNotificationRow{ + Counter: n.GetCounter(), + Position: positionText(n.GetPosition()), + Metadata: md, + }, nil +} + +// channelKindText reads a server that predates the linked kind, and so leaves +// the kind unset, as the independent channel it serves. +func channelKindText(kind notificationpb.ChannelKind) string { + if kind == notificationpb.CHANNEL_KIND_UNSPECIFIED { + kind = notificationpb.CHANNEL_KIND_INDEPENDENT + } + return kind.String() +} + +// printChannelSubscriptions lists the channels a workflow stands on, as its +// description reports them. A workflow that stands on none prints nothing, so +// the section is absent on servers that predate it as well. +func printChannelSubscriptions( + cctx *CommandContext, subs []*workflowpb.ChannelSubscriptionInfo, +) error { + if len(subs) == 0 { + return nil + } + rows := make([]struct { + Channel string + Kind string + LastCounter int64 + PendingCounter string + ScheduledCounter int64 + Listeners int32 + Retained int32 + }, len(subs)) + for i, s := range subs { + rows[i].Channel = s.GetChannel() + rows[i].Kind = channelKindText(s.GetKind()) + rows[i].LastCounter = s.GetLastCounter() + if pending := s.GetPendingNotification(); pending != nil { + rows[i].PendingCounter = strconv.FormatInt(pending.GetCounter(), 10) + } + rows[i].ScheduledCounter = s.GetScheduledCounter() + rows[i].Listeners = s.GetListenerCount() + rows[i].Retained = s.GetRetainedCount() + } + cctx.Printer.Println() + cctx.Printer.Println(color.MagentaString("Notification Channels: %v", len(subs))) + cctx.Printer.Println() + return cctx.Printer.PrintStructured(rows, printer.StructuredOptions{ + Table: &printer.TableOptions{}, + }) +} + +func (c *TemporalChannelNotifyCommand) run(cctx *CommandContext, _ []string) error { + target, err := c.target() + if err != nil { + return err + } + if c.Counter <= 0 { + return fmt.Errorf("--counter must be greater than zero") + } + metadata, err := parseChannelMetadata(c.Metadata) + if err != nil { + return err + } + cl, err := dialClient(cctx, &c.Parent.ClientOptions) + if err != nil { + return err + } + defer cl.Close() + + resp, err := cl.WorkflowService().NotifyChannel(cctx, &workflowservice.NotifyChannelRequest{ + Namespace: c.Parent.Namespace, + Notification: ¬ificationpb.Notification{ + Channel: c.Channel, + Position: []byte(c.Position), + Counter: int64(c.Counter), + Metadata: metadata, + }, + Identity: c.Parent.Identity, + RequestId: uuid.NewString(), + Execution: target.execution, + }) + if err != nil { + return fmt.Errorf("failed notifying %v: %w", target, plainChannelRefusal(err, target)) + } + if cctx.JSONOutput { + return cctx.Printer.PrintStructured(resp, printer.StructuredOptions{}) + } + cctx.Printer.Printlnf("Notified %v. Listeners reached: %v", + target.label(), resp.GetListenerCount()) + return nil +} + +func (c *TemporalChannelDescribeCommand) run(cctx *CommandContext, _ []string) error { + target, err := c.target() + if err != nil { + return err + } + cl, err := dialClient(cctx, &c.Parent.ClientOptions) + if err != nil { + return err + } + defer cl.Close() + + resp, err := cl.WorkflowService().DescribeChannel(cctx, + &workflowservice.DescribeChannelRequest{ + Namespace: c.Parent.Namespace, + Channel: c.Channel, + Execution: target.execution, + }) + if err != nil { + return fmt.Errorf("failed describing %v: %w", target, plainChannelRefusal(err, target)) + } + if cctx.JSONOutput { + return cctx.Printer.PrintStructured(resp, printer.StructuredOptions{}) + } + + info := struct { + Channel string + Kind string + LinkedTo string `cli:",cardOmitEmpty"` + RetainedCount int32 + LatestCounter int64 `cli:",cardOmitEmpty"` + LatestPosition string `cli:",cardOmitEmpty"` + LatestMetadata string `cli:",cardOmitEmpty"` + }{ + Channel: c.Channel, + Kind: channelKindText(resp.GetKind()), + LinkedTo: executionText(resp.GetLinkedTo()), + RetainedCount: resp.GetRetainedCount(), + } + if latest := resp.GetLatest(); latest != nil { + row, err := notificationRow(latest) + if err != nil { + return err + } + info.LatestCounter, info.LatestPosition, info.LatestMetadata = + row.Counter, row.Position, row.Metadata + } + cctx.Printer.Println(color.MagentaString("Channel:")) + if err := cctx.Printer.PrintStructured(info, printer.StructuredOptions{}); err != nil { + return err + } + + type workflowRow struct { + ListenerId string + WorkflowId string + RunId string + Registered time.Time + } + // Headers stay out of the table because they often carry credentials. + type callbackRow struct { + ListenerId string + Url string + Registered time.Time + } + var workflows []workflowRow + var callbacks []callbackRow + for _, l := range resp.GetListeners() { + // A linked channel's owner never registered, so it has no time; the + // conversion would turn that into the Unix epoch. + var registered time.Time + if l.GetRegisteredTime() != nil { + registered = l.GetRegisteredTime().AsTime() + } + if wf := l.GetWorkflow(); wf != nil { + workflows = append(workflows, workflowRow{ + ListenerId: l.GetListenerId(), + WorkflowId: wf.GetWorkflowId(), + RunId: wf.GetRunId(), + Registered: registered, + }) + } else if cb := l.GetCallback(); cb != nil { + callbacks = append(callbacks, callbackRow{ + ListenerId: l.GetListenerId(), + Url: cb.GetNexus().GetUrl(), + Registered: registered, + }) + } + } + sort.Slice(workflows, func(i, j int) bool { + return workflows[i].ListenerId < workflows[j].ListenerId + }) + sort.Slice(callbacks, func(i, j int) bool { + return callbacks[i].ListenerId < callbacks[j].ListenerId + }) + cctx.Printer.Println() + cctx.Printer.Println(color.MagentaString("Workflow listeners: %v", len(workflows))) + if len(workflows) > 0 { + if err := cctx.Printer.PrintStructured(workflows, printer.StructuredOptions{ + Table: &printer.TableOptions{}, + }); err != nil { + return err + } + } + cctx.Printer.Println() + cctx.Printer.Println(color.MagentaString("Callback listeners: %v", len(callbacks))) + if len(callbacks) > 0 { + return cctx.Printer.PrintStructured(callbacks, printer.StructuredOptions{ + Table: &printer.TableOptions{}, + }) + } + return nil +} + +func (c *TemporalChannelListenerAddCommand) run(cctx *CommandContext, _ []string) error { + target, err := c.target() + if err != nil { + return err + } + if c.CallbackUrl == "" { + return fmt.Errorf("--callback-url cannot be empty") + } + headers, err := stringKeysValues(c.Header) + if err != nil { + return fmt.Errorf("invalid --header: %w", err) + } + clientOpts := &c.Parent.Parent.ClientOptions + cl, err := dialClient(cctx, clientOpts) + if err != nil { + return err + } + defer cl.Close() + + resp, err := cl.WorkflowService().RegisterChannelListener(cctx, + &workflowservice.RegisterChannelListenerRequest{ + Namespace: clientOpts.Namespace, + Channel: c.Channel, + Callback: &commonpb.Callback{ + Variant: &commonpb.Callback_Nexus_{Nexus: &commonpb.Callback_Nexus{ + Url: c.CallbackUrl, + Header: headers, + }}, + }, + RequestId: uuid.NewString(), + Identity: clientOpts.Identity, + Execution: target.execution, + }) + if err != nil { + return fmt.Errorf("failed adding a listener to %v: %w", + target, plainChannelRefusal(err, target)) + } + if cctx.JSONOutput { + return cctx.Printer.PrintStructured(resp, printer.StructuredOptions{}) + } + cctx.Printer.Printlnf("Added listener %v to %v", resp.GetListenerId(), target.label()) + return nil +} + +func (c *TemporalChannelListenerRemoveCommand) run(cctx *CommandContext, _ []string) error { + target, err := c.target() + if err != nil { + return err + } + clientOpts := &c.Parent.Parent.ClientOptions + cl, err := dialClient(cctx, clientOpts) + if err != nil { + return err + } + defer cl.Close() + + _, err = cl.WorkflowService().UnregisterChannelListener(cctx, + &workflowservice.UnregisterChannelListenerRequest{ + Namespace: clientOpts.Namespace, + Channel: c.Channel, + ListenerId: c.ListenerId, + Identity: clientOpts.Identity, + Execution: target.execution, + }) + if err != nil { + return fmt.Errorf("failed removing listener %q from %v: %w", + c.ListenerId, target, plainChannelRefusal(err, target)) + } + cctx.Printer.Printlnf("Removed listener %v from %v", c.ListenerId, target.label()) + return nil +} + +// channelPoller hands out a channel's notifications one at a time. Without +// follow it makes one poll; with follow it polls again from the highest +// counter it has seen until the command is interrupted. +type channelPoller struct { + ctx context.Context + cl client.Client + namespace string + target channelTarget + after int64 + wait time.Duration + max int32 + follow bool + buf []*notificationpb.Notification + polled bool +} + +func (p *channelPoller) next() (*notificationpb.Notification, error) { + for len(p.buf) == 0 && (p.follow || !p.polled) { + if err := p.poll(); err != nil { + // An interrupted command is the poller leaving, not a failure. + if p.ctx.Err() != nil { + return nil, nil + } + return nil, fmt.Errorf("failed polling %v: %w", + p.target, plainChannelRefusal(err, p.target)) + } + } + if len(p.buf) == 0 { + return nil, nil + } + n := p.buf[0] + p.buf = p.buf[1:] + return n, nil +} + +func (p *channelPoller) poll() error { + ctx, cancel := context.WithTimeout(p.ctx, p.wait+channelPollGrace) + defer cancel() + resp, err := p.cl.WorkflowService().PollChannel(ctx, &workflowservice.PollChannelRequest{ + Namespace: p.namespace, + Channel: p.target.name, + AfterCounter: p.after, + Wait: durationpb.New(p.wait), + MaxNotifications: p.max, + Execution: p.target.execution, + }) + p.polled = true + if err != nil { + // A poll the Service held past its deadline is an empty answer, the same + // as one it returned with nothing. + if ctx.Err() != nil && p.ctx.Err() == nil { + return nil + } + return err + } + for _, n := range resp.GetNotifications() { + p.after = max(p.after, n.GetCounter()) + } + p.buf = append(p.buf, resp.GetNotifications()...) + return nil +} + +// channelRowIter adapts the poller to the printer's streaming table. +type channelRowIter struct{ poller *channelPoller } + +func (i *channelRowIter) Next() (any, error) { + n, err := i.poller.next() + if n == nil || err != nil { + return nil, err + } + return notificationRow(n) +} + +func (c *TemporalChannelPollCommand) run(cctx *CommandContext, _ []string) error { + target, err := c.target() + if err != nil { + return err + } + if c.AfterCounter < 0 { + return fmt.Errorf("--after-counter cannot be negative") + } + if c.Max < 0 { + return fmt.Errorf("--max cannot be negative") + } + if c.Wait.Duration() <= 0 { + return fmt.Errorf("--wait must be positive") + } + cl, err := dialClient(cctx, &c.Parent.ClientOptions) + if err != nil { + return err + } + defer cl.Close() + + poller := &channelPoller{ + ctx: cctx, + cl: cl, + namespace: c.Parent.Namespace, + target: target, + after: int64(c.AfterCounter), + wait: c.Wait.Duration(), + max: int32(c.Max), + follow: c.Follow, + } + if cctx.JSONOutput { + // This is a listing command subject to json vs jsonl rules + cctx.Printer.StartList() + defer cctx.Printer.EndList() + for { + n, err := poller.next() + if err != nil { + return err + } + if n == nil { + return nil + } + if err := cctx.Printer.PrintStructured(n, printer.StructuredOptions{}); err != nil { + return err + } + } + } + if !c.Follow { + var rows []channelNotificationRow + for { + n, err := poller.next() + if err != nil { + return err + } + if n == nil { + break + } + row, err := notificationRow(n) + if err != nil { + return err + } + rows = append(rows, row) + } + if len(rows) == 0 { + cctx.Printer.Printlnf("No notifications on %v after counter %v", + target.label(), c.AfterCounter) + return nil + } + return cctx.Printer.PrintStructured(rows, printer.StructuredOptions{ + Table: &printer.TableOptions{}, + }) + } + return cctx.Printer.PrintStructuredTableIter( + reflect.TypeOf(channelNotificationRow{}), + &channelRowIter{poller: poller}, + printer.StructuredOptions{ + Table: &printer.TableOptions{ + // Streaming rows cannot be measured first, so the columns + // before the metadata take fixed widths. + FieldWidths: map[string]int{ + "Counter": 12, + "Position": 24, + }, + }, + }, + ) +} diff --git a/internal/temporalcli/commands.channel_test.go b/internal/temporalcli/commands.channel_test.go new file mode 100644 index 000000000..f598daaf9 --- /dev/null +++ b/internal/temporalcli/commands.channel_test.go @@ -0,0 +1,1113 @@ +package temporalcli_test + +import ( + "context" + "encoding/json" + "io" + "net" + "net/http" + "net/http/httptest" + "strings" + "sync" + "testing" + "time" + + "github.com/google/uuid" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "github.com/temporalio/cli/internal/temporalcli" + commonpb "go.temporal.io/api/common/v1" + enumspb "go.temporal.io/api/enums/v1" + notificationpb "go.temporal.io/api/notification/v1" + "go.temporal.io/api/serviceerror" + workflowpb "go.temporal.io/api/workflow/v1" + "go.temporal.io/api/workflowservice/v1" + "go.temporal.io/sdk/client" + "go.temporal.io/sdk/workflow" + streampb "go.temporal.io/server/chasm/lib/stream/gen/streampb/v1" + "google.golang.org/grpc" + "google.golang.org/protobuf/types/known/timestamppb" +) + +// fakeChannelService answers the channel calls from canned responses and +// records what the CLI sent, so the commands are checked without a server +// that carries channels. +type fakeChannelService struct { + workflowservice.UnimplementedWorkflowServiceServer + + mu sync.Mutex + notifies []*workflowservice.NotifyChannelRequest + registers []*workflowservice.RegisterChannelListenerRequest + unregister []*workflowservice.UnregisterChannelListenerRequest + polls []*workflowservice.PollChannelRequest + describes []*workflowservice.DescribeChannelRequest + + err error + describe *workflowservice.DescribeChannelResponse + // pollAnswers are handed out in order; past the end a poll waits for the + // caller to give up, the way an idle channel does. + pollAnswers [][]*notificationpb.Notification + // onPoll runs with the number of polls seen so far. + onPoll func(n int) + // describeWorkflow answers the workflow description with its channels. + describeWorkflow *workflowservice.DescribeWorkflowExecutionResponse +} + +func (f *fakeChannelService) DescribeWorkflowExecution( + context.Context, *workflowservice.DescribeWorkflowExecutionRequest, +) (*workflowservice.DescribeWorkflowExecutionResponse, error) { + f.mu.Lock() + defer f.mu.Unlock() + if f.err != nil { + return nil, f.err + } + return f.describeWorkflow, nil +} + +func (f *fakeChannelService) GetSystemInfo( + context.Context, *workflowservice.GetSystemInfoRequest, +) (*workflowservice.GetSystemInfoResponse, error) { + return &workflowservice.GetSystemInfoResponse{}, nil +} + +func (f *fakeChannelService) NotifyChannel( + _ context.Context, req *workflowservice.NotifyChannelRequest, +) (*workflowservice.NotifyChannelResponse, error) { + f.mu.Lock() + defer f.mu.Unlock() + f.notifies = append(f.notifies, req) + if f.err != nil { + return nil, f.err + } + return &workflowservice.NotifyChannelResponse{ListenerCount: 3}, nil +} + +func (f *fakeChannelService) RegisterChannelListener( + _ context.Context, req *workflowservice.RegisterChannelListenerRequest, +) (*workflowservice.RegisterChannelListenerResponse, error) { + f.mu.Lock() + defer f.mu.Unlock() + f.registers = append(f.registers, req) + if f.err != nil { + return nil, f.err + } + return &workflowservice.RegisterChannelListenerResponse{ListenerId: "listener-7"}, nil +} + +func (f *fakeChannelService) UnregisterChannelListener( + _ context.Context, req *workflowservice.UnregisterChannelListenerRequest, +) (*workflowservice.UnregisterChannelListenerResponse, error) { + f.mu.Lock() + defer f.mu.Unlock() + f.unregister = append(f.unregister, req) + if f.err != nil { + return nil, f.err + } + return &workflowservice.UnregisterChannelListenerResponse{}, nil +} + +func (f *fakeChannelService) DescribeChannel( + _ context.Context, req *workflowservice.DescribeChannelRequest, +) (*workflowservice.DescribeChannelResponse, error) { + f.mu.Lock() + f.describes = append(f.describes, req) + f.mu.Unlock() + if f.err != nil { + return nil, f.err + } + return f.describe, nil +} + +func (f *fakeChannelService) PollChannel( + ctx context.Context, req *workflowservice.PollChannelRequest, +) (*workflowservice.PollChannelResponse, error) { + f.mu.Lock() + f.polls = append(f.polls, req) + n := len(f.polls) + var answer []*notificationpb.Notification + idle := n > len(f.pollAnswers) + if !idle { + answer = f.pollAnswers[n-1] + } + onPoll, err := f.onPoll, f.err + f.mu.Unlock() + if onPoll != nil { + onPoll(n) + } + if err != nil { + return nil, err + } + if idle { + <-ctx.Done() + return nil, ctx.Err() + } + return &workflowservice.PollChannelResponse{Notifications: answer}, nil +} + +func startFakeChannelService(t *testing.T, f *fakeChannelService) string { + return startFakeServices(t, f, nil) +} + +// startFakeServices serves a fake workflow service and, when given, a fake +// stream service on one address, the way the Service's frontend does. +func startFakeServices( + t *testing.T, wf workflowservice.WorkflowServiceServer, st streampb.StreamServiceServer, +) string { + ln, err := net.Listen("tcp", "127.0.0.1:0") + require.NoError(t, err) + // The Service sends its errors as gRPC statuses; a bare server would send + // them as unknown errors and lose their kind. + srv := grpc.NewServer(grpc.UnaryInterceptor(func( + ctx context.Context, req any, _ *grpc.UnaryServerInfo, handler grpc.UnaryHandler, + ) (any, error) { + resp, err := handler(ctx, req) + if err != nil { + return nil, serviceerror.ToStatus(err).Err() + } + return resp, nil + })) + workflowservice.RegisterWorkflowServiceServer(srv, wf) + if st != nil { + streampb.RegisterStreamServiceServer(srv, st) + } + go func() { _ = srv.Serve(ln) }() + t.Cleanup(srv.Stop) + return ln.Addr().String() +} + +func jsonPayload(data string) *commonpb.Payload { + return &commonpb.Payload{ + Metadata: map[string][]byte{"encoding": []byte("json/plain")}, + Data: []byte(data), + } +} + +func TestChannel_ArgumentValidation(t *testing.T) { + f := &fakeChannelService{} + addr := startFakeChannelService(t, f) + h := NewCommandHarness(t) + + for _, tc := range []struct { + args []string + want string + }{ + {[]string{"notify", "--position", "1", "--counter", "1"}, `"channel" not set`}, + {[]string{"notify", "-c", "ch", "--counter", "1"}, `"position" not set`}, + {[]string{"notify", "-c", "ch", "--position", "1", "--counter", "0"}, + "--counter must be greater than zero"}, + {[]string{"notify", "-c", "ch", "--position", "1", "--counter", "1", + "--metadata", "topic"}, "must be KEY=VALUE"}, + {[]string{"notify", "-c", "ch", "--position", "1", "--counter", "1", + "--metadata", "topic=scores"}, "not valid JSON"}, + {[]string{"notify", "-c", "ch", "--position", "1", "--counter", "1", + "--metadata", `a="x"`, "--metadata", `a="y"`}, "more than once"}, + {[]string{"listener", "add", "-c", "ch"}, `"callback-url" not set`}, + {[]string{"listener", "add", "-c", "ch", "--callback-url", "http://x", + "--header", "nope"}, "invalid --header"}, + {[]string{"listener", "remove", "-c", "ch"}, `"listener-id" not set`}, + {[]string{"poll", "-c", "ch", "--after-counter", "-1"}, "cannot be negative"}, + {[]string{"poll", "-c", "ch", "--max", "-1"}, "cannot be negative"}, + {[]string{"poll", "-c", "ch", "--wait", "0s"}, "--wait must be positive"}, + {[]string{"describe"}, `"channel" not set`}, + {[]string{"describe", "-c", "ch", "--run-id", "r1"}, + "--run-id requires --workflow-id or --activity-id"}, + {[]string{"notify", "-c", "ch", "--position", "1", "--counter", "1", "-r", "r1"}, + "--run-id requires --workflow-id or --activity-id"}, + {[]string{"poll", "-c", "ch", "-r", "r1"}, "--run-id requires --workflow-id or --activity-id"}, + {[]string{"listener", "add", "-c", "ch", "--callback-url", "http://x", "-r", "r1"}, + "--run-id requires --workflow-id or --activity-id"}, + {[]string{"listener", "remove", "-c", "ch", "--listener-id", "l", "-r", "r1"}, + "--run-id requires --workflow-id or --activity-id"}, + {[]string{"describe", "-c", "ch", "-w", "wf", "--activity-id", "act"}, + "--workflow-id and --activity-id name different owners"}, + {[]string{"notify", "-c", "ch", "--position", "1", "--counter", "1", "-w", "wf", + "--activity-id", "act"}, "--workflow-id and --activity-id name different owners"}, + {[]string{"poll", "-c", "ch", "-w", "wf", "--activity-id", "act"}, + "--workflow-id and --activity-id name different owners"}, + } { + res := h.Execute(append(append([]string{"channel"}, tc.args...), "--address", addr)...) + require.Error(t, res.Err, tc.args) + assert.Contains(t, res.Err.Error(), tc.want, tc.args) + } + // None of the refused commands reached the Service. + assert.Empty(t, f.notifies) + assert.Empty(t, f.registers) + assert.Empty(t, f.unregister) + assert.Empty(t, f.polls) + assert.Empty(t, f.describes) +} + +func TestChannel_Notify(t *testing.T) { + f := &fakeChannelService{} + addr := startFakeChannelService(t, f) + h := NewCommandHarness(t) + + res := h.Execute( + "channel", "notify", "--address", addr, "--namespace", "ns1", + "-c", "stream/scores", "--position", "42", "--counter", "42", + "--metadata", `topic="scores"`, "--metadata", `batch={"n": 2}`, + ) + require.NoError(t, res.Err) + h.ContainsOnSameLine(res.Stdout.String(), "Notified channel stream/scores", "3") + + require.Len(t, f.notifies, 1) + req := f.notifies[0] + assert.Equal(t, "ns1", req.GetNamespace()) + assert.NotEmpty(t, req.GetIdentity()) + assert.NotEmpty(t, req.GetRequestId()) + n := req.GetNotification() + assert.Equal(t, "stream/scores", n.GetChannel()) + assert.Equal(t, []byte("42"), n.GetPosition()) + assert.Equal(t, int64(42), n.GetCounter()) + require.Len(t, n.GetMetadata(), 2) + assert.Equal(t, "json/plain", string(n.GetMetadata()["topic"].GetMetadata()["encoding"])) + assert.Equal(t, `"scores"`, string(n.GetMetadata()["topic"].GetData())) + assert.Equal(t, `{"n": 2}`, string(n.GetMetadata()["batch"].GetData())) + + // A retried command is a new request, so the Service does not fold it + // with the first. + res = h.Execute( + "channel", "notify", "--address", addr, "-c", "ch", "--position", "p", + "--counter", "1", "-o", "json", + ) + require.NoError(t, res.Err) + var out workflowservice.NotifyChannelResponse + require.NoError(t, temporalcli.UnmarshalProtoJSONWithOptions(res.Stdout.Bytes(), &out, true)) + assert.Equal(t, int32(3), out.GetListenerCount()) + require.Len(t, f.notifies, 2) + assert.NotEqual(t, f.notifies[0].GetRequestId(), f.notifies[1].GetRequestId()) +} + +func TestChannel_Describe(t *testing.T) { + registered := time.Date(2026, 10, 1, 12, 0, 0, 0, time.UTC) + f := &fakeChannelService{describe: &workflowservice.DescribeChannelResponse{ + Listeners: []*notificationpb.ChannelListener{ + { + ListenerId: "l-callback", + Listener: ¬ificationpb.ChannelListener_Callback{Callback: &commonpb.Callback{ + Variant: &commonpb.Callback_Nexus_{Nexus: &commonpb.Callback_Nexus{ + Url: "https://example.com/notify", + Header: map[string]string{"Authorization": "Bearer secret"}, + }}, + }}, + RegisteredTime: timestamppb.New(registered), + }, + { + ListenerId: "l-workflow", + Listener: ¬ificationpb.ChannelListener_Workflow{ + Workflow: ¬ificationpb.WorkflowListener{WorkflowId: "wf-1", RunId: "run-1"}, + }, + RegisteredTime: timestamppb.New(registered), + }, + }, + Latest: ¬ificationpb.Notification{ + Channel: "ch", + Position: []byte{0x00, 0xff}, + Counter: 9, + Metadata: map[string]*commonpb.Payload{ + "topic": jsonPayload(`"scores"`), + "a": jsonPayload(`1`), + }, + }, + RetainedCount: 4, + }} + addr := startFakeChannelService(t, f) + h := NewCommandHarness(t) + + res := h.Execute("channel", "describe", "--address", addr, "-c", "ch") + require.NoError(t, res.Err) + out := res.Stdout.String() + h.ContainsOnSameLine(out, "Channel", "ch") + h.ContainsOnSameLine(out, "RetainedCount", "4") + h.ContainsOnSameLine(out, "LatestCounter", "9") + // Bytes a terminal would mangle print as base64. + h.ContainsOnSameLine(out, "LatestPosition", "base64:AP8=") + h.ContainsOnSameLine(out, "LatestMetadata", `a=1 topic="scores"`) + h.ContainsOnSameLine(out, "Workflow listeners: 1") + h.ContainsOnSameLine(out, "l-workflow", "wf-1", "run-1") + h.ContainsOnSameLine(out, "Callback listeners: 1") + h.ContainsOnSameLine(out, "l-callback", "https://example.com/notify") + assert.NotContains(t, out, "secret", "headers stay out of the text output") + + res = h.Execute("channel", "describe", "--address", addr, "-c", "ch", "-o", "json") + require.NoError(t, res.Err) + var described workflowservice.DescribeChannelResponse + require.NoError(t, temporalcli.UnmarshalProtoJSONWithOptions(res.Stdout.Bytes(), &described, true)) + assert.Equal(t, int32(4), described.GetRetainedCount()) + assert.Len(t, described.GetListeners(), 2) + assert.Equal(t, []byte{0x00, 0xff}, described.GetLatest().GetPosition()) + + // A channel that has seen nothing yet still describes. + f.describe = &workflowservice.DescribeChannelResponse{} + res = h.Execute("channel", "describe", "--address", addr, "-c", "ch") + require.NoError(t, res.Err) + assert.NotContains(t, res.Stdout.String(), "LatestCounter") + h.ContainsOnSameLine(res.Stdout.String(), "Workflow listeners: 0") +} + +func TestChannel_Listener(t *testing.T) { + f := &fakeChannelService{} + addr := startFakeChannelService(t, f) + h := NewCommandHarness(t) + + res := h.Execute( + "channel", "listener", "add", "--address", addr, "-c", "ch", + "--callback-url", "https://example.com/notify", + "--header", "Authorization=Bearer t=1", "--header", "X-Team=ai", + ) + require.NoError(t, res.Err) + h.ContainsOnSameLine(res.Stdout.String(), "Added listener listener-7 to channel ch") + require.Len(t, f.registers, 1) + reg := f.registers[0] + assert.Equal(t, "ch", reg.GetChannel()) + assert.NotEmpty(t, reg.GetRequestId()) + assert.NotEmpty(t, reg.GetIdentity()) + nexus := reg.GetCallback().GetNexus() + assert.Equal(t, "https://example.com/notify", nexus.GetUrl()) + assert.Equal(t, map[string]string{ + "Authorization": "Bearer t=1", + "X-Team": "ai", + }, nexus.GetHeader()) + + res = h.Execute( + "channel", "listener", "remove", "--address", addr, "-c", "ch", + "--listener-id", "listener-7", + ) + require.NoError(t, res.Err) + h.ContainsOnSameLine(res.Stdout.String(), "Removed listener listener-7 from channel ch") + require.Len(t, f.unregister, 1) + assert.Equal(t, "listener-7", f.unregister[0].GetListenerId()) + assert.Equal(t, "ch", f.unregister[0].GetChannel()) +} + +func TestChannel_Poll(t *testing.T) { + f := &fakeChannelService{pollAnswers: [][]*notificationpb.Notification{{ + {Channel: "ch", Position: []byte("offset-4"), Counter: 4, + Metadata: map[string]*commonpb.Payload{"topic": jsonPayload(`"scores"`)}}, + {Channel: "ch", Position: []byte("offset-6"), Counter: 6}, + }}} + addr := startFakeChannelService(t, f) + h := NewCommandHarness(t) + + res := h.Execute( + "channel", "poll", "--address", addr, "-c", "ch", + "--after-counter", "3", "--wait", "5s", "--max", "10", + ) + require.NoError(t, res.Err) + out := res.Stdout.String() + h.ContainsOnSameLine(out, "Counter", "Position", "Metadata") + h.ContainsOnSameLine(out, "4", "offset-4", `topic="scores"`) + h.ContainsOnSameLine(out, "6", "offset-6") + require.Len(t, f.polls, 1) + assert.Equal(t, int64(3), f.polls[0].GetAfterCounter()) + assert.Equal(t, 5*time.Second, f.polls[0].GetWait().AsDuration()) + assert.Equal(t, int32(10), f.polls[0].GetMaxNotifications()) + + // Nothing retained and nothing arriving within the wait reads as such. + f.pollAnswers = append(f.pollAnswers, nil) + res = h.Execute("channel", "poll", "--address", addr, "-c", "ch", "--after-counter", "6") + require.NoError(t, res.Err) + h.ContainsOnSameLine(res.Stdout.String(), "No notifications on channel ch after counter 6") +} + +func TestChannel_PollFollow(t *testing.T) { + h := NewCommandHarness(t) + f := &fakeChannelService{ + pollAnswers: [][]*notificationpb.Notification{ + {{Channel: "ch", Position: []byte("a"), Counter: 2}}, + {}, + { + {Channel: "ch", Position: []byte("b"), Counter: 5}, + {Channel: "ch", Position: []byte("c"), Counter: 3}, + }, + }, + // The fourth poll is the command waiting on an idle channel; stop it + // there the way an interrupt would. + onPoll: func(n int) { + if n == 4 { + h.CancelContext() + } + }, + } + addr := startFakeChannelService(t, f) + + res := h.Execute("channel", "poll", "--address", addr, "-c", "ch", "--follow", "-o", "jsonl") + require.NoError(t, res.Err) + var counters []int64 + for _, raw := range decodeJSONValues(t, res.Stdout.String()) { + var n notificationpb.Notification + require.NoError(t, temporalcli.UnmarshalProtoJSONWithOptions(raw, &n, true)) + counters = append(counters, n.GetCounter()) + } + assert.Equal(t, []int64{2, 5, 3}, counters) + + // Each poll starts after the highest counter seen so far, and an empty + // answer keeps the cursor where it was. + f.mu.Lock() + defer f.mu.Unlock() + require.Len(t, f.polls, 4) + var after []int64 + for _, p := range f.polls { + after = append(after, p.GetAfterCounter()) + } + assert.Equal(t, []int64{0, 2, 2, 5}, after) +} + +func TestChannel_PlainRefusals(t *testing.T) { + f := &fakeChannelService{} + addr := startFakeChannelService(t, f) + h := NewCommandHarness(t) + + f.err = serviceerror.NewNotFound("channel not found") + res := h.Execute("channel", "describe", "--address", addr, "-c", "missing") + require.Error(t, res.Err) + assert.Contains(t, res.Err.Error(), `there is no channel "missing"`) + assert.Contains(t, res.Err.Error(), "The server said: channel not found") + res = h.Execute("channel", "poll", "--address", addr, "-c", "missing") + require.Error(t, res.Err) + assert.Contains(t, res.Err.Error(), `there is no channel "missing"`) + + f.err = serviceerror.NewResourceExhaustedf(enumspb.RESOURCE_EXHAUSTED_CAUSE_CONCURRENT_LIMIT, + "channel already has 1000 listeners, which is the limit") + res = h.Execute( + "channel", "listener", "add", "--address", addr, "-c", "ch", "--callback-url", "http://x", + ) + require.Error(t, res.Err) + assert.Contains(t, res.Err.Error(), "as many listeners as it may hold") + assert.Contains(t, res.Err.Error(), "1000 listeners") + + f.err = &serviceerror.ResourceExhausted{ + Cause: enumspb.RESOURCE_EXHAUSTED_CAUSE_RPS_LIMIT, + Scope: enumspb.RESOURCE_EXHAUSTED_SCOPE_NAMESPACE, + Message: "namespace is over its limit", + } + res = h.Execute( + "channel", "notify", "--address", addr, "-c", "ch", "--position", "1", "--counter", "1", + ) + require.Error(t, res.Err) + assert.Contains(t, res.Err.Error(), "notifying channels faster than it may") + + // Errors the CLI has nothing to add to pass through as they came. + f.err = serviceerror.NewInvalidArgument("notification position exceeds 1024 bytes") + res = h.Execute( + "channel", "notify", "--address", addr, "-c", "ch", "--position", "1", "--counter", "1", + ) + require.Error(t, res.Err) + assert.Contains(t, res.Err.Error(), "position exceeds 1024 bytes") + assert.NotContains(t, res.Err.Error(), "The server said") +} + +// decodeJSONValues splits output holding one JSON value after another. +func decodeJSONValues(t *testing.T, s string) []json.RawMessage { + var out []json.RawMessage + dec := json.NewDecoder(strings.NewReader(s)) + for dec.More() { + var raw json.RawMessage + require.NoError(t, dec.Decode(&raw)) + out = append(out, raw) + } + return out +} + +func (s *SharedServerSuite) TestChannel_NotifyWithoutListenersIsRetained() { + ch := "channel-" + uuid.NewString() + res := s.Execute( + "channel", "notify", "--address", s.Address(), "-c", ch, + "--position", "offset-1", "--counter", "1", "--metadata", `topic="scores"`, + ) + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "Listeners reached", "0") + + res = s.Execute("channel", "describe", "--address", s.Address(), "-c", ch) + s.NoError(res.Err) + out := res.Stdout.String() + s.ContainsOnSameLine(out, "RetainedCount", "1") + s.ContainsOnSameLine(out, "LatestCounter", "1") + s.ContainsOnSameLine(out, "LatestPosition", "offset-1") + s.ContainsOnSameLine(out, "LatestMetadata", `topic="scores"`) + s.ContainsOnSameLine(out, "Callback listeners: 0") +} + +func (s *SharedServerSuite) TestChannel_StaleCounterIsFolded() { + ch := "channel-" + uuid.NewString() + notify := func(position, counter string) { + res := s.Execute( + "channel", "notify", "--address", s.Address(), "-c", ch, + "--position", position, "--counter", counter, + ) + s.NoError(res.Err) + } + notify("p5", "5") + // An equal or lower counter is folded into the one already kept. + notify("p5-again", "5") + notify("p3", "3") + res := s.Execute("channel", "describe", "--address", s.Address(), "-c", ch) + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "RetainedCount", "1") + s.ContainsOnSameLine(res.Stdout.String(), "LatestPosition", "p5") + + notify("p7", "7") + notify("p9", "9") + res = s.Execute( + "channel", "poll", "--address", s.Address(), "-c", ch, + "--after-counter", "5", "--wait", "2s", "-o", "jsonl", + ) + s.NoError(res.Err) + var counters []int64 + for _, raw := range decodeJSONValues(s.T(), res.Stdout.String()) { + var n notificationpb.Notification + s.NoError(temporalcli.UnmarshalProtoJSONWithOptions(raw, &n, true)) + counters = append(counters, n.GetCounter()) + } + s.Equal([]int64{7, 9}, counters) +} + +func (s *SharedServerSuite) TestChannel_PollWaitsForTheNextNotification() { + ch := "channel-" + uuid.NewString() + res := s.Execute( + "channel", "notify", "--address", s.Address(), "-c", ch, "--position", "p1", "--counter", "1", + ) + s.NoError(res.Err) + + done := make(chan *CommandResult, 1) + started := time.Now() + go func() { + done <- s.Execute( + "channel", "poll", "--address", s.Address(), "-c", ch, + "--after-counter", "1", "--wait", "20s", "-o", "json", + ) + }() + // The poll is parked on the channel by the time this lands. + time.Sleep(time.Second) + res = s.Execute( + "channel", "notify", "--address", s.Address(), "-c", ch, "--position", "p2", "--counter", "2", + ) + s.NoError(res.Err) + + select { + case res = <-done: + case <-time.After(60 * time.Second): + s.Fail("poll did not return after the notification") + } + s.NoError(res.Err) + s.Less(time.Since(started), 15*time.Second, "the poll answered on arrival, not at its wait") + raw := decodeJSONValues(s.T(), res.Stdout.String()) + s.Len(raw, 1) + var list []json.RawMessage + s.NoError(json.Unmarshal(raw[0], &list)) + s.Len(list, 1) + var n notificationpb.Notification + s.NoError(temporalcli.UnmarshalProtoJSONWithOptions(list[0], &n, true)) + s.Equal(int64(2), n.GetCounter()) + s.Equal([]byte("p2"), n.GetPosition()) +} + +func (s *SharedServerSuite) TestChannel_CallbackListenerRoundTrip() { + type delivery struct { + auth string + body []byte + } + deliveries := make(chan delivery, 8) + receiver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + body, _ := io.ReadAll(r.Body) + deliveries <- delivery{auth: r.Header.Get("Authorization"), body: body} + })) + defer receiver.Close() + + ch := "channel-" + uuid.NewString() + res := s.Execute( + "channel", "listener", "add", "--address", s.Address(), "-c", ch, + "--callback-url", receiver.URL+"/notify", "--header", "Authorization=Bearer t1", + "-o", "json", + ) + s.NoError(res.Err) + var added workflowservice.RegisterChannelListenerResponse + s.NoError(temporalcli.UnmarshalProtoJSONWithOptions(res.Stdout.Bytes(), &added, true)) + s.NotEmpty(added.GetListenerId()) + + res = s.Execute("channel", "describe", "--address", s.Address(), "-c", ch) + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "Callback listeners: 1") + s.ContainsOnSameLine(res.Stdout.String(), added.GetListenerId(), receiver.URL+"/notify") + + res = s.Execute( + "channel", "notify", "--address", s.Address(), "-c", ch, "--position", "p1", "--counter", "1", + ) + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "Listeners reached", "1") + select { + case d := <-deliveries: + s.Equal("Bearer t1", d.auth) + s.Contains(string(d.body), ch) + case <-time.After(30 * time.Second): + s.Fail("the callback was not called") + } + + res = s.Execute( + "channel", "listener", "remove", "--address", s.Address(), "-c", ch, + "--listener-id", added.GetListenerId(), + ) + s.NoError(res.Err) + res = s.Execute("channel", "describe", "--address", s.Address(), "-c", ch) + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "Callback listeners: 0") +} + +func (s *SharedServerSuite) TestChannel_UnknownChannel() { + res := s.Execute("channel", "describe", "--address", s.Address(), "-c", "never-"+uuid.NewString()) + s.ErrorContains(res.Err, "there is no channel") +} + +func workflowExecution(id, run string) *commonpb.Execution { + return &commonpb.Execution{ + Type: enumspb.EXECUTION_TYPE_WORKFLOW, BusinessId: id, RunId: run, + } +} + +func activityExecution(id, run string) *commonpb.Execution { + return &commonpb.Execution{ + Type: enumspb.EXECUTION_TYPE_ACTIVITY, BusinessId: id, RunId: run, + } +} + +// linkedChannelRequests checks that every call named the owner with the type +// its flag stands for, and the run only where it was given. +func linkedChannelRequests(t *testing.T, f *fakeChannelService, want *commonpb.Execution) { + t.Helper() + f.mu.Lock() + defer f.mu.Unlock() + withRun := []*commonpb.Execution{f.notifies[0].GetExecution()} + withoutRun := []*commonpb.Execution{ + f.describes[0].GetExecution(), f.polls[0].GetExecution(), + f.registers[0].GetExecution(), f.unregister[0].GetExecution(), + } + for _, got := range append(withRun, withoutRun...) { + assert.Equal(t, want.GetType(), got.GetType()) + assert.Equal(t, want.GetBusinessId(), got.GetBusinessId()) + } + assert.Equal(t, want.GetRunId(), withRun[0].GetRunId()) + for _, got := range withoutRun { + assert.Empty(t, got.GetRunId()) + } +} + +func TestChannel_LinkedToWorkflow(t *testing.T) { + f := &fakeChannelService{ + describe: &workflowservice.DescribeChannelResponse{ + Kind: notificationpb.CHANNEL_KIND_LINKED, + LinkedTo: workflowExecution("wf-1", "run-9"), + RetainedCount: 1, + Latest: ¬ificationpb.Notification{Channel: "ch", Position: []byte("p1"), Counter: 1}, + Listeners: []*notificationpb.ChannelListener{{ + ListenerId: "wf-1", + Listener: ¬ificationpb.ChannelListener_Workflow{ + Workflow: ¬ificationpb.WorkflowListener{WorkflowId: "wf-1", RunId: "run-9"}, + }, + }}, + }, + pollAnswers: [][]*notificationpb.Notification{{{Channel: "ch", Counter: 1}}}, + } + addr := startFakeChannelService(t, f) + h := NewCommandHarness(t) + linked := []string{"--address", addr, "-c", "ch", "--workflow-id", "wf-1"} + + res := h.Execute(append([]string{"channel", "notify", "--position", "p1", "--counter", "1", + "--run-id", "run-9"}, linked...)...) + require.NoError(t, res.Err) + h.ContainsOnSameLine(res.Stdout.String(), "Notified channel ch of workflow wf-1") + res = h.Execute(append([]string{"channel", "describe"}, linked...)...) + require.NoError(t, res.Err) + out := res.Stdout.String() + h.ContainsOnSameLine(out, "Kind", "Linked") + h.ContainsOnSameLine(out, "LinkedTo", "workflow wf-1 (run run-9)") + h.ContainsOnSameLine(out, "RetainedCount", "1") + // The owner listens without registering, so its row has no time. + h.ContainsOnSameLine(out, "Workflow listeners: 1") + assert.NotContains(t, out, "ago") + res = h.Execute(append([]string{"channel", "poll"}, linked...)...) + require.NoError(t, res.Err) + res = h.Execute(append([]string{"channel", "listener", "add", "--callback-url", + "http://x"}, linked...)...) + require.NoError(t, res.Err) + h.ContainsOnSameLine(res.Stdout.String(), "to channel ch of workflow wf-1") + res = h.Execute(append([]string{"channel", "listener", "remove", "--listener-id", + "listener-7"}, linked...)...) + require.NoError(t, res.Err) + + linkedChannelRequests(t, f, workflowExecution("wf-1", "run-9")) +} + +func TestChannel_LinkedToActivity(t *testing.T) { + f := &fakeChannelService{ + describe: &workflowservice.DescribeChannelResponse{ + Kind: notificationpb.CHANNEL_KIND_LINKED, + LinkedTo: activityExecution("act-1", ""), + RetainedCount: 2, + Latest: ¬ificationpb.Notification{Channel: "ch", Position: []byte("p2"), Counter: 2}, + }, + pollAnswers: [][]*notificationpb.Notification{{{Channel: "ch", Counter: 2}}}, + } + addr := startFakeChannelService(t, f) + h := NewCommandHarness(t) + linked := []string{"--address", addr, "-c", "ch", "--activity-id", "act-1"} + + res := h.Execute(append([]string{"channel", "notify", "--position", "p2", "--counter", "2", + "--run-id", "run-3"}, linked...)...) + require.NoError(t, res.Err) + h.ContainsOnSameLine(res.Stdout.String(), "Notified channel ch of activity act-1") + res = h.Execute(append([]string{"channel", "describe"}, linked...)...) + require.NoError(t, res.Err) + out := res.Stdout.String() + h.ContainsOnSameLine(out, "Kind", "Linked") + // The owner line carries no run when the Service names none. + h.ContainsOnSameLine(out, "LinkedTo", "activity act-1") + assert.NotContains(t, out, "run ") + h.ContainsOnSameLine(out, "RetainedCount", "2") + res = h.Execute(append([]string{"channel", "poll"}, linked...)...) + require.NoError(t, res.Err) + res = h.Execute(append([]string{"channel", "listener", "add", "--callback-url", + "http://x"}, linked...)...) + require.NoError(t, res.Err) + h.ContainsOnSameLine(res.Stdout.String(), "to channel ch of activity act-1") + res = h.Execute(append([]string{"channel", "listener", "remove", "--listener-id", + "listener-7"}, linked...)...) + require.NoError(t, res.Err) + + linkedChannelRequests(t, f, activityExecution("act-1", "run-3")) + + // The JSON output carries the owner as the Service sent it. + res = h.Execute(append([]string{"channel", "describe", "-o", "json"}, linked...)...) + require.NoError(t, res.Err) + var described workflowservice.DescribeChannelResponse + require.NoError(t, temporalcli.UnmarshalProtoJSONWithOptions(res.Stdout.Bytes(), &described, true)) + assert.Equal(t, enumspb.EXECUTION_TYPE_ACTIVITY, described.GetLinkedTo().GetType()) + assert.Equal(t, "act-1", described.GetLinkedTo().GetBusinessId()) +} + +func TestChannel_IndependentByDefault(t *testing.T) { + f := &fakeChannelService{describe: &workflowservice.DescribeChannelResponse{}} + addr := startFakeChannelService(t, f) + h := NewCommandHarness(t) + + res := h.Execute("channel", "notify", "--address", addr, "-c", "ch", + "--position", "p", "--counter", "1") + require.NoError(t, res.Err) + // A server that predates the linked kind leaves the kind unset. + res = h.Execute("channel", "describe", "--address", addr, "-c", "ch") + require.NoError(t, res.Err) + h.ContainsOnSameLine(res.Stdout.String(), "Kind", "Independent") + assert.NotContains(t, res.Stdout.String(), "LinkedTo") + assert.Nil(t, f.notifies[0].GetExecution()) + assert.Nil(t, f.describes[0].GetExecution()) +} + +func TestChannel_LinkedNotFound(t *testing.T) { + f := &fakeChannelService{err: serviceerror.NewNotFound("workflow execution already completed")} + addr := startFakeChannelService(t, f) + h := NewCommandHarness(t) + + res := h.Execute("channel", "notify", "--address", addr, "-c", "ch", "-w", "gone", + "--position", "p", "--counter", "1") + require.Error(t, res.Err) + assert.Contains(t, res.Err.Error(), `workflow "gone" has no running execution`) + assert.Contains(t, res.Err.Error(), `channel "ch" of workflow "gone"`) + + f.err = serviceerror.NewNotFound("activity execution already completed") + res = h.Execute("channel", "describe", "--address", addr, "-c", "ch", "--activity-id", "gone") + require.Error(t, res.Err) + assert.Contains(t, res.Err.Error(), `activity "gone" has no running execution`) + assert.Contains(t, res.Err.Error(), `channel "ch" of activity "gone"`) +} + +// runningWorkflowDescription is a description with what the text output +// dereferences, for a run that is still open so no close event is fetched. +func runningWorkflowDescription( + subs ...*workflowpb.ChannelSubscriptionInfo, +) *workflowservice.DescribeWorkflowExecutionResponse { + return &workflowservice.DescribeWorkflowExecutionResponse{ + WorkflowExecutionInfo: &workflowpb.WorkflowExecutionInfo{ + Execution: &commonpb.WorkflowExecution{WorkflowId: "wf-1", RunId: "run-1"}, + Type: &commonpb.WorkflowType{Name: "DevWorkflow"}, + Status: enumspb.WORKFLOW_EXECUTION_STATUS_RUNNING, + StartTime: timestamppb.New(time.Date(2026, 10, 2, 9, 0, 0, 0, time.UTC)), + }, + ChannelSubscriptions: subs, + } +} + +func TestChannel_WorkflowDescribeListsChannels(t *testing.T) { + f := &fakeChannelService{describeWorkflow: runningWorkflowDescription( + &workflowpb.ChannelSubscriptionInfo{ + Channel: "demo", + Kind: notificationpb.CHANNEL_KIND_INDEPENDENT, + SubscribedEventId: 5, + LastCounter: 4, + PendingNotification: ¬ificationpb.Notification{Channel: "demo", Counter: 5}, + }, + &workflowpb.ChannelSubscriptionInfo{ + Channel: "stream/scores", + Kind: notificationpb.CHANNEL_KIND_LINKED, + LastCounter: 3, + ScheduledCounter: 0, + ListenerCount: 2, + RetainedCount: 16, + AcceptedCount: 3, + }, + )} + addr := startFakeChannelService(t, f) + h := NewCommandHarness(t) + + res := h.Execute("workflow", "describe", "--address", addr, "-w", "wf-1") + require.NoError(t, res.Err) + out := res.Stdout.String() + h.ContainsOnSameLine(out, "Notification Channels: 2") + h.ContainsOnSameLine(out, "Channel", "Kind", "LastCounter", "PendingCounter", + "ScheduledCounter", "Listeners", "Retained") + // The pending column is blank, not zero, when nothing is pending. + assert.Equal(t, []string{"demo", "Independent", "4", "5", "0", "0", "0"}, + strings.Fields(lineContaining(t, out, "demo"))) + assert.Equal(t, []string{"stream/scores", "Linked", "3", "0", "2", "16"}, + strings.Fields(lineContaining(t, out, "stream/scores"))) + + res = h.Execute("workflow", "describe", "--address", addr, "-w", "wf-1", "-o", "json") + require.NoError(t, res.Err) + var described workflowservice.DescribeWorkflowExecutionResponse + require.NoError(t, temporalcli.UnmarshalProtoJSONWithOptions(res.Stdout.Bytes(), &described, true)) + require.Len(t, described.GetChannelSubscriptions(), 2) + assert.Equal(t, int64(5), described.GetChannelSubscriptions()[0].GetSubscribedEventId()) + assert.Equal(t, int64(5), + described.GetChannelSubscriptions()[0].GetPendingNotification().GetCounter()) + linked := described.GetChannelSubscriptions()[1] + assert.Equal(t, notificationpb.CHANNEL_KIND_LINKED, linked.GetKind()) + assert.Equal(t, int64(3), linked.GetAcceptedCount()) + + // A workflow standing on no channel has no section at all. + f.describeWorkflow = runningWorkflowDescription() + res = h.Execute("workflow", "describe", "--address", addr, "-w", "wf-1") + require.NoError(t, res.Err) + assert.NotContains(t, res.Stdout.String(), "Notification Channels") +} + +func lineContaining(t *testing.T, text, piece string) string { + for _, line := range strings.Split(text, "\n") { + if strings.Contains(line, piece) { + return line + } + } + require.Failf(t, "line not found", "no line contains %q", piece) + return "" +} + +func (s *SharedServerSuite) TestChannel_WorkflowDescribeListsLinked() { + s.Worker().OnDevWorkflow(func(ctx workflow.Context, a any) (any, error) { + workflow.GetSignalChannel(ctx, "finish").Receive(ctx, nil) + return nil, nil + }) + run, err := s.Client.ExecuteWorkflow( + s.Context, + client.StartWorkflowOptions{TaskQueue: s.Worker().Options.TaskQueue}, + DevWorkflow, + "ignored", + ) + s.NoError(err) + describe := func(args ...string) *CommandResult { + return s.Execute(append([]string{"workflow", "describe", "--address", s.Address(), + "-w", run.GetID()}, args...)...) + } + + // A fresh workflow stands on no channel, so the section is absent. + res := describe() + s.NoError(res.Err) + s.NotContains(res.Stdout.String(), "Notification Channels") + + ch := "channel-" + uuid.NewString() + res = s.Execute("channel", "notify", "--address", s.Address(), "-c", ch, + "--workflow-id", run.GetID(), "--position", "p1", "--counter", "1") + s.NoError(res.Err) + + res = describe() + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "Notification Channels: 1") + s.ContainsOnSameLine(res.Stdout.String(), ch, "Linked") + + res = describe("-o", "json") + s.NoError(res.Err) + var described workflowservice.DescribeWorkflowExecutionResponse + s.NoError(temporalcli.UnmarshalProtoJSONWithOptions(res.Stdout.Bytes(), &described, true)) + s.Len(described.GetChannelSubscriptions(), 1) + linked := described.GetChannelSubscriptions()[0] + s.Equal(ch, linked.GetChannel()) + s.Equal(notificationpb.CHANNEL_KIND_LINKED, linked.GetKind()) + s.Equal(int32(1), linked.GetRetainedCount()) + s.Equal(int64(1), linked.GetAcceptedCount()) + + s.NoError(s.Client.SignalWorkflow(s.Context, run.GetID(), "", "finish", nil)) + s.NoError(run.Get(s.Context, nil)) +} + +func (s *SharedServerSuite) TestChannel_LinkedToWorkflow() { + s.Worker().OnDevWorkflow(func(ctx workflow.Context, a any) (any, error) { + workflow.GetSignalChannel(ctx, "finish").Receive(ctx, nil) + return nil, nil + }) + run, err := s.Client.ExecuteWorkflow( + s.Context, + client.StartWorkflowOptions{TaskQueue: s.Worker().Options.TaskQueue}, + DevWorkflow, + "ignored", + ) + s.NoError(err) + ch := "channel-" + uuid.NewString() + linked := []string{"--address", s.Address(), "-c", ch, "--workflow-id", run.GetID()} + channel := func(args ...string) *CommandResult { + return s.Execute(append(append([]string{"channel"}, args...), linked...)...) + } + + // The owner listens by construction, so the first notify reaches it. + res := channel("notify", "--position", "p1", "--counter", "1") + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "Listeners reached", "1") + + res = channel("describe") + s.NoError(res.Err) + out := res.Stdout.String() + s.ContainsOnSameLine(out, "Kind", "Linked") + s.ContainsOnSameLine(out, "LinkedTo", + "workflow "+run.GetID()+" (run "+run.GetRunID()+")") + s.ContainsOnSameLine(out, "RetainedCount", "1") + s.ContainsOnSameLine(out, "LatestPosition", "p1") + s.ContainsOnSameLine(out, "Workflow listeners: 1") + + res = channel("poll", "--after-counter", "0", "--wait", "5s", "-o", "jsonl") + s.NoError(res.Err) + raw := decodeJSONValues(s.T(), res.Stdout.String()) + s.Len(raw, 1) + var n notificationpb.Notification + s.NoError(temporalcli.UnmarshalProtoJSONWithOptions(raw[0], &n, true)) + s.Equal(int64(1), n.GetCounter()) + s.Equal(enumspb.EXECUTION_TYPE_WORKFLOW, n.GetLinkedTo().GetType()) + s.Equal(run.GetID(), n.GetLinkedTo().GetBusinessId()) + + res = channel("listener", "add", "--callback-url", "http://127.0.0.1:1/notify", "-o", "json") + s.NoError(res.Err) + var added workflowservice.RegisterChannelListenerResponse + s.NoError(temporalcli.UnmarshalProtoJSONWithOptions(res.Stdout.Bytes(), &added, true)) + res = channel("describe") + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "Callback listeners: 1") + s.ContainsOnSameLine(res.Stdout.String(), added.GetListenerId(), "http://127.0.0.1:1/notify") + res = channel("listener", "remove", "--listener-id", added.GetListenerId()) + s.NoError(res.Err) + res = channel("describe") + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "Callback listeners: 0") + + // A name nobody has used still exists on a running workflow. + res = s.Execute("channel", "describe", "--address", s.Address(), + "-c", "untouched-"+uuid.NewString(), "--workflow-id", run.GetID()) + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "Kind", "Linked") + s.ContainsOnSameLine(res.Stdout.String(), "RetainedCount", "0") + s.ContainsOnSameLine(res.Stdout.String(), "Workflow listeners: 0") + s.ContainsOnSameLine(res.Stdout.String(), "Callback listeners: 0") + + // The independent channel of the same name is a different channel. + res = s.Execute("channel", "describe", "--address", s.Address(), "-c", ch) + s.ErrorContains(res.Err, "there is no channel") + + res = s.Execute("channel", "describe", "--address", s.Address(), "-c", ch, "--run-id", "r") + s.ErrorContains(res.Err, "--run-id requires --workflow-id or --activity-id") + + s.NoError(s.Client.SignalWorkflow(s.Context, run.GetID(), "", "finish", nil)) + s.NoError(run.Get(s.Context, nil)) +} + +func (s *SharedServerSuite) TestChannel_LinkedToActivity() { + // The activity stays running until the test is done with its channels. + release := make(chan struct{}) + s.Worker().OnDevActivity(func(ctx context.Context, a any) (any, error) { + select { + case <-release: + case <-ctx.Done(): + } + return nil, nil + }) + defer close(release) + activityID := "activity-" + uuid.NewString() + res := s.Execute("activity", "start", "--address", s.Address(), "-o", "json", + "--activity-id", activityID, "--type", "DevActivity", + "--task-queue", s.Worker().Options.TaskQueue, "--start-to-close-timeout", "1m") + s.NoError(res.Err) + var started map[string]any + s.NoError(json.Unmarshal(res.Stdout.Bytes(), &started)) + runID, _ := started["runId"].(string) + s.NotEmpty(runID) + + ch := "channel-" + uuid.NewString() + linked := []string{"--address", s.Address(), "-c", ch, "--activity-id", activityID} + channel := func(args ...string) *CommandResult { + return s.Execute(append(append([]string{"channel"}, args...), linked...)...) + } + + // The activity does not listen on its own channels, so the notify reaches + // nobody and is retained for pollers. + res = channel("notify", "--position", "p1", "--counter", "1") + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "Listeners reached", "0") + + res = channel("describe") + s.NoError(res.Err) + out := res.Stdout.String() + s.ContainsOnSameLine(out, "Kind", "Linked") + s.ContainsOnSameLine(out, "LinkedTo", "activity "+activityID+" (run "+runID+")") + s.ContainsOnSameLine(out, "RetainedCount", "1") + s.ContainsOnSameLine(out, "LatestPosition", "p1") + s.ContainsOnSameLine(out, "Workflow listeners: 0") + + res = channel("poll", "--after-counter", "0", "--wait", "5s", "-o", "jsonl") + s.NoError(res.Err) + raw := decodeJSONValues(s.T(), res.Stdout.String()) + s.Len(raw, 1) + var n notificationpb.Notification + s.NoError(temporalcli.UnmarshalProtoJSONWithOptions(raw[0], &n, true)) + s.Equal(int64(1), n.GetCounter()) + s.Equal(enumspb.EXECUTION_TYPE_ACTIVITY, n.GetLinkedTo().GetType()) + s.Equal(activityID, n.GetLinkedTo().GetBusinessId()) + + res = channel("listener", "add", "--callback-url", "http://127.0.0.1:1/notify", "-o", "json") + s.NoError(res.Err) + var added workflowservice.RegisterChannelListenerResponse + s.NoError(temporalcli.UnmarshalProtoJSONWithOptions(res.Stdout.Bytes(), &added, true)) + res = channel("describe") + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "Callback listeners: 1") + res = channel("listener", "remove", "--listener-id", added.GetListenerId()) + s.NoError(res.Err) + + // The activity's own stream notifies the linked channel named for the + // stream alone, as the describe card says it will. + res = s.Execute("stream", "append", "--address", s.Address(), "--activity-id", activityID, + "--name", "scores", "--input", `{"home": 1}`) + s.NoError(res.Err) + res = s.Execute("stream", "describe", "--address", s.Address(), "--activity-id", activityID, + "--name", "scores") + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "Channel", "stream/scores") + s.ContainsOnSameLine(res.Stdout.String(), "Kind", "Linked") + s.ContainsOnSameLine(res.Stdout.String(), "LinkedTo", "activity "+activityID) + res = s.Execute("channel", "poll", "--address", s.Address(), "-c", "stream/scores", + "--activity-id", activityID, "--after-counter", "0", "--wait", "10s", "-o", "jsonl") + s.NoError(res.Err) + raw = decodeJSONValues(s.T(), res.Stdout.String()) + s.NotEmpty(raw) + var streamed notificationpb.Notification + s.NoError(temporalcli.UnmarshalProtoJSONWithOptions(raw[len(raw)-1], &streamed, true)) + s.Equal(runID+":1", string(streamed.GetPosition())) + s.Equal(enumspb.EXECUTION_TYPE_ACTIVITY, streamed.GetLinkedTo().GetType()) + s.Equal(activityID, streamed.GetLinkedTo().GetBusinessId()) + + // An activity nobody started has no channels to reach. + res = s.Execute("channel", "describe", "--address", s.Address(), "-c", ch, + "--activity-id", "never-"+uuid.NewString()) + s.ErrorContains(res.Err, "has no running execution to reach") +} diff --git a/internal/temporalcli/commands.gen.go b/internal/temporalcli/commands.gen.go index a48152714..59cc77623 100644 --- a/internal/temporalcli/commands.gen.go +++ b/internal/temporalcli/commands.gen.go @@ -97,6 +97,52 @@ func (v *WorkflowReferenceOptions) BuildFlags(f *pflag.FlagSet) { f.StringVarP(&v.RunId, "run-id", "r", "", "Run ID.") } +type ChannelOptions struct { + Channel string + WorkflowId string + ActivityId string + RunId string + FlagSet *pflag.FlagSet +} + +func (v *ChannelOptions) BuildFlags(f *pflag.FlagSet) { + v.FlagSet = f + f.StringVarP(&v.Channel, "channel", "c", "", "Name of the notification channel. Required.") + _ = cobra.MarkFlagRequired(f, "channel") + f.StringVarP(&v.WorkflowId, "workflow-id", "w", "", "Workflow ID of the Workflow Execution the channel is linked to. Without an owner the command uses the independent channel of that name. Cannot use with --activity-id.") + f.StringVar(&v.ActivityId, "activity-id", "", "Activity ID of the standalone Activity the channel is linked to. Cannot use with --workflow-id.") + f.StringVarP(&v.RunId, "run-id", "r", "", "Run ID of the linked execution. Defaults to the current run. Requires --workflow-id or --activity-id.") +} + +type StreamIdOptions struct { + StreamId string + FlagSet *pflag.FlagSet +} + +func (v *StreamIdOptions) BuildFlags(f *pflag.FlagSet) { + v.FlagSet = f + f.StringVarP(&v.StreamId, "stream-id", "s", "", "Stream ID of a standalone Stream. Required.") + _ = cobra.MarkFlagRequired(f, "stream-id") +} + +type StreamReferenceOptions struct { + StreamId string + WorkflowId string + RunId string + ActivityId string + Name string + FlagSet *pflag.FlagSet +} + +func (v *StreamReferenceOptions) BuildFlags(f *pflag.FlagSet) { + v.FlagSet = f + f.StringVarP(&v.StreamId, "stream-id", "s", "", "Stream ID of a standalone Stream. You must set either --stream-id or an owner.") + f.StringVarP(&v.WorkflowId, "workflow-id", "w", "", "Workflow ID of the Workflow Execution owning the Stream. Cannot use with --stream-id.") + f.StringVarP(&v.RunId, "run-id", "r", "", "Run ID of the owner. Defaults to the current run.") + f.StringVar(&v.ActivityId, "activity-id", "", "Activity ID of the Activity owning the Stream. With --workflow-id it names an Activity the Workflow scheduled. On its own it names a standalone Activity.") + f.StringVar(&v.Name, "name", "", "Name of the Stream within its owner. Defaults to the owner's default Stream.") +} + type DeploymentNameOptions struct { Name string FlagSet *pflag.FlagSet @@ -530,12 +576,14 @@ func NewTemporalCommand(cctx *CommandContext) *TemporalCommand { s.Command.Args = cobra.NoArgs s.Command.AddCommand(&NewTemporalActivityCommand(cctx, &s).Command) s.Command.AddCommand(&NewTemporalBatchCommand(cctx, &s).Command) + s.Command.AddCommand(&NewTemporalChannelCommand(cctx, &s).Command) s.Command.AddCommand(&NewTemporalConfigCommand(cctx, &s).Command) s.Command.AddCommand(&NewTemporalEnvCommand(cctx, &s).Command) s.Command.AddCommand(&NewTemporalNexusCommand(cctx, &s).Command) s.Command.AddCommand(&NewTemporalOperatorCommand(cctx, &s).Command) s.Command.AddCommand(&NewTemporalScheduleCommand(cctx, &s).Command) s.Command.AddCommand(&NewTemporalServerCommand(cctx, &s).Command) + s.Command.AddCommand(&NewTemporalStreamCommand(cctx, &s).Command) s.Command.AddCommand(&NewTemporalTaskQueueCommand(cctx, &s).Command) s.Command.AddCommand(&NewTemporalWorkerCommand(cctx, &s).Command) s.Command.AddCommand(&NewTemporalWorkflowCommand(cctx, &s).Command) @@ -1191,6 +1239,213 @@ func NewTemporalBatchTerminateCommand(cctx *CommandContext, parent *TemporalBatc return &s } +type TemporalChannelCommand struct { + Parent *TemporalCommand + Command cobra.Command + cliext.ClientOptions +} + +func NewTemporalChannelCommand(cctx *CommandContext, parent *TemporalCommand) *TemporalChannelCommand { + var s TemporalChannelCommand + s.Parent = parent + s.Command.Use = "channel" + s.Command.Short = "Notify and listen on notification channels" + if hasHighlighting { + s.Command.Long = "A notification channel is a name a writer and its listeners agree on.\nThe writer notifies the channel when a source it writes moves, such as\na Stream gaining records, and never learns who listens. A Workflow\nlistener gets a Workflow Task, a callback listener gets an HTTP call,\nand a client long-polls:\n\n\x1b[1mtemporal channel [command] [options]\x1b[0m\n\nFor example:\n\n\x1b[1mtemporal channel poll \\\n --channel YourChannel \\\n --follow\x1b[0m\n\nA notification tells listeners where the source stands. It carries no\ndata; the listener reads the source itself.\n\nAn independent channel exists on its own, and any number of Workflows,\ncallbacks and clients listen on it. A channel linked to a Workflow\nExecution lives with that Workflow, which listens on it without\nsubscribing. A channel linked to a standalone Activity lives with that\nActivity the same way. Name the owner with \x1b[1m--workflow-id\x1b[0m or\n\x1b[1m--activity-id\x1b[0m on any channel command:\n\n\x1b[1mtemporal channel notify \\\n --channel YourChannel \\\n --workflow-id YourWorkflowId \\\n --position 42 \\\n --counter 42\x1b[0m" + } else { + s.Command.Long = "A notification channel is a name a writer and its listeners agree on.\nThe writer notifies the channel when a source it writes moves, such as\na Stream gaining records, and never learns who listens. A Workflow\nlistener gets a Workflow Task, a callback listener gets an HTTP call,\nand a client long-polls:\n\n```\ntemporal channel [command] [options]\n```\n\nFor example:\n\n```\ntemporal channel poll \\\n --channel YourChannel \\\n --follow\n```\n\nA notification tells listeners where the source stands. It carries no\ndata; the listener reads the source itself.\n\nAn independent channel exists on its own, and any number of Workflows,\ncallbacks and clients listen on it. A channel linked to a Workflow\nExecution lives with that Workflow, which listens on it without\nsubscribing. A channel linked to a standalone Activity lives with that\nActivity the same way. Name the owner with `--workflow-id` or\n`--activity-id` on any channel command:\n\n```\ntemporal channel notify \\\n --channel YourChannel \\\n --workflow-id YourWorkflowId \\\n --position 42 \\\n --counter 42\n```" + } + s.Command.Args = cobra.NoArgs + s.Command.AddCommand(&NewTemporalChannelDescribeCommand(cctx, &s).Command) + s.Command.AddCommand(&NewTemporalChannelListenerCommand(cctx, &s).Command) + s.Command.AddCommand(&NewTemporalChannelNotifyCommand(cctx, &s).Command) + s.Command.AddCommand(&NewTemporalChannelPollCommand(cctx, &s).Command) + s.ClientOptions.BuildFlags(s.Command.PersistentFlags()) + s.ClientOptions.HideFlags() + return &s +} + +type TemporalChannelDescribeCommand struct { + Parent *TemporalChannelCommand + Command cobra.Command + ChannelOptions +} + +func NewTemporalChannelDescribeCommand(cctx *CommandContext, parent *TemporalChannelCommand) *TemporalChannelDescribeCommand { + var s TemporalChannelDescribeCommand + s.Parent = parent + s.Command.DisableFlagsInUseLine = true + s.Command.Use = "describe [flags]" + s.Command.Short = "Show a channel's listeners and latest notification" + if hasHighlighting { + s.Command.Long = "Show who listens on a channel, its latest notification, and how many\nnotifications it keeps for pollers:\n\n\x1b[1mtemporal channel describe \\\n --channel YourChannel\x1b[0m\n\nAn independent channel exists once a writer notifies it or a listener\nregisters on it, and goes away after a while without listeners or\nactivity. A channel linked to a running Workflow Execution or standalone\nActivity always exists, and its description names that owner:\n\n\x1b[1mtemporal channel describe \\\n --channel YourChannel \\\n --workflow-id YourWorkflowId\x1b[0m" + } else { + s.Command.Long = "Show who listens on a channel, its latest notification, and how many\nnotifications it keeps for pollers:\n\n```\ntemporal channel describe \\\n --channel YourChannel\n```\n\nAn independent channel exists once a writer notifies it or a listener\nregisters on it, and goes away after a while without listeners or\nactivity. A channel linked to a running Workflow Execution or standalone\nActivity always exists, and its description names that owner:\n\n```\ntemporal channel describe \\\n --channel YourChannel \\\n --workflow-id YourWorkflowId\n```" + } + s.Command.Args = cobra.NoArgs + s.ChannelOptions.BuildFlags(s.Command.Flags()) + s.Command.Run = func(c *cobra.Command, args []string) { + if err := s.run(cctx, args); err != nil { + cctx.Options.Fail(err) + } + } + return &s +} + +type TemporalChannelListenerCommand struct { + Parent *TemporalChannelCommand + Command cobra.Command +} + +func NewTemporalChannelListenerCommand(cctx *CommandContext, parent *TemporalChannelCommand) *TemporalChannelListenerCommand { + var s TemporalChannelListenerCommand + s.Parent = parent + s.Command.Use = "listener" + s.Command.Short = "Add or remove a channel's callback listeners" + if hasHighlighting { + s.Command.Long = "Register a callback the Service calls with every notification on a\nchannel, or remove one:\n\n\x1b[1mtemporal channel listener [command] [options]\x1b[0m\n\nA Workflow listens on a channel from its own code instead, and stops\nlistening when its run ends." + } else { + s.Command.Long = "Register a callback the Service calls with every notification on a\nchannel, or remove one:\n\n```\ntemporal channel listener [command] [options]\n```\n\nA Workflow listens on a channel from its own code instead, and stops\nlistening when its run ends." + } + s.Command.Args = cobra.NoArgs + s.Command.AddCommand(&NewTemporalChannelListenerAddCommand(cctx, &s).Command) + s.Command.AddCommand(&NewTemporalChannelListenerRemoveCommand(cctx, &s).Command) + return &s +} + +type TemporalChannelListenerAddCommand struct { + Parent *TemporalChannelListenerCommand + Command cobra.Command + ChannelOptions + CallbackUrl string + Header []string +} + +func NewTemporalChannelListenerAddCommand(cctx *CommandContext, parent *TemporalChannelListenerCommand) *TemporalChannelListenerAddCommand { + var s TemporalChannelListenerAddCommand + s.Parent = parent + s.Command.DisableFlagsInUseLine = true + s.Command.Use = "add [flags]" + s.Command.Short = "Register a callback listener" + if hasHighlighting { + s.Command.Long = "Register a callback on a channel. The Service calls the URL with every\nnotification the channel gets from now on:\n\n\x1b[1mtemporal channel listener add \\\n --channel YourChannel \\\n --callback-url https://example.com/notify \\\n --header \"Authorization=Bearer YourToken\"\x1b[0m\n\nThe command prints the listener ID that \x1b[1mtemporal channel listener\nremove\x1b[0m takes.\n\nThe Service only calls addresses its \x1b[1mcallback.allowedAddresses\x1b[0m\ndynamic configuration value allows. The development server allows\n\x1b[1m127.0.0.1\x1b[0m and \x1b[1mlocalhost\x1b[0m on any port over plain HTTP. Another\nService needs the address added, for example:\n\n\x1b[1m--dynamic-config-value \\\n 'callback.allowedAddresses=[{\"Pattern\":\"*.example.com:443\"}]'\x1b[0m" + } else { + s.Command.Long = "Register a callback on a channel. The Service calls the URL with every\nnotification the channel gets from now on:\n\n```\ntemporal channel listener add \\\n --channel YourChannel \\\n --callback-url https://example.com/notify \\\n --header \"Authorization=Bearer YourToken\"\n```\n\nThe command prints the listener ID that `temporal channel listener\nremove` takes.\n\nThe Service only calls addresses its `callback.allowedAddresses`\ndynamic configuration value allows. The development server allows\n`127.0.0.1` and `localhost` on any port over plain HTTP. Another\nService needs the address added, for example:\n\n```\n--dynamic-config-value \\\n 'callback.allowedAddresses=[{\"Pattern\":\"*.example.com:443\"}]'\n```" + } + s.Command.Args = cobra.NoArgs + s.Command.Flags().StringVar(&s.CallbackUrl, "callback-url", "", "URL the Service calls with each notification. Required.") + _ = cobra.MarkFlagRequired(s.Command.Flags(), "callback-url") + s.Command.Flags().StringArrayVar(&s.Header, "header", nil, "Header sent with every call, in `KEY=VALUE` format. Can be passed multiple times.") + s.ChannelOptions.BuildFlags(s.Command.Flags()) + s.Command.Run = func(c *cobra.Command, args []string) { + if err := s.run(cctx, args); err != nil { + cctx.Options.Fail(err) + } + } + return &s +} + +type TemporalChannelListenerRemoveCommand struct { + Parent *TemporalChannelListenerCommand + Command cobra.Command + ChannelOptions + ListenerId string +} + +func NewTemporalChannelListenerRemoveCommand(cctx *CommandContext, parent *TemporalChannelListenerCommand) *TemporalChannelListenerRemoveCommand { + var s TemporalChannelListenerRemoveCommand + s.Parent = parent + s.Command.DisableFlagsInUseLine = true + s.Command.Use = "remove [flags]" + s.Command.Short = "Remove a callback listener" + if hasHighlighting { + s.Command.Long = "Stop calling a callback listener:\n\n\x1b[1mtemporal channel listener remove \\\n --channel YourChannel \\\n --listener-id YourListenerId\x1b[0m\n\nFind the listener ID with \x1b[1mtemporal channel describe\x1b[0m." + } else { + s.Command.Long = "Stop calling a callback listener:\n\n```\ntemporal channel listener remove \\\n --channel YourChannel \\\n --listener-id YourListenerId\n```\n\nFind the listener ID with `temporal channel describe`." + } + s.Command.Args = cobra.NoArgs + s.Command.Flags().StringVar(&s.ListenerId, "listener-id", "", "ID of the listener to remove. Required.") + _ = cobra.MarkFlagRequired(s.Command.Flags(), "listener-id") + s.ChannelOptions.BuildFlags(s.Command.Flags()) + s.Command.Run = func(c *cobra.Command, args []string) { + if err := s.run(cctx, args); err != nil { + cctx.Options.Fail(err) + } + } + return &s +} + +type TemporalChannelNotifyCommand struct { + Parent *TemporalChannelCommand + Command cobra.Command + ChannelOptions + Position string + Counter int + Metadata []string +} + +func NewTemporalChannelNotifyCommand(cctx *CommandContext, parent *TemporalChannelCommand) *TemporalChannelNotifyCommand { + var s TemporalChannelNotifyCommand + s.Parent = parent + s.Command.DisableFlagsInUseLine = true + s.Command.Use = "notify [flags]" + s.Command.Short = "Notify a channel's listeners" + if hasHighlighting { + s.Command.Long = "Tell every listener of a channel that its source moved. The command\nprints how many listeners the notification reached:\n\n\x1b[1mtemporal channel notify \\\n --channel YourChannel \\\n --position 42 \\\n --counter 42 \\\n --metadata 'topic=\"scores\"'\x1b[0m\n\nThe position is where the source stands now, in the writer's own\nterms. The counter orders notifications on the channel: when several\nwait for a listener at once, it gets the one with the highest counter.\nMetadata values are JSON." + } else { + s.Command.Long = "Tell every listener of a channel that its source moved. The command\nprints how many listeners the notification reached:\n\n```\ntemporal channel notify \\\n --channel YourChannel \\\n --position 42 \\\n --counter 42 \\\n --metadata 'topic=\"scores\"'\n```\n\nThe position is where the source stands now, in the writer's own\nterms. The counter orders notifications on the channel: when several\nwait for a listener at once, it gets the one with the highest counter.\nMetadata values are JSON." + } + s.Command.Args = cobra.NoArgs + s.Command.Flags().StringVar(&s.Position, "position", "", "Where the source stands after the write. Sent as text; the Service does not read it. Required.") + _ = cobra.MarkFlagRequired(s.Command.Flags(), "position") + s.Command.Flags().IntVar(&s.Counter, "counter", 0, "Order of this notification among the channel's notifications. Must be greater than zero. Required.") + _ = cobra.MarkFlagRequired(s.Command.Flags(), "counter") + s.Command.Flags().StringArrayVar(&s.Metadata, "metadata", nil, "Detail for listeners, in `KEY=VALUE` format. Values must be JSON. Can be passed multiple times.") + s.ChannelOptions.BuildFlags(s.Command.Flags()) + s.Command.Run = func(c *cobra.Command, args []string) { + if err := s.run(cctx, args); err != nil { + cctx.Options.Fail(err) + } + } + return &s +} + +type TemporalChannelPollCommand struct { + Parent *TemporalChannelCommand + Command cobra.Command + ChannelOptions + AfterCounter int + Wait cliext.FlagDuration + Max int + Follow bool +} + +func NewTemporalChannelPollCommand(cctx *CommandContext, parent *TemporalChannelCommand) *TemporalChannelPollCommand { + var s TemporalChannelPollCommand + s.Parent = parent + s.Command.DisableFlagsInUseLine = true + s.Command.Use = "poll [flags]" + s.Command.Short = "Wait for a channel's notifications" + if hasHighlighting { + s.Command.Long = "Show the notifications a channel keeps, waiting for one when there is\nnone yet:\n\n\x1b[1mtemporal channel poll \\\n --channel YourChannel\x1b[0m\n\nKeep polling and print notifications as they arrive:\n\n\x1b[1mtemporal channel poll \\\n --channel YourChannel \\\n --after-counter 10 \\\n --follow\x1b[0m\n\nEach notification prints with its counter, position and metadata. With\n\x1b[1m--output json\x1b[0m each notification is one JSON object." + } else { + s.Command.Long = "Show the notifications a channel keeps, waiting for one when there is\nnone yet:\n\n```\ntemporal channel poll \\\n --channel YourChannel\n```\n\nKeep polling and print notifications as they arrive:\n\n```\ntemporal channel poll \\\n --channel YourChannel \\\n --after-counter 10 \\\n --follow\n```\n\nEach notification prints with its counter, position and metadata. With\n`--output json` each notification is one JSON object." + } + s.Command.Args = cobra.NoArgs + s.Command.Flags().IntVar(&s.AfterCounter, "after-counter", 0, "Show only notifications with a counter above this one.") + s.Wait = cliext.MustParseFlagDuration("30s") + s.Command.Flags().Var(&s.Wait, "wait", "How long one poll waits for a notification when there is none. The Service may answer sooner.") + s.Command.Flags().IntVar(&s.Max, "max", 0, "Most notifications one poll returns. Default is zero (the Service's limit).") + s.Command.Flags().BoolVarP(&s.Follow, "follow", "f", false, "Keep polling from the last counter seen until interrupted.") + s.ChannelOptions.BuildFlags(s.Command.Flags()) + s.Command.Run = func(c *cobra.Command, args []string) { + if err := s.run(cctx, args); err != nil { + cctx.Options.Fail(err) + } + } + return &s +} + type TemporalConfigCommand struct { Parent *TemporalCommand Command cobra.Command @@ -2848,9 +3103,9 @@ func NewTemporalServerStartDevCommand(cctx *CommandContext, parent *TemporalServ s.Command.Use = "start-dev [flags]" s.Command.Short = "Start Temporal development server" if hasHighlighting { - s.Command.Long = "Run a development Temporal Server on your local system.\n\n\x1b[1m+------------------------------------------------------------------------+\n| WARNING: The development server is not intended for production use. |\n| It skips certain HTTP security checks to make local use simpler. |\n| |\n| For production use, see: |\n| https://docs.temporal.io/production-deployment |\n+------------------------------------------------------------------------+\x1b[0m\n\nView the Web UI for the default configuration at: http://localhost:8233\n\n\x1b[1mtemporal server start-dev\x1b[0m\n\nAdd persistence for Workflow Executions across runs:\n\n\x1b[1mtemporal server start-dev \\\n --db-filename path-to-your-local-persistent-store\x1b[0m\n\nSet the port from the front-end gRPC Service (7233 default):\n\n\x1b[1mtemporal server start-dev \\\n --port 7000\x1b[0m\n\nUse a custom port for the Web UI. The default is the gRPC port (7233 default)\nplus 1000 (8233):\n\n\x1b[1mtemporal server start-dev \\\n --ui-port 3000\x1b[0m" + s.Command.Long = "Run a development Temporal Server on your local system.\n\n\x1b[1m+------------------------------------------------------------------------+\n| WARNING: The development server is not intended for production use. |\n| It skips certain HTTP security checks to make local use simpler. |\n| |\n| For production use, see: |\n| https://docs.temporal.io/production-deployment |\n+------------------------------------------------------------------------+\x1b[0m\n\nView the Web UI for the default configuration at: http://localhost:8233\n\n\x1b[1mtemporal server start-dev\x1b[0m\n\nAdd persistence for Workflow Executions across runs:\n\n\x1b[1mtemporal server start-dev \\\n --db-filename path-to-your-local-persistent-store\x1b[0m\n\nSet the port from the front-end gRPC Service (7233 default):\n\n\x1b[1mtemporal server start-dev \\\n --port 7000\x1b[0m\n\nUse a custom port for the Web UI. The default is the gRPC port (7233 default)\nplus 1000 (8233):\n\n\x1b[1mtemporal server start-dev \\\n --ui-port 3000\x1b[0m\n\nStreams are enabled on the development server. Turn them off with a\ndynamic configuration value:\n\n\x1b[1mtemporal server start-dev \\\n --dynamic-config-value 'stream.enabled=false'\x1b[0m\n\nCallbacks to \x1b[1m127.0.0.1\x1b[0m and \x1b[1mlocalhost\x1b[0m on any port are allowed over\nplain HTTP, so a local receiver can listen on a notification channel.\nReplace the list with a dynamic configuration value:\n\n\x1b[1mtemporal server start-dev \\\n --dynamic-config-value 'callback.allowedAddresses=[]'\x1b[0m" } else { - s.Command.Long = "Run a development Temporal Server on your local system.\n\n```\n+------------------------------------------------------------------------+\n| WARNING: The development server is not intended for production use. |\n| It skips certain HTTP security checks to make local use simpler. |\n| |\n| For production use, see: |\n| https://docs.temporal.io/production-deployment |\n+------------------------------------------------------------------------+\n```\n\nView the Web UI for the default configuration at: http://localhost:8233\n\n```\ntemporal server start-dev\n```\n\nAdd persistence for Workflow Executions across runs:\n\n```\ntemporal server start-dev \\\n --db-filename path-to-your-local-persistent-store\n```\n\nSet the port from the front-end gRPC Service (7233 default):\n\n```\ntemporal server start-dev \\\n --port 7000\n```\n\nUse a custom port for the Web UI. The default is the gRPC port (7233 default)\nplus 1000 (8233):\n\n```\ntemporal server start-dev \\\n --ui-port 3000\n```" + s.Command.Long = "Run a development Temporal Server on your local system.\n\n```\n+------------------------------------------------------------------------+\n| WARNING: The development server is not intended for production use. |\n| It skips certain HTTP security checks to make local use simpler. |\n| |\n| For production use, see: |\n| https://docs.temporal.io/production-deployment |\n+------------------------------------------------------------------------+\n```\n\nView the Web UI for the default configuration at: http://localhost:8233\n\n```\ntemporal server start-dev\n```\n\nAdd persistence for Workflow Executions across runs:\n\n```\ntemporal server start-dev \\\n --db-filename path-to-your-local-persistent-store\n```\n\nSet the port from the front-end gRPC Service (7233 default):\n\n```\ntemporal server start-dev \\\n --port 7000\n```\n\nUse a custom port for the Web UI. The default is the gRPC port (7233 default)\nplus 1000 (8233):\n\n```\ntemporal server start-dev \\\n --ui-port 3000\n```\n\nStreams are enabled on the development server. Turn them off with a\ndynamic configuration value:\n\n```\ntemporal server start-dev \\\n --dynamic-config-value 'stream.enabled=false'\n```\n\nCallbacks to `127.0.0.1` and `localhost` on any port are allowed over\nplain HTTP, so a local receiver can listen on a notification channel.\nReplace the list with a dynamic configuration value:\n\n```\ntemporal server start-dev \\\n --dynamic-config-value 'callback.allowedAddresses=[]'\n```" } s.Command.Args = cobra.NoArgs s.Command.Flags().StringVarP(&s.DbFilename, "db-filename", "f", "", "Path to file for persistent Temporal state store. By default, Workflow Executions are lost when the server process dies.") @@ -2878,6 +3133,298 @@ func NewTemporalServerStartDevCommand(cctx *CommandContext, parent *TemporalServ return &s } +type TemporalStreamCommand struct { + Parent *TemporalCommand + Command cobra.Command + cliext.ClientOptions +} + +func NewTemporalStreamCommand(cctx *CommandContext, parent *TemporalCommand) *TemporalStreamCommand { + var s TemporalStreamCommand + s.Parent = parent + s.Command.Use = "stream" + s.Command.Short = "Read and manage Streams" + if hasHighlighting { + s.Command.Long = "A Stream is an append-only log the Temporal Service carries next to\nWorkflow Executions. A standalone Stream has an ID of its own. An owned\nStream lives inside a Workflow Execution or an Activity and is addressed\nby its owner and a Stream name:\n\n\x1b[1mtemporal stream [command] [options]\x1b[0m\n\nFor example:\n\n\x1b[1mtemporal stream read \\\n --workflow-id YourWorkflowId\x1b[0m\n\nThe Service must enable Streams for the Namespace through the\n\x1b[1mstream.enabled\x1b[0m dynamic configuration value. The development server\nenables them." + } else { + s.Command.Long = "A Stream is an append-only log the Temporal Service carries next to\nWorkflow Executions. A standalone Stream has an ID of its own. An owned\nStream lives inside a Workflow Execution or an Activity and is addressed\nby its owner and a Stream name:\n\n```\ntemporal stream [command] [options]\n```\n\nFor example:\n\n```\ntemporal stream read \\\n --workflow-id YourWorkflowId\n```\n\nThe Service must enable Streams for the Namespace through the\n`stream.enabled` dynamic configuration value. The development server\nenables them." + } + s.Command.Args = cobra.NoArgs + s.Command.AddCommand(&NewTemporalStreamAppendCommand(cctx, &s).Command) + s.Command.AddCommand(&NewTemporalStreamCloseCommand(cctx, &s).Command) + s.Command.AddCommand(&NewTemporalStreamCreateCommand(cctx, &s).Command) + s.Command.AddCommand(&NewTemporalStreamDeleteCommand(cctx, &s).Command) + s.Command.AddCommand(&NewTemporalStreamDescribeCommand(cctx, &s).Command) + s.Command.AddCommand(&NewTemporalStreamListCommand(cctx, &s).Command) + s.Command.AddCommand(&NewTemporalStreamReadCommand(cctx, &s).Command) + s.Command.AddCommand(&NewTemporalStreamTruncateCommand(cctx, &s).Command) + s.ClientOptions.BuildFlags(s.Command.PersistentFlags()) + s.ClientOptions.HideFlags() + return &s +} + +type TemporalStreamAppendCommand struct { + Parent *TemporalStreamCommand + Command cobra.Command + StreamReferenceOptions + PayloadInputOptions + Topic string + ProducerId string + Attempt int + Sequence int + ExpectedOffset int + Finish bool +} + +func NewTemporalStreamAppendCommand(cctx *CommandContext, parent *TemporalStreamCommand) *TemporalStreamAppendCommand { + var s TemporalStreamAppendCommand + s.Parent = parent + s.Command.DisableFlagsInUseLine = true + s.Command.Use = "append [flags]" + s.Command.Short = "Append records to a Stream" + if hasHighlighting { + s.Command.Long = "Append one or more records to a Stream. Each \x1b[1m--input\x1b[0m is one record.\nWithout \x1b[1m--input\x1b[0m or \x1b[1m--input-file\x1b[0m, one record is read per line from\nstdin:\n\n\x1b[1mtemporal stream append \\\n --stream-id YourStreamId \\\n --input '{\"score\": 1}' \\\n --input '{\"score\": 2}'\x1b[0m\n\nAppend to a Stream a Workflow Execution owns, naming the owner instead:\n\n\x1b[1mtemporal stream append \\\n --workflow-id YourWorkflowId \\\n --name scores \\\n --input '{\"score\": 1}'\x1b[0m\n\nSet \x1b[1m--producer-id\x1b[0m and \x1b[1m--sequence\x1b[0m so a retried append is deduplicated\nrather than appended twice. Pass \x1b[1m--finish\x1b[0m to end the producer's writes\non the topic with a \x1b[1mFINISH\x1b[0m record after the values." + } else { + s.Command.Long = "Append one or more records to a Stream. Each `--input` is one record.\nWithout `--input` or `--input-file`, one record is read per line from\nstdin:\n\n```\ntemporal stream append \\\n --stream-id YourStreamId \\\n --input '{\"score\": 1}' \\\n --input '{\"score\": 2}'\n```\n\nAppend to a Stream a Workflow Execution owns, naming the owner instead:\n\n```\ntemporal stream append \\\n --workflow-id YourWorkflowId \\\n --name scores \\\n --input '{\"score\": 1}'\n```\n\nSet `--producer-id` and `--sequence` so a retried append is deduplicated\nrather than appended twice. Pass `--finish` to end the producer's writes\non the topic with a `FINISH` record after the values." + } + s.Command.Args = cobra.NoArgs + s.Command.Flags().StringVar(&s.Topic, "topic", "", "Topic label stored on every record.") + s.Command.Flags().StringVar(&s.ProducerId, "producer-id", "", "Producer identity stored on every record. Together with --sequence it deduplicates a retried append.") + s.Command.Flags().IntVar(&s.Attempt, "attempt", 0, "Producer attempt stored on every record.") + s.Command.Flags().IntVar(&s.Sequence, "sequence", 0, "Sequence of the first record within the producer's attempt. Records that follow take the next numbers.") + s.Command.Flags().IntVar(&s.ExpectedOffset, "expected-offset", 0, "Refuse the append unless the Stream's next offset equals this value. Only applies to a standalone Stream.") + s.Command.Flags().BoolVar(&s.Finish, "finish", false, "Append a `FINISH` record after the values.") + s.StreamReferenceOptions.BuildFlags(s.Command.Flags()) + s.PayloadInputOptions.BuildFlags(s.Command.Flags()) + s.Command.Run = func(c *cobra.Command, args []string) { + if err := s.run(cctx, args); err != nil { + cctx.Options.Fail(err) + } + } + return &s +} + +type TemporalStreamCloseCommand struct { + Parent *TemporalStreamCommand + Command cobra.Command + StreamIdOptions + Reason string +} + +func NewTemporalStreamCloseCommand(cctx *CommandContext, parent *TemporalStreamCommand) *TemporalStreamCloseCommand { + var s TemporalStreamCloseCommand + s.Parent = parent + s.Command.DisableFlagsInUseLine = true + s.Command.Use = "close [flags]" + s.Command.Short = "Close a Stream" + if hasHighlighting { + s.Command.Long = "Close a standalone Stream. A closed Stream accepts no more records and\nstays readable until its retention passes:\n\n\x1b[1mtemporal stream close \\\n --stream-id YourStreamId \\\n --reason \"done\"\x1b[0m" + } else { + s.Command.Long = "Close a standalone Stream. A closed Stream accepts no more records and\nstays readable until its retention passes:\n\n```\ntemporal stream close \\\n --stream-id YourStreamId \\\n --reason \"done\"\n```" + } + s.Command.Args = cobra.NoArgs + s.Command.Flags().StringVar(&s.Reason, "reason", "", "Reason recorded on the Stream.") + s.StreamIdOptions.BuildFlags(s.Command.Flags()) + s.Command.Run = func(c *cobra.Command, args []string) { + if err := s.run(cctx, args); err != nil { + cctx.Options.Fail(err) + } + } + return &s +} + +type TemporalStreamCreateCommand struct { + Parent *TemporalStreamCommand + Command cobra.Command + StreamIdOptions + Retention cliext.FlagDuration + MaxItems int + MaxBytes int +} + +func NewTemporalStreamCreateCommand(cctx *CommandContext, parent *TemporalStreamCommand) *TemporalStreamCreateCommand { + var s TemporalStreamCreateCommand + s.Parent = parent + s.Command.DisableFlagsInUseLine = true + s.Command.Use = "create [flags]" + s.Command.Short = "Create a standalone Stream" + if hasHighlighting { + s.Command.Long = "Create a standalone Stream with an ID of its own:\n\n\x1b[1mtemporal stream create \\\n --stream-id YourStreamId\x1b[0m\n\nRecords older than \x1b[1m--retention\x1b[0m are reclaimed, and a closed Stream is\ndeleted once it passes. Without it the Namespace's retention applies.\nCreating a Stream that already exists with the same lifecycle is an\nidempotent retry; a different lifecycle is refused." + } else { + s.Command.Long = "Create a standalone Stream with an ID of its own:\n\n```\ntemporal stream create \\\n --stream-id YourStreamId\n```\n\nRecords older than `--retention` are reclaimed, and a closed Stream is\ndeleted once it passes. Without it the Namespace's retention applies.\nCreating a Stream that already exists with the same lifecycle is an\nidempotent retry; a different lifecycle is refused." + } + s.Command.Args = cobra.NoArgs + s.Retention = 0 + s.Command.Flags().Var(&s.Retention, "retention", "How long a record stays readable, and how long a closed Stream stays readable before it is deleted. Cannot be longer than the Namespace's retention.") + s.Command.Flags().IntVar(&s.MaxItems, "max-items", 0, "Most records the Stream holds before the oldest are reclaimed. Default is zero (unbounded).") + s.Command.Flags().IntVar(&s.MaxBytes, "max-bytes", 0, "Most bytes the Stream holds. An append past it is refused until records are reclaimed. Default is zero (unbounded).") + s.StreamIdOptions.BuildFlags(s.Command.Flags()) + s.Command.Run = func(c *cobra.Command, args []string) { + if err := s.run(cctx, args); err != nil { + cctx.Options.Fail(err) + } + } + return &s +} + +type TemporalStreamDeleteCommand struct { + Parent *TemporalStreamCommand + Command cobra.Command + StreamIdOptions + Force bool +} + +func NewTemporalStreamDeleteCommand(cctx *CommandContext, parent *TemporalStreamCommand) *TemporalStreamDeleteCommand { + var s TemporalStreamDeleteCommand + s.Parent = parent + s.Command.DisableFlagsInUseLine = true + s.Command.Use = "delete [flags]" + s.Command.Short = "Remove a Stream" + if hasHighlighting { + s.Command.Long = "Delete a standalone Stream and every record it holds:\n\n\x1b[1mtemporal stream delete \\\n --stream-id YourStreamId\x1b[0m\n\nThe call is refused while a Workflow Execution consumes the Stream,\nbecause its replay depends on the records. Pass \x1b[1m--force\x1b[0m to delete it\nanyway." + } else { + s.Command.Long = "Delete a standalone Stream and every record it holds:\n\n```\ntemporal stream delete \\\n --stream-id YourStreamId\n```\n\nThe call is refused while a Workflow Execution consumes the Stream,\nbecause its replay depends on the records. Pass `--force` to delete it\nanyway." + } + s.Command.Args = cobra.NoArgs + s.Command.Flags().BoolVar(&s.Force, "force", false, "Delete even while a Workflow Execution consumes the Stream.") + s.StreamIdOptions.BuildFlags(s.Command.Flags()) + s.Command.Run = func(c *cobra.Command, args []string) { + if err := s.run(cctx, args); err != nil { + cctx.Options.Fail(err) + } + } + return &s +} + +type TemporalStreamDescribeCommand struct { + Parent *TemporalStreamCommand + Command cobra.Command + StreamReferenceOptions +} + +func NewTemporalStreamDescribeCommand(cctx *CommandContext, parent *TemporalStreamCommand) *TemporalStreamDescribeCommand { + var s TemporalStreamDescribeCommand + s.Parent = parent + s.Command.DisableFlagsInUseLine = true + s.Command.Use = "describe [flags]" + s.Command.Short = "Show a Stream's state" + if hasHighlighting { + s.Command.Long = "Show a Stream's frontier, floor, close state, retention, producers and\nconsumers:\n\n\x1b[1mtemporal stream describe \\\n --stream-id YourStreamId\x1b[0m\n\nDescribe a Stream a Workflow Execution owns:\n\n\x1b[1mtemporal stream describe \\\n --workflow-id YourWorkflowId \\\n --name scores\x1b[0m\n\nThe output also names the notification channel the Stream notifies on\neach append and on its close. The name follows from the Stream alone,\nso no call to the Service is needed. A Stream a Workflow Execution\nowns notifies \x1b[1mstream/NAME\x1b[0m, linked to the owner. A Stream of an\nActivity the Workflow scheduled notifies \x1b[1mstream/ACTIVITY_ID/NAME\x1b[0m,\nlinked to the owner too. A standalone Stream notifies the independent\nchannel \x1b[1mstream/STREAM_ID\x1b[0m. Pass the channel, and the owner of a\nlinked one, to \x1b[1mtemporal channel poll\x1b[0m to follow the Stream:\n\n\x1b[1mtemporal channel poll \\\n --channel stream/scores \\\n --workflow-id YourWorkflowId\x1b[0m" + } else { + s.Command.Long = "Show a Stream's frontier, floor, close state, retention, producers and\nconsumers:\n\n```\ntemporal stream describe \\\n --stream-id YourStreamId\n```\n\nDescribe a Stream a Workflow Execution owns:\n\n```\ntemporal stream describe \\\n --workflow-id YourWorkflowId \\\n --name scores\n```\n\nThe output also names the notification channel the Stream notifies on\neach append and on its close. The name follows from the Stream alone,\nso no call to the Service is needed. A Stream a Workflow Execution\nowns notifies `stream/NAME`, linked to the owner. A Stream of an\nActivity the Workflow scheduled notifies `stream/ACTIVITY_ID/NAME`,\nlinked to the owner too. A standalone Stream notifies the independent\nchannel `stream/STREAM_ID`. Pass the channel, and the owner of a\nlinked one, to `temporal channel poll` to follow the Stream:\n\n```\ntemporal channel poll \\\n --channel stream/scores \\\n --workflow-id YourWorkflowId\n```" + } + s.Command.Args = cobra.NoArgs + s.StreamReferenceOptions.BuildFlags(s.Command.Flags()) + s.Command.Run = func(c *cobra.Command, args []string) { + if err := s.run(cctx, args); err != nil { + cctx.Options.Fail(err) + } + } + return &s +} + +type TemporalStreamListCommand struct { + Parent *TemporalStreamCommand + Command cobra.Command + Query string + Limit int + PageSize int +} + +func NewTemporalStreamListCommand(cctx *CommandContext, parent *TemporalStreamCommand) *TemporalStreamListCommand { + var s TemporalStreamListCommand + s.Parent = parent + s.Command.DisableFlagsInUseLine = true + s.Command.Use = "list [flags]" + s.Command.Short = "Show standalone Streams" + if hasHighlighting { + s.Command.Long = "List the standalone Streams in a Namespace. The optional \x1b[1m--query\x1b[0m\nlimits the output to Streams matching a List Filter, where \x1b[1mWorkflowId\x1b[0m\nholds the Stream ID:\n\n\x1b[1mtemporal stream list \\\n --query 'WorkflowId STARTS_WITH \"scores-\"'\x1b[0m\n\nStreams a Workflow Execution owns are not listed here. Describe or read\nthem through their owner." + } else { + s.Command.Long = "List the standalone Streams in a Namespace. The optional `--query`\nlimits the output to Streams matching a List Filter, where `WorkflowId`\nholds the Stream ID:\n\n```\ntemporal stream list \\\n --query 'WorkflowId STARTS_WITH \"scores-\"'\n```\n\nStreams a Workflow Execution owns are not listed here. Describe or read\nthem through their owner." + } + s.Command.Args = cobra.NoArgs + s.Command.Flags().StringVarP(&s.Query, "query", "q", "", "Content for an SQL-like `QUERY` List Filter.") + s.Command.Flags().IntVar(&s.Limit, "limit", 0, "Maximum number of Streams to display.") + s.Command.Flags().IntVar(&s.PageSize, "page-size", 100, "Maximum number of Streams to fetch at a time from the server.") + s.Command.Run = func(c *cobra.Command, args []string) { + if err := s.run(cctx, args); err != nil { + cctx.Options.Fail(err) + } + } + return &s +} + +type TemporalStreamReadCommand struct { + Parent *TemporalStreamCommand + Command cobra.Command + StreamReferenceOptions + FromOffset int + FromTail bool + Last int + Follow bool + Topic []string + Limit int + PageSize int +} + +func NewTemporalStreamReadCommand(cctx *CommandContext, parent *TemporalStreamCommand) *TemporalStreamReadCommand { + var s TemporalStreamReadCommand + s.Parent = parent + s.Command.DisableFlagsInUseLine = true + s.Command.Use = "read [flags]" + s.Command.Short = "Read a Stream's records" + if hasHighlighting { + s.Command.Long = "Read a Stream from the beginning and stop once caught up:\n\n\x1b[1mtemporal stream read \\\n --stream-id YourStreamId\x1b[0m\n\nRead a Stream a Workflow Execution owns and keep following it as records\nland:\n\n\x1b[1mtemporal stream read \\\n --workflow-id YourWorkflowId \\\n --name scores \\\n --follow\x1b[0m\n\nStart at a cursor with \x1b[1m--from-offset\x1b[0m, at the tail with \x1b[1m--from-tail\x1b[0m,\nor at the newest records with \x1b[1m--last\x1b[0m. Each record prints with its\noffset, kind, topic, producer, attempt, sequence and body. With\n\x1b[1m--output json\x1b[0m each record is one JSON object." + } else { + s.Command.Long = "Read a Stream from the beginning and stop once caught up:\n\n```\ntemporal stream read \\\n --stream-id YourStreamId\n```\n\nRead a Stream a Workflow Execution owns and keep following it as records\nland:\n\n```\ntemporal stream read \\\n --workflow-id YourWorkflowId \\\n --name scores \\\n --follow\n```\n\nStart at a cursor with `--from-offset`, at the tail with `--from-tail`,\nor at the newest records with `--last`. Each record prints with its\noffset, kind, topic, producer, attempt, sequence and body. With\n`--output json` each record is one JSON object." + } + s.Command.Args = cobra.NoArgs + s.Command.Flags().IntVar(&s.FromOffset, "from-offset", 0, "Offset to start at, inclusive. Refused when below the Stream's floor.") + s.Command.Flags().BoolVar(&s.FromTail, "from-tail", false, "Start at the tail and show only records appended from now on.") + s.Command.Flags().IntVar(&s.Last, "last", 0, "Start at the newest records, showing at most this many first.") + s.Command.Flags().BoolVarP(&s.Follow, "follow", "f", false, "Keep reading as records land, until the Stream closes.") + s.Command.Flags().StringArrayVar(&s.Topic, "topic", nil, "Show only records on this topic. Can be passed multiple times.") + s.Command.Flags().IntVar(&s.Limit, "limit", 0, "Maximum number of records to display.") + s.Command.Flags().IntVar(&s.PageSize, "page-size", 100, "Maximum number of records to fetch at a time from the server.") + s.StreamReferenceOptions.BuildFlags(s.Command.Flags()) + s.Command.Run = func(c *cobra.Command, args []string) { + if err := s.run(cctx, args); err != nil { + cctx.Options.Fail(err) + } + } + return &s +} + +type TemporalStreamTruncateCommand struct { + Parent *TemporalStreamCommand + Command cobra.Command + StreamIdOptions + To int +} + +func NewTemporalStreamTruncateCommand(cctx *CommandContext, parent *TemporalStreamCommand) *TemporalStreamTruncateCommand { + var s TemporalStreamTruncateCommand + s.Parent = parent + s.Command.DisableFlagsInUseLine = true + s.Command.Use = "truncate [flags]" + s.Command.Short = "Drop the oldest records of a Stream" + if hasHighlighting { + s.Command.Long = "Move the readable floor of a standalone Stream up to an offset. Records\nbelow it are gone:\n\n\x1b[1mtemporal stream truncate \\\n --stream-id YourStreamId \\\n --to 100\x1b[0m\n\nThe call is refused while a Workflow Execution consumes records below\nthe new floor." + } else { + s.Command.Long = "Move the readable floor of a standalone Stream up to an offset. Records\nbelow it are gone:\n\n```\ntemporal stream truncate \\\n --stream-id YourStreamId \\\n --to 100\n```\n\nThe call is refused while a Workflow Execution consumes records below\nthe new floor." + } + s.Command.Args = cobra.NoArgs + s.Command.Flags().IntVar(&s.To, "to", 0, "New floor. Records below this offset are dropped. Required.") + _ = cobra.MarkFlagRequired(s.Command.Flags(), "to") + s.StreamIdOptions.BuildFlags(s.Command.Flags()) + s.Command.Run = func(c *cobra.Command, args []string) { + if err := s.run(cctx, args); err != nil { + cctx.Options.Fail(err) + } + } + return &s +} + type TemporalTaskQueueCommand struct { Parent *TemporalCommand Command cobra.Command diff --git a/internal/temporalcli/commands.stream.go b/internal/temporalcli/commands.stream.go new file mode 100644 index 000000000..fb75ec3ce --- /dev/null +++ b/internal/temporalcli/commands.stream.go @@ -0,0 +1,1053 @@ +package temporalcli + +import ( + "bufio" + "context" + "errors" + "fmt" + "os" + "reflect" + "sort" + "strings" + "time" + + "github.com/fatih/color" + "github.com/mattn/go-isatty" + "github.com/temporalio/cli/cliext" + "github.com/temporalio/cli/internal/printer" + commonpb "go.temporal.io/api/common/v1" + enumspb "go.temporal.io/api/enums/v1" + notificationpb "go.temporal.io/api/notification/v1" + "go.temporal.io/api/serviceerror" + streamapi "go.temporal.io/api/stream/v1" + "go.temporal.io/api/temporalproto" + "go.temporal.io/sdk/converter" + streamlib "go.temporal.io/server/chasm/lib/stream" + streampb "go.temporal.io/server/chasm/lib/stream/gen/streampb/v1" + "google.golang.org/protobuf/types/known/durationpb" +) + +// streamRefusal is a server refusal whose reason token has been read, so the +// message says what to do instead of carrying the token. +type streamRefusal struct { + plain string + detail string + cause error +} + +func (e *streamRefusal) Error() string { + if e.detail == "" { + return e.plain + } + return e.plain + " The server said: " + e.detail +} + +func (e *streamRefusal) Unwrap() error { return e.cause } + +// plainRefusal maps a refusal's reason token to plain language. Every other +// error comes back as it was. +func plainRefusal(err error) error { + var refused *serviceerror.FailedPrecondition + if !errors.As(err, &refused) { + return err + } + reason := streamlib.ReasonOf(refused.Message) + if reason == "" { + return err + } + _, detail, _ := strings.Cut(refused.Message, ": ") + var plain string + switch reason { + case streamlib.ReasonProducerConflict: + plain = "the producer already appended different content at this sequence, so " + + "nothing was written. Use the next sequence, or check that two producers do not " + + "share the id." + case streamlib.ReasonProducerStaleSequence: + plain = "the sequence is below the producer's latest, so nothing was written. A " + + "producer carries one append at a time; retry with its latest sequence or move on " + + "to the next." + case streamlib.ReasonCursorBelowFloor: + plain = "the read starts below the stream's floor, and those records are gone. Start " + + "from the beginning, or at the offset the stream now starts at." + case streamlib.ReasonStreamClosed: + plain = "the stream is closed. Its records stay readable, but nothing more can be " + + "appended." + case streamlib.ReasonPolicyMismatch: + plain = "a stream with this id already exists with a different lifecycle. Use a new " + + "id, or repeat the existing retention, max items and max bytes." + } + return &streamRefusal{plain: plain, detail: detail, cause: err} +} + +// streamTarget is the stream a command names: a standalone stream by its id, +// or a stream an execution owns by that owner and a name. +type streamTarget struct { + streamID string + owner *streampb.StreamOwner + name string +} + +// resolvedName is the name the server resolves an owned stream to when the +// command leaves it out. +func (t streamTarget) resolvedName() string { + if t.name == "" { + return streamlib.DefaultStreamName + } + return t.name +} + +func (t streamTarget) String() string { + if t.streamID != "" { + return fmt.Sprintf("stream %q", t.streamID) + } + name := t.resolvedName() + switch t.owner.GetKind() { + case streampb.STREAM_OWNER_KIND_WORKFLOW_ACTIVITY: + return fmt.Sprintf("stream %q of activity %q in workflow %q", + name, t.owner.GetActivityId(), t.owner.GetId()) + case streampb.STREAM_OWNER_KIND_ACTIVITY: + return fmt.Sprintf("stream %q of activity %q", name, t.owner.GetId()) + default: + return fmt.Sprintf("stream %q of workflow %q", name, t.owner.GetId()) + } +} + +// streamChannelPrefix starts the name of every notification channel a stream +// notifies. +const streamChannelPrefix = "stream/" + +// streamChannel is the notification channel a stream notifies on each append +// and on its close. The server derives the name from the stream's identity +// alone, so the CLI derives the same name without a call and a listener can +// follow a stream before it exists. +type streamChannel struct { + name string + kind notificationpb.ChannelKind + owner *commonpb.Execution +} + +func (t streamTarget) channel() streamChannel { + if t.streamID != "" { + return streamChannel{ + name: streamChannelPrefix + t.streamID, + kind: notificationpb.CHANNEL_KIND_INDEPENDENT, + } + } + owner := &commonpb.Execution{ + Type: enumspb.EXECUTION_TYPE_WORKFLOW, + BusinessId: t.owner.GetId(), + RunId: t.owner.GetRunId(), + } + switch t.owner.GetKind() { + case streampb.STREAM_OWNER_KIND_WORKFLOW_ACTIVITY: + // The activity's stream notifies the workflow that scheduled it, so the + // activity ID is in the name and the workflow is the owner. + return streamChannel{ + name: streamChannelPrefix + t.owner.GetActivityId() + "/" + t.resolvedName(), + kind: notificationpb.CHANNEL_KIND_LINKED, + owner: owner, + } + case streampb.STREAM_OWNER_KIND_ACTIVITY: + // A standalone activity is an execution that holds linked channels of + // its own, since the server resolves the channel's owner from the + // execution a request names. Its stream's channel then needs no + // activity ID in the name. + owner.Type = enumspb.EXECUTION_TYPE_ACTIVITY + return streamChannel{ + name: streamChannelPrefix + t.resolvedName(), + kind: notificationpb.CHANNEL_KIND_LINKED, + owner: owner, + } + default: + return streamChannel{ + name: streamChannelPrefix + t.resolvedName(), + kind: notificationpb.CHANNEL_KIND_LINKED, + owner: owner, + } + } +} + +func (o *StreamReferenceOptions) target() (streamTarget, error) { + hasOwner := o.WorkflowId != "" || o.ActivityId != "" + switch { + case o.StreamId == "" && !hasOwner: + return streamTarget{}, fmt.Errorf( + "must set either --stream-id or an owner with --workflow-id or --activity-id") + case o.StreamId != "" && hasOwner: + return streamTarget{}, fmt.Errorf("cannot set --stream-id with --workflow-id or --activity-id") + case o.StreamId != "" && (o.RunId != "" || o.Name != ""): + return streamTarget{}, fmt.Errorf("--run-id and --name name an owner's stream, not --stream-id") + case o.StreamId != "": + return streamTarget{streamID: o.StreamId}, nil + } + owner := &streampb.StreamOwner{RunId: o.RunId} + switch { + case o.WorkflowId != "" && o.ActivityId != "": + owner.Kind = streampb.STREAM_OWNER_KIND_WORKFLOW_ACTIVITY + owner.Id, owner.ActivityId = o.WorkflowId, o.ActivityId + case o.WorkflowId != "": + owner.Kind = streampb.STREAM_OWNER_KIND_WORKFLOW + owner.Id = o.WorkflowId + default: + owner.Kind = streampb.STREAM_OWNER_KIND_ACTIVITY + owner.Id = o.ActivityId + } + return streamTarget{owner: owner, name: o.Name}, nil +} + +// streamClient reaches the stream service over the connection the SDK dialed, +// so TLS, credentials and headers are the ones every other command uses. +type streamClient struct { + *dialedClient + svc streampb.StreamServiceClient + namespace string +} + +func dialStreamClient(cctx *CommandContext, c *cliext.ClientOptions) (*streamClient, error) { + cl, err := dialClientWithConn(cctx, c) + if err != nil { + return nil, err + } + return &streamClient{ + dialedClient: cl, + svc: streampb.NewStreamServiceClient(cl.Conn), + namespace: c.Namespace, + }, nil +} + +// applyCodec runs every payload the records carry through one codec call, so a +// page costs one round trip to the codec server rather than one per record. +func (s *streamClient) applyCodec(records []*streampb.StreamRecord, encode bool) error { + if s.PayloadCodec == nil { + return nil + } + fn := s.PayloadCodec.Decode + if encode { + fn = s.PayloadCodec.Encode + } + var payloads []*commonpb.Payload + var put []func(*commonpb.Payload) + for _, r := range records { + if r.Body != nil { + payloads = append(payloads, r.Body) + put = append(put, func(p *commonpb.Payload) { r.Body = p }) + } + for k, v := range r.Metadata { + if v == nil { + continue + } + payloads = append(payloads, v) + put = append(put, func(p *commonpb.Payload) { r.Metadata[k] = p }) + } + } + if len(payloads) == 0 { + return nil + } + out, err := fn(payloads) + if err != nil { + return err + } + if len(out) != len(payloads) { + return fmt.Errorf("codec returned %d payloads for %d", len(out), len(payloads)) + } + for i, p := range out { + put[i](p) + } + return nil +} + +func (s *streamClient) decodeRecords(records []*streampb.StreamRecord) error { + return s.applyCodec(records, false) +} + +func (s *streamClient) encodeRecords(records []*streampb.StreamRecord) error { + return s.applyCodec(records, true) +} + +func (s *streamClient) describe( + ctx context.Context, t streamTarget, +) (*streampb.StreamState, error) { + if t.streamID != "" { + resp, err := s.svc.DescribeStream(ctx, &streampb.DescribeStreamRequest{ + FrontendRequest: &streampb.DescribeStreamInput{ + Namespace: s.namespace, + StreamId: t.streamID, + }, + }) + return resp.GetFrontendResponse().GetState(), err + } + resp, err := s.svc.DescribeWorkflowStream(ctx, &streampb.DescribeWorkflowStreamRequest{ + FrontendRequest: &streampb.DescribeWorkflowStreamInput{ + Namespace: s.namespace, + Owner: t.owner, + StreamName: t.name, + }, + }) + return resp.GetFrontendResponse().GetState(), err +} + +func (s *streamClient) poll( + ctx context.Context, + t streamTarget, + from int64, + start *streamapi.StreamStartPosition, + pageSize int32, + topics []string, + wait bool, +) (*streampb.PollMessagesOutput, error) { + if t.streamID != "" { + resp, err := s.svc.PollMessages(ctx, &streampb.PollMessagesRequest{ + FrontendRequest: &streampb.PollMessagesInput{ + Namespace: s.namespace, + StreamId: t.streamID, + FromOffset: from, + StartPosition: start, + MaxMessages: pageSize, + Topics: topics, + WaitNewMessages: wait, + }, + }) + return resp.GetFrontendResponse(), err + } + resp, err := s.svc.PollWorkflowMessages(ctx, &streampb.PollWorkflowMessagesRequest{ + FrontendRequest: &streampb.PollWorkflowMessagesInput{ + Namespace: s.namespace, + Owner: t.owner, + StreamName: t.name, + FromOffset: from, + StartPosition: start, + MaxMessages: pageSize, + Topics: topics, + WaitNewMessages: wait, + }, + }) + return resp.GetFrontendResponse(), err +} + +func (s *streamClient) append( + ctx context.Context, + t streamTarget, + records []*streampb.StreamRecord, + producerID string, + sequence int64, + expectedOffset *int64, +) (*streampb.AddMessagesOutput, error) { + if t.streamID != "" { + in := &streampb.AddMessagesInput{ + Namespace: s.namespace, + StreamId: t.streamID, + Records: records, + ProducerId: producerID, + Sequence: sequence, + } + if expectedOffset != nil { + in.ExpectedOffset, in.UseExpectedOffset = *expectedOffset, true + } + resp, err := s.svc.AddMessages(ctx, &streampb.AddMessagesRequest{FrontendRequest: in}) + return resp.GetFrontendResponse(), err + } + resp, err := s.svc.AddWorkflowMessages(ctx, &streampb.AddWorkflowMessagesRequest{ + FrontendRequest: &streampb.AddWorkflowMessagesInput{ + Namespace: s.namespace, + Owner: t.owner, + StreamName: t.name, + Records: records, + ProducerId: producerID, + Sequence: sequence, + }, + }) + return resp.GetFrontendResponse(), err +} + +func (c *TemporalStreamCreateCommand) run(cctx *CommandContext, _ []string) error { + if c.MaxItems < 0 || c.MaxBytes < 0 { + return fmt.Errorf("--max-items and --max-bytes cannot be negative") + } + cl, err := dialStreamClient(cctx, &c.Parent.ClientOptions) + if err != nil { + return err + } + defer cl.Close() + + lifecycle := &streampb.StreamLifecycle{ + MaxItems: int64(c.MaxItems), + MaxBytes: int64(c.MaxBytes), + } + if c.Retention.Duration() > 0 { + lifecycle.Retention = durationpb.New(c.Retention.Duration()) + } + resp, err := cl.svc.CreateStream(cctx, &streampb.CreateStreamRequest{ + FrontendRequest: &streampb.CreateStreamInput{ + Namespace: cl.namespace, + StreamId: c.StreamId, + Lifecycle: lifecycle, + }, + }) + if err != nil { + return fmt.Errorf("failed creating stream %q: %w", c.StreamId, plainRefusal(err)) + } + if cctx.JSONOutput { + return cctx.Printer.PrintStructured(resp.GetFrontendResponse(), printer.StructuredOptions{}) + } + cctx.Printer.Printlnf("Created stream %v with run id %v", + c.StreamId, resp.GetFrontendResponse().GetRunId()) + return nil +} + +func (c *TemporalStreamListCommand) run(cctx *CommandContext, _ []string) error { + cl, err := dialStreamClient(cctx, &c.Parent.ClientOptions) + if err != nil { + return err + } + defer cl.Close() + + // This is a listing command subject to json vs jsonl rules + cctx.Printer.StartList() + defer cctx.Printer.EndList() + + pageSize := c.PageSize + if c.Limit > 0 && c.Limit < pageSize { + pageSize = c.Limit + } + type row struct { + StreamId string + RunId string + } + var token []byte + var seen int + for pageIndex := 0; ; pageIndex++ { + resp, err := cl.svc.ListStreams(cctx, &streampb.ListStreamsRequest{ + FrontendRequest: &streampb.ListStreamsInput{ + Namespace: cl.namespace, + PageSize: int32(pageSize), + NextPageToken: token, + Query: c.Query, + }, + }) + if err != nil { + return fmt.Errorf("failed listing streams: %w", err) + } + page := resp.GetFrontendResponse() + var rows []row + for _, entry := range page.GetStreams() { + if c.Limit > 0 && seen >= c.Limit { + break + } + seen++ + if cctx.JSONOutput { + _ = cctx.Printer.PrintStructured(entry, printer.StructuredOptions{}) + } else { + rows = append(rows, row{StreamId: entry.GetStreamId(), RunId: entry.GetRunId()}) + } + } + // Print table, headers only on first table + if len(rows) > 0 { + _ = cctx.Printer.PrintStructured(rows, printer.StructuredOptions{ + Table: &printer.TableOptions{NoHeader: pageIndex > 0}, + }) + } + token = page.GetNextPageToken() + if len(token) == 0 || (c.Limit > 0 && seen >= c.Limit) { + return nil + } + } +} + +func (c *TemporalStreamDescribeCommand) run(cctx *CommandContext, _ []string) error { + target, err := c.target() + if err != nil { + return err + } + cl, err := dialStreamClient(cctx, &c.Parent.ClientOptions) + if err != nil { + return err + } + defer cl.Close() + + state, err := cl.describe(cctx, target) + if err != nil { + return fmt.Errorf("failed describing %v: %w", target, err) + } + if reason := state.GetCloseReason(); reason != nil && cl.PayloadCodec != nil { + decoded, err := cl.PayloadCodec.Decode([]*commonpb.Payload{reason}) + if err != nil { + return fmt.Errorf("failed decoding close reason: %w", err) + } + if len(decoded) == 1 { + state.CloseReason = decoded[0] + } + } + if cctx.JSONOutput { + return cctx.Printer.PrintStructured(state, printer.StructuredOptions{}) + } + + // The frontier, the floor and the close state are facts at zero too, so + // they stay on the card; the rest is shown when set. + info := struct { + StreamId string `cli:",cardOmitEmpty"` + WorkflowId string `cli:",cardOmitEmpty"` + ActivityId string `cli:",cardOmitEmpty"` + RunId string `cli:",cardOmitEmpty"` + Name string `cli:",cardOmitEmpty"` + HeadOffset int64 + BaseOffset int64 + Records int64 + HeldBytes int64 + AppendedBytes int64 + Closed bool + CloseTime time.Time `cli:",cardOmitEmpty"` + CloseReason *commonpb.Payload `cli:",cardOmitEmpty"` + RedirectRunId string `cli:",cardOmitEmpty"` + Retention time.Duration `cli:",cardOmitEmpty"` + MaxItems int64 `cli:",cardOmitEmpty"` + MaxBytes int64 `cli:",cardOmitEmpty"` + BudgetMaxItems int64 `cli:",cardOmitEmpty"` + BudgetMaxBytes int64 `cli:",cardOmitEmpty"` + }{ + StreamId: target.streamID, + HeadOffset: state.GetHeadOffset(), + BaseOffset: state.GetBaseOffset(), + Records: state.GetHeadOffset() - state.GetBaseOffset(), + HeldBytes: state.GetHeldBytes(), + AppendedBytes: state.GetAppendedBytes(), + Closed: state.GetClosed(), + CloseReason: state.GetCloseReason(), + RedirectRunId: state.GetRedirectRunId(), + Retention: state.GetLifecycle().GetRetention().AsDuration(), + MaxItems: state.GetLifecycle().GetMaxItems(), + MaxBytes: state.GetLifecycle().GetMaxBytes(), + BudgetMaxItems: state.GetBudget().GetMaxItems(), + BudgetMaxBytes: state.GetBudget().GetMaxBytes(), + } + if state.GetCloseTime() != nil { + info.CloseTime = state.GetCloseTime().AsTime() + } + if owner := target.owner; owner != nil { + info.RunId, info.Name = owner.GetRunId(), target.name + if owner.GetKind() == streampb.STREAM_OWNER_KIND_ACTIVITY { + info.ActivityId = owner.GetId() + } else { + info.WorkflowId, info.ActivityId = owner.GetId(), owner.GetActivityId() + } + } + cctx.Printer.Println(color.MagentaString("Stream:")) + if err := cctx.Printer.PrintStructured(info, printer.StructuredOptions{}); err != nil { + return err + } + + ch := target.channel() + cctx.Printer.Println() + cctx.Printer.Println(color.MagentaString("Notification Channel:")) + if err := cctx.Printer.PrintStructured(struct { + Channel string + Kind string + LinkedTo string `cli:",cardOmitEmpty"` + }{ + Channel: ch.name, + Kind: ch.kind.String(), + LinkedTo: executionText(ch.owner), + }, printer.StructuredOptions{}); err != nil { + return err + } + + type producerRow struct { + ProducerId string + Sequence int64 + FirstOffset int64 + Count int64 + Fenced bool + } + producers := make([]producerRow, 0, len(state.GetProducers())) + for id, p := range state.GetProducers() { + producers = append(producers, producerRow{ + ProducerId: id, + Sequence: p.GetSeq(), + FirstOffset: p.GetFirstOffset(), + Count: p.GetCount(), + Fenced: p.GetFenced(), + }) + } + sort.Slice(producers, func(i, j int) bool { + return producers[i].ProducerId < producers[j].ProducerId + }) + cctx.Printer.Println() + cctx.Printer.Println(color.MagentaString("Producers: %v", len(producers))) + if len(producers) > 0 { + if err := cctx.Printer.PrintStructured(producers, printer.StructuredOptions{ + Table: &printer.TableOptions{}, + }); err != nil { + return err + } + } + + type consumerRow struct { + ConsumerId string + WorkflowId string + RunId string + Offset int64 + ReplayFloor int64 + Active bool + External bool + } + consumers := make([]consumerRow, 0, len(state.GetConsumers())) + for id, c := range state.GetConsumers() { + consumers = append(consumers, consumerRow{ + ConsumerId: id, + WorkflowId: c.GetWorkflowId(), + RunId: c.GetRunId(), + Offset: c.GetOffset(), + ReplayFloor: c.GetReplayFloor(), + Active: c.GetActive(), + External: c.GetExternal(), + }) + } + sort.Slice(consumers, func(i, j int) bool { + return consumers[i].ConsumerId < consumers[j].ConsumerId + }) + cctx.Printer.Println() + cctx.Printer.Println(color.MagentaString("Consumers: %v", len(consumers))) + if len(consumers) > 0 { + return cctx.Printer.PrintStructured(consumers, printer.StructuredOptions{ + Table: &printer.TableOptions{}, + }) + } + return nil +} + +// startPosition settles where the first poll begins. The beginning is the +// default; an explicit offset of zero is allowed and differs from it once the +// floor has moved. +func (c *TemporalStreamReadCommand) startPosition() (*streamapi.StreamStartPosition, error) { + fromOffset := c.Command.Flags().Changed("from-offset") + set := 0 + for _, on := range []bool{fromOffset, c.FromTail, c.Last != 0} { + if on { + set++ + } + } + if set > 1 { + return nil, fmt.Errorf("set at most one of --from-offset, --from-tail and --last") + } + switch { + case fromOffset: + if c.FromOffset < 0 { + return nil, fmt.Errorf("--from-offset cannot be negative") + } + return &streamapi.StreamStartPosition{ + Position: &streamapi.StreamStartPosition_Offset{Offset: int64(c.FromOffset)}, + }, nil + case c.FromTail: + return &streamapi.StreamStartPosition{ + Position: &streamapi.StreamStartPosition_Tail{Tail: true}, + }, nil + case c.Last != 0: + if c.Last < 0 { + return nil, fmt.Errorf("--last must be positive") + } + return &streamapi.StreamStartPosition{ + Position: &streamapi.StreamStartPosition_LastN{LastN: int64(c.Last)}, + }, nil + default: + return &streamapi.StreamStartPosition{ + Position: &streamapi.StreamStartPosition_Earliest{Earliest: true}, + }, nil + } +} + +// streamReader pages through a stream. Without follow it stops once caught up +// with the frontier; with follow it long-polls until the stream closes. +type streamReader struct { + ctx context.Context + cl *streamClient + target streamTarget + start *streamapi.StreamStartPosition + from int64 + topics []string + follow bool + pageSize int32 + limit int + seen int + buf []*streampb.StreamRecord + done bool +} + +func (r *streamReader) next() (*streampb.StreamRecord, error) { + if r.limit > 0 && r.seen >= r.limit { + return nil, nil + } + for len(r.buf) == 0 && !r.done { + if err := r.poll(); err != nil { + // An interrupted command is the reader leaving, not a failure. + if r.ctx.Err() != nil { + r.done = true + break + } + return nil, fmt.Errorf("failed reading %v: %w", r.target, plainRefusal(err)) + } + } + if len(r.buf) == 0 { + return nil, nil + } + rec := r.buf[0] + r.buf = r.buf[1:] + r.seen++ + return rec, nil +} + +func (r *streamReader) poll() error { + out, err := r.cl.poll(r.ctx, r.target, r.from, r.start, r.pageSize, r.topics, r.follow) + if err != nil { + return err + } + // The first poll resolved the start; from here the cursor is an offset. + r.start = nil + r.from = out.GetNextOffset() + if err := r.cl.decodeRecords(out.GetRecords()); err != nil { + return fmt.Errorf("decoding records: %w", err) + } + r.buf = append(r.buf, out.GetRecords()...) + caughtUp := out.GetNextOffset() >= out.GetHeadOffset() + if caughtUp && (out.GetClosed() || !r.follow) { + r.done = true + } + return nil +} + +type streamRecordRow struct { + Offset int64 + Kind string + Topic string + Producer string + Attempt int64 + Sequence int64 + Body string +} + +// streamRowIter adapts the reader to the printer's streaming table. +type streamRowIter struct { + cctx *CommandContext + reader *streamReader +} + +func (i *streamRowIter) Next() (any, error) { + rec, err := i.reader.next() + if rec == nil || err != nil { + return nil, err + } + body, err := payloadText(rec.GetBody()) + if err != nil { + return nil, err + } + return streamRecordRow{ + Offset: rec.GetOffset(), + Kind: recordKindText(rec.GetKind()), + Topic: rec.GetTopic(), + Producer: rec.GetProducerId(), + Attempt: rec.GetAttempt(), + Sequence: rec.GetSequence(), + Body: body, + }, nil +} + +// recordKindText is the kind in the shorthand the API's enums print with, the +// way every other enum in a table reads. +func recordKindText(kind streamapi.StreamRecordKind) string { + if kind == streamapi.STREAM_RECORD_KIND_UNSPECIFIED { + kind = streamapi.STREAM_RECORD_KIND_DATA + } + return kind.String() +} + +// payloadText renders a payload for a table cell the way the printer renders +// payloads elsewhere: shorthand, on one line. +func payloadText(p *commonpb.Payload) (string, error) { + if p == nil { + return "", nil + } + b, err := temporalproto.CustomJSONMarshalOptions{ + Metadata: map[string]any{commonpb.EnablePayloadShorthandMetadataKey: true}, + }.Marshal(p) + if err != nil { + return "", fmt.Errorf("failed rendering payload: %w", err) + } + return string(b), nil +} + +func (c *TemporalStreamReadCommand) run(cctx *CommandContext, _ []string) error { + target, err := c.target() + if err != nil { + return err + } + start, err := c.startPosition() + if err != nil { + return err + } + if c.PageSize <= 0 { + return fmt.Errorf("--page-size must be positive") + } + cl, err := dialStreamClient(cctx, &c.Parent.ClientOptions) + if err != nil { + return err + } + defer cl.Close() + + reader := &streamReader{ + ctx: cctx, + cl: cl, + target: target, + start: start, + topics: c.Topic, + follow: c.Follow, + pageSize: int32(c.PageSize), + limit: c.Limit, + } + if cctx.JSONOutput { + // This is a listing command subject to json vs jsonl rules + cctx.Printer.StartList() + defer cctx.Printer.EndList() + for { + rec, err := reader.next() + if err != nil { + return err + } + if rec == nil { + return nil + } + if err := cctx.Printer.PrintStructured(rec, printer.StructuredOptions{}); err != nil { + return err + } + } + } + return cctx.Printer.PrintStructuredTableIter( + reflect.TypeOf(streamRecordRow{}), + &streamRowIter{cctx: cctx, reader: reader}, + printer.StructuredOptions{ + Table: &printer.TableOptions{ + // Streaming rows cannot be measured first, so the columns + // before the body take fixed widths. + FieldWidths: map[string]int{ + "Offset": 8, + "Kind": 8, + "Topic": 16, + "Producer": 20, + "Attempt": 8, + "Sequence": 10, + }, + }, + }, + ) +} + +// inputPayloads takes the records from --input and --input-file, or one per +// line of stdin when neither is given. +func (c *TemporalStreamAppendCommand) inputPayloads( + cctx *CommandContext, +) ([]*commonpb.Payload, error) { + if len(c.Input) == 0 && len(c.InputFile) == 0 { + lines, err := readStdinLines(cctx) + if err != nil { + return nil, err + } + c.Input = lines + } + if len(c.Input) == 0 && len(c.InputFile) == 0 { + return nil, nil + } + payloads, err := c.buildRawInputPayloads() + if err != nil { + return nil, err + } + return payloads.GetPayloads(), nil +} + +func readStdinLines(cctx *CommandContext) ([]string, error) { + // A terminal with nothing piped in would sit waiting for input the user + // did not mean to type. + if f, ok := cctx.Options.Stdin.(*os.File); ok && isatty.IsTerminal(f.Fd()) { + return nil, nil + } + scanner := bufio.NewScanner(cctx.Options.Stdin) + scanner.Buffer(make([]byte, 0, 64*1024), 16<<20) + var lines []string + for scanner.Scan() { + if line := strings.TrimSpace(scanner.Text()); line != "" { + lines = append(lines, line) + } + } + if err := scanner.Err(); err != nil { + return nil, fmt.Errorf("failed reading stdin: %w", err) + } + return lines, nil +} + +func (c *TemporalStreamAppendCommand) run(cctx *CommandContext, _ []string) error { + target, err := c.target() + if err != nil { + return err + } + var expectedOffset *int64 + if c.Command.Flags().Changed("expected-offset") { + if target.streamID == "" { + return fmt.Errorf("--expected-offset applies to a standalone stream") + } + if c.ExpectedOffset < 0 { + return fmt.Errorf("--expected-offset cannot be negative") + } + v := int64(c.ExpectedOffset) + expectedOffset = &v + } + if c.Attempt < 0 || c.Sequence < 0 { + return fmt.Errorf("--attempt and --sequence cannot be negative") + } + sequenceSet := c.Command.Flags().Changed("sequence") + if sequenceSet && c.ProducerId == "" { + return fmt.Errorf("--sequence needs --producer-id") + } + payloads, err := c.inputPayloads(cctx) + if err != nil { + return err + } + if len(payloads) == 0 && !c.Finish { + return fmt.Errorf( + "no records to append: pass --input, --input-file, lines on stdin, or --finish") + } + + records := make([]*streampb.StreamRecord, 0, len(payloads)+1) + for i, p := range payloads { + rec := &streampb.StreamRecord{ + Body: p, + Topic: c.Topic, + Kind: streamapi.STREAM_RECORD_KIND_DATA, + ProducerId: c.ProducerId, + Attempt: int64(c.Attempt), + } + if sequenceSet { + rec.Sequence = int64(c.Sequence + i) + } + records = append(records, rec) + } + if c.Finish { + rec := &streampb.StreamRecord{ + Topic: c.Topic, + Kind: streamapi.STREAM_RECORD_KIND_FINISH, + ProducerId: c.ProducerId, + Attempt: int64(c.Attempt), + } + if sequenceSet { + rec.Sequence = int64(c.Sequence + len(payloads)) + } + records = append(records, rec) + } + + cl, err := dialStreamClient(cctx, &c.Parent.ClientOptions) + if err != nil { + return err + } + defer cl.Close() + + if err := cl.encodeRecords(records); err != nil { + return fmt.Errorf("failed encoding records: %w", err) + } + var sequence int64 + if sequenceSet { + sequence = int64(c.Sequence) + } + out, err := cl.append(cctx, target, records, c.ProducerId, sequence, expectedOffset) + if err != nil { + return fmt.Errorf("failed appending to %v: %w", target, plainRefusal(err)) + } + if cctx.JSONOutput { + return cctx.Printer.PrintStructured(out, printer.StructuredOptions{}) + } + return cctx.Printer.PrintStructured(struct { + FirstOffset int64 + NextOffset int64 + Count int64 + Deduplicated bool + }{ + FirstOffset: out.GetFirstOffset(), + NextOffset: out.GetNextOffset(), + Count: out.GetCount(), + Deduplicated: out.GetDeduplicated(), + }, printer.StructuredOptions{}) +} + +func (c *TemporalStreamCloseCommand) run(cctx *CommandContext, _ []string) error { + cl, err := dialStreamClient(cctx, &c.Parent.ClientOptions) + if err != nil { + return err + } + defer cl.Close() + + var reason *commonpb.Payload + if c.Reason != "" { + reason, err = converter.GetDefaultDataConverter().ToPayload(c.Reason) + if err != nil { + return fmt.Errorf("failed converting reason: %w", err) + } + if cl.PayloadCodec != nil { + encoded, err := cl.PayloadCodec.Encode([]*commonpb.Payload{reason}) + if err != nil { + return fmt.Errorf("failed encoding reason: %w", err) + } + reason = encoded[0] + } + } + _, err = cl.svc.CloseStream(cctx, &streampb.CloseStreamRequest{ + FrontendRequest: &streampb.CloseStreamInput{ + Namespace: cl.namespace, + StreamId: c.StreamId, + Reason: reason, + }, + }) + if err != nil { + return fmt.Errorf("failed closing stream %q: %w", c.StreamId, err) + } + cctx.Printer.Printlnf("Closed stream %v", c.StreamId) + return nil +} + +func (c *TemporalStreamTruncateCommand) run(cctx *CommandContext, _ []string) error { + if c.To < 0 { + return fmt.Errorf("--to cannot be negative") + } + cl, err := dialStreamClient(cctx, &c.Parent.ClientOptions) + if err != nil { + return err + } + defer cl.Close() + + _, err = cl.svc.TruncateStream(cctx, &streampb.TruncateStreamRequest{ + FrontendRequest: &streampb.TruncateStreamInput{ + Namespace: cl.namespace, + StreamId: c.StreamId, + NewBaseOffset: int64(c.To), + }, + }) + if err != nil { + return fmt.Errorf("failed truncating stream %q: %w", c.StreamId, err) + } + cctx.Printer.Printlnf("Truncated stream %v to offset %v", c.StreamId, c.To) + return nil +} + +func (c *TemporalStreamDeleteCommand) run(cctx *CommandContext, _ []string) error { + cl, err := dialStreamClient(cctx, &c.Parent.ClientOptions) + if err != nil { + return err + } + defer cl.Close() + + _, err = cl.svc.DeleteStream(cctx, &streampb.DeleteStreamRequest{ + FrontendRequest: &streampb.DeleteStreamInput{ + Namespace: cl.namespace, + StreamId: c.StreamId, + Force: c.Force, + }, + }) + if err != nil { + return fmt.Errorf("failed deleting stream %q: %w", c.StreamId, err) + } + cctx.Printer.Printlnf("Deleted stream %v", c.StreamId) + return nil +} diff --git a/internal/temporalcli/commands.stream_test.go b/internal/temporalcli/commands.stream_test.go new file mode 100644 index 000000000..7700cac26 --- /dev/null +++ b/internal/temporalcli/commands.stream_test.go @@ -0,0 +1,477 @@ +package temporalcli_test + +import ( + "context" + "encoding/json" + "fmt" + "strconv" + "strings" + "sync" + "testing" + "time" + + "github.com/google/uuid" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "github.com/temporalio/cli/internal/temporalcli" + notificationpb "go.temporal.io/api/notification/v1" + streamapi "go.temporal.io/api/stream/v1" + "go.temporal.io/sdk/client" + "go.temporal.io/sdk/workflow" + streampb "go.temporal.io/server/chasm/lib/stream/gen/streampb/v1" +) + +// fakeStreamService answers every describe with one canned state and records +// the owners the CLI named, so the output is checked without a server that +// carries streams. +type fakeStreamService struct { + streampb.UnimplementedStreamServiceServer + + mu sync.Mutex + state *streampb.StreamState + owned []*streampb.DescribeWorkflowStreamInput +} + +func (f *fakeStreamService) DescribeStream( + context.Context, *streampb.DescribeStreamRequest, +) (*streampb.DescribeStreamResponse, error) { + f.mu.Lock() + defer f.mu.Unlock() + return &streampb.DescribeStreamResponse{ + FrontendResponse: &streampb.DescribeStreamOutput{State: f.state}, + }, nil +} + +func (f *fakeStreamService) DescribeWorkflowStream( + _ context.Context, req *streampb.DescribeWorkflowStreamRequest, +) (*streampb.DescribeWorkflowStreamResponse, error) { + f.mu.Lock() + defer f.mu.Unlock() + f.owned = append(f.owned, req.GetFrontendRequest()) + return &streampb.DescribeWorkflowStreamResponse{ + FrontendResponse: &streampb.DescribeStreamOutput{State: f.state}, + }, nil +} + +func TestStream_DescribeChannel(t *testing.T) { + st := &fakeStreamService{state: &streampb.StreamState{HeadOffset: 2}} + addr := startFakeServices(t, &fakeChannelService{}, st) + h := NewCommandHarness(t) + // channelSection is the output from the channel card on, so the owner + // asserted on is the channel's and not the stream card's. + channelSection := func(args ...string) string { + res := h.Execute(append([]string{"stream", "describe", "--address", addr}, args...)...) + require.NoError(t, res.Err) + out := res.Stdout.String() + h.ContainsOnSameLine(out, "HeadOffset", "2") + _, section, found := strings.Cut(out, "Notification Channel:") + require.True(t, found, "no channel card in %q", out) + return section + } + + section := channelSection("--stream-id", "scores") + h.ContainsOnSameLine(section, "Channel", "stream/scores") + h.ContainsOnSameLine(section, "Kind", "Independent") + assert.NotContains(t, section, "LinkedTo") + + // The owner's default stream has the name the server resolves it to. + section = channelSection("--workflow-id", "wf-1") + h.ContainsOnSameLine(section, "Channel", "stream/output") + h.ContainsOnSameLine(section, "Kind", "Linked") + h.ContainsOnSameLine(section, "LinkedTo", "workflow wf-1") + assert.NotContains(t, section, "run ") + + section = channelSection("--workflow-id", "wf-1", "--run-id", "run-1", "--name", "scores") + h.ContainsOnSameLine(section, "Channel", "stream/scores") + h.ContainsOnSameLine(section, "Kind", "Linked") + h.ContainsOnSameLine(section, "LinkedTo", "workflow wf-1 (run run-1)") + + // An activity the workflow scheduled notifies the workflow, under a name + // that carries the activity ID. + section = channelSection("--workflow-id", "wf-1", "--activity-id", "act-1", "--name", "scores") + h.ContainsOnSameLine(section, "Channel", "stream/act-1/scores") + h.ContainsOnSameLine(section, "Kind", "Linked") + h.ContainsOnSameLine(section, "LinkedTo", "workflow wf-1") + + // A standalone activity holds linked channels of its own, so its stream + // notifies one named like a workflow's and linked to the activity. + section = channelSection("--activity-id", "act-1", "--run-id", "run-2", "--name", "scores") + h.ContainsOnSameLine(section, "Channel", "stream/scores") + h.ContainsOnSameLine(section, "Kind", "Linked") + h.ContainsOnSameLine(section, "LinkedTo", "activity act-1 (run run-2)") + + // The derivation adds no call: each describe reached the service once. + st.mu.Lock() + defer st.mu.Unlock() + require.Len(t, st.owned, 4) + assert.Equal(t, "", st.owned[0].GetStreamName(), "the server resolves the default name") + assert.Equal(t, streampb.STREAM_OWNER_KIND_ACTIVITY, st.owned[3].GetOwner().GetKind()) +} + +func (s *SharedServerSuite) createStream(id string, args ...string) { + res := s.Execute(append([]string{ + "stream", "create", "--address", s.Address(), "--stream-id", id, + }, args...)...) + s.NoError(res.Err) + s.Contains(res.Stdout.String(), id) +} + +// readStreamJSON runs a read with JSON output and decodes every record. +func (s *SharedServerSuite) readStreamJSON(args ...string) []*streampb.StreamRecord { + res := s.Execute(append([]string{ + "stream", "read", "--address", s.Address(), "-o", "json", + }, args...)...) + s.NoError(res.Err) + var raw []json.RawMessage + s.NoError(json.Unmarshal(res.Stdout.Bytes(), &raw)) + records := make([]*streampb.StreamRecord, len(raw)) + for i, r := range raw { + records[i] = &streampb.StreamRecord{} + s.NoError(temporalcli.UnmarshalProtoJSONWithOptions(r, records[i], true)) + } + return records +} + +func (s *SharedServerSuite) TestStream_StandaloneLifecycle() { + id := "stream-" + uuid.NewString() + s.createStream(id, "--retention", "1h", "--max-bytes", "1048576") + + // The same lifecycle again is a retry; a different one is refused. + res := s.Execute( + "stream", "create", "--address", s.Address(), "--stream-id", id, + "--retention", "1h", "--max-bytes", "1048576", + ) + s.ErrorContains(res.Err, "already exists") + res = s.Execute( + "stream", "create", "--address", s.Address(), "--stream-id", id, + "--retention", "1h", "--max-bytes", "2048", + ) + s.ErrorContains(res.Err, "different lifecycle") + s.NotContains(res.Err.Error(), "STREAM_POLICY_MISMATCH") + + appendArgs := []string{ + "stream", "append", "--address", s.Address(), "--stream-id", id, + "--topic", "scores", "--producer-id", "p1", "--attempt", "1", "--sequence", "1", + "--input", `{"home": 1}`, "--input", `{"home": 2}`, + } + res = s.Execute(appendArgs...) + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "FirstOffset", "0") + s.ContainsOnSameLine(res.Stdout.String(), "NextOffset", "2") + s.ContainsOnSameLine(res.Stdout.String(), "Deduplicated", "false") + + // The same producer retrying the same sequence appends nothing. + res = s.Execute(appendArgs...) + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "Deduplicated", "true") + + // Different content at that sequence is a conflict, and an older sequence + // is stale; both are refused in plain words, without the token. + res = s.Execute( + "stream", "append", "--address", s.Address(), "--stream-id", id, + "--producer-id", "p1", "--sequence", "1", "--input", `{"home": 9}`, + ) + s.ErrorContains(res.Err, "different content at this sequence") + s.NotContains(res.Err.Error(), "STREAM_PRODUCER_CONFLICT") + res = s.Execute( + "stream", "append", "--address", s.Address(), "--stream-id", id, + "--producer-id", "p1", "--sequence", "0", "--input", `{"home": 9}`, + ) + s.ErrorContains(res.Err, "below the producer's latest") + s.NotContains(res.Err.Error(), "STREAM_PRODUCER_STALE_SEQUENCE") + + // One record per stdin line, then the producer's FINISH. + s.Stdin.WriteString("\"third\"\n\n\"fourth\"\n") + res = s.Execute( + "stream", "append", "--address", s.Address(), "--stream-id", id, + "--producer-id", "p2", "--finish", + ) + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "FirstOffset", "2") + s.ContainsOnSameLine(res.Stdout.String(), "Count", "3") + + // Nothing to append is an error rather than an empty batch. + res = s.Execute("stream", "append", "--address", s.Address(), "--stream-id", id) + s.ErrorContains(res.Err, "no records to append") + + // Read from the beginning as a table. + res = s.Execute("stream", "read", "--address", s.Address(), "--stream-id", id) + s.NoError(res.Err) + out := res.Stdout.String() + s.ContainsOnSameLine(out, "Offset", "Kind", "Topic", "Producer", "Attempt", "Sequence", "Body") + s.ContainsOnSameLine(out, "0", "Data", "scores", "p1", "1", "1", `{"home":1}`) + s.ContainsOnSameLine(out, "1", "Data", "scores", "p1", "1", "2", `{"home":2}`) + s.ContainsOnSameLine(out, "2", "Data", "p2", `"third"`) + s.ContainsOnSameLine(out, "3", "Data", "p2", `"fourth"`) + s.ContainsOnSameLine(out, "4", "Finish", "p2") + + // The same records as JSON, with the body in shorthand. + records := s.readStreamJSON("--stream-id", id) + s.Len(records, 5) + s.Equal(int64(3), records[3].GetOffset()) + s.Equal(`"fourth"`, string(records[3].GetBody().GetData())) + s.Equal("p2", records[4].GetProducerId()) + s.Equal(streamapi.STREAM_RECORD_KIND_FINISH, records[4].GetKind()) + + // Start positions and filters. + records = s.readStreamJSON("--stream-id", id, "--last", "2") + s.Len(records, 2) + s.Equal(int64(3), records[0].GetOffset()) + records = s.readStreamJSON("--stream-id", id, "--from-offset", "4") + s.Len(records, 1) + s.Equal(int64(4), records[0].GetOffset()) + records = s.readStreamJSON("--stream-id", id, "--topic", "scores") + s.Len(records, 2) + records = s.readStreamJSON("--stream-id", id, "--limit", "3", "--page-size", "2") + s.Len(records, 3) + records = s.readStreamJSON("--stream-id", id, "--from-tail") + s.Empty(records) + + // Describe shows the frontier and both producers. + res = s.Execute("stream", "describe", "--address", s.Address(), "--stream-id", id) + s.NoError(res.Err) + out = res.Stdout.String() + s.ContainsOnSameLine(out, "StreamId", id) + s.ContainsOnSameLine(out, "HeadOffset", "5") + s.ContainsOnSameLine(out, "Retention", "1h") + s.ContainsOnSameLine(out, "MaxBytes", "1048576") + s.ContainsOnSameLine(out, "HeldBytes") + s.ContainsOnSameLine(out, "Producers", "2") + s.ContainsOnSameLine(out, "p1", "1", "0", "2", "false") + // A FINISH record tells readers the producer is done; only FinishWriting + // fences it on the stream. + s.ContainsOnSameLine(out, "p2", "0", "2", "3", "false") + s.ContainsOnSameLine(out, "Closed", "false") + s.ContainsOnSameLine(out, "Consumers", "0") + + res = s.Execute("stream", "describe", "--address", s.Address(), "--stream-id", id, "-o", "json") + s.NoError(res.Err) + var state streampb.StreamState + s.NoError(temporalcli.UnmarshalProtoJSONWithOptions(res.Stdout.Bytes(), &state, true)) + s.Equal(int64(5), state.GetHeadOffset()) + s.Equal(int64(0), state.GetBaseOffset()) + s.False(state.GetClosed()) + s.Equal(int64(3), state.GetProducers()["p2"].GetCount()) + s.Equal(int64(1048576), state.GetLifecycle().GetMaxBytes()) + s.Positive(state.GetHeldBytes()) + s.Equal(state.GetAppendedBytes(), state.GetHeldBytes()) + + // Truncation moves the floor; the beginning is now the oldest record left, + // and the reclaimed batch no longer counts as held. + res = s.Execute("stream", "truncate", "--address", s.Address(), "--stream-id", id, "--to", "2") + s.NoError(res.Err) + records = s.readStreamJSON("--stream-id", id) + s.Len(records, 3) + s.Equal(int64(2), records[0].GetOffset()) + res = s.Execute("stream", "describe", "--address", s.Address(), "--stream-id", id, "-o", "json") + s.NoError(res.Err) + var truncated streampb.StreamState + s.NoError(temporalcli.UnmarshalProtoJSONWithOptions(res.Stdout.Bytes(), &truncated, true)) + s.Equal(int64(2), truncated.GetBaseOffset()) + s.Less(truncated.GetHeldBytes(), truncated.GetAppendedBytes()) + res = s.Execute( + "stream", "read", "--address", s.Address(), "--stream-id", id, "--from-offset", "0", + ) + s.ErrorContains(res.Err, "below the stream's floor") + s.NotContains(res.Err.Error(), "STREAM_CURSOR_BELOW_FLOOR") + + // Closing seals the stream but keeps it readable. + res = s.Execute("stream", "close", "--address", s.Address(), "--stream-id", id, "--reason", "done") + s.NoError(res.Err) + res = s.Execute("stream", "describe", "--address", s.Address(), "--stream-id", id) + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "Closed", "true") + s.ContainsOnSameLine(res.Stdout.String(), "CloseReason", `"done"`) + res = s.Execute( + "stream", "append", "--address", s.Address(), "--stream-id", id, "--input", `"late"`, + ) + s.ErrorContains(res.Err, "the stream is closed") + s.NotContains(res.Err.Error(), "STREAM_CLOSED") + s.Len(s.readStreamJSON("--stream-id", id), 3) + + // Deleting takes the records with it. + res = s.Execute("stream", "delete", "--address", s.Address(), "--stream-id", id) + s.NoError(res.Err) + res = s.Execute("stream", "describe", "--address", s.Address(), "--stream-id", id) + s.Error(res.Err) +} + +func (s *SharedServerSuite) TestStream_ReadFollow() { + id := "stream-" + uuid.NewString() + s.createStream(id) + res := s.Execute( + "stream", "append", "--address", s.Address(), "--stream-id", id, "--input", `"first"`, + ) + s.NoError(res.Err) + + done := make(chan *CommandResult, 1) + go func() { + done <- s.Execute("stream", "read", "--address", s.Address(), "--stream-id", id, "--follow") + }() + + // The follow is parked on the frontier by the time this lands. + time.Sleep(time.Second) + res = s.Execute( + "stream", "append", "--address", s.Address(), "--stream-id", id, "--input", `"second"`, + ) + s.NoError(res.Err) + res = s.Execute("stream", "close", "--address", s.Address(), "--stream-id", id) + s.NoError(res.Err) + + select { + case res = <-done: + case <-time.After(60 * time.Second): + s.Fail("follow did not end after the stream closed") + } + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "0", "Data", `"first"`) + s.ContainsOnSameLine(res.Stdout.String(), "1", "Data", `"second"`) +} + +func (s *SharedServerSuite) TestStream_OwnedByWorkflow() { + s.Worker().OnDevWorkflow(func(ctx workflow.Context, a any) (any, error) { + workflow.GetSignalChannel(ctx, "finish").Receive(ctx, nil) + return nil, nil + }) + run, err := s.Client.ExecuteWorkflow( + s.Context, + client.StartWorkflowOptions{TaskQueue: s.Worker().Options.TaskQueue}, + DevWorkflow, + "ignored", + ) + s.NoError(err) + + // An owned stream comes into being on its first append. + res := s.Execute( + "stream", "append", "--address", s.Address(), + "--workflow-id", run.GetID(), "--name", "scores", + "--producer-id", "outside", "--input", `{"home": 1}`, "--input", `{"home": 2}`, + ) + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "NextOffset", "2") + + res = s.Execute( + "stream", "read", "--address", s.Address(), + "--workflow-id", run.GetID(), "--run-id", run.GetRunID(), "--name", "scores", + ) + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "0", "Data", "outside", `{"home":1}`) + s.ContainsOnSameLine(res.Stdout.String(), "1", "Data", "outside", `{"home":2}`) + + res = s.Execute( + "stream", "describe", "--address", s.Address(), "--workflow-id", run.GetID(), "--name", "scores", + ) + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "WorkflowId", run.GetID()) + s.ContainsOnSameLine(res.Stdout.String(), "Name", "scores") + s.ContainsOnSameLine(res.Stdout.String(), "HeadOffset", "2") + + // The owner's default stream is a different, still empty, stream. + s.Empty(s.readStreamJSON("--workflow-id", run.GetID())) + + s.NoError(s.Client.SignalWorkflow(s.Context, run.GetID(), "", "finish", nil)) + s.NoError(run.Get(s.Context, nil)) +} + +func (s *SharedServerSuite) TestStream_List() { + prefix := "list-" + uuid.NewString()[:8] + s.createStream(prefix + "-a") + s.createStream(prefix + "-b") + other := "other-" + uuid.NewString() + s.createStream(other) + query := fmt.Sprintf(`WorkflowId STARTS_WITH "%s"`, prefix) + + var out string + s.EventuallyWithT(func(t *assert.CollectT) { + res := s.Execute("stream", "list", "--address", s.Address(), "--query", query) + assert.NoError(t, res.Err) + out = res.Stdout.String() + assert.Contains(t, out, prefix+"-a") + assert.Contains(t, out, prefix+"-b") + }, 10*time.Second, 200*time.Millisecond) + s.ContainsOnSameLine(out, "StreamId", "RunId") + s.NotContains(out, other) + + res := s.Execute( + "stream", "list", "--address", s.Address(), "--query", query, "--limit", "1", "-o", "json", + ) + s.NoError(res.Err) + var entries []json.RawMessage + s.NoError(json.Unmarshal(res.Stdout.Bytes(), &entries)) + s.Len(entries, 1) + var entry streampb.StreamListEntry + s.NoError(temporalcli.UnmarshalProtoJSONWithOptions(entries[0], &entry, true)) + s.Contains(entry.GetStreamId(), prefix) + s.NotEmpty(entry.GetRunId()) +} + +func (s *SharedServerSuite) TestStream_StandaloneNotifiesItsChannel() { + id := "stream-" + uuid.NewString() + s.createStream(id) + ch := "stream/" + id + poll := func(after int64) *notificationpb.Notification { + res := s.Execute("channel", "poll", "--address", s.Address(), "-c", ch, + "--after-counter", strconv.FormatInt(after, 10), "--wait", "10s", "-o", "jsonl") + s.NoError(res.Err) + raw := decodeJSONValues(s.T(), res.Stdout.String()) + s.NotEmpty(raw) + var n notificationpb.Notification + s.NoError(temporalcli.UnmarshalProtoJSONWithOptions(raw[len(raw)-1], &n, true)) + return &n + } + + res := s.Execute("stream", "describe", "--address", s.Address(), "--stream-id", id) + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "Channel", ch) + s.ContainsOnSameLine(res.Stdout.String(), "Kind", "Independent") + + res = s.Execute("stream", "append", "--address", s.Address(), "--stream-id", id, + "--input", `{"home": 1}`) + s.NoError(res.Err) + // A standalone stream's notify reaches its channel through a task of its + // own, so the poll waits for it. + n := poll(0) + s.Equal(ch, n.GetChannel()) + s.GreaterOrEqual(n.GetCounter(), int64(1)) + s.Equal(id+":1", string(n.GetPosition())) + s.Nil(n.GetMetadata()["closed"]) + + res = s.Execute("stream", "close", "--address", s.Address(), "--stream-id", id) + s.NoError(res.Err) + closed := poll(n.GetCounter()) + s.Greater(closed.GetCounter(), n.GetCounter()) + s.Equal(id+":1", string(closed.GetPosition())) + s.Equal("true", string(closed.GetMetadata()["closed"].GetData())) + + res = s.Execute("channel", "describe", "--address", s.Address(), "-c", ch) + s.NoError(res.Err) + s.ContainsOnSameLine(res.Stdout.String(), "Kind", "Independent") + s.ContainsOnSameLine(res.Stdout.String(), "LatestCounter", + strconv.FormatInt(closed.GetCounter(), 10)) +} + +func (s *SharedServerSuite) TestStream_ReferenceValidation() { + res := s.Execute( + "stream", "read", "--address", s.Address(), "--stream-id", "a", "--workflow-id", "b", + ) + s.ErrorContains(res.Err, "cannot set --stream-id") + res = s.Execute("stream", "read", "--address", s.Address()) + s.ErrorContains(res.Err, "must set either --stream-id") + res = s.Execute("stream", "read", "--address", s.Address(), "--stream-id", "a", "--name", "n") + s.ErrorContains(res.Err, "--run-id and --name") + res = s.Execute( + "stream", "read", "--address", s.Address(), "--stream-id", "a", "--from-tail", "--last", "2", + ) + s.ErrorContains(res.Err, "at most one") + res = s.Execute( + "stream", "append", "--address", s.Address(), "--workflow-id", "w", + "--expected-offset", "3", "--input", "1", + ) + s.ErrorContains(res.Err, "--expected-offset applies to a standalone stream") + res = s.Execute("stream", "truncate", "--address", s.Address(), "--to", "1") + s.ErrorContains(res.Err, "stream-id") + res = s.Execute( + "stream", "create", "--address", s.Address(), "--stream-id", "a", "--max-bytes", "-1", + ) + s.ErrorContains(res.Err, "cannot be negative") +} diff --git a/internal/temporalcli/commands.workflow_view.go b/internal/temporalcli/commands.workflow_view.go index c44711a11..ca767934d 100644 --- a/internal/temporalcli/commands.workflow_view.go +++ b/internal/temporalcli/commands.workflow_view.go @@ -252,6 +252,9 @@ func (c *TemporalWorkflowDescribeCommand) run(cctx *CommandContext, args []strin if err := printCallbacks(cctx, resp.Callbacks); err != nil { return err } + if err := printChannelSubscriptions(cctx, resp.ChannelSubscriptions); err != nil { + return err + } if running { cctx.Printer.Println() diff --git a/internal/temporalcli/commands.yaml b/internal/temporalcli/commands.yaml index 3a94ce74b..30ad1c46d 100644 --- a/internal/temporalcli/commands.yaml +++ b/internal/temporalcli/commands.yaml @@ -950,6 +950,244 @@ commands: description: Reason for terminating the batch job. required: true + - name: temporal channel + summary: Notify and listen on notification channels + description: | + A notification channel is a name a writer and its listeners agree on. + The writer notifies the channel when a source it writes moves, such as + a Stream gaining records, and never learns who listens. A Workflow + listener gets a Workflow Task, a callback listener gets an HTTP call, + and a client long-polls: + + ``` + temporal channel [command] [options] + ``` + + For example: + + ``` + temporal channel poll \ + --channel YourChannel \ + --follow + ``` + + A notification tells listeners where the source stands. It carries no + data; the listener reads the source itself. + + An independent channel exists on its own, and any number of Workflows, + callbacks and clients listen on it. A channel linked to a Workflow + Execution lives with that Workflow, which listens on it without + subscribing. A channel linked to a standalone Activity lives with that + Activity the same way. Name the owner with `--workflow-id` or + `--activity-id` on any channel command: + + ``` + temporal channel notify \ + --channel YourChannel \ + --workflow-id YourWorkflowId \ + --position 42 \ + --counter 42 + ``` + option-sets: + - client + docs: + description-header: >- + Temporal Channel commands notify a notification channel, poll it, + describe it, and add or remove its callback listeners. + keywords: + - channel + - channel describe + - channel listener add + - channel listener remove + - channel notify + - channel poll + - cli reference + - command-line-interface-cli + - notification channel + - temporal cli + tags: + - Temporal CLI + - Channels + + - name: temporal channel describe + summary: Show a channel's listeners and latest notification + description: | + Show who listens on a channel, its latest notification, and how many + notifications it keeps for pollers: + + ``` + temporal channel describe \ + --channel YourChannel + ``` + + An independent channel exists once a writer notifies it or a listener + registers on it, and goes away after a while without listeners or + activity. A channel linked to a running Workflow Execution or standalone + Activity always exists, and its description names that owner: + + ``` + temporal channel describe \ + --channel YourChannel \ + --workflow-id YourWorkflowId + ``` + option-sets: + - channel + + - name: temporal channel listener + summary: Add or remove a channel's callback listeners + description: | + Register a callback the Service calls with every notification on a + channel, or remove one: + + ``` + temporal channel listener [command] [options] + ``` + + A Workflow listens on a channel from its own code instead, and stops + listening when its run ends. + + - name: temporal channel listener add + summary: Register a callback listener + description: | + Register a callback on a channel. The Service calls the URL with every + notification the channel gets from now on: + + ``` + temporal channel listener add \ + --channel YourChannel \ + --callback-url https://example.com/notify \ + --header "Authorization=Bearer YourToken" + ``` + + The command prints the listener ID that `temporal channel listener + remove` takes. + + The Service only calls addresses its `callback.allowedAddresses` + dynamic configuration value allows. The development server allows + `127.0.0.1` and `localhost` on any port over plain HTTP. Another + Service needs the address added, for example: + + ``` + --dynamic-config-value \ + 'callback.allowedAddresses=[{"Pattern":"*.example.com:443"}]' + ``` + options: + - name: callback-url + type: string + description: URL the Service calls with each notification. + required: true + - name: header + type: string[] + description: | + Header sent with every call, in `KEY=VALUE` format. + Can be passed multiple times. + option-sets: + - channel + + - name: temporal channel listener remove + summary: Remove a callback listener + description: | + Stop calling a callback listener: + + ``` + temporal channel listener remove \ + --channel YourChannel \ + --listener-id YourListenerId + ``` + + Find the listener ID with `temporal channel describe`. + options: + - name: listener-id + type: string + description: ID of the listener to remove. + required: true + option-sets: + - channel + + - name: temporal channel notify + summary: Notify a channel's listeners + description: | + Tell every listener of a channel that its source moved. The command + prints how many listeners the notification reached: + + ``` + temporal channel notify \ + --channel YourChannel \ + --position 42 \ + --counter 42 \ + --metadata 'topic="scores"' + ``` + + The position is where the source stands now, in the writer's own + terms. The counter orders notifications on the channel: when several + wait for a listener at once, it gets the one with the highest counter. + Metadata values are JSON. + options: + - name: position + type: string + description: | + Where the source stands after the write. + Sent as text; the Service does not read it. + required: true + - name: counter + type: int + description: | + Order of this notification among the channel's notifications. + Must be greater than zero. + required: true + - name: metadata + type: string[] + description: | + Detail for listeners, in `KEY=VALUE` format. + Values must be JSON. + Can be passed multiple times. + option-sets: + - channel + + - name: temporal channel poll + summary: Wait for a channel's notifications + description: | + Show the notifications a channel keeps, waiting for one when there is + none yet: + + ``` + temporal channel poll \ + --channel YourChannel + ``` + + Keep polling and print notifications as they arrive: + + ``` + temporal channel poll \ + --channel YourChannel \ + --after-counter 10 \ + --follow + ``` + + Each notification prints with its counter, position and metadata. With + `--output json` each notification is one JSON object. + options: + - name: after-counter + type: int + description: Show only notifications with a counter above this one. + - name: wait + type: duration + description: | + How long one poll waits for a notification when there is none. + The Service may answer sooner. + default: 30s + - name: max + type: int + description: | + Most notifications one poll returns. + Default is zero (the Service's limit). + - name: follow + short: f + type: bool + description: Keep polling from the last counter seen until interrupted. + option-sets: + - channel + - name: temporal config summary: Manage config files (EXPERIMENTAL) description: | @@ -3380,6 +3618,23 @@ commands: temporal server start-dev \ --ui-port 3000 ``` + + Streams are enabled on the development server. Turn them off with a + dynamic configuration value: + + ``` + temporal server start-dev \ + --dynamic-config-value 'stream.enabled=false' + ``` + + Callbacks to `127.0.0.1` and `localhost` on any port are allowed over + plain HTTP, so a local receiver can listen on a notification channel. + Replace the list with a dynamic configuration value: + + ``` + temporal server start-dev \ + --dynamic-config-value 'callback.allowedAddresses=[]' + ``` options: - name: db-filename short: f @@ -3463,6 +3718,319 @@ commands: attribute type, which is one of the following: Text, Keyword, Int, Double, Bool, Datetime, KeywordList. + - name: temporal stream + summary: Read and manage Streams + description: | + A Stream is an append-only log the Temporal Service carries next to + Workflow Executions. A standalone Stream has an ID of its own. An owned + Stream lives inside a Workflow Execution or an Activity and is addressed + by its owner and a Stream name: + + ``` + temporal stream [command] [options] + ``` + + For example: + + ``` + temporal stream read \ + --workflow-id YourWorkflowId + ``` + + The Service must enable Streams for the Namespace through the + `stream.enabled` dynamic configuration value. The development server + enables them. + option-sets: + - client + docs: + description-header: >- + Temporal Stream commands list, describe, read, append to, close, + truncate and delete the Streams a Temporal Service carries. + keywords: + - cli reference + - command-line-interface-cli + - stream + - stream append + - stream close + - stream create + - stream delete + - stream describe + - stream list + - stream read + - stream truncate + - streams + - temporal cli + tags: + - Temporal CLI + - Streams + + - name: temporal stream append + summary: Append records to a Stream + description: | + Append one or more records to a Stream. Each `--input` is one record. + Without `--input` or `--input-file`, one record is read per line from + stdin: + + ``` + temporal stream append \ + --stream-id YourStreamId \ + --input '{"score": 1}' \ + --input '{"score": 2}' + ``` + + Append to a Stream a Workflow Execution owns, naming the owner instead: + + ``` + temporal stream append \ + --workflow-id YourWorkflowId \ + --name scores \ + --input '{"score": 1}' + ``` + + Set `--producer-id` and `--sequence` so a retried append is deduplicated + rather than appended twice. Pass `--finish` to end the producer's writes + on the topic with a `FINISH` record after the values. + options: + - name: topic + type: string + description: Topic label stored on every record. + - name: producer-id + type: string + description: | + Producer identity stored on every record. + Together with --sequence it deduplicates a retried append. + - name: attempt + type: int + description: Producer attempt stored on every record. + - name: sequence + type: int + description: | + Sequence of the first record within the producer's attempt. + Records that follow take the next numbers. + - name: expected-offset + type: int + description: | + Refuse the append unless the Stream's next offset equals this value. + Only applies to a standalone Stream. + - name: finish + type: bool + description: Append a `FINISH` record after the values. + option-sets: + - stream-reference + - payload-input + + - name: temporal stream close + summary: Close a Stream + description: | + Close a standalone Stream. A closed Stream accepts no more records and + stays readable until its retention passes: + + ``` + temporal stream close \ + --stream-id YourStreamId \ + --reason "done" + ``` + options: + - name: reason + type: string + description: Reason recorded on the Stream. + option-sets: + - stream-id + + - name: temporal stream create + summary: Create a standalone Stream + description: | + Create a standalone Stream with an ID of its own: + + ``` + temporal stream create \ + --stream-id YourStreamId + ``` + + Records older than `--retention` are reclaimed, and a closed Stream is + deleted once it passes. Without it the Namespace's retention applies. + Creating a Stream that already exists with the same lifecycle is an + idempotent retry; a different lifecycle is refused. + options: + - name: retention + type: duration + description: | + How long a record stays readable, and how long a closed Stream + stays readable before it is deleted. + Cannot be longer than the Namespace's retention. + - name: max-items + type: int + description: | + Most records the Stream holds before the oldest are reclaimed. + Default is zero (unbounded). + - name: max-bytes + type: int + description: | + Most bytes the Stream holds. An append past it is refused until + records are reclaimed. Default is zero (unbounded). + option-sets: + - stream-id + + - name: temporal stream delete + summary: Remove a Stream + description: | + Delete a standalone Stream and every record it holds: + + ``` + temporal stream delete \ + --stream-id YourStreamId + ``` + + The call is refused while a Workflow Execution consumes the Stream, + because its replay depends on the records. Pass `--force` to delete it + anyway. + options: + - name: force + type: bool + description: Delete even while a Workflow Execution consumes the Stream. + option-sets: + - stream-id + + - name: temporal stream describe + summary: Show a Stream's state + description: | + Show a Stream's frontier, floor, close state, retention, producers and + consumers: + + ``` + temporal stream describe \ + --stream-id YourStreamId + ``` + + Describe a Stream a Workflow Execution owns: + + ``` + temporal stream describe \ + --workflow-id YourWorkflowId \ + --name scores + ``` + + The output also names the notification channel the Stream notifies on + each append and on its close. The name follows from the Stream alone, + so no call to the Service is needed. A Stream a Workflow Execution + owns notifies `stream/NAME`, linked to the owner. A Stream of an + Activity the Workflow scheduled notifies `stream/ACTIVITY_ID/NAME`, + linked to the owner too. A standalone Stream notifies the independent + channel `stream/STREAM_ID`. Pass the channel, and the owner of a + linked one, to `temporal channel poll` to follow the Stream: + + ``` + temporal channel poll \ + --channel stream/scores \ + --workflow-id YourWorkflowId + ``` + option-sets: + - stream-reference + + - name: temporal stream list + summary: Show standalone Streams + description: | + List the standalone Streams in a Namespace. The optional `--query` + limits the output to Streams matching a List Filter, where `WorkflowId` + holds the Stream ID: + + ``` + temporal stream list \ + --query 'WorkflowId STARTS_WITH "scores-"' + ``` + + Streams a Workflow Execution owns are not listed here. Describe or read + them through their owner. + options: + - name: query + short: q + type: string + description: Content for an SQL-like `QUERY` List Filter. + - name: limit + type: int + description: Maximum number of Streams to display. + - name: page-size + type: int + description: Maximum number of Streams to fetch at a time from the server. + default: 100 + + - name: temporal stream read + summary: Read a Stream's records + description: | + Read a Stream from the beginning and stop once caught up: + + ``` + temporal stream read \ + --stream-id YourStreamId + ``` + + Read a Stream a Workflow Execution owns and keep following it as records + land: + + ``` + temporal stream read \ + --workflow-id YourWorkflowId \ + --name scores \ + --follow + ``` + + Start at a cursor with `--from-offset`, at the tail with `--from-tail`, + or at the newest records with `--last`. Each record prints with its + offset, kind, topic, producer, attempt, sequence and body. With + `--output json` each record is one JSON object. + options: + - name: from-offset + type: int + description: | + Offset to start at, inclusive. + Refused when below the Stream's floor. + - name: from-tail + type: bool + description: Start at the tail and show only records appended from now on. + - name: last + type: int + description: Start at the newest records, showing at most this many first. + - name: follow + short: f + type: bool + description: Keep reading as records land, until the Stream closes. + - name: topic + type: string[] + description: | + Show only records on this topic. + Can be passed multiple times. + - name: limit + type: int + description: Maximum number of records to display. + - name: page-size + type: int + description: Maximum number of records to fetch at a time from the server. + default: 100 + option-sets: + - stream-reference + + - name: temporal stream truncate + summary: Drop the oldest records of a Stream + description: | + Move the readable floor of a standalone Stream up to an offset. Records + below it are gone: + + ``` + temporal stream truncate \ + --stream-id YourStreamId \ + --to 100 + ``` + + The call is refused while a Workflow Execution consumes records below + the new floor. + options: + - name: to + type: int + description: New floor. Records below this offset are dropped. + required: true + option-sets: + - stream-id + - name: temporal task-queue summary: Manage Task Queues description: | @@ -5392,6 +5960,72 @@ option-sets: short: r description: Run ID. + - name: channel + options: + - name: channel + type: string + short: c + description: Name of the notification channel. + required: true + - name: workflow-id + type: string + short: w + description: | + Workflow ID of the Workflow Execution the channel is linked to. + Without an owner the command uses the independent channel of that name. + Cannot use with --activity-id. + - name: activity-id + type: string + description: | + Activity ID of the standalone Activity the channel is linked to. + Cannot use with --workflow-id. + - name: run-id + type: string + short: r + description: | + Run ID of the linked execution. + Defaults to the current run. Requires --workflow-id or --activity-id. + + - name: stream-id + options: + - name: stream-id + type: string + short: s + description: Stream ID of a standalone Stream. + required: true + + - name: stream-reference + options: + - name: stream-id + type: string + short: s + description: | + Stream ID of a standalone Stream. + You must set either --stream-id or an owner. + - name: workflow-id + type: string + short: w + description: | + Workflow ID of the Workflow Execution owning the Stream. + Cannot use with --stream-id. + - name: run-id + type: string + short: r + description: | + Run ID of the owner. + Defaults to the current run. + - name: activity-id + type: string + description: | + Activity ID of the Activity owning the Stream. + With --workflow-id it names an Activity the Workflow scheduled. + On its own it names a standalone Activity. + - name: name + type: string + description: | + Name of the Stream within its owner. + Defaults to the owner's default Stream. + - name: deployment-name options: - name: name