Skip to content

Notified the channel named by a native stream on each change. - #13

Closed
moetemp wants to merge 7 commits into
moe/AI-198-srv-9-inline-handofffrom
moe/AI-198-srv-10-stream-channel
Closed

moetemp wants to merge 7 commits into
moe/AI-198-srv-9-inline-handofffrom
moe/AI-198-srv-10-stream-channel

Conversation

@moetemp

@moetemp moetemp commented Oct 2, 2026

Copy link
Copy Markdown
Owner

This PR makes a native stream notify the notification channel named by the stream on every change.

What changed?

  • A workflow-owned stream <name> notifies the channel stream/<name> linked to the owning run, a stream of an activity the workflow scheduled notifies stream/<activity id>/<name> linked to that run, and a standalone stream <id> notifies the independent channel stream/<id>. A stream a standalone activity owns notifies the independent channel stream/<activity id>/<name>, since a standalone activity has no linked channels.
  • One notification per append call and one for the close. counter is the stream's new change sequence, position is the head after the change in the form a native cursor names it (<run>:<offset>, the stream id in place of the run for a standalone stream), and the close carries closed: true in the metadata. A producer's deduplicated retry is not a change.
  • The owned kinds notify in the write that appends, through the linked channel without handing the notification to the owner: the owning run follows its own streams through its cursor, and a Workflow Task per append would be a cost the stream did not have. Callback listeners are handed each change and the ring keeps it for pollers.
  • The standalone kinds reach their channel through a task on the stream's execution, one outstanding at a time, which reads the latest change when it runs. Tasks of one execution may run in any order and the channel folds a counter below its latest, so one task per change would lose earlier changes from the ring. A burst reaches the channel as its newest change, in order.
  • Dynamic config stream.notifyChannel (default on) turns it off for a namespace, in which case streams create no channel state, and stream.channelRetainedNotifications (default 16) caps the ring of a stream-driven channel.
  • DescribeChannel and the workflow's channel list report stream-driven channels like any other.
  • tests/stream_channel_test.go covers the four owner kinds and the switch, and unit tests cover the change sequence, the notification shape, the coalescing task and the listeners-only linked notify.

Part of AI-198 (epic AI-37).

Why?

Section 8 of the proposal said native streams share the channel's effect, not its mechanism, and left a write to a native stream notifying the channel named by the stream as a follow-up. With it, a callback or a poller follows a native stream exactly as it follows an external one, and a client derives the channel name from the stream's identity without asking the server. The workflow consumers of native streams keep the consumer table and AdvanceConsumerHead. Nothing changes for them.

How did you test it?

  • Unit Tests
  • Staging
  • End to End Tests

go build ./..., golangci-lint and the errortype vet are clean on the touched packages. Unit tests ran for ./chasm/lib/stream/..., ./chasm/lib/channel/..., ./chasm/lib/workflow/..., ./chasm/lib/activity/... and ./service/history/workflow/.... The functional tests ran with -tags disable_grpc_modules,test_dep: an append to a workflow-owned stream shows on the linked channel with the head as position and a counter of one, a second append gives two, a callback registered on it is handed the latest and then each change, the owner gets no Workflow Task and a producer's retry notifies nothing. A standalone stream reaches its independent channel in order through the task, a callback that registers after three appends is handed the third once, and the close is the last change marked closed. A workflow activity's stream notifies its channel and the activity's completion closes it with a last change. A standalone activity's stream notifies its independent channel. With the switch off nothing is created. The stream suites, the notification channel and linked channel suites and the inline handoff cases pass.

Callback and polling listeners learn of a native stream the way they learn of an external one. A stream a workflow owns notifies the linked channel of its name in the write that appends, without waking the owner, which already follows its own streams through its cursor. A standalone stream, and one a standalone activity owns, reach an independent channel through one coalescing task.
@moetemp

moetemp commented Oct 3, 2026

Copy link
Copy Markdown
Owner Author

Replaced by #18, #19, #20, #21, #22, #23, #24, #25, #26, #27, #28, #29, #30, #31, #32, #33, #34, #35, #36 and #37.

Same content, split into 20 PRs in the v3 series: the notification channel first, then the streaming interface, then native streams and the rest. The branch stays as a pin.

@moetemp moetemp closed this Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant