Repository navigation
Conversation
…9-external-interface
…-10-redis-provider
…the clients. Core moves to the protos-only branch commit that carries the notification channel: the Notification message, the five channel calls, the subscribe command and event, the notifications on the scheduled Workflow Task event, and the bridge's command and activation job for them. The vendored tree, the payload visitor and the bridge client regenerate from the pin.
…-workflow-runtime
A workflow subscribes to a notification channel once per run and gets the notifications the server folded for it with each Workflow Task. They travel in History, so a replay sees the same ones at the same points.
Notify, poll, describe, register and unregister go through the outbound interceptor with one input each. Metadata values are encoded with the data converter on the way out and come back as payloads, the shape workflow code sees.
…vider-workflow-streams
…AI-198-streams-provider-nexus
A describe that races the subscribe is answered not found, and a worker whose Core refuses the subscribe command leaves the task in a timeout loop that the worker's shutdown waits on. The case now tolerates the first, names the second and bounds the shutdown.
…vider-workflow-streams
…AI-198-streams-provider-nexus
…mand. The protos-only Core refuses the subscribe command, so the case is gated on a conftest constant the native layers flip with their pin. A client-only case covers what the server keeps for pollers: a notify with no listener is retained and answers zero, a counter at or below the latest is not kept, and a bounded poll above the latest comes back empty.
…vider-workflow-streams
…AI-198-streams-provider-nexus
…ients. Core moves to the core-3 commit that carries the subscribe-notification-channel command machine and the NotificationsReceived job folded from the task's scheduled events. The vendored tree and the payload visitor regenerate from the pin, and the live workflow channel case now runs on this layer.
The server reports a producer conflict as FAILED_PRECONDITION with a reason token, and this layer only knew the INVALID_ARGUMENT a server built before the tokens sent, so the typed error was lost on the current server.
This layer maps the STREAM_CURSOR_BELOW_FLOOR token to StreamCursorError, which is not an RPCError, so the live case written on the wire layer no longer matched the exception it got.
… case. A channel nobody has touched is not found, and the first notify on a fresh channel shows up in describe with its position and counter before any poll.
…vider-workflow-streams
…AI-198-streams-provider-nexus
The stream_reader docstring said nothing crosses continue-as-new, which the Redis provider contradicts: its transport resumes a successor's reader where the predecessor committed.
The entry still said each topic was an input and an output stream, which the provider stopped doing when it moved to one shared log per topic.
The main chain carries the channel, the stream interface with its memory, Workflow Streams and Nexus providers, and native streams.
Both chains share the channel surface and the stream interface, so the merge keeps one copy of each. Core moves to the unified head that carries both lineages, and the clients are regenerated from it.
The union tests against the time-skipping server too, so it carries the fix that gives concurrent result waiters one shared unlock.
The workflow info and the payload converter factory grew on main after the external chain's helpers were written.
On the shared wire zero says a producer doesn't number its records, and the Redis producer does, so its first record is one.
The shared interface reads and writes DEFAULT_TOPIC when a call names no topic, and the Redis handle stores it under that name like any other.
Without TEMPORAL_ADDRESS the cases reached the environment's own server, which has no stream service, and failed on the wire.
The workflow_streams provider answers reads from the workflow itself, so the read has to happen before the worker stops.
The union carries the native provider, so the cases gated on it no longer skip.
The Redis producer notifies the stream's channel itself, so one live case needs no notify by hand. The native case opens its own client with the native provider on it.
Only the union carries the memory, Workflow Streams, native and Redis providers together, so the demo's provider switch and its notes live here.
A getattr default is evaluated whether or not the attribute is there, and protobuf 6 descriptors have is_repeated but no label, so the generator needed a protobuf 3 interpreter.
The union was rebuilt as one PR on main, so the examples move onto it. The tree is the union's plus the examples and their docs.
NexusStreams resolves the endpoint by name through the client, so the id the docs asked for failed the lookup. The docs also give --http as a full URL.
# Conflicts: # streams_demo/README.md # temporalio/bridge/proto/workflow_activation/workflow_activation_pb2.py # temporalio/bridge/proto/workflow_commands/workflow_commands_pb2.py # temporalio/bridge/sdk-core
1 of 3 tasks
Owner
Author
|
Replaced by #112 in the v5 series (phase 1, Signal wake only). The branch stays as a pin. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds the stream examples, one per path, one composed agent, and the June scenarios.
What changed?
examples/streams/path_a_publish.py: a workflow publishes progress and a backend follows it fromlatest().path_b_produce.py: an Activity appends to its workflow's topic, a backend produces and consumes, and the consumer handles a supersession when the first attempt fails.path_c_consume.py: a workflow consumes a topic fed from outside and runs an Activity per record with the cache off.agent.pyandrun.pycompose the three on every provider and behind the Nexus front. The agent races its reader against the generator, so an exhausted generator fails the run.examples/streams/june_scenarios/has one demo per scenario in Roey's June notes. Each file opens with the notes' heading, a status and why.run.pyruns them all on one provider.s2runs the three standalone alternatives on the handle, alt 3 with aStreamRefpassed into the workflow start.s8(b) returns aStreamReffrom a Nexus operation.README.mdmaps each path to its call._setup.pyis the one place a store is named, and every example defines its topics once withstreams.topic().examples/: a changelog line,examplesunder the type checker inpyproject.toml, and the Nexus endpoint setup instreams_demo/README.md.Part of AI-198 (epic AI-37).
Why?
It puts the portability claim in code a reviewer can run, with the same accessor in every context. The June suite answers Roey's scenarios one by one. It says plainly which are implemented, which are emulated and which are unsupported by design. It sits on the union, since it needs every provider at once.
How did you test it?
Link to a test plan if any -
poe lintpasses withexamples/under the type checker. Against a local server built from the stream server PRs, the three path examples andrun.pyexit 0 onworkflow_streams,nativeandredis, andrun.py nexusexits 0 through an endpoint on that server. The June suite exits 0 onnative,workflow_streams,memoryandredis. A provider that can't serve a scenario says so:s2ands5(b) and (c) onworkflow_streams,s1(d) offmemory, ands7onmemory. The worker and consumer that die ins6ands8do so on purpose.