Skip to content

Merged native streams into the Option 7 (external streams) Core. - #36

Closed
moetemp wants to merge 9 commits into
moe/AI-198-o7-core-5-channel-wakefrom
moe/AI-198-un-core
Closed

moetemp wants to merge 9 commits into
moe/AI-198-o7-core-5-channel-wakefrom
moe/AI-198-un-core

Conversation

@moetemp

@moetemp moetemp commented Oct 5, 2026

Copy link
Copy Markdown
Owner

This PR merges the native stream chain (#33, #34, #35) into the Option 7 Core, Max's External Workflow Streams prototype, so one Core speaks both stream prototypes.

What changed?

It's one merge commit, plus a plain merge that brings in the channel tests #29 added. Six files conflicted. The bridge activation proto keeps Max's jobs 17 to 20, then the channel's 21 and the native DeliverStreamRecords at 22. The command proto keeps Max's 23 to 28, then the channel's 29 and 30, the native 31 and 32, and WorkflowStreamChannels at 33. wf/mod.rs lists Max's variants, then the native ones, then the channel's. The protos module, the coverage reporter and managed_run.rs keep both sides. The changelog has the channel entry once under Unreleased, followed by the native stream entries.

The native stream files (streams.rs, both stream machines, protosext, replay/mod.rs, integ_helpers.rs) are #35's, byte for byte, and so is the vendored api. Review the stream content there. This PR is where the two prototypes meet.

Part of AI-198 (epic AI-37).

Why?

One Core that speaks both prototypes lets one Python build run every provider, and the Python union pins this head. Since #28 put Max's commits on the same upstream main the native chain uses, this is a single merge. There's no upstream merge and no replay of the native commits.

How did you test it?

Link to a test plan if any -

  • Unit Tests
  • Staging
  • End to End Tests

cargo fmt --check, cargo lint, cargo test-lint, the workspace lib suite, the Core lib suite with --include-ignored and the CI cargo doc check pass. The coverage report lists the same machines and transitions as the earlier all-in-one Core's. The tree was compared with that Core too: the stream files match byte for byte, and the rest differs only where the channel group takes main's form. The integ suites weren't run, since they need a server.

Core decodes the stream commands, events, slices and consumed ranges only when its vendored api carries them. The type maps in the protos crate name the two new commands and events, which are not ignorable.
A consumed stream range reaches lang as its own job. The Rust SDKs have no stream API yet, so they fail the activation rather than drop records the server will not send again.
A workflow consumes and produces stream records through bridge commands 31 and 32. Each machine holds a reissued command to its recorded event on replay, and an unnamed append learns its stream name from the first event the server records.
The cases cover each command reaching the server, its round trip through replay, and which reissues replay refuses and which it accepts.
The poll response carries the records for the task about to run and re-supplies what earlier tasks consumed, keyed by the completion that recorded each range. Core hands them over in a fixed order, gives a consuming task its own activation on replay, and fails a task whose recorded range arrived without its records.
The cases cover live delivery, replay across history pages, the lookahead to the closing completion, missing and mismatched records, and notifications ordered ahead of a stream range.
One Core carries both the external stream runtime and the native stream commands. The shared proto oneofs say which numbers each tree leaves to the other.
@moetemp

moetemp commented Oct 6, 2026

Copy link
Copy Markdown
Owner Author

Paused per the 2026-10-05 design review (the decision record is on the proposal page, section 14). Resumes in phase 2 with the CHASM notification work. The branch stays as a pin.

@moetemp moetemp closed this Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant