Repository navigation
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Long public query composition chains can overflow the stack during execution.
1 open issue
What changed in this PR
Introduces reusable Kyo query plans to batch requests across composed relations while preserving output positions.
Changes:
- Groups independent requests by dependency stage.
- Migrates interpreter execution to query plans and documents compatibility.
- Adds batching regression tests and test dependencies.
| File | Description |
|---|---|
| kyo-batch/src/test/scala/decrel/reify/KyoBatchSpec.scala | Tests batching, ordering, composition, and plan reuse. |
| kyo-batch/src/main/scala/decrel/reify/KyoBatchQuery.scala | Implements query composition and staged execution. |
| kyo-batch/src/main/scala/decrel/reify/kyoBatch.scala | Integrates query plans into the interpreter. |
| kyo-batch/README.md | Documents batching semantics and migration. |
| build.sbt | Adds Kyo module test dependencies. |
Review effort: Balanced
yoohaemin
force-pushed
the
refactor/typed-interpreter-zip
branch
from
November 2, 2026 13:00
afadfe5 to
953aadd
Compare
yoohaemin
force-pushed
the
fix/kyo-composed-batching
branch
from
November 2, 2026 13:00
3d27c17 to
800b44b
Compare
yoohaemin
force-pushed
the
fix/kyo-composed-batching
branch
from
November 2, 2026 13:34
800b44b to
ec1ecf0
Compare
yoohaemin
force-pushed
the
fix/kyo-composed-batching
branch
from
November 2, 2026 13:48
ec1ecf0 to
d056c32
Compare
| type State = pending.State | ||
| val initial = pending.initial | ||
| val requests = pending.requests | ||
| def resume(state: State): KyoBatchQuery[S, B] = defer(pending.resume(state)).flatMap(f) |
| } | ||
| val groups = fetches.groupBy(_.source).toVector | ||
| Kyo | ||
| .foreach(groups) { case (source, requests) => |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Kyo datasource implementations handled Batch separately for each relation invocation. Zipped branches and overlapping derived groups could not share requests.
Build a reusable query plan with structured datasource/key requests.
foreachcombines independent plans; native grouped sources deduplicate keys and reconstruct result positions. Dependent stages resume after inputs arrive, with trampolined composition and balanced traversal.startingFrom/startingFromManypreserve handled Kyo execution. NewtoBatch/toBatchManyleave Batch to the caller, allowing separate traversals to share an outer handler. Register aKyoBatchSourceonce and pass it to datasource constructors to share its nativetoBatchhandle too. Existing callback signatures remain supported. No persistent cache or concurrent datasource execution guarantee is added.Low-level reification now returns a plan with
.toBatch, rather than a direct effect. Ordinary custom effects useKyoBatchQuery.liftand execute per occurrence. Seekyo-batch/README.mdfor migration and native interoperability.The optional ZQuery interpretation is on
feat/kyo-zquery(no new PR opened), using the same proofs and shared sources without duplicate callbacks. This PR has no ZQuery runtime dependency.Validation: 21 core regressions pass on JVM; Scala.js composition and collection suites pass separately. Coverage includes native source sharing, source isolation, duplicate/empty inputs, nested groups, dependent stages, typed failures, ordinary lifted effects, reruns, 4,096-input traversal and 100,000-deep composition. Changed Scala sources are formatted.