Repository navigation
Accept reusable evaluation closures and validate run options - #139
Merged
Stephen Belanger (Qard) merged 1 commit intoOct 28, 2026
Merged
Stephen Belanger (Qard) merged 1 commit into
Stephen Belanger (Qard) merged 1 commit into
Conversation
Stephen Belanger (Qard)
merged commit Oct 28, 2026
6247c49
into
refactor/eval-case-execution
2 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Async functions and captured closures now implement
Taskdirectly, and(name, callback)tuples implementScorerwith borrowed or owned names. Thetaskandscoreridentity helpers supply lifetime/type context for boxed futures without adding wrapper state. RemoveAsyncFnTaskandSimpleAsyncFnTask; callbacks can borrow their input, hooks and scorer arguments across awaits and be reused concurrently without cloning user values. Update the async example, eval doctest and active recording integration to this shape.Add
EvalOpts::validatewith typedEvalConfigErrorvalues for zero concurrency, blank scorer names and duplicate names. The current evaluator calls it before source access, login-dependent setup or recording. Builtin, custom and tuple scorers share the same validation. Existing synchronous adapters remain transitional until the normalized run context supplies a managed blocking executor.Validation: 365 unit/integration tests and 10 doctests passed with all features, plus strict all-target Clippy, rustdoc, formatting and diff checks. Five new public tests cover borrowed functions, captured concurrent callbacks, both name forms and validation before side effects. Existing custom trait implementations remain valid.
Rust 1.86: 11 focused eval tests passed, both examples compiled, and both eval doctests passed in an isolated checkout with a Rust-compatible dependency resolution. The normal local resolution fails its MSRV check on both this source and parent #138 before compiling the SDK: bon/ICU/time require 1.88 and uuid requires 1.89. No dependency files changed; release preparation still needs to reconcile that resolution with the declared minimum. Older scorer fixtures now bind expected strings explicitly so their lifetimes compile on 1.86.
Stacked on #138. Normalized definitions/case sources, eager local/recorded scheduling, managed blocking callbacks and recorder/reporter separation follow. Planning documents are excluded.