Skip to content

Run synchronous evaluation callbacks on the managed blocking pool - #144

Closed
Stephen Belanger (Qard) wants to merge 1 commit into
refactor/eager-eval-runsfrom
refactor/eval-blocking-callbacks
Closed

Stephen Belanger (Qard) wants to merge 1 commit into
refactor/eager-eval-runsfrom
refactor/eval-blocking-callbacks

Conversation

@Qard

Copy link
Copy Markdown
Contributor

Synchronous evaluation callbacks previously ran inline on the selected async worker, so a callback waiting for a span write could block the worker responsible for delivering it. Evaluation runs now invoke synchronous tasks and scorers on their selected runtime's managed blocking pool. Built-in scorers use that path as well; async functions and closures retain their borrowing contracts.

The task/scorer traits expose synchronous bodies and an explicit dispatch marker, with defaults for existing async implementations. Named scorer wrappers preserve dispatch as well as validated names. Cases and outputs use shared ownership only while invocations are active, then return ownership to the result without adding Clone or serialization bounds. Shared hook metadata, tags, expected values and recording spans remain available to synchronous tasks.

Dropping an invocation aborts queued blocking work. An already-running synchronous callback retains its owned data until it returns; run timeout revokes its recording permission. Blocking panics retain the Tokio JoinError as a case error, allowing later cases and recording finalization to continue. No new public cancellation policy is introduced.

This continues stage 8, stacked on #143. Experiment-backed recording, dataset case sources and remaining CLI adapters are next.

Validation:

  • 392 unit/integration tests and 12 doctests passed with all features.
  • Strict all-target/all-feature Clippy, strict rustdoc, formatting and diff checks passed.
  • Rust 1.86 with an isolated compatible dependency resolution: 54 focused eval tests, both examples and 4 eval doctests passed. No dependency files changed.
  • Regression tests cover single-thread executor progress, selected runtime/thread identity, concurrent synchronous scorers with ordered results, non-cloneable data and shared hooks, queued cancellation, blocking panic, input lifetime after shutdown timeout, revoked write authority and a synchronous callback waiting for recording delivery.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant