Skip to content

Add tool-calling scoping note to the models CI-stub section #621

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
heskew wants to merge 2 commits into
base: main
Choose a base branch
from
+7 −1
Open

Add tool-calling scoping note to the models CI-stub section #621

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
bb79406
Add tool-calling scoping note to the CI-stub section
heskew Aug 7, 2026
1e9a4d5
Split the scoping notes into bold-led sub-paragraphs
heskew Aug 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion reference/models/local-development.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,12 @@ models.registerBackend(

The stub still exercises routing, accounting, and [analytics](./analytics) — only the inference itself is faked.

Two scoping notes. `defineBackend` derives `generate` from a supplied `generateStream` (by draining the stream), but not the reverse — if the suite calls `generateStream()`, give the stub a `generateStream` implementation. And a backend registered under `'generative'` serves only generation: `embed()` resolves the `'embedding'` registry, so embedding tests need their own stub registered with `models.registerBackend('embedding', 'default', …)`.
Three scoping notes:

**Streaming.** `defineBackend` derives `generate` from a supplied `generateStream` (by draining the stream), but not the reverse — if the suite calls `generateStream()`, give the stub a `generateStream` implementation.

**Embedding.** A backend registered under `'generative'` serves only generation: `embed()` resolves the `'embedding'` registry, so embedding tests need their own stub registered with `models.registerBackend('embedding', 'default', …)`.

**Tool calling.** `defineBackend` defaults the `tools` capability to `false`, and a `generate()` call that declares tools [automatically requires that capability](./routing#capability-routing) — so a tool-calling application fails against this stub up front, before its `generate()` ever runs. Tool-calling tests need a stub defined with `tools: true` whose `generate()` returns the tool-call sequence the test expects; leave the flag off a simple reply stub like this one, where the up-front failure is the honest signal.

As with any registered backend, register the stub during component initialization (for example, in `handleApplication`) rather than at a test file's top level: each worker thread keeps its own registry, so registration must run in every thread that serves requests — see [`registerBackend()`](./backends#registerbackend).