fix(brain): correlate Ollama tool results by id, generate schemas, stop faking steer [SC-A4.1] - #75
Merged
Merged
Conversation
…op faking steer
Lane SC-A4, slice 1 of local-brain parity.
Three ways the local Ollama brain quietly disagreed with the host.
1. Tool results were not correlated to tool calls.
The pending result was a single anonymous resolver and the id parameter was
literally discarded:
sendToolResult(_id: string, result: ToolResult): void {
if (this.pending) { ... }
}
Any result satisfied whichever call happened to be waiting. A result
carrying the wrong id was indistinguishable from the right one, and a
duplicate vanished with no trace. The loop awaits calls serially so only one
is ever in flight today, which is what kept this from corrupting runs — but
nothing enforced that, and nothing would have reported it if it broke.
Now keyed by id. An unknown id, or a second result for a call already
settled, emits an error event instead of advancing the loop on a result it
never asked for. close() drains every outstanding waiter rather than one.
Also fixes an ordering fragility: the waiter is now registered BEFORE the
tool_call event is emitted. Registering afterwards only worked because the
consumer resumes on a microtask — a host that replied synchronously would
have found no waiter at all.
2. The advertised tool schemas were hand-written and told the model nothing.
parameters: { type: "object", properties: {}, additionalProperties: true }
Argument names, types, required-ness and bounds were never advertised. The
only description of the real shapes lived in free-text strings that nothing
kept in step with TOOL_DEFINITIONS, and validateToolDefinitionCoverage only
checks name-set equality, not schema fidelity.
ollamaToolSchemas() now generates them from TOOL_DEFINITIONS: per-argument
types, maxLength from maxBytes, integer min/max, an accurate required set,
and additionalProperties:false — the host rejects unknown arguments, so
advertising them as allowed only invited a refusal. A test asserts the
advertised argument set, required set and bounds match the validator for
every tool, so the two cannot drift.
3. control() accepted pause/resume/steer and did nothing.
It returned normally, so the host believed the instruction landed. A dropped
steer then reads to the user as the model ignoring them. This brain runs a
single-pass loop with no interruption point and genuinely cannot honour
these, so it now says so in a visible monologue rather than reporting a
success it did not deliver.
Tests: 5 added — unknown id rejected, duplicate does not advance the loop
twice, close() cannot strand a waiter, control() is visibly honest, and
schemas match TOOL_DEFINITIONS argument-for-argument.
Mutation-checked: restoring the old behaviour (fall back to any waiter when the
id is unknown) fails "a tool result for an unknown id is rejected" — 8 pass /
1 fail; restoring gives 9 / 9.
Gates at this commit:
npm run typecheck exit 0
npm test 927 pass / 0 fail (922 on clean 41a7e26)
Not addressed here, and still open in this lane: chat.ts drops the AbortSignal
on local turns so Ctrl+C is inert until the turn ends; tool_executor.ts uses
blocking spawnSync with no process-group cleanup, so a timed-out test leaves
its children running; and chat.ts re-implements the host loop rather than
sharing code.ts's. The last two are SC-A1's surface and are deliberately not
raced here.
This was referenced Aug 19, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Three ways the local Ollama brain quietly disagreed with the host.
1. Tool results were not correlated to tool calls
The pending result was a single anonymous resolver, and the
idparameter was literally thrown away. Any result satisfied whichever call happened to be waiting.The loop awaits calls serially so only one is in flight today — that is what kept this from corrupting runs. But nothing enforced it, and nothing would have reported it if it broke.
2. The advertised tool schemas told the model nothing
Argument names, types, required-ness and bounds were never advertised. The only description of the real shapes lived in hand-written free text that nothing kept in step with
TOOL_DEFINITIONS.validateToolDefinitionCoverage()only checks name-set equality — not schema fidelity — so the two could drift indefinitely without a test noticing.3.
control()acceptedpause/resume/steerand did nothingIt returned normally, so the host believed the instruction landed. A dropped steer then reads to the user as the model ignoring them.
Contract
Correlation fails closed. Results are keyed by tool-call id. An unknown id — or a second result for a call already settled — emits an
errorevent rather than advancing the loop on a result it never asked for.close()drains every outstanding waiter, not one.Schemas are generated, not written.
ollamaToolSchemas()derives fromTOOL_DEFINITIONS: per-argument types,maxLengthfrommaxBytes, integermin/max, an accuraterequiredset, andadditionalProperties: false— the host rejects unknown arguments, so advertising them as allowed only invited a refusal.Unsupported control is reported, not swallowed. This brain runs a single-pass loop with no interruption point and genuinely cannot honour these. It now says so in a visible monologue instead of reporting a success it did not deliver.
Also fixed: an ordering fragility
The waiter is now registered before the
tool_callevent is emitted. Registering afterwards only worked because the consumer resumes on a microtask — a host that replied synchronously would have found no waiter at all. Nothing pinned that; now the ordering does not matter.Tests
5 added.
close()resolves an outstanding waitercontrol()reports steering is unsupportedTOOL_DEFINITIONSThat last one is the anti-drift guard: it walks all 8 tools and compares the advertised contract against the host validator field by field, so the two cannot diverge again.
Mutation-checked. Restoring the old behaviour — fall back to any waiter when the id is unknown — fails "a tool result for an unknown id is rejected": 8 pass / 1 fail. Restoring gives 9 / 9.
Gates at
c214adc:npm run typechecknpm testBaseline on clean
41a7e261, measured in the same session: 922 / 0.Security notes
TOOL_DEFINITIONSremains the sole validator, and this PR only makes the model's advertised contract match it.additionalProperties: falsenarrows what the model is invited to send; it does not widen what the host accepts.errorevent on a mismatched id is a fail-closed path: the loop does not proceed on an uncorrelated result.Known limits
Deliberately not taken here, because they are SC-A1's surface and racing them would collide:
chat.ts:108drops theAbortSignalon local turns, so Ctrl+C is inert until the turn ends.tool_executor.ts:88uses blockingspawnSyncwith noAbortSignaland no process-group cleanup — a timed-out test leaves its children running, near-certain on Windows viacmd.exe.chat.tsre-implements the host loop rather than sharingcode.ts's, and the two have already diverged (different permission prompt fields, different denial strings, and nofinalVerifyon the REPL path at all).Also still open in this lane:
LocalBrain(the Python path) has no test seam and no turn budget, andhostLoophas no turn or wall-clock limit for either non-Ollama brain.Dependency and merge order
Independent of #72, #73, #74 — branched from
origin/main, no shared files. Any order.Per the integration order this lands after SC-A3 and before SC-A1.