You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix(toolcall-schema-fuzz): apply timeoutMs to reading the response body - #144
timeoutMs in the toolcall_schema_fuzz solver now covers reading the response body, not just receiving headers. A server that sends headers and then stalls produces a 408 ModelError instead of hanging the sample.
What changed?
requestCompletion now executes the request, checks the status, and parses the JSON body inside one withTimeout boundary, and it returns the parsed body. The 408 ModelError conversion and the 429/5xx retry behaviour are unchanged.
solver.test.ts: the timeout test now runs against two fake servers, one that never responds and one that sends 200 headers and then stalls the body.
Why?
Found by Devin Review on openrouter-web#50308, which vendors #143. Without the stalled-body case passing, timeoutMs stops applying as soon as headers arrive. I confirmed the new stalled-body test fails without this fix.
Reviewer focus
Retries now re-run the full request and body read, since the body read is inside retrySalted.
Checklist
Tests cover changed behavior
Public API or configuration changes are backward compatible, or the break is documented
Benchmark changes document dataset provenance and licensing
No credentials, private results, or restricted dataset contents are included
I'll fix CI failures and address comments from users with write access that start with 'Devin'.
Disable automatic comment, CI, and merge conflict monitoring
Original prompt from Abhinav
SYSTEM:
<latest_message>
Abhinav Pola (U090K0G7JF3) [ts=1791572889.759909]: @Devin move the tool call fuzz dataset to the public repo and sync it back to the subtree
</latest_message>
The <latest_message> is the message that you should use to guide your goals + task for this session, and you should use the rest of the slack thread as context.
A [ts=...] marker on a Slack message is that message's timestamp. To act on a specific message with the slack tool (e.g. adding an emoji reaction via the reaction command), pass that value as timestamp along with the Channel ID — no extra lookup call is needed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
timeoutMsin thetoolcall_schema_fuzzsolver now covers reading the response body, not just receiving headers. A server that sends headers and then stalls produces a 408ModelErrorinstead of hanging the sample.What changed?
requestCompletionnow executes the request, checks the status, and parses the JSON body inside onewithTimeoutboundary, and it returns the parsed body. The 408ModelErrorconversion and the 429/5xx retry behaviour are unchanged.solver.test.ts: the timeout test now runs against two fake servers, one that never responds and one that sends 200 headers and then stalls the body.Why?
Found by Devin Review on openrouter-web#50308, which vendors #143. Without the stalled-body case passing,
timeoutMsstops applying as soon as headers arrive. I confirmed the new stalled-body test fails without this fix.Reviewer focus
retrySalted.Checklist
Link to Devin session: https://openrouter.devinenterprise.com/sessions/72a3f13c847645c5b06306d88cec089d
Open in Devin Desktop: https://openrouter.devinenterprise.com/desktop/session/72a3f13c847645c5b06306d88cec089d?variant=devin
Requested by: @abhinav-pola