Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,15 @@
- Malformed fixture `citations` or `files` (not an array, a missing or mistyped field, a non-integer offset, an offset outside the text, or citations on a turn with no text: tool calls only, `blocks` with no text block, or empty content) answer a 500 before any content is written: `aimock_invalid_fixture_citation` or `aimock_invalid_fixture_file`. A non-`image/*` file on the OpenAI Responses API answers a 500 with `aimock_unsupported_generated_file`. This is the same on every wire that reads these keys, with or without a misbehavior fault: the fixture is checked on its original turn before any fault applies (only an invalid misbehavior config answers its own 400 or 501 first). A valid fixture under a fault that changes the reply text is served without its `citations`. Only Chat Completions, Anthropic Messages, Gemini and OpenAI Responses read these keys; other wires ignore them.
- Fixture load validation warns about malformed `citations` and `files`, with the same problem description the 500 carries (the 500 message adds an `aimock: invalid fixture` prefix; the warning adds which 500 the requests answer). The fixture still loads.
- `InvalidFixtureOutputError` (the error behind `aimock_invalid_fixture_citation` and `aimock_invalid_fixture_file`) is exported from `@copilotkit/aimock`.
- Opt-in fixture `nonStreamingTimingAllProviders: true` delays non-streaming replies on OpenAI chat, Anthropic, Responses, Gemini/Vertex, Gemini Interactions, Cohere, Bedrock invoke/converse and Ollama; `nonStreamingTiming` is unchanged (OpenAI chat only); error replies are never delayed. On the OpenAI Responses API the wait also counts the stream frames that fixture `citations` and generated `files` add

### Fixed

- Misbehavior faults on Chat Completions, Anthropic Messages, Gemini and the OpenAI Responses API (HTTP and WebSocket): a fixture the wire rejects (generated `files` on Chat Completions or Anthropic, a non-`image/*` file on Responses, malformed `citations` or `files`) is never faulted. It takes the normal path, is journaled as a 500 that keeps its fixture, and consumes no fault `times` budget or seed ordinal; `aimock_misbehavior_total` does not count it. A non-stream Chat Completions fault is journaled before its timing wait.
- Journal accuracy: a client that hangs up during the timing wait of any non-stream Chat Completions reply (faulted or not) is journaled as interrupted, not as served.
- Client disconnects stop delivery (no more chunks or waits) on OpenAI chat, Anthropic, Responses, Gemini/Vertex, Gemini Interactions, Cohere, Bedrock, Ollama, transcription streams and the Responses, Realtime and Gemini Live WebSockets
- Client disconnects on those surfaces are journaled as `interrupted` with `interruptReason: "client aborted"` (HTTP `status: 0` when nothing was sent); aimock-caused teardowns record `"disconnect"`. Journal snapshots of hung-up requests now carry these fields
- WebSocket: a client TCP FIN without a close frame now closes the connection at once (1006) and ends the turn
- Docker image: the base image is pinned to `node:22.23.4-alpine` by digest. Node 22.23.4 fixes CVE-2026-111497 (Brotli decompression memory bug); the 1.46.0 image shipped Node 22.23.3.

## [1.46.0] - 2026-10-20
Expand Down
10 changes: 10 additions & 0 deletions docs/aws-bedrock/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,16 @@ <h2>URL Patterns</h2>
</tbody>
</table>

<p>
If the client hangs up before the reply is complete, aimock stops delivery and journals
the request with <code>interruptReason: "client aborted"</code> (see
<a href="/error-injection#client-disconnects">Client Disconnects</a>). To delay
InvokeModel and Converse replies, set
<code>nonStreamingTimingAllProviders: true</code> with a <code>streamingProfile</code> or
<code>recordedTimings</code> (see
<a href="/streaming-physics#non-streaming-all-providers">All Providers</a>).
</p>

<h2>Request Format</h2>
<p>
Bedrock Claude requests use the Anthropic Messages format. The
Expand Down
10 changes: 10 additions & 0 deletions docs/chat-completions/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,16 @@ <h2>Endpoint</h2>
</tbody>
</table>

<p>
If the client hangs up before the reply is complete, aimock stops delivery and journals
the request with <code>interruptReason: "client aborted"</code> (see
<a href="/error-injection#client-disconnects">Client Disconnects</a>). To delay
<code>stream: false</code> replies, set <code>nonStreamingTiming: true</code> or
<code>nonStreamingTimingAllProviders: true</code> with a <code>streamingProfile</code> or
<code>recordedTimings</code> (see
<a href="/streaming-physics#non-streaming">Non-Streaming Responses</a>).
</p>

<h2>Unit Test: Text Response</h2>
<p>
Using the programmatic API with vitest, register a fixture and assert on the response.
Expand Down
10 changes: 10 additions & 0 deletions docs/cohere/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,16 @@ <h2>Endpoint</h2>
</tbody>
</table>

<p>
If the client hangs up before the reply is complete, aimock stops delivery and journals
the request with <code>interruptReason: "client aborted"</code> (see
<a href="/error-injection#client-disconnects">Client Disconnects</a>). To delay
<code>stream: false</code> replies, set
<code>nonStreamingTimingAllProviders: true</code> with a <code>streamingProfile</code> or
<code>recordedTimings</code> (see
<a href="/streaming-physics#non-streaming-all-providers">All Providers</a>).
</p>

<h2>Key Features</h2>
<ul>
<li>
Expand Down
46 changes: 44 additions & 2 deletions docs/error-injection/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -188,8 +188,50 @@ <h2>Interruption Behavior</h2>
<code>response.interrupted: true</code> and <code>response.interruptReason</code>
</li>
<li>
A client that disconnects from a Claude Messages stream is recorded the same way, with
<code>response.interruptReason: "client aborted"</code>
A client that disconnects is recorded the same way, with
<code>response.interruptReason: "client aborted"</code> (see
<a href="#client-disconnects">Client Disconnects</a>)
</li>
</ul>

<h2 id="client-disconnects">Client Disconnects</h2>
<p>
When the client hangs up before the reply is complete, aimock stops delivery. It sends no
more chunks and skips the remaining waits. This applies to these endpoints, streaming and
non-streaming:
</p>
<ul>
<li>OpenAI Chat Completions, Anthropic Messages and OpenAI Responses</li>
<li>Gemini and Vertex AI, and Gemini Interactions</li>
<li>Cohere v2 chat</li>
<li>Bedrock InvokeModel and Converse, and their streaming variants</li>
<li>Ollama <code>/api/chat</code> and <code>/api/generate</code></li>
<li>Audio transcription streams</li>
<li>WebSocket: OpenAI Responses, OpenAI Realtime and Gemini Live</li>
</ul>
<p>
On WebSocket, a client that closes its TCP connection without sending a close frame is
also a disconnect. aimock closes the connection at once (code 1006, "Connection lost"),
stops the turn and journals it.
</p>
<p>The journal entry records how the reply ended:</p>
<ul>
<li>
<code>response.interrupted: true</code> and
<code>response.interruptReason: "client aborted"</code> when the client left.
</li>
<li>
On HTTP, <code>response.status: 0</code> when no byte of the reply reached the client.
Otherwise the status stays the one aimock sent.
</li>
<li>
<code>response.interruptReason: "disconnect"</code> when aimock tore the response down
itself, for example with the chaos <code>disconnect</code> action or a server-side
destroy. This is not blamed on the client.
</li>
<li>
Fixture interruptions (<code>truncateAfterChunks</code>,
<code>disconnectAfterMs</code>) keep their own reasons.
</li>
</ul>

Expand Down
15 changes: 14 additions & 1 deletion docs/fixtures/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -1049,10 +1049,23 @@ <h2>Fixture Options</h2>
<td>
Also delay <code>stream: false</code> OpenAI Chat Completions replies by the
fixture's <code>streamingProfile</code> or <code>recordedTimings</code> (default
<code>false</code>). See
<code>false</code>). Other providers ignore it (use
<code>nonStreamingTimingAllProviders</code>). See
<a href="/streaming-physics#non-streaming">Non-Streaming Responses</a>
</td>
</tr>
<tr>
<td>nonStreamingTimingAllProviders</td>
<td>boolean</td>
<td>
Delay <code>stream: false</code> replies the same way on OpenAI Chat Completions,
Anthropic Messages, OpenAI Responses, Gemini, Vertex AI, Gemini Interactions,
Cohere, Bedrock InvokeModel and Converse, and Ollama (default <code>false</code>).
Needs <code>streamingProfile</code> or <code>recordedTimings</code>. Error replies
and audio transcription are not delayed. See
<a href="/streaming-physics#non-streaming-all-providers">All Providers</a>
</td>
</tr>
<tr>
<td>chaos</td>
<td>object</td>
Expand Down
10 changes: 10 additions & 0 deletions docs/gemini-interactions/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,16 @@ <h2>Endpoint</h2>
</tbody>
</table>

<p>
If the client hangs up before the reply is complete, aimock stops delivery and journals
the request with <code>interruptReason: "client aborted"</code> (see
<a href="/error-injection#client-disconnects">Client Disconnects</a>). To delay
<code>stream: false</code> replies, set
<code>nonStreamingTimingAllProviders: true</code> with a <code>streamingProfile</code> or
<code>recordedTimings</code> (see
<a href="/streaming-physics#non-streaming-all-providers">All Providers</a>).
</p>

<h2>Quick Start</h2>
<p>Set up a minimal mock that responds to a Gemini Interactions request:</p>

Expand Down
10 changes: 10 additions & 0 deletions docs/gemini/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,16 @@ <h2>Endpoints</h2>
</tbody>
</table>

<p>
If the client hangs up before the reply is complete, aimock stops delivery and journals
the request with <code>interruptReason: "client aborted"</code> (see
<a href="/error-injection#client-disconnects">Client Disconnects</a>). Gemini Live records
the turn the same way. To delay <code>generateContent</code> replies, set
<code>nonStreamingTimingAllProviders: true</code> with a <code>streamingProfile</code> or
<code>recordedTimings</code> (see
<a href="/streaming-physics#non-streaming-all-providers">All Providers</a>).
</p>

<h2 id="model-catalog">Model Catalog</h2>
<p>
The native catalog contains Google IDs and unrecognized IDs from fixtures, in first-seen
Expand Down
10 changes: 10 additions & 0 deletions docs/ollama/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,16 @@ <h2>Endpoints</h2>
</tbody>
</table>

<p>
If the client hangs up before the reply is complete, aimock stops delivery and journals
the request with <code>interruptReason: "client aborted"</code> (see
<a href="/error-injection#client-disconnects">Client Disconnects</a>). To delay
<code>"stream": false</code> replies, set
<code>nonStreamingTimingAllProviders: true</code> with a <code>streamingProfile</code> or
<code>recordedTimings</code> (see
<a href="/streaming-physics#non-streaming-all-providers">All Providers</a>).
</p>

<h2>Embeddings</h2>
<p>
Use the Ollama SDK's <code>embed({ model, input })</code> with a string or array of
Expand Down
10 changes: 10 additions & 0 deletions docs/openrouter-chat/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,16 @@ <h2>Endpoints</h2>
</tbody>
</table>

<p>
If the client hangs up before the reply is complete, aimock stops delivery and journals
the request with <code>interruptReason: "client aborted"</code> (see
<a href="/error-injection#client-disconnects">Client Disconnects</a>). To delay
<code>stream: false</code> replies, set <code>nonStreamingTiming: true</code> or
<code>nonStreamingTimingAllProviders: true</code> with a <code>streamingProfile</code> or
<code>recordedTimings</code> (see
<a href="/streaming-physics#non-streaming">Non-Streaming Responses</a>).
</p>

<h2>Point the OpenAI SDK at aimock</h2>
<p>
The dominant real-world integration path is the OpenAI SDK with its
Expand Down
10 changes: 10 additions & 0 deletions docs/responses-api/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,16 @@ <h2>Endpoints</h2>
</tbody>
</table>

<p>
If the client hangs up before the reply is complete, aimock stops delivery and journals
the request with <code>interruptReason: "client aborted"</code> (see
<a href="/error-injection#client-disconnects">Client Disconnects</a>). The WebSocket
transport records the turn the same way. To delay HTTP <code>stream: false</code> replies,
set <code>nonStreamingTimingAllProviders: true</code> with a
<code>streamingProfile</code> or <code>recordedTimings</code> (see
<a href="/streaming-physics#non-streaming-all-providers">All Providers</a>).
</p>

<h2>Unit Test: HTTP SSE Text Response</h2>

<div class="code-block">
Expand Down
61 changes: 59 additions & 2 deletions docs/streaming-physics/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -195,9 +195,65 @@ <h2 id="non-streaming">Non-Streaming Responses</h2>
<p>
<code>nonStreamingTiming</code> has an effect only on OpenAI Chat Completions, and only
when the fixture also sets <code>streamingProfile</code> or <code>recordedTimings</code>.
Non-streaming responses from other providers are not delayed.
Non-streaming responses from other providers are not delayed by it.
</p>

<h3 id="non-streaming-all-providers">All Providers</h3>
<p>
To delay <code>stream: false</code> replies on other providers too, set
<code>nonStreamingTimingAllProviders: true</code> on the fixture. It uses the same formula
as <code>nonStreamingTiming: true</code>, and it also needs a
<code>streamingProfile</code> or <code>recordedTimings</code>. It applies to these
non-streaming endpoints:
</p>
<ul>
<li>OpenAI Chat Completions (the same delay as <code>nonStreamingTiming: true</code>)</li>
<li>Anthropic Messages</li>
<li>OpenAI Responses</li>
<li>Gemini and Vertex AI <code>generateContent</code></li>
<li>Gemini Interactions (with <code>stream: false</code>)</li>
<li>Cohere v2 chat</li>
<li>Bedrock InvokeModel and Converse</li>
<li>Ollama <code>/api/chat</code> and <code>/api/generate</code></li>
</ul>
<p>
Audio transcription replies are not delayed. <code>nonStreamingTiming</code> is unchanged:
it still applies to OpenAI Chat Completions only. Setting
<code>nonStreamingTimingAllProviders: false</code> does not turn off
<code>nonStreamingTiming: true</code>.
</p>

<div class="code-block">
<div class="code-block-header">
fixtures/slow-non-streaming-all.json <span class="lang-tag">json</span>
</div>
<pre><code>{
<span class="key">"fixtures"</span>: [
{
<span class="key">"match"</span>: { <span class="key">"userMessage"</span>: <span class="str">"think carefully"</span> },
<span class="key">"response"</span>: { <span class="key">"content"</span>: <span class="str">"Let me think about this..."</span> },
<span class="key">"streamingProfile"</span>: { <span class="key">"ttft"</span>: <span class="num">800</span>, <span class="key">"tps"</span>: <span class="num">50</span> },
<span class="key">"nonStreamingTimingAllProviders"</span>: <span class="kw">true</span>
}
]
}</code></pre>
</div>

<h3 id="non-streaming-errors-and-disconnects">Errors and Disconnects</h3>
<ul>
<li>
Error replies (a fixture <code>error</code> response) are never delayed. aimock sends
them at once, with the fixture status.
</li>
<li>
If the client disconnects during the wait, aimock writes nothing. The journal entry
records <code>response.interrupted: true</code>,
<code>response.interruptReason: "client aborted"</code> and
<code>response.status: 0</code>. See
<a href="/error-injection#client-disconnects">Client Disconnects</a>.
</li>
</ul>

<h2>Interaction with <code>latency</code></h2>
<ul>
<li>
Expand All @@ -216,7 +272,8 @@ <h2>Interaction with <code>latency</code></h2>
<code>latency</code> and <code>--latency</code> are gaps between SSE chunks only. They
never delay a non-streaming response, including the fallback case above. To delay
<code>stream: false</code> replies, set <code>streamingProfile</code> and
<code>nonStreamingTiming: true</code> (see
<code>nonStreamingTiming: true</code> (OpenAI Chat Completions) or
<code>nonStreamingTimingAllProviders: true</code> (see
<a href="#non-streaming">Non-Streaming Responses</a>).
</li>
</ul>
Expand Down
7 changes: 7 additions & 0 deletions docs/transcription/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,13 @@ <h2>Endpoint</h2>
</tbody>
</table>

<p>
If the client hangs up during a streamed transcription (<code>stream=true</code>), aimock
stops the stream and journals the request with
<code>interruptReason: "client aborted"</code> (see
<a href="/error-injection#client-disconnects">Client Disconnects</a>).
</p>

<h2>Unit Test: Simple Transcription</h2>
<p>
Using the programmatic API with vitest, register a fixture and assert on the response.
Expand Down
10 changes: 10 additions & 0 deletions docs/vertex-ai/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,16 @@ <h2>Endpoints</h2>
</tbody>
</table>

<p>
If the client hangs up before the reply is complete, aimock stops delivery and journals
the request with <code>interruptReason: "client aborted"</code> (see
<a href="/error-injection#client-disconnects">Client Disconnects</a>). To delay
<code>generateContent</code> replies, set
<code>nonStreamingTimingAllProviders: true</code> with a <code>streamingProfile</code> or
<code>recordedTimings</code> (see
<a href="/streaming-physics#non-streaming-all-providers">All Providers</a>).
</p>

<h2>URL Pattern Difference</h2>
<p>
The key difference between consumer Gemini and Vertex AI is the URL routing. Consumer
Expand Down
7 changes: 7 additions & 0 deletions docs/websocket/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,13 @@ <h2>Endpoints</h2>
</tbody>
</table>

<p>
On OpenAI Responses, OpenAI Realtime and Gemini Live, if the client closes the connection
during a reply, aimock stops sending and journals the turn with
<code>interruptReason: "client aborted"</code> (see
<a href="/error-injection#client-disconnects">Client Disconnects</a>).
</p>

<h2 id="openai-live">OpenAI GPT-Live</h2>
<p>
GPT-Live uses a primary WebSocket upgrade: <code>GET /v1/live/sessions</code>. Connect to
Expand Down
1 change: 1 addition & 0 deletions etc/aimock-a2a.import.api.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions etc/aimock-a2a.require.api.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading