[jpr5]

Closes #549

Summary

This PR adds raw passthrough fixtures. A fixture response of the form { raw: { wire, events?, json?, status?, headers?, done? } } sends provider events or JSON exactly as written. aimock adds only the framing of the named wire. No overrides, reasoning gating, usage synthesis, provider shaping or drift checks apply to a raw fixture.

Per-wire framing. Each wire uses the same bytes that the typed handlers write for that wire:

• Named SSE (event: <type> then data: <json>): openai-responses, anthropic, cohere.
• Data SSE (data: <json>): openai-chat, gemini, gemini-interactions. On openai-chat, the stream ends with data: [DONE] unless raw.done is false.
• NDJSON: ollama (/api/chat) and ollama-generate (/api/generate). These are two wires because the frames of the two routes have different shapes.
• AWS event stream: bedrock-invoke wraps each event as a chunk frame with base64 bytes. bedrock-converse uses the single key of each event as the frame's event type and its value as the payload.

That gives 10 wires: openai-responses, openai-chat (including the OpenRouter and Azure routes), anthropic, gemini (including Vertex), gemini-interactions, cohere, ollama, ollama-generate, bedrock-invoke and bedrock-converse. The router offers a raw fixture only to the handler for its raw.wire. The handler supplies the wire, never the request body. When a raw fixture matches a request in every way except its wire, and no other fixture matches, the server log names both wires instead of returning a silent 404.

Stream vs JSON. A streaming request gets events and a non-streaming request gets json. If the body that the request needs is missing, aimock answers with the native 500 error of that wire and the code aimock_raw_mode_unavailable.

Status and headers. status defaults to 200 and must be 200-599, except 204 and 304. headers replace the defaults case-insensitively. If headers has no Content-Type, a stream gets the wire's own type and a non-streaming reply gets application/json. aimock rejects headers that it must set itself, because it frames the body.

Error codes. An invalid raw fixture gets a 500 in the wire's own error shape with the code aimock_raw_fixture_invalid. Examples are a factory that returns status: 99, or a factory that returns a raw fixture for another wire. On Gemini, Ollama and OpenRouter the code is prefixed to the message.

Validation. --validate-on-load, addFixturesFromJSON and POST /__aimock/fixtures all check raw fixtures with one shared checker:

• the wire is known, and events or json is present;
• the events are plain JSON objects;
• the per-wire type and single-key rules pass;
• status and the header names and values are valid;
• done is a boolean, and only on openai-chat;
• customToolCalls and responsesBlocks are not next to raw;
• the fixture does not set its own misbehavior.

Unknown keys inside raw and an empty events are warnings. chunkSize and openRouterProcessing on a raw fixture are warnings, because they are ignored. The same check runs again when a fixture is served, so fixtures added with on(), addFixture() or factories are checked too.

Misbehavior skip. Model misbehavior faults skip raw fixtures. aimock skips a server-level fault, serves the raw reply, and records the fault as skipped in the journal. An X-AIMock-Misbehavior header returns 501 aimock_misbehavior_not_applicable. A raw fixture that sets its own misbehavior is an invalid fixture: load validation reports it, and when it is served it gets aimock_raw_fixture_invalid.

Timing rules. latency, streamingProfile, recordedTimings and replaySpeed pace streamed raw events in the same way as typed frames. truncateAfterChunks, disconnectAfterMs and chaos also apply, and each raw event counts as one chunk. A non-streaming raw reply follows nonStreamingTiming like a typed fixture. nonStreamingTiming: true delays the reply on OpenAI Chat Completions only. nonStreamingTimingAllProviders: true delays it on every raw wire. aimock sizes the wait from the served JSON.

Error replies. A raw reply with a status of 400 or more is never delayed, streaming or not. If the reply has a json, a streaming request gets that JSON instead of events, because provider APIs answer errors that way. Error replies are recorded in the journal.

WS Responses. /v1/responses over WebSocket answers a raw fixture with an error event with the code aimock_raw_mode_unavailable and the message "raw fixtures are not yet supported on the WebSocket transport". Before this change, it answered with the generic unknown-type error.

OpenRouter failover. In an OpenRouter models[] fallback chain, an openai-chat raw fixture with a status of 400 or more fails over to the next model. A typed error fixture with the same status does the same.

Compatibility

• Additive only. The feature is opt-in. A response is raw only if raw is a non-null, non-array object and no other response type's marker key (content, toolCalls, error, json, and so on) is present. A typed fixture with a stray raw key is still served, and validated, exactly as before.

• Typed fixtures unchanged. Existing typed fixtures produce output byte-identical to main: an offline probe compared 321 cases (status, headers except date, and body) on the pre-review build, and the no-regression snapshot plus the feat: stop delivery on client disconnect across providers, and opt-in non-streaming timing for all providers #579 byte goldens pass at the final head. The no-regression suite (raw-no-regression.test.ts, with a snapshot) is green on main and on this branch.

• API report. The API report changes are additions only:

  • new exported types RawResponse, RawWireId and WireId (WireId was already declared, and is now exported);
  • RawResponse is added to the FixtureResponse and FixtureFileResponse unions;
  • a new @internal wire field on MatchOptions.

  isRawResponse is internal and not exported.

Red-green evidence

Each wire was reproduced against a real aimock server, with and without --validate-on-load, and with the real provider SDK tests. Before the change, every raw fixture request got a 500 with the body Fixture response did not match any known type. After the change, the authored events or JSON are served with the wire's framing.

Wire	Before (curl, stream / non-stream)	After (curl, stream / non-stream)	SDK tests before	SDK tests after
openai-responses	500 / 500, "did not match any known type"	200 text/event-stream / 200 application/json	16 failed, 1 passed (17)	17 passed (17)
openai-chat	500 / 500, "did not match any known type"	200 text/event-stream, ends data: [DONE] / 200 application/json	14 failed (14)	14 passed (14)
anthropic	500 / 500, "did not match any known type"	200 text/event-stream / 200	9 failed (9)	9 passed (9)
gemini, gemini-interactions	500 / 500 on both, "did not match any known type"	200 / 200 on both	16 failed (16)	16 passed (16)
cohere	500 / 500, "did not match any known type"	200 text/event-stream / 200	21 failed (21), cohere and ollama together	21 passed (21), cohere and ollama together
ollama, ollama-generate	/api/chat 500, /api/generate 500, "did not match any known type"	/api/chat 200, /api/generate 200, application/x-ndjson	(see cohere row)	(see cohere row)
bedrock-invoke, bedrock-converse	500 / 500 on both, "did not match any known type"	200 application/vnd.amazon.eventstream / 200 on both	13 failed (13), the SDK reported UnknownError	13 passed (13)

The split between ollama and ollama-generate has its own red-green. Before the split, the test "/api/generate gets the ollama-generate fixture in generate shape" failed (1 failed, 1 passed). After the split, it passes (2 passed).

The full suite, typecheck, lint, format check, build, goldens/contract tests and the API report check all pass on this branch.

Not in this PR

• Raw fixtures over WebSocket Responses, and record.raw (recording raw fixtures). Both are in spec PR3.
• Bedrock exception framing for raw error replies.

Spec: raw SSE/JSON passthrough fixtures (Notion)

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/@copilotkit/aimock@580

commit: 0a1fe0e