Replayed tool calls are sent in neutral {id, name, arguments} shape; OpenAI-protocol upstreams reject the turn with 400

[gregjkal]

Summary

When a run replays an earlier assistant turn that contained tool calls, the engine renders each tool call as the provider-neutral object {id, name, arguments} with arguments as a JSON object. The gateway forwards that message unchanged to an OpenAI-protocol endpoint, which rejects the request with 400. The first tool-calling turn succeeds; the turn that carries the tool results fails. Any prompt whose section calls a tool and then continues cannot complete through the harness against an OpenAI-protocol upstream.

Observed at master b64c1c9d.

Where

• crates/promptforge/lua/src/projection.rs, wire_message: builds tool_calls as

  { "id": "...", "name": "...", "arguments": { ... } }

  The doc comment describes this as the deliberate "provider-neutral" shape.
• crates/gateway/protocol and crates/gateway/app: nothing rewrites inbound messages[].tool_calls before forwarding. dialect.rs only normalizes model output (tool code fences into tool_calls), not the conversation history sent upstream.

Reproduction

1. Run a --no-default-features gateway with one [[endpoint]] using protocol = "openai" pointing at an OpenAI-compatible provider (observed via OpenRouter routing to openai/gpt-4o-mini, upstream provider reported as OpenAI and Azure) and one [[model]] referencing it.
2. Through harness-api, launch an agent session whose first section advertises a tool and calls models.infer(prose). The model calls the tool. The harness dispatches it and issues the follow-up model turn.
3. The follow-up turn fails with Model turn failed in agent ...; run failed: non-success backend status 400. The gateway responds with:

   {"error":{"code":"upstream_client_error","message":"upstream returned 400","type":"invalid_request_error"}}

The exact request body captured with a logging proxy between the harness and the gateway:

{"messages":[
  {"role":"user","content":"Identify every H2 section in this paper: ..."},
  {"role":"assistant","content":"","tool_calls":[
    {"arguments":{"end_line":5,"name":"Motivation","start_line":4},"id":"call_5I2iA7ZFVozl8V3MzCjdewMn","name":"add_section"},
    {"arguments":{"end_line":8,"name":"Design","start_line":7},"id":"call_h5sc0boZdR3srhwkWjHSurnl","name":"add_section"}]},
  {"role":"tool","tool_call_id":"call_5I2iA7ZFVozl8V3MzCjdewMn","content":"added Motivation"},
  {"role":"tool","tool_call_id":"call_h5sc0boZdR3srhwkWjHSurnl","content":"added Design"}],
 "model":"pg-remote","stream":true,"stream_options":{"include_usage":true},"tool_choice":"auto",
 "tools":[{"type":"function","function":{"name":"add_section","description":"Add a section with its line range","parameters":{...}}}]}

Sending the same conversation with the tool calls in OpenAI wire shape returns 200 from the same gateway and model:

{"role":"assistant","content":"","tool_calls":[
  {"id":"call_abc","type":"function","function":{"name":"add_section","arguments":"{\"end_line\":5,\"name\":\"Motivation\",\"start_line\":4}"}}]}

Expected

Replayed tool calls reach an openai-protocol endpoint as
{"id", "type": "function", "function": {"name", "arguments": "<JSON string>"}}.

Either the engine's projection emits the wire shape for the provider, or the gateway converts the neutral shape per endpoint protocol before forwarding. vLLM's OpenAI server validates tool_calls with the same schema, so self-hosted models are affected as well as hosted ones.

Diagnostics note

The harness surfaces only non-success backend status 400; the body is hidden by design (F5). The gateway logs nothing about the exchange even at RUST_LOG=debug. Finding this took a logging proxy. Some opt-in path to the upstream 400 body, or a gateway debug log line naming the upstream error code, would have shortened the diagnosis considerably.

Context

Found while porting papergate (wg21-paperflow) to harness-api. The runbook crates/papergate/TESTING.md on branch papergate-harness-api records the observation.