Skip to content

models.infer without tools emits no assistant_reply; harness hosts cannot obtain the run's final text #65

Description

@gregjkal

Summary

A Lua models.infer(prose) call in a section that advertises no tools returns the model's text to Lua, but the engine emits only model_turn_completed for that turn. No assistant_reply event is emitted. A host that follows session events through harness-api therefore never sees the reply text, and since harness-api exposes no run outcome value either (RunOutcome::Completed { final_text } lives only in the private run log), there is no way for a host to obtain the run's final text.

Observed at master b64c1c9d.

Where

  • crates/promptforge-api-runtime/src/execute/tools.rs, accept_infer_completion: reports MODEL_TURN_COMPLETED, optionally request/response debug events, then returns Ok(text) for CompletionResult::Text. Nothing emits assistant_reply.
  • Compare crates/promptforge-api-runtime/src/execute/scheduler/chat.rs, Round::text_reply and accept_chat: the tool-advertising chat path emits assistant_reply (and assistant_tool_calls) for the same kind of turn.
  • crates/harness/sessions/src/session.rs forwards every engine event, so the gap is in the engine, not the harness.

Reproduction

Prompt:

---
name: echo
description: Reply with the first line of the paper
promptforge: 0
models:
  writer:
    min_context: 32768
    description: any chat model
---

# Echo

```lua
models.default("writer")
```

## Only

Reply with exactly the first line of this text and nothing else:

{{ args }}

```lua
return models.infer(prose)
```

Launch it as an agent session through harness-api against any chat model and follow the session events. Observed sequence for the Only section:

section_started, lua_shared_load_started, lua_shared_load_succeeded,
lua_chunk_started, model_turn_completed, lua_chunk_succeeded,
lua_teardown_started, lua_teardown_succeeded, section_finished

then run_succeeded. No assistant_reply, no thinking.

The text does arrive in Lua. Replacing the last chunk with

local r = models.infer(prose)
error("PROBE: infer returned " .. #r .. " bytes: " .. r:sub(1, 60))

fails the run with PROBE: infer returned 66 bytes: Everyone needs this. Field experience at three companies sho, so the completion was non-empty and the model turn completed normally.

For contrast, the same run's tool-advertising section emits model_turn_completed followed by assistant_tool_calls, as expected.

Expected

One of:

  1. accept_infer_completion emits assistant_reply for a text result, the same way Round::text_reply does, so hosts see every model reply regardless of whether tools were advertised.
  2. harness-api exposes the run outcome (final_text) to the host after run_succeeded.

Option 1 also keeps the event stream consistent: today a host cannot tell from events whether a model_turn_completed produced text, an empty reply, or a tool batch unless tools were advertised.

Context

Found while porting papergate (wg21-paperflow) to harness-api. papergate reads the run's last assistant_reply event as its report because no outcome value is exposed; with this gap, a prompt whose final step is a tool-less models.infer can never report. The debug response event does carry the raw completion body, which is the only current workaround.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugThings we want less of

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions