Skip to content

fix(tools): persist tool output reference labels and keep keys unique across runs - #629

Open
Shabonasar wants to merge 2 commits into
LibreChat-AI:mainfrom
Shabonasar:fix/stable-tool-output-ref-labels
Open

Shabonasar wants to merge 2 commits into
LibreChat-AI:mainfrom
Shabonasar:fix/stable-tool-output-ref-labels

Conversation

@Shabonasar

@Shabonasar Shabonasar commented Oct 20, 2026 •

Copy link
Copy Markdown

Summary

With tool output references on, ToolNode labels each tool result with [ref: tool<i>turn<n>] in the request it sends live. The label was shown only while the key was in the current run's registry, and every run restarted at turn0. As a result, a restored history lost the labels: the first request of each later turn missed the prompt cache, and Claude dropped thinking blocks bound to the old prefix. Keys also repeated between turns, so {{tool0turn0}} from an old turn silently resolved to the current turn's output. With this PR the label is the stored name of the output: it is saved with the result, restored with the history, and turn numbers continue across runs.

Fixes #627

Related to LibreChat-AI/LibreChat#17102. Base: main, no dependency on other PRs. LibreChat needs no change: the fields ride in the stored tool_call part.

How it works

live:     ToolNode key tool0turn7 -> ON_RUN_STEP_COMPLETED.tool_call { output, ref, unresolved_refs? } -> host stores it
restore:  formatAgentMessages: ref -> ToolMessage.additional_kwargs._refKey
          Run: turn counter = highest restored turn + 1   (keys never repeat)
request:  annotateMessagesForLLM labels by _refKey, no registry lookup, after truncation (byte-equal to live)
  • The completion event carries ref / unresolved_refs for direct tools, event-driven tools (batch and early completion) and eager executions.
  • An old key is not in the registry, so it stays literal with the existing [unresolved refs: ...] warning.
  • Handoff agents share one registry (one seed); subagents have their own run and history.
  • Tests that expected "no label when the key is missing from the registry" now expect the label.
  • Not changed: a placeholder in the same parallel batch as the call it names stays literal without a warning.

Type of change

  • Bug fix

Testing

Tested environments/configuration:

The table comes from an earlier run on a head with the same code. Real Anthropic API through LibreChat, 6 turns, adaptive thinking, code interpreter running code, chained bash_tool calls, MCP tool, restart after turn 3, turn 5 references a key from turn 1. "series" is the LibreChat series plus agents #613, #614, #615, #619 (LibreChat-AI/LibreChat#17102); dev is LibreChat dev with agents main. "before" used a 1h cache TTL, "after" 5m.

model LibreChat agents cache hit misses dropped thinking blocks 400
Haiku 5.5 series before 87.9% 4 28 0
Haiku 5.5 series this PR 92.9% 0 0 (2nd run: 1, from #618) 0
Sonnet 5.5 series before 86.1-88.4% 3 10 0
Sonnet 5.5 series this PR 90.8% 0 0 0
Haiku 5.5 dev before 88.6% 5 0 0
Haiku 5.5 dev this PR 87.5% 5 0 0

With the series, keys run tool0turn0...tool0turn8 without repeats, and the turn-5 probe returns [unresolved refs: tool0turn0] instead of a wrong output. On dev the labels are now stable too; its misses come from other restore differences that the series fixes.

Automated tests:

npx jest src/messages src/llm src/graphs src/tools src/agents src/summarization
npx tsc --noEmit

Risk / compatibility

  • Only with tool output references on. A restored history now carries the labels the live request had. A history stored before this change has no ref and stays unlabelled, as it is today.

Checklist

  • I reviewed my own changes
  • Relevant tests have been added or updated
  • Existing relevant tests pass
  • The change does not introduce new warnings or errors
  • Required dependency changes have been merged/published (none)

… across runs

[ref: toolNturnM] labels were added only when the key was live in the current
run's registry, and every run restarted its turn counter at 0. A restored
history therefore had no labels (the first labelled result of each later turn
changed the prefix and missed the provider cache), and keys repeated between
turns, so {{tool0turn0}} from an old turn silently resolved to the current one.

- ON_RUN_STEP_COMPLETED tool_call now carries `ref` and `unresolved_refs`
  (direct, early, eager and batch completions); the content aggregator keeps them
- formatAgentMessages restores them as `_refKey` / `_unresolvedRefs`
- annotateMessagesForLLM labels by `_refKey` without a registry lookup
- Run seeds the registry turn counter past the highest restored key, so a
  placeholder naming an older turn is reported as unresolved

Fixes LibreChat-AI#627
… across runs

[ref: toolNturnM] labels were added only when the key was live in the current
run's registry, and every run restarted its turn counter at 0. A restored
history therefore had no labels (the first labelled result of each later turn
changed the prefix and missed the provider cache), and keys repeated between
turns, so {{tool0turn0}} from an old turn silently resolved to the current one.

- ON_RUN_STEP_COMPLETED tool_call now carries `ref` and `unresolved_refs`
  (direct, early, eager and batch completions); the content aggregator keeps them
- formatAgentMessages restores them as `_refKey` / `_unresolvedRefs`
- annotateMessagesForLLM labels by `_refKey` without a registry lookup
- Run seeds the registry turn counter past the highest restored key, so a
  placeholder naming an older turn is reported as unresolved

Fixes LibreChat-AI#627
@Shabonasar

Copy link
Copy Markdown
Author

Updated to current main and reduced to simplify the review: no dependency on other PRs, +546 -> +480 lines.

  • Code unchanged (one block in stream.ts written shorter, same behaviour); tests consolidated: the separate ToolNode.persistedReferences.test.ts is folded into the existing ToolNode.outputReferences, formatAgentMessages.tools and toolOutputReferences suites.
  • Checked: SDK suites (4803 passed, the 3 llm.spec.ts suites need API keys); all five SDK PRs together 6987 passed; LibreChat agent suites against all five.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tool output reference labels missing from restored history: cache misses and wrong {{toolNturnM}} substitution

1 participant