Repository navigation
Send a search to the search tool, never to the browser - #841
Closed
guidovizoso wants to merge 6 commits into
Closed
guidovizoso wants to merge 6 commits into
guidovizoso wants to merge 6 commits into
Conversation
holdsWebSearch reads the grant refs a built-in Bot is built with, and researchRoleOfToolName reads the mcp__server__tool names a Bot running its own loop is offered. Either counts a search tool from any research provider, without needing its reader alongside. RESEARCH_APP_NAMES lists the catalogue's search apps by title, the name show_app_access resolves. There is no app called "search", so a Bot with none has to name one, and listing every provider keeps the choice with the administrator. A test holds the list to the catalogue's titles.
Asked to "search for" something in a demo, a Bot drove its browser to a search engine. The computer guidance told it to navigate whenever asked to look at, check or read anything, and never to claim it cannot browse; the one line saying a page is not a search read as a hedge beside them. computerGuidance now takes what the run holds and says one thing: - With a search tool, finding information goes to it, and the computer is for signing in, clicking through, filling in forms, or opening a page the person asked to see. - Without one, the browser never goes to Google, Bing or DuckDuckGo to look something up. The Bot answers from what it knows, says it has not checked it, and calls show_app_access once with one of Parallel Search, Exa Search or Firecrawl so the person can ask for it. "Never claim you cannot browse" is gone; opening a page the person names is still something the Bot does. Every path gets the right wording. Built-in Bots are worded in builtInAgentConfiguration from the tools they are built with, so a narrowed run follows what it was offered. agent-bot and agent-langgraph work it out from the tool names each run offers them, which are the only view they have of their grants.
The tool description repeated the old prompt's "look at, visit, open or check a website", which reads "check the latest on X" as a page to open. It now says to open a particular site and never a search engine, so the tool list agrees with the guidance beside it.
…holds no search tool Search tools the catalogue roles do not recognise (a Composio toolkit, a server added by URL, Exa's advanced search granted alone) would otherwise be contradicted by the prompt.
Collaborator
Author
|
Landed in #847. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Asked to "search for" something in a demo, a Bot drove its browser to a search engine instead of calling the search tool it held. The prompt invited it: the computer guidance said to navigate whenever asked to "look at, check or read" anything and to "never claim you cannot browse", and the one sentence saying a page is not a search read as a hedge beside them. A model given both picks the emphatic one.
Now a Bot is told one thing, chosen by what it holds:
show_app_accessonce with one of Parallel Search, Exa Search or Firecrawl, so the person gets a card to ask an administrator for one. The wording is "if none of your tools searches the web", not "you hold no search tool", because a search tool can arrive unrecognised (a Composio toolkit, a server added by URL) and the Bot should not be told something false about it.computer_navigate's description now says it opens a particular site and is not a web search, so the tool list agrees with the guidance beside it.How each kind of Bot gets the right wording.
COMPUTER_GUIDANCE(a constant) becomescomputerGuidance({ search }). Built-in Bots are worded inbuiltInAgentConfigurationfrom the tools they are built with (holdsWebSearch, from grant refs), so a narrowed run follows what it was offered.agent-botandagent-langgraphwork it out from the tool names each run offers them (researchRoleOfToolName, frommcp__server__tool), which are the only view they have of their grants.RESEARCH_APP_NAMESlists the catalogue's search apps by the titleshow_app_accessresolves; a test fails if one stops matching the catalogue.Where it runs
Boundary and audit
agent-botandagent-langgraphread the tool names in their run input, which the server builds from the grants; a wrong guess there changes only which sentence the Bot reads, not what it may call.Changelog
Unreleased.Merge notes
This removes the
COMPUTER_GUIDANCEexport in favour ofcomputerGuidance({ search }). The calm-computer-start PR adds a test inshared/bot-prompt.test.tsthat referencesCOMPUTER_GUIDANCE(and a line to the guidance's list of acting rules, which here lives inACTING_LINES). Whichever of the two merges second has to switch that test tocomputerGuidance, checking both{ search: true }and{ search: false }.How it was checked
shared/bot-prompt.test.ts(each variant's wording, the unchecked answer and the search-app offer),server/tests/copilot.test.ts(a built-in Bot is worded from the tools it is built with, and a deployment with no computer is told about neither),server/tests/web-research.test.ts(any provider's search tool counts, by ref or by the name a model sees; the offered names are the catalogue's titles), andagent-bot/tests/history.test.tsandagent-langgraph/tests/history.test.ts(the system message follows the run's tools).