From ae9ad693c0888e4976fdbdc202475b4932c93fa4 Mon Sep 17 00:00:00 2001 From: asavor <74147411+asavor@users.noreply.github.com> Date: Wed, 16 Sep 2026 05:13:59 +0000 Subject: [PATCH 1/3] Add String Web Access integration page String Web Access is a hosted MCP server at https://mcp.usestring.ai/v1/mcp that gives a Haystack agent web search, URL fetching, write requests and sitemap crawls, returned as LLM-ready Markdown with anti-bot handling server-side. Like the Unstructured Transform entry, this integration ships no package of its own: it uses mcp-haystack's MCPToolset over Streamable HTTP, authenticating with a String API key through StreamableHttpServerInfo's token parameter. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01We1tQZLVTFFuSpr5piXguP --- integrations/string-web-access.md | 149 ++++++++++++++++++++++++++++++ logos/string-web-access.svg | 13 +++ 2 files changed, 162 insertions(+) create mode 100644 integrations/string-web-access.md create mode 100644 logos/string-web-access.svg diff --git a/integrations/string-web-access.md b/integrations/string-web-access.md new file mode 100644 index 00000000..f4f11ffb --- /dev/null +++ b/integrations/string-web-access.md @@ -0,0 +1,149 @@ +--- +layout: integration +name: String Web Access +description: Search the web, fetch any URL and map a site — clean Markdown, past anti-bot blocks. +authors: + - name: String + socials: + github: usestring +pypi: https://pypi.org/project/mcp-haystack/ +repo: https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mcp +type: Tool Integration +report_issue: https://github.com/usestring/string-ai-mcp/issues +logo: /logos/string-web-access.svg +version: Haystack 2.0 +toc: true +mcp: true +--- +### **Table of Contents** +- [Overview](#overview) +- [Installation](#installation) +- [Usage](#usage) +- [Examples](#examples) +- [License](#license) + +## Overview + +**[String Web Access](https://usestring.ai)** lets an agent read the live web. Search the web, fetch any URL +or send it a write request, and map a site's URLs — all returned as clean, LLM-ready Markdown. Proxy rotation, +anti-bot handling, CAPTCHA solving and JavaScript rendering happen server-side, so the agent gets the page +instead of a block screen. Best for sites that rate-limit, geo-gate or block automated traffic. + +It is exposed as a hosted [Model Context Protocol](https://modelcontextprotocol.io/) server at +`https://mcp.usestring.ai/v1/mcp`, over Streamable HTTP. + +This integration does not ship its own package. It uses `mcp-haystack`'s `MCPToolset` to connect a Haystack +agent or pipeline to that server, so the toolset is discovered at connect time and stays current as String +ships new tools. + +The server currently exposes four tools: + +| Tool | What it does | +| --- | --- | +| `web_access_search` | Search the web and return structured organic results. | +| `web_access_fetch` | Fetch any URL as Markdown, raw body, or a JSON envelope with status and headers. Optional JavaScript rendering, geolocated proxy routing and browser actions. | +| `web_access_request` | Send a `POST`, `PUT` or `PATCH` with a body to a URL — for GraphQL queries and JSON APIs that refuse `GET`. | +| `web_access_sitemap` | Map a whole site's URLs as an asynchronous, quote-then-approve crawl job. | + +## Installation + +```bash +pip install mcp-haystack +``` + +## Usage + +String Web Access supports two ways to authenticate: browser-based OAuth for clients that speak remote MCP +natively, and a static API key sent as an `Authorization: Bearer` header for headless frameworks like Haystack. +Get an API key at [usestring.ai](https://usestring.ai) and pass it through `StreamableHttpServerInfo`'s `token` +parameter, which builds that header for you: + +```python +import os + +from haystack.utils import Secret +from haystack_integrations.tools.mcp import MCPToolset, StreamableHttpServerInfo + +os.environ["STRING_API_KEY"] = "YOUR_STRING_API_KEY" + +server_info = StreamableHttpServerInfo( + url="https://mcp.usestring.ai/v1/mcp", + token=Secret.from_env_var("STRING_API_KEY"), +) +toolset = MCPToolset(server_info=server_info, eager_connect=True) + +for tool in toolset.tools: + print(f"{tool.name}: {tool.description}") +``` + +Pass `tool_names` to narrow the toolset to the tools a given agent should have — for a read-only research +agent, leave `web_access_request` out: + +```python +toolset = MCPToolset( + server_info=server_info, + tool_names=["web_access_search", "web_access_fetch"], +) +``` + +## Examples + +### Fetch a page that blocks ordinary requests + +```python +from haystack.utils import Secret +from haystack_integrations.tools.mcp import MCPTool, StreamableHttpServerInfo + +server_info = StreamableHttpServerInfo( + url="https://mcp.usestring.ai/v1/mcp", + token=Secret.from_env_var("STRING_API_KEY"), +) + +fetch = MCPTool(name="web_access_fetch", server_info=server_info) +result = fetch.invoke(url="https://example.com") + +print(result) +``` + +### Give an Agent the web + +`MCPToolset` is a Haystack `Toolset`, so it goes straight into an `Agent`. The agent picks the right String +tool for each step — searching first, then fetching the pages worth reading. + +```python +from haystack.components.agents import Agent +from haystack.components.generators.chat import OpenAIChatGenerator +from haystack.dataclasses import ChatMessage +from haystack.utils import Secret +from haystack_integrations.tools.mcp import MCPToolset, StreamableHttpServerInfo + +toolset = MCPToolset( + server_info=StreamableHttpServerInfo( + url="https://mcp.usestring.ai/v1/mcp", + token=Secret.from_env_var("STRING_API_KEY"), + ) +) + +agent = Agent( + chat_generator=OpenAIChatGenerator(model="gpt-5-mini"), + tools=toolset, + system_prompt=( + "You research questions using the live web. Search first, then fetch the pages " + "worth reading, and cite every URL you used." + ), +) + +result = agent.run( + messages=[ChatMessage.from_user("What does String Web Access charge for a CAPTCHA solve?")] +) +print(result["messages"][-1].text) +``` + +Because the toolset is discovered from the server at connect time, no code changes when String adds a tool — +the agent sees it on the next run. + +## License + +`mcp-haystack` is distributed under the terms of the +[Apache-2.0](https://github.com/deepset-ai/haystack-core-integrations/blob/main/LICENSE.txt) license. +The String Web Access API is a commercial service; see [usestring.ai](https://usestring.ai) for its terms. diff --git a/logos/string-web-access.svg b/logos/string-web-access.svg new file mode 100644 index 00000000..3fff2176 --- /dev/null +++ b/logos/string-web-access.svg @@ -0,0 +1,13 @@ + +String mark + + + + + + + + + + + From 241f1b4e720009f16aac7e669527f01750309d10 Mon Sep 17 00:00:00 2001 From: asavor <74147411+asavor@users.noreply.github.com> Date: Tue, 22 Sep 2026 14:26:12 +0000 Subject: [PATCH 2/3] Fix the Apache-2.0 license link (points at a nonexistent LICENSE.txt) haystack-core-integrations has a root LICENSE file, not LICENSE.txt, so the blob link 404s. Switched to the SPDX link, matching tavily.md's own license line. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01We1tQZLVTFFuSpr5piXguP --- integrations/string-web-access.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/integrations/string-web-access.md b/integrations/string-web-access.md index f4f11ffb..64658364 100644 --- a/integrations/string-web-access.md +++ b/integrations/string-web-access.md @@ -145,5 +145,5 @@ the agent sees it on the next run. ## License `mcp-haystack` is distributed under the terms of the -[Apache-2.0](https://github.com/deepset-ai/haystack-core-integrations/blob/main/LICENSE.txt) license. +[Apache-2.0](https://spdx.org/licenses/Apache-2.0.html) license. The String Web Access API is a commercial service; see [usestring.ai](https://usestring.ai) for its terms. From 02bb6ca86b741a4145e63ed13dd434178a041967 Mon Sep 17 00:00:00 2001 From: asavor <74147411+asavor@users.noreply.github.com> Date: Tue, 22 Sep 2026 07:51:19 +0000 Subject: [PATCH 3/3] Limit the Agent example to search and fetch The example handed the agent every server tool, including web_access_request, which sends POST, PUT and PATCH requests, so a research prompt could reach a write. It now passes tool_names, as the read-only snippet above it does, and the agent only gets web_access_search and web_access_fetch. The note after the example said new String tools reach the agent automatically. That stops being true once tool_names is set, since MCPToolset skips any tool off the list, so the note now explains the restriction instead. Co-Authored-By: Claude Opus 5 (1M context) --- integrations/string-web-access.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/integrations/string-web-access.md b/integrations/string-web-access.md index 64658364..dae90593 100644 --- a/integrations/string-web-access.md +++ b/integrations/string-web-access.md @@ -121,7 +121,8 @@ toolset = MCPToolset( server_info=StreamableHttpServerInfo( url="https://mcp.usestring.ai/v1/mcp", token=Secret.from_env_var("STRING_API_KEY"), - ) + ), + tool_names=["web_access_search", "web_access_fetch"], ) agent = Agent( @@ -139,8 +140,8 @@ result = agent.run( print(result["messages"][-1].text) ``` -Because the toolset is discovered from the server at connect time, no code changes when String adds a tool — -the agent sees it on the next run. +`tool_names` limits this agent to searching and fetching. `web_access_request` sends `POST`, `PUT` and `PATCH` +requests that can change data on the target site, so give it only to an agent that is meant to write. ## License