Hermes Agent crawl4ai plugin for self-hosted AI agent web extraction #2103
mblauser
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
I built hermes-plugin-crawl4ai to wire crawl4ai into Hermes's web_extract tool as a plugin-registered backend. It calls crawl4ai's /md endpoint for clean markdown extraction, handles JS-rendered pages, and integrates with Hermes's multi-profile architecture.
The blocker: a core load-order bug in Hermes prevents plugin-registered web providers from being selected as backends, even when they're correctly installed and registered. Plugin discovery runs after backend selection, so the provider never surfaces. The symptom is a silent fallback to whatever non-plugin backend is configured (e.g. searxng) with a "search-only backend" error on extract calls.
I submitted a fix in PR #67309. It's a minimal 2-line change that moves plugin discovery before the shared backend-resolution chokepoints. It's been tested end-to-end against a live crawl4ai instance and includes a regression test for the cold-registry edge case. The PR has labels triaged and some initial activity, but it needs a project maintainer to do a final review and merge.
If you're a self-hosted Hermes + crawl4ai user (or know someone on the NousResearch team who maintains Hermes), could you help flag this PR to a maintainer? The fix is small, well-scoped, and unblocks a fully self-hosted search-and-extract stack that a lot of self-hosters want.
Repository: https://github.com/mblauser/hermes-plugin-crawl4ai
PR: NousResearch/hermes-agent#67309
All reactions