Skip to content

feat(catalog): add "What's in the Catalog?" discovery tutorial - #67

Merged
aaitor merged 2 commits into
mainfrom
aaitor/discover-the-catalog
Sep 4, 2026
Merged

feat(catalog): add "What's in the Catalog?" discovery tutorial#67
aaitor merged 2 commits into
mainfrom
aaitor/discover-the-catalog

Conversation

@aaitor

@aaitor aaitor commented Sep 4, 2026

Copy link
Copy Markdown
Member

Why this matters

The two existing Catalog demos show an agent spending money — finding services and paying for them autonomously. But before anything can be paid for, it has to be found. This tutorial teaches that first half: how a person, an agent, or a crawler discovers what's in the Nevermined Catalog — every discovery endpoint being public, unauthenticated, and free. Because it's read-only and costs nothing, its "See it run" panel isn't a watch-only video like the paid demos — it actually runs live in the browser against the real catalog (~160 agents today), so a visitor can type a question and watch it answered three ways in real time. It's the natural on-ramp that sits in front of the paid demos.

What this adds

A single new Catalog tutorial, "What's in the Catalog?" (slug discover-the-catalog):

  • catalog/discover-the-catalog/ — the raw tutorial content, mirroring the two sibling catalog folders: the layered README.md (every curl/MCP/ARD example verified live), run-demo.sh (a free, no-key terminal tour), demo-prompt.txt, the 57s video + .en/.es.srt, and the standalone zero-dependency playground/ kept as the runnable "run it locally" reference.
  • Showcase entry in content/tutorials.ts with the written explanation (learn · how · under-the-hood, sourced from the README) and the guided-tour video (.vtt subtitles derived from the .srt).
  • A functional discovery demo in the browsercomponents/DiscoverPanel.tsx runs the "one question, three altitudes" discovery live: human result cards (by eye), the exact MCP/ARD/REST request + JSON an agent makes (by an agent), the /.well-known/ard.json feed entry (by a crawler), and a live ARD /explore facet histogram of the rails/types/tags behind the answer.
  • A same-origin proxyapp/api/catalog/route.ts forwards the catalog/ARD/MCP calls server-side. This is required: the catalog API's CORS only reflects *.nevermined.app origins, so a browser calling it from the showcase's origin is blocked (exactly what the standalone playground/server.mjs solves). It is SSRF-safe: the client picks an op from a fixed set; the upstream host + path are built entirely server-side, never from client input, so it can't be turned into an open relay. Pure request-building/parsing logic lives in lib/catalog-discovery.mjs with a node self-check.

Design decision (the one that was mine to make)

How should a functional discovery demo fit the showcase model? The live tier's /api/agent + demo-agent.mjs is built for the x402 payment handshake (402 → authorize → settle), which discovery has none of. recap is watch-only. I added a third tier + run variant, discover (option (b) from the brief), with its own panel and /api/catalog proxy referenced from the entry.

This was chosen as the least-invasive option that keeps the shared page model intact for every other tutorial. The touches to shared files (lib/types.ts, app/t/[slug]/page.tsx, app/globals.css, app/page.tsx badge) are strictly additive and gated on === "discover" — they add a new discriminated-union member and new render branches; they do not change how any existing tutorial renders. Verified in a browser: song-from-the-headlines still renders "recap · watch it run" with its video + receipt, and langchain-research-agent still renders its live payment panel — neither shows the discovery panel. Because this is purely additive and was explicitly delegated in the brief ("referenced from the entry"), it is not a reshaping of the shared model, so I proceeded rather than escalating.

Rejected: (a) bolting a "discovery mode" onto the payment-shaped live infrastructure — it would muddy the x402 sandbox with an unrelated concern; and reusing recap — it would mislabel a genuinely interactive, free demo as "watch-only", which is the opposite of the tutorial's point.

Scope

Touches only the new tutorial plus the minimal additive wiring: catalog/discover-the-catalog/, its showcase entry, its media under public/media/discover-the-catalog/, the one added copy line in scripts/sync-media.sh, the new panel + proxy route, and the additive discover tier/kind in the shared type/page/CSS. No other tutorial, the two paid catalog demos, or shared component behavior was modified. The large .mp4 is committed under catalog/ and gitignored under public/media/ (pulled in by npm run sync:media), matching the sibling demos.

Test plan

  • node lib/demo-agent.mjs — existing sandbox self-check still passes.
  • node lib/catalog-discovery.mjs — new self-check for keyword extraction, rail filters, ARD/explore body shape, SSE parsing, MCP unwrap, and feed matching.
  • npm run build — green; type-checks content/tutorials.ts and generates all 11 tutorial pages.
  • npm run sync:media — copies the video + derives .vtt from .srt for the new slug.
  • Verified live in a real browser (npm run dev, headless Chromium) against the production catalog: the readout populated (162 agents · 13 categories · 38 x402 / 124 MPP), the hero question returned real cards, all three altitudes populated, the feed entry matched, and the /explore bars filled. Clicking a preset ("weather") re-queried live and refreshed all panes. Confirmed the two existing catalog/live tutorials render unchanged.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NJVmLG9aiAzymDX4n56duk

The discovery counterpart to the two paid catalog demos: instead of an
agent that *pays*, it shows how you *find* what's in the catalog — the same
live Nevermined catalog read three ways (by eye, by an agent, by a crawler)
over public, unauthenticated, free endpoints.

Because discovery is free and read-only, its "See it run" panel is genuinely
functional in the browser (a new `discover` tier), unlike the watch-only
`recap` demos:

- catalog/discover-the-catalog/ — the raw tutorial (README, run-demo.sh,
  demo-prompt.txt, the 57s video + EN/ES subtitles) and the standalone
  zero-dependency playground kept as the runnable reference.
- showcase: a new `DiscoverPanel` + `/api/catalog` same-origin proxy that
  forwards catalog/ARD/MCP calls server-side (the catalog CORS only reflects
  *.nevermined.app origins). Proxies only the fixed Nevermined upstreams — no
  open relay. Pure request/parse logic lives in lib/catalog-discovery.mjs with
  a `node` self-check.
- Registered in content/tutorials.ts with the written explanation + video.

The `discover` tier/kind is additive: existing live/recap tutorials render
unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NJVmLG9aiAzymDX4n56duk
@aaitor
aaitor enabled auto-merge (squash) September 4, 2026 11:11
@r-marques

Copy link
Copy Markdown
Member

👀 Reviewing

@r-marques r-marques left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated PR review — 🟡 Mergeable with nits

Adds the "What's in the Catalog?" discovery tutorial: a standalone playground, a same-origin proxy route in the showcase, and a new recap video. The security-shaped part — two new server-side proxies — is the part I checked hardest, and both hold up. One real gap: the new video will be missing from the deployed image, and it works locally, which is exactly what hides it.

What I reviewed

  • Diff range origin/main...HEAD — 23 files, +1,951/−5, head ed7778eb.
  • Panel: none — stood on direct verification.

Verified

  • The showcase proxy's SSRF claim is accurate, not just asserted. app/api/catalog/route.ts says the client picks an op from a fixed set and the URL is built server-side; I checked every builder in lib/catalog-discovery.mjs. restRequest puts the term through new URLSearchParams (encoded) and keyword() has already stripped it to [a-z0-9\s]; ardRequest / mcpRequest / exploreRequest all send text in the body against hardcoded paths; normalizeRails is a strict x402|mpp whitelist, so even rails[0] can't inject. No client-controlled data reaches the host or path.
  • The playground proxy forwards no client headers. proxy() constructs a fresh header object (accept, plus content-type on writes) rather than passing req.headers through, so there's no Authorization passthrough; p comes from new URL(...).pathname, which is already normalised, so .. can't escape the API host. serveStatic documents and enforces its own containment.
  • No credentials anywhere in the new route, discovery lib, playground or demo scripts — discovery is genuinely unauthenticated, as the comments say.
  • My round-3 nit on the SRT→VTT conversion was taken exactly, and it works: sed 's/\r$//; /-->/ s/,\([0-9][0-9][0-9]\)/.\1/g' is now anchored to cue lines. I regenerated both new .vtt with it — byte-identical to the committed files.
  • The verify job added in #65 covers this PR — it triggers on showcase/**, and next build type-checks the new route, DiscoverPanel and the expanded tutorials.ts. That is the CI gap from two rounds ago now doing its job.

Verdict: 🟡 Mergeable with nits — 0 blockers, 1 should-fix, 1 nit.

🧭 Inline comments (2)

Each is posted on its line in Files changed.

  • 🟡 SHOULD FIX — The new recap video won't be in the deployed image — the Dockerfile copies mp4s by name and wasn't updated — showcase/content/tutorials.ts:626
  • 💡 NITlisten(PORT) binds every interface while the README promises localhost — catalog/discover-the-catalog/playground/server.mjs:112

Comment thread showcase/content/tutorials.ts
Comment thread catalog/discover-the-catalog/playground/server.mjs Outdated
@r-marques
r-marques disabled auto-merge September 4, 2026 11:23

@r-marques r-marques left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Approved with comments — no blockers; 1 should-fix, 1 nit(s) left inline. Auto-merge was disabled so these can be looked at first.

Inline review: #67 (review)

…o localhost

Address review nits on PR #67:

- showcase/Dockerfile: the recap/discover videos are gitignored under
  showcase/public and copied by name from catalog/; add the discover-the-catalog
  mp4 COPY line. Without it the image builds green but the tutorial's <video>
  404s in the deployed image (it only resolved locally via sync-media.sh).
- playground/server.mjs: default-bind to 127.0.0.1 (was 0.0.0.0) so the teaching
  artifact matches its own "http://localhost" banner/README; HOST env still
  allows 0.0.0.0 for container/VM deploys. README documents it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NJVmLG9aiAzymDX4n56duk
@aaitor
aaitor merged commit b319df2 into main Sep 4, 2026
1 check passed
@aaitor
aaitor deleted the aaitor/discover-the-catalog branch September 4, 2026 11:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants