DX-3122: Serve product root URLs as pages, add per-product llms.txt

.github/workflows/llms-check.yml

@@ -41,12 +41,12 @@ jobs:
         run: |
           git config user.name "github-actions[bot]"
           git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
-          git add llms.txt llms-full.txt
-          git commit -m "chore(llms): regenerate llms.txt and llms-full.txt"
+          git add llms.txt llms-full.txt ':(glob)*/llms.txt'
+          git commit -m "chore(llms): regenerate llms.txt files"
           git push
 
       - name: Fail if llms.txt was stale
         if: steps.check.outcome == 'failure'
         run: |
-          echo "::error::llms.txt / llms-full.txt were out of date. A regeneration commit was pushed to this PR — please review it and re-run CI."
+          echo "::error::llms.txt files were out of date. A regeneration commit was pushed to this PR; please review it and re-run CI."
           exit 1

README.mdx

@@ -25,7 +25,7 @@ The site deploys from `main`. Every pull request gets a Docs7 preview link and a
 
 ## llms.txt
 
-`llms.txt` and `llms-full.txt` are generated from `docs.json` and the MDX files. Don't edit them by hand. After changing pages, run:
+`llms.txt`, `llms-full.txt`, and each product's `<product>/llms.txt` are generated from `docs.json` and the MDX files. Don't edit them by hand. After changing pages, run:
 
 ```sh
 cd llms && npm install && npm run build

blob/index.mdx

@@ -0,0 +1,24 @@
+---
+title: "Upstash Blob"
+sidebarTitle: "Overview"
+description: "S3-compatible object storage with a server client and direct browser uploads."
+---
+
+Upstash Blob is S3-compatible object storage. `@upstash/blob` has a `Bucket` client for your server, and an upload handler plus React hooks that upload from the browser straight to storage.
+
+## Start here
+
+- [Quickstart](/blob/overall/quickstart): create a bucket, upload a file from the browser, and read it back.
+- [Pricing & Limits](/blob/overall/pricing)
+
+## Sections
+
+- [Bucket](/blob/bucket/connecting): connect, then write, read, delete, and cache objects.
+- [Browser Uploads](/blob/uploads/upload-handler): the upload handler and client, constraints, large files, and abandoned uploads.
+- [Recipes](/blob/recipes/overview): profile pictures, product images, site assets, AI images, attachments, video, private documents, and exports.
+- [Reference](/blob/reference/types): types, errors, and how signing works.
+
+## For AI agents
+
+- [/docs/blob/llms.txt](https://upstash.com/docs/blob/llms.txt) lists every Blob page.
+- Append `.md` to any page URL for its Markdown, for example [/docs/blob/overall/quickstart.md](https://upstash.com/docs/blob/overall/quickstart.md).

blob/llms.txt

@@ -0,0 +1,45 @@
+# Upstash Blob
+
+> S3-compatible object storage with a server client and direct browser uploads.
+
+Every Upstash Blob page, grouped by sidebar section. Links point to the Markdown version of each page; drop the `.md` suffix for HTML. The index of all Upstash docs is https://upstash.com/docs/llms.txt. Search them with GET https://upstash.com/docs/search?q=<query>.
+
+## Introduction
+
+- [Upstash Blob](https://upstash.com/docs/blob.md): S3-compatible object storage with a server client and direct browser uploads.
+- [Quickstart](https://upstash.com/docs/blob/overall/quickstart.md): Create a bucket, upload a file straight from the browser, and read it back.
+- [Pricing & Limits](https://upstash.com/docs/blob/overall/pricing.md): What Upstash Blob costs, and where the current limits are listed.
+
+## Bucket
+
+- [Connecting](https://upstash.com/docs/blob/bucket/connecting.md): Build a Bucket client from environment variables, on Node, on Cloudflare Workers, or through an S3 client.
+- [Writing](https://upstash.com/docs/blob/bucket/writing.md): put, unique paths, metadata, conditional writes, copy and move, and signed upload URLs.
+- [Reading](https://upstash.com/docs/blob/bucket/reading.md): get, info, exists and list, public URLs, and signed read URLs on a private bucket.
+- [Deleting](https://upstash.com/docs/blob/bucket/deleting.md): Delete one path, an array or a prefix, and clear out incomplete multipart uploads.
+- [Caching](https://upstash.com/docs/blob/bucket/caching.md): immutable, revalidate and no-store, where each one can be set, and what a CDN keeps.
+
+## Browser Uploads
+
+- [Upload Handler](https://upstash.com/docs/blob/uploads/upload-handler.md): Mount the upload route, authorize the upload in onBeforeUpload, and record the file in onUploadComplete.
+- [Upload Client](https://upstash.com/docs/blob/uploads/upload-client.md): useUpload, useServerUpload and uploadHooks, for uploading from the browser straight to storage.
+- [Constraints](https://upstash.com/docs/blob/uploads/constraints.md): maxSize, contentTypes and byte sniffing, enforced on your server before anything is signed.
+- [Large Files](https://upstash.com/docs/blob/uploads/large-files.md): Multipart uploads from the browser, with progress, pause, resume, retries and cancel.
+- [Abandoned Uploads](https://upstash.com/docs/blob/uploads/abandoned-uploads.md): Sweep the incomplete multipart uploads and pending rows left behind by uploads that never finished.
+
+## Recipes
+
+- [Overview](https://upstash.com/docs/blob/recipes/overview.md): Pick a recipe by the shape of your data, and the three rules that run under all of them.
+- [Profile Pictures](https://upstash.com/docs/blob/recipes/avatars.md): One picture per user, overwritten in place, uploaded from the browser, with the old one out of the cache.
+- [Product Images](https://upstash.com/docs/blob/recipes/product-images.md): Several ordered images per product, uploaded from the browser and rendered from your own rows.
+- [Site Assets and CMS Media](https://upstash.com/docs/blob/recipes/site-assets.md): Editor uploads for a CMS, and site assets written by a deploy script.
+- [AI-Generated Images](https://upstash.com/docs/blob/recipes/ai-images.md): Keep what a model generated, before the provider's temporary URL expires.
+- [File Attachments](https://upstash.com/docs/blob/recipes/attachments.md): Several files per thread, uploaded from the browser as multipart, one row per attachment.
+- [Video Uploads](https://upstash.com/docs/blob/recipes/video.md): Multi-gigabyte video uploaded straight from the browser, with progress and resume.
+- [Private Documents](https://upstash.com/docs/blob/recipes/private-documents.md): One document per owner on a private bucket, handed out through short-lived signed URLs.
+- [Generated Exports](https://upstash.com/docs/blob/recipes/exports.md): A background job builds the file, and the download link stops working after a day.
+
+## Reference
+
+- [Types](https://upstash.com/docs/blob/reference/types.md): Size, Duration, CacheOption, the record types, and the phases an upload moves through.
+- [Errors](https://upstash.com/docs/blob/reference/errors.md): BlobError codes, what reaches the browser, and the platform body limits behind a 413.
+- [How Signing Works](https://upstash.com/docs/blob/reference/signing.md): What the bucket token signs, how the direct upload handshake works, and what the browser gets to see.

box/index.mdx

@@ -0,0 +1,27 @@
+---
+title: "Upstash Box"
+sidebarTitle: "Overview"
+description: "Secure, isolated cloud containers with a built-in AI agent."
+---
+
+Every Upstash Box is a secure, isolated cloud container with an AI agent built in. Each box has its own filesystem, shell, git, network stack, and runtime. You send prompts or commands, the box runs them, and you get structured results back.
+
+## Start here
+
+- [Quickstart](/box/overall/quickstart): get an API key, install the SDK, create a box, and run your first task.
+- [Box Basics](/box/overall/how-it-works): architecture, billing, and lifecycle.
+- [Use Cases](/box/overall/use-cases)
+- [Pricing & Limits](/box/overall/pricing)
+
+## Sections
+
+- [Basics](/box/overall/agent): the agent, custom agents and harnesses, git, shell, live sessions, filesystem, and snapshots.
+- [Lifecycle](/box/overall/preview): public URLs, schedules, keep alive, and ephemeral boxes.
+- [Security](/box/overall/security): secrets, network policy, and attached headers.
+- [Browser](/box/overall/browser/overview): a cloud browser with tabs, page reading, AI actions, live view, recordings, and CDP.
+- [Guides](/box/guides/remote-development): end-to-end setups for coding agents, frameworks, and agent platforms.
+
+## For AI agents
+
+- [/docs/box/llms.txt](https://upstash.com/docs/box/llms.txt) lists every Box page.
+- Append `.md` to any page URL for its Markdown, for example [/docs/box/overall/quickstart.md](https://upstash.com/docs/box/overall/quickstart.md).

box/llms.txt

@@ -0,0 +1,78 @@
+# Upstash Box
+
+> Secure, isolated cloud containers with a built-in AI agent.
+
+Every Upstash Box page, grouped by sidebar section. Links point to the Markdown version of each page; drop the `.md` suffix for HTML. The index of all Upstash docs is https://upstash.com/docs/llms.txt. Search them with GET https://upstash.com/docs/search?q=<query>.
+
+## Introduction
+
+- [Upstash Box](https://upstash.com/docs/box.md): Secure, isolated cloud containers with a built-in AI agent.
+- [Quickstart](https://upstash.com/docs/box/overall/quickstart.md)
+- [Box Basics](https://upstash.com/docs/box/overall/how-it-works.md)
+- [Use Cases](https://upstash.com/docs/box/overall/use-cases.md)
+- [Pricing & Limits](https://upstash.com/docs/box/overall/pricing.md)
+
+## Basics
+
+- [Agent](https://upstash.com/docs/box/overall/agent.md)
+- [Git](https://upstash.com/docs/box/overall/git.md)
+- [Shell](https://upstash.com/docs/box/overall/shell.md)
+- [Live Sessions](https://upstash.com/docs/box/overall/live-sessions.md)
+- [Filesystem](https://upstash.com/docs/box/overall/files.md)
+- [Snapshots](https://upstash.com/docs/box/overall/snapshots.md)
+
+## Basics > Custom Agents
+
+- [How to Add a Custom Agent](https://upstash.com/docs/box/overall/custom-agent.md)
+- [Pi](https://upstash.com/docs/box/overall/custom-harness/pi.md)
+- [Gemini](https://upstash.com/docs/box/overall/custom-harness/gemini.md)
+- [Copilot CLI](https://upstash.com/docs/box/overall/custom-harness/copilot.md)
+- [Aider](https://upstash.com/docs/box/overall/custom-harness/aider.md)
+- [Goose](https://upstash.com/docs/box/overall/custom-harness/goose.md)
+- [Amp](https://upstash.com/docs/box/overall/custom-harness/amp.md)
+- [CrewAI](https://upstash.com/docs/box/overall/custom-harness/crewai.md)
+- [Pydantic AI](https://upstash.com/docs/box/overall/custom-harness/pydantic-ai.md)
+
+## Lifecycle
+
+- [Public URLs](https://upstash.com/docs/box/overall/preview.md)
+- [Schedules](https://upstash.com/docs/box/overall/schedules.md)
+- [Keep Alive](https://upstash.com/docs/box/overall/keep-alive.md)
+- [Ephemeral Box](https://upstash.com/docs/box/overall/ephemeral-box.md)
+
+## Security
+
+- [Security & Secrets](https://upstash.com/docs/box/overall/security.md)
+- [Network Policy](https://upstash.com/docs/box/overall/network-policy.md)
+- [Attach Headers](https://upstash.com/docs/box/overall/attach-headers.md)
+
+## Browser
+
+- [Browser](https://upstash.com/docs/box/overall/browser/overview.md)
+- [Tabs & Navigation](https://upstash.com/docs/box/overall/browser/tabs.md)
+- [Reading Pages](https://upstash.com/docs/box/overall/browser/reading-pages.md)
+- [AI Actions](https://upstash.com/docs/box/overall/browser/ai-actions.md)
+- [Live View](https://upstash.com/docs/box/overall/browser/live-view.md)
+- [Recordings](https://upstash.com/docs/box/overall/browser/recordings.md)
+- [Connect over CDP](https://upstash.com/docs/box/overall/browser/connect.md)
+- [Cookbook](https://upstash.com/docs/box/overall/browser/cookbook.md): Runnable browser examples grouped by use case. Each one is self-contained: paste it, run it, read the output.
+
+## Guides
+
+- [Remote Development Environment for AI Coding Agents](https://upstash.com/docs/box/guides/remote-development.md): Turn an Upstash Box into a remote development environment. Your app runs in the cloud, you open it in your browser, and your coding agent does the work.
+- [Host T3 Code in a Remote Sandbox](https://upstash.com/docs/box/guides/t3-code-setup.md): Run T3 Code in an Upstash Box and open it from any browser or phone. The box sleeps when you are away and wakes when you open the URL again.
+- [Run a Next.js App in a Cloud Sandbox with a Public URL](https://upstash.com/docs/box/guides/nextjs-setup.md): Scaffold a Next.js app inside an Upstash Box, run the dev server, and open it from any browser through a public URL, all with the Box SDK.
+- [Build an AI Code Review Agent for Pull Requests](https://upstash.com/docs/box/guides/code-review-agent.md): Build an AI code review agent with Upstash Box that clones a repo, inspects the pull request diff, and returns structured findings with severity and suggested fixes.
+- [Sandbox Backend for LangChain Deep Agents](https://upstash.com/docs/box/guides/langchain-deep-agents.md): Use langchain-upstash-box as a Deep Agents sandbox backend, so the agent's shell and file tools run in an isolated Upstash Box instead of on your machine.
+- [Playwright Web Scraping with a Cloud Browser](https://upstash.com/docs/box/guides/web-scraping-playwright.md): Scrape JavaScript-heavy sites with the managed headless Chromium in Upstash Box. Extract structured data through the SDK, then connect Playwright over CDP.
+- [Run a Browser Use Agent on a Daily Schedule](https://upstash.com/docs/box/guides/browser-use.md): Run a Browser Use agent inside Upstash Box and schedule it with cron. The example agent reads Hacker News every morning and saves a report to the box.
+- [Vercel AI SDK Code Interpreter and Shell Tools](https://upstash.com/docs/box/guides/ai-sdk-setup.md): Give Vercel AI SDK agents a cloud sandbox to run Python, JavaScript, and shell commands, manage files, and share app previews with Upstash Box.
+- [Run TanStack AI Coding Agents in a Cloud Sandbox](https://upstash.com/docs/box/guides/tanstack-setup.md): Run TanStack AI coding agents like Claude Code, Codex, and OpenCode inside Upstash Box with the native @tanstack/ai-sandbox-upstash-box provider.
+- [Vercel Eve Sandbox for Isolated Code Execution](https://upstash.com/docs/box/guides/eve-setup.md): Use @upstash/agentkit-eve to run Vercel Eve agent code in Upstash Box, with deny-all egress, setup baked into a snapshot, and one box per session.
+- [Run Pi Agent Tool Calls in a Remote Sandbox](https://upstash.com/docs/box/guides/pi-setup.md): Use the @upstash/box-pi extension to run every Pi tool call in an Upstash Box, while the agent and your model API keys stay on your machine.
+- [Self-Host an Always-On OpenClaw Agent in the Cloud](https://upstash.com/docs/box/guides/openclaw-setup.md): Install OpenClaw in a keep-alive Upstash Box over SSH, open its dashboard from your machine, and restart it automatically with an init script.
+- [Self-Host Hermes Agent in a Cloud Sandbox](https://upstash.com/docs/box/guides/hermes-setup.md): Install the Hermes agent by Nous Research in a keep-alive Upstash Box over SSH, and restart it automatically with an init script.
+- [Run Herdr Coding Agents in Isolated Cloud Sandboxes](https://upstash.com/docs/box/guides/herdr-setup.md): Use the Upstash Box plugin for Herdr to run coding agents from a Git worktree in their own box, then apply their changes locally as a reviewed patch.
+- [Self-Hosted Sandboxes for Claude Managed Agents](https://upstash.com/docs/box/guides/claude-managed-agents.md): Use Upstash Box as a self-hosted sandbox for Claude Managed Agents. Claude runs on Anthropic's side, and each session's bash commands and file edits run in its own box.
+- [Self-Hosted Sandboxes for the OpenAI Agents API](https://upstash.com/docs/box/guides/openai-agents-api.md): Use Upstash Box as a self-hosted sandbox for the OpenAI Agents API. The agent runs on OpenAI's side, and each session's commands run in its own box.
+- [In-App Coding Agent with CopilotKit](https://upstash.com/docs/box/guides/copilotkit.md): Add a CopilotKit chat to your Next.js app where each user talks to a coding agent running in their own Upstash Box, next to its shell and files.

docs.json

@@ -86,6 +86,7 @@
                       {
                         "group": "Overall",
                         "pages": [
+                          "redis/index",
                           "redis/overall/getstarted",
                           "redis/overall/pricing",
                           "redis/overall/compatibility",
@@ -1427,6 +1428,7 @@
                       {
                         "group": "Overall",
                         "pages": [
+                          "vector/index",
                           "vector/overall/getstarted",
                           "vector/overall/whatisvector",
                           "vector/overall/pricing",
@@ -1594,6 +1596,7 @@
                       {
                         "group": "Overall",
                         "pages": [
+                          "qstash/index",
                           "qstash/overall/getstarted",
                           "qstash/overall/usecases",
                           "qstash/overall/pricing",
@@ -1747,6 +1750,7 @@
                   {
                     "group": "Getting Started",
                     "pages": [
+                      "workflow/index",
                       "workflow/getstarted",
                       {
                         "group": "Quickstarts",
@@ -2011,6 +2015,7 @@
                       {
                         "group": "Overall",
                         "pages": [
+                          "search/index",
                           "search/overall/getstarted",
                           "search/overall/whatisupstashsearch",
                           "search/overall/pricing"
@@ -2112,6 +2117,7 @@
                   {
                     "group": "Introduction",
                     "pages": [
+                      "box/index",
                       "box/overall/quickstart",
                       "box/overall/how-it-works",
                       "box/overall/use-cases",
@@ -2204,6 +2210,7 @@
                   {
                     "group": "Introduction",
                     "pages": [
+                      "blob/index",
                       "blob/overall/quickstart",
                       "blob/overall/pricing"
                     ]
@@ -2263,6 +2270,7 @@
                   {
                     "group": "Realtime",
                     "pages": [
+                      "realtime/index",
                       "realtime/overall/quickstart",
                       "realtime/features/client-side",
                       "realtime/features/server-side",