Skip to content

v5 data platform, Phase 6: two-step add-tool — identify in chat, research in the background - #38

Merged
philosophercode merged 2 commits into
mainfrom
v5/data-platform-phase-6
Sep 24, 2026
Merged

philosophercode merged 2 commits into
mainfrom
v5/data-platform-phase-6

Conversation

@philosophercode

Copy link
Copy Markdown
Owner

Stacked on #37 (Phase 5). Implements §5.4 of the data-platform spec; as-built details are in the 2026-09-23 amendment.

What changes

  • Chat identifies only. identify_tools (tools.add) creates pending_tools rows owned by the caller, attaches the turn's photos, runs the duplicate check (normalized name+brand, then pg_trgm), and renders IntakeTableCard: select all/some/none, inline edit via PATCH /api/pending-tools/[id], and a required choice for each duplicate. research_tool and propose_listing are removed. create_tool is MCP-only and writes a Postgres draft, which removes the last request-path Notion write.
  • Research runs in the background. POST /api/pending-tools/research checks ownership (or tools.approve) and the limits (25 per request, 100 per person per day, counted from an append-only ledger under an advisory lock), then starts researchBatch on the Workflow SDK:
    • Each item takes two steps: search (≤4), then fetch and verify (≤4), with a 240s abort each and maxRetries = 2.
    • Items run three at a time, using chunked Promise.allSettled.
    • Confidence is computed in code, never taken from the model.
  • Review and approve. /admin/intake polls while research runs, and offers Retry, Discard and duplicate resolution for leftover items. /admin/intake/[id] offers Approve, Approve as draft, Add unit, Research again and Discard, with the low-confidence gate ("I've checked this" + note). Approval is one transaction; it is audited (pending.approved, tool.published) and invalidates the catalogue.
  • Cron. Fails research abandoned for 24 h, expires identified items after 14 days, deletes discarded items after 30 days, and releases their photos to the orphan sweep. pending_tools is included in the backup.
  • Migrations 0005_pending_tools.sql and 0006_research_ledger.sql.

Tests

  • Unit and integration coverage for the research engine, routes, data layer and cron, plus one in-process @workflow/vitest test.
  • E2E scenario 5 (e2e/intake.spec.ts) stubs the Anthropic API at the provider boundary. Everything else runs for real, including the workflow on the local world.
  • Gate with env vars unset: lint 0 errors, 163 files / 2131 tests, 68 Playwright, spec:coverage 0 undocumented, build green (/tools/[id] and all /admin routes are Partial Prerender).

Before production

  • copyToPublic (turning a pending photo into a public blob) needs one manual check on a preview deploy.
  • Photos sent before a clarifying question are not claimed (a known gap, recorded in the amendment).

🤖 Generated with Claude Code

https://claude.ai/code/session_01DNV63U6ERy45Gj2TThBfyC

…he background

The chat now only identifies. identify_tools writes pending_tools rows owned by
the caller and renders an editable, selectable intake table; research_tool and
propose_listing are gone, and create_tool is MCP-only and writes a Postgres
draft. Research selected (N) starts researchBatch on the Workflow SDK: two
steps per item (4 searches, then 4 fetches and link verification), a 240s
abort, retryable vs fatal errors, and confidence computed in code. Admins
review on /admin/intake and approve from /admin/intake/[id], behind the
low-confidence gate.

The 100-a-day limit counts a ledger (research_requests) under an advisory
lock, so Research again costs again and concurrent presses cannot overshoot.
Each item carries the request that owns it, so two runs never both pay for it.
The daily cron fails abandoned research, expires identified items after 14
days and deletes discarded ones after 30.

Migrations 0005 (pending_tools) and 0006 (research ledger). E2E scenario 5
stubs the model at the provider boundary and runs everything else for real.

Gate: lint 0 errors, 163 files / 2131 tests, 68 Playwright, spec:coverage 0
undocumented, build green with every /admin route still Partial Prerender.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DNV63U6ERy45Gj2TThBfyC
Copilot AI lite review requested due to automatic review settings September 23, 2026 11:15
@vercel

vercel Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
makerlab-tools Ready Ready Preview Sep 24, 2026 1:07am UTC
makerlab-tools-v5 Ready Ready Preview Sep 24, 2026 1:07am UTC

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2a7901d98d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +984 to +986
if (row.status !== "researched" || row.duplicateResolution === "add_unit") {
throw new Refusal("not_editable");
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Block approval while a duplicate remains unresolved

When an admin edits the name or brand of an already researched item, updatePendingTool reruns duplicate detection and can attach a new match while clearing duplicateResolution to null, without changing the researched status. This condition rejects only add_unit, so the still-enabled approval path can immediately create a second catalogue tool without the required duplicate decision; also reject rows that have either duplicate ID with no resolution.

AGENTS.md reference: v5/AGENTS.md:L328-L339

Useful? React with 👍 / 👎.

Comment thread v5/messages/en.json
"actionCreateAnyway": "No, create a new tool",
"actionOpenDraft": "Open draft in Notion"
"actionOpenDraft": "Open draft in Notion",
"status": {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Update every locale alongside the new intake strings

This commit adds the new intake table, status, error, and admin-review copy only to en.json; none of the other 11 locale files changed, so users selecting those supported locales receive English throughout the new workflow via the fallback instead of localized UI. Add the corresponding keys to all message files as required by the repository convention.

AGENTS.md reference: AGENTS.md:L77-L85

Useful? React with 👍 / 👎.

@philosophercode
philosophercode changed the base branch from v5/data-platform-phase-5 to main September 24, 2026 01:03
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DNV63U6ERy45Gj2TThBfyC

This branch was successfully deployed

2 active deployments
Preview – makerlab-tools-v5 — f8c4ad8d Deployed Sep 24, 2026 by vercel[bot]
Preview – makerlab-tools — f8c4ad8d Deployed Sep 24, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants