Skip to content

feat: group lints in get_advisors response - #390

Open
mattrossman wants to merge 4 commits into
mainfrom
mattrossman/ai-1136-group-lints-in-get_advisors-to-reduce-response-bloat
Open

feat: group lints in get_advisors response#390
mattrossman wants to merge 4 commits into
mainfrom
mattrossman/ai-1136-group-lints-in-get_advisors-to-reduce-response-bloat

Conversation

@mattrossman

@mattrossman mattrossman commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

Problem

get_advisors returns a list with one object per finding, each carrying the same keys. The lint SQL emits title, description and remediation as literals on each row, so 300 instances of the same lint means those same three strings 300 times.

See diagnosis thread with metrics demonstrating that is is the longest tail context hog in studio's AI Assistant.

Fix

Groups findings by lint.

  • Findings only merge when all seven shared fields match
  • Merged findings appear as a findings list on the lints entry, with convenience count
  • Drops cache_key, which primarily exists for the dashboard UI

Before

{ "lints": [
  { "name": "lint_a", "title": "", "level": "WARN", "facing": "", "categories": [""], "description": "", "remediation": "",
    "detail": "… object_1 …", "metadata": { "name": "object_1" }, "cache_key": "" },
  { "name": "lint_a", "title": "", "level": "WARN", "facing": "", "categories": [""], "description": "", "remediation": "",
    "detail": "… object_2 …", "metadata": { "name": "object_2" }, "cache_key": "" },
  { "name": "lint_a", "title": "", "level": "WARN", "facing": "", "categories": [""], "description": "", "remediation": "",
    "detail": "… object_3 …", "metadata": { "name": "object_3" }, "cache_key": "" },
  { "name": "lint_a", "title": "", "level": "WARN", "facing": "", "categories": [""], "description": "", "remediation": "",
    "detail": "… object_4 …", "metadata": { "name": "object_4" }, "cache_key": "" },
  { "name": "lint_a", "title": "", "level": "WARN", "facing": "", "categories": [""], "description": "", "remediation": "",
    "detail": "… object_5 …", "metadata": { "name": "object_5" }, "cache_key": "" },
  // ...296 more, each repeating the same 7 shared fields
]}

After

{ "lints": [
  { "name": "lint_a", "title": "", "level": "WARN", "facing": "", "categories": [""], "description": "", "remediation": "",
    "count": 301,
    "findings": [
      { "detail": "… object_1 …", "metadata": { "name": "object_1" } },
      { "detail": "… object_2 …", "metadata": { "name": "object_2" } },
      { "detail": "… object_3 …", "metadata": { "name": "object_3" } },
      { "detail": "… object_4 …", "metadata": { "name": "object_4" } },
      { "detail": "… object_5 …", "metadata": { "name": "object_5" } },
      // ...296 more
    ] }
]}

In the turn cited above, the response shrinks to 293,485 chars, ~73k tokens.

How to review

Take a look at the new tests to understand the expecting grouping behavior.

Point an MCP client at the preview build below then call get_advisors for a project with some advisors, and observe the shape.

Screenshot from my testing in MCP Inspector:

CleanShot 2026-08-28 at 12 42 40@2x

Follow-ups

  • Even with the shrunken response, tool outputs are unbounded and can flood context. The worst recent call returned 10,649 findings, where even with grouping it's ~2.5x over Assistant's context limit. This exploration will continue in AI-865.
  • detail is only included per-finding because pg-meta puts index names, policy names, roles and actions in prose instead of metadata. If we fill those in, detail could be dropped entirely, making that same payload ~4x smaller

tools/list output is untouched so no ChatGPT resubmission needed.

Closes AI-1136

@mattrossman mattrossman changed the title feat: group lints in get_advisors response feat: group lints in get_advisors response Aug 28, 2026
@coveralls

coveralls commented Aug 28, 2026

Copy link
Copy Markdown

Coverage Report for CI Build 33192641335

Coverage increased (+0.08%) to 96.595%

Details

  • Coverage increased (+0.08%) from the base build.
  • Patch coverage: 54 of 54 lines across 1 file are fully covered (100%).
  • No coverage regressions found.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

No coverage regressions found.


Coverage Stats

Coverage Status
Relevant Lines: 3089
Covered Lines: 3004
Line Coverage: 97.25%
Relevant Branches: 406
Covered Branches: 372
Branch Coverage: 91.63%
Branches in Coverage %: Yes
Coverage Strength: 50.3 hits per line

💛 - Coveralls

@mattrossman mattrossman added the publish-preview Runs `publish-preview` workflow to publish preview packages via https://pkg.pr.new/ label Aug 28, 2026
@pkg-pr-new

pkg-pr-new Bot commented Aug 28, 2026

Copy link
Copy Markdown

Open in StackBlitz

pnpm add https://pkg.pr.new/@supabase/mcp-server-postgrest@8e4b8bc
pnpm add https://pkg.pr.new/@supabase/mcp-server-supabase@8e4b8bc
pnpm add https://pkg.pr.new/@supabase/mcp-utils@8e4b8bc

commit: 8e4b8bc

@mattrossman
mattrossman marked this pull request as ready for review August 28, 2026 17:04
@mattrossman
mattrossman requested a review from a team as a code owner August 28, 2026 17:04
@lokicik

lokicik commented Aug 29, 2026

Copy link
Copy Markdown

I tested this against Base 30baa1f and the current PR head 8e4b8bc.

I used the same 301 advisor findings on both sides and asked the agent to fetch both security and performance advisors, then return every group with its count, severity, and a sample object.

The Base response was about 178k characters, while the grouped PR response was about 41k.

Model Base PR
DeepSeek V4 Flash 0/3 3/3
GPT-5.6 Luna 0/3 3/3
Qwen 3.5 Flash 0/3 3/3
MiMo v2.5 0/3 3/3

On Base, none of the attempts managed to return the complete summary within the result budget. With this PR, all 12 did.

The grouping helped a lot here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

publish-preview Runs `publish-preview` workflow to publish preview packages via https://pkg.pr.new/

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants