Skip to content

refactor(skills): remove dated prompt patterns found by prompt audit - #173

Merged
ryzizub merged 3 commits into
mainfrom
session/fierce-quail-qwmo
Oct 6, 2026
Merged

ryzizub merged 3 commits into
mainfrom
session/fierce-quail-qwmo

Conversation

@ryzizub

@ryzizub ryzizub commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

What

Remove dated prompt patterns from all 15 skills and the reviewer agent.

Why

The skills carry duplicated rules, failure-mode lists and incident-specific wording from eval tuning. Current models read those as extra constraints.

Changes

  • Close the unclosed code fence in animations that broke its anti-patterns section
  • Drop model: pins from every skill, so each answers on the session model
  • Rewrite the five descriptions at the 1024-char cap as intent categories
  • Replace repo paths in green-gate with skill-name references
  • Remove duplicated rules, worked examples, and one editor-facing note
  • Move the WCAG criterion tables to accessibility/references/wcag-criteria.md
  • AGENTS.md, evals/README.md: pin count and structure updated

Evals

Commit 1 (two-arm, create-project + very-good-analysis-upgrade): every with-arm case ≥ 0.90, negative controls 1.00 in both arms. Other seven skills, with-arm: 41/49 at 1.00, rest match main.

Commit 2 (two-arm, the five rewritten descriptions): 36 cases, zero routing misses, 30 at 1.00, lowest 0.81. Other seven skills, with-arm: 37/45 at 1.00, lowest material-theming-refactors-hardcoded-widget at 0.75, which scores 0.79 with its old sonnet pin and fails the same two graders, so the pin is not the cause.

Pre-existing on main: dart-flutter-sdk-upgrade-reports-pub-get-conflict-instead-of-resolving scores 0.67. The model reads the pasted error as a toolchain mismatch. Needs its own fix.

🤖 Generated with Claude Code

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@ryzizub
ryzizub requested a review from a team as a code owner October 5, 2026 20:08
ryzizub and others added 2 commits October 5, 2026 22:16
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

@marcossevilla marcossevilla left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice deletions!

@ryzizub
ryzizub merged commit 496a3c6 into main Oct 6, 2026
5 checks passed
@ryzizub
ryzizub deleted the session/fierce-quail-qwmo branch October 6, 2026 07:57
@vgvbot vgvbot mentioned this pull request Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants