Skip to content

Rewrite hand-written skill copy from behaviors to labels - #7

Merged
shunuen0 merged 4 commits into
mainfrom
shun/labels-skill-copy-e323
Oct 13, 2026
Merged

shunuen0 merged 4 commits into
mainfrom
shun/labels-skill-copy-e323

Conversation

@shunuen0

@shunuen0 shunuen0 commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Copy-only companion to Judgment's Behaviors → Labels rename. Wording matches the judgment-plugin draft (JudgmentLabs/judgment-plugin#1). Noun lists say labels. A judge applies a label; the automation condition is Judge applied label. Judge type copy uses Agent judge / Code judge. Rubric scoring is an Agent judge, not a prompt scorer.

Merge this only after judgment-mono #5273 deploys. The "Labels or scores" row links to https://docs.judgmentlabs.ai/documentation/judges/monitor-judge-results, and that target goes live with #5273. https://docs.judgmentlabs.ai/documentation/monitoring is already live.

The generated skills/judgeval-jql/* files are unchanged. They are marked "Do not edit" and come with the jql follow-up, which regenerates them from the upstream jql repo. The search_traces wire field stays "behaviors". Paths, the code-judges keyword, and the live /prompt-scorers docs URL are unchanged. User feedback is captured as scores, because judges apply labels.

Before / after

  • README.md: "production traces, behaviors, prompts, automations…" → "production traces, labels, prompts, automations…"
  • README.md: "evaluations, code judges, datasets" → "evaluations, Code judges, datasets"
  • README.md: "using code judges" → "using Code judges"
  • README.md: "Create or debug Python code judges" → "Create or debug Python Code judges"
  • skills/judgment/references/tracing.md: "capture as scores or behavior signals" → "capture as scores"
  • skills/judgment/references/tracing.md: "Determines scoring or behavior-monitoring follow-up" → "Determines whether to follow up with a score or a label"
  • skills/judgment/references/tracing.md: "Behavior or score signals | Enables quality filtering and production monitoring | …/performance/agent-behavior-monitoring" → "Labels or scores | Filter by label or score and track how often each label is applied | https://docs.judgmentlabs.ai/documentation/judges/monitor-judge-results"
  • skills/judgment/references/tracing.md: "Behaviors: Filter traces or sessions by monitored behavior" → "Labels: Filter traces or sessions by the labels judges applied"
  • skills/judgment/references/tracing.md: "Route failures, behaviors, or thresholds…" → "Route failures, a "Judge applied label" condition, or thresholds…"
  • skills/judgment/references/code-judges.md: "# Judgment Code Judges" → "# Judgment Code judges"
  • skills/judgment/references/code-judges.md: "## Current Support" → "## Current support"
  • skills/judgment/references/code-judges.md: "## When To Use A Code Judge" → "## When to use a Code judge"
  • skills/judgment/references/code-judges.md: "## Minimal Pattern" → "## Minimal pattern"
  • skills/judgment/references/code-judges.md: "## Trace-Aware Pattern" → "## Trace-aware pattern"
  • skills/judgment/references/code-judges.md: "## Packaging Guidance" → "## Packaging guidance"
  • skills/judgment/references/code-judges.md: "creating Python code judges" → "creating Python Code judges"
  • skills/judgment/references/code-judges.md: "Use a code judge when…" → "Use a Code judge when…"
  • skills/judgment/references/agent-testing.md: "Custom code judges can also use the fields copied" → "Code judges can also use the fields copied"
  • skills/judgment/references/evaluations.md: "prompt or hosted scorer for rubric-based…" → "Agent judge or hosted scorer for rubric-based…"
  • skills/judgment/references/evaluations.md: "Python code judge for deterministic logic…" → "Python Code judge for deterministic logic…"
  • skills/judgment/references/evaluations.md: "Use prompt scorers for qualitative rubrics…" → "Use Agent judges for qualitative rubrics…"
  • skills/judgment/references/evaluations.md: "Use code judges for exact checks…" → "Use Code judges for exact checks…"
  • skills/judgment/references/docs.md: "Prompt scorers:" → "Agent judges:" (URL stays …/evaluation/prompt-scorers)
  • skills/judgment/SKILL.md: "evaluations, code judges, datasets" → "evaluations, Code judges, datasets"
  • skills/judgment/SKILL.md: "creating code judges" → "creating Code judges"
  • skills/judgment/SKILL.md: "evaluations or code judges" → "evaluations or Code judges"
  • skills/judgment/SKILL.md: "Use prompt/hosted scorers for rubric-based judgment and Python code judges…" → "Use Agent judges for rubric-based judgment and Python Code judges…"
  • skills/judgment/SKILL.md: "Creating Python code judges" → "Creating Python Code judges" (path references/code-judges.md unchanged)
  • .claude-plugin/plugin.json: "evaluations, code judges, MCP" → "evaluations, Code judges, MCP"
  • .cursor-plugin/plugin.json: "evaluations, code judges, MCP" → "evaluations, Code judges, MCP"
  • .claude-plugin/marketplace.json: "evaluations, code judges, MCP" → "evaluations, Code judges, MCP"
  • .claude-plugin/marketplace.json: "using code judges" → "using Code judges"
  • skills/mcp-server-best-practices/SKILL.md: "traces, behaviors, sessions…" → "traces, labels, sessions…"
  • skills/mcp-server-best-practices/SKILL.md: // Behaviors (any of the listed judge/value pairs) → // Labels (any of the listed judge/value pairs)
Open in Web Open in Cursor 

cursoragent and others added 4 commits October 7, 2026 22:15
Hand-written skill and README lines now describe labels a judge applies, matching the judgment-plugin wording. The search_traces wire field stays "behaviors", and generated judgeval-jql files are untouched.

Co-authored-by: shunuen0 <shunuen0@users.noreply.github.com>
User feedback is captured as scores, because judges apply labels. Code judge headings in the hand-written skills match the judgment-plugin wording.

Co-authored-by: shunuen0 <shunuen0@users.noreply.github.com>
Rubric scoring in the judgment skill is described as Agent judges. Paths, the code-judges keyword, and the behaviors filter field are unchanged.

Co-authored-by: shunuen0 <shunuen0@users.noreply.github.com>
The evaluations and docs references no longer say prompt scorer. The live /prompt-scorers docs URL stays.

Co-authored-by: shunuen0 <shunuen0@users.noreply.github.com>
@luna-jl
luna-jl marked this pull request as ready for review October 8, 2026 20:45
@luna-jl
luna-jl requested a review from alanzhang25 October 8, 2026 20:45
@nlemoff
nlemoff self-requested a review October 13, 2026 01:27

@nlemoff nlemoff left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@shunuen0
shunuen0 merged commit 2b37238 into main Oct 13, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants