Skip to content

feat(claude_sdk): add CSDK-021, CSDK-022 TypeScript description quality rules - #90

Open
bradAGI wants to merge 1 commit into
trustabl:mainfrom
bradAGI:feat/claude-sdk-ts-description-quality
Open

feat(claude_sdk): add CSDK-021, CSDK-022 TypeScript description quality rules#90
bradAGI wants to merge 1 commit into
trustabl:mainfrom
bradAGI:feat/claude-sdk-ts-description-quality

Conversation

@bradAGI

@bradAGI bradAGI commented Aug 24, 2026

Copy link
Copy Markdown

CSDK-017/018 (merged in #40) cover the Python side of description quality; the TypeScript half was missing. CSDK-014 only checks that a description exists, so a tool whose description reads "TODO: describe this tool." or "Gets data." passes today — while sitting in exactly the state CSDK-014 exists to prevent.

The gap costs more on the TypeScript side than in Python, and CSDK-014's own explanation already says why: there's no docstring fallback, so the description argument is the entire prompt-side account of the tool. In Python a thin docstring at least sits next to readable source; here the string is the interface.

CSDK-022 also names the mitigation people assume covers this: the Zod input schema doesn't compensate. It constrains the shape of the arguments once the model has decided to call this tool, and says nothing about whether calling it was the right move. Schema validation passing and the run being wrong are compatible.

Both predicates confirmed working against TypeScript tools rather than assumed — has_description_text reads ToolDef.Description, which for TS is the explicit description argument.

Verification — engine built at main:

$ trustabl rules validate .
OK: 85 rule pack(s), 208 rule(s) valid under rule schema version 14

Fire (one tool described "TODO: describe this tool.", one "Gets data."): CSDK-021, CSDK-022, CSDK-203
Silent (full description naming when to prefer the neighboring tool): CSDK-203

(CSDK-203 is the pre-existing missing-CLAUDE.md repo rule.)

CSDK-022 pairs description_length_lt: 40 with has_docstring: true so it doesn't double-report against CSDK-014 on an empty description, same as CSDK-018.

No new predicates, so no schema_version bump.

…ty rules

CSDK-017/018 cover the Python side; the TypeScript half was missing.
CSDK-014 only checks that a description exists, so a tool whose
description reads "TODO: describe this tool." or "Gets data." passes today
while leaving the tool in exactly the state CSDK-014 exists to prevent.

The gap costs more on the TypeScript side than in Python, and the rule
text says so: CSDK-014's own explanation notes there is no docstring
fallback, so the description argument is the entire prompt-side account of
the tool. The Zod input schema does not compensate — it constrains the
shape of the arguments once the model has decided to call this tool, and
says nothing about whether calling it was the right move.
@jhumel-code

Copy link
Copy Markdown
Collaborator

Thanks @bradAGI. The framing is right, and the Zod paragraph in CSDK-022 is the part I would have written myself: the schema constrains the arguments after the model has already decided to call the tool, and says nothing about whether calling it was the right move.

Checked against main. Both rules carry every required field, language: typescript is stated explicitly, claude_sdk_tool is valid at tool scope, and all three predicates (has_description_text, has_docstring, description_length_lt) already exist in the engine, so leaving schema_version at 14 is correct. CSDK-021 and CSDK-022 are free on main and no other open PR claims them. Severity and confidence match CSDK-017 and CSDK-018 exactly, which is what I want for the TypeScript twins.

Two notes. has_description_text is a case-insensitive substring test, so a tool whose real description mentions a todo list will fire CSDK-021. That is inherited from CSDK-017 and acceptable at low severity, but worth remembering if it turns up in the corpus. Your own #83 and #84 append CSDK-019 and CSDK-020 to the same file, so expect a small rebase depending on merge order.

This should land together with trustabl/trustabl#167. It also needs a rationale doc in trustabl-rulebook before it ships, and the maintainers can take that part.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants