fix(strands-command): fail bug-verifier runs that apply no label or comment#77
Open
JackYPCOnline wants to merge 1 commit into
Open
Conversation
…omment A bug-verify run could finish green while applying nothing to the issue: the agent described a deferred triage label in its report without ever invoking add_issue_labels. Because the deferred-write record is only written when the tool is actually called, no write_operations artifact was produced, finalize had nothing to replay, and the issue got no label or comment -- with no error anywhere. Track write-tool invocations and, for bug-verifier sessions, fail the run unless it applied a label or comment this run, or (for a resumed session) a triage label from a prior run already exists on the issue. Session resumption is preserved.
Contributor
Author
|
/strandly-the-agent is this fix correct? |
mehtarac
approved these changes
Jul 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
A
bug-verifyrun can finish green while doing nothing to the issue. The agent describes applying a deferred triage label in its report, but never actually callsadd_issue_labels. Since the deferred-write record is only written inside the tool, nowrite_operations.jsonlis produced,finalizefinds no artifact to replay, and the issue ends up with no label and no comment — with no error surfaced anywhere. Observed on harness-sdk#3216: all jobs succeeded, the agent reported "Labels applied: bug-needs-info (deferred)", yet nothing landed.What this does
Adds a post-run guard for
bug-verifiersessions. A run must end up applying a label or comment to the issue, or it fails (non-zero exit).bug-verifieris affected; other modes return early.Out of scope
Does not address the SOP wording that nudges the model to narrate a deferred write instead of calling the tool, nor the case where a label is applied but doesn't exist in the repo (fails later in
write_executor). Happy to follow up separately.