Fix Phase 3 behavioral-claim verification and command-substitution rejection - #93
Merged
Merged
Conversation
…docs Phase 3's Localization verification only confirmed a named symbol exists, not that the issue's description of its behavior was correct past that point — add a third verification step so downstream doc cycles don't paraphrase a wrong claim straight into project docs. Phase 4 step 5 and Phase 6 prescribed a `--body "$(cat <<'EOF' ... EOF)"` form that at least one harness's command classifier rejects outright before it reaches the shell; switch both to `--body-file` / `-F` via a Write-tool scratch file, and name that constraint next to the existing scratch-file-handling rule. Also name the sandbox's session-scoped filesystem restriction (gardener-dispatched sessions) as a trigger for the Phase 4 anchor-cannot-run fallback, and add a retry note for transient `gh` network errors to Edge cases. Closes #90 Closes #91 Closes #92 Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Member
Author
|
Self-review rubric:
Summary: mechanically small, well-scoped fix for three self-audit gaps. The one real gap is that manual fixture-regeneration validation could not run in this sandboxed session — flagged UNVERIFIED and this PR is left open for a human (or a future cycle with fixture access) to run that check before merge, per this PR's own anchor-cannot-run rule. |
Member
Author
|
Skip reasons for the remaining open issues not picked for this cycle (triage recorded 2026-08-01):
|
5 tasks
dmccoystephenson
added a commit
that referenced
this pull request
Aug 3, 2026
…docs (#93) Phase 3's Localization verification only confirmed a named symbol exists, not that the issue's description of its behavior was correct past that point — add a third verification step so downstream doc cycles don't paraphrase a wrong claim straight into project docs. Phase 4 step 5 and Phase 6 prescribed a `--body "$(cat <<'EOF' ... EOF)"` form that at least one harness's command classifier rejects outright before it reaches the shell; switch both to `--body-file` / `-F` via a Write-tool scratch file, and name that constraint next to the existing scratch-file-handling rule. Also name the sandbox's session-scoped filesystem restriction (gardener-dispatched sessions) as a trigger for the Phase 4 anchor-cannot-run fallback, and add a retry note for transient `gh` network errors to Edge cases. Closes #90 Closes #91 Closes #92 Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three small
template-ruleself-audit gaps, all localized tocreate-dev-loop.mdprose and none requiring aCLAUDE.mdedit:--body "$(cat <<'EOF' ... EOF)", which at least one harness's command classifier rejects outright asContains command_substitutionbefore it reaches the shell — confirmed live in this very cycle (see test plan). Replaced both with aWrite-tool scratch file +--body-file/-F, added a standing rule next to the existing scratch-file-handling bullet, and added a "retry transientghnetwork errors" Edge case.Test plan
python3 scripts/check_docs.py— passes (every{{placeholder}}has a Step 4 row, README's "What it does" still 1:1 with Steps)./create-dev-loopfixture regeneration not run. This PR changes template surface (Phase 3 Localization verification, Phase 4 self-review/anchor-cannot-run text, Phase 6 commit form, Edge cases) that the CLAUDE.md "Testing changes" checklist would normally exercise by regenerating a fixture skill (e.g. againstdpm-dev-looporherald-dev-loop) and diffing. This session's sandbox restricts filesystem/tool access to only this checkout —ls ~/local-skills/,mkdir(even for a path inside this checkout), and cloning an external fixture all fail with "Claude Code may only [list/create] in the allowed working directories for this session." This is exactly the gap [self-audit] Phase 4's anchor-cannot-run guidance doesn't name session-sandbox path restriction as a trigger #90 (this PR) documents, and the same gap the prior cycle's PR Document gh pr create workaround for restricted fetch refspecs #89 hit. Per the anchor-cannot-run gating this PR itself edits: template surface changed → do not auto-merge, hand to a human who can run the fixture.... && echo "exit: $?") was rejected mid-cycle with the same class of classifier block the issue describes, before any template change was applied — direct evidence the prescribed heredoc form is unsafe to keep as-written.git diff) — every changed line is one of the three issues' named surface area; no unrelated formatting/renames.Closes #90
Closes #91
Closes #92
No RESEARCH.md finding directly covers harness command-classifier behavior or session-sandbox path restriction (#90, #91) — these are execution-environment constraints observed empirically in live sessions, not covered by the cited literature. #92 extends the existing localization-verification instruction grounded in RESEARCH.md §3 (fault localization as the dominant uncontaminated-benchmark failure mode); no new citation needed since it sharpens an already-cited finding's application rather than introducing a new claim.
This PR description was drafted during a Gardener session (Stephenson-Software/gardener).