Repository navigation
Description
For Scott
What: The pre-push security scan in josui-skills crashed instead of finishing, and because it is designed to fail closed, it refused every push to that repo until I went around it by hand.
Why it matters: The scan failing closed is correct — a gate that silently passes when it breaks is worse than no gate. But a gate that cannot run is indistinguishable, from the user's seat, from a gate that found a problem, and the only way past it is --no-verify, which skips all the local gates rather than the one that is broken. Teach people that reflex and the next real finding gets skipped the same way. If we never fix it, the fallback becomes habit.
Recommendation: Give the scan enough memory to finish, and when it still cannot run, let it say so and defer to CI rather than block. Size S.
Needs from you: nothing.
Where it fits
Feature or program: standalone, python-sdlc tooling. Related: crunchtools/josui-skills#91, which had to be pushed with --no-verify and merged on CI's verdict instead.
Why now: it blocks pushes to josui-skills today, and that repo holds the skills every session loads. It can wait only as long as nobody needs to change a skill.
For the agent
Problem: sdlc-push runs codeql database create then codeql database analyze over python-security-extended.qls. In josui-skills the database builds cleanly, then analyze dies:
[2026-11-08 11:27:28] Calling plumbing command: codeql resolve ram --dataset=/cache/codeql/josui-skills/db-python
[2026-11-08 11:27:28] Plumbing command codeql resolve ram completed:
[ "-J-Xmx1253M" ]
[2026-11-08 11:27:28] Plumbing command codeql database run-queries completed with status 70.
[2026-11-08 11:27:28] Exiting with code 70
The gate then reports GATE CANNOT RUN: codeql database analyze failed (exit 2); nothing was judged and the push is refused. Failed on 3 of 3 push attempts this session.
1253 MB of evaluator heap is the number to look at. The repo has 4583 *.py files. Partial results were still written (results/ appeared 12s after exit 70), so this looks like the evaluator being killed mid-run rather than a malformed query set.
Two separate faults, and the first is already fixed by hand:
- A previous run had been killed mid-build and left
/cache/codeql/josui-skillsholdingtrap/andworking/with nodb-python. CodeQL then refused to overwrite it:Ran with database overwriting enabled, but the directory does not appear to be a CodeQL database or database cluster. Please check you do indeed wish to delete it, and do so manually.I deleted the directory; creation works again. A half-built cache should not require a human to clear it — the gate should detect an unfinalized database and rebuild it. - Analyze then failed as above, which is the open part.
Measured, so the scope is known rather than guessed: this is not a general failure and not a simple function of repo size. Every other repo in crunchtools-hook-cache/codeql/ has a complete db-python + results + src.zip, including trentina at 8245 *.py files — more than josui-skills. So something about this repo's closure, not its file count, is pushing the evaluator over. Worth finding out what before raising the limit blindly.
C=~/.local/share/containers/storage/volumes/crunchtools-hook-cache/_data/codeql
for d in "$C"/*/; do [ -d "$d/db-python" ] || echo "unfinalized: $(basename $d)"; done
Files: the sdlc launcher's codeql gate and wherever --ram / container memory for it is set; [tool.sdlc.push] in a consuming repo's pyproject.toml for how the gate is selected.
Proposed fix:
- Raise the evaluator's memory, or pass
--ramexplicitly instead of lettingcodeql resolve ramderive 1253 MB from a constrained container. - Treat an unfinalized database in the cache as a cache miss: detect missing
db-pythonand rebuild rather than failing. - Separate "the gate ran and found nothing" from "the gate could not run". A gate that cannot run should fail the push only when no CI will judge the same commit; otherwise it should warn loudly, name CI as the judge, and let the push through. Blocking with no selective override is what makes
--no-verifythe only door.
Done when:
git pushon josui-skills passessdlc-pushwith no--no-verify- Deleting
db-pythonfrom a cached database and pushing rebuilds it instead of erroring - A gate forced to fail to start produces a distinct, non-blocking verdict naming CI, verified on a scratch branch
Out of scope: changing which queries the suite runs, .trivyignore, and any narrowing of scanned paths — reducing coverage to fit the memory limit is the wrong fix and would quietly shrink what gets scanned.
Activity
fatherlinux commentedon Nov 8, 2026
Item 3 is unblocked. I approved the XII amendment as CTO on crunchtools/constitution#99; it merged as constitution#100 and
constitution.mdnow carries exit3, "could not run, deferred to CI", at version 1.35.0.The amendment grants
3on tighter terms than this issue proposed, so the implementation has to match them:3is granted per check by python-sdlc and is never a status a tool returns, today only CodeQL; it requires push tier, outside CI, that check also in[tool.sdlc.ci], a workflow line that actually runsbin/sdlc ci(not a comment or a job name — the Gatehouse finding on #34), and the tool itself broke. A config error stays2. Anywhere else3is2.So the work here is to restore 6a11c8a plus the 33c9b9e shellcheck fix on top of main, with
ci_judges()tightened to thatbin/sdlc cimatch. Items 1 and 2 ship in #34 as they stand.One sequencing note: the constitution refs were bumped to
@v1.35.0on main but the annotated tag is not cut yet, so pin python-sdlc's constitution reference only oncev1.35.0exists.fatherlinux commentedon Nov 8, 2026
Correcting the sequencing note above.
v1.35.0is now tagged and released (annotated tag on d7261ac), so the literal condition I gave is met — but do not move python-sdlc's constitution pin on the strength of it.Creating the missing release objects for 1.33.0 and 1.34.0 exposed that the whole fleet, this repo included, is still inheriting v1.31.0;
scripts/fleet-drift.pyhad been measuring against v1.32.0 because it readsreleases/latestand those two versions had tags with no release. The fleet bump is one decision for all 70 repos, it retagslatest/mainand touches systemd units on gloval, and it is Scott's call. It is in front of him now.So: restoring 6a11c8a + 33c9b9e with the tightened
ci_judges()is unblocked and can land against the current pin. The pin bump rides with the fleet, separately.fatherlinux commentedon Nov 8, 2026
Reopening: this closed on a commit message, not on the work.
#34's branch kept 6a11c8a's original message, which ended "Closes #33", after the deferral code had been stripped out of the diff. So merging 16c4c2f closed this issue while items 1 and 2 shipped and item 3 did not. Verified against
origin/main: noDeferredError, nocodeql_broke, noci_judges, noSDLC_TIER, noDEFERRED.What shipped in #34: the
--rambudget and the per-projectflockwith a proven-empty database directory. Those close items 1 and 2, and they fix the josui-skills blocker.What remains, and it is now unblocked rather than waiting:
Constitution XII was amended and ratified — exit
3, "could not run, deferred to CI", in v1.35.0 (crunchtools/constitution#99, merged as #100). The implementation has to match the amendment's terms, which are tighter than this issue originally proposed:3is granted per check by python-sdlc and is never a status a tool returns, today only CodeQL; it requires push tier, outside CI, that check also present in[tool.sdlc.ci], a workflow line that actually runsbin/sdlc ci(not a comment or a job name), and the tool itself to have broken. A configuration error stays2. Anywhere else3is2.The reference implementation is 6a11c8a plus the 33c9b9e shellcheck fix, in #34's history. It needs restoring as the deferral subset only, since its memory and locking halves are already on main:
DEFERRED/DeferredError,ci_judges(),codeql_broke(), theSDLC_TIERexport and pass-through,run_step's demotion of a non-push3to2, thedeferred()warning, the3branches inrun_serial/run_lane/_close_check,run_plan's own_plan_rc, the usage text,docs/codeql.md, and thedeferral()case intests/test_codeql_resilience.pyplus theSDLC_TIERhost-contract case.ci_judges()gets thebin/sdlc citightening, which the reference version does not have.Note for whoever picks this up: this repo carries the
ashigaru-observetopic, so it is watch-only and Ashigaru will not take this itself.Done when: a CodeQL forced to fail to start at the push tier, in a repo whose CI runs
bin/sdlc ci, prints DEFERRED TO CI, records it inlast-push.json, and lets the push through; the same failure undersdlc ci, undersdlc codeql, withCIset, or in a repo without that workflow line still exits2and blocks; and a config error still exits2.fatherlinux commentedon Nov 8, 2026
Items 1 and 2 shipped in #34 and are released in 0.5.4 (#35).
--ram= ¾ of the memory budget: 6144 MB at the default 8g, a 4.6 GB heap where it used to get 1253 MB.db-pythondeleted, and again with a read-onlytrap/left behind.On the cause: a clean josui-skills checkout analyses in 9s (13 tracked
.pyfiles), and CodeQL already leaves out.venvand gitignored worktrees. So the 4583 files were not the cause. Two runs sharing/cache/codeql/josui-skillsfits the evidence best, includingresults/appearing after the exit 70.Item 3, the non-blocking "deferred to CI" status, contradicts constitution XII ("Every check exits 0, 1, or 2"). It is proposed as an amendment in crunchtools/constitution#99 rather than shipped here. Leaving this open until that is decided.
feat: a push may defer a broken CodeQL to the required `sdlc ci` (#33)
fix: read the parsed workflow, not its text, before deferring (#33)
feat: a push may defer a broken CodeQL to the required `sdlc ci` (#33) (