From 1409cf7ef4a3bca09e6ef325c592e2d655e71c8f Mon Sep 17 00:00:00 2001 From: Scott McCarty Date: Wed, 4 Nov 2026 09:02:30 -0500 Subject: [PATCH 1/5] feat: v0.3.0 a command's checks run in lanes, at the same time `sdlc ci` ran every check one after another, so trentina's image build -- which depends on none of mypy, pytest, semgrep or pip-audit -- waited for all four. The job took the sum of its checks rather than the length of its longest chain: 5m09s on the self-hosted runner, 16m05s on a GitHub-hosted one, of which the image chain was 2m34s and 11m21s. Each plan line now carries a lane as its second field. Steps of one lane run in sequence, in plan order; lanes run at the same time. The image assigns them, by two rules that name no repo: an image-build check and every pytest-in-image check that runs the image it tags are one lane, and everything else is the main lane -- which is where uv sync happens, so two syncs can never land in one venv at once. A repo with no image-build check gets one lane, and one lane is the launcher's sequential path: the same run, the same output, as before. The CPU budget is for the whole gate and is divided between the lanes, so two lanes take the share of the machine one lane used to. Each lane's output is buffered and printed whole, in plan order, and each block ends with how long its check took, because buffering is what the `== ` timestamps used to tell you. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R7Y1jesjfgxHeqtKJCvay6 --- .gitignore | 5 + CHANGELOG.md | 52 ++++++++- Containerfile | 2 +- IMAGE_TAG | 2 +- README.md | 35 +++++- __pycache__/sdlccpython-314.pyc | Bin 37362 -> 0 bytes bin/sdlc | 198 ++++++++++++++++++++++++++++++-- pyproject.toml | 2 +- sdlc | 79 ++++++++++++- tests/contract.sh | 53 +++++++++ tests/test_host_contract.sh | 170 +++++++++++++++++++++++++-- tests/test_plan.py | 178 ++++++++++++++++++++++++++++ 12 files changed, 744 insertions(+), 32 deletions(-) delete mode 100644 __pycache__/sdlccpython-314.pyc diff --git a/.gitignore b/.gitignore index 7b3690a..353e6e4 100644 --- a/.gitignore +++ b/.gitignore @@ -5,3 +5,8 @@ # gourmand refuses to cache without this entry and then writes the directory, # which is also why its hook mounts the tree read-write. .gourmand-cache/ + +# `python3 sdlc` and the unit tests import this repo's own `sdlc` module, which +# leaves bytecode beside it. A committed .pyc was tracked until v0.3.0. +__pycache__/ +*.pyc diff --git a/CHANGELOG.md b/CHANGELOG.md index d922535..4bba27e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,55 @@ All notable changes to this project are documented here, following ## [Unreleased] +## [0.3.0] - 2026-11-04 + +### Changed + +- **A command's checks run in lanes, at the same time.** `sdlc ci` used to run + every check one after another, so trentina's image build -- which depends on + none of mypy, pytest, semgrep or pip-audit -- waited for all four. The job + took the sum of its checks instead of the length of its longest chain: + 5m09s on the self-hosted runner, 16m05s on a GitHub-hosted one, of which the + image chain was 2m34s and 11m21s. + + Each plan line now carries a lane as its second field, + `check TAB lane TAB mode TAB payload`. Steps of one lane run in sequence, in + plan order; lanes run at the same time. The image assigns them, by two rules + that name no repo: + + - an `image-build` check and every `pytest-in-image` check that runs the + image it tags are one lane, because the build has to finish before the + tests start and two builds writing one tag must not race; + - everything else is the `main` lane, in plan order, which is where `uv sync` + happens -- mypy, pytest and pip-audit each sync the project venv in the + shared cache volume, and one lane makes those sequential. uv does lock the + target environment, but a gate should not bet its venv on a lock it did not + take. + + A repo with no image-build check gets one lane, and one lane is the + launcher's sequential path: the same run, the same output, as before. `sdlc + push` has no image chain, so push is unchanged too. + +- **The CPU budget is for the whole gate, not for each container.** It is + divided between the lanes, floor of one CPU each, so two lanes take the same + share of the machine that one lane took when the checks were sequential. + `sdlc ci` on a laptop is no busier than it was. + +- **Each lane's output is buffered and printed whole, in plan order**, as each + check finishes -- a log that cuts two lanes into each other is a log nobody + can read. Buffering collapses the timestamps people were reading the + `== ` markers for, so every block now ends with `== passed in + Ns`, or with how long it ran before it failed. + +- `sdlc tiers` prints each check's lane when a command has more than one. + +### Fixed + +- Two lanes starting on a machine with no cache volume could both try to create + it, and the one that lost the race called it a gate that could not run. +- A `__pycache__/*.pyc` was committed, and nothing ignored it. Running + `python3 sdlc` or the unit tests left a modified tracked file behind. + ## [0.2.2] - 2026-11-04 ### Fixed @@ -182,7 +231,8 @@ cached tag was never refreshed — that gate had been running a two-week-old reviewer while CI pulled fresh. Local and CI were different reviewers on the same diff. -[Unreleased]: https://github.com/crunchtools/python-sdlc/compare/v0.2.2...HEAD +[Unreleased]: https://github.com/crunchtools/python-sdlc/compare/v0.3.0...HEAD +[0.3.0]: https://github.com/crunchtools/python-sdlc/releases/tag/v0.3.0 [0.2.2]: https://github.com/crunchtools/python-sdlc/releases/tag/v0.2.2 [0.2.1]: https://github.com/crunchtools/python-sdlc/releases/tag/v0.2.1 [0.2.0]: https://github.com/crunchtools/python-sdlc/releases/tag/v0.2.0 diff --git a/Containerfile b/Containerfile index 2586ab5..49c4431 100644 --- a/Containerfile +++ b/Containerfile @@ -44,7 +44,7 @@ LABEL description="Every CrunchTools Python SDLC check in one image: ruff, mypy, LABEL org.opencontainers.image.source="https://github.com/crunchtools/python-sdlc" LABEL org.opencontainers.image.description="Every CrunchTools Python SDLC check in one image." LABEL org.opencontainers.image.licenses="AGPL-3.0-or-later" -LABEL version="0.2.2" +LABEL version="0.3.0" # What the image carries, readable without starting anything. `sdlc versions` # prints these and a consuming repo's CI pins them. diff --git a/IMAGE_TAG b/IMAGE_TAG index ee1372d..0d91a54 100644 --- a/IMAGE_TAG +++ b/IMAGE_TAG @@ -1 +1 @@ -0.2.2 +0.3.0 diff --git a/README.md b/README.md index 44f835d..82e5d63 100644 --- a/README.md +++ b/README.md @@ -36,7 +36,7 @@ default_install_hook_types: [pre-commit, pre-push] repos: - repo: https://github.com/crunchtools/python-sdlc - rev: v0.2.2 + rev: v0.3.0 hooks: - id: sdlc-commit - id: sdlc-push @@ -69,7 +69,33 @@ laptop and CI both. One pin. | `push` | mypy, pytest, gourmand, gatehouse, semgrep, pip-audit, constitution, codeql | | `ci` | mypy, pytest, semgrep, pip-audit, constitution, codeql | -`sdlc tiers` prints what each command runs in the repo you are in. +`sdlc tiers` prints what each command runs in the repo you are in, and which +lane each check runs in. + +## Lanes + +A command's checks run in lanes. Steps of one lane run in sequence; lanes run +at the same time, so a command takes about as long as its longest chain rather +than the sum of everything in it. + +The image decides the lanes, by two rules: + +- An `image-build` check and every `pytest-in-image` check that runs the image + it tags are **one lane** — the build has to finish before the tests start. +- Everything else is the `main` lane, in the order `tiers` prints. That is + where `uv sync` happens, and one lane keeps two syncs out of one venv. + +A repo with no `image-build` check has one lane and runs exactly as it did +before lanes existed. So does `sdlc push`, which never builds a container. + +The CPU budget is for the whole gate, not for each container: it is divided +between the lanes, floor of one CPU each. Two lanes are no busier on your +laptop than one was. + +Each lane's output is buffered and printed whole, in the order `tiers` lists +the checks, so two lanes never cut into each other's log. Each block ends with +how long its check took, because buffering is what the `== ` timestamps +used to tell you. ## Configuring it @@ -192,7 +218,10 @@ then runs the reviewer CI runs. A developer machine needs `git`, `podman`, `pre-commit` and `gh`. `bin/sdlc` is POSIX shell, runs only `git` and `podman`, and refuses any step that is not a `podman build` or `podman run`. `tests/test_host_contract.sh` proves it with a -restricted `PATH`. +restricted `PATH`, and proves the lane executor there too: that the lanes +really overlap, that a `2` in one beats a `1` in another whichever finishes +first, that a failure in one lane does not stop the other, and that no two +lanes' output interleaves. ## Licence diff --git a/__pycache__/sdlccpython-314.pyc b/__pycache__/sdlccpython-314.pyc deleted file mode 100644 index 57304124fabf714dcb7da9ea8d490da299b0eb87..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 37362 zcmd75dsJN4nJ0Rx>eRca;tdi=90>^t1cij&=w*aJLJz=42q`ual&AvA2C95cA&H8U zkjLC0I~kQa$*A1+jOmVN8qc^p%Dw4{dRA7qopBP+{8gxs!4=Jody;imXV#jvP$ab- z&)n(z`}TPisAS9O>9y{W*k{){`|Pv#K6`)f@7r6vc`gn|(O>+x=l*?|<965a@-Uraxv}#Cz?7;$ILNv73IwxmY9W|`54d6)|i!@Z7~}=+hcZicElX)?2I|t z*%fo)Z0T?x^Ta%AU3sy*^PY9wd24s2)&%3+U1j98oXCsT9cJa%4wG_@<$umPY@+>< zItoO`=V*mfbZIq+?$7a@C+oSq&+(i$>p9=&crHKdxq{E}Tw&I8MW5rj;;iRNKF4#V z#&eDKSfS{^zcRmB%)>uA`|yv>1^7qjB0ncS(`6D%@Q;4GMKhi&63fdtaa9>Nnjb5^ zz{N@~m|~?D%(1c(ZXGAC#=VO3sFUdbPL~`6`yL5C!Ddv3#Yt9{FmP zUn6cnek04T6>E^+#PU_*X3;Fx;;&BJ(r6Luai@XqjGAMD=*O={R&JfR75Qx}zg}!Y zeml!oi#w3t$?_Y-UC8fd`HkWpDunIPdzVo$W!TCC|=lmA2=zM*nE7l~Q!IQnL-tFgi zh-delhEzfa1BOxI?-Vh!O42Oh~!03<=84_Z+*eXqo503OiBBQdla zoNN__1F?|b}@x zD27|(p-@oPpV%Y|!{L}KBH?=MazuJwKqr}t&K|L)wWl6!3lEM^BdNicM}?Hv2xo!rJNiY3mGMwmX&PnI&<=f(Am`^h-W%o zt|n^MI1Lv1BV0uDFc>uC61~wcw6L$J4B=p494!#==~$cSi_mCqWy2qlQHvF=2}(a>mkJR}T9LKu4~6jCY~k6erdCxy{SL3|#)QlG=y)y~kmdFG_yaWEWxqT!APk34GJ?(y2GD)BO3?JnQbeu(a!3ebG%kiD zJTf{d1j9o^A*mVjR(1`%_@Y3a9}va@Q5o|jI2IU3HzJqEg>6kc8@4s=M%DOMp)28- zu&rK@Bf|Q@$iyg{9$06WBG0oScVUoX7|)Qjo(+Y>R+22NH)fYItMu8j-Y*)+ zuN;~nkr?`nYBA;2wqV8t!-H(;1%xKQtJaipv-uIAk0N*CH{TtIjbxkz3;M;j?yk6% zt>TP}U1@D;J=PYtH4X+a)iaLa$b^KR1~Vn>?$NF@;;EL-BmKQ?;^~gA&WwFx{P}SV zLdHqsJQBgg%9JWaTYB1#b)7lgrkCZViJ_rPf%4)r?d|#vyBr$B+>2(4*}cTCLj#6N#+VEexu=qR9N;MjL~gn-AT!1Fau>{dz5&#NYn9y7rV*290z$Tc zECAZ5<$p^kAj6Lb#zGm39FsCODHNNK#^2;H&bFZlhAt*k-Rm)+a z#%?L{Txc*RKMv+HaGLS-_p4to)ZZ_yK}~J=lUE~|;y&W?P4)LENm`cZG-F}E?&B`U zxQtoMh|m?^H0U8JNNS63droQAccre4z!>$+&F24?N!wX)qdEoYq9alOjm zST3v;`Q4!O>4=(p%-tR(pPlYAZBl;@s~8aFyxAp`d{#Nu5@2jwcfq6OG|b*z^!R&G zOOHi+YY^N@CD#L=qX`vV0dd5c6J2z zV6e{;NTM|gOdAV~jskiCR|cd>0HIMJX26!v=n&x4Spe-31;P=y8xLK33UGs)X{~1+b;i3 z!Yl;EC&thS7u^%0Xt~k@Mq!{r2HXOI4D1MGIUIRs4q^00r&_-TnP-u zMkf_0WnlfW!1KTx0bvs=N7ptfNSm;ja>4J)m>Sx9YWa+nA>532Fc8JU3iU@OV$q3M zhL2%|X3n!F8BBYs zr`i_np09oB17G2i*M5gujFDniga6c`mIK0*J(?%20L0r@i0&+M^ z_Z84|o{O18E@r-PA!ZRxF&?z9RkRSu%y?TFL=z{*+1SRr#Xwlb6a}=*iQ(AXdlbX@ zg~OpiKvbGzz__g~ogkk2K|J{-YC>nNE#v6#C&=92pKic72sd90?puJUA2+O;OduQA@L)lLfpZ~jc2%LYX{fYP7Y`ymv{ z>itIQ@5bpt>ixz zd0u4auu13eoq?w?NIf7Q2A(Al=qDcg`~dLm#ZfQ>%NW?M&Ni?wgQEcnz*G(e#)K$i zkcdBbX-p6V1X8d;VcTL7vDV`Oa1PPPXn1fEGzx$%abVHW0MMe)&>##PZ9CP`+0oz9 z-O+!d?F$1g0@Fm#$P?ihnlKyLiTXyM)&}DC6edh9DFGO} zl&b;(G=&`ltpuV$5mqz|hXCiogDSN#G#U!U8ZMKxLLLP2jiD=eX@po$kQ(46@mh44 z@w(vsCdNQ>#r#5>A#fmZ0DCHcff@>cair4hl~QP2{)jBggvC0{HSO&c?8UpEwZwpyL#yT1a1m%pAreQx(rBWRh ze8$()b?W31P$^Isr;aSTeb@G1-GA-iO9vNy1=CI<5F76LO0RwCr7tCWUcd0l zg_~!S7t(7EqMfJ&E?@;k3JI!CgJ)uAXV1#gE1X_s*1i=gp=Rzw_egP)8Q|<;L1RL%;yot(WFxflPG&4AJCgt0*;HaN-)c+464;wy_iSqrknwGuX zJ9{e*H(GvLTyVI~^3ysi^2^AdKM0jchfw=xVzR3!q+>F>$#suILZ}|6M==?VH~y^m ziAH%mCVL_*2A;K&>m%ZU9irKQ>_j4{WWwPVZT8aYq{XO;WYvjU&wvkGJt|C4xw980 z!dUBAUw-K%UJ(E5<#SmGlHiU249H+P#(*VTRSIZf;0~OSwqUg7=wBwQtB3ec5%Vxu zqf2s_)DiE-)ky{<_Cd-xG*mB)O-3i{0q6*TIHZGoV$L(?Z55zV|p%`&3Rp>zqjyC z*F6j8X?&E7Qbi!9`zp1Roq4mX71WjOF>zBqPQ%b`^(w z;vB`mKMRiVJR`k6bEsHNAE&z<2X%zkdd z+>X{p`969pJ9_zSdM_7cmYdc)0m{0U?=+vl$8>UQxu`YD_gb~L4}&7cO42Dw5z;5n zOXgn7a(by3QkDs5jVH`eJ`8Xgf4haz90Gj@VMvOMEvJ_1h37*d$Q1}nL8{9%5^S-^ zKsfE;QHZ7KCPa0}$<1tvv$?Fz>x+;zD_rJyWV|6Bk|0^YnowS9MqpopR}gN5KlM>GXcd#Bxw}4*i;=yW{O+# zaNdGzXRn@3oKEMhpK8BrFG$q9R=r?fd)vNtv9LT*K5hG8wUGAid8H=t^cy|1E&ux5 z!iJr58+I;i*nRDV=@(w9nL3*C?)jCwP(dey$up_-`)~E$I+pUa%{$r`-Gx(~zyF<` z^X-*s4*a0HrI~-H%!c$Ik3>H45yBKtZ09k8^3cHh&O#fSBhNIx&I7oV_g{8AWdp)rzgY7EcB zr3O&Dptng6#prwCQfT~=O0-fI0O%5svASTH0!6qK83iXoAZQW~UYyjQg%U+00aQhV z-{8_D5RMY`qL)V$#sg0?d@2hBiy-KQ=$El{5a|XO^aA{FLA?R-=JB@Hp8mGZ-VU*= z^Hf`Bj}RD?BC@P%v#3WZ<)OY-2IPZkYT^ZyAS zAO=B~3^Jt>qg|*V#I_!$Xi=MyG5Iqdw2JAJ`j2#o5>2N}Y3_z*czS8v(h!}LHd47F zD))t+V_lv7N4m~-p6qHlaynyWTG0%Dtm{-;oNtVcMQgnob41SA@i2Wf1|St0gGpp9 z@}!*csGaQ(j}JvM*0I2I5c66gu5TI9{(pYOr-1aUTePEeC^=XgGv8e8-7sxW^LNNW9rCVU*UqU zYR*@+;M+Lo+nDm!%mm)DENtF0w|URP=7V#a52iL9N_n1|YP;+5eoML*zZy@wR!_Aq z^7d~Y|LXA%J*!`9eZxA-|7%aWvSHD);f=w!Hr}lHbN{X2t=_k{zdiAbFTL}n)R}XT z5~q)zPqkl2?d!i&*PpH#SoEx$*$ts_UQwcbDbMb7J+L^fwxwduW1lelulkqglG^0`VrDo}&Dol3G+ai8%P9dQ`1*T~B%kdsgPzNq!op{%~= zK+OMpzk+DfRiNN0qSgxT9N#S%eb91Z-d<#b)ohmN=nQLWnPL*?&W*n2cpi5=+2X|9 zrOsKm1HD0nsUu$~E}>5##buxV)}BY-YH!M7XxDHj2m30umOo()jukb0qmR@l>_82y z-%*><2GN0dgR=Es~wLGU=vElWsDWMU$)vGKnU^Qd|x}5{I))k~D}qU7gUF z+B25H%Rv=>hsV(=X&r`IB1kPYP=W#H7=DGy@+Ha>te0MIjnV@O&-vch+-I8MX5)?KwEnUV`TWkL9mr?GwUqMn^ zAQ2xd)l$Nw0)&gDE0jzkfvHMIcU#7Gy5~sOnI0)l*Qp~JD;X7qq;|>|fQ1WnMq=$` z8D+@YGQRGRG!~Z0#6!7SN2MX%B?AqT%hlSYZ&Hn{|AcEK!YwuU!8MP77t5=gKxOmaKYE;4Jn|A9+yB<>g;He)af*r*h6y z30dB=uROc3#-Cc_pFKS9X}s(48F%MBYZuEZZhZ0j7g0^BZ1c>;tH*xxB=3LVw$aO$ zyj=O}tH+iKIfv()U-{}+l6CX;4aA>SCj#HANtCBn?YtTMapZ@Q)Xw&|11aCJc}E9H z9uHkTlsuMlZ%FYQK7LTbdDf7eaNDv6{V8AL`;M)5-K*&y32BTidqx(n&v zR=T$)DbH;CmzR=V)Xy#;u>oM={u6v$y4OFCDfa{b|8&r2o-r zp)~Ix(x2rC?fWf1YjjY$rxfYW^C|!HRVGSTm$&b>{Cu|+`DL^Wk9sXns(q|gRI-d5 zG*q%rCZYWAc#*D?EF}pg5;EMuX$BfuoR-rSD9>4y)>-en*c3DH*9w&zL?urotn@G} zL$$IaTUZI2#ni1+8(Ikt0!&Z@csm_YGng)7y&O%Jb7ta{f(6TKWOtp}5MQ)G2#P=8 z9-Xn(Fb4!vMq8{sBwJewG@cC6bl5(A?IL(vd$B6vF*f%O-g>uH!y~x8uY6Sk>8s3vp9q z^pe`ztgd4$2A`KsVQNX8Na7{gY8bU*=wjjyGS?6e2&3w2!O$gjFrWnrJ5RBLAYV=-QXBetG8d^q14VMy4im6(%+$N)s=n z?5k#q=j|Ku07^{{Epe8TjY#rp{@!9`Qg%lTX^T8X8hB3?)Xc~;A%OM;N5h<>fkAG+ zq8++f*iyy4Q{_Hvw!Bm4Z8>0h=YSRYe>@~+$eSdTitPP~bVW9R4jmFZ2ofVnj4#n? zwwV3;6i7V8hS#u-5lvZ~3^0juz>&n-W$K0KO|uxe430x8>@{8B_FM+s1n;#>ZDcWO z5utgYzgk;-`(b8Dmput-k=F}QB$GKZ(SMr^f(xqlP5$^A$a3V#@j*;znjMfkH8g;- zY#`-iK!(*04O;xmkCdVcLohDy0RW%}nQjAmrxz)fc@a+p^}1#oix2{ec~;hnB+_v#8Xp0acFJ*#FO1vh+dANu?oG_iX1Lb z>oaC)Fk|XxVi>Fl#rViWHEEh(=?xTPUz(UjXpVSC%DH)-uLb>dJHPpD%e?2vSKFoz zFWQ~cPk-YwrumZDly@3BB#Sb;ea0NXd|?NzKYU@+eF2%rnc@(>%rf6nr0|TcW#(HJ zG7b7H>2q=R4CTo#2T3BhM5m<8ei^M}O+!MmeAc=@X00=3sKG5~sew2V9F%DGn>EKJ z2zw1&j6UR|1$yinI%ZPfV7J$ z1aD{<3SSX2CIMfZq$>(@Vr7?M88<+eMk%5Bk>Q_%;|7~0Z<^VN!w8Es88Q|fK&CoB zV*T`jel`WAAL0%h=MH41xVwD*H#+ZnN)m&~{j(cymZfURKljSH znb52Ksmg|w(?8EQ-gSF#`?k&QO!>B@-Ay;Q-GLUJ-)Vf7KWNfO{|1fu%;{0Fe0un4 zLiiIgnjU_#0msQ`2P$O8|1YKoDOdgG-cOqzcIi#DSNgY<{5vFZ51SPrV4$HBq_^lg zn-C02TokZqSKlSwfq~&$TD~_3xSAAfoQqJ0Wz7AhH zVM{tvzSrJtv;!Sv2MB!y z7+k^AuyD}Yr+dIb?4u(qQ8(|zRUHUuU;}_bXA2AffRg&62bv>LV$wm$XX%@Cn>e+# zI6*`7slrB0qXU*f*DsORZg&pxV6VF4f!V=S{3MpP z4N$d6{|Z-XeG1$oQjh5w7?M-Kpe{67q5n|u=U>nhwUm(jPh!FsUO`QSPNkca&`7Q@ zSGUFbsgcrO;TD^RwEVOc#6tr-&#$`cDtWnKsuchoJIH_ESbS%F(T_cI^t6yAQDO)z?1% zn0_*&9gs4F?Quk{hTb|Gwt+Z-l~6AT^Qf(t7cGns_Y)y5+WU%?X90;E*=s)+d?9s~ zCKB0;$AW;iu$!@wh~96m*L#kZ*!pysb2cD)9t$bF8Jl8Gm2nU)M4Cj^Melgm;nNw@ z){I-Z-w*RJIc{!vArAC2)E4(Bmm99Y4Jalh(V8rF+T1X%01i8&wO0Tb=_e?Ss-p4y z@klcRHIS8Df&fCQ&07g+Hlb|B3aRUutiX^r=v7xJnWTh~={=M?M+w0i2v{#toV z`TFH#JY80|P_{KywsrR6Lh0_g(%tWu?!D@ow%jeRNVdI-%(q$-mY0vcoCh1RvgY4t z04obXZJCkN?k&VId#~-jx;t@TX2)#lj6dz(27N66R{1wNe`j{NZ4j73D_DfwRO?bH zgs0Q;H@=wo@>?5juDVm#yinUVSKId9L-@3cG* zqVnlG?y3}DWx$4iLKxMP4WrDaJ@+^yi6cvNN}guFjCNqC*`dSAr|>fyMrE%mgO0%Z z`ZUVU952!B3>7gAsQHP}QE-9;3>6CBrd*R=z?E9F!TrHlhbEIuL!>&JxMKZUebRfl z%(@7$#ij|kJz<_-$Eft}(6`^jVDC=xyNyo%6AtzlvIjfgR84M~)rlT*(yUGnvR_6! z&|r3`2*G;)89a@_)~@F?xap=(v*O5w0yKb5uAEO~sHey!h`?rtfkG&>N;TI!nhBiHp@L3}Xzc|rx(sw$vVgeFAo>h+LMA{>I!XdTs*{3e zw~3C&f?8+%@5IohaEQ#$L&8P)@(+$ELJxBIh8$R8W;_s(GW!Q;2_s|BqbcS)^a#Z^ zfB?CJiA1#v5GVqwEpyOh)sc`6A{00nKw3EziecghH4{PQ9%^tw78x3iz+nT`AOeCg z2qzBm&IrIolw`76I0T3&$3t+8hlm*x8O6}>BA$e`xMFt{mJJ7Uzbn20W{Ha1B{@n^ zr1GX^3tO=v6{n|#D!`*h2x?vrh&B_iGhZz5+lZR|w01nglNM&Ys$p|KT#kmq!*Pe^sulNXCeO+pA>OdUzhEDX z{YjOgx}s?hsx^^!GYjMZM;Q2f#Y?pN%Y4A%#oAlQ35*2-k%i6Dw=k9&vpm8GR=8F} zg_FfG$4j+`=mrGZY6~U$U~tmG#W-ZhYL5W3U~v(V;&}}Xh`9nUGWkGcXsFhuP_+!! zFj{p4Pb*#sObv64elj-s>y#TsqS{eYgb5lfCX$l=ll@10Hs(UTnp8}4uoGt*6qP)oW^2A~)&ax*!eZjs< z2U4tBmKgSQqANR~MYj&JjIvta;D~|jqmCHCeGC-pz>S1uxx#7~G?_tp_3Czg+Nfyf zdZPivVrTu*O2C>AGa$>g!#(jKinUPWmlcXsRj;M%oS?o{l-EZQb%u1?WDn)qwi`qb z`Drp!CTO3@hm-|MVlGn9MhzJLO8<50-v|iu6C>L#jX)a&X5dKEyDN$}P-WzS zssw357#Wa~YOAz3Jw#NJ`V5v`iq(-)6+ussc`z#=vsb@MqEFNVUWf6?{|rCk#fC}` zHdFx`fKU+1@odW~^=fnclnap$1Qrjc?FcEc;qTQw+1-Jck`=jTHULWo78*&5wUq5I zJ2)C5x8Y`lu!>tK8FjUg3J`Vyq(&iaBh-$;2)P#PW6tz^p(}9HB-&VUYt^miLZf&e z8+MEUbIidxfF_|ziWs7P5rW%($O&?ZKfwu73@A1{Ecxv}-~d@UfFa5fe;Po$5jSA9 zOhGxaXw1K$t2P<&Rp9($0shilW7v2zP>!VrBtn5aGEQhNmGOQAtb@wLbm?cerC;L_ z28P~6hEbW;0w|DH$U-49sDMlq%6MO4#&HHPe9@Etz|Q+@Up+c?7}i%M=lhP`9~5s` zvT{~0v_1v4k6fIi07NY5Fe^Uv7GHb*>hrJFyiqjsh3~ITuil#WZhL4o=QSIQystmT?ML;0b96te9qQ9{+%^-NdIu<{F4FkKO-Pcrl4esoo6~~ zy2sKaG#m752C5A@KMKfoM)^1FVMDj^7#b0}35`b7Kn=h;kB8au2hginvDx&hO0DXQ zE)anZ;ziz|>I?(|+EEA08aNiM8twQ9&1MUHfWhhwjE0#Ko_J0IY=9zgNeK@zQymJ@ z2Fn$?2U5j|!-lwbFnA(gx1N=^up|e;J7V>!L5e7TPgdn*uA;wyEN}p18N72{gv$u5 zYAE0t1EtJH6z~qVQ&{3M-midT<|(3h)+vmq0)piz!PQGFDiK*1A=k8*0u)66Bb2%a zBQVcWeCVE=2qLuM7)2CRz&YG#h;(G&bP`Y)hDGFl7YIh8SmXP2D<()AX}|!zRQn?U z{Oqyuj|EllddNM=+CfGr1G4FH!CRe~Unx#55eeR|tZi^NC#D=DjMgbOR#X%N3`<^A zz+SBzj4>-Rml7ulo`;w4t>h5|55y2_CStIWAg`J1R>iCQitqbm<5*}6?{3V#i#Er7 zE2#2#9yzHusND@IB439@WJi`Z$S^cw6mB`){W=n0o{KnT9NMQ|rb_k3h`86Mlk|5e z%2w_dkOAB)=JJaZtv8NdKbq`J)t^ciccy(^Q^$y;-Sh3P1^0$I_l5;`&24uLk+ho= ztCNS|KR)lM2W6S?0YaVo{u)?{{^;5N>-@aC_3f5B?)DVlZY+A0w&G~HqC-z+d+}rd z=*QUrXtyZ<07NM%Zs<&!8#)_pK@-?Pkdn2f3F#CpDIi_T>4HlxtQzQvn$NCdtl%~V zJj`lcpm~NG$HOe9 zW}{1{N$h})F7$Z_VPnnWbzD7`T;Bt;&)s81PqW-9;i2J)S`S%I{!BS%S?^ZH1wW*m zBNG<^+9+sG$Z7h0leC`FOPSCOxv z_98WIs9%QnPVt7a+MhFE~=#M-fuLrRTJVU}+GKC{V z*^1^reUeP$0v9Q$sE#ibZ-~x}L%Rh50(&f8082MDa?1hexL|1;_d$q?01Shn1}PGW z9q^OpKLRYmyc{xic%6<;#zXNEs79C-%mLPHR6}AGlFPzE4-)DFaaThFg=RrOk^|CT zQ_uf~dS0o#ra^||2K*)_$qy!{kR`Jvc{{^0 zAg@Od^>q~SFUJs8CL535k_MF#>~TQF*)c z54g)vKnzfCid!t%J=Lj-W)l4~T9PG>t~MK;cJ&17OQAlbhxfLuR5>!uKH&9+1Rvi-mwGz zht7gUN8zF)|E|M5efVYjqN8Hbv1-v#vgC7kZD?b$RstM=TB2&vRg`E;>`tMqtN6R( zjc2YubK}DG3o|Ef7Ntveriyo^9J}$n&o*`RfuAd{gXXgI&l}QZd*=9J$h&#>g0DG+ z{}p>vtDEoe%~MD3x=Rv6GZQ~K@^;ZL%HAn^Z%yjl`E<*L^#1;N_dtpt`1rvIlWGkt zn%L+ZxOMSw!haRM_3V4*_qM;cG3D!Ca6CQdc$ygoi>6OxB0+xIxTdX!`&o^$P$kbG6#uZy?+;#1bDRE1j~xw zK(Ir8^0>D0(0GAd&K!oIDZ-!THMzAa7=k>dmpjc<9Q0uz8pxWtTyX|Y^Kcx=dZ~s4 zwAz#iAWBqpBO<{*)6@KMF39R7m&@%Y(Sq3P4w28-<>>7oinLnPHh~9P3j^p=#Ux@9 zt*7#V;B!8^vK8v{|ANcX37pnZ3y2F^*zCyV;JS`GyH!JSdW(<&$MzY|f&8##j}F2f zs$WyD2Vb+2JMn9dkTHZ+O3!G%1-^-*M<^IiKsenZcCI!X)BxNHV$$Rx;?Ka4;Xff3Mfd{8y76EGDGtHI&}ER~5JG38p4O8HvrW;t z2s06IB~AtW3lmO(wF2`sIHKYy@QcLKK{ky@Adrb`{vPDz5*{BMoq+4f(UY(@At@3( z86m@Otg9Pvkp^<0uC7s$0X5cvX+lgQ#@mgChQSV@g#olJ7-}Ga4vW{VwA3%0qW5dT zS77)HM;2A8q-Zt`j0rCnnEYY@6|PTbo0zx_J&`A(6t;wo&saTclrd6-wA2T2ahx%V z@JSpBv-o82(d-aZtl$^Ug7lzxYYYdm2M;J>KXU(2Ue4YLF(T?3W1KTZYa`@~G1KCw zw<@?lFgO^BX2(S%DHi4klC&|?C&?N$LyVg z%XvFAZZ8y^Ht)slE8LW|iaUWd%jPsx{W~pTlhlG+3W&61%>Jzz(_n^YMnF_$jWDGk zRp)*R)^{Z;9n4hG-0oM#I_Tfe=CORxuNSaknXterAO99@NEO!qNob}Z$^VPJ-w*ws z86SSVpx6|*)E#=ZJ#K0A*8!4~x)_uN%$r!uUACD3Nd_|(NLn-Aej20xSOigZheKE3 z82}+3nU%1a4=)=N;H*TDKVxMpjYvBR^)oKnafiihr3g4rsYc@2?UxEsf+2u6@chS9 z-2Js&;i_rtqObVc7q5Qt6@B0-UUU~FV)O2b%h*|7G(+)xq0U2Qx)wzw#BnEYDN7(}nfdMy%oL{!Yo_gF zJPGx5(!SuXyX~%f>*5ba|Ly3l-Ea5a*>~c-a|{07d4De|y;p)}-7Doh1qQQjj4h&8Kxwz0D65gxn>qP>)BwjNV6>Q(pRNIgd__oTn>_C(Gs0V zkcX4eG8AKnjve#SRoozR zxZ{7>1KSjtsQgEuD@{@c8_2CK0BDDq4>)y<&;(Sf0Hwk)9G z(7>2_1-OA=hzed@1CWFP3Fr|5DRR=_7w3cylK*Ay{5$>5eb{z!RrlM*%!C-EeUSLM!wRdRme4aXljQmg7_kG!A1 zZJLMoa$@JV4nPv0*muXX_NCUvyn;l}xBBS9&TpSoexLrHd z+;n5_AMH){eCN;|&-#~IvHA*{UaEQH)9zZ7Q~Uy}}vG|ZBPU@%D*(g>-WpNm=$ z?l)_>D^J0Q*Y@}){Tx8Yb<(aHLipaSSSKrqz94K05sih<)rOq!wrlS`W$NLh)~pbB ztnDYB0%2>If#@33Zr!m#>jfDjYjtU39E3!ROi6MXCg-YEdmhOBMF6Lq&xy`nNN+?} zw!zJb?`Xg%XCK{Ky^oFxjqe$o8afaXLl{9Dz16=9w9hd6VcKU%;?C|?uMMgCs{6Rq>4l{^Qg06H|A)^faspe8iZl9zmfs=YH5+%(8=owNs zhg_s>#uha!-XVJn;5}|4t|2`DDv||mW@0nci@;w4kBEf^R3^lwcO4-{usa2{N?}CM zR!xRQc!T6fh_fr5!ncz;De0!YDcnP#0PDN;zAkLeyI416}CIB-fne5$je?k1Y1FHJ+|$|CQ= zlNwty1YaM2W&GxFay-5EVA^+R!Pkc9hHp19*Xq@kkgODxFprnobisy&g1S^e-K_a` zLEW_VZvC#C6F&(q)VJQQZ%tTlkc}bYg9&p!VWxE6xB0yijld++!r03&XRMlYSItUKNpG2o*57p!7c}2f(X*YUw`ZNX)2s zjQ<(sK8w6PNGS9ON)ie!Ho@T%$)cxZ$%6BTd8JEMoM8y%$zQTl&cS)hmYkGxac)uD~ePY>Z+Uc%H59H)fy zXhrdo+T|59MRX7}+A)kzmYrZLer7zRo0eetM=AwytxHsf1k=g{WTHnl_x_ZweIJS9 z_JZ)MfIJG&`90mDgiW{zGSIN*b5>`{xo)0c4==cc?X?rHo|^a6eRY}dx)Pa(Opz5I zIux|O_Ux z-RoX%#FFz(c{ysq*HNi^Pp>EEQ{GQ|D(4fvtZ~YFOs)B!yH;%NtB;e{;3Uq@X;+Cp zp+j>Iys%t%YdZnxkfkRt?9BtjEb8?lE$($9E$MYZr>b)R5WQPh)M0Hioq#5tsA%M1 z|3VC-4@%i5^p?vtPMMpiiV7u*^d(KoxRx$ zs?g=sklGLpjs~cjQOLq0uJ)Fbr`rs94kP+e36su1>`hW(zi>1hJ2r6<#CoeDwN^-b z0nt-w`wCTKfy&shXhM9&WNScsgvLb<%McQ=O;9P;76j3dJz-B7*iJ#YFr;@zkK5YLAqDV6522r}!kNvDYdY z@scQ>Nv^@Y(bL%eK-DCu`T-U)nOVM#V2f1YbdT84+JkRNOcLoD&<~gLIh6p)QEUwb z^$%3!Z2uIrj@japzC-s0L#RZIilY93JsNsXHU=1?fQz+HV3owen2D+kPTjDzwL{+! zR^7K1N*L%J@I}zK0E-l@C>dx|R~H!oYvzDzoG)19HnY$^#P{k`1s=up381c_c$w4~ zR7d(9ECOthimnK3?@xSdrUSvJ&_;<(_W+CqFO&H*TOgRV3WbCzlO>yEG!-T&cvn1q z1%d7Lk3~#0SyiwNYQZ$K@co@_XSKH=8a92bea1vW9c<&oXgD+oRu~>~w1+U$MDoU0(OcWL8%NRL&gpRN^I2_GD2`>oHWZ=5baEUbOfslCf6s`OPPF4gQ#Dz zzk~z&7#Km2j}( zA)zvW>7)t`SnY<=kii8u@(N5)zEwaY%UO7Gzi>Jfb7gIZl?@>gB909B>DwvGPu1Zm z=IOMJ&vP(NmCgbkDt_v9$Yorxtesf7PY=`mFfT=f-C%^QLBi`21?>7pVMpPdh((1R zjN#x2TNe66G<@_6knSM8gJQYC=bb(>^j1<#`deI;$p~24e?hZSQ8|HXOc;q0M*99W zG7vxbE!Mqv5wb1*Qariw^~P5kZ|+VurdRJv`}Q*dL!K8KMtXgaGUONEa9npJOJ@of z@;5-}UAp?l$n}xr7u%YqNY?))2uWbq({q=rtj^(vG4l6RMob0QPV=vo>bAE zn-|%w8+)$rNmcsOMU4wZJ5xp2WN4wVd9JYe*3P%9(}nG*wz&MpvFpc@=hDSB3&r)R z;(9c(Xxm)TwwqgS$?2ljX~apdNVYEcYH#~$7xT-K+rH;X*gwcGzw0ST98SoIP;y(c zC)qx;o=G~^&z_!bOBFWFd$uElNaE>lAAh;+K^a%(hs%G##*bEW4j;_q6PCo$#NJfp z$&~lhyrc71ZrIkq0d)PmqnaE*u_xd0#6)5^`Ao`1A;W%UcPp2%&5ZLY(*p}{wcT@Z zp1hAA9H8xzo=Ld{5{&F+W-r@rR^Kd1`F1Zj_S|;tVa6_dl_+>$t2t`oeqky&y2bK~ zLhsQU%P(rI$j4y-H8j+~7EavF@`{jA`Z1;li_#$dKgbPY7U(+_lVsgOd|4$jkCT+F z+!j!JB5B8;61PPXTatD(ki`pkUdfIAF)e{NC~FA@aal_|VqP9cUY}w< z&IQ?KmVF@Fl_Qt)y*55*b+zVzX0!_0Oja-KUZ`GgmY81e4SJ?4D4;^v_(;zRpQLA5 zZfzL77mgU?xfhdKwf6xqWV-{Yqqlr_pdf??%fyjyXp{40gb9Sg(!a zph(lzDkLm`=Y_~K%jGFLyxxajXW`_1jlQY2_+UqZir%4!63kJ?WF zw=yOa`)Fsm@|8=c(eU_)h{)`%z>$L^J#d55_Iea(*ARuR04dl;8~QfsM6N?)BN64Oxei9$e+!JNh4$gs#l4LR>&u>A2=IVij3{uuyFJSjap2PxiJG+gGUKev7ZExu~*>(h5ZRKZRJvt0)138rOwn)R+$7ejo zlzvji69@(=5)-?Yaj`-yGL!TaJ_e&2GxnYq@n~BQwyZhaa-!{s^bS6#w4IW_pd?O7 z1Cor(*u0u;`Ixb`cb(}x0&3M9h)VBLIc90yj;b@Jm_!glv5>Z6(qnW-Ih|_k zpX!?D2jD7x?ZDLoFCUsZ@*!duIie1bc2BU(C+@`i(h>)=}NpfZ-<9_UZONFfL#e{2 z-uJZNJ!|Ulo+UG%x0d-e-t%(#1s|?n`+EH!*QdSvUa3z!{iEG;b zZ|1u`&oMjyu3$q-dz;h8?eC8luvx22A%czA!0s46G;C%_T$iz{0446#y$47g9I7?7 z*EUPdR0O-D4^3cK1niD35m_l6$MaGXC0FPtv&VTrxy_WYiQ9`DGo%~ACa8*bZ4Es@ z#y45617D$^U33G!B+Bf^?n)=yS~{ht>1sD6Rg~m85qvSj#1C)$X@^wm>+uEy? zdxMf6Qt~58{)&>HQSu%obChH#`86efN69}>LV!{+79zziQ?xK1OX{Q>v`(a5xRK#W z@~3Ewk5WOlN()YeCND-H>gu47?9xP33FFHMV#RdOv0vF_{-AUoRpCD;{{Z8OO;Sy! zdwi*hf4G%1RsWjv{gQM4l5?{EdB5Zee#yCh$$9?^SN?0x{kL{l@YDVdq|Ht4o3*Ds zThsP!Q~U>r_8?w+=IS%oE?m8kJUP2Lowq&h*fC{=V$teLm~XhQyKZ=|ds9`rZaLHW zhZlKYqVUG*>#J|9y}mXj?70Pv^N~f~Wz?Rk@ZWSI#_b}XH_djJqEO(ewL5Nx)4r!3 z^SEdFq7fTs+s(ac?}0`5IIq7}ceU=C|EizttUA)JU2rXMn$fm$YTN4Mvsh^K?%u?+ zDbG5*FyTz@OZhg@Tec*>m@3$^$a`*kcT!zD5)~=enh$ZO7@-{t%u{&}%kxe3OPe{T zYs$Lla3TSt>v~`_nVvNzz5vDx=Z6k#Y){F?hhFSq{%{ZWeSKJrtu!C5#m0S0TWS9$ z8@Ikj+uf<6O55G3p!6f!-Ra?w2|F-6Y(fn1hm{Dk{LqUix(~ewuKLhlZW12uLY%aR zF2wJ8Sc7Of4{H!&;$gRm9F!jlix(MUDVPg6K5?* z)Gs1lCPvEz_M7)ik1X=ugo_5Cf=_ z&AJ8wh^{$aa$GO|PU-uO@_SZ}U;QyDpRv(~*}CQf0&DALs%J`4-c7TEKX~rV=k7T6 zs?Q`=FE}c1J1YOjqPvnfR;?~}op$qG^NG9x6ZgHXWgTwrKe_G4*IE8kVeN5~Hi1sWPUpU diff --git a/bin/sdlc b/bin/sdlc index c6ebdc4..6462e83 100755 --- a/bin/sdlc +++ b/bin/sdlc @@ -134,8 +134,12 @@ esac CACHE_VOLUME=${SDLC_CACHE_VOLUME:-crunchtools-hook-cache} need_volume() { - podman volume exists "$1" 2>/dev/null || podman volume create "$1" >/dev/null \ - || die "cannot create the podman volume $1" + podman volume exists "$1" 2>/dev/null && return 0 + podman volume create "$1" >/dev/null 2>&1 && return 0 + # Two lanes reach this at the same moment on a cold machine and one of them + # loses the create. Losing that race is not a failure; not having the volume + # afterwards is. + podman volume exists "$1" 2>/dev/null || die "cannot create the podman volume $1" } # ------------------------------------------------------------------ podman run @@ -249,8 +253,8 @@ t_gatehouse() { # # Which checks run is decided by the repo's [tool.sdlc.commit], [tool.sdlc.push] # and [tool.sdlc.ci] blocks, and this shell cannot read TOML. So it asks the -# image for a plan -- one line per step, `check TAB mode TAB payload` -- and -# carries it out. The modes: +# image for a plan -- one line per step, `check TAB lane TAB mode TAB payload` -- +# and carries it out. The lane is handled under "the lanes" below. The modes: # # image-ro | image-rw `sdlc ` in the image, the tree ro or rw # local a check this file answers itself @@ -310,21 +314,18 @@ run_step() { # mode payload -> the step's exit status esac } -run_plan() { # commit | push | ci | - _plan=$(run_in_image ro nostdin "$IMAGE" plan "$1") \ - || die "the image could not plan '$1'; nothing was checked" - [ -n "$_plan" ] || die "the image planned nothing for '$1'" - +run_serial() { # the plan -> its worst step status _worst=0 _failed= _ifs=$IFS set -f IFS=' ' - for _line in $_plan; do + for _line in $1; do IFS=$_ifs _check=${_line%%"$TAB"*} _rest=${_line#*"$TAB"} + _rest=${_rest#*"$TAB"} # the lane; one lane, so nothing to decide _mode=${_rest%%"$TAB"*} _payload=${_rest#*"$TAB"} # One failed step ends that check; the next check still runs. @@ -344,6 +345,183 @@ run_plan() { # commit | push | ci | return "$_worst" } +# ------------------------------------------------------------------- the lanes +# +# The plan's second field is a lane. Steps of one lane run in sequence, in plan +# order; lanes run at the same time. Which steps share a lane is the image's +# decision (see "the lanes" in `sdlc`), so no check name appears in this file -- +# a repo with one lane never reaches any of this and runs through run_serial +# exactly as it did before lanes existed. +# +# Two things change when there is more than one lane: +# +# - The CPU budget is DIVIDED between them. The budget is for the whole gate, +# not for each container: two lanes at half the budget take the same share +# of the machine that one lane at the whole budget took, so a laptop running +# `sdlc ci` is no worse off than before. +# - Every lane's output is buffered to a file per CHECK and printed whole, in +# plan order, as each check finishes. Two lanes writing to one terminal +# produce a log nobody can read. The cost is that the `== ` markers +# inside a block all arrive at once, so each block ends with how long its +# check took -- which is the number people were reading off those markers. + +OUT= # the lane scratch directory, while lanes are running + +now_s() { date +%s 2>/dev/null || printf 0; } + +_lanes_running() { # 0 while any lane has not yet written its done marker + _j=0 + while [ "$_j" -lt "$NLANES" ]; do + _j=$((_j + 1)) + if [ ! -f "$OUT/lane-done.$_j" ]; then return 0; fi + done + return 1 +} + +_close_check() { # the open check's block is complete + if [ -n "${_cur:-}" ]; then + printf '%s\n' "$_cur_rc" > "$OUT/rc.$_cur" + _cur= + fi +} + +run_lane() { # lane-index steps-file lane-cpus -> the lane's worst + _k=$1 + GATE_CPUS=$3 + _worst=0 + _failed= + _cur= + _cur_rc=0 + # However this subshell ends -- a step's `die` exits it where it stands -- the + # check it was on must get a verdict file and the lane must get its done + # marker, or the printer waits for a block that is never coming. + trap 'printf "2\n" > "$OUT/rc.${_cur:-0}"; printf "2\n" > "$OUT/lane-done.$_k"' EXIT + + while IFS= read -r _line; do + _check=${_line%%"$TAB"*} + _rest=${_line#*"$TAB"} + _rest=${_rest#*"$TAB"} # drop the lane: this whole file is one + _mode=${_rest%%"$TAB"*} + _payload=${_rest#*"$TAB"} + case " $_failed " in *" $_check "*) continue ;; esac + _i=$(awk -v c="$_check" '$0==c {print NR; exit}' "$OUT/order") || _i= + [ -n "$_i" ] || die "the image planned a step for a check its own plan does not list: $_check" + # A check's steps are contiguous within its lane, so a different index means + # the previous block is finished and can be printed. + if [ "$_i" != "${_cur:-}" ]; then _close_check; _cur=$_i; _cur_rc=0; fi + _t0=$(now_s) + _rc=0 + run_step "$_mode" "$_payload" >> "$OUT/out.$_i" 2>&1 || _rc=$? + _took=$(( $(now_s) - _t0 )) + if [ "$_rc" -ne 0 ]; then + _failed="$_failed $_check" + _cur_rc=$_rc + printf '\n%s: %s %s after %ss\n' "$ME" "$_check" \ + "$([ "$_rc" -eq 2 ] && echo 'COULD NOT RUN' || echo 'FAILED')" "$_took" \ + >> "$OUT/out.$_i" + # A later 1 must not mask an earlier 2, in this lane or any other. + if [ "$_rc" -eq 2 ] || [ "$_worst" -eq 2 ]; then _worst=2; else _worst=1; fi + else + printf '\n== %s passed in %ss\n' "$_check" "$_took" >> "$OUT/out.$_i" + fi + done < "$2" + + _close_check + trap - EXIT + printf '%s\n' "$_worst" > "$OUT/lane-done.$_k" + return "$_worst" +} + +run_lanes() { # the plan -> the worst status of every lane + OUT=$(mktemp -d) || die "cannot create a scratch directory for the lane logs" + trap 'rm -rf "$OUT"' EXIT HUP INT TERM + + # The checks in plan order, first appearance: the order blocks are printed in. + printf '%s\n' "$1" | cut -f1 | awk '!seen[$0]++' > "$OUT/order" + printf '%s\n' "$1" | cut -f2 | awk '!seen[$0]++' > "$OUT/lanes" + NCHECKS=$(awk 'END {print NR}' "$OUT/order") + NLANES=$(awk 'END {print NR}' "$OUT/lanes") + + # Floor of one, so a budget smaller than the lane count does overshoot it by + # a CPU or two. The alternative is running the lanes one after another on a + # single-CPU budget, which is the thing this exists to stop doing. + _lane_cpus=$(( GATE_CPUS / NLANES )) + [ "$_lane_cpus" -lt 1 ] && _lane_cpus=1 + + # Once, here, rather than from whichever lane gets there first. + need_volume "$CACHE_VOLUME" + + printf '%s: %s lanes at the same time, %s CPU(s) each\n' "$ME" "$NLANES" "$_lane_cpus" + _k=0 + _pids= + while IFS= read -r _lane; do + _k=$((_k + 1)) + printf '%s\n' "$1" | awk -F"$TAB" -v l="$_lane" '$2 == l' > "$OUT/steps.$_k" + printf '%s: lane %s: %s\n' "$ME" "$_lane" \ + "$(cut -f1 "$OUT/steps.$_k" | awk '!seen[$0]++' | tr '\n' ' ')" + run_lane "$_k" "$OUT/steps.$_k" "$_lane_cpus" & + _pids="$_pids $!" + done < "$OUT/lanes" + + # Print each block whole, in plan order, as soon as it is finished. A lane + # that finishes early is HELD BACK, never dropped and never cut into another + # lane's output; the lanes themselves keep running while this waits. + _i=1 + while [ "$_i" -le "$NCHECKS" ]; do + if [ -f "$OUT/rc.$_i" ]; then + if [ -f "$OUT/out.$_i" ]; then cat "$OUT/out.$_i"; fi + _i=$((_i + 1)) + continue + fi + _lanes_running || break + sleep 1 + done + # Whatever the loop above left: a lane that `die`d stopped where it stood, so + # the checks behind it in that lane have no block at all. Say so rather than + # print nothing, which reads exactly like a check that passed. + while [ "$_i" -le "$NCHECKS" ]; do + if [ -f "$OUT/out.$_i" ]; then + cat "$OUT/out.$_i" + else + printf '\n%s: %s NEVER RAN: its lane stopped before reaching it\n' "$ME" \ + "$(awk -v n="$_i" 'NR == n {print; exit}' "$OUT/order")" >&2 + fi + _i=$((_i + 1)) + done + + # Collected after every lane has been reaped, so which lane finished first + # cannot decide the answer: a 2 anywhere is a 2. + _worst=0 + for _p in $_pids; do + _rc=0 + wait "$_p" || _rc=$? + case "$_rc" in + 0) ;; + 2) _worst=2 ;; + *) [ "$_worst" -eq 2 ] || _worst=1 ;; + esac + done + + rm -rf "$OUT" + trap - EXIT HUP INT TERM + return "$_worst" +} + +run_plan() { # commit | push | ci | + _plan=$(run_in_image ro nostdin "$IMAGE" plan "$1") \ + || die "the image could not plan '$1'; nothing was checked" + [ -n "$_plan" ] || die "the image planned nothing for '$1'" + + _n=$(printf '%s\n' "$_plan" | awk -F"$TAB" '!seen[$2]++ {n++} END {print n + 0}') + _rc=0 + if [ "${_n:-1}" -le 1 ]; then + run_serial "$_plan" || _rc=$? + else + run_lanes "$_plan" || _rc=$? + fi + return "$_rc" +} + # ---------------------------------------------------------------------- dispatch case "${1:---help}" in diff --git a/pyproject.toml b/pyproject.toml index 582a207..6613584 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -10,7 +10,7 @@ [project] name = "python-sdlc" -version = "0.2.2" +version = "0.3.0" description = "One container image and one script for every CrunchTools Python SDLC check" requires-python = ">=3.14" license = "AGPL-3.0-or-later" diff --git a/sdlc b/sdlc index 5f07af4..e770c68 100755 --- a/sdlc +++ b/sdlc @@ -280,12 +280,13 @@ def kind_of(cfg: dict, name: str) -> str | None: # -------------------------------------------------------------------- the plan # # The host half (bin/sdlc) is POSIX shell and cannot read TOML, so it asks this -# script what to do and then does it. One line per step, three fields split by a +# script what to do and then does it. One line per step, four fields split by a # tab: # -# TAB TAB +# TAB TAB TAB # -# where mode is: +# where lane is the group the step runs in -- see "the lanes" below -- and mode +# is: # image-ro / image-rw run `sdlc ` in this image, tree ro or rw # local bin/sdlc's own check named # host is a podman argv, its words joined by @@ -321,27 +322,93 @@ def plan_lines(cfg: dict, check: str) -> list[tuple[str, str, str]]: return [(check, "image-rw" if check in WRITES_TREE else "image-ro", check)] +# ------------------------------------------------------------------- the lanes +# +# `sdlc ci` used to run every check one after another, so trentina's image build +# -- which depends on none of mypy, pytest, semgrep or pip-audit -- waited for +# all four. The job took the sum of its checks instead of the length of its +# longest chain: 5m09s on a self-hosted runner, 16m05s on a GitHub-hosted one. +# +# So each plan line carries a lane. Steps of one lane run in sequence, in plan +# order; lanes run at the same time. The decision is made HERE, in the image, +# because this is the half that knows what each check does -- bin/sdlc only +# carries it out, and no check name appears in that file. +# +# Two rules, and neither names a repo: +# +# - An `image-build` check and every `pytest-in-image` check that runs the +# image it tags are ONE lane. The build has to finish before the tests +# start, and two builds writing one tag must not race, so a shared tag is a +# shared lane. +# - Everything else goes in MAIN_LANE, in plan order. That is where `uv sync` +# happens: mypy, pytest and pip-audit each sync the project venv in the +# shared cache volume, and one lane makes those sequential. uv does lock the +# target environment (uv 0.13, "Cache safety"), but a gate should not bet +# its venv on a lock it did not take. +# +# A repo with no image-build check therefore gets one lane, and one lane is the +# launcher's sequential path -- the same run, the same output, as before lanes +# existed. `sdlc push` has no image chain either, so push is unchanged too. + +MAIN_LANE = "main" + + +def lane_of(cfg: dict, checks: list[str]) -> dict[str, str]: + """check -> lane, for one command's resolved check list. + + An image chain is keyed by the image reference, not by position: a + `pytest-in-image` may be listed before the `image-build` that makes what it + runs, and two builds may legitimately produce two different tags. + """ + lanes: dict[str, str] = {} + by_tag: dict[str, str] = {} + for check in checks: + if kind_of(cfg, check) != "image-build": + continue + tag = str(check_settings(cfg, check).get("tag", default_image_tag())) + lanes[check] = by_tag.setdefault(tag, check) + for check in checks: + if kind_of(cfg, check) != "pytest-in-image": + continue + image = str(check_settings(cfg, check).get("image", default_image_tag())) + # No build of that image in this command: the repo is testing something + # already built, so it owes nothing to another lane and gets its own. + lanes[check] = by_tag.get(image, check) + return {check: lanes.get(check, MAIN_LANE) for check in checks} + + def t_plan(target: str) -> int: cfg = sdlc_config() checks = resolve(cfg, target) if target in COMMANDS else [target] for check in checks: if check not in BUILTINS and kind_of(cfg, check) is None: cannot(f"unknown check: {check}") + lanes = lane_of(cfg, checks) + for check in checks: + lane = plain(lanes[check], "a lane name") for name, mode, payload in plan_lines(cfg, check): plain(name, "a check name") if mode != "host": plain(payload, name) - print(f"{name}\t{mode}\t{payload}") + print(f"{name}\t{lane}\t{mode}\t{payload}") return CLEAN def t_tiers() -> int: cfg = sdlc_config() for command in COMMANDS: + checks = resolve(cfg, command) + lanes = lane_of(cfg, checks) + parallel = len(set(lanes.values())) > 1 print(f"sdlc {command}:") - for check in resolve(cfg, command): + for check in checks: kind = kind_of(cfg, check) - print(f" {check}" + (f" ({kind})" if kind else "")) + note = f" ({kind})" if kind else "" + if parallel: + note += f" [lane {lanes[check]}]" + print(f" {check}{note}") + if parallel: + print(f" {len(set(lanes.values()))} lanes, run at the same time") if skipped(): print(f"\nSDLC_SKIP is skipping: {', '.join(sorted(skipped()))}") return CLEAN diff --git a/tests/contract.sh b/tests/contract.sh index 8806ccf..e487e95 100755 --- a/tests/contract.sh +++ b/tests/contract.sh @@ -255,6 +255,59 @@ case "$plan" in *) ok "sdlc plan ci leaves out the two reviewers" ;; esac +# A repo with no image chain gets ONE lane, which is the launcher's sequential +# path: the same run it had before lanes existed. +lanes=$(podman run --rm -v "$FIXTURE:/repo:ro,z" -w /repo --entrypoint="" \ + "$IMAGE" sdlc plan ci 2>&1 | cut -f2 | sort -u | tr '\n' ' ') +if [ "$lanes" = "main " ]; then + ok "a repo with no image build plans one lane" +else + bad "a repo with no image build planned lanes '$lanes', want 'main '" +fi + +# And a repo with one gets two: the build and the tests that run what it built +# in their own lane, every uv sync in the other. This is the seam between the +# TOML the image reads and the parallel executor in the shell, so it is checked +# on the real image rather than only in the unit test. +CHAIN=$(mktemp -d) +mkdir -p "$CHAIN/tests" +printf 'FROM scratch\n' > "$CHAIN/Containerfile" +cat > "$CHAIN/pyproject.toml" <<'TOML' +[project] +name = "chain" +version = "0.0.0" + +[tool.sdlc.ci] +checks = ["image", "image-tests"] + +[tool.sdlc.ci.image] +kind = "image-build" + +[tool.sdlc.ci.image-tests] +kind = "pytest-in-image" +tests = ["tests/"] +TOML + +chain_plan=$(podman run --rm -v "$CHAIN:/repo:ro,z" -w /repo --entrypoint="" \ + -e SDLC_HOST_REPO="$CHAIN" -e SDLC_CACHE_VOLUME=contract-test \ + "$IMAGE" sdlc plan ci 2>&1) +rm -rf "$CHAIN" +nlanes=$(printf '%s\n' "$chain_plan" | cut -f2 | sort -u | grep -c .) +off_main=$(printf '%s\n' "$chain_plan" | awk -F'\t' '$2 != "main" {print $1}' | sort -u | tr '\n' ' ') +syncers=$(printf '%s\n' "$chain_plan" | awk -F'\t' ' + $1 == "mypy" || $1 == "pytest" || $1 == "pip-audit" {print $2}' | sort -u | grep -c .) +if [ "$nlanes" = 2 ] && [ "$off_main" = "image image-tests " ]; then + ok "an image chain plans a second lane, and only the chain is in it" +else + bad "an image chain planned $nlanes lane(s), off-main: '$off_main'" + printf '%s\n' "$chain_plan" | sed 's/^/ /' +fi +if [ "$syncers" = 1 ]; then + ok "mypy, pytest and pip-audit share a lane, so two uv syncs cannot race" +else + bad "the checks that run uv sync were spread across $syncers lanes" +fi + # And it must still FAIL on something broken -- a gate that passes everything is # the other way to be useless. cat > "$FIXTURE/src/broken.py" <<'PY' diff --git a/tests/test_host_contract.sh b/tests/test_host_contract.sh index 2707aa4..bb7c8b8 100755 --- a/tests/test_host_contract.sh +++ b/tests/test_host_contract.sh @@ -46,30 +46,71 @@ done # POSIX utilities. The rule is about language toolchains and linters, not about # rebuilding coreutils -- every machine with a shell has these. -for prog in sh awk sed tr grep basename dirname nproc cat mktemp rm ln head id; do +POSIX_UTILS="sh awk sed tr grep basename dirname nproc cat mktemp rm ln head id + cut sleep date wc" +for prog in $POSIX_UTILS; do if full=$(command -v "$prog" 2>/dev/null); then ln -sf "$full" "$SHIM/$prog"; fi done # A fake podman that records what it was asked to do and succeeds. Every target # that needs a container must reach THIS and nothing else. Asked for a plan, it # answers with one step of each mode, so the launcher's whole executor runs. +# +# A plan line is `check TAB lane TAB mode TAB payload`. The plans whose names +# begin `lanes-` exercise the parallel executor: their steps print markers, +# sleep and exit on command, so a test can pin what the launcher does with a +# lane that fails, a lane that is slow, and two lanes writing one terminal. TAB=$(printf '\t') US=$(printf '\037') cat > "$SHIM/podman" <> "$LOG" +for _last in "\$@"; do :; done case " \$* " in *" plan gatehouse-direct "*) - printf 'gatehouse${TAB}local${TAB}gatehouse\n' ;; + printf 'gatehouse${TAB}main${TAB}local${TAB}gatehouse\n' ;; *" plan evil "*) - printf 'evil${TAB}host${TAB}sh${US}-c${US}touch $SHIM/pwned\n' ;; + printf 'evil${TAB}main${TAB}host${TAB}sh${US}-c${US}touch $SHIM/pwned\n' ;; *" plan subst "*) - printf 'subst${TAB}host${TAB}podman${US}run${US}\\\$(touch $SHIM/pwned)\n' ;; + printf 'subst${TAB}main${TAB}host${TAB}podman${US}run${US}\\\$(touch $SHIM/pwned)\n' ;; + *" plan lanes-interleave "*) + printf 'a1${TAB}one${TAB}image-ro${TAB}tick-a1\n' + printf 'a2${TAB}one${TAB}image-ro${TAB}tick-a2\n' + printf 'b1${TAB}two${TAB}image-ro${TAB}tick-b1\n' + printf 'b2${TAB}two${TAB}image-ro${TAB}tick-b2\n' ;; + *" plan lanes-two-first "*) + printf 'x${TAB}one${TAB}image-ro${TAB}rc-two\n' + printf 'y${TAB}two${TAB}image-ro${TAB}slow-rc-one\n' ;; + *" plan lanes-one-first "*) + printf 'x${TAB}one${TAB}image-ro${TAB}rc-one\n' + printf 'y${TAB}two${TAB}image-ro${TAB}slow-rc-two\n' ;; + *" plan lanes-keeps-going "*) + printf 'x${TAB}one${TAB}image-ro${TAB}rc-one\n' + printf 'x${TAB}one${TAB}image-ro${TAB}must-not-run\n' + printf 'w${TAB}one${TAB}image-ro${TAB}after-the-failure\n' + printf 'y${TAB}two${TAB}image-ro${TAB}other-lane-first\n' + printf 'z${TAB}two${TAB}image-ro${TAB}other-lane-second\n' ;; + *" plan lanes-budget "*) + printf 'p${TAB}one${TAB}host${TAB}podman${US}build${US}-t${US}localhost/x:sdlc${US}.\n' + printf 'q${TAB}two${TAB}image-ro${TAB}quiet-ok\n' ;; *" plan "*) - printf 'mypy${TAB}image-ro${TAB}mypy\n' - printf 'ruff-check${TAB}image-rw${TAB}ruff-check\n' - printf 'image${TAB}host${TAB}podman${US}build${US}-t${US}localhost/x:sdlc${US}.\n' - printf 'image-tests${TAB}host${TAB}podman${US}run${US}--rm${US}localhost/x:sdlc${US}-m${US}pytest\n' ;; + printf 'mypy${TAB}main${TAB}image-ro${TAB}mypy\n' + printf 'ruff-check${TAB}main${TAB}image-rw${TAB}ruff-check\n' + printf 'image${TAB}main${TAB}host${TAB}podman${US}build${US}-t${US}localhost/x:sdlc${US}.\n' + printf 'image-tests${TAB}main${TAB}host${TAB}podman${US}run${US}--rm${US}localhost/x:sdlc${US}-m${US}pytest\n' ;; +esac +case "\$_last" in + tick-*) + printf 'START %s\n' "\$_last" >> "$SHIM/race.log" + printf 'BEGIN %s\n' "\$_last"; sleep 1; printf 'END %s\n' "\$_last" + printf 'DONE %s\n' "\$_last" >> "$SHIM/race.log" ;; + rc-one) printf 'RAN rc-one\n'; exit 1 ;; + rc-two) printf 'RAN rc-two\n'; exit 2 ;; + slow-rc-one) printf 'RAN slow-rc-one\n'; sleep 2; exit 1 ;; + slow-rc-two) printf 'RAN slow-rc-two\n'; sleep 2; exit 2 ;; + must-not-run) printf 'RAN must-not-run\n'; touch "$SHIM/must-not-run" ;; + after-the-failure|other-lane-first|other-lane-second) + printf 'RAN %s\n' "\$_last" ;; esac exit 0 EOF @@ -113,6 +154,117 @@ else bad "a host step's word was evaluated as shell" fi +printf '\n== the lanes\n\n' + +# Steps of one lane run in sequence; lanes run at the same time. What has to +# hold is not the speed -- it is that the answer and the log do not depend on +# which lane happened to finish first. + +RUN=$(mktemp -d) +trap 'rm -rf "$SHIM" "$RUN"' EXIT 0 + +lanes() { # target -> exit status, output in $RUN/out + rm -f "$SHIM/must-not-run" + _rc=0 + PATH="$SHIM" SDLC_CPUS=4 sh bin/sdlc "$1" > "$RUN/out" 2>&1 || _rc=$? + return "$_rc" +} + +# A single-lane plan takes the sequential path: no lane banner, and the log is +# the one a repo without an image chain has always had. +lanes ci && rc=0 || rc=$? +if ! grep -q 'lanes at the same time' "$RUN/out"; then + ok "a single-lane plan does not go through the lane executor" +else + bad "a single-lane plan was split into lanes" +fi + +# Two lanes, four checks, each step printing BEGIN and END a second apart. Run +# unbuffered the two lanes would alternate; what must come out is each check's +# block whole, in PLAN order -- a1 a2 b1 b2 -- however the lanes ran. +rm -f "$SHIM/race.log" +lanes lanes-interleave && rc=0 || rc=$? + +# First: the lanes really did overlap, so the ordering assertion below is +# testing the buffering and not a sequential run that could not interleave if +# it tried. Each step records when it started and finished in one shared file; +# both lanes' first steps must have started before either had finished. +if [ "$(head -2 "$SHIM/race.log" | grep -c '^START ')" -eq 2 ]; then + ok "both lanes were running at the same time" +else + bad "the lanes ran one after another: $(tr '\n' ' ' < "$SHIM/race.log")" +fi + +got=$(grep -E '^(BEGIN|END) ' "$RUN/out" | tr '\n' ' ') +want="BEGIN tick-a1 END tick-a1 BEGIN tick-a2 END tick-a2 BEGIN tick-b1 END tick-b1 BEGIN tick-b2 END tick-b2 " +if [ "$got" = "$want" ]; then + ok "each check's block is printed whole, in plan order, never interleaved" +else + bad "lane output interleaved or reordered: $got" +fi +if [ "$rc" -eq 0 ]; then + ok "two passing lanes exit 0" +else + bad "two passing lanes exited $rc" +fi +# And the blocks must carry a duration, because buffering collapses the +# timestamps people were reading the `== ` markers for. +if grep -q '^== a1 passed in [0-9]*s' "$RUN/out"; then + ok "a block ends with how long its check took" +else + bad "a block does not report its check's duration" +fi + +# A 2 anywhere wins over a 1, in either finishing order. Fast-2/slow-1 and +# fast-1/slow-2 are two different code paths through the aggregation, and the +# second is the one a naive "last writer wins" gets wrong. +lanes lanes-two-first && rc=0 || rc=$? +if [ "$rc" -eq 2 ]; then + ok "a 2 that finishes first beats a 1 that finishes last" +else + bad "fast 2 + slow 1 exited $rc, should be 2" +fi +lanes lanes-one-first && rc=0 || rc=$? +if [ "$rc" -eq 2 ]; then + ok "a 2 that finishes last beats a 1 that finished first" +else + bad "fast 1 + slow 2 exited $rc, should be 2" +fi + +# A failed step ends its OWN check. The next check in that lane still runs, and +# so does everything in every other lane -- that rule is older than the lanes +# and the lanes must not quietly narrow it. +lanes lanes-keeps-going && rc=0 || rc=$? +if [ "$rc" -eq 1 ]; then + ok "a lane that finds something exits 1 when nothing could not run" +else + bad "a found-something run exited $rc, should be 1" +fi +if [ -e "$SHIM/must-not-run" ]; then + bad "a failed check's remaining steps still ran" +else + ok "a failed step ends its own check" +fi +for marker in after-the-failure other-lane-first other-lane-second; do + if grep -q "RAN $marker" "$RUN/out"; then + ok "$marker ran despite the failure in lane one" + else + bad "$marker was skipped because another check failed" + fi +done + +# The CPU budget is for the whole gate, not for each container: two lanes out +# of a budget of 4 get 2 each, so a laptop running `sdlc ci` is no worse off +# than it was when the checks were sequential. +: > "$LOG" +lanes lanes-budget || true +if grep -q -- '--cpus 2 ' "$LOG" && grep -q -- '--cpu-quota 200000' "$LOG"; then + ok "a budget of 4 is split 2 and 2 between two lanes" +else + bad "the lanes did not divide the CPU budget" + grep -o -- '--cpus [0-9]*\|--cpu-quota [0-9]*' "$LOG" | sort -u | sed 's/^/ /' +fi + # A container that succeeds must report success, whatever the caller's last # exit status was. Gatehouse runs right after `git diff --quiet`, which exits 1 # whenever there IS a diff to review; that 1 leaked through a shared variable @@ -122,7 +274,7 @@ GITSHIM=$(mktemp -d) for prog in pre-commit gh; do [ -e "$SHIM/$prog" ] && ln -s "$SHIM/$prog" "$GITSHIM/$prog" done -for prog in sh awk sed tr grep basename dirname nproc cat mktemp rm ln head id podman; do +for prog in $POSIX_UTILS podman; do [ -e "$SHIM/$prog" ] && ln -s "$(readlink -f "$SHIM/$prog")" "$GITSHIM/$prog" done cat > "$GITSHIM/git" < dict: S.host_step("x", ["podman", "run", "a b"]) == ("x", "host", "podman\x1frun\x1fa b"), ) +# ------------------------------------------------------------------- the lanes +# +# Steps of one lane run in sequence; lanes run at the same time. Getting this +# wrong is not a slow job, it is a check that ran against an image that had not +# been built yet, or two `uv sync`es in one venv. So every rule is pinned. + +no_image = S.lane_of(empty, S.resolve(empty, "ci")) +check( + "a repo with no image-build gets one lane", + set(no_image.values()) == {S.MAIN_LANE}, + str(no_image), +) + +chain = { + "ci": { + "checks": ["image", "image-tests"], + "image": {"kind": "image-build"}, + "image-tests": {"kind": "pytest-in-image", "tests": ["tests/"]}, + } +} +lanes = S.lane_of(chain, S.resolve(chain, "ci")) +check( + "an image chain is one lane, so the tests follow the build", + lanes["image"] == lanes["image-tests"], + str(lanes), +) +check("the image chain is not the main lane", lanes["image"] != S.MAIN_LANE, str(lanes)) +check( + "the other checks stay in the main lane", + all(lanes[c] == S.MAIN_LANE for c in S.resolve(chain, "ci") if not c.startswith("image")), + str(lanes), +) +check( + "every uv sync is in one lane, so two cannot race in one venv", + len({lanes[c] for c in ("mypy", "pytest", "pip-audit")}) == 1, + str(lanes), +) + +# A pytest-in-image listed BEFORE its build still joins that build's lane: the +# chain is keyed by the image reference, not by the order the repo wrote. +reordered = { + "ci": { + "checks": ["image-tests", "image"], + "image": {"kind": "image-build"}, + "image-tests": {"kind": "pytest-in-image", "tests": ["tests/"]}, + } +} +lanes = S.lane_of(reordered, S.resolve(reordered, "ci")) +check("a chain written backwards is still one lane", lanes["image"] == lanes["image-tests"]) + +# Two builds writing ONE tag share a lane: concurrent builds of the same tag +# would race to leave the last one standing. +same_tag = { + "ci": { + "checks": ["a", "b"], + "a": {"kind": "image-build"}, + "b": {"kind": "image-build"}, + } +} +lanes = S.lane_of(same_tag, S.resolve(same_tag, "ci")) +check("two builds of one tag share a lane", lanes["a"] == lanes["b"], str(lanes)) + +two_tags = { + "ci": { + "checks": ["a", "b"], + "a": {"kind": "image-build", "tag": "localhost/a:sdlc"}, + "b": {"kind": "image-build", "tag": "localhost/b:sdlc"}, + } +} +lanes = S.lane_of(two_tags, S.resolve(two_tags, "ci")) +check("two builds of two tags get two lanes", lanes["a"] != lanes["b"], str(lanes)) + +# Tests against an image nothing in this command builds owe no lane anything. +prebuilt = { + "ci": { + "checks": ["image-tests"], + "image-tests": { + "kind": "pytest-in-image", + "tests": ["tests/"], + "image": "quay.io/x/y:1", + }, + } +} +lanes = S.lane_of(prebuilt, S.resolve(prebuilt, "ci")) +check( + "tests on a prebuilt image get their own lane", + lanes["image-tests"] not in (S.MAIN_LANE, None) and lanes["image-tests"] == "image-tests", + str(lanes), +) + +# ------------------------------------------------- the plan the launcher reads +# +# Four tab-separated fields now, and the launcher splits on exactly that. A +# plan line that does not have them is a step the shell mis-reads. + + +def plan_of(target: str, pyproject: str, **env) -> list[list[str]]: + with tempfile.TemporaryDirectory() as root: + Path(root, "pyproject.toml").write_text(pyproject) + Path(root, "tests").mkdir() + Path(root, "Containerfile").write_text("FROM scratch\n") + S.REPO = Path(root) + saved = dict(os.environ) + os.environ.update( + {"SDLC_HOST_REPO": root, "SDLC_CACHE_VOLUME": "testvol", "SDLC_PROJECT": "fx", **env} + ) + buf = io.StringIO() + try: + with contextlib.redirect_stdout(buf): + S.t_plan(target) + finally: + os.environ.clear() + os.environ.update(saved) + return [line.split("\t") for line in buf.getvalue().splitlines()] + + +TRENTINA = """ +[project] +name = "fx" +version = "0.0.0" + +[tool.sdlc.ci] +checks = ["image", "image-tests"] +skip = ["codeql"] + +[tool.sdlc.ci.image] +kind = "image-build" + +[tool.sdlc.ci.image-tests] +kind = "pytest-in-image" +tests = ["tests/"] +""" + +PLAN_FIELDS = 4 # check, lane, mode, payload -- what bin/sdlc splits on +CHAIN_LANES = 2 # the main lane, and the image chain's + +rows = plan_of("ci", TRENTINA) +check( + "every plan line has four fields", + all(len(r) == PLAN_FIELDS for r in rows), + str(rows), +) +check( + "the modes are still the third field", + {r[2] for r in rows} <= {"image-ro", "image-rw", "local", "host"}, + str({r[2] for r in rows}), +) +lanes_seen = [r[1] for r in rows] +check( + "the plan has exactly two lanes", + len(set(lanes_seen)) == CHAIN_LANES, + str(set(lanes_seen)), +) +check( + "the build and its tests are the only steps off the main lane", + {r[0] for r in rows if r[1] != S.MAIN_LANE} == {"image", "image-tests"}, + str(lanes_seen), +) +# The launcher groups a lane's steps by check and prints blocks in plan order, +# which both assume a check's steps are contiguous within its lane. +for lane in set(lanes_seen): + seq = [r[0] for r in rows if r[1] == lane] + runs = [c for i, c in enumerate(seq) if i == 0 or seq[i - 1] != c] + check(f"lane {lane}: each check's steps are contiguous", len(runs) == len(set(runs)), str(seq)) + +single = plan_of("push", '[project]\nname = "fx"\nversion = "0.0.0"\n') +check( + "push is one lane, so it runs exactly as it did before", + {r[1] for r in single} == {S.MAIN_LANE}, + str({r[1] for r in single}), +) + def main() -> int: print() From eb7742e74cbb58f5fcd05729843bb5d4e7031c27 Mon Sep 17 00:00:00 2001 From: Scott McCarty Date: Wed, 4 Nov 2026 09:09:46 -0500 Subject: [PATCH 2/5] fix: nothing runs beside a check that rewrites the tree, and the step index is resolved once Two findings from the first round on this branch. A check in WRITES_TREE -- ruff-check, ruff-format, gourmand -- now takes the whole command sequential. A build context or an in-image mount reading those files while ruff rewrites them is a check judging a tree that no longer exists. `sdlc commit` is ruff, so commit is sequential by definition, which is what "seconds, offline, free" wanted anyway. Reachable today only by a config nobody has written, which is the moment to close it. And each lane step was starting an awk to scan the whole check-order file for its own index -- O(N*N) and a process per step, next to steps that each start a container. The index is now resolved by the single awk that already splits the plan by lane and written as the step's first field, which is less shell, not more. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R7Y1jesjfgxHeqtKJCvay6 --- CHANGELOG.md | 6 ++++++ README.md | 5 +++++ bin/sdlc | 18 ++++++++++++------ sdlc | 9 +++++++++ tests/test_plan.py | 19 +++++++++++++++++++ 5 files changed, 51 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4bba27e..fef0f19 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -31,6 +31,12 @@ All notable changes to this project are documented here, following target environment, but a gate should not bet its venv on a lock it did not take. + And one refusal: a command that rewrites the tree gets no lanes at all. ruff + rewrites your source and gourmand writes its cache, and a build context or an + in-image mount reading those files while they change is a check judging a + tree that no longer exists. `sdlc commit` is ruff, so commit stays + sequential, which is what "seconds, offline, free" wanted anyway. + A repo with no image-build check gets one lane, and one lane is the launcher's sequential path: the same run, the same output, as before. `sdlc push` has no image chain, so push is unchanged too. diff --git a/README.md b/README.md index 82e5d63..fbd859d 100644 --- a/README.md +++ b/README.md @@ -85,6 +85,11 @@ The image decides the lanes, by two rules: - Everything else is the `main` lane, in the order `tiers` prints. That is where `uv sync` happens, and one lane keeps two syncs out of one venv. +A command that rewrites the tree — anything running `ruff-check`, +`ruff-format` or `gourmand` — gets no lanes at all, because a build context +reading those files while they change is judging a tree that no longer exists. +That makes `sdlc commit` sequential by definition. + A repo with no `image-build` check has one lane and runs exactly as it did before lanes existed. So does `sdlc push`, which never builds a container. diff --git a/bin/sdlc b/bin/sdlc index 6462e83..c9c6483 100755 --- a/bin/sdlc +++ b/bin/sdlc @@ -397,15 +397,17 @@ run_lane() { # lane-index steps-file lane-cpus -> the lane's worst # marker, or the printer waits for a block that is never coming. trap 'printf "2\n" > "$OUT/rc.${_cur:-0}"; printf "2\n" > "$OUT/lane-done.$_k"' EXIT + # `index TAB check TAB mode TAB payload`: the lane is gone, this file is one, + # and the index into the print order was resolved when the file was written + # rather than by an awk per step. while IFS= read -r _line; do - _check=${_line%%"$TAB"*} + _i=${_line%%"$TAB"*} _rest=${_line#*"$TAB"} - _rest=${_rest#*"$TAB"} # drop the lane: this whole file is one + _check=${_rest%%"$TAB"*} + _rest=${_rest#*"$TAB"} _mode=${_rest%%"$TAB"*} _payload=${_rest#*"$TAB"} case " $_failed " in *" $_check "*) continue ;; esac - _i=$(awk -v c="$_check" '$0==c {print NR; exit}' "$OUT/order") || _i= - [ -n "$_i" ] || die "the image planned a step for a check its own plan does not list: $_check" # A check's steps are contiguous within its lane, so a different index means # the previous block is finished and can be printed. if [ "$_i" != "${_cur:-}" ]; then _close_check; _cur=$_i; _cur_rc=0; fi @@ -456,9 +458,13 @@ run_lanes() { # the plan -> the worst status of every lane _pids= while IFS= read -r _lane; do _k=$((_k + 1)) - printf '%s\n' "$1" | awk -F"$TAB" -v l="$_lane" '$2 == l' > "$OUT/steps.$_k" + # Number each check by first appearance in the WHOLE plan -- the order the + # blocks print in -- and keep this lane's steps, index first. + printf '%s\n' "$1" | awk -F"$TAB" -v OFS="$TAB" -v l="$_lane" ' + !($1 in idx) { idx[$1] = ++n } + $2 == l { print idx[$1], $1, $3, $4 }' > "$OUT/steps.$_k" printf '%s: lane %s: %s\n' "$ME" "$_lane" \ - "$(cut -f1 "$OUT/steps.$_k" | awk '!seen[$0]++' | tr '\n' ' ')" + "$(cut -f2 "$OUT/steps.$_k" | awk '!seen[$0]++' | tr '\n' ' ')" run_lane "$_k" "$OUT/steps.$_k" "$_lane_cpus" & _pids="$_pids $!" done < "$OUT/lanes" diff --git a/sdlc b/sdlc index e770c68..9d37060 100755 --- a/sdlc +++ b/sdlc @@ -346,6 +346,12 @@ def plan_lines(cfg: dict, check: str) -> list[tuple[str, str, str]]: # target environment (uv 0.13, "Cache safety"), but a gate should not bet # its venv on a lock it did not take. # +# And one refusal: a command that REWRITES THE TREE gets no lanes at all. ruff +# rewrites your source and gourmand writes its cache, and a build context or an +# in-image mount reading those files while they change is a check judging a tree +# that no longer exists. `sdlc commit` is ruff, so commit is always sequential, +# which is what "seconds, offline, free" wanted anyway. +# # A repo with no image-build check therefore gets one lane, and one lane is the # launcher's sequential path -- the same run, the same output, as before lanes # existed. `sdlc push` has no image chain either, so push is unchanged too. @@ -360,6 +366,9 @@ def lane_of(cfg: dict, checks: list[str]) -> dict[str, str]: `pytest-in-image` may be listed before the `image-build` that makes what it runs, and two builds may legitimately produce two different tags. """ + # Nothing runs beside a check that rewrites the tree: one lane, in order. + if any(check in WRITES_TREE for check in checks): + return dict.fromkeys(checks, MAIN_LANE) lanes: dict[str, str] = {} by_tag: dict[str, str] = {} for check in checks: diff --git a/tests/test_plan.py b/tests/test_plan.py index 061f720..c8f2189 100644 --- a/tests/test_plan.py +++ b/tests/test_plan.py @@ -273,6 +273,25 @@ def sarif(results: list[tuple[str, str, str]]) -> dict: str(lanes), ) +# A check that rewrites the tree takes the whole command sequential. A build +# context or an in-image mount reading those files while ruff rewrites them is +# a check judging a tree that no longer exists. +rewrites = { + "ci": { + "checks": ["ruff-format", "image", "image-tests"], + "image": {"kind": "image-build"}, + "image-tests": {"kind": "pytest-in-image", "tests": ["tests/"]}, + } +} +lanes = S.lane_of(rewrites, S.resolve(rewrites, "ci")) +check( + "a check that rewrites the tree gets no lanes beside it", + set(lanes.values()) == {S.MAIN_LANE}, + str(lanes), +) +commit = S.lane_of(empty, S.resolve(empty, "commit")) +check("commit is always one lane", set(commit.values()) == {S.MAIN_LANE}, str(commit)) + # ------------------------------------------------- the plan the launcher reads # # Four tab-separated fields now, and the launcher splits on exactly that. A From c8c3aadb510ccf0d07306f0eaa8798666055cb64 Mon Sep 17 00:00:00 2001 From: Scott McCarty Date: Wed, 4 Nov 2026 09:12:31 -0500 Subject: [PATCH 3/5] fix: the printer cannot wait forever for a lane that cannot say it is done A lane wrote its done marker after clearing its EXIT trap, so a failed write in that window left no marker at all -- and `_lanes_running` would then return "still running" forever while the printer slept a second at a time. The trap now stays armed through that write: it writes the same marker, and the only reader asks whether the file exists. And the printer gives up ORDERING after an hour with no block finishing, rather than polling for one. `wait` below it still blocks for each lane's real status, so giving up on the order never guesses an answer. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R7Y1jesjfgxHeqtKJCvay6 --- bin/sdlc | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/bin/sdlc b/bin/sdlc index c9c6483..c376fc1 100755 --- a/bin/sdlc +++ b/bin/sdlc @@ -429,7 +429,9 @@ run_lane() { # lane-index steps-file lane-cpus -> the lane's worst done < "$2" _close_check - trap - EXIT + # The trap stays armed through this: it writes the same marker, and its only + # reader asks whether the file exists. Clearing it first would leave a window + # where a failed write means no marker at all, and the printer waiting on one. printf '%s\n' "$_worst" > "$OUT/lane-done.$_k" return "$_worst" } @@ -473,13 +475,20 @@ run_lanes() { # the plan -> the worst status of every lane # that finishes early is HELD BACK, never dropped and never cut into another # lane's output; the lanes themselves keep running while this waits. _i=1 + _stall=0 while [ "$_i" -le "$NCHECKS" ]; do if [ -f "$OUT/rc.$_i" ]; then if [ -f "$OUT/out.$_i" ]; then cat "$OUT/out.$_i"; fi _i=$((_i + 1)) + _stall=0 continue fi _lanes_running || break + # An hour with no block finishing is not a slow check, it is a lane that + # cannot say it is done. Give up ORDERING and print what there is; `wait` + # below still blocks for each lane's real status, so nothing is guessed. + [ "$_stall" -ge 3600 ] && break + _stall=$((_stall + 1)) sleep 1 done # Whatever the loop above left: a lane that `die`d stopped where it stood, so From f3931e0365df40753a2eef21608b935ec07a2d56 Mon Sep 17 00:00:00 2001 From: Scott McCarty Date: Wed, 4 Nov 2026 09:34:14 -0500 Subject: [PATCH 4/5] fix: cover the printer's stall fallback, and reap before printing what is left Gatehouse: the stall threshold had no test. It could not have one at 3600 seconds, so it is now SDLC_LANE_STALL, and the host contract test reaches the boundary with a one-second threshold against a lane that takes six. Answering it found the better shape. The remaining blocks are now printed AFTER every lane has been reaped, not before: the status is the answer and is never guessed from what a file happened to contain, and once every lane has stopped nothing is still writing, so a late block is printed whole instead of being read out from under the process writing it. The test pins that -- the late block's last line is written five seconds after the printer gave up, so it can only be there if the block was printed after the reap -- and fails if the two are swapped back. The launcher also says when it stops waiting on the order, rather than silently printing out of sequence. Only a lane killed outright reaches any of this: SIGKILL, the OOM killer, a cancelled CI job. Every other ending runs the lane's EXIT trap. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R7Y1jesjfgxHeqtKJCvay6 --- CHANGELOG.md | 9 +++++++ README.md | 6 +++++ bin/sdlc | 52 ++++++++++++++++++++++++------------- tests/test_host_contract.sh | 44 +++++++++++++++++++++++++++++-- 4 files changed, 91 insertions(+), 20 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fef0f19..99600cb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -54,6 +54,15 @@ All notable changes to this project are documented here, following - `sdlc tiers` prints each check's lane when a command has more than one. +### Added + +- `SDLC_LANE_STALL` (default 3600 seconds): how long the printer waits for a + lane that has stopped answering. Only a lane killed outright -- SIGKILL, the + OOM killer, a cancelled CI job -- gets there, because every other ending runs + the lane's cleanup. At the threshold the launcher says so, waits for every + lane to stop and prints what is left; each lane's real exit status is + collected either way. + ### Fixed - Two lanes starting on a machine with no cache volume could both try to create diff --git a/README.md b/README.md index fbd859d..29524b4 100644 --- a/README.md +++ b/README.md @@ -102,6 +102,12 @@ the checks, so two lanes never cut into each other's log. Each block ends with how long its check took, because buffering is what the `== ` timestamps used to tell you. +A lane killed outright — SIGKILL, the OOM killer, a cancelled CI job — runs no +cleanup and never reports, so the printer stops waiting on the order after +`SDLC_LANE_STALL` seconds (default 3600), says so, waits for every lane to +stop and then prints what is left. Each lane's real exit status is still +collected either way. + ## Configuring it One rule for all three blocks: diff --git a/bin/sdlc b/bin/sdlc index c376fc1..fe92f66 100755 --- a/bin/sdlc +++ b/bin/sdlc @@ -367,6 +367,14 @@ run_serial() { # the plan -> its worst step status OUT= # the lane scratch directory, while lanes are running +# How long the printer waits for a lane that has stopped answering before it +# stops waiting on the order. Only a lane killed outright -- SIGKILL, the OOM +# killer, a cancelled CI job -- gets here, because every other ending runs the +# lane's EXIT trap. Settable so the test can reach the boundary without +# waiting an hour for it. +STALL_MAX=${SDLC_LANE_STALL:-3600} +case "$STALL_MAX" in ''|*[!0-9]*) die "SDLC_LANE_STALL must be a whole number" ;; esac + now_s() { date +%s 2>/dev/null || printf 0; } _lanes_running() { # 0 while any lane has not yet written its done marker @@ -484,28 +492,23 @@ run_lanes() { # the plan -> the worst status of every lane continue fi _lanes_running || break - # An hour with no block finishing is not a slow check, it is a lane that - # cannot say it is done. Give up ORDERING and print what there is; `wait` - # below still blocks for each lane's real status, so nothing is guessed. - [ "$_stall" -ge 3600 ] && break + # A lane SIGKILLed outright runs no EXIT trap, so it never says it is done + # and this would poll for it forever. After STALL_MAX seconds with no block + # finishing, stop waiting on the order and fall through to the reap. + if [ "$_stall" -ge "$STALL_MAX" ]; then + printf '%s: a lane has not reported for %ss, so it is not going to.\n' \ + "$ME" "$STALL_MAX" + printf '%s: waiting for every lane to stop, then printing what is left.\n' "$ME" + break + fi _stall=$((_stall + 1)) sleep 1 done - # Whatever the loop above left: a lane that `die`d stopped where it stood, so - # the checks behind it in that lane have no block at all. Say so rather than - # print nothing, which reads exactly like a check that passed. - while [ "$_i" -le "$NCHECKS" ]; do - if [ -f "$OUT/out.$_i" ]; then - cat "$OUT/out.$_i" - else - printf '\n%s: %s NEVER RAN: its lane stopped before reaching it\n' "$ME" \ - "$(awk -v n="$_i" 'NR == n {print; exit}' "$OUT/order")" >&2 - fi - _i=$((_i + 1)) - done - # Collected after every lane has been reaped, so which lane finished first - # cannot decide the answer: a 2 anywhere is a 2. + # Reaped BEFORE the remaining blocks are printed, for two reasons: the status + # is the answer and is never guessed from what a file happened to contain, + # and once every lane has stopped nothing is still writing, so the blocks + # below are whole and still in plan order. _worst=0 for _p in $_pids; do _rc=0 @@ -517,6 +520,19 @@ run_lanes() { # the plan -> the worst status of every lane esac done + # Whatever the loop above did not reach: a lane that stopped where it stood + # leaves the checks behind it with no block at all. Say so rather than print + # nothing, which reads exactly like a check that passed. + while [ "$_i" -le "$NCHECKS" ]; do + if [ -f "$OUT/out.$_i" ]; then + cat "$OUT/out.$_i" + else + printf '\n%s: %s NEVER RAN: its lane stopped before reaching it\n' "$ME" \ + "$(awk -v n="$_i" 'NR == n {print; exit}' "$OUT/order")" >&2 + fi + _i=$((_i + 1)) + done + rm -rf "$OUT" trap - EXIT HUP INT TERM return "$_worst" diff --git a/tests/test_host_contract.sh b/tests/test_host_contract.sh index bb7c8b8..c208101 100755 --- a/tests/test_host_contract.sh +++ b/tests/test_host_contract.sh @@ -90,6 +90,9 @@ case " \$* " in printf 'w${TAB}one${TAB}image-ro${TAB}after-the-failure\n' printf 'y${TAB}two${TAB}image-ro${TAB}other-lane-first\n' printf 'z${TAB}two${TAB}image-ro${TAB}other-lane-second\n' ;; + *" plan lanes-stall "*) + printf 'f${TAB}one${TAB}image-ro${TAB}quiet-ok\n' + printf 's${TAB}two${TAB}image-ro${TAB}very-slow-rc-one\n' ;; *" plan lanes-budget "*) printf 'p${TAB}one${TAB}host${TAB}podman${US}build${US}-t${US}localhost/x:sdlc${US}.\n' printf 'q${TAB}two${TAB}image-ro${TAB}quiet-ok\n' ;; @@ -107,6 +110,7 @@ case "\$_last" in rc-one) printf 'RAN rc-one\n'; exit 1 ;; rc-two) printf 'RAN rc-two\n'; exit 2 ;; slow-rc-one) printf 'RAN slow-rc-one\n'; sleep 2; exit 1 ;; + very-slow-rc-one) printf 'RAN very-slow-rc-one\n'; sleep 6; printf 'LAST LINE\n'; exit 1 ;; slow-rc-two) printf 'RAN slow-rc-two\n'; sleep 2; exit 2 ;; must-not-run) printf 'RAN must-not-run\n'; touch "$SHIM/must-not-run" ;; after-the-failure|other-lane-first|other-lane-second) @@ -163,10 +167,11 @@ printf '\n== the lanes\n\n' RUN=$(mktemp -d) trap 'rm -rf "$SHIM" "$RUN"' EXIT 0 -lanes() { # target -> exit status, output in $RUN/out +lanes() { # target [stall-seconds] -> exit status, output in $RUN/out rm -f "$SHIM/must-not-run" _rc=0 - PATH="$SHIM" SDLC_CPUS=4 sh bin/sdlc "$1" > "$RUN/out" 2>&1 || _rc=$? + PATH="$SHIM" SDLC_CPUS=4 SDLC_LANE_STALL=${2:-3600} \ + sh bin/sdlc "$1" > "$RUN/out" 2>&1 || _rc=$? return "$_rc" } @@ -214,6 +219,41 @@ if grep -q '^== a1 passed in [0-9]*s' "$RUN/out"; then else bad "a block does not report its check's duration" fi +if ! grep -q 'has not reported for' "$RUN/out"; then + ok "a healthy run never says a lane stopped answering" +else + bad "a healthy run claimed a lane had stopped answering" +fi + +# A lane killed outright runs no EXIT trap, so it never says it is done and the +# printer would poll for it forever. At the threshold it stops waiting on the +# ORDER -- and must still reap every lane for its real status, and still print +# the late block WHOLE rather than reading a file another process is writing. +# Reached here with a one-second threshold and a lane that takes six. +lanes lanes-stall 1 && rc=0 || rc=$? +if grep -q 'has not reported for 1s' "$RUN/out"; then + ok "the printer stops waiting on the order and says so" +else + bad "the stall threshold was never reached" +fi +if [ "$rc" -eq 1 ]; then + ok "a lane's status is still collected after the order is given up" +else + bad "giving up on the order lost the lane's status (exit $rc)" +fi +if grep -q '^== f passed in [0-9]*s' "$RUN/out"; then + ok "the block that did finish in time is still printed" +else + bad "the finished block was dropped when the order was given up" +fi +# LAST LINE is written six seconds in, after the printer gave up at one. It can +# only be here if the block was printed after the lane was reaped. +if grep -q '^LAST LINE$' "$RUN/out" && grep -q 'sdlc: s FAILED after' "$RUN/out"; then + ok "the late block is printed whole, after its lane was reaped" +else + bad "the late block was read while its lane was still writing it" + sed 's/^/ /' "$RUN/out" +fi # A 2 anywhere wins over a 1, in either finishing order. Fast-2/slow-1 and # fast-1/slow-2 are two different code paths through the aggregation, and the From 867a3662c9a0ba18b70f20671246f948480d18ee Mon Sep 17 00:00:00 2001 From: Scott McCarty Date: Wed, 4 Nov 2026 09:40:36 -0500 Subject: [PATCH 5/5] docs: the stall threshold measures silence, not a killed lane Gatehouse, and it is my error rather than a wording nit. The timer counts seconds with no check FINISHING, so a check that honestly takes longer than the threshold trips it too -- the claim that only an outright-killed lane reaches it was false in bin/sdlc's comment, the message it prints, the CHANGELOG and the README alike. All four now say what is measured. A killed lane is the case worth guarding, because it runs no EXIT trap and so never reports at all, leaving the printer polling for a marker that is never coming. A slow check is the other way in, which is why the default is an hour, why the message names SDLC_LANE_STALL, and why tripping it is cosmetic: the order is abandoned, the verdict is not, because the reap is the authority. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01R7Y1jesjfgxHeqtKJCvay6 --- CHANGELOG.md | 15 +++++++++------ README.md | 13 ++++++++----- bin/sdlc | 27 ++++++++++++++++----------- tests/test_host_contract.sh | 22 ++++++++++++---------- 4 files changed, 45 insertions(+), 32 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 99600cb..634d3b4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -56,12 +56,15 @@ All notable changes to this project are documented here, following ### Added -- `SDLC_LANE_STALL` (default 3600 seconds): how long the printer waits for a - lane that has stopped answering. Only a lane killed outright -- SIGKILL, the - OOM killer, a cancelled CI job -- gets there, because every other ending runs - the lane's cleanup. At the threshold the launcher says so, waits for every - lane to stop and prints what is left; each lane's real exit status is - collected either way. +- `SDLC_LANE_STALL` (default 3600 seconds): how long the printer waits with no + check finishing before it stops waiting on the order. What it measures is + silence, and two things cause it -- a lane killed outright (SIGKILL, the OOM + killer, a cancelled CI job), which runs no cleanup and so never reports at + all, or one check that honestly takes longer than the threshold. The first is + the one worth guarding, because without the guard the printer polls for a + report that is never coming. Tripping it is cosmetic: the launcher says so, + waits for every lane to stop and prints what is left, possibly out of plan + order, and every lane's real exit status is still collected. ### Fixed diff --git a/README.md b/README.md index 29524b4..5a35b57 100644 --- a/README.md +++ b/README.md @@ -102,11 +102,14 @@ the checks, so two lanes never cut into each other's log. Each block ends with how long its check took, because buffering is what the `== ` timestamps used to tell you. -A lane killed outright — SIGKILL, the OOM killer, a cancelled CI job — runs no -cleanup and never reports, so the printer stops waiting on the order after -`SDLC_LANE_STALL` seconds (default 3600), says so, waits for every lane to -stop and then prints what is left. Each lane's real exit status is still -collected either way. +If no check finishes for `SDLC_LANE_STALL` seconds (default 3600) the printer +stops waiting on the order, says so, waits for every lane to stop and then +prints what is left. That guards against a lane killed outright — SIGKILL, the +OOM killer, a cancelled CI job — which runs no cleanup and so never reports at +all, leaving the printer waiting for a report that is never coming. One check +honestly taking longer than the threshold reaches it too; raise the variable if +that is your repo. Either way only the order is given up, never the verdict: +every lane's real exit status is still collected. ## Configuring it diff --git a/bin/sdlc b/bin/sdlc index fe92f66..5335775 100755 --- a/bin/sdlc +++ b/bin/sdlc @@ -367,11 +367,15 @@ run_serial() { # the plan -> its worst step status OUT= # the lane scratch directory, while lanes are running -# How long the printer waits for a lane that has stopped answering before it -# stops waiting on the order. Only a lane killed outright -- SIGKILL, the OOM -# killer, a cancelled CI job -- gets here, because every other ending runs the -# lane's EXIT trap. Settable so the test can reach the boundary without -# waiting an hour for it. +# How long the printer waits with NO CHECK FINISHING before it stops waiting on +# the order. What it measures is silence, not death, and two things cause it: +# a lane killed outright -- SIGKILL, the OOM killer, a cancelled CI job -- which +# runs no EXIT trap and so never reports at all; or a single check that honestly +# takes longer than this. The first is the one worth guarding, because without +# the guard the printer polls for a marker that is never coming. The second is +# why the default is an hour and why tripping it is cosmetic: the order is +# abandoned, the exit status is not, because the reap below is the authority. +# Settable so the test can reach the boundary without waiting an hour for it. STALL_MAX=${SDLC_LANE_STALL:-3600} case "$STALL_MAX" in ''|*[!0-9]*) die "SDLC_LANE_STALL must be a whole number" ;; esac @@ -492,13 +496,14 @@ run_lanes() { # the plan -> the worst status of every lane continue fi _lanes_running || break - # A lane SIGKILLed outright runs no EXIT trap, so it never says it is done - # and this would poll for it forever. After STALL_MAX seconds with no block - # finishing, stop waiting on the order and fall through to the reap. + # Nothing has finished for STALL_MAX seconds. Either a lane was killed and + # will never report, or a check is honestly taking that long. Either way, + # stop waiting on the order and fall through to the reap. if [ "$_stall" -ge "$STALL_MAX" ]; then - printf '%s: a lane has not reported for %ss, so it is not going to.\n' \ - "$ME" "$STALL_MAX" - printf '%s: waiting for every lane to stop, then printing what is left.\n' "$ME" + printf '%s: no check has finished for %ss.\n' "$ME" "$STALL_MAX" + printf '%s: waiting for every lane to stop, then printing what is left,\n' "$ME" + printf '%s: which may not be in plan order. Raise SDLC_LANE_STALL if a\n' "$ME" + printf '%s: check here really does take that long.\n' "$ME" break fi _stall=$((_stall + 1)) diff --git a/tests/test_host_contract.sh b/tests/test_host_contract.sh index c208101..f22d62c 100755 --- a/tests/test_host_contract.sh +++ b/tests/test_host_contract.sh @@ -219,20 +219,22 @@ if grep -q '^== a1 passed in [0-9]*s' "$RUN/out"; then else bad "a block does not report its check's duration" fi -if ! grep -q 'has not reported for' "$RUN/out"; then - ok "a healthy run never says a lane stopped answering" +if ! grep -q 'no check has finished for' "$RUN/out"; then + ok "a healthy run never claims nothing is finishing" else - bad "a healthy run claimed a lane had stopped answering" + bad "a healthy run claimed no check had finished" fi -# A lane killed outright runs no EXIT trap, so it never says it is done and the -# printer would poll for it forever. At the threshold it stops waiting on the -# ORDER -- and must still reap every lane for its real status, and still print -# the late block WHOLE rather than reading a file another process is writing. -# Reached here with a one-second threshold and a lane that takes six. +# Nothing finishing for STALL_MAX seconds. The case worth guarding is a lane +# killed outright, which runs no EXIT trap and so never reports at all, leaving +# the printer polling for a marker that is never coming. At the threshold it +# gives up the ORDER -- and must still reap every lane for its real status, and +# still print the late block WHOLE rather than reading a file another process +# is writing. Reached here with a one-second threshold and a check that takes +# six, which is the OTHER way in and the cheaper one to arrange. lanes lanes-stall 1 && rc=0 || rc=$? -if grep -q 'has not reported for 1s' "$RUN/out"; then - ok "the printer stops waiting on the order and says so" +if grep -q 'no check has finished for 1s' "$RUN/out"; then + ok "the printer gives up the order and says what it measured" else bad "the stall threshold was never reached" fi