Repository navigation
fix: the host clones the constitution, so the gate needs no warm cache #26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
0f993d1
0e58fd4
ccaf13b
b6fc416
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1 @@ | ||
| 0.5.0 | ||
| 0.5.1 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -172,6 +172,48 @@ need_volume() { | |
| podman volume exists "$1" 2>/dev/null || die "cannot create the podman volume $1" | ||
| } | ||
|
|
||
| # ------------------------------------------------- the constitution, by the host | ||
| # | ||
| # `validate` clones crunchtools/constitution at the tag the repo pins, into the | ||
| # cache volume. From inside the container that clone takes whatever route the | ||
| # network gives a container, and on a gateway seat that is a CONNECT proxy that | ||
| # refuses github.com: the route to GitHub there is a git `insteadOf` in the | ||
| # HOST's git config, which the container never reads. A warm cache volume hid | ||
| # it; an empty one, or a repo moving to a tag nobody has fetched, was a gate | ||
| # that could not run (claude-agent#9, developer3, 2026-11-06). | ||
| # | ||
| # So the host's git makes the clone, once per tag, and the checkout is mounted | ||
| # read-only at the path `validate` already looks in. The container still gets | ||
| # no git config and no credentials -- what crosses is a public repo's files. | ||
| # Done once here, before any lane forks. Two sdlc runs -- two repos on one tag | ||
| # -- can still meet, so each clones into its own directory and publishes it | ||
| # with `ln -sn`, which only one of them can win; the loser removes only its own | ||
| # copy and uses the winner's. Every failure is silent on purpose: without the | ||
| # mount `validate` tries its own clone and reports a 2 exactly as it did before. | ||
| CONSTITUTION_MOUNT= | ||
| host_constitution() { | ||
| _manifest=$REPO/.specify/memory/constitution.md | ||
| [ -f "$_manifest" ] || return 0 | ||
| _tag=$(awk '/^>[ \t]*\*\*Inherits:\*\*/ { | ||
| if (match($0, /v[0-9]+\.[0-9]+\.[0-9]+/)) print substr($0, RSTART, RLENGTH) | ||
| exit | ||
| }' "$_manifest") | ||
| [ -n "$_tag" ] || return 0 | ||
| _root=${XDG_CACHE_HOME:-$HOME/.cache}/python-sdlc/constitution | ||
| _dir=$_root/$_tag | ||
| if [ ! -f "$_dir/constitution.md" ]; then | ||
| mkdir -p "$_root" 2>/dev/null || return 0 | ||
| _own=$(mktemp -d "$_root/.$_tag.XXXXXX" 2>/dev/null) || return 0 | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. MEDIUM (Performance Check): Concurrent runs that need the same uncached tag each create a temporary directory and clone the full shallow repository before racing to publish it. The losing run then deletes its clone, so concurrent runs cause redundant network and disk work. Suggestion: Serialize cache population per tag, then recheck whether the cache exists before cloning. Alternatively, use a mechanism that lets later runs wait for and reuse the in-progress clone. Evidence:
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. not a bug: the duplicate clone only happens when two runs start on a cold cache for a tag nobody has fetched, which is about once per seat per constitution release. It costs one There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. MEDIUM (Test Coverage): The new concurrent cache-publication logic has no corresponding test coverage in the changes shown. Its behavior depends on two runs racing to publish the symlink, including the losing run cleaning up only its own clone and then using the winner's checkout. Suggestion: Add a test that runs two cache population attempts for the same tag concurrently and verifies that one checkout is published, the loser does not remove the winner's data, and both runs can use the published constitution. Also cover clone failure cleanup. Evidence:
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. fixed in ccaf13b
|
||
| git clone --quiet --depth 1 --branch "$_tag" \ | ||
| https://github.com/crunchtools/constitution "$_own" >/dev/null 2>&1 \ | ||
| || { rm -rf "$_own"; return 0; } | ||
| ln -sn "${_own##*/}" "$_dir" 2>/dev/null || rm -rf "$_own" | ||
| [ -f "$_dir/constitution.md" ] || return 0 | ||
| fi | ||
| CONSTITUTION_MOUNT="$_dir:/cache/constitution/$_tag:ro,z" | ||
| } | ||
| host_constitution | ||
|
|
||
| # ------------------------------------------------------------------ podman run | ||
| # | ||
| # `z`, not `Z`: more than one of these can mount the same tree at once, and `Z` | ||
|
|
@@ -221,6 +263,7 @@ run_in_image() { | |
| --cpus "$GATE_CPUS" --memory "$GATE_MEMORY" --cpu-shares 256 \ | ||
| -v "$REPO:/repo:$_mode,z" -w /repo \ | ||
| -v "$CACHE_VOLUME:/cache" \ | ||
| ${CONSTITUTION_MOUNT:+-v "$CONSTITUTION_MOUNT"} \ | ||
| --tmpfs /tmp:rw,exec,size=4g,mode=1777 \ | ||
| -e HOME=/tmp \ | ||
| -e PYTHONPYCACHEPREFIX=/tmp/pycache \ | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -607,6 +607,57 @@ else | |
| bad "$locked of ${runs:-0} gate containers dropped capabilities" | ||
| fi | ||
|
|
||
| # The constitution is cloned by the HOST's git and published with `ln -sn`, so | ||
| # runs that meet on a cold cache never delete each other's clone. Three run at | ||
| # once against a git whose clone is slow and offline; what must be left is one | ||
| # checkout, one link to it, and no loser's copy. | ||
| CLONESHIM=$(mktemp -d) | ||
| CLONECACHE=$(mktemp -d) | ||
| for prog in $POSIX_UTILS podman pre-commit gh; do | ||
| [ -e "$SHIM/$prog" ] && ln -s "$(readlink -f "$SHIM/$prog")" "$CLONESHIM/$prog" | ||
| done | ||
| REALGIT=$(command -v git) | ||
| cat > "$CLONESHIM/git" <<EOF | ||
| #!/bin/sh | ||
| case "\$1" in | ||
| clone) for dest; do :; done; sleep 1; : > "\$dest/constitution.md" ;; | ||
| *) exec "$REALGIT" "\$@" ;; | ||
| esac | ||
| EOF | ||
| chmod +x "$CLONESHIM/git" | ||
| # The check's own verdict depends on the checkout (CI has no pre-push hook), so | ||
| # each concurrent run must match one that had a cache to itself. | ||
| _solo=$(mktemp -d) | ||
| PATH="$CLONESHIM" XDG_CACHE_HOME="$_solo" sh bin/sdlc hooks-installed >/dev/null 2>&1 \ | ||
| && _want=0 || _want=$? | ||
| rm -rf "$_solo" | ||
| _pids= | ||
| for _ in 1 2 3; do | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. MEDIUM (Test Coverage): The concurrency test verifies the final cache layout but never verifies that all three Suggestion: Capture each background process ID and check the result of each corresponding Evidence:
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. fixed in b6fc416 Each background run is now waited on by its PID, and its exit status is compared with a baseline run that had a cache to itself. Comparing with a baseline rather than 0 is deliberate: |
||
| PATH="$CLONESHIM" XDG_CACHE_HOME="$CLONECACHE" sh bin/sdlc hooks-installed >/dev/null 2>&1 & | ||
| _pids="$_pids $!" | ||
| done | ||
| _odd=0 | ||
| for _pid in $_pids; do | ||
| wait "$_pid" && _got=0 || _got=$? | ||
| [ "$_got" = "$_want" ] || _odd=$((_odd + 1)) | ||
| done | ||
| if [ "$_odd" = 0 ]; then | ||
| ok "three runs racing one cold cache each exit $_want, as a run alone does" | ||
| else | ||
| bad "$_odd of 3 runs racing one cold cache did not exit $_want" | ||
| fi | ||
| _root="$CLONECACHE/python-sdlc/constitution" | ||
| _links=$(find "$_root" -mindepth 1 -maxdepth 1 -type l | wc -l | tr -d ' ') | ||
| _dirs=$(find "$_root" -mindepth 1 -maxdepth 1 -type d | wc -l | tr -d ' ') | ||
| _nested=$(find "$_root" -mindepth 2 -name '.v*' | wc -l | tr -d ' ') | ||
| if [ "$_links" = 1 ] && [ "$_dirs" = 1 ] && [ "$_nested" = 0 ] \ | ||
| && [ -f "$(find "$_root" -mindepth 1 -maxdepth 1 -type l)/constitution.md" ]; then | ||
| ok "three runs on a cold cache leave one constitution checkout and one link" | ||
| else | ||
| bad "concurrent constitution clones left $_links link(s), $_dirs dir(s), $_nested nested" | ||
| fi | ||
| rm -rf "$CLONESHIM" "$CLONECACHE" | ||
|
|
||
| printf '\n' | ||
| if [ "$FAILED" -eq 0 ]; then | ||
| printf 'host contract PASSED\n'; exit 0 | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
MEDIUM (Bug Hunter): Concurrent
sdlcinvocations can delete or interfere with each other's in-progress clone. Both can observe the missing file, then one invocation'srm -rfcan remove the directory while the other is cloning; a failed clone can likewise remove the other invocation's checkout. On seats where the container cannot reach GitHub, this can make the constitution gate fail intermittently.Suggestion: Serialize cache population with a per-tag lock, or clone into a unique temporary directory and atomically publish it without deleting a directory another process may be populating.
Evidence:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed in 0e58fd4
Each run now clones into its own
mktemp -ddirectory and publishes it withln -sn, so only one run can win. The loser deletes only its own copy and uses the winner's. Nothing removes a shared path anymore. I tested three runs started at once on an empty cache: all three passed and left one checkout and one link.