Skip to content

(standardize-repo): Report each repo's devcontainer definition, CI image and staleness, and queue rebuilds #1670

Closed

Description

@evanharmon1

Problem

Running containers go stale after a template update: the maintainer rarely rebuilds, and the staleness check is warn-only and not surfaced at session start. A standardize run should verify, per repo, that the devcontainer definition rendered, that the devcontainer builds and devcontainer-verify passed on the PR head, that the ruleset requires devcontainer-verify (seven managed repos did not on 2026-10-26), and after merge that the devcontainer-build run on the default branch for the merged head succeeded and pushed (read from the workflow run, no packages scope), so the GHCR image matches the definition. It should list in the fleet report a rebuild queue, one command per workspace (devcontainer up for the CLI path, the Coder rebuild for Coder workspaces, or task devcontainer:rebuild once harmon-init#1833 ships), ordered after a wave's merges. Before any lane starts, the orchestrator's own container must pass the staleness probe and meet the release's tool floors; lanes in a stale container fail the new gates and look like template bugs. It should never rebuild inside a run: an agent cannot rebuild the container it runs in, a rebuild mid-wave kills every lane in the workspace, and the hard rules reserve process termination for the maintainer.

Acceptance criteria

  • [CI] The fleet run checks the orchestrator's own container against the staleness probe and the release's tool floors before creating the epic, and stops with the rebuild command when it fails.
  • [CI] The per-repo report has a devcontainer row: definition rendered, devcontainer builds and devcontainer-verify on the head, whether the ruleset requires devcontainer-verify, the default-branch devcontainer-build run for the merged head (succeeded and pushed), and whether the staleness probe can run.
  • [CI] The fleet report lists the rebuild queue with one command per workspace the decisions file names, ordered after a wave's merges.
  • [CI] SKILL.md states that a run never rebuilds a container.

Verify

Re-read the ADR and spec named under Provenance for the current design, and re-establish every path, task name or observation above with git grep in the target checkout (and gh for GitHub state) before acting; a renamed path means the claim needs re-checking, not that the problem is gone.

Out of scope

SessionStart staleness summary and host-side rebuild task (harmon-init).

Provenance

Filed from the 2026-10-26 standardize-repo redesign session (harmon-devkit, worktree/clear-river-0602). Design record: harmon-devkit ADR docs/decisions/2026-10-26-fleet-standardization-runs.md and spec specs/standardize-repo-fleet.md.

Activity

  1. added
    featureNew feature or request
    ai-generatedCreated or authored by an AI agent
    layer:integrationExternal boundary: webhooks, API clients, credentials
    domain:standardizationKeeping repos current with conventions: standardize-repo, drift audits, adoption
    area:standardize-repoThe standardize-repo skill and its standards catalog
    impact:mediumImpact: clear benefit to a meaningful share of users or goals
    risk:lowRisk: a mistake is contained, caught quickly, and cheap to undo
    complexity:sComplexity: small and local; a clear approach across a few files
    on Oct 26, 2026
  2. added
    tier:economyModel tier: cheapest qualified hosted model first; escalation allowed
    on Oct 26, 2026
  3. self-assigned this
    on Nov 2, 2026
  4. evanharmon1 commented on Nov 2, 2026

    @evanharmon1
    OwnerAuthor

    Claiming — starting implementation on branch feat/1670-devcontainer-report (session fleet-m6-foundations).

    Preflight (§3):

    • credentials/scopes: unverified — the fleet operator's GitHub credential needs repository, ruleset and Actions read access; the build workflow token needs image publication permission. Scopes were not verified.
    • human/out-of-band steps: the maintainer rebuilds each workspace after its wave merges; no container rebuild runs inside the fleet run.
    • plan-vs-apply blind spots: static checks cannot prove the container builds or the image was pushed; inspect head CI and the merged-head default-branch workflow run.
    • stale references: the ADR and fleet spec still describe this work; template ownership of prospective implementation files is unproven without an identified current source checkout. The repository is a Copier consumer of harmon-init at 362211f89b114eab65390eb013d2be8634c6e16c, not a root Copier template source.

    Claim record (for /wrap — undo only what this claim added):

    • harness: codex-cli
    • model: gpt-6.1-terra
    • family: gpt
    • runtime environment: coder
    • session: fleet-m6-foundations
    • dispatched to: none
    • assignee added by this claim: yes
    • claim: label added by this claim: claim:gpt
    • claim: model label added by this claim: n/a
    • claim: label displaced by this claim: none
    • assignee logins owned by this claim chain: evanharmon1
    • claim: label owned by this claim chain: claim:gpt
    • claim: model label owned by this claim chain: n/a
    • claim: label displaced by this claim chain: none
  5. 10 remaining items

  6. evanharmon1 commented on Nov 3, 2026

    @evanharmon1
    OwnerAuthor

    Dev flow v2 evidence: review round 1 for run-1670-devcontainer-report at 1cc912e (finder codex-verification).

    {
      "run_id": "run-1670-devcontainer-report",
      "stage": "review",
      "round": 1,
      "reviewed_head": "1cc912e05798b70194b6916679d0774a2ccbf9e8",
      "passes": [
        {
          "schema": 2,
          "role": "reviewer",
          "status": "completed",
          "head": "1cc912e05798b70194b6916679d0774a2ccbf9e8",
          "produced_at": "2026-11-03T03:13:56Z",
          "producer": {
            "harness": "codex-cli",
            "model": "gpt-6-astra",
            "tier": "frontier"
          },
          "run": {
            "run_id": "run-1670-devcontainer-report",
            "initiated_by": "human"
          },
          "payload": {
            "stage": "review",
            "round": 1,
            "reviewed_head": "1cc912e05798b70194b6916679d0774a2ccbf9e8",
            "finder": "codex-verification",
            "slot": "codex-verification",
            "findings": [
              {
                "id": "review-r1-codex-verification-1",
                "path": "ai/skills/repo/standardize-repo/assets/fleet-report.py",
                "line": 239,
                "class": "correctness",
                "provenance": "original",
                "fingerprint": "new",
                "priority": "P2",
                "recommended_disposition": "fix",
                "evidence": "Codex [P2] 'Preserve significant whitespace in rebuild commands' (fleet-report.py:238-240): the rebuild command is rendered as ordinary Markdown table text via text(), so GitHub's rendered view collapses repeated spaces inside a quoted argument (e.g. devcontainer up --workspace-folder '/work/my  app' copies as a different path); escaping Markdown punctuation alone does not deliver the 'verbatim' display references/fleet-report.md promises; asks for a whitespace-preserving representation plus a regression test. Reviewer check via git show 1cc912e0:ai/skills/repo/standardize-repo/assets/fleet-report.py: line 239 builds the cell with text(entry[\"command\"]); text() (lines 26-33) html-escapes, backslash-escapes \\`*_{}[]()|!#@ and replaces CR/LF, but keeps consecutive spaces as raw spaces, which HTML rendering collapses. Running text() on the example returns the double space unchanged in source. Confirmed as described."
              },
              {
                "id": "review-r1-codex-verification-2",
                "path": "ai/skills/repo/standardize-repo/assets/devcontainer-probe.sh",
                "line": 92,
                "class": "correctness",
                "provenance": "original",
                "fingerprint": "new",
                "priority": "P2",
                "recommended_disposition": "fix",
                "evidence": "Codex [P2] 'Retain other known workflow-run conclusions' (devcontainer-probe.sh:91-93): when the merged-head devcontainer-build run completes cancelled, timed_out or action_required, the API returns a definite conclusion, but the whitelist reports 'unknown', making a known unsuccessful run indistinguishable from missing evidence; asks to preserve those conclusions (push still unknown) and test them. Reviewer check via git show 1cc912e0:ai/skills/repo/standardize-repo/assets/devcontainer-probe.sh: line 92 is `result = conclusion if conclusion in ('success', 'failure', 'skipped') else 'unknown'`, so any other completed conclusion (cancelled, timed_out, action_required, neutral, stale, startup_failure) is mapped to 'unknown'; test-devcontainer-probe.sh test_image_conclusions covers only failure/skipped/missing-run. Confirmed as described."
              }
            ],
            "counts": {
              "P0": 0,
              "P1": 0,
              "P2": 2,
              "P3": 0
            }
          }
        }
      ],
      "adjudication": {
        "schema": 2,
        "run_id": "run-1670-devcontainer-report",
        "stage": "review",
        "round": 1,
        "reviewed_head": "1cc912e05798b70194b6916679d0774a2ccbf9e8",
        "adjudications": [
          {
            "finding_id": "review-r1-codex-verification-1",
            "reviewer_priority": "P2",
            "adjudicated_priority": "P2",
            "disposition": "fix",
            "reason": "Confirmed by reading: rebuild commands render through text() into a plain Markdown table cell, so GitHub collapses repeated spaces, and $…$ / ~…~ can render as math or strikethrough; the copied command differs from the decisions file. Render each command as a code span whose backtick fence is longer than any backtick run inside, with pipes escaped for the table and padding where it starts or ends with a backtick or space, and test it.",
            "evidence": "fleet-report.py@1cc912e0:239; text() line 33",
            "override": null
          },
          {
            "finding_id": "review-r1-codex-verification-2",
            "reviewer_priority": "P2",
            "adjudicated_priority": "P2",
            "disposition": "fix",
            "reason": "Confirmed by reading: only success, failure and skipped pass through; a completed run concluding cancelled, timed_out, action_required, neutral, stale or startup_failure reads as unknown, hiding an observed result. Report any completed conclusion from that documented set verbatim, keep unknown for a missing, in-progress or unrecognised run, and test cancelled and timed_out.",
            "evidence": "devcontainer-probe.sh@1cc912e0:92",
            "override": null
          }
        ]
      },
      "exit": {
        "stage": "review",
        "rounds_counted": 1,
        "next_round": 2,
        "substitutions": [],
        "split_candidate": {
          "detected": false,
          "round": 1,
          "mechanism": null,
          "concentration": 0,
          "provenance_share": 0,
          "introduced_by_rounds": [],
          "finding_ids": [],
          "consecutive_rounds": [
            1
          ],
          "reason": "no_gating_findings"
        },
        "outcome": "continue",
        "reason": "below_threshold",
        "action": "dispatch",
        "corrections": [],
        "diagnostics": [],
        "verified_findings": [
          {
            "id": "review-r1-codex-verification-1",
            "provenance_status": "verified",
            "verified_provenance": "original",
            "fingerprint_status": "verified",
            "verified_fingerprint": "new"
          },
          {
            "id": "review-r1-codex-verification-2",
            "provenance_status": "verified",
            "verified_provenance": "original",
            "fingerprint_status": "verified",
            "verified_fingerprint": "new"
          }
        ]
      }
    }

    Round

    review round 1 — reviewed 1cc912e
    Finding Reviewer Adjudicated Classification Evidence Action Provenance
    review-r1-codex-verification-1 P2 P2 correctness fleet-report.py@1cc912e0:239; text() line 33 fix — Confirmed by reading: rebuild commands render through text() into a plain Markdown table cell, so GitHub collapses repeated spaces, and $…$ / … can render as math or strikethrough; the copied command differs from the decisions file. Render each command as a code span whose backtick fence is longer than any backtick run inside, with pipes escaped for the table and padding where it starts or ends with a backtick or space, and test it. original (verified)
    review-r1-codex-verification-2 P2 P2 correctness devcontainer-probe.sh@1cc912e0:92 fix — Confirmed by reading: only success, failure and skipped pass through; a completed run concluding cancelled, timed_out, action_required, neutral, stale or startup_failure reads as unknown, hiding an observed result. Report any completed conclusion from that documented set verbatim, keep unknown for a missing, in-progress or unrecognised run, and test cancelled and timed_out. original (verified)

    Exit: continue — below_threshold (rounds counted: 1; next round: 2)

  7. evanharmon1 commented on Nov 3, 2026

    @evanharmon1
    OwnerAuthor

    Dev flow v2 evidence: review round 2 for run-1670-devcontainer-report at 800cf34 (finder codex-verification).

    {
      "run_id": "run-1670-devcontainer-report",
      "stage": "review",
      "round": 2,
      "reviewed_head": "800cf34ba113ad96b48a0b9134f7140131abf0eb",
      "passes": [
        {
          "schema": 2,
          "role": "reviewer",
          "status": "completed",
          "head": "800cf34ba113ad96b48a0b9134f7140131abf0eb",
          "produced_at": "2026-11-03T13:58:49Z",
          "producer": {
            "harness": "codex-cli",
            "model": "gpt-6-astra",
            "tier": "frontier"
          },
          "run": {
            "run_id": "run-1670-devcontainer-report",
            "initiated_by": "human"
          },
          "payload": {
            "stage": "review",
            "round": 2,
            "reviewed_head": "800cf34ba113ad96b48a0b9134f7140131abf0eb",
            "finder": "codex-verification",
            "slot": "codex-verification",
            "findings": [],
            "counts": {
              "P0": 0,
              "P1": 0,
              "P2": 0,
              "P3": 0
            }
          }
        }
      ],
      "adjudication": {
        "schema": 2,
        "run_id": "run-1670-devcontainer-report",
        "stage": "review",
        "round": 2,
        "reviewed_head": "800cf34ba113ad96b48a0b9134f7140131abf0eb",
        "adjudications": []
      }
    }
  8. github-actions commented on Nov 3, 2026

    @github-actions

    Claim released — PR #1823 merged (claiming work unit complete). (Supersedes the claim record above.)

    Released by claim-release automation:

    • claim:gpt label: removed
    • assignee evanharmon1: removed
  9. github-actions commented on Nov 3, 2026

    @github-actions

    Claim released — issue closed (completed). (Supersedes the claim record above.)

    Released by claim-release automation:

    • claim:gpt label: removed
    • assignee evanharmon1: removed
  10. evanharmon1 commented on Nov 3, 2026

    @evanharmon1
    OwnerAuthor

    Slate membership:

    • session: fleet-m6-foundations
    • slate: m6-fleet-foundations
    • action: remove
  11. evanharmon1 commented on Nov 3, 2026

    @evanharmon1
    OwnerAuthor

    Slate membership:

    • session: fleet-m6-agy
    • slate: m6-fleet-foundations
    • action: remove
  12. removed
    slatedIssue is in an orchestrator slate plan (membership only, not a state)
    on Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ai-generatedCreated or authored by an AI agentarea:standardize-repoThe standardize-repo skill and its standards catalogcomplexity:sComplexity: small and local; a clear approach across a few filesdomain:standardizationKeeping repos current with conventions: standardize-repo, drift audits, adoptionfeatureNew feature or requestimpact:mediumImpact: clear benefit to a meaningful share of users or goalslayer:integrationExternal boundary: webhooks, API clients, credentialspriority-ai:p2Priority (AI): worth doing, not blockingrisk:lowRisk: a mistake is contained, caught quickly, and cheap to undotier:economyModel tier: cheapest qualified hosted model first; escalation allowed

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions