Skip to content

(standardize-repo): Provision the model labels before the draft PR instead of after merge #1531

Closed

Description

@evanharmon1

Problem

A standardize-repo update cannot reach its readiness gate until the repository's live labels include the model labels the implementer record writes. The skill provisions those labels only after the update PR merges.

  • implement/assets/record-implementer.sh labels the issue and the PR with exactly one model-family:*, model:* and model-version:* label, and at most one model-effort:* label.
  • The integrate stage re-checks that record before the gate (harmon-devkit#1262).
  • A template update that adds a model line, such as model:sol or model-version:6.1, ships the label-registry entry in files. The live label does not exist yet.
  • references/mode-update.md §6 runs task setup:github-labels only once the update PR has merged. It gives a reason: that task is gh label create --force, which overwrites the color and description of existing labels.

The writer therefore refuses, and the lane stops to ask. In the fleet rollout every wave-2 and wave-3 lane stopped on this. The orchestrator ran task setup:github-labels by hand in each repository, for example ponderous-content (221 ok), lawnomator-site (218 ok), omator (227 ok) and harmon-infra (222 ok).

Creating a label that does not exist yet is not the hazard the post-merge rule guards against. Overwriting an existing one is.

Current violation (observed 2026-10-17)

references/mode-update.md §6 gates every metadata task, labels included, on the merged PR. No step before the draft PR provisions the model labels the implementer record needs.

Acceptance criteria

  • [CI] The label setup has an additive mode that creates only missing labels and never changes an existing label's color or description. A test pins that an existing label is left byte-identical.
  • [CI] mode-update.md adds a step before the draft PR. It runs that additive mode for the target release's label vocabulary, after the maintainer approves it, and records the result.
  • [CI] The step accepts one approval covering a whole fleet run, so each lane does not ask again.
  • [CI] §6 keeps the post-merge --force reconciliation unchanged, and says the additive step does not replace it.

Verify

grep -n 'Run this only once the update PR has merged' ai/skills/repo/standardize-repo/references/mode-update.md
grep -n 'setup:github-labels' ai/skills/repo/standardize-repo/references/mode-update.md

While this is open, the first command matches in §6, and every setup:github-labels hit is in §6 or later.

Out of scope

Retired-label migration and the project and issue-field reconciliation, which stay post-merge.

Provenance

Retro of the session on 2026-10-17; the PR and its linked issues carried no Dev flow v2 evidence marker. The run was the 2026-10-13/16 standardize-repo fleet rollout of harmon-init v5.7.2 to v5.8.2 across 13 repositories. Among its PRs were harmonops/harmon-infra#862, sommerlawn/sommerlawn-site#527 and ponderousdev/lawnomator-site#255.

Activity

  1. added
    featureNew feature or request
    ai-generatedCreated or authored by an AI agent
    domain:standardizationKeeping repos current with conventions: standardize-repo, drift audits, adoption
    area:standardize-repoThe standardize-repo skill and its standards catalog
    layer:noneInapplicable: the work is not in a single stack slice
    impact:mediumImpact: clear benefit to a meaningful share of users or goals
    risk:mediumRisk: a mistake breaks a feature or needs a careful rollback
    complexity:sComplexity: small and local; a clear approach across a few files
    tier:standardModel tier: reliable general-purpose coding model first
    on Oct 18, 2026
  2. self-assigned this
    on Nov 2, 2026
  3. evanharmon1 commented on Nov 2, 2026

    @evanharmon1
    OwnerAuthor

    Claiming — starting implementation on branch feat/1531-additive-model-labels (session fleet-m6-foundations).

    Preflight (§3):

    • credentials/scopes: unverified — implementing session GitHub access reports ADMIN; consumer operators need a gh credential with repository label-write permission, which has not been checked across the fleet. No Project permission is required for this claim.
    • human/out-of-band steps: additive live provisioning needs maintainer approval; the implementation must accept one approval covering the fleet. No human-only prerequisite blocks this claim.
    • plan-vs-apply blind spots: local/stub tests cannot prove consumer live label-write authorization or API outcomes; verify live results after approved provisioning and record failures.
    • stale references: none in the reported post-merge restriction; mode-update.md:3442 still gates metadata reconciliation and the label helper still uses --force. Template provenance: harmon-init at immutable baseline 362211f89b114eab65390eb013d2be8634c6e16c; consumer only, no root Copier source manifest. Matching template remote verified and current default f9003319b20290c236c96ab28d5514b8efb873b9 searched.

    Findings:

    1. correction — scripts/setup-github-labels.sh exists in harmon-init's root and template payload at both the recorded baseline and current default. Implement its additive mode upstream with the root twin; harmon-devkit owns ai/skills/repo/standardize-repo/references/mode-update.md (the template checkout carries a vendored skill copy).
    2. note — open (standardize-repo): Run the GitHub-side reconcile phase in every mode, probing in lanes and writing from the parent #1669 overlaps fleet approval and parent-owned GitHub writes; coordinate that boundary in the forthcoming lane brief. Open PR feat(standardize-repo): keep a fleet report on the epic and a per-repo report on each tracking issue #1793 adds fleet reports but does not touch this issue's guide/helper targets. No linked implementation PR, foreign assignee, or claim marker found. Checkout is clean and current with origin/main 0c79878.

    Claim record (for /wrap — undo only what this claim added):

    • harness: codex-cli
    • model: gpt-6.1-sol
    • family: gpt
    • runtime environment: coder
    • session: fleet-m6-foundations
    • dispatched to: none
    • assignee added by this claim: yes
    • claim: label added by this claim: claim:gpt
    • claim: model label added by this claim: n/a
    • claim: label displaced by this claim: none
    • assignee logins owned by this claim chain: evanharmon1
    • claim: label owned by this claim chain: claim:gpt
    • claim: model label owned by this claim chain: n/a
    • claim: label displaced by this claim chain: none
  4. evanharmon1 commented on Nov 2, 2026

    @evanharmon1
    OwnerAuthor

    Slate membership:

    • session: fleet-m6-foundations
    • slate: m6-fleet-foundations
    • action: add
  5. added
    slatedIssue is in an orchestrator slate plan (membership only, not a state)
    on Nov 2, 2026
  6. evanharmon1 commented on Nov 3, 2026

    @evanharmon1
    OwnerAuthor

    Dev flow v2 evidence: challenge round 1 for run-1531-additive-model-labels at 9e0d274 (finder codex-adversarial).

    {
      "run_id": "run-1531-additive-model-labels",
      "stage": "challenge",
      "round": 1,
      "reviewed_head": "9e0d2740f87c0ce9c39a3fecea57284deae75e5c",
      "passes": [
        {
          "schema": 2,
          "role": "challenger",
          "status": "completed",
          "head": "9e0d2740f87c0ce9c39a3fecea57284deae75e5c",
          "produced_at": "2026-11-03T13:59:18Z",
          "producer": {
            "harness": "codex-cli",
            "model": "gpt-6-astra",
            "tier": "apex"
          },
          "run": {
            "run_id": "run-1531-additive-model-labels",
            "initiated_by": "human"
          },
          "payload": {
            "stage": "challenge",
            "round": 1,
            "reviewed_head": "9e0d2740f87c0ce9c39a3fecea57284deae75e5c",
            "finder": "codex-adversarial",
            "slot": "codex-adversarial",
            "findings": [
              {
                "id": "challenge-r1-codex-adversarial-1",
                "path": "ai/skills/repo/standardize-repo/SKILL.md",
                "line": 79,
                "class": "consistency",
                "provenance": "original",
                "fingerprint": "new",
                "priority": "P2",
                "recommended_disposition": "fix",
                "evidence": "Both `docs/decisions/2026-10-26-fleet-standardization-runs.md` and `specs/standardize-repo-fleet.md` (§ Start-of-run questions) state that `SKILL.md` lists the ordered question set with each question's default, its decisions-file key, and the later stop it removes. `ai/skills/repo/standardize-repo/assets/fleet-decisions.py` adds `label_provisioning` as the 6th question in `QUESTIONS` (shifting the question count from 10 to 11), and `references/fleet-decisions.md` + `references/mode-update.md` document it, but `SKILL.md` lines 69-79 still list only the previous 9 rows in the ordered question table and omit `label_provisioning`."
              }
            ],
            "counts": {
              "P0": 0,
              "P1": 0,
              "P2": 1,
              "P3": 0
            },
            "attack_scenarios": [
              {
                "id": "as-1",
                "description": "Case-insensitive collision & existing-label preservation: verify that when a live label exists with different casing (`Model:Sol`) and custom color/description, `ensure-missing-labels.sh` matches it case-insensitively (`ascii_downcase`), records it in `already_present`, and never invokes `gh label create --force`, `gh label edit`, or `gh label delete`.",
                "outcome": "held",
                "finding_id": null
              },
              {
                "id": "as-2",
                "description": "Truncated or malformed live label listing: verify that `ensure-missing-labels.sh` requests `--limit 10000` and refuses before any write when `gh label list` fails, returns non-array JSON, or returns `length == 10000` (which could hide a subsequent page and cause an existing label to be treated as missing).",
                "outcome": "held",
                "finding_id": null
              },
              {
                "id": "as-3",
                "description": "Missing or failing target renderer / malformed renderer output: verify that `ensure-missing-labels.sh` fails closed before listing or creating labels if `$checkout/scripts/label-registry-render.mjs` is missing, exits non-zero, emits records violating `^(?<name>[^|]+)\\|(?<color>[a-fA-F0-9]{6})\\|(?<description>.*)$`, or contains control characters or case-insensitive duplicate names.",
                "outcome": "held",
                "finding_id": null
              },
              {
                "id": "as-4",
                "description": "Mid-flight `gh label create` failure & partial progress accounting: verify that if a label creation fails after earlier labels were created, `ensure-missing-labels.sh` halts immediately without attempting rollback/deletion, emits the already-created names in `created` alongside the failed label in `refused`, and exits 1.",
                "outcome": "held",
                "finding_id": null
              },
              {
                "id": "as-5",
                "description": "Bash 3.2 `set -u` portability and profile flag forwarding: verify that `renderer_args=(labels)` is never empty under `nounset`, and optional `--foreman` and `--release-please` flags are forwarded cleanly to `label-registry-render.mjs`.",
                "outcome": "held",
                "finding_id": null
              },
              {
                "id": "as-6",
                "description": "Fleet approval gating and start-of-run question table synchronization: verify that `label_provisioning` defaults to `withheld` (never implicit approval), is validated against `$defs.label_provisioning` (`approved` | `withheld`), is reusable across repositories within the same release in `fleet-decisions.py`, and is documented consistently across `SKILL.md`, `references/fleet-decisions.md`, and `references/mode-update.md`.",
                "outcome": "surfaced-finding",
                "finding_id": "challenge-r1-codex-adversarial-1"
              }
            ]
          }
        }
      ],
      "adjudication": {
        "schema": 2,
        "run_id": "run-1531-additive-model-labels",
        "stage": "challenge",
        "round": 1,
        "reviewed_head": "9e0d2740f87c0ce9c39a3fecea57284deae75e5c",
        "adjudications": [
          {
            "finding_id": "challenge-r1-codex-adversarial-1",
            "reviewer_priority": "P2",
            "adjudicated_priority": "P2",
            "disposition": "defer",
            "classification": "confirmed",
            "override": null,
            "reason": "Confirmed: ai/skills/repo/standardize-repo/SKILL.md lines 69-79 list the 9 start-of-run questions without label_provisioning, while fleet-decisions.py, references/fleet-decisions.md, and references/mode-update.md include it. Deferred to integration because SKILL.md is owned by wave-1 lane-1670 (#1670) and outside lane-1531's fence; will be settled at integration by filing a follow-up or after #1670 merges.",
            "evidence": "Verified in ai/skills/repo/standardize-repo/SKILL.md:69-79 and plan.json revision 233 (SKILL.md is fenced to lane-1670, not lane-1531)."
          }
        ]
      }
    }
  7. evanharmon1 commented on Nov 3, 2026

    @evanharmon1
    OwnerAuthor

    Dev flow v2 evidence: review round 1 for run-1531-additive-model-labels at 9e0d274 (finder codex-verification).

    {
      "run_id": "run-1531-additive-model-labels",
      "stage": "review",
      "round": 1,
      "reviewed_head": "9e0d2740f87c0ce9c39a3fecea57284deae75e5c",
      "passes": [
        {
          "schema": 2,
          "role": "reviewer",
          "status": "completed",
          "head": "9e0d2740f87c0ce9c39a3fecea57284deae75e5c",
          "produced_at": "2026-11-03T14:10:26Z",
          "producer": {
            "harness": "codex-cli",
            "model": "gpt-6.1-sol",
            "tier": "frontier"
          },
          "run": {
            "run_id": "run-1531-additive-model-labels",
            "initiated_by": "human"
          },
          "payload": {
            "stage": "review",
            "round": 1,
            "reviewed_head": "9e0d2740f87c0ce9c39a3fecea57284deae75e5c",
            "finder": "codex-verification",
            "slot": "codex-verification",
            "findings": [],
            "counts": {
              "P0": 0,
              "P1": 0,
              "P2": 0,
              "P3": 0
            }
          }
        }
      ],
      "adjudication": {
        "schema": 2,
        "run_id": "run-1531-additive-model-labels",
        "stage": "review",
        "round": 1,
        "reviewed_head": "9e0d2740f87c0ce9c39a3fecea57284deae75e5c",
        "adjudications": []
      }
    }
  8. github-actions commented on Nov 3, 2026

    @github-actions

    Claim released — issue closed (completed). (Supersedes the claim record above.)

    Released by claim-release automation:

    • claim:gpt label: removed
    • assignee evanharmon1: removed
  9. evanharmon1 commented on Nov 3, 2026

    @evanharmon1
    OwnerAuthor

    Slate membership:

    • session: fleet-m6-foundations
    • slate: m6-fleet-foundations
    • action: remove
  10. evanharmon1 commented on Nov 3, 2026

    @evanharmon1
    OwnerAuthor

    Slate membership:

    • session: fleet-m6-agy
    • slate: m6-fleet-foundations
    • action: remove
  11. removed
    slatedIssue is in an orchestrator slate plan (membership only, not a state)
    on Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ai-generatedCreated or authored by an AI agentarea:standardize-repoThe standardize-repo skill and its standards catalogcomplexity:sComplexity: small and local; a clear approach across a few filesdomain:standardizationKeeping repos current with conventions: standardize-repo, drift audits, adoptionfeatureNew feature or requestimpact:mediumImpact: clear benefit to a meaningful share of users or goalslayer:noneInapplicable: the work is not in a single stack slicepriority-ai:p2Priority (AI): worth doing, not blockingrisk:mediumRisk: a mistake breaks a feature or needs a careful rollbacktier:standardModel tier: reliable general-purpose coding model first

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions