Skip to content

Improve Open Actions compatibility reviews and implementation guidance - #1627

Open
kelos-bot[bot] wants to merge 12 commits into
mainfrom
open-actions-config-update-latest
Open

kelos-bot[bot] wants to merge 12 commits into
mainfrom
open-actions-config-update-latest

Conversation

@kelos-bot

@kelos-bot kelos-bot Bot commented Aug 9, 2026 •

Copy link
Copy Markdown
Contributor

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

Keeps Open Actions agents aligned with full documented GitHub Actions compatibility and incorporates recurring implementation lessons from PR reviews. Shared implementation guidance lives in agentconfig.yaml; the two reviewer configurations carry the corresponding review checks. The final diff touches only those three files under self-development/open-actions/, and shared skills remain in self-development/base-agent.yaml.

  • Keep intentional compatibility regressions eligible for review. The implementation review of Open Actions #159 and its API review identified lost trigger reporting, Checks-tab visibility, GitHub rerun controls, and collapsed execution identities. A later #159 review stopped raising the remaining reporting/rerun gaps because they were intentional, documented, and tracked. The #163 API review repeated that reasoning for the narrower newest-execution reporting contract. The merged diffs retain the narrower reporting model. Code-review criteria now allow documented compatibility findings despite author intent or preference, and both reviewer prompts explicitly check trigger coverage, separate workflow/job reporting, Checks visibility, and GitHub UI/CLI reruns. Shared guidance clarifies that rejection, an alternative, or a tracking issue does not close a gap. Verified against GitHub's current workflow reporting model, Checks versus commit statuses, multiple-event behavior, and rerun contract. Existing compatibility text in the planner, triage, user, strategist, and configuration-update prompts is preserved.

  • Filter controller watch events to avoid redundant reconciles while preserving execution changes. The #159 review identified an unfiltered WorkflowRun watch that issued an uncached list and enqueued peers on every update. The #173 review found the same overhead on the reporter's WorkflowJob watch, including a self-trigger from its own reporting acknowledgement. The #173 resolution applied the event filter and added coverage for completion combined with an acknowledgement; the merged diff wires the predicate into the reporter, WorkflowRun, and Runner controllers. The existing shared performance rule now covers filtering by changes relevant to each controller and testing that mixed acknowledgement/execution updates still reconcile.

  • Test Console forms and polling through requests derived from the rendered page. The #172 review found that dispatch tests bypassed the rendered loaded-selection field, leaving the source-run form contract untested. The #176 review found that manually constructed POST requests used LF-only values and missed browser textarea CRLF conversion. The #179 review found a related page/handler mismatch: polling the newest 100 runs discarded durations for rows still displayed after new runs arrived. Both independent review paths agreed. These reviews expose the same testing gap: isolated handler requests can bypass state retained by the rendered page. The shared Console testing instruction requires rendered hidden-field values, browser form encoding, and persisted-result assertions; for polling, it also requires requests derived from the rendered page, new resources arriving before refresh, and updates for still-visible resources through completion. The merged Update license from BSL 1.1 to Apache 2.0 #179 head dff2257 selects the displayed run identities and adds TestConsoleWorkflowRunListPollsDisplayedRuns, confirming the finding and fix. The #172 response and #176 response, together with the merged tests, confirm the fixes. Checked GitHub's current dispatch input contract and variable limits; the browser encoding detail comes from the HTML form submission specification. GitHub's current workflow run documentation describes observing run progress and completion. The polling instruction addresses Open Actions' browser/handler contract; it does not prescribe GitHub's polling mechanism or cadence.

  • Verify that positive test fixtures can occur in production and pass the downstream eligibility checks. The #178 code review found that a push + TriggerInvalid fixture claimed to prove a reporting path that could never occur: the condition producer handles only workflow_dispatch, workflow_call, and schedule, while the reporter excludes those events. The merged diff removes that reporting branch and uses WorkflowInvalid for the no-Console-URL case, retaining meaningful positive coverage. This repeats the fixture-fidelity problem in the #172 review and #176 review, where hand-built requests bypassed rendered form state or browser encoding. One sentence extends the existing shared production-path testing instruction to require reachable inputs and states. GitHub's current workflow run documentation requires failed runs for invalid workflow files on new commits; a fabricated internal trigger-error state does not establish that observable behavior. This testing clarification adds no exception to the full-compatibility requirement.

  • Apply GitHub's workflow eligibility rules on every path that creates WorkflowRuns. The #182 code review and API review independently found (P2) that open-actions run create WORKFLOW validates only the path's shape (cmd/open-actions/run_create.go:144 at 4145506). As a result, any .yaml/.yml file in the repository that declares workflow_dispatch can be dispatched, while webhook and schedule discovery list only direct children of Project.spec.workflowDirectory. Both reviews note that the Console dispatch path already has the same gap, so two separately built creation paths have now missed the rule. The API review also found that only the CLI enforces the default-branch dispatch rule; the Console and direct creation do not. A new shared rule requires every WorkflowRun creation path, including the Console and CLI, to accept only direct children of the workflow directory and to require manually dispatched workflows to be on the default branch. It also requires tests showing that files outside the directory or in its subdirectories are rejected. GitHub's current docs require workflow files to be stored in .github/workflows, state that subdirectories of the workflows directory are not supported, and require the workflow to be on the default branch for workflow_dispatch. The unchecked path was confirmed in the CLI UX: Missing client-side validation for enum flags (--type, --credential-type) #182 diff (validateRunCreate), and the non-recursive listings were confirmed in internal/webhook/delivery.go (workflowFilesAtRevision, git ls-tree without -r) and internal/controller/schedule_controller.go (discoverScheduledWorkflows) on Open Actions main. CLI UX: Missing client-side validation for enum flags (--type, --credential-type) #182's current head 9208a6a (September 27) adds the direct-child check to run create, and TestRunCreateWorkflowDirectoryConformance rejects root, elsewhere-in-repository, nested, and similar-prefix paths, confirming the finding for the CLI. The Console dispatch handler on Open Actions main (61e073d, which includes the merged CLI UX: 'axon run' should provide next-step guidance after task creation #183) still checks only the path's shape and reads the workflow at the selected revision, so the rule still applies there.

  • Build Console forms opened from an existing run from the workflow revision the new run will use. The #183 review (P3) found that a form started from an existing run counted the source run's workflow snapshot as loaded. Optional inputs the source run omitted were submitted with the snapshot's defaults, and inputs added at the latest commit stayed hidden, while the new run executed at the ref's latest commit. The #187 review (P2, head e679d01) found the same mismatch in the new default captions: the form showed the snapshot's Default: values, which the run would not receive when the ref head declared different defaults. Both findings concern the same source-run prefill path, and both authors fixed them. CLI UX: 'axon run' should provide next-step guidance after task creation #183's merged head 281e010 submits only the inputs the source run supplied. Wait for terminating resources before server-side apply #187's merged head 07d5810 asks users to load the workflow before it shows defaults and adds TestConsoleDispatchLoadsCurrentDefaultsForSnapshotInputs, which covers defaults changed, added, or removed since the source run. The shared Console form instruction now requires source-run forms to present and submit inputs according to the revision that runs. They must submit only the inputs the source run supplied and show declarations or defaults only after reading them at that revision. Tests must cover snapshots whose inputs or defaults differ from the ref head. GitHub's current docs set GITHUB_SHA for workflow_dispatch to the last commit on the dispatched branch or tag, and they apply the defaults defined in the workflow file when inputs are omitted, as does the REST dispatch contract. A form that shows or submits an older snapshot's defaults therefore misstates the documented dispatch result.

The shared implementation guidance retains these review-backed lessons:

  • Treat GitHub commit-status contexts as external identities used by required checks. Reviews: #163, #163, #163, #164, #164.
  • Recover accepted external reports after local status persistence fails, for both initial reports and later transitions, and use only the latest report for the identity and expected creator. Reviews: #159, #159, #163, #163.
  • Treat persisted and versioned cross-component data as upgrade contracts, preserving in-flight and pre-upgrade data and positively testing current and fallback paths. Reviews: #45, #47, #47, #49, #55, #55.
  • Require workflow parsing to reject unsupported or ambiguous input instead of silently ignoring it or applying last-value-wins behavior. Reviews: #1, #1, #1.
  • Keep durable repository documentation aligned with observable behavior, supported syntax, enforced limits, and operational prerequisites. Reviews: #3, #3, #13, #54, #54, #62.
  • Preserve retryable dependency failures consistently across reconciliation phases and test their observable retry state. Reviews: #64, #64.
  • Keep request, reconcile, and retry hot paths bounded by capping or paginating list-backed responses, placing no-op checks before expensive reads, and persisting completed sub-work. Reviews: #52, #47, #70.
  • Exercise integration-dependent behavior through the installed Kind path and test security boundaries through their real enforcement layer. Reviews: #53, #55, #73, #74.
  • Bound repository-controlled parsing and expression evaluation at every stage: source size, AST depth and node count, incremental evaluation output, and post-interpolation field and aggregate budgets. Reviews: #45, #45.
  • Follow GitHub Actions' documented expression coercion and error semantics. Reviews: #74.
  • Preserve GitHub API file-type semantics and validate network-sensitive Git behavior with realistic remotes. Reviews: #77.
  • Retain positive authorization coverage for every supported credential path and keep deployed credential wiring covered end to end. Reviews: #78.
  • Version every controller-to-runner interface, including job plans, CLI arguments, environment variables, mounted files, and credentials, and test both skew directions. Reviews: #79, #79, #93.
  • Bound combinatorial traversal independently of accepted output so filters cannot hide unbounded work. Reviews: #95, #45.
  • Keep PR descriptions and release notes as accurate as repository documentation, including changed defaults, upgrade requirements, and access boundaries. Reviews: #79, #94, #94.
  • Preserve positive coverage for each distinct production path when repurposing tests or fixtures. Reviews: #78, #91.
  • Make create-and-follow-up workflows retry-safe across stale caches, concurrent reconciles, and partial failures. Reviews: #108, #93, #92.
  • Compare omitted and operator-configured behavior across every resource-creation path before adding optional API or configuration fields. Reviews: #105, #105, #92, #96, #96. For optional Helm values, the shared rule also requires absence to remain valid in the values schema and templates, with schema and rendering coverage using base-release values that lack the key. The #174 API review and code review independently found that requiring the added Console boolean broke helm upgrade --reuse-values; the tests loaded only updated defaults. This repeats the omitted-setting upgrade failure class identified in Introduce user CRD #105. Helm’s implementation replaces incoming chart defaults with the previous release’s coalesced values on that path. The author’s response and merged diff confirm optional schema validation, absence-as-disabled rendering, and missing-key coverage.
  • Preserve scalar meaning and matrix identity across workflow YAML, expression results, webhook JSON, and persisted intermediate data. The #95 review found numeric values changing representation after deferred-plan serialization; the #169 review finds the same class of mismatch between expression-produced float64 values and reloaded json.Number values, rejecting selective reruns even though YAML-literal tests pass. The shared rule requires consistent, value-preserving identity normalization and separate coverage of YAML literals and fromJSON numeric matrices through planning, persistence/restart, and selective reruns, including large integers and small fractions. GitHub documents numeric expressions and fromJSON, matrices from job outputs, and selective reruns; the internal representation mismatch is therefore a compatibility gap. Related earlier review: #93.
  • Define concurrency order independently of reconcile arrival and intermediate status presence, and test adversarial arrival sequences. Reviews: #102, #104.
  • Treat condition-reason tables and supported expression/result lists as exhaustive documentation contracts. Reviews: #104, #107, #100.
  • Handle permanently missing referenced state according to its lifecycle. Reviews: #89, #94, #95.
  • Treat the current GitHub Actions documentation as the normative workflow specification and classify every missing or different documented behavior as a compatibility gap. Reviews: #103, #121.
  • Preserve independent workflow candidates represented by one webhook. Reviews: #121.
  • Classify GitHub rate-limit responses precisely and preserve unrelated aggregated failures. Reviews: #122.
  • Preserve the causal meaning of terminal and non-running job states instead of treating status presence or a coarse terminal, cancelled, or skipped predicate as proof of success, reuse, or lifecycle progress. Reviews: #127, #100, #102, #104, #141.
  • Exercise every independent validation constraint through the real enforcement layer, including numeric bounds and Kubernetes structural list or map semantics. Reviews: #130, #74.
  • Exercise shared behavior through every materially distinct production caller and cover path-specific lifecycle and backing-store assumptions. Reviews: #133, #91, #78.
  • Test every ordered source or fallback with distinguishable fixtures and assert which source wins, rather than only checking a final value that another branch can produce. Reviews: #135, #136.
  • Do not let conditionally skipped or optional-tool-dependent tests serve as the only coverage of deterministic logic. Reviews: #140, #101.
  • Populate phase-sensitive expression contexts from actual execution state at every supported evaluation site instead of substituting optimistic defaults. Reviews: #144.
  • Treat runnable examples as maintained contracts that remain executable and continue demonstrating behavior outside the provided baseline. Reviews: #124, #143.
  • Match Open Actions' actual structured-logging convention without inventing an initial-capital requirement. Reviews: #150, #150.
  • Test exact cross-component textual contracts at both the producer and consumer. Reviews: #149, #140.
  • Require feature-specific terminal reasons to be proven by authoritative lifecycle state or field-specific API details. Reviews: #156, #157.
  • Scope bearer credentials to the permissions and actual enforced lifetime of the receiving path. Reviews: #156, #79.
  • Preserve existing GitHub Actions-compatible paths when replacing a mechanism instead of trading one behavior for another or treating documentation and tracking as permission to regress. Reviews: #159, #159.

Which issue(s) this PR is related to:

N/A

Special notes for your reviewer:

Validation passed: make verify (generated artifacts, formatting, module metadata, YAML, shell formatting, Console frontend build, and go vet), go test ./internal/examples/... (self-development manifest checks), git diff --check, a file-scope check, and a PR-template check.

The final branch changes only three configuration files under self-development/open-actions/: agentconfig.yaml, open-actions-reviewer.yaml, and open-actions-api-reviewer.yaml. The branch is rebased onto main, which removed the Claude reviewer spawners. That removal left their earlier copies of the review checks in this PR with nothing to modify, and the surviving reviewers carry the checks unchanged. The workflow-eligibility guidance is a new bullet next to the existing creation-path rule in agentconfig.yaml, and the source-run form guidance extends the existing Console form bullet; shared skills remain in the base configuration.

Reviewed both requested recent-PR lists and collected diffs, formal reviews, inline comments, and conversations for the five PRs active September 12–19, 2026: #174 and #176–#179. None currently carries generated-by-kelos. Substantive reviews are sticky issue comments by kelos-bot; the formal-review and inline-comment endpoints returned no entries. #177 has no review comments. Rechecked #172 as supporting evidence for the recurring Console page/handler testing gap. The #179 review describes revision 7e057a8; the merged head dff2257 fixes the polling selection and adds coverage for new runs arriving and displayed runs completing. The #178 review describes revision ee6c545; the merged head 8044600 removes the unreachable reporting branch and corrects its documentation and test fixture.

Also reviewed activity from September 19–24, 2026: open PRs #181 and #182 and issue #180. #181 and #180 have no reviews or comments. #182's substantive reviews are kelos-bot sticky issue comments at 4145506; the formal-review and inline-comment endpoints returned no entries. Neither PR carries generated-by-kelos. The comments and timelines of earlier PRs have no new activity since the last update.

Existing guidance already covers #171's documentation and installed precedence coverage, #172/#176's Console form testing, #173's watch filtering and cancellation/deletion documentation, and #174's omitted Helm setting. The namespace-scoping and invalid-ConfigMap filtering suggestions in #176 were declined by the author, and flag deprecation feedback conflicts with the stated maintainer direction; none motivates a new rule. #177's screenshot instruction has no review evidence, so it is not duplicated here. #178's naming, required-check warning, and exact invalid-file activity scope are isolated suggestions or fit existing guidance; no separate rules are added for them. #182's other findings fit existing guidance:

  • CLI-created runs ignore the operator's --workflow-run-ttl-seconds-after-finished default. The existing creation-path rule already requires comparing omitted and operator-configured behavior.
  • The docs describe a GitHub write-access check that only the CLI enforces. Existing guidance already forbids claiming security properties a path does not enforce.
  • Retries with a fixed --name are not idempotent. The existing retry-safe creation and documentation-alignment rules cover this.
  • A --name help assertion also matches --namespace. The code reviewer's existing vacuous-substring check caught it.

The #174 code review excludes a documented authorization difference because it is intentional. Existing guidance already keeps such gaps eligible for review: GitHub requires repository write access for manual dispatch and reruns. Documenting an opt-in exception or tracking it does not establish compatibility, so no exception is added to this configuration.

Also reviewed activity from September 24–27, 2026: merged PR #183, the updated head of #182, and issues #180 and #168. Neither PR carries generated-by-kelos, and the formal-review and inline-comment endpoints returned no entries. #182 received no new reviews. Its new head 9208a6a also resolves the other September 24 findings that existing guidance covers: named retries now reuse a matching run, the docs name Kubernetes RBAC as the authorization boundary, and the docs state that the operator TTL default does not apply to CLI-created runs. The #183 review raised two P3 findings, and the merged head 281e010 addresses both:

  • Starting from an existing run submitted the old snapshot's defaults for optional inputs that run had omitted, while the new run used the latest commit. The merged head includes only the inputs the source run supplied, plus required inputs. Wait for terminating resources before server-side apply #187 repeated this finding, and together they motivate the source-run form guidance above.
  • A resubmission resolves the ref and revalidates the workflow before the existing-run check, so a deleted ref or a changed workflow returns an error instead of the documented redirect. The existing hot-path rule already puts idempotency checks before external reads. The author narrowed the documentation instead, one of the two fixes the review suggested. This finding does not motivate a new rule.

Issue #168's update refreshes a strategist assessment, and #180 contains only a /kelos pick-up command. Neither contains review feedback.

Also reviewed activity from September 27–29, 2026: merged PR #186, open PRs #184, #185, and #187, and issue #168. None of the PRs carries generated-by-kelos. Substantive reviews are kelos-bot sticky issue comments; the formal-review and inline-comment endpoints returned no entries. #184 has no reviews or comments, #181 and #182 have no new activity, and #168's update refreshes a strategist assessment without review feedback. Apart from the #187 finding above, no finding motivates a new rule:

  • The #186 review raised three P3 findings, and the merged head fd51beb addresses all three. The individual-job rerun path returned Kubernetes API failures as 409 Conflict with raw error text, unlike the sibling failed-jobs path. The existing rule to classify dependency and API failures consistently covers this, and the merged head sends these failures through the Console's logged generic error response. The chart README's outdated description of anonymous rerun scope falls under the existing documentation-alignment rule. The untested lineage-break guard falls under the existing production-path testing rules, and the merged head adds a mismatched previous-run UID case.
  • The #185 review approved with one optional P3 suggestion: live step counters use the viewer's clock, while job counters use a server-computed reference. No earlier review raised this, so it is a one-off and adds no rule.

Also reviewed activity from September 29–30, 2026. #185 and #187 merged, and there are no new reviews, review comments, or formal-review or inline-comment entries. #187 merged at 07d5810, the post-review head cited above, which confirms the source-run form fix. #185 merged at c91e007, rebased onto #186 after the review of 02055d9; its live step counters still use the viewer's clock. No review of #167 or #179 raised client-clock skew, so this remains a one-off. #181, #182, and #184 have no new activity. Issues #76 and #168 received only title and body refreshes from the fake-user and strategist agents, with no review feedback. The configuration files are unchanged in this pass.

Does this PR introduce a user-facing change?

NONE

🤖 Generated with Claude Code

@gjkim42

gjkim42 commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

/kelos squash-commits

@kelos-bot

kelos-bot Bot commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor Author

🤖 Kelos Task Status

Task kelos-squash-commits-issue-comment-0a8b9b2677e8 has succeeded. ✅

@kelos-bot
kelos-bot Bot force-pushed the open-actions-config-update-latest branch from e4bd217 to a1b6016 Compare September 4, 2026 00:05
@kelos-bot

kelos-bot Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

Squash complete.\n\nRebased onto origin/main and squashed to a single commit.

@kelos-bot
kelos-bot Bot force-pushed the open-actions-config-update-latest branch from a1b6016 to 46e1db3 Compare September 4, 2026 18:12
@github-actions github-actions Bot added release-note-none and removed needs-release-note Indicates a PR lacks a release-note block labels Sep 4, 2026
@kelos-bot kelos-bot Bot changed the title Improve Open Actions agent implementation guidance Improve Open Actions compatibility reviews and implementation guidance Sep 7, 2026
gjkim42 and others added 2 commits September 24, 2026 18:04
…aths

Open Actions PR reviews found that a new CLI dispatch path accepted any
YAML file in the repository, while webhook and schedule discovery accept
only direct children of the Project's workflow directory. The Console
dispatch path has the same gap, and the default-branch dispatch rule is
enforced by only some creation paths. GitHub loads workflows only from the
workflows directory and requires manually dispatched workflows to be on
the default branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Open Actions reviews of #183 and #187 found the same defect in the Console
Run workflow form opened from an existing run: the form presented or
submitted the source run's workflow snapshot while the new run executed at
the latest commit on the ref. Extend the shared Console form guidance to
require input handling and default captions from the revision that runs,
with tests where the snapshot and ref head differ.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant