Fix analytics-review on workflow_dispatch: pass resolved PR context to the agent - #152
Merged
Merged
Conversation
…o the agent Dry-run 32015295640 (PR #146) went green end-to-end after the v0.86.2 recompile, but the agent posted nothing: the resolve pre-step's AW_PR_NUMBER/AW_HEAD_REF land in step env only, which never reaches the agent sandbox, and on workflow_dispatch the event payload has no PR object. The agent concluded "no associated pull request ... nothing to review" and noop'd. Fix 1: the resolve step (now id: resolve_pr) also writes step outputs and /tmp/gh-aw/context/run-context.md — the context dir is the one channel proven to reach the agent (it read jira-ticket.md in the same run). The prompt body names run-context.md as the source of truth for PR identity, treats a dispatch run with a resolved PR as a normal review, and reserves noop for runs where no PR resolves at all. (${{ steps.* }} interpolation into the body cannot work: the prompt is rendered in the activation job, the steps run in the agent job.) Fix 2: the same run showed "JIRA returned HTTP 404 for ENG-909" — ambiguous, since JIRA answers 404 for missing ticket, missing project permission, AND bad credentials. On any non-200 the step now probes /rest/api/3/myself and writes an auth diagnosis into the marker (credentials rejected vs authenticated-but-cannot-see vs unreachable), reporting HTTP codes only, never credential values. Still never fails the job. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…e + auto-recovery) Root cause of the ENG-909 404: JIRA_API_TOKEN is a scoped Atlassian API token, and scoped tokens must target https://api.atlassian.com/ex/jira/<cloudId> instead of https://<site>.atlassian.net. Atlassian answers 404 (not 403) when a scoped token hits the site-host form, which mimics a missing issue. - Replace the /rest/api/3/myself probe (not callable with granular read:issue scopes -> would false-report valid credentials as broken) with GET https://api.atlassian.com/oauth/token/accessible-resources, which works for both token types and returns each site's cloudId. - Auto-recovery: when the issue fetch fails and the probe can resolve a cloudId (configured URL matches a site, cloudId embedded in an ex/jira URL, or single-site token), retry via the scoped-token endpoint - so either token type works with either JIRA_BASE_URL form. - Self-diagnosing markers on hard failure: credentials rejected (401/403, mentions <=365-day scoped-token expiry), base URL matches none of the token's sites (lists reachable site URLs + cloudIds and the exact value to set), ticket not visible/nonexistent (tried both endpoints), or api.atlassian.com unreachable. HTTP codes, site URLs and cloudIds only; never credential values. - Comments fetch now uses the possibly-recovered base URL. - README: JIRA_BASE_URL form guidance per token type. - New helper .github/scripts/jira_sites.py (cloud-id resolution + site summary; always exits 0). All six failure/recovery branches exercised locally against a mock Atlassian server using the step script extracted verbatim from the workflow; every path exits 0 and writes the marker. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
amjithtitus09
added a commit
that referenced
this pull request
Aug 17, 2026
…o inconclusive arms Per review of #152: a failed accessible-resources probe is not proof the credentials are bad (the endpoint may not accept basic auth for every token type), so the 401/403 arm no longer asserts "credentials rejected" - it now enumerates likely causes. Every inconclusive arm (401/403, unreachable, unexpected code) additionally surfaces the scoped/unscoped two-URL rule whenever the configured base URL host is not api.atlassian.com, and enumerates the three real possibilities: wrong base URL for the token type, no access to the project, or nonexistent ticket. Project name derived from the ticket key for accuracy. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
amjithtitus09
added a commit
that referenced
this pull request
Aug 17, 2026
…nt_info
Item 1 (highest severity): GitHub Actions invokes run: steps as
`bash -e {0}`, so errexit is inherited — `set -uo pipefail` does NOT
clear it, only an explicit `set +e` does. The KEY extraction pipeline
(grep exits 1 on branches without ENG-nnn) therefore killed the
"never fails" JIRA step before its no_ticket fallback could run,
hard-failing the agent job for every non-ENG-branch PR (observed in
run 32017183079 on PR #152). Fix: explicit `set +e` in the JIRA and
lint steps with a comment citing the run, `|| true` on the grep
pipeline as belt-and-braces, and an audit of all pre-steps (the
resolve and skill-move steps stay deliberately strict). Reproduced
locally under `bash -e`: main's version exits 1, this version exits 0
and writes the marker.
Item 2: the two-URL rule is now empirically confirmed (scoped API
tokens are ignored entirely on *.atlassian.net — anonymous and
authenticated requests return identical status codes). Replace the
accessible-resources probe (OAuth-Bearer-only; returns 401 uniformly
for no-auth/bogus-basic/bogus-bearer, so its 401 says nothing about
credentials) with the public, unauthenticated
GET <site>/_edge/tenant_info for cloudId discovery, and auto-retry
via https://api.atlassian.com/ex/jira/<cloudId>. Remove the now
unused .github/scripts/jira_sites.py.
Item 3: carry forward the hedged diagnostics from orphaned commit
c75b882 (pushed after the #152 squash-merge), now decisive where the
evidence allows: when the configured host is *.atlassian.net, the
marker recommends the ex/jira form outright; arms stay hedged only
where JIRA's 404-for-unviewable behaviour makes causes genuinely
indistinguishable.
README: add the _edge/tenant_info cloudId discovery command.
Recompiled with gh-aw v0.86.2 (0 errors, 0 warnings).
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
amjithtitus09
added a commit
that referenced
this pull request
Aug 17, 2026
…nt_info (#153) Item 1 (highest severity): GitHub Actions invokes run: steps as `bash -e {0}`, so errexit is inherited — `set -uo pipefail` does NOT clear it, only an explicit `set +e` does. The KEY extraction pipeline (grep exits 1 on branches without ENG-nnn) therefore killed the "never fails" JIRA step before its no_ticket fallback could run, hard-failing the agent job for every non-ENG-branch PR (observed in run 32017183079 on PR #152). Fix: explicit `set +e` in the JIRA and lint steps with a comment citing the run, `|| true` on the grep pipeline as belt-and-braces, and an audit of all pre-steps (the resolve and skill-move steps stay deliberately strict). Reproduced locally under `bash -e`: main's version exits 1, this version exits 0 and writes the marker. Item 2: the two-URL rule is now empirically confirmed (scoped API tokens are ignored entirely on *.atlassian.net — anonymous and authenticated requests return identical status codes). Replace the accessible-resources probe (OAuth-Bearer-only; returns 401 uniformly for no-auth/bogus-basic/bogus-bearer, so its 401 says nothing about credentials) with the public, unauthenticated GET <site>/_edge/tenant_info for cloudId discovery, and auto-retry via https://api.atlassian.com/ex/jira/<cloudId>. Remove the now unused .github/scripts/jira_sites.py. Item 3: carry forward the hedged diagnostics from orphaned commit c75b882 (pushed after the #152 squash-merge), now decisive where the evidence allows: when the configured host is *.atlassian.net, the marker recommends the ex/jira form outright; arms stay hedged only where JIRA's 404-for-unviewable behaviour makes causes genuinely indistinguishable. README: add the _edge/tenant_info cloudId discovery command. Recompiled with gh-aw v0.86.2 (0 errors, 0 warnings). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #148/#151. Dry-run 32015295640 (workflow_dispatch, PR #146) went all-green — the v0.86.2 recompile fixed the ENOENT — but no review was posted. Two bugs, both fixed here.
Bug 1 (blocking): resolved PR context never reached the agent
The "Resolve PR context" pre-step worked (
AW_PR_NUMBER: 146,AW_HEAD_REF: ENG-909in every pre-step's env), but those are per-stepenv:vars — they don't exist inside the agent sandbox, and onworkflow_dispatchthe event payload has no PR either. The agent's own words from the run log:…and it called
noop. So the manual/backfill dispatch path could never work.Fix — use the proven file channel (the agent already reads
/tmp/gh-aw/context/):/tmp/gh-aw/context/run-context.md(event name, PR number, head ref, head SHA — or an explicitNONE RESOLVEDmarker), and exposes proper stepoutputs(id: resolve_pr+$GITHUB_OUTPUT) for any future step-level consumers.run-context.md, never from the event payload, treats a dispatch run with a resolved PR as a normal first-review/re-review, and reservesnoopfor runs whererun-context.mdgenuinely says no PR.(Interpolating
${{ steps.resolve_pr.outputs.* }}into the prompt was not possible: gh-aw renders the prompt in the activation job, while custom steps run later in the agent job — the file channel is the correct mechanism.)Bug 2: JIRA 404 for ENG-909 — scoped-token root cause + self-diagnosing markers + auto-recovery
Root cause identified:
JIRA_API_TOKENis an Atlassian API token with scopes (read:issue-details:jira,read:issue-field-values:jira,read:comment:jira). Per Atlassian's docs, scoped tokens must target a different base URL than unscoped ones:JIRA_BASE_URLformhttps://<site>.atlassian.nethttps://api.atlassian.com/ex/jira/<cloudId>A scoped token sent to the
*.atlassian.netform gets HTTP 404 — the same status JIRA uses for missing/unviewable issues — which fully explains the dry-run'sHTTP 404 for ENG-909with valid credentials.Fixes in the JIRA pre-step:
GET https://api.atlassian.com/oauth/token/accessible-resources(fixed absolute URL, independent ofJIRA_BASE_URL; returns each reachable site'surl+id=cloudId). If a cloudId can be resolved (configured URL matches a site, cloudId embedded in anex/jiraURL, or single-site token), retry viahttps://api.atlassian.com/ex/jira/<cloudId>— so either token type now works with eitherJIRA_BASE_URLform. Subsequent comment fetches use the recovered base.jira-ticket.md. All arms are worded as likely causes, never verdicts (a failed probe is not proof the credentials are bad — the endpoint may not accept basic auth for every token type), and report only HTTP codes, site URLs, cloudIds — never secret values:JIRA_EMAILdoesn't match the token's account, or the token expired (scoped tokens expire ≤365 days).https://api.atlassian.com/ex/jira/<cloudId>, no trailing slash, no/restsuffix).api.atlassian.com(host shape only — the URL value is never printed)./rest/api/3/myselfprobe is removed: it isn't callable with these granular scopes and would have false-reported valid credentials as broken..github/scripts/jira_sites.py(cloudId resolution + site summary; always exits 0). README updated with the base-URL form guidance.Admin action (recommended)
Set
JIRA_BASE_URLtohttps://api.atlassian.com/ex/jira/<cloudId>for the scoped token. The auto-recovery makes the current value work anyway (single-site tokens recover unconditionally), and if anything is still off, the marker in the run log / review context now says exactly what to change.Verification
All pre-steps were already green in the dry-run; the agent demonstrably reads
/tmp/gh-aw/context/(itcat'djira-ticket.mdandlint-report.mdin its tool log) — only the PR identity was missing, whichrun-context.mdnow supplies. The new JIRA logic was exercised locally against a mock Atlassian server using the step script extracted verbatim from the workflow: recovery path renders the full ticket; all failure branches write their marker and exit 0.After merge:
Expected: the agent reads
run-context.md, identifies PR #146, and posts a review;jira-ticket.mdcontains either the rendered ENG-909 ticket (via auto-recovery) or a precise likely-cause diagnosis instead of a bare 404.Compiled with gh-aw v0.86.2,
0 warnings.