Skip to content

Add agentic PR reviewer (gh-aw): requirement-fidelity + CARE SQL review - #148

Merged
amjithtitus09 merged 1 commit into
mainfrom
amjithtitus09-analytics-review-bot
Aug 17, 2026
Merged

Add agentic PR reviewer (gh-aw): requirement-fidelity + CARE SQL review#148
amjithtitus09 merged 1 commit into
mainfrom
amjithtitus09-analytics-review-bot

Conversation

@amjithtitus09

Copy link
Copy Markdown
Member

What this does

Adds a GitHub Agentic Workflows (gh-aw) reviewer that automatically reviews every query PR in this repo, modeled on the battle-tested reviewer in ohcnetwork/care_fe. It posts ≤8 prioritized inline comments plus one consolidated review, tracks its own findings across pushes (re-reviews only the delta, resolves fixed threads), and answers when humans reply or @-mention it.

Branch convention note: this repo's convention is branch name = JIRA ticket ID. This PR is meta-work on the repo's own tooling — no JIRA ticket exists for it, so the convention deliberately doesn't apply here.

The four lenses

  1. Requirement fidelity (the differentiator) — a pre-step extracts ENG-\d+ from the PR branch name, fetches the JIRA ticket (summary, description, comments) and renders it to markdown. The agent reconstructs what was asked — metric, grain, filters, facility scope — and verifies the SQL delivers exactly that. No ticket in the branch → one finding about the missing linkage, review continues.
  2. SQL correctness — defers to the care-sql-code-review skill (checked out at a pinned SHA): soft-delete/entered_in_error traps, facility scoping, join fan-out, sargability, magic IDs. Uncertain schema claims are verified against ohcnetwork/care models via the API, not guessed. A deterministic sqlglot lint pre-step feeds parse/template errors in as input.
  3. Documentation — TEMPLATE.md sections accurate (not just present), magic IDs explained in Notes, Parameters table matches the {{variables}} actually in the SQL, Metabase optional-filter syntax kept commented per repo convention.
  4. Repo hygiene — correct domain folder (knows the AccoutingAccounting consolidation from Move accounting docs from Care/Accouting into Care/Accounting #144), snake_case + deployment suffix (_ssmm/_pallium/_kc), one query per PR.

Proportionality is explicit in the agent file: a fine PR gets a clean pass, no manufactured findings.

Security model

  • pull_request_target with base-only checkout — the workflow only ever checks out trusted main; PR content is read via the API at head SHA and never placed on disk. This is the same mitigation care_fe uses.
  • JIRA secrets isolatedJIRA_* secrets appear only in the ticket-fetch pre-step's env, never in the agent's env. The agent job runs with read-only permissions; all writes go through gh-aw safe-outputs (capped: 8 inline comments, 1 review, 8 replies, 8 resolves).
  • Untrusted input handling — PR content, diffs, and the JIRA ticket text itself are treated as data, not instructions; injection attempts get reported as findings.
  • Event-derived values (branch names etc.) are passed via env vars, never ${{ }}-interpolated into scripts.

Security review note (compiler-mandated)

gh aw compile gates on new secrets; these were reviewed and approved:

  • New secrets referenced: JIRA_API_TOKEN, JIRA_BASE_URL, JIRA_EMAIL — read-only JIRA REST fetch of the ticket named by the branch, confined to one pre-step's env, registered with gh-aw's log-redaction step. Missing/wrong secrets degrade to a NO TICKET FOUND marker; the job never fails and the agent never sees the credentials.
  • actions/checkout bumped to v7.0.1 (pinned by full SHA in the lock file); ohcnetwork/skills checked out at pinned SHA 30f437fb55e9216a964476bfb7fcf46992051a2f.

Admin prerequisites (before this works end-to-end)

Set three repo secrets (values not invented here — admin supplies them):

  1. JIRA_BASE_URL — e.g. the org's https://<org>.atlassian.net
  2. JIRA_EMAIL — service account email for the API token
  3. JIRA_API_TOKEN — Atlassian API token for that account

Plus the org-level COPILOT_GITHUB_TOKEN engine credential — already in use by care_fe's reviewer, assumed available to this repo.

Without the JIRA secrets the bot still reviews (lenses 2–4); lens 1 degrades to a "ticket context unavailable" note.

Verification done here

  • gh aw compile (v0.85.4): 0 errors, 0 warnings; .lock.yml + actions-lock.json committed.
  • Lint script run against real repo files — correctly flags the stray trailing fences in bed_occupancy_ssmm.md / labtest_revenue_ssmm.md, clean parse on gendersplit_pallium.md.
  • JIRA extraction dry-run: all fallback paths (ENG-909, embedded lowercase eng-1234, no ticket, no PR context, no secrets, connection failure) write the NO TICKET FOUND marker and exit 0.
  • ADF renderer tested against a realistic JIRA v3 payload (bold, bullets, mentions, comments).

No workflow runs were triggered. Post-merge dry-run plan: workflow_dispatch with the pr_number input against #146 and #147 to validate the full loop on live PRs before relying on the PR triggers.

Post-v0 roadmap

  • JIRA comment-back: post the review verdict to the ticket.
  • Schema snapshot regeneration: keep the skill's care-schema.md in sync with ohcnetwork/care migrations.
  • Seeded-Postgres validation: actually execute changed queries against a seeded schema instead of parse-only linting.

Agentic workflow that reviews every query PR through four lenses:
JIRA requirement fidelity (branch name = ticket ID), CARE SQL
correctness (defers to the ohcnetwork/skills care-sql-code-review
skill), TEMPLATE.md conformance, and repo hygiene.

- .github/workflows/analytics-review.md: gh-aw source. pull_request_target
  with base-only checkout (never PR head), JIRA secrets confined to the
  fetch pre-step, sqlglot lint pre-step, pinned skills checkout,
  safe-outputs as the only write channel.
- .github/agents/analytics-review.agent.md: the four review lenses with
  proportionality guidance.
- .github/scripts/lint_queries.py: deterministic sqlglot parse +
  template-structure lint of changed query docs (never fails the job).
- .github/scripts/render_jira_ticket.py: ADF JSON -> markdown renderer.
- .github/workflows/analytics-review.lock.yml + .github/aw/actions-lock.json:
  compiled with gh-aw v0.85.4, zero warnings.
- README.md: PR Review Bot section; fix stale Accouting/ typo in the
  structure diagram (folder was consolidated in #144).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant