Skip to content

fix(copilot): expose only account-available models - #4698

Open
hgaol wants to merge 5 commits into
apache:mainfrom
hgaol:fix/github-copilot-available-models
Open

fix(copilot): expose only account-available models#4698
hgaol wants to merge 5 commits into
apache:mainfrom
hgaol:fix/github-copilot-available-models

Conversation

@hgaol

@hgaol hgaol commented Sep 4, 2026

Copy link
Copy Markdown
Member

Title

fix(copilot): expose only account-available models

Description

Summary

GitHub Copilot model discovery could expose models the connected account could not use.

This fixes both sources:

  1. Provider response filtering

    • Policy-free models remain valid.
    • A present policy must be exactly enabled.
    • disabled, unconfigured, unknown, and malformed policies are excluded.
    • Existing picker, tool-call, and supported-wire requirements remain unchanged.
  2. Catalog persistence

    • Copilot's filtered /models response is treated as the account's authoritative usable catalog.
    • The first account-scoped discovery replaces bootstrap fallback IDs with the available account models.
    • Later refreshes remove withdrawn IDs while preserving still-available user selections.
    • Newly introduced models remain opt-in after the initial discovery.
    • If an unavailable default must be replaced, reconciliation selects an available model.

The generic reconciliation behavior for other providers is unchanged: their model lists may be partial, so omitted user-selected IDs remain preserved.

Fixes #4697

Root cause

toGitHubCopilotModelInfo rejected only:

policy?.state === 'disabled'

That admitted unconfigured models even though Maka cannot accept their policy terms.

Separately, reconcileConnectionAfterModelFetch intentionally preserved every previously enabled model. Interactive Copilot connections start with fallback IDs, so models absent from the account response were added back into the projected catalog.

Verification

The regressions fail without this change and pass with it:

  • Provider-contract fixture with:
    • policy-free model;
    • enabled model;
    • disabled model;
    • unconfigured model;
    • hidden-picker model;
    • tool-incompatible model;
    • unsupported endpoint.
  • Core authoritative-reconciliation tests:
    • initial fallback replacement;
    • withdrawn model removal;
    • still-available user choice preservation;
    • newly introduced model remains opt-in.
  • Runtime Policy integration test proving unavailable bootstrap IDs are removed from persisted Copilot state.

Checks run:

  • Provider conformance matrix: 126 passed
  • @maka/core: 782 passed
  • @maka/storage: 1107 tests, 0 failures
  • @maka/runtime: 3176 tests, 0 failures
  • Full workspace build: passed
  • Full workspace typecheck: passed
  • Lint: passed
  • Format check: passed
  • ASF source-header check: passed

The focused regressions and full typecheck/lint/format were rerun after merging the latest main.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: pi with an OpenAI model assisted diagnosis, primary-source comparison against GitHub's Copilot client, implementation, test creation, and verification. The human contributor remains responsible for the change.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — GitHub Copilot model lists now omit models that are unavailable to the connected account.
  • No

GitHub's model policy has three historical states: enabled, disabled, and
unconfigured. Maka cannot accept model policy terms, so exclude every present
policy that is not enabled while continuing to admit policy-free current
responses.

Treat the filtered Copilot /models response as an authoritative account
catalog during persistence. The first fetch replaces bootstrap fallback IDs;
later refreshes remove withdrawn IDs without automatically opting the user
into newly introduced models.

Cover the mixed provider payload, authoritative reconciliation, and real
Runtime Policy model-fetch commit.
@github-actions github-actions Bot added the effort/M Under 500 readable lines label Sep 4, 2026
The required CLI packaging gate failed when npm's advisory bulk endpoint
returned HTTP 503 after holding the request for five minutes. The first retry
implementation capped every attempt at 90 seconds, which guaranteed hosted
runner timeouts before npm could return that response.

Run the audit before compilation, allow each request its observed five-minute
response window (bounded at 330 seconds), and make one retry after two seconds.
The roughly eleven-minute worst case fits the 45-minute required job without
spending the workspace-build budget first.

Vulnerabilities and deterministic config/auth failures still fail immediately;
408/429/5xx and transport errors retry once, then fail closed with the real
diagnostic. Release-contract tests pin classification and the two-attempt cap.
@hgaol
hgaol force-pushed the fix/github-copilot-available-models branch from 0583cb8 to ae320e1 Compare September 4, 2026 05:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/M Under 500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(copilot): model catalog includes unavailable and unconfigured models

1 participant