Skip to content

copilot: VS Code models from a Model Provider Service or model location, Claude on Anthropic Messages - #1358

Merged
xi-db merged 7 commits into
mainfrom
xi-db/stack/vscode-mps
Oct 30, 2026
Merged

xi-db merged 7 commits into
mainfrom
xi-db/stack/vscode-mps

Conversation

@xi-db

@xi-db xi-db commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

🥞 Stacked PR

Use this link to review incremental changes.


Summary

ug configure --agents vscode can now give Copilot Chat in VS Code the models of a Model Provider Service or of a Unity Catalog model location, not only the workspace's system.ai models. Every Claude model now goes through the Anthropic Messages API.

Choosing the models

  • --provider <catalog>.<schema>.<name> uses the models of a Model Provider Service, chosen the way ug copilot --provider chooses them:
    • the same provider types are accepted;
    • relayed (subscription) services and Bedrock services with no Claude target are refused;
    • for a service that allows any model, the models come from the gateway's listing.
  • --model-location <catalog>.<schema> uses the model services in that schema, as ug copilot --model-location lists them.
  • Each configure uses one source. The two flags can't be combined, and they work only with --agents vscode. Running configure without either flag goes back to system.ai.
  • The choice isn't saved in ug's state: it lives in the installed extension's ug-config.json, and nothing else reads it.
  • Claude models from a Model Provider Service or a model location get a 200K window, as in Copilot CLI, because such a service may not serve the 1M window.

Routing

  • With a Model Provider Service, every request carries the Databricks-Model-Provider-Service header.
  • That service's OpenAI models use /ai-gateway/openai/v1. The gateway doesn't serve Model Provider Services on /ai-gateway/mlflow/v1.

Anthropic Messages for Claude

  • Every Claude model, including system.ai ones, now calls /ai-gateway/anthropic/v1/messages, the route Claude Code and Copilot CLI use.
  • Each request marks two prompt-cache points: after the system prompt and after the last message.
  • Images in tool results are sent inside the tool_result.
  • The content-block workaround that Chat Completions needs for thinking models doesn't apply here.
  • A reply counts as finished by the same rules as the other two formats:
    • only end_turn, tool_use and stop_sequence finish normally;
    • max_tokens reports the output limit, and refusal reads like a content-filter stop;
    • tool calls are held back until message_stop, so a cut-off stream runs no tool.

Changes to the config and shared code

  • ug-config.json gains a top-level provider and a per-model family. Without family, the extension would derive the model's name from the text after the last dot, which turns gpt-5.5 into 5.
  • provider_service_catalog takes a tool argument, and VS Code is added to the provider types the gateway listing accepts.
  • The 200K cap for Claude moves into catalog_model_limits, which Copilot CLI and VS Code share.

Managed config for VS Code is a follow-up.

Testing

  • Unit tests: 31 ug VS Code tests, 37 extension tests, plus the full ug suite.
    • Model Provider Service targets and allow-all listings; Bedrock Claude names and limits.
    • Services VS Code can't use, the model-location listing, and the flag checks.
    • The Anthropic Messages request: system prompt, images, tool results, merging consecutive messages of the same role, cache points.
    • The Anthropic Messages stream: text, dropped thinking, tool-call fragments, every stop reason, error events, cut-off streams.
    • Routing and headers with and without a provider.
  • Copilot Chat 0.71 in VS Code, on a staging workspace, in Agent mode:
    • Claude Haiku 5.5 on Anthropic Messages created a file.
    • Claude Haiku 5.5 viewed an image through copilot_viewImage and wrote its colour ("red").
    • GPT 6 Luna on Responses created a file.
  • Direct requests to the gateway's Anthropic route with a system.ai model:
    • streamed events and stop reasons (end_turn, tool_use, max_tokens);
    • a max_tokens of 128K was accepted;
    • two identical requests: the second read 8,053 cached tokens;
    • an image inside a tool_result;
    • two user messages in a row.
  • Not yet tested end to end through a working Model Provider Service. The routes were checked by probing the gateway:
    • it rejects mlflow/v1 for Model Provider Services;
    • it serves openai/v1 for OpenAI-type services;
    • it serves anthropic/v1/messages for Anthropic and Bedrock services.

This pull request and its description were written by Isaac.

… and Claude on Anthropic Messages

ug configure --agents vscode takes --provider <catalog.schema.name> or
--model-location <catalog.schema>, picking the models the way ug copilot does.
The extension sends a provider's header, uses the gateway's OpenAI route for
its OpenAI models, and calls every Claude model through Anthropic Messages with
prompt-cache points.

Co-authored-by: Isaac <no-reply@databricks.com>
@xi-db xi-db added the ug-review Run the automated UG review label Oct 29, 2026
Comment thread src/ucode/agents/vscode.py Resolved
Comment thread src/ucode/cli.py Resolved
Comment thread vscode-extension/src/extension.ts Resolved
Comment thread vscode-extension/src/extension.ts Resolved
Comment thread vscode-extension/src/extension.ts Resolved
Comment thread vscode-extension/src/formats.ts Resolved
@github-actions

Copy link
Copy Markdown
Contributor

Polly AI Review

Blocking issues

  • P2 — require an accepted stop reason before releasing tools. At vscode-extension/src/formats.ts:348, missing/null reasons bypass validation; message_stop alone then releases buffered calls. This breaks the stated completion contract. Track an accepted reason and require it alongside message_stop. Extend the failure test and update the Claude answers fixture accordingly.

Security vulnerabilities

No additional concrete vulnerabilities found by static inspection. The completion-validation issue above affects the intended fail-closed tool boundary.

Non-blocking notes

  • Can merging preserve result-first ordering across messages? At vscode-extension/src/formats.ts:301, [result(c1), text] followed by [result(c2)] becomes [result(c1), text, result(c2)]. Whether Copilot produces this ordering is unverified; extend puts a tool's images in its result, results before text, and joins messages of the same role with this boundary.
  • Can TestConfigure.test_reruns_reinstall_with_the_new_setup cover provider → location → flagless configuration? The current scenario does not verify clearing a previously installed provider.

Approach

The shared Copilot catalog, provider validation, and model-limit helpers fit the outcome. One concrete simplification: share Claude release-suffix normalization instead of duplicating its regex in src/ucode/agents/vscode.py:59 and the existing Copilot implementation.

Summary

Diff size (computed): 12 files; +689 / -114 lines (803 changed text lines); 0 binary files.

Category Files Added Deleted Share of changed text lines
Tests and test support 3 373 34 50.7%
Documentation 1 10 5 1.9%
Dependencies and lockfiles 0 0 0 0.0%
Implementation / other 8 306 75 47.4%

Shares use additions + deletions, including generated text and lockfiles. Binary files have no line count. Tests include fixtures/helpers under test paths and colocated test/spec files; categories are path-based, not coverage measurements.

Changes

VS Code configuration gains provider-service and model-location sources, explicit model families, and provider-aware routing. Claude requests move to Anthropic Messages, including request translation and streamed completion handling. Python and TypeScript tests, extension documentation, and the bundled JavaScript accompany those changes; no dependencies change.

The size is proportionate to adding both configuration and transport behavior. Shared release-suffix normalization is a concrete opportunity to reduce duplication without removing functionality. Added or modified source comments are brief; no conciseness issue was found.

Tests

Test-by-test assessment

Assessment is static; tests were not run. The PR reports automated and manual checks, but their execution was not independently verified. The table accounts for changed tests and support edits, including renamed cases and parameterized inputs.

Test / case Behavior protected Layer Needed? Action / rationale
tests/test_agent_vscode.py:70 — code fixture and import additions Synthetic authentication for shared catalog selection Test support keep Avoids real token lookup while retaining selection and packaging behavior.
tests/test_agent_vscode.py:90 — TestConfigure.test_installs_the_extension_with_the_workspace_and_its_claude_and_gpt_models Default configuration includes families and Anthropic routing Component keep Checks the packaged VSIX configuration rather than only an internal builder.
tests/test_agent_vscode.py:131 — TestConfigure.test_models_with_the_same_label_show_their_ids Distinct picker identities after catalog reuse Component keep Preserves collision behavior while adapting expected ordering.
tests/test_agent_vscode.py:152 — TestConfigure.test_a_provider_services_claude_targets_with_a_200k_cap_and_its_name Bedrock IDs, Claude filtering, external limits, and provider persistence Component keep Shared resolver coverage alone does not protect VS Code serialization.
tests/test_agent_vscode.py:193 — TestConfigure.test_a_openai_provider_services_models_on_responses_and_chat_completions Allow-all listing and GPT API/family mapping Component keep Covers gateway discovery and the dotted gpt-5.5 family regression.
tests/test_agent_vscode.py:229 — TestConfigure.test_keeps_the_installed_setup_for_a_provider_service_it_cant_use: Gemini, relayed Anthropic, and denied OpenAI listing Unusable sources leave the installed setup unchanged Component keep The cases share a preservation contract but exercise distinct rejection causes.
tests/test_agent_vscode.py:247 — TestConfigure.test_a_model_locations_services_with_a_200k_cap Location forwarding, custom IDs, external Claude limits, and omitted provider Component keep Protects location-specific adapter behavior, including unknown-name services.
tests/test_agent_vscode.py:452 — test_configure_agents_vscode_refuses_a_workspace_the_machine_isnt_managed_for Workspace ownership refusal before setup CLI component keep Stub changes preserve the existing ownership-boundary assertion.
tests/test_agent_vscode.py:474 — test_configure_agents_vscode_sets_up_the_workspace_then_vscode, dry_run=False/True Source forwarding and dry-run suppression CLI component keep Tests orchestration separately from packaging and protects the mutation boundary.
tests/test_agent_vscode.py:526 — test_configure_refuses_a_model_source_vscode_cant_take: Claude-only selection and omitted agents Source options require VS Code selection CLI component keep Covers explicit wrong-agent and omitted-agent inputs before setup.
tests/test_agent_vscode.py:526 — test_configure_refuses_a_model_source_vscode_cant_take: both source flags Mutual exclusion CLI component keep Prevents ambiguous source selection.
tests/test_agent_vscode.py:526 — test_configure_refuses_a_model_source_vscode_cant_take: three-part location Location shape validation CLI component keep Protects the public schema argument contract.
vscode-extension/test/extension.test.ts:7 — imports; vscode-extension/test/extension.test.ts:15 — stubFetch Request-header inspection Test support keep Extends the existing transport fake instead of adding another harness.
vscode-extension/test/extension.test.ts:41 — yields each event's JSON until [DONE] SSE completion-marker parsing Unit keep Path-argument adaptation retains existing parser coverage.
vscode-extension/test/extension.test.ts:47 — keeps a last event that has no trailing newline Final buffered SSE event Unit keep Protects a distinct finalization boundary.
vscode-extension/test/extension.test.ts:53 — retries once with a new token after a 401, then asks the user to sign in Token renewal, bounded retry, and request destination Component keep Preserves authentication recovery and terminal-failure coverage.
vscode-extension/test/extension.test.ts:67 — reports other gateway errors with their body Non-authentication HTTP diagnostics Unit keep Protects actionable errors independently of format translation.
vscode-extension/test/extension.test.ts:91 — keeps well-formed models, names, families and token limits, the provider, and trims the workspace URL New fields alongside malformed-entry filtering Unit keep Renames and extends the existing mixed-validity configuration scenario.
vscode-extension/test/extension.test.ts:121 — gives no models for a missing or malformed file Safe defaults, including empty provider Unit keep Updates the existing malformed-file boundary without redundant setup.
vscode-extension/test/extension.test.ts:130 — settings; vscode-extension/test/extension.test.ts:156 — answers; vscode-extension/test/extension.test.ts:162 — askEach Shared request setup across three APIs Test support keep Reuses request-driving code; update Claude’s response fixture with an accepted stop reason.
vscode-extension/test/extension.test.ts:141 — lists the configured models by name, or by id without one, or none with a hint when settings are missing Model registration, explicit family, limits, and empty-state hint Unit keep Protects the VS Code-facing model contract separately from parsing.
vscode-extension/test/extension.test.ts:176 — sends each model's turns to its API with the User-Agent and its output limit Default routes, absent provider header, attribution, and output limits Component keep Extends the renamed routing case and retains older-GPT wiring coverage.
vscode-extension/test/extension.test.ts:192 — routes a Model Provider Service's models with its header, and OpenAI ones on the OpenAI route Provider headers across APIs and provider-specific OpenAI routes Component keep Exercises transport selection without duplicating translator payload assertions.
vscode-extension/test/formats.test.ts:3 — imports; vscode-extension/test/formats.test.ts:191 — cached Translator access and expected cache marker Test support keep Uses existing conversation/event primitives with minimal added setup.
vscode-extension/test/formats.test.ts:193 — sends the system prompt, conversation, images and tools, with cache points after the system prompt and the last message Anthropic payload and cache placement Unit keep Wire-format assertions belong at the translator layer.
vscode-extension/test/formats.test.ts:221 — puts a tool's images in its result, results before text, and joins messages of the same role Nested images, empty results, ordering, and role merging Unit keep Extend with results split across consecutive messages containing text.
vscode-extension/test/formats.test.ts:260 — leaves out empty messages, the system prompt and tools when there are none Minimal requests and omitted optional fields Unit keep Covers branches absent from the populated payload scenario.
vscode-extension/test/formats.test.ts:265 — streams text, drops thinking, and assembles tool calls from their fragments once the message stops Thinking suppression, fragmented calls, and successful completion Unit keep Protects successful multi-call assembly and deferred emission.
vscode-extension/test/formats.test.ts:291 — fails on a token-limit stop, a refusal, any other stop reason, an error event, and a stream without message_stop Limit, refusal, unsupported stop, error, and cutoff handling Unit keep Extend with missing/null stop reasons followed by message_stop; assert no tool emission.

Existing coverage protects default-discovery failures, reinstall/revert, authentication variants, shared catalog failures, Copilot’s external Claude limits, token renewal, and common tool-argument validation. Those assertions do not need duplicate E2E tests.

The remaining concrete additions are the completion-validation regression and configuration transition noted above. Split-result merging is a useful translator boundary case, though its occurrence in Copilot input remains unverified. VS Code’s no-Claude Bedrock rejection has indirect protection through shared resolver coverage.

After fixes, verify the malformed stream emits no tools and that provider → location → flagless reconfiguration clears provider routing. Run focused Python and extension checks in a credential-free checkout with dependencies already installed.

Scope

All changed files support the stated configuration and Anthropic-routing outcome; no clearly unrelated changes were found. Managed VS Code configuration remains the declared follow-up. No linked issue narrows that outcome, and the truncated description does not establish additional scope exclusions.


Automated review by Polly · workflow run

…s, retry a refused output limit

A model's real output limit can be below the one ug configures, which Anthropic
Messages rejects; the gateway names the limit it allows, so retry once with it.
--provider and --model-location now refuse any other model agent in the same
run, and the per-model api picks its route through a lookup.

Co-authored-by: Isaac <no-reply@databricks.com>
@xi-db
xi-db marked this pull request as ready for review October 29, 2026 21:10
Comment thread src/ucode/agents/copilot.py Resolved
Comment thread src/ucode/agents/copilot.py Resolved
Comment thread src/ucode/agents/vscode.py Resolved
xi-db and others added 2 commits October 30, 2026 09:14
Co-authored-by: Isaac <no-reply@databricks.com>
…ond set of names

Co-authored-by: Isaac <no-reply@databricks.com>
@xi-db
xi-db enabled auto-merge October 30, 2026 09:19
@xi-db
xi-db disabled auto-merge October 30, 2026 09:47
xi-db and others added 3 commits October 30, 2026 10:25
A service's targets are the provider's ids, which is what its users know it by,
and its regional variants differ only by a prefix a readable label drops.

Co-authored-by: Isaac <no-reply@databricks.com>
A system-role input item is refused outright by Microsoft Foundry, which made
every turn through a Foundry service fail; OpenAI and Azure OpenAI accept the
prompt either way.

Co-authored-by: Isaac <no-reply@databricks.com>
Azure names a regional deployment `eu/gpt-5.6-luna`, which no longer hid the
model behind its region: such a model was classified for Chat Completions, which
also switched off image input, and its family reached Copilot Chat unrecognisable.

Co-authored-by: Isaac <no-reply@databricks.com>
@xi-db
xi-db enabled auto-merge October 30, 2026 10:54
@xi-db
xi-db added this pull request to the merge queue Oct 30, 2026
Merged via the queue into main with commit 6fc6bf4 Oct 30, 2026
59 checks passed
@xi-db
xi-db deleted the xi-db/stack/vscode-mps branch October 30, 2026 11:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ug-review Run the automated UG review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants