Skip to content

Upgrade gpt-latest to three-tier model failover - #25

Merged
Paul Yuknewicz (paulyuk) merged 6 commits into
mainfrom
copilot/upgrade-gpt-latest-alias
Oct 25, 2026
Merged

Paul Yuknewicz (paulyuk) merged 6 commits into
mainfrom
copilot/upgrade-gpt-latest-alias

Conversation

Copilot AI commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Configure the default gpt-latest alias to fail over from gpt-6.1-sol to gpt-6-sol, then to gpt-5.4-mini. Provision and register all three deployments with the requested versions and capacities.

  • Infrastructure: Add the secondary model parameters, deployment, Gateway registration, and outputs. Keep gpt-mini-latest directly routed to the mini deployment.
  • Routing: Set each backend to weight 100:
    gpt-6.1-sol (priority 0) → gpt-6-sol (priority 1) → gpt-5.4-mini (priority 2)
    
  • Tests and documentation: Assert the three routes and model defaults; document the failover chain, model table, and fallback guidance.

Co-authored-by: paulyuk <1968137+paulyuk@users.noreply.github.com>
Copilot AI changed the title [WIP] Update gpt-latest alias to 3-tier failover Upgrade gpt-latest to three-tier model failover Oct 24, 2026
@paulyuk

Copy link
Copy Markdown
Member

Copilot Please also add developer-friendly README.md files at each Bicep folder level so developers can understand and deploy either the full stack or any individual Bicep module (azd or standalone az deployment):

  1. infra/README.md (Subscription-scoped root orchestration):

    • Overview of the 3 resource groups/modules (foundry-models/, foundry-agents/, ai-gateway/) and scripts/ lifecycle hooks.
    • How to deploy all infrastructure via azd provision (with preprovision/postprovision hooks) or directly via az deployment sub create --location eastus2 --template-file infra/main.bicep --parameters environmentName=dailydigest location=eastus2 principalId="$(az ad signed-in-user show --query id -o tsv)".
    • Links to each child module README (foundry-models/README.md, foundry-agents/README.md, ai-gateway/README.md, scripts/README.md).
  2. infra/foundry-models/README.md (Resource-group-scoped model account + deployments):

    • What it provisions: model-only Azure AI Services account (allowProjectManagement: false, disableLocalAuth: true), the 3 model deployments (gpt-6.1-sol, gpt-6-sol, gpt-5.4-mini), and optional developer RBAC.
    • Copy-pasteable az group create + az deployment group create command to deploy infra/foundry-models/main.bicep standalone, plus key outputs (aiServicesId, aiServicesEndpoint, modelDeploymentIds).
  3. infra/foundry-agents/README.md (Resource-group-scoped Foundry project + hosted agent runtime):

    • What it provisions: project-enabled AI Services account (allowProjectManagement: true), Foundry project, capability host, Storage Account, Premium ACR, Log Analytics + Application Insights, and RBAC assignments.
    • Copy-pasteable az group create + az deployment group create command to deploy infra/foundry-agents/main.bicep standalone, plus key outputs (projectEndpoint, containerRegistryEndpoint, aiProjectResourceId).
  4. infra/ai-gateway/README.md (Resource-group-scoped standalone AI Gateway + model routing):

    • What it provisions: Microsoft.ApiManagement/aigateways (Consumption SKU), Microsoft.Web/connectorGateways, cross-RG Cognitive Services User RBAC (foundry-user-role.bicep), Foundry modelProviders/foundry with child catalog models, gpt-latest (3-tier failover: gpt-6.1-sol -> gpt-6-sol -> gpt-5.4-mini) and gpt-mini-latest aliases, apiKeys/default, and Application Insights telemetry exporter.
    • Copy-pasteable az group create + az deployment group create command to deploy infra/ai-gateway/main.bicep standalone (both gateway-only and wired to an existing Foundry model account), plus how to run ./infra/scripts/configure-ai-gateway.sh to wire up the GitHub MCP ToolServer and Foundry Toolbox.
  5. infra/scripts/README.md (Preprovision, postprovision, and postdown hooks):

    • Concise explanation and standalone usage for manage-ai-gateway-lifecycle.sh / .ps1 (preprovision failed-gateway cleanup and postdown identity-cleanup wait) and configure-ai-gateway.sh / .ps1 (retrieving apiKeys/default, registering the GitHub MCP toolServers/github, creating the Foundry project connection and Toolbox repo-digest-tools).

Co-authored-by: paulyuk <1968137+paulyuk@users.noreply.github.com>

Copilot AI commented Oct 24, 2026

Copy link
Copy Markdown
Contributor Author

Copilot Please also add developer-friendly README.md files at each Bicep folder level so developers can understand and deploy either the full stack or any individual Bicep module (azd or standalo...

Added the five infrastructure READMEs with full-stack and standalone deployment examples, module outputs, Gateway routing, and script lifecycle/configuration guidance. Committed as fd4c19a.

@paulyuk

Copy link
Copy Markdown
Member

Copilot Update the 5 infra/**/README.md files so every README instructs commands relative to its own folder (assuming the user has cd'd into that folder):

  1. infra/README.md: Assume the working directory is infra/:
    • Note that azd provision still works from infra/ (since azd walks up to find ../azure.yaml and deploys the full stack + hooks).
    • For direct subscription deployment from infra/, use --template-file ./main.bicep.
  2. infra/foundry-models/README.md: Assume the working directory is infra/foundry-models/ and use --template-file ./main.bicep.
  3. infra/foundry-agents/README.md: Assume the working directory is infra/foundry-agents/ and use --template-file ./main.bicep.
  4. infra/ai-gateway/README.md: Assume the working directory is infra/ai-gateway/:
    • Use --template-file ./main.bicep in both az deployment group create commands.
    • For the optional postprovision script reference, use bash ../scripts/configure-ai-gateway.sh (or note running from ../scripts).
  5. infra/scripts/README.md: Assume the working directory is infra/scripts/ and use ./manage-ai-gateway-lifecycle.sh, ./manage-ai-gateway-lifecycle.ps1, ./configure-ai-gateway.sh, and ./configure-ai-gateway.ps1 (since both scripts resolve REPO_ROOT from BASH_SOURCE[0] / $PSScriptRoot, they work when run directly from infra/scripts/).

Co-authored-by: paulyuk <1968137+paulyuk@users.noreply.github.com>

Copilot AI commented Oct 25, 2026

Copy link
Copy Markdown
Contributor Author

Copilot Update the 5 infra/**/README.md files so every README instructs commands relative to its own folder (assuming the user has cd'd into that folder):

  1. infra/README.md: Assume the wo...

Updated all five READMEs with their expected working directories and folder-relative deployment/script paths. The infra README now explains azd discovery from infra/. Committed as 5248276.

@paulyuk

Copy link
Copy Markdown
Member

Copilot In the root README.md, update the opening overview and the ## How it works section to clearly explain the hub-and-spoke architecture where Foundry agents call models and MCP servers exclusively through AI Gateway with no direct access path to them:

  1. Replace the opening paragraph under # Daily Repo Digest with AI Gateway with:
This sample demonstrates a **hub-and-spoke governance architecture** for Foundry Hosted Agents using **AI Gateway** as the central hub. The Microsoft Agent Framework repo-digest agent runs in an isolated Foundry project spoke (`infra/foundry-agents/`) with zero model deployments and no direct credentials to backing models or external MCP servers. All model inference (`gpt-latest` three-tier failover across `gpt-6.1-sol` → `gpt-6-sol` → `gpt-5.4-mini`, plus `gpt-mini-latest`) and read-only GitHub MCP tool calls (`via Foundry Toolbox`) are routed **exclusively through AI Gateway (`infra/ai-gateway/`)** — there is no other access path from the agent to the model account (`infra/foundry-models/`) or to GitHub MCP.
  1. Under ## How it works, add a brief hub-and-spoke breakdown right before the See IMPLEMENTATION_NOTES.md... paragraph:
- **Hub (`infra/ai-gateway/`)**: Standalone AI Gateway (`Microsoft.ApiManagement/aigateways`) that enforces central routing, token rate limits, failover (`gpt-latest`), and MCP tool brokering (`toolServers/github`). Only AI Gateway's managed identity has RBAC access (`Cognitive Services OpenAI User`) to the model account, and only AI Gateway holds the GitHub MCP token.
- **Agent Spoke (`infra/foundry-agents/`)**: Hosts the Foundry project, ACR, Storage, and Hosted Agent runtime. Its project account has no model deployments (`modelDeployments: []`) and no RBAC assignment on the model account, ensuring the agent cannot bypass AI Gateway to call models or GitHub MCP directly.
- **Model Spoke (`infra/foundry-models/`)**: Dedicated model-only Azure AI Services account (`allowProjectManagement: false`, `disableLocalAuth: true`) hosting `gpt-6.1-sol`, `gpt-6-sol`, and `gpt-5.4-mini`, accessible to the agent only through AI Gateway.

Co-authored-by: paulyuk <1968137+paulyuk@users.noreply.github.com>

Copilot AI commented Oct 25, 2026

Copy link
Copy Markdown
Contributor Author

Copilot In the root README.md, update the opening overview and the ## How it works section to clearly explain the hub-and-spoke architecture where Foundry agents call models and MCP servers e...

Updated the root README with the hub-and-spoke overview and Hub, Agent Spoke, and Model Spoke breakdown. I described the Gateway's actual cross-resource permission as the configured Foundry User role. Committed as ccf3d89.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1b3ab80e-64db-4e12-8e50-8fe293ced64b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Upgrade gpt-latest alias to 3-tier failover (gpt-6.1-sol -> gpt-6-sol -> gpt-5.4-mini)

2 participants