[Agentic Story] Split Azure Databricks connector into Genie Space and Genie One #269

Summary

Replace the Azure Databricks Foundry connector with separate Genie Space and Genie One catalog entries, using distinct connector identifiers and the explicitly selected non-Unity Gateway endpoints.

Retrospective status, October 23, 2026: implementation and local static validation are complete; the feature branch is published on dgepstein/MCP and draft PR #270 targets Azure/MCP:main. DO NOT MERGE - NOT READY FOR ADMIN MERGE. Nothing has been merged. Direct writes to origin remain unavailable; fork publication resolves availability for inspection, not runtime or merge readiness.

Originating Prompt

Privacy-preserving paraphrase of the original request; local filesystem paths and non-public research-handoff references are intentionally omitted:

Reimplement the existing Azure Databricks Foundry connector for current connector conventions and Azure Databricks, particularly Genie One. Validate the research against public documentation, clarify material decisions, follow repository contribution guidance, and work on a feature branch without submitting a PR.

Subsequent human decisions:

• Replace the original entry with azure-databricks-geniespace-mcp-server.json and add azure-databricks-genieone-mcp-server.json.
• Preserve both per-space and workspace-wide options, with a new assumed connectorName for each.
• Use the non-Unity Gateway Genie One endpoint and correct the initially misspelled filename.
• Do not change community/registry/servers.json.
• Subsequently authorized committing and pushing the branch to origin, but still no PR.
• Requested this retrospective Agentic-Agile backlog issue.
• Subsequently authorized fork publication and a draft PR with an explicit not-ready-for-admin-merge title. This supersedes the earlier no-PR boundary for a draft only; marking ready, enabling auto-merge, or merging is not authorized.

Context / Motivation

The original connector targets one Genie space and does not represent the distinct workspace-wide Genie One option. It also mixes a delegated permission with .default and contains HTTP OAuth authority placeholders whose substitution behavior is unverified.

This change separates the catalog identities, makes transport and OAuth metadata explicit, and avoids silently replacing per-space behavior with a workspace-wide endpoint.

Endpoint lifecycle: the selected non-Unity Genie One beta endpoint is documented as deprecated and scheduled to retire on October 31, 2026. The user explicitly confirmed this endpoint after that limitation was surfaced. This story does not implement the replacement Unity Gateway service or claim that the legacy endpoint is a long-term supported route.

Scope

Files to Create or Modify

• Remove partners/servers/azure-databricks-mcp-server.json.
• Create partners/servers/azure-databricks-geniespace-mcp-server.json as its per-space replacement.
• Create partners/servers/azure-databricks-genieone-mcp-server.json as the separate Genie One entry.

Interfaces to Implement

Both manifests declare streamable-http, Entra v2 HTTPS authorization/token/refresh URLs, the Azure Databricks audience 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d, its delegated user_impersonation permission, offline_access, authSchemas: ["OAuth2"], and preview metadata.

These are proposed catalog bindings. Their presence does not provision managed connector registrations or prove successful runtime authentication.

Invariants to Preserve

• Retain access to one curated Genie space through the space-ID endpoint.
• Keep manifest names and authentication keys globally unique.
• Preserve Microsoft vendor, icon, legal, support, visibility, and version metadata.
• Keep the repository schema, workflows, unrelated connectors, and community registry unchanged.
• Do not represent catalog metadata or local validation as evidence of a functioning managed OAuth integration.

Acceptance Criteria

• Remove the original manifest and add the two correctly named replacement manifests.
• Use distinct new manifest names, OAuth keys, and assumed connector names; neither entry reuses foundrydatabricksmcp.
• Use the exact per-space and no-space-ID endpoint patterns above; neither remote uses Unity Gateway.
• Declare transport, audience, OAuth2, HTTPS authority/token/refresh URLs, delegated permission, and refresh scope without mixing .default with dynamic permission scopes.
• Disclose the legacy Genie One beta endpoint's October 31, 2026 retirement in its summary and description.
• Pass schema validation across all 115 partner manifests, including enabled URI-format validation and a negative control that rejects an invalid icon URI.
• Verify 115 globally unique server names and 123 globally unique security-scheme keys, valid JSON, field limits, whitespace, and the exact three-path change scope.
• Preserve community/registry/servers.json unchanged and commit only the approved manifest changes.
• Publish the feature branch to the personal fork for inspection and verify the remote commit matches the validated implementation. Direct origin publication remains unavailable.
• Open [NOT READY FOR MERGE] Split Databricks Genie connectors #270 as a draft with [DO NOT MERGE - NOT READY FOR ADMIN MERGE] in the title, the exact validated head, and auto-merge disabled.

Negative Constraints

• Does NOT implement Genie One through /ai-gateway/mcp-services/system.ai.genie_one_mcp.
• Does NOT copy Unity Gateway-specific tools, policies, or governance claims into the legacy entry.
• Does NOT modify the community registry, schemas, validation workflow, or repository dependencies.
• Does NOT provision connector registrations, consent applications, credentials, cloud resources, or permissions.
• Does NOT claim live consent, effective user identity, token refresh, transport negotiation, tool invocation, data access, or cross-tenant support has been tested.
• Does NOT mark the PR ready, enable auto-merge, merge it, perform an administrator merge, or close this issue prematurely. Only draft publication has been authorized.
• Does NOT publish non-public research, internal communications, or local session artifacts.

Dependencies

• Direct-origin limitation: pushes to Azure/MCP returned HTTP 403, including an explicit attempt with the issue-author account. The branch is now published on dgepstein/MCP, while origin remains unchanged.
• Hosted validation: the validator run for [NOT READY FOR MERGE] Split Databricks Genie connectors #270 reports action_required with no jobs executed. Maintainer action is outstanding; no hosted CI pass is claimed.
• Runtime follow-up: connector owners must confirm/provision the assumed managed registrations and validate the exact Foundry/Databricks flow before operational readiness can be claimed.
• Lifecycle follow-up: the selected beta endpoint's retirement constrains usable deployment; Unity Gateway migration remains outside this story.

Implementation / Validation Evidence

• Published feature branch: dgepstein/MCP:update-azure-databricks-genie-connectors.
• Draft PR: [NOT READY FOR MERGE] Split Databricks Genie connectors #270, DO NOT MERGE - NOT READY FOR ADMIN MERGE. Verified open, draft, exact head, and no auto-merge.
• Origin publication retry (October 23, 2026): an explicit push using dgepstein returned HTTP 403; the named feature branch was verified absent from origin. Subsequent fork publication does not change origin permissions or create that branch on origin.
• Published head commit: 9f4696f282e4648db391e6afa34f47178b5e8876 (Split Azure Databricks connector into Genie Space and Genie One). Verified the fork branch matches this local commit and its comparison with Azure/MCP:main contains only the three intended manifest paths.
• Ran the workflow-equivalent schema command using python -m check_jsonschema --verbose --schemafile partners\server-schema.json with all partner JSON paths expanded by PowerShell.
• Used check-jsonschema 0.38.2 in an isolated, non-repository environment. A negative control exposed absent optional URI support; after adding rfc3986-validator 0.1.1 there, all 115 manifests passed and an intentionally invalid icon URI was rejected.
• Requirement assertions and global uniqueness checks passed. The complete deletion and both additions were reviewed; git diff --check passed. The committed working tree is clean.
• Validation was repeated before committing and full catalog schema validation was repeated before fork publication. Local Python was 3.14.7; the repository workflow specifies 3.12. Hosted validator run currently reports action_required with zero jobs. Live runtime integration has not been run.

File Ownership

Public References / Format

• Agentic Story starting template. Azure/MCP had no active issue template when this issue was authored.
• Genie One MCP documentation and beta retirement notice.
• Per-space Genie Agent MCP documentation.
• Azure Databricks Entra resource and delegated permission.
• Entra scopes, offline_access, and .default restrictions.