Repository navigation
feat(codex): prepare braintrust for the official plugin directory - #240
Open
Abhijeet Prasad (AbhiPrasad) wants to merge 3 commits into
Open
Abhijeet Prasad (AbhiPrasad) wants to merge 3 commits into
Abhijeet Prasad (AbhiPrasad) wants to merge 3 commits into
Conversation
> 🤖 **This text was AI-generated.** Add the package needed to publish Braintrust in the official ChatGPT and Codex plugin directory. It lets users query traces, manage prompts and datasets, and run evaluations from a conversation. Once approved and published, install Braintrust from the directory, sign in with your Braintrust account, and select it in a new conversation. Try "Show me my Braintrust projects" or "Summarize recent errors in my project." The plugin uses your account's permissions and needs no API key or local software. Installing it does not enable session tracing; bt trace enable codex continues to install only the existing tracing plugin. Keep both plugins in braintrustdata/braintrust-codex-plugin, built from this monorepo with the same release version. Extend the existing release approval process to create a ZIP containing only the new MCP plugin: ```text approve and merge release PR -> attach braintrust-<version>.zip to braintrust-codex-plugin's release -> download ZIP and upload it to OpenAI for review -> publish in the official directory after approval ``` Attach a checksum alongside the ZIP and allow attachment retries without repeating deployment. CI and test releases provide downloadable workflow artifacts. Build a local ZIP with make codex-submission. OpenAI needs a working server address to test the plugin. The package provides Braintrust's North America (US) address, https://api.braintrust.dev/mcp. Organizations using Europe or their own Braintrust deployment need a different address. We plan one public listing where organization admins can configure that address, rather than separate regional plugins. OpenAI must still confirm Braintrust's access to this feature, the address formats it supports, and whether North America can be preselected. GitHub installs currently use the North America address; admins needing another address must configure their own package copy. OpenAI submission and publication remain manual. Before submission, complete the separate server review and sign-in/tool checks with authorized test accounts. This change prepares the package; it does not publish a directory listing. Docs: - [Braintrust MCP setup](https://www.braintrust.dev/docs/integrations/developer-tools/mcp) - [Connect and test in ChatGPT](https://developers.openai.com/plugins/deploy/connect-chatgpt) - [Submit and publish the plugin](https://developers.openai.com/plugins/deploy/submission) - [Configurable server addresses](https://developers.openai.com/plugins/deploy/app-review#template-mcp-server-urls)
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: db0a3531e2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
> 🤖 **This text was AI-generated.** Codex installs the Braintrust plugin but drops its MCP server when mcp.json contains server-level auth extensions. Keep the GitHub package compatible with Codex and add OpenAI's documented OAuth declaration only when building the submission ZIP: ```text GitHub package -> server URL -> Codex discovers OAuth submission ZIP -> server URL + OAuth DCR declaration -> OpenAI portal ``` Validate the two formats separately. The ZIP builder leaves the source package untouched and continues to produce a reproducible MCP-only archive. Existing tracing setup and the reviewed release flow stay the same. Add an installed-plugin check that reads the MCP server through Codex's loader, since plugin add/list alone can succeed with a broken server entry. Run it in CI using the existing resolved Codex installation. Authentication and tool calls still require authorized integration checks. Refs: openai/codex#53658 Refs: https://linear.app/braintrustdata/issue/SDK-699/add-official-braintrust-chatgpt-marketplace-plugin
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Braintrust’s existing Codex plugin records coding sessions, but it does not give ChatGPT tools for working with your Braintrust data:
Add a separate Braintrust MCP plugin for publication in the official ChatGPT and Codex plugin directory. It lets you query traces, investigate failures, manage prompts and datasets, and run evaluations from a conversation.
Once approved and published, install Braintrust from the directory, sign in with your Braintrust account, and select the plugin in a new conversation. Try “Show me my Braintrust projects” or “Summarize recent errors in my project.” Tools use your account’s permissions and can read or change data. No API key or local Braintrust CLI installation is required.
After the GitHub distribution is released, Codex users can also install it with:
MCP sign-in is separate from tracing authentication. Installing Braintrust does not enable tracing, and
bt trace enable codexcontinues to select onlytrace-codex@braintrust-codex-plugins.Use the server address for your organization:
https://api.braintrust.dev/mcphttps://api-eu.braintrust.dev/mcpThe GitHub package currently uses North America. Admins using another address need a local/team package copy with that address configured. The intended public listing would use OpenAI’s template MCP server URLs, which let workspace admins configure the changing parts of the server address. OpenAI must still confirm Braintrust’s eligibility, supported address formats, and whether North America can be preselected. The North America address provided for OpenAI’s review does not establish an installation default.
Keep this monorepo as the source of truth and release both packages together:
The ZIP contains only the MCP plugin, with
plugin.jsonat its root. Build it locally withmake codex-submission; files appear underdist/submissions/. Attachment retries reuse the saved artifact without repeating deployment. CI and test releases provide downloadable workflow artifacts; test releases create no tags or GitHub Releases.The GitHub package uses server-discovered OAuth because current Codex versions reject explicit sign-in metadata. ZIP creation adds the OAuth registration settings required by the submission package without changing the GitHub package.
OpenAI upload, submission, and publication remain manual. Before submission, complete the separate server audit and sign-in/tool checks with authorized test accounts. This PR prepares the package; it does not publish a directory listing.
Further guidance:
Refs: SDK-699