Skip to content

fix: replace the gpt-5-mini runtime default model before OpenAI retires it - #255

Open
alexcastgonz (pcbeingused333) wants to merge 1 commit into
braintrustdata:mainfrom
pcbeingused333:fix/default-model-gpt-5-mini-retirement
Open

alexcastgonz (pcbeingused333) wants to merge 1 commit into
braintrustdata:mainfrom
pcbeingused333:fix/default-model-gpt-5-mini-retirement

Conversation

@pcbeingused333

Copy link
Copy Markdown

GPT-5 mini, GPT-5 nano and GPT-5 are retired by OpenAI on 2026-12-11 (replacements: gpt-5.6-terra, gpt-5.6-luna, gpt-5.6-sol). The SDK fallback model, get_default_model() in Python and getDefaultModel() in TypeScript, is still gpt-5-mini. After the retirement, any scorer that runs without init(default_model=...) or an explicit model would fail.

#230 moved the exported DEFAULT_MODEL constant to gpt-6-luna and kept this fallback on purpose. This PR changes the fallback now, because its model goes away:

  • get_default_model() / getDefaultModel() return gpt-5.6-terra, OpenAI's replacement for gpt-5-mini. It keeps the gpt-5 prefix, so the routing to the Responses API does not change.
  • The deprecated DEFAULT_RAGAS_MODEL (gpt-5-nano) becomes gpt-5.6-luna, its replacement.
  • Docstrings and SCORERS.md name the new defaults.

If you prefer one default for both the constant and the fallback, gpt-6-luna is a two-string change; it does not start with gpt-5, so it would use the Chat Completions path unless use_responses_api is set. Say which one you prefer and the PR will follow it.

Tests: the Python tests for the fallback (test_oai.py, test_init_models.py) and the TypeScript ones (oai.test.ts, init-models.test.ts) now expect the new default; the four Python ones fail on main. Python: those two files pass. TypeScript: oai.test.ts, init-models.test.ts and llm.test.ts pass. The tests that call the live API (ragas.test.ts, the Python test_llm.py ones) need an OpenAI key and were not run, so the full suites were not compared against the previous state.

This change was prepared with AI assistance (Claude Code).

…es it

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant