Skip to content

fix: allow explicit thread render overrides - #247

Open
Aryan R. Ganjoo (AryanRG) wants to merge 1 commit into
braintrustdata:mainfrom
AryanRG:fix/python-thread-override
Open

Aryan R. Ganjoo (AryanRG) wants to merge 1 commit into
braintrustdata:mainfrom
AryanRG:fix/python-thread-override

Conversation

@AryanRG

Copy link
Copy Markdown

When LLMClassifier receives a trace and evaluation-time thread or thread_count, its separate keyword expansions raise TypeError before rendering. Merge the computed variables and explicit arguments into one mapping in both sync and async request assembly, so explicit values take precedence and other computed values remain available.

Add three regressions covering sync and async thread/count overrides and an independent count-only collision. The combined cases also verify that the computed full thread retains its system message.

Validation on CPython 3.14.4, macOS arm64, with lock-pinned dependencies and networking denied:

  • New tests: three exact duplicate-keyword failures before the fix; three passes afterward.
  • Full test_llm.py: 18 passed, four provider integrations intentionally unrun, one missing-credentials failure in test_factuality.
  • Python suite: 91 collected without errors; 74 passed, 16 provider integrations intentionally unrun, the same missing-credentials failure. That failure also reproduces on untouched base source. Existing fake-key test_oai.py tests remained in scope.
  • Black 24.10.0, Ruff 0.12.7, direct EOF/whitespace checks, and git diff --check passed.

The complete dev/scipy/litellm closure was installed from verified lock-pinned wheels. The documentation extra remains incomplete because its pinned docstring-parser==0.11 and watchdog==6.0.0 have no compatible locked wheels. Real-provider integration, other Python versions, TypeScript tests, aggregate pre-commit, and codespell were not run. No provider credentials were supplied.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant