Skip to content

test: bump vcrpy and drop the aiohttp stub patches 8.5.0 fixes - #997

Merged
Abhijeet Prasad (AbhiPrasad) merged 1 commit into
braintrustdata:mainfrom
pylaterreur:vcrpy-850-drop-aiohttp-stub-patches
Oct 27, 2026
Merged

Abhijeet Prasad (AbhiPrasad) merged 1 commit into
braintrustdata:mainfrom
pylaterreur:vcrpy-850-drop-aiohttp-stub-patches

Conversation

@pylaterreur

@pylaterreur Pierre-Yves Le Borgne (pylaterreur) commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #834. VCR.py 8.5.0 is out with kevin1024/vcrpy#1055, which fixes kevin1024/vcrpy#502 (an aiohttp response streamed while recording came back empty) and kevin1024/vcrpy#927 (a replayed response.content restarted on each access, so google-genai's readline() loop never ended). Since #834, _patch_vcr_aiohttp_stubs() in py/src/braintrust/conftest.py skips its patches for those on such releases, and its docstring says to delete them once uv.lock has one.

Changes

  • Add vcrpy>=8.5.0 to the test dependency group. Since chore(deps): weekly dependency update #998, uv.lock already has VCR.py 9.2.0 on Python 3.11 and newer, and 8.5.0 on Python 3.10, which VCR.py 9 doesn't support; it only gains the new requirement.
  • Remove the patches for problems 1, 3 and 4 (cached content, no-op MockStream.set_exception, record_response replacement), and the shims that only older VCR.py needed: the AsyncStreamReaderMixin placeholder, the stream_writer constructor and MockStream.iter_chunked. VCR.py 8.5.0 does all of these itself. The gzip decoding in text() and read() stays.
  • This no longer moves the exclude-newer cutoff: it's rebased onto chore(deps): weekly dependency update #998, whose cutoff (2026-10-26T06:02:31Z) already covers 9.2.0. On the old base, the cutoff the earlier push set (2026-10-25T22:30:00Z) also let test_pydantic_ai_integration(latest) install anthropic 2.3.0 next to the old pydantic-ai==2.57.0 pin. Anthropic 2 made BetaMessageParam a Union, so test_agent_run_anthropic_reasoning_tokens failed with TypeError: Cannot instantiate typing.Union in the previous CI run; chore(deps): weekly dependency update #998 moved the pin to 2.61.0, which works with it.

VCR.py 9.0 has some breaking changes (release notes). Nothing in this repo matches on the CannotOverwriteExistingCassetteException message or uses filter_post_data_parameters, and the sessions below pass with 9.2.0.

Testing

nox sessions as CI runs them (uv run --project ./py nox -f ./py/noxfile.py -s ..., Python 3.12, cassette replay):

Session main (VCR.py 9.2.0 since #998) this PR
test_litellm(latest) 43 passed 43 passed
test_litellm(1.74.0) 30 passed, 13 skipped 30 passed, 13 skipped
test_dspy(latest) 15 passed 15 passed
test_google_genai(latest) 44 passed, 2 skipped 44 passed, 2 skipped
test_huggingface_hub(0.32.0) 17 passed, 2 skipped 17 passed, 2 skipped
test_google_adk(latest) 16 passed 16 passed
test_google_adk(2.6.3) 13 passed, 3 skipped 13 passed, 3 skipped
test_google_adk(1.14.1) 13 passed, 3 skipped 13 passed, 3 skipped

All six of CI's nox shards for Python 3.12 on Linux (py/scripts/nox-matrix.py <n> 6 --python-version 3.12 --os ubuntu-24.04 --exclude-static-checks --exclude-session test_api_codegen) pass on this branch, as on main: 90 sessions, with the same installed packages in every session. On Python 3.10, where the lock has 8.5.0, the 3.10 shards pass too (75 sessions), and test_litellm(1.74.0), test_google_genai(latest) and test_huggingface_hub(0.32.0) have the same counts as above. test_pydantic_ai_integration(latest) passes on Python 3.11 to 3.15 (61 passed).

These replays don't reach the bugs the patches were for: without the patches, test_litellm (both versions), test_dspy(latest), test_google_genai(latest) and test_google_adk(latest) also pass on VCR.py 8.4.0. test_google_genai has no aiohttp, since google-genai makes it optional. With aiohttp added to test_google_genai(latest), as with google-genai[aiohttp], this PR on VCR.py 8.4.0 hangs in test_basic_completion[async_stream] (kevin1024/vcrpy#927). It passes on 8.5.0 (44 passed, 2 skipped), as main does on 8.4.0.

Pre-commit hooks pass on the changed files, and pylint --errors-only is clean on conftest.py.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GwUvhmzgfkiZ6vo4uGyesN

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-25T22:41:02.474334Z e86691e PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@pylaterreur Pierre-Yves Le Borgne (pylaterreur) changed the title test: bump vcrpy to 8.5.0 and drop the aiohttp stub patches it fixes test: bump vcrpy and drop the aiohttp stub patches 8.5.0 fixes Oct 25, 2026
vcrpy 8.5.0 includes kevin1024/vcrpy#1055, which fixes
kevin1024/vcrpy#502 and kevin1024/vcrpy#927: a response streamed while
recording keeps its body, a replayed response keeps a single aiohttp
StreamReader as its content, so google-genai's readline() loop ends,
and closing the response no longer poisons that stream. Since braintrustdata#834,
conftest.py has skipped its patches for those (problems 1, 3 and 4) on
such releases. Require vcrpy 8.5.0 in the test group and remove them,
along with the shims only older vcrpy needed: the
AsyncStreamReaderMixin placeholder, the stream_writer constructor and
MockStream.iter_chunked. The gzip decoding in text() and read() stays.

The weekly dependency update (braintrustdata#998) moved uv's exclude-newer cutoff
past vcrpy 9.2.0's upload, so uv.lock already has vcrpy 9.2.0 on
Python 3.11 and newer, and 8.5.0 on Python 3.10, which vcrpy 9 doesn't
support. It only gains the new requirement, in the groups that include
test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GwUvhmzgfkiZ6vo4uGyesN

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AbhiPrasad
Abhijeet Prasad (AbhiPrasad) merged commit bd4ff59 into braintrustdata:main Oct 27, 2026
98 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

aiohttp streaming fails on initial cassette recording

2 participants