Skip to content

FIX: Decode video frames and audio off the event loop in video scorers - #3359

Merged
Richard Lundeen (richlundeen) merged 3 commits into
microsoft:mainfrom
Shizoqua:claude/wizardly-brahmagupta-5obhxm-video-scorer
Oct 25, 2026
Merged

Richard Lundeen (richlundeen) merged 3 commits into
microsoft:mainfrom
Shizoqua:claude/wizardly-brahmagupta-5obhxm-video-scorer

Conversation

@Shizoqua

Copy link
Copy Markdown
Contributor

Description

Fixes #3358.

VideoHelper ran its two media decodes directly on the event loop: _extract_frames (OpenCV) in _score_frames_async, and AudioTranscriptHelper.extract_audio_from_video (PyAV) in _score_video_audio_async. Both now run through asyncio.to_thread, which is what the style guide asks for with cv2 and av and what AudioTranscriptHelper._transcribe_audio_async already does for its own PyAV conversion.

With the setup from the issue, the longest event loop stall while scoring a 1080p clip goes from about 2.4 s to about 150 ms. Scores, rationales and temp file cleanup are unchanged.

Tests and Documentation

Added test_video_scorer_decodes_frames_and_audio_off_the_event_loop to tests/unit/score/test_video_scorer.py. It wraps both extractors to record which thread they run on, scores the sample video with an image scorer and an audio scorer, and asserts that neither ran on the event loop thread. Existing video scorer tests are unchanged.

No docs or notebooks changed, so JupyText was not needed.

@Shizoqua
Lanre Shittu (Shizoqua) force-pushed the claude/wizardly-brahmagupta-5obhxm-video-scorer branch from f8084c5 to 9bd0f3b Compare October 19, 2026 22:28
Wait for off-loop audio extraction before propagating cancellation, then remove generated audio without deleting the source. Cover single and repeated cancellation with regression tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@richlundeen
Richard Lundeen (richlundeen) added this pull request to the merge queue Oct 25, 2026
Merged via the queue into microsoft:main with commit 4d08c74 Oct 25, 2026
65 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: Video scorers block the event loop while decoding frames and audio

2 participants