Skip to content

fix(m3): score a single pair on multiple devices - #1616

Open
michaelkillgta wants to merge 1 commit into
FlagOpen:masterfrom
michaelkillgta:m3-single-pair-multi-device
Open

michaelkillgta wants to merge 1 commit into
FlagOpen:masterfrom
michaelkillgta:m3-single-pair-multi-device

Conversation

@michaelkillgta

Copy link
Copy Markdown

Fixes #1615

BGEM3FlagModel.compute_score passed a single (query, passage) tuple straight to the multi-process pool when more than one device is set. The pool iterated over the tuple, the workers received bare strings and failed with IndexError, and the parent then waited on the output queue forever.

The multi-device path now wraps a single pair before dispatch and unwraps the result afterwards, so it returns one float per mode, the same as on a single device.

Tests: added tests/test_m3_compute_score_multi_device.py, which stubs the pool (no model download). The single-pair test fails on master and passes with this change, and the pair-list test passes on both. I also reproduced the original hang end to end with BAAI/bge-m3 on devices=["cpu", "cpu"].

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BGE-M3 compute_score hangs on a single pair with multiple devices

1 participant