Skip to content

fix: sort unscored retriever results last - #13090

Open
linhongyu510 wants to merge 1 commit into
deepset-ai:mainfrom
linhongyu510:fix/retriever-none-score-ordering
Open

linhongyu510 wants to merge 1 commit into
deepset-ai:mainfrom
linhongyu510:fix/retriever-none-score-ordering

Conversation

@linhongyu510

Copy link
Copy Markdown
Contributor

What this PR does

Fixes #13089.

MultiQueryEmbeddingRetriever, MultiQueryTextRetriever, and TextEmbeddingRetriever used doc.score or 0.0 when sorting results. That places score=None alongside zero and therefore above valid negative retrieval scores.

This PR follows Haystack's existing convention in _deduplicate_documents, DocumentJoiner, and AnswerJoiner: a missing score sorts as negative infinity, while every numeric score—including 0.0 and negative values—keeps its real ordering.

The change covers both sync and async execution paths for all three retriever wrappers. It does not modify the underlying retrievers or their scoring.

Why this matters

Negative retrieval scores are supported. In particular, unscaled BM25Okapi may return them. Before this fix, combining or wrapping such results could promote an unscored document above a genuinely scored match:

before: positive, unscored, zero, negative
after:  positive, zero, negative, unscored

Tests

Added paired sync/async regression tests for:

  • MultiQueryEmbeddingRetriever
  • MultiQueryTextRetriever
  • TextEmbeddingRetriever

Each test mixes positive, zero, negative, and missing scores.

Verification performed:

  • UV_EXCLUDE_NEWER=false hatch run test:unit <six affected test files>
    • 60 passed, 12 deselected
  • Six focused regression tests
    • all 6 fail on the pre-fix implementation
    • all 6 pass with this change
  • Targeted mypy check
    • no issues in 9 source files
  • Ruff check
    • all checks passed
  • git diff --check
    • clean

UV_EXCLUDE_NEWER=false was only needed locally because the repository's 24-hour supply-chain cutoff filtered the already-published build dependency while resolving against the current system date; no dependency or project configuration was changed.

Scope

  • No public API change.
  • No score normalization or score mutation.
  • Numeric ordering remains unchanged.
  • Only None moves after all numeric scores.

AI assistance

Developed with AI-assisted tooling. I reviewed and understood the final changes and independently ran the reproducer, counterfactual regression test, unit tests, lint, and type checks.

@linhongyu510
linhongyu510 requested a review from a team as a code owner October 2, 2026 16:54
@linhongyu510
linhongyu510 requested review from bogdankostic and removed request for a team October 2, 2026 16:54
@vercel

vercel Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

@linhongyu510 is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions github-actions Bot added topic:tests type:documentation Improvements on the docs labels Oct 2, 2026
@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  haystack/components/retrievers
  multi_query_embedding_retriever.py
  multi_query_text_retriever.py
  text_embedding_retriever.py
Project Total  

This report was generated by python-coverage-comment-action

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic:tests type:documentation Improvements on the docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Multi-query retrievers rank unscored documents above valid negative scores

1 participant