Skip to content

Apply an explicitly configured zero relevance threshold - #1996

Open
Afloat16 wants to merge 1 commit into
evidentlyai:mainfrom
Afloat16:fix/zero-relevance-threshold-20261112
Open

Afloat16 wants to merge 1 commit into
evidentlyai:mainfrom
Afloat16:fix/zero-relevance-threshold-20261112

Conversation

@Afloat16

Copy link
Copy Markdown

Problem

The shared recommendation preprocessing uses a truthiness check for min_rel_score. An explicitly configured zero threshold is skipped, so items with a zero rating are treated as irrelevant. This can also exclude users who should have relevant feedback and changes the resulting recommendation metrics.

Change

Apply the threshold whenever min_rel_score is not None, including zero. Preserve the existing >= comparison, optional binary conversion, and no-feedback-user policy. Other threshold values and the unconfigured behavior remain unchanged.

Validation

  • 20 new cases on the upstream base: 8 failed, 12 passed.
  • All recommendation metric tests with the fix: 58 passed on CPU.
  • Direct preprocessing cases cover None, negative, zero and positive thresholds with both binary/graded data and both no-feedback policies; input frames are unchanged.
  • Native current/reference Report cases check Precision, Recall, Hit Rate, MRR and NDCG against hand-calculated values for three users, with both scores and explicit ranks.
  • Repository-pinned Ruff 0.3.7 lint/format and git diff --check pass.

This corrects the handling of an explicit zero threshold in the existing recommendation preprocessing. It does not change ranking, tie handling, metric formulas, or the meaning of a missing threshold. The full repository and notebook smoke suites were not run.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant