You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
AzureContentFilterScorer only sets evaluation_file_mapping inside evaluate_async. A scorer you just built for one category has no mapping, so get_scorer_metrics() returns None even though the registry has its entry with a matching eval hash.
scorer=AzureContentFilterScorer(harm_categories=[TextCategory.HATE])
scorer.evaluation_file_mapping# Nonescorer.get_scorer_metrics() # None, but hate_speech_metrics.jsonl has this eval hash
Same for violence, sexual and self harm. Setting the mapping in the constructor for single-category scorers fixes it, like the Likert scorers already do.
Activity