Skip to content

BUG: get_all_harm_metrics finds nothing for SEXUAL_CONTENT and REPRESENTATIONAL #3542

Open

Description

The sexual and representational Likert scorers (and the Azure sexual category) use the canonical names SEXUAL_CONTENT and REPRESENTATIONAL as their harm_category. find_harm_metrics_by_eval_hash maps those to sexual_metrics.jsonl and representational_metrics.jsonl, but get_all_harm_metrics just builds "{harm_category}_metrics.jsonl", so it looks for files that don't exist and returns an empty list.

from pyrit.score import get_all_harm_metrics
len(get_all_harm_metrics("SEXUAL_CONTENT"))    # 0
len(get_all_harm_metrics("sexual"))            # 5
len(get_all_harm_metrics("REPRESENTATIONAL"))  # 0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions