Repository navigation
BUG: red_team_social_bias multi-turn prompts have no role, so their seed groups fail validation #3555
Description
Describe the bug
_RedTeamSocialBiasDataset (pyrit/datasets/seed_datasets/remote/red_team_social_bias_dataset.py) turns each "Multi Turn" row into one SeedPrompt per user turn, sharing a prompt_group_id, with sequence=0, 1, 2, ..., but it does not set their role. SeedGroup requires a role on each sequence of a group with more than one sequence (_enforce_consistent_role), so every multi-turn group fails validation:
ValueError: No roles set for sequence 0 in a multi-sequence group. Please ensure at least one prompt within a sequence has an assigned role.
Fetching the dataset works, but anything that builds its seed groups raises: SeedDataset.seed_groups, and memory.get_seed_groups_async(dataset_name="red_team_social_bias") after the dataset is added to memory. Since one bad group makes the whole call fail, the single-prompt rows of the dataset cannot be loaded as seed groups either. The upstream dataset has 111 "Multi Turn" rows, each with two or more user turns (the turns are named system_1, user_2, assistant_3, user_4, ...).
The other remote loaders that build multi-sequence groups (for example cyberseceval_prompt_injection_dataset.py and mask_dataset.py) set role on each prompt.
Steps/Code to Reproduce
import asyncio
from unittest.mock import AsyncMock, patch
from pyrit.datasets.seed_datasets.remote.red_team_social_bias_dataset import _RedTeamSocialBiasDataset
from pyrit.memory import CentralMemory
from pyrit.setup import IN_MEMORY, initialize_pyrit_async
# One "Multi Turn" row in the format of svannie678/red_team_repo_social_bias_prompts
rows = [
{
"prompt": "[{'role': 'system_1', 'body': 'You are a helpful assistant.'}, "
"{'role': 'user_2', 'body': 'First turn'}, {'role': 'assistant_3', 'body': 'Response'}, "
"{'role': 'user_4', 'body': 'Second turn'}]",
"prompt_type": "Multi Turn",
"categorization": "discrimination",
"organization": "org",
}
]
async def main():
await initialize_pyrit_async(memory_db_type=IN_MEMORY)
loader = _RedTeamSocialBiasDataset()
with patch.object(loader, "_fetch_from_huggingface_async", new=AsyncMock(return_value=rows)):
dataset = await loader.fetch_dataset_async()
print("roles:", [seed.role for seed in dataset.seeds])
memory = CentralMemory.get_memory_instance()
await memory.add_seed_datasets_to_memory_async(datasets=[dataset], added_by="repro")
groups = await memory.get_seed_groups_async(dataset_name="red_team_social_bias")
print("groups:", [[(p.role, p.value) for p in g.prompts] for g in groups])
asyncio.run(main())Expected Results
roles: ['user', 'user']
groups: [[('user', 'First turn'), ('user', 'Second turn')]]
Actual Results
roles: [None, None]
Traceback (most recent call last):
...
pydantic_core._pydantic_core.ValidationError: 1 validation error for SeedGroup
Value error, No roles set for sequence 0 in a multi-sequence group. Please ensure at least one prompt within a sequence has an assigned role. [type=value_error, ...]
Screenshots
N/A
Versions
- OS: Windows 11
- Python version: 3.11.9
- PyRIT version: installed from main (d2c6fff) in editable mode
Activity