You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
BUG: SeedGroup fails on seed prompts with metadata=None #3508
SeedPrompt.metadata is typed dict[str, Any] | None, so it can be None. SeedGroup passes it straight into MessagePiece(prompt_metadata=...), which only accepts a dict. So next_message, user_messages, prepended_conversation and AttackParameters.from_seed_group_async raise a ValidationError for those seeds.
This happens with real data. The PKU-SafeRLHF loader sets metadata=None for prompts that have no harm category. With the default settings, that's 15,827 of its 38,641 seeds.
Steps/Code to Reproduce
frompyrit.modelsimportSeedGroup, SeedPromptgroup=SeedGroup(seeds=[SeedPrompt(value="How do I bake bread?", data_type="text", metadata=None)])
print(group.next_message)
Expected Results
The message is built with empty prompt_metadata, the same as for a prompt that has no metadata set.
Actual Results
pydantic_core._pydantic_core.ValidationError: 1 validation error for MessagePiece
prompt_metadata
Input should be a valid dictionary [type=dict_type, input_value=None, input_type=NoneType]
Versions
OS: macOS 26.4.1 (arm64)
Python 3.13.16
PyRIT 1.2.0.dev0, main (6a23196), editable install
I have a one-line fix (prompt.metadata or {} where the message pieces are built) with a test. Happy to open a PR.
Activity