Repository navigation
BUG: OpenAIChatTarget, OpenAICompletionTarget, and LiteLLMChatTarget do not validate presence_penalty or frequency_penalty #3498
Description
Describe the bug
OpenAIChatTarget, OpenAICompletionTarget, and LiteLLMChatTarget document both presence_penalty and frequency_penalty as numbers between -2.0 and 2.0 (the valid range defined by OpenAI and LiteLLM):
presence_penalty (float): Number between -2.0 and 2.0. Positive values penalize new tokens...
frequency_penalty (float): Number between -2.0 and 2.0. Positive values penalize new tokens...However, unlike temperature and top_p which are validated at construction time via validate_temperature() and validate_top_p(), neither presence_penalty nor frequency_penalty is validated. Passing an out-of-bounds value (e.g. presence_penalty=5.0 or frequency_penalty=-4.0), nan, or inf constructs the target without error, records the invalid parameter into the target's ComponentIdentifier, and only fails later when requests hit the provider API.
Steps/Code to Reproduce
from pyrit.prompt_target import OpenAIChatTarget
# Constructs without error despite invalid penalty values:
target = OpenAIChatTarget(
model_name=gpt-4o,
endpoint=https://test.openai.com,
api_key=test,
presence_penalty=5.0,
frequency_penalty=-4.0,
)
print(target.get_identifier().params[presence_penalty]) # 5.0Expected Results
Target construction should raise a PyritException if presence_penalty or frequency_penalty is not between -2.0 and 2.0 (inclusive), matching the existing behavior of validate_temperature and validate_top_p.
Proposed Fix
- Add
validate_presence_penaltyandvalidate_frequency_penaltyhelpers topyrit.prompt_target.common.utils(validating that the value is between -2.0 and 2.0 inclusive when not None). - Call both validation helpers in the constructors of
OpenAIChatTarget,OpenAICompletionTarget, andLiteLLMChatTarget. - Add unit tests for both helpers and target constructors.
Activity
its-amann commentedon Oct 24, 2026
Hi! I'd like to work on this. Roman Lutz (@romanlutz), could you assign it to me? I'll follow up with a PR that includes a fix and tests.
nithin42 commentedon Oct 25, 2026
Hey Aman (@its-amann) and maintainers (Roman Lutz (@romanlutz)),
I had already drafted and tested this fix locally when filing the issue and was just waiting on initial maintainer feedback before opening a PR. Since the branch and tests were already complete, I went ahead and submitted #3530 to save everyone time.
Happy to have you review it or collaborate if there is anything you would like to tweak!
romanlutz commentedon Oct 26, 2026
In an ideal world, I would have preferred an indication on the issue for whether you want to fix this yourself Nithin (@nithin42) . Aman (@its-amann) jumped in to help. Since your PR was first and you acknowledged Aman (@its-amann) 's work and added him as a contributor I'm going to go ahead and close Aman (@its-amann) 's PR. Thank you both. For next time, Aman (@its-amann) you did everything right. Feel free to jump on any other issue.
nithin42 commentedon Oct 26, 2026
Thanks for the heads up Roman Lutz (@romanlutz), totally fair point. Next time I will add a quick note in the issue that I am working on the PR so there is no confusion. Appreciate you taking a look at #3530!