litellm: add abliteration.ai models to the model list - #575
Open
abliteration-ai wants to merge 1 commit into
Open
litellm: add abliteration.ai models to the model list#575abliteration-ai wants to merge 1 commit into
abliteration-ai wants to merge 1 commit into
Conversation
Add abliterated-model and abliterated-model-large to the litellm model_list, routed through the abliteration provider (merged upstream in BerriAI/litellm#18678). Keys are read from ABLIT_KEY and wired through the k8s secret, values templates, compose env template, and the local setup script, following the existing provider pattern.
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds two models from abliteration.ai to the litellm proxy model list:
abliterated-modelabliterated-model-largeAbliteration is an OpenAI-compatible API hosting less-restrictive LLMs, useful for security-research workloads where refusals get in the way of fuzzing/patching tasks.
How
Abliteration is a first-class litellm provider upstream (merged in BerriAI/litellm#18678), so no custom routing is needed — the entries use the standard
abliteration/<model>model string and litellm's built-in base URL (https://api.abliteration.ai/v1). The API key is read fromABLIT_KEY.Following the existing per-provider pattern, the key is also wired through:
deployment/k8s/values.yaml(mirrored model list)deployment/k8s/templates/litellm-secret.yamland thevalues-*.templatefilesdeployment/env.templateanddev/docker-compose/env.templatescripts/common.shinteractive setup (optional key, counts toward the at-least-one-LLM-key check)deployment/k8s/README.mddocsEntries carry conservative rate limits (tpm 2,000,000 / rpm 150, matching
gemini-pro).Compatibility note
The abliteration provider landed in litellm in January 2026. The pinned proxy images in this repo (
litellm_stable_release_branch-v1.57.8-stablein compose, litellm-helm chart 0.1.783 in k8s) predate that merge, so these entries will only resolve once the litellm image/chart is bumped to a release that includes the provider. Happy to include that bump here if you'd like.Validation
yaml.safe_loadonlitellm/litellm_config.yamlanddeployment/k8s/values.yaml; both model lists checked for uniquemodel_nameand well-formedlitellm_params(27 models each, 2 abliteration).values-*.templatefiles render throughenvsubstand parse as YAML.pre-commit runon all changed files passes (check-yaml, shellcheck, shfmt, etc.).