Skip to content

Add Anthropic count_tokens endpoint - #630

Open
iLoveBug wants to merge 3 commits into
antirez:mainfrom
iLoveBug:codex/count-tokens-api
Open

Add Anthropic count_tokens endpoint#630
iLoveBug wants to merge 3 commits into
antirez:mainfrom
iLoveBug:codex/count-tokens-api

Conversation

@iLoveBug

Copy link
Copy Markdown

Summary

  • add the Anthropic-compatible POST /v1/messages/count_tokens endpoint
  • reuse the /v1/messages parser, model-family prompt renderer, and loaded model tokenizer without queueing generation
  • add a reusable integration test that compares count_tokens with total Anthropic input usage, including cache fields

Test plan

  • make -B ds4-server ds4_test
  • ./ds4_test --server
  • DeepSeek V4 Flash IQ2_XXS/Q2_K integration test with ctx=4096: 344 input tokens
  • DeepSeek V4 Flash IQ2_XXS/Q2_K integration test with ctx=32768: 344 input tokens

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant