Skip to content

Read turbopuffer query result ids from the raw response - #879

Closed
owenhalpert wants to merge 1 commit into
zilliztech:mainfrom
owenhalpert:turbopuffer-raw-response
Closed

owenhalpert wants to merge 1 commit into
zilliztech:mainfrom
owenhalpert:turbopuffer-raw-response

Conversation

@owenhalpert

@owenhalpert owenhalpert commented Sep 21, 2026 •

Copy link
Copy Markdown

The turbopuffer client builds a pydantic model per returned row, but only the ids are used. Interleaved A/B on the same queries, the model construction is about 28% of client CPU per query at top_k=100 (1.30ms -> 0.94ms), rising to 47% at top_k=1000.

Reading the ids off the raw response instead: 2,910 -> 3,700 QPS on Cohere 1M (768d), recall unchanged at 0.9045. No new dependencies, loads is already imported.

Marking as draft while I confirm there are no edge cases with the multitenant and payload profile paths.

@sre-ci-robot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: owenhalpert
To complete the pull request process, please assign xuanyang-cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @xuanyang-cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@owenhalpert

Copy link
Copy Markdown
Author

Closing this — my benchmark numbers were not measured rigorously enough. Re-running as a proper alternating A/B shows a much smaller difference than the PR claimed. Will revisit if it holds up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants