Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: eddy-aws The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
eddy-aws
force-pushed
the
add-dynamodb-vector-client
branch
2 times, most recently
from
October 6, 2026 03:46
4670917 to
a456e3b
Compare
Add a DynamoDB client that benchmarks DynamoDB's GA vector search feature
using the standard boto3 SDK. Validated end-to-end against a live DynamoDB
vector index in us-east-1 (VectorDBBench Performance1536D50K: recall 1.0,
max QPS 63.3, p99 79.8ms).
- create_table with VectorIndexes (VectorAttribute={AttributeName}, SearchSchema
list of HASH/INLINE_FILTER elements, DistanceFunction COSINE/EUCLIDEAN/
DOT_PRODUCT) + search_vectors
- default boto3 credential chain; optional explicit keys / session token
- batch_write_item loading (25-item batches); throttling handled by boto3
standard retry mode plus bounded exponential backoff with jitter on
UnprocessedItems re-drive, raising if the table cannot drain
- SearchSchema partition key (HASH): --partition-count > 1 defines a vector
index partition key, spreads vectors id%%N, and scopes each SearchVectors
call to ONE randomly chosen partition value (realistic single-partition
access; recall ~1/N vs whole-dataset ground truth)
- INLINE_FILTER equality label filter; NumGE not advertised (search '=' only)
- cosine / euclidean / dotproduct metrics; TopK capped at the API max of 100
- registered in the DB enum and CLI; 'dynamodb' extra (boto3>=1.40.0)
- offline unit tests for config, metrics, partition key, scoped search,
filters, throttling backoff, and CLI wiring
eddy-aws
force-pushed
the
add-dynamodb-vector-client
branch
from
October 6, 2026 03:51
a456e3b to
503bba4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an Amazon DynamoDB vector search client, benchmarking DynamoDB's GA
vector index feature (
CreateTablewithVectorIndexes+SearchVectors) viathe standard
boto3SDK. No external vector store is required — vectors live inthe DynamoDB table alongside operational data.
What's included
vectordb_bench/backend/clients/dynamodb/—dynamodb.py,config.py,cli.pyDBenum (init_cls/config_cls/case_config_cls) andthe
vectordbbenchCLIdynamodboptional-dependency extra (boto3>=1.40.0)tests/test_dynamodb.pyFeatures
boto3credential chain by default; optional explicitaccess key / secret / session token.
cosine/euclidean(L2) /dotproduct(IP), mapped to theCOSINE/EUCLIDEAN/DOT_PRODUCTdistance functions.batch_write_itemin 25-item batches with two layers of throttlingdefence — boto3
standardretry mode plus bounded exponential backoff (withfull jitter) on
UnprocessedItemsre-drive.--partition-count): when > 1, defines a SearchSchemaHASH(vector index partition key), spreads vectors uniformly acrosspartition values (
id % N), and scopes eachSearchVectorscall to onerandomly chosen partition value — the realistic single-partition access
pattern the feature is built for (one scoped call per query, not a
whole-index fan-out).
real-world partition keys are usually skewed, so the figures represent the
best-case even distribution rather than skewed production behaviour.
computes recall against whole-dataset ground truth, so recall is expected
to be ~1/N under partitioning; QPS/latency reflect the scoped access
pattern. Use
--partition-count 1(default) for a whole-index recallbenchmark with no partition key.
INLINE_FILTERSearchSchema element(
SearchConditionExpressionsupports only=, soNumGEis intentionally notadvertised).
TopKcapped at the API maximum of 100.Testing
Offline unit tests cover config/metric parsing, the partition-key SearchSchema,
single-partition-scoped search, filter translation, throttling backoff, and CLI
wiring.
Validated end-to-end against a live DynamoDB vector index in
us-east-1:20k synthetic vectors — search returns the correct nearest neighbours.
Real VectorDBBench run —
Performance1536D50K(OpenAI, 50k × 1536-dim,COSINE, k=10), no partition key:
Notes
BillingMode=PAY_PER_REQUEST.until results are returned (the benchmark's load→search ordering handles this).