Vector DB Engine

A FastAPI service that lets users create, read, update, and delete document libraries and perform k-Nearest Neighbor vector search. Packaged as a Docker image and deployed via Helm into Kubernetes.

Table of Contents

1. Prerequisites
2. Local Development
3. Docker
4. Kubernetes + Helm Deployment
5. Rotating the API Key
6. Cleanup
7. In-Memory Store & Concurrency
8. Pydantic Models & Validation
9. Service Layer
10. Search Algorithms
11. Error Handling & HTTP Semantics
12. Trade-offs & Future Work

---

Prerequisites

• Git
• Python 3.11
• Docker & Docker Desktop (with Kubernetes enabled)
• kubectl CLI (configured to talk to your local cluster)
• Helm 3

Verify your setup:

python3 --version
docker --version
kubectl version --client
helm version

Local Development

1. Clone & set up a virtual environment

   git clone https://github.com/jballo/vector-db-engine.git
   cd vector-db-engine
   python3 -m venv .venv
   source .venv/bin/activate
   pip install --upgrade pip
   pip install -r requirements.txt

2. Set your API key

   export API_KEY=<uuigen generated key>
   export COHERE_KEY=<cohere key from platform>

3. Run the FastAPI app locally

   uvicorn app.main:app --reload --port 8000

   In another shell:

   curl -H "X-Key: $API_KEY" http://localhost:8000/health
   # → {"status":"ok"}

   Browse the OpenAPI docs at http://localhost:8000/docs

Docker

1. Build the Docker image and push

   export IMAGE=yourdockerid/vector-db-engine
   export TAG=0.1.0
   
   docker build --pull --rm \
   -t $IMAGE:$TAG \
   -t $IMAGE:latest \
   .

   docker push $IMAGE:$TAG
   docker push $IMAGE:latest

2. Run the container

   docker run --rm \
   -e API_KEY=$API_KEY \
   -e COHERE_KEY=$COHERE_KEY \
   -p 8000:8000 \
   $IMAGE:$TAG

   Test

   export REAL_API_KEY=<api key generated by uuigen>
   curl -X GET "http://localhost:8000/health" \
   -H "X-Key: $REAL_API_KEY"
   # → {"status":"ok"}

Kubernetes + Helm Deployment

We inject the API key at runtime via a pre-created Kubernetes Secret.

1. Create the namespace

   kubectl create namespace vector-db-engine \
   --dry-run=client -o yaml \
   | kubectl apply -f -

2. Create or update the API key Secret

   export REAL_API_KEY=<api key generated by uuigen>
   export COHERE_API_KEY=<cohere_key>
   kubectl create secret generic vector-db-engine-secret \
   --from-literal=API_KEY="$REAL_API_KEY" \
   --from-literal=COHERE_KEY="$COHERE_API_KEY" \
   -n vector-db-engine \
   --dry-run=client -o yaml \
   | kubectl apply -f -

3. Deploy with Helm

   export TAG=0.1.0
   
   helm upgrade --install vector-db-engine \
   ./vector-db-engine-chart \
   --namespace vector-db-engine \
   --set image.repository=yourdockerid/vector-db-engine \
   --set image.tag=$TAG \
   --set existingSecret=true

4. Access the service In one terminal:

   kubectl port-forward svc/vector-db-engine 8000:80 \
   -n vector-db-engine

   In another:

   export REAL_API_KEY=<api key generated by uuigen>
   curl -X GET "http://localhost:8000/health" \
   -H "X-Key: $REAL_API_KEY"
       # → {"status":"ok"}

Rotating the API Key

1. Update the Secret

   export REAL_API_KEY=<api key generated by uuigen>
   export COHERE_API_KEY=<cohere_key>
   kubectl create secret generic vector-db-engine-secret \
   --from-literal=API_KEY="$REAL_API_KEY" \
   --from-literal=COHERE_KEY="$COHERE_API_KEY" \
   -n vector-db-engine \
   --dry-run=client -o yaml \
   | kubectl apply -f -

2. Restart your Pods

   kubectl rollout restart deployment/vector-db-engine \
   -n vector-db-engine

Project Structure and Architecture

app/
├── routers/         # HTTP layer: one APIRouter per resource
├── services/        # Business logic: CRUD, search orchestration
├── store/           # In-memory data store with thread-safety
├── utils/           # Pure-algorithm code (kNN)
└── models/          # Pydantic schemas (Base/Create/Update/Read)

+-------------------------------------------------------------+
|                        Client (User)                        |
+--------------------------+----------------------------------+
                           |
                           v
+-------------------------------------------------------------+
|                        FastAPI Routers                      |
|  - /libraries                                               |
|  - /{library_id}/documents                                  |
|  - /{document_id}/chunks                                    |
|  - /health                                                  |
+--------------------------+----------------------------------+
                           |
                           v
+-------------------------------------------------------------+
|                      Service Layer                          |
|  - library_service.py                                       |
|  - document_service.py                                      |
|  - chunk_service.py                                         |
|  - search_service.py                                        |
|  (Business logic, validation, orchestration)                |
+--------------------------+----------------------------------+
                           |
                           v
+-------------------------------------------------------------+
|                    In-Memory Store (store/in_memory.py)     |
|  - _libraries: Dict[UUID, Library]                          |
|  - _documents: Dict[UUID, Document]                         |
|  - _chunks: Dict[UUID, Chunk]                               |
|  - Thread-safe with RLock                                   |
+--------------------------+----------------------------------+
                           |
                           v
+-------------------------------------------------------------+
|                        Models (Pydantic)                    |
|  - Library, LibraryCreate, LibraryUpdate                    |
|  - Document, DocumentCreate, DocumentUpdate                 |
|  - Chunk, ChunkCreate, ChunkUpdate                          |
|  - SearchRequest, SearchResult                              |
+--------------------------+----------------------------------+
                           |
                           v
+-------------------------------------------------------------+
|                        Utils (utils/knn.py)                 |
|  - Brute-force kNN search                                   |
|  - VP-Tree search                                           |
|  - Distance metrics (cosine, L2)                            |
+--------------------------+----------------------------------+
                           |
                           v
+-------------------------------------------------------------+
|                  External Embedding Service                 |
|                  (Cohere API via cohere.Client)             |
+-------------------------------------------------------------+


Data Flow:
1. Client sends HTTP request (CRUD/search).
2. FastAPI router parses/validates, calls service.
3. Service layer enforces invariants, calls store/utils.
4. Store manages in-memory state (thread-safe).
5. For search, utils/knn.py runs kNN on chunk embeddings.
6. For embedding, Cohere API is called as needed.

• Routers only parse/validate and call services.
• Services enforce invariants, call store & utils.
• Store manages raw state under a single RLock.
• Utils hold algorithms that can be swapped out (e.g. brute-force ↔ ball-tree).

In-Memory Store & Concurrency

• Single Python process with store/in_memory.py.
• All write operations (save_, attach_, remove_*) wrapped in a module-level RLock.
• Atomic multi-step operations (e.g. save+attach) exposed as single store methods.
• Trade-off: simple PoC, but a real deployment needs a persistent, distributed store.

Pydantic Models & Validation

• Base / Create / Update / Read variants per resource:

  • LibraryBase → LibraryCreate → Library (with id)
  • DocumentBase → DocumentCreate → Document (with id, library_id)
  • ChunkBase → ChunkCreate → Chunk (with id, library_id, document_id)

• orm_mode = True on Read models for easy ORM integration later.

• conlist(float, min_items=1) to enforce non-empty embeddings.

• New TextSearchRequest for routes that accept raw text and embed it server-side.

Service Layer

• Keeps routers thin: service functions return/accept Pydantic models directly.
• .model_dump(exclude_none=True) + .model_copy(update=…) for partial updates.
• Raises KeyError on missing resources → routers map to 404 Not Found.

Search Algorithms

• Brute-Force kNN (utils/knn.py):

  • Score all candidates, full sort, slice top-k.
  • Complexity: O(N·d + N log N). Great for N ≲ 10 000.

• VP-Tree (Ball-Tree):

  • build_vptree partitions points by median radius around a vantage point.
  • vptree_knn prunes branches whose balls cannot improve the top-k.
  • Build: O(N log N). Query: ≈ O(log N) for moderate dims (d ≲ 200).

• Algorithm Dispatch in _run_knn: clients choose "brute" or "vptree" via SearchRequest.algorithm.

Error Handling & HTTP Semantics

• 404 for missing libraries/documents/chunks.
• 422 for invalid UUIDs, missing required fields.
• Clear use of status_code, response_model, and HTTPException.

Trade-offs & Future Work

• In-memory store → trivial but ephemeral. Next step: Redis or SQL (SQLAlchemy).
• Single-process → simple locking, but limited scale. Future: leader-follower or sharding.
• Search → brute-force then VP-Tree; later introduction of approximate methods (HNSW) for massive dims.
• Embedding → synchronous calls to Cohere; consider async / batched / cached strategies for high throughput