Local-first performance engineering for telemetry analysis, AI-assisted workload design, runnable load-test generation, real engine execution, bottleneck detection, root-cause analysis, and grounded optimization recommendations.
This project is designed to run locally with open-source software and locally hosted AI models. No commercial AI API key, hosted database, hosted vector database, or paid SaaS service is required.
- Overview
- Architecture
- Technology
- Repository Layout
- Requirements
- Quick Start
- Windows Setup
- Configuration
- Local AI
- Demo Workflow
- API Overview
- Telemetry Analysis
- AI and RAG
- Test Generation and Execution
- Database
- Testing
- Security
- Status and Limitations
- Troubleshooting
The platform supports this engineering workflow:
- Register and create a performance-testing project.
- Upload CSV, JSON, or JSONL telemetry.
- Normalize request, latency, resource, database, and error data.
- Calculate throughput, averages, weighted percentiles, error rate, and saturation indicators deterministically.
- Detect CPU, database, API, and connection-pool bottlenecks with evidence.
- Generate a validated peak-load scenario through Ollama.
- Generate and persist runnable k6 JavaScript or JMeter
.jmxartifacts. - Execute a configured local k6 or JMeter binary without shell execution.
- Capture status, stdout, stderr, cancellation, and normalized test metrics.
- Stream execution updates through Server-Sent Events.
- Compare baseline and candidate execution metrics.
- Ingest technical documents, create local embeddings, retrieve relevant chunks, and generate grounded recommendations.
- Run a structured root-cause workflow over deterministic evidence and retrieved knowledge.
Quantitative analysis never depends on an LLM. AI explains and extends measured evidence; it does not invent measurements.
flowchart LR
Browser[React dashboard] --> API[FastAPI API]
API --> Postgres[(PostgreSQL)]
API --> Redis[(Redis)]
API --> Ollama[Ollama]
API --> Chroma[(ChromaDB service)]
API --> Engine[k6 or JMeter]
API --> Analysis[Deterministic analysis]
API --> Agents[Structured AI workflows]
Agents --> Ollama
Agents --> Knowledge[Local knowledge retrieval]
PostgreSQL is the system of record. Redis is provisioned for future worker and streaming scale-out. Ollama is accessed through the LLMProvider abstraction; agents never call a commercial provider directly.
| Area | Technology |
|---|---|
| Frontend | React, TypeScript, Vite, React Router, CSS variables |
| API | Python 3.12+, FastAPI, Pydantic |
| Persistence | PostgreSQL, SQLAlchemy async, Alembic |
| Authentication | JWT, Argon2 password hashing |
| Local LLM | Ollama, default qwen3:8b |
| Local embeddings | Ollama, default nomic-embed-text |
| Agent workflow | LangGraph-compatible root-cause workflow |
| Load testing | k6 and Apache JMeter |
| Infrastructure | Docker Compose, Redis, ChromaDB service |
.
├── backend/
│ ├── app/
│ │ ├── api/ # HTTP routes and authorization boundaries
│ │ ├── core/ # settings, database, security
│ │ ├── models/ # SQLAlchemy entities
│ │ ├── providers/ # Ollama and LLM provider abstraction
│ │ ├── schemas/ # Pydantic API and AI contracts
│ │ ├── services/ # analysis, RAG, scripts, execution, comparison
│ │ └── main.py
│ ├── migrations/ # Alembic revisions
│ ├── tests/ # backend tests
│ └── requirements.txt
├── frontend/
│ ├── src/ # React application, API client, styles
│ └── package.json
├── data/demo/ # baseline, peak, stress, and mixed telemetry
├── scripts/ # local setup scripts
├── docker-compose.yml
├── .env.example
└── README.md
- Docker Desktop and Docker Compose
- Node.js 20 or newer
- Python 3.12 or newer
- 8 GB RAM minimum for CPU-only local development; 16 GB or more is recommended for
qwen3:8b - Additional storage for Ollama models
- Optional: k6 and/or JMeter installed locally for test execution
GPU acceleration is optional. Ollama supports CPU-only hardware.
From the repository root:
Copy-Item .env.example .env
docker compose up --buildIn a second terminal, download the default local models:
docker compose exec ollama ollama pull qwen3:8b
docker compose exec ollama ollama pull nomic-embed-textOpen:
- Frontend: http://localhost:5173
- FastAPI Swagger UI: http://localhost:8001/docs
- FastAPI OpenAPI JSON: http://localhost:8001/openapi.json
- Ollama: http://localhost:11434
- ChromaDB service: http://localhost:8000
Stop the stack:
docker compose downReset local database, model, and vector volumes:
docker compose down -vCopy-Item .env.example .env
.\scripts\setup.ps1The script starts infrastructure, pulls qwen3:8b and nomic-embed-text, and builds the application containers.
For direct development without Docker:
Set-Location backend
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
uvicorn app.main:app --reload --port 8001In another terminal:
Set-Location frontend
npm install
npm run devPostgreSQL and Ollama still need to be available for the complete workflow.
Copy .env.example to .env. Important settings:
| Variable | Purpose | Default |
|---|---|---|
DATABASE_URL |
Async PostgreSQL connection | Local PostgreSQL |
REDIS_URL |
Redis connection | redis://localhost:6379/0 |
OLLAMA_BASE_URL |
Ollama HTTP endpoint | http://localhost:11434 |
LLM_MODEL |
Local reasoning model | qwen3:8b |
EMBEDDING_MODEL |
Local embedding model | nomic-embed-text |
CHROMA_URL |
Local ChromaDB endpoint | http://localhost:8000 |
JWT_SECRET_KEY |
JWT signing secret | Change for non-local use |
K6_EXECUTABLE |
k6 executable name/path | k6 |
JMETER_EXECUTABLE |
JMeter executable name/path | jmeter |
CORS_ORIGINS |
Allowed frontend origins | http://localhost:5173 |
Never commit .env or API credentials.
The application communicates with Ollama over HTTP:
LLM_MODEL=qwen3:8b
EMBEDDING_MODEL=nomic-embed-text
OLLAMA_BASE_URL=http://localhost:11434
Models can be changed without source edits:
docker compose exec ollama ollama pull qwen3:14bSet LLM_MODEL=qwen3:14b in .env and restart the backend.
If Ollama is unavailable, deterministic telemetry analysis, project management, script generation, and engine execution remain available. AI-specific endpoints return a controlled error instead of silently using a paid provider.
- Start Docker Compose and open http://localhost:5173.
- Register an account.
- Create a project named
E-Commerce Demo. - Upload
data/demo/baseline-telemetry.csvand run analysis. - Upload
data/demo/peak-telemetry.csvand run analysis again. - Upload
data/demo/stress-telemetry.csvto expose stronger saturation. - Review P50/P95/P99 latency, throughput, error rate, CPU, database latency, and connection-pool evidence.
- Generate a scenario with Ollama.
- Generate a k6 script or JMeter plan.
- Execute the artifact if the selected engine is installed.
- Watch execution status through the dashboard or SSE endpoint.
- Compare two completed executions through the comparison API.
- Add PostgreSQL/JVM/Redis documentation to the knowledge base.
- Generate grounded recommendations and root-cause analysis.
The mixed fixture, data/demo/ecommerce-telemetry.csv, combines normal traffic with CPU saturation, database latency spikes, elevated errors, and pool saturation.
All /api routes except registration and login require a Bearer JWT.
POST /api/auth/register
POST /api/auth/login
GET /api/auth/me
GET /api/projects
POST /api/projects
GET /api/projects/{project_id}
PATCH /api/projects/{project_id}
DELETE /api/projects/{project_id}
POST /api/projects/{project_id}/telemetry/upload
GET /api/projects/{project_id}/telemetry
POST /api/projects/{project_id}/telemetry/analysis
GET /api/projects/{project_id}/telemetry/analysis/latest
POST /api/projects/{project_id}/telemetry/analysis/{analysis_id}/root-cause
Accepted uploads are .csv, .json, and .jsonl, limited to 10 MB. Supported normalized fields include timestamp, service, endpoint, request_count, latency_ms, status_code, cpu_percent, memory_percent, database_latency_ms, and connection_pool_percent.
POST /api/projects/{project_id}/scenarios/generate
POST /api/projects/{project_id}/tests/generate
POST /api/tests/{script_id}/execute
GET /api/tests/executions/{execution_id}
POST /api/tests/executions/{execution_id}/cancel
GET /api/tests/executions/{execution_id}/events
GET /api/tests/executions/{candidate_id}/compare/{baseline_id}
The SSE endpoint returns text/event-stream events containing execution status, normalized metrics, and recent output. The execution service uses asyncio.create_subprocess_exec; arbitrary shell commands are never accepted.
POST /api/projects/{project_id}/knowledge/documents
POST /api/projects/{project_id}/knowledge/query
POST /api/projects/{project_id}/recommendations/generate
Measurements are calculated in Python:
- Average, minimum, and maximum latency
- Weighted P50, P90, P95, and P99 latency
- Requests per second and throughput
- Successful and failed request counts
- Error rate
- CPU and memory averages
- Database latency and connection-pool utilization
- Deterministic bottleneck severity and confidence
Current rules include CPU saturation at 85% or higher, database latency at 150 ms or higher when API P95 is at least 500 ms, connection-pool utilization at 85% or higher, and error rate at 5% or higher.
The provider boundary in backend/app/providers/ exposes structured generation and embeddings. Scenario, recommendation, and root-cause outputs are validated with Pydantic and receive one bounded correction attempt when structured output is invalid.
Knowledge documents are chunked, embedded with Ollama, and stored with metadata and embeddings in PostgreSQL JSONB records. Retrieval uses cosine similarity, with lexical fallback when embeddings are unavailable. ChromaDB is included in Compose as a local vector-service option; the current persisted retrieval path does not require a hosted vector database.
The root-cause workflow consumes persisted deterministic metrics and bottleneck evidence. It must not recalculate or invent measurements.
Generated k6 scripts include HTTP endpoints, methods, virtual-user stages, ramp and duration settings, P95 thresholds, and error thresholds. Generated JMeter plans include thread groups, virtual users, ramp time, duration, and HTTP samplers.
Execution output is parsed into normalized metrics. Missing k6 or JMeter binaries fail explicitly with exit code 127; successful runs are never simulated.
The schema uses UUID primary keys and timestamp fields where appropriate:
users
projects
telemetry_records
performance_analyses
generated_test_scripts
test_executions
test_metrics
knowledge_documents
document_chunks
recommendations
Alembic revisions are in backend/migrations/versions/. Local startup also creates missing tables through SQLAlchemy metadata for developer convenience.
Backend tests:
Set-Location backend
python -m pytest -qCoverage includes password hashing, JWT behavior, deterministic analysis, bottleneck rules, structured AI contracts, k6/JMeter generation, k6/JMeter result parsing, RAG ranking, recommendation fallback, and execution comparison deltas.
Frontend build:
Set-Location frontend
npm install
npm run build- JWT-protected API routes
- Argon2 password hashing
- Project-level ownership checks
- Pydantic input validation
- 10 MB telemetry upload limit
- Supported file-extension validation
- No arbitrary shell command input
- Engine executable configuration through environment variables
- No committed secrets or required commercial API keys
- Configurable CORS
Do not expose the local development JWT secret or engine endpoints to an untrusted network without hardening deployment settings.
Implemented:
- Authentication, projects, authorization, PostgreSQL persistence, and migrations
- Telemetry ingestion, deterministic analysis, and multiple demo datasets
- Ollama provider abstraction and structured scenario generation
- k6 and JMeter artifact generation
- Real subprocess execution, cancellation, parsed metrics, and SSE updates
- Execution comparison API
- Local embeddings, knowledge retrieval, recommendations, and root-cause workflow
- Backend regression tests and frontend production build
Known limitations:
- The frontend is still centered around one dashboard rather than fully separated route-specific views for every product area.
- PostgreSQL JSONB embeddings are the active retrieval store; ChromaDB is provisioned but not the active storage adapter.
- Distributed worker execution through Redis/Celery is not enabled; local execution uses in-process asyncio tasks.
- Live Docker, k6, JMeter, Ollama, and ChromaDB smoke tests require those tools to be installed and running on the developer machine.
- Frontend component and browser integration coverage is smaller than backend unit coverage.
docker compose exec ollama ollama list
docker compose exec ollama ollama pull qwen3:8b
docker compose exec ollama ollama pull nomic-embed-textdocker compose ps
docker compose logs postgres
docker compose restart postgres backendInstall k6 or JMeter locally, or set executable paths in .env:
K6_EXECUTABLE=C:\path\to\k6.exe
JMETER_EXECUTABLE=C:\path\to\jmeter.bat
docker compose down -v
docker compose up --build