An autonomous, real-time voice-driven technical interview platform powered by WebRTC, OpenAI Realtime API, Deepgram, and Google Gemini.
Traditional technical screening tools rely on static forms, asynchronous video recordings, or generic text chatbots that lack conversational depth.
AI Technical Interviewer conducts adaptive, low-latency spoken technical interviews in real time. By ingesting a candidate's GitHub profile prior to the call, the system tailors its questions directly to the candidate's real-world repositories and architectural decisions. Over the course of the call, it manages a bi-directional WebRTC audio channel with OpenAI's Realtime model, maintains real-time transcript synchronization, and produces a structured analytical score and feedback report using Google Gemini.
- Direct WebRTC Voice Streaming: Peer-to-peer audio connection between the browser and OpenAI Realtime API (
gpt-realtime) delivering sub-second conversational latency and natural voice turn-taking. - Dynamic Candidate Profiling: Automatically scrapes public GitHub repositories, metadata, and star counts to ground interview questions in the candidate's real technical background.
- Sideband Control & Session Sync: A dedicated WebSocket sideband synchronizes runtime instructions with the OpenAI session and captures synthesized assistant speech transcripts as they occur.
- Live User Transcription: Streams candidate microphone audio concurrently to Deepgram over WebSockets for fast and accurate live speech-to-text.
- Reactive Audio Visualizer: Uses the Web Audio API (
AnalyserNode) to compute RMS decibel levels in real time, driving an interactive animated voice orb. - Automated Structured Evaluation: Upon completion, the conversation transcript is analyzed by Google Gemini using strict Zod JSON schemas to yield numerical ratings (out of 10) and actionable qualitative feedback.
sequenceDiagram
autonumber
actor Candidate as Candidate (Browser)
participant Backend as Express Backend
participant GitHub as GitHub API
participant OpenAI as OpenAI Realtime API
participant Deepgram as Deepgram STT
participant DB as PostgreSQL (Prisma)
participant Gemini as Google Gemini
Note over Candidate,Backend: 1. Pre-Interview Profiling
Candidate->>Backend: Submit GitHub Profile URL
Backend->>GitHub: Scrape Repositories & Metadata
GitHub-->>Backend: Repository Data
Backend->>DB: Store Session & GitHub Context
Backend-->>Candidate: Return interviewId
Note over Candidate,OpenAI: 2. Real-Time Spoken Interview
Candidate->>Backend: Send WebRTC SDP Offer
Backend->>OpenAI: Forward SDP to Realtime Session Endpoint
OpenAI-->>Backend: Return SDP Answer & call_id
Backend->>OpenAI: Connect WebSocket Sideband (Inject Context)
Backend-->>Candidate: Return SDP Answer
Candidate<<-->>OpenAI: Direct Bi-Directional WebRTC Audio Stream
par User Transcription
Candidate->>Deepgram: Stream Mic Audio via WebSocket
Deepgram-->>Candidate: Live User Transcript
Candidate->>Backend: Post User Utterances
Backend->>DB: Save User Messages
and AI Transcript Capture
OpenAI->>Backend: Stream Assistant Events over Sideband
Backend->>DB: Save Assistant Messages
end
Note over Candidate,Gemini: 3. Post-Interview Evaluation
Candidate->>Backend: Request Results (interviewId)
Backend->>DB: Fetch Full Conversation Transcript
Backend->>Gemini: Run Evaluation (Zod JSON Schema)
Gemini-->>Backend: Structured Score & Feedback
Backend->>DB: Persist Score & Feedback (Status = Done)
Backend-->>Candidate: Return Score, Feedback & Transcript
| Layer | Technologies |
|---|---|
| Monorepo & Tooling | Turborepo, Bun, TypeScript |
| Frontend | React 19, Vite, Tailwind CSS, Radix UI, Lucide React, Sonner |
| Backend | Express, Bun Runtime, Prisma ORM, PostgreSQL |
| Voice & Real-Time | WebRTC, Web Audio API, OpenAI Realtime API, Deepgram WebSocket API |
| AI Evaluation | Google Gemini Flash (@google/genai), Zod |
ai-interviewer/
├── apps/
│ ├── backend/
│ │ ├── prisma/
│ │ │ └── schema.prisma # Database models (Interview, Message)
│ │ ├── scrapers/
│ │ │ └── github.ts # GitHub repo scraper with proxy support
│ │ ├── db.ts # Prisma client singleton
│ │ ├── index.ts # Express REST & SDP relay server
│ │ ├── result.ts # Gemini structured evaluation engine
│ │ ├── sideband.ts # OpenAI Realtime WebSocket sideband controller
│ │ └── types.ts # Request validation schemas
│ └── frontend/
│ └── src/
│ ├── components/
│ │ ├── Form.tsx # GitHub URL submission & session initiation
│ │ ├── Interview.tsx # WebRTC audio room & Deepgram transcription
│ │ ├── Result.tsx # Performance scorecard & transcript viewer
│ │ ├── VoiceOrb.tsx # Audio-reactive canvas/CSS visualizer
│ │ └── ui/ # Reusable UI component library
│ ├── App.tsx # Router configuration
│ └── lib/config.ts # Frontend runtime configuration
├── packages/
│ ├── eslint-config/ # Shared ESLint configs
│ ├── typescript-config/ # Shared TypeScript tsconfig bases
│ └── ui/ # Shared UI components
├── turbo.json # Turborepo task pipeline definition
└── package.json # Monorepo workspaces configuration
- Bun (v1.3+ recommended)
- PostgreSQL database instance
- API Keys:
- OpenAI API Key (with Realtime API access)
- Google Gemini API Key
- Deepgram API Key
-
Clone the repository:
git clone <your-repo-url> cd ai-interviewer
-
Install monorepo dependencies:
bun install
Create an .env file in apps/backend/.env:
# Database
DATABASE_URL="postgresql://user:password@localhost:5432/ai_interviewer?schema=public"
# AI & Speech Services
OPENAI_KEY="sk-..."
GEMINI_API_KEY="..."
# Scraper Proxy (Optional / Required if scraping rate limits apply)
PROXY_URL="http://proxy-host:port"Configure the backend URL in apps/frontend/src/lib/config.ts (defaults to http://localhost:3001):
export const BACKEND_URL = "http://localhost:3001";Run Prisma migrations from the backend workspace:
cd apps/backend
bunx prisma migrate dev --name init
cd ../..Start both frontend and backend concurrently via Turborepo:
bun dev- Frontend: Accessible at
http://localhost:3000(orhttp://localhost:5173) - Backend: Running on
http://localhost:3001
Ingests a candidate's GitHub profile, scrapes repository metadata, and creates a pre-interview record.
- Request Body:
{ "github": "https://github.com/username" } - Response:
{ "id": "uuid-v4-session-id" }
Exchanges WebRTC SDP with OpenAI Realtime API and initializes the sideband WebSocket.
- Headers:
Content-Type: application/sdp - Request Body: Raw SDP offer string from browser.
- Response: Raw SDP answer string from OpenAI.
Records a candidate's transcribed statement into the database.
- Request Body:
{ "message": "Transcribed speech text" }
Retrieves or triggers the Gemini evaluation for the completed interview session.
- Response:
{ "score": 8, "feedback": "Strong explanation of asynchronous systems, but could expand on database indexing...", "transcript": [ { "type": "Assistant", "content": "Can you walk me through how you designed the database schema for your project?", "createdAt": "2026-09-09T18:30:00Z" }, { "type": "User", "content": "In that repository, I used PostgreSQL with Prisma...", "createdAt": "2026-09-09T18:30:20Z" } ], "status": "Done" }
- Ephemeral client-token generation for Deepgram on the backend to avoid storing credentials in client state.
- Concurrency locking and background worker queues (e.g., BullMQ) for post-interview evaluations.
- Deep codebase parsing (analyzing
package.json, architecture diagrams, and commit histories rather than only top-level repo metadata). - Interactive live code execution environment and whiteboard integration during the interview.
This project is licensed under the MIT License.