An AI-powered customer support triage system built with LangGraph and Gemini. It classifies incoming customer queries, retrieves relevant knowledge, assesses risk, auto-resolves low-risk cases, and escalates the rest to a human manager through a real approval workflow backed by Jira.
A customer query comes in through the Support Operations page. The system:
- Understands the query — classifies category, extracts intent (Gemini).
- Retrieves knowledge — searches a ChromaDB knowledge base for relevant, verified answers.
- Assesses risk and sentiment — flags payment/security-sensitive or angry-customer cases.
- Decides — a deterministic Supervisor (rule-based, not free-form LLM) decides whether to auto-resolve or escalate, based on
config/risk_policy.yaml. - Checks for duplicates — avoids creating a new ticket for an issue that's already being tracked.
- Investigates root cause — for escalated technical issues, generates advisory hypotheses (never presented as confirmed fact).
- Escalates to Jira — creates a real Jira Cloud ticket (or runs in DEMO MODE if Jira isn't configured).
- Pauses for human review — a manager approves or rejects the case in the Manager Review page. Rejections loop back for another round of review rather than dead-ending.
- Responds to the customer — a clean, customer-safe response is generated (internal
APPROVE:/REJECT:review markers are stripped before the customer ever sees it). - Feeds back into the knowledge base — manager-approved resolutions become a pending knowledge entry, which only becomes searchable, verified knowledge after a separate admin approves it in the Knowledge Approval page.
Every case is persisted (LangGraph SQLite checkpointing), so a case can be reopened, resumed, or reviewed at any time — not just within the browser tab that created it.
The workflow is a directed graph of nodes (see graph/workflow.py, graph/routing.py):
query_understanding → rag ─┐
→ risk_priority ─┤→ supervisor ─┬→ auto_resolve → knowledge_feedback → END
→ sentiment ─────┘ └→ duplicate_check ─┬→ jira
└→ root_cause → jira
↓
human_review ⇄ (loops on reject)
↓
response → knowledge_feedback → END
- Deterministic Supervisor (
agents/supervisor.py): routing rules come fromconfig/risk_policy.yaml, not an LLM making the call itself — payment/security categories, high risk, or low confidence always escalate, regardless of what the LLM "feels." - Human-in-the-loop: implemented with LangGraph's real
interrupt()/resume_case(), persisted to SQLite — a paused case survives an app restart. - Knowledge feedback gate: a manager's phrasing of a fix never becomes "verified company knowledge" on its own. It's queued as pending, and a second, separate approval step (a different person, in principle) promotes it into ChromaDB as
verified=true.
| Agent | Responsibility |
|---|---|
query_understanding.py |
Category + intent extraction (Gemini structured output) |
risk_priority.py, sentiment.py |
Risk level and customer sentiment analysis |
supervisor.py |
Deterministic auto-resolve vs. escalate decision |
auto_resolution.py |
Grounded response generation for auto-resolved cases |
jira_escalation.py |
Category-to-team routing, Jira payload construction |
duplicate_detection.py |
Semantic duplicate detection against past cases |
root_cause.py |
Advisory root-cause hypotheses (never confirmed fact) |
knowledge_feedback.py |
Pending → approved knowledge base entry lifecycle |
case_registry.py |
Merges the audit log + live LangGraph state into a manager-facing case queue |
| Page | File | Role | Access |
|---|---|---|---|
| Support Operations | streamlit_app.py |
Support agent | Password-gated (SUPPORT_TEAM_PASSWORD) |
| Manager Review | pages/manager_review.py |
Manager | Password-gated (MANAGER_PASSWORD) |
| Knowledge Approval | pages/knowledge_approval.py |
Admin | Open (add a gate here if needed for your deployment) |
| Overview / Dashboard | pages/dashboard.py |
Support agent or Manager | Password-gated (either password) |
| Customer Portal | pages/customer_portal.py |
Customer | Open — the Case ID itself is the access token, like an order-tracking link |
Shared theming lives in ui_theme.py (fonts, pill/badge system, layout — no emoji, professional B2B look).
- ChromaDB (
chroma_db/) — knowledge base, embedded via Gemini's embedding API (not a local ONNX model, to avoid a large download). - SQLite (
support_ops_checkpoints.db) — LangGraph workflow checkpoints (case state, pause/resume). - Audit log (
data/logs/audit_log.jsonl) — append-only, structured business events (CASE_CREATED,DECISION_MADE,JIRA_CREATED, etc.) used by the Dashboard and Manager Review's case queue. knowledge_data/pending_entries.json— the pending → approved/rejected knowledge entry queue.
| Component | Library | Version |
|---|---|---|
| LLM orchestration | langgraph |
1.2.11 |
| LangGraph SQLite checkpointing | langgraph-checkpoint-sqlite |
3.1.1 |
| LLM framework | langchain, langchain-google-genai |
1.3.18 / 4.3.7 |
| LLM + embeddings | google-genai (Gemini) |
2.20.0 |
| Vector store | chromadb |
1.5.9 |
| Issue tracking | jira |
3.10.5 |
| UI | streamlit |
1.62.0 |
| Config / validation | pydantic, pydantic-settings, PyYAML |
2.13.5 / 2.7.1 / 6.0.2 |
| Testing | pytest, pytest-mock |
8.3.4 / 3.14.0 |
Python 3.11 required.
cd support-ops-agent
python -m venv .venv
(Set-ExecutionPolicy -Scope Process -ExecutionPolicy RemoteSigned) ; (& .venv\Scripts\Activate.ps1)pip install -r requirements.txtCopy .env.example to .env and fill in your values:
Copy-Item .env.example .env| Variable | Required? | Notes |
|---|---|---|
GOOGLE_API_KEY |
Yes | Free tier key at https://aistudio.google.com/apikey. ~20 requests/day on the free tier — prefer running the test suite (which mocks Gemini) over repeated real end-to-end runs. |
GEMINI_MODEL |
Yes | Defaults to gemini-3.6-flash. |
JIRA_BASE_URL, JIRA_EMAIL, JIRA_API_TOKEN, JIRA_PROJECT_KEY |
No | Leave blank to run Jira escalation in DEMO MODE (mocked responses, clearly labeled). All four must be set for real Jira Cloud integration — a partially-configured Jira is never treated as "connected." |
CHROMA_PERSIST_DIR, CHROMA_COLLECTION_NAME |
No | Defaults provided; local persistent ChromaDB store. |
SQLITE_DB_PATH |
No | Default provided. |
LOG_LEVEL, APP_ENV |
No | Defaults to INFO / development. |
SUPPORT_TEAM_PASSWORD |
Yes, for internal pages | Gates Support Operations and (together with MANAGER_PASSWORD) the Dashboard. |
MANAGER_PASSWORD |
Yes, for internal pages | Gates Manager Review and (together with SUPPORT_TEAM_PASSWORD) the Dashboard. |
Internal pages fail closed: if a password isn't set in
.env, that page blocks access entirely rather than silently letting everyone in.
python -m pytest -vTests use tests/conftest.py, which redirects the audit log and checkpoint DB to a temporary directory per test — the test suite never touches production data/logs/audit_log.jsonl or support_ops_checkpoints.db.
streamlit run streamlit_app.pyStreamlit's multi-page sidebar will show all four pages: streamlit app (Support Operations), Manager Review, Knowledge Approval, Dashboard, and Customer Portal.
Auto-resolved case: On Support Operations, submit a low-risk query (e.g. an account FAQ question). The Supervisor should auto-resolve it directly, with a generated, grounded response.
Escalated case → manager approval: Submit a query mentioning a payment or security issue. The Supervisor escalates it, a Jira ticket is created, and the case pauses at human_review. Open Manager Review, find the case in the queue (or by Case ID), and Approve or Reject it. A rejection loops the case back for another round; an approval generates the final customer-facing response.
Knowledge feedback loop: After approving a case in Manager Review, open Knowledge Approval. The manager's resolution appears as a pending entry. Approving it here ingests it into ChromaDB as verified knowledge, immediately retrievable by the RAG agent for future similar queries.
Customer status check: Open Customer Portal (no login required) and look up a Case ID to see its status (RESOLVED / IN REVIEW) without any internal details (risk level, Jira ticket, manager notes) exposed.
Dashboard: Sign in with either the support team or manager password to see aggregate case volume, auto-resolve vs. escalate breakdown, category and risk distribution, and duplicate/Jira stats — all computed live from the audit trail.
- Authentication is a simple shared-password gate, not a real auth system — no individual accounts, no password hashing, no session persistence across browser restarts. Sufficient for an internal demo/prototype, not for production use.
- Gemini free-tier quota is limited (~20 requests/day on
gemini-3.6-flash). Automated tests mock the LLM; real end-to-end runs should be used sparingly. - Audit log / checkpoint DB may still contain development test data (case IDs like
HITL-*,case-phase*) from earlier testing before per-test isolation (conftest.py) was added. This can inflate Dashboard figures and add noise to the Manager Review queue until manually cleaned up. - Root-cause hypotheses are advisory only — they are LLM-generated possibilities for a human to investigate, never confirmed facts, and are labeled as such in the UI.
- No automated evaluation suite yet for classification/escalation/duplicate-detection accuracy against a labeled dataset.
- No Docker packaging yet — the app runs directly via a local Python virtual environment.
- Knowledge Approval page is not password-gated in the current version — add a gate (see
auth.py) if deploying this beyond a local demo.
support-ops-agent/
├── agents/ # LangGraph node implementations (one concern each)
├── config/ # risk_policy.yaml, routing_rules.yaml
├── graph/ # LangGraph state, workflow graph, routing logic
├── jira_client/ # Jira Cloud REST client + escalation payload builder
├── knowledge_data/ # Seed docs + pending knowledge entries
├── pages/ # Manager Review, Knowledge Approval, Dashboard, Customer Portal
├── rag/ # ChromaDB client, ingestion, retrieval
├── tests/ # pytest suite (conftest.py isolates test data)
├── utils/ # settings, logging/audit, errors, LLM helpers
├── auth.py # Shared password-gate helper for internal pages
├── ui_theme.py # Shared Streamlit theme/CSS across all pages
├── streamlit_app.py # Support Operations page (app entry point)
├── requirements.txt
└── .env.example