I build ML systems end to end and measure them honestly: a Malayalam voice assistant (speech in, speech out), the ASR benchmark behind it, local LLM apps, and vision and IoT models running on edge hardware. PhD in Computational Biology · ML Engineer & Educator at ICT Academy of Kerala · writing at intuitivetutorial.com
Most Malayalam speakers will never have a GPU, so everything here is built and measured on 2 CPU cores.
🎙️ speech ─► ASR (my Whisper fine-tune / IndicConformer) ─► LangGraph agent + hybrid RAG ─► my VITS voice ─► 🔊 speech
Kerala Voice Helpdesk · LLM agent + RAG · 🎙️ Try it live
A voice assistant that answers Malayalam questions about panchayat services (K-SMART), the weather and the news.
- LangGraph agent with explicit routing, BM25 + BGE-M3 hybrid retrieval, and Gemma running locally through llama.cpp
- Reliability by design: fees are calculated in code, not by the LLM; the model must answer
UNSUREwhen the retrieved passage doesn't contain the answer, and that becomes a refusal; weather and news numbers go out through templates - Ingestion pipeline: downloads panchayat citizen charters, runs Malayalam OCR with Tesseract and turns them into ~600 passages with page citations, plus a checker that verifies knowledge-base entries against the source pages
- Error-driven: routing, retrieval and ASR-correction fixes come from reading real conversation logs, and are tested by an eval suite (routing, hit@3, refusals, 100-question answer eval, multi-turn conversations); a whole spoken turn takes 1.7–6.2 s on a shared CPU
malayalam-asr-eval · Benchmark
A reproducible, leakage-free benchmark of 6 Malayalam ASR models on accuracy and 2-core CPU speed.
- WER/CER with bootstrap 95% CIs, a column that excludes each model's own training corpora, RTF and peak RAM; one
transcribe()interface over Whisper, CTranslate2, IndicConformer and wav2vec2; CI-tested adapters - Findings: IndicConformer matches my Whisper fine-tune's accuracy at ~14× the speed; one model's 39.7% WER becomes 52.9% on corpora it didn't train on; a CTranslate2 run silently truncated transcripts at 224 tokens
- Choosing per clip between two recognisers with a character LM: 31.2% → 26.4% WER (significant, no retraining). I deployed it, then reverted it after measuring that it slowed the voice assistant
Also in the stack: my fine-tuned whisper-small-malayalam-v2 and its training corpus, the assistant's voice (mms-tts-malayalam-sajil, a VITS fine-tune), and IndicF5-TTS-Pipeline: zero-shot voice cloning in 11 Indian languages, sped up on CPU (EPSS step schedules, cached preprocessing, core pinning, bf16) from a real-time factor of ~83× down to single digits. Write-ups: leakage in my test set · a bug in my own evaluation
PyTorch Transformers Whisper CTranslate2 IndicConformer ONNX Runtime LangGraph llama.cpp BM25 BGE-M3 VITS Tesseract
| Project | What it shows |
|---|---|
| 🎓 AI Assignment Grader · video | Grades Jupyter notebooks against a rubric with a local LLM (Qwen2.5-Coder via Ollama): typed JSON output, score guardrails in Python, optional sandboxed code execution; Docker + GitHub Actions deploy to HF Spaces |
| 🔍 OmniSense | Semantic search inside videos: faster-whisper transcription → MiniLM embeddings → FAISS; click a hit and the player jumps to that moment |
| 💬 ICTSahayi | GPT-2 implemented in plain PyTorch and instruction fine-tuned on an FAQ, evaluated against the base model and a retrieval baseline. An honest result: fine-tuning learned the format, not the facts, which makes the case for RAG |
| ✍️ MiniGPT-JAX · demo | Decoder-only transformer built from scratch in JAX / Flax NNX and trained on 200k stories on a free GPU, with pre-tokenised, memory-mapped batches to remove the input bottleneck |
| Project | What it shows |
|---|---|
| 🚗 Dash-Cam Segmenter | DeepLabV3+ and U-Net for Indian road scenes (20 classes, mean IoU 0.659) on my own Kaggle dataset; video or YouTube URL in, segmented video out |
| 🌧️ Acoustic Rain Gauge | Estimates rainfall from sound (ICFOSS R&D team): Raspberry Pi data acquisition, LSTM on STFT features (R² 0.97), LoRaWAN uplink (LoraWANPi) and a live Grafana dashboard. Published at IEEE RAICS 2024 |
| ✋ Hand Gesture Recognition | Touch-free control of an information display: a 3D CNN trained on Jester (8 gestures) reads live webcam clips and drives the pages over Flask-SocketIO; ONNX export for lighter runtimes |
- ML & AI Knowledge Graph: an interactive D3 map of ML concepts, prerequisites and resources (repo)
- ML Visual Lab: interactive visual explainers for entropy, eigenvectors and PCA
| Area | Tools |
|---|---|
| LLMs & agents | LangGraph, Ollama, llama.cpp, Hugging Face Transformers, RAG (BM25, dense, hybrid), FAISS, structured JSON output |
| Speech | Whisper fine-tuning, CTranslate2 / faster-whisper, IndicConformer, wav2vec2, VITS and F5-TTS, WER/CER evaluation |
| Deep learning | PyTorch, JAX / Flax, TensorFlow / Keras, scikit-learn |
| Vision & edge | OpenCV, segmentation, 3D CNNs, Raspberry Pi, LoRaWAN |
| Shipping | Docker, Gradio, Flask, Hugging Face Spaces, GitHub Actions, pytest, Linux |
- Edge Analytics Enabled Acoustic Rain Gauge, IEEE RAICS 2024
- Active Noise Control, International Journal of Biomedical and Clinical Engineering
- Making the Malayalam voice assistant faster and more accurate on a shared CPU
- Measuring, not guessing: leakage-free test sets, confidence intervals, error analysis from real logs
- Open to Applied ML / Speech AI / GenAI engineering roles and to collaborations on Indic-language AI