Multilingual Document Categorization & Tagging System

An end-to-end NLP system that reads documents in English, French, Spanish, or German, classifies them into one of 7 topic categories, and extracts context-aware tags (people, organizations, places, dates, key concepts) - in real time, with a live monitoring dashboard.

83.95% test accuracy across 4 languages · 15 docs/sec on CPU · 66.6 ms avg latency · 4× model compression via quantization

Highlights

• Transfer learning with TensorFlow - fine-tuned distilbert-base-multilingual-cased (pre-trained on 104 languages) with a custom 7-class head. A single model classifies all four languages; no per-language classifiers needed.
• Context-aware tagging with SpaCy - automatic language detection routes each document to a language-specific NER model. Tags fuse named entities with lemmatized keyword extraction (hybrid ML + rule-based approach), correctly handling different NER label schemes across languages (OntoNotes PERSON/GPE vs WikiNER PER/LOC).
• Real-time micro-batching pipeline - documents are cleaned, classified, and tagged in batches, amortizing per-call overhead. Benchmarked end-to-end on the full held-out test set.
• Model optimization - dynamic-range quantization (TFLite) compresses the model from 541 MB → 136 MB with negligible accuracy impact; efficient architecture choices throughout (DistilBERT over BERT, 128-token sequences, dynamic padding).
• Production-minded engineering - reproducible seeds, leak-free data handling (pre-defined splits, test set touched once), graceful error handling for empty inputs and unsupported languages, best-validation checkpointing, single-source-of-truth label mappings.

Results

Metric	Value
Test accuracy (7 classes, 4 languages, 816 docs)	83.95%
Throughput (end-to-end pipeline, laptop CPU)	15.0 docs/sec
Average latency per document	66.6 ms
Language detection accuracy	100% (816/816)
Model size (quantized)	541 MB → 136 MB

Per-language accuracy - en 86.8% · de 85.3% · es 82.4% · fr 81.4%

• tightly clustered, demonstrating genuine cross-lingual transfer rather than English-only performance.

Error analysis shows misclassifications concentrate between semantically adjacent categories (science/technology ↔ travel, travel ↔ entertainment), i.e. the model's mistakes are plausible boundary cases, not noise.

Architecture

                          ┌─────────────────────┐
 incoming documents ──►   │  text_preprocessing │  clean + normalize (NFC)
                          └──────────┬──────────┘
                                     │
                          ┌──────────▼──────────┐
                          │  language detection │  langdetect (seeded)
                          └──────────┬──────────┘
                            ┌────────┴────────┐
                 ┌──────────▼─────┐   ┌───────▼────────────┐
                 │ TextClassifier │   │  DocumentTagger    │
                 │ TF DistilBERT  │   │  SpaCy en/fr/es/de │
                 │ (batched)      │   │  NER + keywords    │
                 └──────────┬─────┘   └───────┬────────────┘
                            └────────┬────────┘
                          ┌──────────▼──────────┐
                          │  DocumentPipeline   │  merged results
                          └──────────┬──────────┘
                                     │
                          ┌──────────▼──────────┐
                          │ Streamlit dashboard │  live · simulation · metrics
                          └─────────────────────┘

document-categorization/
├── data/
│   ├── raw_documents/          # raw SIB-200 CSVs (generated)
│   └── processed_data/         # cleaned train/val/test CSVs (generated)
├── models/
│   ├── text_classifier.py      # TF inference engine + TFLite quantization
│   ├── tagger.py               # multi-language SpaCy NER/keyword tagger
│   ├── pipeline.py             # real-time pipeline + benchmark
│   └── saved/                  # fine-tuned model (generated, not committed)
├── utils/
│   ├── data_loader.py          # downloads SIB-200 (4 languages)
│   ├── text_preprocessing.py   # cleaning, label encoding, language detection
│   └── transfer_learning.py    # fine-tunes DistilBERT (TensorFlow/Keras)
├── app/
│   └── real_time_dashboard.py  # Streamlit dashboard
├── docs/screenshots/
├── README.md
└── requirements.txt

Dashboard

Three tabs:

1. Live processing - paste any document; see category, confidence, detected language, tags, contributing entities (interpretability), and latency.
2. Simulation - stream batches of test documents through the pipeline; watch category/language/tag distributions build in real time.
3. Model performance - benchmark stats, per-language accuracy computed live on the test set, and a misclassification table for error analysis.

Dataset

SIB-200 topic classification benchmark, languages eng_Latn, fra_Latn, spa_Latn, deu_Latn. Categories: entertainment, geography, health, politics, science/technology, sports, travel. The dataset's pre-defined train/validation/test splits are preserved end-to-end (2,804 / 396 / 816 documents after preprocessing) - no re-splitting, no leakage.

Setup

python -m venv venv
source venv/Scripts/activate        # Windows Git Bash
pip install -r requirements.txt

# SpaCy language models (direct URLs - reliable on all networks)
pip install https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.7.1/en_core_web_sm-3.7.1-py3-none-any.whl
pip install https://github.com/explosion/spacy-models/releases/download/fr_core_news_sm-3.7.0/fr_core_news_sm-3.7.0-py3-none-any.whl
pip install https://github.com/explosion/spacy-models/releases/download/es_core_news_sm-3.7.0/es_core_news_sm-3.7.0-py3-none-any.whl
pip install https://github.com/explosion/spacy-models/releases/download/de_core_news_sm-3.7.0/de_core_news_sm-3.7.0-py3-none-any.whl

Usage

# 1. Download the dataset
python utils/data_loader.py

# 2. Preprocess (writes train/validation/test CSVs)
python utils/text_preprocessing.py

# 3. Fine-tune the classifier (~75 min on CPU, 3 epochs,
#    best-validation checkpointing)
python -m utils.transfer_learning

# 4. (optional) Inference smoke test + build the quantized TFLite model
python -m models.text_classifier

# 5. (optional) Tagger smoke test
python -m models.tagger

# 6. Benchmark the full pipeline on the held-out test set
python -m models.pipeline

# 7. Launch the dashboard
streamlit run app/real_time_dashboard.py

Engineering Notes

• Error handling - empty/invalid documents return category: unknown with empty tags; unsupported or undetectable languages skip tagging gracefully and surface a dashboard warning; missing data/model files raise actionable messages naming the command to run first.
• Reproducibility - fixed seeds for random, numpy, TensorFlow, and langdetect; pinned dependency versions; deterministic preprocessing (pipeline benchmark accuracy matches training-time evaluation to four decimal places).
• Known limitations & upgrade paths - small SpaCy models (_sm) trade some NER precision for speed (_md/_trf are drop-in upgrades); 128-token truncation suits SIB-200's sentence-length documents (raise MAX_LENGTH for longer texts); streaming is simulated with micro-batches - a production deployment would place a message queue (e.g., Kafka) in front of the same pipeline.

Tech Stack

TensorFlow/Keras · Hugging Face Transformers (DistilBERT) · SpaCy · langdetect · pandas / NumPy · scikit-learn · Streamlit · TFLite