SPAM DETECTION MACHINE LEARNING PROJECT

This is a spam detection project that classifies spam emails using Naive Bayes and Logistic Regression. The machine learning models are trained using two types of Naive Bayes (Multinomial Naive Bayes on Bag-of-Words features and Bernoulli Naive Bayes on presence/absence features) and Logistic Regression.

OVERVIEW

Email spam filtering is one of the earliest and most practical applications of text classification in machine learning. This project implements a spam detection system using three approaches:

1. Multinomial Naive Bayes
2. Bernoulli Naive Bayes
3. Logistic Regression (MCAP) with L2 regularization The system uses the Enron email dataset (enron1, enron2, enron4) and applies natural language preprocessing, feature extraction, and supervised classification. Models are evaluated on accuracy, precision, recall, and F1-score.

PROJECT STRUCTURE

spam_filter/ │── spam_filter.py # Main script (preprocessing, feature extraction, models, evaluation) │── README.md # Documentation │── enron1/ # Dataset folder (train/test/ham/spam) │── enron2/ │── enron4/ │── REPORT │── cs6375_project1_ai_transcript.pdf │── *.csv (12 csv files)

Each dataset follows the structure: enronX/ ├── train/ | ├── ham/ | └── spam/ └── test/ ├── ham/ └── spam/

CSV FILES

• enron1_bow_train.csv
• enron1_bow_test.csv
• enron1_bernoulli_train.csv
• enron1_bernoulli_test.csv
• enron2_bow_train.csv
• enron2_bow_test.csv
• enron2_bernoulli_train.csv
• enron2_bernoulli_test.csv
• enron4_bow_train.csv
• enron4_bow_test.csv
• enron4_bernoulli_train.csv
• enron4_bernoulli_test.csv

DEPENDENCIES

This project is written in Python 3.9+ and the requirements are: ● Python 3.9+ ● Python library numpy for numerical computing ● Python library nltk for text preprocessing (tokenization, stopwords) ● Python library os, re, and csv Installation: The following should be installed for executing the code:

• pip install numpy
• pip install nltk The code automatically downloads necessary NLTK data, but you can also run manually:
• import nltk
• nltk.download("punkt")
• nltk.download("stopwords")

HOW TO RUN THE PROJECT

Run Full Pipeline (Preprocessing + Feature Extraction + Logistic Regression) Execute the below command to run the code: python spam_filter.py This will:

1. Build a global vocabulary from all training datasets.
2. Generate Bag-of-Words (BoW) and Bernoulli features.
3. Train Multinomial Naive Bayes model
4. Train Logistic Regression models with MCAP (Using both BoW and Bernoulli).
5. Evaluate on enron1, enron2, enron4 test sets.

STEPS USED IN THIS PROJECT:

1. Preprocessing ● Convert text to lowercase ● Remove punctuation and non-alphabetic characters ● Tokenize text using NLTK ● Remove stopwords
2. Feature Extraction ● Bag-of-Words (BoW): counts frequency of each word ● Bernoulli: binary indicator of word presence
3. Dataset Preparation ● Build a global vocabulary from all datasets ● Save features in CSV files for reproducibility
4. Models Implemented ● Multinomial Naive Bayes: suitable for word counts ● Bernoulli Naive Bayes: suitable for binary word presence ● Logistic Regression (MCAP): with L2 regularization (λ tuned) ○ Bernoulli representation ○ Bag-of-Words representation
5. Training & Evaluation ● Models are trained separately on each dataset ● Logistic Regression is trained on combined datasets for robustness

METRICS USED

We evaluate models using: ● Accuracy: (TP + TN) / Total ● Precision: TP / (TP + FP) measures how many predicted spams were correct ● Recall (Sensitivity): TP / (TP + FN) measures how many actual spams were detected ● F1-Score: Harmonic mean of precision and recall

CUSTOMIZATION

• Change dataset_roots in spam_filter.py to add/remove datasets.
• Adjust learning rate (lr), regularization λ, or iterations in Logistic Regression.
• Try different smoothing α values in Naive Bayes.

CONCLUSION

This project demonstrates:

• How text preprocessing impacts machine learning models.
• Comparison between generative models (Naive Bayes) and discriminative models (Logistic Regression).
• Effectiveness of simple models in spam detection tasks.