07_textClassification_pickle.py

#!/usr/bin/env python
# coding: utf-8

"""
# ════════════════════════════════════════════════════════════════════════════════
# PROJECT: TEXT CLASSIFICATION WITH SENTIMENT ANALYSIS
# ════════════════════════════════════════════════════════════════════════════════
#
# 1. EXECUTIVE SUMMARY
# ────────────────────────────────────────────────────────────────────────────────
# OBJECTIVE:
#   To build a complete text classification pipeline that predicts the sentiment 
#   (positive/negative) of movie reviews. The project demonstrates the full 
#   lifecycle of an NLP project: data loading, preprocessing, feature extraction 
#   (BOW + TF-IDF), model training (Logistic Regression), evaluation, and 
#   model persistence (saving via Pickle).
#
# PROBLEM STATEMENT & CONTEXT:
#   Sentiment analysis is a crucial task in NLP for understanding public opinion, 
#   customer feedback, and social media trends. The challenge is to convert 
#   unstructured text into a format that machine learning algorithms can understand 
#   and then classify it accurately.
#
# ALGORITHM OVERVIEW:
#   - Preprocessing: Regex-based cleaning and stopword removal.
#   - Vectorization: 
#       1. Bag-of-Words (CountVectorizer): Counts word occurrences.
#       2. TF-IDF (TfidfTransformer): Normalizes counts by their global rarity.
#   - Model: Logistic Regression (a robust linear classifier for high-dimensional text data).
#
# DATASET:
#   - Source: 'txt_sentoken' (likely the Cornell Movie Review Data).
#   - Structure: Folders for 'pos' and 'neg' reviews.
#   - Size: 2000 documents (1000 positive, 1000 negative).
#
# EXPECTED OUTCOMES:
#   - A trained model capable of classifying new text.
#   - Serialized model files (.pickle) for deployment.
#   - Evaluation metrics (Confusion Matrix, Classification Report).
#
# PREREQUISITES:
#   - Python 3.x
#   - Libraries: numpy, nltk, sklearn, pickle
#   - Data: 'txt_sentoken' directory must be present in the working directory.
#
# ────────────────────────────────────────────────────────────────────────────────
# TABLE OF CONTENTS
# ────────────────────────────────────────────────────────────────────────────────
# 1. Theoretical Background (Stopwords)
# 2. Setup & Library Imports
# 3. Data Loading
# 4. Data Preprocessing (Cleaning Pipeline)
# 5. Feature Engineering (BOW & TF-IDF)
# 6. Model Training (Logistic Regression)
# 7. Model Evaluation
# 8. Inference on New Data
# 9. Model Persistence (Pickling)
# ════════════════════════════════════════════════════════════════════════════════
"""

# ════════════════════════════════════════════════════════════════════════════════
# STEP 1: THEORETICAL BACKGROUND
# ════════════════════════════════════════════════════════════════════════════════

# WHAT: Explain the concept of Stopwords.
# WHY: Stopword removal is a key preprocessing step in this script.

# #  Text Classification (Stopwords)

# # #### Stopwords are commonly used words in a language—such as "the", "is", "in", "and"—that typically carry little meaningful information about the content or intent of a text. Their frequent occurrence means they often do not contribute to distinguishing between different categories in a classification task.

# # ### Why Remove Stopwords in Text Classification?
# # ##### Reduce Noise: Removing stopwords helps reduce noise in the data, allowing models to focus on the more informative words that are likely to determine the class of the text.

# # ##### Improve Efficiency: Filtering out stopwords reduces the number of features, leading to faster and more efficient computation, especially with large datasets.

# # ##### Enhance Model Performance: In many traditional text classification approaches (e.g., bag-of-words, TF-IDF), removing stopwords can improve accuracy by focusing on content words that carry more semantic meaning.

# # ### Typical Workflow
# # ##### The general preprocessing steps for text classification involving stopwords are:
 
# # ##### Lowercasing: Convert all text to lowercase to ensure uniformity.
 
# # ##### Tokenization: Split text into individual words or tokens.

# # ##### Stopword Removal: Filter out stopwords using predefined lists (e.g., from NLTK, SpaCy, or custom lists).

# # ##### Stemming/Lemmatization: Reduce words to their base or root form (optional, but common).

# # ##### Vectorization: Convert text into numerical features using methods like Bag-of-Words (BOW) or TF-IDF


# ════════════════════════════════════════════════════════════════════════════════
# STEP 2: SETUP & LIBRARY IMPORTS
# ════════════════════════════════════════════════════════════════════════════════

# WHAT: Import necessary libraries.
# WHY:
#   - 'numpy': For array manipulation.
#   - 're': For regular expressions (text cleaning).
#   - 'nltk.corpus': To get the list of stopwords.
#   - 'sklearn.datasets': To easily load data from a folder structure.

import numpy as np
import re
from nltk.corpus import stopwords
from sklearn.datasets import load_files


# ════════════════════════════════════════════════════════════════════════════════
# STEP 3: DATA LOADING
# ════════════════════════════════════════════════════════════════════════════════
# WHAT: Load the dataset from the disk.
# WHY: We need labeled data for supervised learning.
# FUNCTION: load_files checks for subdirectories (e.g., 'pos', 'neg') and assigns 
#           labels automatically (0 and 1).

# download dataset (Reviews=positive,negative)
reviews  = load_files('txt_sentoken')

# Display the loaded bunch object (contains data, target, target_names, etc.)
# reviews

# Split data (text content) and labels (0 or 1)
X = reviews.data
y = reviews.target

# Check the size of the dataset
# len(X)
# y.shape

# Inspect the first sample (raw bytes)
# X[0]


# ════════════════════════════════════════════════════════════════════════════════
# STEP 4: DATA PREPROCESSING (CLEANING PIPELINE)
# ════════════════════════════════════════════════════════════════════════════════
# WHAT: Clean the raw text data.
# WHY: Raw text contains noise (punctuation, special characters, inconsistent casing) 
#      that confuses the model.
#
# PIPELINE STEPS:
#   1. Remove non-word characters (keep only a-z, A-Z, 0-9).
#   2. Lowercase everything.
#   3. Remove single isolated letters (often artifacts of cleaning).
#   4. Remove 'b' prefix (artifact of byte-string conversion).
#   5. Normalize whitespace (remove extra spaces).

corpus = []  # Initialize an empty list to store the cleaned reviews

for i in range(0, len(X)):
    # Remove all non-word characters (everything except numbers and letters)
    review = re.sub(r'\W', ' ', str(X[i]))
    
    # Convert all characters to lowercase to ensure uniformity
    review = review.lower()
    
    # Remove single characters surrounded by spaces (isolated letters)
    review = re.sub(r'\s+[a-z]\s+', ' ', review)
    
    # Remove leading 'b' character (common in byte strings)
    review = re.sub(r'^b\s+', ' ', review)
    
    # Remove leading and trailing spaces
    review = re.sub(r'^\s+|\s+$', '', review)
    
    # Replace multiple spaces with a single space
    review = re.sub(r'\s+', ' ', review)
    
    # Append the cleaned review to the corpus
    corpus.append(review)

# Inspect a cleaned review
# review
# corpus


# ════════════════════════════════════════════════════════════════════════════════
# STEP 5: FEATURE ENGINEERING (BOW & TF-IDF)
# ════════════════════════════════════════════════════════════════════════════════
# WHAT: Convert text to numbers.
# APPROACH:
#   1. CountVectorizer (Bag-of-Words): Counts word frequencies.
#   2. TfidfTransformer: Re-weights counts based on document frequency.

from sklearn.feature_extraction.text import CountVectorizer

# Initialize CountVectorizer with specific parameters for text feature extraction
# PARAMETERS:
#   - max_features=1500: Keep only the top 1500 words (dimensionality reduction).
#   - min_df=3: Ignore words appearing in < 3 docs (too rare).
#   - max_df=0.6: Ignore words appearing in > 60% of docs (too common).
#   - stop_words: Remove standard English stopwords.
vectorizer = CountVectorizer(
    max_features=1500,               
    min_df=3,                       
    max_df=0.6,                     
    stop_words=stopwords.words('english')  
)

# Fit the CountVectorizer to the cleaned text corpus and transform the corpus into a feature matrix
X = vectorizer.fit_transform(corpus)

# Inspect the sparse matrix
# X
# X.toarray()
# X.shape

# --------------------------------------------------------------------------------
# TF-IDF TRANSFORMATION
# --------------------------------------------------------------------------------
# WHAT: Apply TF-IDF weighting.
# WHY: To penalize words that are frequent across all documents (less discriminative).

from sklearn.feature_extraction.text import TfidfTransformer

# The TfidfTransformer in scikit-learn is used to convert a matrix of raw term counts (such as those produced by CountVectorizer) into a matrix of TF-IDF features.
transformer = TfidfTransformer()

X = transformer.fit_transform(X)

# Inspect the TF-IDF matrix
# X
# X.toarray()


# ════════════════════════════════════════════════════════════════════════════════
# STEP 6: MODEL TRAINING (LOGISTIC REGRESSION)
# ════════════════════════════════════════════════════════════════════════════════
# WHAT: Train a classifier.
# WHY: To learn the relationship between features (words) and labels (sentiment).
# SPLIT: 80% Training, 20% Testing.

from sklearn.model_selection import train_test_split

X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.2,random_state=0)

# print(X_train)
# print(X_train.shape)
# X_test.shape

# use LogisticRegression 
from sklearn.linear_model import LogisticRegression

model = LogisticRegression()
model.fit(X_train,y_train)


# ════════════════════════════════════════════════════════════════════════════════
# STEP 7: MODEL EVALUATION
# ════════════════════════════════════════════════════════════════════════════════
# WHAT: Assess how well the model performs on unseen data.
# METRICS:
#   - Confusion Matrix: Shows True Positives, True Negatives, False Positives, False Negatives.
#   - Classification Report: Precision, Recall, F1-Score.

# prediction
y_pred = model.predict(X_test)

# y_pred
# y_test

# compare test vs predicted data 
from sklearn.metrics import confusion_matrix,classification_report

print("CONFUSION MATRIX:")
print(confusion_matrix(y_test,y_pred))

print("\nCLASSIFICATION REPORT:")
print(classification_report(y_test,y_pred))


# ════════════════════════════════════════════════════════════════════════════════
# STEP 8: INFERENCE ON NEW DATA
# ════════════════════════════════════════════════════════════════════════════════
# WHAT: Test the model with a custom sentence.
# WHY: To verify it works in a real-world scenario.
# CRITICAL STEP: Must use the SAME vectorizer and transformer fitted on training data.

# check model prediction level 
sample=['i am a good boy,im not sleep well']

# sample

# Transform new data using the fitted vectorizer
test = vectorizer.transform(sample)

# test
# test.toarray()

test = transformer.transform(test)

# test.toarray()

print("\nPREDICTION FOR SAMPLE:", sample)
print("PREDICTED CLASS:", model.predict(test))


# ════════════════════════════════════════════════════════════════════════════════
# STEP 9: MODEL PERSISTENCE (PICKLING)
# ════════════════════════════════════════════════════════════════════════════════
# WHAT: Save the trained components to disk.
# WHY: So we can reuse the model later without retraining (e.g., in a web app).
# COMPONENTS TO SAVE:
#   1. Model (The logic)
#   2. Vectorizer (The vocabulary)
#   3. Transformer (The IDF weights)

import pickle

# save files in database

# Open a file named 'model.pickle' in write-binary mode ('wb')
# This file will be used to save the trained model object
with open('model.pickle', 'wb') as f:
    # Serialize the 'model' object and write it to the file
    pickle.dump(model, f)

# Open a file named 'vectorizer.pickle' in write-binary mode ('wb')
# This file will be used to save the fitted vectorizer object
with open('vectorizer.pickle', 'wb') as f:
    # Serialize the 'vectorizer' object and write it to the file
    pickle.dump(vectorizer, f)

# Open a file named 'tfidf.pickle' in write-binary mode ('wb')
# This file will be used to save the fitted transformer object (e.g., a TF-IDF transformer)
with open('tfidf.pickle', 'wb') as f:
    # Serialize the 'transformer' object and write it to the file
    pickle.dump(transformer, f)


"""
# ════════════════════════════════════════════════════════════════════════════════
# FINAL SUMMARY & CONCLUSION
# ════════════════════════════════════════════════════════════════════════════════
#
# RECAP:
#   We successfully built a sentiment analysis model.
#   1. Loaded movie reviews.
#   2. Cleaned text (removed noise and stopwords).
#   3. Converted text to TF-IDF features (top 1500 words).
#   4. Trained a Logistic Regression classifier.
#   5. Evaluated performance (typically > 80% accuracy on this dataset).
#   6. Saved the model for future use.
#
# KEY TAKEAWAYS:
#   - Preprocessing is critical. The regex cleaning ensured quality input.
#   - TF-IDF is superior to simple counts as it handles common words better.
#   - Saving the vectorizer is as important as saving the model; without it, 
#     the model cannot understand new text.
#
# PRACTICAL APPLICATION:
#   - This 'model.pickle' can be loaded into a Flask/Streamlit app to classify 
#     user feedback in real-time.
#
# ════════════════════════════════════════════════════════════════════════════════
"""