iamnotpalak/personapath-mobilenetv2-emotion-fer2013

PersonaPath Emotion Detection (MobileNetV2 - FER2013)

Overview

This model is part of the PersonaPath project — a multi-modal AI system for assessing speaking skills such as confidence, pronunciation, and emotional expression.

This repository contains a fine-tuned MobileNetV2 model trained on the FER-2013 dataset for facial emotion classification.

Task

• Problem Type: Multi-class classification
• Input: Facial images (224 × 224 RGB)
• Output: 7 emotion classes

Classes:

• Angry
• Disgust
• Fear
• Happy
• Sad
• Surprise
• Neutral

---

Model Details

• Base Model: MobileNetV2 (pretrained on ImageNet)
• Framework: TensorFlow / Keras
• Strategy: Transfer Learning + Fine-Tuning

Fine-Tuning Approach

• Initial training with frozen base layers (~52% validation accuracy)

• Progressive unfreezing of top layers:

  • Top 30 layers → ~60.66% accuracy
  • Top 50 layers → further improvement (ongoing experimentation)

• Lower layers retained for general feature extraction

• Higher layers adapted for emotion-specific patterns

---

Dataset

• FER-2013
• Source: Hugging Face
• 35,000+ grayscale facial images

Preprocessing

• Resized to 224×224
• Converted to RGB
• Data augmentation:
  • Horizontal flip
  • Rotation (±10°)
  • Zoom (±10%)

Challenge

• Severe class imbalance (e.g., Disgust class underrepresented)
• Handled using class weighting

---

Training Details

• Loss: Sparse Categorical Crossentropy

• Optimizer: Adam

  • Base training: lr = 1e-4
  • Fine-tuning: lr = 1e-5

• Callbacks:

  • EarlyStopping
  • ReduceLROnPlateau
  • ModelCheckpoint

---

Results

Phase	Validation Accuracy
Frozen Base	~52%
Fine-Tuned (Top 30)	~60.66%
Fine-Tuned (Top 50)	Ongoing

---

Usage

from tensorflow.keras.models import load_model
import cv2
import numpy as np

model = load_model("path_to_model")

img = cv2.imread("image.jpg")
img = cv2.resize(img, (224, 224))
img = img / 255.0
img = np.expand_dims(img, axis=0)

pred = model.predict(img)

Role in PersonaPath

This model is integrated into Phase 2 of the PersonaPath system.

• Extract frames from input video at 3 FPS
• Perform emotion classification on each frame
• Aggregate predictions across frames to estimate an overall confidence score

---

Limitations

• Moderate performance (~60% validation accuracy), affected by:
  • Class imbalance in FER-2013
  • Low-resolution facial images
• Generalization may drop in real-world conditions (lighting, pose variations)
• Emotion predictions are used as a proxy for confidence, not a direct measurement

---

Future Improvements

• Explore deeper fine-tuning or alternative architectures
• Train on higher-quality datasets (e.g., RAF-DB, AffectNet)
• Integrate with audio-based features (e.g., LSTM on MFCC) for multi-modal confidence estimation