---
license: apache-2.0
datasets:
- AutumnQiu/fer2013
language:
- en
metrics:
- accuracy
base_model:
- Synaptics/MobileNetV2
tags:
- emotion-detection
- mobilenetv2
- tranfer-learning
- personapath
---
# PersonaPath Emotion Detection (MobileNetV2 - FER2013)

## Overview
This model is part of the **PersonaPath** project — a multi-modal AI system for assessing speaking skills such as confidence, pronunciation, and emotional expression.

This repository contains a **fine-tuned MobileNetV2 model** trained on the FER-2013 dataset for **facial emotion classification**.
---
## Task
- **Problem Type:** Multi-class classification  
- **Input:** Facial images (224 × 224 RGB)  
- **Output:** 7 emotion classes  

**Classes:**
- Angry  
- Disgust  
- Fear  
- Happy  
- Sad  
- Surprise  
- Neutral  

---
## Model Details
- **Base Model:** MobileNetV2 (pretrained on ImageNet)  
- **Framework:** TensorFlow / Keras  
- **Strategy:** Transfer Learning + Fine-Tuning  

### Fine-Tuning Approach
- Initial training with frozen base layers (~52% validation accuracy)  
- Progressive unfreezing of top layers:
  - Top 30 layers → ~60.66% accuracy  
  - Top 50 layers → further improvement (ongoing experimentation)

- Lower layers retained for general feature extraction  
- Higher layers adapted for emotion-specific patterns  

---

## Dataset
- **FER-2013**
- Source: Hugging Face  
- 35,000+ grayscale facial images  

### Preprocessing
- Resized to **224×224**
- Converted to **RGB**
- Data augmentation:
  - Horizontal flip  
  - Rotation (±10°)  
  - Zoom (±10%)  

### Challenge
- Severe class imbalance (e.g., *Disgust* class underrepresented)  
- Handled using **class weighting**

---

## Training Details
- **Loss:** Sparse Categorical Crossentropy  
- **Optimizer:** Adam  
  - Base training: lr = 1e-4  
  - Fine-tuning: lr = 1e-5  

- **Callbacks:**
  - EarlyStopping  
  - ReduceLROnPlateau  
  - ModelCheckpoint  

---

## Results
| Phase                     | Validation Accuracy |
|--------------------------|-------------------|
| Frozen Base              | ~52%              |
| Fine-Tuned (Top 30)      | ~60.66%           |
| Fine-Tuned (Top 50)      | Ongoing           |

---

## Usage

```python
from tensorflow.keras.models import load_model
import cv2
import numpy as np

model = load_model("path_to_model")

img = cv2.imread("image.jpg")
img = cv2.resize(img, (224, 224))
img = img / 255.0
img = np.expand_dims(img, axis=0)

pred = model.predict(img)
```

##  Role in PersonaPath

This model is integrated into **Phase 2** of the PersonaPath system.

- Extract frames from input video at **3 FPS**
- Perform **emotion classification** on each frame
- Aggregate predictions across frames to estimate an overall **confidence score**

---

## Limitations

- Moderate performance (~60% validation accuracy), affected by:
  - Class imbalance in FER-2013
  - Low-resolution facial images
- Generalization may drop in real-world conditions (lighting, pose variations)
- Emotion predictions are used as a **proxy for confidence**, not a direct measurement

---

##  Future Improvements

- Explore deeper fine-tuning or alternative architectures
- Train on higher-quality datasets (e.g., RAF-DB, AffectNet)
- Integrate with audio-based features (e.g., **LSTM on MFCC**) for multi-modal confidence estimation