MSG1999/m25csa011-transformer-english-hindi

🚀 English → Hindi Transformer

---

📌 Overview

This model implements a Transformer-based Neural Machine Translation (NMT) system for English → Hindi translation using PyTorch.

Optimized using Ray Tune + Optuna + ASHA, achieving high BLEU score with reduced training time.

---

🏗️ Model Architecture

• Encoder–Decoder Transformer
• 6 Encoder + 6 Decoder layers
• Multi-head attention
• Feed-forward network
• Positional encoding
• Residual connections + LayerNorm

Configuration

• d_model: 512
• num_heads: 4
• num_layers: 6
• d_ff: 4096
• dropout: 0.054

---

⚙️ Training Details

• Dataset: ~13,186 English-Hindi sentence pairs
• Optimizer: AdamW
• Loss: CrossEntropy (ignore padding)
• Scheduler: CosineAnnealingLR
• Device: GPU

---

📊 Results

Metric	Baseline	Tuned
Epochs	100	30
Time	129.42 min	79.31 min
Loss	0.0972	0.0959
BLEU	68.02	90.38

---

⚙️ Best Hyperparameters

• LR: 0.0001009
• Batch size: 64
• Heads: 4
• d_ff: 4096
• Dropout: 0.054
• Weight decay: 0.000261

---

🧪 Sample Outputs

• EN: I love you HI: मैं तुमसे प्यार करता हूँ

• EN: What is your name? HI: आपका नाम क्या है?

• EN: How are you? HI: आप कैसे हैं?

---

📂 Files

• M25CSA011_ass_4_best_model.pth
• en_vocab.pkl
• hi_vocab.pkl
• best_config.json

---

👤 Author

Mahek Shankesh Gadiya M.Tech AI – IIT Jodhpur

---

📚 Assignment

Transformer Optimization using Ray Tune + Optuna