# 🚀 English → Hindi Transformer

![Python](https://img.shields.io/badge/Python-3.10-blue)
![PyTorch](https://img.shields.io/badge/PyTorch-Deep%20Learning-red)
![Task](https://img.shields.io/badge/Task-Translation-green)
![BLEU](https://img.shields.io/badge/BLEU-90.38-brightgreen)
![Optimization](https://img.shields.io/badge/RayTune+Optuna-Used-orange)

---

## 📌 Overview

This model implements a **Transformer-based Neural Machine Translation (NMT)** system for English → Hindi translation using PyTorch.

Optimized using **Ray Tune + Optuna + ASHA**, achieving high BLEU score with reduced training time.

---

## 🏗️ Model Architecture

* Encoder–Decoder Transformer
* 6 Encoder + 6 Decoder layers
* Multi-head attention
* Feed-forward network
* Positional encoding
* Residual connections + LayerNorm

### Configuration

* d_model: 512
* num_heads: 4
* num_layers: 6
* d_ff: 4096
* dropout: 0.054

---

## ⚙️ Training Details

* Dataset: ~13,186 English-Hindi sentence pairs
* Optimizer: AdamW
* Loss: CrossEntropy (ignore padding)
* Scheduler: CosineAnnealingLR
* Device: GPU

---

## 📊 Results

| Metric | Baseline   | Tuned      |
| ------ | ---------- | ---------- |
| Epochs | 100        | 30         |
| Time   | 129.42 min | 79.31 min  |
| Loss   | 0.0972     | **0.0959** |
| BLEU   | 68.02      | **90.38**  |

---

## ⚙️ Best Hyperparameters

* LR: 0.0001009
* Batch size: 64
* Heads: 4
* d_ff: 4096
* Dropout: 0.054
* Weight decay: 0.000261

---

## 🧪 Sample Outputs

* EN: I love you
  HI: मैं तुमसे प्यार करता हूँ

* EN: What is your name?
  HI: आपका नाम क्या है?

* EN: How are you?
  HI: आप कैसे हैं?

---

## 📂 Files

* M25CSA011_ass_4_best_model.pth
* en_vocab.pkl
* hi_vocab.pkl
* best_config.json

---

## 👤 Author

Mahek Shankesh Gadiya
M.Tech AI – IIT Jodhpur

---

## 📚 Assignment

Transformer Optimization using Ray Tune + Optuna