README.md

---
tags:
- pytorch
- mnist
- image-classification
- computer-vision
- knowledge-distillation
library_name: pytorch
---

# MNIST Distilled Student Model

A neural network trained on the MNIST dataset using knowledge distillation from a teacher model.

## Model Description

This is a StudentNet model trained on MNIST using knowledge distillation with the following architecture:
- Fully connected: 28 × 28 → 128 → 10 (output)
- ReLU activation

The model was trained using knowledge distillation, combining:
- KL divergence between student and teacher logits (with temperature scaling)
- Cross-entropy loss on true labels

## Training Details

### Training Hyperparameters
- **Batch size**: 128
- **Epochs**: 10
- **Learning rate**: 0.001
- **Weight decay**: 0.0
- **Optimizer**: AdamW
- **Training set size**: 50,000
- **Validation set size**: 10,000
- **Test set size**: 10,000
- **Device**: cuda
- **Seed**: 42

### Distillation Parameters
- **Temperature**: 3.0
- **Alpha (KL weight)**: 0.5

The loss function is: `loss = alpha × KL_loss + (1 - alpha) × CE_loss`

### Results
- **Test Accuracy**: 0.9785
- **Test Loss**: 0.0808

## Usage

```python
import torch
from pathlib import Path

# Download the model
model_path = "model.pt"
state_dict = torch.load(model_path)

# Load into your StudentNet architecture
# (you'll need to define the StudentNet class from the training script)
model = StudentNet()
model.load_state_dict(state_dict)
model.eval()

# Make predictions
with torch.no_grad():
    predictions = model(images)
```

## Dataset

The model was trained on the [MNIST dataset](http://yann.lecun.com/exdb/mnist/), which contains 70,000 grayscale images of handwritten digits (0-9), each 28x28 pixels.

## Model Card Authors

Generated automatically during training.