File size: 1,787 Bytes
8fe7bd6 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 | ---
tags:
- pytorch
- mnist
- image-classification
- computer-vision
- knowledge-distillation
library_name: pytorch
---
# MNIST Distilled Student Model
A neural network trained on the MNIST dataset using knowledge distillation from a teacher model.
## Model Description
This is a StudentNet model trained on MNIST using knowledge distillation with the following architecture:
- Fully connected: 28 × 28 → 128 → 10 (output)
- ReLU activation
The model was trained using knowledge distillation, combining:
- KL divergence between student and teacher logits (with temperature scaling)
- Cross-entropy loss on true labels
## Training Details
### Training Hyperparameters
- **Batch size**: 128
- **Epochs**: 10
- **Learning rate**: 0.001
- **Weight decay**: 0.0
- **Optimizer**: AdamW
- **Training set size**: 50,000
- **Validation set size**: 10,000
- **Test set size**: 10,000
- **Device**: cuda
- **Seed**: 42
### Distillation Parameters
- **Temperature**: 3.0
- **Alpha (KL weight)**: 0.5
The loss function is: `loss = alpha × KL_loss + (1 - alpha) × CE_loss`
### Results
- **Test Accuracy**: 0.9785
- **Test Loss**: 0.0808
## Usage
```python
import torch
from pathlib import Path
# Download the model
model_path = "model.pt"
state_dict = torch.load(model_path)
# Load into your StudentNet architecture
# (you'll need to define the StudentNet class from the training script)
model = StudentNet()
model.load_state_dict(state_dict)
model.eval()
# Make predictions
with torch.no_grad():
predictions = model(images)
```
## Dataset
The model was trained on the [MNIST dataset](http://yann.lecun.com/exdb/mnist/), which contains 70,000 grayscale images of handwritten digits (0-9), each 28x28 pixels.
## Model Card Authors
Generated automatically during training.
|