---
tags:
- sft
- gsm8k
- math
- ml-intern
base_model: Qwen/Qwen2.5-0.5B
---

# Qwen2.5-0.5B GSM8K SFT

Supervised fine-tuned model for grade-school math reasoning on GSM8K.

## Results

| Model | GSM8K test exact-match accuracy | N eval |
|-------|-----------------------------------|--------|
| Base (Qwen/Qwen2.5-0.5B) | 0.0008 (1/1319) | 1319 |
| Tuned (pngwn/qwen2.5-0.5b-gsm8k-sft) | 0.3472 (458/1319) | 1319 |

## Training details

- **Dataset:** openai/gsm8k (main config)
- **Train split:** 7473 samples
- **Test split:** 1319 samples
- **Epochs:** 3
- **Learning rate:** 2e-5
- **Batch size:** 4 per device
- **Gradient accumulation:** 4
- **Max sequence length:** 1024
- **Decoding:** greedy (do_sample=False, max_new_tokens=256)
- **Answer extraction:** regex `####\s*(-?\d+(?:,\d+)*(?:\.\d+)?)`

## Eval script

The exact eval script used for both baseline and tuned evaluation is included in this repository as `eval_gsm8k.py`.

<!-- ml-intern-provenance -->
## Generated by ML Intern

This model repository was generated by [ML Intern](https://github.com/huggingface/ml-intern), an agent for machine learning research and development on the Hugging Face Hub.

- Try ML Intern: https://smolagents-ml-intern.hf.space
- Source code: https://github.com/huggingface/ml-intern

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = 'pngwn/qwen2.5-0.5b-gsm8k-sft'
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
```

For non-causal architectures, replace `AutoModelForCausalLM` with the appropriate `AutoModel` class.