---
license: apache-2.0
base_model: allenai/Olmo-3-1025-7B
language:
- en
library_name: transformers
---


This repo is a collection of SFT checkpoints produced by sweeping unique training samples vs epochs, following the setup from the paper:

Data Repetition Beats Data Scaling in Long-CoT Supervised Fine-Tuning
https://arxiv.org/abs/2602.11149



### Default model

The repo root contains the weights and config for the default variant trained with 16 epochs on 800 samples.
Calling `from_pretrained(repo_id)` loads this checkpoint.

### Variants

Each subfolder follows:

s{N}_e{M}

where:
- s{N} means N unique samples
- e{M} means M epochs

Example names:
- s3200_e8 means 3200 unique samples trained for 8 epochs
- s12800_e1 means 12800 unique samples trained for 1 epoch

## How to load

Load the default model (root):

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "dakopi/olmo3-7b_data-repetition"

tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)
```

Load a specific variant (subfolder):

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "dakopi/olmo3-7b_data-repetition"
variant = "s6400_e4"

tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder=variant)
model = AutoModelForCausalLM.from_pretrained(repo_id, subfolder=variant)
```


## Reproducibility and code

Official training and evaluation code:
https://github.com/dkopi/data-repetition

## Citation

```
@misc{kopiczko2026datarepetitionbeatsdata,
  title        = {Data Repetition Beats Data Scaling in Long-CoT Supervised Fine-Tuning},
  author       = {Dawid J. Kopiczko and Sagar Vaze and Tijmen Blankevoort and Yuki M. Asano},
  year         = {2026},
  eprint       = {2602.11149},
  archivePrefix= {arXiv},
  primaryClass = {cs.CL},
  url          = {https://arxiv.org/abs/2602.11149}
}
```