README.md

---
base_model: openai-community/gpt2
library_name: peft
pipeline_tag: text-generation
license: apache-2.0
tags:
  - lora
  - peft
  - gpt2
  - math
  - dataset-curation
datasets:
  - kurtos-ai/gsm-gpt2-rff
---

# gsm-gpt2-rff

## Model Overview

This repository demonstrates that model-based data selection can significantly improve data efficiency under iso-compute training budgets. See the associated training dataset [gsm-gpt2-rff Dataset](https://huggingface.co/datasets/kurtos-ai/gsm-gpt2-rff).

This repository contains LoRA adapters for [`gpt2`](https://huggingface.co/openai-community/gpt2) trained on the `kurtos-ai/gsm-gpt2-rff` dataset family under multiple data-selection configurations (`gamma_0.01`, `gamma_0.02`, `gamma_0.05`, `gamma_0.1`, and `full`).

Each configuration includes two checkpoints:
- `*_best`: checkpoint selected by best validation loss during training.
- `*_last`: final checkpoint at the end of training.

All adapters are PEFT LoRA adapters and are intended to be loaded on top of the GPT-2 base model.

## Table of Available Adapters

All adapters have `gpt2` as base model and an adapter size of `3,253,104 bytes`.

| Adapter Path | Config | Checkpoint | Eval Loss | Eval Accuracy |
| --- | --- | --- | ---: | ---: |
| `gamma_0.01_best` | `gamma_0.01` | Best (by eval loss) | 5.8880 | 0.72% |
| `gamma_0.01_last` | `gamma_0.01` | Last | 6.3623 | 4.45% |
| `gamma_0.02_best` | `gamma_0.02` | Best (by eval loss) | 5.8692 | 1.6% |
| `gamma_0.02_last` | `gamma_0.02` | Last | 6.0615 | 3.25% |
| `gamma_0.05_best` | `gamma_0.05` | Best (by eval loss) | 5.8248 | 0.92% |
| `gamma_0.05_last` | `gamma_0.05` | Last | 5.8541 | 1.45% |
| `gamma_0.1_best` | `gamma_0.1` | Best (by eval loss) | 5.8283 | 1.01% |
| `gamma_0.1_last` | `gamma_0.1` | Last | 5.8506 | 1.43% |
| `full_best` | `full` | Best (by eval loss) | 6.1236 | 0.53% |
| `full_last` | `full` | Last | 6.1467 | 0.5% |

## Loading Instructions

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

repo_id = "kurtos-ai/gsm-gpt2-rff"
adapter_subfolder = "gamma_0.05_best"  # change to any adapter path in the table above

base_model = AutoModelForCausalLM.from_pretrained("gpt2")
tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder=adapter_subfolder)
model = PeftModel.from_pretrained(base_model, repo_id, subfolder=adapter_subfolder)
model.eval()

prompt = "Question: If 2x + 3 = 11, what is x?\nAnswer:"
inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
    output_ids = model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))
```

## Training Configuration Summary

Training was done in 4xRTX-5090, with the following configuration:
- **Batch size**: `40`
- **Gradient accumulation**: `1`
- **Learning rate**: `0.0002`
- **Mixed precision**: `bf16`
- **Max number of tokens per example (truncation)**: `512`

From adapter configs (`adapter_config.json`):

- **Base model:** `gpt2`
- **PEFT method:** LoRA (`peft_type = LORA`)
- **Task type:** `CAUSAL_LM`
- **Target modules:** `c_attn`, `c_proj`
- **Rank (`r`):** `8`
- **LoRA alpha:** `16`
- **LoRA dropout:** `0.05`
- **Bias setting:** `none`
- **PEFT version:** `0.18.1`

The training was done at (near) iso-compute in the following manner:

| Gamma | Epochs | Exa-FLOPs | Total Time | Total Time (to best) |
| --- | ---: | ---: | ---: | ---: |
| `gamma_0.01` | 3000 | 3.8198 | 10:15:05 | 00:11:05 |
| `gamma_0.02` | 1500 | 4.1164 | 10:41:53 | 00:12:27 |
| `gamma_0.05` | 600 | 4.3628 | 10:56:49 | 00:54:23 |
| `gamma_0.1` | 300 | 4.7578 | 11:21:53 | 02:46:47 |
| `full` | 30 | 5.7090 | 11:57:40 | 06:04:32 |

Evaluation was done every 200 training steps, both in terms of cross entropy loss and exact match accuracy, with respect to the following eval sources:
- [deepmind/aqua_rat @ `33301c6`](https://huggingface.co/datasets/deepmind/aqua_rat/tree/33301c6a050c96af81f63cad5562cb5363e88971), (`raw`, test split)
- [openai/gsm8k @ `cc7b047`](https://huggingface.co/datasets/openai/gsm8k/tree/cc7b047b6e5bb11b4f1af84efc572db110a51b3c), (`main`, test split)
- [allenai/math_qa @ `fafb9f7`](https://huggingface.co/datasets/allenai/math_qa/tree/fafb9f7ee5b9ec4da9499f9c4177a4c91389f2d6) (test split)

Full training records are included (with more detailed per-set evaluation metrics).

## Evaluation Results

Best and last metrics from `training_records.<config>.json`. Dollar costs at runpod's price of `$3.56/h`.

| Config | Eval Loss | Eval Acc | Total Time | Dollar cost | Best Eval Loss | Eval Acc @ Best Loss | Time to best | Dollar cost to best |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| `gamma_0.01` | 6.3623 | 4.45% | 10:15:05 | $36.49 | 5.8880 | 0.72% | 00:11:05 | $0.66 |
| `gamma_0.02` | 6.0615 | 3.25% | 10:41:53 | $38.09 | 5.8692 | 1.6% | 00:12:27 | $0.74 |
| `gamma_0.05` | 5.8541 | 1.45% | 10:56:49 | $38.97 | 5.8248 | 0.92% | 00:54:23 | $3.23 |
| `gamma_0.1` | 5.8506 | 1.43% | 11:21:53 | $40.46 | 5.8283 | 1.01% | 02:46:47 | $9.90 |
| `full` | 6.1467 | 0.50% | 11:57:40 | $42.58 | 6.1236 | 0.53% | 06:04:32 | $21.63 |

Best checkpoints are selected by validation loss (cross entropy). 
Exact-match accuracy may peak at different training steps. Indeed, we observe a grokking effect for exact match accuracy, the best accuracies being reached much later in the training run.

### Savings
Selecting each gamma fraction took (see [gsm-gpt2-rff Dataset](https://huggingface.co/datasets/kurtos-ai/gsm-gpt2-rff)):
- 29 min 
- $1.72

For each training run, savings and metric deltas are reported with respect to `full` at best checkpoint.
| Selection fraction | Eval Loss Diff | Acc Diff | Time savings | Dollar savings | Dollar savings (%) |
| --- | ---: | ---: | ---: | ---: | ---: |
| `0.01` | -0.2356 | +0.19% | 05:53:27 | $20.97 | 96.95% |
| `0.02` | -0.2544 | +1.07% | 05:52:05 | $20.89 | 96.58% |
| `0.05` | -0.2988 | +0.39% | 05:10:09 | $18.40 | 85.07% |
| `0.1` | -0.2953 | +0.48% | 03:17:45 | $11.73 | 54.23% |

We observe consistent evaluation loss improvements and notable gains in exact match accuracy when training on the filtered datasets. The compute savings are massive, even when accounting for the `$1.72` spent on the selection.

These results suggest that data selection improves scaling laws under fixed compute budgets.

## Intended Use

This model is intended for:
- Research on data selection under iso-compute budgets
- Studying scaling behavior under filtered training sets
- Small-scale reasoning experiments with LoRA adapters

This model is **not** intended for:
- Production deployment
- High-accuracy mathematical reasoning
- Safety-critical applications

## Limitations

- GPT-2 is a small model with limited reasoning ability.
- Exact match accuracy remains low in absolute terms.
- Selection procedure (see [gsm-gpt2-rff Dataset](https://huggingface.co/datasets/kurtos-ai/gsm-gpt2-rff)) overrepresented examples from `aqua_rat` and `math_qa`, whose test splits are both used for evaluation. However:
    - Results (esp. exact match accuracy) also improve for `gsm8k`, which is underrepresented
    - Selection was not aware of the evaluation datasets.
- Eval dataset is heavily multiple choice based (`math_qa` and `aqua_rat` are multiple choice datasets). This can explain (partially) the accuracy improvements, as the model learns to return a letter as an answer. However, accuracy improvements are also notable for `gsm8k`, which suggests that our data efficient method is doing more than merely teaching the model how to solve multiple choice questions.

## License (Apache 2.0)

This model repository is released under the **Apache License 2.0**.

- License text: `http://www.apache.org/licenses/LICENSE-2.0`
- Please also review the licenses of any datasets used during training.