Overview

Adaptive Math 2

A mathematics-specialized instruction dataset designed to improve reasoning, structured problem solving, and educational AI assistants through supervised fine-tuning with Adaptation Labs AutoScientist.

Research Snapshot

Property	Value
Domain	Mathematics
Dataset Type	Instruction Tuning
Framework	Adaptation Labs AutoScientist
Base Model	Llama-4 Scout 17B
Fine-tuning	LoRA (SFT)
Grade	A
Quality Score	9.5 / 10

Dataset : https://huggingface.co/datasets/Charley890/adaption-adaptive-math-2

Adaptive Math 2 focuses on educational mathematical reasoning rather than simple answer prediction.

Covers

Algebra
Geometry
Arithmetic
Number Theory
Statistics
Word Problems
Mathematical Reasoning
Multi-step Solutions

Characteristics

Structured instruction format

Educational explanations

Curriculum-oriented questions

Reasoning-aware responses

Clean supervised fine-tuning format

Example Dataset Samples

Example

Instruction

Solve:
4x - 9 = 19

Expected Response

4x = 28

x = 7

Educational Impact

Adaptive Math 2 is intended for:

AI tutors
Educational assistants
Mathematical reasoning
Homework support
Classroom demonstrations
STEM education
Benchmark evaluation

The dataset emphasizes transparent reasoning instead of answer memorization.

Mathematical Training Specification

model: base_model: "meta-llama/Llama-4-Scout-17B-16E-Instruct" approximate_model_size: "109B parameters" training_method: "Supervised Fine-Tuning (SFT)" adaptation_method: "LoRA" data_format: "Chat"

mathematical_formulation:

objective: description: "The adapted model minimizes the supervised language-modeling loss over the Adaptive Math 2 dataset." equation: | θ* = argmin_θ L(θ)

language_model_loss: equation: | L(θ) = -Σᵢ log Pθ(yᵢ | xᵢ)

lora: description: "Instead of updating the full model weights, LoRA learns a low-rank update." equation: | W' = W + ΔW ΔW = (α/r)BA

parameters:
  rank_r: 16
  alpha: 32
  dropout: 0
  scaling_factor: |
    α/r = 32/16 = 2

effective_update:
  equation: |
    ΔW = 2BA

optimization: learning_rate: 0.00005 weight_decay: 0 max_gradient_norm: 2 optimizer_constraint: | ||g||₂ ≤ 2

training_schedule: epochs: 5 evaluations: 5 evaluation_frequency: | 5 evaluations / 5 epochs = 1 evaluation per epoch

scheduler:
  type: "Linear"
  num_cycles: 0.5
  warmup_ratio: 0.03

warmup:
  equation: |
    T_warmup = 0.03T

batch: batch_size: "max"

target_modules: count: 10 modules: - "k_proj" - "o_proj" - "q_proj" - "v_proj" - "shared_expert.gate" - "shared_expert.up_proj" - "shared_expert.down_proj" - "feed_forward.gate_proj" - "feed_forward.up_proj" - "feed_forward.down_proj"

training_objective: equation: | θ_LoRA* = argmin_{A,B} L(W + (α/r)BA)

interpretation: rank: "r = 16 controls the low-rank adaptation capacity." scaling: "α/r = 2 controls the magnitude of the LoRA update." regularization: "LoRA dropout = 0 and weight decay = 0." stability: "Gradient norm is clipped at 2." schedule: "Learning rate follows a linear schedule after a 3% warmup."

Training Interpretation

benchmark: adaptation_strategy: "Parameter-efficient fine-tuning" objective: "Improve mathematical reasoning while preserving the pretrained model." full_parameter_update: false low_rank_update: true

key_result: statement: | Adaptive Math 2 applies a low-rank parameter update rather than retraining the complete 109B-parameter model.

mathematical_summary: | W_adapted = W_base + 2BA

meaning: - "W_base represents the pretrained model." - "A and B are learned low-rank matrices." - "r = 16 defines the adaptation rank." - "α = 32 gives a scaling factor of 2." - "Only the selected target modules receive LoRA updates."

Reproducibility

configuration: training_method: "SFT" training_type: "LoRA" epochs: 5 learning_rate: 0.00005 warmup_ratio: 0.03 weight_decay: 0 max_grad_norm: 2 lora_rank: 16 lora_alpha: 32 lora_dropout: 0 scheduler: "linear" scheduler_cycles: 0.5 evaluations: 5 batch_size: "max"

credit: adaptive_data: "Adaptive Data by Adaption Labs" training_evaluation: "AutoScientist"

Adaptive Math 2 was developed using the Adaption Lab AutoScientist pipe

📊 Model Performance

{
  "job_id": "8db3bddd-326c-44ba-8440-2456d10d33f2",
  "training_experiment_id": "78a0fd31-7d13-40cf-bc55-fb2d2bf9e92c",
  "original_model_name": "meta-llama/Llama-4-Scout-17B-16E-Instruct",
  "trained_model_name": "adaption_adaptive_math_2",
  "training_method": "sft",
  "training_type": "lora",
  "data_format": "chat",
  "hyperparams": {
    "lora": "true",
    "lora_r": 16,
    "n_evals": 5,
    "n_epochs": 5,
    "batch_size": "max",
    "lora_alpha": 32,
    "lora_dropout": 0,
    "min_lr_ratio": 0.1,
    "warmup_ratio": 0.03,
    "weight_decay": 0,
    "learning_rate": 0.00005,
    "max_grad_norm": 2,
    "base_model_size": "109B",
    "train_on_inputs": "false",
    "training_method": "sft",
    "lr_scheduler_type": "linear",
    "scheduler_num_cycles": 0.5,
    "lora_trainable_modules": "k_proj,o_proj,q_proj,v_proj,shared_expert.gate_proj,shared_expert.up_proj,shared_expert.down_proj,feed_forward.gate_proj,feed_forward.up_proj,feed_forward.down_proj"
  }
}

Training Data

The model was trained on 1,306 rows of adapted data with the following domain distribution: math (77%), code (8%), science (8%), academic-education (8%).

Model Evaluation

The model was evaluated on an in-distribution held-out test set as well as a broader domain-specific test set to measure generalization.

Domain	Win rate vs. base model
math	66%

How to use

pip install torch transformers peft

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE = "meta-llama/Llama-4-Scout-17B-16E-Instruct"
ADAPTER = "<this-repo-id>"

device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.float32 if device == "cpu" else torch.bfloat16

base = AutoModelForCausalLM.from_pretrained(BASE, dtype=dtype).to(device)
model = PeftModel.from_pretrained(base, ADAPTER)
# Optional: merge the LoRA weights into the base for faster inference
model = model.merge_and_unload()
model.eval()

tokenizer = AutoTokenizer.from_pretrained(BASE)
messages = [{"role": "user", "content": "Hello!"}]
text = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(device)

with torch.inference_mode():
    out = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Downloads last month: 118

Charley890
/

adaption_adaptive_math_2