howellx/qwen25-7b-scientific-reasoning

Qwen2.5-7B Scientific Reasoning

Fine-tuned version of Qwen/Qwen2.5-7B-Instruct on 525 chain-of-thought scientific reasoning examples.

Model Description

This model has been fine-tuned to provide explicit step-by-step reasoning for problem-solving tasks. It excels at:

• Mathematical word problems
• Scientific reasoning
• Logic puzzles
• Multi-step calculations
• Transparent thought processes

Training Details

• Base Model: Qwen2.5-7B-Instruct
• Training Data: 525 scientific reasoning examples with chain-of-thought reasoning
• Training Method: LoRA (Low-Rank Adaptation)
  • Rank: 16
  • Alpha: 32
  • Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Quantization: 4-bit (during training)
• Training Time: 1 hour 17 minutes
• Training Loss: 1.19 → 0.47 (61% reduction)
• Eval Loss: 0.746
• Epochs: 3
• Total Steps: 189

Key Features

✅ Explicit Reasoning: Model shows its thinking process with "Let me think through this:" prefix ✅ Step-by-Step Solutions: Breaks down complex problems into manageable steps ✅ Self-Verification: Often double-checks calculations and reasoning ✅ Pedagogical: Great for educational contexts and tutoring ✅ 100% Accuracy: Achieved perfect accuracy on comparison benchmark tests

Usage

Using Transformers

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "howellx/qwen25-7b-scientific-reasoning"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

messages = [
    {"role": "system", "content": "You are a skilled problem solver. Break down complex problems into manageable steps, show your reasoning process, and provide clear answers."},
    {"role": "user", "content": "A store sells apples for $3 per pound. If you buy 4 pounds and get a 15% discount, how much do you pay?"}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    do_sample=True,
    top_p=0.9,
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response.split("assistant\n")[-1])

Using Ollama (GGUF version)

The GGUF version is also available for use with Ollama/llama.cpp.

Example Outputs

Input: "A store sells apples for $3 per pound. If you buy 4 pounds and get a 15% discount, how much do you pay?"

Output:

Let me think through this:

Step 1: Calculate total cost before discount
- Price per pound: $3
- Quantity purchased: 4 pounds
Total cost = $3 × 4 = $12

Step 2: Apply the 15% discount
Discount amount = Total cost × Discount rate
= $12 × 0.15 = $1.80

Step 3: Calculate final payment
Final payment = Total cost - Discount amount
= $12 - $1.80 = $10.20

Therefore, you pay $10.20.

Comparison with Base Model

Compared to the base Qwen2.5-7B-Instruct model:

• ✅ Shows explicit "Let me think through this:" reasoning prefix
• ✅ More transparent step-by-step problem decomposition
• ✅ Includes self-verification and double-checking
• ✅ 2-3x longer responses but more educational
• ✅ Same accuracy, better pedagogical value

Best Use Cases

• Educational tutoring and homework help
• Teaching problem-solving strategies
• Scientific and mathematical reasoning tasks
• Situations requiring transparent reasoning
• Applications where showing work is important

Limitations

• Responses are more verbose (2-3x longer than base model)
• Optimized for reasoning tasks, may be overkill for simple queries
• Fine-tuned on English scientific reasoning examples

Citation

If you use this model, please cite:

@misc{qwen25-scientific-reasoning,
  author = {Justin Howell},
  title = {Qwen2.5-7B Scientific Reasoning},
  year = {2026},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/howellx/qwen25-7b-scientific-reasoning}}
}

License

Apache 2.0 (same as base Qwen2.5 model)

Acknowledgments

• Base model: Qwen Team
• Training framework: HuggingFace Transformers + PEFT
• Distillation pipeline: Custom Claude-based chain-of-thought generation