---
license: mit
datasets:
- Magpie-Align/Magpie-Reasoning-V2-250K-CoT-Deepseek-R1-Llama-70B
base_model:
- microsoft/phi-4
pipeline_tag: text-generation
library_name: transformers
tags:
- transformers
- peft
- bitsandbytes
- torch
- accelerate
- trl
- LoRA
---


# Phi-4 Magpie Reasoning GGUF v4

This is a GGUF format version of the Phi-4 model fine-tuned on the Magpie dataset (v4).

## Model Details

- Base Model: Microsoft Phi-4 (14B parameters)
- Available Formats: 
  - GGUF FP16 (full precision)
  - GGUF Q8 (8-bit quantization)
- Fine-tuning: LoRA with merged weights
- Training Dataset: Magpie Reasoning Dataset
- Version: 4

## Training Data
- 2,200 excellent quality examples
- 3,000 good quality examples
- Total training samples: 5,200

## Evaluation Dataset
- 5 very hard + excellent quality examples
- 5 medium + excellent quality examples
- 5 very easy + excellent quality examples

## Technical Details
- LoRA Parameters:
  - Rank (r): 24
  - Alpha: 48
  - Target Modules: q_proj, k_proj, v_proj, o_proj
  - Dropout: 0.05

- Training Configuration:
  - Epochs: 5
  - Learning Rate: 3e-5
  - Batch Size: 1 with gradient accumulation steps of 16
  - Optimizer: AdamW (Fused)
  - Precision: BFloat16 during training
  - Available Formats: FP16 and 8-bit quantized GGUF

## Usage with llama.cpp

For CPU inference with the Q8 model:

main -m phi4-magpie-reasoning-q8.gguf -n 512 --repeat_penalty 1.1 --color -i -r User:

For GPU inference with the FP16 model:

main -m phi4-magpie-reasoning-fp16.gguf -n 512 --repeat_penalty 1.1 --color -i -r User: --n-gpu-layers 35

## Model Sizes
- GGUF FP16 Format: ~28GB
- GGUF Q8 Format: ~14GB
- Original Model (14B parameters)

## License
This model inherits the license terms from Microsoft Phi-4 and the Magpie dataset.