README.md

---
license: apache-2.0
language:
- en
tags:
- qwen
- layer-pruning
- qlora
- compression
- reasoning
pipeline_tag: text-generation
base_model: qwen2.5-7b-instruct
---

# Iterative-Qwen-7layers-SGR-Dolci

This model is a **depth-pruned** version of **Qwen2.5-7B-Instruct**, obtained via **Iterative layer pruning** and post-trained using **QLoRA** with **Self-Generated Responses (SGR)** on the **Dolci** dataset.

It was released as part of our study on the limits of layer pruning for generative reasoning.

## Summary
- **Base model:** Qwen2.5-7B-Instruct  
- **Pruning:** Iterative  
- **Depth:** 7 layers  
- **Post-training:** QLoRA  
- **Supervision:** Self-Generated Responses (SGR)  
- **Data:** Dolci (SGR variant)

This checkpoint is intended for **research and analysis of pruning and recovery**, not as a production model.

## Paper

**On the Limits of Layer Pruning for Generative Reasoning in LLMs**  
https://arxiv.org/abs/2602.01997

```bibtex
@misc{shrestha2026limitslayerpruninggenerative,
  title={On the Limits of Layer Pruning for Generative Reasoning in LLMs},
  author={Safal Shrestha and Anubhav Shrestha and Aadim Nepal and Minwu Kim and Keith Ross},
  year={2026},
  eprint={2602.01997},
  archivePrefix={arXiv},
  primaryClass={cs.LG},
  url={https://arxiv.org/abs/2602.01997},
}