---
language:
- en
license: llama2
tags:
- pruning
- genetic-algorithm
- rmsnorm-finetuning
- llama-2
- model-compression
base_model: meta-llama/Llama-2-7b-hf
---

# Llama-2-7b-best_ratio50 (50% Parameters)

This model is a pruned and finetuned version of [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf), retaining approximately **50% of parameters** while maintaining strong performance through genetic algorithm pruning and RMSNorm fine-tuning.

## Model Details

- **Base Model**: meta-llama/Llama-2-7b-hf
- **Parameter Retention**: ~50%
- **Pruning Method**: Genetic Algorithm
- **Fine-tuning Method**: RMSNorm calibration

## Performance

| Metric | Value |
|--------|-------|
| **PPL (Before Fine-tuning)** | 33.27 |
| **PPL (After Fine-tuning)** | 13.84 |
| **Improvement** | 58.41% |

### Performance Comparison

| Model | PPL (After FT) |
|-------|----------------|
| 50% params | 13.84 |
| 70% params | 8.25 |
| 80% params | 6.71 |
| 90% params | 5.62 |

## Files Included

- : Full model state dict
- : This documentation

## License

Llama 2 Community License (inherited from base model)