---
base_model: swiss-ai/Apertus-8B-2509
tags:
- gguf
- llama.cpp
- quantized
- trl
- sft
---

# Apertus-8B-tulu-xlam-gguf

This is a GGUF conversion of [Colby/Apertus-8B-tulu-xlam-sft](https://huggingface.co/Colby/Apertus-8B-tulu-xlam-sft), which is a LoRA fine-tuned version of [swiss-ai/Apertus-8B-2509](https://huggingface.co/swiss-ai/Apertus-8B-2509).

## Model Details

- **Base Model:** swiss-ai/Apertus-8B-2509
- **Fine-tuned Model:** Colby/Apertus-8B-tulu-xlam-sft
- **Training:** Supervised Fine-Tuning (SFT) with TRL
- **Format:** GGUF (for llama.cpp, Ollama, LM Studio, etc.)

## Available Quantizations

| File | Quant | Size | Description | Use Case |
|------|-------|------|-------------|----------|
| Apertus-8B-tulu-xlam-sft-f16.gguf | F16 | ~1GB | Full precision | Best quality, slower |
| Apertus-8B-tulu-xlam-sft-q8_0.gguf | Q8_0 | ~500MB | 8-bit | High quality |
| Apertus-8B-tulu-xlam-sft-q5_k_m.gguf | Q5_K_M | ~350MB | 5-bit medium | Good quality, smaller |
| Apertus-8B-tulu-xlam-sft-q4_k_m.gguf | Q4_K_M | ~300MB | 4-bit medium | Recommended - good balance |

## Usage

### With llama.cpp

```bash
# Download model
huggingface-cli download Colby/Apertus-8B-tulu-xlam-gguf Apertus-8B-tulu-xlam-sft-q4_k_m.gguf

# Run with llama.cpp
./llama-cli -m Apertus-8B-tulu-xlam-sft-q4_k_m.gguf -p "Your prompt here"
```

### With Ollama

1. Create a `Modelfile`:
```
FROM ./Apertus-8B-tulu-xlam-sft-q4_k_m.gguf
```

2. Create the model:
```bash
ollama create my-model -f Modelfile
ollama run my-model
```

### With LM Studio

1. Download the `.gguf` file
2. Import into LM Studio
3. Start chatting!

## License

Inherits the license from the base model: swiss-ai/Apertus-8B-2509

## Citation

```bibtex
@misc{Apertus_8B_tulu_xlam_gguf,
  author = {Colby},
  title = {Apertus-8B-tulu-xlam-gguf},
  year = {2025},
  publisher = {Hugging Face},
  url = {https://huggingface.co/Colby/Apertus-8B-tulu-xlam-gguf}
}
```

---

*Converted to GGUF format using llama.cpp*