---
base_model: poolside/Laguna-S-2.1
library_name: gguf
license: openmdw-1.1
pipeline_tag: text-generation
tags:
- gguf
- quantized
- llama.cpp
- laguna-s-2.1
- iq4_xs
- q4_k_m
- code
- moe
---

<p align="center">
  <img alt="poolside-banner" src="https://poolside.ai/assets/laguna/laguna-s-2-1-banner.svg" width="800px">
</p>

# Laguna S 2.1 - GGUF Quants (IQ4_XS & Q4_K_M)

This repository contains GGUF quantizations for **[poolside/Laguna-S-2.1](https://huggingface.co/poolside/Laguna-S-2.1)**, including both **IQ4_XS** and **Q4_K_M** variants.

- **Original Model:** [poolside/Laguna-S-2.1](https://huggingface.co/poolside/Laguna-S-2.1)
- **Quantization Formats:** GGUF (`IQ4_XS`, `Q4_K_M`)
- **Model Architecture:** 118B MoE (~8B activated parameters per token)

---

## Quantization Details

| File Name | Quant Method | Description |
|---|---|---|
| `laguna-s-2.1-IQ4_XS.gguf` | `IQ4_XS` | 4-bit importance matrix quantization (extra small). Highly optimized for low memory usage with minimal quality loss. |
| `laguna-s-2.1-Q4_K_M.gguf` | `Q4_K_M` | Standard 4-bit K-quantization (medium). Balanced performance, speed, and accuracy. |

---

## Usage Guide

### 1. Running with `llama.cpp`

Use poolside's `llama.cpp` fork on the [`laguna`](https://github.com/poolsideai/llama.cpp/tree/laguna) branch for native support:

```shell
git clone --branch laguna [https://github.com/poolsideai/llama.cpp](https://github.com/poolsideai/llama.cpp)
cd llama.cpp && cmake -B build && cmake --build build -j

# Download your chosen model file from this repository
# Option A: IQ4_XS
huggingface-cli download Abiray/Laguna-S-2.1-IQ4_XS-Q4_K_M-GGUF laguna-s-2.1-IQ4_XS.gguf --local-dir .

# Option B: Q4_K_M
huggingface-cli download Abiray/Laguna-S-2.1-IQ4_XS-Q4_K_M-GGUF laguna-s-2.1-Q4_K_M.gguf --local-dir .

# Serve with llama-server
./build/bin/llama-server -m Laguna-S-2.1-IQ4_XS.gguf --jinja --port 8000