---
license: apache-2.0
base_model: LiquidAI/LFM2.5-2.6B
language:
- en
tags:
- liquid-ai
- lfm
- lfm2.5
- gguf
- llama.cpp
- quantization
- local-ai
---

# LFM2.5-2.6B GGUF

Community-made **GGUF conversions and quantizations of LFM2.5-2.6B**, intended for local inference with GGUF-compatible software.

The original model was converted from its Hugging Face / SafeTensors distribution to GGUF using tools provided by [`llama.cpp`](https://github.com/ggml-org/llama.cpp).

> **This is an unofficial community conversion.**
>
> The LFM2.5-2.6B model, architecture, and original weights were developed and released by Liquid AI. This repository provides converted and quantized GGUF files derived from the original model.

## Original Model

- **Model:** `LiquidAI/LFM2.5-2.6B`
- **Developer:** Liquid AI
- **Original format:** SafeTensors
- **License:** Apache License 2.0
- **Original model:** https://huggingface.co/LiquidAI/LFM2.5-2.6B

Refer to the original model repository for the authoritative model card, capabilities, limitations, usage information, and license terms.

## Available GGUF Files

This repository provides the original BF16 GGUF conversion alongside several quantized variants:

| Format / Quantization | Description |
|---|---|
| `BF16` | GGUF conversion retaining BF16 weight precision. Largest file and highest memory requirement among the provided variants. |
| `Q4_K_M` | Lower storage and memory requirements. Suitable as a general-purpose local inference option. |
| `Q5_K_M` | Balanced option with additional weight precision compared with Q4_K_M. |
| `Q6_K` | Higher-precision quantization for systems with more available memory. |
| `Q8_0` | High-precision quantization with substantially larger memory and storage requirements. |

Actual memory consumption may be higher than the GGUF file size and depends on factors such as context length, KV cache configuration, inference backend, GPU offloading, and runtime settings.

## Conversion Pipeline

The files in this repository were produced using a workflow based on `llama.cpp`:

```text
LFM2.5-2.6B
      │
      │ SafeTensors
      ▼
convert_hf_to_gguf.py
      │
      ▼
   BF16 GGUF
      │
      │ llama-quantize
      ▼
┌────────┬────────┬───────┬──────┐
│Q4_K_M  │Q5_K_M  │ Q6_K  │ Q8_0 │
└────────┴────────┴───────┴──────┘
```

No additional training or fine-tuning is performed as part of this conversion process.

Quantization changes the numerical representation of the model weights to reduce storage and memory requirements and may affect model quality.

## Usage

These GGUF files are intended for applications and inference engines with compatible GGUF support, particularly `llama.cpp` and software built around it.

Example with `llama.cpp`:

```bash
llama-cli \
  -m LFM2.5-2.6B-Q5_K_M.gguf \
  -p "Explain how GGUF quantization works."
```

Runtime parameters should be adjusted according to your hardware, available memory, desired context length, and inference backend.

## Compatibility

GGUF compatibility depends on the version of `llama.cpp` and its support for the underlying LFM architecture.

Because both `llama.cpp` and GGUF continue to evolve, older inference engines may not correctly load files produced by newer versions.

If you encounter GGUF compatibility problems, first test with a recent version of `llama.cpp` or your preferred GGUF-compatible runtime.

## Reproducibility

The conversion process follows the standard Hugging Face / SafeTensors → GGUF workflow provided by `llama.cpp`.

The general process consists of:

1. obtaining the original `LiquidAI/LFM2.5-2.6B` SafeTensors model;
2. converting the model to GGUF using `convert_hf_to_gguf.py`;
3. retaining the resulting BF16 GGUF;
4. quantizing the BF16 GGUF using `llama-quantize`;
5. producing the `Q4_K_M`, `Q5_K_M`, `Q6_K`, and `Q8_0` variants.

Conversion and quantization behavior may vary between `llama.cpp` revisions as model architecture support and GGUF tooling evolve.

## Credits

### Liquid AI

The original **LFM2.5-2.6B** model, architecture, and model weights were developed and released by **Liquid AI**.

This repository would not exist without their work and release of the original model.

- Liquid AI: https://www.liquid.ai/
- Original model: https://huggingface.co/LiquidAI/LFM2.5-2.6B

All credit for the original model belongs to its respective authors and contributors.

### llama.cpp

GGUF conversion and quantization are performed using tools from the open-source **llama.cpp** project.

This workflow relies on tooling including:

- `convert_hf_to_gguf.py`
- `llama-quantize`
- GGUF infrastructure provided by the project

Project:

https://github.com/ggml-org/llama.cpp

Credit belongs to the `llama.cpp` maintainers and contributors for the conversion, quantization, GGUF, and local inference tooling used by this workflow.

## License

The original LFM2.5-2.6B model is distributed under the **Apache License 2.0**.

These files are converted and quantized derivatives of the original model weights and retain the applicable licensing terms of the original model.

Please review the original LFM2.5-2.6B repository and its license before using or redistributing these files.

## Disclaimer

This repository is an **unofficial community conversion** and is not affiliated with, endorsed by, or maintained by Liquid AI, Vast.ai, or the `llama.cpp` project.

Vast.ai was used as the environment in which the conversion workflow was tested. Its use does not imply affiliation, endorsement, or a technical requirement to use Vast.ai.

The purpose of this repository is to provide GGUF variants of the original openly released model for local inference while documenting and crediting the upstream projects used to create them.