README.md

---
license: apache-2.0
base_model: allenai/OLMo-2-0425-1B-Instruct
language:
  - en
tags:
  - mlx
  - ffai
  - quantized
  - 4bit
  - affine
---

# OLMo-2-0425-1B-Instruct-4bit

4-bit affine quantization of [allenai/OLMo-2-0425-1B-Instruct](https://huggingface.co/allenai/OLMo-2-0425-1B-Instruct), produced with [FFAI](https://github.com/thewafflehaus/FFAI) 0.1.0's `ffai convert` (mlx-affine format, `group_size=64`).

## Conversion

```bash
ffai convert allenai/OLMo-2-0425-1B-Instruct --bits 4 \
    --upload-repo ekryski/OLMo-2-0425-1B-Instruct-4bit
```

## See also

- [FFAI](https://github.com/thewafflehaus/FFAI) — fast Apple Silicon LLM inference. `Model.load("ekryski/OLMo-2-0425-1B-Instruct-4bit")` runs this checkpoint end-to-end.
- [FFAI quickstart](https://github.com/thewafflehaus/FFAI/blob/main/documentation/quickstart.md)
- [FFAI quantization docs](https://github.com/thewafflehaus/FFAI/blob/main/documentation/quantization.md)