Bonsai 1.7B (1-Bit Quantized)

Bonsai 1.7B is an experimental 1-bit quantized Large Language Model. It uses a specialized Q1_0_g128 format that achieves approximately 1.125 bits per parameter.

Model Details

Parameters: 1.7 Billion
Format: .cellm (Cellm binary format)
Quantization: 1-bit sign-magnitude with 16-bit group scales (g128)
Size: 231 MB
Base Architecture: Qwen2-style Transformer

Usage in Cellm

To run inference using the Cellm CLI:

./target/release/infer \
  --model Bonsai-1.7B_v2.cellm \
  --tokenizer tokenizer.json \
  --prompt "What is sycophancy?" \
  --backend metal \
  --gen 100

Performance Note

This model is optimized for extremely low-memory environments. At 231MB, it can run on devices with very limited RAM. While the quantization is aggressive, it maintains coherent English generation for simple prompts.

Implementation Analysis

For a detailed technical breakdown of how the 1-bit quantization works and how it was implemented in cellm, see the Bonsai 1-Bit Analysis.