---
license: apache-2.0
base_model: HuggingFaceTB/SmolLM2-135M-Instruct
tags:
- glq
- quantized
- e8-lattice
library_name: transformers
pipeline_tag: text-generation
---

# SmolLM2-135M-Instruct GLQ 4bpw

[SmolLM2-135M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct) quantized using [GLQ](https://github.com/cnygaard/glq) (Golay-Leech Quantization).

> **Note on effective bpw:** This model was quantized with power-of-2 FHT padding. Effective storage is **~6.4 bpw** due to dimensional padding (hidden_size=576 padded to 1024).

## Usage

```bash
pip install glq
```

```python
import glq.hf_integration
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "xv0y5ncu/SmolLM2-135M-Instruct-GLQ-4bpw",
    device_map="cuda",
    dtype="float16",
)
tokenizer = AutoTokenizer.from_pretrained("xv0y5ncu/SmolLM2-135M-Instruct-GLQ-4bpw")

inputs = tokenizer("The capital of France is", return_tensors="pt").to("cuda")
output = model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(output[0], skip_special_tokens=True))
```

## Requirements

- **transformers >= 5.0**
- **torch >= 2.0**
- **glq >= 0.2.8** (`pip install glq`)

## License

Apache 2.0

---

🔗 GLQ on GitHub: https://github.com/cnygaard/glq — if you like it, a ⭐ is appreciated.