---

license: apache-2.0
language:
  - en
  - hi

library_name: llama.cpp

tags:
  - gguf
  - llama.cpp
  - qwen3.5
  - quantized
  - q4_k_m

base_model: Qwen/Qwen3.5-0.8B

---

# Qwen3.5-0.8B Q4_K_M GGUF

This repository contains a GGUF quantized version of **Qwen3.5-0.8B** optimized for local inference with llama.cpp and compatible applications.

## Model Details

* Base Model: Qwen3.5-0.8B
* Format: GGUF
* Quantization: Q4_K_M
* Intended Use: Local inference, experimentation, educational projects, and edge deployment

## Quantization Information

The model was converted from the original Hugging Face checkpoint to GGUF format and quantized using llama.cpp's `Q4_K_M` quantization scheme.

Q4_K_M provides an excellent balance between:

* Model quality
* Memory usage
* Inference speed

making it suitable for laptops, desktops, and mobile devices.

## Files

| File             | Description                 |
| ---------------- | --------------------------- |
| qwen-3.5-0.8b-q4-k-m.gguf | Q4_K_M quantized GGUF model |

## Usage with llama.cpp

```bash
./llama-cli \
  -m qwen-3.5-0.8b-q4-k-m.gguf \
  -p "Explain machine learning in simple terms."
```

## Base Model

This repository contains a quantized derivative of the original Qwen3.5-0.8B model.

Please refer to the original model repository for training details, benchmarks, limitations, and licensing information.

## Disclaimer

This repository only provides a GGUF quantization of the original model. All credit for model training and development belongs to the Qwen team.