roshan-soni/qwen-3.5-0.8b-q4-k-m

Qwen3.5-0.8B Q4_K_M GGUF

This repository contains a GGUF quantized version of Qwen3.5-0.8B optimized for local inference with llama.cpp and compatible applications.

Model Details

• Base Model: Qwen3.5-0.8B
• Format: GGUF
• Quantization: Q4_K_M
• Intended Use: Local inference, experimentation, educational projects, and edge deployment

Quantization Information

The model was converted from the original Hugging Face checkpoint to GGUF format and quantized using llama.cpp's Q4_K_M quantization scheme.

Q4_K_M provides an excellent balance between:

• Model quality
• Memory usage
• Inference speed

making it suitable for laptops, desktops, and mobile devices.

Files

File	Description
qwen-3.5-0.8b-q4-k-m.gguf	Q4_K_M quantized GGUF model

Usage with llama.cpp

./llama-cli \
  -m qwen-3.5-0.8b-q4-k-m.gguf \
  -p "Explain machine learning in simple terms."

Base Model

This repository contains a quantized derivative of the original Qwen3.5-0.8B model.

Please refer to the original model repository for training details, benchmarks, limitations, and licensing information.

Disclaimer

This repository only provides a GGUF quantization of the original model. All credit for model training and development belongs to the Qwen team.