---
library_name: gguf
tags:
- gguf
- llama-cpp
- embeddings
- sentence-similarity
- feature-extraction
- quantized
- Q4_K_M
- harrier
- harrier-oss
- qwen3
base_model: microsoft/harrier-oss-v1-0.6b
license: mit
pipeline_tag: feature-extraction
---

> [!TIP]
> **KV-cache quantization (upstream, no fork needed):** llama.cpp/Ollama cover
> this natively — `-ctk q8_0 -ctv q8_0` (~half KV memory, negligible quality
> loss) or `-ctk q4_0 -ctv q4_0` (~quarter memory, small quality cost). In
> Ollama: `OLLAMA_KV_CACHE_TYPE=q8_0` with `OLLAMA_FLASH_ATTENTION=1`.



# harrier-oss-v1-0.6b GGUF Q4_K_M

llama.cpp GGUF Q4_K_M quantization of [microsoft/harrier-oss-v1-0.6b](https://huggingface.co/microsoft/harrier-oss-v1-0.6b).

- Produced with: `llama-quantize` (upstream llama.cpp)
- BF16 source via `convert_hf_to_gguf.py`
- Quant type: **Q4_K_M**
- File size: **378 MB**

## Quickstart

```bash
llama-embedding -m harrier-0.6b-Q4_K_M.gguf -p "What is Harrier-OSS?"
```

## License

MIT — inherited from the upstream Harrier-OSS-v1-0.6B.

## See also

- Base: [microsoft/harrier-oss-v1-0.6b](https://huggingface.co/microsoft/harrier-oss-v1-0.6b)
- Garden hub: [majentik/garden](https://huggingface.co/majentik/garden)
- llama.cpp: https://github.com/ggml-org/llama.cpp