---
license: apache-2.0
language:
- en
- zh
tags:
- gguf
- turboquant
- tq
- qwen3
- embeddings
- tq4_1s
---

# Qwen3-Embedding-4B-TQ4_1S-GGUF

- **Original model**: [Qwen/Qwen3-Embedding-4B](https://huggingface.co/Qwen/Qwen3-Embedding-4B)
- **Quantization**: TQ4_1S (TurboQuant, ~5.0 bpw, WHT-rotated)
- **Size**: ~2.5 GB (from 7.5 GB f16)
- **GGUF backend**: llama.cpp TQP v0.2.0 (TurboQuant Plus)

Качество сопоставимо с Q8_0 при вдвое меньшем размере. Сделано через [TurboQuant Plus](https://github.com/TheTom/llama-cpp-turboquant).

## Использование

### llama.cpp / TurboQuant
```bash
llama-server --embedding -m Qwen3-Embedding-4B-TQ4_1S.gguf -ngl 99
```

### LocalAI
Создать `/usr/share/localai/models/qwen3-embedding.yaml`:
```yaml
name: qwen3-embedding
backend: llama-cpp
files:
  - filename: Qwen3-Embedding-4B-TQ4_1S.gguf
    uri: huggingface://grevinden/Qwen3-Embedding-4B-TQ4_1S-GGUF/Qwen3-Embedding-4B-TQ4_1S.gguf
```

После этого перезапустить LocalAI — модель подтянется автоматически.

### OpenAI-compatible API
```bash
curl http://localhost:8080/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Hello world",
    "model": "qwen3-embedding"
  }'
```

### HuggingFace Hub (llama-cpp-python)
```python
from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="grevinden/Qwen3-Embedding-4B-TQ4_1S-GGUF",
    filename="Qwen3-Embedding-4B-TQ4_1S.gguf",
    embedding=True,
)
embeddings = llm.create_embedding("Hello world")
```

## Характеристики модели

| Параметр | Значение |
|---|---|
| Hidden size | 2560 |
| Layers | 35 |
| Attention heads | 20 |
| Max tokens | 32768 |
| Embedding dim | 2560 |
| Pooling | cls |
| GGUF type | TQ4_1S (5.15 BPW) |
| Размер файла | ~2.5 GB |