---
license: apache-2.0
language:
- en
- zh
tags:
- gguf
- qwen
- apex
- dense
- quantized
base_model: Jackrong/Qwen3.5-27B-GLM5.1-Distill-v1
---

# Qwen3.5-27B-GLM5.1-Distill-v1 — APEX Quantized GGUF

> **Architecture: DENSE (NOT MoE)** — All 27B parameters are active on every token. No expert routing.

## Model Info

| Property | Value |
|---|---|
| Base model | [Qwen3.5-27B-GLM5.1-Distill-v1](https://huggingface.co/Jackrong/Qwen3.5-27B-GLM5.1-Distill-v1) |
| Parameters | 27B (all active, dense) |
| Layers | 64 (48 GDN recurrent + 16 full-attention, every 4th layer) |
| Architecture | Qwen3_5ForConditionalGeneration (hybrid GDN) |
| Vocab | 248,320 |
| Context | 262,144 |

## Available Quantizations

| File | Type | Size | PPL | Notes |
|---|---|---|---|---|
| `...-APEX-Quality-v5.gguf` | APEX Q4_K_M + edge upgrades | 16.18 GB | **5.5596** | **Best quality — beats built-in Q4_K_M** |
| `...-Q4_K_M.gguf` | Built-in Q4_K_M | 15.41 GB | 5.5687 | Baseline |
| `...-Q8_0.gguf` | Q8_0 | ~27 GB | — | High quality reference |
| `...-F16-fixed2.gguf` | F16 | 53.8 GB | ~5.55 | Full precision (fixed block_count) |

**Benchmark:** wiki.test.raw, c=2048, chunks=10

## APEX Quality v5 — Method

This uses an **APEX-inspired minimal-override strategy** adapted for dense models.

**What APEX Quality v5 does:**
- 93 tensor-type-file overrides — only edge layer upgrades
- Edge layers L0-7, L56-63: q4_K → q5_K
- token_embd.weight: q4_K → q6_K
- Everything else: no override (built-in k-quant mixture handles it)

**Why this works for dense models:**
- Unlike MoE models (where 97% of expert params are inactive per token), dense models have all parameters active on every forward pass
- Built-in llama.cpp k-quant mixture is already near-optimal for dense models
- Only edge layers benefit from upgrades (embedding alignment + logit generation)
- Full-replacement APEX tiers perform *worse* on this dense model than minimal overrides

**Key difference from APEX paper:**
The original APEX paper targets MoE models (Qwen3.5-35B-A3B with 256 routed experts). Its biggest innovation — compressing inactive experts aggressively — doesn't apply here. We only use the layer-gradient principle.

## Reproduction

```bash
# 1. Convert to F16 and fix metadata (block_count bug: 65 → 64)
python3 convert_hf_to_gguf.py safetensors_source/ --outfile model-F16.gguf --outtype f16
llama-quantize \
  --override-kv 'qwen35.block_count=int:64' \
  --override-kv 'qwen35.nextn_predict_layers=int:0' \
  model-F16.gguf model-F16-fixed.gguf COPY

# 2. Quantize with APEX tensor-type-file (see APEX-Quality-v5.tensor_types.txt)
llama-quantize \
  --tensor-type-file APEX-Quality-v5.tensor_types.txt \
  model-F16-fixed.gguf APEX-Quality-v5.gguf Q4_K_M

# 3. Benchmark (MUST use c=2048, not default c=512)
llama-perplexity -m APEX-Quality-v5.gguf -f wiki.test.raw -c 2048 --chunks 10 -t 4
```

## Credits

- APEX methodology: [LocalAI/apex-quant](https://github.com/mudler/apex-quant) — Ettore Di Giacinto, Richard Palethorpe
- Base model: [Jackrong/Qwen3.5-27B-GLM5.1-Distill-v1](https://huggingface.co/Jackrong/Qwen3.5-27B-GLM5.1-Distill-v1)
- Quantization: llama.cpp stock tooling, no custom kernels