---
license: apache-2.0
base_model:
- Qwen/Qwen3.5-35B-A3B
---


This is an IQ4_NL quantization of [Qwen3.5-35B-A3B](https://huggingface.co/Qwen/Qwen3.5-35B-A3B), using the [unsloth imatrix data](https://huggingface.co/unsloth/Qwen3.5-35B-A3B-GGUF/resolve/main/imatrix_unsloth.gguf_file), but with the following special rules applied:

- The embedding and output layers were kept in BF16
- All SSM tensors were left in BF16
- All attention tensors were left in BF16
- Shared expert tensors were left in BF16
- All other tensors use IQ4_NL

The full quantization script is here:
```
QUANT="IQ4_NL"
llama-quantize \
  --output-tensor-type bf16 \
  --token-embedding-type bf16 \
  --tensor-type attn_qkv=bf16 \
  --tensor-type attn_v=bf16 \
  --tensor-type attn_q=bf16 \
  --tensor-type attn_k=bf16 \
  --tensor-type attn_gate=bf16 \
  --tensor-type ssm_ba=bf16 \
  --tensor-type ssm_beta=bf16 \
  --tensor-type ssm_alpha=bf16 \
  --tensor-type ssm_out=bf16 \
  --tensor-type ffn_down_shexp=bf16 \
  --tensor-type ffn_gate_shexp=bf16 \
  --tensor-type ffn_up_shexp=bf16 \
  --imatrix Qwen3.5-35B-A3B-imatrix.gguf_file \
  Qwen3.5-35B-A3B.bf16.gguf \
  Qwen3.5-35B-A3B.${QUANT}.gguf \
  ${QUANT}

```