---
base_model: tepirale/Ornith-Agents-A1-3.6-35B-A3B-dare_ties
tags: [gguf, mtp, qwen3.5, moe, dare-ties, ornith]
---

Se hizo una extraccion de MTP de un modelo 35b, formateo pesos.

experimento 20

creo que elimine el template de los modelos


el injerto MTP del otro modelo me gusto con este modelo , me falta recrear este modelo para agregar el template

en algun momento lo borrare

```bash
wget -O /content/chat_template.jinja tepirale/Ornith-Agents-A1-3.6-35B-A3B-dare_ties
```

## Uso (llama.cpp con MTP)

```bash
llama-server -m Ornith-Agents-A1-3.6-35B-A3B-dare_ties-Q4_K_M.gguf \
  --spec-type draft-mtp --spec-draft-n-max 2 \
  -ngl 99 -fa on -c 65536 --jinja \
  --cache-type-k q8_0 --cache-type-v q8_0
```

```bash
/content/llama.cpp/build-cuda/bin/llama-server \
  -m /content/work/gguf/Ornith-Agents-A1-3.6-35B-A3B-dare_ties-Q4_K_M.gguf \
  --chat-template-file /content/chat_template.jinja \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --spec-draft-n-min 1 \
  --host 0.0.0.0 --port 8080 \
  --n-gpu-layers 999 \
  --ctx-size 20000 \
  --flash-attn on \
  --cont-batching \
  --parallel 4 \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --metrics
```bash

## Quants

| Quant | Tamaño |
|-------|--------|
| `Q4_K_M` | 20.5 GiB |
| `Q5_K_M` | 23.9 GiB |
| `Q6_K` | 27.4 GiB |
| `Q8_0` | 35.2 GiB |


## GPU 3090 24GB | uso 22GB libre ~2GB  | modelo Q4_K_M.gguf

 max_tok | compl_tok |  time(s) |  tok/s medido |  tok/s nativo | draft acc% |   finish
---------------------------------------------------------------------------------------
     512 |       512 |     3.11 |        164.64 | 185.81154644562835 |      58.6% |   length
    1024 |      1024 |     6.38 |         160.6 | 168.1821675659376 |      49.6% |   length
    1536 |      1536 |     9.23 |        166.42 | 174.37983333556605 |      52.5% |   length
    2048 |      2048 |    11.82 |        173.27 | 180.66270396669316 |      56.3% |   length
    2560 |      2560 |    15.35 |        166.77 | 173.19459183649997 |      52.7% |   length
    3072 |      3072 |    18.23 |        168.52 | 174.07312295115594 |      53.1% |   length
    3584 |      3584 |    21.77 |        164.61 | 170.3211904082557 |      51.6% |   length
    4096 |      4096 |    24.66 |        166.11 | 171.41060296776368 |      52.3% |   length
    4608 |      4608 |    26.66 |        172.83 | 178.31869156802577 |      56.8% |   length
    5120 |      5120 |    30.05 |        170.39 | 175.3896167291549 |      54.9% |   length
    5632 |      5632 |     34.0 |        165.65 | 170.00963609304574 |      52.2% |   length
    6144 |      6144 |    37.16 |        165.34 | 170.1135424832823 |      52.4% |   length
    6656 |      6656 |     40.8 |        163.14 | 171.59782245044528 |      53.3% |   length
    7168 |      7168 |    47.57 |        150.68 | 153.01051061387676 |      54.1% |   length