--- license: apache-2.0 arxiv: 2608.23283 language: - en - zh base_model: apodex/Apodex-1.1-mini base_model_relation: quantized pipeline_tag: text-generation library_name: gguf tags: - gguf - llama.cpp - qwen3.5 - moe - multimodal - mtp - apodex --- # Apodex-1.1-mini GGUF GGUF quantizations of [apodex/Apodex-1.1-mini](https://huggingface.co/apodex/Apodex-1.1-mini), a 35.95B-parameter Qwen3.5 MoE model designed for research, data, files, code, and tool-driven work. Every model file includes the checkpoint's native MTP/NextN head. Image input requires the F16 vision projector included in this repository. ## Q4 Selective Q4 Selective is a compact custom Q4 build. It uses importance-matrix-guided expert quantization and preserves selected tensors at higher precision. It is an alternative to the standard Q4_K_M; Q4_K_M remains the recommended default. ## Upstream benchmarks Both figures are reproduced from the upstream model card. They report evaluations performed with [FrontierAgent](https://github.com/ApodexAI/FrontierAgent), Apodex's open-source agent runtime and evaluation suite, using its ReAct and Agent Team workflows. They do not measure the GGUF files in this repository. ![Apodex-1.1 benchmark results](./assets/apodex-1.1-benchmarks.png) *Upstream Apodex-1.1 evaluation results using FrontierAgent's ReAct and Agent Team workflows.* ![Apodex-1.1-mini benchmark results](./assets/apodex-1.1-mini-benchmarks.png) *Upstream Apodex-1.1-mini evaluation results using FrontierAgent. With the Agent Team workflow, the model scored 27.7 on APEX-Agent, 50.2 on FrontierFinance, and 51.7 on FrontierScience-Research.* ## Files | File | Size | Notes | |---|---:|---| | `Apodex-1.1-mini-IQ1_M.gguf` | 8.82 GB | Ultra-low-memory, importance-aware build; substantial quality trade-off | | `Apodex-1.1-mini-IQ2_M.gguf` | 12.2 GB | Importance-aware; smallest recommended build | | `Apodex-1.1-mini-Q2_K.gguf` | 13.2 GB | Maximum-compression legacy K-quant | | `Apodex-1.1-mini-IQ3_M.gguf` | 16.0 GB | Importance-aware balance for constrained systems | | `Apodex-1.1-mini-Q3_K_M.gguf` | 17.2 GB | Standard Q3 K-quant | | `Apodex-1.1-mini-Q4_K_S.gguf` | 20.4 GB | Smaller Q4 option | | `Apodex-1.1-mini-Q4_Selective-Q4_K_M.gguf` | 19.8 GB | Compact custom Q4 alternative | | `Apodex-1.1-mini-Q4_K_M.gguf` | 21.7 GB | Recommended default | | `Apodex-1.1-mini-Q5_K_S.gguf` | 24.6 GB | Smaller Q5 option | | `Apodex-1.1-mini-Q5_K_M.gguf` | 25.3 GB | Higher-quality Q5 option | | `Apodex-1.1-mini-Q6_K.gguf` | 29.2 GB | High quality | | `Apodex-1.1-mini-Q8_0.gguf` | 37.8 GB | Near-lossless reference quantization | | `mmproj-Apodex-1.1-mini-F16.gguf` | 899 MB | F16 vision projector | | `imatrix/Apodex-1.1-mini-imatrix-Q6_K-wiki-100x512.gguf` | 192 MB | Importance matrix used by the IQ builds | IQ1_M, IQ2_M, and IQ3_M use an importance matrix. Their embedded MTP layer is retained at Q5_K because that auxiliary layer is not exercised while collecting the matrix. IQ1_M is a 1.75-bit-per-weight format intended for severe memory constraints; use IQ2_M or higher when possible. ## Hardware guidance | Available VRAM | Suggested starting point | Expected placement | |---:|---|---| | 8 GB | IQ1_M | CPU/GPU hybrid offload | | 12 GB | IQ1_M or IQ2_M | IQ1_M full offload; IQ2_M may require hybrid offload | | 16 GB | IQ2_M or IQ3_M | IQ2_M full offload; IQ3_M may require hybrid offload | These are starting points, not guarantees. Context length, KV-cache type, backend overhead, and other GPU workloads also consume memory. Prefer Q4_K_M or higher when system memory permits. ## Chat template The included `chat_template.jinja` is synchronized with [upstream revision `62583b4`](https://huggingface.co/apodex/Apodex-1.1-mini/commit/62583b464707ad58848bc2ab066747d8d5b02533). Use it with a current llama.cpp build to enable the upstream identity, thinking-history, and tool-argument behavior. ```bash llama-server -m Apodex-1.1-mini-Q4_K_M.gguf --jinja --chat-template-file chat_template.jinja ``` ## Usage Use llama.cpp commit `f280b26983ad0fdb705a0d9ebf0503e76f2899b0` or a newer compatible revision. Text generation: ```bash llama-cli \ -m Apodex-1.1-mini-Q4_K_M.gguf \ -c 4096 -n 512 --temp 1.0 --top-p 0.95 \ --jinja --chat-template-file chat_template.jinja \ -p "Explain why reproducible builds matter." ``` MTP speculative decoding requires no separate draft model: ```bash llama-server \ -m Apodex-1.1-mini-Q4_K_M.gguf \ -c 32768 --host 0.0.0.0 --port 8080 \ --spec-type draft-mtp --spec-draft-n-max 3 \ --jinja --chat-template-file chat_template.jinja ``` Image input: ```bash llama-mtmd-cli \ -m Apodex-1.1-mini-Q4_K_M.gguf \ -mm mmproj-Apodex-1.1-mini-F16.gguf \ --image image.jpg -p "Describe this image." \ -c 4096 -n 256 ``` ## Checksums | File | SHA-256 | |---|---| | `Apodex-1.1-mini-IQ1_M.gguf` | `1e84d8adf7837e96fb18712882a8a114becc7e53554372e2f612c5e0c6276cd4` | | `Apodex-1.1-mini-IQ2_M.gguf` | `450afff9cd60e19ad485b4be4c22adf561df35072e4b4879bcc981e6fad88a8f` | | `Apodex-1.1-mini-Q2_K.gguf` | `cbe8ea39285cf03d24d119ac69f6a2d9f126777d31c7486138cbaea7b6897619` | | `Apodex-1.1-mini-IQ3_M.gguf` | `8620c43276492c59be49269b0cce52ca4f6698c73154751274fa73eb831fb38a` | | `Apodex-1.1-mini-Q3_K_M.gguf` | `c8c51f058f2fb4be7f064dedd0dcd8c07eb690dbc985335bb7f15f3b5754381b` | | `Apodex-1.1-mini-Q4_K_S.gguf` | `9d0e92ea8b9de608de713b960bb2005277636a2d0ba3c815d5f2e58cb4b97c6a` | | `Apodex-1.1-mini-Q4_Selective-Q4_K_M.gguf` | `1226a6dad081cddc4406aa6376152364b1d991cfecc51df2ba485770c336e0f2` | | `Apodex-1.1-mini-Q4_K_M.gguf` | `13a580b1ab9350b90085c221cf66e8715a594b2428e5a80a7a956d8e0420be16` | | `Apodex-1.1-mini-Q5_K_S.gguf` | `69df27cae68b3c012c9e1d508835d62d51044c55b735a14eca397c8375619e8c` | | `Apodex-1.1-mini-Q5_K_M.gguf` | `b55bea4022cd4963af12f4446887966ebcdbf0ea1f326c79fcf8abba48f38cb7` | | `Apodex-1.1-mini-Q6_K.gguf` | `a29a4071b7a034e7f15a8379dd2d70407dbeb20ab4d1340bf4954fc747a4ed73` | | `Apodex-1.1-mini-Q8_0.gguf` | `75340c1561b8e6cb8b6314299a6c4a865ba49f0109e2193dcef48d3478564068` | | `mmproj-Apodex-1.1-mini-F16.gguf` | `d0ccf814138020651bcab603d9ce5c080a0ada46f482c116f0aca0fbb84e091e` | | `imatrix/Apodex-1.1-mini-imatrix-Q6_K-wiki-100x512.gguf` | `ecc7798b0c08125e4cfc3f25ba060f5bc46ea09cccca7f558aaacebf6dd5d244` | ```bash sha256sum -c SHA256SUMS ``` ## License The source model was released by [Apodex AI](https://huggingface.co/apodex) under the [Apache License 2.0](https://huggingface.co/apodex/Apodex-1.1-mini/blob/27479c1e48453eb8172e03aaeb54eab8ce9419d6/LICENSE). The same license and attribution requirements apply to these conversions.