alexop1000/Agents-A1-Q8_0-GGUF

Agents-A1 — Q8_0 GGUF (with vision)

GGUF quantization of InternScience/Agents-A1 for llama.cpp, including the vision projector so the model keeps its multimodal (image + text) capability.

File	Type	Size	Notes
agents-a1-q8_0.gguf	text model, Q8_0	36.9 GB	chat template + tokenizer embedded
mmproj-Agents-A1-f16.gguf	vision projector, F16	0.9 GB	required for image input
chat_template.jinja	chat template	—	already baked into the GGUF; loose copy for --chat-template-file overrides

Converted with llama.cpp/convert_hf_to_gguf.py (architecture qwen35moe, registered upstream as Qwen3_5MoeForConditionalGeneration) using --no-mtp.

The upstream config.json declares mtp_num_hidden_layers: 1, but the published checkpoint contains no MTP / nextn weights (no layers.40.*, no eh_proj/enorm/hnorm/shared_head; highest layer index is 39). Any converter that honors the config bumps block_count to 41 and writes nextn_predict_layers, yet cannot fill block 40 — yielding a file that declares 41 blocks but ships 40, which fails to load with missing tensor 'blk.40.attn_norm.weight'. --no-mtp keeps block_count=40 to match the 40 real layers that actually shipped. MTP is only a speculative-decoding accelerator, so nothing is lost for normal inference.

The chat template embedded in the GGUF is byte-identical to the upstream chat_template.jinja and includes the vision-token, tool-call, and <think> reasoning logic.

Agents-A1 uses a hybrid linear-attention + MoE architecture (Qwen3.5-MoE family). You need a recent llama.cpp build that supports the qwen35moe inference graph — converter support landing does not guarantee your local binary can run it. If the model fails to load, update and rebuild llama.cpp from master.

Usage

Text + vision (recommended)

Download both GGUFs, then point --mmproj at the projector:

llama-server -m agents-a1-q8_0.gguf --mmproj mmproj-Agents-A1-f16.gguf -c 8192 -ngl 99

Then send images via the web UI or the OpenAI-compatible image_url message format.

Text only

llama-cli -m agents-a1-q8_0.gguf -p "Hello"

Omitting --mmproj runs the language model alone and frees the compute/KV the vision encoder would use.

---

About the original model

Agents-A1 is a 35B Mixture-of-Experts agentic model from InternScience, built to scale heterogeneous agentic abilities across long-horizon search, engineering, scientific research, instruction following, and tool-calling. Despite the ~35B class, it reports competitive-to-SOTA results against much larger frontier systems on benchmarks such as Seal-0, HiPhO, FrontierScience, IFBench, BrowseComp, and GAIA.

• 🏠 Homepage: https://internscience.github.io/Agents-A1/
• 📄 Technical Report: https://arxiv.org/abs/2606.30616
• 💻 GitHub: https://github.com/InternScience/Agents-A1

See the original model card for full details, benchmarks, and licensing. All credit for the model goes to InternScience; this repo only provides GGUF conversions. License: Apache-2.0.