jackasda211233/Agents-A1-IQ4_NL

Agents-A1 IQ4_NL GGUF

An importance-matrix-calibrated IQ4_NL quantization of InternScience/Agents-A1, a 35B-parameter Mixture-of-Experts model (3B active) built on Qwen3.5-MoE and trained for long-horizon agentic tasks — multi-step search, engineering, scientific research, and tool-calling.

About the base model

Agents-A1 is developed by the AI for Science Center at Shanghai AI Laboratory (InternScience). It uses a "scale the horizon, not the parameters" approach — training on synthetic long-horizon trajectories (avg 45K tokens) via multi-teacher domain-routed on-policy distillation across 6 heterogeneous agent domains. See the original model card and paper for full details.

Architecture: Qwen3.5-MoE (hybrid linear-attention + full-attention, 256 experts with 8 active per token, 1 MTP layer), 262K context, multimodal (vision encoder included separately in the original repo).

Quantization

This is an IQ4_NL (4.50 bpw non-linear) quantization, calibrated with an importance matrix to preserve coding and reasoning quality at reduced bit-width. Produced using llama-quantize from the official Q8_0 GGUF with --allow-requantize --imatrix.

Quant	Size	BPW
Official Q8_0	35 GB	8.5
Official Q4_K_M	21 GB	4.87
This — IQ4_NL	20 GB	4.50

Evaluation

The model was tested on an autonomous coding-agent eval harness with 11 tasks across two categories:

Coding tasks (7) — algorithm implementation from spec

The agent receives a README + test file, then autonomously reads files, writes code, runs tests, and debugs using a tool-call loop (<read>, <write>, <bash> tags with pytest verification).

Research & debugging tasks (4) — diagnostic reasoning

Custom tasks designed to test multi-step debugging and research synthesis:

• Debug tasks: bugged codebases with subtle logic errors (off-by-one, stale state, contract violations across modules)
• Research tasks: contradicting specifications that must be reconciled into a correct implementation

Results — all three quants

All quants tested with the same seed (42), temperature (0.6), top-p (0.95), max 25 turns per task, 8192 max new tokens.

Task	Type	Q8_0	Q4_K_M	IQ4_NL
debug_ratelimit	Debug	11/11 ✅ (8 turns)	11/11 ✅ (21 turns)	11/11 ✅ (6 turns)
debug_fsm	Debug	13/13 ✅ (25 turns)	13/13 ✅ (17 turns)	13/13 ✅ (25 turns)
research_logspec	Research	18/18 ✅ (18 turns)	18/18 ✅ (13 turns)	18/18 ✅ (25 turns)
research_wire	Research	15/15 ✅ (12 turns)	15/15 ✅ (16 turns)	15/15 ✅ (12 turns)
minijson	Coding	8/8 ✅ (8 turns)	8/8 ✅ (9 turns)	8/8 ✅ (7 turns)
graph	Coding	7/7 ✅ (6 turns)	7/7 ✅ (5 turns)	7/7 ✅ (6 turns)
tracker	Coding	9/9 ✅ (13 turns)	9/9 ✅ (9 turns)	9/9 ✅ (8 turns)
mdlist	Coding	9/9 ✅ (6 turns)	9/9 ✅ (6 turns)	9/9 ✅ (6 turns)
taskq	Coding	6/6 ✅ (23 turns)	6/6 ✅ (8 turns)	6/6 ✅ (6 turns)
lru	Coding	5/5 ✅ (5 turns)	5/5 ✅ (5 turns)	5/5 ✅ (5 turns)
calc	Coding	7/7 ✅ (12 turns)	7/7 ✅ (9 turns)	7/7 ✅ (11 turns)

Summary	Q8_0	Q4_K_M	IQ4_NL
Pass rate	11/11 (100%)	11/11 (100%)	11/11 (100%)
Tests passed	108/108	108/108	108/108
Total turns	142	118	117
Avg turns/task	12.9	10.7	10.6
Size	35 GB	21 GB	20 GB

Takeaway: IQ4_NL matches or exceeds both higher-bit quants on task-level quality while being the smallest. The imatrix calibration improved turn efficiency on diagnostic tasks (notably debug_ratelimit: 6 turns vs 21 for Q4_K_M, and taskq: 6 turns vs 23 for Q8_0).

Usage

llama-server \
  -m Agents-A1-IQ4_NL.gguf \
  --port 8080 \
  -ngl 99 \
  -c 32768 \
  -b 512 -ub 128 \
  -ctk f16 -ctv f16 \
  --jinja \
  --reasoning-format deepseek

This is a reasoning model — it generates <think> blocks. The --reasoning-format deepseek flag separates reasoning into a reasoning_content field on the API.

For multimodal use, pair with the mmproj from LordNeel/Agents-A1-GGUF.

Credits

• Base model: InternScience/Agents-A1 by the AI for Science Center at Shanghai AI Laboratory
• Paper: Scaling the Horizon, Not the Parameters
• Q8_0 source: InternScience/Agents-A1-Q8_0-GGUF
• License: Apache 2.0 (inherited from base model)

License

Apache 2.0 — same as the base model.