Agents-A1 IQ2_M GGUF (imatrix) + agentic SWE eval + context Modelfiles

.gitattributes

@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+Agents-A1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
+Agents-A1.imatrix filter=lfs diff=lfs merge=lfs -text
+banner.png filter=lfs diff=lfs merge=lfs -text

Agents-A1-IQ2_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6d4f33072cf991885b823b30b04aa0d4a7ac649a575803c392d7d403c8f7d156
+size 11659235200

Agents-A1.imatrix

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:511924f1da9b8d05a7485b7bbe0bd3a152c4c6ba624c4472bdf408875bd37176
+size 192223904

CONTEXT.md

@@ -0,0 +1,26 @@
+# Context size guide — Agents-A1-IQ2_M
+
+The model's native context is **262,144 tokens** (256K). You almost never need that much, and KV-cache memory grows with the context window, so pick the smallest window that fits your task.
+
+## Ready-made Modelfiles (Ollama)
+| file | num_ctx | use for |
+|---|---|---|
+| `Modelfile.8k`   | 8,192   | quick Q&A, short snippets — lightest KV |
+| `Modelfile.32k`  | 32,768  | most agentic/coding sessions (recommended default) |
+| `Modelfile.128k` | 131,072 | long files / large repos |
+| `Modelfile.native` | 262,144 | full native context (heaviest KV, needs lots of RAM) |
+
+```bash
+# build a context-sized model in Ollama
+ollama create agents-a1-32k -f Modelfile.32k
+ollama run agents-a1-32k
+```
+
+## llama.cpp
+```bash
+llama-server -m Agents-A1-IQ2_M.gguf -c 32768 --jinja --temp 1.0 --top-p 0.95
+# bump -c up to 262144 for the full window
+```
+
+## Rough KV-cache cost
+KV memory scales roughly linearly with `num_ctx`. On top of the ~11.7 GB of weights, budget a few hundred MB at 8K, ~1–2 GB at 32K, and several GB at 128K+. If you hit OOM, lower `num_ctx` first.

Modelfile.128k

@@ -0,0 +1,4 @@
+FROM ./Agents-A1-IQ2_M.gguf
+PARAMETER num_ctx 131072
+PARAMETER temperature 1.0
+PARAMETER top_p 0.95

Modelfile.32k

@@ -0,0 +1,4 @@
+FROM ./Agents-A1-IQ2_M.gguf
+PARAMETER num_ctx 32768
+PARAMETER temperature 1.0
+PARAMETER top_p 0.95

Modelfile.8k

@@ -0,0 +1,4 @@
+FROM ./Agents-A1-IQ2_M.gguf
+PARAMETER num_ctx 8192
+PARAMETER temperature 1.0
+PARAMETER top_p 0.95

Modelfile.native

@@ -0,0 +1,4 @@
+FROM ./Agents-A1-IQ2_M.gguf
+PARAMETER num_ctx 262144
+PARAMETER temperature 1.0
+PARAMETER top_p 0.95

README.md

@@ -0,0 +1,76 @@
+---
+license: apache-2.0
+base_model: InternScience/Agents-A1
+tags:
+  - gguf
+  - ollama
+  - llama.cpp
+  - agentic-coding
+  - reasoning
+  - qwen3_5_moe
+  - moe
+language:
+  - en
+---
+
+![banner](banner.png)
+
+# Agents-A1 — IQ2_M GGUF
+
+> **InternScience's Agents-A1** (a ~35B-A3B hybrid-attention MoE agent model) quantized to a single **IQ2_M GGUF (~11.7 GB)** that runs on a 16 GB machine. It **solves real SWE-bench Verified tasks** in an agentic Claude-Code-style loop. This is **InternScience's model** — ours is the quant + imatrix + eval + ready-to-run Modelfiles. (No other GGUF existed when this was published, so this is the first-mover GGUF; if unsloth/others ship one, ours adds an imatrix, a documented agentic eval, and configurable-context Modelfiles.)
+
+## 📦 Which quant should I pick?
+This repo ships the **IQ2_M** — the sweet spot for a ~35B MoE: smallest that still runs real agentic loops sharply, made with an importance matrix (imatrix).
+
+| file | bits | size | pick this if… |
+|---|---|---|---|
+| `Agents-A1-IQ2_M.gguf` | ~2.7 | **~11.7 GB** | **the one in this repo** — fits 16 GB RAM/VRAM, still does agentic coding |
+
+<!-- Quant names: K-quants (Q*_K) = good general quants. I-quants (IQ*) = smaller at the same quality but need an imatrix (we ship `Agents-A1.imatrix`) and a bit more CPU. Higher number = more bits = better + bigger. -->
+
+## 🧮 Will it fit? (RAM/VRAM cheat-sheet)
+Total memory ≈ weights (~11.7 GB) + KV-cache (grows with context).
+| you have | comfortable context |
+|---|---|
+| 16 GB | ~8–32K |
+| 24 GB | ~64–128K |
+| 32 GB+ | up to native 256K |
+
+If you OOM, lower `num_ctx` (see [`CONTEXT.md`](CONTEXT.md)).
+
+## 🧠 Context size (native 262,144 / 256K)
+Configurable — Modelfiles at **8K / 32K / 128K / native** are included; full guide in [`CONTEXT.md`](CONTEXT.md). 32K is the recommended default for agentic/coding work.
+
+## 🚀 How to run it
+```bash
+# Ollama (build a 32K-context model from the included Modelfile)
+ollama create agents-a1-32k -f Modelfile.32k
+ollama run agents-a1-32k
+
+# llama.cpp (use a recent build — this is a qwen3_5_moe hybrid-attention MoE)
+llama-server -m Agents-A1-IQ2_M.gguf -c 32768 --jinja --temp 1.0 --top-p 0.95
+```
+**Recommended sampling**: temperature **1.0**, top_p **0.95**. This is a reasoning model — it emits a thinking channel before its answer; keep temperature high (≈1.0). Low temperature (≤0.2) tends to cause repetition loops.
+
+**Agentic use**: native tool-calling works out of the box with the model's bundled chat template — pass your tools via the standard OpenAI/`tools` parameter (or Anthropic-style via a bridge) and it returns proper `tool_calls`. No text-injection workaround needed.
+
+> ⚠️ **Needs a recent llama.cpp.** Agents-A1 is a `qwen3_5_moe` hybrid (gated-delta-net linear-attention layers + MoE, every-4th-layer full attention). It requires a llama.cpp build new enough to include the `qwen35moe` graph. Older builds will fail to load it.
+
+## ⚠️ Good to know
+- **Strengths**: agentic coding / tool use — runs clean multi-turn Claude-Code-style loops, makes decisive single tool calls per turn, doesn't degenerate. Long native context (256K). Efficient: ~3B active params (MoE) so it's fast for its size.
+- **Limits**: this is a **2-bit** quant of a small-active-params MoE — it will not match a full-precision 70B. On the hardest, multi-file SWE tasks it often explores well but doesn't always land the exact fix. Use ≥ Q4 if you have the RAM and need max fidelity (not shipped here; quantize from the source if needed).
+- **Quirk**: emits an internal reasoning/thinking channel; if your client shows it, you can strip everything before the final answer.
+
+## 📊 Evaluation methodology (MANDATORY — never lose how it was measured)
+- **Score**: **1/5** on a fail-fast **swe-mix pre-screen** subset (5 SWE-bench Verified django instances). Resolved: **django__django-11099** (1/1 tests). The gate to publish is **resolve ≥ 1** — this build passed. All 5 ran clean agentic loops (37–39 tool calls each, **0 empty/degenerate turns**, no crash or loop).
+- **Exact instances** (pre-screen): `django__django-11133` (✗), `django__django-11099` (✓), `django__django-13590` (✗), `django__django-13933` (✗), `django__django-14608` (✗). The full 30-instance discriminating probe spec is in [`swe_mix.json`](swe_mix.json) — 6 continuity anchors + 24 discriminating (10 repos); the pre-screen runs the first 5.
+- **Harness**: Claude Code + agent-bridge.js (Anthropic→OpenAI translation, context eviction) → llama.cpp `llama-server`, in terminal-bench / Docker, native `tools` calling, 80-message cap per task.
+- **Sampling**: temperature **1.0**, top_p **0.95**. **Tool format**: native (`tools` param; not bypassed — the model's chat template emits proper `tool_calls`).
+- **Quant**: **IQ2_M**, imatrix computed on a code+general corpus (273×512-tok chunks, final PPL ≈ 1.82 on the calib set), **11.66 GB**, requantized from a near-lossless Q8_0 intermediate.
+- **Date**: 2026-06-29. *Caveat: small/hard local probe, relative ranking — NOT 1:1 with the official SWE-bench leaderboard.*
+
+## 🗒️ Changelog
+- 2026-06-29 v1: initial IQ2_M release. SWE-mix pre-screen: **1/5** (resolved `django__django-11099`, 1/1 tests) — gate passed. Clean agentic loops on all 5 (native tool calls, 0 degenerate turns). Probe spec in `swe_mix.json`.
+
+## 📚 Credit & license
+Model, weights, and training: **© InternScience** — [InternScience/Agents-A1](https://huggingface.co/InternScience/Agents-A1). Quant (IQ2_M) + importance matrix + agentic eval + context Modelfiles: KikoCis. **apache-2.0** (same as upstream). No weights were modified beyond quantization — this is a faithful re-quant, not a fine-tune.

swe_mix.json

@@ -0,0 +1,157 @@
+{
+  "description": "6 continuity-anchor instances (the original probe, comparable to all prior experiments) + 24 discriminating (15x '1-4h' + 9x '15min-1h'). 10+ repos.",
+  "anchor_6": [
+    {
+      "instance_id": "django__django-11133",
+      "repo": "django/django",
+      "difficulty": "<15 min fix"
+    },
+    {
+      "instance_id": "django__django-11099",
+      "repo": "django/django",
+      "difficulty": "<15 min fix"
+    },
+    {
+      "instance_id": "django__django-13590",
+      "repo": "django/django",
+      "difficulty": "15 min - 1 hour"
+    },
+    {
+      "instance_id": "django__django-13933",
+      "repo": "django/django",
+      "difficulty": "<15 min fix"
+    },
+    {
+      "instance_id": "django__django-14608",
+      "repo": "django/django",
+      "difficulty": "<15 min fix"
+    },
+    {
+      "instance_id": "django__django-14752",
+      "repo": "django/django",
+      "difficulty": "<15 min fix"
+    }
+  ],
+  "discriminating_24": [
+    {
+      "instance_id": "astropy__astropy-13398",
+      "repo": "astropy/astropy",
+      "difficulty": "1-4 hours"
+    },
+    {
+      "instance_id": "astropy__astropy-13579",
+      "repo": "astropy/astropy",
+      "difficulty": "1-4 hours"
+    },
+    {
+      "instance_id": "astropy__astropy-14369",
+      "repo": "astropy/astropy",
+      "difficulty": "1-4 hours"
+    },
+    {
+      "instance_id": "django__django-10554",
+      "repo": "django/django",
+      "difficulty": "1-4 hours"
+    },
+    {
+      "instance_id": "django__django-11138",
+      "repo": "django/django",
+      "difficulty": "1-4 hours"
+    },
+    {
+      "instance_id": "django__django-11400",
+      "repo": "django/django",
+      "difficulty": "1-4 hours"
+    },
+    {
+      "instance_id": "pydata__xarray-3993",
+      "repo": "pydata/xarray",
+      "difficulty": "1-4 hours"
+    },
+    {
+      "instance_id": "pylint-dev__pylint-4551",
+      "repo": "pylint-dev/pylint",
+      "difficulty": "1-4 hours"
+    },
+    {
+      "instance_id": "pylint-dev__pylint-8898",
+      "repo": "pylint-dev/pylint",
+      "difficulty": "1-4 hours"
+    },
+    {
+      "instance_id": "pytest-dev__pytest-10356",
+      "repo": "pytest-dev/pytest",
+      "difficulty": "1-4 hours"
+    },
+    {
+      "instance_id": "pytest-dev__pytest-5787",
+      "repo": "pytest-dev/pytest",
+      "difficulty": "1-4 hours"
+    },
+    {
+      "instance_id": "pytest-dev__pytest-6197",
+      "repo": "pytest-dev/pytest",
+      "difficulty": "1-4 hours"
+    },
+    {
+      "instance_id": "scikit-learn__scikit-learn-25102",
+      "repo": "scikit-learn/scikit-learn",
+      "difficulty": "1-4 hours"
+    },
+    {
+      "instance_id": "sphinx-doc__sphinx-11510",
+      "repo": "sphinx-doc/sphinx",
+      "difficulty": "1-4 hours"
+    },
+    {
+      "instance_id": "sphinx-doc__sphinx-8548",
+      "repo": "sphinx-doc/sphinx",
+      "difficulty": "1-4 hours"
+    },
+    {
+      "instance_id": "astropy__astropy-12907",
+      "repo": "astropy/astropy",
+      "difficulty": "15 min - 1 hour"
+    },
+    {
+      "instance_id": "astropy__astropy-13033",
+      "repo": "astropy/astropy",
+      "difficulty": "15 min - 1 hour"
+    },
+    {
+      "instance_id": "django__django-10973",
+      "repo": "django/django",
+      "difficulty": "15 min - 1 hour"
+    },
+    {
+      "instance_id": "django__django-11087",
+      "repo": "django/django",
+      "difficulty": "15 min - 1 hour"
+    },
+    {
+      "instance_id": "matplotlib__matplotlib-14623",
+      "repo": "matplotlib/matplotlib",
+      "difficulty": "15 min - 1 hour"
+    },
+    {
+      "instance_id": "matplotlib__matplotlib-20488",
+      "repo": "matplotlib/matplotlib",
+      "difficulty": "15 min - 1 hour"
+    },
+    {
+      "instance_id": "mwaskom__seaborn-3069",
+      "repo": "mwaskom/seaborn",
+      "difficulty": "15 min - 1 hour"
+    },
+    {
+      "instance_id": "mwaskom__seaborn-3187",
+      "repo": "mwaskom/seaborn",
+      "difficulty": "15 min - 1 hour"
+    },
+    {
+      "instance_id": "psf__requests-2931",
+      "repo": "psf/requests",
+      "difficulty": "15 min - 1 hour"
+    }
+  ]
+}