Instructions to use jackasda211233/Agents-A1-IQ4_NL with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use jackasda211233/Agents-A1-IQ4_NL with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf jackasda211233/Agents-A1-IQ4_NL:IQ4_NL
# Run inference directly in the terminal:
llama cli -hf jackasda211233/Agents-A1-IQ4_NL:IQ4_NL

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf jackasda211233/Agents-A1-IQ4_NL:IQ4_NL
# Run inference directly in the terminal:
llama cli -hf jackasda211233/Agents-A1-IQ4_NL:IQ4_NL

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf jackasda211233/Agents-A1-IQ4_NL:IQ4_NL
# Run inference directly in the terminal:
./llama-cli -hf jackasda211233/Agents-A1-IQ4_NL:IQ4_NL

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf jackasda211233/Agents-A1-IQ4_NL:IQ4_NL
# Run inference directly in the terminal:
./build/bin/llama-cli -hf jackasda211233/Agents-A1-IQ4_NL:IQ4_NL

Use Docker

docker model run hf.co/jackasda211233/Agents-A1-IQ4_NL:IQ4_NL

LM Studio
Jan

vLLM

How to use jackasda211233/Agents-A1-IQ4_NL with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jackasda211233/Agents-A1-IQ4_NL"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jackasda211233/Agents-A1-IQ4_NL",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/jackasda211233/Agents-A1-IQ4_NL:IQ4_NL

Ollama
How to use jackasda211233/Agents-A1-IQ4_NL with Ollama:
```
ollama run hf.co/jackasda211233/Agents-A1-IQ4_NL:IQ4_NL
```

Unsloth Studio

How to use jackasda211233/Agents-A1-IQ4_NL with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for jackasda211233/Agents-A1-IQ4_NL to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for jackasda211233/Agents-A1-IQ4_NL to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for jackasda211233/Agents-A1-IQ4_NL to start chatting

How to use jackasda211233/Agents-A1-IQ4_NL with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf jackasda211233/Agents-A1-IQ4_NL:IQ4_NL

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "jackasda211233/Agents-A1-IQ4_NL:IQ4_NL"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use jackasda211233/Agents-A1-IQ4_NL with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf jackasda211233/Agents-A1-IQ4_NL:IQ4_NL

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default jackasda211233/Agents-A1-IQ4_NL:IQ4_NL

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use jackasda211233/Agents-A1-IQ4_NL with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf jackasda211233/Agents-A1-IQ4_NL:IQ4_NL

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "jackasda211233/Agents-A1-IQ4_NL:IQ4_NL" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use jackasda211233/Agents-A1-IQ4_NL with Docker Model Runner:
```
docker model run hf.co/jackasda211233/Agents-A1-IQ4_NL:IQ4_NL
```

Lemonade

How to use jackasda211233/Agents-A1-IQ4_NL with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull jackasda211233/Agents-A1-IQ4_NL:IQ4_NL

Run and chat with the model

lemonade run user.Agents-A1-IQ4_NL-IQ4_NL

List all available models

lemonade list

Agents-A1 IQ4_NL GGUF

An importance-matrix-calibrated IQ4_NL quantization of InternScience/Agents-A1, a 35B-parameter Mixture-of-Experts model (3B active) built on Qwen3.5-MoE and trained for long-horizon agentic tasks — multi-step search, engineering, scientific research, and tool-calling.

About the base model

Agents-A1 is developed by the AI for Science Center at Shanghai AI Laboratory (InternScience). It uses a "scale the horizon, not the parameters" approach — training on synthetic long-horizon trajectories (avg 45K tokens) via multi-teacher domain-routed on-policy distillation across 6 heterogeneous agent domains. See the original model card and paper for full details.

Architecture: Qwen3.5-MoE (hybrid linear-attention + full-attention, 256 experts with 8 active per token, 1 MTP layer), 262K context, multimodal (vision encoder included separately in the original repo).

Quantization

This is an IQ4_NL (4.50 bpw non-linear) quantization, calibrated with an importance matrix to preserve coding and reasoning quality at reduced bit-width. Produced using llama-quantize from the official Q8_0 GGUF with --allow-requantize --imatrix.

Quant	Size	BPW
Official Q8_0	35 GB	8.5
Official Q4_K_M	21 GB	4.87
This — IQ4_NL	20 GB	4.50

Evaluation

The model was tested on an autonomous coding-agent eval harness with 11 tasks across two categories:

Coding tasks (7) — algorithm implementation from spec

The agent receives a README + test file, then autonomously reads files, writes code, runs tests, and debugs using a tool-call loop (<read>, <write>, <bash> tags with pytest verification).

Research & debugging tasks (4) — diagnostic reasoning

Custom tasks designed to test multi-step debugging and research synthesis:

Debug tasks: bugged codebases with subtle logic errors (off-by-one, stale state, contract violations across modules)
Research tasks: contradicting specifications that must be reconciled into a correct implementation

Results — all three quants

All quants tested with the same seed (42), temperature (0.6), top-p (0.95), max 25 turns per task, 8192 max new tokens.

Task	Type	Q8_0	Q4_K_M	IQ4_NL
debug_ratelimit	Debug	11/11 ✅ (8 turns)	11/11 ✅ (21 turns)	11/11 ✅ (6 turns)
debug_fsm	Debug	13/13 ✅ (25 turns)	13/13 ✅ (17 turns)	13/13 ✅ (25 turns)
research_logspec	Research	18/18 ✅ (18 turns)	18/18 ✅ (13 turns)	18/18 ✅ (25 turns)
research_wire	Research	15/15 ✅ (12 turns)	15/15 ✅ (16 turns)	15/15 ✅ (12 turns)
minijson	Coding	8/8 ✅ (8 turns)	8/8 ✅ (9 turns)	8/8 ✅ (7 turns)
graph	Coding	7/7 ✅ (6 turns)	7/7 ✅ (5 turns)	7/7 ✅ (6 turns)
tracker	Coding	9/9 ✅ (13 turns)	9/9 ✅ (9 turns)	9/9 ✅ (8 turns)
mdlist	Coding	9/9 ✅ (6 turns)	9/9 ✅ (6 turns)	9/9 ✅ (6 turns)
taskq	Coding	6/6 ✅ (23 turns)	6/6 ✅ (8 turns)	6/6 ✅ (6 turns)
lru	Coding	5/5 ✅ (5 turns)	5/5 ✅ (5 turns)	5/5 ✅ (5 turns)
calc	Coding	7/7 ✅ (12 turns)	7/7 ✅ (9 turns)	7/7 ✅ (11 turns)

Summary	Q8_0	Q4_K_M	IQ4_NL
Pass rate	11/11 (100%)	11/11 (100%)	11/11 (100%)
Tests passed	108/108	108/108	108/108
Total turns	142	118	117
Avg turns/task	12.9	10.7	10.6
Size	35 GB	21 GB	20 GB

Takeaway: IQ4_NL matches or exceeds both higher-bit quants on task-level quality while being the smallest. The imatrix calibration improved turn efficiency on diagnostic tasks (notably debug_ratelimit: 6 turns vs 21 for Q4_K_M, and taskq: 6 turns vs 23 for Q8_0).

Usage

llama-server \
  -m Agents-A1-IQ4_NL.gguf \
  --port 8080 \
  -ngl 99 \
  -c 32768 \
  -b 512 -ub 128 \
  -ctk f16 -ctv f16 \
  --jinja \
  --reasoning-format deepseek

This is a reasoning model — it generates <think> blocks. The --reasoning-format deepseek flag separates reasoning into a reasoning_content field on the API.

For multimodal use, pair with the mmproj from LordNeel/Agents-A1-GGUF.

Credits

Base model: InternScience/Agents-A1 by the AI for Science Center at Shanghai AI Laboratory
Paper: Scaling the Horizon, Not the Parameters
Q8_0 source: InternScience/Agents-A1-Q8_0-GGUF
License: Apache 2.0 (inherited from base model)

License

Apache 2.0 — same as the base model.

Downloads last month: 455

GGUF

Model size

35B params

Architecture

qwen35moe

Hardware compatibility

4-bit

Model tree for jackasda211233/Agents-A1-IQ4_NL

Base model

InternScience/Agents-A1

Quantized

(70)

this model

Paper for jackasda211233/Agents-A1-IQ4_NL

Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent

Paper • 2606.30616 • Published Jun 29 • 103