Instructions to use InternScience/Agents-A1-Q4_K_M-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use InternScience/Agents-A1-Q4_K_M-GGUF with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="InternScience/Agents-A1-Q4_K_M-GGUF")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("InternScience/Agents-A1-Q4_K_M-GGUF", dtype="auto")

llama-cpp-python

How to use InternScience/Agents-A1-Q4_K_M-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="InternScience/Agents-A1-Q4_K_M-GGUF",
	filename="Agents-A1-Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use InternScience/Agents-A1-Q4_K_M-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf InternScience/Agents-A1-Q4_K_M-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf InternScience/Agents-A1-Q4_K_M-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf InternScience/Agents-A1-Q4_K_M-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf InternScience/Agents-A1-Q4_K_M-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf InternScience/Agents-A1-Q4_K_M-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf InternScience/Agents-A1-Q4_K_M-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf InternScience/Agents-A1-Q4_K_M-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf InternScience/Agents-A1-Q4_K_M-GGUF:Q4_K_M

Use Docker

docker model run hf.co/InternScience/Agents-A1-Q4_K_M-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use InternScience/Agents-A1-Q4_K_M-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "InternScience/Agents-A1-Q4_K_M-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "InternScience/Agents-A1-Q4_K_M-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/InternScience/Agents-A1-Q4_K_M-GGUF:Q4_K_M

SGLang

How to use InternScience/Agents-A1-Q4_K_M-GGUF with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "InternScience/Agents-A1-Q4_K_M-GGUF" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "InternScience/Agents-A1-Q4_K_M-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "InternScience/Agents-A1-Q4_K_M-GGUF" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "InternScience/Agents-A1-Q4_K_M-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use InternScience/Agents-A1-Q4_K_M-GGUF with Ollama:
```
ollama run hf.co/InternScience/Agents-A1-Q4_K_M-GGUF:Q4_K_M
```

Unsloth Studio

How to use InternScience/Agents-A1-Q4_K_M-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for InternScience/Agents-A1-Q4_K_M-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for InternScience/Agents-A1-Q4_K_M-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for InternScience/Agents-A1-Q4_K_M-GGUF to start chatting

How to use InternScience/Agents-A1-Q4_K_M-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf InternScience/Agents-A1-Q4_K_M-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "InternScience/Agents-A1-Q4_K_M-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use InternScience/Agents-A1-Q4_K_M-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf InternScience/Agents-A1-Q4_K_M-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default InternScience/Agents-A1-Q4_K_M-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use InternScience/Agents-A1-Q4_K_M-GGUF with Docker Model Runner:
```
docker model run hf.co/InternScience/Agents-A1-Q4_K_M-GGUF:Q4_K_M
```

Lemonade

How to use InternScience/Agents-A1-Q4_K_M-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull InternScience/Agents-A1-Q4_K_M-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Agents-A1-Q4_K_M-GGUF-Q4_K_M

List all available models

lemonade list

Agents-A1: Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent

🏠 Homepage |

Technical Report

Hugging Face |

Github |

ModelScope

This repository contains model weights and configuration files for Agents-A1 in the Hugging Face Transformers format.

These artifacts are compatible with Hugging Face Transformers, vLLM, SGLang, etc.

Agents‑A1 is a 35B Mixture‑of‑Experts agentic model from InternScience, built to scale heterogeneous agentic abilities across multiple domains including Long‑horizon Search, Engineering, Scientific Research, Instruction Following, and Tool-calling. We investigate agent-horizon scaling from two perspectives: scaling long-horizon trajectories and scaling heterogeneous agent abilities.

From the scaling of long-horizon trajectories, Agents‑A1 is trained with the assistance of a domain-grounded knowledge-action infrastructure that jointly constructs actions, observations, and verifier outcomes, turning the agent's process into a trainable target. From the scaling of heterogeneous agent abilities, Agents‑A1 presents a three-stage training paradigm for building scalable general-purpose agentic model. First, we perform full-domain supervised fine-tuning to align the base model with broad agentic behaviors. Second, we train domain-level teacher models to capture specialized expertise in each domain. Third, we propose multi-teacher multi-domain on-policy distillation with heterogeneity-aware optimization to improve knowledge transfer efficiency across different domains.

Highlights

Agentic Reasoning: Agents-A1 excels at decomposing complex tasks into executable sub-steps, planning ahead, and adapting its strategy based on intermediate results.
Tool Use: Natively supports function calling and tool integration, enabling seamless interaction with APIs, code interpreters, search engines, and other external tools.
Scientific and Professional Reasoning: Handles tool-integrated scientific reasoning and professional knowledge question answering.
Instruction Following: Precisely follows detailed, multi-constraint instructions across diverse domains.

We welcome developers and enterprises to integrate and try Agents-A1 and share their feedback.

Performance

We evaluate Agents-A1 in real-world agentic and research-oriented workflows across six directions — long-horizon search, engineering tasks, scientific research, instruction following, general agentic tasks, and scientific agentic tasks. Despite operating in the ~35B model class, Agents-A1 delivers highly competitive performance against frontier-scale systems such as GPT-5.5, DeepSeek-V4-pro, and Kimi-K2.6. It achieves overall SOTA results on several challenging benchmarks, including Seal-0 (56.4), HiPhO (46.4), FrontierScience-Olympiad (79.0), FrontierScience-Research (40.00), IFBench (80.6), and IFEval (94.8), while also ranking as the best among comparable models on a broad range of tasks such as BrowseComp (75.5), XBench-DS-2510 (86.0), GAIA (96.0), SciCode (44.3), HLE with tools (47.6), and MolBench-bind (56.8). These results show that Agents-A1 combines strong long-horizon search ability, robust scientific reasoning, and reliable instruction following, establishing it as a highly capable and efficient agentic model that narrows the gap with much larger frontier models.

🥇 Overall SOTA 🟢 Best Among Comparable Models (~35B)

Benchmark	📏 Comparable Models (~35B)			🚀 Larger-scale Models				⭐ Ours
Benchmark	Qwen3.5-35B-A3B	Qwen3.6-35B-A3B	Nex-N2-mini	Step-3.5-Flash	Kimi-K2.6	DeepSeek-V4-pro(Max)	GPT-5.5(xhigh)	Agents-A1
🔍 Long-horizon Search
BrowseComp	61.0	67.93	74.1	69.0	83.2	83.4	🥇 84.4	🟢 75.51
XBench-DS-2510	77.0	71.0	82.0	56.3	🥇 90.0	🥇 90.0	84.0	🟢 86.0
Seal0	41.4	38.74	49.55	36.94	50.45	54.95	42.34	🥇 56.36
GAIA	59.8	78.64	82.52	84.5	80.58	🥇 98.06	87.38	🟢 96.04
⚙️ Engineering Tasks
SciCode	37.7	35.8	29.9	40.4	53.5	50.0	🥇 56.1	🟢 44.33
MLE-Lite	24.24	34.85	34.85	54.55	62.12	63.64	🥇 72.73	🟢 43.94
🧪 Scientific Research
HLE w/ tools	47.4	36.2	32.0	23.1	🥇 54.0	48.2	52.2	🟢 47.6
HiPhO	37.0	37.7	38.5	38.3	41.1	38.7	43.3	🥇 46.4
FrontierScience-Olympiad	64.5	60.3	52.0	61.0	73.0	76.0	78.0	🥇 79.0
FrontierScience-Research	2.5	2.9	5.0	6.7	17.9	13.3	26.7	🥇 40.0
📋 Instruction Following
IFBench	70.2	64.4	54.08	64.6	71.77	73.47	75.9	🥇 80.61
LongBench-v2	59.0	57.7	59.6	57.5	62.0	🥇 64.3	-	🟢 60.2
IFEval	91.9	91.3	88.4	93.53	94.45	93.35	93.35	🥇 94.82
🤖 General Agentic Tasks
τ²-Bench	🟢 81.2	79.0	74.53	75.77	81.93	🥇 82.2	81.63	79.81
VitaBench	31.9	35.6	23.0	30.0	35.63	🥇 49.04	45.0	🟢 38.75
🔬 Scientific Agentic Tasks
MatTools	21.0	15.9	34.1	44.93	63.8	47.1	🥇 68.8	🟢 47.1
MolBench-bind	46.0	48.7	51.4	45.95	21.6	37.8	🥇 62.2	🟢 56.8

Usage

SGLang

SGLang is a fast serving framework for large language models and vision language models.

Install SGLang with uv:

uv venv --python 3.12 --seed --managed-python
source .venv/bin/activate

uv pip install sglang

See its documentation for more details.

The following commands create API endpoints at http://localhost:8000/v1:

Standard Version (1 GPUs, 262K context):

python -m sglang.launch_server \
  --model-path InternScience/Agents-A1 \
  --port 8000 \
  --tp-size 1 \
  --mem-fraction-static 0.8 \
  --context-length 262144 \
  --reasoning-parser qwen3

Tool Use:

python -m sglang.launch_server \
  --model-path InternScience/Agents-A1 \
  --port 8000 \
  --tp-size 1 \
  --mem-fraction-static 0.8 \
  --context-length 262144 \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder

vLLM

vLLM is a high-throughput and memory-efficient inference and serving engine for LLMs.

Install vLLM from the main branch via uv:

uv venv --python 3.12 --seed --managed-python
source .venv/bin/activate

uv pip install vllm --torch-backend=auto

See its documentation for more details.

The following commands create API endpoints at http://localhost:8000/v1:

Standard Version (1 GPUs, 262K context):

vllm serve InternScience/Agents-A1 \
  --port 8000 \
  --tensor-parallel-size 1 \
  --max-model-len 262144 \
  --reasoning-parser qwen3

Tool Call:

vllm serve InternScience/Agents-A1 \
  --port 8000 \
  --tensor-parallel-size 1 \
  --max-model-len 262144 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder

Text-Only (skips vision encoder to free KV cache memory):

vllm serve InternScience/Agents-A1 \
  --port 8000 \
  --tensor-parallel-size 1 \
  --max-model-len 262144 \
  --reasoning-parser qwen3 \
  --language-model-only

Recommended Sampling Parameters

For the best generation quality, we recommend the following sampling parameters:

temperature: 0.85
top_p: 0.95
top_k: 20
min_p: 0.0
presence_penalty: 1.1
repetition_penalty: 1.0

Agent Capability Evaluation

To provide the community with a unified agent evaluation codebase for fair comparison, we have also open-sourced an evaluation framework for assessing agentic models across core capabilities, including tool use and multi-step reasoning. The evaluation code is included in the Agents-A1/evaluation of this repository.

We use this framework to evaluate the released model under a standardized and reproducible setting. Specifically, the model is tested on a set of agent-oriented tasks that require it to understand user goals, decompose complex instructions, interact with tools or environments when necessary, and produce final results. The evaluation results reported in Model Card are generated using the open-source framework above, so that users can reproduce the experiments, compare other models under the same protocol, and further extend the benchmark for new agent scenarios. (Note that: To ensure a fair comparison, we report the benchmark results from their original technical reports. If a model does not report the corresponding benchmark results, we evaluate it using the same evaluation protocol as our model.)

For detailed evaluation scripts, task definitions, metrics, and reproduction instructions, please refer to the evaluation codebase.

Citation

If you find our work helpful, feel free to give us a cite.

Downloads last month: -

GGUF

Model size

35B params

Architecture

qwen35moe

Hardware compatibility

4-bit

Collection including InternScience/Agents-A1-Q4_K_M-GGUF

Agents-A1

Collection

7 items • Updated about 13 hours ago • 5

Paper for InternScience/Agents-A1-Q4_K_M-GGUF

Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent

Paper • 2606.30616 • Published 3 days ago • 73