Instructions to use TheStageAI/Qwen3.5-2B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use TheStageAI/Qwen3.5-2B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf TheStageAI/Qwen3.5-2B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf TheStageAI/Qwen3.5-2B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf TheStageAI/Qwen3.5-2B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf TheStageAI/Qwen3.5-2B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf TheStageAI/Qwen3.5-2B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf TheStageAI/Qwen3.5-2B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf TheStageAI/Qwen3.5-2B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf TheStageAI/Qwen3.5-2B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/TheStageAI/Qwen3.5-2B-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use TheStageAI/Qwen3.5-2B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TheStageAI/Qwen3.5-2B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TheStageAI/Qwen3.5-2B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/TheStageAI/Qwen3.5-2B-GGUF:Q4_K_M

Ollama
How to use TheStageAI/Qwen3.5-2B-GGUF with Ollama:
```
ollama run hf.co/TheStageAI/Qwen3.5-2B-GGUF:Q4_K_M
```

Unsloth Studio

How to use TheStageAI/Qwen3.5-2B-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for TheStageAI/Qwen3.5-2B-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for TheStageAI/Qwen3.5-2B-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for TheStageAI/Qwen3.5-2B-GGUF to start chatting

How to use TheStageAI/Qwen3.5-2B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf TheStageAI/Qwen3.5-2B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "TheStageAI/Qwen3.5-2B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

OpenClaw new

How to use TheStageAI/Qwen3.5-2B-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf TheStageAI/Qwen3.5-2B-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "TheStageAI/Qwen3.5-2B-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use TheStageAI/Qwen3.5-2B-GGUF with Docker Model Runner:
```
docker model run hf.co/TheStageAI/Qwen3.5-2B-GGUF:Q4_K_M
```

Lemonade

How to use TheStageAI/Qwen3.5-2B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull TheStageAI/Qwen3.5-2B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Qwen3.5-2B-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use TheStageAI/Qwen3.5-2B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf TheStageAI/Qwen3.5-2B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default TheStageAI/Qwen3.5-2B-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

Qwen3.5-2B-GGUF

File size: 7,998 Bytes

19bf190
 
 
 
 
 
 
9eea191
 
19bf190
 
9eea191

<svg width="222" height="64" viewBox="0 0 222 64" fill="none" xmlns="http://www.w3.org/2000/svg">
<foreignObject x="0" y="0" width="0" height="0"><div xmlns="http://www.w3.org/1999/xhtml" style="backdrop-filter:blur(26px);clip-path:url(#bgblur_0_5480_3558_clip_path);height:100%;width:100%"></div></foreignObject><g data-figma-bg-blur-radius="52">
<rect width="222" height="64" rx="32" fill="#FFF844"/>
<rect x="8" y="8" width="48" height="48" rx="24" fill="black"/>
<path d="M32 35.0002L29 32.0002M32 35.0002C33.3968 34.469 34.7369 33.799 36 33.0002M32 35.0002V40.0002C32 40.0002 35.03 39.4502 36 38.0002C37.08 36.3802 36 33.0002 36 33.0002M29 32.0002C29.5321 30.6197 30.2022 29.2963 31 28.0502C32.1652 26.1872 33.7876 24.6533 35.713 23.5943C37.6384 22.5354 39.8027 21.9866 42 22.0002C42 24.7202 41.22 29.5002 36 33.0002M29 32.0002H24C24 32.0002 24.55 28.9702 26 28.0002C27.62 26.9202 31 28.0002 31 28.0002M24.5 36.5002C23 37.7602 22.5 41.5002 22.5 41.5002C22.5 41.5002 26.24 41.0002 27.5 39.5002C28.21 38.6602 28.2 37.3702 27.41 36.5902C27.0213 36.2193 26.5093 36.0049 25.9722 35.9883C25.4352 35.9717 24.9109 36.154 24.5 36.5002Z" stroke="#FFF844" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"/>
<path d="M76.2028 39.7642C74.913 39.7642 73.8022 39.4886 72.8704 38.9375C71.9442 38.3807 71.2283 37.5994 70.7227 36.5938C70.2227 35.5824 69.9727 34.3977 69.9727 33.0398C69.9727 31.6989 70.2227 30.517 70.7227 29.4943C71.2283 28.4716 71.9329 27.6733 72.8363 27.0994C73.7454 26.5256 74.8079 26.2386 76.0238 26.2386C76.7624 26.2386 77.4783 26.3608 78.1715 26.6051C78.8647 26.8494 79.4869 27.233 80.038 27.7557C80.5891 28.2784 81.0238 28.9574 81.342 29.7926C81.6602 30.6222 81.8192 31.6307 81.8192 32.8182V33.7216H71.413V31.8125H79.3221C79.3221 31.142 79.1857 30.5483 78.913 30.0312C78.6403 29.5085 78.2567 29.0966 77.7624 28.7955C77.2738 28.4943 76.6999 28.3438 76.0408 28.3438C75.3249 28.3438 74.6999 28.5199 74.1658 28.8722C73.6374 29.2187 73.2283 29.6733 72.9386 30.2358C72.6545 30.7926 72.5124 31.3977 72.5124 32.0511V33.5426C72.5124 34.4176 72.6658 35.1619 72.9727 35.7756C73.2852 36.3892 73.7198 36.858 74.2766 37.1818C74.8335 37.5 75.484 37.6591 76.2283 37.6591C76.7113 37.6591 77.1516 37.5909 77.5494 37.4545C77.9471 37.3125 78.2908 37.1023 78.5806 36.8239C78.8704 36.5455 79.092 36.2017 79.2454 35.7926L81.6573 36.2273C81.4641 36.9375 81.1175 37.5597 80.6175 38.0938C80.1232 38.6222 79.5011 39.0341 78.7511 39.3295C78.0067 39.6193 77.1573 39.7642 76.2028 39.7642ZM89.5472 39.7557C88.4904 39.7557 87.5472 39.4858 86.7177 38.946C85.8938 38.4006 85.2461 37.625 84.7745 36.6193C84.3086 35.608 84.0756 34.3949 84.0756 32.9801C84.0756 31.5653 84.3114 30.3551 84.783 29.3494C85.2603 28.3437 85.9137 27.5739 86.7433 27.0398C87.5728 26.5057 88.5131 26.2386 89.5643 26.2386C90.3768 26.2386 91.0302 26.375 91.5245 26.6477C92.0245 26.9148 92.4109 27.2273 92.6836 27.5852C92.962 27.9432 93.1779 28.2585 93.3313 28.5312H93.4847V22.0455H96.033V39.5H93.5444V37.4631H93.3313C93.1779 37.7415 92.9563 38.0597 92.6665 38.4176C92.3825 38.7756 91.9904 39.0881 91.4904 39.3551C90.9904 39.6222 90.3427 39.7557 89.5472 39.7557ZM90.1097 37.5824C90.8427 37.5824 91.462 37.3892 91.9677 37.0028C92.479 36.6108 92.8654 36.0682 93.1268 35.375C93.3938 34.6818 93.5273 33.875 93.5273 32.9545C93.5273 32.0455 93.3967 31.25 93.1353 30.5682C92.8739 29.8864 92.4904 29.3551 91.9847 28.9744C91.479 28.5937 90.854 28.4034 90.1097 28.4034C89.3427 28.4034 88.7035 28.6023 88.1921 29C87.6808 29.3977 87.2944 29.9403 87.033 30.6278C86.7773 31.3153 86.6495 32.0909 86.6495 32.9545C86.6495 33.8295 86.7802 34.6165 87.0415 35.3153C87.3029 36.0142 87.6893 36.5682 88.2006 36.9773C88.7177 37.3807 89.354 37.5824 90.1097 37.5824ZM105.159 44.6818C104.119 44.6818 103.224 44.5455 102.474 44.2727C101.73 44 101.122 43.6392 100.65 43.1903C100.179 42.7415 99.8263 42.25 99.5934 41.7159L101.784 40.8125C101.937 41.0625 102.142 41.3267 102.397 41.6051C102.659 41.8892 103.011 42.1307 103.454 42.3295C103.903 42.5284 104.48 42.6278 105.184 42.6278C106.15 42.6278 106.949 42.392 107.579 41.9205C108.21 41.4545 108.525 40.7102 108.525 39.6875V37.1136H108.363C108.21 37.392 107.988 37.7017 107.699 38.0426C107.414 38.3835 107.022 38.679 106.522 38.929C106.022 39.179 105.372 39.304 104.571 39.304C103.537 39.304 102.605 39.0625 101.775 38.5795C100.951 38.0909 100.298 37.3722 99.815 36.4233C99.3377 35.4687 99.0991 34.2955 99.0991 32.9034C99.0991 31.5114 99.3349 30.3182 99.8065 29.3239C100.284 28.3295 100.937 27.5682 101.767 27.0398C102.596 26.5057 103.537 26.2386 104.588 26.2386C105.4 26.2386 106.056 26.375 106.556 26.6477C107.056 26.9148 107.446 27.2273 107.724 27.5852C108.008 27.9432 108.227 28.2585 108.38 28.5312H108.568V26.4091H111.065V39.7898C111.065 40.9148 110.804 41.8381 110.281 42.5597C109.758 43.2813 109.051 43.8153 108.159 44.1619C107.272 44.5085 106.272 44.6818 105.159 44.6818ZM105.133 37.1903C105.866 37.1903 106.485 37.0199 106.991 36.679C107.502 36.3324 107.889 35.8381 108.15 35.196C108.417 34.5483 108.551 33.7727 108.551 32.8693C108.551 31.9886 108.42 31.2131 108.159 30.5426C107.897 29.8722 107.514 29.3494 107.008 28.9744C106.502 28.5937 105.877 28.4034 105.133 28.4034C104.366 28.4034 103.727 28.6023 103.216 29C102.704 29.392 102.318 29.9261 102.056 30.6023C101.801 31.2784 101.673 32.0341 101.673 32.8693C101.673 33.7273 101.804 34.4801 102.065 35.1278C102.326 35.7756 102.713 36.2812 103.224 36.6449C103.741 37.0085 104.377 37.1903 105.133 37.1903ZM120.125 39.7642C118.835 39.7642 117.724 39.4886 116.792 38.9375C115.866 38.3807 115.15 37.5994 114.645 36.5938C114.145 35.5824 113.895 34.3977 113.895 33.0398C113.895 31.6989 114.145 30.517 114.645 29.4943C115.15 28.4716 115.855 27.6733 116.758 27.0994C117.667 26.5256 118.73 26.2386 119.946 26.2386C120.684 26.2386 121.4 26.3608 122.093 26.6051C122.787 26.8494 123.409 27.233 123.96 27.7557C124.511 28.2784 124.946 28.9574 125.264 29.7926C125.582 30.6222 125.741 31.6307 125.741 32.8182V33.7216H115.335V31.8125H123.244C123.244 31.142 123.108 30.5483 122.835 30.0312C122.562 29.5085 122.179 29.0966 121.684 28.7955C121.196 28.4943 120.622 28.3438 119.963 28.3438C119.247 28.3438 118.622 28.5199 118.088 28.8722C117.559 29.2187 117.15 29.6733 116.86 30.2358C116.576 30.7926 116.434 31.3977 116.434 32.0511V33.5426C116.434 34.4176 116.588 35.1619 116.895 35.7756C117.207 36.3892 117.642 36.858 118.199 37.1818C118.755 37.5 119.406 37.6591 120.15 37.6591C120.633 37.6591 121.074 37.5909 121.471 37.4545C121.869 37.3125 122.213 37.1023 122.502 36.8239C122.792 36.5455 123.014 36.2017 123.167 35.7926L125.579 36.2273C125.386 36.9375 125.039 37.5597 124.539 38.0938C124.045 38.6222 123.423 39.0341 122.673 39.3295C121.929 39.6193 121.079 39.7642 120.125 39.7642ZM136.282 31.0199V33.2273H128.543V31.0199H136.282ZM142.226 22.0455V39.5H139.678V22.0455H142.226ZM145.654 39.5V26.4091H148.1V28.5398H148.262C148.535 27.8182 148.981 27.2557 149.6 26.8523C150.22 26.4432 150.961 26.2386 151.825 26.2386C152.7 26.2386 153.433 26.4432 154.024 26.8523C154.62 27.2614 155.061 27.8239 155.345 28.5398H155.481C155.794 27.8409 156.291 27.2841 156.973 26.8693C157.654 26.4489 158.467 26.2386 159.41 26.2386C160.598 26.2386 161.566 26.6108 162.316 27.3551C163.072 28.0994 163.45 29.2216 163.45 30.7216V39.5H160.902V30.9602C160.902 30.0739 160.66 29.4318 160.177 29.0341C159.694 28.6364 159.118 28.4375 158.447 28.4375C157.618 28.4375 156.973 28.6932 156.512 29.2045C156.052 29.7102 155.822 30.3608 155.822 31.1562V39.5H153.282V30.7983C153.282 30.0881 153.061 29.517 152.618 29.0852C152.174 28.6534 151.598 28.4375 150.887 28.4375C150.404 28.4375 149.958 28.5653 149.549 28.821C149.146 29.071 148.819 29.4205 148.569 29.8693C148.325 30.3182 148.203 30.8381 148.203 31.429V39.5H145.654Z" fill="#0C0C0C"/>
<path d="M179 31H193M187 37L193 31L187 25" stroke="#0C0C0C" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"/>
</g>
<defs>
<clipPath id="bgblur_0_5480_3558_clip_path" transform="translate(0 0)"><rect width="222" height="64" rx="32"/>
</clipPath></defs>
</svg>