Instructions to use inclusionAI/Sing-Guard-4b-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use inclusionAI/Sing-Guard-4b-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="inclusionAI/Sing-Guard-4b-GGUF",
	filename="Sing-Guard-4b-F16.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use inclusionAI/Sing-Guard-4b-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf inclusionAI/Sing-Guard-4b-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf inclusionAI/Sing-Guard-4b-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf inclusionAI/Sing-Guard-4b-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf inclusionAI/Sing-Guard-4b-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf inclusionAI/Sing-Guard-4b-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf inclusionAI/Sing-Guard-4b-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf inclusionAI/Sing-Guard-4b-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf inclusionAI/Sing-Guard-4b-GGUF:Q4_K_M

Use Docker

docker model run hf.co/inclusionAI/Sing-Guard-4b-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use inclusionAI/Sing-Guard-4b-GGUF with Ollama:
```
ollama run hf.co/inclusionAI/Sing-Guard-4b-GGUF:Q4_K_M
```

Unsloth Studio

How to use inclusionAI/Sing-Guard-4b-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for inclusionAI/Sing-Guard-4b-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for inclusionAI/Sing-Guard-4b-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for inclusionAI/Sing-Guard-4b-GGUF to start chatting

How to use inclusionAI/Sing-Guard-4b-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf inclusionAI/Sing-Guard-4b-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "inclusionAI/Sing-Guard-4b-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use inclusionAI/Sing-Guard-4b-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf inclusionAI/Sing-Guard-4b-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default inclusionAI/Sing-Guard-4b-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use inclusionAI/Sing-Guard-4b-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf inclusionAI/Sing-Guard-4b-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "inclusionAI/Sing-Guard-4b-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use inclusionAI/Sing-Guard-4b-GGUF with Docker Model Runner:
```
docker model run hf.co/inclusionAI/Sing-Guard-4b-GGUF:Q4_K_M
```

Lemonade

How to use inclusionAI/Sing-Guard-4b-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull inclusionAI/Sing-Guard-4b-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Sing-Guard-4b-GGUF-Q4_K_M

List all available models

lemonade list

Sing-Guard-4b-GGUF

File size: 1,892 Bytes

157cbe5

<svg width="1024" height="1024" viewBox="0 0 1024 1024" fill="none" xmlns="http://www.w3.org/2000/svg">
  <path d="M512 94C586 147 669 184 765 201C781 204 793 217 794 234C806 471 741 660 596 806C571 831 543 855 512 878C481 855 453 831 428 806C283 660 218 471 230 234C231 217 243 204 259 201C355 184 438 147 512 94Z" fill="#07847F"/>
  <path d="M512 126C577 173 654 207 744 224C752 226 758 232 759 240C768 456 710 629 577 764C557 784 535 804 512 822C489 804 467 784 447 764C314 629 256 456 265 240C266 232 272 226 280 224C370 207 447 173 512 126Z" fill="url(#shieldGradient)"/>
  <path d="M512 270C455 270 408 285 379 311C350 337 335 370 335 408C335 446 348 476 374 498C400 520 437 537 484 549L540 563C570 571 591 580 604 591C617 602 623 615 623 632C623 651 614 666 597 677C580 688 555 693 524 693C492 693 461 687 432 676C403 665 378 649 357 629L308 695C334 720 365 739 402 752C439 765 478 772 519 772C557 772 591 766 622 754C653 742 677 724 695 701C713 678 722 651 722 619C722 581 709 551 683 528C657 505 619 487 569 474L513 460C481 452 458 442 444 431C430 420 423 406 423 389C423 372 432 358 449 348C466 338 489 333 519 333C548 333 576 338 603 348C630 358 654 372 676 391L725 325C700 307 671 293 638 284C605 275 563 270 512 270Z" fill="white"/>
  <path d="M108 275L123 329L177 344L123 359L108 413L93 359L39 344L93 329L108 275Z" fill="#07847F"/>
  <path d="M184 600L200 657L257 673L200 689L184 746L168 689L111 673L168 657L184 600Z" fill="#07847F"/>
  <path d="M925 275L940 329L994 344L940 359L925 413L910 359L856 344L910 329L925 275Z" fill="#07847F"/>
  <path d="M842 638L855 686L903 699L855 712L842 760L829 712L781 699L829 686L842 638Z" fill="#07847F"/>
  <defs>
    <linearGradient id="shieldGradient" x1="317" y1="176" x2="707" y2="789" gradientUnits="userSpaceOnUse">
      <stop stop-color="#16C7BD"/>
      <stop offset="1" stop-color="#07847F"/>
    </linearGradient>
  </defs>
</svg>