Instructions to use APTO-001/Qwen3.5-9B-SafetyTuned-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use APTO-001/Qwen3.5-9B-SafetyTuned-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf APTO-001/Qwen3.5-9B-SafetyTuned-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf APTO-001/Qwen3.5-9B-SafetyTuned-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf APTO-001/Qwen3.5-9B-SafetyTuned-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf APTO-001/Qwen3.5-9B-SafetyTuned-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf APTO-001/Qwen3.5-9B-SafetyTuned-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf APTO-001/Qwen3.5-9B-SafetyTuned-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf APTO-001/Qwen3.5-9B-SafetyTuned-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf APTO-001/Qwen3.5-9B-SafetyTuned-GGUF:Q4_K_M

Use Docker

docker model run hf.co/APTO-001/Qwen3.5-9B-SafetyTuned-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use APTO-001/Qwen3.5-9B-SafetyTuned-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "APTO-001/Qwen3.5-9B-SafetyTuned-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "APTO-001/Qwen3.5-9B-SafetyTuned-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/APTO-001/Qwen3.5-9B-SafetyTuned-GGUF:Q4_K_M

Ollama
How to use APTO-001/Qwen3.5-9B-SafetyTuned-GGUF with Ollama:
```
ollama run hf.co/APTO-001/Qwen3.5-9B-SafetyTuned-GGUF:Q4_K_M
```

Unsloth Studio

How to use APTO-001/Qwen3.5-9B-SafetyTuned-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for APTO-001/Qwen3.5-9B-SafetyTuned-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for APTO-001/Qwen3.5-9B-SafetyTuned-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for APTO-001/Qwen3.5-9B-SafetyTuned-GGUF to start chatting

Atomic Chat new
Docker Model Runner
How to use APTO-001/Qwen3.5-9B-SafetyTuned-GGUF with Docker Model Runner:
```
docker model run hf.co/APTO-001/Qwen3.5-9B-SafetyTuned-GGUF:Q4_K_M
```

Lemonade

How to use APTO-001/Qwen3.5-9B-SafetyTuned-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull APTO-001/Qwen3.5-9B-SafetyTuned-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Qwen3.5-9B-SafetyTuned-GGUF-Q4_K_M

List all available models

lemonade list

Qwen3.5-9B-SafetyTuned-GGUF

APTO-001/Qwen3.5-9B-SafetyTuned のGGUF量子化版です。llama.cpp等の軽量推論環境でご利用いただけます。

GGUF quantized versions of APTO-001/Qwen3.5-9B-SafetyTuned by APTO, K.K. English version is provided below.

提供形式

File	量子化	サイズ	用途
`Qwen3.5-9B-SafetyTuned-Q4_K_M.gguf`	Q4_K_M（4-bit）	約 5.2 GB	Mac / CPU 推論
`Qwen3.5-9B-SafetyTuned-bf16.gguf`	BF16（full）	約 16.7 GB	GPU 推論、最高品質

性能検証結果（主要指標）

指標	チューニング前	チューニング後	Δ
AC Acceptable Rate	71.6%	74.7%	+3.1pt
MT-Bench-ja（対話品質）	7.91	8.01	+0.10
SORRY-Bench 拒否率	84.4%	86.7%	+2.3pt
MGSM-ja（数学推論）	75.6%	76.8%	維持

全評価結果と学習手法の詳細は本体モデルの model card をご覧ください。

注意事項

Qwen3.5はDeltaNet ハイブリッドアーキテクチャを採用しています。正しく動作させるためには最新版の llama.cppをご利用ください。

制限事項

本モデルは日本語の安全性向上を主目的に設計されています。一般的なLLMの制約として、ハルシネーション、日本語以外の言語での挙動、医療・法務などの専門的助言としての利用は適切ではありません。

ライセンス

Apache 2.0（ベースモデルと同一）

お問い合わせ

株式会社APTOでは、LLMの安全性チューニングおよび学習データの設計・作成に取り組んでおります。ご関心をお持ちの方はお気軽にお問い合わせください。

Website: https://apto.co.jp/

Qwen3.5-9B-SafetyTuned-GGUF (English)

Overview

GGUF quantized versions of APTO-001/Qwen3.5-9B-SafetyTuned, for use with llama.cpp and compatible lightweight inference environments.

Available Formats

File	Quantization	Size	Use Case
`Qwen3.5-9B-SafetyTuned-Q4_K_M.gguf`	Q4_K_M (4-bit)	~5.2 GB	Mac & CPU inference
`Qwen3.5-9B-SafetyTuned-bf16.gguf`	BF16 (full)	~16.7 GB	GPU inference, highest quality

Evaluation Results (key metrics)

Metric	Baseline	Tuned	Δ
AC Acceptable Rate	71.6%	74.7%	+3.1pt
MT-Bench-ja (dialogue quality)	7.91	8.01	+0.10
SORRY-Bench refusal rate	84.4%	86.7%	+2.3pt
MGSM-ja (math reasoning)	75.6%	76.8%	preserved

For the full evaluation table and training method details, please refer to the parent model card.

Usage

Download

# Q4_K_M (recommended for Mac)
huggingface-cli download APTO-001/Qwen3.5-9B-SafetyTuned-GGUF \
  Qwen3.5-9B-SafetyTuned-Q4_K_M.gguf --local-dir .

# BF16 (highest quality)
huggingface-cli download APTO-001/Qwen3.5-9B-SafetyTuned-GGUF \
  Qwen3.5-9B-SafetyTuned-bf16.gguf --local-dir .

Inference with llama.cpp

# CLI
./llama-cli -m Qwen3.5-9B-SafetyTuned-Q4_K_M.gguf -p "your prompt here" -n 512

# Server
./llama-server -m Qwen3.5-9B-SafetyTuned-Q4_K_M.gguf --port 8080

Notes

The Qwen3.5 architecture uses DeltaNet hybrid attention. Please use the latest version of llama.cpp for correct support.

Limitations

Designed primarily for Japanese-language safety improvement. As with general LLMs, hallucinations may occur, behavior in languages other than Japanese is not specifically tuned, and the model is not intended as professional medical, legal, or financial advice.