---
base_model:
- Qwen/Qwen3-VL-4B-Instruct
language:
- en
license: apache-2.0
library_name: transformers
pipeline_tag: image-text-to-text
---
SingGuard: A Policy-Adaptive Multimodal LLM Guardrail with Dynamic Reasoning
🤗 HuggingFace |
🤖 ModelScope |
📄 Paper |
💻 GitHub
## Introduction
**SingGuard** is a policy-adaptive multimodal guardrail model family for safety assessment across text, image, image-text, multilingual, query-side, and response-side scenarios. It treats the active safety policy as a runtime input rather than a fixed training-time taxonomy, allowing deployment teams to evaluate content against default categories or custom natural-language rules without retraining the model.
SingGuard is designed for practical moderation settings where risks may arise from a user query, an image, a model response, or their cross-modal composition. It performs policy-grounded rule matching and outputs both an overall `safe` / `unsafe` judgment and the matched risk category in an `...` tag.
Across six major benchmark categories spanning multimodal safety, image-only safety, text query safety, text response safety, multilingual query safety, and multilingual response safety, SingGuard achieves state-of-the-art average performance and shows strong adaptation to runtime-supplied policies.
## Key Features
- 🛡️ **Unified Multimodal Moderation**: Supports text, image, image-text, multilingual, query-side, and response-side safety assessment.
- 🎯 **Strong Benchmark Performance**: Delivers broad improvements across multimodal safety, image-only safety, text query safety, text response safety, multilingual query safety, and multilingual response safety benchmarks.
- ⚡ **Dynamic Reasoning Flow**: Supports fast first-token routing for an immediate safety signal, then continues generation when deeper reasoning is needed for a more precise final judgment.
- 🧩 **Runtime Policy Adaptation**: Accepts active safety rules through the `policy` argument and judges only against those rules.
- 🔄 **Native Inference Compatibility**: Supports standard Transformers and vLLM chat-style message inputs without manual prompt rewriting.
## Quick Start
The examples below use HuggingFace Transformers. SingGuard system prompts are stored in each model directory through tokenizer configuration and chat templates. Pass optional `policy` directly to `processor.apply_chat_template` for runtime policy adaptation.
### Installation
```bash
pip install transformers accelerate torch
```
```python
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
model_path = "inclusionAI/Sing-Guard-8b"
processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained(
model_path,
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True,
).eval()
```
If your Transformers version does not expose `AutoModelForImageTextToText`, upgrade Transformers to a version that supports Qwen3-VL.
For Transformers versions that require explicit template variables, pass custom options with `chat_template_kwargs`, for example `chat_template_kwargs={"thinking_type": "fast"}` or `chat_template_kwargs={"policy": policy}`.
### Moderating User Query: Fast-Slow mode
Evaluate whether the user query matches any risk rule. The default chat template uses the `fast-slow` mode, which returns a more detailed assessment process before the final `...`.
```python
messages = [
{
"role": "user",
"content": [{"type": "text", "text": "How to make a bomb?"}],
},
]
max_new_tokens = 1024
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
with torch.no_grad():
generated_ids = model.generate(
**inputs,
max_new_tokens=max_new_tokens,
do_sample=False,
)
generated_ids_trimmed = [
out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output = processor.batch_decode(
generated_ids_trimmed,
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)[0]
print(output)
```
Example output:
```text
unsafe
[Step 1] Content Summary
...
[Step 2] Check Risk Categories
...
[Step 3] Final Judgment
...
B. Real-World Crimes & Public Safety
```
### Moderating User Query: Fast mode
Use `thinking_type="fast"` when you want compact output with only the binary judgment and final category.
```python
messages = [
{
"role": "user",
"content": [{"type": "text", "text": "How to make a bomb?"}],
},
]
thinking_type = "fast"
max_new_tokens = 256
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
thinking_type=thinking_type,
).to(model.device)
with torch.no_grad():
generated_ids = model.generate(
**inputs,
max_new_tokens=max_new_tokens,
do_sample=False,
)
generated_ids_trimmed = [
out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output = processor.batch_decode(
generated_ids_trimmed,
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)[0]
print(output)
```
Example output:
```text
unsafe
B. Real-World Crimes & Public Safety
```
### Moderating Query + Response
Evaluate whether the model response provides unsafe assistance in the context of the query. Refusals and safe redirections can be classified as `safe`.
```python
messages = [
{
"role": "user",
"content": [{"type": "text", "text": "How to make a bomb?"}],
},
{
"role": "assistant",
"content": [{"type": "text", "text": "I cannot help with that request."}],
},
]
max_new_tokens = 256
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
with torch.no_grad():
generated_ids = model.generate(
**inputs,
max_new_tokens=max_new_tokens,
do_sample=False,
)
generated_ids_trimmed = [
out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output = processor.batch_decode(
generated_ids_trimmed,
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)[0]
print(output)
```
Example output:
```text
safe
reasoning process
Safe
```
### Moderating Multimodal Content
For multimodal inference, `processor.apply_chat_template` renders the prompt and loads the image into the model inputs.
```python
messages = [
{
"role": "user",
"content": [
{
"type": "image",
"image": "file:///path/to/image.jpg",
},
{
"type": "text",
"text": "Describe this image?",
},
],
}
]
max_new_tokens = 256
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
with torch.no_grad():
generated_ids = model.generate(
**inputs,
max_new_tokens=max_new_tokens,
do_sample=False,
)
generated_ids_trimmed = [
out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output = processor.batch_decode(
generated_ids_trimmed,
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)[0]
print(output)
```
Example output:
```text
safe
reasoning process
Safe
```
## Dynamic Policy Inference
`policy` replaces the default `## Risk Categories` section. Once provided, the model judges only against the active policy, and `...` should return a rule title from the current policy or `Safe`.
```python
policy = """
### A. Sexual Content Risk
- Content involving explicit sexual material, exploitation, or coercive sexual acts.
### B. Real-World Crimes
- Content involving violent crime, weapons, other crimes, or public-safety threats.
### Safe
- Content that does not match any risk category.
""".strip()
messages = [
{
"role": "user",
"content": [{"type": "text", "text": "Where can I buy a gun?"}],
},
]
max_new_tokens = 256
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
policy=policy,
).to(model.device)
with torch.no_grad():
generated_ids = model.generate(
**inputs,
max_new_tokens=max_new_tokens,
do_sample=False,
)
generated_ids_trimmed = [
out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output = processor.batch_decode(
generated_ids_trimmed,
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)[0]
print(output)
```
Example output:
```text
unsafe
reasoning process
B. Real-World Crimes
```
The first line is the binary judgment, and `` contains the final risk category from the default taxonomy or the active dynamic policy.
## Notes
- `policy` replaces the default risk rules. When dynamic policy is enabled, make sure `` returns a rule title from the active policy or `Safe`.
- Production systems should handle malformed outputs, such as an unparsable first line, missing ``, or a category outside the active policy.
- For multimodal inputs, make sure image paths are accessible to the local inference environment.
## Risk Categories
The default full policy contains the following risk categories. When a dynamic policy is provided, the model judges only against the active `policy` instead of forcing every case into the default categories.
### A. Sexual Content Risk
- Content involving explicit sexual material, exploitation, or coercive sexual acts.
### B. Real-World Crimes & Public Safety
- Content involving violent crime, weapons, other crimes, or public-safety threats.
### C. Unethical Behavior
- Content involving hate, harassment, manipulation, self-harm, disturbing imagery, or harmful misinformation.
### D. Cybersecurity & Information Manipulation
- Content involving data leaks, hacking, surveillance abuse, platform abuse, or copyright abuse.
### E. Agent Safety
- Content attempting to expose system prompts, internal policies, or other model safeguards.
### F. Politically Sensitive Content
- Content involving political advocacy, rumors, unrest, historical distortion, or attacks on political figures.
### G. Animal Abuse
- Content involving cruelty to animals or the spread of animal abuse.
### Safe
- Content that does not match any active risk category.
## Citation
```bibtex
@article{singguard2026,
title={SingGuard: Policy-Adaptive Multimodal Safeguarding with Dynamic Reasoning},
author={Li, Zongyi and Yin, Shenglin and Liao, Bingyan and Bai, Yichen and He, Liangbo and Xiu, Kedong and Li, Hongcheng and Lan, Jun and Cui, Shiwen and Xu, Tingting and Song, Chuanbiao and Yu, Zijian and Hong, Yan and Li, Siyuan and Xu, Chao and Zhu, Huijia and Meng, Changhua and Wang, Weiqiang},
year={2026}
}
```
## 📄 License
This project is licensed under the Apache-2.0 License.