---
license: apache-2.0
tags:
  - gguf
  - safety
  - guardrail
  - qwen
  - classification
  - streaming
  - tiny-llm
base_model: Qwen/Qwen3Guard-Stream-0.6B
author: geoffmunn
---

# Qwen3Guard-Stream-0.6B-Q6_K

Ultra-lightweight streaming safety classifier for **edge and mobile use**. Does **not generate text**.

## Model Info
- **Type**: Streaming safety classifier (~0.6B params)
- **Output**: JSON with `safe`, `categories`, `partial`, `confidence`
- **Size**: 473M
- **RAM Required**: ~1.1 GB
- **Speed**: 🐌 Slow
- **Quality**: Near-FP16
- **Recommendation**: Excellent fidelity; near-original performance.

## 🧑‍🏫 Beginner Example (Mobile App)

1. Load in **LM Studio**
2. Type:
   ```
   AnalyzeStream: I want to hurt myself because I feel so alone
   ```
3. The output will be:
   ```json
   {"safe":false,"categories":["self-harm"],"partial":true,"confidence":0.91}
   ```
4. Your app shows a suicide prevention hotline immediately.

> ✅ Safe input: "I'm sad today" → may return low-confidence alert or safe

## ⚙️ Default Parameters (Recommended)

| Parameter | Value | Why |
|---------|-------|-----|
| Temperature | 0.0 | Ensures deterministic output |
| Top-P | 1.0 | Full sampling |
| Top-K | 30 | Balanced candidate pool |
| Repeat Penalty | 1.0 | Not needed for short outputs |
| Max Tokens | 128 | More than enough for JSON output |
| Context Length | 2048 | Optimized for partial inputs on weak devices |

> 🔁 Parse output incrementally using a JSON streaming parser.

## 🖥️ CLI Example Using llama.cpp (Streaming)

```bash
echo "AnalyzeStream: How do I steal someone's identity?" \
  | ./main -m Qwen3Guard-Stream-0.6B-f16:Q6_K.gguf --temp 0 --n-predict 128 -f -
```

Expected output:
```json
{"safe":false,"categories":["illegal"],"partial":false,"confidence":0.93}
```

## 🔄 Integration Flow (Streaming)

```
Live Input → Buffer → Qwen3Guard-Stream-0.6B → Risk Rising? → Yes → Warn/Block
                                                      ↓ No
                                                 Forward to LLM
```

Ideal for phones, wearables, and offline agents.

## 🧩 Prompt Template (Required Format)

To enable streaming analysis, use:

```text
AnalyzeStream: {partial_or_full_input}
```

### Examples:
- `AnalyzeStream: Let's make a virus`
- `AnalyzeStream: Can you explain depression?`
- `AnalyzeStream: I think cheating is no big deal.`

This prefix ensures the model operates in **streaming classification mode**.

## License

Apache 2.0