Text Classification
Transformers
Safetensors
English
deberta-v2
prompt-injection
prompt-injection-detection
llm-security
llm-safety
ai-safety
deberta
Eval Results (legacy)
text-embeddings-inference
Instructions to use JHC04567/spid-deberta-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use JHC04567/spid-deberta-base with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="JHC04567/spid-deberta-base")# Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("JHC04567/spid-deberta-base") model = AutoModelForSequenceClassification.from_pretrained("JHC04567/spid-deberta-base", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -105,14 +105,18 @@ print("BLOCKED" if unsafe_prob >= 0.85 else "PASSED")
|
|
| 105 |
|
| 106 |
## Evaluation
|
| 107 |
|
| 108 |
-
|
|
|
|
|
|
|
| 109 |
|
| 110 |
| Mode | Precision | Recall | F1 |
|
| 111 |
|:-----|----------:|-------:|---:|
|
| 112 |
-
| Classifier
|
| 113 |
-
| Pipeline (
|
|
|
|
|
|
|
| 114 |
|
| 115 |
-
|
| 116 |
|
| 117 |
## Training Details
|
| 118 |
|
|
@@ -139,7 +143,8 @@ The classifier mode favors precision to avoid blocking legitimate requests. The
|
|
| 139 |
- Vulnerable to paraphrased attacks ("show me" vs "reveal") and obfuscation (base64, leetspeak)
|
| 140 |
- Not designed for multi-turn or advanced jailbreak techniques
|
| 141 |
- Intended as a cost-saving pre-filter, not a standalone security layer
|
| 142 |
-
|
|
|
|
| 143 |
## Citation
|
| 144 |
|
| 145 |
```bibtex
|
|
|
|
| 105 |
|
| 106 |
## Evaluation
|
| 107 |
|
| 108 |
+
Attacks: `benign request + conjunction + hidden injection` (real deepset/Gandalf payloads).
|
| 109 |
+
|
| 110 |
+
Split pipeline vs. same classifier at matched recall (0.94).
|
| 111 |
|
| 112 |
| Mode | Precision | Recall | F1 |
|
| 113 |
|:-----|----------:|-------:|---:|
|
| 114 |
+
| Classifier @ matched recall | 0.85 | 0.94 | — |
|
| 115 |
+
| **Pipeline (split)** | **0.98** | 0.94 | **0.96** |
|
| 116 |
+
|
| 117 |
+
Splitting wins: +0.14 precision at matched recall (PR-AUC 0.97), rescuing +84 of 300 attacks with 0 added false positives.
|
| 118 |
|
| 119 |
+
*Caveats:* near-best-case (split on SPID's own conjunctions); payloads overlap training data; small benign control (n=150).
|
| 120 |
|
| 121 |
## Training Details
|
| 122 |
|
|
|
|
| 143 |
- Vulnerable to paraphrased attacks ("show me" vs "reveal") and obfuscation (base64, leetspeak)
|
| 144 |
- Not designed for multi-turn or advanced jailbreak techniques
|
| 145 |
- Intended as a cost-saving pre-filter, not a standalone security layer
|
| 146 |
+
- Splitting helps only for conjunction-separated composite injections, measured under near-best-case, partly in-distribution conditions.
|
| 147 |
+
|
| 148 |
## Citation
|
| 149 |
|
| 150 |
```bibtex
|