Text Classification
Transformers
Safetensors
English
deberta-v2
safety
biosecurity
content-classification
constitutional-classifiers
deberta-v3
dual-use
nsabb
biology
Eval Results (legacy)
text-embeddings-inference
Instructions to use jang1563/constitutional-bioguard-deberta-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use jang1563/constitutional-bioguard-deberta-v1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="jang1563/constitutional-bioguard-deberta-v1")# Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("jang1563/constitutional-bioguard-deberta-v1") model = AutoModelForSequenceClassification.from_pretrained("jang1563/constitutional-bioguard-deberta-v1", device_map="auto") - Notebooks
- Google Colab
- Kaggle
docs(readme): replace em dashes with cleaner punctuation
Browse files
README.md
CHANGED
|
@@ -165,7 +165,7 @@ text = normalize_text(f"{query} [SEP] {response}")
|
|
| 165 |
|
| 166 |
- **Source**: Synthetic examples generated by Claude API from a 56-rule biosafety constitution
|
| 167 |
- **Constitution**: Covers all 7 NSABB dual-use research categories with explicit permitted/restricted/boundary rules
|
| 168 |
-
- **Size**: 4,267 total
|
| 169 |
- **Class balance**: ~68% UNSAFE, ~32% SAFE (class weights applied during training)
|
| 170 |
- **Splits**: Stratified by NSABB category and fine label
|
| 171 |
- **Augmentation**: Translation (5 languages), jailbreak templates, formality variation, prefill attacks
|
|
@@ -179,7 +179,7 @@ The dataset is not publicly released; the generation pipeline is open-source and
|
|
| 179 |
|
| 180 |
1. **Encoding bypass**: ROT13 achieves 47.9% ASR, URL-encode 29.2%. Use `preprocessing.normalize_text()` to mitigate.
|
| 181 |
2. **Synthetic-only training**: All examples are Claude-generated; real-world distribution shift is uncharacterized.
|
| 182 |
-
3. **External validation gap**: External kappa = 0.414 vs. target 0.80. The benchmark (BioThreat-Eval) uses response-based labeling; this classifier labels queries
|
| 183 |
4. **English-centric**: Evaluation is English-only despite multilingual augmentation in training.
|
| 184 |
5. **Single LLM training data**: All data from Claude; cross-LLM calibration is unknown.
|
| 185 |
6. **Not a complete defense**: Mean adversarial ASR = 9.79%; should be used as one layer in a broader safety system.
|
|
@@ -188,7 +188,7 @@ The dataset is not publicly released; the generation pipeline is open-source and
|
|
| 188 |
|
| 189 |
## Ethical Considerations
|
| 190 |
|
| 191 |
-
This model detects potentially dangerous biological content to support biosafety compliance in AI systems. The training data contains synthetic descriptions of potentially harmful topics
|
| 192 |
|
| 193 |
Do not use this model to identify exploitable gaps in biosafety systems for malicious purposes, or as the sole safety mechanism in contexts where a false negative could enable serious harm.
|
| 194 |
|
|
|
|
| 165 |
|
| 166 |
- **Source**: Synthetic examples generated by Claude API from a 56-rule biosafety constitution
|
| 167 |
- **Constitution**: Covers all 7 NSABB dual-use research categories with explicit permitted/restricted/boundary rules
|
| 168 |
+
- **Size**: 4,267 total: 2,968 train / 635 val / 664 test
|
| 169 |
- **Class balance**: ~68% UNSAFE, ~32% SAFE (class weights applied during training)
|
| 170 |
- **Splits**: Stratified by NSABB category and fine label
|
| 171 |
- **Augmentation**: Translation (5 languages), jailbreak templates, formality variation, prefill attacks
|
|
|
|
| 179 |
|
| 180 |
1. **Encoding bypass**: ROT13 achieves 47.9% ASR, URL-encode 29.2%. Use `preprocessing.normalize_text()` to mitigate.
|
| 181 |
2. **Synthetic-only training**: All examples are Claude-generated; real-world distribution shift is uncharacterized.
|
| 182 |
+
3. **External validation gap**: External kappa = 0.414 vs. target 0.80. The benchmark (BioThreat-Eval) uses response-based labeling; this classifier labels queries: an architectural mismatch that accounts for the gap. See [GitHub README](https://github.com/jang1563/constitutional-bioguard#limitations) for full explanation.
|
| 183 |
4. **English-centric**: Evaluation is English-only despite multilingual augmentation in training.
|
| 184 |
5. **Single LLM training data**: All data from Claude; cross-LLM calibration is unknown.
|
| 185 |
6. **Not a complete defense**: Mean adversarial ASR = 9.79%; should be used as one layer in a broader safety system.
|
|
|
|
| 188 |
|
| 189 |
## Ethical Considerations
|
| 190 |
|
| 191 |
+
This model detects potentially dangerous biological content to support biosafety compliance in AI systems. The training data contains synthetic descriptions of potentially harmful topics: necessary to teach the classifier what to flag, not to enable harm.
|
| 192 |
|
| 193 |
Do not use this model to identify exploitable gaps in biosafety systems for malicious purposes, or as the sole safety mechanism in contexts where a false negative could enable serious harm.
|
| 194 |
|