jang1563 commited on
Commit
e375eb3
·
verified ·
1 Parent(s): b5dba97

docs(readme): replace em dashes with cleaner punctuation

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -165,7 +165,7 @@ text = normalize_text(f"{query} [SEP] {response}")
165
 
166
  - **Source**: Synthetic examples generated by Claude API from a 56-rule biosafety constitution
167
  - **Constitution**: Covers all 7 NSABB dual-use research categories with explicit permitted/restricted/boundary rules
168
- - **Size**: 4,267 total 2,968 train / 635 val / 664 test
169
  - **Class balance**: ~68% UNSAFE, ~32% SAFE (class weights applied during training)
170
  - **Splits**: Stratified by NSABB category and fine label
171
  - **Augmentation**: Translation (5 languages), jailbreak templates, formality variation, prefill attacks
@@ -179,7 +179,7 @@ The dataset is not publicly released; the generation pipeline is open-source and
179
 
180
  1. **Encoding bypass**: ROT13 achieves 47.9% ASR, URL-encode 29.2%. Use `preprocessing.normalize_text()` to mitigate.
181
  2. **Synthetic-only training**: All examples are Claude-generated; real-world distribution shift is uncharacterized.
182
- 3. **External validation gap**: External kappa = 0.414 vs. target 0.80. The benchmark (BioThreat-Eval) uses response-based labeling; this classifier labels queries an architectural mismatch that accounts for the gap. See [GitHub README](https://github.com/jang1563/constitutional-bioguard#limitations) for full explanation.
183
  4. **English-centric**: Evaluation is English-only despite multilingual augmentation in training.
184
  5. **Single LLM training data**: All data from Claude; cross-LLM calibration is unknown.
185
  6. **Not a complete defense**: Mean adversarial ASR = 9.79%; should be used as one layer in a broader safety system.
@@ -188,7 +188,7 @@ The dataset is not publicly released; the generation pipeline is open-source and
188
 
189
  ## Ethical Considerations
190
 
191
- This model detects potentially dangerous biological content to support biosafety compliance in AI systems. The training data contains synthetic descriptions of potentially harmful topics necessary to teach the classifier what to flag, not to enable harm.
192
 
193
  Do not use this model to identify exploitable gaps in biosafety systems for malicious purposes, or as the sole safety mechanism in contexts where a false negative could enable serious harm.
194
 
 
165
 
166
  - **Source**: Synthetic examples generated by Claude API from a 56-rule biosafety constitution
167
  - **Constitution**: Covers all 7 NSABB dual-use research categories with explicit permitted/restricted/boundary rules
168
+ - **Size**: 4,267 total: 2,968 train / 635 val / 664 test
169
  - **Class balance**: ~68% UNSAFE, ~32% SAFE (class weights applied during training)
170
  - **Splits**: Stratified by NSABB category and fine label
171
  - **Augmentation**: Translation (5 languages), jailbreak templates, formality variation, prefill attacks
 
179
 
180
  1. **Encoding bypass**: ROT13 achieves 47.9% ASR, URL-encode 29.2%. Use `preprocessing.normalize_text()` to mitigate.
181
  2. **Synthetic-only training**: All examples are Claude-generated; real-world distribution shift is uncharacterized.
182
+ 3. **External validation gap**: External kappa = 0.414 vs. target 0.80. The benchmark (BioThreat-Eval) uses response-based labeling; this classifier labels queries: an architectural mismatch that accounts for the gap. See [GitHub README](https://github.com/jang1563/constitutional-bioguard#limitations) for full explanation.
183
  4. **English-centric**: Evaluation is English-only despite multilingual augmentation in training.
184
  5. **Single LLM training data**: All data from Claude; cross-LLM calibration is unknown.
185
  6. **Not a complete defense**: Mean adversarial ASR = 9.79%; should be used as one layer in a broader safety system.
 
188
 
189
  ## Ethical Considerations
190
 
191
+ This model detects potentially dangerous biological content to support biosafety compliance in AI systems. The training data contains synthetic descriptions of potentially harmful topics: necessary to teach the classifier what to flag, not to enable harm.
192
 
193
  Do not use this model to identify exploitable gaps in biosafety systems for malicious purposes, or as the sole safety mechanism in contexts where a false negative could enable serious harm.
194