| --- |
| language: |
| - en |
| license: apache-2.0 |
| base_model: microsoft/deberta-v3-small |
| tags: |
| - text-classification |
| - literary-analysis |
| - content-moderation |
| - explicitness-detection |
| - deberta-v3 |
| - pytorch |
| - focal-loss |
| pipeline_tag: text-classification |
| model-index: |
| - name: deberta-v3-small-explicit-classifier-v2 |
| results: |
| - task: |
| type: text-classification |
| name: Literary Explicitness Classification |
| dataset: |
| name: Custom Literary Dataset (Deduplicated) |
| type: custom |
| metrics: |
| - type: accuracy |
| value: 0.818 |
| name: Accuracy |
| - type: f1 |
| value: 0.754 |
| name: Macro F1 |
| - type: f1 |
| value: 0.816 |
| name: Weighted F1 |
| widget: |
| - text: "Content warning: This story contains mature themes including explicit sexual content and violence." |
| example_title: "Content Disclaimer" |
| - text: "His hand lingered on hers as he helped her from the carriage, their fingers intertwining despite propriety." |
| example_title: "Suggestive Romance" |
| - text: "She gasped as he traced kisses down her neck, his hands exploring the curves of her body with growing urgency." |
| example_title: "Explicit Sexual" |
| - text: "The morning mist drifted across the Yorkshire moors as Elizabeth walked the familiar path to the village." |
| example_title: "Non-Explicit Literary" |
| --- |
| |
| # Literary Content Classifier - DeBERTa v3 Small (v2.0) |
|
|
| An improved fine-tuned DeBERTa-v3-small model for sophisticated literary content analysis across 7 categories of explicitness. This v2.0 model features **significant improvements** over the original, including focal loss training, extended epochs, and data quality enhancements. |
|
|
| ## 🚀 Key Improvements in v2.0 |
|
|
| - **+4.5% accuracy improvement** (81.8% vs 77.3%) |
| - **+6.4% macro F1 improvement** (0.754 vs 0.709) |
| - **+21% improvement on violent content** (F1: 0.581 vs 0.478) |
| - **+19% improvement on suggestive content** (F1: 0.476 vs 0.400) |
| - **Focal loss training** for better minority class performance |
| - **Clean dataset** with cross-split contamination resolved |
| - **Extended training** (4.79 epochs vs 1.1 epochs) |
|
|
| ## Model Description |
|
|
| This model provides nuanced classification of textual content across 7 categories, enabling sophisticated analysis for digital humanities, content curation, and literary research applications. |
|
|
| ### Categories |
|
|
| | ID | Category | Description | F1 Score | |
| |----|----------|-------------|----------| |
| | 0 | EXPLICIT-DISCLAIMER | Content warnings and age restriction notices | **0.977** | |
| | 1 | EXPLICIT-OFFENSIVE | Profanity, crude language, offensive content | **0.813** | |
| | 2 | EXPLICIT-SEXUAL | Graphic sexual content and detailed intimate scenes | **0.930** | |
| | 3 | EXPLICIT-VIOLENT | Violent or disturbing content | **0.581** | |
| | 4 | NON-EXPLICIT | Clean, family-friendly content | **0.851** | |
| | 5 | SEXUAL-REFERENCE | Mentions of sexual topics without graphic description | **0.652** | |
| | 6 | SUGGESTIVE | Mild innuendo or romantic themes without explicit detail | **0.476** | |
|
|
| ## Performance Metrics |
|
|
| ### Overall Performance |
| - **Accuracy**: 81.8% |
| - **Macro F1**: 0.754 |
| - **Weighted F1**: 0.816 |
|
|
| ### Detailed Results (Test Set) |
| ``` |
| precision recall f1-score support |
| EXPLICIT-DISCLAIMER 0.95 1.00 0.98 19 |
| EXPLICIT-OFFENSIVE 0.82 0.88 0.81 414 |
| EXPLICIT-SEXUAL 0.93 0.91 0.93 514 |
| EXPLICIT-VIOLENT 0.44 0.62 0.58 24 |
| NON-EXPLICIT 0.77 0.87 0.85 683 |
| SEXUAL-REFERENCE 0.63 0.73 0.65 212 |
| SUGGESTIVE 0.37 0.46 0.48 134 |
| |
| accuracy 0.82 2000 |
| macro avg 0.65 0.78 0.75 2000 |
| weighted avg 0.75 0.82 0.82 2000 |
| ``` |
|
|
| ## Training Details |
|
|
| ### Model Architecture |
| - **Base Model**: microsoft/deberta-v3-small |
| - **Parameters**: 141.9M (6 layers, 768 hidden, 12 attention heads) |
| - **Vocabulary**: 128,100 tokens |
| - **Max Sequence Length**: 512 tokens |
|
|
| ### Training Configuration |
| - **Training Method**: Focal Loss (γ=2.0) for class imbalance |
| - **Epochs**: 4.79 (early stopped) |
| - **Learning Rate**: 5e-5 with cosine schedule |
| - **Batch Size**: 16 (effective 32 with gradient accumulation) |
| - **Warmup Steps**: 1,000 |
| - **Weight Decay**: 0.01 |
| - **Early Stopping**: Patience 5 on macro F1 |
|
|
| ### Dataset |
| - **Total Samples**: 119,023 (after deduplication) |
| - **Training**: 83,316 samples |
| - **Validation**: 17,853 samples |
| - **Test**: 17,854 samples |
| - **Data Quality**: Cross-split contamination eliminated (2,127 duplicates removed) |
|
|
| ### Training Environment |
| - **Framework**: PyTorch + Transformers |
| - **Hardware**: Apple Silicon (MPS) |
| - **Training Time**: ~13.7 hours |
|
|
| ## Usage |
|
|
| ```python |
| from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline |
| |
| # Load model and tokenizer |
| model_id = "your-username/deberta-v3-small-explicit-classifier-v2" |
| model = AutoModelForSequenceClassification.from_pretrained(model_id) |
| tokenizer = AutoTokenizer.from_pretrained(model_id) |
| |
| # Create classification pipeline |
| classifier = pipeline( |
| "text-classification", |
| model=model, |
| tokenizer=tokenizer, |
| return_all_scores=True, |
| truncation=True |
| ) |
| |
| # Single classification |
| text = "His hand lingered on hers as he helped her from the carriage." |
| result = classifier(text) |
| print(f"Top prediction: {result[0]['label']} ({result[0]['score']:.3f})") |
| |
| # All class probabilities |
| for class_result in result: |
| print(f"{class_result['label']}: {class_result['score']:.3f}") |
| ``` |
|
|
| ### Recommended Thresholds (F1-Optimized) |
|
|
| For applications requiring specific precision/recall trade-offs: |
|
|
| | Class | Optimal Threshold | Precision | Recall | F1 | |
| |-------|------------------|-----------|--------|-----| |
| | EXPLICIT-DISCLAIMER | 0.995 | 0.950 | 1.000 | 0.974 | |
| | EXPLICIT-OFFENSIVE | 0.626 | 0.819 | 0.829 | 0.824 | |
| | EXPLICIT-SEXUAL | 0.456 | 0.927 | 0.911 | 0.919 | |
| | EXPLICIT-VIOLENT | 0.105 | 0.441 | 0.625 | 0.517 | |
| | NON-EXPLICIT | 0.103 | 0.768 | 0.874 | 0.818 | |
| | SEXUAL-REFERENCE | 0.355 | 0.629 | 0.726 | 0.674 | |
| | SUGGESTIVE | 0.530 | 0.370 | 0.455 | 0.408 | |
|
|
| ## Model Files |
|
|
| - `model.safetensors`: Model weights in SafeTensors format |
| - `config.json`: Model configuration with proper label mappings |
| - `tokenizer.json`, `spm.model`: SentencePiece tokenizer files |
| - `label_mapping.json`: Label ID to name mapping reference |
|
|
| ## Limitations & Considerations |
|
|
| 1. **Challenging Distinctions**: SUGGESTIVE vs SEXUAL-REFERENCE categories remain difficult to distinguish due to conceptual overlap |
| 2. **Minority Classes**: EXPLICIT-VIOLENT and SUGGESTIVE classes have lower F1 scores due to limited training data |
| 3. **Context Dependency**: Short text snippets may lack sufficient context for accurate classification |
| 4. **Domain Specificity**: Optimized for literary and review content; performance may vary on other text types |
| 5. **Language**: English text only |
|
|
| ## Evaluation Artifacts |
|
|
| The model includes comprehensive evaluation materials: |
| - Confusion matrix visualization |
| - Per-class precision-recall curves |
| - ROC curves for all categories |
| - Calibration analysis |
| - Recommended decision thresholds |
|
|
| ## Ethical Use |
|
|
| This model is designed for: |
| - Academic research and digital humanities |
| - Content curation and library science applications |
| - Literary analysis and publishing workflows |
| - Educational content assessment |
|
|
| **Important**: This model should be used responsibly with human oversight for content moderation decisions. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{literary-explicit-classifier-v2-2025, |
| title={Literary Content Analysis: Improved Multi-Class Classification with Focal Loss}, |
| author={Explicit Content Research Team}, |
| year={2025}, |
| note={DeBERTa-v3-small fine-tuned for literary explicitness detection} |
| } |
| ``` |
|
|
| ## License |
|
|
| This model is released under the Apache 2.0 license. |
