Update README.md

README.md

@@ -1,199 +1,231 @@
 ---
 library_name: transformers
-tags: []
+language:
+- en
+license: apache-2.0
+base_model: distilbert-base-uncased
+datasets:
+- jigsaw-toxic-comment-classification-challenge
+tags:
+- text-classification
+- multi-label-classification
+- content-moderation
+- toxic-comment-detection
+- distilbert
+metrics:
+- f1
+- roc_auc
 ---
 
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
+# lakresi/toxic-comment-moderator
 
+A DistilBERT model fine tuned for multi label toxic comment classification across six categories. Trained on the Jigsaw Toxic Comment Classification dataset. Deployed as a production REST API.
 
 ## Model Details
 
 ### Model Description
 
-<!-- Provide a longer summary of what this model is. -->
+distilbert-base-uncased fine tuned for multi label classification of toxic content. A linear classification head maps the CLS token representation to six independent output logits, one per toxicity category. Sigmoid activation produces per label probabilities between 0 and 1. Each label is classified independently so a comment can be flagged for multiple categories simultaneously.
 
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+Trained with BCEWithLogitsLoss and per label pos_weight to handle significant class imbalance. Per label decision thresholds were tuned on the validation set post training and stored in thresholds.json.
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+- **Developed by:** Lloyd Akresi
+- **Model type:** Text classification, multi label
+- **Language:** English
+- **License:** Apache 2.0
+- **Finetuned from:** distilbert-base-uncased
 
-### Model Sources [optional]
+### Model Sources
 
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+- **Repository:** [GitHub](https://github.com/lloydakresi/toxic_comment_moderator_api)
+- **Demo:** [Live API](https://lakresi-toxic-comment-moderator.hf.space) | [Interactive Docs](https://lakresi-toxic-comment-moderator.hf.space/docs)
 
 ## Uses
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
 ### Direct Use
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+Classifying text across six toxicity categories for content moderation pipelines. The model accepts raw English text and returns per label probabilities and binary flags based on tuned decision thresholds.
 
-[More Information Needed]
+### Downstream Use
 
-### Downstream Use [optional]
+Can be used as a baseline for further fine tuning on domain specific moderation tasks. The model weights and tokenizer can be loaded directly via HuggingFace Transformers as shown in the How to Get Started section below.
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+### Out of Scope Use
 
-[More Information Needed]
+- Automated content removal without human review for borderline or LGBTQ+ identity related sentences. See the bias section below.
+- Languages other than English.
+- Deployment without awareness of the known bias issues documented below.
 
-### Out-of-Scope Use
+## Bias, Risks, and Limitations
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+The model was evaluated against a synthetic probe set of 1,400 sentences spanning 25 demographic groups across religion, race and ethnicity, gender and sexuality, and nationality.
 
-[More Information Needed]
+12 of 1,400 sentences (0.9%) were flagged. All flags were on the toxic label only and concentrated entirely in three groups: Gay, Lesbian, and Transgender. No sentence from any of the other 22 groups tested was flagged in any category.
 
-## Bias, Risks, and Limitations
+The effect is driven by a copula construction stating identity directly. "The new manager is Gay" scored 0.818 toxic probability. The structurally identical "The new manager is Muslim" scored 0.007. This is a documented artifact of the Jigsaw training corpus where constructions like "that's so gay" are disproportionately represented as insults, causing the model to associate the is/are + LGBTQ+ identity term pattern with toxicity independent of surrounding sentiment or context.
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+Additional limitations:
 
-[More Information Needed]
+- Comments exceeding 512 tokens are truncated and content beyond the limit is not considered
+- Trained on Wikipedia comments so performance may vary on platform specific language or slang
+- Rare categories threat and severe_toxic have lower F1 scores due to data scarcity and should be treated as indicative rather than definitive
 
 ### Recommendations
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+Do not use this model for automated content removal on sentences that state an LGBTQ+ identity using an is/are construction. Route these to human review instead. A post hoc calibration layer or allowlist rule for this specific construction is a reasonable short term mitigation before any automated action is taken.
 
 ## How to Get Started with the Model
 
-Use the code below to get started with the model.
-
-[More Information Needed]
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+from huggingface_hub import hf_hub_download
+import torch
+import json
+
+model_name = "lakresi/toxic-comment-moderator"
+
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForSequenceClassification.from_pretrained(model_name)
+model.eval()
+
+file_path = hf_hub_download(repo_id=model_name, filename="thresholds.json")
+with open(file_path) as f:
+    thresholds = json.load(f)
+
+label_cols = ["toxic", "severe_toxic", "obscene", "threat", "insult", "identity_hate"]
+
+def classify(text: str):
+    inputs = tokenizer(
+        text,
+        return_tensors="pt",
+        truncation=True,
+        padding="max_length",
+        max_length=512
+    )
+    with torch.no_grad():
+        logits = model(**inputs).logits
+    probs = torch.sigmoid(logits).squeeze().tolist()
+    return {
+        label: {
+            "probability": round(prob, 4),
+            "flagged": prob >= thresholds[label]
+        }
+        for label, prob in zip(label_cols, probs)
+    }
+
+result = classify("You are absolutely worthless and should be ashamed of yourself")
+print(result)
+```
 
 ## Training Details
 
 ### Training Data
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+Jigsaw Toxic Comment Classification Challenge dataset. Approximately 145k training examples and 16k validation examples. Wikipedia comments labeled across six binary toxicity categories.
 
-[More Information Needed]
+Class distribution is highly imbalanced: toxic appears in ~10.2% of comments, obscene in ~5.9%, insult in ~4.9%, severe_toxic in ~1.0%, identity_hate in ~0.9%, and threat in ~0.3%. Per label pos_weight was applied inversely proportional to each label's positive frequency to address this.
 
 ### Training Procedure
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
+#### Preprocessing
 
+Text tokenized with the distilbert-base-uncased tokenizer. Maximum sequence length of 512 tokens with truncation and padding. Comments exceeding 512 tokens are truncated from the right.
 
 #### Training Hyperparameters
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+- **Training regime:** fp32
+- **Epochs:** 5
+- **Batch size:** 256 effective (128 per device across 2x RTX 3090)
+- **Optimizer:** AdamW
+- **Loss function:** BCEWithLogitsLoss with per label pos_weight
+- **Primary metric for best model:** F1 Macro
 
-#### Speeds, Sizes, Times [optional]
+#### Speeds, Sizes, Times
 
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
+- **Hardware:** 2x NVIDIA RTX 3090 (24GB VRAM each) via Vast.ai
+- **Training time:** ~34 minutes (2,845 steps at ~1.57 iterations per second)
+- **Evaluation speed:** ~410 samples per second on validation set
 
 ## Evaluation
 
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
+### Testing Data, Factors and Metrics
 
 #### Testing Data
 
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
+16,180 held out examples from the Jigsaw Toxic Comment Classification dataset, stratified by label to preserve class distribution.
 
 #### Factors
 
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
+Performance disaggregated by toxicity category. Rare labels evaluated separately given their low positive frequency.
 
 #### Metrics
 
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
+F1 score per label and overall macro F1. ROC AUC to measure discriminative capability independent of threshold. Both metrics reported because the model can have high ROC AUC (strong ranking ability) while having lower F1 macro (threshold sensitivity), which is exactly what was observed here.
 
 ### Results
 
-[More Information Needed]
-
-#### Summary
+**Epoch by epoch training dynamics:**
 
+| Epoch | Train Loss | Eval Loss | F1 Macro | ROC AUC |
+|---|---|---|---|---|
+| 1 | 3.917 | 1.598 | 0.632 | 0.989 |
+| 2 | 1.438 | 1.546 | 0.670 | 0.990 |
+| 3 | 1.116 | 1.687 | 0.671 | 0.989 |
+| 4 | 0.857 | 1.880 | 0.684 | 0.987 |
+| 5 | 0.697 | 1.934 | 0.687 | 0.987 |
 
+**Per label results after threshold tuning:**
 
-## Model Examination [optional]
+| Category | Threshold | F1 Score | Class Frequency |
+|---|---|---|---|
+| obscene | 0.44 | 0.849 | ~5.9% |
+| toxic | 0.52 | 0.840 | ~10.2% |
+| insult | 0.52 | 0.765 | ~4.9% |
+| identity_hate | 0.26 | 0.589 | ~0.9% |
+| threat | 0.36 | 0.561 | ~0.3% |
+| severe_toxic | 0.46 | 0.558 | ~1.0% |
+| **Overall** | | **0.687 macro / 0.987 ROC AUC** | |
 
-<!-- Relevant interpretability work for the model goes here -->
+#### Summary
 
-[More Information Needed]
+Strong performance on frequent labels (toxic, obscene, insult) with F1 above 0.76. Rare labels (threat, severe_toxic, identity_hate) underperform due to data scarcity rather than modeling limitations, as evidenced by the consistently high ROC AUC of 0.987 across all labels. Per label threshold tuning meaningfully improved F1 over the default 0.5 cutoff, particularly for rare labels where lower thresholds improve recall at acceptable precision cost.
 
 ## Environmental Impact
 
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+- **Hardware type:** 2x NVIDIA RTX 3090 via Vast.ai
+- **Hours used:** ~0.57 hours (34 minutes)
+- **Cloud provider:** Vast.ai
+- **Compute region:** Not specified
+- **Carbon emitted:** Estimated minimal given short training duration
 
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
+## Technical Specifications
 
 ### Model Architecture and Objective
 
-[More Information Needed]
+distilbert-base-uncased with a single linear classification head. Input text is tokenized and passed through 6 transformer layers. The CLS token representation from the final layer is passed through the linear head to produce 6 logits. Sigmoid activation converts logits to independent per label probabilities. Binary cross entropy loss with logits and per label positive weighting.
 
 ### Compute Infrastructure
 
-[More Information Needed]
-
 #### Hardware
 
-[More Information Needed]
+2x NVIDIA RTX 3090 (24GB VRAM each) via Vast.ai. Multi GPU training managed by HuggingFace Accelerate.
 
 #### Software
 
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
+- Python 3.10
+- PyTorch
+- HuggingFace Transformers
+- HuggingFace Accelerate
+- HuggingFace Trainer
 
-## More Information [optional]
+## More Information
 
-[More Information Needed]
+Full technical breakdown including training code, bias analysis methodology, API documentation, and Docker deployment instructions are available in the [GitHub repository](https://github.com/lloydakresi/toxic_comment_moderator_api).
 
-## Model Card Authors [optional]
+## Model Card Authors
 
-[More Information Needed]
+Lloyd Akresi
 
 ## Model Card Contact
 
-[More Information Needed]
+[GitHub](https://github.com/lloydakresi)