gospelgit/African-Languages-Sentiment-Classifier

African Languages Sentiment Classifier (Hausa, Yorùbá, Swahili)

A LoRA-adapted sentiment classifier for Hausa, Yorùbá, and Swahili, fine-tuned on a combined dataset of 46,725 rows stitched from three independent sources across three different domains, built to reduce the single-domain (Twitter-only) bias common in existing African-language sentiment resources.

Model Details

• Base model: togethercomputer/gpt-oss-20b-bf16
• Adapter type: LoRA (PEFT), rank 64, alpha 128, target modules q_proj/k_proj/v_proj/o_proj
• Task formulation: causal LM, prompt → single-word completion (the model generates the sentiment label as its next-token completion)
• Languages: Hausa, Yorùbá, Swahili
• License: CC-BY-4.0
• Produced via: Adaption Labs AutoScientist (Language category submission)
• AutoScientist training run ID: adaption_gpt_oss_20b_ha_yo_sw_sentiment_1eb424c7

Training Data

This model was trained on a combined dataset built from three sources:

Source	Domain	Languages	Rows
AfriSenti	Twitter	Hausa, Yorùbá, Swahili	40,290
NollySenti	Nollywood movie reviews (human-translated)	Hausa, Yorùbá	2,510
Neurotech-HQ Swahili	Social media / product reviews (back-translated)	Swahili	3,925

3-class labels (positive / negative / neutral), 70/15/15 train/dev/test split per language, stratified by label.

Note on training data adaptation: this adapter was trained on a version of this data that was adapted once via Adaption Labs' AutoScientist — its "Adaptive Data" step rewrote the original rows into enhanced_prompt/enhanced_completion pairs (15,280 rows after this process) as part of its data-and-recipe co-optimization loop. The dataset used to produce this result is included in this repo, alongside the model weights (see Files and versions).

Training Procedure

• 5 epochs, 585 total steps
• Train/eval loss decreased steadily across all 5 epochs (eval loss: 0.828 → 0.787 → 0.769 → 0.760 → 0.757)
• Learning rate: warm-up then decay schedule
• Framework: PEFT 0.15.1

Evaluation (AutoScientist internal metrics)

These are AutoScientist's own judge-based scores comparing the base model against the fine-tuned ("adapted") model — not standard accuracy/F1:

Metric	Before (base)	After (adapted)
Quality score (0–10 scale)	3.0	6.9 (+130% relative)
Grade	E	C
Percentile	1.3	8.4
Win rate — on this dataset	44	57
Win rate — general category (all tasks)	52	48

Read this table carefully: task-specific quality improved substantially (grade E→C, +130% relative quality score), but the general-category win rate slightly dropped (52→48), meaning the adaptation traded a small amount of general-purpose capability for sentiment-task performance. This is disclosed deliberately — don't assume "adapted" is strictly better in every dimension.

Intended Use

Sentiment classification (positive/negative/neutral) for short-form text in Hausa, Yorùbá, or Swahili, primarily for research and benchmarking purposes within the AutoScientist Challenge. Not validated for production deployment.

How to Use

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained("togethercomputer/gpt-oss-20b-bf16")
model = PeftModel.from_pretrained(base_model, "gospelgit/African-Languages-Sentiment-Classifier")
tokenizer = AutoTokenizer.from_pretrained("gospelgit/African-Languages-Sentiment-Classifier")

prompt = "Classify the sentiment of this text as positive, negative, or neutral: <your text here>"
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=5)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Limitations

• Evaluated via AutoScientist's internal judge/win-rate system, not an external, reproducible benchmark — independent verification is recommended before relying on these numbers.
• Trained on an evolved/rewritten version of the source data, not the raw human-annotated labels directly.
• Slight general-capability regression observed post-adaptation (see table above).
• Swahili has less underlying data than Hausa/Yorùbá — performance may be less stable for that language.

Citation

If you use this model, please also cite the original dataset sources listed in the dataset card.