nahidstaq/bge-m3-bangla

BGE-M3 Bangla Cross-Script Fine-tuned

A fine-tuned version of BAAI/bge-m3 optimized for Bangla cross-script semantic search. The model understands and aligns text across:

• Pure Bangla (বাংলা)
• Banglish (Romanized Bangla)
• Mixed code-switching (Bangla + English)
• Informal/colloquial Bangla
• Noisy/typo-heavy text
• English (same intent)

Key Results

Metric	Base BGE-M3	Fine-tuned	Improvement
Avg cross-script similarity	0.689	0.788	+0.098
Bangla ↔ Banglish	0.652	0.861	+0.209
Banglish ↔ English	0.680	0.919	+0.239
Typos ↔ English	0.612	0.856	+0.244
Dev triplet accuracy	95.30%	98.74%	+3.44%
Retrieval Hit@1	1.000	1.000	—

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("nahidstaq/bge-m3-bangla")

# These all mean the same thing - the model knows that!
sentences = [
    "আজকের আবহাওয়া কেমন?",           # Pure Bangla
    "ajker weather kemon?",             # Banglish
    "আজকে weather কেমন হবে?",          # Mixed
    "what is today's weather?",         # English
]

embeddings = model.encode(sentences, normalize_embeddings=True)

# Compute similarity
from sentence_transformers.util import cos_sim
print(cos_sim(embeddings, embeddings))
# All pairs will show high similarity (>0.85)

Training Details

• Base model: BAAI/bge-m3 (568M params, XLM-RoBERTa)
• Method: LoRA fine-tuning (rank=32, alpha=64)
• LoRA targets: query, key, value, dense layers
• Loss: MultipleNegativesRankingLoss (temperature=0.05)
• Training data: 39,082 examples
  • 27,082 augmented cross-script pairs (6 script variants per intent)
  • 12,000 synthetic cross-script parallel pairs generated using Gemini 2.5 Flash and Claude Opus 4.5
  • 10 domains: food, travel, health, tech, education, sports, society, ecommerce, government, agriculture
• Epochs: 3
• Batch size: 8
• GPU: NVIDIA RTX 4090 (24GB)
• Training time: ~3 hours
• Best checkpoint: 98.74% triplet accuracy on dev set

Intended Use

• Semantic search over Bangla content (government services, FAQs, documents)
• Cross-script information retrieval (user types in Banglish, retrieves Bangla docs)
• Bangla chatbot retrieval-augmented generation (RAG)
• Bangla text clustering and deduplication across scripts

Limitations

• Trained primarily on Bangladesh-specific domains (government, daily life, services)
• Within-group coherence is lower than base model (by design — embeddings are more discriminative)
• Pure Bangla ↔ informal Bangla similarity slightly decreased (-0.12)
• Not optimized for long documents (best for queries and short passages)

Credits

Built by Md Nahid Hasan (@nahidstaq | nahidscript.com).

Based on BAAI/bge-m3.

Citation

@misc{bge-m3-bangla-cross-script,
  author = {Hasan, Md Nahid},
  title = {BGE-M3 Bangla Cross-Script Fine-tuned},
  year = {2025},
  url = {https://huggingface.co/nahidstaq/bge-m3-bangla},
  note = {Fine-tuned on cross-script Bangla data for semantic search}
}