---
language:
  - bn
license: cc-by-4.0
library_name: transformers
pipeline_tag: fill-mask
tags:
  - bengali
  - bangla
  - BnLM-F
  - bnlm
  - bangla-model
  - small-bangla-model
  - bengali-nlp
  - bnlp
  - low-resource
  - masked-language-modeling
  - pretrained
  - language-model
  - distilbert
datasets:
  - nahid-hub/B-CORE-bengali-corpus
model-index:
  - name: BnLM-F
    results: []
---

# BnLM-F: Bangla Language Model (135M)

**BnLM-F** is a 135M-parameter Bangla-specific pretrained language model, one of three models in the **BnLM** suite (**B**e**n**gali **L**anguage **M**odel) introduced alongside the **BLUGE** benchmark and **B-CORE** pretraining corpus. BnLM models are pretrained from scratch on Bangla text using the Masked Language Modeling (MLM) objective, and are designed for efficient, low-resource NLP without relying on large multilingual models.

See the [BLUGE collection](https://huggingface.co/collections/nahid-hub/bluge) for the full release — evaluation tasks, pretraining corpus, tokenizers, and all three BnLM variants (BnLM-F, BnLM-M, BnLM-C).

## Model Details

| | |
|---|---|
| Base architecture | DistilBERT-multilingual |
| Parameters | 135M |
| Tokenizer | Custom Bangla WordPiece, 120K vocabulary |
| Objective | Masked Language Modeling (MLM) |
| Max sequence length | 512 tokens |
| Pretraining data | [B-CORE](https://huggingface.co/datasets/nahid-hub/B-CORE-bengali-corpus) — 4.32B tokens, 52GB |

Despite its compact size, BnLM-F achieves state-of-the-art results across BLUGE and four cross-lingual/Indic benchmarks, while requiring 44–91% fewer FLOPs and ~2x faster inference than competitive multilingual baselines. Full training configuration and hyperparameters are available in the paper.

## Usage

### Fine-tuning for sentiment classification

Example using [BLUGE-TSC](https://huggingface.co/datasets/nahid-hub/BLUGE-bengali-sentiment-classification):

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("nahid-hub/BnLM-F-135m")
model = AutoModelForSequenceClassification.from_pretrained(
    "nahid-hub/BnLM-F-135m",
    num_labels=3  # 0: neutral, 1: positive, 2: negative
)

text = "খাবারটা অসাধারণ ছিল, আমি খুব খুশি!"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)

with torch.no_grad():
    logits = model(**inputs).logits

predicted_class = logits.argmax(dim=-1).item()
labels = {0: "neutral", 1: "positive", 2: "negative"}
print(labels[predicted_class])
```

> Note: `AutoModelForSequenceClassification` attaches a randomly initialized classification head on top of the pretrained BnLM-F encoder — you'll need to fine-tune on labeled data (e.g. BLUGE-TSC) before this produces meaningful predictions. Skip straight to inference only if loading your own fine-tuned checkpoint.

### Loading the base model (for masked-language-modeling or custom heads)

```python
from transformers import AutoTokenizer, AutoModelForMaskedLM

tokenizer = AutoTokenizer.from_pretrained("nahid-hub/BnLM-F-135m")
model = AutoModelForMaskedLM.from_pretrained("nahid-hub/BnLM-F-135m")
```

## Training Data

Pretrained exclusively on **B-CORE**, a 16.5M-document, 4.32-billion-token Bangla corpus built via a reproducible multi-stage pipeline achieving a 22.4% reduction in corpus volume through quality filtering and cross-corpus deduplication.

## Evaluation

Evaluated on **BLUGE** (7-task Bangla NLU benchmark) and 4 cross-lingual/Indic benchmarks. See the paper for full results tables.

| Task | Metric | Score |
|---|---|---|
| [fill in from your paper's results tables] | | |

## Related Models

- [BnLM-M](https://huggingface.co/nahid-hub/BnLM-M-135m) — 135M parameters, larger-vocabulary variant tuned for different training regime
- [BnLM-C](https://huggingface.co/nahid-hub/BnLM-C-66m) — 66M parameters, compact variant with 30.5K-vocabulary tokenizer

## Limitations

[e.g. domain coverage of B-CORE, max sequence length, any known biases — fill in from your paper's limitations section]

## License

Released under **CC BY 4.0**.

## Citation

If you use this model, please cite:

```bibtex
@ARTICLE{BnLM-BLUGE-B-CORE,
  author={Hossain, Nahid and Faisal Kabir, Md.},
  journal={IEEE Access}, 
  title={Efficient Monolingual Pretraining in Low-Resource Settings Through Morphology-Aware Tokenization, Principled Corpus Denoising, and Benchmark-Driven Evaluation}, 
  year={2026},
  volume={14},
  number={},
  pages={91979-92003},
  keywords={Modeling;Multilingual;Training;Cleaning;Vocabulary;Labeling;Tokenization;Computational linguistics;Pipelines;Conferences;B-CORE;BLUGE;BnLM;corpus;evaluation benchmark;pretrained models},
  doi={10.1109/ACCESS.2026.3701520}
}
```