---
language:
- ru
- ba
tags:
- translation
- nllb
- qlora
- peft
- loresmt
license: mit
base_model: facebook/nllb-200-1.3B
datasets:
- AigizK/bashkir-russian-parallel-corpora
metrics:
- chrf
widget:
- text: "Искусственный интеллект меняет мир."
  example_title: "AI Example"
- text: "Привет, как твои дела?"
  example_title: "Greeting"
model-index:
- name: DevLake NLLB-1.3B
  results:
  - task:
      type: translation
      name: Translation
    dataset:
      name: LoResMT 2026
      type: loresmt-2026
    metrics:
    - name: CHRF++
      type: chrf
      value: 52.67
---

# DevLake: NLLB-200 (1.3B) for Russian-Bashkir Translation

<div align="center">

| **Current Model** | **Architecture** | **Focus** |
|:---:|:---:|:---:|
| 🔴 **Large (This Model)** | **NLLB-1.3B (QLoRA)** | **Best Quality (SOTA)** |
| [🟡 Medium Model](https://huggingface.co/Voldis/m2m100-rus-bak) | M2M-100 (418M) | Balanced |
| [🟢 Small Model](https://huggingface.co/Voldis/marian-rus-bak) | MarianMT (77M) | Fastest / CPU |

</div>

## Model Description

This is the **High-Performance** model submitted by Team **DevLake** for the **LoResMT 2026 Shared Task**. It achieved the highest score in our experiments (**52.67 CHRF++**), significantly outperforming standard baselines.

It is a fine-tuned version of [NLLB-200-1.3B-Distilled](https://huggingface.co/facebook/nllb-200-1.3B), trained using **QLoRA** (4-bit quantization) on a rigorously filtered subset of the Russian-Bashkir parallel corpus.

- **Paper/Code:** [GitHub Repository](https://github.com/Voldisoriginal/LoResMT-2026-Russian-Bashkir)
- **Developed by:** DevLake Team
- **Language Pair:** Russian (`rus_Cyrl`) $\to$ Bashkir (`bak_Cyrl`)

## Performance

| Model | Size | CHRF++ | Note |
| :--- | :--- | :--- | :--- |
| **DevLake NLLB** | **1.3B** | **52.67** | **Best Morphology & Syntax** |
| DevLake M2M | 418M | 48.80 | Good baseline |
| DevLake Marian | 77M | 43.15 | Fast but hallucinates |

## Usage

Due to the use of QLoRA, this model requires `peft` and `bitsandbytes`.

```bash
pip install torch transformers peft bitsandbytes accelerate
```

```python
import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from peft import PeftModel

# 1. Load Base Model (NLLB)
base_model_id = "facebook/nllb-200-1.3B"
model = AutoModelForSeq2SeqLM.from_pretrained(
    base_model_id,
    load_in_4bit=True,
    device_map="auto"
)

# 2. Load DevLake Adapters
adapter_model_id = "Voldis/nllb-1.3b-rus-bak"
model = PeftModel.from_pretrained(model, adapter_model_id)
tokenizer = AutoTokenizer.from_pretrained(adapter_model_id)

# 3. Inference
text = "Утром я выпил чашку кофе."
inputs = tokenizer(text, return_tensors="pt").to("cuda")

with torch.no_grad():
    generated_tokens = model.generate(
        **inputs,
        forced_bos_token_id=tokenizer.convert_tokens_to_ids("bak_Cyrl"),
        max_length=128
    )

print(tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)[0])
```

## Training Details

- **Filtering:** We used a BERT-based metric to select only the top 486k sentence pairs (Semantic Similarity $\ge 0.80$).
- **Method:** QLoRA (Rank=64, Alpha=64).
- **Hardware:** Trained on a single NVIDIA RTX 3080.

## Citation

```bibtex
@inproceedings{tyurin-2026-devlake,
    title = "{D}ev{L}ake at {L}o{R}es{MT} 2026: The Impact of Pre-training and Model Scale on {R}ussian-{B}ashkir Low-Resource Translation",
    author = "Tyurin, Vyacheslav",
    booktitle = "Proceedings for the Ninth Workshop on Technologies for Machine Translation of Low Resource Languages (LoResMT 2026)",
    month = mar,
    year = "2026",
    address = "Rabat, Morocco",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.loresmt-1.18",
    doi = "10.18653/v1/2026.loresmt-1.18",
    pages = "209--212",
}
```