---
license: cc-by-nc-4.0
library_name: transformers
pipeline_tag: translation
base_model: facebook/nllb-200-distilled-600M
language:
- ami
- bnn
- ckv
- dru
- pwn
- pyu
- ssf
- sxr
- szy
- tao
- tay
- trv
- tsu
- xnb
- xsy
- zh
tags:
- translation
- nllb-200
- formosan-languages
- low-resource
metrics:
- bleu
- chrf
- ter
model-index:
- name: nllb200-formosan-zh-spm8k
  results:
  - task:
      type: translation
      name: Translation
    dataset:
      name: FormosanBank private no-Bible hard test
      type: private-no-bible-hard-test
      split: test
    metrics:
    - type: bleu
      name: sacreBLEU
      value: 15.134699
    - type: chrf
      name: chrF2
      value: 15.382060
    - type: ter
      name: TER
      value: 79.847105
---

# nllb200-formosan-zh-spm8k

**Direction:** Formosan to Traditional Chinese<br>
**Base model:** [`facebook/nllb-200-distilled-600M`](https://huggingface.co/facebook/nllb-200-distilled-600M)<br>
**Recipe:** `nllb200-spm8k-directional-v5`<br>
**Release:** `20260816-193900`, validation-selected step 180,000

This is a directional model for 15 Formosan languages. It uses the
`private_no_bible` leakage-controlled corpus, Formosan-aware 8k SentencePiece extension, balanced
language/source sampling, direction/language controls, fixed coarse domain
tags, and dialect tags. Domain and dialect metadata use independent training
dropout so `unknown` and `default` are learned inference conditions. The model
weights are public. The training corpus is distributed separately to
authorized FormosanBank members through the access-controlled
[`FormosanBank/formosan-mt-private`](https://huggingface.co/datasets/FormosanBank/formosan-mt-private)
dataset and is not included with the weights.

## Model details

| Item | Value |
|---|---|
| Base revision | `f8d333a098d19b4fd9a8b18f94170487ad3f821d` |
| Training rows | 625,460 |
| Effective batch size | 64 |
| Maximum sequence length | 384 |
| Learning rate | 2e-05 |
| Precision | `bf16` |
| Checkpoint selection | Validation `chrF2` |
| Formosan text | `kindOf=standard`, `formosan-mt-standard-v3` |
| Corpus SHA-256 | `0024e7fcd35958455a2a060b20999e94b5e8a4364e7f54ba97ef27810746d932` |
| Training profile SHA-256 | `4253b1b136fedebc589da942b80df71945212d1f4b92a8e98ec6fb160712f14b` |

## Usage

```python
import torch
from transformers import AutoModelForSeq2SeqLM, NllbTokenizer
from formosan_mt_inference import normalize_formosan


model_id = "FormosanBank/nllb200-formosan-zh-spm8k"
tokenizer = NllbTokenizer.from_pretrained(model_id, use_fast=False)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)
model.to("cuda" if torch.cuda.is_available() else "cpu")
NLLB_LIDS = {'ami': 'ami_Latn', 'bnn': 'bnn_Latn', 'ckv': 'ckv_Latn', 'dru': 'dru_Latn', 'pwn': 'pwn_Latn', 'pyu': 'pyu_Latn', 'ssf': 'ssf_Latn', 'sxr': 'sxr_Latn', 'szy': 'szy_Latn', 'tao': 'tao_Latn', 'tay': 'tay_Latn', 'trv': 'trv_Latn', 'tsu': 'tsu_Latn', 'xnb': 'xnb_Latn', 'xsy': 'xsy_Latn'}

def translate(text, lang_code, source_bucket="unknown", dialect="default"):
    text = normalize_formosan(text, lang_code)
    tokenizer.src_lang = NLLB_LIDS[lang_code]
    source_bucket = source_bucket if source_bucket in ('dictionary', 'classroom', 'narrative', 'linguistic', 'education', 'media', 'culture', 'religious', 'unknown') else "unknown"
    domain_tag = f"<dom_{source_bucket}>"
    if tokenizer.convert_tokens_to_ids(domain_tag) == tokenizer.unk_token_id:
        domain_tag = "<dom_unknown>"
    dialect_tag = f"<dialect_{dialect}>"
    if tokenizer.convert_tokens_to_ids(dialect_tag) == tokenizer.unk_token_id:
        dialect_tag = "<dialect_default>"
    prompt = (
        f"<to_zh> <src_{lang_code}> {domain_tag} {dialect_tag} {text}"
    )
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    output = model.generate(
        **inputs,
        decoder_start_token_id=tokenizer.eos_token_id,
        forced_bos_token_id=tokenizer.convert_tokens_to_ids('zho_Hant'),
        max_new_tokens=256,
        num_beams=4,
    )
    return tokenizer.batch_decode(output, skip_special_tokens=True)[0]

print(translate("Pa'araw cingra.", "ami"))
```

The control tags are part of the training contract. Use `unknown` and `default`
when source bucket or dialect metadata is unavailable.

## Evaluation

The best checkpoint was selected on validation chrF2. Test and validation
contain only eligible, human-translated sentence pairs. Synthetic pivots and
lexical entries are train-only.
The headline result uses `default` metadata controls, so it does not
assume access to test-set domain or dialect labels.

| Split | Rows |
|---|---:|
| Train | 625,460 |
| Test | 73,591 |
| Validate | 36,797 |

| Scope | BLEU | chrF2 | TER |
|---|---:|---:|---:|
| Hard test | 15.13 | 15.38 | 79.85 |
| Selection validation | 9.55 | 10.50 | 86.50 |

Test empty-output rate: 0.0190%.


| Language | Samples | BLEU | chrF2 | TER |
|---|---:|---:|---:|---:|
| `ami` | 14,348 | 13.03 | 14.08 | 80.09 |
| `bnn` | 6,048 | 17.27 | 16.77 | 77.89 |
| `ckv` | 2,871 | 21.78 | 20.30 | 68.64 |
| `dru` | 6,588 | 12.08 | 13.05 | 91.76 |
| `pwn` | 5,945 | 12.89 | 12.82 | 87.55 |
| `pyu` | 4,334 | 17.67 | 18.11 | 77.29 |
| `ssf` | 2,150 | 20.17 | 19.20 | 68.65 |
| `sxr` | 1,968 | 12.38 | 12.83 | 86.25 |
| `szy` | 2,300 | 19.09 | 19.14 | 72.29 |
| `tao` | 2,360 | 13.46 | 13.84 | 82.59 |
| `tay` | 7,161 | 12.94 | 13.52 | 86.05 |
| `trv` | 10,728 | 14.59 | 15.53 | 75.77 |
| `tsu` | 2,048 | 14.35 | 15.18 | 81.42 |
| `xnb` | 2,590 | 22.91 | 21.43 | 64.09 |
| `xsy` | 2,152 | 17.85 | 17.01 | 73.92 |

The corpus gate enforces standard-tier Formosan text, 5/10
validation/test proportions from all deduplicated pairs, capacity-aware source
balance, sentence-only evaluation, and
zero exact, skeleton, one-edit, or configured high character n-gram
train/evaluation conflicts. Document overlap is diagnostic. This release
passed all leakage gates: exact
0, skeleton
0, one-edit
0, character n-gram
0. Document overlap:
1439.

See `eval/metrics.json` for sacreBLEU signatures, per-language, source,
dialect, and length diagnostics. `publication.json` records the corpus,
profile, run, and checkpoint hashes used for this release.

## Intended use

This model supports research, corpus development, and assisted translation for
the 15 included Formosan languages. It is designed for the exact prompt and
generation contract shown above.

## Limitations

Outputs require knowledgeable speaker review. Aggregate metrics hide large
differences among languages and domains. This model is not suitable for
authoritative, medical, legal, or safety-critical translation.