---
license: apache-2.0
base_model: dsfsi/nllb_200_distilled_600m-eng-zul
language:
- en
- zu
pipeline_tag: translation
tags: [onnx, translation, africanlp, dsfsi, nllb, m2m_100]
---

# nllb_200_distilled_600m-eng-zul-onnx

ONNX export of [`dsfsi/nllb_200_distilled_600m-eng-zul`](https://huggingface.co/dsfsi/nllb_200_distilled_600m-eng-zul).

Translation direction: **English (en) -> Zulu (zu)**.

## Attribution

The original model was trained by [DSFSI](https://dsfsi.github.io/) (Data
Science for Social Impact research group, University of Pretoria), a
fine-tune of `facebook/nllb-200-distilled-600M` on the `afriscience-mt`
dataset. Licence: **Apache-2.0**, as declared by the original repository.
This repository only converts the weights to ONNX. All credit for the
model belongs to DSFSI.

## Export

```
optimum-cli export onnx --model dsfsi/nllb_200_distilled_600m-eng-zul \
  --task text2text-generation-with-past --no-post-process <outdir>
```

The exported fp32 export uses external data (`decoder_model.onnx_data`,
`decoder_with_past_model.onnx_data`) since the ~600M-parameter checkpoint
exceeds the 2 GB single-file protobuf limit. The int8 quantized files
required the same treatment: `onnxruntime.quantization.quantize_dynamic`
writes external data as `<name>.onnx.data` (dot), but `from_pretrained`
looks for `<name>.onnx_data` (underscore) - the file was renamed and the
`location` field inside the ONNX proto was rewritten (including one
`Constant` node's tensor attribute, which is not a graph initializer and is
easy to miss) so the int8 build loads correctly. Verified with a real load
from a cache cleared beforehand.

## Files

| Path | Precision | Size |
|---|---|---|
| `*.onnx` + `*.onnx_data` (root) | fp32 | ~7 GB |
| `int8/*.onnx` + `int8/*.onnx_data` | int8 dynamic | ~5.5 GB |

## Parity

10 general-domain English sentences, PyTorch original vs ONNX, exact string
match against `AutoModelForSeq2SeqLM.generate()`.

```yaml
parity:
  sample_size: 10
  metric: exact_match
  fp32_greedy: 1.00   # 10/10
  fp32_beam4:  1.00   # 10/10
  int8_greedy: 0.80   # 8/10
  int8_beam4:  0.80   # 8/10
```

| Decoding | fp32 (n=10) | int8 (n=10) |
|---|---|---|
| greedy | 100% (10/10) | 80% (8/10) |
| beam=4 | 100% (10/10) | 80% (8/10) |

fp32 is an exact reproduction of the original model. int8 disagreements
were inspected by hand and are paraphrase-level Zulu word variation
(e.g. ref "Izingane zazidlala zijabule ensimini." vs int8 "Izingane
zazidlala ngenjabulo ensimini." - same meaning, different word for
"happily"), not degradations.

**Caveat on sample size:** n=10 exact-match is a small, noisy estimate,
not a FLORES-scale benchmark. A separate measurement campaign on this
project found n=5 exact-match numbers for another model swing from 100%
to 14% when re-measured at n=100 on FLORES-200. Treat the percentages
above as an initial signal only - a proper FLORES + chrF evaluation
supersedes it wherever available.

## Selecting the language

This is an NLLB checkpoint, not M2M100: the target language is **not**
baked into `forced_bos_token_id` in `generation_config.json` - you must
pass it explicitly at generate time using NLLB's `flores_code` language
tags (`eng_Latn`, `zul_Latn`, not plain ISO 639-1 codes):

```python
tokenizer.src_lang = "eng_Latn"
forced_bos_token_id = tokenizer.convert_tokens_to_ids("zul_Latn")
model.generate(**inputs, forced_bos_token_id=forced_bos_token_id, ...)
```

## Usage

```python
from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForSeq2SeqLM

repo = "TigreGotico/nllb_200_distilled_600m-eng-zul-onnx"
tok = AutoTokenizer.from_pretrained(repo)
tok.src_lang = "eng_Latn"
model = ORTModelForSeq2SeqLM.from_pretrained(repo, use_cache=True, use_merged=False)

enc = tok("The weather is very nice today.", return_tensors="pt")
tgt_id = tok.convert_tokens_to_ids("zul_Latn")
out = model.generate(**enc, forced_bos_token_id=tgt_id, num_beams=4, max_new_tokens=64)
print(tok.batch_decode(out, skip_special_tokens=True)[0])
# -> Isimo sezulu sihle kakhulu namuhla.
```

For the int8 build, pass `subfolder="int8"`.