---
license: apache-2.0
base_model: Helsinki-NLP/opus-mt-az-en
base_model_relation: quantized
tags:
  - onnx
  - translation
  - marian
library_name: transformers
pipeline_tag: translation
language:
  - az
  - en
---

# opus-mt-az-en-onnx

ONNX export (fp32 + dynamic int8) of [`Helsinki-NLP/opus-mt-az-en`](https://huggingface.co/Helsinki-NLP/opus-mt-az-en),
a Marian (az -> en) translation model from the Helsinki-NLP OPUS-MT project.

**License:** apache-2.0, inherited unchanged from the base model.

## Contents

| Files | What it is |
|---|---|
| `encoder_model.onnx`, `decoder_model.onnx`, `decoder_with_past_model.onnx` | ONNX, float32 |
| `int8/` | same three graphs, dynamic int8 (`QInt8`, `MatMul` only, `/lm_head/MatMul` excluded) |
| `source.spm`, `target.spm`, `vocab.json` | `MarianTokenizer` over the original sentencepiece models |

fp32 size: 485 MB (three graphs) | int8 size: 295 MB

## Usage

```python
from optimum.onnxruntime import ORTModelForSeq2SeqLM
from transformers import AutoTokenizer

repo = "TigreGotico/opus-mt-az-en-onnx"
tok = AutoTokenizer.from_pretrained(repo)
model = ORTModelForSeq2SeqLM.from_pretrained(repo, use_cache=True, use_merged=False)  # fp32
# int8: ORTModelForSeq2SeqLM.from_pretrained(repo, subfolder="int8", use_cache=True, use_merged=False)
inputs = tok("Bu gün hava çox gözəldir.", return_tensors="pt")
out = model.generate(**inputs, num_beams=4, max_new_tokens=64)
print(tok.decode(out[0], skip_special_tokens=True))
```

## Parity with the original PyTorch model

10 general-domain **Azerbaijani** sentences (source language for this az->en
pair), exact-string-match of generated output against `MarianMTModel.generate()`
on the original `Helsinki-NLP/opus-mt-az-en` checkpoint.

```yaml
parity:
  fp32_greedy: 1.00   # 10/10
  fp32_beam4:  1.00   # 10/10
  int8_greedy: 0.70   # 7/10
  int8_beam4:  0.90   # 9/10
```

| Decoding | fp32 exact match | int8 exact match |
|---|---|---|
| greedy (num_beams=1) | 10/10 (100.0%) | 7/10 (70.0%) |
| beam=4 | 10/10 (100.0%) | 9/10 (90.0%) |

fp32 is a faithful reproduction of the original model at both decoding
settings. int8 disagreements were inspected by hand, not just counted: they
are genuinely **different, not worse** - e.g. beam=4 on "Bu axşam iki
nəfərlik masa sifariş etmək istəyirəm." gives ref "...pay two men a
denarii." vs int8 "...pay two men a denunciation." (a near-miss on one rare
word), and greedy on "Bu, maşın tərcüməsi üçün test cümləsidir." gives ref
"...test for the drive." vs int8 "...test for the translation work." (int8
is arguably closer to the correct meaning there). No degenerate or garbled
output was observed in any int8 sample.

An earlier version of this card reported int8 40%/40% from a harness bug -
the parity script had reused a fixed **English** sentence set across all
opus-mt pairs, but this model's source language is Azerbaijani, not
English, so that measurement compared the model against out-of-distribution
input and was discarded. The numbers above are the corrected, re-measured
values using real Azerbaijani source sentences.