opus-mt-az-en-onnx

ONNX export (fp32 + dynamic int8) of Helsinki-NLP/opus-mt-az-en, a Marian (az -> en) translation model from the Helsinki-NLP OPUS-MT project.

License: apache-2.0, inherited unchanged from the base model.

Contents

Files	What it is
encoder_model.onnx, decoder_model.onnx, decoder_with_past_model.onnx	ONNX, float32
int8/	same three graphs, dynamic int8 (QInt8, MatMul only, /lm_head/MatMul excluded)
source.spm, target.spm, vocab.json	MarianTokenizer over the original sentencepiece models

fp32 size: 485 MB (three graphs) | int8 size: 295 MB

Usage

from optimum.onnxruntime import ORTModelForSeq2SeqLM
from transformers import AutoTokenizer

repo = "TigreGotico/opus-mt-az-en-onnx"
tok = AutoTokenizer.from_pretrained(repo)
model = ORTModelForSeq2SeqLM.from_pretrained(repo, use_cache=True, use_merged=False)  # fp32
# int8: ORTModelForSeq2SeqLM.from_pretrained(repo, subfolder="int8", use_cache=True, use_merged=False)
inputs = tok("Bu gün hava çox gözəldir.", return_tensors="pt")
out = model.generate(**inputs, num_beams=4, max_new_tokens=64)
print(tok.decode(out[0], skip_special_tokens=True))

Parity with the original PyTorch model

10 general-domain Azerbaijani sentences (source language for this az->en pair), exact-string-match of generated output against MarianMTModel.generate() on the original Helsinki-NLP/opus-mt-az-en checkpoint.

parity:
  fp32_greedy: 1.00   # 10/10
  fp32_beam4:  1.00   # 10/10
  int8_greedy: 0.70   # 7/10
  int8_beam4:  0.90   # 9/10

Decoding	fp32 exact match	int8 exact match
greedy (num_beams=1)	10/10 (100.0%)	7/10 (70.0%)
beam=4	10/10 (100.0%)	9/10 (90.0%)

fp32 is a faithful reproduction of the original model at both decoding settings. int8 disagreements were inspected by hand, not just counted: they are genuinely different, not worse - e.g. beam=4 on "Bu axşam iki nəfərlik masa sifariş etmək istəyirəm." gives ref "...pay two men a denarii." vs int8 "...pay two men a denunciation." (a near-miss on one rare word), and greedy on "Bu, maşın tərcüməsi üçün test cümləsidir." gives ref "...test for the drive." vs int8 "...test for the translation work." (int8 is arguably closer to the correct meaning there). No degenerate or garbled output was observed in any int8 sample.

An earlier version of this card reported int8 40%/40% from a harness bug - the parity script had reused a fixed English sentence set across all opus-mt pairs, but this model's source language is Azerbaijani, not English, so that measurement compared the model against out-of-distribution input and was discarded. The numbers above are the corrected, re-measured values using real Azerbaijani source sentences.