---
language:
- ca
- it
license: apache-2.0
tags:
- translation
- onnx
- marian
base_model: Helsinki-NLP/opus-mt-ca-it
---

# opus-mt-ca-it-onnx

ONNX export of [Helsinki-NLP/opus-mt-ca-it](https://huggingface.co/Helsinki-NLP/opus-mt-ca-it), for use with `optimum.onnxruntime.ORTModelForSeq2SeqLM`.

## Licence

Apache-2.0, taken from the base model card ([Helsinki-NLP/opus-mt-ca-it](https://huggingface.co/Helsinki-NLP/opus-mt-ca-it)).

## Export command

```
optimum-cli export onnx --model Helsinki-NLP/opus-mt-ca-it --task text2text-generation-with-past --no-post-process ca-it
```

int8 dynamic quantization was then applied to each ONNX file with `onnxruntime.quantization.quantize_dynamic` (`QUInt8`).

## File layout

```
encoder_model.onnx
decoder_model.onnx
decoder_with_past_model.onnx
config.json / generation_config.json
source.spm / target.spm / vocab.json / tokenizer_config.json / special_tokens_map.json
int8/
  encoder_model.onnx
  decoder_model.onnx
  decoder_with_past_model.onnx
```

## Sanity check

3 short sentences, PyTorch original vs this ONNX export, `num_beams=4, max_new_tokens=48`:

| input | PyTorch | ONNX fp32 | ONNX int8 |
|---|---|---|---|
| Hola, com estas? | Ehi, come stai? | Ehi, come stai? | Ehi, come stai? |
| El gos corre al parc. | Il cane va al parco. | Il cane va al parco. | Il cane va al parco. |
| Bon dia a tothom. | Buongiorno a tutti. | Buongiorno a tutti. | Buongiorno a tutti. |

fp32 matches PyTorch on: 3/3. int8 output recorded above (not gated).

## Usage

```python
from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForSeq2SeqLM

model_id = "TigreGotico/opus-mt-ca-it-onnx"
tok = AutoTokenizer.from_pretrained(model_id)
model = ORTModelForSeq2SeqLM.from_pretrained(model_id)

inputs = tok("Hola, com estas?", return_tensors="pt")
out = model.generate(**inputs, num_beams=4, max_new_tokens=48)
print(tok.decode(out[0], skip_special_tokens=True))
```

For the int8 quantized version, load from the `int8/` subfolder:

```python
model = ORTModelForSeq2SeqLM.from_pretrained(model_id, subfolder="int8")
```