lopatnov/nllb-200-distilled-600M-onnx

NLLB Model 600M for Lopatnov.Translate

This repository contains an ONNX-converted version of the NLLB model, optimized for the Lopatnov.Translate service. These files are converted from the original weights in the facebook/nllb-200-distilled-600M repository.

Conversion recipe

conda create -n model_export python=3.10 -y
conda activate model_export
pip install "optimum[onnxruntime,export,openvino]" transformers sentencepiece
optimum-cli export onnx --model facebook/nllb-200-distilled-600M --task seq2seq-lm --dtype fp32 --opset 18 ./model/nllb-200-distilled-600M-onnx

Configuration

To configure model for the service, please modify appsettings.json:

Example configuration:

{
  "Models": {
    "nllb-600M": {
      "Type": "NLLB",
      "Path": "../../models/translate/nllb-200-distilled-600M",
      "EncoderFile": "encoder_model.onnx",
      "DecoderFile": "decoder_model.onnx",
      "TokenizerFile": "sentencepiece.bpe.model",
      "TokenizerConfigFile": "tokenizer.json",
      "MaxTokens": 512,
      "BeamSize": 1
    }
  },
  // These configurations are optional
  "Translation": {
    "AllowedModels": [ "nllb-600M" ], // Whitelist allowed models if needed
    "DefaultModel": "nllb-600M", // Set this model as the default
    "WarmUp": [ "nllb-600M" ] // Load the model into memory on startup
  }
}

Note: Make sure the Path correctly points to the location of your downloaded .onnx files relative to application.

Documentation

For advanced configuration options, visit the Official Documentation.