Add Nemotron-3.5 ASR streaming multilingual bundle + model card

.gitattributes

@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+decoder.onnx.data filter=lfs diff=lfs merge=lfs -text
+encoder.onnx.data filter=lfs diff=lfs merge=lfs -text
+joint.onnx.data filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,136 @@
+---
+license: other
+license_name: nvidia-open-model-license
+license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
+language:
+- en
+- es
+- zh
+- hi
+- ar
+- fr
+- de
+- ja
+- ru
+- pt
+- ko
+- it
+- nl
+- pl
+- tr
+- uk
+- ro
+- cs
+- hu
+- sv
+- da
+- fi
+- th
+- vi
+- id
+tags:
+- automatic-speech-recognition
+- streaming-asr
+- cache-aware
+- multilingual
+- FastConformer
+- RNNT
+- onnx
+base_model: nvidia/nemotron-3.5-asr-streaming-0.6b
+library_name: onnxruntime
+pipeline_tag: automatic-speech-recognition
+---
+
+# Nemotron-3.5-ASR-Streaming-Multilingual-0.6B — ONNX (FP16)
+
+Cache-aware **streaming** multilingual speech recognition. A 0.6 B FastConformer-RNNT
+encoder with a 128-slot **language prompt** (one slot per language/locale), exported to
+ONNX in **FP16**. FP16 is lossless versus the FP32 source (identical WER/CER) at half the
+download size, and is hardware-accelerated on GPU / NPU / Android-NNAPI.
+
+- **Architecture**: cache-aware FastConformer encoder (24 layers, 1024 hidden, 8× subsampling) + RNN-T decoder/joint
+- **Streaming**: 320 ms chunk, 240 ms lookahead, left attention context 56, right context 3
+- **Languages**: 100+ via the prompt dictionary (`languages.json`); benchmarked on 6 below
+- **Audio**: 16 kHz mono, 128-bin log-mel front end
+
+## Model
+
+| | |
+|---|---|
+| Parameters | ~0.6 B |
+| Format | ONNX (external-data weights) |
+| Precision | FP16 |
+| Bundle size | ~1.25 GB |
+| Sample rate | 16 kHz mono |
+| Chunk / lookahead | 320 ms / 240 ms |
+
+## Files
+
+| File | Size | Description |
+|---|---|---|
+| `encoder.onnx` + `encoder.onnx.data` | ~1.18 GB | Cache-aware FastConformer encoder (FP16) |
+| `decoder.onnx` + `decoder.onnx.data` | ~30 MB | RNN-T prediction network |
+| `joint.onnx` + `joint.onnx.data` | ~19 MB | RNN-T joint network |
+| `config.json` | <1 KB | Model + streaming config (mel, chunk, cache sizes) |
+| `languages.json` | ~2 KB | Locale → prompt-slot dictionary (128 slots) |
+| `vocab.json` | ~230 KB | 13 087-token BPE vocabulary |
+
+## Performance
+
+FLEURS test, 320 ms streaming, **CPU**, n=30 per language. FP16 is bit-equivalent to FP32.
+For Japanese, **CER** is the meaningful metric (no word boundaries).
+
+| Language | WER % | CER % |
+|---|---|---|
+| English (en-US) | 9.92 | 5.65 |
+| German (de-DE) | 12.68 | 7.40 |
+| French (fr-FR) | 15.93 | 6.02 |
+| Arabic (ar-EG) | 14.02 | 3.74 |
+| Hindi (hi-IN) | 7.37 | 4.46 |
+| Japanese (ja-JP) | — | 16.28 |
+
+Resource profile (8.4 s utterance, ONNX Runtime CPU): encoder ~87 ms/chunk (RTF ~0.27),
+peak RSS ~3.4 GB. Note: ONNX Runtime up-converts FP16→FP32 on CPU (no native FP16 CPU
+kernels), so FP16's runtime wins are realized on GPU / NPU / NNAPI — on CPU its benefit is
+the smaller download. For lowest CPU latency/RAM, use the INT8 build.
+
+## Usage
+
+```python
+import onnxruntime as ort
+
+so = ort.SessionOptions()
+enc = ort.InferenceSession("encoder.onnx", so, providers=["CPUExecutionProvider"])
+dec = ort.InferenceSession("decoder.onnx", so, providers=["CPUExecutionProvider"])
+joint = ort.InferenceSession("joint.onnx", so, providers=["CPUExecutionProvider"])
+
+# Pick the language prompt slot from languages.json, e.g. "en-US" -> 0, "ja-JP" -> 10.
+# Front end: 128-bin log-mel (n_fft=512, win=400, hop=160, preemph=0.97), 16 kHz mono.
+# Streaming contract (per chunk): feed 320 ms of audio + the carried encoder caches
+# (attention / conv / pre-cache), then run the RNN-T greedy loop over the 4 emitted frames.
+# Port wiring for the encoder caches is described by config.json's "streaming" block.
+```
+
+Production streaming, cache management and RNN-T greedy decoding are handled by the
+**[speech-android](https://github.com/soniqo/speech-android)** SDK.
+
+## Source
+
+Converted from **[nvidia/nemotron-3.5-asr-streaming-0.6b](https://huggingface.co/nvidia/nemotron-3.5-asr-streaming-0.6b)**
+(NVIDIA NeMo). Licensed under the
+[NVIDIA Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/).
+
+## Related models
+
+| Variant | Repo |
+|---|---|
+| ONNX · FP16 (this) | `soniqo/Nemotron-3.5-ASR-Streaming-Multilingual-0.6B-ONNX-FP16` |
+| ONNX · INT8 | [soniqo/…-ONNX-INT8](https://huggingface.co/soniqo/Nemotron-3.5-ASR-Streaming-Multilingual-0.6B-ONNX-INT8) |
+| LiteRT · FP16 | [soniqo/…-LiteRT-FP16](https://huggingface.co/soniqo/Nemotron-3.5-ASR-Streaming-Multilingual-0.6B-LiteRT-FP16) |
+| LiteRT · INT8 | [soniqo/…-LiteRT-INT8](https://huggingface.co/soniqo/Nemotron-3.5-ASR-Streaming-Multilingual-0.6B-LiteRT-INT8) |
+
+## Links
+
+- [speech-android](https://github.com/soniqo/speech-android) — Android SDK
+- [soniqo.audio](https://soniqo.audio) — website
+- [blog](https://soniqo.audio/blog) — blog

config.json

@@ -0,0 +1,28 @@
+{
+  "modelName": "nemotron-3.5-asr-streaming-multilingual-0.6b",
+  "numMelBins": 128,
+  "sampleRate": 16000,
+  "nFFT": 512,
+  "hopLength": 160,
+  "winLength": 400,
+  "preEmphasis": 0.97,
+  "encoderHidden": 1024,
+  "encoderLayers": 24,
+  "subsamplingFactor": 8,
+  "attentionLeftContext": 56,
+  "convCacheSize": 8,
+  "decoderHidden": 640,
+  "decoderLayers": 2,
+  "vocabSize": 13087,
+  "blankTokenId": 13087,
+  "numPrompts": 128,
+  "streaming": {
+    "chunkMs": 320,
+    "chunkSize": 4,
+    "rightContext": 3,
+    "lookaheadMs": 240,
+    "melFrames": 32,
+    "preCacheSize": 9,
+    "outputFrames": 4
+  }
+}

decoder.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:965f7cb2fa4b04364d1562c76e9d756e7f6c50548e0436c6c379cb4189bf06fb
+size 7040

decoder.onnx.data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cafdb046f5dc849be2c0183dd3da17befa3f3a4dc5cead867793c23a69f5b929
+size 29880320

encoder.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:851c598c39729c0925b4d2f066821b1283676ed307d9f38c7d0174b6d90330e8
+size 22130711

encoder.onnx.data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4e0ae7c84854c5d322f9a168283e47c4f2da603a5b1f4ce402cad66f939e4b92
+size 1236396032

joint.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a4ae1a699ab5aee9e85499c51907ec58714806c54638c7b5357510d7dded60db
+size 3207

joint.onnx.data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35eb3b1c090cc72a451f7250b7ee0e50573cebb3bb4efd9cf90ae6c6d0374d92
+size 18911296

languages.json

@@ -0,0 +1,127 @@
+{
+  "numPrompts": 128,
+  "promptDictionary": {
+    "en-US": 0,
+    "en": 0,
+    "en-GB": 1,
+    "enGB": 1,
+    "es-ES": 2,
+    "esES": 2,
+    "es-US": 3,
+    "es": 3,
+    "zh-CN": 4,
+    "zh-ZH": 4,
+    "zh-TW": 5,
+    "hi-IN": 6,
+    "hi": 6,
+    "hi-HI": 6,
+    "ar-AR": 7,
+    "ar": 7,
+    "fr-FR": 8,
+    "fr": 8,
+    "de-DE": 9,
+    "de": 9,
+    "ja-JP": 10,
+    "ja-JA": 10,
+    "ru-RU": 11,
+    "ru": 11,
+    "pt-BR": 12,
+    "pt-PT": 13,
+    "pt": 13,
+    "ko-KR": 14,
+    "ko": 14,
+    "ko-KO": 14,
+    "it-IT": 15,
+    "it": 15,
+    "nl-NL": 16,
+    "nl": 16,
+    "pl-PL": 17,
+    "pl": 17,
+    "tr-TR": 18,
+    "tr": 18,
+    "uk-UA": 19,
+    "uk": 19,
+    "ro-RO": 20,
+    "ro": 20,
+    "el-GR": 21,
+    "el": 21,
+    "cs-CZ": 22,
+    "cs": 22,
+    "hu-HU": 23,
+    "hu": 23,
+    "sv-SE": 24,
+    "sv": 24,
+    "da-DK": 25,
+    "da": 25,
+    "fi-FI": 26,
+    "fi": 26,
+    "no-NO": 27,
+    "no": 27,
+    "nb-NO": 103,
+    "nb": 103,
+    "nn-NO": 104,
+    "nn": 104,
+    "sk-SK": 28,
+    "sk": 28,
+    "hr-HR": 29,
+    "hr": 29,
+    "bg-BG": 30,
+    "bg": 30,
+    "lt-LT": 31,
+    "lt": 31,
+    "et-EE": 60,
+    "et": 60,
+    "lv-LV": 61,
+    "lv": 61,
+    "sl-SI": 62,
+    "sl": 62,
+    "th-TH": 32,
+    "vi-VN": 33,
+    "id-ID": 34,
+    "ms-MY": 35,
+    "bn-IN": 36,
+    "ur-PK": 37,
+    "fa-IR": 38,
+    "ta-IN": 39,
+    "te-IN": 40,
+    "mr-IN": 41,
+    "gu-IN": 42,
+    "kn-IN": 43,
+    "ml-IN": 44,
+    "si-LK": 45,
+    "ne-NP": 46,
+    "km-KH": 47,
+    "sw-KE": 48,
+    "am-ET": 49,
+    "ha-NG": 50,
+    "zu-ZA": 51,
+    "yo-NG": 52,
+    "ig-NG": 53,
+    "af-ZA": 54,
+    "rw-RW": 55,
+    "so-SO": 56,
+    "ny-MW": 57,
+    "ln-CD": 58,
+    "or-KE": 59,
+    "he-IL": 64,
+    "ku-TR": 65,
+    "az-AZ": 66,
+    "ka-GE": 67,
+    "hy-AM": 68,
+    "uz-UZ": 69,
+    "tg-TJ": 70,
+    "ky-KG": 71,
+    "qu-PE": 80,
+    "ay-BO": 81,
+    "gn-PY": 82,
+    "nah-MX": 83,
+    "mi-NZ": 96,
+    "haw-US": 97,
+    "sm-WS": 98,
+    "to-TO": 99,
+    "fr-CA": 100,
+    "mt-MT": 102,
+    "auto": 101
+  },
+  "autoSlot": 101
+}