v2: q4-mix LiteRT of the zh-TW FINE-TUNED model (CV zh-TW CER 13.90 vs ~38 for v1 warm-start; fp32 ref 12.03)

README.md

@@ -2,69 +2,62 @@
 license: other
 license_name: nvidia-open-model-license
 license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
-base_model: nvidia/nemotron-3.5-asr-streaming-0.6b
-language:
-  - zh
-  - en
-  - ja
-  - ko
-tags:
-  - litert
-  - tflite
-  - automatic-speech-recognition
-  - streaming-asr
-  - FastConformer
-  - RNNT
-  - zh-tw
-  - traditional-chinese
+base_model: Luigi/nemotron-3.5-asr-streaming-0.6b-zhtw
+language: [zh, en, de, fr, es, ar, ja, ko]
+tags: [litert, tflite, automatic-speech-recognition, streaming-asr, FastConformer, RNNT, zh-tw, q4]
 pipeline_tag: automatic-speech-recognition
 library_name: litert
 ---
 
-# Nemotron-3.5-ASR-Streaming-0.6B — LiteRT q4-mix (zh-TW variant)
+# Nemotron-3.5-ASR — LiteRT q4-mix, zh-TW fine-tuned (v2)
 
-zh-TW-enabled variant of
-[Luigi/nemotron-asr-litert](https://huggingface.co/Luigi/nemotron-asr-litert)
-(the faithful q4-mix port). The base model's **zh-TW** language slot (5) is
-untrained (100% CER); this variant **warm-starts it from the working zh-CN slot**
-— a lossless copy of the zh-CN column of the prompt projector into the zh-TW
-column, baked into `nemotron_prompt_fuse_fp32.tflite`. Nothing else changes
-(encoder / decoder / joint are byte-identical to the base repo).
+On-device (Android) LiteRT build of
+[Luigi/nemotron-3.5-asr-streaming-0.6b-zhtw](https://huggingface.co/Luigi/nemotron-3.5-asr-streaming-0.6b-zhtw)
+— the zh-TW **fine-tuned** Nemotron (3.2× better Taiwan Mandarin than base).
+**q4-mix**: INT4 encoder + fp16 decoder/joint + fp32 prompt-fusion. Bundle **663 MB**.
 
-**Result:** the `zh-TW` slot now works natively at **15.78% CER end-to-end** (on
-par with zh-CN), up from 100% (dead). Output is Simplified Chinese; add OpenCC
-`s2t` for Traditional.
+Supersedes v1 (which packaged the zero-training warm-start).
 
-> This is v1.1 — a zero-training warm-start. A future v2 fine-tune will add native
-> Traditional output + far-field/meeting-domain robustness (IVOD).
+## Accuracy
 
-## Files
+| build | CV zh-TW CER |
+|---|---|
+| fp32 fine-tuned model (reference) | 12.03 |
+| **this q4-mix LiteRT** | **13.90** (+1.87 INT4 cost) |
+| v1 q4-mix (base warm-start) | ~38 |
+
+Other languages (fp32 FT, vs base): ko/de/ja/hi/en all improve, ar flat,
+fr/es +1.5 — see the [source model card](https://huggingface.co/Luigi/nemotron-3.5-asr-streaming-0.6b-zhtw).
 
-Identical layout to the base repo; only `nemotron_prompt_fuse_fp32.tflite` differs.
+## Files
 
 | file | role | precision | size |
 |---|---|---|---|
-| `nemotron_encoder_q4.tflite` | FastConformer encoder | INT4 FC (QAT) + fp32 convs | 569 MB |
-| `nemotron_prompt_fuse_fp32.tflite` | 128-slot prompt fusion (**zh-TW warm-started**) | fp32 | 17 MB |
-| `nemotron_decoder_fp16.tflite` | RNN-T prediction net | fp16 | 29 MB |
-| `nemotron_joint_fp16.tflite` | RNN-T joint (→ vocab 13088) | fp16 | 17 MB |
+| `nemotron_encoder_q4.tflite` | FastConformer encoder | INT4 FC (blockwise-128) + fp32 convs | 596 MB |
+| `nemotron_prompt_fuse_fp32.tflite` | 128-slot language-prompt fusion | fp32 | 18 MB |
+| `nemotron_decoder_fp16.tflite` | RNN-T prediction net | fp16 | 31 MB |
+| `nemotron_joint_fp16.tflite` | RNN-T joint (→ vocab 13088) | fp16 | 18 MB |
 | `tokenizer.json` / `processor_config.json` / `config.json` | runtime | — | — |
 
-## Usage
+## Run
 
-Same 4-graph RNN-T greedy pipeline as the base repo (see the
-[port + runner](https://github.com/vieenrose/LiteRT/tree/nemotron/litert/samples/asr/nemotron)):
+Port, runner and integration note:
+https://github.com/vieenrose/LiteRT/tree/nemotron/litert/samples/asr/nemotron
 
 ```bash
-python -m nemotron.runner --models ./ --wav clip.wav --lang zh-TW --s2t
+python -m nemotron.runner --models ./ --wav clip.wav --lang zh-TW --s2t --itn
 ```
 
+**Output is Simplified Chinese** — the tokenizer cannot represent many common
+Traditional characters, so apply OpenCC `s2t` (`--s2t`) for Traditional.
+
 The INT4 encoder needs the LiteRT-Next `CompiledModel` runtime (Android NNAPI /
 XNNPACK-QD8); the classic `Interpreter` cannot allocate dynamic-range INT4.
+Set the CPU thread count explicitly (big-core count on big.LITTLE).
 
 ## Provenance
 
-Base: nvidia/nemotron-3.5-asr-streaming-0.6b (NVIDIA Open Model License). The INT4
-encoder is QAT'd (label-based RNN-T self-distillation); the zh-TW slot is warm-
-started from zh-CN (weight copy, no training). Everything else is
-numerics-preserving conversion.
+Base: nvidia/nemotron-3.5-asr-streaming-0.6b (NVIDIA Open Model License),
+fine-tuned for zh-TW (see source model card), then exported with
+`litert_torch` 0.9.1 + `ai_edge_quantizer` 0.7.0 (INT4 weights pre-baked to the
+export grid before quantization).

config.json

@@ -4,6 +4,7 @@
   ],
   "blank_token_id": 13087,
   "decoder_hidden_size": 640,
+  "default_prompt_id": 101,
   "dtype": "float32",
   "durations": [],
   "encoder_config": {
@@ -15,6 +16,7 @@
     "default_num_lookahead_tokens": 3,
     "dropout": 0.1,
     "dropout_positions": 0.0,
+    "dtype": "float32",
     "hidden_act": "silu",
     "hidden_size": 1024,
     "initializer_range": 0.02,
@@ -48,7 +50,6 @@
   "num_prompts": 128,
   "pad_token_id": 0,
   "prompt_intermediate_size": 2048,
-  "transformers_version": "5.13.0.dev0",
-  "vocab_size": 13088,
-  "default_prompt_id": 101
-}
+  "transformers_version": "5.14.1",
+  "vocab_size": 13088
+}

nemotron_decoder_fp16.tflite

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:93a3b8d91ab604305cb5c3014c4fb389cfd7b6a7397dc2e6e22df459f3f8859f
+oid sha256:e92dfa900ebd9d7cd87429c9bb7c304b7e3fa61dc233c74f2e074fbb4342222b
 size 30733936

nemotron_encoder_q4.tflite

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9e817d29ab20013de9962a8c347e7f68f9a896eef1e29ffcf9b0e0a0f1ef691c
+oid sha256:b1b3c93add91ee2253c8d6d24172614a83f6572720dea0150fb34285be53a0c2
 size 596222224

nemotron_joint_fp16.tflite

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8e5f46279a288ff5c0ce11f581fcd3753308c2151559046c90ca62aa632d48f2
+oid sha256:d728fb09aa034b85b1549772fef6cfc4f85d7df0faf59c6db4ad2e7fbbfdc848
 size 18121056

nemotron_prompt_fuse_fp32.tflite

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b4dcdc802c4124b0b0b0ff5b56344705f9b15f10c9f4983aa3503176b2e9e9b9
+oid sha256:21c59326f8633c3824f9e92dcaded6148978dcd53591846c85c9b1ac982a1bba
 size 17841444