Luigi/nemotron-asr-litert-zhtw

Nemotron-3.5-ASR — LiteRT q4-mix, zh-TW fine-tuned (v2)

On-device (Android) LiteRT build of Luigi/nemotron-3.5-asr-streaming-0.6b-zhtw — the zh-TW fine-tuned Nemotron (3.2× better Taiwan Mandarin than base). q4-mix: INT4 encoder + fp16 decoder/joint + fp32 prompt-fusion. Bundle 663 MB.

Supersedes v1 (which packaged the zero-training warm-start).

Accuracy

build	CV zh-TW CER (n=120)
fp32 fine-tuned model (reference)	11.33
this q4-mix LiteRT	13.20 (+1.9 INT4 cost)
v1 q4-mix (base warm-start)	~38

Per language-prompt slot on this build — all within ~0.7 CER, pick either:

slot	q4-mix CER	fp32 CER
zh-CN (4)	13.20	11.57
auto (101)	13.79	11.33
zh-TW (5)	13.90	12.03

auto was the worst slot on the base model (50.58 CER) and is repaired by the fine-tune — training used prompt_mode: unified, which trains the auto path alongside the explicit language ID. Use auto when the speaker may switch languages, or an explicit slot when you know it.

Other languages (fp32 FT, vs base): ko/de/ja/hi/en all improve, ar flat, fr/es +1.5 — see the source model card.

Files

file	role	precision	size
nemotron_encoder_q4.tflite	FastConformer encoder	INT4 FC (blockwise-128) + fp32 convs	596 MB
nemotron_prompt_fuse_fp32.tflite	128-slot language-prompt fusion	fp32	18 MB
nemotron_decoder_fp16.tflite	RNN-T prediction net	fp16	31 MB
nemotron_joint_fp16.tflite	RNN-T joint (→ vocab 13088)	fp16	18 MB
tokenizer.json / processor_config.json / config.json	runtime	—	—

Run

Port, runner and integration note: https://github.com/vieenrose/LiteRT/tree/nemotron/litert/samples/asr/nemotron

python -m nemotron.runner --models ./ --wav clip.wav --lang zh-CN --s2t --itn
# --lang auto also works well (see slot table); --s2t gives Traditional output

Output is Simplified Chinese — the tokenizer cannot represent many common Traditional characters, so apply OpenCC s2t (--s2t) for Traditional.

The INT4 encoder needs the LiteRT-Next CompiledModel runtime (Android NNAPI / XNNPACK-QD8); the classic Interpreter cannot allocate dynamic-range INT4. Set the CPU thread count explicitly (big-core count on big.LITTLE).

Provenance

Base: nvidia/nemotron-3.5-asr-streaming-0.6b (NVIDIA Open Model License), fine-tuned for zh-TW (see source model card), then exported with litert_torch 0.9.1 + ai_edge_quantizer 0.7.0 (INT4 weights pre-baked to the export grid before quantization).