nenad1002/feather-0.2b-de-onnx-int8

Feather 0.2B — German ONNX (INT8)

Feather is Microsoft's newest family of mini speech-recognition models, designed for fast on-device transcription with a tiny CPU footprint. This checkpoint — feather-0.2b-onnx-int8 — is the German-specialised member of the family: a streaming Conformer-Transducer (RNN-T) of about 200 M parameters, quantised to INT8 weight-only and packaged for onnxruntime-genai.

The encoder is a "LiteConformer" — a slimmed-down Conformer variant that diverges from the standard NVIDIA Nemotron Conformer in three ways, specifically chosen to reduce the per-layer op count on CPU:

Block	Standard Nemotron Conformer	Feather LiteConformer encoder
FFN	Macaron: two FFN blocks per layer (feed_forward1 + feed_forward2, each with fc_factor = 0.5)	Single FFN per layer (feed_forward2 only, fc_factor = 1.0) — drops one LayerNorm + two Linear ops per layer
Norm	nn.LayerNorm (weight + bias, mean-subtract + variance-normalise)	nn.RMSNorm (weight-only, no bias, no mean subtraction) — fewer ops + no centring pass
Activation	Swish / SiLU (x * sigmoid(x)) in FFN and convolution	GELU (matches faster ORT fused BiasGelu kernel) in FFN and convolution

Across the 20-layer encoder this removes roughly 40 LayerNorm sub-ops (centring

• bias add) and 40 Linear projections worth of FFN compute compared to the Macaron baseline, while RMSNorm and GELU map directly to fused SkipSimplifiedLayerNormalization / BiasGelu kernels in ONNX Runtime CPU EP.

After export, the encoder is graph-fused with OrtTransformersOptimization (model_type=conformer, MHA fusion disabled for accuracy) and the encoder weights are quantised with MatMulNBitsQuantizer (bits=8, block_size=32, symmetric, k-quant). The decoder and joint networks are kept FP32 — they are tiny and benefit little from quantisation. A bundled Silero VAD is included for optional voice-activity-driven streaming.

Model details

Property	Value
Architecture	Conformer-Transducer (RNN-T), LiteConformer encoder + RNN-T decoder + joint
Parameters	~200 M
Language	German (de)
Sample rate	16 kHz
Mel features	128
Encoder layers / dim / heads	20 / 768 / 12
Predictor (decoder)	2-layer LSTM, hidden 640
Tokenizer	SentencePiece BPE, vocab 8192 (+1 blank)
Streaming chunk	560 ms (chunk_samples=8960 @ 16 kHz)
Left context	70 frames
Subsampling	depthwise-striding, factor 8
Encoder quantisation	INT8 weight-only (MatMulNBits, block 32, symmetric)
Decoder / joint dtype	FP32
Inference runtime	onnxruntime-genai (Microsoft)
File size (encoder)	~358 MB (graph + external data)

Evaluation — German test sets, full splits

All numbers are streaming, greedy-RNNT, CPU, no VAD. Reference normaliser is BasicMultilingualTextNormalizer from huggingface/open_asr_leaderboard.

Dataset	Split	Utts	Hours	WER	RTF (CPU)
FLEURS de_de	test	862	3.15	10.22 %	14.9×
Common Voice 17 de	test	16,206	27.96	10.28 %	12.5×
MLS german	test	3,394	14.29	9.28 %	14.1×
VoxPopuli de	test	1,968	4.92	13.75 %	13.1×

RTF measured single-threaded on a Linux x86-64 CPU host with onnxruntime-genai's streaming Generator. No batching.

Comparison vs NVIDIA Nemotron Multilingual (German)

For reference, the official nvidia/nemotron-asr-streaming-multilingual-0.6b model — same architecture family, ~3× the parameters, covers 40+ languages — evaluated on the same German test splits scores:

Dataset	Nemotron Multilingual (0.6B)	Feather 0.2B (this model)	Δ (Feather − Nemotron)
FLEURS de_de	12.25 %	10.22 %	−2.03
Common Voice 17 de	12.80 %	10.28 %	−2.52
MLS german	9.91 %	9.28 %	−0.63
VoxPopuli de	16.90 %	13.75 %	−3.15

Feather is German-specialised and a third of the size in terms of number of parameters, smaller than int4 quantised version of multilingual Nemotron, and it both transcribes German more accurately and runs noticeably faster on CPU, in our streaming benchmarks Feather achieves ~2× the RTF of Nemotron Multilingual 0.6B ONNX INT4 on the same CPU under identical decoding settings.

Usage (onnxruntime-genai)

pip install onnxruntime-genai soundfile huggingface_hub

from huggingface_hub import snapshot_download
import onnxruntime_genai as og
import soundfile as sf
import numpy as np

model_dir = snapshot_download("nenad1002/feather-0.2b-onnx-int8")

# Load audio (16 kHz mono float32)
audio, sr = sf.read("your_audio.wav", dtype="float32")
if audio.ndim > 1:
    audio = audio.mean(axis=1)
assert sr == 16000

config = og.Config(model_dir)
model = og.Model(config)
tokenizer = og.Tokenizer(model)
params = og.GeneratorParams(model)

processor = og.StreamingProcessor(model)
processor.set_option("use_vad", "false")
generator = og.Generator(model, params)
tok_stream = tokenizer.create_stream()

CHUNK = 8960  # 560 ms @ 16 kHz, matches genai_config.json
text = ""
def drain():
    global text
    while not generator.is_done():
        generator.generate_next_token()
        toks = generator.get_next_tokens()
        if len(toks) > 0:
            t = tok_stream.decode(toks[0])
            if t: text += t

for s in range(0, len(audio), CHUNK):
    inputs = processor.process(audio[s:s+CHUNK].astype(np.float32))
    if inputs is not None:
        generator.set_inputs(inputs); drain()
inputs = processor.flush()
if inputs is not None:
    generator.set_inputs(inputs); drain()

print(text)

Or use the reference script nemotron_speech.py from the onnxruntime-genai repository:

python nemotron_speech.py \
    --model_path /path/to/feather-0.2b-onnx-int8 \
    --audio_file your_audio.wav \
    --use_vad false -e cpu

Files

File	Purpose
encoder.onnx (+ .data)	INT8-quantised LiteConformer encoder
decoder.onnx (+ .data)	FP32 RNN-T predictor
joint.onnx (+ .data)	FP32 RNN-T joiner
silero_vad.onnx	Optional VAD (Silero, ONNX)
genai_config.json	onnxruntime-genai model config
audio_processor_config.json	Audio front-end / mel config
tokenizer.json, tokenizer_config.json, vocab.txt	SentencePiece BPE tokenizer files

Limitations

• German only. Inference on other languages is unsupported and will produce garbled output.
• Streaming chunk = 560 ms. The encoder caches are sized for this latency setting and cannot be changed at inference time.
• CPU-optimised. The encoder is INT8 weight-only quantised and graph-fused for ONNX Runtime CPU EP; GPU EP will run but will not benefit from the optimisations.
• Reference text normalisation. WER numbers above use the open-asr-leaderboard BasicMultilingualTextNormalizer; other normalisers (e.g. Whisper-style) will give different absolute numbers.

License

MIT.

Acknowledgements

• Base architecture: NVIDIA Nemotron Conformer-Transducer family.
• Optimisation stack: Microsoft Olive + ONNX Runtime + onnxruntime-genai.
• Voice activity: Silero VAD.

Author

• Nenad Banfic — CoreAI, Microsoft