onnx-community/feather-0.2b-de-onnx-int8

Feather 0.2B — German ONNX (INT8)

Feather is Microsoft's newest family of mini speech-recognition models, designed for fast on-device transcription with a tiny CPU footprint. This checkpoint — feather-0.2b-onnx-int8 — is the German-specialised member of the family: a streaming Conformer-Transducer (RNN-T) of about 200 M parameters, quantised to INT8 weight-only and packaged for onnxruntime-genai.

The encoder is a "LiteConformer" — a slimmed-down Conformer variant that diverges from the standard NVIDIA Nemotron Conformer in three ways, specifically chosen to reduce the per-layer op count on CPU:

Block	Standard Nemotron Conformer	Feather LiteConformer encoder
FFN	Macaron: two FFN blocks per layer (feed_forward1 + feed_forward2, each with fc_factor = 0.5)	Single FFN per layer (feed_forward2 only, fc_factor = 1.0) — drops one LayerNorm + two Linear ops per layer
Norm	nn.LayerNorm (weight + bias, mean-subtract + variance-normalise)	nn.RMSNorm (weight-only, no bias, no mean subtraction) — fewer ops + no centring pass
Activation	Swish / SiLU (x * sigmoid(x)) in FFN and convolution	GELU (matches faster ORT fused BiasGelu kernel) in FFN and convolution

Across the 20-layer encoder this removes roughly 40 LayerNorm sub-ops (centring

• bias add) and 40 Linear projections worth of FFN compute compared to the Macaron baseline, while RMSNorm and GELU map directly to fused SkipSimplifiedLayerNormalization / BiasGelu kernels in ONNX Runtime CPU EP.

After export, the encoder is graph-fused with OrtTransformersOptimization (model_type=conformer, MHA fusion disabled for accuracy) and the encoder weights are quantised with MatMulNBitsQuantizer (bits=8, block_size=32, symmetric, k-quant). The decoder and joint networks are kept FP32 — they are tiny and benefit little from quantisation. A bundled Silero VAD is included for optional voice-activity-driven streaming.

Model details

Property	Value
Architecture	Conformer-Transducer (RNN-T), LiteConformer encoder + RNN-T decoder + joint
Parameters	~200 M
Language	German (de)
Sample rate	16 kHz
Mel features	128
Encoder layers / dim / heads	20 / 768 / 12
Predictor (decoder)	2-layer LSTM, hidden 640
Tokenizer	SentencePiece BPE, vocab 8192 (+1 blank)
Streaming chunk	560 ms (chunk_samples=8960 @ 16 kHz)
Left context	70 frames
Subsampling	depthwise-striding, factor 8
Encoder quantisation	INT8 weight-only (MatMulNBits, block 32, symmetric)
Decoder / joint dtype	FP32
Inference runtime	onnxruntime-genai (Microsoft)
File size (encoder)	~358 MB (graph + external data)

Evaluation — German test sets, full splits

All numbers are streaming, greedy-RNNT, CPU, no VAD. Reference normaliser is BasicMultilingualTextNormalizer from huggingface/open_asr_leaderboard.

Dataset	Split	Utts	Hours	WER	RTF (CPU)
FLEURS de_de	test	862	3.15	10.22 %	14.9×
Common Voice 17 de	test	16,206	27.96	10.28 %	12.5×
MLS german	test	3,394	14.29	9.28 %	14.1×
VoxPopuli de	test	1,968	4.92	13.75 %	13.1×

RTF measured single-threaded on a Linux x86-64 CPU host with onnxruntime-genai's streaming Generator. No batching.

Comparison vs NVIDIA Nemotron Multilingual (German)

For reference, the official nvidia/nemotron-asr-streaming-multilingual-0.6b model — same architecture family, ~3× the parameters, covers 40+ languages — evaluated on the same German test splits scores:

Dataset	Nemotron Multilingual (0.6B) ONNX INT4	Feather 0.2B (this model)	Δ (Feather − Nemotron)
FLEURS de_de	12.25 %	10.22 %	−2.03
Common Voice 17 de	12.80 %	10.28 %	−2.52
MLS german	9.91 %	9.28 %	−0.63
VoxPopuli de	16.90 %	13.75 %	−3.15

Feather is German-specialised and a third of the size in terms of number of parameters, so it both transcribes German more accurately and runs noticeably faster on CPU — in our streaming benchmarks Feather achieves ~2× the RTF of Nemotron Multilingual 0.6B ONNX INT4 on the same CPU under identical decoding settings.

Usage (onnxruntime-genai)

pip install onnxruntime-genai soundfile huggingface_hub

from huggingface_hub import snapshot_download
import onnxruntime_genai as og
import soundfile as sf
import numpy as np

model_dir = snapshot_download("nenad1002/feather-0.2b-onnx-int8")

# Load audio (16 kHz mono float32)
audio, sr = sf.read("your_audio.wav", dtype="float32")
if audio.ndim > 1:
    audio = audio.mean(axis=1)
assert sr == 16000

config = og.Config(model_dir)
model = og.Model(config)
tokenizer = og.Tokenizer(model)
params = og.GeneratorParams(model)

processor = og.StreamingProcessor(model)
processor.set_option("use_vad", "false")
generator = og.Generator(model, params)
tok_stream = tokenizer.create_stream()

CHUNK = 8960  # 560 ms @ 16 kHz, matches genai_config.json
text = ""
def drain():
    global text
    while not generator.is_done():
        generator.generate_next_token()
        toks = generator.get_next_tokens()
        if len(toks) > 0:
            t = tok_stream.decode(toks[0])
            if t: text += t

for s in range(0, len(audio), CHUNK):
    inputs = processor.process(audio[s:s+CHUNK].astype(np.float32))
    if inputs is not None:
        generator.set_inputs(inputs); drain()
inputs = processor.flush()
if inputs is not None:
    generator.set_inputs(inputs); drain()

print(text)

Or use the reference script nemotron_speech.py from the onnxruntime-genai repository:

python nemotron_speech.py \
    --model_path /path/to/feather-0.2b-onnx-int8 \
    --audio_file your_audio.wav \
    --use_vad false -e cpu

Files

File	Purpose
encoder.onnx (+ .data)	INT8-quantised LiteConformer encoder
decoder.onnx (+ .data)	FP32 RNN-T predictor
joint.onnx (+ .data)	FP32 RNN-T joiner
silero_vad.onnx	Optional VAD (Silero, ONNX)
genai_config.json	onnxruntime-genai model config
audio_processor_config.json	Audio front-end / mel config
tokenizer.json, tokenizer_config.json, vocab.txt	SentencePiece BPE tokenizer files

Training data

Feather 0.2B German was trained on a nearly 10,000-hour German speech mixture drawn from several complementary sources:

Source	Role in the mixture
Multilingual LibriSpeech (MLS)	Read audiobook-style German speech
Common Voice	Crowd-sourced German speech with diverse speakers and recording conditions
VoxPopuli	Parliamentary / broadcast-style German speech
CML-TTS	Synthetic German speech
YODAS / Granary	Large-scale German speech data

This mixture was selected to improve robustness across domains while keeping the model specialised for German-only streaming ASR.

Responsible AI / demographic evaluation

The model was evaluated on Common Voice 17 German using available demographic metadata across gender, age, and accent/dialect. The overall WER on the full Common Voice German test split is 10.28%.

Gender

Performance remains relatively consistent across gender groups. Male speakers achieve 12.75% WER, female speakers achieve 13.89% WER, and samples with unknown gender metadata achieve 10.06% WER.

While the male and female subsets show moderately higher WER than the overall average, these subsets represent a substantially smaller portion of the evaluation set compared to samples with unknown metadata. This limits the statistical strength of subgroup comparisons. No severe gender-specific degradation is observed.

Age

Age-group WER ranges from 10.64% to 15.04%. The fifties group performs best at 10.64% WER, while the forties group shows the highest WER at 15.04% WER. Most age groups cluster between roughly 11% and 14% WER, indicating generally stable performance across age demographics despite uneven subgroup sizes.

Accent / dialect

The model demonstrates robust performance across major German-speaking regional variants. Standard German (Deutschland Deutsch) achieves 9.09% WER, Austrian German achieves 10.22% WER, and Swiss German achieves 10.99% WER.

The spread between the evaluated accent groups remains below 2 percentage points, indicating good generalisation across the evaluated German dialectal varieties.

Important caveat

The majority of evaluation samples contain unknown demographic metadata, which is inherent to the Common Voice dataset and limits the statistical power of this subgroup analysis. Some demographic subsets, particularly female speakers and certain age or accent categories, contain relatively small sample counts and should therefore be interpreted cautiously.

Summary

The model demonstrates stable performance across the evaluated demographic dimensions and German-speaking regional accents. No subgroup with meaningful representation exhibits catastrophic degradation, and the results indicate reasonable robustness for a low-latency streaming German ASR system.

Limitations

• German only. Inference on other languages is unsupported and will produce garbled output.
• Streaming chunk = 560 ms. The encoder caches are sized for this latency setting and cannot be changed at inference time.
• CPU-optimised. The encoder is INT8 weight-only quantised and graph-fused for ONNX Runtime CPU EP; GPU EP will run but will not benefit from the optimisations.
• Inconsistent capitalization and punctuation. The model might not always transcribe words with correct capitalization or output appropriate punctuation.

License

MIT.

Acknowledgements

• Base architecture: NVIDIA Nemotron Conformer-Transducer family.
• Optimisation stack: Microsoft Olive + ONNX Runtime + onnxruntime-genai.
• Voice activity: Silero VAD.

Author

• Nenad Banfic — CoreAI, Microsoft
• For any issues or questions, open a thread or send an email to nebanfic@microsoft.com