manassehzw/sna-qwen-3-asr-1.7b

manassehzw/sna-qwen-asr-1.7b

Shona Qwen3-ASR 1.7B is a Shona (sna) automatic speech recognition model fine-tuned from Qwen/Qwen3-ASR-1.7B on manassehzw/sna-dataset-annotated.

Overview

This release contains checkpoint 760 from run 20260803T140000Z. It was selected by generated validation WER and occurs at approximately two epochs. Qwen3-ASR adapted rapidly to Shona and substantially outperformed its zero-shot behavior on the project dataset. Evaluation on FLEURS also revealed a meaningful domain and speaker-generalization gap, so the in-domain score should not be treated as a universal Shona ASR result.

Model Details

• Curated by: Manasseh Changachirere (Harare Institute of Technology)
• Base model: Qwen/Qwen3-ASR-1.7B
• Base model revision: 7278e1e70fe206f11671096ffdd38061171dd6e5
• Training dataset: manassehzw/sna-dataset-annotated
• Dataset revision: f91b1a79cbac15520d3c808b56e5192bf903f280
• Training run: 20260803T140000Z
• Released checkpoint: checkpoint-760
• Checkpoint epoch: approximately 1.996
• Selection criterion: lowest generated WER among retained checkpoints
• Framework: qwen-asr==0.0.6, Transformers, PyTorch

Evaluation

Text for the project evaluations was normalized with Unicode NFKC, case folding, punctuation and symbol removal, and whitespace collapse.

Evaluation	Checkpoint	Examples	WER	CER
In-domain validation generation probe	760	256	25.20%	5.08%
Full in-domain test set	855	1,565	25.51%	4.97%
PazaBench v2 / FLEURS sn_zw test	855	925	52.85%	13.57%
Zero-shot base model, full in-domain test set	base	1,565	106.58%	31.64%

The full-test and FLEURS numbers are included as nearby-run context and were measured on checkpoint 855, not the released checkpoint. Checkpoints 760 and 855 were effectively tied on the 256-example generation probe: 25.198% versus 25.215% WER, a difference of one word error. A full external evaluation of checkpoint 760 has not yet been recorded.

Training Summary

• Training examples: 12,170
• Validation examples: 1,504
• Held-out test examples: 1,565
• Training audio: approximately 62.63 hours
• Learning rate: 2e-5
• Microbatch size: 4
• Gradient accumulation: 8
• Effective batch size: 32
• Precision: bfloat16
• Scheduler: linear with 2% warmup
• Planned epochs: 3
• Released checkpoint: approximately 2 epochs / 760 optimizer steps
• Checkpoint evaluation loss: 0.17747

Example Usage

Install the official Qwen ASR package:

pip install -U qwen-asr

import torch
from qwen_asr import Qwen3ASRModel

model = Qwen3ASRModel.from_pretrained(
    "manassehzw/sna-qwen-asr-1.7b",
    dtype=torch.bfloat16,
    device_map="cuda:0",
    max_inference_batch_size=16,
    max_new_tokens=256,
)

results = model.transcribe(audio="sample.wav", language=None)
print(results[0].text)

The recorded evaluations used language=None. Although the fine-tuning targets identify the output as Shona, Shona was not in the base model's original list of supported language arguments, so automatic language handling is the tested path.

Limitations

• External FLEURS performance is substantially weaker than in-domain performance.
• The training corpus is relatively small and may not cover Zimbabwe's full range of speakers, accents, recording conditions, and conversational domains.
• Code-switching was not separately quantified for this checkpoint.
• Long-form, streaming, noisy, telephone, and far-field behavior require further evaluation.
• Orthographically close substitutions can produce a low CER while still being penalized heavily by WER.

License

The model is released under Apache-2.0, matching the base Qwen3-ASR model.