File size: 4,888 Bytes
e15b122 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 | ---
language:
- sna
license: apache-2.0
library_name: qwen-asr
pipeline_tag: automatic-speech-recognition
tags:
- automatic-speech-recognition
- audio
- speech
- shona
- qwen3-asr
- transformers
base_model: Qwen/Qwen3-ASR-1.7B
datasets:
- manassehzw/sna-dataset-annotated
metrics:
- wer
- cer
model-index:
- name: Shona Qwen3-ASR 1.7B
results:
- task:
type: automatic-speech-recognition
name: Automatic Speech Recognition
dataset:
name: Shona Annotated Validation Probe
type: manassehzw/sna-dataset-annotated
split: validation
metrics:
- name: Normalized Word Error Rate
type: wer
value: 0.2519838056680162
- name: Normalized Character Error Rate
type: cer
value: 0.050787599076953946
---
# manassehzw/sna-qwen-asr-1.7b
Shona Qwen3-ASR 1.7B is a Shona (`sna`) automatic speech recognition model
fine-tuned from [`Qwen/Qwen3-ASR-1.7B`](https://huggingface.co/Qwen/Qwen3-ASR-1.7B)
on [`manassehzw/sna-dataset-annotated`](https://huggingface.co/datasets/manassehzw/sna-dataset-annotated).
## Overview
This release contains checkpoint 760 from run `20260803T140000Z`. It was selected
by generated validation WER and occurs at approximately two epochs. Qwen3-ASR
adapted rapidly to Shona and substantially outperformed its zero-shot behavior on
the project dataset. Evaluation on FLEURS also revealed a meaningful domain and
speaker-generalization gap, so the in-domain score should not be treated as a
universal Shona ASR result.
## Model Details
- **Curated by:** [Manasseh Changachirere (Harare Institute of Technology)](https://www.manasseh.dev/)
- **Base model:** [`Qwen/Qwen3-ASR-1.7B`](https://huggingface.co/Qwen/Qwen3-ASR-1.7B)
- **Base model revision:** `7278e1e70fe206f11671096ffdd38061171dd6e5`
- **Training dataset:** [`manassehzw/sna-dataset-annotated`](https://huggingface.co/datasets/manassehzw/sna-dataset-annotated)
- **Dataset revision:** `f91b1a79cbac15520d3c808b56e5192bf903f280`
- **Training run:** `20260803T140000Z`
- **Released checkpoint:** `checkpoint-760`
- **Checkpoint epoch:** approximately `1.996`
- **Selection criterion:** lowest generated WER among retained checkpoints
- **Framework:** `qwen-asr==0.0.6`, Transformers, PyTorch
## Evaluation
Text for the project evaluations was normalized with Unicode NFKC, case folding,
punctuation and symbol removal, and whitespace collapse.
| Evaluation | Checkpoint | Examples | WER | CER |
|---|---:|---:|---:|---:|
| In-domain validation generation probe | 760 | 256 | **25.20%** | **5.08%** |
| Full in-domain test set | 855 | 1,565 | 25.51% | 4.97% |
| PazaBench v2 / FLEURS `sn_zw` test | 855 | 925 | 52.85% | 13.57% |
| Zero-shot base model, full in-domain test set | base | 1,565 | 106.58% | 31.64% |
The full-test and FLEURS numbers are included as nearby-run context and were
measured on checkpoint 855, not the released checkpoint. Checkpoints 760 and 855
were effectively tied on the 256-example generation probe: 25.198% versus
25.215% WER, a difference of one word error. A full external evaluation of
checkpoint 760 has not yet been recorded.
## Training Summary
- **Training examples:** 12,170
- **Validation examples:** 1,504
- **Held-out test examples:** 1,565
- **Training audio:** approximately 62.63 hours
- **Learning rate:** `2e-5`
- **Microbatch size:** 4
- **Gradient accumulation:** 8
- **Effective batch size:** 32
- **Precision:** bfloat16
- **Scheduler:** linear with 2% warmup
- **Planned epochs:** 3
- **Released checkpoint:** approximately 2 epochs / 760 optimizer steps
- **Checkpoint evaluation loss:** `0.17747`
## Example Usage
Install the official Qwen ASR package:
```bash
pip install -U qwen-asr
```
```python
import torch
from qwen_asr import Qwen3ASRModel
model = Qwen3ASRModel.from_pretrained(
"manassehzw/sna-qwen-asr-1.7b",
dtype=torch.bfloat16,
device_map="cuda:0",
max_inference_batch_size=16,
max_new_tokens=256,
)
results = model.transcribe(audio="sample.wav", language=None)
print(results[0].text)
```
The recorded evaluations used `language=None`. Although the fine-tuning targets
identify the output as Shona, Shona was not in the base model's original list of
supported language arguments, so automatic language handling is the tested path.
## Limitations
- External FLEURS performance is substantially weaker than in-domain performance.
- The training corpus is relatively small and may not cover Zimbabwe's full range
of speakers, accents, recording conditions, and conversational domains.
- Code-switching was not separately quantified for this checkpoint.
- Long-form, streaming, noisy, telephone, and far-field behavior require further
evaluation.
- Orthographically close substitutions can produce a low CER while still being
penalized heavily by WER.
## License
The model is released under Apache-2.0, matching the base Qwen3-ASR model.
|