README.md

---
language:
- sna
license: apache-2.0
library_name: qwen-asr
pipeline_tag: automatic-speech-recognition
tags:
- automatic-speech-recognition
- audio
- speech
- shona
- qwen3-asr
- transformers
base_model: Qwen/Qwen3-ASR-1.7B
datasets:
- manassehzw/sna-dataset-annotated
metrics:
- wer
- cer
model-index:
- name: Shona Qwen3-ASR 1.7B
  results:
  - task:
      type: automatic-speech-recognition
      name: Automatic Speech Recognition
    dataset:
      name: Shona Annotated Validation Probe
      type: manassehzw/sna-dataset-annotated
      split: validation
    metrics:
    - name: Normalized Word Error Rate
      type: wer
      value: 0.2519838056680162
    - name: Normalized Character Error Rate
      type: cer
      value: 0.050787599076953946
---

# manassehzw/sna-qwen-asr-1.7b

Shona Qwen3-ASR 1.7B is a Shona (`sna`) automatic speech recognition model
fine-tuned from [`Qwen/Qwen3-ASR-1.7B`](https://huggingface.co/Qwen/Qwen3-ASR-1.7B)
on [`manassehzw/sna-dataset-annotated`](https://huggingface.co/datasets/manassehzw/sna-dataset-annotated).

## Overview

This release contains checkpoint 760 from run `20260803T140000Z`. It was selected
by generated validation WER and occurs at approximately two epochs. Qwen3-ASR
adapted rapidly to Shona and substantially outperformed its zero-shot behavior on
the project dataset. Evaluation on FLEURS also revealed a meaningful domain and
speaker-generalization gap, so the in-domain score should not be treated as a
universal Shona ASR result.

## Model Details

- **Curated by:** [Manasseh Changachirere (Harare Institute of Technology)](https://www.manasseh.dev/)
- **Base model:** [`Qwen/Qwen3-ASR-1.7B`](https://huggingface.co/Qwen/Qwen3-ASR-1.7B)
- **Base model revision:** `7278e1e70fe206f11671096ffdd38061171dd6e5`
- **Training dataset:** [`manassehzw/sna-dataset-annotated`](https://huggingface.co/datasets/manassehzw/sna-dataset-annotated)
- **Dataset revision:** `f91b1a79cbac15520d3c808b56e5192bf903f280`
- **Training run:** `20260803T140000Z`
- **Released checkpoint:** `checkpoint-760`
- **Checkpoint epoch:** approximately `1.996`
- **Selection criterion:** lowest generated WER among retained checkpoints
- **Framework:** `qwen-asr==0.0.6`, Transformers, PyTorch

## Evaluation

Text for the project evaluations was normalized with Unicode NFKC, case folding,
punctuation and symbol removal, and whitespace collapse.

| Evaluation | Checkpoint | Examples | WER | CER |
|---|---:|---:|---:|---:|
| In-domain validation generation probe | 760 | 256 | **25.20%** | **5.08%** |
| Full in-domain test set | 855 | 1,565 | 25.51% | 4.97% |
| PazaBench v2 / FLEURS `sn_zw` test | 855 | 925 | 52.85% | 13.57% |
| Zero-shot base model, full in-domain test set | base | 1,565 | 106.58% | 31.64% |

The full-test and FLEURS numbers are included as nearby-run context and were
measured on checkpoint 855, not the released checkpoint. Checkpoints 760 and 855
were effectively tied on the 256-example generation probe: 25.198% versus
25.215% WER, a difference of one word error. A full external evaluation of
checkpoint 760 has not yet been recorded.

## Training Summary

- **Training examples:** 12,170
- **Validation examples:** 1,504
- **Held-out test examples:** 1,565
- **Training audio:** approximately 62.63 hours
- **Learning rate:** `2e-5`
- **Microbatch size:** 4
- **Gradient accumulation:** 8
- **Effective batch size:** 32
- **Precision:** bfloat16
- **Scheduler:** linear with 2% warmup
- **Planned epochs:** 3
- **Released checkpoint:** approximately 2 epochs / 760 optimizer steps
- **Checkpoint evaluation loss:** `0.17747`

## Example Usage

Install the official Qwen ASR package:

```bash
pip install -U qwen-asr
```

```python
import torch
from qwen_asr import Qwen3ASRModel

model = Qwen3ASRModel.from_pretrained(
    "manassehzw/sna-qwen-asr-1.7b",
    dtype=torch.bfloat16,
    device_map="cuda:0",
    max_inference_batch_size=16,
    max_new_tokens=256,
)

results = model.transcribe(audio="sample.wav", language=None)
print(results[0].text)
```

The recorded evaluations used `language=None`. Although the fine-tuning targets
identify the output as Shona, Shona was not in the base model's original list of
supported language arguments, so automatic language handling is the tested path.

## Limitations

- External FLEURS performance is substantially weaker than in-domain performance.
- The training corpus is relatively small and may not cover Zimbabwe's full range
  of speakers, accents, recording conditions, and conversational domains.
- Code-switching was not separately quantified for this checkpoint.
- Long-form, streaming, noisy, telephone, and far-field behavior require further
  evaluation.
- Orthographically close substitutions can produce a low CER while still being
  penalized heavily by WER.

## License

The model is released under Apache-2.0, matching the base Qwen3-ASR model.