cantonese-asr-lab/whisper-small-cantonese-w500-adaptive

Whisper-small Cantonese W500 Adaptive

This is the exact model weight and inference configuration used by W500_Adaptive_RAW_WINNER, which scored 69.49 in a Cantonese ASR competition evaluation.

The model keeps the original Whisper-small architecture and tokenizer. It was adapted with a source-alternating curriculum:

• WenetSpeech-Yue steps freeze the Decoder and output projection and update only the Encoder, expanding acoustic, accent, speaker, and recording-domain coverage.
• Official short-utterance steps unfreeze the full model, continually correcting Cantonese orthography, insertions, repetition, and EOS behavior.

The released checkpoint is the raw 50-hour Wenet milestone selected using the fixed validation set and stability guardrails.

Competition result

The platform evaluated a hidden 200-item sample.

Metric	Value
Final score	69.49
CER	0.2473053892
Sentence accuracy (edit-distance tolerance 2)	0.3400

These numbers are specific to that competition protocol and are not a general Cantonese ASR benchmark.

Local diagnostics

The validation, Public, and OOD sets below were used for local selection and diagnostics; Public and OOD did not rerank checkpoints.

Split	Samples	CER	tol2	tol1	exact
Fixed validation	702	0.085934	0.854701	0.709402	0.454416
Public diagnostic	1,900	0.081328	0.894211	0.751579	0.450000
OOD panel	2,000	0.340429	0.331500	0.159000	0.042000

Reproducibility hashes

• Original flat submission ZIP SHA-256: b4fda8ac549d37d8cac9797950636d50ca28f74d8e5e76e2479c15de9b956bc5
• model.safetensors SHA-256: a0f29a5a011213d5e4de34c40a02d42247255645f06e649e092d2dc495094370
• config.json SHA-256: 53b4eb5c1c63510e9541417174df6ed709cd59e0b052492446ca1287088ee023
• generation_config.json SHA-256: 3f2ced827b5a4b0241c4f2f8883cf13da2b00ece23c341d663333c9b07b8de64
• predict.py SHA-256: 075d465a775f4ebff1f817d86ab16d3f8da0977257cdcab1870312aaa9c78546

The model was copied from the scored package without re-saving or converting the weights. Raw-checkpoint versus extracted-package predictions and token IDs matched on a fixed 32-item offline subset.

Offline inference

Download the repository first because the bundled entry point intentionally loads all model assets with local_files_only=True:

hf download Vanxun-Hank/whisper-small-cantonese-w500-adaptive \
  --local-dir whisper-small-cantonese-w500-adaptive

python whisper-small-cantonese-w500-adaptive/predict.py \
  --model_dir whisper-small-cantonese-w500-adaptive \
  --audio_dir /path/to/audio \
  --test_list /path/to/test.jsonl \
  --output_jsonl predictions.jsonl

Input JSONL rows require an audio_path field. Output rows contain audio_path and pred_text. Use --diagnostics-jsonl to additionally save generated token IDs without changing the official prediction output.

The standalone defaults used by the scored entry point are language=zh, task=transcribe, num_beams=1, and max_length=225. The bundled generation configuration also supplies no_repeat_ngram_size=4 and repetition_penalty=1.05.

Source code

Training, evaluation, checkpoint selection, inference, package verification, and a detailed method description are available at Vanxun-Hank/cantonese-asr.

Limitations

• The model is optimized for Cantonese transcription under a competition-specific short-utterance protocol.
• It uses Whisper's Chinese language token because the original tokenizer does not expose a dedicated Cantonese language token.
• Performance can vary by writing convention, domain, microphone, noise, speaker population, and utterance length.
• Training data is not included in this repository. Users must follow the upstream dataset terms and applicable privacy requirements.