katrintomanek commited on
Commit
808a31c
·
verified ·
1 Parent(s): e6b46e1

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +18 -8
README.md CHANGED
@@ -1,5 +1,5 @@
1
  ---
2
- language: sw
3
  library_name: transformers
4
  pipeline_tag: automatic-speech-recognition
5
  tags:
@@ -12,27 +12,36 @@ metrics:
12
  - wer
13
  ---
14
 
15
- This is a fine-tuned version of [**`openai/whisper-large-v3`**](https://huggingface.co/openai/whisper-large-v3) for `Ugandan Luganda` (standard speech). It is part of CDLI's effort to make speech technology work for people whose speech is underserved by mainstream ASR systems.
16
 
17
  All CDLI models and datasets can be found on [**CDLI's HuggingFace page**](https://huggingface.co/cdli).
18
 
19
- ## Evaluation
 
 
20
 
21
- This model was evaluated on the **`test`** split of [`google/WaxalNLP`](https://huggingface.co/datasets/google/WaxalNLP), a speech dataset for African Languages. The `lug_asr` subset of the dataset was used for fine-tuning and evaluation. Examples longer than 30 seconds were excluded.
22
- For decoding we ran Whisper with `language=sw`, `task=transcribe`, greedy search (`num_beams=1`, `do_sample=False`).
23
 
24
- Note: We fine-tuned the Whisper model for the Swahili (sw) language as out of all languages supported by Whisper, this is the language most similar to Luganda.
25
 
 
 
 
 
 
26
  - **Examples evaluated:** 503
27
  - **Speakers:** 194
28
 
29
- ### Metrics
 
 
 
30
 
31
  We report two complementary word error rate (WER) metrics, both computed on text
32
  normalized with Whisper's `BasicTextNormalizer`:
33
 
34
  - **Standard (corpus-level) WER** — the usual error rate, pooling all reference
35
- words and edit errors across the entire test set.
36
  - **Per-utterance averaged WER** — WER computed *separately for each utterance*,
37
  each capped at 1.0, then averaged across utterances.
38
 
@@ -41,6 +50,7 @@ The per-utterance averaged WER bounds each utterance to
41
  without a few catastrophic utterances dominating — but it is not a true error
42
  rate and isn't directly comparable to other published WER, hence we report the standard, corpus-level WER as well.
43
 
 
44
 
45
  ### Overall Results
46
 
 
1
  ---
2
+ language: lg
3
  library_name: transformers
4
  pipeline_tag: automatic-speech-recognition
5
  tags:
 
12
  - wer
13
  ---
14
 
15
+ This is a fine-tuned version of [**`openai/whisper-large-v3`**](https://huggingface.co/openai/whisper-large-v3) for `Luganda` non-standard speech. It is part of CDLI's effort to make speech technology work for people whose speech is underserved by mainstream ASR systems.
16
 
17
  All CDLI models and datasets can be found on [**CDLI's HuggingFace page**](https://huggingface.co/cdli).
18
 
19
+ ## Dataset
20
+
21
+ The model has been fine-tuned using [`google/WaxalNLP`](https://huggingface.co/datasets/google/WaxalNLP), a dataset of speech samples of people living with impaired speech across a range of impairment **severity levels** and **etiologies**. The `lug_asr` subset of the dataset was used.
22
 
23
+ ## Training
 
24
 
25
+ The *train* split was used for training, and the *dev* split for selecting the best checkpoint. **Note:** This Whisper model was fine-tuned and is decoded using the **Swahili** (`sw`) language setting out of all languages Whisper supports, the one most similar to Luganda.
26
 
27
+ All model parameters (encoder, decoder, and output projection) were fine-tuned, with SpecAugment enabled.
28
+
29
+ ## Evaluation
30
+
31
+ This model was evaluated on the **`test`** split of the dataset. Utterances longer than 30 seconds were excluded:
32
  - **Examples evaluated:** 503
33
  - **Speakers:** 194
34
 
35
+ For decoding we ran Whisper with `language=sw`, `task=transcribe`,
36
+ greedy search (`num_beams=1`, `do_sample=False`).
37
+
38
+
39
 
40
  We report two complementary word error rate (WER) metrics, both computed on text
41
  normalized with Whisper's `BasicTextNormalizer`:
42
 
43
  - **Standard (corpus-level) WER** — the usual error rate, pooling all reference
44
+ words and edit errors across the entire test set.
45
  - **Per-utterance averaged WER** — WER computed *separately for each utterance*,
46
  each capped at 1.0, then averaged across utterances.
47
 
 
50
  without a few catastrophic utterances dominating — but it is not a true error
51
  rate and isn't directly comparable to other published WER, hence we report the standard, corpus-level WER as well.
52
 
53
+ ## Results
54
 
55
  ### Overall Results
56