README.md

metadata

language:
  - ms
  - en
  - zh
  - ta
datasets:
  - mesolitica/Malaysian-STT-Whisper
  - malaysia-ai/STT-Whisper
base_model:
  - openai/whisper-large-v3-turbo

Malaysian Finetune Whisper Large V3 Turbo

Finetune Whisper Large V3 Turbo on Malaysian context.

Improvement

1. Distilled from Whisper Large V3 on Malaysian and Science context.
2. Better translation for Malay, Manglish, Mandarin, Tamil and Science context.
3. Word level timestamp, introduced <|transcribeprecise|> token, a new task!
4. Better filler words, introduced <|transcribenonverbal|> token, a new task!

how we finetuned it?

We done 2 phases,

1. Finetune on mesolitica/Malaysian-STT-Whisper

• Revision 267552e0f093068519a816112c2741939d057f48
• WanDB at https://wandb.ai/huseinzol05/malaysian-whisper-large-v3-turbo-v3

2. Stage 2, Annealing on 5% from mesolitica/Malaysian-STT-Whisper and 100% from mesolitica/Malaysian-STT-Whisper-Stage2

• Revision 5f6ca0596f01527ec2d013662cfc168d5f754461
• WanDB at https://wandb.ai/huseinzol05/malaysian-whisper-large-v3-turbo-v3-stage2

3. Stage 3, Annealing on 5% from mesolitica/Malaysian-STT-Whisper and 10% from mesolitica/Malaysian-STT-Whisper-Stage2 and 100% on AMI only mesolitica/Speech-Nonverbal-Whisper

• WanDB at https://wandb.ai/huseinzol05/malaysian-whisper-large-v3-turbo-v3-stage3