Configuration Parsing Warning:In adapter_config.json: "peft.task_type" must be a string
Whisper Large V3 Turbo — Polish Medical ASR (LoRA)
LoRA fine-tuned adapter for openai/whisper-large-v3-turbo on Polish medical and general speech.
Performance
| Eval Set | WER | CER |
|---|---|---|
| Fair eval (5 datasets, 3205 samples) | 12.54% | 4.12% |
| Baseline (no fine-tuning) | 17.56% | 5.38% |
5.0pp absolute WER improvement (-28.5% relative) over the base model.
Fair eval includes: admed_anoni (1000), admed_human (1000), bigos (800), youtube (261), gemini (144).
Training Details
- Base model: openai/whisper-large-v3-turbo (809M params)
- Method: LoRA (encoder + decoder), r=64, alpha=128, dropout=0.0
- Trainable params: 49M (5.7% of total)
- Target modules: q_proj, k_proj, v_proj, fc1, fc2
- Learning rate: 2e-4 (cosine decay)
- Epochs: 5
- Batch size: 16
- GPU: NVIDIA A100-SXM4-40GB
- Training data: ~29k samples (admed_anoni 8.5k + admed_human 5.7k + youtube 3.7k + gemini 1.3k + bigos 10k)
Usage
from transformers import WhisperForConditionalGeneration, WhisperProcessor
from peft import PeftModel
processor = WhisperProcessor.from_pretrained("openai/whisper-large-v3-turbo")
model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-large-v3-turbo")
model = PeftModel.from_pretrained(model, "lion-ai/whisper-large-v3-turbo-med-pl-lora")
# Transcribe audio
input_features = processor(audio_array, sampling_rate=16000, return_tensors="pt").input_features
predicted_ids = model.generate(input_features, language="pl", task="transcribe")
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
- Downloads last month
- -