mohammedaly22/whisper-large-v3-turbo-egyptian-code-switching

Whisper Large V3 Turbo — Egyptian Arabic Code-Switching

Fine-tuned openai/whisper-large-v3-turbo for Egyptian Arabic with Arabic↔English code-switching support.

Model Details

Property	Value
Base model	openai/whisper-large-v3-turbo (809M params)
Fine-tuned on	MohamedRashad/arabic-english-code-switching
Language	Egyptian Arabic + English (code-switching)
Training	Encoder frozen, decoder fine-tuned
Precision	bfloat16

What Does It Do?

The base Whisper model transcribes everything in Arabic script, even English words. This fine-tuned version outputs English words in English when the speaker code-switches.

Example

Audio content: A speaker saying "تعالوا نبني مع بعض برنامج توصيل أكل... من غير ما نكتب ولا سطر code... عن طريق منصة Replit... منزلين عندهم Agent 4"

Model	Output
Base whisper-large-v3-turbo	تعالوا نبني مع بعض برنامج توصيل أكل من غير ما نكتب ولا سطر كود عن طريق منصة ريبليت منزلين عندهم إيجنت فور
This model	تعالوا نبني مع بعض برنامج توصيل أكل من غير ما نكتب ولا سطر code عن طريق منصة Replit منزلين عندهم Agent 4

Usage

Using pipeline (simplest)

from transformers import pipeline, AutoProcessor
import torch

model_id = "mohammedaly22/whisper-large-v3-turbo-egyptian-code-switching"

# Manually load the processor
processor = AutoProcessor.from_pretrained(model_id)

pipe = pipeline(
    "automatic-speech-recognition",
    model=model_id,
    processor=processor,
    torch_dtype=torch.float16, # Better for CUDA
    device="cuda",
    chunk_length_s=30,
)

result = pipe(
    "/path/to/audio.wav", 
    generate_kwargs={"language": "arabic", "task": "transcribe"},
)

print(result["text"])

Using Processor + Model

import torch
import librosa
from transformers import WhisperProcessor, WhisperForConditionalGeneration

model_id = "mohammedaly22/whisper-large-v3-turbo-egyptian-code-switching"

processor = WhisperProcessor.from_pretrained(model_id)
model = WhisperForConditionalGeneration.from_pretrained(model_id, torch_dtype=torch.bfloat16).to("cuda")

# Load audio
audio, sr = librosa.load("path/to/audio.wav", sr=16000)

# Process
input_features = processor.feature_extractor(
    audio, sampling_rate=16000, return_tensors="pt"
).input_features.to(device="cuda", dtype=torch.bfloat16)

# Generate
with torch.no_grad():
    generated_ids = model.generate(
        input_features,
        max_new_tokens=444,
        language="ar",
        task="transcribe",
    )

# Decode
text = processor.tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(text)

Batch Inference

import torch
import librosa
from transformers import WhisperProcessor, WhisperForConditionalGeneration

model_id = "mohammedaly22/whisper-large-v3-turbo-egyptian-code-switching"
processor = WhisperProcessor.from_pretrained(model_id)
model = WhisperForConditionalGeneration.from_pretrained(model_id, torch_dtype=torch.bfloat16).to("cuda")

audio_files = ["audio1.wav", "audio2.wav", "audio3.wav"]
audios = [librosa.load(f, sr=16000)[0] for f in audio_files]

inputs = processor.feature_extractor(
    audios, sampling_rate=16000, return_tensors="pt", padding=True
)
input_features = inputs.input_features.to(device="cuda", dtype=torch.bfloat16)

with torch.no_grad():
    generated_ids = model.generate(
        input_features,
        max_new_tokens=444,
        language="ar",
        task="transcribe",
    )

texts = processor.tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
for t in texts:
    print(t)

Training Details

• Method: Encoder frozen, decoder-only fine-tuning
• Dataset: 12,180 train / 300 eval samples from MohamedRashad/arabic-english-code-switching
• Learning rate: 1e-5 with cosine schedule
• Batch size: 16 (8 × 2 gradient accumulation)
• Epochs: 5
• Precision: bfloat16

Limitations

• Optimized for Egyptian Arabic dialect — may not generalize well to other Arabic dialects
• Code-switching performance is best for common English terms
• Audio should be 16kHz mono for best results
• Maximum audio length: 30 seconds per segment

Citation

If you use this model, please cite the base model and dataset:

@misc{whisper-large-v3-turbo-egyptian-cs,
  title={Whisper Large V3 Turbo — Egyptian Arabic Code-Switching},
  author={Mohammed Aly},
  year={2025},
  url={https://huggingface.co/mohammedaly22/whisper-large-v3-turbo-egyptian-code-switching}
}