Slamford/mtg-voice-whisper

MTG Voice — fine-tuned Whisper (base.en) for Magic card names

A LoRA fine-tune of openai/whisper-base.en that recognizes spoken Magic: The Gathering card names — the fantasy proper nouns ("Atraxa", "Korvold", "Tergrid") that stock Whisper mangles. Exported to GGML for whisper.cpp, so it runs fully offline and real-time on-device (it's the recognizer in the MTG Voice app, on desktop and Android).

⚠️ Unofficial fan project. Not affiliated with, endorsed, or sponsored by Wizards of the Coast. Magic: The Gathering is © Wizards of the Coast. This model only recognizes spoken card names — it contains no card text, art, or other game content.

Why

Stock Whisper handles English well but fumbles fantasy names. Fine-tuning a small (base.en) model on card names lets the fast, mobile-friendly model handle the hard proper nouns — accuracy without the latency of a bigger model.

Results (held-out benchmark)

50 cards across difficulty tiers, synthesized with a held-out TTS voice (one not used in training), then run through the app's fuzzy/phonetic matcher:

Model	Overall	fantasy	legendary	relative speed
stock base.en	60%	6/12	3/12	fast
stock small.en	58%	5/12	2/12	~3× slower
this model (base.en, fine-tuned)	78%	8/12	6/12	fast

+18 points over stock base.en on an unseen voice (real generalization, not memorization), beats the larger small.en by +20 at ~3× the speed, and doubles both stock models on legendary names.

Usage (whisper.cpp)

Load ggml-base.en-mtg.bin like any ggml Whisper model — 16 kHz mono audio in, English-only:

./whisper-cli -m ggml-base.en-mtg.bin -f audio.wav

Or from code (whisper.cpp, whisper-jni, or any binding): point whisper_init_from_file at the .bin. It pairs best with a fuzzy/phonetic name matcher downstream to recover near-misses.

Files

• ggml-base.en-mtg.bin — f16, ~141 MB. Recommended (fastest on modern fp16-capable CPUs/phones).
• A q5_1 quantized build (~57 MB) is smaller but slightly slower on flagship hardware, so f16 is the default.

Training

• Method: LoRA (r=16) on the attention projections of whisper-base.en; adapter merged back into the base weights and converted to GGML.
• Data: ~8.7k Magic card names (every legendary + every possessive + a broad random sample), each spoken isolated and in natural carrier phrases ("I'll play X", "casting X"), synthesized with Piper neural TTS across several speakers. On-the-fly audio augmentation (noise / gain / pitch / time-stretch) at train time, so it learns card-name phonetics, not TTS timbre.
• Eval honesty: the benchmark above uses a Piper voice held out from training (train ≠ test).

Limitations

• English only (base.en).
• Trained on synthetic (TTS) speech; real-world accuracy on live mics varies — the companion app pairs it with a fuzzy/phonetic matcher that recovers near-misses.
• Recognizes card names, not rules text.

Attribution & license

• Base model: openai/whisper-base.en (MIT).
• Training audio: Piper TTS voices.
• This fine-tune is released under the MIT License.