---
license: apache-2.0
tags:
- executorch
- xnnpack
- pte
- on-device
- automatic-speech-recognition
base_model:
- openai/whisper-tiny
---
# Whisper-tiny — ExecuTorch (encoder + decoder)

Speech recognition in two `.pte` files: the encoder runs once per 30-second window, the
decoder once per generated token. Putting them in one graph would re-encode the audio on
every step.

| graph | build | file | size (MB) | corr vs fp32 eager | ms | eager ms |
|---|---|---|---|---|---|---|
| encoder | XNNPACK fp32 | `whisper_tiny_encoder_xnnpack_fp32.pte` | 32.9 | 1.000000 | 60.2 | 22.1 |
| encoder | XNNPACK fp16 | `whisper_tiny_encoder_xnnpack_fp16.pte` | 17.6 | 1.000000 | 110.5 | 22.8 |
| encoder | XNNPACK int8 | `whisper_tiny_encoder_xnnpack_int8.pte` | 11.7 | 0.999439 | 58.9 | 22.0 |
| encoder | Core ML | `whisper_tiny_encoder_coreml_all.pte` | 16.6 | 0.999992 | 13.5 | 22.3 |
| decoder | XNNPACK fp32 | `whisper_tiny_decoder_xnnpack_fp32.pte` | 198.0 | 1.000000 | 20.6 | 11.9 |
| decoder | XNNPACK fp16 | `whisper_tiny_decoder_xnnpack_fp16.pte` | 99.1 | 0.999991 | 48.7 | 12.1 |
| decoder | Core ML | `whisper_tiny_decoder_coreml_all.pte` | 59.3 | 0.999892 | 3.0 | 11.8 |

Every file takes and returns fp32 tensors (token ids stay int64), so any encoder pairs with
any decoder. The lightest working pair is 71.0 MB.

- **Source**: [openai/whisper-tiny](https://huggingface.co/openai/whisper-tiny)
- **License**: Apache-2.0
- **Encoder input**: log-mel spectrogram `[1, 80, 3000]` — 30 s at 16 kHz, 80 mel bins, hop
  160, window 400, exactly what `WhisperFeatureExtractor` produces
- **Decoder input**: the encoder output plus `decoder_input_ids [1, 128]` int64,
  left-aligned and padded. Start with `<|startoftranscript|>`, a language token,
  `<|transcribe|>`, `<|notimestamps|>`.

## Decoding

No KV cache: the decoder is a static graph over a fixed 128-token window, so a greedy step
is take `argmax` of row `len-1`, append it, run again. Stop at `<|endoftext|>` (50257). 128
tokens covers a 30-second window of ordinary speech; past that, start a new window.

That costs a full 128-position forward pass per token, which is the price of a static graph
that runs unchanged across runtimes and precisions.

## Verification (Mac arm64, executorch 1.4.0, torch 2.13.0)

The two wrappers compose back to `WhisperForConditionalGeneration` exactly — max_abs_diff
**0.000e+00** — and every graph matches torch fp32 eager at the correlations above. Timings
are medians over 5 runs in one process: a relative reference, not a device number.

## Two things worth knowing about the sizes

**The decoder `.pte` is larger than the decoder's weights.** Whisper ties `proj_out.weight`
to `decoder.embed_tokens.weight`, but the two uses need different representations: an
embedding table the portable kernels index into, and the same values packed into the XNNPACK
delegate's blob for the output matmul. Tying them in PyTorch does not tie them here.
Referencing the weight through `F.linear` instead of the `proj_out` module does not either —
exported both ways, whisper-tiny's decoder comes out at 198.0 MB exactly.



## Conversion

```bash
python convert/export_whisper.py tiny
```

The ExecuTorch tree ships a single-graph Whisper example under `examples/models/whisper`;
this is that model with the halves separated.

(conversion scripts: [executorch-models](https://github.com/john-rocky/executorch-models))