Upload README.md with huggingface_hub

README.md

@@ -9,84 +9,66 @@ tags:
 base_model:
 - openai/whisper-tiny
 ---
-# Whisper-tiny — ExecuTorch XNNPACK (encoder + decoder)
+# Whisper-tiny — ExecuTorch (encoder + decoder)
 
-Speech recognition in two `.pte` files: the encoder runs once per 30-second window,
-the decoder once per generated token.
+Speech recognition in two `.pte` files: the encoder runs once per 30-second window, the
+decoder once per generated token. Putting them in one graph would re-encode the audio on
+every step.
 
-| graph | precision | file | size (MB) | corr vs fp32 eager |
-|-------|-----------|------|-----------|--------------------|
-| encoder | fp32 | `whisper_tiny_encoder_xnnpack_fp32.pte` | 32.9 | 1.000000 |
-| encoder | fp16 | `whisper_tiny_encoder_xnnpack_fp16.pte` | 17.6 | 0.999999 |
-| encoder | int8 | `whisper_tiny_encoder_xnnpack_int8.pte` | 11.7 | 0.999454 |
-| decoder | fp32 | `whisper_tiny_decoder_xnnpack_fp32.pte` | 198.0 | 1.000000 |
-| decoder | fp16 | `whisper_tiny_decoder_xnnpack_fp16.pte` | 99.1 | 0.999988 |
+| graph | build | file | size (MB) | corr vs fp32 eager | ms | eager ms |
+|---|---|---|---|---|---|---|
+| encoder | XNNPACK fp32 | `whisper_tiny_encoder_xnnpack_fp32.pte` | 32.9 | 1.000000 | 60.2 | 22.1 |
+| encoder | XNNPACK fp16 | `whisper_tiny_encoder_xnnpack_fp16.pte` | 17.6 | 1.000000 | 110.5 | 22.8 |
+| encoder | XNNPACK int8 | `whisper_tiny_encoder_xnnpack_int8.pte` | 11.7 | 0.999439 | 58.9 | 22.0 |
+| encoder | Core ML | `whisper_tiny_encoder_coreml_all.pte` | 16.6 | 0.999992 | 13.5 | 22.3 |
+| decoder | XNNPACK fp32 | `whisper_tiny_decoder_xnnpack_fp32.pte` | 198.0 | 1.000000 | 20.6 | 11.9 |
+| decoder | XNNPACK fp16 | `whisper_tiny_decoder_xnnpack_fp16.pte` | 99.1 | 0.999991 | 48.7 | 12.1 |
+| decoder | Core ML | `whisper_tiny_decoder_coreml_all.pte` | 59.3 | 0.999892 | 3.0 | 11.8 |
 
-Every file takes and returns fp32 tensors (token ids stay int64), so any encoder
-pairs with any decoder. The lightest working pair is 110.8 MB.
+Every file takes and returns fp32 tensors (token ids stay int64), so any encoder pairs with
+any decoder. The lightest working pair is 71.0 MB.
 
 - **Source**: [openai/whisper-tiny](https://huggingface.co/openai/whisper-tiny)
 - **License**: Apache-2.0
-- **Encoder input**: log-mel spectrogram `[1,80,3000]` — 30 s at 16 kHz, 80 mel bins,
-  hop 160, window 400. This is exactly what `WhisperFeatureExtractor` produces; pad
-  or trim audio to 30 s as it does.
-- **Encoder output**: `encoder_hidden_states [1,1500,384]`
-- **Decoder input**: the encoder output plus `decoder_input_ids [1,128]` int64,
-  left-aligned and padded. Start the sequence with
-  `<|startoftranscript|>`, a language token, `<|transcribe|>`, `<|notimestamps|>`.
-- **Decoder output**: `logits [1,128,51865]`
+- **Encoder input**: log-mel spectrogram `[1, 80, 3000]` — 30 s at 16 kHz, 80 mel bins, hop
+  160, window 400, exactly what `WhisperFeatureExtractor` produces
+- **Decoder input**: the encoder output plus `decoder_input_ids [1, 128]` int64,
+  left-aligned and padded. Start with `<|startoftranscript|>`, a language token,
+  `<|transcribe|>`, `<|notimestamps|>`.
 
 ## Decoding
 
-There is no KV cache. The decoder is a static graph over a fixed 128-token window,
-so a greedy step is: take `argmax` of row `len-1`, append it, run again. Stop at
-`<|endoftext|>` (50257). 128 tokens covers a 30-second window of ordinary speech
-with room to spare; for longer audio, start a new window.
+No KV cache: the decoder is a static graph over a fixed 128-token window, so a greedy step
+is take `argmax` of row `len-1`, append it, run again. Stop at `<|endoftext|>` (50257). 128
+tokens covers a 30-second window of ordinary speech; past that, start a new window.
 
-That costs a full 128-position forward pass per token. On a 37M-parameter model
-this is cheap enough to be practical, and it keeps the graph static — which is what
-lets the same file run unchanged across runtimes and precisions.
+That costs a full 128-position forward pass per token, which is the price of a static graph
+that runs unchanged across runtimes and precisions.
 
 ## Verification (Mac arm64, executorch 1.4.0, torch 2.13.0)
 
-The two wrappers compose back to `WhisperForConditionalGeneration` exactly
-(max_abs_diff 0.000e+00), and every graph matches torch fp32 eager at the
-correlations in the table above.
-
-Median over 5 runs, Mac arm64 single process — a relative reference, not a device
-number: encoder 54.5 ms (torch eager 20.5 ms), decoder 18.1 ms (eager 12.1 ms).
+The two wrappers compose back to `WhisperForConditionalGeneration` exactly — max_abs_diff
+**0.000e+00** — and every graph matches torch fp32 eager at the correlations above. Timings
+are medians over 5 runs in one process: a relative reference, not a device number.
 
 ## Two things worth knowing about the sizes
 
-**The decoder .pte is larger than the decoder's weights.** Its parameters come to
-118 MB, and the file is 198 MB. Whisper ties `proj_out.weight` to
-`decoder.embed_tokens.weight` — one 19.9M-parameter tensor — but the two uses need
-different representations in the `.pte`: an embedding table the portable kernels
-index into, and the same values packed into the XNNPACK delegate's blob for the
-output matmul. Tying them in PyTorch does not tie them here, and referencing the
-embedding weight directly through `F.linear` does not either.
-
-**The decoder has no int8 build.** PT2E puts an observer on the int64
-`decoder_input_ids` feeding the token embedding, and the lookup then refuses a float
-index (`tensors used as indices must be long, int, byte or bool`). The encoder takes
-float mel input and quantizes without complaint, which is where the size is worth
-taking anyway.
-
-## Conversion
+**The decoder `.pte` is larger than the decoder's weights.** Whisper ties `proj_out.weight`
+to `decoder.embed_tokens.weight`, but the two uses need different representations: an
+embedding table the portable kernels index into, and the same values packed into the XNNPACK
+delegate's blob for the output matmul. Tying them in PyTorch does not tie them here.
+Referencing the weight through `F.linear` instead of the `proj_out` module does not either —
+exported both ways, whisper-tiny's decoder comes out at 198.0 MB exactly.
 
-torch.export → to_edge_transform_and_lower(XnnpackPartitioner) → .pte
-(conversion script: [executorch-models](https://github.com/john-rocky/executorch-models))
 
-The ExecuTorch tree ships a single-graph Whisper example under
-`examples/models/whisper`. This is that model with the halves separated, because a
-combined graph re-encodes the audio on every decoded token.
 
-<!-- funnel:v1 -->
-
----
+## Conversion
 
-**More models in this format:** [ExecuTorch Model Zoo](https://huggingface.co/collections/mlboydaisuke/executorch-model-zoo-6a7ff328390b63075ffeae5e) — 31 models, each with the recipe that produced it.
+```bash
+python convert/export_whisper.py tiny
+```
 
-**Want a different model on-device?** [Open a request](https://github.com/john-rocky/on-device-requests) — free, open weights only; the export and its measured numbers get published publicly.
+The ExecuTorch tree ships a single-graph Whisper example under `examples/models/whisper`;
+this is that model with the halves separated.
 
-<!-- /funnel:v1 -->
+(conversion scripts: [executorch-models](https://github.com/john-rocky/executorch-models))