Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -32,13 +32,14 @@ for use with [WinSTT](https://github.com/dahshury/WinSTT).
|
|
| 32 |
|
| 33 |
## Precisions
|
| 34 |
|
| 35 |
-
- `
|
|
|
|
| 36 |
|
| 37 |
-
|
| 38 |
-
ONNX Runtime CPU EP up-casts fp16→fp32, so
|
| 39 |
-
`export_merged.py` + `quantize.py` scripts here reproduce fp32 or any other precision on demand.
|
| 40 |
|
| 41 |
Verified numerically against the original PyTorch model (encoder/decoder parity ~1e-5 in fp32) and by
|
| 42 |
-
end-to-end greedy decoding.
|
|
|
|
| 43 |
|
| 44 |
License: Apache-2.0 (inherited from the base model).
|
|
|
|
| 32 |
|
| 33 |
## Precisions
|
| 34 |
|
| 35 |
+
- `` (default) — fp32. Best accuracy; also the fastest on the CPU EP this model runs on.
|
| 36 |
+
- `q4` — 4-bit (`MatMulNBitsQuantizer`, block_size=32, symmetric), ~2.2 GB.
|
| 37 |
|
| 38 |
+
fp16 / q4f16 are not shipped: the model is CPU-only (its attention kernel is not DirectML-compatible)
|
| 39 |
+
and the ONNX Runtime CPU EP up-casts fp16→fp32, so they'd be slower and larger with no benefit.
|
|
|
|
| 40 |
|
| 41 |
Verified numerically against the original PyTorch model (encoder/decoder parity ~1e-5 in fp32) and by
|
| 42 |
+
end-to-end greedy decoding. See `export_merged.py` / `quantize.py` to reproduce.
|
| 43 |
+
|
| 44 |
|
| 45 |
License: Apache-2.0 (inherited from the base model).
|