Masterx commited on
Commit
6018579
·
verified ·
1 Parent(s): c177310

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +6 -5
README.md CHANGED
@@ -32,13 +32,14 @@ for use with [WinSTT](https://github.com/dahshury/WinSTT).
32
 
33
  ## Precisions
34
 
35
- - `q4` — 4-bit (`MatMulNBitsQuantizer`, block_size=32, symmetric), ~2.2 GB. Recommended default.
 
36
 
37
- Only q4 is published. The model is CPU-only (its attention kernel is not DirectML-compatible) and the
38
- ONNX Runtime CPU EP up-casts fp16→fp32, so fp16 / q4f16 would be slower and larger with no benefit. The
39
- `export_merged.py` + `quantize.py` scripts here reproduce fp32 or any other precision on demand.
40
 
41
  Verified numerically against the original PyTorch model (encoder/decoder parity ~1e-5 in fp32) and by
42
- end-to-end greedy decoding.
 
43
 
44
  License: Apache-2.0 (inherited from the base model).
 
32
 
33
  ## Precisions
34
 
35
+ - `` (default) fp32. Best accuracy; also the fastest on the CPU EP this model runs on.
36
+ - `q4` — 4-bit (`MatMulNBitsQuantizer`, block_size=32, symmetric), ~2.2 GB.
37
 
38
+ fp16 / q4f16 are not shipped: the model is CPU-only (its attention kernel is not DirectML-compatible)
39
+ and the ONNX Runtime CPU EP up-casts fp16→fp32, so they'd be slower and larger with no benefit.
 
40
 
41
  Verified numerically against the original PyTorch model (encoder/decoder parity ~1e-5 in fp32) and by
42
+ end-to-end greedy decoding. See `export_merged.py` / `quantize.py` to reproduce.
43
+
44
 
45
  License: Apache-2.0 (inherited from the base model).