Update X-Voice GGUF model card
Browse files
README.md
CHANGED
|
@@ -38,6 +38,23 @@ The quantized files use a conservative X-Voice policy: large GGML `mul_mat`
|
|
| 38 |
matrix weights are converted, while conv, norm, bias, embedding, positional, and
|
| 39 |
small tensors remain in their source type.
|
| 40 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
## Quick Start
|
| 42 |
|
| 43 |
```bash
|
|
|
|
| 38 |
matrix weights are converted, while conv, norm, bias, embedding, positional, and
|
| 39 |
small tensors remain in their source type.
|
| 40 |
|
| 41 |
+
## Quantization Benchmark
|
| 42 |
+
|
| 43 |
+
Single-run local benchmark on an RTX 4060 Ti with `NVIDIA_TF32_OVERRIDE=0`,
|
| 44 |
+
`--preset product`, `cfg_nonlayered`, 32 sampler steps, the default zh sample
|
| 45 |
+
text, and `/root/code/ggbond/models/test.wav`:
|
| 46 |
+
|
| 47 |
+
| model | size | wall | sampler | load | mel max abs vs f32 | mel mean abs vs f32 | wav max abs vs f32 | wav mean abs vs f32 |
|
| 48 |
+
|---|---:|---:|---:|---:|---:|---:|---:|---:|
|
| 49 |
+
| f32 | 1544.1 MiB | 10.74s | 9.222s | 0.515s | 0 | 0 | 0 | 0 |
|
| 50 |
+
| f16 | 1060.5 MiB | 8.89s | 7.316s | 0.513s | 1.40578 | 0.01232 | 0.89212 | 0.01086 |
|
| 51 |
+
| q8_0 | 847.4 MiB | 8.52s | 7.268s | 0.303s | 0.78026 | 0.02397 | 1.01920 | 0.01708 |
|
| 52 |
+
| q6_k | 790.0 MiB | 8.66s | 7.407s | 0.301s | 0.89031 | 0.04256 | 0.68326 | 0.02387 |
|
| 53 |
+
| q4_k | 728.8 MiB | 8.61s | 7.277s | 0.319s | 5.72519 | 0.28411 | 1.22644 | 0.07608 |
|
| 54 |
+
|
| 55 |
+
Interpretation: f16/q8_0/q6_k are the practical release candidates. q4_k is
|
| 56 |
+
size-first and should be auditioned carefully before use as a quality default.
|
| 57 |
+
|
| 58 |
## Quick Start
|
| 59 |
|
| 60 |
```bash
|