File size: 2,643 Bytes
c6b160c 63bbcf9 c6b160c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 | ---
license: apache-2.0
language:
- zh
- en
base_model: funasr/paraformer-zh
pipeline_tag: automatic-speech-recognition
tags:
- paraformer
- funasr
- gguf
- quantized
---
# paraformer-zh GGUF
Experimental low-memory GGUF quantizations of
[`funasr/paraformer-zh`](https://huggingface.co/funasr/paraformer-zh).
The vocabulary and CMVN tensors are embedded in each file.
## Files
| File | Size | Quantization | Intended use |
| --- | ---: | --- | --- |
| `paraformer-q5_0.gguf` | 149.7 MiB | Q5_0 matrices, remaining tensors F32 | Recommended low-memory balance |
| `paraformer-q4_0.gguf` | 124.3 MiB | Q4_0 matrices, remaining tensors F32 | Experimental minimum-memory option |
The original exporter only supports F32, F16, and Q8_0. These files were
exported directly from the official FP32 checkpoint with an extended exporter;
they were not requantized from Q8.
## Measured behavior
Measurements used the persistent CPU Paraformer GGUF runtime on an Intel
Core i7-14700KF and a 30-second Mandarin/English mixed-language clip.
| Quantization | Resident peak working set | Recognition time |
| --- | ---: | ---: |
| Q8_0 upstream baseline | 252.8 MiB | 996 ms |
| Q5_0 | 176.5 MiB | 1,201 ms |
| Q4_0 | 151.1 MiB | 1,009 ms |
Across 59 clips from the same recording, Q5_0 and Q4_0 each matched the Q8_0
token sequence exactly on 18 clips. Their aggregate token edit distance from
Q8_0 was 8.8% and 10.1%, respectively. This is output drift relative to Q8,
not a labeled word-error-rate measurement. Validate the quantization with your
own language and acoustic conditions before replacing Q8 in quality-sensitive
deployments.
## Compatibility
These files use the custom `paraformer` GGUF architecture implemented by the
FunASR `runtime/llama.cpp/paraformer` runtime. They are not language models and
cannot be loaded by general-purpose llama.cpp applications.
The compatible runtime is FunASR's `runtime/llama.cpp/paraformer`
implementation. The extended export script used for these files is included
in this repository.
## Source integrity
The export input was the official `model.pt` file:
```text
size: 880502012 bytes
sha256: 5bba782a5e9196166233b9ab12ba04cadff9ef9212b4ff6153ed9290ff679025
```
Exported files:
```text
paraformer-q5_0.gguf
size: 156967168 bytes
sha256: 1f2309eacd761c1f4184177c718321cd6ad3c07e7b17d6f796e5fb15565906ec
paraformer-q4_0.gguf
size: 130313216 bytes
sha256: 992562722aa2c4e88158245a0fc5be0e1338c580db808a97c6262f6864317584
```
## License
The base model and the FunASR exporter/runtime are provided under Apache-2.0.
See `LICENSE` and the base model repository for details.
|