cjpais commited on
Commit
d69eae3
·
verified ·
1 Parent(s): b16638d

Re-quantize K-presets with Q8_0 fallback (was F16)

Browse files

Updated quantize policy: tensors with ne0 not divisible by 256 now use Q8_0 fallback instead of F16. K-preset GGUFs are 5-7% smaller. WER unchanged within bootstrap CI; full re-score pending on bench machine.

README.md CHANGED
@@ -39,7 +39,7 @@ Ported from upstream commit
39
  [76b8b23](https://huggingface.co/CohereLabs/cohere-transcribe-03-2026/commit/76b8b23),
40
  pinned 2026-04-16.
41
  Validated against the Transformers reference at transcribe.cpp commit
42
- [929d922](https://github.com/handy-computer/transcribe.cpp/tree/929d922)
43
  on 2026-04-18.
44
 
45
  Offline multilingual speech-to-text covering 14 languages (English, French,
@@ -56,9 +56,9 @@ mono WAV and produces a transcript. Decoding is autoregressive.
56
  | BF16 | [cohere-transcribe-03-2026-BF16.gguf](https://huggingface.co/handy-computer/cohere-transcribe-03-2026-gguf/resolve/main/cohere-transcribe-03-2026-BF16.gguf) | 4.10 GB | 1.26% |
57
  | F16 | [cohere-transcribe-03-2026-F16.gguf](https://huggingface.co/handy-computer/cohere-transcribe-03-2026-gguf/resolve/main/cohere-transcribe-03-2026-F16.gguf) | 4.11 GB | 1.26% |
58
  | Q8_0 | [cohere-transcribe-03-2026-Q8_0.gguf](https://huggingface.co/handy-computer/cohere-transcribe-03-2026-gguf/resolve/main/cohere-transcribe-03-2026-Q8_0.gguf) | 2.41 GB | 1.27% |
59
- | Q6_K | [cohere-transcribe-03-2026-Q6_K.gguf](https://huggingface.co/handy-computer/cohere-transcribe-03-2026-gguf/resolve/main/cohere-transcribe-03-2026-Q6_K.gguf) | 1.97 GB | 1.27% |
60
- | Q5_K_M | [cohere-transcribe-03-2026-Q5_K_M.gguf](https://huggingface.co/handy-computer/cohere-transcribe-03-2026-gguf/resolve/main/cohere-transcribe-03-2026-Q5_K_M.gguf) | 1.76 GB | 1.25% |
61
- | Q4_K_M | [cohere-transcribe-03-2026-Q4_K_M.gguf](https://huggingface.co/handy-computer/cohere-transcribe-03-2026-gguf/resolve/main/cohere-transcribe-03-2026-Q4_K_M.gguf) | 1.55 GB | 1.25% |
62
 
63
  WER measured on the full LibriSpeech test-clean split (2620 utterances) with
64
  greedy decoding and no external LM. BF16 reference baseline: 1.26%.
 
39
  [76b8b23](https://huggingface.co/CohereLabs/cohere-transcribe-03-2026/commit/76b8b23),
40
  pinned 2026-04-16.
41
  Validated against the Transformers reference at transcribe.cpp commit
42
+ [bf0d0b7](https://github.com/handy-computer/transcribe.cpp/tree/bf0d0b7)
43
  on 2026-04-18.
44
 
45
  Offline multilingual speech-to-text covering 14 languages (English, French,
 
56
  | BF16 | [cohere-transcribe-03-2026-BF16.gguf](https://huggingface.co/handy-computer/cohere-transcribe-03-2026-gguf/resolve/main/cohere-transcribe-03-2026-BF16.gguf) | 4.10 GB | 1.26% |
57
  | F16 | [cohere-transcribe-03-2026-F16.gguf](https://huggingface.co/handy-computer/cohere-transcribe-03-2026-gguf/resolve/main/cohere-transcribe-03-2026-F16.gguf) | 4.11 GB | 1.26% |
58
  | Q8_0 | [cohere-transcribe-03-2026-Q8_0.gguf](https://huggingface.co/handy-computer/cohere-transcribe-03-2026-gguf/resolve/main/cohere-transcribe-03-2026-Q8_0.gguf) | 2.41 GB | 1.27% |
59
+ | Q6_K | [cohere-transcribe-03-2026-Q6_K.gguf](https://huggingface.co/handy-computer/cohere-transcribe-03-2026-gguf/resolve/main/cohere-transcribe-03-2026-Q6_K.gguf) | 1.84 GB | 1.27% |
60
+ | Q5_K_M | [cohere-transcribe-03-2026-Q5_K_M.gguf](https://huggingface.co/handy-computer/cohere-transcribe-03-2026-gguf/resolve/main/cohere-transcribe-03-2026-Q5_K_M.gguf) | 1.65 GB | 1.25% |
61
+ | Q4_K_M | [cohere-transcribe-03-2026-Q4_K_M.gguf](https://huggingface.co/handy-computer/cohere-transcribe-03-2026-gguf/resolve/main/cohere-transcribe-03-2026-Q4_K_M.gguf) | 1.45 GB | 1.25% |
62
 
63
  WER measured on the full LibriSpeech test-clean split (2620 utterances) with
64
  greedy decoding and no external LM. BF16 reference baseline: 1.26%.
cohere-transcribe-03-2026-Q4_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5134197d51aeb71dd6b764500222bd42d274931fe2d33389afcff628ff3252ce
3
- size 1549773856
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e29d5008f121976a70c14759b0a506327fbc44975c0791b87a972ae06eb8ebd8
3
+ size 1558162464
cohere-transcribe-03-2026-Q5_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:41193f0315140925c430fc5cc0bff62e7407453a38a5f11f5b992990e9f88c6d
3
- size 1763978272
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5f9b6f0895e26e8a41581d6cf2835b8983df38c5cfd27b6d4d4dbefb96b1fbad
3
+ size 1770269728
cohere-transcribe-03-2026-Q6_K.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a3abc361140a76a1cb69a326542fe96a14ec7a985cbd04acd885105bf79268b0
3
  size 1972524064
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1b54fa846b0813efa4fcd6ce1510c99bbaa67e2ef20323714069ce3b8c9846e4
3
  size 1972524064