Hugging Face's logo

RepublicOfKorokke
/
GLM-4.7-Flash-REAP-23B-A3B-oQ4

Text Generation
MLX
Safetensors
English
glm4_moe_lite
glm
MOE
pruning
compression
oq
quantized
conversational
4-bit precision
Model card Files
xet
 Community
RepublicOfKorokke commited on
Commit
f648ae5
·
verified ·
1 Parent(s): dbdd89d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +40 -23
README.md CHANGED
@@ -36,32 +36,49 @@ This model was quantized using [oQ](https://github.com/jundot/omlx) mixed-precis
36
  - **Bits**: 4
37
  - **Group size**: 64
38
  - **Format**: MLX safetensors
39
-
40
  # Benchmark
41
 
42
- | Model | File size | MMLU | JMMLU | HELLASWAG | ARC_CHALLENGE | GSM8K |
43
- | ------------------------------------ | --------- | ----- | ----- | --------- | ------------- | ----- |
44
- | GLM-4.7-Flash-REAP-23B-A3B-6bit | 17.43 GB | 62.3% | 46.0% | 53.0% | 73.0% | 96.7% |
45
- | GLM-4.7-Flash-REAP-23B-A3B-oQ3.5 | 10.62 GB | 57.7% | 49.3% | 60.0% | 65.0% | 93.3% |
46
- | GLM-4.7-Flash-REAP-23B-A3B-oQ4 | 12.51 GB | 63.3% | 39.7% | 51.0% | 70.0% | 90.0% |
 
 
47
 
48
  ## Detail
49
 
50
- | Model | Benchmark | Accuracy | Correct | Total | Time(s) |
51
- | ------------------------------------ | ------------- | -------- | ------- | ----- | ------- |
52
- | GLM-4.7-Flash-REAP-23B-A3B-6bit | MMLU | 62.3% | 187 | 300 | 505.9 |
53
- | GLM-4.7-Flash-REAP-23B-A3B-6bit | JMMLU | 46.0% | 138 | 300 | 239.7 |
54
- | GLM-4.7-Flash-REAP-23B-A3B-6bit | HELLASWAG | 53.0% | 53 | 100 | 114.7 |
55
- | GLM-4.7-Flash-REAP-23B-A3B-6bit | ARC_CHALLENGE | 73.0% | 73 | 100 | 64.4 |
56
- | GLM-4.7-Flash-REAP-23B-A3B-6bit | GSM8K | 96.7% | 29 | 30 | 88.6 |
57
- | GLM-4.7-Flash-REAP-23B-A3B-oQ3.5 | MMLU | 57.7% | 173 | 300 | 555.1 |
58
- | GLM-4.7-Flash-REAP-23B-A3B-oQ3.5 | JMMLU | 49.3% | 148 | 300 | 252.4 |
59
- | GLM-4.7-Flash-REAP-23B-A3B-oQ3.5 | HELLASWAG | 60.0% | 60 | 100 | 107.2 |
60
- | GLM-4.7-Flash-REAP-23B-A3B-oQ3.5 | GSM8K | 93.3% | 28 | 30 | 76.4 |
61
- | GLM-4.7-Flash-REAP-23B-A3B-oQ3.5 | ARC_CHALLENGE | 65.0% | 65 | 100 | 61.5 |
62
- | GLM-4.7-Flash-REAP-23B-A3B-oQ4 | MMLU | 63.3% | 190 | 300 | 550.7 |
63
- | GLM-4.7-Flash-REAP-23B-A3B-oQ4 | JMMLU | 39.7% | 119 | 300 | 250.9 |
64
- | GLM-4.7-Flash-REAP-23B-A3B-oQ4 | HELLASWAG | 51.0% | 51 | 100 | 103.4 |
65
- | GLM-4.7-Flash-REAP-23B-A3B-oQ4 | ARC_CHALLENGE | 70.0% | 70 | 100 | 59.8 |
66
- | GLM-4.7-Flash-REAP-23B-A3B-oQ4 | GSM8K | 90.0% | 27 | 30 | 75.6 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
67
 
 
36
  - **Bits**: 4
37
  - **Group size**: 64
38
  - **Format**: MLX safetensors
39
+ -
40
  # Benchmark
41
 
42
+ | Model | File size | MMLU | JMMLU | HELLASWAG | GSM8K | ARC_CHALLENGE |
43
+ | -------------------------------------------------------------- | --------- | ----- | ----- | --------- | ----- | ------------- |
44
+ | GLM-4.7-Flash-REAP-23B-A3B-6bit | 17.43 GB | 62.3% | 46.0% | 53.0% | 96.7% | 73.0% |
45
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ3 | 9.91 GB | 53.3% | 38.3% | 47.7% | 73.3% | 73.3% |
46
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ3.5 | 10.62 GB | 57.7% | 49.3% | 60.0% | 93.3% | 65.0% |
47
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ4 | 12.51 GB | 59.3% | 43.0% | 53.3% | 87.7% | 78.7% |
48
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ5 | 15.21 GB | 61.0% | 45.3% | 59.0% | 90.0% | 81.0% |
49
 
50
  ## Detail
51
 
52
+ | Model | Benchmark | Accuracy | Correct | Total | Time(s) |
53
+ | -------------------------------------------------------------- | ------------- | -------- | ------- | ----- | ------- |
54
+ | GLM-4.7-Flash-REAP-23B-A3B-6bit | MMLU | 62.3% | 187 | 300 | 505.9 |
55
+ | GLM-4.7-Flash-REAP-23B-A3B-6bit | JMMLU | 46.0% | 138 | 300 | 239.7 |
56
+ | GLM-4.7-Flash-REAP-23B-A3B-6bit | HELLASWAG | 53.0% | 53 | 100 | 114.7 |
57
+ | GLM-4.7-Flash-REAP-23B-A3B-6bit | GSM8K | 96.7% | 29 | 30 | 88.6 |
58
+ | GLM-4.7-Flash-REAP-23B-A3B-6bit | ARC_CHALLENGE | 73.0% | 73 | 100 | 64.4 |
59
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ3 | MMLU | 53.3% | 160 | 300 | 602.7 |
60
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ3 | JMMLU | 38.3% | 115 | 300 | 255.7 |
61
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ3 | HELLASWAG | 47.7% | 143 | 300 | 346.8 |
62
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ3 | ARC_CHALLENGE | 73.3% | 220 | 300 | 204.8 |
63
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ3 | GSM8K | 73.3% | 220 | 300 | 1029.3 |
64
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ3.5 | MMLU | 57.7% | 173 | 300 | 555.1 |
65
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ3.5 | JMMLU | 49.3% | 148 | 300 | 252.4 |
66
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ3.5 | HELLASWAG | 60.0% | 60 | 100 | 107.2 |
67
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ3.5 | GSM8K | 93.3% | 28 | 30 | 76.4 |
68
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ3.5 | ARC_CHALLENGE | 65.0% | 65 | 100 | 61.5 |
69
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ4 | MMLU | 63.3% | 190 | 300 | 550.7 |
70
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ4 | JMMLU | 39.7% | 119 | 300 | 250.9 |
71
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ4 | HELLASWAG | 51.0% | 51 | 100 | 103.4 |
72
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ4 | GSM8K | 90.0% | 27 | 30 | 75.6 |
73
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ4 | ARC_CHALLENGE | 70.0% | 70 | 100 | 59.8 |
74
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ4 | MMLU | 59.3% | 178 | 300 | 547.7 |
75
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ4 | JMMLU | 43.0% | 129 | 300 | 232.6 |
76
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ4 | HELLASWAG | 53.3% | 160 | 300 | 300.5 |
77
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ4 | ARC_CHALLENGE | 78.7% | 236 | 300 | 179.7 |
78
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ4 | GSM8K | 87.7% | 263 | 300 | 748.4 |
79
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ5 | MMLU | 61.0% | 183 | 300 | 617.8 |
80
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ5 | JMMLU | 45.3% | 136 | 300 | 273 |
81
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ5 | HELLASWAG | 59.0% | 177 | 300 | 353.6 |
82
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ5 | ARC_CHALLENGE | 81.0% | 243 | 300 | 201.2 |
83
+ | GLM-4.7-Flash-REAP-23B-A3B-oQ5 | GSM8K | 90.0% | 270 | 300 | 1001.1 |
84