Hakureirm commited on
Commit
6edb39d
·
verified ·
1 Parent(s): 86415f2

Finalize model card (accuracy notes) for public release

Browse files
Files changed (1) hide show
  1. README.md +16 -7
README.md CHANGED
@@ -1,17 +1,26 @@
1
  ---
2
  license: apache-2.0
3
- frameworks: [pytorch]
4
- tasks: [text-generation]
5
- base_model: [BlinkDL/rwkv7-g1]
 
 
 
 
 
6
  ---
7
 
8
  # RWKV-7 G1 7.2B — int4 GPTQ for rwkv-sglang (fits a 16 GB card)
9
 
10
- Hand-written **weight-only int4** (GPTQ-calibrated) quantization of BlinkDL's RWKV-7 "Goose"
11
- G1 7.2B, for the [rwkv-sglang](https://github.com/Hakureirm/rwkv-sglang) serving overlay.
 
12
 
13
- - **Accuracy: greedy token-for-token EXACT** on the oracle fixture (8/8); 192/192 projections
14
- GPTQ-quantized (streamed Hessian calibration, no RTN fallback).
 
 
 
15
  - **Runs a 7.2B model on a 16 GB GPU:** checkpoint **4.6 GB** (vs 14.4 GB fp16); the int4 path
16
  serves 7.2B in well under 16 GB total. On a real T4 (16 GB) the sibling RTN build measured
17
  greedy 8/8 EXACT, 32.9 tok/s bsz1, peak 6.7 GB.
 
1
  ---
2
  license: apache-2.0
3
+ base_model: BlinkDL/rwkv7-g1
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - rwkv
7
+ - rwkv7
8
+ - int4
9
+ - gptq
10
+ - quantized
11
  ---
12
 
13
  # RWKV-7 G1 7.2B — int4 GPTQ for rwkv-sglang (fits a 16 GB card)
14
 
15
+ Hand-written **weight-only int4** (GPTQ-calibrated, symmetric, group 64) quantization of
16
+ BlinkDL's RWKV-7 "Goose" G1 7.2B, for the
17
+ [rwkv-sglang](https://github.com/Hakureirm/rwkv-sglang) serving overlay.
18
 
19
+ - **Accuracy: near-lossless at this size.** MATH500 avg@64 (500 problems × 64 rollouts):
20
+ **61.08% vs fp16's 64.18% (−3.1pt)** the same symmetric GPTQ scheme that loses 25.6pt at
21
+ 1.5B costs three points at 7.2B; bigger models quantize better, measured not assumed.
22
+ Greedy decoding is **token-for-token EXACT** on the oracle fixture (8/8); 192/192
23
+ projections GPTQ-quantized (streamed Hessian calibration, no RTN fallback).
24
  - **Runs a 7.2B model on a 16 GB GPU:** checkpoint **4.6 GB** (vs 14.4 GB fp16); the int4 path
25
  serves 7.2B in well under 16 GB total. On a real T4 (16 GB) the sibling RTN build measured
26
  greedy 8/8 EXACT, 32.9 tok/s bsz1, peak 6.7 GB.