Commit ·
d5c3680
1
Parent(s): 1b5f0e2
Update README.md (#6)
Browse files- Update README.md (c22a1b989dfbd5f8371fc23ac789a8fb70ccb3c2)
Co-authored-by: CarreyZhan <CarreyZhan@users.noreply.huggingface.co>
README.md
CHANGED
|
@@ -29,18 +29,18 @@ This repository hosts the [**LiteRT-LM**](https://ai.google.dev/edge/litert-lm)
|
|
| 29 |
|
| 30 |
## Quantization Benchmark
|
| 31 |
|
| 32 |
-
The table below compares the accuracy of the two LiteRT-LM quantized models on the same evaluation benchmark. Scores are reported as percentages.
|
| 33 |
-
|
| 34 |
-
| Category |
|
| 35 |
-
| --- | --- | --- |
|
| 36 |
-
| Overall | 63.78 | 60.91 |
|
| 37 |
-
| STEM | 56.35 | 53.36 |
|
| 38 |
-
| Social Science | 73.65 | 70.87 |
|
| 39 |
-
| Humanities | 66.68 | 64.35 |
|
| 40 |
-
| Other | 65.40 | 62.14 |
|
| 41 |
-
| Hard | 43.79 | 44.23 |
|
| 42 |
-
|
| 43 |
-
W8 uses dynamic weight-only INT8 quantization (`wi8`) with FP32 activations (`afp32`). W4 uses mixed INT4-block32 (linear) / INT8 (embed and lmhead) quantization (`wi4b32_wi8`) with FP32 activations (`afp32`). Thinking mode has been turned off to reduce context usage.
|
| 44 |
|
| 45 |
## What is MiniCPM?
|
| 46 |
|
|
|
|
| 29 |
|
| 30 |
## Quantization Benchmark
|
| 31 |
|
| 32 |
+
The table below compares the accuracy of the FP baseline and the two LiteRT-LM quantized models on the same evaluation benchmark. Scores are reported as percentages.
|
| 33 |
+
|
| 34 |
+
| Category | FP (bf16) | W8 | W4 |
|
| 35 |
+
| --- | --- | --- | --- |
|
| 36 |
+
| Overall | 66.16 | 63.78 | 60.91 |
|
| 37 |
+
| STEM | 58.07 | 56.35 | 53.36 |
|
| 38 |
+
| Social Science | 77.91 | 73.65 | 70.87 |
|
| 39 |
+
| Humanities | 67.65 | 66.68 | 64.35 |
|
| 40 |
+
| Other | 68.71 | 65.40 | 62.14 |
|
| 41 |
+
| Hard | 47.08 | 43.79 | 44.23 |
|
| 42 |
+
|
| 43 |
+
FP (bf16) is the full-precision baseline. W8 uses dynamic weight-only INT8 quantization (`wi8`) with FP32 activations (`afp32`). W4 uses mixed INT4-block32 (linear) / INT8 (embed and lmhead) quantization (`wi4b32_wi8`) with FP32 activations (`afp32`). Thinking mode has been turned off to reduce context usage. All evaluations use a temperature of 0.6, top-p of 0.95, and a random seed of 0.
|
| 44 |
|
| 45 |
## What is MiniCPM?
|
| 46 |
|