Update README.md
Browse files
README.md
CHANGED
|
@@ -6,6 +6,17 @@ license: apache-2.0
|
|
| 6 |
|
| 7 |
<Fill-me for Mathstral-7B-v0.1>
|
| 8 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 9 |
## Installation
|
| 10 |
|
| 11 |
It is recommended to use `mistralai/mathstral-7B-v0.1` with [mistral-inference](https://github.com/mistralai/mistral-inference)
|
|
|
|
| 6 |
|
| 7 |
<Fill-me for Mathstral-7B-v0.1>
|
| 8 |
|
| 9 |
+
## Evaluation
|
| 10 |
+
We evaluate Mathstral 7B and models of the similar size on industry-standard benchmarks.
|
| 11 |
+
| Benchmarks | MATH | GSM8K (8-shot) | Odyssey Math | GRE Math | AMC 2023 | AIME 2024
|
| 12 |
+
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
|
| 13 |
+
| Mathstral 7B | **56.6** | 77.1 | **37.2** | 56.9 | **42.4** | **2/30** |
|
| 14 |
+
| DeepSeek Math 7B | 44.4 | **80.6** | 27.6 | 44.6 | 28.0 | 0/30 |
|
| 15 |
+
| Llama3 8B | 28.4 | 75.4 | 24.0 | 26.2 | 34.4 | 0/30 |
|
| 16 |
+
| GLM4 9B | 50.2 | 48.8 | 18.9 | 46.2 | 36.0 | 1/30 |
|
| 17 |
+
| QWen2 7B | **56.8** | 32.7 | 24.8 | **58.5** | 35.2 | **2/30** |
|
| 18 |
+
| Gemma2 9B | 48.3 | 69.5 | 18.6 | 52.3 | 31.2 | 1/30 |
|
| 19 |
+
|
| 20 |
## Installation
|
| 21 |
|
| 22 |
It is recommended to use `mistralai/mathstral-7B-v0.1` with [mistral-inference](https://github.com/mistralai/mistral-inference)
|