Update README.md
Browse files
README.md
CHANGED
|
@@ -19,12 +19,11 @@ base_model:
|
|
| 19 |
- **KV cache quantization:** OCP FP8
|
| 20 |
- **Calibration Dataset:** [Pile](https://huggingface.co/datasets/mit-han-lab/pile-val-backup)
|
| 21 |
|
| 22 |
-
|
| 23 |
-
|
| 24 |
|
| 25 |
# Model Quantization
|
| 26 |
|
| 27 |
-
|
| 28 |
|
| 29 |
**Quantization scripts:**
|
| 30 |
```
|
|
|
|
| 19 |
- **KV cache quantization:** OCP FP8
|
| 20 |
- **Calibration Dataset:** [Pile](https://huggingface.co/datasets/mit-han-lab/pile-val-backup)
|
| 21 |
|
| 22 |
+
This model is a quantized version of [meta-llama/Llama-3.1-405B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-405B-Instruct),optimized using [AMD-Quark](https://quark.docs.amd.com/latest/index.html) framework with MXFP4 quantization.
|
|
|
|
| 23 |
|
| 24 |
# Model Quantization
|
| 25 |
|
| 26 |
+
The model was quantized from [meta-llama/Llama-3.1-405B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-405B-Instruct) using [AMD-Quark](https://quark.docs.amd.com/latest/index.html). Weights and activations were quantized to MXFP4, and KV caches were quantized to FP8. The AutoSmoothQuant algorithm was applied to enhance accuracy during quantization.
|
| 27 |
|
| 28 |
**Quantization scripts:**
|
| 29 |
```
|