Update README.md

86f3e9f verified 6 months ago

5.73 kB

license: apache-2.0
base_model: mistralai/Ministral-3-3B-Instruct-2512-BF16
base_model_relation: quantized
quantized_by: UnstableLlama
tags:
  - exl3

>> mistralai/Ministral-3-8B-Instruct-2512

[QUANT :: UNSTABLELLAMA]

// REPO

EXL3 quantizations of mistralai's Ministral-3-8b-Instruct-2512-BF16.

Quantized with exllamav3 0.0.18.

[BRANCH]	[GiB]	[K/L_DIV]	[PPL]
2.10bpw	4.4	no	measurements
3.00bpw	4.77	0.05864883	7.89712533
3.15bpw	4.89	0.04512541	7.81142826
4.00bpw	5.63	0.0162187	7.61291338
6.00bpw	7.36	0.0162187	7.51715039
bf16	16	0	x

Use HF-CLI to pull specific branches to your local machine:

huggingface-cli download UnstableLlama/Ministral-3-8B-Instruct-2512-exl3 --revision "3.00bpw" --local-dir ./