| Quant   | Size (MB) | PPL       | Size (%) | Accuracy (%) | PPL error rate |
| ------- | --------- | --------- | -------- | ------------ | -------------- |
| IQ1_S   | 489       | 436.7686  | 14.40    | 8.68         | 9.93           |
| IQ1_M   | 515       | 188.5540  | 15.17    | 20.11        | 4.19           |
| IQ2_XXS | 560       | 131.2225  | 16.49    | 28.90        | 2.98           |
| IQ2_XS  | 597       | 76.8887   | 17.58    | 49.32        | 1.62           |
| IQ2_S   | 633       | 61.0257   | 18.65    | 62.14        | 1.26           |
| IQ2_M   | 668       | 54.0953   | 19.67    | 70.10        | 1.11           |
| Q2_K_S  | 683       | 62.3147   | 20.12    | 60.85        | 1.32           |
| Q2_K    | 718       | 54.6519   | 21.15    | 69.39        | 1.14           |
| IQ3_XXS | 733       | 44.2555   | 21.59    | 85.69        | 0.90           |
| IQ3_XS  | 793       | 41.2959   | 23.36    | 91.83        | 0.82           |
| Q3_K_S  | 821       | 42.9523   | 24.18    | 88.29        | 0.87           |
| IQ3_S   | 822       | 41.0938   | 24.21    | 92.28        | 0.82           |
| IQ3_M   | 836       | 40.2980   | 24.62    | 94.10        | 0.80           |
| Q3_K_M  | 881       | 41.7783   | 25.95    | 90.77        | 0.86           |
| Q3_K_L  | 935       | 42.7356   | 27.54    | 88.73        | 0.88           |
| IQ4_XS  | 972       | 40.5138   | 28.63    | 93.60        | 0.83           |
| IQ4_NL  | 1018      | 40.9880   | 29.99    | 92.52        | 0.84           |
| Q4_0    | 1019      | 40.6287   | 30.01    | 93.33        | 0.84           |
| Q4_K_S  | 1021      | 39.5842   | 30.07    | 95.80        | 0.81           |
| Q4_K_M  | 1065      | 39.5447   | 31.37    | 95.89        | 0.80           |
| Q4_1    | 1108      | 39.0552   | 32.64    | 97.10        | 0.79           |
| Q5_K_S  | 1200      | 39.0222   | 35.35    | 97.18        | 0.80           |
| Q5_0    | 1203      | 39.4241   | 35.43    | 96.19        | 0.80           |
| Q5_K_M  | 1225      | 38.6085   | 36.08    | 98.22        | 0.79           |
| Q5_1    | 1292      | 38.5234   | 38.06    | 98.43        | 0.78           |
| Q6_K    | 1396      | 38.1520   | 41.12    | 99.39        | 0.77           |
| Q8_0    | 1806      | 37.9817   | 53.19    | 99.84        | 0.77           |
| F16     | 3395      | 37.9208   | 100      | 100          | 0.77           |