Add Local SOTA comparison section and chart

README.md

@@ -14,6 +14,27 @@ tags:
 pipeline_tag: image-text-to-text
 ---
 
+## Local SOTA for 48GB Macs — Intelligence Benchmark Comparison
+
+This model is part of a benchmark comparison of the best local MLX-quantized LLMs that fit in 48GB unified memory on Apple Silicon. All benchmarks run in instruct mode (no thinking) with n=50 samples per benchmark.
+
+![SOTA Comparison](sota_comparison_instruct.png)
+
+| Benchmark | Samples | Agents-A1 6bit-XL | Gemma-4 26B 6bit-XL | Huihui-Qwen3.6 6bit-XL | Ornith-35B 6bit-XL | Qwen3.6-27B oQ4e | Qwen3.6-35B 6bit-XL | Qwen3.6-35B oQ4e | Qwen3.6-35B oQ4e-XL | Qwen3.6-35B oQ6 |
+|-----------|:-------:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
+| MMLU | 50/14042 | 66% | **76%** | 74% | 64% | 74% | 64% | 66% | 72% | 64% |
+| MMLU_PRO | 50/12032 | 58% | **82%** | 66% | 66% | 56% | 64% | 60% | 64% | 60% |
+| ARC_CHALLENGE | 50/1172 | 90% | 90% | **92%** | **92%** | 88% | 90% | **92%** | **92%** | 90% |
+| HUMANEVAL | 50/164 | 90% | **98%** | 84% | 78% | **92%** | 78% | **92%** | 90% | 66% |
+| MBPP | 50/500 | 70% | 82% | 78% | 78% | **86%** | 78% | 80% | 76% | 76% |
+| **Average** | | 74.8% | **85.6%** | 78.8% | 75.6% | 79.2% | 74.8% | 78.0% | 78.8% | 71.2% |
+
+**Collection:** [Local SOTA for 48GB Macs](https://huggingface.co/collections/leonsarmiento/local-sota-for-48gb-macs-6a5fb58390dd01e1fc35d55e)
+
+> ⚠️ n=50 sampling means wide confidence intervals (±~13% at 95% CI). Differences under ~6 points may not be statistically significant. Models using data-aware quantization (oQ/oQe) may be calibrated on benchmark-like data — their scores carry a benchmaxxing caveat. The BaseQuant_XL variants (data-agnostic) provide the most honest generalization estimates.
+
+
+
 # leonsarmiento/Agents-A1-6bit-XL-mlx
 
 This model was converted to MLX format from [`InternScience/Agents-A1`](https://huggingface.co/InternScience/Agents-A1) using **BaseQuant_XL 6/8-bit mixed quantization** optimized for Apple Silicon. The vision encoder is preserved and quantized at 6-bit, making this a full multimodal model.