Add 4-bit MLX conversion of Phi-4-multimodal-instruct

README.md

@@ -145,7 +145,7 @@ pip install mlx-vlm>=0.1.0 mlx>=0.22.0
 
 ## Benchmark Results
 
-All evaluations conducted on a single Apple Silicon device using our [evaluation harness](https://github.com/ferox-ai/phi4mm-mlx). Scores are computed on a 100-sample subset of each benchmark. Microsoft's reference scores are reported on the full dataset using PyTorch FP16 — direct comparison should account for both the precision difference and sample-size variance.
+Evaluated with our internal evaluation harness on a single Apple Silicon device. Scores are computed on a 100-sample subset of each benchmark. Microsoft's reference scores are reported on the full dataset using PyTorch FP16 — direct comparison should account for both the precision difference and sample-size variance.
 
 | Benchmark | This Model (4-bit) | bf16 | Microsoft FP16 (full dataset) | Metric |
 |-----------|:------------------:|:----:|:-----------------------------:|--------|
@@ -166,7 +166,7 @@ Across all benchmarks, 4-bit quantization produces a mean accuracy delta of −2
 
 ### Note on MMMU
 
-The MMMU scores (24.0% 4-bit, 31.0% bf16) are significantly below Microsoft's reference (55.1%). Since the bf16 variant is lossless, this gap is not attributable to quantization or weight conversion. We attribute it to a combination of: (1) answer-extraction sensitivity in our evaluation harness for MMMU's multiple-choice format, and (2) variance inherent to a 100-sample evaluation subset. We are investigating the extraction logic and plan to re-evaluate on the full validation split. This does not reflect the model's actual MMMU capability.
+The 100-sample MMMU scores (24.0% 4-bit, 31.0% bf16) fall well below Microsoft's reported 55.1%. To isolate the cause, we ran a **full 900-sample MMMU validation** on the lossless bf16 variant and obtained **27.9%** — consistent with the subset, which confirms the gap is **not** caused by quantization or weight conversion. We were unable to reproduce Microsoft's 55.1% and attribute the difference to evaluation-harness and answer-extraction handling for MMMU's multiple-choice format (prompt formatting and option parsing), rather than to the model's underlying capability — which is better reflected by the document-, chart-, OCR-, and science-focused benchmarks above.
 
 ## Architecture
 
@@ -190,7 +190,7 @@ Weights are converted from `microsoft/Phi-4-multimodal-instruct` using a determi
 5. Quantize backbone to 4-bit (SigLIP excluded)
 6. Serialize as MLX safetensors
 
-Full conversion and quantization scripts are available in the [project repository](https://github.com/ferox-ai/phi4mm-mlx).
+The conversion and quantization pipeline is deterministic and fully reproducible from the base model.
 
 ## Intended Use
 
@@ -220,10 +220,10 @@ This model is designed for **local, on-device vision-language inference** on App
 If you use this model in your work, please cite:
 
 ```bibtex
-@misc{feroxai2025phi4mlx,
+@misc{feroxai2026phi4mlx,
   title={Phi-4-Multimodal-Instruct MLX Conversion},
   author={Ferox AI},
-  year={2025},
+  year={2026},
   url={https://huggingface.co/ferox-ai/Phi-4-multimodal-instruct-mlx-4bit},
   note={4-bit quantized MLX port of microsoft/Phi-4-multimodal-instruct}
 }