Looking forward to the mxfp4 quantization format.
#2
by markxixihaha - opened
It would be even better if there were mxfp4, as this quantization format generates the fastest speed and smallest size on my Mac.
Here - https://huggingface.co/TheCluster/Qwen3.6-35B-A3B-Heretic-MLX-mxfp4.
According to a research by Unsloth, the Qwen 3.5 (and likely 3.6) hybrid models degrade significantly starting with mxfp4, which is why I didn't include that option.
Thanks for your hard work. I'll keep the points you mentioned in mind.