Instructions to use Jundot/Qwen3.6-27B-oQ4e-mtp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use Jundot/Qwen3.6-27B-oQ4e-mtp with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir Qwen3.6-27B-oQ4e-mtp Jundot/Qwen3.6-27B-oQ4e-mtp
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
Qwen3.6-27B-oQ4e-mtp
This model was quantized using oQ (oMLX v0.4.5.dev1) mixed-precision quantization.
Quantization details
- Model type: qwen3_5
- Bits: 4
- Group size: 64
- Format: MLX safetensors
Benchmark
All runs used thinking off. Evaluation used the oMLX standard benchmark runner with greedy decoding, so the same model artifacts reproduce the same results. Disk size is shown under each quantization label. Original is shown as a reference and is excluded from bold best-score marking. Bold values mark the best quantized score within each model for each benchmark, including the average; ties are all bolded.
| Model | Quantization | MMLU 1000 | Winogrande 300 | HumanEval 164 | MBPP 300 | Average % |
|---|---|---|---|---|---|---|
| Qwen3.6-27B | Original (55.6GB) |
877/1000 87.70% |
241/300 80.33% |
149/164 90.85% |
250/300 83.33% |
85.56% |
| mlx-lm 4bit (16.1GB) |
865/1000 86.50% |
239/300 79.67% |
153/164 93.29% |
244/300 81.33% |
85.20% | |
| oQ4 (16.7GB) |
869/1000 86.90% |
239/300 79.67% |
151/164 92.07% |
244/300 81.33% |
84.99% | |
| oQ4e imatrix (16.7GB) |
868/1000 86.80% |
239/300 79.67% |
152/164 92.68% |
250/300 83.33% |
85.62% |
- Downloads last month
- 218
Model size
5B params
Tensor type
BF16
·
U32 ·
Hardware compatibility
Log In to add your hardware
4-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support