Instructions to use vanch007/gemma-4-E4B-it-heretic-mlx-mxfp8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use vanch007/gemma-4-E4B-it-heretic-mlx-mxfp8 with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir gemma-4-E4B-it-heretic-mlx-mxfp8 vanch007/gemma-4-E4B-it-heretic-mlx-mxfp8
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
gemma-4-E4B-it-heretic-mlx-mxfp8
Converted from coder3101/gemma-4-E4B-it-heretic with mlx_vlm.convert.
- Quantization: mxfp8
- q_bits: 8
- q_mode: mxfp8
- Source type: Gemma 4 multimodal (Gemma4ForConditionalGeneration)
- Converter env: /Users/vanch/.cache/lm-studio/.venvs/gemma4-mlx
- Validation date: 2026-04-26
Validation Summary
- Status: passed local text and vision smoke tests
- Vision metadata present:
processor_config.jsonandpreprocessor_config.json - Peak memory across sampled tests: 9.593 GB
Validation Matrix
| Case | Output | Prompt TPS | Generation TPS | Peak Memory |
|---|---|---|---|---|
| Text: one-sentence model description | Gemma 4 is a large language model developed by Google DeepMind. | 150.998 | 62.448 | 8.885 GB |
| Text: two-bullet structured answer | * The model belongs to the Gemma family of lightweight, open-weights models. * It possesses a multimodal capability, meaning it can process and understand different types of input, such as text and potentially images or audio depending on the specific version. |
171.767 | 60.052 | 8.897 GB |
| Vision: red image color identification | The image is red. | 562.027 | 66.977 | 9.593 GB |
| Vision: blue image color identification | The image is blue. | 558.614 | 67.212 | 9.593 GB |
Local Artifacts
- Full verification logs:
/Users/vanch/models/gemma4-e4b-heretic-full-verify/mxfp8-*.log
- Downloads last month
- 15
Model size
3B params
Tensor type
BF16
·
U8 ·
U32 ·
Hardware compatibility
Log In to add your hardware
8-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for vanch007/gemma-4-E4B-it-heretic-mlx-mxfp8
Base model
google/gemma-4-E4B Finetuned
google/gemma-4-E4B-it Finetuned
coder3101/gemma-4-E4B-it-heretic