🦆 zecanard/gemma-4-26B-A4B-it-uncensored-abliterix-MLX-3bit-affine
This model was converted to MLX from wangzhang/gemma-4-26B-A4B-it-abliterix using mlx-vlm version 0.4.4.
Please refer to the original model card for more details.
🌟 Quality
Quantized vision language model with 4.364 bits per weight.
mlx_vlm.convert --quantize --q-bits 3 --q-group-size 32 --q-mode affine
🛠️ Customizations
This quant is aware of the current date, and also enables thinking (if available). You may disable this behavior by deleting the following line from the chat template:
{%- set enable_thinking = true %}
You may also need to adjust your environment’s Reasoning Section Parsing to recognize <|channel>thought as the Start String, and <channel|> as the End String.
🖥️ Use with mlx
pip install -U mlx-vlm
mlx_vlm.generate --model zecanard/gemma-4-26B-A4B-it-uncensored-abliterix-MLX-3bit-affine --max-tokens 100 --temperature 0 --prompt "Describe this image." --image <path_to_image>
- Downloads last month
- 765
Model size
5B params
Tensor type
BF16
·
U32 ·
Hardware compatibility
Log In to add your hardware
3-bit
Model tree for zecanard/gemma-4-26B-A4B-it-uncensored-abliterix-MLX-3bit-int3-affine
Base model
google/gemma-4-26B-A4B Finetuned
google/gemma-4-26B-A4B-it Quantized
wangzhang/gemma-4-26B-A4B-it-abliterix
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("zecanard/gemma-4-26B-A4B-it-uncensored-abliterix-MLX-3bit-int3-affine") config = load_config("zecanard/gemma-4-26B-A4B-it-uncensored-abliterix-MLX-3bit-int3-affine") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output)