--- library_name: mlx license: apache-2.0 license_link: https://ai.google.dev/gemma/docs/gemma_4_license pipeline_tag: image-text-to-text base_model: google/gemma-4-12B-it-qat-q4_0-unquantized tags: - mlx - gemma4 - qat - q4 - apple-silicon --- # mavis-ai/Gemma4-12B-QAT-Q4 This repository contains an MLX-ready **QAT Q4** distribution of Google's Gemma 4 12B Unified instruction-tuned model, prepared for local inference on Apple Silicon Macs. This is not a newly fine-tuned model. It is a quantized MLX redistribution converted from Google's official **Quantization-Aware Training (QAT)** unquantized checkpoint. No structural modifications or additional training have been applied to the model architecture. ## Important Notice This repository is hosted primarily as a dedicated engine source for the R.E.V.I.S. application ecosystem. You are free to download and use this model package for your own local MLX projects or workflows, subject to the Apache License 2.0 and Google's Gemma terms. For the original model cards, architecture details, intended usage, limitations, and evaluation information, refer to the official upstream models: - Base model: - Official QAT source checkpoint: - Official QAT Q4 GGUF reference: ## Quantization This package uses MLX affine quantization with a mixed layout chosen to follow Google's official QAT Q4 GGUF release as closely as practical in MLX: - Source checkpoint: `google/gemma-4-12B-it-qat-q4_0-unquantized` - Reference layout: `google/gemma-4-12B-it-qat-q4_0-gguf` - Default language tensor quantization: MLX affine Q4 - Group size: `64` - Mode: `affine` - Token embeddings: MLX affine Q6 - Audio, vision, and multimodal projection components: BF16 - Norms, biases, and small non-linear tensors: preserved in their source floating-point representation In practical terms, this is not a plain "quantize every linear layer to Q4" package. The embedding table is intentionally kept at higher precision, and the multimodal side of the model is not aggressively quantized. This keeps the MLX package closer to the official Google QAT Q4 distribution while still reducing local storage and memory requirements. ## Optimized for R.E.V.I.S. (Local Cognitive OS) We host and test this model package to serve as a local reasoning and judgment engine for **R.E.V.I.S.** **R.E.V.I.S.** is a 100% local Cognitive OS for Multi-Agentic AI. It transforms your Mac devices into a distributed Agentic Swarm via zero-config Wi-Fi clustering, allowing you to run heavy AI workloads like recursive web research, dynamic RAG generation, and multi-step logic without killing single-machine performance. If you are interested in pushing the limits of local AI and open-weight models, check out our project. - Official Website: - Watch the 13-min Raw Demo (Multi-node Dynamic RAG): - Follow our updates on X: ## Usage Install or update the MLX runtime you use for Gemma 4 / multimodal models: ```bash pip install -U mlx mlx-lm mlx-vlm huggingface_hub hf_xet ``` Download the model: ```bash hf download mavis-ai/Gemma4-12B-QAT-Q4 \ --local-dir ~/Models/mlx/Gemma4-12B-QAT-Q4 ``` Run a quick text generation test: ```bash python -m mlx_vlm.generate \ --model ~/Models/mlx/Gemma4-12B-QAT-Q4 \ --max-tokens 256 \ --temperature 0.0 \ --prompt "日本語で短く自己紹介してください。" ``` ## Usage Notes Gemma 4 12B Unified is a practical middle point between the smaller E2B / E4B models and the larger 26B MoE / 31B dense models. It is useful when local workflows need stronger reasoning and judgment while still remaining manageable on Apple Silicon systems with enough unified memory. For lighter repeated routing or extraction checks, the E2B / E4B QAT Q4 variants may be faster. For heavier long-document reasoning, the 26B / 31B QAT Q4 variants may be more appropriate if your Mac has enough unified memory. ## Files Recommended repository files: ```text README.md LICENSE config.json generation_config.json model-00001-of-00002.safetensors model-00002-of-00002.safetensors model.safetensors.index.json processor_config.json tokenizer.json tokenizer_config.json chat_template.jinja ``` ## License This repository redistributes a quantized derivative of Google's Gemma 4 12B instruction-tuned QAT release, which is distributed by Google under the **Apache License 2.0**. This derivative is likewise distributed under the Apache License 2.0. A copy of the license is included in the `LICENSE` file in this repository, and can also be found at . ## Modification Notice Compared with the official Google QAT source checkpoint, this repository applies the following packaging modification: ```text The source checkpoint was converted to MLX format and quantized with a GGUF-like mixed Q4/Q6/BF16 policy for local MLX inference. ``` No fine-tuning, additional training, or architecture-level modification has been applied.