---
library_name: mlx
license: apache-2.0
license_link: https://ai.google.dev/gemma/docs/gemma_4_license
pipeline_tag: image-text-to-text
base_model: google/gemma-4-26B-A4B-it-qat-q4_0-gguf
tags:
- mlx
- gemma4
- qat
- q4
- moe
- apple-silicon
---

# mavis-ai/Gemma4-26B-MoE-QATQ4

This repository provides an MLX-ready **QAT Q4** build of Google's Gemma 4 26B A4B instruction-tuned Mixture-of-Experts model for local inference on Apple Silicon Macs.

Unlike a plain post-training 4-bit quantized checkpoint, this model is based on Google's **Quantization-Aware Training (QAT)** Q4 release. The goal is not only to reduce the storage and memory footprint, but also to preserve the model's practical judgment, instruction-following behavior, and reasoning stability under low-bit inference.

**💡 Notice for Everyone**
While this repository is hosted primarily as a dedicated engine source for our application ecosystem (**R.E.V.I.S.**), **you are completely free to download and use this model normally** for your own local MLX projects or workflows.

For more details on the official source model, please refer to Google's Gemma 4 model pages:

- [google/gemma-4-26B-A4B-it](https://huggingface.co/google/gemma-4-26B-A4B-it)
- [google/gemma-4-26B-A4B-it-qat-q4_0-gguf](https://huggingface.co/google/gemma-4-26B-A4B-it-qat-q4_0-gguf)

## Why QAT Q4?

R.E.V.I.S. is designed around local, evidence-grounded AI workflows where the model does not need to memorize everything. Instead, the model should be strong at judging, reading, filtering, structuring, and reasoning over the local context it is given.

QAT Q4 is especially interesting for this direction because it aims to keep more of the model's behavior intact after quantization, compared with ordinary post-training quantization. In practical local workflows, this can matter for tasks such as:

- judging whether a passage is real body text or navigation / advertisement noise
- checking whether a Markdown extraction is complete enough for RAG
- preserving instruction-following behavior in low-bit inference
- restructuring long documents into useful sections
- acting as a second-opinion evaluator for local agents

## 🚀 Optimized for R.E.V.I.S. (Local Cognitive OS)

We prepared and tested this QAT Q4 model to serve as a local reasoning and judgment engine for **R.E.V.I.S.**

**R.E.V.I.S.** is a 100% local Cognitive OS for Multi-Agentic AI. It transforms your Mac devices into a distributed Agentic Swarm via zero-config Wi-Fi clustering, allowing you to run heavy AI workloads—like recursive web research, dynamic RAG generation, and multi-step logic—without killing single-machine performance.

If you are interested in pushing the limits of local AI and open-weight models like this one, check out our project!

- **🌐 Official Website:** [https://mavis-ai.co.jp/revis/](https://mavis-ai.co.jp/revis/)
- **▶️ Watch the 13-min Raw Demo (Multi-node Dynamic RAG):** [https://x.gd/LxaBF](https://x.gd/LxaBF)
- **🐦 Follow our updates on X:** [@mavis_ai_jp](https://x.com/mavis_ai_jp)

## Usage

Install or update the MLX runtime you use for Gemma 4 / multimodal models:

```bash
pip install -U mlx mlx-lm mlx-vlm huggingface_hub hf_xet
```

Download the model:

```bash
hf download mavis-ai/Gemma4-26B-MoE-QATQ4 \
  --local-dir ~/Models/mlx/Gemma4-26B-MoE-QATQ4
```

Run a quick text generation test:

```bash
python -m mlx_vlm.generate \
  --model ~/Models/mlx/Gemma4-26B-MoE-QATQ4 \
  --max-tokens 256 \
  --temperature 0.0 \
  --prompt "日本語で短く自己紹介してください。"
```

## Notes

Gemma 4 26B A4B is a Mixture-of-Experts model. Although only a subset of parameters is active per token, the full model still needs to be stored and loaded. On Apple Silicon, practical speed and context length depend heavily on unified memory size, MLX / mlx-vlm versions, prompt length, KV cache usage, and generation settings.

For day-to-day local RAG, a smaller Gemma 4 E2B / E4B QAT model may be faster. This 26B MoE QAT Q4 build is best suited for heavier judgment tasks, long-document inspection, extraction-quality review, and second-opinion reasoning.

## License

This model is a derivative of Google's Gemma 4 26B A4B instruction-tuned QAT Q4 release, which is distributed by Google under the **Apache License 2.0**.

This derivative is likewise distributed under the Apache License 2.0. A copy of the license is included in the `LICENSE` file in this repository, and can also be found at <https://www.apache.org/licenses/LICENSE-2.0>.

**Modification notice:** This repository adapts the official Gemma 4 26B A4B instruction-tuned QAT Q4 model for MLX-based local inference. No additional fine-tuning or architectural changes were applied by MAVIS.