README.md

---
license: gemma
base_model: google/gemma-4-31b-it
library_name: mlx
tags:
- mlx
- vision
pipeline_tag: image-text-to-text
---

# ToPo-ToPo/gemma-4-31b-it-mlx-bf16

MLX **bf16** conversion of [`google/gemma-4-31b-it`](https://huggingface.co/google/gemma-4-31b-it) for Apple Silicon (mlx-vlm).

## Provenance (self-converted from official weights)
- Source: [`google/gemma-4-31b-it`](https://huggingface.co/google/gemma-4-31b-it) (license: gemma)
- Tool: `mlx-vlm 0.6.3` — `mlx_vlm.convert --hf-path google/gemma-4-31b-it --mlx-path . --dtype bfloat16`
- Effective: **16.0 bits/weight**
- Validation: reproduced geometrically exact CAD output in an agentic CAD+FEM pipeline
  (volumes match the reference mlx-community conversion).

## Usage
```python
from mlx_vlm import load, generate
model, processor = load("ToPo-ToPo/gemma-4-31b-it-mlx-bf16")
```

## License
This is a derivative of Google **Gemma**. Use is governed by the [Gemma Terms of Use](https://ai.google.dev/gemma/terms) and the [Gemma Prohibited Use Policy](https://ai.google.dev/gemma/prohibited_use_policy). Weights were converted/quantized to MLX format (modification notice per the Gemma Terms).

## ⚡ Faster generation with MTP (speculative decoding, lossless)

**Recommended drafter: `google/gemma-4-31b-it-assistant`** — Google's official MTP drafter for this
model. It loads **directly in mlx-vlm (no conversion needed)** and gives up to
~3x faster generation (≈1.4–1.5x measured on short prompts); output is
**identical** to non-MTP decoding.

```python
# requires:  pip install "mlx-vlm>=0.6.3"
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model, processor = load("ToPo-ToPo/gemma-4-31b-it-mlx-bf16")
draft_model, _   = load("google/gemma-4-31b-it-assistant")
config = load_config("ToPo-ToPo/gemma-4-31b-it-mlx-bf16")

prompt = apply_chat_template(processor, config, "Hello!", num_images=0)
out = generate(model, processor, prompt,
               draft_model=draft_model, draft_kind="mtp", max_tokens=256)
```

CLI (draft_kind auto-detected):
`mlx_vlm.generate --model ToPo-ToPo/gemma-4-31b-it-mlx-bf16 --draft-model google/gemma-4-31b-it-assistant`

### Notes
- `draft_kind="mtp"` is required in the Python API (the CLI auto-detects it).
- Use **this model's own** drafter above — drafters are size-specific and not interchangeable across Gemma 4 variants.
- Needs **mlx-vlm >= 0.6.3**. MTP is lossless — if output differs from non-MTP, your versions are mismatched.