HarmenWessels/gemma-4-E4B-it-qat-int4-ov

Gemma 4 E4B QAT — OpenVINO INT4 (q4_0-matched scheme)

OpenVINO IR conversion of google/gemma-4-E4B-it-qat-q4_0-unquantized — Google's quantization-aware-trained Gemma 4 E4B checkpoint, converted with the QAT-matched scheme (symmetric, group size 32):

optimum-cli export openvino -m google/gemma-4-E4B-it-qat-q4_0-unquantized \
  --task image-text-to-text --weight-format int4 --sym --group-size 32 \
  gemma-4-E4B-it-qat-int4-ov

QAT means this int4 build is trained to be quantized, so the quality loss is far smaller than post-training quantization at the same bit width.

Conversion note

Loader warnings about "missing" k/v projections on upper layers are Gemma 4's tied KV-shared weights (verified benign). Serving and tooling: core-ultra-llm-server.

Usage (OpenVINO GenAI)

import openvino_genai as ov_genai

pipe = ov_genai.VLMPipeline("gemma-4-E4B-it-qat-int4-ov", "GPU", CACHE_DIR="./.ovcache")
print(pipe.generate("Explain Python decorators in three sentences.", max_new_tokens=128))

VLM-shaped IR — requires VLMPipeline even for text-only use.

Provenance & license

• Base: Google's Gemma 4 E4B QAT checkpoint (released 2026-04); weights are governed by the Gemma Terms of Use
• Conversion date: 2026-06-06; optimum-intel git-master, transformers 5.5.0
• No finetuning — direct quantization of Google's QAT weights