metadata
license: gemma
base_model: google/gemma-4-E4B-it-qat-q4_0-unquantized
base_model_relation: quantized
tags:
- openvino
- int4
- qat
- intel
- arc
- igpu
- core-ultra
pipeline_tag: image-text-to-text
Gemma 4 E4B QAT — OpenVINO INT4 (q4_0-matched scheme)
OpenVINO IR conversion of google/gemma-4-E4B-it-qat-q4_0-unquantized — Google's quantization-aware-trained Gemma 4 E4B checkpoint, converted with the QAT-matched scheme (symmetric, group size 32):
optimum-cli export openvino -m google/gemma-4-E4B-it-qat-q4_0-unquantized \
--task image-text-to-text --weight-format int4 --sym --group-size 32 \
gemma-4-E4B-it-qat-int4-ov
QAT means this int4 build is trained to be quantized, so the quality loss is far smaller than post-training quantization at the same bit width.
Conversion note
Loader warnings about "missing" k/v projections on upper layers are Gemma 4's tied KV-shared weights (verified benign). Serving and tooling: core-ultra-llm-server.
Usage (OpenVINO GenAI)
import openvino_genai as ov_genai
pipe = ov_genai.VLMPipeline("gemma-4-E4B-it-qat-int4-ov", "GPU", CACHE_DIR="./.ovcache")
print(pipe.generate("Explain Python decorators in three sentences.", max_new_tokens=128))
VLM-shaped IR — requires VLMPipeline even for text-only use.
Provenance & license
- Base: Google's Gemma 4 E4B QAT checkpoint (released 2026-04); weights are governed by the Gemma Terms of Use
- Conversion date: 2026-06-06; optimum-intel git-master, transformers 5.5.0
- No finetuning — direct quantization of Google's QAT weights