---
license: other
license_name: health-ai-developer-foundations
license_link: https://developers.google.com/health-ai-developer-foundations/terms
extra_gated_heading: Access MedGemma on Hugging Face
extra_gated_prompt: >-
  To access MedGemma on Hugging Face, you're required to review and agree to
  Health AI Developer Foundations' terms of use. To do this, please ensure
  you're logged in to Hugging Face and click below. Requests are processed
  immediately.
extra_gated_button_content: Acknowledge license
base_model: google/medgemma-1.5-4b-it
library_name: litert-lm
pipeline_tag: text-generation
tags:
  - medical
  - litert
  - litert-lm
  - android
  - on-device
  - gemma3
  - clinical-reasoning
---

# MedGemma 1.5 4B IT — LiteRT-LM (.litertlm)

On-device conversion of [google/medgemma-1.5-4b-it](https://huggingface.co/google/medgemma-1.5-4b-it)
to the `.litertlm` format for [LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM),
targeting offline-first health-Q&A use on low-connectivity, low-RAM Android phones.

Converted by [TobiasLogic](https://huggingface.co/TobiasLogic), as requested in
[litert-community/gemma-4-E2B-it-litert-lm · discussion #39](https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm/discussions/39).

> ## ⚠️ Medical disclaimer — read before use
>
> MedGemma is a **developer foundation model for decision support and
> education. It is not a medical device, not a diagnostic tool, and not a
> substitute for professional medical advice, diagnosis, or treatment.**
> Outputs may be incorrect, incomplete, or unsafe if acted on without
> independent clinical verification. Developers **must validate this model
> for their specific use case — including the effects of the int4
> quantization applied here — before any real-world deployment**, and must
> comply with the [Health AI Developer Foundations terms of use](https://developers.google.com/health-ai-developer-foundations/terms).
> MedGemma is prompt-sensitive and **not tuned for multi-turn conversational
> use**; phrase each query as a single, self-contained question.

## License and gating

This model is a derivative of MedGemma and remains governed by the
**Health AI Developer Foundations (HAI-DEF) terms of use** — it is **not**
Apache-2.0. This repository preserves the same gated access as the source
model: you must review and accept the HAI-DEF terms before downloading.
Source model: [google/medgemma-1.5-4b-it](https://huggingface.co/google/medgemma-1.5-4b-it).

## Files

| File | Variant | Size (bytes) | Size |
|---|---|---|---|
| `medgemma-1.5-4b-it_q4_block32_ekv2048.litertlm` | Text-only, int4 (block-32) weights | 2,583,871,056 | 2.41 GiB |
| `medgemma-1.5-4b-it_q4_block32_vision_ekv2048.litertlm` | Text + SigLIP vision encoder (int8 weight-only) | 3,023,069,488 | 2.82 GiB |

![On-disk size: bf16 source vs int4 .litertlm variants](size_comparison.png)

## Conversion details

- **Source model:** `google/medgemma-1.5-4b-it` (Gemma 3–based, `Gemma3ForConditionalGeneration`;
  text decoder: 34 layers, hidden 2560, vocab 262,208, 128K context, sliding window 1024;
  vision: SigLIP, 27 layers, 896×896)
- **Tools:** `litert-torch-nightly v0.10.0.dev20260711`, `litert-lm v0.14.0`
  (installed via `uv tool install`), Python 3.11 (tool venv)
- **Text-only export command:**

```bash
litert-torch export_hf \
  --model=google/medgemma-1.5-4b-it \
  --output_dir=/tmp/medgemma15_4b_text \
  --externalize_embedder \
  --quantization_recipe=dynamic_wi4b32_afp32 \
  --prefill_lengths=32,128,512,1024 \
  --cache_length=2048
```

- **Quantization:** `dynamic_wi4b32_afp32` — int4 weights, block size 32,
  dynamic-range fp32 activations. Embedder externalized and quantized.
- **KV cache:** exported with `cache_length=2048` (ekv2048) to bound runtime
  memory on low-RAM devices; prefill signatures at 32/128/512/1024 tokens.
- **Chat template:** the source model's Jinja chat template is bundled in the
  `.litertlm`; `litert-lm` applies it automatically. No template override was
  needed for coherent output.
- **Multimodal variant:** exported with
  `--task=image_text_to_text --vision_encoder_quantization_recipe=weight_only_wi8_afp32`
  (int8 weight-only for the SigLIP tower, per exporter guidance for CPU
  execution of conv ops). Adds the SigLIP-400M vision encoder (27 layers,
  896×896 input, 256 image tokens) plus a small vision adapter (2.87 MiB).
  **Size increase: +0.41 GiB (2.41 → 2.82 GiB), ~17%.**

  Full multimodal export command:

  ```bash
  litert-torch export_hf \
    --model=google/medgemma-1.5-4b-it \
    --output_dir=/tmp/medgemma15_4b_mm \
    --task=image_text_to_text \
    --externalize_embedder \
    --quantization_recipe=dynamic_wi4b32_afp32 \
    --vision_encoder_quantization_recipe=weight_only_wi8_afp32 \
    --prefill_lengths=32,128,512,1024 \
    --cache_length=2048
  ```

  > **GPU/NPU realistically needed for the multimodal variant on low-end
  > phones.** The SigLIP encoder is convolution/matmul-heavy and turns each
  > 896×896 image into 256 tokens that then prefill the decoder. On a
  > CPU-only low-end device this makes time-to-first-token for an image query
  > painfully long; the exporter itself distinguishes CPU vs GPU vision recipes
  > for this reason. Pass `--vision-backend gpu` (or NPU where available) and
  > keep text-only queries on CPU. If the target fleet is low-RAM / CPU-only,
  > **ship the text-only variant and treat multimodal as an opt-in for
  > capable devices.**

  Multimodal smoke test (CPU vision backend) passed: prompted with a test
  image and *"Describe what you see in this image,"* the model returned an
  accurate description of the image contents — the vision encoder → decoder
  path works end-to-end in the `.litertlm`.

## Quality verification

Sanity check (text-only artifact, CPU backend):
prompt *"How do you differentiate bacterial from viral pneumonia?"* produced a
coherent, well-structured, clinically reasonable answer with correct Gemma
chat formatting and clean termination.

### Eval parity (ai-edge-eval 0.0.1, lm-eval harness)

Task: `mmlu_clinical_knowledge_generative` (MMLU clinical knowledge, generative
protocol — the model writes the answer; chat template applied on both sides),
first 50 questions.

![Eval parity: int4 on-device vs bf16 source, both 0.58](eval_parity.png)

| Model | Runner / backend | Metric | Score |
|---|---|---|---|
| `.litertlm` int4 (this repo) | litert-lm, CPU | acc (letter extraction) | **0.58 (29/50)** |
| `google/medgemma-1.5-4b-it` bf16 | HF transformers, CUDA | acc (letter extraction) | 0.58 (29/50) |
| `google/medgemma-1.5-4b-it` bf16 (reference, classic MC loglikelihood, no chat template) | HF transformers, CUDA | acc | 0.54 |

**Result: the int4 on-device model matched the bf16 source model exactly on this
50-question set (0.58 vs 0.58).** The int4 run produced 0 server errors and 5
unparseable answers (vs 4 for bf16) — i.e. essentially identical behavior. This
is a smoke-test level of parity, not a guarantee; run a larger, task-specific
eval before deployment.

The generative task's stock `get_response` filter does not parse MedGemma's
prose answers ("The correct answer is **B**…"), so accuracy was computed from
the harness-recorded sample outputs with an identical answer-letter extractor
for both models (unparseable responses counted as wrong).

> Note: 50-question samples give a coarse parity signal (±~7 pp stderr), not a
> leaderboard number. Validate on your own task before deployment.

### Benchmark (indicative only — server CPU, not a phone)

`litert-lm benchmark`, CPU backend, AMD EPYC 7K62 (shared cloud slice),
prefill 256 / decode 128 tokens:

| Metric | Value |
|---|---|
| Prefill speed | 12.04 tokens/s |
| Decode speed | 5.74 tokens/s |
| Engine init time | 19.9 s |
| Time to first token | 21.4 s |

Expect different (often better) numbers on modern Arm big cores with XNNPack;
measure on your target devices. ~2.6 GB of weights must fit in memory/mmap:
devices with <4 GB free RAM will struggle; the text-only variant is the right
choice for low-RAM phones.

## Running it

```bash
# after accepting the HAI-DEF terms on this repo's page
litert-lm run --from-huggingface-repo litert-community/MedGemma-1.5-4B-IT \
  medgemma-1.5-4b-it_q4_block32_ekv2048.litertlm \
  --prompt "How do you differentiate bacterial from viral pneumonia?"
```

On Android, use the LiteRT-LM engine (or MediaPipe LLM Inference API) and load
the `.litertlm` from local storage. GPU/NPU backends help the multimodal
variant substantially; the text-only variant runs acceptably on CPU.

## Intended use & limitations

- Offline health question answering, patient education, clinical decision
  *support* — always with a human professional in the loop.
- Single-turn prompts; the model is not optimized for multi-turn chat.
- English-centric training; validate for other languages.
- int4 quantization can shift factual recall and calibration relative to the
  bf16 source model — the parity eval above is a smoke test, not a clearance.

## Citation

Source model and documentation: [MedGemma on HAI-DEF](https://developers.google.com/health-ai-developer-foundations/medgemma)
Converted and packaged with [LiteRT-Torch](https://github.com/google-ai-edge/LiteRT-Torch) / [LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM).