README.md

---
license: apache-2.0
language:
  - en
  - zh
base_model:
  - openbmb/MiniCPM5-1B
pipeline_tag: text-generation
library_name: litert
tags:
  - minicpm
  - minicpm5
  - litert
  - tflite
  - on-device
  - edge-ai
---

# MiniCPM5-1B (LiteRT-LM)

This repository hosts the [**LiteRT-LM**](https://ai.google.dev/edge/litert-lm) (LiteRT formerly known as TensorFlow Lite) version of **MiniCPM5-1B**, optimized for fully on-device inference on mobile and edge hardware.

---

## Available Models

* **`minicpm_dynamic_wi8_afp32_gpu_opt.litertlm`**: This model features dynamic weight-only INT8 quantization (wi8) with FP32 activations (afp32), heavily optimized for GPU execution.
* **`minicpm_wi4b32_wi8_afp32.litertlm`**: This model features mixed INT4-block32(linear)/INT8(embed and lmhead) quantization (wi4b32_wi8) with FP32 activations (afp32).

## Quantization Benchmark

The table below compares the accuracy of the FP baseline and the two LiteRT-LM quantized models on the same evaluation benchmark. Scores are reported as percentages.

| Category | FP (bf16) | W8 | W4 |
| --- | --- | --- | --- |
| Overall | 66.16 | 63.78 | 60.91 |
| STEM | 58.07 | 56.35 | 53.36 |
| Social Science | 77.91 | 73.65 | 70.87 |
| Humanities | 67.65 | 66.68 | 64.35 |
| Other | 68.71 | 65.40 | 62.14 |
| Hard | 47.08 | 43.79 | 44.23 |

FP (bf16) is the full-precision baseline. W8 uses dynamic weight-only INT8 quantization (`wi8`) with FP32 activations (`afp32`). W4 uses mixed INT4-block32 (linear) / INT8 (embed and lmhead) quantization (`wi4b32_wi8`) with FP32 activations (`afp32`). Thinking mode has been turned off to reduce context usage. All evaluations use a temperature of 0.6, top-p of 0.95, and a random seed of 0.

## What is MiniCPM?

**MiniCPM5-1B** is the first model in the **MiniCPM5** series from [OpenBMB](https://huggingface.co/openbmb). It is a dense **1B-parameter** Transformer built specifically for **on-device, local, and resource-constrained deployment**, while reaching **1B-class open-source SOTA** in its size class.

### Highlights

- 🏆 **1B-class open-source SOTA** — strongest in tool use, code generation, and difficult reasoning among comparable open models.
- 🧠 **Hybrid Reasoning** — a single checkpoint serves as both a fast assistant and a deliberate reasoner via a built-in `<think>` template (`enable_thinking`).
- 📏 **Long context** — native **131,072**-token context length.
- 📱 **Built for the edge** — compact footprint designed for local assistants, coding agents, and tool-use workflows.

### Model Information

| Item | Value |
| --- | --- |
| Type | Causal Language Model |
| Architecture | Standard `LlamaForCausalLM` |
| Parameters | 1,080,632,832 (~1B) |
| Non-Embedding Parameters | 679,552,512 |
| Layers | 24 |
| Attention Heads (GQA) | 16 (Q) / 2 (KV) |
| Context Length | 131,072 |

---

## Use the model

### Edge Gallery App (Android)

1. **Get the App**: Install the [app](https://play.google.com/store/apps/details?id=com.google.ai.edge.gallery&pli=1) from Google Play or download the latest APK from the [GitHub releases page](https://github.com/google-ai-edge/gallery/releases).
2. **Importing the Model**: Navigate to the **Model manager** within the app and click the **"+" (plus)** icon in the bottom-right corner. Two options will appear:
   * **Import from HF (Recommended)**: Select this option, and a dialog box will appear showing an example Hugging Face model URL. Enter the HF link for the desired `.litertlm` model and click submit. The model will then appear in your list, and you can proceed to download it (a Hugging Face account login is required).
   * **From local model file**: First, download the `.litertlm` model directly to your Android device, OR download it to your computer and push it via ADB (e.g., `adb push minicpm_dynamic_wi8_afp32_gpu_opt.litertlm /sdcard/Download/`). Then, select this option, choose the downloaded file from your storage, configure your preferred parameters, and tap **"Import"**.

For full details on importing models and other features, see the [Edge Gallery App Wiki](https://github.com/google-ai-edge/gallery/wiki).

To build the demo app from source, please follow the [instructions](https://github.com/google-ai-edge/gallery/blob/main/README.md) from the GitHub repository.

### Try It (Desktop/CLI)

Install `uv` and run the model directly from the LiteRT-LM command line:

```bash
uv tool install litert-lm
uvx litert-lm run --from-huggingface-repo=litert-community/MiniCPM5-1B minicpm_dynamic_wi8_afp32_gpu_opt.litertlm --prompt="What is the capital of France?"
```

## Links

- 🤗 Original model (BF16): [openbmb/MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B)
- 📦 GitHub: [OpenBMB/MiniCPM](https://github.com/OpenBMB/MiniCPM)
- 🛠️ LiteRT docs: [ai.google.dev/edge/litert](https://ai.google.dev/edge/litert)

---

## License

Released under the **Apache-2.0 License**, consistent with the upstream [openbmb/MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B).

## Citation

```bibtex
@article{minicpm4,
  title={MiniCPM4: Ultra-efficient LLMs on end devices},
  author={MiniCPM, Team},
  journal={arXiv preprint arXiv:2506.07900},
  year={2025}
}
```