Instructions to use openbmb/MiniCPM5-1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use openbmb/MiniCPM5-1B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="openbmb/MiniCPM5-1B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("openbmb/MiniCPM5-1B")
model = AutoModelForCausalLM.from_pretrained("openbmb/MiniCPM5-1B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use openbmb/MiniCPM5-1B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "openbmb/MiniCPM5-1B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openbmb/MiniCPM5-1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/openbmb/MiniCPM5-1B

SGLang

How to use openbmb/MiniCPM5-1B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "openbmb/MiniCPM5-1B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openbmb/MiniCPM5-1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "openbmb/MiniCPM5-1B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openbmb/MiniCPM5-1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use openbmb/MiniCPM5-1B with Docker Model Runner:
```
docker model run hf.co/openbmb/MiniCPM5-1B
```

OpenBMB-LLM commited on 3 days ago

Commit

5155e74

verified ·

1 Parent(s): 68a0509

Fix config _name_or_path to openbmb/MiniCPM5-1B

Browse files

Files changed (2) hide show

.ms_upload_cache +1 -0
config.json +1 -1

.ms_upload_cache ADDED Viewed

	@@ -0,0 +1 @@

+ {"version": 3, "repo_id": "OpenBMB/MiniCPM5-1B", "files": {"config.json|1779420286.5717523|729": {"hash": "3737ad159bfe7826f6ebc2f802aaf60546aa5691ca9cd07932520847e4c09558", "size": 729, "status": "c"}, "generation_config.json|1779088120.0|202": {"hash": "aac691a24af6d9efb4d5e44562bb523781df44ea6182bdae1a667131c09c4d3a", "size": 202, "status": "c"}, "chat_template.jinja|1779087697.0|9062": {"hash": "7451a05cf1e28a79d97d7c0bc951028c0b1915119bf9046acd06a0e3d931f47c", "size": 9062, "status": "c"}, "model-00000-of-00001.safetensors|1779087697.0|2161290912": {"hash": "7ab8fd86563125929be78aeec8cb3969c7ed2ead3be1ab9d3ec0a9fa69c8660d", "size": 2161290912, "status": "c"}, "model.safetensors.index.json|1779087691.0|18004": {"hash": "162add042e75abc3d571c4a8679523fa4f1ffc55d1fea25fc6658a19d6e957ee", "size": 18004, "status": "c"}, "special_tokens_map.json|1779087697.0|551": {"hash": "82d96d7a9e6ced037f12394b7ea6a5b02e6ca87e0d11edaa8d60d9be857ce7db", "size": 551, "status": "c"}, "tokenizer.json|1779087697.0|9894271": {"hash": "3e065a558a034185fe299917b398685c1facd0169a9eea1e629eb30c171fed81", "size": 9894271, "status": "c"}, "tokenizer_config.json|1779420286.573849|94416": {"hash": "094efb3cf1ff412284cc5945fc99dff58673a912760d04483a04aa1c716f66fd", "size": 94416, "status": "c"}, "config.json|1779509027.7066157|728": {"hash": "5b702f18b5d79e36e2f6858c4c4086c4c51547433780e315feb920cf607cf798", "size": 728, "status": "c"}, "generation_config.json|1779516522.3885784|213": {"hash": "92afd6424501426eddcf7e1542f013d19e5987544977b4ee7bd26359bd5fd2ab", "size": 213, "status": "c"}, "config.json|1779595199.6800644|726": {"hash": "6a6509b646cb3169616c5ffc3196e7ccaf9d4d6bc17b266581d241a31c217714", "size": 726, "status": "c"}}}

config.json CHANGED Viewed

@@ -1,5 +1,5 @@
 {
-    "_name_or_path": "openbmb/MiniCPM5-0.9B",
     "architectures": [
         "LlamaForCausalLM"
     ],

 {
+    "_name_or_path": "openbmb/MiniCPM5-1B",
     "architectures": [
         "LlamaForCausalLM"
     ],