skjortan/MiniCPM5-1B-ONNX

MiniCPM5-1B ONNX (web / transformers.js)

openbmb/MiniCPM5-1B exported to ONNX for in-browser inference with transformers.js on WebGPU.

• onnx/model_q4f16.onnx — 4-bit MatMuls + fp16 (888 MB), ~30 tok/s on WebGPU. Use dtype: "q4f16".

import { AutoTokenizer, AutoModelForCausalLM } from "@huggingface/transformers";
const tokenizer = await AutoTokenizer.from_pretrained("skjortan/MiniCPM5-1B-ONNX");
const model = await AutoModelForCausalLM.from_pretrained("skjortan/MiniCPM5-1B-ONNX",
  { device: "webgpu", dtype: "q4f16" });

Exported with optimum-cli export onnx --task text-generation-with-past, quantized with onnxruntime MatMulNBitsQuantizer (block 32) and converted to fp16 after layernorm fusion. Chat template (with enable_thinking switch and native XML tool calling) is inlined in tokenizer_config.json. KV cache I/O is fp32 (transformers_js_config.kv_cache_dtype).

Built for local-web-llm-bot — a fully in-browser voice assistant (VAD + STT + LLM + TTS, all ONNX on WebGPU).