Bonsai-8B `bonsai_q1_f32` for MLC/WebLLM

This repository contains an experimental MLC/WebLLM conversion of prism-ml/Bonsai-8B-unpacked. It is a browser-runtime artifact, not a new model, fine-tune, GGUF, MLX, or ONNX mirror.

The weights use the local bonsai_q1_f32 format: binary signs packed into uint32 words with one FP32 scale per 128-wide group. Linear layers, embeddings, and the final lm head are stored in this format.

Artifact Summary

Field	Value
Source checkpoint	`prism-ml/Bonsai-8B-unpacked`
Architecture	Qwen3-shaped decoder
MLC model type	`qwen3`
Quantization	`bonsai_q1_f32`
Conversation template	`qwen3_nothink`
Context window in config	`32768`
Prefill chunk in config	`2048`
Total parameters	8,188,548,096
Quantized parameter size	1.193 GB
Bits per parameter	1.251
Parameter shards	37
Artifact size	about 1.1 GB
WebGPU library	`libs/bonsai-8b-bonsai_q1_f32-webgpu.wasm`
WebGPU wasm size	8,205,907 bytes

Runtime Requirement

This artifact requires a Bonsai q1-capable MLC/WebLLM runtime. It is not expected to work with an unmodified upstream WebLLM release until the Bonsai q1 quantization/runtime support is upstreamed.

Use this repository when you control the WebLLM runtime and want to test browser-local Bonsai inference through WebGPU.

WebLLM App Config

const appConfig = {
  model_list: [
    {
      model: "https://huggingface.co/welcoma/Bonsai-8B-bonsai_q1_f32-MLC/resolve/main/",
      model_id: "Bonsai-8B-q1-MLC",
      model_lib:
        "https://huggingface.co/welcoma/Bonsai-8B-bonsai_q1_f32-MLC/resolve/main/libs/bonsai-8b-bonsai_q1_f32-webgpu.wasm",
      overrides: {
        context_window_size: 4096,
        prefill_chunk_size: 512,
      },
    },
  ],
};

The smaller override values above are intended for local browser smoke tests. Increase them only after measuring browser memory and cache behavior on the target device.

Validation

The 8B artifact was converted and WebGPU-compiled on the GCP MLC/WebLLM builder VM, not on a local laptop.

Source download path: /opt/gemma4-webllm/models/Bonsai-8B-unpacked
Artifact path: /opt/gemma4-webllm/dist/Bonsai-8B-bonsai_q1_f32-MLC
Conversion peak RAM: 9.188 GB on CPU
Weight conversion completed with 37 committed parameter shards
gen_config completed with Qwen3 tokenizer files copied into the artifact
compile --device webgpu generated the wasm library listed above
Compile estimate without KV cache: 2854.23 MB
Compile estimate with 4K KV cache: 4006.23 MB
Hugging Face round-trip check confirmed README, WebGPU wasm, and no accidental resolve/ mirror path.

Limitations

This is an experimental runtime artifact, not a general transformers model checkpoint.
Quality evaluation is limited to conversion/runtime smoke checks; no benchmark score is claimed by this repository.
Browser success depends on WebGPU support, available GPU memory, cache quota, and a compatible patched WebLLM runtime.
The ternary Bonsai family is not represented by this q1 format. Ternary models need a separate 2-bit/ternary MLC path.

Attribution

Original model by Prism ML:

MLC/WebLLM conversion by welcoma.

Downloads last month: 4

Model tree for welcoma/Bonsai-8B-bonsai_q1_f32-MLC

Base model

prism-ml/Bonsai-8B-unpacked

Quantized

(13)

this model

Bonsai-8B bonsai_q1_f32 for MLC/WebLLM