Qwen3.5-0.8B-English-Trimmed

A vocabulary-trimmed derivative of Qwen/Qwen3.5-0.8B, built for deployments that only need English + code, not all 201 languages the base tokenizer supports.

This model changes only the token embedding / output (lm_head) matrix (tied in the base model). Every other weight — the full vision tower and all 24 text-decoder layers — is copied unmodified from the base model. No fine-tuning was performed.

Modified from the original. This is a derivative of Qwen/Qwen3.5-0.8B (Apache 2.0). See Attribution below.

Update (2026-07-29): an earlier version of this checkpoint was accidentally exported as a text-only model — it was built by loading the base model via AutoModelForCausalLM, which silently loads only the inner text decoder and drops the vision tower entirely, even though the model card claimed vision was preserved. That was wrong. This version is rebuilt from the full Qwen3_5ForConditionalGeneration class (via AutoModelForImageTextToText + AutoProcessor), so the vision tower and multimodal wiring are genuinely intact this time. Thanks to @danielnobbe for flagging it.

What changed

	Original	This model
Vocabulary size	248,320	48,704	5.10x smaller
Embedding/lm_head params	254.3M	49.9M	5.10x smaller
Vision tower	Unchanged	Unchanged	untouched
Text decoder layers	Unchanged	Unchanged	untouched

The vocabulary was reduced by mining real token-frequency statistics from a diverse English + code corpus and keeping only the tokens actually needed, rounded up to a multiple of 64 for GPU alignment. Only model.language_model.embed_tokens.weight and lm_head.weight (tied) were sliced to the kept token ids; the vision tower (model.visual.*) and every text-decoder layer are byte-identical to the base model. The tokenizer's BPE merge rules were rebuilt to match — since the base tokenizer is byte-level BPE, dropped tokens don't cause hard failures; text that would have used a removed token falls back to more, smaller sub-word/byte pieces that remain in the vocabulary.

Verified working

• Text generation via transformers (AutoModelForImageTextToText) — confirmed.
• Image + text generation via transformers — confirmed, real vision forward pass, no shape/config errors.
• mlx_lm (Apple Silicon, text-only) — confirmed, ~150-190 tok/s on an M4 Pro.
• mlx_vlm (Apple Silicon, image+text) — loads and runs correctly (confirmed no crash, correct architecture dispatch). Note: on this small 0.8B model, both mlx_vlm and plain transformers occasionally produce degenerate/empty output after an empty <think></think> block on short prompts -- this reproduces identically on the unmodified base model, so it's a pre-existing quirk of this checkpoint's thinking-mode chat template / small model size, not something introduced by trimming.

Intended use / limitations

• Good for: English text generation and chat, image understanding, and code generation/completion across Python, JavaScript, Java, C++, Go, Rust, Bash, and SQL.
• Not good for: any non-English language. Non-English text still produces output (byte-level fallback prevents hard failures) but quality degrades substantially (see benchmarks below).
• No fine-tuning was applied. All numbers below are zero-shot.

Benchmarks (text-only; from the original vocab-trimming validation)

Measured on an Apple M4 Pro (24GB unified memory), bf16, via PyTorch MPS backend for the text-only comparison. fla/causal_conv1d fast kernels are CUDA-only and unavailable on this hardware, so the Gated DeltaNet backbone ran via the transformers PyTorch fallback path for both models — this makes the absolute tok/s numbers conservative.

Quality (bits-per-byte, zero-shot, no fine-tuning — lower is better, tokenization-invariant)

Eval set	Base model	This model	Change
English (WikiText-103, held-out)	0.937	1.070	+14.2%
English (Dolly-15k instructions, held-out)	0.757	0.915	+20.9%
English (Frankenstein, held-out)	1.046	1.150	+9.9%
Code (Python, held-out)	0.729	0.959	+31.6%
Code (JavaScript, held-out)	0.718	0.865	+20.4%
French (out-of-domain)	1.225	1.613	+31.6%
Spanish (out-of-domain)	1.151	1.478	+28.4%

Decode speed (bf16, MPS, greedy decoding, 40 tokens, text-only)

Prompt	Base tok/s	This model tok/s	Speedup
English (chat)	46.76	51.65	+10.5%
English (prose)	48.60	52.49	+8.0%
Code (Python)	51.27	53.13	+3.6%
Code (JavaScript)	45.92	53.09	+15.6%

Methodology

1. Mined token frequency from ~10.7M characters of real data: WikiText-103 (encyclopedic English), Databricks Dolly-15k (instruction/chat English), 3 public-domain novels, and idiomatic code samples across 8 languages, plus a curated emoji set.
2. Always-keep set, independent of frequency: all special/chat/multimodal tokens, all 256 primitive byte-level tokens (required for byte-level BPE's no-hard-OOV guarantee), and a curated safety-net list of ~40 common cross-language keywords/operators.
3. Coverage-based selection: remaining tokens ranked by frequency, kept until reaching 99.9% of mined token occurrences, rounded up to a multiple of 64.
4. Loaded the full multimodal model (AutoModelForImageTextToText, class Qwen3_5ForConditionalGeneration) and sliced only the nested model.language_model.embed_tokens.weight / lm_head.weight to the kept token ids. Vision tower and all decoder layers untouched.
5. Rebuilt the tokenizer inside a full AutoProcessor (image processor + video processor + tokenizer + chat template all bundled correctly).
6. Verified: round-trip encode/decode correctness (including multi-byte UTF-8 like emoji), a real image+text generation forward pass, held-out coverage across 7 datasets never used in mining, and a hand-checked list of ~100 common English words and cross-language code keywords/operators.

Usage

from transformers import AutoModelForImageTextToText, AutoProcessor
import torch

model_id = "sahilchachra/Qwen3.5-0.8B-English-trimmed"
proc = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype=torch.bfloat16)

messages = [{"role": "user", "content": [
    {"type": "image"},
    {"type": "text", "text": "Describe this image."}
]}]
prompt = proc.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = proc(text=[prompt], images=[your_pil_image], return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=200)
print(proc.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0])

Text-only usage (no image) works the same way, or via mlx_lm/mlx_vlm on Apple Silicon.

Attribution & License

This model is a derivative of Qwen/Qwen3.5-0.8B, licensed under Apache License 2.0. In accordance with the license:

• This is a modified version of the original work — only the token embedding / lm_head matrix and tokenizer vocabulary were changed; all other weights, including the full vision tower, are unmodified copies of the original.
• Distributed under the same Apache License 2.0.
• All original copyright and attribution notices are retained.

See the Apache 2.0 license text for full terms.

See also: sahilchachra/Qwen3.5-9B-English-trimmed — the larger sibling release.