Instructions to use Elsephire/Qwen3.5-0.8B-vocabulary-trimming with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Elsephire/Qwen3.5-0.8B-vocabulary-trimming with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Elsephire/Qwen3.5-0.8B-vocabulary-trimming")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Elsephire/Qwen3.5-0.8B-vocabulary-trimming")
model = AutoModelForMultimodalLM.from_pretrained("Elsephire/Qwen3.5-0.8B-vocabulary-trimming")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Elsephire/Qwen3.5-0.8B-vocabulary-trimming with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Elsephire/Qwen3.5-0.8B-vocabulary-trimming"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Elsephire/Qwen3.5-0.8B-vocabulary-trimming",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Elsephire/Qwen3.5-0.8B-vocabulary-trimming

SGLang

How to use Elsephire/Qwen3.5-0.8B-vocabulary-trimming with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Elsephire/Qwen3.5-0.8B-vocabulary-trimming" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Elsephire/Qwen3.5-0.8B-vocabulary-trimming",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Elsephire/Qwen3.5-0.8B-vocabulary-trimming" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Elsephire/Qwen3.5-0.8B-vocabulary-trimming",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Elsephire/Qwen3.5-0.8B-vocabulary-trimming with Docker Model Runner:
```
docker model run hf.co/Elsephire/Qwen3.5-0.8B-vocabulary-trimming
```

Qwen3.5-0.8B-vocabulary-trimming / README.md

Elsephire

Update README.md

e2df296 verified 17 days ago

6.81 kB

library_name: transformers
license: mit
license_link: https://huggingface.co/Qwen/Qwen3.5-0.8B/blob/main/LICENSE
pipeline_tag: image-text-to-text
base_model:
  - Qwen/Qwen3.5-0.8B
language:
  - en
  - fr
  - de
  - es
  - it
  - pl
  - nl
  - ro
  - hu
  - el
tags:
  - text-generation-inference

Qwen3.5-0.8B-Trimmed

A vocabulary-trimmed version of Qwen3.5-0.8B with 41.3% fewer tokens, keeping only Latin (EN, FR, DE, ES, IT, PT, NL, PL, RO, HU) and Greek scripts while preserving all original capabilities in those languages.

📊 Trimming Statistics

Metric	Original	Trimmed	Gain
Vocabulary size	248,077 tokens	145,572 tokens	−41.3%
File size	3,876.2 MB	3,475.8 MB	−400.4 MB (10.33%)
Parameters	2,032.2 M	2,032.2 M	unchanged
Architecture	Dense	Dense	unchanged

Original vocabulary distribution by script

Script / Language	Tokens	% of vocabulary	Status
Latin (EN, FR, DE, ES, IT, PT, NL, PL, RO, HU)	144,029	58.1%	✅ Kept
Chinese / Japanese / Korean	65,722	26.5%	❌ Removed
Cyrillic (RU, UK, BG, SR, MK)	18,580	7.5%	❌ Removed
Arabic	8,817	3.6%	❌ Removed
Thai	5,741	2.3%	❌ Removed
Greek	1,543	0.6%	✅ Kept
Devanagari (Hindi, Nepali, Marathi)	959	0.4%	❌ Removed
Bengali	531	0.2%	❌ Removed
Hebrew	520	0.2%	❌ Removed
Tamil	268	0.1%	❌ Removed
Malayalam	205	0.1%	❌ Removed
Telugu	188	0.1%	❌ Removed
Georgian	186	0.1%	❌ Removed
Burmese	147	0.1%	❌ Removed
Kannada	144	0.1%	❌ Removed
Gujarati	116	0.0%	❌ Removed
Armenian	88	0.0%	❌ Removed
Khmer	79	0.0%	❌ Removed
Sinhala	77	0.0%	❌ Removed
Gurmukhi (Punjabi)	65	0.0%	❌ Removed
Lao	37	0.0%	❌ Removed
Ethiopic (Ge'ez)	25	0.0%	❌ Removed
Tibetan	10	0.0%	❌ Removed

🎯 Why Trim?

Eliminating random Asian characters

One of the most common issues with general multilingual models is the sporadic appearance of Chinese, Japanese, or Korean characters in outputs meant for Latin-script languages — even when the prompt has nothing to do with those languages. This happens because the model occasionally explores parts of the vocabulary that are never actively used for the target task. By permanently removing those tokens, this issue is resolved at a structural level: the model simply cannot generate them anymore, as they no longer exist in its output space.

Faster generation and prompt processing

Reducing the vocabulary size has a direct and measurable impact on speed:

Prompt processing (prefill) — The embedding matrix is smaller, which reduces load time and memory access during the prefill phase.
Token-by-token generation (decoding) — The search for the best token in the output distribution (softmax + argmax) operates over a 41.3% smaller space. This translates into faster decoding, especially on hardware where the softmax computation is the bottleneck.

Reduced memory footprint

Storage — The model file is 400.4 MB smaller (10.33% reduction).
RAM / VRAM — The embedding matrix (lm_head + embed_tokens) occupies proportionally less memory. On systems with tight memory constraints, this reduction can make it possible to run the model where it previously wouldn't fit, or free up memory for other processes.

Potential performance improvement (1–3%)

Trimming the vocabulary may lead to a slight improvement in overall model performance on the retained languages. The hypothesis is that reducing the output space concentrates probability mass more effectively on relevant tokens, reducing the statistical noise introduced by thousands of tokens that are never used in the task context. Gains in the range of 1–3% may be observed on Latin-language benchmarks (perplexity, answer accuracy, coherence).

🔧 Methodology

Trimming was performed using the [vocab-trimmer] (release soon) tool, which follows this pipeline:

Vocabulary analysis — Each token in the original tokenizer is classified by Unicode script (Latin, CJK, Cyrillic, Arabic, etc.).
Selective filtering — Only Latin and Greek scripts are kept. Special tokens (EOS, BOS, PAD, etc.) and multimodal tokens are always preserved.
ID remapping — A continuous old_id → new_id mapping is built to compact the vocabulary with no gaps.
Weight slicing — Only the rows of the embedding matrices corresponding to retained tokens are extracted. The lm_head and embed_tokens layers are recalibrated.
Configuration reconstruction — tokenizer_config.json, tokenizer.json, config.json, and the vocabulary are rebuilt with the new IDs. Chat templates and special tokens are preserved verbatim.
Verification — Consistency between tokenizer and weights is validated (special token IDs < vocab_size, functional chat template, etc.).

⚠️ Limitations

Unsupported languages — This model cannot generate text in Chinese, Japanese, Korean, Russian, Arabic, Thai, Hindi, Bengali, Hebrew, or any other language whose tokens were removed. Prompts in these languages will not be understood correctly.
Supported languages — Functional languages include English, French, German, Spanish, Italian, Portuguese, Dutch, Polish, Romanian, Hungarian, and Greek.
Code & Math — Programming and mathematical tokens are preserved (they primarily use Latin encoding). No loss of capability is expected in these domains.
Emojis & Symbols — Common Unicode symbols and emojis are retained if they fall within the selected scripts.

💡 Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "your-namespace/Qwen3.5-0.8B-Trimmed"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

inputs = tokenizer("Explain quantum computing in simple terms.", return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

📝 License

This model is derived from Qwen/Qwen3.5-0.8B and follows its original license. The trimming tool [vocab-trimmer]release soon is available under the MIT License.

🔗 References

Original model : Qwen/Qwen3.5-0.8B
Trimming tool : [vocab-trimmer] soon
Qwen Team : Alibaba Cloud – Qwen Team