Qwen3.6 35B-A3B OpenVINO INT4

This repository contains an unofficial OpenVINO™ IR conversion of Qwen/Qwen3.6-35B-A3B with INT4 weight compression.

The model is intended for local vision-language generation using OpenVINO and Optimum Intel. It can take an image plus a text prompt and generate a natural-language response.

Original model

• Original model: Qwen/Qwen3.6-35B-A3B
• Model family: Qwen3.6
• Architecture: sparse Mixture-of-Experts vision-language model
• Converted format: OpenVINO IR
• Weight format: INT4
• Task: image-text-to-text / vision-language generation

This is an unofficial converted model repository. Please refer to the original model card for full model details, license terms, intended use, safety notes, and limitations.

Model summary

Qwen3.6-35B-A3B is a sparse Mixture-of-Experts model from the Qwen3.6 family. The upstream model is distributed on Hugging Face and is intended for multimodal use cases including image understanding and text generation.

This OpenVINO version is designed for local inference on Intel hardware through the Optimum Intel OpenVINO runtime path.

Conversion

This model was converted with Optimum Intel using the OpenVINO export path.

optimum-cli export openvino \
  --model Qwen/Qwen3.6-35B-A3B \
  --weight-format int4 \
  --trust-remote-code \
  ov_Qwen3.6-35B-A3B_int4
  
Runtime note

This model currently uses the Optimum Intel OVModelForVisualCausalLM path for inference.

The tested environment used a Qwen3.5/Qwen3.6 support branch of Optimum Intel and nightly OpenVINO packages:
pip install -U git+https://github.com/rkazants/optimum-intel.git@support_qwen3_5
pip install --pre -U openvino openvino-tokenizers nncf --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
pip install transformers==5.2.0
pip install requests torchvision opencv-python
OpenVINO Model Server compatibility is not claimed for this model. In local testing, OVMS 2026.1 rejected this model type during VLM initialization with an unsupported model-type error, while direct Optimum Intel OpenVINO inference worked.

Quantization
Weight format: INT4
Export tool: Optimum Intel
Runtime: Optimum Intel OpenVINO
Model format: OpenVINO IR

INT4 compression is intended to reduce model size and memory usage compared with higher precision weights. As with any converted and quantized model, quality and numerical behavior may differ from the original model and should be validated for your use case.
Usage
Run from inside the model directory
python examples/run_qwen36_vlm_openvino.py \
  --model-dir . \
  --device CPU \
  --max-new-tokens 64 \
  --prompt "What animal is on the candy? Answer in one sentence."
  
Notes
This is a vision-language model, not a text-only model.
Depending on package versions, you may see:
The following generation flags are not valid and may be ignored: ['top_p', 'top_k']
The demo script uses deterministic generation with do_sample=False.
OpenVINO Model Server compatibility is not claimed unless separately validated.
Limitations

This repository inherits the limitations of the original Qwen/Qwen3.6-35B-A3B model. Additional differences may arise from OpenVINO conversion, INT4 compression, and runtime configuration.

Attribution

This is an unofficial OpenVINO conversion of the original Qwen3.6 model. All rights to the original model, training, and licensing remain with the original authors.