OPOD-Qwen3-Omni-30B-A3B

OPOD (On-Policy Omni Distillation) consolidates separate text, image, and audio teachers into a single omni-modal student. This model is the 30B-A3B student from the paper OPOD: On-Policy Omni Distillation, post-trained from Qwen/Qwen3-Omni-30B-A3B-Instruct.

It reaches an overall average of 70.8 across twelve text / vision / audio / omni-modal benchmarks (+3.8 over the base model), and beats the base model and pooled-data GRPO on all 12 benchmarks. The teachers are discarded after training — inference uses one model, with no teacher-ensemble latency or memory.

📄 Paper: arXiv:2607.20918
💻 Code: github.com/VincentZhao2002/OPOD
🤗 Other scales: OPOD-Qwen2.5-Omni-7B · OPOD-Qwen2.5-Omni-3B

Method

Post-training on pooled multimodal data often fails to preserve the strengths of modality specialists, and standard on-policy distillation does not extend cleanly to several teachers: their guidance conflicts on a shared backbone, and symmetric distribution matching imposes a teacher ceiling. OPOD addresses this with three components:

Component	Description
One-Sided Teacher Guidance	The token constraint applies only where the routed teacher assigns a higher likelihood than the student, so the student is never pulled back on tokens it already handles better.
Adaptive Modality Control	Each modality gets its own constraint budget $\epsilon_m$ and dual weight $\beta_m$, calibrated from a 10-step warm-up (here: 0.020 / 0.191 / 0.254 for text / image / audio — a 12.7× range).
Verification Reward	The routed teacher also verifies the trajectory: correctness-gated answer confidence plus ungated reasoning gain.

Each student rollout is routed by its input modality to the matching teacher (each obtained by applying GRPO to that modality's data from the same base model).

Results

Accuracy (%) on the paper's twelve benchmarks. Bold = best in the block; the three specialist teachers are shown as references (they are not deployable as one model).

Model	AIME25	AIME26	HQA	MMLU-Pro	GPQA	MMMU	MathV.	ChartQA	A-OKVQA	MMAU	AVQA	OmniBench	Avg.
Text teacher	47.9	57.7	41.5	76.8	65.8	67.7	76.3	86.5	87.7	76.0	79.8	61.1	68.7
Image teacher	47.9	52.1	27.3	77.3	64.8	68.2	77.3	87.8	88.6	75.7	80.8	61.5	67.4
Audio teacher	52.1	59.2	27.3	76.9	64.0	67.4	76.4	87.2	88.9	79.1	84.4	55.6	68.2
Base (Qwen3-Omni-30B-A3B-Instruct)	49.2	52.9	26.8	77.0	65.3	67.9	76.3	85.8	87.6	75.6	79.3	60.3	67.0
GRPO (pooled data)	49.2	53.8	28.0	76.7	65.3	67.9	76.7	86.2	87.9	76.2	80.5	59.7	67.3
Native OPD	51.7	55.8	41.1	77.5	64.9	68.4	76.4	73.4	88.0	75.8	81.2	60.3	67.9
ExOPD	50.0	64.2	41.0	77.1	63.5	68.0	76.5	87.2	88.7	65.1	83.2	58.8	68.6
OPOD (this model)	55.0	63.3	41.1	78.2	68.7	68.7	77.6	87.6	89.1	76.6	82.6	61.1	70.8

Highlights:

+3.8 over base, +3.5 over pooled GRPO, +2.9 over native OPD, +2.2 over ExOPD, and +2.1 over the strongest specialist teacher.
Wins over base and GRPO on all 12 benchmarks; ranks 1st or 2nd on 11 of 12 even when the specialist teachers are included in the comparison.
Beats a per-benchmark oracle over the three teachers (70.3) by 0.5 points with a single model.
Improves every modality group over base: text +7.0, vision +1.4, audio +2.2, omni +0.8.

Evaluation protocol. Accuracy, temperature 0.7, top-p 0.7, max prompt / response length 8,192 tokens, identical for all methods. Training prompts are disjoint from every benchmark (exact and normalized-string overlaps removed).

Usage

This model keeps the Qwen3-Omni architecture and processor, so it is a drop-in replacement for the base model.

Prompt format (important)

The model was RL-trained to reason inside <think></think> and put the final answer inside <answer></answer>. Use a system prompt of the same form, otherwise accuracy will drop and answers become harder to parse. The prompts used in the paper are in eval/prompts/system_prompts.py.

For image / audio / omni multiple-choice questions:

Please think about this question as if you were a human pondering deeply, carefully
considering BOTH the visual and the audio information before answering, engaging in an
internal dialogue using expressions such as let me think, wait, hmm, oh I see, or let's
break it down, including self-reflection or verification in the reasoning process,
providing the detailed reasoning between the <think> </think> tags, and finally giving
only the single option letter (e.g., A, B, C, D, etc.) as the final answer within the
<answer> </answer> tags.

For text problems:

You are a careful problem solver. Think step by step inside <think> </think> tags, then
output the final answer in the exact format requested at the end of the user prompt (do
not add any extra wrappers around the final answer).

transformers

import soundfile as sf
from transformers import Qwen3OmniMoeForConditionalGeneration, Qwen3OmniMoeProcessor

model_id = "Tung111/OPOD-Qwen3-Omni-30B-A3B"
model = Qwen3OmniMoeForConditionalGeneration.from_pretrained(
    model_id, dtype="auto", device_map="auto", attn_implementation="flash_attention_2",
)
processor = Qwen3OmniMoeProcessor.from_pretrained(model_id)

SYSTEM = (
    "You are a careful problem solver. Think step by step inside <think> </think> tags, "
    "then output the final answer in the exact format requested at the end of the user "
    "prompt (do not add any extra wrappers around the final answer)."
)

conversation = [
    {"role": "system", "content": [{"type": "text", "text": SYSTEM}]},
    {"role": "user", "content": [
        {"type": "image", "image": "https://example.com/chart.png"},
        {"type": "text", "text": "What is the highest value in the chart? "
                                 "Put the final answer in <answer></answer>."},
    ]},
]

text = processor.apply_chat_template(conversation, add_generation_prompt=True, tokenize=False)
inputs = processor(text=text, return_tensors="pt").to(model.device)

ids = model.generate(**inputs, max_new_tokens=8192, temperature=0.7, top_p=0.7, do_sample=True)
print(processor.batch_decode(ids[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0])

Use Qwen3OmniMoeProcessor's multimodal loading utilities (process_mm_info) for audio and video inputs, exactly as with the base model.

vLLM

vllm serve Tung111/OPOD-Qwen3-Omni-30B-A3B \
  --tensor-parallel-size 4 --max-model-len 16384 \
  --allowed-local-media-path / --trust-remote-code

Then query the OpenAI-compatible endpoint with the system prompts above.

Training details


Base model	Qwen3-Omni-30B-A3B-Instruct (MoE)
Teachers	3 × modality specialists (text / image / audio), each GRPO-trained from the same base
Training data	6,028 modality-balanced verifiable prompts (2,029 text / 2,000 image / 1,999 audio) from public sources
Hardware	32 × NVIDIA H20 (16 for the student, the rest hosting teacher servers)
Rollouts	8 samples per prompt, rollout batch 16, global batch 64
Lengths	8,192 max prompt, 8,192 max response
Verification reward	$w_A = w_B = 0.2$, clipping $C = 2.0$
Modality control	$\epsilon_{\min}=0.02$, 10-step warm-up, $\beta \in [0.1, 1.5]$
Regularization	reference KL kept throughout
Precision	bfloat16

Limitations

Post-training targets the thinker (text/reasoning) pathway. The talker / speech-generation components are inherited unchanged from the base model and were not optimized or evaluated.
Optimized for verifiable reasoning with the <think> / <answer> format; free-form chat, long-form generation, and speech output are outside the evaluated scope.
Single run per configuration with a fixed seed; no multi-seed variance is reported.
Inherits the base model's biases and knowledge cutoff. Not evaluated for safety alignment.

Citation

@article{zhao2026opod,
  title   = {OPOD: On-Policy Omni Distillation},
  author  = {Zhao, Tong and Hu, Yuyang and Zhu, Yutao and Li, Reed and
             Liang, Haijin and Shi, Haibo and Lu, Yu and Dou, Zhicheng},
  journal = {arXiv preprint arXiv:2607.20918},
  year    = {2026},
  url     = {https://arxiv.org/abs/2607.20918}
}