Add T-Search-FP8 model

Co-authored-by: alexm <alexm@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,39 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+assets/t-search-benchmarks.png filter=lfs diff=lfs merge=lfs -text
+assets/t-search-latency-quality.png filter=lfs diff=lfs merge=lfs -text
+assets/t-search-rl-training.png filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,183 @@
+---
+license: apache-2.0
+language:
+- en
+- ru
+pipeline_tag: text-generation
+library_name: transformers
+tags:
+- fp8
+- 35b-a3b
+- mixture-of-experts
+- moe
+- agent
+- search-agent
+- agentic-search
+- search
+- information-retrieval
+- retrieval
+- tool-use
+- function-calling
+- rag
+base_model:
+- t-tech/T-Search
+base_model_relation: quantized
+---
+
+# T-Search-FP8
+
+**🚨 T-Search-FP8 is designed for use with the [official T-Search harness](https://github.com/turbo-llm/t-search-harness). The source T-Search checkpoint did not show noticeable differences relative to the Qwen3.6-35B-A3B base model on general-purpose benchmarks.**
+
+**🚨 Users are advised to exercise caution and are responsible for any additional training and oversight required to ensure the model's responses meet acceptable ethical and safety standards. The responsibility for incorporating this model into industrial or commercial solutions lies entirely with those who choose to deploy it.**
+
+## **Highlights**
+
+We introduce **T-Search-FP8**, an agentic retriever built for difficult multi-step search in English and Russian. It plans and executes searches across multiple rounds with a configurable search budget.
+
+- **FP8 checkpoint:** block-wise quantization for lower-precision deployment.
+- **Agentic retrieval:** issues queries, inspects results, tracks coverage, carries compact evidence and search state across rounds, and returns a ranked chunk set.
+- **Retrieval quality:** outperforms the base model and larger open models on average across the evaluated retrieval benchmarks.
+- **Retriever robustness:** trained and evaluated with multiple retrievers, maintaining strong retrieval quality across configurations.
+
+![T-Search benchmark recall](https://huggingface.co/t-tech/T-Search/raw/main/assets/t-search-benchmarks.svg)
+
+## Description
+
+T-Search-FP8 is built on **Qwen3.6-35B-A3B** and trained on fully synthetic search tasks generated for the same harness used at inference time.
+
+The model is responsible for collecting evidence. Given a question and access to a corpus search backend, it formulates queries, reads retrieved snippets, decides which chunks are worth preserving, and returns a ranked evidence set. A downstream generator, reranker, or full-text fetcher can then consume that ranking.
+
+## ⛷️ T-Search Harness
+
+The [official T-Search harness](https://github.com/turbo-llm/t-search-harness) implements the inference protocol used for training and evaluation. The retriever operates in rounds. Within each round, it follows a ReAct loop with an approximately 32K-token context budget and three tools:
+
+- `search_corpus` searches the corpus and returns document chunks;
+- `save_and_advance` preserves important chunks and starts a new round;
+- `finalize_ranking` ends the search and returns a ranked evidence set.
+
+The agent sees the original question, previously saved chunks, and the current coverage state: which parts of the question are supported by evidence and which remain open. It chooses search queries, inspects retrieved chunks, and decides whether to continue searching, preserve state for another round, or finalize.
+
+At 75% context utilization, `search_corpus` is locked. The agent must either call `finalize_ranking` or use `save_and_advance` to build compact memory for the next round: preserved chunks with reasons, covered and open parts of the question, previous attempts, and a useful next step. Full tool history and intermediate noise do not cross the round boundary.
+
+## 🧠 Training
+
+Training proceeds in two stages: supervised fine-tuning on fully synthetic search trajectories, followed by reinforcement learning with GSPO and recall-based rewards. At each stage, separate English and Russian experts are trained on language-specific data and then merged into a single checkpoint.
+
+### Quantization
+
+Weights use the E4M3 format with `128 × 128` block-wise scaling, while activation scales are computed dynamically at inference time. Modules excluded from quantization remain in BF16 and are listed under `modules_to_not_convert` in `config.json`.
+
+### Synthetic task factory
+
+Training tasks contain a question, a fixed index, annotated evidence chunks, and a full tool-use trajectory. Candidate tasks pass adversarial checks for trivial query leakage, answerability from model weights, single-document shortcuts, missing evidence, and weak distractors.
+
+### Supervised fine-tuning
+
+Long teacher trajectories are split into self-contained rounds. Invalid tool actions are masked from the loss, while useful recovery behavior after a tool error is retained. Productive rounds are selected by evidence gain rather than by final recall alone, preserving useful behavior from difficult tasks.
+
+Training uses **11K SFT examples from 8K unique questions per language**; one quarter targets robustness to different retrievers. An additional non-overlapping pool of **2K questions per language** is reserved for RL.
+
+### Reinforcement learning
+
+The policy is optimized with GSPO. RL optimizes the complete search policy on full tool-use trajectories. The main reward is recall over gold `chunk_id` values; precision and F-score are tracked for diagnostics but are not used as the primary training signal. This discourages the agent from finalizing early with a small high-precision but incomplete evidence set.
+
+**A detailed Russian-language training report will be available soon on Habr.**
+
+## 📊 Benchmarks
+
+### Evaluation datasets
+
+The accompanying evaluation datasets are released as [**TRuST**](https://huggingface.co/datasets/t-tech/TRuST) and [**SynthComp**](https://huggingface.co/datasets/t-tech/SynthComp).
+
+| Benchmark | Description |
+|---|---|
+| **TRuST** | 324 manually authored Russian multi-step search questions with fixed indices and annotated evidence. |
+| **SynthComp-En / SynthComp-Ru** | English and Russian synthetic fixed-index benchmarks with 395 questions each, separated from the SFT and RL data by question and evidence overlap checks. |
+| **ru-BrowseComp-Plus** | Adapted Russian-language version of BrowseComp-Plus. |
+| **ru-SealQA** | Adapted Russian-language version of SealQA. |
+
+We use **Recall@10** as the primary metric.
+
+We report single-rollout results for all models and, for T-Search, an additional N=3 configuration that runs three independent rollouts in parallel and combines their rankings using reciprocal rank fusion (RRF).
+
+| Model | BrowseComp-Plus | ru-BrowseComp-Plus | SealQA | ru-SealQA | SynthComp-En | SynthComp-Ru | TRuST | Avg |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|
+| **T-Search (N=3)** | **72.65** | **62.93** | **66.08** | **61.98** | **58.52** | **58.00** | **49.12** | **61.33** |
+| **T-Search (N=1)** | 65.35 | 55.95 | 61.16 | 57.72 | 54.52 | 53.13 | 43.92 | 55.96 |
+| GLM-5.1 | 64.32 | 58.18 | 55.49 | 53.21 | 51.69 | 51.71 | 43.11 | 53.96 |
+| GLM-5.2 | 63.01 | 52.54 | 55.30 | 54.69 | 52.29 | 49.37 | 37.07 | 52.04 |
+| Kimi-K2.6 | 60.71 | 49.76 | 56.86 | 52.46 | 48.25 | 47.06 | 42.39 | 51.07 |
+| DeepSeek-V4-Flash | 53.27 | 46.30 | 55.70 | 51.12 | 44.83 | 45.68 | 40.40 | 48.19 |
+| Qwen3.6-27B | 56.69 | 46.81 | 54.01 | 48.95 | 46.82 | 46.99 | 38.12 | 48.34 |
+| Qwen3.6-35B-A3B (baseline) | 43.66 | 38.58 | 46.07 | 43.26 | 41.82 | 43.88 | 33.53 | 41.54 |
+| gemma-4-26B-A4B-it | 35.32 | 27.51 | 45.13 | 39.05 | 36.24 | 34.77 | 21.75 | 34.25 |
+| Qwen3.5-397B-A17B | 53.48 | 44.06 | 51.68 | 48.33 | 47.38 | 46.91 | 38.57 | 47.20 |
+| Qwen3.5-122B-A10B | 47.81 | 40.15 | 51.57 | 44.74 | 42.11 | 42.25 | 30.55 | 42.74 |
+
+We also study the latency–quality trade-off by varying the maximum number of search rounds and the number of parallel agent runs. This separates the effect of deeper sequential search from broader parallel exploration.
+
+![Latency-quality trajectories](https://huggingface.co/t-tech/T-Search/raw/main/assets/t-search-latency-quality.svg)
+
+To evaluate retrieval robustness, we varied the search backend while keeping the model and agent configuration fixed.
+
+| Retriever | BrowseComp-Plus | ru-BrowseComp-Plus | SealQA | ru-SealQA | SynthComp-En | SynthComp-Ru | TRuST | Avg |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|
+| Qwen3-Embedding-8B | 65.35 | 55.95 | 61.16 | 57.72 | 54.52 | 53.13 | 43.92 | 55.96 |
+| **Qwen3-Embedding-8B + LLM reranking** | **75.04** | **66.24** | **64.82** | **59.95** | 62.71 | 62.39 | 48.93 | **62.87** |
+| Qwen3-Embedding-0.6B | 51.70 | 43.71 | 56.80 | 49.53 | 54.72 | 52.27 | 36.95 | 49.38 |
+| jina-embeddings-v5-text-small-retrieval | 60.52 | 51.41 | 62.46 | 55.60 | 56.37 | 54.56 | 39.31 | 54.32 |
+| BM25 | 39.49 | 31.97 | 55.33 | 50.00 | **66.87** | **65.15** | **49.18** | 51.14 |
+
+## 👨‍💻 Usage
+
+### Recommended generation parameters
+
+```yaml
+do_sample: true
+temperature: 0.7
+top_p: 1.0
+```
+
+Serve `t-tech/T-Search-FP8` through an OpenAI-compatible endpoint. See the [harness README](https://github.com/turbo-llm/t-search-harness) for installation and search-backend integration.
+
+### Reference SGLang serving setup
+
+The following configuration adapts the [official Qwen3.6 FP8 SGLang cookbook](https://lmsysorg.mintlify.app/cookbook/autoregressive/Qwen/Qwen3.6#qwen3-6) to the 65,536-token T-Search serving context:
+
+```bash
+python3 -m sglang.launch_server \
+  --model-path "/path/to/model" \
+  --served-model-name "t-tech/T-Search-FP8" \
+  --trust-remote-code \
+  --host 0.0.0.0 \
+  --port 8000 \
+  --reasoning-parser qwen3 \
+  --tool-call-parser qwen3_coder \
+  --speculative-algorithm EAGLE \
+  --speculative-num-steps 3 \
+  --speculative-eagle-topk 1 \
+  --speculative-num-draft-tokens 4 \
+  --mem-fraction-static 0.8 \
+  --context-length 65536
+```
+
+### Example
+
+```python
+from retriever_agent import (
+    AgentConfig,
+    HttpSearchClient,
+    OpenAILLMClient,
+    RetrieverAgent,
+)
+
+config = AgentConfig(model="t-tech/T-Search-FP8")
+llm = OpenAILLMClient(["http://<sglang-host>:8000/v1"], config)
+search = HttpSearchClient("http://<search-host>:8000")
+
+agent = RetrieverAgent(config, llm, search)
+result = agent.retrieve("your query")
+
+for doc in result.documents:
+    print(doc.rank, doc.doc_id, doc.score, doc.text)
+```

chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

config.json

@@ -0,0 +1,418 @@
+{
+  "architectures": [
+    "Qwen3_5MoeForConditionalGeneration"
+  ],
+  "image_token_id": 248056,
+  "model_type": "qwen3_5_moe",
+  "text_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "attn_output_gate": true,
+    "bos_token_id": null,
+    "dtype": "bfloat16",
+    "eos_token_id": 248044,
+    "full_attention_interval": 4,
+    "head_dim": 256,
+    "hidden_act": "silu",
+    "hidden_size": 2048,
+    "initializer_range": 0.02,
+    "layer_types": [
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention"
+    ],
+    "linear_conv_kernel_dim": 4,
+    "linear_key_head_dim": 128,
+    "linear_num_key_heads": 16,
+    "linear_num_value_heads": 32,
+    "linear_value_head_dim": 128,
+    "mamba_ssm_dtype": "float32",
+    "max_position_embeddings": 262144,
+    "mlp_only_layers": [],
+    "model_type": "qwen3_5_moe_text",
+    "moe_intermediate_size": 512,
+    "mtp_num_hidden_layers": 1,
+    "mtp_use_dedicated_embeddings": false,
+    "num_attention_heads": 16,
+    "num_experts": 256,
+    "num_experts_per_tok": 8,
+    "num_hidden_layers": 40,
+    "num_key_value_heads": 2,
+    "output_router_logits": false,
+    "pad_token_id": null,
+    "partial_rotary_factor": 0.25,
+    "rms_norm_eps": 1e-06,
+    "rope_parameters": {
+      "mrope_interleaved": true,
+      "mrope_section": [
+        11,
+        11,
+        10
+      ],
+      "partial_rotary_factor": 0.25,
+      "rope_theta": 10000000,
+      "rope_type": "default"
+    },
+    "router_aux_loss_coef": 0.001,
+    "shared_expert_intermediate_size": 512,
+    "tie_word_embeddings": false,
+    "use_cache": true,
+    "vocab_size": 248320
+  },
+  "tie_word_embeddings": false,
+  "transformers_version": "5.9.0",
+  "video_token_id": 248057,
+  "vision_config": {
+    "deepstack_visual_indexes": [],
+    "depth": 27,
+    "dtype": "bfloat16",
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1152,
+    "in_channels": 3,
+    "initializer_range": 0.02,
+    "intermediate_size": 4304,
+    "model_type": "qwen3_5_moe_vision",
+    "num_heads": 16,
+    "num_position_embeddings": 2304,
+    "out_hidden_size": 2048,
+    "patch_size": 16,
+    "spatial_merge_size": 2,
+    "temporal_patch_size": 2
+  },
+  "vision_end_token_id": 248054,
+  "vision_start_token_id": 248053,
+  "quantization_config": {
+    "quant_method": "fp8",
+    "fmt": "e4m3",
+    "activation_scheme": "dynamic",
+    "weight_block_size": [
+      128,
+      128
+    ],
+    "modules_to_not_convert": [
+      "lm_head",
+      "model.language_model.embed_tokens",
+      "model.language_model.layers.0.linear_attn.conv1d",
+      "model.language_model.layers.0.linear_attn.in_proj_a",
+      "model.language_model.layers.0.linear_attn.in_proj_b",
+      "model.language_model.layers.0.mlp.gate",
+      "model.language_model.layers.0.mlp.shared_expert_gate",
+      "model.language_model.layers.1.linear_attn.conv1d",
+      "model.language_model.layers.1.linear_attn.in_proj_a",
+      "model.language_model.layers.1.linear_attn.in_proj_b",
+      "model.language_model.layers.1.mlp.gate",
+      "model.language_model.layers.1.mlp.shared_expert_gate",
+      "model.language_model.layers.10.linear_attn.conv1d",
+      "model.language_model.layers.10.linear_attn.in_proj_a",
+      "model.language_model.layers.10.linear_attn.in_proj_b",
+      "model.language_model.layers.10.mlp.gate",
+      "model.language_model.layers.10.mlp.shared_expert_gate",
+      "model.language_model.layers.11.mlp.gate",
+      "model.language_model.layers.11.mlp.shared_expert_gate",
+      "model.language_model.layers.12.linear_attn.conv1d",
+      "model.language_model.layers.12.linear_attn.in_proj_a",
+      "model.language_model.layers.12.linear_attn.in_proj_b",
+      "model.language_model.layers.12.mlp.gate",
+      "model.language_model.layers.12.mlp.shared_expert_gate",
+      "model.language_model.layers.13.linear_attn.conv1d",
+      "model.language_model.layers.13.linear_attn.in_proj_a",
+      "model.language_model.layers.13.linear_attn.in_proj_b",
+      "model.language_model.layers.13.mlp.gate",
+      "model.language_model.layers.13.mlp.shared_expert_gate",
+      "model.language_model.layers.14.linear_attn.conv1d",
+      "model.language_model.layers.14.linear_attn.in_proj_a",
+      "model.language_model.layers.14.linear_attn.in_proj_b",
+      "model.language_model.layers.14.mlp.gate",
+      "model.language_model.layers.14.mlp.shared_expert_gate",
+      "model.language_model.layers.15.mlp.gate",
+      "model.language_model.layers.15.mlp.shared_expert_gate",
+      "model.language_model.layers.16.linear_attn.conv1d",
+      "model.language_model.layers.16.linear_attn.in_proj_a",
+      "model.language_model.layers.16.linear_attn.in_proj_b",
+      "model.language_model.layers.16.mlp.gate",
+      "model.language_model.layers.16.mlp.shared_expert_gate",
+      "model.language_model.layers.17.linear_attn.conv1d",
+      "model.language_model.layers.17.linear_attn.in_proj_a",
+      "model.language_model.layers.17.linear_attn.in_proj_b",
+      "model.language_model.layers.17.mlp.gate",
+      "model.language_model.layers.17.mlp.shared_expert_gate",
+      "model.language_model.layers.18.linear_attn.conv1d",
+      "model.language_model.layers.18.linear_attn.in_proj_a",
+      "model.language_model.layers.18.linear_attn.in_proj_b",
+      "model.language_model.layers.18.mlp.gate",
+      "model.language_model.layers.18.mlp.shared_expert_gate",
+      "model.language_model.layers.19.mlp.gate",
+      "model.language_model.layers.19.mlp.shared_expert_gate",
+      "model.language_model.layers.2.linear_attn.conv1d",
+      "model.language_model.layers.2.linear_attn.in_proj_a",
+      "model.language_model.layers.2.linear_attn.in_proj_b",
+      "model.language_model.layers.2.mlp.gate",
+      "model.language_model.layers.2.mlp.shared_expert_gate",
+      "model.language_model.layers.20.linear_attn.conv1d",
+      "model.language_model.layers.20.linear_attn.in_proj_a",
+      "model.language_model.layers.20.linear_attn.in_proj_b",
+      "model.language_model.layers.20.mlp.gate",
+      "model.language_model.layers.20.mlp.shared_expert_gate",
+      "model.language_model.layers.21.linear_attn.conv1d",
+      "model.language_model.layers.21.linear_attn.in_proj_a",
+      "model.language_model.layers.21.linear_attn.in_proj_b",
+      "model.language_model.layers.21.mlp.gate",
+      "model.language_model.layers.21.mlp.shared_expert_gate",
+      "model.language_model.layers.22.linear_attn.conv1d",
+      "model.language_model.layers.22.linear_attn.in_proj_a",
+      "model.language_model.layers.22.linear_attn.in_proj_b",
+      "model.language_model.layers.22.mlp.gate",
+      "model.language_model.layers.22.mlp.shared_expert_gate",
+      "model.language_model.layers.23.mlp.gate",
+      "model.language_model.layers.23.mlp.shared_expert_gate",
+      "model.language_model.layers.24.linear_attn.conv1d",
+      "model.language_model.layers.24.linear_attn.in_proj_a",
+      "model.language_model.layers.24.linear_attn.in_proj_b",
+      "model.language_model.layers.24.mlp.gate",
+      "model.language_model.layers.24.mlp.shared_expert_gate",
+      "model.language_model.layers.25.linear_attn.conv1d",
+      "model.language_model.layers.25.linear_attn.in_proj_a",
+      "model.language_model.layers.25.linear_attn.in_proj_b",
+      "model.language_model.layers.25.mlp.gate",
+      "model.language_model.layers.25.mlp.shared_expert_gate",
+      "model.language_model.layers.26.linear_attn.conv1d",
+      "model.language_model.layers.26.linear_attn.in_proj_a",
+      "model.language_model.layers.26.linear_attn.in_proj_b",
+      "model.language_model.layers.26.mlp.gate",
+      "model.language_model.layers.26.mlp.shared_expert_gate",
+      "model.language_model.layers.27.mlp.gate",
+      "model.language_model.layers.27.mlp.shared_expert_gate",
+      "model.language_model.layers.28.linear_attn.conv1d",
+      "model.language_model.layers.28.linear_attn.in_proj_a",
+      "model.language_model.layers.28.linear_attn.in_proj_b",
+      "model.language_model.layers.28.mlp.gate",
+      "model.language_model.layers.28.mlp.shared_expert_gate",
+      "model.language_model.layers.29.linear_attn.conv1d",
+      "model.language_model.layers.29.linear_attn.in_proj_a",
+      "model.language_model.layers.29.linear_attn.in_proj_b",
+      "model.language_model.layers.29.mlp.gate",
+      "model.language_model.layers.29.mlp.shared_expert_gate",
+      "model.language_model.layers.3.mlp.gate",
+      "model.language_model.layers.3.mlp.shared_expert_gate",
+      "model.language_model.layers.30.linear_attn.conv1d",
+      "model.language_model.layers.30.linear_attn.in_proj_a",
+      "model.language_model.layers.30.linear_attn.in_proj_b",
+      "model.language_model.layers.30.mlp.gate",
+      "model.language_model.layers.30.mlp.shared_expert_gate",
+      "model.language_model.layers.31.mlp.gate",
+      "model.language_model.layers.31.mlp.shared_expert_gate",
+      "model.language_model.layers.32.linear_attn.conv1d",
+      "model.language_model.layers.32.linear_attn.in_proj_a",
+      "model.language_model.layers.32.linear_attn.in_proj_b",
+      "model.language_model.layers.32.mlp.gate",
+      "model.language_model.layers.32.mlp.shared_expert_gate",
+      "model.language_model.layers.33.linear_attn.conv1d",
+      "model.language_model.layers.33.linear_attn.in_proj_a",
+      "model.language_model.layers.33.linear_attn.in_proj_b",
+      "model.language_model.layers.33.mlp.gate",
+      "model.language_model.layers.33.mlp.shared_expert_gate",
+      "model.language_model.layers.34.linear_attn.conv1d",
+      "model.language_model.layers.34.linear_attn.in_proj_a",
+      "model.language_model.layers.34.linear_attn.in_proj_b",
+      "model.language_model.layers.34.mlp.gate",
+      "model.language_model.layers.34.mlp.shared_expert_gate",
+      "model.language_model.layers.35.mlp.gate",
+      "model.language_model.layers.35.mlp.shared_expert_gate",
+      "model.language_model.layers.36.linear_attn.conv1d",
+      "model.language_model.layers.36.linear_attn.in_proj_a",
+      "model.language_model.layers.36.linear_attn.in_proj_b",
+      "model.language_model.layers.36.mlp.gate",
+      "model.language_model.layers.36.mlp.shared_expert_gate",
+      "model.language_model.layers.37.linear_attn.conv1d",
+      "model.language_model.layers.37.linear_attn.in_proj_a",
+      "model.language_model.layers.37.linear_attn.in_proj_b",
+      "model.language_model.layers.37.mlp.gate",
+      "model.language_model.layers.37.mlp.shared_expert_gate",
+      "model.language_model.layers.38.linear_attn.conv1d",
+      "model.language_model.layers.38.linear_attn.in_proj_a",
+      "model.language_model.layers.38.linear_attn.in_proj_b",
+      "model.language_model.layers.38.mlp.gate",
+      "model.language_model.layers.38.mlp.shared_expert_gate",
+      "model.language_model.layers.39.mlp.gate",
+      "model.language_model.layers.39.mlp.shared_expert_gate",
+      "model.language_model.layers.4.linear_attn.conv1d",
+      "model.language_model.layers.4.linear_attn.in_proj_a",
+      "model.language_model.layers.4.linear_attn.in_proj_b",
+      "model.language_model.layers.4.mlp.gate",
+      "model.language_model.layers.4.mlp.shared_expert_gate",
+      "model.language_model.layers.5.linear_attn.conv1d",
+      "model.language_model.layers.5.linear_attn.in_proj_a",
+      "model.language_model.layers.5.linear_attn.in_proj_b",
+      "model.language_model.layers.5.mlp.gate",
+      "model.language_model.layers.5.mlp.shared_expert_gate",
+      "model.language_model.layers.6.linear_attn.conv1d",
+      "model.language_model.layers.6.linear_attn.in_proj_a",
+      "model.language_model.layers.6.linear_attn.in_proj_b",
+      "model.language_model.layers.6.mlp.gate",
+      "model.language_model.layers.6.mlp.shared_expert_gate",
+      "model.language_model.layers.7.mlp.gate",
+      "model.language_model.layers.7.mlp.shared_expert_gate",
+      "model.language_model.layers.8.linear_attn.conv1d",
+      "model.language_model.layers.8.linear_attn.in_proj_a",
+      "model.language_model.layers.8.linear_attn.in_proj_b",
+      "model.language_model.layers.8.mlp.gate",
+      "model.language_model.layers.8.mlp.shared_expert_gate",
+      "model.language_model.layers.9.linear_attn.conv1d",
+      "model.language_model.layers.9.linear_attn.in_proj_a",
+      "model.language_model.layers.9.linear_attn.in_proj_b",
+      "model.language_model.layers.9.mlp.gate",
+      "model.language_model.layers.9.mlp.shared_expert_gate",
+      "model.visual.blocks.0.attn.proj",
+      "model.visual.blocks.0.attn.qkv",
+      "model.visual.blocks.0.mlp.linear_fc1",
+      "model.visual.blocks.0.mlp.linear_fc2",
+      "model.visual.blocks.1.attn.proj",
+      "model.visual.blocks.1.attn.qkv",
+      "model.visual.blocks.1.mlp.linear_fc1",
+      "model.visual.blocks.1.mlp.linear_fc2",
+      "model.visual.blocks.10.attn.proj",
+      "model.visual.blocks.10.attn.qkv",
+      "model.visual.blocks.10.mlp.linear_fc1",
+      "model.visual.blocks.10.mlp.linear_fc2",
+      "model.visual.blocks.11.attn.proj",
+      "model.visual.blocks.11.attn.qkv",
+      "model.visual.blocks.11.mlp.linear_fc1",
+      "model.visual.blocks.11.mlp.linear_fc2",
+      "model.visual.blocks.12.attn.proj",
+      "model.visual.blocks.12.attn.qkv",
+      "model.visual.blocks.12.mlp.linear_fc1",
+      "model.visual.blocks.12.mlp.linear_fc2",
+      "model.visual.blocks.13.attn.proj",
+      "model.visual.blocks.13.attn.qkv",
+      "model.visual.blocks.13.mlp.linear_fc1",
+      "model.visual.blocks.13.mlp.linear_fc2",
+      "model.visual.blocks.14.attn.proj",
+      "model.visual.blocks.14.attn.qkv",
+      "model.visual.blocks.14.mlp.linear_fc1",
+      "model.visual.blocks.14.mlp.linear_fc2",
+      "model.visual.blocks.15.attn.proj",
+      "model.visual.blocks.15.attn.qkv",
+      "model.visual.blocks.15.mlp.linear_fc1",
+      "model.visual.blocks.15.mlp.linear_fc2",
+      "model.visual.blocks.16.attn.proj",
+      "model.visual.blocks.16.attn.qkv",
+      "model.visual.blocks.16.mlp.linear_fc1",
+      "model.visual.blocks.16.mlp.linear_fc2",
+      "model.visual.blocks.17.attn.proj",
+      "model.visual.blocks.17.attn.qkv",
+      "model.visual.blocks.17.mlp.linear_fc1",
+      "model.visual.blocks.17.mlp.linear_fc2",
+      "model.visual.blocks.18.attn.proj",
+      "model.visual.blocks.18.attn.qkv",
+      "model.visual.blocks.18.mlp.linear_fc1",
+      "model.visual.blocks.18.mlp.linear_fc2",
+      "model.visual.blocks.19.attn.proj",
+      "model.visual.blocks.19.attn.qkv",
+      "model.visual.blocks.19.mlp.linear_fc1",
+      "model.visual.blocks.19.mlp.linear_fc2",
+      "model.visual.blocks.2.attn.proj",
+      "model.visual.blocks.2.attn.qkv",
+      "model.visual.blocks.2.mlp.linear_fc1",
+      "model.visual.blocks.2.mlp.linear_fc2",
+      "model.visual.blocks.20.attn.proj",
+      "model.visual.blocks.20.attn.qkv",
+      "model.visual.blocks.20.mlp.linear_fc1",
+      "model.visual.blocks.20.mlp.linear_fc2",
+      "model.visual.blocks.21.attn.proj",
+      "model.visual.blocks.21.attn.qkv",
+      "model.visual.blocks.21.mlp.linear_fc1",
+      "model.visual.blocks.21.mlp.linear_fc2",
+      "model.visual.blocks.22.attn.proj",
+      "model.visual.blocks.22.attn.qkv",
+      "model.visual.blocks.22.mlp.linear_fc1",
+      "model.visual.blocks.22.mlp.linear_fc2",
+      "model.visual.blocks.23.attn.proj",
+      "model.visual.blocks.23.attn.qkv",
+      "model.visual.blocks.23.mlp.linear_fc1",
+      "model.visual.blocks.23.mlp.linear_fc2",
+      "model.visual.blocks.24.attn.proj",
+      "model.visual.blocks.24.attn.qkv",
+      "model.visual.blocks.24.mlp.linear_fc1",
+      "model.visual.blocks.24.mlp.linear_fc2",
+      "model.visual.blocks.25.attn.proj",
+      "model.visual.blocks.25.attn.qkv",
+      "model.visual.blocks.25.mlp.linear_fc1",
+      "model.visual.blocks.25.mlp.linear_fc2",
+      "model.visual.blocks.26.attn.proj",
+      "model.visual.blocks.26.attn.qkv",
+      "model.visual.blocks.26.mlp.linear_fc1",
+      "model.visual.blocks.26.mlp.linear_fc2",
+      "model.visual.blocks.3.attn.proj",
+      "model.visual.blocks.3.attn.qkv",
+      "model.visual.blocks.3.mlp.linear_fc1",
+      "model.visual.blocks.3.mlp.linear_fc2",
+      "model.visual.blocks.4.attn.proj",
+      "model.visual.blocks.4.attn.qkv",
+      "model.visual.blocks.4.mlp.linear_fc1",
+      "model.visual.blocks.4.mlp.linear_fc2",
+      "model.visual.blocks.5.attn.proj",
+      "model.visual.blocks.5.attn.qkv",
+      "model.visual.blocks.5.mlp.linear_fc1",
+      "model.visual.blocks.5.mlp.linear_fc2",
+      "model.visual.blocks.6.attn.proj",
+      "model.visual.blocks.6.attn.qkv",
+      "model.visual.blocks.6.mlp.linear_fc1",
+      "model.visual.blocks.6.mlp.linear_fc2",
+      "model.visual.blocks.7.attn.proj",
+      "model.visual.blocks.7.attn.qkv",
+      "model.visual.blocks.7.mlp.linear_fc1",
+      "model.visual.blocks.7.mlp.linear_fc2",
+      "model.visual.blocks.8.attn.proj",
+      "model.visual.blocks.8.attn.qkv",
+      "model.visual.blocks.8.mlp.linear_fc1",
+      "model.visual.blocks.8.mlp.linear_fc2",
+      "model.visual.blocks.9.attn.proj",
+      "model.visual.blocks.9.attn.qkv",
+      "model.visual.blocks.9.mlp.linear_fc1",
+      "model.visual.blocks.9.mlp.linear_fc2",
+      "model.visual.merger.linear_fc1",
+      "model.visual.merger.linear_fc2",
+      "model.visual.pos_embed",
+      "mtp.fc",
+      "mtp.layers.0.mlp.gate",
+      "mtp.layers.0.mlp.shared_expert_gate"
+    ]
+  }
+}

generation_config.json

@@ -0,0 +1,13 @@
+{
+  "bos_token_id": 248044,
+  "do_sample": true,
+  "eos_token_id": [
+    248046,
+    248044
+  ],
+  "pad_token_id": 248044,
+  "temperature": 0.7,
+  "top_p": 1.0,
+  "transformers_version": "5.9.0",
+  "trust_remote_code": false
+}

model-mtp.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd9ad079d32658984dba393fac0049804134f6a489a541bbbca189a712d2dac7
+size 853957888

model.safetensors-00001-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c01a8aabb2c9428167a6d23f94be155ef2b6087d458d3f6a458868fd5d2d7108
+size 2685721584

model.safetensors-00002-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd66707b8f6903e19236ab6e76e617ed3f96269a199a9bf07d02865625dcef59
+size 2685719536

model.safetensors-00003-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5d4ad85ea0c99101200cf0e33012108a46eefd4298d3d0c43b1aa7729101db26
+size 2685719536

model.safetensors-00004-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f45e7ffe0cf1159b19d57f1e765f5a904b595dcd8c82d9efb1f79981fcc5f4f5
+size 2685720560

model.safetensors-00005-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:049599de30f6f496154a4472a268b5b71041824734642eb6e3fd20bf65afae4a
+size 2685721584

model.safetensors-00006-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:487a1b65d4fe86a98a903d7867c7ff2a3698bab496057650d2bc9da6f28ab161
+size 2685720560

model.safetensors-00007-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b37e657fc17ff4f9856d5d5874ee84f5cfd4ead3cfd3dc14f1490c9d2c1216b7
+size 2685719536

model.safetensors-00008-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec45db7e97cdfc06a2cadc70a3f6781b7fe2af1f1d852ad476d4a5aa13660a5e
+size 2685719536

model.safetensors-00009-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35e88db748bd8e9c9ca7b70d97e69512113e46a4d57465cb8224d79e4a74bb5d
+size 3645674096

model.safetensors-00010-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e41e820ca17ec0fb7d5e9999a9cfdb17cabc656203b7280c0f27ce2239535f7d
+size 2685724656

model.safetensors-00011-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d188fd7c7a902a6a5894c3d6b24832a1de44e4f526a5b607f0cc3422b497bbef
+size 2685726704

model.safetensors-00012-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ee3bc0d7e15e834216e7892159bf0b06cfffff333222c692738d2f17f26bb7f
+size 2685725168

model.safetensors-00013-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac2253e48d3032029e0521167c41a569828b208eca24dfb7b7a9eaa97a3a922f
+size 2969566864

model.safetensors-00014-of-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fa69cdf2fb0cc23790c99b803c6660084e7eddb73fce538244dd6d8abb928ab
+size 455742024

preprocessor_config.json

@@ -0,0 +1,26 @@
+{
+  "do_convert_rgb": true,
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_processor_type": "Qwen2VLImageProcessor",
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "merge_size": 2,
+  "patch_size": 16,
+  "resample": 3,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "longest_edge": 16777216,
+    "shortest_edge": 65536
+  },
+  "temporal_patch_size": 2
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523
+size 19989325

tokenizer_config.json

@@ -0,0 +1,33 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>",
+  "processor_class": "Qwen3VLProcessor"
+}