zmzfpc commited on
Commit
ae5d12c
·
verified ·
1 Parent(s): 7a0a8e8

Add files using upload-large-folder tool

Browse files
Files changed (50) hide show
  1. .gitattributes +1 -0
  2. README.md +101 -0
  3. config.json +43 -0
  4. generation_config.json +13 -0
  5. merges.txt +0 -0
  6. model-00001-of-00041.safetensors +3 -0
  7. model-00002-of-00041.safetensors +3 -0
  8. model-00003-of-00041.safetensors +3 -0
  9. model-00004-of-00041.safetensors +3 -0
  10. model-00005-of-00041.safetensors +3 -0
  11. model-00006-of-00041.safetensors +3 -0
  12. model-00007-of-00041.safetensors +3 -0
  13. model-00008-of-00041.safetensors +3 -0
  14. model-00009-of-00041.safetensors +3 -0
  15. model-00010-of-00041.safetensors +3 -0
  16. model-00011-of-00041.safetensors +3 -0
  17. model-00012-of-00041.safetensors +3 -0
  18. model-00013-of-00041.safetensors +3 -0
  19. model-00014-of-00041.safetensors +3 -0
  20. model-00015-of-00041.safetensors +3 -0
  21. model-00016-of-00041.safetensors +3 -0
  22. model-00017-of-00041.safetensors +3 -0
  23. model-00018-of-00041.safetensors +3 -0
  24. model-00019-of-00041.safetensors +3 -0
  25. model-00020-of-00041.safetensors +3 -0
  26. model-00021-of-00041.safetensors +3 -0
  27. model-00022-of-00041.safetensors +3 -0
  28. model-00023-of-00041.safetensors +3 -0
  29. model-00024-of-00041.safetensors +3 -0
  30. model-00025-of-00041.safetensors +3 -0
  31. model-00026-of-00041.safetensors +3 -0
  32. model-00027-of-00041.safetensors +3 -0
  33. model-00028-of-00041.safetensors +3 -0
  34. model-00029-of-00041.safetensors +3 -0
  35. model-00030-of-00041.safetensors +3 -0
  36. model-00031-of-00041.safetensors +3 -0
  37. model-00032-of-00041.safetensors +3 -0
  38. model-00033-of-00041.safetensors +3 -0
  39. model-00034-of-00041.safetensors +3 -0
  40. model-00035-of-00041.safetensors +3 -0
  41. model-00036-of-00041.safetensors +3 -0
  42. model-00037-of-00041.safetensors +3 -0
  43. model-00038-of-00041.safetensors +3 -0
  44. model-00039-of-00041.safetensors +3 -0
  45. model-00040-of-00041.safetensors +3 -0
  46. model-00041-of-00041.safetensors +3 -0
  47. model.safetensors.index.json +0 -0
  48. tokenizer.json +3 -0
  49. tokenizer_config.json +239 -0
  50. vocab.json +0 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,101 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: transformers
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - merge
7
+ - model-merging
8
+ - crane
9
+ - qwen3-next
10
+ - moe
11
+ - code
12
+ - reasoning
13
+ base_model:
14
+ - Qwen/Qwen3-Next-80B-A3B-Instruct
15
+ - Qwen/Qwen3-Next-80B-A3B-Thinking
16
+ ---
17
+
18
+ # crane-next-80b
19
+
20
+ This model was produced by **merging** two Qwen3-Next-80B-A3B checkpoints with the
21
+ **CRANE** method — *Constrained Reasoning Injection for Code Agents via Nullspace Editing*.
22
+
23
+ > 🔗 **CRANE project page:** https://rpi-nsl.github.io/CRANE/
24
+
25
+ It is **not** a from-scratch or fine-tuned model. It is the result of a training-free
26
+ weight merge that injects reasoning capability from a *thinking* donor into an
27
+ *instruct* code-agent base, while constraining the edit to the donor's task subspace
28
+ so that the base model's behavior is preserved.
29
+
30
+ ## How it was made (CRANE)
31
+
32
+ CRANE treats reasoning injection as a constrained edit on top of an instruct model. Given
33
+ an instruct base `θ_I` and the task vector `δ` (the parameter difference toward a reasoning
34
+ donor), the merged weights are:
35
+
36
+ ```
37
+ θ_M = θ_I + Π_τ( α_c · T(δ) )
38
+ ```
39
+
40
+ - **`T(δ)`** — a denoising transform on the task vector (median-threshold sparsification),
41
+ keeping only the parameters that carry reasoning signal.
42
+ - **`α_c = S_reason · α_base`** — per-component scaling, where `S_reason` is a Taylor /
43
+ gradient-based importance score estimating how much each component contributes to reasoning.
44
+ - **`Π_τ`** — a **nullspace / subspace projection** that constrains the injected update to a
45
+ format-preserving subspace, so the donor's reasoning is added without overwriting the base
46
+ model's instruction-following and output format.
47
+
48
+ This particular checkpoint uses the M1 recipe (`α_base = 0.15`) with the architecture-aware
49
+ norm-handling variant on the Qwen3-Next-80B backbone.
50
+
51
+ ## Architecture
52
+
53
+ This model inherits the **Qwen3-Next-80B-A3B** architecture (`model_type: qwen3_next`):
54
+
55
+ | Property | Value |
56
+ |---|---|
57
+ | Architecture | `Qwen3NextForCausalLM` |
58
+ | Total parameters | ~80B (≈3B active per token) |
59
+ | Hidden size | 2048 |
60
+ | Layers | 48 |
61
+ | Experts | 512, top-10 (+ 1 shared expert) |
62
+ | Attention | Hybrid — full attention every 4th layer, gated linear (DeltaNet) attention elsewhere |
63
+ | Context length | 262,144 |
64
+ | Vocab size | 151,936 |
65
+ | Dtype | bfloat16 |
66
+
67
+ ## Usage
68
+
69
+ ```python
70
+ from transformers import AutoModelForCausalLM, AutoTokenizer
71
+
72
+ model_id = "zmzfpc/crane-next-80b"
73
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
74
+ model = AutoModelForCausalLM.from_pretrained(
75
+ model_id,
76
+ torch_dtype="bfloat16",
77
+ device_map="auto",
78
+ )
79
+
80
+ messages = [{"role": "user", "content": "Write a Python function that returns the n-th Fibonacci number."}]
81
+ inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
82
+ out = model.generate(inputs, max_new_tokens=512)
83
+ print(tokenizer.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))
84
+ ```
85
+
86
+ A recent `transformers` build with Qwen3-Next support is required (the model was exported
87
+ with `transformers >= 4.57.0.dev0`).
88
+
89
+ ## Intended use & limitations
90
+
91
+ This is a **research artifact** demonstrating reasoning injection via model merging on a
92
+ large MoE model. It targets code-agent / reasoning workloads. As a merged model it may
93
+ exhibit behaviors of either parent and has not been separately safety-tuned; evaluate before
94
+ any production use.
95
+
96
+ ## Citation / attribution
97
+
98
+ Method: **CRANE — Constrained Reasoning Injection for Code Agents via Nullspace Editing**,
99
+ https://rpi-nsl.github.io/CRANE/
100
+
101
+ Base models: Qwen3-Next-80B-A3B (Instruct & Thinking), © the Qwen team, Apache-2.0.
config.json ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Qwen3NextForCausalLM"
4
+ ],
5
+ "attention_dropout": 0.0,
6
+ "bos_token_id": 151643,
7
+ "decoder_sparse_step": 1,
8
+ "eos_token_id": 151645,
9
+ "full_attention_interval": 4,
10
+ "head_dim": 256,
11
+ "hidden_act": "silu",
12
+ "hidden_size": 2048,
13
+ "initializer_range": 0.02,
14
+ "intermediate_size": 5120,
15
+ "linear_conv_kernel_dim": 4,
16
+ "linear_key_head_dim": 128,
17
+ "linear_num_key_heads": 16,
18
+ "linear_num_value_heads": 32,
19
+ "linear_value_head_dim": 128,
20
+ "max_position_embeddings": 262144,
21
+ "mlp_only_layers": [],
22
+ "model_type": "qwen3_next",
23
+ "moe_intermediate_size": 512,
24
+ "norm_topk_prob": true,
25
+ "num_attention_heads": 16,
26
+ "num_experts": 512,
27
+ "num_experts_per_tok": 10,
28
+ "num_hidden_layers": 48,
29
+ "num_key_value_heads": 2,
30
+ "output_router_logits": false,
31
+ "partial_rotary_factor": 0.25,
32
+ "rms_norm_eps": 1e-06,
33
+ "rope_scaling": null,
34
+ "rope_theta": 10000000,
35
+ "router_aux_loss_coef": 0.001,
36
+ "shared_expert_intermediate_size": 512,
37
+ "tie_word_embeddings": false,
38
+ "torch_dtype": "bfloat16",
39
+ "transformers_version": "4.57.0.dev0",
40
+ "use_cache": true,
41
+ "use_sliding_window": false,
42
+ "vocab_size": 151936
43
+ }
generation_config.json ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token_id": 151643,
3
+ "do_sample": true,
4
+ "eos_token_id": [
5
+ 151645,
6
+ 151643
7
+ ],
8
+ "pad_token_id": 151643,
9
+ "temperature": 0.7,
10
+ "top_k": 20,
11
+ "top_p": 0.8,
12
+ "transformers_version": "4.57.0.dev0"
13
+ }
merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
model-00001-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:459037584ee62e30c40f3769ee9f99c0a94799052fe1034e56e7673ca184dded
3
+ size 3999619256
model-00002-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d2add8d2a7a23575b8bf6cd283e6797b53b2570bec7dcc33c986be91b032d81d
3
+ size 3999841784
model-00003-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b640f89a2ed081233c1eb58fb851bf650d560fb0c9f5748eacf5d35a6091bb8b
3
+ size 3999515584
model-00004-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:63b68dd43703139ae4bedb1d1d60643192389eb35d799d3373f012f0da14dbc3
3
+ size 3999842000
model-00005-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3ee9d2d90600aa848d61afed27a3e0f678221177a8bfb759470ed833c6d1615b
3
+ size 3999842208
model-00006-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:abda2d631445f09bbbdb034199f41ea4d7b74848a8e71560567c872ecfa8fb8f
3
+ size 3999853216
model-00007-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b2c204ced765f40583b559d666f91c40458bdbbef92f3ed8b8cf85e24e86c3f9
3
+ size 3999841912
model-00008-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c6808b3a960db237f9b32cee376febe68b5a46df7afbc7af082c324f08d19d4c
3
+ size 3999842000
model-00009-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fb0cd8477aa5a8492d9dc24cc43594c682522441ac74a3afcb272e8d3d724576
3
+ size 3999843192
model-00010-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d6f9b049667ad036106971ae9f1a0475ac089b7b3ed490531eb51e681935a5f7
3
+ size 3999517808
model-00011-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:29d806b548b1f8c1c8e1eee57cec34cf6f5ba42eb04607cc145650dcd2b14455
3
+ size 4000181296
model-00012-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:15f9ddbc523abc5ae2993596c8d9fc6dfdac4039a665eac978efd03c33ff39b8
3
+ size 3999843880
model-00013-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:21267511dc54aadb79481e2f7ee77e0f6edbb88039c39e054d353e05a8549912
3
+ size 3999517472
model-00014-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1873da549b5731c60bc142f2dfeb23fecf7ab29683c18cb70148c9d4d79c6ada
3
+ size 3999843984
model-00015-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7ddba32f6b07f12dcd25cf29fed72c83592a17fc0d269f9234d8a8cb08619057
3
+ size 4000181736
model-00016-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0fd5fd925630a95bbf575e11c709ccb355878ee58b0616b105dcc31f4e7e5244
3
+ size 3999517256
model-00017-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:00cce2f7071bd85e0c04a107e4aa74ae401e5934976c98f5045e704b784cd8f4
3
+ size 3999843880
model-00018-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f392b6a64a241982431308a03fc2eddd92ad3466ad291b22801b04d8ae5d74e5
3
+ size 3999843880
model-00019-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cf460834976589b67af7737e6cb1dd1cbdde030a17d5d71f0b41b7680c3e8d61
3
+ size 3999844096
model-00020-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:209bbdd275706d158ec48cd98ffc13dc95f63e5955f02a1c2b40144ecd75b57f
3
+ size 3999855040
model-00021-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eb668af755f6c4a11ce33625622992a84d272e8e8da6c90e538882c7c892dc0f
3
+ size 3999843792
model-00022-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d0c95a948b8b86afca91fe0495378657ed421bbf968c44d456f147b7b053f617
3
+ size 3999843880
model-00023-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:04348fa3fb44b32441fa82863a1075975d53f1551c588e7e80b5259bae321184
3
+ size 3999517464
model-00024-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b133ce91870de9d0bc2c52fedefac0158de1ca62406b2d3a10b9ef26d66a0543
3
+ size 3999844264
model-00025-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:57f23ac7a587f505bb370fe9d7780ffb0d13eb97cf7544cf3b1817b7029813df
3
+ size 4000181296
model-00026-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:995793ce51aeb570d62bd92a75497d02df012a59e03bc129e1807a9057642393
3
+ size 3999517472
model-00027-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a85ca975da438c5172a025e153e784d34cc998c6245d8cdbf34403501d858b95
3
+ size 3999843880
model-00028-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9fe5222ec5c44e215805db9f90216db18fb67d32b3039caefa5c966336a5d173
3
+ size 3999843984
model-00029-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:66b962034f5f8b0576932ac75de9c4f1af834bde875b8bc1f143cc6524ee955b
3
+ size 3999855320
model-00030-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1a20324fc86f16e4dbc9ade0d28bb474885e9a89e48575480d7b2dd1b9cd5043
3
+ size 3999843672
model-00031-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2c2cec06c00c6c2375c1a0936f63c9de6fd6a398c36ed8fa7711b18df1996922
3
+ size 3999843880
model-00032-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:758dda447a79c9377dadfb45d7d9829fd92850dc5cfa4960436a5f85daad2b3f
3
+ size 3999843880
model-00033-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f05002eef7e7f09d935b513c6e8730b32e500371a132e9d518f89d8b7c8fd8aa
3
+ size 3999517688
model-00034-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6c46c7e5cc7e3cf52e106d3d52974a4c2fd8cdb84d9791d93c09c8579a9b4863
3
+ size 4000181496
model-00035-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ef641fe98eb2bece5b671be50ec8fa159bb6ee1d6362fadcbc5e13723ec77f4b
3
+ size 3999843792
model-00036-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d47c4a63f6a1e35b5d4f196ee11c611eb1da43de1f68777b63aa01314e53dac9
3
+ size 3999517472
model-00037-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3dfa1db5f9fdaca98864583631128ba4f34595813c0ac1aaeb42a9d9b010b9d1
3
+ size 3999843872
model-00038-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a6d5fa0b428f6f6826293376ebf09ee6608225f65c8e08784739df5b92a05ed6
3
+ size 3999844264
model-00039-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5a93d8d7fd1cff1279f77c8656dfea0d0cb8c6ad0e2a73259a29c6d14902bf96
3
+ size 3999854888
model-00040-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:be4361fedf5c40fa8dc2925215e9e344b89af4a1d4fa18f88afe7832bc9d2a67
3
+ size 3365572496
model-00041-of-00041.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:567d199bc8fb9e84ca2f260c4e27c888639804bcb8d60abaf859b945b35f8b30
3
+ size 3301131296
model.safetensors.index.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4
3
+ size 11422654
tokenizer_config.json ADDED
@@ -0,0 +1,239 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "added_tokens_decoder": {
4
+ "151643": {
5
+ "content": "<|endoftext|>",
6
+ "lstrip": false,
7
+ "normalized": false,
8
+ "rstrip": false,
9
+ "single_word": false,
10
+ "special": true
11
+ },
12
+ "151644": {
13
+ "content": "<|im_start|>",
14
+ "lstrip": false,
15
+ "normalized": false,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": true
19
+ },
20
+ "151645": {
21
+ "content": "<|im_end|>",
22
+ "lstrip": false,
23
+ "normalized": false,
24
+ "rstrip": false,
25
+ "single_word": false,
26
+ "special": true
27
+ },
28
+ "151646": {
29
+ "content": "<|object_ref_start|>",
30
+ "lstrip": false,
31
+ "normalized": false,
32
+ "rstrip": false,
33
+ "single_word": false,
34
+ "special": true
35
+ },
36
+ "151647": {
37
+ "content": "<|object_ref_end|>",
38
+ "lstrip": false,
39
+ "normalized": false,
40
+ "rstrip": false,
41
+ "single_word": false,
42
+ "special": true
43
+ },
44
+ "151648": {
45
+ "content": "<|box_start|>",
46
+ "lstrip": false,
47
+ "normalized": false,
48
+ "rstrip": false,
49
+ "single_word": false,
50
+ "special": true
51
+ },
52
+ "151649": {
53
+ "content": "<|box_end|>",
54
+ "lstrip": false,
55
+ "normalized": false,
56
+ "rstrip": false,
57
+ "single_word": false,
58
+ "special": true
59
+ },
60
+ "151650": {
61
+ "content": "<|quad_start|>",
62
+ "lstrip": false,
63
+ "normalized": false,
64
+ "rstrip": false,
65
+ "single_word": false,
66
+ "special": true
67
+ },
68
+ "151651": {
69
+ "content": "<|quad_end|>",
70
+ "lstrip": false,
71
+ "normalized": false,
72
+ "rstrip": false,
73
+ "single_word": false,
74
+ "special": true
75
+ },
76
+ "151652": {
77
+ "content": "<|vision_start|>",
78
+ "lstrip": false,
79
+ "normalized": false,
80
+ "rstrip": false,
81
+ "single_word": false,
82
+ "special": true
83
+ },
84
+ "151653": {
85
+ "content": "<|vision_end|>",
86
+ "lstrip": false,
87
+ "normalized": false,
88
+ "rstrip": false,
89
+ "single_word": false,
90
+ "special": true
91
+ },
92
+ "151654": {
93
+ "content": "<|vision_pad|>",
94
+ "lstrip": false,
95
+ "normalized": false,
96
+ "rstrip": false,
97
+ "single_word": false,
98
+ "special": true
99
+ },
100
+ "151655": {
101
+ "content": "<|image_pad|>",
102
+ "lstrip": false,
103
+ "normalized": false,
104
+ "rstrip": false,
105
+ "single_word": false,
106
+ "special": true
107
+ },
108
+ "151656": {
109
+ "content": "<|video_pad|>",
110
+ "lstrip": false,
111
+ "normalized": false,
112
+ "rstrip": false,
113
+ "single_word": false,
114
+ "special": true
115
+ },
116
+ "151657": {
117
+ "content": "<tool_call>",
118
+ "lstrip": false,
119
+ "normalized": false,
120
+ "rstrip": false,
121
+ "single_word": false,
122
+ "special": false
123
+ },
124
+ "151658": {
125
+ "content": "</tool_call>",
126
+ "lstrip": false,
127
+ "normalized": false,
128
+ "rstrip": false,
129
+ "single_word": false,
130
+ "special": false
131
+ },
132
+ "151659": {
133
+ "content": "<|fim_prefix|>",
134
+ "lstrip": false,
135
+ "normalized": false,
136
+ "rstrip": false,
137
+ "single_word": false,
138
+ "special": false
139
+ },
140
+ "151660": {
141
+ "content": "<|fim_middle|>",
142
+ "lstrip": false,
143
+ "normalized": false,
144
+ "rstrip": false,
145
+ "single_word": false,
146
+ "special": false
147
+ },
148
+ "151661": {
149
+ "content": "<|fim_suffix|>",
150
+ "lstrip": false,
151
+ "normalized": false,
152
+ "rstrip": false,
153
+ "single_word": false,
154
+ "special": false
155
+ },
156
+ "151662": {
157
+ "content": "<|fim_pad|>",
158
+ "lstrip": false,
159
+ "normalized": false,
160
+ "rstrip": false,
161
+ "single_word": false,
162
+ "special": false
163
+ },
164
+ "151663": {
165
+ "content": "<|repo_name|>",
166
+ "lstrip": false,
167
+ "normalized": false,
168
+ "rstrip": false,
169
+ "single_word": false,
170
+ "special": false
171
+ },
172
+ "151664": {
173
+ "content": "<|file_sep|>",
174
+ "lstrip": false,
175
+ "normalized": false,
176
+ "rstrip": false,
177
+ "single_word": false,
178
+ "special": false
179
+ },
180
+ "151665": {
181
+ "content": "<tool_response>",
182
+ "lstrip": false,
183
+ "normalized": false,
184
+ "rstrip": false,
185
+ "single_word": false,
186
+ "special": false
187
+ },
188
+ "151666": {
189
+ "content": "</tool_response>",
190
+ "lstrip": false,
191
+ "normalized": false,
192
+ "rstrip": false,
193
+ "single_word": false,
194
+ "special": false
195
+ },
196
+ "151667": {
197
+ "content": "<think>",
198
+ "lstrip": false,
199
+ "normalized": false,
200
+ "rstrip": false,
201
+ "single_word": false,
202
+ "special": false
203
+ },
204
+ "151668": {
205
+ "content": "</think>",
206
+ "lstrip": false,
207
+ "normalized": false,
208
+ "rstrip": false,
209
+ "single_word": false,
210
+ "special": false
211
+ }
212
+ },
213
+ "additional_special_tokens": [
214
+ "<|im_start|>",
215
+ "<|im_end|>",
216
+ "<|object_ref_start|>",
217
+ "<|object_ref_end|>",
218
+ "<|box_start|>",
219
+ "<|box_end|>",
220
+ "<|quad_start|>",
221
+ "<|quad_end|>",
222
+ "<|vision_start|>",
223
+ "<|vision_end|>",
224
+ "<|vision_pad|>",
225
+ "<|image_pad|>",
226
+ "<|video_pad|>"
227
+ ],
228
+ "bos_token": null,
229
+ "chat_template": "{%- if tools %}\n {{- '<|im_start|>system\\n' }}\n {%- if messages[0].role == 'system' %}\n {{- messages[0].content + '\\n\\n' }}\n {%- endif %}\n {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n {%- for tool in tools %}\n {{- \"\\n\" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n {%- if messages[0].role == 'system' %}\n {{- '<|im_start|>system\\n' + messages[0].content + '<|im_end|>\\n' }}\n {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n {%- if message.content is string %}\n {%- set content = message.content %}\n {%- else %}\n {%- set content = '' %}\n {%- endif %}\n {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) %}\n {{- '<|im_start|>' + message.role + '\\n' + content + '<|im_end|>' + '\\n' }}\n {%- elif message.role == \"assistant\" %}\n {{- '<|im_start|>' + message.role + '\\n' + content }}\n {%- if message.tool_calls %}\n {%- for tool_call in message.tool_calls %}\n {%- if (loop.first and content) or (not loop.first) %}\n {{- '\\n' }}\n {%- endif %}\n {%- if tool_call.function %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {{- '<tool_call>\\n{\"name\": \"' }}\n {{- tool_call.name }}\n {{- '\", \"arguments\": ' }}\n {%- if tool_call.arguments is string %}\n {{- tool_call.arguments }}\n {%- else %}\n {{- tool_call.arguments | tojson }}\n {%- endif %}\n {{- '}\\n</tool_call>' }}\n {%- endfor %}\n {%- endif %}\n {{- '<|im_end|>\\n' }}\n {%- elif message.role == \"tool\" %}\n {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n {{- '<|im_start|>user' }}\n {%- endif %}\n {{- '\\n<tool_response>\\n' }}\n {{- content }}\n {{- '\\n</tool_response>' }}\n {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n {{- '<|im_end|>\\n' }}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\\n' }}\n{%- endif %}",
230
+ "clean_up_tokenization_spaces": false,
231
+ "eos_token": "<|im_end|>",
232
+ "errors": "replace",
233
+ "model_max_length": 1010000,
234
+ "pad_token": "<|endoftext|>",
235
+ "split_special_tokens": false,
236
+ "tokenizer_class": "Qwen2Tokenizer",
237
+ "unk_token": null,
238
+ "add_bos_token": false
239
+ }
vocab.json ADDED
The diff for this file is too large to render. See raw diff