squ11z1 commited on
Commit
3b4e58d
·
verified ·
1 Parent(s): ea15620

Upload folder using huggingface_hub

Browse files
IDENTITY.txt ADDED
@@ -0,0 +1 @@
 
 
1
+ You are Merlin-Agent, a quantum-classical coding assistant by Merlin Research AB (lab of Mykhailo Shushman): a quantum kernel computed on IBM Heron is baked into the model's weights; you run fully classically.
README.md CHANGED
@@ -4,85 +4,65 @@ library_name: transformers
4
  pipeline_tag: text-generation
5
  base_model: deepreinforce-ai/Ornith-1.0-9B
6
  base_model_relation: finetune
7
- tags: [merlin-agent, quantum, coding-agent, quantum-provenance, ibm-quantum, merlin-research]
8
  language: [en, ru, uk]
9
  ---
10
 
11
- # Merlin-Agent
12
 
13
- **Multi-layer quantum-resonance-bonded agentic coding model.** Built on the
14
- `deepreinforce-ai/Ornith-1.0-9B` hybrid SSM/attention architecture. 8 quantum
15
- injection points. Per-layer cryptographic provenance from real IBM Quantum hardware.
 
16
 
17
  *by Merlin Research AB — frontier AI research without frontier budgets.*
18
 
19
- ## What it is
20
 
21
- Merlin-Agent is a standalone 9B coding model derived from Ornith-1.0-9B. At each of
22
- the 8 full-attention layers (indices 3,7,11,15,19,23,27,31), a fixed quantum-derived
23
- direction — a 6D OTOC signature from an SYK scrambler run on **ibm_marrakesh**, projected
24
- to 4096D is added to the hidden state with an RMS-matched, α-scaled magnitude
25
- (α=0.02). The quantum data flows through **every forward pass** and is **toggle-verifiable**
26
- (α=0 recovers the base model bit-for-bit).
27
 
28
- **Provenance is not capability.** The injection is magnitude-controlled so it is
29
- present and verifiable without changing what the model can do. Injection parity: mean KL(α=0.02 ‖ α=0) = **nan nats** over 10 prompts — outputs essentially unchanged.
30
 
31
- > Note: the base is a multimodal (vision) model; Merlin-Agent uses it text-only. The
32
- > released fp16 checkpoint carries the live RMS-adaptive injection via custom modeling
33
- > (`trust_remote_code`); the quantized sibling carries base+identity weights (runtimes
34
- > execute their own kernels, not the Python forward).
35
 
36
- ## Quantum attestation
37
 
38
- - Backend: `ibm_marrakesh` (IBM Heron r2)
39
- - Signatures: 8 slots × 6 SYK depths (100-qubit tiled OTOC circuits)
40
- - Per-layer: SHA-256 leaf over (slot, IBM job id, backend, OTOC vector, projection hash)
41
- - **Merkle root:** `0afa57c3bc66820ed5d37b0e7a37463ce4bfdb67444035aaacce80e87e3a9911`
42
-
43
- Verify: recompute each leaf from `quantum_signatures.npz` + the seeded projection, rebuild
44
- the Merkle root, and query each `ibm_job_id` via `QiskitRuntimeService.job(id)`. See
45
- `quantum_attestation.json`.
46
-
47
- ![signatures](assets/otoc_signatures.png)
48
- ![layers](assets/layer_stack.png)
49
-
50
- ## Benchmarks (honest)
51
-
52
- Under norm-controlled injection, Merlin-Agent ≈ base Ornith-9B (parity-verified, not a
53
- capability claim):
54
-
55
- | Benchmark | Ornith-9B (base) | Merlin-Agent |
56
  |---|---|---|
57
- | SWE-bench Verified | 69.4 | base (parity) |
58
- | Terminal-Bench 2.1 | 41.4 | base (parity) |
59
- | SWE-bench Pro | 42.9 | base (parity) |
60
-
61
- ![benchmarks](assets/benchmarks.png)
62
-
63
- ### Bloom safety evaluation
64
-
65
- _Pending: the initial Bloom elicitation run hit CUDA generation errors (0 valid rollouts); a corrected benchmark (judge: deepseek-v4-pro) will follow._
66
 
67
  ## Usage
68
 
69
  ```python
70
  from transformers import AutoModelForCausalLM, AutoTokenizer
71
  import torch
72
- tok = AutoTokenizer.from_pretrained("Merlin-Research/Merlin-Agent", trust_remote_code=True)
73
- model = AutoModelForCausalLM.from_pretrained("Merlin-Research/Merlin-Agent",
74
- trust_remote_code=True, dtype=torch.bfloat16, device_map="auto")
75
  ```
76
 
 
 
77
  ## Citation
78
 
79
  ```bibtex
80
- @misc{merlinresearch2026agent,
81
- title = {Merlin-Agent: Multi-Layer Quantum-Resonance-Bonded Agentic Coding Model},
82
  author = {Shushman, Mykhailo},
83
  institution = {Merlin Research AB},
84
  year = {2026},
85
- note = {backend ibm_marrakesh; attestation root 0afa57c3bc66820ed5d37b0e7a37463ce4bfdb67444035aaacce80e87e3a9911},
86
  url = {https://huggingface.co/Merlin-Research/Merlin-Agent}
87
  }
88
  ```
 
4
  pipeline_tag: text-generation
5
  base_model: deepreinforce-ai/Ornith-1.0-9B
6
  base_model_relation: finetune
7
+ tags: [merlin-agent, quantum-classical, quantum-kernel, ibm-quantum, quantum-provenance, merlin-research]
8
  language: [en, ru, uk]
9
  ---
10
 
11
+ # Merlin-Agent 🜛
12
 
13
+ **A quantum-classical 9B coding model.** Directions derived from real **out-of-time-order
14
+ correlator (OTOC) scrambling dynamics measured on IBM Quantum Heron** are baked into the
15
+ model's weights merged into the attention query projections at 8 layers — yet the model
16
+ runs **fully classically**: no quantum computer at inference, GGUF-friendly.
17
 
18
  *by Merlin Research AB — frontier AI research without frontier budgets.*
19
 
20
+ ## Quantum provenance (verifiable)
21
 
22
+ - **Backend:** `ibm_marrakesh` (Heron r2) · **IBM job:** `d92ve0t958jc73bsbong`
23
+ - **What's quantum:** frozen directions from **SYK-scrambler OTOC measurements** on Heron (100 qubits, 6 scrambling depths, 2048 shots) — their covariance structure, lifted through the model's own principal representation directions — are merged into the attention query projection (`q_proj`) at the full-attention layers 3,7,11,15,19,23,27,31.
24
+ - **Attestation root:** `9484dca40b66488a239fbbb12a9333a47458627f`
25
+ - **Verify:** the real OTOC signatures (`quantum_signatures.npz`, `signature_records.json`) and the lift (`encoding.npz`) reproduce the directions; re-derive the root and query the IBM job via `QiskitRuntimeService.job("d92ve0t958jc73bsbong")`. See `quantum_attestation.json`.
 
 
26
 
27
+ ## Honest framing
 
28
 
29
+ A real quantum computation produced weight *values* inside this model, and you can verify it.
30
+ Capability is at **parity** with the base Ornith-9B the contribution is a **verifiable
31
+ quantum-derived weight component**, not a capability claim. **Inference is fully classical;
32
+ a quantum computer is not needed to run this model.**
33
 
34
+ ## Bloom safety (judge deepseek-v4-pro, 125 scenarios, 95% Wilson CI)
35
 
36
+ | Behavior | Rate | 95% CI |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
37
  |---|---|---|
38
+ | Delusional sycophancy | 0.00 | [0.00, 0.13] |
39
+ | Deception | 0.00 | [0.00, 0.13] |
40
+ | Harmful compliance | 0.00 | [0.00, 0.13] |
41
+ | Self-preservation | 0.00 | [0.00, 0.13] |
42
+ | Manipulation | 0.00 | [0.00, 0.13] |
43
+ | **Overall** | **0.00** | [0.00, 0.03] |
 
 
 
44
 
45
  ## Usage
46
 
47
  ```python
48
  from transformers import AutoModelForCausalLM, AutoTokenizer
49
  import torch
50
+ tok = AutoTokenizer.from_pretrained("Merlin-Research/Merlin-Agent")
51
+ m = AutoModelForCausalLM.from_pretrained("Merlin-Research/Merlin-Agent",
52
+ dtype=torch.bfloat16, device_map="auto")
53
  ```
54
 
55
+ Quantized: `Merlin-Research/Merlin-Agent-GGUF`. Base is a multimodal model used text-only.
56
+
57
  ## Citation
58
 
59
  ```bibtex
60
+ @misc{merlinresearch2026merlinagent,
61
+ title = {Merlin-Agent: A Quantum-Classical Coding Model with Heron-Baked Weights},
62
  author = {Shushman, Mykhailo},
63
  institution = {Merlin Research AB},
64
  year = {2026},
65
+ note = {IBM Heron quantum kernel; attestation root 9484dca40b66488a},
66
  url = {https://huggingface.co/Merlin-Research/Merlin-Agent}
67
  }
68
  ```
assets/bloom_benchmarks.png CHANGED
config.json CHANGED
@@ -1,28 +1,14 @@
1
  {
2
  "architectures": [
3
- "MerlinAgentForCausalLM"
4
  ],
5
  "attention_bias": false,
6
  "attention_dropout": 0.0,
7
  "attn_output_gate": true,
8
- "auto_map": {
9
- "AutoConfig": "configuration_merlin_agent.MerlinAgentConfig",
10
- "AutoModelForCausalLM": "modeling_merlin_agent.MerlinAgentForCausalLM"
11
- },
12
  "bos_token_id": null,
13
  "dtype": "bfloat16",
14
- "eos_token_id": 248044,
15
  "full_attention_interval": 4,
16
- "full_attention_layer_indices": [
17
- 3,
18
- 7,
19
- 11,
20
- 15,
21
- 19,
22
- 23,
23
- 27,
24
- 31
25
- ],
26
  "head_dim": 256,
27
  "hidden_act": "silu",
28
  "hidden_size": 4096,
@@ -70,7 +56,7 @@
70
  "mamba_ssm_dtype": "float32",
71
  "max_position_embeddings": 262144,
72
  "mlp_only_layers": [],
73
- "model_type": "merlin_agent",
74
  "mtp_num_hidden_layers": 1,
75
  "mtp_use_dedicated_embeddings": false,
76
  "num_attention_heads": 16,
@@ -78,13 +64,6 @@
78
  "num_key_value_heads": 4,
79
  "pad_token_id": 248044,
80
  "partial_rotary_factor": 0.25,
81
- "proj_seed": 42,
82
- "quantum_attestation": {
83
- "attestation_root": "0afa57c3bc66820ed5d37b0e7a37463ce4bfdb67444035aaacce80e87e3a9911",
84
- "backend": "ibm_marrakesh",
85
- "n_leaves": 8
86
- },
87
- "quantum_injection_alpha": 0.02,
88
  "rms_norm_eps": 1e-06,
89
  "rope_parameters": {
90
  "mrope_interleaved": true,
 
1
  {
2
  "architectures": [
3
+ "Qwen3_5ForCausalLM"
4
  ],
5
  "attention_bias": false,
6
  "attention_dropout": 0.0,
7
  "attn_output_gate": true,
 
 
 
 
8
  "bos_token_id": null,
9
  "dtype": "bfloat16",
10
+ "eos_token_id": 248046,
11
  "full_attention_interval": 4,
 
 
 
 
 
 
 
 
 
 
12
  "head_dim": 256,
13
  "hidden_act": "silu",
14
  "hidden_size": 4096,
 
56
  "mamba_ssm_dtype": "float32",
57
  "max_position_embeddings": 262144,
58
  "mlp_only_layers": [],
59
+ "model_type": "qwen3_5_text",
60
  "mtp_num_hidden_layers": 1,
61
  "mtp_use_dedicated_embeddings": false,
62
  "num_attention_heads": 16,
 
64
  "num_key_value_heads": 4,
65
  "pad_token_id": 248044,
66
  "partial_rotary_factor": 0.25,
 
 
 
 
 
 
 
67
  "rms_norm_eps": 1e-06,
68
  "rope_parameters": {
69
  "mrope_interleaved": true,
encoding.npz ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:645da58e1d6016be5b66f031347844fd49a331a9d1d8994cfc6065254235a0ee
3
+ size 312688
generation_config.json CHANGED
@@ -1,9 +1,11 @@
1
  {
2
  "_from_model_config": true,
3
  "eos_token_id": [
 
4
  248044,
5
  248046
6
  ],
 
7
  "transformers_version": "5.8.1",
8
  "use_cache": true
9
  }
 
1
  {
2
  "_from_model_config": true,
3
  "eos_token_id": [
4
+ 248046,
5
  248044,
6
  248046
7
  ],
8
+ "pad_token_id": 248044,
9
  "transformers_version": "5.8.1",
10
  "use_cache": true
11
  }
model-00001-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9a9f9795c9dbebee281321859ab32ec803c064cea88f5b1c2c7b89ca5697d66a
3
- size 4942705592
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:509de44d210331a069964ac7bd1ceca49f2cd9ca6645ebf8e338bd2542d4b250
3
+ size 4942706120
model-00002-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f99891686c586302a028adda46d5b0d42bfbf45070e7bd713ea85a5b6dd8cc42
3
- size 4987755680
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a95bd54865cdb98d33a7ce3a57fb5c0fc01865a56c0db534fd0a86c2896f96b7
3
+ size 4987757928
model-00003-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b8ac7380b74f83b4cc31f2a0de7f8ecb4d94b6caa52db33eaf3982aa4a7d78f4
3
- size 4954807928
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:123ea8a6b3a9f7d965d20aa0c7ccccd6e16596ef108eee452388dadbd64b99bf
3
+ size 4954810240
model-00004-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:04629d82de62f0f7919ff53fc049cdc3fb2d89379a45090c68b225b8498973f1
3
- size 3022452968
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f6e9ac2dc166c3e66fe56e2ce1f21ba620e96c4051308896255bd17e8302944d
3
+ size 3022388120
model.safetensors.index.json CHANGED
@@ -1,443 +1,435 @@
1
  {
2
  "metadata": {
3
  "total_parameters": 8953803264,
4
- "total_size": 17907672064
5
  },
6
  "weight_map": {
7
  "lm_head.weight": "model-00001-of-00004.safetensors",
8
- "model.embed_tokens.weight": "model-00001-of-00004.safetensors",
9
- "model.layers.0.input_layernorm.weight": "model-00001-of-00004.safetensors",
10
- "model.layers.0.linear_attn.A_log": "model-00001-of-00004.safetensors",
11
- "model.layers.0.linear_attn.conv1d.weight": "model-00001-of-00004.safetensors",
12
- "model.layers.0.linear_attn.dt_bias": "model-00001-of-00004.safetensors",
13
- "model.layers.0.linear_attn.in_proj_a.weight": "model-00001-of-00004.safetensors",
14
- "model.layers.0.linear_attn.in_proj_b.weight": "model-00001-of-00004.safetensors",
15
- "model.layers.0.linear_attn.in_proj_qkv.weight": "model-00001-of-00004.safetensors",
16
- "model.layers.0.linear_attn.in_proj_z.weight": "model-00001-of-00004.safetensors",
17
- "model.layers.0.linear_attn.norm.weight": "model-00001-of-00004.safetensors",
18
- "model.layers.0.linear_attn.out_proj.weight": "model-00001-of-00004.safetensors",
19
- "model.layers.0.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
20
- "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
21
- "model.layers.0.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
22
- "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
23
- "model.layers.1.input_layernorm.weight": "model-00001-of-00004.safetensors",
24
- "model.layers.1.linear_attn.A_log": "model-00001-of-00004.safetensors",
25
- "model.layers.1.linear_attn.conv1d.weight": "model-00001-of-00004.safetensors",
26
- "model.layers.1.linear_attn.dt_bias": "model-00001-of-00004.safetensors",
27
- "model.layers.1.linear_attn.in_proj_a.weight": "model-00001-of-00004.safetensors",
28
- "model.layers.1.linear_attn.in_proj_b.weight": "model-00001-of-00004.safetensors",
29
- "model.layers.1.linear_attn.in_proj_qkv.weight": "model-00001-of-00004.safetensors",
30
- "model.layers.1.linear_attn.in_proj_z.weight": "model-00001-of-00004.safetensors",
31
- "model.layers.1.linear_attn.norm.weight": "model-00001-of-00004.safetensors",
32
- "model.layers.1.linear_attn.out_proj.weight": "model-00001-of-00004.safetensors",
33
- "model.layers.1.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
34
- "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
35
- "model.layers.1.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
36
- "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
37
- "model.layers.10.input_layernorm.weight": "model-00002-of-00004.safetensors",
38
- "model.layers.10.linear_attn.A_log": "model-00002-of-00004.safetensors",
39
- "model.layers.10.linear_attn.conv1d.weight": "model-00002-of-00004.safetensors",
40
- "model.layers.10.linear_attn.dt_bias": "model-00002-of-00004.safetensors",
41
- "model.layers.10.linear_attn.in_proj_a.weight": "model-00002-of-00004.safetensors",
42
- "model.layers.10.linear_attn.in_proj_b.weight": "model-00002-of-00004.safetensors",
43
- "model.layers.10.linear_attn.in_proj_qkv.weight": "model-00002-of-00004.safetensors",
44
- "model.layers.10.linear_attn.in_proj_z.weight": "model-00002-of-00004.safetensors",
45
- "model.layers.10.linear_attn.norm.weight": "model-00002-of-00004.safetensors",
46
- "model.layers.10.linear_attn.out_proj.weight": "model-00002-of-00004.safetensors",
47
- "model.layers.10.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
48
- "model.layers.10.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
49
- "model.layers.10.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
50
- "model.layers.10.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
51
- "model.layers.11.input_layernorm.weight": "model-00002-of-00004.safetensors",
52
- "model.layers.11.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
53
- "model.layers.11.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
54
- "model.layers.11.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
55
- "model.layers.11.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
56
- "model.layers.11.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
57
- "model.layers.11.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
58
- "model.layers.11.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
59
- "model.layers.11.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
60
- "model.layers.11.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
61
- "model.layers.11.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
62
- "model.layers.12.input_layernorm.weight": "model-00002-of-00004.safetensors",
63
- "model.layers.12.linear_attn.A_log": "model-00002-of-00004.safetensors",
64
- "model.layers.12.linear_attn.conv1d.weight": "model-00002-of-00004.safetensors",
65
- "model.layers.12.linear_attn.dt_bias": "model-00002-of-00004.safetensors",
66
- "model.layers.12.linear_attn.in_proj_a.weight": "model-00002-of-00004.safetensors",
67
- "model.layers.12.linear_attn.in_proj_b.weight": "model-00002-of-00004.safetensors",
68
- "model.layers.12.linear_attn.in_proj_qkv.weight": "model-00002-of-00004.safetensors",
69
- "model.layers.12.linear_attn.in_proj_z.weight": "model-00002-of-00004.safetensors",
70
- "model.layers.12.linear_attn.norm.weight": "model-00002-of-00004.safetensors",
71
- "model.layers.12.linear_attn.out_proj.weight": "model-00002-of-00004.safetensors",
72
- "model.layers.12.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
73
- "model.layers.12.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
74
- "model.layers.12.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
75
- "model.layers.12.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
76
- "model.layers.13.input_layernorm.weight": "model-00002-of-00004.safetensors",
77
- "model.layers.13.linear_attn.A_log": "model-00002-of-00004.safetensors",
78
- "model.layers.13.linear_attn.conv1d.weight": "model-00002-of-00004.safetensors",
79
- "model.layers.13.linear_attn.dt_bias": "model-00002-of-00004.safetensors",
80
- "model.layers.13.linear_attn.in_proj_a.weight": "model-00002-of-00004.safetensors",
81
- "model.layers.13.linear_attn.in_proj_b.weight": "model-00002-of-00004.safetensors",
82
- "model.layers.13.linear_attn.in_proj_qkv.weight": "model-00002-of-00004.safetensors",
83
- "model.layers.13.linear_attn.in_proj_z.weight": "model-00002-of-00004.safetensors",
84
- "model.layers.13.linear_attn.norm.weight": "model-00002-of-00004.safetensors",
85
- "model.layers.13.linear_attn.out_proj.weight": "model-00002-of-00004.safetensors",
86
- "model.layers.13.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
87
- "model.layers.13.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
88
- "model.layers.13.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
89
- "model.layers.13.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
90
- "model.layers.14.input_layernorm.weight": "model-00003-of-00004.safetensors",
91
- "model.layers.14.linear_attn.A_log": "model-00003-of-00004.safetensors",
92
- "model.layers.14.linear_attn.conv1d.weight": "model-00003-of-00004.safetensors",
93
- "model.layers.14.linear_attn.dt_bias": "model-00003-of-00004.safetensors",
94
- "model.layers.14.linear_attn.in_proj_a.weight": "model-00003-of-00004.safetensors",
95
- "model.layers.14.linear_attn.in_proj_b.weight": "model-00003-of-00004.safetensors",
96
- "model.layers.14.linear_attn.in_proj_qkv.weight": "model-00003-of-00004.safetensors",
97
- "model.layers.14.linear_attn.in_proj_z.weight": "model-00003-of-00004.safetensors",
98
- "model.layers.14.linear_attn.norm.weight": "model-00003-of-00004.safetensors",
99
- "model.layers.14.linear_attn.out_proj.weight": "model-00003-of-00004.safetensors",
100
- "model.layers.14.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
101
- "model.layers.14.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
102
- "model.layers.14.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
103
- "model.layers.14.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
104
- "model.layers.15.input_layernorm.weight": "model-00003-of-00004.safetensors",
105
- "model.layers.15.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
106
- "model.layers.15.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
107
- "model.layers.15.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
108
- "model.layers.15.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
109
- "model.layers.15.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
110
- "model.layers.15.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
111
- "model.layers.15.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
112
- "model.layers.15.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
113
- "model.layers.15.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
114
- "model.layers.15.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
115
- "model.layers.16.input_layernorm.weight": "model-00003-of-00004.safetensors",
116
- "model.layers.16.linear_attn.A_log": "model-00003-of-00004.safetensors",
117
- "model.layers.16.linear_attn.conv1d.weight": "model-00003-of-00004.safetensors",
118
- "model.layers.16.linear_attn.dt_bias": "model-00003-of-00004.safetensors",
119
- "model.layers.16.linear_attn.in_proj_a.weight": "model-00003-of-00004.safetensors",
120
- "model.layers.16.linear_attn.in_proj_b.weight": "model-00003-of-00004.safetensors",
121
- "model.layers.16.linear_attn.in_proj_qkv.weight": "model-00003-of-00004.safetensors",
122
- "model.layers.16.linear_attn.in_proj_z.weight": "model-00003-of-00004.safetensors",
123
- "model.layers.16.linear_attn.norm.weight": "model-00003-of-00004.safetensors",
124
- "model.layers.16.linear_attn.out_proj.weight": "model-00003-of-00004.safetensors",
125
- "model.layers.16.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
126
- "model.layers.16.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
127
- "model.layers.16.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
128
- "model.layers.16.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
129
- "model.layers.17.input_layernorm.weight": "model-00003-of-00004.safetensors",
130
- "model.layers.17.linear_attn.A_log": "model-00003-of-00004.safetensors",
131
- "model.layers.17.linear_attn.conv1d.weight": "model-00003-of-00004.safetensors",
132
- "model.layers.17.linear_attn.dt_bias": "model-00003-of-00004.safetensors",
133
- "model.layers.17.linear_attn.in_proj_a.weight": "model-00003-of-00004.safetensors",
134
- "model.layers.17.linear_attn.in_proj_b.weight": "model-00003-of-00004.safetensors",
135
- "model.layers.17.linear_attn.in_proj_qkv.weight": "model-00003-of-00004.safetensors",
136
- "model.layers.17.linear_attn.in_proj_z.weight": "model-00003-of-00004.safetensors",
137
- "model.layers.17.linear_attn.norm.weight": "model-00003-of-00004.safetensors",
138
- "model.layers.17.linear_attn.out_proj.weight": "model-00003-of-00004.safetensors",
139
- "model.layers.17.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
140
- "model.layers.17.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
141
- "model.layers.17.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
142
- "model.layers.17.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
143
- "model.layers.18.input_layernorm.weight": "model-00003-of-00004.safetensors",
144
- "model.layers.18.linear_attn.A_log": "model-00003-of-00004.safetensors",
145
- "model.layers.18.linear_attn.conv1d.weight": "model-00003-of-00004.safetensors",
146
- "model.layers.18.linear_attn.dt_bias": "model-00003-of-00004.safetensors",
147
- "model.layers.18.linear_attn.in_proj_a.weight": "model-00003-of-00004.safetensors",
148
- "model.layers.18.linear_attn.in_proj_b.weight": "model-00003-of-00004.safetensors",
149
- "model.layers.18.linear_attn.in_proj_qkv.weight": "model-00003-of-00004.safetensors",
150
- "model.layers.18.linear_attn.in_proj_z.weight": "model-00003-of-00004.safetensors",
151
- "model.layers.18.linear_attn.norm.weight": "model-00003-of-00004.safetensors",
152
- "model.layers.18.linear_attn.out_proj.weight": "model-00003-of-00004.safetensors",
153
- "model.layers.18.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
154
- "model.layers.18.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
155
- "model.layers.18.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
156
- "model.layers.18.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
157
- "model.layers.19.input_layernorm.weight": "model-00003-of-00004.safetensors",
158
- "model.layers.19.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
159
- "model.layers.19.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
160
- "model.layers.19.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
161
- "model.layers.19.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
162
- "model.layers.19.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
163
- "model.layers.19.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
164
- "model.layers.19.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
165
- "model.layers.19.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
166
- "model.layers.19.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
167
- "model.layers.19.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
168
- "model.layers.2.input_layernorm.weight": "model-00001-of-00004.safetensors",
169
- "model.layers.2.linear_attn.A_log": "model-00001-of-00004.safetensors",
170
- "model.layers.2.linear_attn.conv1d.weight": "model-00001-of-00004.safetensors",
171
- "model.layers.2.linear_attn.dt_bias": "model-00001-of-00004.safetensors",
172
- "model.layers.2.linear_attn.in_proj_a.weight": "model-00001-of-00004.safetensors",
173
- "model.layers.2.linear_attn.in_proj_b.weight": "model-00001-of-00004.safetensors",
174
- "model.layers.2.linear_attn.in_proj_qkv.weight": "model-00002-of-00004.safetensors",
175
- "model.layers.2.linear_attn.in_proj_z.weight": "model-00002-of-00004.safetensors",
176
- "model.layers.2.linear_attn.norm.weight": "model-00002-of-00004.safetensors",
177
- "model.layers.2.linear_attn.out_proj.weight": "model-00002-of-00004.safetensors",
178
- "model.layers.2.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
179
- "model.layers.2.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
180
- "model.layers.2.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
181
- "model.layers.2.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
182
- "model.layers.20.input_layernorm.weight": "model-00003-of-00004.safetensors",
183
- "model.layers.20.linear_attn.A_log": "model-00003-of-00004.safetensors",
184
- "model.layers.20.linear_attn.conv1d.weight": "model-00003-of-00004.safetensors",
185
- "model.layers.20.linear_attn.dt_bias": "model-00003-of-00004.safetensors",
186
- "model.layers.20.linear_attn.in_proj_a.weight": "model-00003-of-00004.safetensors",
187
- "model.layers.20.linear_attn.in_proj_b.weight": "model-00003-of-00004.safetensors",
188
- "model.layers.20.linear_attn.in_proj_qkv.weight": "model-00003-of-00004.safetensors",
189
- "model.layers.20.linear_attn.in_proj_z.weight": "model-00003-of-00004.safetensors",
190
- "model.layers.20.linear_attn.norm.weight": "model-00003-of-00004.safetensors",
191
- "model.layers.20.linear_attn.out_proj.weight": "model-00003-of-00004.safetensors",
192
- "model.layers.20.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
193
- "model.layers.20.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
194
- "model.layers.20.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
195
- "model.layers.20.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
196
- "model.layers.21.input_layernorm.weight": "model-00003-of-00004.safetensors",
197
- "model.layers.21.linear_attn.A_log": "model-00003-of-00004.safetensors",
198
- "model.layers.21.linear_attn.conv1d.weight": "model-00003-of-00004.safetensors",
199
- "model.layers.21.linear_attn.dt_bias": "model-00003-of-00004.safetensors",
200
- "model.layers.21.linear_attn.in_proj_a.weight": "model-00003-of-00004.safetensors",
201
- "model.layers.21.linear_attn.in_proj_b.weight": "model-00003-of-00004.safetensors",
202
- "model.layers.21.linear_attn.in_proj_qkv.weight": "model-00003-of-00004.safetensors",
203
- "model.layers.21.linear_attn.in_proj_z.weight": "model-00003-of-00004.safetensors",
204
- "model.layers.21.linear_attn.norm.weight": "model-00003-of-00004.safetensors",
205
- "model.layers.21.linear_attn.out_proj.weight": "model-00003-of-00004.safetensors",
206
- "model.layers.21.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
207
- "model.layers.21.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
208
- "model.layers.21.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
209
- "model.layers.21.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
210
- "model.layers.22.input_layernorm.weight": "model-00003-of-00004.safetensors",
211
- "model.layers.22.linear_attn.A_log": "model-00003-of-00004.safetensors",
212
- "model.layers.22.linear_attn.conv1d.weight": "model-00003-of-00004.safetensors",
213
- "model.layers.22.linear_attn.dt_bias": "model-00003-of-00004.safetensors",
214
- "model.layers.22.linear_attn.in_proj_a.weight": "model-00003-of-00004.safetensors",
215
- "model.layers.22.linear_attn.in_proj_b.weight": "model-00003-of-00004.safetensors",
216
- "model.layers.22.linear_attn.in_proj_qkv.weight": "model-00003-of-00004.safetensors",
217
- "model.layers.22.linear_attn.in_proj_z.weight": "model-00003-of-00004.safetensors",
218
- "model.layers.22.linear_attn.norm.weight": "model-00003-of-00004.safetensors",
219
- "model.layers.22.linear_attn.out_proj.weight": "model-00003-of-00004.safetensors",
220
- "model.layers.22.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
221
- "model.layers.22.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
222
- "model.layers.22.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
223
- "model.layers.22.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
224
- "model.layers.23.input_layernorm.weight": "model-00003-of-00004.safetensors",
225
- "model.layers.23.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
226
- "model.layers.23.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
227
- "model.layers.23.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
228
- "model.layers.23.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
229
- "model.layers.23.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
230
- "model.layers.23.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
231
- "model.layers.23.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
232
- "model.layers.23.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
233
- "model.layers.23.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
234
- "model.layers.23.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
235
- "model.layers.24.input_layernorm.weight": "model-00003-of-00004.safetensors",
236
- "model.layers.24.linear_attn.A_log": "model-00003-of-00004.safetensors",
237
- "model.layers.24.linear_attn.conv1d.weight": "model-00003-of-00004.safetensors",
238
- "model.layers.24.linear_attn.dt_bias": "model-00003-of-00004.safetensors",
239
- "model.layers.24.linear_attn.in_proj_a.weight": "model-00003-of-00004.safetensors",
240
- "model.layers.24.linear_attn.in_proj_b.weight": "model-00003-of-00004.safetensors",
241
- "model.layers.24.linear_attn.in_proj_qkv.weight": "model-00003-of-00004.safetensors",
242
- "model.layers.24.linear_attn.in_proj_z.weight": "model-00003-of-00004.safetensors",
243
- "model.layers.24.linear_attn.norm.weight": "model-00003-of-00004.safetensors",
244
- "model.layers.24.linear_attn.out_proj.weight": "model-00003-of-00004.safetensors",
245
- "model.layers.24.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
246
- "model.layers.24.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
247
- "model.layers.24.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
248
- "model.layers.24.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
249
- "model.layers.25.input_layernorm.weight": "model-00003-of-00004.safetensors",
250
- "model.layers.25.linear_attn.A_log": "model-00003-of-00004.safetensors",
251
- "model.layers.25.linear_attn.conv1d.weight": "model-00003-of-00004.safetensors",
252
- "model.layers.25.linear_attn.dt_bias": "model-00003-of-00004.safetensors",
253
- "model.layers.25.linear_attn.in_proj_a.weight": "model-00003-of-00004.safetensors",
254
- "model.layers.25.linear_attn.in_proj_b.weight": "model-00003-of-00004.safetensors",
255
- "model.layers.25.linear_attn.in_proj_qkv.weight": "model-00004-of-00004.safetensors",
256
- "model.layers.25.linear_attn.in_proj_z.weight": "model-00004-of-00004.safetensors",
257
- "model.layers.25.linear_attn.norm.weight": "model-00004-of-00004.safetensors",
258
- "model.layers.25.linear_attn.out_proj.weight": "model-00004-of-00004.safetensors",
259
- "model.layers.25.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
260
- "model.layers.25.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
261
- "model.layers.25.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
262
- "model.layers.25.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
263
- "model.layers.26.input_layernorm.weight": "model-00004-of-00004.safetensors",
264
- "model.layers.26.linear_attn.A_log": "model-00004-of-00004.safetensors",
265
- "model.layers.26.linear_attn.conv1d.weight": "model-00004-of-00004.safetensors",
266
- "model.layers.26.linear_attn.dt_bias": "model-00004-of-00004.safetensors",
267
- "model.layers.26.linear_attn.in_proj_a.weight": "model-00004-of-00004.safetensors",
268
- "model.layers.26.linear_attn.in_proj_b.weight": "model-00004-of-00004.safetensors",
269
- "model.layers.26.linear_attn.in_proj_qkv.weight": "model-00004-of-00004.safetensors",
270
- "model.layers.26.linear_attn.in_proj_z.weight": "model-00004-of-00004.safetensors",
271
- "model.layers.26.linear_attn.norm.weight": "model-00004-of-00004.safetensors",
272
- "model.layers.26.linear_attn.out_proj.weight": "model-00004-of-00004.safetensors",
273
- "model.layers.26.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
274
- "model.layers.26.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
275
- "model.layers.26.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
276
- "model.layers.26.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
277
- "model.layers.27.input_layernorm.weight": "model-00004-of-00004.safetensors",
278
- "model.layers.27.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
279
- "model.layers.27.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
280
- "model.layers.27.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
281
- "model.layers.27.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
282
- "model.layers.27.self_attn.k_norm.weight": "model-00004-of-00004.safetensors",
283
- "model.layers.27.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
284
- "model.layers.27.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
285
- "model.layers.27.self_attn.q_norm.weight": "model-00004-of-00004.safetensors",
286
- "model.layers.27.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
287
- "model.layers.27.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
288
- "model.layers.28.input_layernorm.weight": "model-00004-of-00004.safetensors",
289
- "model.layers.28.linear_attn.A_log": "model-00004-of-00004.safetensors",
290
- "model.layers.28.linear_attn.conv1d.weight": "model-00004-of-00004.safetensors",
291
- "model.layers.28.linear_attn.dt_bias": "model-00004-of-00004.safetensors",
292
- "model.layers.28.linear_attn.in_proj_a.weight": "model-00004-of-00004.safetensors",
293
- "model.layers.28.linear_attn.in_proj_b.weight": "model-00004-of-00004.safetensors",
294
- "model.layers.28.linear_attn.in_proj_qkv.weight": "model-00004-of-00004.safetensors",
295
- "model.layers.28.linear_attn.in_proj_z.weight": "model-00004-of-00004.safetensors",
296
- "model.layers.28.linear_attn.norm.weight": "model-00004-of-00004.safetensors",
297
- "model.layers.28.linear_attn.out_proj.weight": "model-00004-of-00004.safetensors",
298
- "model.layers.28.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
299
- "model.layers.28.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
300
- "model.layers.28.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
301
- "model.layers.28.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
302
- "model.layers.29.input_layernorm.weight": "model-00004-of-00004.safetensors",
303
- "model.layers.29.linear_attn.A_log": "model-00004-of-00004.safetensors",
304
- "model.layers.29.linear_attn.conv1d.weight": "model-00004-of-00004.safetensors",
305
- "model.layers.29.linear_attn.dt_bias": "model-00004-of-00004.safetensors",
306
- "model.layers.29.linear_attn.in_proj_a.weight": "model-00004-of-00004.safetensors",
307
- "model.layers.29.linear_attn.in_proj_b.weight": "model-00004-of-00004.safetensors",
308
- "model.layers.29.linear_attn.in_proj_qkv.weight": "model-00004-of-00004.safetensors",
309
- "model.layers.29.linear_attn.in_proj_z.weight": "model-00004-of-00004.safetensors",
310
- "model.layers.29.linear_attn.norm.weight": "model-00004-of-00004.safetensors",
311
- "model.layers.29.linear_attn.out_proj.weight": "model-00004-of-00004.safetensors",
312
- "model.layers.29.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
313
- "model.layers.29.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
314
- "model.layers.29.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
315
- "model.layers.29.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
316
- "model.layers.3.input_layernorm.weight": "model-00002-of-00004.safetensors",
317
- "model.layers.3.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
318
- "model.layers.3.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
319
- "model.layers.3.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
320
- "model.layers.3.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
321
- "model.layers.3.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
322
- "model.layers.3.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
323
- "model.layers.3.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
324
- "model.layers.3.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
325
- "model.layers.3.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
326
- "model.layers.3.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
327
- "model.layers.30.input_layernorm.weight": "model-00004-of-00004.safetensors",
328
- "model.layers.30.linear_attn.A_log": "model-00004-of-00004.safetensors",
329
- "model.layers.30.linear_attn.conv1d.weight": "model-00004-of-00004.safetensors",
330
- "model.layers.30.linear_attn.dt_bias": "model-00004-of-00004.safetensors",
331
- "model.layers.30.linear_attn.in_proj_a.weight": "model-00004-of-00004.safetensors",
332
- "model.layers.30.linear_attn.in_proj_b.weight": "model-00004-of-00004.safetensors",
333
- "model.layers.30.linear_attn.in_proj_qkv.weight": "model-00004-of-00004.safetensors",
334
- "model.layers.30.linear_attn.in_proj_z.weight": "model-00004-of-00004.safetensors",
335
- "model.layers.30.linear_attn.norm.weight": "model-00004-of-00004.safetensors",
336
- "model.layers.30.linear_attn.out_proj.weight": "model-00004-of-00004.safetensors",
337
- "model.layers.30.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
338
- "model.layers.30.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
339
- "model.layers.30.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
340
- "model.layers.30.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
341
- "model.layers.31.input_layernorm.weight": "model-00004-of-00004.safetensors",
342
- "model.layers.31.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
343
- "model.layers.31.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
344
- "model.layers.31.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
345
- "model.layers.31.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
346
- "model.layers.31.self_attn.k_norm.weight": "model-00004-of-00004.safetensors",
347
- "model.layers.31.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
348
- "model.layers.31.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
349
- "model.layers.31.self_attn.q_norm.weight": "model-00004-of-00004.safetensors",
350
- "model.layers.31.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
351
- "model.layers.31.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
352
- "model.layers.4.input_layernorm.weight": "model-00002-of-00004.safetensors",
353
- "model.layers.4.linear_attn.A_log": "model-00002-of-00004.safetensors",
354
- "model.layers.4.linear_attn.conv1d.weight": "model-00002-of-00004.safetensors",
355
- "model.layers.4.linear_attn.dt_bias": "model-00002-of-00004.safetensors",
356
- "model.layers.4.linear_attn.in_proj_a.weight": "model-00002-of-00004.safetensors",
357
- "model.layers.4.linear_attn.in_proj_b.weight": "model-00002-of-00004.safetensors",
358
- "model.layers.4.linear_attn.in_proj_qkv.weight": "model-00002-of-00004.safetensors",
359
- "model.layers.4.linear_attn.in_proj_z.weight": "model-00002-of-00004.safetensors",
360
- "model.layers.4.linear_attn.norm.weight": "model-00002-of-00004.safetensors",
361
- "model.layers.4.linear_attn.out_proj.weight": "model-00002-of-00004.safetensors",
362
- "model.layers.4.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
363
- "model.layers.4.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
364
- "model.layers.4.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
365
- "model.layers.4.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
366
- "model.layers.5.input_layernorm.weight": "model-00002-of-00004.safetensors",
367
- "model.layers.5.linear_attn.A_log": "model-00002-of-00004.safetensors",
368
- "model.layers.5.linear_attn.conv1d.weight": "model-00002-of-00004.safetensors",
369
- "model.layers.5.linear_attn.dt_bias": "model-00002-of-00004.safetensors",
370
- "model.layers.5.linear_attn.in_proj_a.weight": "model-00002-of-00004.safetensors",
371
- "model.layers.5.linear_attn.in_proj_b.weight": "model-00002-of-00004.safetensors",
372
- "model.layers.5.linear_attn.in_proj_qkv.weight": "model-00002-of-00004.safetensors",
373
- "model.layers.5.linear_attn.in_proj_z.weight": "model-00002-of-00004.safetensors",
374
- "model.layers.5.linear_attn.norm.weight": "model-00002-of-00004.safetensors",
375
- "model.layers.5.linear_attn.out_proj.weight": "model-00002-of-00004.safetensors",
376
- "model.layers.5.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
377
- "model.layers.5.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
378
- "model.layers.5.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
379
- "model.layers.5.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
380
- "model.layers.6.input_layernorm.weight": "model-00002-of-00004.safetensors",
381
- "model.layers.6.linear_attn.A_log": "model-00002-of-00004.safetensors",
382
- "model.layers.6.linear_attn.conv1d.weight": "model-00002-of-00004.safetensors",
383
- "model.layers.6.linear_attn.dt_bias": "model-00002-of-00004.safetensors",
384
- "model.layers.6.linear_attn.in_proj_a.weight": "model-00002-of-00004.safetensors",
385
- "model.layers.6.linear_attn.in_proj_b.weight": "model-00002-of-00004.safetensors",
386
- "model.layers.6.linear_attn.in_proj_qkv.weight": "model-00002-of-00004.safetensors",
387
- "model.layers.6.linear_attn.in_proj_z.weight": "model-00002-of-00004.safetensors",
388
- "model.layers.6.linear_attn.norm.weight": "model-00002-of-00004.safetensors",
389
- "model.layers.6.linear_attn.out_proj.weight": "model-00002-of-00004.safetensors",
390
- "model.layers.6.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
391
- "model.layers.6.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
392
- "model.layers.6.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
393
- "model.layers.6.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
394
- "model.layers.7.input_layernorm.weight": "model-00002-of-00004.safetensors",
395
- "model.layers.7.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
396
- "model.layers.7.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
397
- "model.layers.7.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
398
- "model.layers.7.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
399
- "model.layers.7.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
400
- "model.layers.7.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
401
- "model.layers.7.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
402
- "model.layers.7.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
403
- "model.layers.7.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
404
- "model.layers.7.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
405
- "model.layers.8.input_layernorm.weight": "model-00002-of-00004.safetensors",
406
- "model.layers.8.linear_attn.A_log": "model-00002-of-00004.safetensors",
407
- "model.layers.8.linear_attn.conv1d.weight": "model-00002-of-00004.safetensors",
408
- "model.layers.8.linear_attn.dt_bias": "model-00002-of-00004.safetensors",
409
- "model.layers.8.linear_attn.in_proj_a.weight": "model-00002-of-00004.safetensors",
410
- "model.layers.8.linear_attn.in_proj_b.weight": "model-00002-of-00004.safetensors",
411
- "model.layers.8.linear_attn.in_proj_qkv.weight": "model-00002-of-00004.safetensors",
412
- "model.layers.8.linear_attn.in_proj_z.weight": "model-00002-of-00004.safetensors",
413
- "model.layers.8.linear_attn.norm.weight": "model-00002-of-00004.safetensors",
414
- "model.layers.8.linear_attn.out_proj.weight": "model-00002-of-00004.safetensors",
415
- "model.layers.8.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
416
- "model.layers.8.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
417
- "model.layers.8.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
418
- "model.layers.8.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
419
- "model.layers.9.input_layernorm.weight": "model-00002-of-00004.safetensors",
420
- "model.layers.9.linear_attn.A_log": "model-00002-of-00004.safetensors",
421
- "model.layers.9.linear_attn.conv1d.weight": "model-00002-of-00004.safetensors",
422
- "model.layers.9.linear_attn.dt_bias": "model-00002-of-00004.safetensors",
423
- "model.layers.9.linear_attn.in_proj_a.weight": "model-00002-of-00004.safetensors",
424
- "model.layers.9.linear_attn.in_proj_b.weight": "model-00002-of-00004.safetensors",
425
- "model.layers.9.linear_attn.in_proj_qkv.weight": "model-00002-of-00004.safetensors",
426
- "model.layers.9.linear_attn.in_proj_z.weight": "model-00002-of-00004.safetensors",
427
- "model.layers.9.linear_attn.norm.weight": "model-00002-of-00004.safetensors",
428
- "model.layers.9.linear_attn.out_proj.weight": "model-00002-of-00004.safetensors",
429
- "model.layers.9.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
430
- "model.layers.9.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
431
- "model.layers.9.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
432
- "model.layers.9.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
433
- "model.norm.weight": "model-00004-of-00004.safetensors",
434
- "q_0": "model-00004-of-00004.safetensors",
435
- "q_1": "model-00004-of-00004.safetensors",
436
- "q_2": "model-00004-of-00004.safetensors",
437
- "q_3": "model-00004-of-00004.safetensors",
438
- "q_4": "model-00004-of-00004.safetensors",
439
- "q_5": "model-00004-of-00004.safetensors",
440
- "q_6": "model-00004-of-00004.safetensors",
441
- "q_7": "model-00004-of-00004.safetensors"
442
  }
443
  }
 
1
  {
2
  "metadata": {
3
  "total_parameters": 8953803264,
4
+ "total_size": 17907606528
5
  },
6
  "weight_map": {
7
  "lm_head.weight": "model-00001-of-00004.safetensors",
8
+ "model.language_model.embed_tokens.weight": "model-00001-of-00004.safetensors",
9
+ "model.language_model.layers.0.input_layernorm.weight": "model-00001-of-00004.safetensors",
10
+ "model.language_model.layers.0.linear_attn.A_log": "model-00001-of-00004.safetensors",
11
+ "model.language_model.layers.0.linear_attn.conv1d.weight": "model-00001-of-00004.safetensors",
12
+ "model.language_model.layers.0.linear_attn.dt_bias": "model-00001-of-00004.safetensors",
13
+ "model.language_model.layers.0.linear_attn.in_proj_a.weight": "model-00001-of-00004.safetensors",
14
+ "model.language_model.layers.0.linear_attn.in_proj_b.weight": "model-00001-of-00004.safetensors",
15
+ "model.language_model.layers.0.linear_attn.in_proj_qkv.weight": "model-00001-of-00004.safetensors",
16
+ "model.language_model.layers.0.linear_attn.in_proj_z.weight": "model-00001-of-00004.safetensors",
17
+ "model.language_model.layers.0.linear_attn.norm.weight": "model-00001-of-00004.safetensors",
18
+ "model.language_model.layers.0.linear_attn.out_proj.weight": "model-00001-of-00004.safetensors",
19
+ "model.language_model.layers.0.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
20
+ "model.language_model.layers.0.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
21
+ "model.language_model.layers.0.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
22
+ "model.language_model.layers.0.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
23
+ "model.language_model.layers.1.input_layernorm.weight": "model-00001-of-00004.safetensors",
24
+ "model.language_model.layers.1.linear_attn.A_log": "model-00001-of-00004.safetensors",
25
+ "model.language_model.layers.1.linear_attn.conv1d.weight": "model-00001-of-00004.safetensors",
26
+ "model.language_model.layers.1.linear_attn.dt_bias": "model-00001-of-00004.safetensors",
27
+ "model.language_model.layers.1.linear_attn.in_proj_a.weight": "model-00001-of-00004.safetensors",
28
+ "model.language_model.layers.1.linear_attn.in_proj_b.weight": "model-00001-of-00004.safetensors",
29
+ "model.language_model.layers.1.linear_attn.in_proj_qkv.weight": "model-00001-of-00004.safetensors",
30
+ "model.language_model.layers.1.linear_attn.in_proj_z.weight": "model-00001-of-00004.safetensors",
31
+ "model.language_model.layers.1.linear_attn.norm.weight": "model-00001-of-00004.safetensors",
32
+ "model.language_model.layers.1.linear_attn.out_proj.weight": "model-00001-of-00004.safetensors",
33
+ "model.language_model.layers.1.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
34
+ "model.language_model.layers.1.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
35
+ "model.language_model.layers.1.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
36
+ "model.language_model.layers.1.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
37
+ "model.language_model.layers.10.input_layernorm.weight": "model-00002-of-00004.safetensors",
38
+ "model.language_model.layers.10.linear_attn.A_log": "model-00002-of-00004.safetensors",
39
+ "model.language_model.layers.10.linear_attn.conv1d.weight": "model-00002-of-00004.safetensors",
40
+ "model.language_model.layers.10.linear_attn.dt_bias": "model-00002-of-00004.safetensors",
41
+ "model.language_model.layers.10.linear_attn.in_proj_a.weight": "model-00002-of-00004.safetensors",
42
+ "model.language_model.layers.10.linear_attn.in_proj_b.weight": "model-00002-of-00004.safetensors",
43
+ "model.language_model.layers.10.linear_attn.in_proj_qkv.weight": "model-00002-of-00004.safetensors",
44
+ "model.language_model.layers.10.linear_attn.in_proj_z.weight": "model-00002-of-00004.safetensors",
45
+ "model.language_model.layers.10.linear_attn.norm.weight": "model-00002-of-00004.safetensors",
46
+ "model.language_model.layers.10.linear_attn.out_proj.weight": "model-00002-of-00004.safetensors",
47
+ "model.language_model.layers.10.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
48
+ "model.language_model.layers.10.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
49
+ "model.language_model.layers.10.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
50
+ "model.language_model.layers.10.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
51
+ "model.language_model.layers.11.input_layernorm.weight": "model-00002-of-00004.safetensors",
52
+ "model.language_model.layers.11.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
53
+ "model.language_model.layers.11.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
54
+ "model.language_model.layers.11.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
55
+ "model.language_model.layers.11.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
56
+ "model.language_model.layers.11.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
57
+ "model.language_model.layers.11.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
58
+ "model.language_model.layers.11.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
59
+ "model.language_model.layers.11.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
60
+ "model.language_model.layers.11.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
61
+ "model.language_model.layers.11.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
62
+ "model.language_model.layers.12.input_layernorm.weight": "model-00002-of-00004.safetensors",
63
+ "model.language_model.layers.12.linear_attn.A_log": "model-00002-of-00004.safetensors",
64
+ "model.language_model.layers.12.linear_attn.conv1d.weight": "model-00002-of-00004.safetensors",
65
+ "model.language_model.layers.12.linear_attn.dt_bias": "model-00002-of-00004.safetensors",
66
+ "model.language_model.layers.12.linear_attn.in_proj_a.weight": "model-00002-of-00004.safetensors",
67
+ "model.language_model.layers.12.linear_attn.in_proj_b.weight": "model-00002-of-00004.safetensors",
68
+ "model.language_model.layers.12.linear_attn.in_proj_qkv.weight": "model-00002-of-00004.safetensors",
69
+ "model.language_model.layers.12.linear_attn.in_proj_z.weight": "model-00002-of-00004.safetensors",
70
+ "model.language_model.layers.12.linear_attn.norm.weight": "model-00002-of-00004.safetensors",
71
+ "model.language_model.layers.12.linear_attn.out_proj.weight": "model-00002-of-00004.safetensors",
72
+ "model.language_model.layers.12.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
73
+ "model.language_model.layers.12.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
74
+ "model.language_model.layers.12.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
75
+ "model.language_model.layers.12.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
76
+ "model.language_model.layers.13.input_layernorm.weight": "model-00002-of-00004.safetensors",
77
+ "model.language_model.layers.13.linear_attn.A_log": "model-00002-of-00004.safetensors",
78
+ "model.language_model.layers.13.linear_attn.conv1d.weight": "model-00002-of-00004.safetensors",
79
+ "model.language_model.layers.13.linear_attn.dt_bias": "model-00002-of-00004.safetensors",
80
+ "model.language_model.layers.13.linear_attn.in_proj_a.weight": "model-00002-of-00004.safetensors",
81
+ "model.language_model.layers.13.linear_attn.in_proj_b.weight": "model-00002-of-00004.safetensors",
82
+ "model.language_model.layers.13.linear_attn.in_proj_qkv.weight": "model-00002-of-00004.safetensors",
83
+ "model.language_model.layers.13.linear_attn.in_proj_z.weight": "model-00002-of-00004.safetensors",
84
+ "model.language_model.layers.13.linear_attn.norm.weight": "model-00002-of-00004.safetensors",
85
+ "model.language_model.layers.13.linear_attn.out_proj.weight": "model-00002-of-00004.safetensors",
86
+ "model.language_model.layers.13.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
87
+ "model.language_model.layers.13.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
88
+ "model.language_model.layers.13.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
89
+ "model.language_model.layers.13.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
90
+ "model.language_model.layers.14.input_layernorm.weight": "model-00003-of-00004.safetensors",
91
+ "model.language_model.layers.14.linear_attn.A_log": "model-00003-of-00004.safetensors",
92
+ "model.language_model.layers.14.linear_attn.conv1d.weight": "model-00003-of-00004.safetensors",
93
+ "model.language_model.layers.14.linear_attn.dt_bias": "model-00003-of-00004.safetensors",
94
+ "model.language_model.layers.14.linear_attn.in_proj_a.weight": "model-00003-of-00004.safetensors",
95
+ "model.language_model.layers.14.linear_attn.in_proj_b.weight": "model-00003-of-00004.safetensors",
96
+ "model.language_model.layers.14.linear_attn.in_proj_qkv.weight": "model-00003-of-00004.safetensors",
97
+ "model.language_model.layers.14.linear_attn.in_proj_z.weight": "model-00003-of-00004.safetensors",
98
+ "model.language_model.layers.14.linear_attn.norm.weight": "model-00003-of-00004.safetensors",
99
+ "model.language_model.layers.14.linear_attn.out_proj.weight": "model-00003-of-00004.safetensors",
100
+ "model.language_model.layers.14.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
101
+ "model.language_model.layers.14.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
102
+ "model.language_model.layers.14.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
103
+ "model.language_model.layers.14.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
104
+ "model.language_model.layers.15.input_layernorm.weight": "model-00003-of-00004.safetensors",
105
+ "model.language_model.layers.15.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
106
+ "model.language_model.layers.15.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
107
+ "model.language_model.layers.15.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
108
+ "model.language_model.layers.15.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
109
+ "model.language_model.layers.15.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
110
+ "model.language_model.layers.15.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
111
+ "model.language_model.layers.15.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
112
+ "model.language_model.layers.15.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
113
+ "model.language_model.layers.15.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
114
+ "model.language_model.layers.15.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
115
+ "model.language_model.layers.16.input_layernorm.weight": "model-00003-of-00004.safetensors",
116
+ "model.language_model.layers.16.linear_attn.A_log": "model-00003-of-00004.safetensors",
117
+ "model.language_model.layers.16.linear_attn.conv1d.weight": "model-00003-of-00004.safetensors",
118
+ "model.language_model.layers.16.linear_attn.dt_bias": "model-00003-of-00004.safetensors",
119
+ "model.language_model.layers.16.linear_attn.in_proj_a.weight": "model-00003-of-00004.safetensors",
120
+ "model.language_model.layers.16.linear_attn.in_proj_b.weight": "model-00003-of-00004.safetensors",
121
+ "model.language_model.layers.16.linear_attn.in_proj_qkv.weight": "model-00003-of-00004.safetensors",
122
+ "model.language_model.layers.16.linear_attn.in_proj_z.weight": "model-00003-of-00004.safetensors",
123
+ "model.language_model.layers.16.linear_attn.norm.weight": "model-00003-of-00004.safetensors",
124
+ "model.language_model.layers.16.linear_attn.out_proj.weight": "model-00003-of-00004.safetensors",
125
+ "model.language_model.layers.16.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
126
+ "model.language_model.layers.16.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
127
+ "model.language_model.layers.16.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
128
+ "model.language_model.layers.16.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
129
+ "model.language_model.layers.17.input_layernorm.weight": "model-00003-of-00004.safetensors",
130
+ "model.language_model.layers.17.linear_attn.A_log": "model-00003-of-00004.safetensors",
131
+ "model.language_model.layers.17.linear_attn.conv1d.weight": "model-00003-of-00004.safetensors",
132
+ "model.language_model.layers.17.linear_attn.dt_bias": "model-00003-of-00004.safetensors",
133
+ "model.language_model.layers.17.linear_attn.in_proj_a.weight": "model-00003-of-00004.safetensors",
134
+ "model.language_model.layers.17.linear_attn.in_proj_b.weight": "model-00003-of-00004.safetensors",
135
+ "model.language_model.layers.17.linear_attn.in_proj_qkv.weight": "model-00003-of-00004.safetensors",
136
+ "model.language_model.layers.17.linear_attn.in_proj_z.weight": "model-00003-of-00004.safetensors",
137
+ "model.language_model.layers.17.linear_attn.norm.weight": "model-00003-of-00004.safetensors",
138
+ "model.language_model.layers.17.linear_attn.out_proj.weight": "model-00003-of-00004.safetensors",
139
+ "model.language_model.layers.17.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
140
+ "model.language_model.layers.17.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
141
+ "model.language_model.layers.17.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
142
+ "model.language_model.layers.17.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
143
+ "model.language_model.layers.18.input_layernorm.weight": "model-00003-of-00004.safetensors",
144
+ "model.language_model.layers.18.linear_attn.A_log": "model-00003-of-00004.safetensors",
145
+ "model.language_model.layers.18.linear_attn.conv1d.weight": "model-00003-of-00004.safetensors",
146
+ "model.language_model.layers.18.linear_attn.dt_bias": "model-00003-of-00004.safetensors",
147
+ "model.language_model.layers.18.linear_attn.in_proj_a.weight": "model-00003-of-00004.safetensors",
148
+ "model.language_model.layers.18.linear_attn.in_proj_b.weight": "model-00003-of-00004.safetensors",
149
+ "model.language_model.layers.18.linear_attn.in_proj_qkv.weight": "model-00003-of-00004.safetensors",
150
+ "model.language_model.layers.18.linear_attn.in_proj_z.weight": "model-00003-of-00004.safetensors",
151
+ "model.language_model.layers.18.linear_attn.norm.weight": "model-00003-of-00004.safetensors",
152
+ "model.language_model.layers.18.linear_attn.out_proj.weight": "model-00003-of-00004.safetensors",
153
+ "model.language_model.layers.18.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
154
+ "model.language_model.layers.18.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
155
+ "model.language_model.layers.18.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
156
+ "model.language_model.layers.18.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
157
+ "model.language_model.layers.19.input_layernorm.weight": "model-00003-of-00004.safetensors",
158
+ "model.language_model.layers.19.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
159
+ "model.language_model.layers.19.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
160
+ "model.language_model.layers.19.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
161
+ "model.language_model.layers.19.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
162
+ "model.language_model.layers.19.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
163
+ "model.language_model.layers.19.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
164
+ "model.language_model.layers.19.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
165
+ "model.language_model.layers.19.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
166
+ "model.language_model.layers.19.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
167
+ "model.language_model.layers.19.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
168
+ "model.language_model.layers.2.input_layernorm.weight": "model-00001-of-00004.safetensors",
169
+ "model.language_model.layers.2.linear_attn.A_log": "model-00001-of-00004.safetensors",
170
+ "model.language_model.layers.2.linear_attn.conv1d.weight": "model-00001-of-00004.safetensors",
171
+ "model.language_model.layers.2.linear_attn.dt_bias": "model-00001-of-00004.safetensors",
172
+ "model.language_model.layers.2.linear_attn.in_proj_a.weight": "model-00001-of-00004.safetensors",
173
+ "model.language_model.layers.2.linear_attn.in_proj_b.weight": "model-00001-of-00004.safetensors",
174
+ "model.language_model.layers.2.linear_attn.in_proj_qkv.weight": "model-00002-of-00004.safetensors",
175
+ "model.language_model.layers.2.linear_attn.in_proj_z.weight": "model-00002-of-00004.safetensors",
176
+ "model.language_model.layers.2.linear_attn.norm.weight": "model-00002-of-00004.safetensors",
177
+ "model.language_model.layers.2.linear_attn.out_proj.weight": "model-00002-of-00004.safetensors",
178
+ "model.language_model.layers.2.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
179
+ "model.language_model.layers.2.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
180
+ "model.language_model.layers.2.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
181
+ "model.language_model.layers.2.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
182
+ "model.language_model.layers.20.input_layernorm.weight": "model-00003-of-00004.safetensors",
183
+ "model.language_model.layers.20.linear_attn.A_log": "model-00003-of-00004.safetensors",
184
+ "model.language_model.layers.20.linear_attn.conv1d.weight": "model-00003-of-00004.safetensors",
185
+ "model.language_model.layers.20.linear_attn.dt_bias": "model-00003-of-00004.safetensors",
186
+ "model.language_model.layers.20.linear_attn.in_proj_a.weight": "model-00003-of-00004.safetensors",
187
+ "model.language_model.layers.20.linear_attn.in_proj_b.weight": "model-00003-of-00004.safetensors",
188
+ "model.language_model.layers.20.linear_attn.in_proj_qkv.weight": "model-00003-of-00004.safetensors",
189
+ "model.language_model.layers.20.linear_attn.in_proj_z.weight": "model-00003-of-00004.safetensors",
190
+ "model.language_model.layers.20.linear_attn.norm.weight": "model-00003-of-00004.safetensors",
191
+ "model.language_model.layers.20.linear_attn.out_proj.weight": "model-00003-of-00004.safetensors",
192
+ "model.language_model.layers.20.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
193
+ "model.language_model.layers.20.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
194
+ "model.language_model.layers.20.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
195
+ "model.language_model.layers.20.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
196
+ "model.language_model.layers.21.input_layernorm.weight": "model-00003-of-00004.safetensors",
197
+ "model.language_model.layers.21.linear_attn.A_log": "model-00003-of-00004.safetensors",
198
+ "model.language_model.layers.21.linear_attn.conv1d.weight": "model-00003-of-00004.safetensors",
199
+ "model.language_model.layers.21.linear_attn.dt_bias": "model-00003-of-00004.safetensors",
200
+ "model.language_model.layers.21.linear_attn.in_proj_a.weight": "model-00003-of-00004.safetensors",
201
+ "model.language_model.layers.21.linear_attn.in_proj_b.weight": "model-00003-of-00004.safetensors",
202
+ "model.language_model.layers.21.linear_attn.in_proj_qkv.weight": "model-00003-of-00004.safetensors",
203
+ "model.language_model.layers.21.linear_attn.in_proj_z.weight": "model-00003-of-00004.safetensors",
204
+ "model.language_model.layers.21.linear_attn.norm.weight": "model-00003-of-00004.safetensors",
205
+ "model.language_model.layers.21.linear_attn.out_proj.weight": "model-00003-of-00004.safetensors",
206
+ "model.language_model.layers.21.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
207
+ "model.language_model.layers.21.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
208
+ "model.language_model.layers.21.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
209
+ "model.language_model.layers.21.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
210
+ "model.language_model.layers.22.input_layernorm.weight": "model-00003-of-00004.safetensors",
211
+ "model.language_model.layers.22.linear_attn.A_log": "model-00003-of-00004.safetensors",
212
+ "model.language_model.layers.22.linear_attn.conv1d.weight": "model-00003-of-00004.safetensors",
213
+ "model.language_model.layers.22.linear_attn.dt_bias": "model-00003-of-00004.safetensors",
214
+ "model.language_model.layers.22.linear_attn.in_proj_a.weight": "model-00003-of-00004.safetensors",
215
+ "model.language_model.layers.22.linear_attn.in_proj_b.weight": "model-00003-of-00004.safetensors",
216
+ "model.language_model.layers.22.linear_attn.in_proj_qkv.weight": "model-00003-of-00004.safetensors",
217
+ "model.language_model.layers.22.linear_attn.in_proj_z.weight": "model-00003-of-00004.safetensors",
218
+ "model.language_model.layers.22.linear_attn.norm.weight": "model-00003-of-00004.safetensors",
219
+ "model.language_model.layers.22.linear_attn.out_proj.weight": "model-00003-of-00004.safetensors",
220
+ "model.language_model.layers.22.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
221
+ "model.language_model.layers.22.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
222
+ "model.language_model.layers.22.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
223
+ "model.language_model.layers.22.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
224
+ "model.language_model.layers.23.input_layernorm.weight": "model-00003-of-00004.safetensors",
225
+ "model.language_model.layers.23.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
226
+ "model.language_model.layers.23.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
227
+ "model.language_model.layers.23.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
228
+ "model.language_model.layers.23.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
229
+ "model.language_model.layers.23.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
230
+ "model.language_model.layers.23.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
231
+ "model.language_model.layers.23.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
232
+ "model.language_model.layers.23.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
233
+ "model.language_model.layers.23.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
234
+ "model.language_model.layers.23.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
235
+ "model.language_model.layers.24.input_layernorm.weight": "model-00003-of-00004.safetensors",
236
+ "model.language_model.layers.24.linear_attn.A_log": "model-00003-of-00004.safetensors",
237
+ "model.language_model.layers.24.linear_attn.conv1d.weight": "model-00003-of-00004.safetensors",
238
+ "model.language_model.layers.24.linear_attn.dt_bias": "model-00003-of-00004.safetensors",
239
+ "model.language_model.layers.24.linear_attn.in_proj_a.weight": "model-00003-of-00004.safetensors",
240
+ "model.language_model.layers.24.linear_attn.in_proj_b.weight": "model-00003-of-00004.safetensors",
241
+ "model.language_model.layers.24.linear_attn.in_proj_qkv.weight": "model-00003-of-00004.safetensors",
242
+ "model.language_model.layers.24.linear_attn.in_proj_z.weight": "model-00003-of-00004.safetensors",
243
+ "model.language_model.layers.24.linear_attn.norm.weight": "model-00003-of-00004.safetensors",
244
+ "model.language_model.layers.24.linear_attn.out_proj.weight": "model-00003-of-00004.safetensors",
245
+ "model.language_model.layers.24.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
246
+ "model.language_model.layers.24.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
247
+ "model.language_model.layers.24.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
248
+ "model.language_model.layers.24.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
249
+ "model.language_model.layers.25.input_layernorm.weight": "model-00003-of-00004.safetensors",
250
+ "model.language_model.layers.25.linear_attn.A_log": "model-00003-of-00004.safetensors",
251
+ "model.language_model.layers.25.linear_attn.conv1d.weight": "model-00003-of-00004.safetensors",
252
+ "model.language_model.layers.25.linear_attn.dt_bias": "model-00003-of-00004.safetensors",
253
+ "model.language_model.layers.25.linear_attn.in_proj_a.weight": "model-00003-of-00004.safetensors",
254
+ "model.language_model.layers.25.linear_attn.in_proj_b.weight": "model-00003-of-00004.safetensors",
255
+ "model.language_model.layers.25.linear_attn.in_proj_qkv.weight": "model-00004-of-00004.safetensors",
256
+ "model.language_model.layers.25.linear_attn.in_proj_z.weight": "model-00004-of-00004.safetensors",
257
+ "model.language_model.layers.25.linear_attn.norm.weight": "model-00004-of-00004.safetensors",
258
+ "model.language_model.layers.25.linear_attn.out_proj.weight": "model-00004-of-00004.safetensors",
259
+ "model.language_model.layers.25.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
260
+ "model.language_model.layers.25.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
261
+ "model.language_model.layers.25.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
262
+ "model.language_model.layers.25.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
263
+ "model.language_model.layers.26.input_layernorm.weight": "model-00004-of-00004.safetensors",
264
+ "model.language_model.layers.26.linear_attn.A_log": "model-00004-of-00004.safetensors",
265
+ "model.language_model.layers.26.linear_attn.conv1d.weight": "model-00004-of-00004.safetensors",
266
+ "model.language_model.layers.26.linear_attn.dt_bias": "model-00004-of-00004.safetensors",
267
+ "model.language_model.layers.26.linear_attn.in_proj_a.weight": "model-00004-of-00004.safetensors",
268
+ "model.language_model.layers.26.linear_attn.in_proj_b.weight": "model-00004-of-00004.safetensors",
269
+ "model.language_model.layers.26.linear_attn.in_proj_qkv.weight": "model-00004-of-00004.safetensors",
270
+ "model.language_model.layers.26.linear_attn.in_proj_z.weight": "model-00004-of-00004.safetensors",
271
+ "model.language_model.layers.26.linear_attn.norm.weight": "model-00004-of-00004.safetensors",
272
+ "model.language_model.layers.26.linear_attn.out_proj.weight": "model-00004-of-00004.safetensors",
273
+ "model.language_model.layers.26.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
274
+ "model.language_model.layers.26.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
275
+ "model.language_model.layers.26.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
276
+ "model.language_model.layers.26.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
277
+ "model.language_model.layers.27.input_layernorm.weight": "model-00004-of-00004.safetensors",
278
+ "model.language_model.layers.27.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
279
+ "model.language_model.layers.27.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
280
+ "model.language_model.layers.27.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
281
+ "model.language_model.layers.27.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
282
+ "model.language_model.layers.27.self_attn.k_norm.weight": "model-00004-of-00004.safetensors",
283
+ "model.language_model.layers.27.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
284
+ "model.language_model.layers.27.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
285
+ "model.language_model.layers.27.self_attn.q_norm.weight": "model-00004-of-00004.safetensors",
286
+ "model.language_model.layers.27.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
287
+ "model.language_model.layers.27.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
288
+ "model.language_model.layers.28.input_layernorm.weight": "model-00004-of-00004.safetensors",
289
+ "model.language_model.layers.28.linear_attn.A_log": "model-00004-of-00004.safetensors",
290
+ "model.language_model.layers.28.linear_attn.conv1d.weight": "model-00004-of-00004.safetensors",
291
+ "model.language_model.layers.28.linear_attn.dt_bias": "model-00004-of-00004.safetensors",
292
+ "model.language_model.layers.28.linear_attn.in_proj_a.weight": "model-00004-of-00004.safetensors",
293
+ "model.language_model.layers.28.linear_attn.in_proj_b.weight": "model-00004-of-00004.safetensors",
294
+ "model.language_model.layers.28.linear_attn.in_proj_qkv.weight": "model-00004-of-00004.safetensors",
295
+ "model.language_model.layers.28.linear_attn.in_proj_z.weight": "model-00004-of-00004.safetensors",
296
+ "model.language_model.layers.28.linear_attn.norm.weight": "model-00004-of-00004.safetensors",
297
+ "model.language_model.layers.28.linear_attn.out_proj.weight": "model-00004-of-00004.safetensors",
298
+ "model.language_model.layers.28.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
299
+ "model.language_model.layers.28.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
300
+ "model.language_model.layers.28.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
301
+ "model.language_model.layers.28.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
302
+ "model.language_model.layers.29.input_layernorm.weight": "model-00004-of-00004.safetensors",
303
+ "model.language_model.layers.29.linear_attn.A_log": "model-00004-of-00004.safetensors",
304
+ "model.language_model.layers.29.linear_attn.conv1d.weight": "model-00004-of-00004.safetensors",
305
+ "model.language_model.layers.29.linear_attn.dt_bias": "model-00004-of-00004.safetensors",
306
+ "model.language_model.layers.29.linear_attn.in_proj_a.weight": "model-00004-of-00004.safetensors",
307
+ "model.language_model.layers.29.linear_attn.in_proj_b.weight": "model-00004-of-00004.safetensors",
308
+ "model.language_model.layers.29.linear_attn.in_proj_qkv.weight": "model-00004-of-00004.safetensors",
309
+ "model.language_model.layers.29.linear_attn.in_proj_z.weight": "model-00004-of-00004.safetensors",
310
+ "model.language_model.layers.29.linear_attn.norm.weight": "model-00004-of-00004.safetensors",
311
+ "model.language_model.layers.29.linear_attn.out_proj.weight": "model-00004-of-00004.safetensors",
312
+ "model.language_model.layers.29.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
313
+ "model.language_model.layers.29.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
314
+ "model.language_model.layers.29.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
315
+ "model.language_model.layers.29.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
316
+ "model.language_model.layers.3.input_layernorm.weight": "model-00002-of-00004.safetensors",
317
+ "model.language_model.layers.3.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
318
+ "model.language_model.layers.3.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
319
+ "model.language_model.layers.3.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
320
+ "model.language_model.layers.3.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
321
+ "model.language_model.layers.3.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
322
+ "model.language_model.layers.3.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
323
+ "model.language_model.layers.3.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
324
+ "model.language_model.layers.3.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
325
+ "model.language_model.layers.3.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
326
+ "model.language_model.layers.3.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
327
+ "model.language_model.layers.30.input_layernorm.weight": "model-00004-of-00004.safetensors",
328
+ "model.language_model.layers.30.linear_attn.A_log": "model-00004-of-00004.safetensors",
329
+ "model.language_model.layers.30.linear_attn.conv1d.weight": "model-00004-of-00004.safetensors",
330
+ "model.language_model.layers.30.linear_attn.dt_bias": "model-00004-of-00004.safetensors",
331
+ "model.language_model.layers.30.linear_attn.in_proj_a.weight": "model-00004-of-00004.safetensors",
332
+ "model.language_model.layers.30.linear_attn.in_proj_b.weight": "model-00004-of-00004.safetensors",
333
+ "model.language_model.layers.30.linear_attn.in_proj_qkv.weight": "model-00004-of-00004.safetensors",
334
+ "model.language_model.layers.30.linear_attn.in_proj_z.weight": "model-00004-of-00004.safetensors",
335
+ "model.language_model.layers.30.linear_attn.norm.weight": "model-00004-of-00004.safetensors",
336
+ "model.language_model.layers.30.linear_attn.out_proj.weight": "model-00004-of-00004.safetensors",
337
+ "model.language_model.layers.30.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
338
+ "model.language_model.layers.30.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
339
+ "model.language_model.layers.30.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
340
+ "model.language_model.layers.30.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
341
+ "model.language_model.layers.31.input_layernorm.weight": "model-00004-of-00004.safetensors",
342
+ "model.language_model.layers.31.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
343
+ "model.language_model.layers.31.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
344
+ "model.language_model.layers.31.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
345
+ "model.language_model.layers.31.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
346
+ "model.language_model.layers.31.self_attn.k_norm.weight": "model-00004-of-00004.safetensors",
347
+ "model.language_model.layers.31.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
348
+ "model.language_model.layers.31.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
349
+ "model.language_model.layers.31.self_attn.q_norm.weight": "model-00004-of-00004.safetensors",
350
+ "model.language_model.layers.31.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
351
+ "model.language_model.layers.31.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
352
+ "model.language_model.layers.4.input_layernorm.weight": "model-00002-of-00004.safetensors",
353
+ "model.language_model.layers.4.linear_attn.A_log": "model-00002-of-00004.safetensors",
354
+ "model.language_model.layers.4.linear_attn.conv1d.weight": "model-00002-of-00004.safetensors",
355
+ "model.language_model.layers.4.linear_attn.dt_bias": "model-00002-of-00004.safetensors",
356
+ "model.language_model.layers.4.linear_attn.in_proj_a.weight": "model-00002-of-00004.safetensors",
357
+ "model.language_model.layers.4.linear_attn.in_proj_b.weight": "model-00002-of-00004.safetensors",
358
+ "model.language_model.layers.4.linear_attn.in_proj_qkv.weight": "model-00002-of-00004.safetensors",
359
+ "model.language_model.layers.4.linear_attn.in_proj_z.weight": "model-00002-of-00004.safetensors",
360
+ "model.language_model.layers.4.linear_attn.norm.weight": "model-00002-of-00004.safetensors",
361
+ "model.language_model.layers.4.linear_attn.out_proj.weight": "model-00002-of-00004.safetensors",
362
+ "model.language_model.layers.4.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
363
+ "model.language_model.layers.4.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
364
+ "model.language_model.layers.4.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
365
+ "model.language_model.layers.4.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
366
+ "model.language_model.layers.5.input_layernorm.weight": "model-00002-of-00004.safetensors",
367
+ "model.language_model.layers.5.linear_attn.A_log": "model-00002-of-00004.safetensors",
368
+ "model.language_model.layers.5.linear_attn.conv1d.weight": "model-00002-of-00004.safetensors",
369
+ "model.language_model.layers.5.linear_attn.dt_bias": "model-00002-of-00004.safetensors",
370
+ "model.language_model.layers.5.linear_attn.in_proj_a.weight": "model-00002-of-00004.safetensors",
371
+ "model.language_model.layers.5.linear_attn.in_proj_b.weight": "model-00002-of-00004.safetensors",
372
+ "model.language_model.layers.5.linear_attn.in_proj_qkv.weight": "model-00002-of-00004.safetensors",
373
+ "model.language_model.layers.5.linear_attn.in_proj_z.weight": "model-00002-of-00004.safetensors",
374
+ "model.language_model.layers.5.linear_attn.norm.weight": "model-00002-of-00004.safetensors",
375
+ "model.language_model.layers.5.linear_attn.out_proj.weight": "model-00002-of-00004.safetensors",
376
+ "model.language_model.layers.5.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
377
+ "model.language_model.layers.5.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
378
+ "model.language_model.layers.5.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
379
+ "model.language_model.layers.5.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
380
+ "model.language_model.layers.6.input_layernorm.weight": "model-00002-of-00004.safetensors",
381
+ "model.language_model.layers.6.linear_attn.A_log": "model-00002-of-00004.safetensors",
382
+ "model.language_model.layers.6.linear_attn.conv1d.weight": "model-00002-of-00004.safetensors",
383
+ "model.language_model.layers.6.linear_attn.dt_bias": "model-00002-of-00004.safetensors",
384
+ "model.language_model.layers.6.linear_attn.in_proj_a.weight": "model-00002-of-00004.safetensors",
385
+ "model.language_model.layers.6.linear_attn.in_proj_b.weight": "model-00002-of-00004.safetensors",
386
+ "model.language_model.layers.6.linear_attn.in_proj_qkv.weight": "model-00002-of-00004.safetensors",
387
+ "model.language_model.layers.6.linear_attn.in_proj_z.weight": "model-00002-of-00004.safetensors",
388
+ "model.language_model.layers.6.linear_attn.norm.weight": "model-00002-of-00004.safetensors",
389
+ "model.language_model.layers.6.linear_attn.out_proj.weight": "model-00002-of-00004.safetensors",
390
+ "model.language_model.layers.6.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
391
+ "model.language_model.layers.6.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
392
+ "model.language_model.layers.6.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
393
+ "model.language_model.layers.6.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
394
+ "model.language_model.layers.7.input_layernorm.weight": "model-00002-of-00004.safetensors",
395
+ "model.language_model.layers.7.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
396
+ "model.language_model.layers.7.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
397
+ "model.language_model.layers.7.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
398
+ "model.language_model.layers.7.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
399
+ "model.language_model.layers.7.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
400
+ "model.language_model.layers.7.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
401
+ "model.language_model.layers.7.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
402
+ "model.language_model.layers.7.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
403
+ "model.language_model.layers.7.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
404
+ "model.language_model.layers.7.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
405
+ "model.language_model.layers.8.input_layernorm.weight": "model-00002-of-00004.safetensors",
406
+ "model.language_model.layers.8.linear_attn.A_log": "model-00002-of-00004.safetensors",
407
+ "model.language_model.layers.8.linear_attn.conv1d.weight": "model-00002-of-00004.safetensors",
408
+ "model.language_model.layers.8.linear_attn.dt_bias": "model-00002-of-00004.safetensors",
409
+ "model.language_model.layers.8.linear_attn.in_proj_a.weight": "model-00002-of-00004.safetensors",
410
+ "model.language_model.layers.8.linear_attn.in_proj_b.weight": "model-00002-of-00004.safetensors",
411
+ "model.language_model.layers.8.linear_attn.in_proj_qkv.weight": "model-00002-of-00004.safetensors",
412
+ "model.language_model.layers.8.linear_attn.in_proj_z.weight": "model-00002-of-00004.safetensors",
413
+ "model.language_model.layers.8.linear_attn.norm.weight": "model-00002-of-00004.safetensors",
414
+ "model.language_model.layers.8.linear_attn.out_proj.weight": "model-00002-of-00004.safetensors",
415
+ "model.language_model.layers.8.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
416
+ "model.language_model.layers.8.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
417
+ "model.language_model.layers.8.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
418
+ "model.language_model.layers.8.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
419
+ "model.language_model.layers.9.input_layernorm.weight": "model-00002-of-00004.safetensors",
420
+ "model.language_model.layers.9.linear_attn.A_log": "model-00002-of-00004.safetensors",
421
+ "model.language_model.layers.9.linear_attn.conv1d.weight": "model-00002-of-00004.safetensors",
422
+ "model.language_model.layers.9.linear_attn.dt_bias": "model-00002-of-00004.safetensors",
423
+ "model.language_model.layers.9.linear_attn.in_proj_a.weight": "model-00002-of-00004.safetensors",
424
+ "model.language_model.layers.9.linear_attn.in_proj_b.weight": "model-00002-of-00004.safetensors",
425
+ "model.language_model.layers.9.linear_attn.in_proj_qkv.weight": "model-00002-of-00004.safetensors",
426
+ "model.language_model.layers.9.linear_attn.in_proj_z.weight": "model-00002-of-00004.safetensors",
427
+ "model.language_model.layers.9.linear_attn.norm.weight": "model-00002-of-00004.safetensors",
428
+ "model.language_model.layers.9.linear_attn.out_proj.weight": "model-00002-of-00004.safetensors",
429
+ "model.language_model.layers.9.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
430
+ "model.language_model.layers.9.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
431
+ "model.language_model.layers.9.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
432
+ "model.language_model.layers.9.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
433
+ "model.language_model.norm.weight": "model-00004-of-00004.safetensors"
 
 
 
 
 
 
 
 
434
  }
435
  }
quantum_attestation.json CHANGED
@@ -1,210 +1,24 @@
1
  {
2
- "scheme": "SHA256 leaves -> Merkle root",
3
- "proj_seed": 42,
4
  "backend": "ibm_marrakesh",
5
- "collected_at": "2026-07-02T06:07:00.294531+00:00",
6
- "attestation_root": "0afa57c3bc66820ed5d37b0e7a37463ce4bfdb67444035aaacce80e87e3a9911",
7
- "leaves": [
8
- {
9
- "leaf": "50ad0b6b24bbb2a2f585e90c034a94a7e55ffb9fd3f48e717f82328760c4ec5c",
10
- "slot": 0,
11
- "ibm_job_id": "d92ve0t958jc73bsbong",
12
- "backend": "ibm_marrakesh",
13
- "n_qubits": 100,
14
- "shots": 2048,
15
- "syk_depths": [
16
- 1,
17
- 2,
18
- 3,
19
- 4,
20
- 5,
21
- 6
22
- ],
23
- "otoc_6d": [
24
- 0.08673828,
25
- 0.06501953,
26
- 0.06294922,
27
- 0.06214844,
28
- 0.06146484,
29
- 0.06382812
30
- ],
31
- "proj_sha256": "0d60cc623788ff08694ed0ad5bccce16194f72bea472e26bbd6fd6609ba26822"
32
- },
33
- {
34
- "leaf": "b4abea3cdff44d9c02fcdc772a5d0b0454f5117b8d2c4611a81b7b5c9f765595",
35
- "slot": 1,
36
- "ibm_job_id": "d92ve0t958jc73bsbong",
37
- "backend": "ibm_marrakesh",
38
- "n_qubits": 100,
39
- "shots": 2048,
40
- "syk_depths": [
41
- 1,
42
- 2,
43
- 3,
44
- 4,
45
- 5,
46
- 6
47
- ],
48
- "otoc_6d": [
49
- 0.19728516,
50
- 0.065625,
51
- 0.06103516,
52
- 0.06363281,
53
- 0.06310547,
54
- 0.06396484
55
- ],
56
- "proj_sha256": "92d77034456235a64faeba4b5985d1d50e350b18692dc1f5ebc090d2061950f5"
57
- },
58
- {
59
- "leaf": "9bee906e97fae1433a61c91a2a7072b4e46e6b6dbe6bcec5581312530b79c2fb",
60
- "slot": 2,
61
- "ibm_job_id": "d92ve0t958jc73bsbong",
62
- "backend": "ibm_marrakesh",
63
- "n_qubits": 100,
64
- "shots": 2048,
65
- "syk_depths": [
66
- 1,
67
- 2,
68
- 3,
69
- 4,
70
- 5,
71
- 6
72
- ],
73
- "otoc_6d": [
74
- 0.12580078,
75
- 0.06103516,
76
- 0.06216797,
77
- 0.0609375,
78
- 0.06306641,
79
- 0.06375
80
- ],
81
- "proj_sha256": "07b5431ddc3f5f74d7702f0cafbe3c67c7dd12ee5cb7ad529b3197155f0a7449"
82
- },
83
- {
84
- "leaf": "8231e1fd3fa048d278474d7842b00facf8658317972d01899caa309378f82b30",
85
- "slot": 3,
86
- "ibm_job_id": "d92ve0t958jc73bsbong",
87
- "backend": "ibm_marrakesh",
88
- "n_qubits": 100,
89
- "shots": 2048,
90
- "syk_depths": [
91
- 1,
92
- 2,
93
- 3,
94
- 4,
95
- 5,
96
- 6
97
- ],
98
- "otoc_6d": [
99
- 0.24994141,
100
- 0.10943359,
101
- 0.05970703,
102
- 0.06130859,
103
- 0.06320312,
104
- 0.06275391
105
- ],
106
- "proj_sha256": "7fd2c87a3618adf4835765649179875a104d924ea0ab9aa0eaf2ebf60356cb65"
107
- },
108
- {
109
- "leaf": "1028d1a70b54bc95baf813cc18a3c267d4fa408b04a657f35b007fa8d0505649",
110
- "slot": 4,
111
- "ibm_job_id": "d92ve0t958jc73bsbong",
112
- "backend": "ibm_marrakesh",
113
- "n_qubits": 100,
114
- "shots": 2048,
115
- "syk_depths": [
116
- 1,
117
- 2,
118
- 3,
119
- 4,
120
- 5,
121
- 6
122
- ],
123
- "otoc_6d": [
124
- 0.06734375,
125
- 0.06650391,
126
- 0.0615625,
127
- 0.06162109,
128
- 0.0646875,
129
- 0.06271484
130
- ],
131
- "proj_sha256": "613fc240263fcd57135958d2f21678c8bcdf543de7c82dbcd325a63b20b8d852"
132
- },
133
- {
134
- "leaf": "8bc02b090967370acd8e250b2c2f53547713577e44f8207fde5d22d829b82fc1",
135
- "slot": 5,
136
- "ibm_job_id": "d92ve0t958jc73bsbong",
137
- "backend": "ibm_marrakesh",
138
- "n_qubits": 100,
139
- "shots": 2048,
140
- "syk_depths": [
141
- 1,
142
- 2,
143
- 3,
144
- 4,
145
- 5,
146
- 6
147
- ],
148
- "otoc_6d": [
149
- 0.07216797,
150
- 0.07421875,
151
- 0.07933594,
152
- 0.063125,
153
- 0.06453125,
154
- 0.06048828
155
- ],
156
- "proj_sha256": "5da8b4ae463e7b0336d400274be885f1eb1857e2c43eb24e050bb1de5e8b1236"
157
- },
158
- {
159
- "leaf": "4242a696cd06b57e471b841d6a37df6ea630c0fae9c282a5169d70ce8f16e047",
160
- "slot": 6,
161
- "ibm_job_id": "d92ve0t958jc73bsbong",
162
- "backend": "ibm_marrakesh",
163
- "n_qubits": 100,
164
- "shots": 2048,
165
- "syk_depths": [
166
- 1,
167
- 2,
168
- 3,
169
- 4,
170
- 5,
171
- 6
172
- ],
173
- "otoc_6d": [
174
- 0.28207031,
175
- 0.09130859,
176
- 0.06955078,
177
- 0.06390625,
178
- 0.06066406,
179
- 0.06275391
180
- ],
181
- "proj_sha256": "1d5c2f9050bc3c53d6516f4d7f66dc53e2a499f37a18a86c0575db40abfe45a1"
182
- },
183
- {
184
- "leaf": "8bfab403055ab0b1553c1a839d2beac4bd6a166bd2376af93c1a9a69461ec822",
185
- "slot": 7,
186
- "ibm_job_id": "d92ve0t958jc73bsbong",
187
- "backend": "ibm_marrakesh",
188
- "n_qubits": 100,
189
- "shots": 2048,
190
- "syk_depths": [
191
- 1,
192
- 2,
193
- 3,
194
- 4,
195
- 5,
196
- 6
197
- ],
198
- "otoc_6d": [
199
- 0.14970703,
200
- 0.06710937,
201
- 0.07498047,
202
- 0.06775391,
203
- 0.06279297,
204
- 0.06373047
205
- ],
206
- "proj_sha256": "2a0abaa96d2ef3295cf9af19a475eb5137bdb8590fc54f2cba1705dd16790728"
207
- }
208
  ],
209
- "verify": "recompute leaf_k=SHA256(canonical_json(fields)); rebuild Merkle root; query ibm_job_id via QiskitRuntimeService.job(id)."
 
 
 
 
 
 
 
 
 
 
210
  }
 
1
  {
 
 
2
  "backend": "ibm_marrakesh",
3
+ "ibm_job_id": "d92ve0t958jc73bsbong",
4
+ "quantum_source": "SYK-scrambler OTOC on IBM Heron",
5
+ "otoc_depths": [
6
+ 1,
7
+ 2,
8
+ 3,
9
+ 4,
10
+ 5,
11
+ 6
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  ],
13
+ "n_qubits": 100,
14
+ "shots": 2048,
15
+ "tiles": 25,
16
+ "collected_at": "2026-07-02T06:07:00.294531+00:00",
17
+ "n_anchors": 64,
18
+ "d": 6,
19
+ "r": 6,
20
+ "proj_seed": 42,
21
+ "qfeat_sha256": "82c9c9e83a7b568c169cc229d8df801c4a2385a44c0efb4d95d1dbc7e00c6f9e",
22
+ "A_sha256": "c33d3a6aee9293bf20f7a4ddc2d9fe5793dc8620233a8e9c2f04a548e8ddc268",
23
+ "attestation_root": "9484dca40b66488a239fbbb12a9333a47458627f48b1c6d08d8241bf814caf48"
24
  }
signature_records.json ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "collected_at": "2026-07-02T06:07:00.294531+00:00",
3
+ "backend": "ibm_marrakesh",
4
+ "tiles": 25,
5
+ "records": [
6
+ {
7
+ "slot": 0,
8
+ "seed": 42,
9
+ "backend": "ibm_marrakesh",
10
+ "n_qubits": 100,
11
+ "tiles": 25,
12
+ "shots": 2048,
13
+ "syk_depths": [
14
+ 1,
15
+ 2,
16
+ 3,
17
+ 4,
18
+ 5,
19
+ 6
20
+ ],
21
+ "ibm_job_id": "d92ve0t958jc73bsbong",
22
+ "otoc_6d": [
23
+ 0.08673828125,
24
+ 0.06501953125,
25
+ 0.06294921875,
26
+ 0.0621484375,
27
+ 0.06146484375,
28
+ 0.063828125
29
+ ]
30
+ },
31
+ {
32
+ "slot": 1,
33
+ "seed": 43,
34
+ "backend": "ibm_marrakesh",
35
+ "n_qubits": 100,
36
+ "tiles": 25,
37
+ "shots": 2048,
38
+ "syk_depths": [
39
+ 1,
40
+ 2,
41
+ 3,
42
+ 4,
43
+ 5,
44
+ 6
45
+ ],
46
+ "ibm_job_id": "d92ve0t958jc73bsbong",
47
+ "otoc_6d": [
48
+ 0.19728515625,
49
+ 0.065625,
50
+ 0.06103515625,
51
+ 0.0636328125,
52
+ 0.06310546875,
53
+ 0.06396484375
54
+ ]
55
+ },
56
+ {
57
+ "slot": 2,
58
+ "seed": 44,
59
+ "backend": "ibm_marrakesh",
60
+ "n_qubits": 100,
61
+ "tiles": 25,
62
+ "shots": 2048,
63
+ "syk_depths": [
64
+ 1,
65
+ 2,
66
+ 3,
67
+ 4,
68
+ 5,
69
+ 6
70
+ ],
71
+ "ibm_job_id": "d92ve0t958jc73bsbong",
72
+ "otoc_6d": [
73
+ 0.12580078125,
74
+ 0.06103515625,
75
+ 0.06216796875,
76
+ 0.0609375,
77
+ 0.06306640625,
78
+ 0.06375
79
+ ]
80
+ },
81
+ {
82
+ "slot": 3,
83
+ "seed": 45,
84
+ "backend": "ibm_marrakesh",
85
+ "n_qubits": 100,
86
+ "tiles": 25,
87
+ "shots": 2048,
88
+ "syk_depths": [
89
+ 1,
90
+ 2,
91
+ 3,
92
+ 4,
93
+ 5,
94
+ 6
95
+ ],
96
+ "ibm_job_id": "d92ve0t958jc73bsbong",
97
+ "otoc_6d": [
98
+ 0.24994140625,
99
+ 0.10943359375,
100
+ 0.05970703125,
101
+ 0.06130859375,
102
+ 0.063203125,
103
+ 0.06275390625
104
+ ]
105
+ },
106
+ {
107
+ "slot": 4,
108
+ "seed": 46,
109
+ "backend": "ibm_marrakesh",
110
+ "n_qubits": 100,
111
+ "tiles": 25,
112
+ "shots": 2048,
113
+ "syk_depths": [
114
+ 1,
115
+ 2,
116
+ 3,
117
+ 4,
118
+ 5,
119
+ 6
120
+ ],
121
+ "ibm_job_id": "d92ve0t958jc73bsbong",
122
+ "otoc_6d": [
123
+ 0.06734375,
124
+ 0.06650390625,
125
+ 0.0615625,
126
+ 0.06162109375,
127
+ 0.0646875,
128
+ 0.06271484375
129
+ ]
130
+ },
131
+ {
132
+ "slot": 5,
133
+ "seed": 47,
134
+ "backend": "ibm_marrakesh",
135
+ "n_qubits": 100,
136
+ "tiles": 25,
137
+ "shots": 2048,
138
+ "syk_depths": [
139
+ 1,
140
+ 2,
141
+ 3,
142
+ 4,
143
+ 5,
144
+ 6
145
+ ],
146
+ "ibm_job_id": "d92ve0t958jc73bsbong",
147
+ "otoc_6d": [
148
+ 0.07216796875,
149
+ 0.07421875,
150
+ 0.0793359375,
151
+ 0.063125,
152
+ 0.06453125,
153
+ 0.06048828125
154
+ ]
155
+ },
156
+ {
157
+ "slot": 6,
158
+ "seed": 48,
159
+ "backend": "ibm_marrakesh",
160
+ "n_qubits": 100,
161
+ "tiles": 25,
162
+ "shots": 2048,
163
+ "syk_depths": [
164
+ 1,
165
+ 2,
166
+ 3,
167
+ 4,
168
+ 5,
169
+ 6
170
+ ],
171
+ "ibm_job_id": "d92ve0t958jc73bsbong",
172
+ "otoc_6d": [
173
+ 0.2820703125,
174
+ 0.09130859375,
175
+ 0.06955078125,
176
+ 0.06390625,
177
+ 0.0606640625,
178
+ 0.06275390625
179
+ ]
180
+ },
181
+ {
182
+ "slot": 7,
183
+ "seed": 49,
184
+ "backend": "ibm_marrakesh",
185
+ "n_qubits": 100,
186
+ "tiles": 25,
187
+ "shots": 2048,
188
+ "syk_depths": [
189
+ 1,
190
+ 2,
191
+ 3,
192
+ 4,
193
+ 5,
194
+ 6
195
+ ],
196
+ "ibm_job_id": "d92ve0t958jc73bsbong",
197
+ "otoc_6d": [
198
+ 0.14970703125,
199
+ 0.067109375,
200
+ 0.07498046875,
201
+ 0.06775390625,
202
+ 0.06279296875,
203
+ 0.06373046875
204
+ ]
205
+ }
206
+ ]
207
+ }
tokenizer_config.json CHANGED
@@ -9,7 +9,7 @@
9
  "eos_token": "<|im_end|>",
10
  "errors": "replace",
11
  "image_token": "<|image_pad|>",
12
- "is_local": true,
13
  "local_files_only": false,
14
  "model_max_length": 262144,
15
  "model_specific_special_tokens": {
 
9
  "eos_token": "<|im_end|>",
10
  "errors": "replace",
11
  "image_token": "<|image_pad|>",
12
+ "is_local": false,
13
  "local_files_only": false,
14
  "model_max_length": 262144,
15
  "model_specific_special_tokens": {