Instructions to use eaddario/gemma-4-E4B-it-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use eaddario/gemma-4-E4B-it-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf eaddario/gemma-4-E4B-it-GGUF:F16
# Run inference directly in the terminal:
llama cli -hf eaddario/gemma-4-E4B-it-GGUF:F16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf eaddario/gemma-4-E4B-it-GGUF:F16
# Run inference directly in the terminal:
llama cli -hf eaddario/gemma-4-E4B-it-GGUF:F16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf eaddario/gemma-4-E4B-it-GGUF:F16
# Run inference directly in the terminal:
./llama-cli -hf eaddario/gemma-4-E4B-it-GGUF:F16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf eaddario/gemma-4-E4B-it-GGUF:F16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf eaddario/gemma-4-E4B-it-GGUF:F16

Use Docker

docker model run hf.co/eaddario/gemma-4-E4B-it-GGUF:F16

LM Studio
Jan
Ollama
How to use eaddario/gemma-4-E4B-it-GGUF with Ollama:
```
ollama run hf.co/eaddario/gemma-4-E4B-it-GGUF:F16
```
Unsloth Desktop

How to use eaddario/gemma-4-E4B-it-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf eaddario/gemma-4-E4B-it-GGUF:F16

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "eaddario/gemma-4-E4B-it-GGUF:F16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use eaddario/gemma-4-E4B-it-GGUF with Docker Model Runner:
```
docker model run hf.co/eaddario/gemma-4-E4B-it-GGUF:F16
```

Lemonade

How to use eaddario/gemma-4-E4B-it-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull eaddario/gemma-4-E4B-it-GGUF:F16

Run and chat with the model

lemonade run user.gemma-4-E4B-it-GGUF-F16

List all available models

lemonade list

Hermes Agent

How to use eaddario/gemma-4-E4B-it-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf eaddario/gemma-4-E4B-it-GGUF:F16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default eaddario/gemma-4-E4B-it-GGUF:F16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use eaddario/gemma-4-E4B-it-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf eaddario/gemma-4-E4B-it-GGUF:F16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "eaddario/gemma-4-E4B-it-GGUF:F16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

gemma-4-E4B-it-GGUF / scores /gemma-4-E4B-it-Q4_K.md

eaddario

Add GGUF internal file structure

8399b2d verified 4 months ago

preview code

Raw

History Blame Contribute Delete

206 kB

gemma-4-E4B-it-Q4_K.gguf - GGUF Internal File Dump

Endian: LITTLE endian

Key Value Metadata Store

There are 52 key-value pairs in this file

POS	TYPE	Count	Key	Value
1	UINT32	1	GGUF.version	3
2	UINT64	1	GGUF.tensor_count	720
3	UINT64	1	GGUF.kv_count	49
4	STRING	1	general.architecture	`gemma4`
5	STRING	1	general.type	`model`
6	INT32	1	general.sampling.top_k	64
7	FLOAT32	1	general.sampling.top_p	0.95
8	FLOAT32	1	general.sampling.temp	1.0
9	STRING	1	general.name	`Gemma 4 E4B It`
10	STRING	1	general.size_label	`7.5B`
11	STRING	1	general.license	`apache-2.0`
12	STRING	1	general.license.link	`https://ai.google.dev/gemma/docs/gemma_4_license`
13	[STRING]	1	general.tags	[ `any-to-any` ]
14	UINT32	1	gemma4.block_count	42
15	UINT32	1	gemma4.context_length	131072
16	UINT32	1	gemma4.embedding_length	2560
17	UINT32	1	gemma4.feed_forward_length	10240
18	UINT32	1	gemma4.attention.head_count	8
19	UINT32	1	gemma4.attention.head_count_kv	2
20	FLOAT32	1	gemma4.rope.freq_base	1e+06
21	FLOAT32	1	gemma4.rope.freq_base_swa	10000.0
22	FLOAT32	1	gemma4.attention.layer_norm_rms_epsilon	1e-06
23	UINT32	1	gemma4.attention.key_length	512
24	UINT32	1	gemma4.attention.value_length	512
25	FLOAT32	1	gemma4.final_logit_softcapping	30.0
26	UINT32	1	gemma4.attention.sliding_window	512
27	UINT32	1	gemma4.attention.shared_kv_layers	18
28	UINT32	1	gemma4.embedding_length_per_layer_input	256
29	[BOOL]	42	gemma4.attention.sliding_window_pattern	[ True, True, True, True, True, False, True, ... ]
30	UINT32	1	gemma4.attention.key_length_swa	256
31	UINT32	1	gemma4.attention.value_length_swa	256
32	UINT32	1	gemma4.rope.dimension_count	512
33	UINT32	1	gemma4.rope.dimension_count_swa	256
34	STRING	1	tokenizer.ggml.model	`gemma4`
35	[STRING]	262144	tokenizer.ggml.tokens	[ `<pad>`, `<eos>`, `<bos>`, `<unk>`, `<mask>`, ... ]
36	[FLOAT32]	262144	tokenizer.ggml.scores	[ -1000.0, -1000.0, -1000.0, -1000.0, -1000.0, -1000.0, -1000.0, ... ]
37	[INT32]	262144	tokenizer.ggml.token_type	[ 3, 3, 3, 3, 3, 1, 1, ... ]
38	[STRING]	514906	tokenizer.ggml.merges	[ `...`, `▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁`...`▁▁▁▁▁▁▁▁▁▁▁▁▁ ▁`, `...`, `...`, `▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁`...`▁▁▁▁▁▁▁▁▁▁▁▁ ▁▁`, ... ]
39	UINT32	1	tokenizer.ggml.bos_token_id	2
40	UINT32	1	tokenizer.ggml.eos_token_id	1
41	UINT32	1	tokenizer.ggml.unknown_token_id	3
42	UINT32	1	tokenizer.ggml.padding_token_id	0
43	UINT32	1	tokenizer.ggml.mask_token_id	4
44	STRING	1	tokenizer.chat_template	`{%- macro format_parameters(pr`... `{%- endif -%} {%- endif -%}`
45	BOOL	1	tokenizer.ggml.add_space_prefix	False
46	BOOL	1	tokenizer.ggml.add_bos_token	True
47	UINT32	1	general.quantization_version	2
48	UINT32	1	general.file_type	15
49	STRING	1	quantize.imatrix.file	`gemma-4-E4B-it-WIP/imatrix/ima`...`rix-gemma-4-E4B-it-medium.gguf`
50	STRING	1	quantize.imatrix.dataset	`../datasets/imatrix/combined_eur_medium.txt`
51	UINT32	1	quantize.imatrix.entries_count	342
52	UINT32	1	quantize.imatrix.chunks_count	2471

Tensors Overview ~8B Elements

Total number of elements in all tensors: 7518069290 Elements

/Users/ed/Development/AI/hf/gemma-4-E4B-it-WIP/gemma-4-E4B-it-Q4_K.gguf - GGUF Internal File Dump
- Key Value Metadata Store
- Tensors Overview ~8B Elements

Tensor Data Offset

This table contains the offset and data segment relative to start of file

T_ID	Tensor Layer Name	Data Offset (B)	Data Size (B)
0	output_norm.weight	0xf173e0	0x2800
1	per_layer_model_proj.weight	0xf19be0	0x3480000
2	per_layer_proj_norm.weight	0x4399be0	0x400
3	per_layer_token_embd.weight	0x4399fe0	0x5e800000
4	rope_freqs.weight	0x62b99fe0	0x400
5	token_embd.weight	0x62b9a3e0	0x16800000
6	blk.0.attn_k.weight	0x7939a3e0	0xb4000
7	blk.0.attn_k_norm.weight	0x7944e3e0	0x400
8	blk.0.attn_norm.weight	0x7944e7e0	0x2800
9	blk.0.attn_output.weight	0x79450fe0	0x2d0000
10	blk.0.attn_q.weight	0x79720fe0	0x2a8000
11	blk.0.attn_q_norm.weight	0x799c8fe0	0x400
12	blk.0.attn_v.weight	0x799c93e0	0xb4000
13	blk.0.ffn_down.weight	0x79a7d3e0	0xd48000
14	blk.0.ffn_gate.weight	0x7a7c53e0	0xd48000
15	blk.0.ffn_norm.weight	0x7b50d3e0	0x2800
16	blk.0.ffn_up.weight	0x7b50fbe0	0xd48000
17	blk.0.inp_gate.weight	0x7c257be0	0x5a000
18	blk.0.layer_output_scale.weight	0x7c2b1be0	0x4
19	blk.0.post_attention_norm.weight	0x7c2b1c00	0x2800
20	blk.0.post_ffw_norm.weight	0x7c2b4400	0x2800
21	blk.0.post_norm.weight	0x7c2b6c00	0x2800
22	blk.0.proj.weight	0x7c2b9400	0x78000
23	blk.1.attn_k.weight	0x7c331400	0xb4000
24	blk.1.attn_k_norm.weight	0x7c3e5400	0x400
25	blk.1.attn_norm.weight	0x7c3e5800	0x2800
26	blk.1.attn_output.weight	0x7c3e8000	0x370000
27	blk.1.attn_q.weight	0x7c758000	0x2d0000
28	blk.1.attn_q_norm.weight	0x7ca28000	0x400
29	blk.1.attn_v.weight	0x7ca28400	0xdc000
30	blk.1.ffn_down.weight	0x7cb04400	0xd48000
31	blk.1.ffn_gate.weight	0x7d84c400	0xd48000
32	blk.1.ffn_norm.weight	0x7e594400	0x2800
33	blk.1.ffn_up.weight	0x7e596c00	0xd48000
34	blk.1.inp_gate.weight	0x7f2dec00	0x5a000
35	blk.1.layer_output_scale.weight	0x7f338c00	0x4
36	blk.1.post_attention_norm.weight	0x7f338c20	0x2800
37	blk.1.post_ffw_norm.weight	0x7f33b420	0x2800
38	blk.1.post_norm.weight	0x7f33dc20	0x2800
39	blk.1.proj.weight	0x7f340420	0x78000
40	blk.2.attn_k.weight	0x7f3b8420	0xb4000
41	blk.2.attn_k_norm.weight	0x7f46c420	0x400
42	blk.2.attn_norm.weight	0x7f46c820	0x2800
43	blk.2.attn_output.weight	0x7f46f020	0x370000
44	blk.2.attn_q.weight	0x7f7df020	0x2d0000
45	blk.2.attn_q_norm.weight	0x7faaf020	0x400
46	blk.2.attn_v.weight	0x7faaf420	0xdc000
47	blk.2.ffn_down.weight	0x7fb8b420	0xd48000
48	blk.2.ffn_gate.weight	0x808d3420	0xd48000
49	blk.2.ffn_norm.weight	0x8161b420	0x2800
50	blk.2.ffn_up.weight	0x8161dc20	0xd48000
51	blk.2.inp_gate.weight	0x82365c20	0x6e000
52	blk.2.layer_output_scale.weight	0x823d3c20	0x4
53	blk.2.post_attention_norm.weight	0x823d3c40	0x2800
54	blk.2.post_ffw_norm.weight	0x823d6440	0x2800
55	blk.2.post_norm.weight	0x823d8c40	0x2800
56	blk.2.proj.weight	0x823db440	0x78000
57	blk.3.attn_k.weight	0x82453440	0xaa000
58	blk.3.attn_k_norm.weight	0x824fd440	0x400
59	blk.3.attn_norm.weight	0x824fd840	0x2800
60	blk.3.attn_output.weight	0x82500040	0x370000
61	blk.3.attn_q.weight	0x82870040	0x2a8000
62	blk.3.attn_q_norm.weight	0x82b18040	0x400
63	blk.3.attn_v.weight	0x82b18440	0xdc000
64	blk.3.ffn_down.weight	0x82bf4440	0xd48000
65	blk.3.ffn_gate.weight	0x8393c440	0xd48000
66	blk.3.ffn_norm.weight	0x84684440	0x2800
67	blk.3.ffn_up.weight	0x84686c40	0xe10000
68	blk.3.inp_gate.weight	0x85496c40	0x5a000
69	blk.3.layer_output_scale.weight	0x854f0c40	0x4
70	blk.3.post_attention_norm.weight	0x854f0c60	0x2800
71	blk.3.post_ffw_norm.weight	0x854f3460	0x2800
72	blk.3.post_norm.weight	0x854f5c60	0x2800
73	blk.3.proj.weight	0x854f8460	0x6e000
74	blk.4.attn_k.weight	0x85566460	0xaa000
75	blk.4.attn_k_norm.weight	0x85610460	0x400
76	blk.4.attn_norm.weight	0x85610860	0x2800
77	blk.4.attn_output.weight	0x85613060	0x370000
78	blk.4.attn_q.weight	0x85983060	0x2a8000
79	blk.4.attn_q_norm.weight	0x85c2b060	0x400
80	blk.4.attn_v.weight	0x85c2b460	0xb4000
81	blk.4.ffn_down.weight	0x85cdf460	0xd48000
82	blk.4.ffn_gate.weight	0x86a27460	0xd48000
83	blk.4.ffn_norm.weight	0x8776f460	0x2800
84	blk.4.ffn_up.weight	0x87771c60	0xd48000
85	blk.4.inp_gate.weight	0x884b9c60	0x5a000
86	blk.4.layer_output_scale.weight	0x88513c60	0x4
87	blk.4.post_attention_norm.weight	0x88513c80	0x2800
88	blk.4.post_ffw_norm.weight	0x88516480	0x2800
89	blk.4.post_norm.weight	0x88518c80	0x2800
90	blk.4.proj.weight	0x8851b480	0x6e000
91	blk.5.attn_k.weight	0x88589480	0x154000
92	blk.5.attn_k_norm.weight	0x886dd480	0x800
93	blk.5.attn_norm.weight	0x886ddc80	0x2800
94	blk.5.attn_output.weight	0x886e0480	0x550000
95	blk.5.attn_q.weight	0x88c30480	0x550000
96	blk.5.attn_q_norm.weight	0x89180480	0x800
97	blk.5.attn_v.weight	0x89180c80	0x168000
98	blk.5.ffn_down.weight	0x892e8c80	0xd48000
99	blk.5.ffn_gate.weight	0x8a030c80	0xd48000
100	blk.5.ffn_norm.weight	0x8ad78c80	0x2800
101	blk.5.ffn_up.weight	0x8ad7b480	0x1130000
102	blk.5.inp_gate.weight	0x8beab480	0x6e000
103	blk.5.layer_output_scale.weight	0x8bf19480	0x4
104	blk.5.post_attention_norm.weight	0x8bf194a0	0x2800
105	blk.5.post_ffw_norm.weight	0x8bf1bca0	0x2800
106	blk.5.post_norm.weight	0x8bf1e4a0	0x2800
107	blk.5.proj.weight	0x8bf20ca0	0x64000
108	blk.6.attn_k.weight	0x8bf84ca0	0xaa000
109	blk.6.attn_k_norm.weight	0x8c02eca0	0x400
110	blk.6.attn_norm.weight	0x8c02f0a0	0x2800
111	blk.6.attn_output.weight	0x8c0318a0	0x370000
112	blk.6.attn_q.weight	0x8c3a18a0	0x2a8000
113	blk.6.attn_q_norm.weight	0x8c6498a0	0x400
114	blk.6.attn_v.weight	0x8c649ca0	0xdc000
115	blk.6.ffn_down.weight	0x8c725ca0	0xd48000
116	blk.6.ffn_gate.weight	0x8d46dca0	0xd48000
117	blk.6.ffn_norm.weight	0x8e1b5ca0	0x2800
118	blk.6.ffn_up.weight	0x8e1b84a0	0x1130000
119	blk.6.inp_gate.weight	0x8f2e84a0	0x6e000
120	blk.6.layer_output_scale.weight	0x8f3564a0	0x4
121	blk.6.post_attention_norm.weight	0x8f3564c0	0x2800
122	blk.6.post_ffw_norm.weight	0x8f358cc0	0x2800
123	blk.6.post_norm.weight	0x8f35b4c0	0x2800
124	blk.6.proj.weight	0x8f35dcc0	0x6e000
125	blk.7.attn_k.weight	0x8f3cbcc0	0xaa000
126	blk.7.attn_k_norm.weight	0x8f475cc0	0x400
127	blk.7.attn_norm.weight	0x8f4760c0	0x2800
128	blk.7.attn_output.weight	0x8f4788c0	0x370000
129	blk.7.attn_q.weight	0x8f7e88c0	0x2a8000
130	blk.7.attn_q_norm.weight	0x8fa908c0	0x400
131	blk.7.attn_v.weight	0x8fa90cc0	0xaa000
132	blk.7.ffn_down.weight	0x8fb3acc0	0xd48000
133	blk.7.ffn_gate.weight	0x90882cc0	0xd48000
134	blk.7.ffn_norm.weight	0x915cacc0	0x2800
135	blk.7.ffn_up.weight	0x915cd4c0	0xd48000
136	blk.7.inp_gate.weight	0x923154c0	0x6e000
137	blk.7.layer_output_scale.weight	0x923834c0	0x4
138	blk.7.post_attention_norm.weight	0x923834e0	0x2800
139	blk.7.post_ffw_norm.weight	0x92385ce0	0x2800
140	blk.7.post_norm.weight	0x923884e0	0x2800
141	blk.7.proj.weight	0x9238ace0	0x78000
142	blk.8.attn_k.weight	0x92402ce0	0xaa000
143	blk.8.attn_k_norm.weight	0x924acce0	0x400
144	blk.8.attn_norm.weight	0x924ad0e0	0x2800
145	blk.8.attn_output.weight	0x924af8e0	0x370000
146	blk.8.attn_q.weight	0x9281f8e0	0x2a8000
147	blk.8.attn_q_norm.weight	0x92ac78e0	0x400
148	blk.8.attn_v.weight	0x92ac7ce0	0xb4000
149	blk.8.ffn_down.weight	0x92b7bce0	0xd48000
150	blk.8.ffn_gate.weight	0x938c3ce0	0xd48000
151	blk.8.ffn_norm.weight	0x9460bce0	0x2800
152	blk.8.ffn_up.weight	0x9460e4e0	0xd48000
153	blk.8.inp_gate.weight	0x953564e0	0x5a000
154	blk.8.layer_output_scale.weight	0x953b04e0	0x4
155	blk.8.post_attention_norm.weight	0x953b0500	0x2800
156	blk.8.post_ffw_norm.weight	0x953b2d00	0x2800
157	blk.8.post_norm.weight	0x953b5500	0x2800
158	blk.8.proj.weight	0x953b7d00	0x6e000
159	blk.9.attn_k.weight	0x95425d00	0xaa000
160	blk.9.attn_k_norm.weight	0x954cfd00	0x400
161	blk.9.attn_norm.weight	0x954d0100	0x2800
162	blk.9.attn_output.weight	0x954d2900	0x370000
163	blk.9.attn_q.weight	0x95842900	0x2a8000
164	blk.9.attn_q_norm.weight	0x95aea900	0x400
165	blk.9.attn_v.weight	0x95aead00	0xb4000
166	blk.9.ffn_down.weight	0x95b9ed00	0xd48000
167	blk.9.ffn_gate.weight	0x968e6d00	0xd48000
168	blk.9.ffn_norm.weight	0x9762ed00	0x2800
169	blk.9.ffn_up.weight	0x97631500	0xd48000
170	blk.9.inp_gate.weight	0x98379500	0x5a000
171	blk.9.layer_output_scale.weight	0x983d3500	0x4
172	blk.9.post_attention_norm.weight	0x983d3520	0x2800
173	blk.9.post_ffw_norm.weight	0x983d5d20	0x2800
174	blk.9.post_norm.weight	0x983d8520	0x2800
175	blk.9.proj.weight	0x983dad20	0x64000
176	blk.10.attn_k.weight	0x9843ed20	0xaa000
177	blk.10.attn_k_norm.weight	0x984e8d20	0x400
178	blk.10.attn_norm.weight	0x984e9120	0x2800
179	blk.10.attn_output.weight	0x984eb920	0x370000
180	blk.10.attn_q.weight	0x9885b920	0x2d0000
181	blk.10.attn_q_norm.weight	0x98b2b920	0x400
182	blk.10.attn_v.weight	0x98b2bd20	0xdc000
183	blk.10.ffn_down.weight	0x98c07d20	0xd48000
184	blk.10.ffn_gate.weight	0x9994fd20	0xd48000
185	blk.10.ffn_norm.weight	0x9a697d20	0x2800
186	blk.10.ffn_up.weight	0x9a69a520	0xd48000
187	blk.10.inp_gate.weight	0x9b3e2520	0x6e000
188	blk.10.layer_output_scale.weight	0x9b450520	0x4
189	blk.10.post_attention_norm.weight	0x9b450540	0x2800
190	blk.10.post_ffw_norm.weight	0x9b452d40	0x2800
191	blk.10.post_norm.weight	0x9b455540	0x2800
192	blk.10.proj.weight	0x9b457d40	0x78000
193	blk.11.attn_k.weight	0x9b4cfd40	0x154000
194	blk.11.attn_k_norm.weight	0x9b623d40	0x800
195	blk.11.attn_norm.weight	0x9b624540	0x2800
196	blk.11.attn_output.weight	0x9b626d40	0x550000
197	blk.11.attn_q.weight	0x9bb76d40	0x550000
198	blk.11.attn_q_norm.weight	0x9c0c6d40	0x800
199	blk.11.attn_v.weight	0x9c0c7540	0x1b8000
200	blk.11.ffn_down.weight	0x9c27f540	0xd48000
201	blk.11.ffn_gate.weight	0x9cfc7540	0xd48000
202	blk.11.ffn_norm.weight	0x9dd0f540	0x2800
203	blk.11.ffn_up.weight	0x9dd11d40	0xd48000
204	blk.11.inp_gate.weight	0x9ea59d40	0x5a000
205	blk.11.layer_output_scale.weight	0x9eab3d40	0x4
206	blk.11.post_attention_norm.weight	0x9eab3d60	0x2800
207	blk.11.post_ffw_norm.weight	0x9eab6560	0x2800
208	blk.11.post_norm.weight	0x9eab8d60	0x2800
209	blk.11.proj.weight	0x9eabb560	0x64000
210	blk.12.attn_k.weight	0x9eb1f560	0xb4000
211	blk.12.attn_k_norm.weight	0x9ebd3560	0x400
212	blk.12.attn_norm.weight	0x9ebd3960	0x2800
213	blk.12.attn_output.weight	0x9ebd6160	0x370000
214	blk.12.attn_q.weight	0x9ef46160	0x2d0000
215	blk.12.attn_q_norm.weight	0x9f216160	0x400
216	blk.12.attn_v.weight	0x9f216560	0xdc000
217	blk.12.ffn_down.weight	0x9f2f2560	0xd48000
218	blk.12.ffn_gate.weight	0xa003a560	0xd48000
219	blk.12.ffn_norm.weight	0xa0d82560	0x2800
220	blk.12.ffn_up.weight	0xa0d84d60	0xd48000
221	blk.12.inp_gate.weight	0xa1accd60	0x5a000
222	blk.12.layer_output_scale.weight	0xa1b26d60	0x4
223	blk.12.post_attention_norm.weight	0xa1b26d80	0x2800
224	blk.12.post_ffw_norm.weight	0xa1b29580	0x2800
225	blk.12.post_norm.weight	0xa1b2bd80	0x2800
226	blk.12.proj.weight	0xa1b2e580	0x78000
227	blk.13.attn_k.weight	0xa1ba6580	0xb4000
228	blk.13.attn_k_norm.weight	0xa1c5a580	0x400
229	blk.13.attn_norm.weight	0xa1c5a980	0x2800
230	blk.13.attn_output.weight	0xa1c5d180	0x370000
231	blk.13.attn_q.weight	0xa1fcd180	0x2d0000
232	blk.13.attn_q_norm.weight	0xa229d180	0x400
233	blk.13.attn_v.weight	0xa229d580	0xdc000
234	blk.13.ffn_down.weight	0xa2379580	0xd48000
235	blk.13.ffn_gate.weight	0xa30c1580	0xd48000
236	blk.13.ffn_norm.weight	0xa3e09580	0x2800
237	blk.13.ffn_up.weight	0xa3e0bd80	0xd48000
238	blk.13.inp_gate.weight	0xa4b53d80	0x6e000
239	blk.13.layer_output_scale.weight	0xa4bc1d80	0x4
240	blk.13.post_attention_norm.weight	0xa4bc1da0	0x2800
241	blk.13.post_ffw_norm.weight	0xa4bc45a0	0x2800
242	blk.13.post_norm.weight	0xa4bc6da0	0x2800
243	blk.13.proj.weight	0xa4bc95a0	0xaa000
244	blk.14.attn_k.weight	0xa4c735a0	0xb4000
245	blk.14.attn_k_norm.weight	0xa4d275a0	0x400
246	blk.14.attn_norm.weight	0xa4d279a0	0x2800
247	blk.14.attn_output.weight	0xa4d2a1a0	0x370000
248	blk.14.attn_q.weight	0xa509a1a0	0x2a8000
249	blk.14.attn_q_norm.weight	0xa53421a0	0x400
250	blk.14.attn_v.weight	0xa53425a0	0xdc000
251	blk.14.ffn_down.weight	0xa541e5a0	0xd48000
252	blk.14.ffn_gate.weight	0xa61665a0	0xd48000
253	blk.14.ffn_norm.weight	0xa6eae5a0	0x2800
254	blk.14.ffn_up.weight	0xa6eb0da0	0xd48000
255	blk.14.inp_gate.weight	0xa7bf8da0	0x6e000
256	blk.14.layer_output_scale.weight	0xa7c66da0	0x4
257	blk.14.post_attention_norm.weight	0xa7c66dc0	0x2800
258	blk.14.post_ffw_norm.weight	0xa7c695c0	0x2800
259	blk.14.post_norm.weight	0xa7c6bdc0	0x2800
260	blk.14.proj.weight	0xa7c6e5c0	0x78000
261	blk.15.attn_k.weight	0xa7ce65c0	0xb4000
262	blk.15.attn_k_norm.weight	0xa7d9a5c0	0x400
263	blk.15.attn_norm.weight	0xa7d9a9c0	0x2800
264	blk.15.attn_output.weight	0xa7d9d1c0	0x370000
265	blk.15.attn_q.weight	0xa810d1c0	0x2a8000
266	blk.15.attn_q_norm.weight	0xa83b51c0	0x400
267	blk.15.attn_v.weight	0xa83b55c0	0xdc000
268	blk.15.ffn_down.weight	0xa84915c0	0xd48000
269	blk.15.ffn_gate.weight	0xa91d95c0	0xd48000
270	blk.15.ffn_norm.weight	0xa9f215c0	0x2800
271	blk.15.ffn_up.weight	0xa9f23dc0	0xd48000
272	blk.15.inp_gate.weight	0xaac6bdc0	0x5a000
273	blk.15.layer_output_scale.weight	0xaacc5dc0	0x4
274	blk.15.post_attention_norm.weight	0xaacc5de0	0x2800
275	blk.15.post_ffw_norm.weight	0xaacc85e0	0x2800
276	blk.15.post_norm.weight	0xaaccade0	0x2800
277	blk.15.proj.weight	0xaaccd5e0	0x140000
278	blk.16.attn_k.weight	0xaae0d5e0	0xb4000
279	blk.16.attn_k_norm.weight	0xaaec15e0	0x400
280	blk.16.attn_norm.weight	0xaaec19e0	0x2800
281	blk.16.attn_output.weight	0xaaec41e0	0x370000
282	blk.16.attn_q.weight	0xab2341e0	0x2d0000
283	blk.16.attn_q_norm.weight	0xab5041e0	0x400
284	blk.16.attn_v.weight	0xab5045e0	0xdc000
285	blk.16.ffn_down.weight	0xab5e05e0	0xd48000
286	blk.16.ffn_gate.weight	0xac3285e0	0xd48000
287	blk.16.ffn_norm.weight	0xad0705e0	0x2800
288	blk.16.ffn_up.weight	0xad072de0	0xd48000
289	blk.16.inp_gate.weight	0xaddbade0	0x5a000
290	blk.16.layer_output_scale.weight	0xade14de0	0x4
291	blk.16.post_attention_norm.weight	0xade14e00	0x2800
292	blk.16.post_ffw_norm.weight	0xade17600	0x2800
293	blk.16.post_norm.weight	0xade19e00	0x2800
294	blk.16.proj.weight	0xade1c600	0x78000
295	blk.17.attn_k.weight	0xade94600	0x154000
296	blk.17.attn_k_norm.weight	0xadfe8600	0x800
297	blk.17.attn_norm.weight	0xadfe8e00	0x2800
298	blk.17.attn_output.weight	0xadfeb600	0x550000
299	blk.17.attn_q.weight	0xae53b600	0x550000
300	blk.17.attn_q_norm.weight	0xaea8b600	0x800
301	blk.17.attn_v.weight	0xaea8be00	0x1b8000
302	blk.17.ffn_down.weight	0xaec43e00	0xd48000
303	blk.17.ffn_gate.weight	0xaf98be00	0xd48000
304	blk.17.ffn_norm.weight	0xb06d3e00	0x2800
305	blk.17.ffn_up.weight	0xb06d6600	0xd48000
306	blk.17.inp_gate.weight	0xb141e600	0x6e000
307	blk.17.layer_output_scale.weight	0xb148c600	0x4
308	blk.17.post_attention_norm.weight	0xb148c620	0x2800
309	blk.17.post_ffw_norm.weight	0xb148ee20	0x2800
310	blk.17.post_norm.weight	0xb1491620	0x2800
311	blk.17.proj.weight	0xb1493e20	0x78000
312	blk.18.attn_k.weight	0xb150be20	0xb4000
313	blk.18.attn_k_norm.weight	0xb15bfe20	0x400
314	blk.18.attn_norm.weight	0xb15c0220	0x2800
315	blk.18.attn_output.weight	0xb15c2a20	0x370000
316	blk.18.attn_q.weight	0xb1932a20	0x2d0000
317	blk.18.attn_q_norm.weight	0xb1c02a20	0x400
318	blk.18.attn_v.weight	0xb1c02e20	0xdc000
319	blk.18.ffn_down.weight	0xb1cdee20	0xd48000
320	blk.18.ffn_gate.weight	0xb2a26e20	0xd48000
321	blk.18.ffn_norm.weight	0xb376ee20	0x2800
322	blk.18.ffn_up.weight	0xb3771620	0xd48000
323	blk.18.inp_gate.weight	0xb44b9620	0x83400
324	blk.18.layer_output_scale.weight	0xb453ca20	0x4
325	blk.18.post_attention_norm.weight	0xb453ca40	0x2800
326	blk.18.post_ffw_norm.weight	0xb453f240	0x2800
327	blk.18.post_norm.weight	0xb4541a40	0x2800
328	blk.18.proj.weight	0xb4544240	0x78000
329	blk.19.attn_k.weight	0xb45bc240	0xb4000
330	blk.19.attn_k_norm.weight	0xb4670240	0x400
331	blk.19.attn_norm.weight	0xb4670640	0x2800
332	blk.19.attn_output.weight	0xb4672e40	0x370000
333	blk.19.attn_q.weight	0xb49e2e40	0x2d0000
334	blk.19.attn_q_norm.weight	0xb4cb2e40	0x400
335	blk.19.attn_v.weight	0xb4cb3240	0xdc000
336	blk.19.ffn_down.weight	0xb4d8f240	0xd48000
337	blk.19.ffn_gate.weight	0xb5ad7240	0xd48000
338	blk.19.ffn_norm.weight	0xb681f240	0x2800
339	blk.19.ffn_up.weight	0xb6821a40	0xd48000
340	blk.19.inp_gate.weight	0xb7569a40	0x6e000
341	blk.19.layer_output_scale.weight	0xb75d7a40	0x4
342	blk.19.post_attention_norm.weight	0xb75d7a60	0x2800
343	blk.19.post_ffw_norm.weight	0xb75da260	0x2800
344	blk.19.post_norm.weight	0xb75dca60	0x2800
345	blk.19.proj.weight	0xb75df260	0x83400
346	blk.20.attn_k.weight	0xb7662660	0xb4000
347	blk.20.attn_k_norm.weight	0xb7716660	0x400
348	blk.20.attn_norm.weight	0xb7716a60	0x2800
349	blk.20.attn_output.weight	0xb7719260	0x370000
350	blk.20.attn_q.weight	0xb7a89260	0x2d0000
351	blk.20.attn_q_norm.weight	0xb7d59260	0x400
352	blk.20.attn_v.weight	0xb7d59660	0xdc000
353	blk.20.ffn_down.weight	0xb7e35660	0xd48000
354	blk.20.ffn_gate.weight	0xb8b7d660	0xd48000
355	blk.20.ffn_norm.weight	0xb98c5660	0x2800
356	blk.20.ffn_up.weight	0xb98c7e60	0xd48000
357	blk.20.inp_gate.weight	0xba60fe60	0x6e000
358	blk.20.layer_output_scale.weight	0xba67de60	0x4
359	blk.20.post_attention_norm.weight	0xba67de80	0x2800
360	blk.20.post_ffw_norm.weight	0xba680680	0x2800
361	blk.20.post_norm.weight	0xba682e80	0x2800
362	blk.20.proj.weight	0xba685680	0x78000
363	blk.21.attn_k.weight	0xba6fd680	0xb4000
364	blk.21.attn_k_norm.weight	0xba7b1680	0x400
365	blk.21.attn_norm.weight	0xba7b1a80	0x2800
366	blk.21.attn_output.weight	0xba7b4280	0x370000
367	blk.21.attn_q.weight	0xbab24280	0x2a8000
368	blk.21.attn_q_norm.weight	0xbadcc280	0x400
369	blk.21.attn_v.weight	0xbadcc680	0xdc000
370	blk.21.ffn_down.weight	0xbaea8680	0xd48000
371	blk.21.ffn_gate.weight	0xbbbf0680	0xd48000
372	blk.21.ffn_norm.weight	0xbc938680	0x2800
373	blk.21.ffn_up.weight	0xbc93ae80	0x1130000
374	blk.21.inp_gate.weight	0xbda6ae80	0x5a000
375	blk.21.layer_output_scale.weight	0xbdac4e80	0x4
376	blk.21.post_attention_norm.weight	0xbdac4ea0	0x2800
377	blk.21.post_ffw_norm.weight	0xbdac76a0	0x2800
378	blk.21.post_norm.weight	0xbdac9ea0	0x2800
379	blk.21.proj.weight	0xbdacc6a0	0xaa000
380	blk.22.attn_k.weight	0xbdb766a0	0xdc000
381	blk.22.attn_k_norm.weight	0xbdc526a0	0x400
382	blk.22.attn_norm.weight	0xbdc52aa0	0x2800
383	blk.22.attn_output.weight	0xbdc552a0	0x370000
384	blk.22.attn_q.weight	0xbdfc52a0	0x370000
385	blk.22.attn_q_norm.weight	0xbe3352a0	0x400
386	blk.22.attn_v.weight	0xbe3356a0	0xdc000
387	blk.22.ffn_down.weight	0xbe4116a0	0xd48000
388	blk.22.ffn_gate.weight	0xbf1596a0	0xd48000
389	blk.22.ffn_norm.weight	0xbfea16a0	0x2800
390	blk.22.ffn_up.weight	0xbfea3ea0	0xd48000
391	blk.22.inp_gate.weight	0xc0bebea0	0x5a000
392	blk.22.layer_output_scale.weight	0xc0c45ea0	0x4
393	blk.22.post_attention_norm.weight	0xc0c45ec0	0x2800
394	blk.22.post_ffw_norm.weight	0xc0c486c0	0x2800
395	blk.22.post_norm.weight	0xc0c4aec0	0x2800
396	blk.22.proj.weight	0xc0c4d6c0	0x6e000
397	blk.23.attn_k.weight	0xc0cbb6c0	0x1b8000
398	blk.23.attn_k_norm.weight	0xc0e736c0	0x800
399	blk.23.attn_norm.weight	0xc0e73ec0	0x2800
400	blk.23.attn_output.weight	0xc0e766c0	0x550000
401	blk.23.attn_q.weight	0xc13c66c0	0x6e0000
402	blk.23.attn_q_norm.weight	0xc1aa66c0	0x800
403	blk.23.attn_v.weight	0xc1aa6ec0	0x1b8000
404	blk.23.ffn_down.weight	0xc1c5eec0	0xd48000
405	blk.23.ffn_gate.weight	0xc29a6ec0	0xd48000
406	blk.23.ffn_norm.weight	0xc36eeec0	0x2800
407	blk.23.ffn_up.weight	0xc36f16c0	0x1130000
408	blk.23.inp_gate.weight	0xc48216c0	0x6e000
409	blk.23.layer_output_scale.weight	0xc488f6c0	0x4
410	blk.23.post_attention_norm.weight	0xc488f6e0	0x2800
411	blk.23.post_ffw_norm.weight	0xc4891ee0	0x2800
412	blk.23.post_norm.weight	0xc48946e0	0x2800
413	blk.23.proj.weight	0xc4896ee0	0x78000
414	blk.24.attn_k.weight	0xc490eee0	0xdc000
415	blk.24.attn_k_norm.weight	0xc49eaee0	0x400
416	blk.24.attn_norm.weight	0xc49eb2e0	0x2800
417	blk.24.attn_output.weight	0xc49edae0	0x370000
418	blk.24.attn_q.weight	0xc4d5dae0	0x2d0000
419	blk.24.attn_q_norm.weight	0xc502dae0	0x400
420	blk.24.attn_v.weight	0xc502dee0	0xb4000
421	blk.24.ffn_down.weight	0xc50e1ee0	0xd48000
422	blk.24.ffn_gate.weight	0xc5e29ee0	0xd48000
423	blk.24.ffn_norm.weight	0xc6b71ee0	0x2800
424	blk.24.ffn_up.weight	0xc6b746e0	0x1130000
425	blk.24.inp_gate.weight	0xc7ca46e0	0x6e000
426	blk.24.layer_output_scale.weight	0xc7d126e0	0x4
427	blk.24.post_attention_norm.weight	0xc7d12700	0x2800
428	blk.24.post_ffw_norm.weight	0xc7d14f00	0x2800
429	blk.24.post_norm.weight	0xc7d17700	0x2800
430	blk.24.proj.weight	0xc7d19f00	0x6e000
431	blk.25.attn_k.weight	0xc7d87f00	0xdc000
432	blk.25.attn_k_norm.weight	0xc7e63f00	0x400
433	blk.25.attn_norm.weight	0xc7e64300	0x2800
434	blk.25.attn_output.weight	0xc7e66b00	0x370000
435	blk.25.attn_q.weight	0xc81d6b00	0x2d0000
436	blk.25.attn_q_norm.weight	0xc84a6b00	0x400
437	blk.25.attn_v.weight	0xc84a6f00	0xb4000
438	blk.25.ffn_down.weight	0xc855af00	0xd48000
439	blk.25.ffn_gate.weight	0xc92a2f00	0xd48000
440	blk.25.ffn_norm.weight	0xc9feaf00	0x2800
441	blk.25.ffn_up.weight	0xc9fed700	0xd48000
442	blk.25.inp_gate.weight	0xcad35700	0x5a000
443	blk.25.layer_output_scale.weight	0xcad8f700	0x4
444	blk.25.post_attention_norm.weight	0xcad8f720	0x2800
445	blk.25.post_ffw_norm.weight	0xcad91f20	0x2800
446	blk.25.post_norm.weight	0xcad94720	0x2800
447	blk.25.proj.weight	0xcad96f20	0x78000
448	blk.26.attn_k.weight	0xcae0ef20	0xdc000
449	blk.26.attn_k_norm.weight	0xcaeeaf20	0x400
450	blk.26.attn_norm.weight	0xcaeeb320	0x2800
451	blk.26.attn_output.weight	0xcaeedb20	0x370000
452	blk.26.attn_q.weight	0xcb25db20	0x2d0000
453	blk.26.attn_q_norm.weight	0xcb52db20	0x400
454	blk.26.attn_v.weight	0xcb52df20	0xb4000
455	blk.26.ffn_down.weight	0xcb5e1f20	0xd48000
456	blk.26.ffn_gate.weight	0xcc329f20	0xd48000
457	blk.26.ffn_norm.weight	0xcd071f20	0x2800
458	blk.26.ffn_up.weight	0xcd074720	0xe10000
459	blk.26.inp_gate.weight	0xcde84720	0x5a000
460	blk.26.layer_output_scale.weight	0xcdede720	0x4
461	blk.26.post_attention_norm.weight	0xcdede740	0x2800
462	blk.26.post_ffw_norm.weight	0xcdee0f40	0x2800
463	blk.26.post_norm.weight	0xcdee3740	0x2800
464	blk.26.proj.weight	0xcdee5f40	0x78000
465	blk.27.attn_k.weight	0xcdf5df40	0xdc000
466	blk.27.attn_k_norm.weight	0xce039f40	0x400
467	blk.27.attn_norm.weight	0xce03a340	0x2800
468	blk.27.attn_output.weight	0xce03cb40	0x370000
469	blk.27.attn_q.weight	0xce3acb40	0x2d0000
470	blk.27.attn_q_norm.weight	0xce67cb40	0x400
471	blk.27.attn_v.weight	0xce67cf40	0xb4000
472	blk.27.ffn_down.weight	0xce730f40	0xd48000
473	blk.27.ffn_gate.weight	0xcf478f40	0xd48000
474	blk.27.ffn_norm.weight	0xd01c0f40	0x2800
475	blk.27.ffn_up.weight	0xd01c3740	0xe10000
476	blk.27.inp_gate.weight	0xd0fd3740	0x5a000
477	blk.27.layer_output_scale.weight	0xd102d740	0x4
478	blk.27.post_attention_norm.weight	0xd102d760	0x2800
479	blk.27.post_ffw_norm.weight	0xd102ff60	0x2800
480	blk.27.post_norm.weight	0xd1032760	0x2800
481	blk.27.proj.weight	0xd1034f60	0x78000
482	blk.28.attn_k.weight	0xd10acf60	0xdc000
483	blk.28.attn_k_norm.weight	0xd1188f60	0x400
484	blk.28.attn_norm.weight	0xd1189360	0x2800
485	blk.28.attn_output.weight	0xd118bb60	0x370000
486	blk.28.attn_q.weight	0xd14fbb60	0x2d0000
487	blk.28.attn_q_norm.weight	0xd17cbb60	0x400
488	blk.28.attn_v.weight	0xd17cbf60	0xb4000
489	blk.28.ffn_down.weight	0xd187ff60	0xd48000
490	blk.28.ffn_gate.weight	0xd25c7f60	0xd48000
491	blk.28.ffn_norm.weight	0xd330ff60	0x2800
492	blk.28.ffn_up.weight	0xd3312760	0xd48000
493	blk.28.inp_gate.weight	0xd405a760	0x5a000
494	blk.28.layer_output_scale.weight	0xd40b4760	0x4
495	blk.28.post_attention_norm.weight	0xd40b4780	0x2800
496	blk.28.post_ffw_norm.weight	0xd40b6f80	0x2800
497	blk.28.post_norm.weight	0xd40b9780	0x2800
498	blk.28.proj.weight	0xd40bbf80	0x78000
499	blk.29.attn_k.weight	0xd4133f80	0x1b8000
500	blk.29.attn_k_norm.weight	0xd42ebf80	0x800
501	blk.29.attn_norm.weight	0xd42ec780	0x2800
502	blk.29.attn_output.weight	0xd42eef80	0x550000
503	blk.29.attn_q.weight	0xd483ef80	0x550000
504	blk.29.attn_q_norm.weight	0xd4d8ef80	0x800
505	blk.29.attn_v.weight	0xd4d8f780	0x168000
506	blk.29.ffn_down.weight	0xd4ef7780	0xd48000
507	blk.29.ffn_gate.weight	0xd5c3f780	0xd48000
508	blk.29.ffn_norm.weight	0xd6987780	0x2800
509	blk.29.ffn_up.weight	0xd6989f80	0xd48000
510	blk.29.inp_gate.weight	0xd76d1f80	0x5a000
511	blk.29.layer_output_scale.weight	0xd772bf80	0x4
512	blk.29.post_attention_norm.weight	0xd772bfa0	0x2800
513	blk.29.post_ffw_norm.weight	0xd772e7a0	0x2800
514	blk.29.post_norm.weight	0xd7730fa0	0x2800
515	blk.29.proj.weight	0xd77337a0	0x78000
516	blk.30.attn_k.weight	0xd77ab7a0	0xdc000
517	blk.30.attn_k_norm.weight	0xd78877a0	0x400
518	blk.30.attn_norm.weight	0xd7887ba0	0x2800
519	blk.30.attn_output.weight	0xd788a3a0	0x370000
520	blk.30.attn_q.weight	0xd7bfa3a0	0x2d0000
521	blk.30.attn_q_norm.weight	0xd7eca3a0	0x400
522	blk.30.attn_v.weight	0xd7eca7a0	0xb4000
523	blk.30.ffn_down.weight	0xd7f7e7a0	0xd48000
524	blk.30.ffn_gate.weight	0xd8cc67a0	0xd48000
525	blk.30.ffn_norm.weight	0xd9a0e7a0	0x2800
526	blk.30.ffn_up.weight	0xd9a10fa0	0xd48000
527	blk.30.inp_gate.weight	0xda758fa0	0x5a000
528	blk.30.layer_output_scale.weight	0xda7b2fa0	0x4
529	blk.30.post_attention_norm.weight	0xda7b2fc0	0x2800
530	blk.30.post_ffw_norm.weight	0xda7b57c0	0x2800
531	blk.30.post_norm.weight	0xda7b7fc0	0x2800
532	blk.30.proj.weight	0xda7ba7c0	0x6e000
533	blk.31.attn_k.weight	0xda8287c0	0xdc000
534	blk.31.attn_k_norm.weight	0xda9047c0	0x400
535	blk.31.attn_norm.weight	0xda904bc0	0x2800
536	blk.31.attn_output.weight	0xda9073c0	0x370000
537	blk.31.attn_q.weight	0xdac773c0	0x2d0000
538	blk.31.attn_q_norm.weight	0xdaf473c0	0x400
539	blk.31.attn_v.weight	0xdaf477c0	0xb4000
540	blk.31.ffn_down.weight	0xdaffb7c0	0xd48000
541	blk.31.ffn_gate.weight	0xdbd437c0	0xd48000
542	blk.31.ffn_norm.weight	0xdca8b7c0	0x2800
543	blk.31.ffn_up.weight	0xdca8dfc0	0xd48000
544	blk.31.inp_gate.weight	0xdd7d5fc0	0x5a000
545	blk.31.layer_output_scale.weight	0xdd82ffc0	0x4
546	blk.31.post_attention_norm.weight	0xdd82ffe0	0x2800
547	blk.31.post_ffw_norm.weight	0xdd8327e0	0x2800
548	blk.31.post_norm.weight	0xdd834fe0	0x2800
549	blk.31.proj.weight	0xdd8377e0	0x78000
550	blk.32.attn_k.weight	0xdd8af7e0	0xdc000
551	blk.32.attn_k_norm.weight	0xdd98b7e0	0x400
552	blk.32.attn_norm.weight	0xdd98bbe0	0x2800
553	blk.32.attn_output.weight	0xdd98e3e0	0x370000
554	blk.32.attn_q.weight	0xddcfe3e0	0x2a8000
555	blk.32.attn_q_norm.weight	0xddfa63e0	0x400
556	blk.32.attn_v.weight	0xddfa67e0	0xb4000
557	blk.32.ffn_down.weight	0xde05a7e0	0xd48000
558	blk.32.ffn_gate.weight	0xdeda27e0	0xd48000
559	blk.32.ffn_norm.weight	0xdfaea7e0	0x2800
560	blk.32.ffn_up.weight	0xdfaecfe0	0xd48000
561	blk.32.inp_gate.weight	0xe0834fe0	0x5a000
562	blk.32.layer_output_scale.weight	0xe088efe0	0x4
563	blk.32.post_attention_norm.weight	0xe088f000	0x2800
564	blk.32.post_ffw_norm.weight	0xe0891800	0x2800
565	blk.32.post_norm.weight	0xe0894000	0x2800
566	blk.32.proj.weight	0xe0896800	0x78000
567	blk.33.attn_k.weight	0xe090e800	0xdc000
568	blk.33.attn_k_norm.weight	0xe09ea800	0x400
569	blk.33.attn_norm.weight	0xe09eac00	0x2800
570	blk.33.attn_output.weight	0xe09ed400	0x370000
571	blk.33.attn_q.weight	0xe0d5d400	0x2a8000
572	blk.33.attn_q_norm.weight	0xe1005400	0x400
573	blk.33.attn_v.weight	0xe1005800	0xb4000
574	blk.33.ffn_down.weight	0xe10b9800	0xd48000
575	blk.33.ffn_gate.weight	0xe1e01800	0xd48000
576	blk.33.ffn_norm.weight	0xe2b49800	0x2800
577	blk.33.ffn_up.weight	0xe2b4c000	0xd48000
578	blk.33.inp_gate.weight	0xe3894000	0x5a000
579	blk.33.layer_output_scale.weight	0xe38ee000	0x4
580	blk.33.post_attention_norm.weight	0xe38ee020	0x2800
581	blk.33.post_ffw_norm.weight	0xe38f0820	0x2800
582	blk.33.post_norm.weight	0xe38f3020	0x2800
583	blk.33.proj.weight	0xe38f5820	0x64000
584	blk.34.attn_k.weight	0xe3959820	0xdc000
585	blk.34.attn_k_norm.weight	0xe3a35820	0x400
586	blk.34.attn_norm.weight	0xe3a35c20	0x2800
587	blk.34.attn_output.weight	0xe3a38420	0x370000
588	blk.34.attn_q.weight	0xe3da8420	0x2d0000
589	blk.34.attn_q_norm.weight	0xe4078420	0x400
590	blk.34.attn_v.weight	0xe4078820	0xb4000
591	blk.34.ffn_down.weight	0xe412c820	0xd48000
592	blk.34.ffn_gate.weight	0xe4e74820	0xd48000
593	blk.34.ffn_norm.weight	0xe5bbc820	0x2800
594	blk.34.ffn_up.weight	0xe5bbf020	0xd48000
595	blk.34.inp_gate.weight	0xe6907020	0x5a000
596	blk.34.layer_output_scale.weight	0xe6961020	0x4
597	blk.34.post_attention_norm.weight	0xe6961040	0x2800
598	blk.34.post_ffw_norm.weight	0xe6963840	0x2800
599	blk.34.post_norm.weight	0xe6966040	0x2800
600	blk.34.proj.weight	0xe6968840	0x6e000
601	blk.35.attn_k.weight	0xe69d6840	0x1b8000
602	blk.35.attn_k_norm.weight	0xe6b8e840	0x800
603	blk.35.attn_norm.weight	0xe6b8f040	0x2800
604	blk.35.attn_output.weight	0xe6b91840	0x550000
605	blk.35.attn_q.weight	0xe70e1840	0x5a0000
606	blk.35.attn_q_norm.weight	0xe7681840	0x800
607	blk.35.attn_v.weight	0xe7682040	0x168000
608	blk.35.ffn_down.weight	0xe77ea040	0xd48000
609	blk.35.ffn_gate.weight	0xe8532040	0xd48000
610	blk.35.ffn_norm.weight	0xe927a040	0x2800
611	blk.35.ffn_up.weight	0xe927c840	0xd48000
612	blk.35.inp_gate.weight	0xe9fc4840	0x5a000
613	blk.35.layer_output_scale.weight	0xea01e840	0x4
614	blk.35.post_attention_norm.weight	0xea01e860	0x2800
615	blk.35.post_ffw_norm.weight	0xea021060	0x2800
616	blk.35.post_norm.weight	0xea023860	0x2800
617	blk.35.proj.weight	0xea026060	0x64000
618	blk.36.attn_k.weight	0xea08a060	0xdc000
619	blk.36.attn_k_norm.weight	0xea166060	0x400
620	blk.36.attn_norm.weight	0xea166460	0x2800
621	blk.36.attn_output.weight	0xea168c60	0x370000
622	blk.36.attn_q.weight	0xea4d8c60	0x2a8000
623	blk.36.attn_q_norm.weight	0xea780c60	0x400
624	blk.36.attn_v.weight	0xea781060	0xb4000
625	blk.36.ffn_down.weight	0xea835060	0xd48000
626	blk.36.ffn_gate.weight	0xeb57d060	0xd48000
627	blk.36.ffn_norm.weight	0xec2c5060	0x2800
628	blk.36.ffn_up.weight	0xec2c7860	0xd48000
629	blk.36.inp_gate.weight	0xed00f860	0x5a000
630	blk.36.layer_output_scale.weight	0xed069860	0x4
631	blk.36.post_attention_norm.weight	0xed069880	0x2800
632	blk.36.post_ffw_norm.weight	0xed06c080	0x2800
633	blk.36.post_norm.weight	0xed06e880	0x2800
634	blk.36.proj.weight	0xed071080	0x6e000
635	blk.37.attn_k.weight	0xed0df080	0xdc000
636	blk.37.attn_k_norm.weight	0xed1bb080	0x400
637	blk.37.attn_norm.weight	0xed1bb480	0x2800
638	blk.37.attn_output.weight	0xed1bdc80	0x370000
639	blk.37.attn_q.weight	0xed52dc80	0x2a8000
640	blk.37.attn_q_norm.weight	0xed7d5c80	0x400
641	blk.37.attn_v.weight	0xed7d6080	0xb4000
642	blk.37.ffn_down.weight	0xed88a080	0xd48000
643	blk.37.ffn_gate.weight	0xee5d2080	0xd48000
644	blk.37.ffn_norm.weight	0xef31a080	0x2800
645	blk.37.ffn_up.weight	0xef31c880	0xd48000
646	blk.37.inp_gate.weight	0xf0064880	0x5a000
647	blk.37.layer_output_scale.weight	0xf00be880	0x4
648	blk.37.post_attention_norm.weight	0xf00be8a0	0x2800
649	blk.37.post_ffw_norm.weight	0xf00c10a0	0x2800
650	blk.37.post_norm.weight	0xf00c38a0	0x2800
651	blk.37.proj.weight	0xf00c60a0	0x78000
652	blk.38.attn_k.weight	0xf013e0a0	0xdc000
653	blk.38.attn_k_norm.weight	0xf021a0a0	0x400
654	blk.38.attn_norm.weight	0xf021a4a0	0x2800
655	blk.38.attn_output.weight	0xf021cca0	0x370000
656	blk.38.attn_q.weight	0xf058cca0	0x2d0000
657	blk.38.attn_q_norm.weight	0xf085cca0	0x400
658	blk.38.attn_v.weight	0xf085d0a0	0xb4000
659	blk.38.ffn_down.weight	0xf09110a0	0xd48000
660	blk.38.ffn_gate.weight	0xf16590a0	0xd48000
661	blk.38.ffn_norm.weight	0xf23a10a0	0x2800
662	blk.38.ffn_up.weight	0xf23a38a0	0xd48000
663	blk.38.inp_gate.weight	0xf30eb8a0	0x5a000
664	blk.38.layer_output_scale.weight	0xf31458a0	0x4
665	blk.38.post_attention_norm.weight	0xf31458c0	0x2800
666	blk.38.post_ffw_norm.weight	0xf31480c0	0x2800
667	blk.38.post_norm.weight	0xf314a8c0	0x2800
668	blk.38.proj.weight	0xf314d0c0	0x78000
669	blk.39.attn_k.weight	0xf31c50c0	0xdc000
670	blk.39.attn_k_norm.weight	0xf32a10c0	0x400
671	blk.39.attn_norm.weight	0xf32a14c0	0x2800
672	blk.39.attn_output.weight	0xf32a3cc0	0x370000
673	blk.39.attn_q.weight	0xf3613cc0	0x2d0000
674	blk.39.attn_q_norm.weight	0xf38e3cc0	0x400
675	blk.39.attn_v.weight	0xf38e40c0	0xb4000
676	blk.39.ffn_down.weight	0xf39980c0	0xd48000
677	blk.39.ffn_gate.weight	0xf46e00c0	0xd48000
678	blk.39.ffn_norm.weight	0xf54280c0	0x2800
679	blk.39.ffn_up.weight	0xf542a8c0	0xd48000
680	blk.39.inp_gate.weight	0xf61728c0	0x5a000
681	blk.39.layer_output_scale.weight	0xf61cc8c0	0x4
682	blk.39.post_attention_norm.weight	0xf61cc8e0	0x2800
683	blk.39.post_ffw_norm.weight	0xf61cf0e0	0x2800
684	blk.39.post_norm.weight	0xf61d18e0	0x2800
685	blk.39.proj.weight	0xf61d40e0	0x78000
686	blk.40.attn_k.weight	0xf624c0e0	0xdc000
687	blk.40.attn_k_norm.weight	0xf63280e0	0x400
688	blk.40.attn_norm.weight	0xf63284e0	0x2800
689	blk.40.attn_output.weight	0xf632ace0	0x370000
690	blk.40.attn_q.weight	0xf669ace0	0x2d0000
691	blk.40.attn_q_norm.weight	0xf696ace0	0x400
692	blk.40.attn_v.weight	0xf696b0e0	0xb4000
693	blk.40.ffn_down.weight	0xf6a1f0e0	0xd48000
694	blk.40.ffn_gate.weight	0xf77670e0	0xe10000
695	blk.40.ffn_norm.weight	0xf85770e0	0x2800
696	blk.40.ffn_up.weight	0xf85798e0	0x1130000
697	blk.40.inp_gate.weight	0xf96a98e0	0x5a000
698	blk.40.layer_output_scale.weight	0xf97038e0	0x4
699	blk.40.post_attention_norm.weight	0xf9703900	0x2800
700	blk.40.post_ffw_norm.weight	0xf9706100	0x2800
701	blk.40.post_norm.weight	0xf9708900	0x2800
702	blk.40.proj.weight	0xf970b100	0x78000
703	blk.41.attn_k.weight	0xf9783100	0x1b8000
704	blk.41.attn_k_norm.weight	0xf993b100	0x800
705	blk.41.attn_norm.weight	0xf993b900	0x2800
706	blk.41.attn_output.weight	0xf993e100	0x5a0000
707	blk.41.attn_q.weight	0xf9ede100	0x5a0000
708	blk.41.attn_q_norm.weight	0xfa47e100	0x800
709	blk.41.attn_v.weight	0xfa47e900	0x168000
710	blk.41.ffn_down.weight	0xfa5e6900	0xd48000
711	blk.41.ffn_gate.weight	0xfb32e900	0xe10000
712	blk.41.ffn_norm.weight	0xfc13e900	0x2800
713	blk.41.ffn_up.weight	0xfc141100	0xe10000
714	blk.41.inp_gate.weight	0xfcf51100	0x44c00
715	blk.41.layer_output_scale.weight	0xfcf95d00	0x4
716	blk.41.post_attention_norm.weight	0xfcf95d20	0x2800
717	blk.41.post_ffw_norm.weight	0xfcf98520	0x2800
718	blk.41.post_norm.weight	0xfcf9ad20	0x2800
719	blk.41.proj.weight	0xfcf9d520	0x78000

Base Tensor Group : ~4B Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
0	output_norm.weight	Output Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
1	per_layer_model_proj.weight	Per_Layer_Model_Proj (W)	( ~28M) 27525120	2560 x 10752 x 1 x 1	F16	16.0000
2	per_layer_proj_norm.weight	Per_Layer_Proj_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
3	per_layer_token_embd.weight	Per_Layer_Token_Embd (W)	( ~3B) 2818572288	10752 x 262144 x 1 x 1	Q4_K	4.5000
4	rope_freqs.weight	Rope_Freqs (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
5	token_embd.weight	Token Embedding (W)	(~671M) 671088640	2560 x 262144 x 1 x 1	Q4_K	4.5000

Total elements in base: ( ~4B) 3517189120
Percentage of total elements: 46.78%
Bits per Weight (BPW) for base: 4.5900 bits

Block 0 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
6	blk.0.attn_k.weight	Block 0 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
7	blk.0.attn_k_norm.weight	Block 0 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
8	blk.0.attn_norm.weight	Block 0 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
9	blk.0.attn_output.weight	Block 0 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q4_K	4.5000
10	blk.0.attn_q.weight	Block 0 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	IQ4_XS	4.2500
11	blk.0.attn_q_norm.weight	Block 0 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
12	blk.0.attn_v.weight	Block 0 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
13	blk.0.ffn_down.weight	Block 0 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
14	blk.0.ffn_gate.weight	Block 0 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
15	blk.0.ffn_norm.weight	Block 0 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
16	blk.0.ffn_up.weight	Block 0 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
17	blk.0.inp_gate.weight	Block 0 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
18	blk.0.layer_output_scale.weight	Block 0 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
19	blk.0.post_attention_norm.weight	Block 0 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
20	blk.0.post_ffw_norm.weight	Block 0 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
21	blk.0.post_norm.weight	Block 0 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
22	blk.0.proj.weight	Block 0 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.0: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.0: 4.2892 bits

Block 1 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
23	blk.1.attn_k.weight	Block 1 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
24	blk.1.attn_k_norm.weight	Block 1 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
25	blk.1.attn_norm.weight	Block 1 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
26	blk.1.attn_output.weight	Block 1 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
27	blk.1.attn_q.weight	Block 1 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	Q4_K	4.5000
28	blk.1.attn_q_norm.weight	Block 1 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
29	blk.1.attn_v.weight	Block 1 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
30	blk.1.ffn_down.weight	Block 1 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
31	blk.1.ffn_gate.weight	Block 1 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
32	blk.1.ffn_norm.weight	Block 1 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
33	blk.1.ffn_up.weight	Block 1 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
34	blk.1.inp_gate.weight	Block 1 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
35	blk.1.layer_output_scale.weight	Block 1 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
36	blk.1.post_attention_norm.weight	Block 1 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
37	blk.1.post_ffw_norm.weight	Block 1 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
38	blk.1.post_norm.weight	Block 1 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
39	blk.1.proj.weight	Block 1 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.1: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.1: 4.3737 bits

Block 2 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
40	blk.2.attn_k.weight	Block 2 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
41	blk.2.attn_k_norm.weight	Block 2 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
42	blk.2.attn_norm.weight	Block 2 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
43	blk.2.attn_output.weight	Block 2 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
44	blk.2.attn_q.weight	Block 2 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	Q4_K	4.5000
45	blk.2.attn_q_norm.weight	Block 2 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
46	blk.2.attn_v.weight	Block 2 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
47	blk.2.ffn_down.weight	Block 2 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
48	blk.2.ffn_gate.weight	Block 2 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
49	blk.2.ffn_norm.weight	Block 2 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
50	blk.2.ffn_up.weight	Block 2 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
51	blk.2.inp_gate.weight	Block 2 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q5_K	5.5000
52	blk.2.layer_output_scale.weight	Block 2 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
53	blk.2.post_attention_norm.weight	Block 2 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
54	blk.2.post_ffw_norm.weight	Block 2 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
55	blk.2.post_norm.weight	Block 2 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
56	blk.2.proj.weight	Block 2 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.2: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.2: 4.3807 bits

Block 3 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
57	blk.3.attn_k.weight	Block 3 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	IQ4_XS	4.2500
58	blk.3.attn_k_norm.weight	Block 3 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
59	blk.3.attn_norm.weight	Block 3 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
60	blk.3.attn_output.weight	Block 3 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
61	blk.3.attn_q.weight	Block 3 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	IQ4_XS	4.2500
62	blk.3.attn_q_norm.weight	Block 3 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
63	blk.3.attn_v.weight	Block 3 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
64	blk.3.ffn_down.weight	Block 3 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
65	blk.3.ffn_gate.weight	Block 3 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
66	blk.3.ffn_norm.weight	Block 3 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
67	blk.3.ffn_up.weight	Block 3 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	Q4_K	4.5000
68	blk.3.inp_gate.weight	Block 3 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
69	blk.3.layer_output_scale.weight	Block 3 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
70	blk.3.post_attention_norm.weight	Block 3 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
71	blk.3.post_ffw_norm.weight	Block 3 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
72	blk.3.post_norm.weight	Block 3 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
73	blk.3.proj.weight	Block 3 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_K	5.5000

Total elements in blk.3: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.3: 4.4230 bits

Block 4 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
74	blk.4.attn_k.weight	Block 4 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	IQ4_XS	4.2500
75	blk.4.attn_k_norm.weight	Block 4 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
76	blk.4.attn_norm.weight	Block 4 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
77	blk.4.attn_output.weight	Block 4 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
78	blk.4.attn_q.weight	Block 4 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	IQ4_XS	4.2500
79	blk.4.attn_q_norm.weight	Block 4 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
80	blk.4.attn_v.weight	Block 4 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
81	blk.4.ffn_down.weight	Block 4 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
82	blk.4.ffn_gate.weight	Block 4 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
83	blk.4.ffn_norm.weight	Block 4 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
84	blk.4.ffn_up.weight	Block 4 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
85	blk.4.inp_gate.weight	Block 4 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
86	blk.4.layer_output_scale.weight	Block 4 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
87	blk.4.post_attention_norm.weight	Block 4 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
88	blk.4.post_ffw_norm.weight	Block 4 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
89	blk.4.post_norm.weight	Block 4 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
90	blk.4.proj.weight	Block 4 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_K	5.5000

Total elements in blk.4: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.4: 4.3385 bits

Block 5 Tensor Group : ~106M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
91	blk.5.attn_k.weight	Block 5 Attention Key (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	IQ4_XS	4.2500
92	blk.5.attn_k_norm.weight	Block 5 Attn_K_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
93	blk.5.attn_norm.weight	Block 5 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
94	blk.5.attn_output.weight	Block 5 Attention Output (W)	( ~10M) 10485760	4096 x 2560 x 1 x 1	IQ4_XS	4.2500
95	blk.5.attn_q.weight	Block 5 Attention Query (W)	( ~10M) 10485760	2560 x 4096 x 1 x 1	IQ4_XS	4.2500
96	blk.5.attn_q_norm.weight	Block 5 Attn_Q_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
97	blk.5.attn_v.weight	Block 5 Attention Value (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	Q4_K	4.5000
98	blk.5.ffn_down.weight	Block 5 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
99	blk.5.ffn_gate.weight	Block 5 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
100	blk.5.ffn_norm.weight	Block 5 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
101	blk.5.ffn_up.weight	Block 5 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	Q5_K	5.5000
102	blk.5.inp_gate.weight	Block 5 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q5_K	5.5000
103	blk.5.layer_output_scale.weight	Block 5 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
104	blk.5.post_attention_norm.weight	Block 5 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
105	blk.5.post_ffw_norm.weight	Block 5 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
106	blk.5.post_norm.weight	Block 5 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
107	blk.5.proj.weight	Block 5 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q4_1	5.0000

Total elements in blk.5: (~106M) 106182145
Percentage of total elements: 1.41%
Bits per Weight (BPW) for blk.5: 4.5807 bits

Block 6 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
108	blk.6.attn_k.weight	Block 6 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	IQ4_XS	4.2500
109	blk.6.attn_k_norm.weight	Block 6 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
110	blk.6.attn_norm.weight	Block 6 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
111	blk.6.attn_output.weight	Block 6 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
112	blk.6.attn_q.weight	Block 6 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	IQ4_XS	4.2500
113	blk.6.attn_q_norm.weight	Block 6 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
114	blk.6.attn_v.weight	Block 6 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
115	blk.6.ffn_down.weight	Block 6 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
116	blk.6.ffn_gate.weight	Block 6 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
117	blk.6.ffn_norm.weight	Block 6 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
118	blk.6.ffn_up.weight	Block 6 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	Q5_K	5.5000
119	blk.6.inp_gate.weight	Block 6 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q5_K	5.5000
120	blk.6.layer_output_scale.weight	Block 6 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
121	blk.6.post_attention_norm.weight	Block 6 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
122	blk.6.post_ffw_norm.weight	Block 6 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
123	blk.6.post_norm.weight	Block 6 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
124	blk.6.proj.weight	Block 6 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_K	5.5000

Total elements in blk.6: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.6: 4.7117 bits

Block 7 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
125	blk.7.attn_k.weight	Block 7 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	IQ4_XS	4.2500
126	blk.7.attn_k_norm.weight	Block 7 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
127	blk.7.attn_norm.weight	Block 7 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
128	blk.7.attn_output.weight	Block 7 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
129	blk.7.attn_q.weight	Block 7 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	IQ4_XS	4.2500
130	blk.7.attn_q_norm.weight	Block 7 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
131	blk.7.attn_v.weight	Block 7 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	IQ4_XS	4.2500
132	blk.7.ffn_down.weight	Block 7 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
133	blk.7.ffn_gate.weight	Block 7 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
134	blk.7.ffn_norm.weight	Block 7 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
135	blk.7.ffn_up.weight	Block 7 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
136	blk.7.inp_gate.weight	Block 7 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q5_K	5.5000
137	blk.7.layer_output_scale.weight	Block 7 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
138	blk.7.post_attention_norm.weight	Block 7 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
139	blk.7.post_ffw_norm.weight	Block 7 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
140	blk.7.post_norm.weight	Block 7 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
141	blk.7.proj.weight	Block 7 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.7: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.7: 4.3455 bits

Block 8 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
142	blk.8.attn_k.weight	Block 8 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	IQ4_XS	4.2500
143	blk.8.attn_k_norm.weight	Block 8 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
144	blk.8.attn_norm.weight	Block 8 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
145	blk.8.attn_output.weight	Block 8 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
146	blk.8.attn_q.weight	Block 8 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	IQ4_XS	4.2500
147	blk.8.attn_q_norm.weight	Block 8 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
148	blk.8.attn_v.weight	Block 8 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
149	blk.8.ffn_down.weight	Block 8 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
150	blk.8.ffn_gate.weight	Block 8 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
151	blk.8.ffn_norm.weight	Block 8 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
152	blk.8.ffn_up.weight	Block 8 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
153	blk.8.inp_gate.weight	Block 8 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
154	blk.8.layer_output_scale.weight	Block 8 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
155	blk.8.post_attention_norm.weight	Block 8 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
156	blk.8.post_ffw_norm.weight	Block 8 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
157	blk.8.post_norm.weight	Block 8 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
158	blk.8.proj.weight	Block 8 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_K	5.5000

Total elements in blk.8: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.8: 4.3385 bits

Block 9 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
159	blk.9.attn_k.weight	Block 9 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	IQ4_XS	4.2500
160	blk.9.attn_k_norm.weight	Block 9 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
161	blk.9.attn_norm.weight	Block 9 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
162	blk.9.attn_output.weight	Block 9 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
163	blk.9.attn_q.weight	Block 9 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	IQ4_XS	4.2500
164	blk.9.attn_q_norm.weight	Block 9 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
165	blk.9.attn_v.weight	Block 9 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
166	blk.9.ffn_down.weight	Block 9 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
167	blk.9.ffn_gate.weight	Block 9 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
168	blk.9.ffn_norm.weight	Block 9 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
169	blk.9.ffn_up.weight	Block 9 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
170	blk.9.inp_gate.weight	Block 9 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
171	blk.9.layer_output_scale.weight	Block 9 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
172	blk.9.post_attention_norm.weight	Block 9 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
173	blk.9.post_ffw_norm.weight	Block 9 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
174	blk.9.post_norm.weight	Block 9 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
175	blk.9.proj.weight	Block 9 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q4_1	5.0000

Total elements in blk.9: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.9: 4.3349 bits

Block 10 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
176	blk.10.attn_k.weight	Block 10 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	IQ4_XS	4.2500
177	blk.10.attn_k_norm.weight	Block 10 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
178	blk.10.attn_norm.weight	Block 10 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
179	blk.10.attn_output.weight	Block 10 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
180	blk.10.attn_q.weight	Block 10 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	Q4_K	4.5000
181	blk.10.attn_q_norm.weight	Block 10 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
182	blk.10.attn_v.weight	Block 10 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
183	blk.10.ffn_down.weight	Block 10 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
184	blk.10.ffn_gate.weight	Block 10 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
185	blk.10.ffn_norm.weight	Block 10 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
186	blk.10.ffn_up.weight	Block 10 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
187	blk.10.inp_gate.weight	Block 10 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q5_K	5.5000
188	blk.10.layer_output_scale.weight	Block 10 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
189	blk.10.post_attention_norm.weight	Block 10 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
190	blk.10.post_ffw_norm.weight	Block 10 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
191	blk.10.post_norm.weight	Block 10 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
192	blk.10.proj.weight	Block 10 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.10: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.10: 4.3772 bits

Block 11 Tensor Group : ~106M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
193	blk.11.attn_k.weight	Block 11 Attention Key (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	IQ4_XS	4.2500
194	blk.11.attn_k_norm.weight	Block 11 Attn_K_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
195	blk.11.attn_norm.weight	Block 11 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
196	blk.11.attn_output.weight	Block 11 Attention Output (W)	( ~10M) 10485760	4096 x 2560 x 1 x 1	IQ4_XS	4.2500
197	blk.11.attn_q.weight	Block 11 Attention Query (W)	( ~10M) 10485760	2560 x 4096 x 1 x 1	IQ4_XS	4.2500
198	blk.11.attn_q_norm.weight	Block 11 Attn_Q_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
199	blk.11.attn_v.weight	Block 11 Attention Value (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	Q5_K	5.5000
200	blk.11.ffn_down.weight	Block 11 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
201	blk.11.ffn_gate.weight	Block 11 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
202	blk.11.ffn_norm.weight	Block 11 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
203	blk.11.ffn_up.weight	Block 11 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
204	blk.11.inp_gate.weight	Block 11 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
205	blk.11.layer_output_scale.weight	Block 11 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
206	blk.11.post_attention_norm.weight	Block 11 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
207	blk.11.post_ffw_norm.weight	Block 11 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
208	blk.11.post_norm.weight	Block 11 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
209	blk.11.proj.weight	Block 11 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q4_1	5.0000

Total elements in blk.11: (~106M) 106182145
Percentage of total elements: 1.41%
Bits per Weight (BPW) for blk.11: 4.2906 bits

Block 12 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
210	blk.12.attn_k.weight	Block 12 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
211	blk.12.attn_k_norm.weight	Block 12 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
212	blk.12.attn_norm.weight	Block 12 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
213	blk.12.attn_output.weight	Block 12 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
214	blk.12.attn_q.weight	Block 12 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	Q4_K	4.5000
215	blk.12.attn_q_norm.weight	Block 12 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
216	blk.12.attn_v.weight	Block 12 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
217	blk.12.ffn_down.weight	Block 12 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
218	blk.12.ffn_gate.weight	Block 12 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
219	blk.12.ffn_norm.weight	Block 12 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
220	blk.12.ffn_up.weight	Block 12 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
221	blk.12.inp_gate.weight	Block 12 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
222	blk.12.layer_output_scale.weight	Block 12 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
223	blk.12.post_attention_norm.weight	Block 12 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
224	blk.12.post_ffw_norm.weight	Block 12 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
225	blk.12.post_norm.weight	Block 12 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
226	blk.12.proj.weight	Block 12 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.12: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.12: 4.3737 bits

Block 13 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
227	blk.13.attn_k.weight	Block 13 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
228	blk.13.attn_k_norm.weight	Block 13 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
229	blk.13.attn_norm.weight	Block 13 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
230	blk.13.attn_output.weight	Block 13 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
231	blk.13.attn_q.weight	Block 13 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	Q4_K	4.5000
232	blk.13.attn_q_norm.weight	Block 13 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
233	blk.13.attn_v.weight	Block 13 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
234	blk.13.ffn_down.weight	Block 13 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
235	blk.13.ffn_gate.weight	Block 13 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
236	blk.13.ffn_norm.weight	Block 13 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
237	blk.13.ffn_up.weight	Block 13 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
238	blk.13.inp_gate.weight	Block 13 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q5_K	5.5000
239	blk.13.layer_output_scale.weight	Block 13 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
240	blk.13.post_attention_norm.weight	Block 13 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
241	blk.13.post_ffw_norm.weight	Block 13 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
242	blk.13.post_norm.weight	Block 13 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
243	blk.13.proj.weight	Block 13 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q8_0	8.5000

Total elements in blk.13: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.13: 4.3983 bits

Block 14 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
244	blk.14.attn_k.weight	Block 14 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
245	blk.14.attn_k_norm.weight	Block 14 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
246	blk.14.attn_norm.weight	Block 14 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
247	blk.14.attn_output.weight	Block 14 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
248	blk.14.attn_q.weight	Block 14 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	IQ4_XS	4.2500
249	blk.14.attn_q_norm.weight	Block 14 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
250	blk.14.attn_v.weight	Block 14 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
251	blk.14.ffn_down.weight	Block 14 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
252	blk.14.ffn_gate.weight	Block 14 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
253	blk.14.ffn_norm.weight	Block 14 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
254	blk.14.ffn_up.weight	Block 14 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
255	blk.14.inp_gate.weight	Block 14 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q5_K	5.5000
256	blk.14.layer_output_scale.weight	Block 14 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
257	blk.14.post_attention_norm.weight	Block 14 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
258	blk.14.post_ffw_norm.weight	Block 14 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
259	blk.14.post_norm.weight	Block 14 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
260	blk.14.proj.weight	Block 14 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.14: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.14: 4.3666 bits

Block 15 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
261	blk.15.attn_k.weight	Block 15 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
262	blk.15.attn_k_norm.weight	Block 15 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
263	blk.15.attn_norm.weight	Block 15 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
264	blk.15.attn_output.weight	Block 15 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
265	blk.15.attn_q.weight	Block 15 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	IQ4_XS	4.2500
266	blk.15.attn_q_norm.weight	Block 15 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
267	blk.15.attn_v.weight	Block 15 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
268	blk.15.ffn_down.weight	Block 15 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
269	blk.15.ffn_gate.weight	Block 15 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
270	blk.15.ffn_norm.weight	Block 15 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
271	blk.15.ffn_up.weight	Block 15 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
272	blk.15.inp_gate.weight	Block 15 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
273	blk.15.layer_output_scale.weight	Block 15 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
274	blk.15.post_attention_norm.weight	Block 15 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
275	blk.15.post_ffw_norm.weight	Block 15 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
276	blk.15.post_norm.weight	Block 15 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
277	blk.15.proj.weight	Block 15 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000

Total elements in blk.15: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.15: 4.4300 bits

Block 16 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
278	blk.16.attn_k.weight	Block 16 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
279	blk.16.attn_k_norm.weight	Block 16 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
280	blk.16.attn_norm.weight	Block 16 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
281	blk.16.attn_output.weight	Block 16 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
282	blk.16.attn_q.weight	Block 16 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	Q4_K	4.5000
283	blk.16.attn_q_norm.weight	Block 16 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
284	blk.16.attn_v.weight	Block 16 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
285	blk.16.ffn_down.weight	Block 16 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
286	blk.16.ffn_gate.weight	Block 16 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
287	blk.16.ffn_norm.weight	Block 16 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
288	blk.16.ffn_up.weight	Block 16 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
289	blk.16.inp_gate.weight	Block 16 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
290	blk.16.layer_output_scale.weight	Block 16 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
291	blk.16.post_attention_norm.weight	Block 16 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
292	blk.16.post_ffw_norm.weight	Block 16 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
293	blk.16.post_norm.weight	Block 16 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
294	blk.16.proj.weight	Block 16 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.16: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.16: 4.3737 bits

Block 17 Tensor Group : ~106M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
295	blk.17.attn_k.weight	Block 17 Attention Key (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	IQ4_XS	4.2500
296	blk.17.attn_k_norm.weight	Block 17 Attn_K_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
297	blk.17.attn_norm.weight	Block 17 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
298	blk.17.attn_output.weight	Block 17 Attention Output (W)	( ~10M) 10485760	4096 x 2560 x 1 x 1	IQ4_XS	4.2500
299	blk.17.attn_q.weight	Block 17 Attention Query (W)	( ~10M) 10485760	2560 x 4096 x 1 x 1	IQ4_XS	4.2500
300	blk.17.attn_q_norm.weight	Block 17 Attn_Q_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
301	blk.17.attn_v.weight	Block 17 Attention Value (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	Q5_K	5.5000
302	blk.17.ffn_down.weight	Block 17 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
303	blk.17.ffn_gate.weight	Block 17 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
304	blk.17.ffn_norm.weight	Block 17 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
305	blk.17.ffn_up.weight	Block 17 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
306	blk.17.inp_gate.weight	Block 17 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q5_K	5.5000
307	blk.17.layer_output_scale.weight	Block 17 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
308	blk.17.post_attention_norm.weight	Block 17 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
309	blk.17.post_ffw_norm.weight	Block 17 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
310	blk.17.post_norm.weight	Block 17 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
311	blk.17.proj.weight	Block 17 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.17: (~106M) 106182145
Percentage of total elements: 1.41%
Bits per Weight (BPW) for blk.17: 4.3030 bits

Block 18 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
312	blk.18.attn_k.weight	Block 18 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
313	blk.18.attn_k_norm.weight	Block 18 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
314	blk.18.attn_norm.weight	Block 18 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
315	blk.18.attn_output.weight	Block 18 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
316	blk.18.attn_q.weight	Block 18 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	Q4_K	4.5000
317	blk.18.attn_q_norm.weight	Block 18 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
318	blk.18.attn_v.weight	Block 18 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
319	blk.18.ffn_down.weight	Block 18 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
320	blk.18.ffn_gate.weight	Block 18 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
321	blk.18.ffn_norm.weight	Block 18 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
322	blk.18.ffn_up.weight	Block 18 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
323	blk.18.inp_gate.weight	Block 18 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q6_K	6.5625
324	blk.18.layer_output_scale.weight	Block 18 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
325	blk.18.post_attention_norm.weight	Block 18 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
326	blk.18.post_ffw_norm.weight	Block 18 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
327	blk.18.post_norm.weight	Block 18 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
328	blk.18.proj.weight	Block 18 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.18: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.18: 4.3882 bits

Block 19 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
329	blk.19.attn_k.weight	Block 19 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
330	blk.19.attn_k_norm.weight	Block 19 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
331	blk.19.attn_norm.weight	Block 19 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
332	blk.19.attn_output.weight	Block 19 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
333	blk.19.attn_q.weight	Block 19 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	Q4_K	4.5000
334	blk.19.attn_q_norm.weight	Block 19 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
335	blk.19.attn_v.weight	Block 19 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
336	blk.19.ffn_down.weight	Block 19 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
337	blk.19.ffn_gate.weight	Block 19 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
338	blk.19.ffn_norm.weight	Block 19 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
339	blk.19.ffn_up.weight	Block 19 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
340	blk.19.inp_gate.weight	Block 19 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q5_K	5.5000
341	blk.19.layer_output_scale.weight	Block 19 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
342	blk.19.post_attention_norm.weight	Block 19 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
343	blk.19.post_ffw_norm.weight	Block 19 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
344	blk.19.post_norm.weight	Block 19 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
345	blk.19.proj.weight	Block 19 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q6_K	6.5625

Total elements in blk.19: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.19: 4.3847 bits

Block 20 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
346	blk.20.attn_k.weight	Block 20 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
347	blk.20.attn_k_norm.weight	Block 20 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
348	blk.20.attn_norm.weight	Block 20 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
349	blk.20.attn_output.weight	Block 20 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
350	blk.20.attn_q.weight	Block 20 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	Q4_K	4.5000
351	blk.20.attn_q_norm.weight	Block 20 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
352	blk.20.attn_v.weight	Block 20 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
353	blk.20.ffn_down.weight	Block 20 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
354	blk.20.ffn_gate.weight	Block 20 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
355	blk.20.ffn_norm.weight	Block 20 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
356	blk.20.ffn_up.weight	Block 20 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
357	blk.20.inp_gate.weight	Block 20 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q5_K	5.5000
358	blk.20.layer_output_scale.weight	Block 20 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
359	blk.20.post_attention_norm.weight	Block 20 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
360	blk.20.post_ffw_norm.weight	Block 20 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
361	blk.20.post_norm.weight	Block 20 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
362	blk.20.proj.weight	Block 20 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.20: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.20: 4.3807 bits

Block 21 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
363	blk.21.attn_k.weight	Block 21 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
364	blk.21.attn_k_norm.weight	Block 21 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
365	blk.21.attn_norm.weight	Block 21 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
366	blk.21.attn_output.weight	Block 21 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
367	blk.21.attn_q.weight	Block 21 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	IQ4_XS	4.2500
368	blk.21.attn_q_norm.weight	Block 21 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
369	blk.21.attn_v.weight	Block 21 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
370	blk.21.ffn_down.weight	Block 21 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
371	blk.21.ffn_gate.weight	Block 21 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
372	blk.21.ffn_norm.weight	Block 21 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
373	blk.21.ffn_up.weight	Block 21 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	Q5_K	5.5000
374	blk.21.inp_gate.weight	Block 21 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
375	blk.21.layer_output_scale.weight	Block 21 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
376	blk.21.post_attention_norm.weight	Block 21 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
377	blk.21.post_ffw_norm.weight	Block 21 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
378	blk.21.post_norm.weight	Block 21 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
379	blk.21.proj.weight	Block 21 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q8_0	8.5000

Total elements in blk.21: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.21: 4.7293 bits

Block 22 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
380	blk.22.attn_k.weight	Block 22 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
381	blk.22.attn_k_norm.weight	Block 22 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
382	blk.22.attn_norm.weight	Block 22 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
383	blk.22.attn_output.weight	Block 22 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
384	blk.22.attn_q.weight	Block 22 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	Q5_K	5.5000
385	blk.22.attn_q_norm.weight	Block 22 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
386	blk.22.attn_v.weight	Block 22 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
387	blk.22.ffn_down.weight	Block 22 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
388	blk.22.ffn_gate.weight	Block 22 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
389	blk.22.ffn_norm.weight	Block 22 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
390	blk.22.ffn_up.weight	Block 22 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
391	blk.22.inp_gate.weight	Block 22 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
392	blk.22.layer_output_scale.weight	Block 22 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
393	blk.22.post_attention_norm.weight	Block 22 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
394	blk.22.post_ffw_norm.weight	Block 22 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
395	blk.22.post_norm.weight	Block 22 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
396	blk.22.proj.weight	Block 22 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_K	5.5000

Total elements in blk.22: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.22: 4.4406 bits

Block 23 Tensor Group : ~106M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
397	blk.23.attn_k.weight	Block 23 Attention Key (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	Q5_K	5.5000
398	blk.23.attn_k_norm.weight	Block 23 Attn_K_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
399	blk.23.attn_norm.weight	Block 23 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
400	blk.23.attn_output.weight	Block 23 Attention Output (W)	( ~10M) 10485760	4096 x 2560 x 1 x 1	IQ4_XS	4.2500
401	blk.23.attn_q.weight	Block 23 Attention Query (W)	( ~10M) 10485760	2560 x 4096 x 1 x 1	Q5_K	5.5000
402	blk.23.attn_q_norm.weight	Block 23 Attn_Q_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
403	blk.23.attn_v.weight	Block 23 Attention Value (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	Q5_K	5.5000
404	blk.23.ffn_down.weight	Block 23 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
405	blk.23.ffn_gate.weight	Block 23 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
406	blk.23.ffn_norm.weight	Block 23 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
407	blk.23.ffn_up.weight	Block 23 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	Q5_K	5.5000
408	blk.23.inp_gate.weight	Block 23 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q5_K	5.5000
409	blk.23.layer_output_scale.weight	Block 23 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
410	blk.23.post_attention_norm.weight	Block 23 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
411	blk.23.post_ffw_norm.weight	Block 23 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
412	blk.23.post_norm.weight	Block 23 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
413	blk.23.proj.weight	Block 23 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.23: (~106M) 106182145
Percentage of total elements: 1.41%
Bits per Weight (BPW) for blk.23: 4.7659 bits

Block 24 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
414	blk.24.attn_k.weight	Block 24 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
415	blk.24.attn_k_norm.weight	Block 24 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
416	blk.24.attn_norm.weight	Block 24 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
417	blk.24.attn_output.weight	Block 24 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
418	blk.24.attn_q.weight	Block 24 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	Q4_K	4.5000
419	blk.24.attn_q_norm.weight	Block 24 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
420	blk.24.attn_v.weight	Block 24 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
421	blk.24.ffn_down.weight	Block 24 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
422	blk.24.ffn_gate.weight	Block 24 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
423	blk.24.ffn_norm.weight	Block 24 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
424	blk.24.ffn_up.weight	Block 24 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	Q5_K	5.5000
425	blk.24.inp_gate.weight	Block 24 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q5_K	5.5000
426	blk.24.layer_output_scale.weight	Block 24 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
427	blk.24.post_attention_norm.weight	Block 24 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
428	blk.24.post_ffw_norm.weight	Block 24 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
429	blk.24.post_norm.weight	Block 24 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
430	blk.24.proj.weight	Block 24 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_K	5.5000

Total elements in blk.24: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.24: 4.7293 bits

Block 25 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
431	blk.25.attn_k.weight	Block 25 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
432	blk.25.attn_k_norm.weight	Block 25 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
433	blk.25.attn_norm.weight	Block 25 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
434	blk.25.attn_output.weight	Block 25 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
435	blk.25.attn_q.weight	Block 25 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	Q4_K	4.5000
436	blk.25.attn_q_norm.weight	Block 25 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
437	blk.25.attn_v.weight	Block 25 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
438	blk.25.ffn_down.weight	Block 25 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
439	blk.25.ffn_gate.weight	Block 25 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
440	blk.25.ffn_norm.weight	Block 25 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
441	blk.25.ffn_up.weight	Block 25 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
442	blk.25.inp_gate.weight	Block 25 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
443	blk.25.layer_output_scale.weight	Block 25 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
444	blk.25.post_attention_norm.weight	Block 25 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
445	blk.25.post_ffw_norm.weight	Block 25 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
446	blk.25.post_norm.weight	Block 25 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
447	blk.25.proj.weight	Block 25 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.25: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.25: 4.3737 bits

Block 26 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
448	blk.26.attn_k.weight	Block 26 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
449	blk.26.attn_k_norm.weight	Block 26 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
450	blk.26.attn_norm.weight	Block 26 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
451	blk.26.attn_output.weight	Block 26 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
452	blk.26.attn_q.weight	Block 26 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	Q4_K	4.5000
453	blk.26.attn_q_norm.weight	Block 26 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
454	blk.26.attn_v.weight	Block 26 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
455	blk.26.ffn_down.weight	Block 26 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
456	blk.26.ffn_gate.weight	Block 26 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
457	blk.26.ffn_norm.weight	Block 26 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
458	blk.26.ffn_up.weight	Block 26 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	Q4_K	4.5000
459	blk.26.inp_gate.weight	Block 26 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
460	blk.26.layer_output_scale.weight	Block 26 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
461	blk.26.post_attention_norm.weight	Block 26 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
462	blk.26.post_ffw_norm.weight	Block 26 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
463	blk.26.post_norm.weight	Block 26 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
464	blk.26.proj.weight	Block 26 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.26: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.26: 4.4441 bits

Block 27 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
465	blk.27.attn_k.weight	Block 27 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
466	blk.27.attn_k_norm.weight	Block 27 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
467	blk.27.attn_norm.weight	Block 27 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
468	blk.27.attn_output.weight	Block 27 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
469	blk.27.attn_q.weight	Block 27 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	Q4_K	4.5000
470	blk.27.attn_q_norm.weight	Block 27 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
471	blk.27.attn_v.weight	Block 27 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
472	blk.27.ffn_down.weight	Block 27 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
473	blk.27.ffn_gate.weight	Block 27 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
474	blk.27.ffn_norm.weight	Block 27 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
475	blk.27.ffn_up.weight	Block 27 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	Q4_K	4.5000
476	blk.27.inp_gate.weight	Block 27 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
477	blk.27.layer_output_scale.weight	Block 27 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
478	blk.27.post_attention_norm.weight	Block 27 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
479	blk.27.post_ffw_norm.weight	Block 27 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
480	blk.27.post_norm.weight	Block 27 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
481	blk.27.proj.weight	Block 27 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.27: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.27: 4.4441 bits

Block 28 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
482	blk.28.attn_k.weight	Block 28 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
483	blk.28.attn_k_norm.weight	Block 28 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
484	blk.28.attn_norm.weight	Block 28 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
485	blk.28.attn_output.weight	Block 28 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
486	blk.28.attn_q.weight	Block 28 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	Q4_K	4.5000
487	blk.28.attn_q_norm.weight	Block 28 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
488	blk.28.attn_v.weight	Block 28 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
489	blk.28.ffn_down.weight	Block 28 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
490	blk.28.ffn_gate.weight	Block 28 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
491	blk.28.ffn_norm.weight	Block 28 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
492	blk.28.ffn_up.weight	Block 28 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
493	blk.28.inp_gate.weight	Block 28 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
494	blk.28.layer_output_scale.weight	Block 28 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
495	blk.28.post_attention_norm.weight	Block 28 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
496	blk.28.post_ffw_norm.weight	Block 28 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
497	blk.28.post_norm.weight	Block 28 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
498	blk.28.proj.weight	Block 28 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.28: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.28: 4.3737 bits

Block 29 Tensor Group : ~106M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
499	blk.29.attn_k.weight	Block 29 Attention Key (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	Q5_K	5.5000
500	blk.29.attn_k_norm.weight	Block 29 Attn_K_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
501	blk.29.attn_norm.weight	Block 29 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
502	blk.29.attn_output.weight	Block 29 Attention Output (W)	( ~10M) 10485760	4096 x 2560 x 1 x 1	IQ4_XS	4.2500
503	blk.29.attn_q.weight	Block 29 Attention Query (W)	( ~10M) 10485760	2560 x 4096 x 1 x 1	IQ4_XS	4.2500
504	blk.29.attn_q_norm.weight	Block 29 Attn_Q_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
505	blk.29.attn_v.weight	Block 29 Attention Value (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	Q4_K	4.5000
506	blk.29.ffn_down.weight	Block 29 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
507	blk.29.ffn_gate.weight	Block 29 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
508	blk.29.ffn_norm.weight	Block 29 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
509	blk.29.ffn_up.weight	Block 29 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
510	blk.29.inp_gate.weight	Block 29 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
511	blk.29.layer_output_scale.weight	Block 29 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
512	blk.29.post_attention_norm.weight	Block 29 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
513	blk.29.post_ffw_norm.weight	Block 29 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
514	blk.29.post_norm.weight	Block 29 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
515	blk.29.proj.weight	Block 29 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.29: (~106M) 106182145
Percentage of total elements: 1.41%
Bits per Weight (BPW) for blk.29: 4.3030 bits

Block 30 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
516	blk.30.attn_k.weight	Block 30 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
517	blk.30.attn_k_norm.weight	Block 30 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
518	blk.30.attn_norm.weight	Block 30 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
519	blk.30.attn_output.weight	Block 30 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
520	blk.30.attn_q.weight	Block 30 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	Q4_K	4.5000
521	blk.30.attn_q_norm.weight	Block 30 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
522	blk.30.attn_v.weight	Block 30 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
523	blk.30.ffn_down.weight	Block 30 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
524	blk.30.ffn_gate.weight	Block 30 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
525	blk.30.ffn_norm.weight	Block 30 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
526	blk.30.ffn_up.weight	Block 30 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
527	blk.30.inp_gate.weight	Block 30 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
528	blk.30.layer_output_scale.weight	Block 30 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
529	blk.30.post_attention_norm.weight	Block 30 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
530	blk.30.post_ffw_norm.weight	Block 30 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
531	blk.30.post_norm.weight	Block 30 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
532	blk.30.proj.weight	Block 30 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_K	5.5000

Total elements in blk.30: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.30: 4.3701 bits

Block 31 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
533	blk.31.attn_k.weight	Block 31 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
534	blk.31.attn_k_norm.weight	Block 31 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
535	blk.31.attn_norm.weight	Block 31 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
536	blk.31.attn_output.weight	Block 31 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
537	blk.31.attn_q.weight	Block 31 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	Q4_K	4.5000
538	blk.31.attn_q_norm.weight	Block 31 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
539	blk.31.attn_v.weight	Block 31 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
540	blk.31.ffn_down.weight	Block 31 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
541	blk.31.ffn_gate.weight	Block 31 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
542	blk.31.ffn_norm.weight	Block 31 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
543	blk.31.ffn_up.weight	Block 31 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
544	blk.31.inp_gate.weight	Block 31 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
545	blk.31.layer_output_scale.weight	Block 31 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
546	blk.31.post_attention_norm.weight	Block 31 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
547	blk.31.post_ffw_norm.weight	Block 31 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
548	blk.31.post_norm.weight	Block 31 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
549	blk.31.proj.weight	Block 31 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.31: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.31: 4.3737 bits

Block 32 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
550	blk.32.attn_k.weight	Block 32 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
551	blk.32.attn_k_norm.weight	Block 32 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
552	blk.32.attn_norm.weight	Block 32 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
553	blk.32.attn_output.weight	Block 32 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
554	blk.32.attn_q.weight	Block 32 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	IQ4_XS	4.2500
555	blk.32.attn_q_norm.weight	Block 32 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
556	blk.32.attn_v.weight	Block 32 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
557	blk.32.ffn_down.weight	Block 32 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
558	blk.32.ffn_gate.weight	Block 32 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
559	blk.32.ffn_norm.weight	Block 32 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
560	blk.32.ffn_up.weight	Block 32 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
561	blk.32.inp_gate.weight	Block 32 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
562	blk.32.layer_output_scale.weight	Block 32 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
563	blk.32.post_attention_norm.weight	Block 32 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
564	blk.32.post_ffw_norm.weight	Block 32 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
565	blk.32.post_norm.weight	Block 32 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
566	blk.32.proj.weight	Block 32 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.32: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.32: 4.3596 bits

Block 33 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
567	blk.33.attn_k.weight	Block 33 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
568	blk.33.attn_k_norm.weight	Block 33 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
569	blk.33.attn_norm.weight	Block 33 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
570	blk.33.attn_output.weight	Block 33 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
571	blk.33.attn_q.weight	Block 33 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	IQ4_XS	4.2500
572	blk.33.attn_q_norm.weight	Block 33 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
573	blk.33.attn_v.weight	Block 33 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
574	blk.33.ffn_down.weight	Block 33 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
575	blk.33.ffn_gate.weight	Block 33 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
576	blk.33.ffn_norm.weight	Block 33 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
577	blk.33.ffn_up.weight	Block 33 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
578	blk.33.inp_gate.weight	Block 33 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
579	blk.33.layer_output_scale.weight	Block 33 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
580	blk.33.post_attention_norm.weight	Block 33 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
581	blk.33.post_ffw_norm.weight	Block 33 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
582	blk.33.post_norm.weight	Block 33 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
583	blk.33.proj.weight	Block 33 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q4_1	5.0000

Total elements in blk.33: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.33: 4.3525 bits

Block 34 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
584	blk.34.attn_k.weight	Block 34 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
585	blk.34.attn_k_norm.weight	Block 34 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
586	blk.34.attn_norm.weight	Block 34 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
587	blk.34.attn_output.weight	Block 34 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
588	blk.34.attn_q.weight	Block 34 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	Q4_K	4.5000
589	blk.34.attn_q_norm.weight	Block 34 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
590	blk.34.attn_v.weight	Block 34 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
591	blk.34.ffn_down.weight	Block 34 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
592	blk.34.ffn_gate.weight	Block 34 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
593	blk.34.ffn_norm.weight	Block 34 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
594	blk.34.ffn_up.weight	Block 34 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
595	blk.34.inp_gate.weight	Block 34 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
596	blk.34.layer_output_scale.weight	Block 34 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
597	blk.34.post_attention_norm.weight	Block 34 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
598	blk.34.post_ffw_norm.weight	Block 34 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
599	blk.34.post_norm.weight	Block 34 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
600	blk.34.proj.weight	Block 34 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_K	5.5000

Total elements in blk.34: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.34: 4.3701 bits

Block 35 Tensor Group : ~106M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
601	blk.35.attn_k.weight	Block 35 Attention Key (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	Q5_K	5.5000
602	blk.35.attn_k_norm.weight	Block 35 Attn_K_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
603	blk.35.attn_norm.weight	Block 35 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
604	blk.35.attn_output.weight	Block 35 Attention Output (W)	( ~10M) 10485760	4096 x 2560 x 1 x 1	IQ4_XS	4.2500
605	blk.35.attn_q.weight	Block 35 Attention Query (W)	( ~10M) 10485760	2560 x 4096 x 1 x 1	Q4_K	4.5000
606	blk.35.attn_q_norm.weight	Block 35 Attn_Q_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
607	blk.35.attn_v.weight	Block 35 Attention Value (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	Q4_K	4.5000
608	blk.35.ffn_down.weight	Block 35 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
609	blk.35.ffn_gate.weight	Block 35 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
610	blk.35.ffn_norm.weight	Block 35 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
611	blk.35.ffn_up.weight	Block 35 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
612	blk.35.inp_gate.weight	Block 35 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
613	blk.35.layer_output_scale.weight	Block 35 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
614	blk.35.post_attention_norm.weight	Block 35 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
615	blk.35.post_ffw_norm.weight	Block 35 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
616	blk.35.post_norm.weight	Block 35 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
617	blk.35.proj.weight	Block 35 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q4_1	5.0000

Total elements in blk.35: (~106M) 106182145
Percentage of total elements: 1.41%
Bits per Weight (BPW) for blk.35: 4.3215 bits

Block 36 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
618	blk.36.attn_k.weight	Block 36 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
619	blk.36.attn_k_norm.weight	Block 36 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
620	blk.36.attn_norm.weight	Block 36 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
621	blk.36.attn_output.weight	Block 36 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
622	blk.36.attn_q.weight	Block 36 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	IQ4_XS	4.2500
623	blk.36.attn_q_norm.weight	Block 36 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
624	blk.36.attn_v.weight	Block 36 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
625	blk.36.ffn_down.weight	Block 36 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
626	blk.36.ffn_gate.weight	Block 36 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
627	blk.36.ffn_norm.weight	Block 36 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
628	blk.36.ffn_up.weight	Block 36 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
629	blk.36.inp_gate.weight	Block 36 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
630	blk.36.layer_output_scale.weight	Block 36 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
631	blk.36.post_attention_norm.weight	Block 36 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
632	blk.36.post_ffw_norm.weight	Block 36 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
633	blk.36.post_norm.weight	Block 36 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
634	blk.36.proj.weight	Block 36 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_K	5.5000

Total elements in blk.36: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.36: 4.3561 bits

Block 37 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
635	blk.37.attn_k.weight	Block 37 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
636	blk.37.attn_k_norm.weight	Block 37 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
637	blk.37.attn_norm.weight	Block 37 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
638	blk.37.attn_output.weight	Block 37 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
639	blk.37.attn_q.weight	Block 37 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	IQ4_XS	4.2500
640	blk.37.attn_q_norm.weight	Block 37 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
641	blk.37.attn_v.weight	Block 37 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
642	blk.37.ffn_down.weight	Block 37 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
643	blk.37.ffn_gate.weight	Block 37 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
644	blk.37.ffn_norm.weight	Block 37 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
645	blk.37.ffn_up.weight	Block 37 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
646	blk.37.inp_gate.weight	Block 37 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
647	blk.37.layer_output_scale.weight	Block 37 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
648	blk.37.post_attention_norm.weight	Block 37 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
649	blk.37.post_ffw_norm.weight	Block 37 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
650	blk.37.post_norm.weight	Block 37 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
651	blk.37.proj.weight	Block 37 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.37: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.37: 4.3596 bits

Block 38 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
652	blk.38.attn_k.weight	Block 38 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
653	blk.38.attn_k_norm.weight	Block 38 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
654	blk.38.attn_norm.weight	Block 38 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
655	blk.38.attn_output.weight	Block 38 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
656	blk.38.attn_q.weight	Block 38 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	Q4_K	4.5000
657	blk.38.attn_q_norm.weight	Block 38 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
658	blk.38.attn_v.weight	Block 38 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
659	blk.38.ffn_down.weight	Block 38 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
660	blk.38.ffn_gate.weight	Block 38 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
661	blk.38.ffn_norm.weight	Block 38 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
662	blk.38.ffn_up.weight	Block 38 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
663	blk.38.inp_gate.weight	Block 38 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
664	blk.38.layer_output_scale.weight	Block 38 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
665	blk.38.post_attention_norm.weight	Block 38 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
666	blk.38.post_ffw_norm.weight	Block 38 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
667	blk.38.post_norm.weight	Block 38 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
668	blk.38.proj.weight	Block 38 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.38: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.38: 4.3737 bits

Block 39 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
669	blk.39.attn_k.weight	Block 39 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
670	blk.39.attn_k_norm.weight	Block 39 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
671	blk.39.attn_norm.weight	Block 39 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
672	blk.39.attn_output.weight	Block 39 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
673	blk.39.attn_q.weight	Block 39 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	Q4_K	4.5000
674	blk.39.attn_q_norm.weight	Block 39 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
675	blk.39.attn_v.weight	Block 39 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
676	blk.39.ffn_down.weight	Block 39 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
677	blk.39.ffn_gate.weight	Block 39 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
678	blk.39.ffn_norm.weight	Block 39 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
679	blk.39.ffn_up.weight	Block 39 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	IQ4_XS	4.2500
680	blk.39.inp_gate.weight	Block 39 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
681	blk.39.layer_output_scale.weight	Block 39 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
682	blk.39.post_attention_norm.weight	Block 39 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
683	blk.39.post_ffw_norm.weight	Block 39 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
684	blk.39.post_norm.weight	Block 39 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
685	blk.39.proj.weight	Block 39 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.39: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.39: 4.3737 bits

Block 40 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
686	blk.40.attn_k.weight	Block 40 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q5_K	5.5000
687	blk.40.attn_k_norm.weight	Block 40 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
688	blk.40.attn_norm.weight	Block 40 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
689	blk.40.attn_output.weight	Block 40 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	Q5_K	5.5000
690	blk.40.attn_q.weight	Block 40 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	Q4_K	4.5000
691	blk.40.attn_q_norm.weight	Block 40 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
692	blk.40.attn_v.weight	Block 40 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	Q4_K	4.5000
693	blk.40.ffn_down.weight	Block 40 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
694	blk.40.ffn_gate.weight	Block 40 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	Q4_K	4.5000
695	blk.40.ffn_norm.weight	Block 40 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
696	blk.40.ffn_up.weight	Block 40 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	Q5_K	5.5000
697	blk.40.inp_gate.weight	Block 40 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q4_K	4.5000
698	blk.40.layer_output_scale.weight	Block 40 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
699	blk.40.post_attention_norm.weight	Block 40 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
700	blk.40.post_ffw_norm.weight	Block 40 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
701	blk.40.post_norm.weight	Block 40 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
702	blk.40.proj.weight	Block 40 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.40: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.40: 4.7961 bits

Block 41 Tensor Group : ~106M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
703	blk.41.attn_k.weight	Block 41 Attention Key (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	Q5_K	5.5000
704	blk.41.attn_k_norm.weight	Block 41 Attn_K_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
705	blk.41.attn_norm.weight	Block 41 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
706	blk.41.attn_output.weight	Block 41 Attention Output (W)	( ~10M) 10485760	4096 x 2560 x 1 x 1	Q4_K	4.5000
707	blk.41.attn_q.weight	Block 41 Attention Query (W)	( ~10M) 10485760	2560 x 4096 x 1 x 1	Q4_K	4.5000
708	blk.41.attn_q_norm.weight	Block 41 Attn_Q_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
709	blk.41.attn_v.weight	Block 41 Attention Value (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	Q4_K	4.5000
710	blk.41.ffn_down.weight	Block 41 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	IQ4_XS	4.2500
711	blk.41.ffn_gate.weight	Block 41 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	Q4_K	4.5000
712	blk.41.ffn_norm.weight	Block 41 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
713	blk.41.ffn_up.weight	Block 41 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	Q4_K	4.5000
714	blk.41.inp_gate.weight	Block 41 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	Q3_K	3.4375
715	blk.41.layer_output_scale.weight	Block 41 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
716	blk.41.post_attention_norm.weight	Block 41 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
717	blk.41.post_ffw_norm.weight	Block 41 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
718	blk.41.post_norm.weight	Block 41 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
719	blk.41.proj.weight	Block 41 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	Q5_1	6.0000

Total elements in blk.41: (~106M) 106182145
Percentage of total elements: 1.41%
Bits per Weight (BPW) for blk.41: 4.4692 bits

Total BPW for gemma-4-E4B-it-Q4_K.gguf: 4.5000 bits