Instructions to use eaddario/gemma-4-E4B-it-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use eaddario/gemma-4-E4B-it-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf eaddario/gemma-4-E4B-it-GGUF:F16
# Run inference directly in the terminal:
llama cli -hf eaddario/gemma-4-E4B-it-GGUF:F16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf eaddario/gemma-4-E4B-it-GGUF:F16
# Run inference directly in the terminal:
llama cli -hf eaddario/gemma-4-E4B-it-GGUF:F16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf eaddario/gemma-4-E4B-it-GGUF:F16
# Run inference directly in the terminal:
./llama-cli -hf eaddario/gemma-4-E4B-it-GGUF:F16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf eaddario/gemma-4-E4B-it-GGUF:F16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf eaddario/gemma-4-E4B-it-GGUF:F16

Use Docker

docker model run hf.co/eaddario/gemma-4-E4B-it-GGUF:F16

LM Studio
Jan
Ollama
How to use eaddario/gemma-4-E4B-it-GGUF with Ollama:
```
ollama run hf.co/eaddario/gemma-4-E4B-it-GGUF:F16
```
Unsloth Desktop

How to use eaddario/gemma-4-E4B-it-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf eaddario/gemma-4-E4B-it-GGUF:F16

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "eaddario/gemma-4-E4B-it-GGUF:F16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use eaddario/gemma-4-E4B-it-GGUF with Docker Model Runner:
```
docker model run hf.co/eaddario/gemma-4-E4B-it-GGUF:F16
```

Lemonade

How to use eaddario/gemma-4-E4B-it-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull eaddario/gemma-4-E4B-it-GGUF:F16

Run and chat with the model

lemonade run user.gemma-4-E4B-it-GGUF-F16

List all available models

lemonade list

Hermes Agent

How to use eaddario/gemma-4-E4B-it-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf eaddario/gemma-4-E4B-it-GGUF:F16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default eaddario/gemma-4-E4B-it-GGUF:F16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use eaddario/gemma-4-E4B-it-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf eaddario/gemma-4-E4B-it-GGUF:F16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "eaddario/gemma-4-E4B-it-GGUF:F16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

gemma-4-E4B-it-GGUF / scores /gemma-4-E4B-it-F16.md

eaddario

Add GGUF internal file structure

8399b2d verified 4 months ago

preview code

Raw

History Blame Contribute Delete

204 kB

gemma-4-E4B-it-F16.gguf - GGUF Internal File Dump

Endian: LITTLE endian

Key Value Metadata Store

There are 48 key-value pairs in this file

POS	TYPE	Count	Key	Value
1	UINT32	1	GGUF.version	3
2	UINT64	1	GGUF.tensor_count	720
3	UINT64	1	GGUF.kv_count	45
4	STRING	1	general.architecture	`gemma4`
5	STRING	1	general.type	`model`
6	INT32	1	general.sampling.top_k	64
7	FLOAT32	1	general.sampling.top_p	0.95
8	FLOAT32	1	general.sampling.temp	1.0
9	STRING	1	general.name	`Gemma 4 E4B It`
10	STRING	1	general.size_label	`7.5B`
11	STRING	1	general.license	`apache-2.0`
12	STRING	1	general.license.link	`https://ai.google.dev/gemma/docs/gemma_4_license`
13	[STRING]	1	general.tags	[ `any-to-any` ]
14	UINT32	1	gemma4.block_count	42
15	UINT32	1	gemma4.context_length	131072
16	UINT32	1	gemma4.embedding_length	2560
17	UINT32	1	gemma4.feed_forward_length	10240
18	UINT32	1	gemma4.attention.head_count	8
19	UINT32	1	gemma4.attention.head_count_kv	2
20	FLOAT32	1	gemma4.rope.freq_base	1e+06
21	FLOAT32	1	gemma4.rope.freq_base_swa	10000.0
22	FLOAT32	1	gemma4.attention.layer_norm_rms_epsilon	1e-06
23	UINT32	1	gemma4.attention.key_length	512
24	UINT32	1	gemma4.attention.value_length	512
25	UINT32	1	general.file_type	1
26	FLOAT32	1	gemma4.final_logit_softcapping	30.0
27	UINT32	1	gemma4.attention.sliding_window	512
28	UINT32	1	gemma4.attention.shared_kv_layers	18
29	UINT32	1	gemma4.embedding_length_per_layer_input	256
30	[BOOL]	42	gemma4.attention.sliding_window_pattern	[ True, True, True, True, True, False, True, ... ]
31	UINT32	1	gemma4.attention.key_length_swa	256
32	UINT32	1	gemma4.attention.value_length_swa	256
33	UINT32	1	gemma4.rope.dimension_count	512
34	UINT32	1	gemma4.rope.dimension_count_swa	256
35	UINT32	1	general.quantization_version	2
36	STRING	1	tokenizer.ggml.model	`gemma4`
37	[STRING]	262144	tokenizer.ggml.tokens	[ `<pad>`, `<eos>`, `<bos>`, `<unk>`, `<mask>`, ... ]
38	[FLOAT32]	262144	tokenizer.ggml.scores	[ -1000.0, -1000.0, -1000.0, -1000.0, -1000.0, -1000.0, -1000.0, ... ]
39	[INT32]	262144	tokenizer.ggml.token_type	[ 3, 3, 3, 3, 3, 1, 1, ... ]
40	[STRING]	514906	tokenizer.ggml.merges	[ `...`, `▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁`...`▁▁▁▁▁▁▁▁▁▁▁▁▁ ▁`, `...`, `...`, `▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁`...`▁▁▁▁▁▁▁▁▁▁▁▁ ▁▁`, ... ]
41	UINT32	1	tokenizer.ggml.bos_token_id	2
42	UINT32	1	tokenizer.ggml.eos_token_id	1
43	UINT32	1	tokenizer.ggml.unknown_token_id	3
44	UINT32	1	tokenizer.ggml.padding_token_id	0
45	UINT32	1	tokenizer.ggml.mask_token_id	4
46	STRING	1	tokenizer.chat_template	`{%- macro format_parameters(pr`... `{%- endif -%} {%- endif -%}`
47	BOOL	1	tokenizer.ggml.add_space_prefix	False
48	BOOL	1	tokenizer.ggml.add_bos_token	True

Tensors Overview ~8B Elements

Total number of elements in all tensors: 7518069290 Elements

gemma-4-E4B-it-F16.gguf - GGUF Internal File Dump
- Key Value Metadata Store
- Tensors Overview ~8B Elements

Tensor Data Offset

This table contains the offset and data segment relative to start of file

T_ID	Tensor Layer Name	Data Offset (B)	Data Size (B)
0	rope_freqs.weight	0xf172c0	0x400
1	token_embd.weight	0xf176c0	0x50000000
2	per_layer_token_embd.weight	0x50f176c0	0x150000000
3	blk.0.attn_norm.weight	0x1a0f176c0	0x2800
4	blk.0.layer_output_scale.weight	0x1a0f19ec0	0x4
5	blk.0.ffn_down.weight	0x1a0f19ee0	0x3200000
6	blk.0.ffn_gate.weight	0x1a4119ee0	0x3200000
7	blk.0.ffn_up.weight	0x1a7319ee0	0x3200000
8	blk.0.inp_gate.weight	0x1aa519ee0	0x140000
9	blk.0.proj.weight	0x1aa659ee0	0x140000
10	blk.0.post_attention_norm.weight	0x1aa799ee0	0x2800
11	blk.0.post_ffw_norm.weight	0x1aa79c6e0	0x2800
12	blk.0.post_norm.weight	0x1aa79eee0	0x2800
13	blk.0.ffn_norm.weight	0x1aa7a16e0	0x2800
14	blk.0.attn_k_norm.weight	0x1aa7a3ee0	0x400
15	blk.0.attn_k.weight	0x1aa7a42e0	0x280000
16	blk.0.attn_output.weight	0x1aaa242e0	0xa00000
17	blk.0.attn_q_norm.weight	0x1ab4242e0	0x400
18	blk.0.attn_q.weight	0x1ab4246e0	0xa00000
19	blk.0.attn_v.weight	0x1abe246e0	0x280000
20	blk.1.attn_norm.weight	0x1ac0a46e0	0x2800
21	blk.1.layer_output_scale.weight	0x1ac0a6ee0	0x4
22	blk.1.ffn_down.weight	0x1ac0a6f00	0x3200000
23	blk.1.ffn_gate.weight	0x1af2a6f00	0x3200000
24	blk.1.ffn_up.weight	0x1b24a6f00	0x3200000
25	blk.1.inp_gate.weight	0x1b56a6f00	0x140000
26	blk.1.proj.weight	0x1b57e6f00	0x140000
27	blk.1.post_attention_norm.weight	0x1b5926f00	0x2800
28	blk.1.post_ffw_norm.weight	0x1b5929700	0x2800
29	blk.1.post_norm.weight	0x1b592bf00	0x2800
30	blk.1.ffn_norm.weight	0x1b592e700	0x2800
31	blk.1.attn_k_norm.weight	0x1b5930f00	0x400
32	blk.1.attn_k.weight	0x1b5931300	0x280000
33	blk.1.attn_output.weight	0x1b5bb1300	0xa00000
34	blk.1.attn_q_norm.weight	0x1b65b1300	0x400
35	blk.1.attn_q.weight	0x1b65b1700	0xa00000
36	blk.1.attn_v.weight	0x1b6fb1700	0x280000
37	blk.10.attn_norm.weight	0x1b7231700	0x2800
38	blk.10.layer_output_scale.weight	0x1b7233f00	0x4
39	blk.10.ffn_down.weight	0x1b7233f20	0x3200000
40	blk.10.ffn_gate.weight	0x1ba433f20	0x3200000
41	blk.10.ffn_up.weight	0x1bd633f20	0x3200000
42	blk.10.inp_gate.weight	0x1c0833f20	0x140000
43	blk.10.proj.weight	0x1c0973f20	0x140000
44	blk.10.post_attention_norm.weight	0x1c0ab3f20	0x2800
45	blk.10.post_ffw_norm.weight	0x1c0ab6720	0x2800
46	blk.10.post_norm.weight	0x1c0ab8f20	0x2800
47	blk.10.ffn_norm.weight	0x1c0abb720	0x2800
48	blk.10.attn_k_norm.weight	0x1c0abdf20	0x400
49	blk.10.attn_k.weight	0x1c0abe320	0x280000
50	blk.10.attn_output.weight	0x1c0d3e320	0xa00000
51	blk.10.attn_q_norm.weight	0x1c173e320	0x400
52	blk.10.attn_q.weight	0x1c173e720	0xa00000
53	blk.10.attn_v.weight	0x1c213e720	0x280000
54	blk.11.attn_norm.weight	0x1c23be720	0x2800
55	blk.11.layer_output_scale.weight	0x1c23c0f20	0x4
56	blk.11.ffn_down.weight	0x1c23c0f40	0x3200000
57	blk.11.ffn_gate.weight	0x1c55c0f40	0x3200000
58	blk.11.ffn_up.weight	0x1c87c0f40	0x3200000
59	blk.11.inp_gate.weight	0x1cb9c0f40	0x140000
60	blk.11.proj.weight	0x1cbb00f40	0x140000
61	blk.11.post_attention_norm.weight	0x1cbc40f40	0x2800
62	blk.11.post_ffw_norm.weight	0x1cbc43740	0x2800
63	blk.11.post_norm.weight	0x1cbc45f40	0x2800
64	blk.11.ffn_norm.weight	0x1cbc48740	0x2800
65	blk.11.attn_k_norm.weight	0x1cbc4af40	0x800
66	blk.11.attn_k.weight	0x1cbc4b740	0x500000
67	blk.11.attn_output.weight	0x1cc14b740	0x1400000
68	blk.11.attn_q_norm.weight	0x1cd54b740	0x800
69	blk.11.attn_q.weight	0x1cd54bf40	0x1400000
70	blk.11.attn_v.weight	0x1ce94bf40	0x500000
71	blk.12.attn_norm.weight	0x1cee4bf40	0x2800
72	blk.12.layer_output_scale.weight	0x1cee4e740	0x4
73	blk.12.ffn_down.weight	0x1cee4e760	0x3200000
74	blk.12.ffn_gate.weight	0x1d204e760	0x3200000
75	blk.12.ffn_up.weight	0x1d524e760	0x3200000
76	blk.12.inp_gate.weight	0x1d844e760	0x140000
77	blk.12.proj.weight	0x1d858e760	0x140000
78	blk.12.post_attention_norm.weight	0x1d86ce760	0x2800
79	blk.12.post_ffw_norm.weight	0x1d86d0f60	0x2800
80	blk.12.post_norm.weight	0x1d86d3760	0x2800
81	blk.12.ffn_norm.weight	0x1d86d5f60	0x2800
82	blk.12.attn_k_norm.weight	0x1d86d8760	0x400
83	blk.12.attn_k.weight	0x1d86d8b60	0x280000
84	blk.12.attn_output.weight	0x1d8958b60	0xa00000
85	blk.12.attn_q_norm.weight	0x1d9358b60	0x400
86	blk.12.attn_q.weight	0x1d9358f60	0xa00000
87	blk.12.attn_v.weight	0x1d9d58f60	0x280000
88	blk.13.attn_norm.weight	0x1d9fd8f60	0x2800
89	blk.13.layer_output_scale.weight	0x1d9fdb760	0x4
90	blk.13.ffn_down.weight	0x1d9fdb780	0x3200000
91	blk.13.ffn_gate.weight	0x1dd1db780	0x3200000
92	blk.13.ffn_up.weight	0x1e03db780	0x3200000
93	blk.13.inp_gate.weight	0x1e35db780	0x140000
94	blk.13.proj.weight	0x1e371b780	0x140000
95	blk.13.post_attention_norm.weight	0x1e385b780	0x2800
96	blk.13.post_ffw_norm.weight	0x1e385df80	0x2800
97	blk.13.post_norm.weight	0x1e3860780	0x2800
98	blk.13.ffn_norm.weight	0x1e3862f80	0x2800
99	blk.13.attn_k_norm.weight	0x1e3865780	0x400
100	blk.13.attn_k.weight	0x1e3865b80	0x280000
101	blk.13.attn_output.weight	0x1e3ae5b80	0xa00000
102	blk.13.attn_q_norm.weight	0x1e44e5b80	0x400
103	blk.13.attn_q.weight	0x1e44e5f80	0xa00000
104	blk.13.attn_v.weight	0x1e4ee5f80	0x280000
105	blk.14.attn_norm.weight	0x1e5165f80	0x2800
106	blk.14.layer_output_scale.weight	0x1e5168780	0x4
107	blk.14.ffn_down.weight	0x1e51687a0	0x3200000
108	blk.14.ffn_gate.weight	0x1e83687a0	0x3200000
109	blk.14.ffn_up.weight	0x1eb5687a0	0x3200000
110	blk.14.inp_gate.weight	0x1ee7687a0	0x140000
111	blk.14.proj.weight	0x1ee8a87a0	0x140000
112	blk.14.post_attention_norm.weight	0x1ee9e87a0	0x2800
113	blk.14.post_ffw_norm.weight	0x1ee9eafa0	0x2800
114	blk.14.post_norm.weight	0x1ee9ed7a0	0x2800
115	blk.14.ffn_norm.weight	0x1ee9effa0	0x2800
116	blk.14.attn_k_norm.weight	0x1ee9f27a0	0x400
117	blk.14.attn_k.weight	0x1ee9f2ba0	0x280000
118	blk.14.attn_output.weight	0x1eec72ba0	0xa00000
119	blk.14.attn_q_norm.weight	0x1ef672ba0	0x400
120	blk.14.attn_q.weight	0x1ef672fa0	0xa00000
121	blk.14.attn_v.weight	0x1f0072fa0	0x280000
122	blk.15.attn_norm.weight	0x1f02f2fa0	0x2800
123	blk.15.layer_output_scale.weight	0x1f02f57a0	0x4
124	blk.15.ffn_down.weight	0x1f02f57c0	0x3200000
125	blk.15.ffn_gate.weight	0x1f34f57c0	0x3200000
126	blk.15.ffn_up.weight	0x1f66f57c0	0x3200000
127	blk.15.inp_gate.weight	0x1f98f57c0	0x140000
128	blk.15.proj.weight	0x1f9a357c0	0x140000
129	blk.15.post_attention_norm.weight	0x1f9b757c0	0x2800
130	blk.15.post_ffw_norm.weight	0x1f9b77fc0	0x2800
131	blk.15.post_norm.weight	0x1f9b7a7c0	0x2800
132	blk.15.ffn_norm.weight	0x1f9b7cfc0	0x2800
133	blk.15.attn_k_norm.weight	0x1f9b7f7c0	0x400
134	blk.15.attn_k.weight	0x1f9b7fbc0	0x280000
135	blk.15.attn_output.weight	0x1f9dffbc0	0xa00000
136	blk.15.attn_q_norm.weight	0x1fa7ffbc0	0x400
137	blk.15.attn_q.weight	0x1fa7fffc0	0xa00000
138	blk.15.attn_v.weight	0x1fb1fffc0	0x280000
139	blk.16.attn_norm.weight	0x1fb47ffc0	0x2800
140	blk.16.layer_output_scale.weight	0x1fb4827c0	0x4
141	blk.16.ffn_down.weight	0x1fb4827e0	0x3200000
142	blk.16.ffn_gate.weight	0x1fe6827e0	0x3200000
143	blk.16.ffn_up.weight	0x2018827e0	0x3200000
144	blk.16.inp_gate.weight	0x204a827e0	0x140000
145	blk.16.proj.weight	0x204bc27e0	0x140000
146	blk.16.post_attention_norm.weight	0x204d027e0	0x2800
147	blk.16.post_ffw_norm.weight	0x204d04fe0	0x2800
148	blk.16.post_norm.weight	0x204d077e0	0x2800
149	blk.16.ffn_norm.weight	0x204d09fe0	0x2800
150	blk.16.attn_k_norm.weight	0x204d0c7e0	0x400
151	blk.16.attn_k.weight	0x204d0cbe0	0x280000
152	blk.16.attn_output.weight	0x204f8cbe0	0xa00000
153	blk.16.attn_q_norm.weight	0x20598cbe0	0x400
154	blk.16.attn_q.weight	0x20598cfe0	0xa00000
155	blk.16.attn_v.weight	0x20638cfe0	0x280000
156	blk.17.attn_norm.weight	0x20660cfe0	0x2800
157	blk.17.layer_output_scale.weight	0x20660f7e0	0x4
158	blk.17.ffn_down.weight	0x20660f800	0x3200000
159	blk.17.ffn_gate.weight	0x20980f800	0x3200000
160	blk.17.ffn_up.weight	0x20ca0f800	0x3200000
161	blk.17.inp_gate.weight	0x20fc0f800	0x140000
162	blk.17.proj.weight	0x20fd4f800	0x140000
163	blk.17.post_attention_norm.weight	0x20fe8f800	0x2800
164	blk.17.post_ffw_norm.weight	0x20fe92000	0x2800
165	blk.17.post_norm.weight	0x20fe94800	0x2800
166	blk.17.ffn_norm.weight	0x20fe97000	0x2800
167	blk.17.attn_k_norm.weight	0x20fe99800	0x800
168	blk.17.attn_k.weight	0x20fe9a000	0x500000
169	blk.17.attn_output.weight	0x21039a000	0x1400000
170	blk.17.attn_q_norm.weight	0x21179a000	0x800
171	blk.17.attn_q.weight	0x21179a800	0x1400000
172	blk.17.attn_v.weight	0x212b9a800	0x500000
173	blk.18.attn_norm.weight	0x21309a800	0x2800
174	blk.18.layer_output_scale.weight	0x21309d000	0x4
175	blk.18.ffn_down.weight	0x21309d020	0x3200000
176	blk.18.ffn_gate.weight	0x21629d020	0x3200000
177	blk.18.ffn_up.weight	0x21949d020	0x3200000
178	blk.18.inp_gate.weight	0x21c69d020	0x140000
179	blk.18.proj.weight	0x21c7dd020	0x140000
180	blk.18.post_attention_norm.weight	0x21c91d020	0x2800
181	blk.18.post_ffw_norm.weight	0x21c91f820	0x2800
182	blk.18.post_norm.weight	0x21c922020	0x2800
183	blk.18.ffn_norm.weight	0x21c924820	0x2800
184	blk.18.attn_k_norm.weight	0x21c927020	0x400
185	blk.18.attn_k.weight	0x21c927420	0x280000
186	blk.18.attn_output.weight	0x21cba7420	0xa00000
187	blk.18.attn_q_norm.weight	0x21d5a7420	0x400
188	blk.18.attn_q.weight	0x21d5a7820	0xa00000
189	blk.18.attn_v.weight	0x21dfa7820	0x280000
190	blk.19.attn_norm.weight	0x21e227820	0x2800
191	blk.19.layer_output_scale.weight	0x21e22a020	0x4
192	blk.19.ffn_down.weight	0x21e22a040	0x3200000
193	blk.19.ffn_gate.weight	0x22142a040	0x3200000
194	blk.19.ffn_up.weight	0x22462a040	0x3200000
195	blk.19.inp_gate.weight	0x22782a040	0x140000
196	blk.19.proj.weight	0x22796a040	0x140000
197	blk.19.post_attention_norm.weight	0x227aaa040	0x2800
198	blk.19.post_ffw_norm.weight	0x227aac840	0x2800
199	blk.19.post_norm.weight	0x227aaf040	0x2800
200	blk.19.ffn_norm.weight	0x227ab1840	0x2800
201	blk.19.attn_k_norm.weight	0x227ab4040	0x400
202	blk.19.attn_k.weight	0x227ab4440	0x280000
203	blk.19.attn_output.weight	0x227d34440	0xa00000
204	blk.19.attn_q_norm.weight	0x228734440	0x400
205	blk.19.attn_q.weight	0x228734840	0xa00000
206	blk.19.attn_v.weight	0x229134840	0x280000
207	blk.2.attn_norm.weight	0x2293b4840	0x2800
208	blk.2.layer_output_scale.weight	0x2293b7040	0x4
209	blk.2.ffn_down.weight	0x2293b7060	0x3200000
210	blk.2.ffn_gate.weight	0x22c5b7060	0x3200000
211	blk.2.ffn_up.weight	0x22f7b7060	0x3200000
212	blk.2.inp_gate.weight	0x2329b7060	0x140000
213	blk.2.proj.weight	0x232af7060	0x140000
214	blk.2.post_attention_norm.weight	0x232c37060	0x2800
215	blk.2.post_ffw_norm.weight	0x232c39860	0x2800
216	blk.2.post_norm.weight	0x232c3c060	0x2800
217	blk.2.ffn_norm.weight	0x232c3e860	0x2800
218	blk.2.attn_k_norm.weight	0x232c41060	0x400
219	blk.2.attn_k.weight	0x232c41460	0x280000
220	blk.2.attn_output.weight	0x232ec1460	0xa00000
221	blk.2.attn_q_norm.weight	0x2338c1460	0x400
222	blk.2.attn_q.weight	0x2338c1860	0xa00000
223	blk.2.attn_v.weight	0x2342c1860	0x280000
224	blk.20.attn_norm.weight	0x234541860	0x2800
225	blk.20.layer_output_scale.weight	0x234544060	0x4
226	blk.20.ffn_down.weight	0x234544080	0x3200000
227	blk.20.ffn_gate.weight	0x237744080	0x3200000
228	blk.20.ffn_up.weight	0x23a944080	0x3200000
229	blk.20.inp_gate.weight	0x23db44080	0x140000
230	blk.20.proj.weight	0x23dc84080	0x140000
231	blk.20.post_attention_norm.weight	0x23ddc4080	0x2800
232	blk.20.post_ffw_norm.weight	0x23ddc6880	0x2800
233	blk.20.post_norm.weight	0x23ddc9080	0x2800
234	blk.20.ffn_norm.weight	0x23ddcb880	0x2800
235	blk.20.attn_k_norm.weight	0x23ddce080	0x400
236	blk.20.attn_k.weight	0x23ddce480	0x280000
237	blk.20.attn_output.weight	0x23e04e480	0xa00000
238	blk.20.attn_q_norm.weight	0x23ea4e480	0x400
239	blk.20.attn_q.weight	0x23ea4e880	0xa00000
240	blk.20.attn_v.weight	0x23f44e880	0x280000
241	blk.21.attn_norm.weight	0x23f6ce880	0x2800
242	blk.21.layer_output_scale.weight	0x23f6d1080	0x4
243	blk.21.ffn_down.weight	0x23f6d10a0	0x3200000
244	blk.21.ffn_gate.weight	0x2428d10a0	0x3200000
245	blk.21.ffn_up.weight	0x245ad10a0	0x3200000
246	blk.21.inp_gate.weight	0x248cd10a0	0x140000
247	blk.21.proj.weight	0x248e110a0	0x140000
248	blk.21.post_attention_norm.weight	0x248f510a0	0x2800
249	blk.21.post_ffw_norm.weight	0x248f538a0	0x2800
250	blk.21.post_norm.weight	0x248f560a0	0x2800
251	blk.21.ffn_norm.weight	0x248f588a0	0x2800
252	blk.21.attn_k_norm.weight	0x248f5b0a0	0x400
253	blk.21.attn_k.weight	0x248f5b4a0	0x280000
254	blk.21.attn_output.weight	0x2491db4a0	0xa00000
255	blk.21.attn_q_norm.weight	0x249bdb4a0	0x400
256	blk.21.attn_q.weight	0x249bdb8a0	0xa00000
257	blk.21.attn_v.weight	0x24a5db8a0	0x280000
258	blk.22.attn_norm.weight	0x24a85b8a0	0x2800
259	blk.22.layer_output_scale.weight	0x24a85e0a0	0x4
260	blk.22.ffn_down.weight	0x24a85e0c0	0x3200000
261	blk.22.ffn_gate.weight	0x24da5e0c0	0x3200000
262	blk.22.ffn_up.weight	0x250c5e0c0	0x3200000
263	blk.22.inp_gate.weight	0x253e5e0c0	0x140000
264	blk.22.proj.weight	0x253f9e0c0	0x140000
265	blk.22.post_attention_norm.weight	0x2540de0c0	0x2800
266	blk.22.post_ffw_norm.weight	0x2540e08c0	0x2800
267	blk.22.post_norm.weight	0x2540e30c0	0x2800
268	blk.22.ffn_norm.weight	0x2540e58c0	0x2800
269	blk.22.attn_k_norm.weight	0x2540e80c0	0x400
270	blk.22.attn_k.weight	0x2540e84c0	0x280000
271	blk.22.attn_output.weight	0x2543684c0	0xa00000
272	blk.22.attn_q_norm.weight	0x254d684c0	0x400
273	blk.22.attn_q.weight	0x254d688c0	0xa00000
274	blk.22.attn_v.weight	0x2557688c0	0x280000
275	blk.23.attn_norm.weight	0x2559e88c0	0x2800
276	blk.23.layer_output_scale.weight	0x2559eb0c0	0x4
277	blk.23.ffn_down.weight	0x2559eb0e0	0x3200000
278	blk.23.ffn_gate.weight	0x258beb0e0	0x3200000
279	blk.23.ffn_up.weight	0x25bdeb0e0	0x3200000
280	blk.23.inp_gate.weight	0x25efeb0e0	0x140000
281	blk.23.proj.weight	0x25f12b0e0	0x140000
282	blk.23.post_attention_norm.weight	0x25f26b0e0	0x2800
283	blk.23.post_ffw_norm.weight	0x25f26d8e0	0x2800
284	blk.23.post_norm.weight	0x25f2700e0	0x2800
285	blk.23.ffn_norm.weight	0x25f2728e0	0x2800
286	blk.23.attn_k_norm.weight	0x25f2750e0	0x800
287	blk.23.attn_k.weight	0x25f2758e0	0x500000
288	blk.23.attn_output.weight	0x25f7758e0	0x1400000
289	blk.23.attn_q_norm.weight	0x260b758e0	0x800
290	blk.23.attn_q.weight	0x260b760e0	0x1400000
291	blk.23.attn_v.weight	0x261f760e0	0x500000
292	blk.24.attn_norm.weight	0x2624760e0	0x2800
293	blk.24.layer_output_scale.weight	0x2624788e0	0x4
294	blk.24.ffn_down.weight	0x262478900	0x3200000
295	blk.24.ffn_gate.weight	0x265678900	0x3200000
296	blk.24.ffn_up.weight	0x268878900	0x3200000
297	blk.24.inp_gate.weight	0x26ba78900	0x140000
298	blk.24.proj.weight	0x26bbb8900	0x140000
299	blk.24.post_attention_norm.weight	0x26bcf8900	0x2800
300	blk.24.post_ffw_norm.weight	0x26bcfb100	0x2800
301	blk.24.post_norm.weight	0x26bcfd900	0x2800
302	blk.24.ffn_norm.weight	0x26bd00100	0x2800
303	blk.24.attn_k_norm.weight	0x26bd02900	0x400
304	blk.24.attn_k.weight	0x26bd02d00	0x280000
305	blk.24.attn_output.weight	0x26bf82d00	0xa00000
306	blk.24.attn_q_norm.weight	0x26c982d00	0x400
307	blk.24.attn_q.weight	0x26c983100	0xa00000
308	blk.24.attn_v.weight	0x26d383100	0x280000
309	blk.25.attn_norm.weight	0x26d603100	0x2800
310	blk.25.layer_output_scale.weight	0x26d605900	0x4
311	blk.25.ffn_down.weight	0x26d605920	0x3200000
312	blk.25.ffn_gate.weight	0x270805920	0x3200000
313	blk.25.ffn_up.weight	0x273a05920	0x3200000
314	blk.25.inp_gate.weight	0x276c05920	0x140000
315	blk.25.proj.weight	0x276d45920	0x140000
316	blk.25.post_attention_norm.weight	0x276e85920	0x2800
317	blk.25.post_ffw_norm.weight	0x276e88120	0x2800
318	blk.25.post_norm.weight	0x276e8a920	0x2800
319	blk.25.ffn_norm.weight	0x276e8d120	0x2800
320	blk.25.attn_k_norm.weight	0x276e8f920	0x400
321	blk.25.attn_k.weight	0x276e8fd20	0x280000
322	blk.25.attn_output.weight	0x27710fd20	0xa00000
323	blk.25.attn_q_norm.weight	0x277b0fd20	0x400
324	blk.25.attn_q.weight	0x277b10120	0xa00000
325	blk.25.attn_v.weight	0x278510120	0x280000
326	blk.26.attn_norm.weight	0x278790120	0x2800
327	blk.26.layer_output_scale.weight	0x278792920	0x4
328	blk.26.ffn_down.weight	0x278792940	0x3200000
329	blk.26.ffn_gate.weight	0x27b992940	0x3200000
330	blk.26.ffn_up.weight	0x27eb92940	0x3200000
331	blk.26.inp_gate.weight	0x281d92940	0x140000
332	blk.26.proj.weight	0x281ed2940	0x140000
333	blk.26.post_attention_norm.weight	0x282012940	0x2800
334	blk.26.post_ffw_norm.weight	0x282015140	0x2800
335	blk.26.post_norm.weight	0x282017940	0x2800
336	blk.26.ffn_norm.weight	0x28201a140	0x2800
337	blk.26.attn_k_norm.weight	0x28201c940	0x400
338	blk.26.attn_k.weight	0x28201cd40	0x280000
339	blk.26.attn_output.weight	0x28229cd40	0xa00000
340	blk.26.attn_q_norm.weight	0x282c9cd40	0x400
341	blk.26.attn_q.weight	0x282c9d140	0xa00000
342	blk.26.attn_v.weight	0x28369d140	0x280000
343	blk.27.attn_norm.weight	0x28391d140	0x2800
344	blk.27.layer_output_scale.weight	0x28391f940	0x4
345	blk.27.ffn_down.weight	0x28391f960	0x3200000
346	blk.27.ffn_gate.weight	0x286b1f960	0x3200000
347	blk.27.ffn_up.weight	0x289d1f960	0x3200000
348	blk.27.inp_gate.weight	0x28cf1f960	0x140000
349	blk.27.proj.weight	0x28d05f960	0x140000
350	blk.27.post_attention_norm.weight	0x28d19f960	0x2800
351	blk.27.post_ffw_norm.weight	0x28d1a2160	0x2800
352	blk.27.post_norm.weight	0x28d1a4960	0x2800
353	blk.27.ffn_norm.weight	0x28d1a7160	0x2800
354	blk.27.attn_k_norm.weight	0x28d1a9960	0x400
355	blk.27.attn_k.weight	0x28d1a9d60	0x280000
356	blk.27.attn_output.weight	0x28d429d60	0xa00000
357	blk.27.attn_q_norm.weight	0x28de29d60	0x400
358	blk.27.attn_q.weight	0x28de2a160	0xa00000
359	blk.27.attn_v.weight	0x28e82a160	0x280000
360	blk.28.attn_norm.weight	0x28eaaa160	0x2800
361	blk.28.layer_output_scale.weight	0x28eaac960	0x4
362	blk.28.ffn_down.weight	0x28eaac980	0x3200000
363	blk.28.ffn_gate.weight	0x291cac980	0x3200000
364	blk.28.ffn_up.weight	0x294eac980	0x3200000
365	blk.28.inp_gate.weight	0x2980ac980	0x140000
366	blk.28.proj.weight	0x2981ec980	0x140000
367	blk.28.post_attention_norm.weight	0x29832c980	0x2800
368	blk.28.post_ffw_norm.weight	0x29832f180	0x2800
369	blk.28.post_norm.weight	0x298331980	0x2800
370	blk.28.ffn_norm.weight	0x298334180	0x2800
371	blk.28.attn_k_norm.weight	0x298336980	0x400
372	blk.28.attn_k.weight	0x298336d80	0x280000
373	blk.28.attn_output.weight	0x2985b6d80	0xa00000
374	blk.28.attn_q_norm.weight	0x298fb6d80	0x400
375	blk.28.attn_q.weight	0x298fb7180	0xa00000
376	blk.28.attn_v.weight	0x2999b7180	0x280000
377	blk.29.attn_norm.weight	0x299c37180	0x2800
378	blk.29.layer_output_scale.weight	0x299c39980	0x4
379	blk.29.ffn_down.weight	0x299c399a0	0x3200000
380	blk.29.ffn_gate.weight	0x29ce399a0	0x3200000
381	blk.29.ffn_up.weight	0x2a00399a0	0x3200000
382	blk.29.inp_gate.weight	0x2a32399a0	0x140000
383	blk.29.proj.weight	0x2a33799a0	0x140000
384	blk.29.post_attention_norm.weight	0x2a34b99a0	0x2800
385	blk.29.post_ffw_norm.weight	0x2a34bc1a0	0x2800
386	blk.29.post_norm.weight	0x2a34be9a0	0x2800
387	blk.29.ffn_norm.weight	0x2a34c11a0	0x2800
388	blk.29.attn_k_norm.weight	0x2a34c39a0	0x800
389	blk.29.attn_k.weight	0x2a34c41a0	0x500000
390	blk.29.attn_output.weight	0x2a39c41a0	0x1400000
391	blk.29.attn_q_norm.weight	0x2a4dc41a0	0x800
392	blk.29.attn_q.weight	0x2a4dc49a0	0x1400000
393	blk.29.attn_v.weight	0x2a61c49a0	0x500000
394	blk.3.attn_norm.weight	0x2a66c49a0	0x2800
395	blk.3.layer_output_scale.weight	0x2a66c71a0	0x4
396	blk.3.ffn_down.weight	0x2a66c71c0	0x3200000
397	blk.3.ffn_gate.weight	0x2a98c71c0	0x3200000
398	blk.3.ffn_up.weight	0x2acac71c0	0x3200000
399	blk.3.inp_gate.weight	0x2afcc71c0	0x140000
400	blk.3.proj.weight	0x2afe071c0	0x140000
401	blk.3.post_attention_norm.weight	0x2aff471c0	0x2800
402	blk.3.post_ffw_norm.weight	0x2aff499c0	0x2800
403	blk.3.post_norm.weight	0x2aff4c1c0	0x2800
404	blk.3.ffn_norm.weight	0x2aff4e9c0	0x2800
405	blk.3.attn_k_norm.weight	0x2aff511c0	0x400
406	blk.3.attn_k.weight	0x2aff515c0	0x280000
407	blk.3.attn_output.weight	0x2b01d15c0	0xa00000
408	blk.3.attn_q_norm.weight	0x2b0bd15c0	0x400
409	blk.3.attn_q.weight	0x2b0bd19c0	0xa00000
410	blk.3.attn_v.weight	0x2b15d19c0	0x280000
411	blk.30.attn_norm.weight	0x2b18519c0	0x2800
412	blk.30.layer_output_scale.weight	0x2b18541c0	0x4
413	blk.30.ffn_down.weight	0x2b18541e0	0x3200000
414	blk.30.ffn_gate.weight	0x2b4a541e0	0x3200000
415	blk.30.ffn_up.weight	0x2b7c541e0	0x3200000
416	blk.30.inp_gate.weight	0x2bae541e0	0x140000
417	blk.30.proj.weight	0x2baf941e0	0x140000
418	blk.30.post_attention_norm.weight	0x2bb0d41e0	0x2800
419	blk.30.post_ffw_norm.weight	0x2bb0d69e0	0x2800
420	blk.30.post_norm.weight	0x2bb0d91e0	0x2800
421	blk.30.ffn_norm.weight	0x2bb0db9e0	0x2800
422	blk.30.attn_k_norm.weight	0x2bb0de1e0	0x400
423	blk.30.attn_k.weight	0x2bb0de5e0	0x280000
424	blk.30.attn_output.weight	0x2bb35e5e0	0xa00000
425	blk.30.attn_q_norm.weight	0x2bbd5e5e0	0x400
426	blk.30.attn_q.weight	0x2bbd5e9e0	0xa00000
427	blk.30.attn_v.weight	0x2bc75e9e0	0x280000
428	blk.31.attn_norm.weight	0x2bc9de9e0	0x2800
429	blk.31.layer_output_scale.weight	0x2bc9e11e0	0x4
430	blk.31.ffn_down.weight	0x2bc9e1200	0x3200000
431	blk.31.ffn_gate.weight	0x2bfbe1200	0x3200000
432	blk.31.ffn_up.weight	0x2c2de1200	0x3200000
433	blk.31.inp_gate.weight	0x2c5fe1200	0x140000
434	blk.31.proj.weight	0x2c6121200	0x140000
435	blk.31.post_attention_norm.weight	0x2c6261200	0x2800
436	blk.31.post_ffw_norm.weight	0x2c6263a00	0x2800
437	blk.31.post_norm.weight	0x2c6266200	0x2800
438	blk.31.ffn_norm.weight	0x2c6268a00	0x2800
439	blk.31.attn_k_norm.weight	0x2c626b200	0x400
440	blk.31.attn_k.weight	0x2c626b600	0x280000
441	blk.31.attn_output.weight	0x2c64eb600	0xa00000
442	blk.31.attn_q_norm.weight	0x2c6eeb600	0x400
443	blk.31.attn_q.weight	0x2c6eeba00	0xa00000
444	blk.31.attn_v.weight	0x2c78eba00	0x280000
445	blk.32.attn_norm.weight	0x2c7b6ba00	0x2800
446	blk.32.layer_output_scale.weight	0x2c7b6e200	0x4
447	blk.32.ffn_down.weight	0x2c7b6e220	0x3200000
448	blk.32.ffn_gate.weight	0x2cad6e220	0x3200000
449	blk.32.ffn_up.weight	0x2cdf6e220	0x3200000
450	blk.32.inp_gate.weight	0x2d116e220	0x140000
451	blk.32.proj.weight	0x2d12ae220	0x140000
452	blk.32.post_attention_norm.weight	0x2d13ee220	0x2800
453	blk.32.post_ffw_norm.weight	0x2d13f0a20	0x2800
454	blk.32.post_norm.weight	0x2d13f3220	0x2800
455	blk.32.ffn_norm.weight	0x2d13f5a20	0x2800
456	blk.32.attn_k_norm.weight	0x2d13f8220	0x400
457	blk.32.attn_k.weight	0x2d13f8620	0x280000
458	blk.32.attn_output.weight	0x2d1678620	0xa00000
459	blk.32.attn_q_norm.weight	0x2d2078620	0x400
460	blk.32.attn_q.weight	0x2d2078a20	0xa00000
461	blk.32.attn_v.weight	0x2d2a78a20	0x280000
462	blk.33.attn_norm.weight	0x2d2cf8a20	0x2800
463	blk.33.layer_output_scale.weight	0x2d2cfb220	0x4
464	blk.33.ffn_down.weight	0x2d2cfb240	0x3200000
465	blk.33.ffn_gate.weight	0x2d5efb240	0x3200000
466	blk.33.ffn_up.weight	0x2d90fb240	0x3200000
467	blk.33.inp_gate.weight	0x2dc2fb240	0x140000
468	blk.33.proj.weight	0x2dc43b240	0x140000
469	blk.33.post_attention_norm.weight	0x2dc57b240	0x2800
470	blk.33.post_ffw_norm.weight	0x2dc57da40	0x2800
471	blk.33.post_norm.weight	0x2dc580240	0x2800
472	blk.33.ffn_norm.weight	0x2dc582a40	0x2800
473	blk.33.attn_k_norm.weight	0x2dc585240	0x400
474	blk.33.attn_k.weight	0x2dc585640	0x280000
475	blk.33.attn_output.weight	0x2dc805640	0xa00000
476	blk.33.attn_q_norm.weight	0x2dd205640	0x400
477	blk.33.attn_q.weight	0x2dd205a40	0xa00000
478	blk.33.attn_v.weight	0x2ddc05a40	0x280000
479	blk.34.attn_norm.weight	0x2dde85a40	0x2800
480	blk.34.layer_output_scale.weight	0x2dde88240	0x4
481	blk.34.ffn_down.weight	0x2dde88260	0x3200000
482	blk.34.ffn_gate.weight	0x2e1088260	0x3200000
483	blk.34.ffn_up.weight	0x2e4288260	0x3200000
484	blk.34.inp_gate.weight	0x2e7488260	0x140000
485	blk.34.proj.weight	0x2e75c8260	0x140000
486	blk.34.post_attention_norm.weight	0x2e7708260	0x2800
487	blk.34.post_ffw_norm.weight	0x2e770aa60	0x2800
488	blk.34.post_norm.weight	0x2e770d260	0x2800
489	blk.34.ffn_norm.weight	0x2e770fa60	0x2800
490	blk.34.attn_k_norm.weight	0x2e7712260	0x400
491	blk.34.attn_k.weight	0x2e7712660	0x280000
492	blk.34.attn_output.weight	0x2e7992660	0xa00000
493	blk.34.attn_q_norm.weight	0x2e8392660	0x400
494	blk.34.attn_q.weight	0x2e8392a60	0xa00000
495	blk.34.attn_v.weight	0x2e8d92a60	0x280000
496	blk.35.attn_norm.weight	0x2e9012a60	0x2800
497	blk.35.layer_output_scale.weight	0x2e9015260	0x4
498	blk.35.ffn_down.weight	0x2e9015280	0x3200000
499	blk.35.ffn_gate.weight	0x2ec215280	0x3200000
500	blk.35.ffn_up.weight	0x2ef415280	0x3200000
501	blk.35.inp_gate.weight	0x2f2615280	0x140000
502	blk.35.proj.weight	0x2f2755280	0x140000
503	blk.35.post_attention_norm.weight	0x2f2895280	0x2800
504	blk.35.post_ffw_norm.weight	0x2f2897a80	0x2800
505	blk.35.post_norm.weight	0x2f289a280	0x2800
506	blk.35.ffn_norm.weight	0x2f289ca80	0x2800
507	blk.35.attn_k_norm.weight	0x2f289f280	0x800
508	blk.35.attn_k.weight	0x2f289fa80	0x500000
509	blk.35.attn_output.weight	0x2f2d9fa80	0x1400000
510	blk.35.attn_q_norm.weight	0x2f419fa80	0x800
511	blk.35.attn_q.weight	0x2f41a0280	0x1400000
512	blk.35.attn_v.weight	0x2f55a0280	0x500000
513	blk.36.attn_norm.weight	0x2f5aa0280	0x2800
514	blk.36.layer_output_scale.weight	0x2f5aa2a80	0x4
515	blk.36.ffn_down.weight	0x2f5aa2aa0	0x3200000
516	blk.36.ffn_gate.weight	0x2f8ca2aa0	0x3200000
517	blk.36.ffn_up.weight	0x2fbea2aa0	0x3200000
518	blk.36.inp_gate.weight	0x2ff0a2aa0	0x140000
519	blk.36.proj.weight	0x2ff1e2aa0	0x140000
520	blk.36.post_attention_norm.weight	0x2ff322aa0	0x2800
521	blk.36.post_ffw_norm.weight	0x2ff3252a0	0x2800
522	blk.36.post_norm.weight	0x2ff327aa0	0x2800
523	blk.36.ffn_norm.weight	0x2ff32a2a0	0x2800
524	blk.36.attn_k_norm.weight	0x2ff32caa0	0x400
525	blk.36.attn_k.weight	0x2ff32cea0	0x280000
526	blk.36.attn_output.weight	0x2ff5acea0	0xa00000
527	blk.36.attn_q_norm.weight	0x2fffacea0	0x400
528	blk.36.attn_q.weight	0x2fffad2a0	0xa00000
529	blk.36.attn_v.weight	0x3009ad2a0	0x280000
530	blk.37.attn_norm.weight	0x300c2d2a0	0x2800
531	blk.37.layer_output_scale.weight	0x300c2faa0	0x4
532	blk.37.ffn_down.weight	0x300c2fac0	0x3200000
533	blk.37.ffn_gate.weight	0x303e2fac0	0x3200000
534	blk.37.ffn_up.weight	0x30702fac0	0x3200000
535	blk.37.inp_gate.weight	0x30a22fac0	0x140000
536	blk.37.proj.weight	0x30a36fac0	0x140000
537	blk.37.post_attention_norm.weight	0x30a4afac0	0x2800
538	blk.37.post_ffw_norm.weight	0x30a4b22c0	0x2800
539	blk.37.post_norm.weight	0x30a4b4ac0	0x2800
540	blk.37.ffn_norm.weight	0x30a4b72c0	0x2800
541	blk.37.attn_k_norm.weight	0x30a4b9ac0	0x400
542	blk.37.attn_k.weight	0x30a4b9ec0	0x280000
543	blk.37.attn_output.weight	0x30a739ec0	0xa00000
544	blk.37.attn_q_norm.weight	0x30b139ec0	0x400
545	blk.37.attn_q.weight	0x30b13a2c0	0xa00000
546	blk.37.attn_v.weight	0x30bb3a2c0	0x280000
547	blk.38.attn_norm.weight	0x30bdba2c0	0x2800
548	blk.38.layer_output_scale.weight	0x30bdbcac0	0x4
549	blk.38.ffn_down.weight	0x30bdbcae0	0x3200000
550	blk.38.ffn_gate.weight	0x30efbcae0	0x3200000
551	blk.38.ffn_up.weight	0x3121bcae0	0x3200000
552	blk.38.inp_gate.weight	0x3153bcae0	0x140000
553	blk.38.proj.weight	0x3154fcae0	0x140000
554	blk.38.post_attention_norm.weight	0x31563cae0	0x2800
555	blk.38.post_ffw_norm.weight	0x31563f2e0	0x2800
556	blk.38.post_norm.weight	0x315641ae0	0x2800
557	blk.38.ffn_norm.weight	0x3156442e0	0x2800
558	blk.38.attn_k_norm.weight	0x315646ae0	0x400
559	blk.38.attn_k.weight	0x315646ee0	0x280000
560	blk.38.attn_output.weight	0x3158c6ee0	0xa00000
561	blk.38.attn_q_norm.weight	0x3162c6ee0	0x400
562	blk.38.attn_q.weight	0x3162c72e0	0xa00000
563	blk.38.attn_v.weight	0x316cc72e0	0x280000
564	blk.39.attn_norm.weight	0x316f472e0	0x2800
565	blk.39.layer_output_scale.weight	0x316f49ae0	0x4
566	blk.39.ffn_down.weight	0x316f49b00	0x3200000
567	blk.39.ffn_gate.weight	0x31a149b00	0x3200000
568	blk.39.ffn_up.weight	0x31d349b00	0x3200000
569	blk.39.inp_gate.weight	0x320549b00	0x140000
570	blk.39.proj.weight	0x320689b00	0x140000
571	blk.39.post_attention_norm.weight	0x3207c9b00	0x2800
572	blk.39.post_ffw_norm.weight	0x3207cc300	0x2800
573	blk.39.post_norm.weight	0x3207ceb00	0x2800
574	blk.39.ffn_norm.weight	0x3207d1300	0x2800
575	blk.39.attn_k_norm.weight	0x3207d3b00	0x400
576	blk.39.attn_k.weight	0x3207d3f00	0x280000
577	blk.39.attn_output.weight	0x320a53f00	0xa00000
578	blk.39.attn_q_norm.weight	0x321453f00	0x400
579	blk.39.attn_q.weight	0x321454300	0xa00000
580	blk.39.attn_v.weight	0x321e54300	0x280000
581	blk.4.attn_norm.weight	0x3220d4300	0x2800
582	blk.4.layer_output_scale.weight	0x3220d6b00	0x4
583	blk.4.ffn_down.weight	0x3220d6b20	0x3200000
584	blk.4.ffn_gate.weight	0x3252d6b20	0x3200000
585	blk.4.ffn_up.weight	0x3284d6b20	0x3200000
586	blk.4.inp_gate.weight	0x32b6d6b20	0x140000
587	blk.4.proj.weight	0x32b816b20	0x140000
588	blk.4.post_attention_norm.weight	0x32b956b20	0x2800
589	blk.4.post_ffw_norm.weight	0x32b959320	0x2800
590	blk.4.post_norm.weight	0x32b95bb20	0x2800
591	blk.4.ffn_norm.weight	0x32b95e320	0x2800
592	blk.4.attn_k_norm.weight	0x32b960b20	0x400
593	blk.4.attn_k.weight	0x32b960f20	0x280000
594	blk.4.attn_output.weight	0x32bbe0f20	0xa00000
595	blk.4.attn_q_norm.weight	0x32c5e0f20	0x400
596	blk.4.attn_q.weight	0x32c5e1320	0xa00000
597	blk.4.attn_v.weight	0x32cfe1320	0x280000
598	blk.40.attn_norm.weight	0x32d261320	0x2800
599	blk.40.layer_output_scale.weight	0x32d263b20	0x4
600	blk.40.ffn_down.weight	0x32d263b40	0x3200000
601	blk.40.ffn_gate.weight	0x330463b40	0x3200000
602	blk.40.ffn_up.weight	0x333663b40	0x3200000
603	blk.40.inp_gate.weight	0x336863b40	0x140000
604	blk.40.proj.weight	0x3369a3b40	0x140000
605	blk.40.post_attention_norm.weight	0x336ae3b40	0x2800
606	blk.40.post_ffw_norm.weight	0x336ae6340	0x2800
607	blk.40.post_norm.weight	0x336ae8b40	0x2800
608	blk.40.ffn_norm.weight	0x336aeb340	0x2800
609	blk.40.attn_k_norm.weight	0x336aedb40	0x400
610	blk.40.attn_k.weight	0x336aedf40	0x280000
611	blk.40.attn_output.weight	0x336d6df40	0xa00000
612	blk.40.attn_q_norm.weight	0x33776df40	0x400
613	blk.40.attn_q.weight	0x33776e340	0xa00000
614	blk.40.attn_v.weight	0x33816e340	0x280000
615	blk.41.attn_norm.weight	0x3383ee340	0x2800
616	blk.41.layer_output_scale.weight	0x3383f0b40	0x4
617	blk.41.ffn_down.weight	0x3383f0b60	0x3200000
618	blk.41.ffn_gate.weight	0x33b5f0b60	0x3200000
619	blk.41.ffn_up.weight	0x33e7f0b60	0x3200000
620	blk.41.inp_gate.weight	0x3419f0b60	0x140000
621	blk.41.proj.weight	0x341b30b60	0x140000
622	blk.41.post_attention_norm.weight	0x341c70b60	0x2800
623	blk.41.post_ffw_norm.weight	0x341c73360	0x2800
624	blk.41.post_norm.weight	0x341c75b60	0x2800
625	blk.41.ffn_norm.weight	0x341c78360	0x2800
626	blk.41.attn_k_norm.weight	0x341c7ab60	0x800
627	blk.41.attn_k.weight	0x341c7b360	0x500000
628	blk.41.attn_output.weight	0x34217b360	0x1400000
629	blk.41.attn_q_norm.weight	0x34357b360	0x800
630	blk.41.attn_q.weight	0x34357bb60	0x1400000
631	blk.41.attn_v.weight	0x34497bb60	0x500000
632	blk.5.attn_norm.weight	0x344e7bb60	0x2800
633	blk.5.layer_output_scale.weight	0x344e7e360	0x4
634	blk.5.ffn_down.weight	0x344e7e380	0x3200000
635	blk.5.ffn_gate.weight	0x34807e380	0x3200000
636	blk.5.ffn_up.weight	0x34b27e380	0x3200000
637	blk.5.inp_gate.weight	0x34e47e380	0x140000
638	blk.5.proj.weight	0x34e5be380	0x140000
639	blk.5.post_attention_norm.weight	0x34e6fe380	0x2800
640	blk.5.post_ffw_norm.weight	0x34e700b80	0x2800
641	blk.5.post_norm.weight	0x34e703380	0x2800
642	blk.5.ffn_norm.weight	0x34e705b80	0x2800
643	blk.5.attn_k_norm.weight	0x34e708380	0x800
644	blk.5.attn_k.weight	0x34e708b80	0x500000
645	blk.5.attn_output.weight	0x34ec08b80	0x1400000
646	blk.5.attn_q_norm.weight	0x350008b80	0x800
647	blk.5.attn_q.weight	0x350009380	0x1400000
648	blk.5.attn_v.weight	0x351409380	0x500000
649	blk.6.attn_norm.weight	0x351909380	0x2800
650	blk.6.layer_output_scale.weight	0x35190bb80	0x4
651	blk.6.ffn_down.weight	0x35190bba0	0x3200000
652	blk.6.ffn_gate.weight	0x354b0bba0	0x3200000
653	blk.6.ffn_up.weight	0x357d0bba0	0x3200000
654	blk.6.inp_gate.weight	0x35af0bba0	0x140000
655	blk.6.proj.weight	0x35b04bba0	0x140000
656	blk.6.post_attention_norm.weight	0x35b18bba0	0x2800
657	blk.6.post_ffw_norm.weight	0x35b18e3a0	0x2800
658	blk.6.post_norm.weight	0x35b190ba0	0x2800
659	blk.6.ffn_norm.weight	0x35b1933a0	0x2800
660	blk.6.attn_k_norm.weight	0x35b195ba0	0x400
661	blk.6.attn_k.weight	0x35b195fa0	0x280000
662	blk.6.attn_output.weight	0x35b415fa0	0xa00000
663	blk.6.attn_q_norm.weight	0x35be15fa0	0x400
664	blk.6.attn_q.weight	0x35be163a0	0xa00000
665	blk.6.attn_v.weight	0x35c8163a0	0x280000
666	blk.7.attn_norm.weight	0x35ca963a0	0x2800
667	blk.7.layer_output_scale.weight	0x35ca98ba0	0x4
668	blk.7.ffn_down.weight	0x35ca98bc0	0x3200000
669	blk.7.ffn_gate.weight	0x35fc98bc0	0x3200000
670	blk.7.ffn_up.weight	0x362e98bc0	0x3200000
671	blk.7.inp_gate.weight	0x366098bc0	0x140000
672	blk.7.proj.weight	0x3661d8bc0	0x140000
673	blk.7.post_attention_norm.weight	0x366318bc0	0x2800
674	blk.7.post_ffw_norm.weight	0x36631b3c0	0x2800
675	blk.7.post_norm.weight	0x36631dbc0	0x2800
676	blk.7.ffn_norm.weight	0x3663203c0	0x2800
677	blk.7.attn_k_norm.weight	0x366322bc0	0x400
678	blk.7.attn_k.weight	0x366322fc0	0x280000
679	blk.7.attn_output.weight	0x3665a2fc0	0xa00000
680	blk.7.attn_q_norm.weight	0x366fa2fc0	0x400
681	blk.7.attn_q.weight	0x366fa33c0	0xa00000
682	blk.7.attn_v.weight	0x3679a33c0	0x280000
683	blk.8.attn_norm.weight	0x367c233c0	0x2800
684	blk.8.layer_output_scale.weight	0x367c25bc0	0x4
685	blk.8.ffn_down.weight	0x367c25be0	0x3200000
686	blk.8.ffn_gate.weight	0x36ae25be0	0x3200000
687	blk.8.ffn_up.weight	0x36e025be0	0x3200000
688	blk.8.inp_gate.weight	0x371225be0	0x140000
689	blk.8.proj.weight	0x371365be0	0x140000
690	blk.8.post_attention_norm.weight	0x3714a5be0	0x2800
691	blk.8.post_ffw_norm.weight	0x3714a83e0	0x2800
692	blk.8.post_norm.weight	0x3714aabe0	0x2800
693	blk.8.ffn_norm.weight	0x3714ad3e0	0x2800
694	blk.8.attn_k_norm.weight	0x3714afbe0	0x400
695	blk.8.attn_k.weight	0x3714affe0	0x280000
696	blk.8.attn_output.weight	0x37172ffe0	0xa00000
697	blk.8.attn_q_norm.weight	0x37212ffe0	0x400
698	blk.8.attn_q.weight	0x3721303e0	0xa00000
699	blk.8.attn_v.weight	0x372b303e0	0x280000
700	blk.9.attn_norm.weight	0x372db03e0	0x2800
701	blk.9.layer_output_scale.weight	0x372db2be0	0x4
702	blk.9.ffn_down.weight	0x372db2c00	0x3200000
703	blk.9.ffn_gate.weight	0x375fb2c00	0x3200000
704	blk.9.ffn_up.weight	0x3791b2c00	0x3200000
705	blk.9.inp_gate.weight	0x37c3b2c00	0x140000
706	blk.9.proj.weight	0x37c4f2c00	0x140000
707	blk.9.post_attention_norm.weight	0x37c632c00	0x2800
708	blk.9.post_ffw_norm.weight	0x37c635400	0x2800
709	blk.9.post_norm.weight	0x37c637c00	0x2800
710	blk.9.ffn_norm.weight	0x37c63a400	0x2800
711	blk.9.attn_k_norm.weight	0x37c63cc00	0x400
712	blk.9.attn_k.weight	0x37c63d000	0x280000
713	blk.9.attn_output.weight	0x37c8bd000	0xa00000
714	blk.9.attn_q_norm.weight	0x37d2bd000	0x400
715	blk.9.attn_q.weight	0x37d2bd400	0xa00000
716	blk.9.attn_v.weight	0x37dcbd400	0x280000
717	output_norm.weight	0x37df3d400	0x2800
718	per_layer_model_proj.weight	0x37df3fc00	0x3480000
719	per_layer_proj_norm.weight	0x3813bfc00	0x400

Base Tensor Group : ~4B Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
0	rope_freqs.weight	Rope_Freqs (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
1	token_embd.weight	Token Embedding (W)	(~671M) 671088640	2560 x 262144 x 1 x 1	F16	16.0000
2	per_layer_token_embd.weight	Per_Layer_Token_Embd (W)	( ~3B) 2818572288	10752 x 262144 x 1 x 1	F16	16.0000
717	output_norm.weight	Output Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
718	per_layer_model_proj.weight	Per_Layer_Model_Proj (W)	( ~28M) 27525120	2560 x 10752 x 1 x 1	F16	16.0000
719	per_layer_proj_norm.weight	Per_Layer_Proj_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000

Total elements in base: ( ~4B) 3517189120
Percentage of total elements: 46.78%
Bits per Weight (BPW) for base: 16.0000 bits

Block 0 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
3	blk.0.attn_norm.weight	Block 0 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
4	blk.0.layer_output_scale.weight	Block 0 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
5	blk.0.ffn_down.weight	Block 0 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
6	blk.0.ffn_gate.weight	Block 0 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
7	blk.0.ffn_up.weight	Block 0 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
8	blk.0.inp_gate.weight	Block 0 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
9	blk.0.proj.weight	Block 0 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
10	blk.0.post_attention_norm.weight	Block 0 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
11	blk.0.post_ffw_norm.weight	Block 0 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
12	blk.0.post_norm.weight	Block 0 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
13	blk.0.ffn_norm.weight	Block 0 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
14	blk.0.attn_k_norm.weight	Block 0 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
15	blk.0.attn_k.weight	Block 0 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
16	blk.0.attn_output.weight	Block 0 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
17	blk.0.attn_q_norm.weight	Block 0 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
18	blk.0.attn_q.weight	Block 0 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
19	blk.0.attn_v.weight	Block 0 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.0: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.0: 16.0023 bits

Block 1 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
20	blk.1.attn_norm.weight	Block 1 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
21	blk.1.layer_output_scale.weight	Block 1 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
22	blk.1.ffn_down.weight	Block 1 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
23	blk.1.ffn_gate.weight	Block 1 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
24	blk.1.ffn_up.weight	Block 1 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
25	blk.1.inp_gate.weight	Block 1 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
26	blk.1.proj.weight	Block 1 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
27	blk.1.post_attention_norm.weight	Block 1 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
28	blk.1.post_ffw_norm.weight	Block 1 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
29	blk.1.post_norm.weight	Block 1 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
30	blk.1.ffn_norm.weight	Block 1 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
31	blk.1.attn_k_norm.weight	Block 1 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
32	blk.1.attn_k.weight	Block 1 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
33	blk.1.attn_output.weight	Block 1 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
34	blk.1.attn_q_norm.weight	Block 1 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
35	blk.1.attn_q.weight	Block 1 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
36	blk.1.attn_v.weight	Block 1 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.1: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.1: 16.0023 bits

Block 10 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
37	blk.10.attn_norm.weight	Block 10 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
38	blk.10.layer_output_scale.weight	Block 10 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
39	blk.10.ffn_down.weight	Block 10 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
40	blk.10.ffn_gate.weight	Block 10 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
41	blk.10.ffn_up.weight	Block 10 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
42	blk.10.inp_gate.weight	Block 10 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
43	blk.10.proj.weight	Block 10 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
44	blk.10.post_attention_norm.weight	Block 10 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
45	blk.10.post_ffw_norm.weight	Block 10 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
46	blk.10.post_norm.weight	Block 10 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
47	blk.10.ffn_norm.weight	Block 10 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
48	blk.10.attn_k_norm.weight	Block 10 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
49	blk.10.attn_k.weight	Block 10 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
50	blk.10.attn_output.weight	Block 10 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
51	blk.10.attn_q_norm.weight	Block 10 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
52	blk.10.attn_q.weight	Block 10 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
53	blk.10.attn_v.weight	Block 10 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.10: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.10: 16.0023 bits

Block 11 Tensor Group : ~106M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
54	blk.11.attn_norm.weight	Block 11 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
55	blk.11.layer_output_scale.weight	Block 11 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
56	blk.11.ffn_down.weight	Block 11 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
57	blk.11.ffn_gate.weight	Block 11 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
58	blk.11.ffn_up.weight	Block 11 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
59	blk.11.inp_gate.weight	Block 11 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
60	blk.11.proj.weight	Block 11 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
61	blk.11.post_attention_norm.weight	Block 11 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
62	blk.11.post_ffw_norm.weight	Block 11 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
63	blk.11.post_norm.weight	Block 11 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
64	blk.11.ffn_norm.weight	Block 11 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
65	blk.11.attn_k_norm.weight	Block 11 Attn_K_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
66	blk.11.attn_k.weight	Block 11 Attention Key (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	F16	16.0000
67	blk.11.attn_output.weight	Block 11 Attention Output (W)	( ~10M) 10485760	4096 x 2560 x 1 x 1	F16	16.0000
68	blk.11.attn_q_norm.weight	Block 11 Attn_Q_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
69	blk.11.attn_q.weight	Block 11 Attention Query (W)	( ~10M) 10485760	2560 x 4096 x 1 x 1	F16	16.0000
70	blk.11.attn_v.weight	Block 11 Attention Value (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	F16	16.0000

Total elements in blk.11: (~106M) 106182145
Percentage of total elements: 1.41%
Bits per Weight (BPW) for blk.11: 16.0021 bits

Block 12 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
71	blk.12.attn_norm.weight	Block 12 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
72	blk.12.layer_output_scale.weight	Block 12 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
73	blk.12.ffn_down.weight	Block 12 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
74	blk.12.ffn_gate.weight	Block 12 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
75	blk.12.ffn_up.weight	Block 12 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
76	blk.12.inp_gate.weight	Block 12 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
77	blk.12.proj.weight	Block 12 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
78	blk.12.post_attention_norm.weight	Block 12 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
79	blk.12.post_ffw_norm.weight	Block 12 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
80	blk.12.post_norm.weight	Block 12 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
81	blk.12.ffn_norm.weight	Block 12 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
82	blk.12.attn_k_norm.weight	Block 12 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
83	blk.12.attn_k.weight	Block 12 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
84	blk.12.attn_output.weight	Block 12 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
85	blk.12.attn_q_norm.weight	Block 12 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
86	blk.12.attn_q.weight	Block 12 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
87	blk.12.attn_v.weight	Block 12 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.12: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.12: 16.0023 bits

Block 13 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
88	blk.13.attn_norm.weight	Block 13 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
89	blk.13.layer_output_scale.weight	Block 13 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
90	blk.13.ffn_down.weight	Block 13 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
91	blk.13.ffn_gate.weight	Block 13 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
92	blk.13.ffn_up.weight	Block 13 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
93	blk.13.inp_gate.weight	Block 13 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
94	blk.13.proj.weight	Block 13 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
95	blk.13.post_attention_norm.weight	Block 13 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
96	blk.13.post_ffw_norm.weight	Block 13 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
97	blk.13.post_norm.weight	Block 13 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
98	blk.13.ffn_norm.weight	Block 13 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
99	blk.13.attn_k_norm.weight	Block 13 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
100	blk.13.attn_k.weight	Block 13 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
101	blk.13.attn_output.weight	Block 13 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
102	blk.13.attn_q_norm.weight	Block 13 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
103	blk.13.attn_q.weight	Block 13 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
104	blk.13.attn_v.weight	Block 13 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.13: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.13: 16.0023 bits

Block 14 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
105	blk.14.attn_norm.weight	Block 14 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
106	blk.14.layer_output_scale.weight	Block 14 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
107	blk.14.ffn_down.weight	Block 14 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
108	blk.14.ffn_gate.weight	Block 14 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
109	blk.14.ffn_up.weight	Block 14 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
110	blk.14.inp_gate.weight	Block 14 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
111	blk.14.proj.weight	Block 14 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
112	blk.14.post_attention_norm.weight	Block 14 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
113	blk.14.post_ffw_norm.weight	Block 14 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
114	blk.14.post_norm.weight	Block 14 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
115	blk.14.ffn_norm.weight	Block 14 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
116	blk.14.attn_k_norm.weight	Block 14 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
117	blk.14.attn_k.weight	Block 14 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
118	blk.14.attn_output.weight	Block 14 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
119	blk.14.attn_q_norm.weight	Block 14 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
120	blk.14.attn_q.weight	Block 14 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
121	blk.14.attn_v.weight	Block 14 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.14: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.14: 16.0023 bits

Block 15 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
122	blk.15.attn_norm.weight	Block 15 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
123	blk.15.layer_output_scale.weight	Block 15 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
124	blk.15.ffn_down.weight	Block 15 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
125	blk.15.ffn_gate.weight	Block 15 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
126	blk.15.ffn_up.weight	Block 15 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
127	blk.15.inp_gate.weight	Block 15 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
128	blk.15.proj.weight	Block 15 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
129	blk.15.post_attention_norm.weight	Block 15 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
130	blk.15.post_ffw_norm.weight	Block 15 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
131	blk.15.post_norm.weight	Block 15 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
132	blk.15.ffn_norm.weight	Block 15 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
133	blk.15.attn_k_norm.weight	Block 15 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
134	blk.15.attn_k.weight	Block 15 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
135	blk.15.attn_output.weight	Block 15 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
136	blk.15.attn_q_norm.weight	Block 15 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
137	blk.15.attn_q.weight	Block 15 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
138	blk.15.attn_v.weight	Block 15 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.15: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.15: 16.0023 bits

Block 16 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
139	blk.16.attn_norm.weight	Block 16 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
140	blk.16.layer_output_scale.weight	Block 16 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
141	blk.16.ffn_down.weight	Block 16 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
142	blk.16.ffn_gate.weight	Block 16 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
143	blk.16.ffn_up.weight	Block 16 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
144	blk.16.inp_gate.weight	Block 16 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
145	blk.16.proj.weight	Block 16 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
146	blk.16.post_attention_norm.weight	Block 16 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
147	blk.16.post_ffw_norm.weight	Block 16 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
148	blk.16.post_norm.weight	Block 16 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
149	blk.16.ffn_norm.weight	Block 16 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
150	blk.16.attn_k_norm.weight	Block 16 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
151	blk.16.attn_k.weight	Block 16 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
152	blk.16.attn_output.weight	Block 16 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
153	blk.16.attn_q_norm.weight	Block 16 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
154	blk.16.attn_q.weight	Block 16 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
155	blk.16.attn_v.weight	Block 16 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.16: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.16: 16.0023 bits

Block 17 Tensor Group : ~106M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
156	blk.17.attn_norm.weight	Block 17 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
157	blk.17.layer_output_scale.weight	Block 17 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
158	blk.17.ffn_down.weight	Block 17 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
159	blk.17.ffn_gate.weight	Block 17 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
160	blk.17.ffn_up.weight	Block 17 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
161	blk.17.inp_gate.weight	Block 17 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
162	blk.17.proj.weight	Block 17 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
163	blk.17.post_attention_norm.weight	Block 17 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
164	blk.17.post_ffw_norm.weight	Block 17 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
165	blk.17.post_norm.weight	Block 17 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
166	blk.17.ffn_norm.weight	Block 17 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
167	blk.17.attn_k_norm.weight	Block 17 Attn_K_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
168	blk.17.attn_k.weight	Block 17 Attention Key (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	F16	16.0000
169	blk.17.attn_output.weight	Block 17 Attention Output (W)	( ~10M) 10485760	4096 x 2560 x 1 x 1	F16	16.0000
170	blk.17.attn_q_norm.weight	Block 17 Attn_Q_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
171	blk.17.attn_q.weight	Block 17 Attention Query (W)	( ~10M) 10485760	2560 x 4096 x 1 x 1	F16	16.0000
172	blk.17.attn_v.weight	Block 17 Attention Value (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	F16	16.0000

Total elements in blk.17: (~106M) 106182145
Percentage of total elements: 1.41%
Bits per Weight (BPW) for blk.17: 16.0021 bits

Block 18 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
173	blk.18.attn_norm.weight	Block 18 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
174	blk.18.layer_output_scale.weight	Block 18 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
175	blk.18.ffn_down.weight	Block 18 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
176	blk.18.ffn_gate.weight	Block 18 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
177	blk.18.ffn_up.weight	Block 18 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
178	blk.18.inp_gate.weight	Block 18 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
179	blk.18.proj.weight	Block 18 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
180	blk.18.post_attention_norm.weight	Block 18 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
181	blk.18.post_ffw_norm.weight	Block 18 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
182	blk.18.post_norm.weight	Block 18 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
183	blk.18.ffn_norm.weight	Block 18 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
184	blk.18.attn_k_norm.weight	Block 18 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
185	blk.18.attn_k.weight	Block 18 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
186	blk.18.attn_output.weight	Block 18 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
187	blk.18.attn_q_norm.weight	Block 18 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
188	blk.18.attn_q.weight	Block 18 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
189	blk.18.attn_v.weight	Block 18 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.18: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.18: 16.0023 bits

Block 19 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
190	blk.19.attn_norm.weight	Block 19 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
191	blk.19.layer_output_scale.weight	Block 19 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
192	blk.19.ffn_down.weight	Block 19 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
193	blk.19.ffn_gate.weight	Block 19 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
194	blk.19.ffn_up.weight	Block 19 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
195	blk.19.inp_gate.weight	Block 19 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
196	blk.19.proj.weight	Block 19 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
197	blk.19.post_attention_norm.weight	Block 19 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
198	blk.19.post_ffw_norm.weight	Block 19 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
199	blk.19.post_norm.weight	Block 19 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
200	blk.19.ffn_norm.weight	Block 19 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
201	blk.19.attn_k_norm.weight	Block 19 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
202	blk.19.attn_k.weight	Block 19 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
203	blk.19.attn_output.weight	Block 19 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
204	blk.19.attn_q_norm.weight	Block 19 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
205	blk.19.attn_q.weight	Block 19 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
206	blk.19.attn_v.weight	Block 19 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.19: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.19: 16.0023 bits

Block 2 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
207	blk.2.attn_norm.weight	Block 2 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
208	blk.2.layer_output_scale.weight	Block 2 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
209	blk.2.ffn_down.weight	Block 2 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
210	blk.2.ffn_gate.weight	Block 2 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
211	blk.2.ffn_up.weight	Block 2 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
212	blk.2.inp_gate.weight	Block 2 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
213	blk.2.proj.weight	Block 2 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
214	blk.2.post_attention_norm.weight	Block 2 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
215	blk.2.post_ffw_norm.weight	Block 2 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
216	blk.2.post_norm.weight	Block 2 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
217	blk.2.ffn_norm.weight	Block 2 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
218	blk.2.attn_k_norm.weight	Block 2 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
219	blk.2.attn_k.weight	Block 2 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
220	blk.2.attn_output.weight	Block 2 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
221	blk.2.attn_q_norm.weight	Block 2 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
222	blk.2.attn_q.weight	Block 2 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
223	blk.2.attn_v.weight	Block 2 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.2: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.2: 16.0023 bits

Block 20 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
224	blk.20.attn_norm.weight	Block 20 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
225	blk.20.layer_output_scale.weight	Block 20 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
226	blk.20.ffn_down.weight	Block 20 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
227	blk.20.ffn_gate.weight	Block 20 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
228	blk.20.ffn_up.weight	Block 20 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
229	blk.20.inp_gate.weight	Block 20 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
230	blk.20.proj.weight	Block 20 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
231	blk.20.post_attention_norm.weight	Block 20 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
232	blk.20.post_ffw_norm.weight	Block 20 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
233	blk.20.post_norm.weight	Block 20 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
234	blk.20.ffn_norm.weight	Block 20 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
235	blk.20.attn_k_norm.weight	Block 20 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
236	blk.20.attn_k.weight	Block 20 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
237	blk.20.attn_output.weight	Block 20 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
238	blk.20.attn_q_norm.weight	Block 20 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
239	blk.20.attn_q.weight	Block 20 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
240	blk.20.attn_v.weight	Block 20 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.20: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.20: 16.0023 bits

Block 21 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
241	blk.21.attn_norm.weight	Block 21 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
242	blk.21.layer_output_scale.weight	Block 21 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
243	blk.21.ffn_down.weight	Block 21 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
244	blk.21.ffn_gate.weight	Block 21 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
245	blk.21.ffn_up.weight	Block 21 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
246	blk.21.inp_gate.weight	Block 21 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
247	blk.21.proj.weight	Block 21 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
248	blk.21.post_attention_norm.weight	Block 21 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
249	blk.21.post_ffw_norm.weight	Block 21 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
250	blk.21.post_norm.weight	Block 21 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
251	blk.21.ffn_norm.weight	Block 21 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
252	blk.21.attn_k_norm.weight	Block 21 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
253	blk.21.attn_k.weight	Block 21 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
254	blk.21.attn_output.weight	Block 21 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
255	blk.21.attn_q_norm.weight	Block 21 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
256	blk.21.attn_q.weight	Block 21 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
257	blk.21.attn_v.weight	Block 21 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.21: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.21: 16.0023 bits

Block 22 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
258	blk.22.attn_norm.weight	Block 22 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
259	blk.22.layer_output_scale.weight	Block 22 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
260	blk.22.ffn_down.weight	Block 22 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
261	blk.22.ffn_gate.weight	Block 22 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
262	blk.22.ffn_up.weight	Block 22 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
263	blk.22.inp_gate.weight	Block 22 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
264	blk.22.proj.weight	Block 22 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
265	blk.22.post_attention_norm.weight	Block 22 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
266	blk.22.post_ffw_norm.weight	Block 22 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
267	blk.22.post_norm.weight	Block 22 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
268	blk.22.ffn_norm.weight	Block 22 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
269	blk.22.attn_k_norm.weight	Block 22 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
270	blk.22.attn_k.weight	Block 22 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
271	blk.22.attn_output.weight	Block 22 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
272	blk.22.attn_q_norm.weight	Block 22 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
273	blk.22.attn_q.weight	Block 22 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
274	blk.22.attn_v.weight	Block 22 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.22: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.22: 16.0023 bits

Block 23 Tensor Group : ~106M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
275	blk.23.attn_norm.weight	Block 23 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
276	blk.23.layer_output_scale.weight	Block 23 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
277	blk.23.ffn_down.weight	Block 23 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
278	blk.23.ffn_gate.weight	Block 23 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
279	blk.23.ffn_up.weight	Block 23 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
280	blk.23.inp_gate.weight	Block 23 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
281	blk.23.proj.weight	Block 23 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
282	blk.23.post_attention_norm.weight	Block 23 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
283	blk.23.post_ffw_norm.weight	Block 23 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
284	blk.23.post_norm.weight	Block 23 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
285	blk.23.ffn_norm.weight	Block 23 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
286	blk.23.attn_k_norm.weight	Block 23 Attn_K_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
287	blk.23.attn_k.weight	Block 23 Attention Key (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	F16	16.0000
288	blk.23.attn_output.weight	Block 23 Attention Output (W)	( ~10M) 10485760	4096 x 2560 x 1 x 1	F16	16.0000
289	blk.23.attn_q_norm.weight	Block 23 Attn_Q_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
290	blk.23.attn_q.weight	Block 23 Attention Query (W)	( ~10M) 10485760	2560 x 4096 x 1 x 1	F16	16.0000
291	blk.23.attn_v.weight	Block 23 Attention Value (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	F16	16.0000

Total elements in blk.23: (~106M) 106182145
Percentage of total elements: 1.41%
Bits per Weight (BPW) for blk.23: 16.0021 bits

Block 24 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
292	blk.24.attn_norm.weight	Block 24 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
293	blk.24.layer_output_scale.weight	Block 24 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
294	blk.24.ffn_down.weight	Block 24 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
295	blk.24.ffn_gate.weight	Block 24 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
296	blk.24.ffn_up.weight	Block 24 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
297	blk.24.inp_gate.weight	Block 24 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
298	blk.24.proj.weight	Block 24 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
299	blk.24.post_attention_norm.weight	Block 24 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
300	blk.24.post_ffw_norm.weight	Block 24 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
301	blk.24.post_norm.weight	Block 24 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
302	blk.24.ffn_norm.weight	Block 24 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
303	blk.24.attn_k_norm.weight	Block 24 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
304	blk.24.attn_k.weight	Block 24 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
305	blk.24.attn_output.weight	Block 24 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
306	blk.24.attn_q_norm.weight	Block 24 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
307	blk.24.attn_q.weight	Block 24 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
308	blk.24.attn_v.weight	Block 24 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.24: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.24: 16.0023 bits

Block 25 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
309	blk.25.attn_norm.weight	Block 25 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
310	blk.25.layer_output_scale.weight	Block 25 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
311	blk.25.ffn_down.weight	Block 25 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
312	blk.25.ffn_gate.weight	Block 25 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
313	blk.25.ffn_up.weight	Block 25 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
314	blk.25.inp_gate.weight	Block 25 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
315	blk.25.proj.weight	Block 25 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
316	blk.25.post_attention_norm.weight	Block 25 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
317	blk.25.post_ffw_norm.weight	Block 25 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
318	blk.25.post_norm.weight	Block 25 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
319	blk.25.ffn_norm.weight	Block 25 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
320	blk.25.attn_k_norm.weight	Block 25 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
321	blk.25.attn_k.weight	Block 25 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
322	blk.25.attn_output.weight	Block 25 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
323	blk.25.attn_q_norm.weight	Block 25 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
324	blk.25.attn_q.weight	Block 25 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
325	blk.25.attn_v.weight	Block 25 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.25: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.25: 16.0023 bits

Block 26 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
326	blk.26.attn_norm.weight	Block 26 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
327	blk.26.layer_output_scale.weight	Block 26 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
328	blk.26.ffn_down.weight	Block 26 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
329	blk.26.ffn_gate.weight	Block 26 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
330	blk.26.ffn_up.weight	Block 26 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
331	blk.26.inp_gate.weight	Block 26 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
332	blk.26.proj.weight	Block 26 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
333	blk.26.post_attention_norm.weight	Block 26 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
334	blk.26.post_ffw_norm.weight	Block 26 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
335	blk.26.post_norm.weight	Block 26 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
336	blk.26.ffn_norm.weight	Block 26 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
337	blk.26.attn_k_norm.weight	Block 26 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
338	blk.26.attn_k.weight	Block 26 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
339	blk.26.attn_output.weight	Block 26 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
340	blk.26.attn_q_norm.weight	Block 26 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
341	blk.26.attn_q.weight	Block 26 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
342	blk.26.attn_v.weight	Block 26 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.26: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.26: 16.0023 bits

Block 27 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
343	blk.27.attn_norm.weight	Block 27 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
344	blk.27.layer_output_scale.weight	Block 27 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
345	blk.27.ffn_down.weight	Block 27 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
346	blk.27.ffn_gate.weight	Block 27 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
347	blk.27.ffn_up.weight	Block 27 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
348	blk.27.inp_gate.weight	Block 27 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
349	blk.27.proj.weight	Block 27 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
350	blk.27.post_attention_norm.weight	Block 27 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
351	blk.27.post_ffw_norm.weight	Block 27 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
352	blk.27.post_norm.weight	Block 27 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
353	blk.27.ffn_norm.weight	Block 27 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
354	blk.27.attn_k_norm.weight	Block 27 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
355	blk.27.attn_k.weight	Block 27 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
356	blk.27.attn_output.weight	Block 27 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
357	blk.27.attn_q_norm.weight	Block 27 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
358	blk.27.attn_q.weight	Block 27 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
359	blk.27.attn_v.weight	Block 27 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.27: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.27: 16.0023 bits

Block 28 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
360	blk.28.attn_norm.weight	Block 28 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
361	blk.28.layer_output_scale.weight	Block 28 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
362	blk.28.ffn_down.weight	Block 28 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
363	blk.28.ffn_gate.weight	Block 28 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
364	blk.28.ffn_up.weight	Block 28 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
365	blk.28.inp_gate.weight	Block 28 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
366	blk.28.proj.weight	Block 28 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
367	blk.28.post_attention_norm.weight	Block 28 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
368	blk.28.post_ffw_norm.weight	Block 28 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
369	blk.28.post_norm.weight	Block 28 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
370	blk.28.ffn_norm.weight	Block 28 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
371	blk.28.attn_k_norm.weight	Block 28 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
372	blk.28.attn_k.weight	Block 28 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
373	blk.28.attn_output.weight	Block 28 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
374	blk.28.attn_q_norm.weight	Block 28 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
375	blk.28.attn_q.weight	Block 28 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
376	blk.28.attn_v.weight	Block 28 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.28: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.28: 16.0023 bits

Block 29 Tensor Group : ~106M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
377	blk.29.attn_norm.weight	Block 29 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
378	blk.29.layer_output_scale.weight	Block 29 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
379	blk.29.ffn_down.weight	Block 29 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
380	blk.29.ffn_gate.weight	Block 29 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
381	blk.29.ffn_up.weight	Block 29 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
382	blk.29.inp_gate.weight	Block 29 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
383	blk.29.proj.weight	Block 29 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
384	blk.29.post_attention_norm.weight	Block 29 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
385	blk.29.post_ffw_norm.weight	Block 29 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
386	blk.29.post_norm.weight	Block 29 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
387	blk.29.ffn_norm.weight	Block 29 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
388	blk.29.attn_k_norm.weight	Block 29 Attn_K_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
389	blk.29.attn_k.weight	Block 29 Attention Key (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	F16	16.0000
390	blk.29.attn_output.weight	Block 29 Attention Output (W)	( ~10M) 10485760	4096 x 2560 x 1 x 1	F16	16.0000
391	blk.29.attn_q_norm.weight	Block 29 Attn_Q_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
392	blk.29.attn_q.weight	Block 29 Attention Query (W)	( ~10M) 10485760	2560 x 4096 x 1 x 1	F16	16.0000
393	blk.29.attn_v.weight	Block 29 Attention Value (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	F16	16.0000

Total elements in blk.29: (~106M) 106182145
Percentage of total elements: 1.41%
Bits per Weight (BPW) for blk.29: 16.0021 bits

Block 3 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
394	blk.3.attn_norm.weight	Block 3 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
395	blk.3.layer_output_scale.weight	Block 3 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
396	blk.3.ffn_down.weight	Block 3 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
397	blk.3.ffn_gate.weight	Block 3 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
398	blk.3.ffn_up.weight	Block 3 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
399	blk.3.inp_gate.weight	Block 3 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
400	blk.3.proj.weight	Block 3 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
401	blk.3.post_attention_norm.weight	Block 3 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
402	blk.3.post_ffw_norm.weight	Block 3 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
403	blk.3.post_norm.weight	Block 3 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
404	blk.3.ffn_norm.weight	Block 3 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
405	blk.3.attn_k_norm.weight	Block 3 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
406	blk.3.attn_k.weight	Block 3 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
407	blk.3.attn_output.weight	Block 3 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
408	blk.3.attn_q_norm.weight	Block 3 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
409	blk.3.attn_q.weight	Block 3 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
410	blk.3.attn_v.weight	Block 3 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.3: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.3: 16.0023 bits

Block 30 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
411	blk.30.attn_norm.weight	Block 30 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
412	blk.30.layer_output_scale.weight	Block 30 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
413	blk.30.ffn_down.weight	Block 30 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
414	blk.30.ffn_gate.weight	Block 30 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
415	blk.30.ffn_up.weight	Block 30 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
416	blk.30.inp_gate.weight	Block 30 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
417	blk.30.proj.weight	Block 30 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
418	blk.30.post_attention_norm.weight	Block 30 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
419	blk.30.post_ffw_norm.weight	Block 30 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
420	blk.30.post_norm.weight	Block 30 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
421	blk.30.ffn_norm.weight	Block 30 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
422	blk.30.attn_k_norm.weight	Block 30 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
423	blk.30.attn_k.weight	Block 30 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
424	blk.30.attn_output.weight	Block 30 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
425	blk.30.attn_q_norm.weight	Block 30 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
426	blk.30.attn_q.weight	Block 30 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
427	blk.30.attn_v.weight	Block 30 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.30: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.30: 16.0023 bits

Block 31 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
428	blk.31.attn_norm.weight	Block 31 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
429	blk.31.layer_output_scale.weight	Block 31 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
430	blk.31.ffn_down.weight	Block 31 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
431	blk.31.ffn_gate.weight	Block 31 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
432	blk.31.ffn_up.weight	Block 31 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
433	blk.31.inp_gate.weight	Block 31 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
434	blk.31.proj.weight	Block 31 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
435	blk.31.post_attention_norm.weight	Block 31 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
436	blk.31.post_ffw_norm.weight	Block 31 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
437	blk.31.post_norm.weight	Block 31 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
438	blk.31.ffn_norm.weight	Block 31 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
439	blk.31.attn_k_norm.weight	Block 31 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
440	blk.31.attn_k.weight	Block 31 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
441	blk.31.attn_output.weight	Block 31 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
442	blk.31.attn_q_norm.weight	Block 31 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
443	blk.31.attn_q.weight	Block 31 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
444	blk.31.attn_v.weight	Block 31 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.31: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.31: 16.0023 bits

Block 32 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
445	blk.32.attn_norm.weight	Block 32 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
446	blk.32.layer_output_scale.weight	Block 32 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
447	blk.32.ffn_down.weight	Block 32 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
448	blk.32.ffn_gate.weight	Block 32 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
449	blk.32.ffn_up.weight	Block 32 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
450	blk.32.inp_gate.weight	Block 32 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
451	blk.32.proj.weight	Block 32 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
452	blk.32.post_attention_norm.weight	Block 32 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
453	blk.32.post_ffw_norm.weight	Block 32 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
454	blk.32.post_norm.weight	Block 32 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
455	blk.32.ffn_norm.weight	Block 32 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
456	blk.32.attn_k_norm.weight	Block 32 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
457	blk.32.attn_k.weight	Block 32 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
458	blk.32.attn_output.weight	Block 32 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
459	blk.32.attn_q_norm.weight	Block 32 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
460	blk.32.attn_q.weight	Block 32 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
461	blk.32.attn_v.weight	Block 32 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.32: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.32: 16.0023 bits

Block 33 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
462	blk.33.attn_norm.weight	Block 33 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
463	blk.33.layer_output_scale.weight	Block 33 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
464	blk.33.ffn_down.weight	Block 33 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
465	blk.33.ffn_gate.weight	Block 33 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
466	blk.33.ffn_up.weight	Block 33 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
467	blk.33.inp_gate.weight	Block 33 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
468	blk.33.proj.weight	Block 33 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
469	blk.33.post_attention_norm.weight	Block 33 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
470	blk.33.post_ffw_norm.weight	Block 33 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
471	blk.33.post_norm.weight	Block 33 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
472	blk.33.ffn_norm.weight	Block 33 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
473	blk.33.attn_k_norm.weight	Block 33 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
474	blk.33.attn_k.weight	Block 33 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
475	blk.33.attn_output.weight	Block 33 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
476	blk.33.attn_q_norm.weight	Block 33 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
477	blk.33.attn_q.weight	Block 33 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
478	blk.33.attn_v.weight	Block 33 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.33: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.33: 16.0023 bits

Block 34 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
479	blk.34.attn_norm.weight	Block 34 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
480	blk.34.layer_output_scale.weight	Block 34 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
481	blk.34.ffn_down.weight	Block 34 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
482	blk.34.ffn_gate.weight	Block 34 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
483	blk.34.ffn_up.weight	Block 34 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
484	blk.34.inp_gate.weight	Block 34 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
485	blk.34.proj.weight	Block 34 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
486	blk.34.post_attention_norm.weight	Block 34 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
487	blk.34.post_ffw_norm.weight	Block 34 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
488	blk.34.post_norm.weight	Block 34 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
489	blk.34.ffn_norm.weight	Block 34 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
490	blk.34.attn_k_norm.weight	Block 34 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
491	blk.34.attn_k.weight	Block 34 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
492	blk.34.attn_output.weight	Block 34 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
493	blk.34.attn_q_norm.weight	Block 34 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
494	blk.34.attn_q.weight	Block 34 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
495	blk.34.attn_v.weight	Block 34 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.34: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.34: 16.0023 bits

Block 35 Tensor Group : ~106M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
496	blk.35.attn_norm.weight	Block 35 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
497	blk.35.layer_output_scale.weight	Block 35 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
498	blk.35.ffn_down.weight	Block 35 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
499	blk.35.ffn_gate.weight	Block 35 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
500	blk.35.ffn_up.weight	Block 35 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
501	blk.35.inp_gate.weight	Block 35 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
502	blk.35.proj.weight	Block 35 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
503	blk.35.post_attention_norm.weight	Block 35 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
504	blk.35.post_ffw_norm.weight	Block 35 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
505	blk.35.post_norm.weight	Block 35 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
506	blk.35.ffn_norm.weight	Block 35 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
507	blk.35.attn_k_norm.weight	Block 35 Attn_K_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
508	blk.35.attn_k.weight	Block 35 Attention Key (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	F16	16.0000
509	blk.35.attn_output.weight	Block 35 Attention Output (W)	( ~10M) 10485760	4096 x 2560 x 1 x 1	F16	16.0000
510	blk.35.attn_q_norm.weight	Block 35 Attn_Q_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
511	blk.35.attn_q.weight	Block 35 Attention Query (W)	( ~10M) 10485760	2560 x 4096 x 1 x 1	F16	16.0000
512	blk.35.attn_v.weight	Block 35 Attention Value (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	F16	16.0000

Total elements in blk.35: (~106M) 106182145
Percentage of total elements: 1.41%
Bits per Weight (BPW) for blk.35: 16.0021 bits

Block 36 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
513	blk.36.attn_norm.weight	Block 36 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
514	blk.36.layer_output_scale.weight	Block 36 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
515	blk.36.ffn_down.weight	Block 36 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
516	blk.36.ffn_gate.weight	Block 36 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
517	blk.36.ffn_up.weight	Block 36 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
518	blk.36.inp_gate.weight	Block 36 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
519	blk.36.proj.weight	Block 36 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
520	blk.36.post_attention_norm.weight	Block 36 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
521	blk.36.post_ffw_norm.weight	Block 36 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
522	blk.36.post_norm.weight	Block 36 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
523	blk.36.ffn_norm.weight	Block 36 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
524	blk.36.attn_k_norm.weight	Block 36 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
525	blk.36.attn_k.weight	Block 36 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
526	blk.36.attn_output.weight	Block 36 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
527	blk.36.attn_q_norm.weight	Block 36 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
528	blk.36.attn_q.weight	Block 36 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
529	blk.36.attn_v.weight	Block 36 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.36: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.36: 16.0023 bits

Block 37 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
530	blk.37.attn_norm.weight	Block 37 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
531	blk.37.layer_output_scale.weight	Block 37 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
532	blk.37.ffn_down.weight	Block 37 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
533	blk.37.ffn_gate.weight	Block 37 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
534	blk.37.ffn_up.weight	Block 37 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
535	blk.37.inp_gate.weight	Block 37 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
536	blk.37.proj.weight	Block 37 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
537	blk.37.post_attention_norm.weight	Block 37 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
538	blk.37.post_ffw_norm.weight	Block 37 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
539	blk.37.post_norm.weight	Block 37 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
540	blk.37.ffn_norm.weight	Block 37 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
541	blk.37.attn_k_norm.weight	Block 37 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
542	blk.37.attn_k.weight	Block 37 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
543	blk.37.attn_output.weight	Block 37 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
544	blk.37.attn_q_norm.weight	Block 37 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
545	blk.37.attn_q.weight	Block 37 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
546	blk.37.attn_v.weight	Block 37 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.37: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.37: 16.0023 bits

Block 38 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
547	blk.38.attn_norm.weight	Block 38 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
548	blk.38.layer_output_scale.weight	Block 38 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
549	blk.38.ffn_down.weight	Block 38 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
550	blk.38.ffn_gate.weight	Block 38 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
551	blk.38.ffn_up.weight	Block 38 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
552	blk.38.inp_gate.weight	Block 38 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
553	blk.38.proj.weight	Block 38 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
554	blk.38.post_attention_norm.weight	Block 38 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
555	blk.38.post_ffw_norm.weight	Block 38 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
556	blk.38.post_norm.weight	Block 38 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
557	blk.38.ffn_norm.weight	Block 38 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
558	blk.38.attn_k_norm.weight	Block 38 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
559	blk.38.attn_k.weight	Block 38 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
560	blk.38.attn_output.weight	Block 38 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
561	blk.38.attn_q_norm.weight	Block 38 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
562	blk.38.attn_q.weight	Block 38 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
563	blk.38.attn_v.weight	Block 38 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.38: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.38: 16.0023 bits

Block 39 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
564	blk.39.attn_norm.weight	Block 39 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
565	blk.39.layer_output_scale.weight	Block 39 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
566	blk.39.ffn_down.weight	Block 39 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
567	blk.39.ffn_gate.weight	Block 39 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
568	blk.39.ffn_up.weight	Block 39 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
569	blk.39.inp_gate.weight	Block 39 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
570	blk.39.proj.weight	Block 39 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
571	blk.39.post_attention_norm.weight	Block 39 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
572	blk.39.post_ffw_norm.weight	Block 39 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
573	blk.39.post_norm.weight	Block 39 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
574	blk.39.ffn_norm.weight	Block 39 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
575	blk.39.attn_k_norm.weight	Block 39 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
576	blk.39.attn_k.weight	Block 39 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
577	blk.39.attn_output.weight	Block 39 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
578	blk.39.attn_q_norm.weight	Block 39 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
579	blk.39.attn_q.weight	Block 39 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
580	blk.39.attn_v.weight	Block 39 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.39: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.39: 16.0023 bits

Block 4 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
581	blk.4.attn_norm.weight	Block 4 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
582	blk.4.layer_output_scale.weight	Block 4 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
583	blk.4.ffn_down.weight	Block 4 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
584	blk.4.ffn_gate.weight	Block 4 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
585	blk.4.ffn_up.weight	Block 4 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
586	blk.4.inp_gate.weight	Block 4 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
587	blk.4.proj.weight	Block 4 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
588	blk.4.post_attention_norm.weight	Block 4 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
589	blk.4.post_ffw_norm.weight	Block 4 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
590	blk.4.post_norm.weight	Block 4 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
591	blk.4.ffn_norm.weight	Block 4 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
592	blk.4.attn_k_norm.weight	Block 4 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
593	blk.4.attn_k.weight	Block 4 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
594	blk.4.attn_output.weight	Block 4 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
595	blk.4.attn_q_norm.weight	Block 4 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
596	blk.4.attn_q.weight	Block 4 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
597	blk.4.attn_v.weight	Block 4 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.4: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.4: 16.0023 bits

Block 40 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
598	blk.40.attn_norm.weight	Block 40 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
599	blk.40.layer_output_scale.weight	Block 40 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
600	blk.40.ffn_down.weight	Block 40 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
601	blk.40.ffn_gate.weight	Block 40 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
602	blk.40.ffn_up.weight	Block 40 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
603	blk.40.inp_gate.weight	Block 40 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
604	blk.40.proj.weight	Block 40 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
605	blk.40.post_attention_norm.weight	Block 40 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
606	blk.40.post_ffw_norm.weight	Block 40 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
607	blk.40.post_norm.weight	Block 40 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
608	blk.40.ffn_norm.weight	Block 40 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
609	blk.40.attn_k_norm.weight	Block 40 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
610	blk.40.attn_k.weight	Block 40 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
611	blk.40.attn_output.weight	Block 40 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
612	blk.40.attn_q_norm.weight	Block 40 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
613	blk.40.attn_q.weight	Block 40 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
614	blk.40.attn_v.weight	Block 40 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.40: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.40: 16.0023 bits

Block 41 Tensor Group : ~106M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
615	blk.41.attn_norm.weight	Block 41 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
616	blk.41.layer_output_scale.weight	Block 41 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
617	blk.41.ffn_down.weight	Block 41 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
618	blk.41.ffn_gate.weight	Block 41 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
619	blk.41.ffn_up.weight	Block 41 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
620	blk.41.inp_gate.weight	Block 41 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
621	blk.41.proj.weight	Block 41 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
622	blk.41.post_attention_norm.weight	Block 41 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
623	blk.41.post_ffw_norm.weight	Block 41 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
624	blk.41.post_norm.weight	Block 41 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
625	blk.41.ffn_norm.weight	Block 41 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
626	blk.41.attn_k_norm.weight	Block 41 Attn_K_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
627	blk.41.attn_k.weight	Block 41 Attention Key (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	F16	16.0000
628	blk.41.attn_output.weight	Block 41 Attention Output (W)	( ~10M) 10485760	4096 x 2560 x 1 x 1	F16	16.0000
629	blk.41.attn_q_norm.weight	Block 41 Attn_Q_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
630	blk.41.attn_q.weight	Block 41 Attention Query (W)	( ~10M) 10485760	2560 x 4096 x 1 x 1	F16	16.0000
631	blk.41.attn_v.weight	Block 41 Attention Value (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	F16	16.0000

Total elements in blk.41: (~106M) 106182145
Percentage of total elements: 1.41%
Bits per Weight (BPW) for blk.41: 16.0021 bits

Block 5 Tensor Group : ~106M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
632	blk.5.attn_norm.weight	Block 5 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
633	blk.5.layer_output_scale.weight	Block 5 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
634	blk.5.ffn_down.weight	Block 5 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
635	blk.5.ffn_gate.weight	Block 5 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
636	blk.5.ffn_up.weight	Block 5 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
637	blk.5.inp_gate.weight	Block 5 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
638	blk.5.proj.weight	Block 5 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
639	blk.5.post_attention_norm.weight	Block 5 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
640	blk.5.post_ffw_norm.weight	Block 5 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
641	blk.5.post_norm.weight	Block 5 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
642	blk.5.ffn_norm.weight	Block 5 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
643	blk.5.attn_k_norm.weight	Block 5 Attn_K_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
644	blk.5.attn_k.weight	Block 5 Attention Key (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	F16	16.0000
645	blk.5.attn_output.weight	Block 5 Attention Output (W)	( ~10M) 10485760	4096 x 2560 x 1 x 1	F16	16.0000
646	blk.5.attn_q_norm.weight	Block 5 Attn_Q_Norm (W)	( 512) 512	512 x 1 x 1 x 1	F32	32.0000
647	blk.5.attn_q.weight	Block 5 Attention Query (W)	( ~10M) 10485760	2560 x 4096 x 1 x 1	F16	16.0000
648	blk.5.attn_v.weight	Block 5 Attention Value (W)	( ~3M) 2621440	2560 x 1024 x 1 x 1	F16	16.0000

Total elements in blk.5: (~106M) 106182145
Percentage of total elements: 1.41%
Bits per Weight (BPW) for blk.5: 16.0021 bits

Block 6 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
649	blk.6.attn_norm.weight	Block 6 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
650	blk.6.layer_output_scale.weight	Block 6 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
651	blk.6.ffn_down.weight	Block 6 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
652	blk.6.ffn_gate.weight	Block 6 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
653	blk.6.ffn_up.weight	Block 6 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
654	blk.6.inp_gate.weight	Block 6 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
655	blk.6.proj.weight	Block 6 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
656	blk.6.post_attention_norm.weight	Block 6 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
657	blk.6.post_ffw_norm.weight	Block 6 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
658	blk.6.post_norm.weight	Block 6 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
659	blk.6.ffn_norm.weight	Block 6 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
660	blk.6.attn_k_norm.weight	Block 6 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
661	blk.6.attn_k.weight	Block 6 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
662	blk.6.attn_output.weight	Block 6 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
663	blk.6.attn_q_norm.weight	Block 6 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
664	blk.6.attn_q.weight	Block 6 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
665	blk.6.attn_v.weight	Block 6 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.6: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.6: 16.0023 bits

Block 7 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
666	blk.7.attn_norm.weight	Block 7 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
667	blk.7.layer_output_scale.weight	Block 7 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
668	blk.7.ffn_down.weight	Block 7 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
669	blk.7.ffn_gate.weight	Block 7 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
670	blk.7.ffn_up.weight	Block 7 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
671	blk.7.inp_gate.weight	Block 7 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
672	blk.7.proj.weight	Block 7 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
673	blk.7.post_attention_norm.weight	Block 7 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
674	blk.7.post_ffw_norm.weight	Block 7 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
675	blk.7.post_norm.weight	Block 7 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
676	blk.7.ffn_norm.weight	Block 7 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
677	blk.7.attn_k_norm.weight	Block 7 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
678	blk.7.attn_k.weight	Block 7 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
679	blk.7.attn_output.weight	Block 7 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
680	blk.7.attn_q_norm.weight	Block 7 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
681	blk.7.attn_q.weight	Block 7 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
682	blk.7.attn_v.weight	Block 7 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.7: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.7: 16.0023 bits

Block 8 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
683	blk.8.attn_norm.weight	Block 8 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
684	blk.8.layer_output_scale.weight	Block 8 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
685	blk.8.ffn_down.weight	Block 8 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
686	blk.8.ffn_gate.weight	Block 8 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
687	blk.8.ffn_up.weight	Block 8 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
688	blk.8.inp_gate.weight	Block 8 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
689	blk.8.proj.weight	Block 8 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
690	blk.8.post_attention_norm.weight	Block 8 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
691	blk.8.post_ffw_norm.weight	Block 8 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
692	blk.8.post_norm.weight	Block 8 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
693	blk.8.ffn_norm.weight	Block 8 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
694	blk.8.attn_k_norm.weight	Block 8 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
695	blk.8.attn_k.weight	Block 8 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
696	blk.8.attn_output.weight	Block 8 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
697	blk.8.attn_q_norm.weight	Block 8 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
698	blk.8.attn_q.weight	Block 8 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
699	blk.8.attn_v.weight	Block 8 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.8: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.8: 16.0023 bits

Block 9 Tensor Group : ~93M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type	BPW
700	blk.9.attn_norm.weight	Block 9 Attention Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
701	blk.9.layer_output_scale.weight	Block 9 Layer_Output_Scale (W)	( 1) 1	1 x 1 x 1 x 1	F32	32.0000
702	blk.9.ffn_down.weight	Block 9 Feed-Forward Network "Down" (W)	( ~26M) 26214400	10240 x 2560 x 1 x 1	F16	16.0000
703	blk.9.ffn_gate.weight	Block 9 Feed-Forward Network "Gate" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
704	blk.9.ffn_up.weight	Block 9 Feed-Forward Network "Up" (W)	( ~26M) 26214400	2560 x 10240 x 1 x 1	F16	16.0000
705	blk.9.inp_gate.weight	Block 9 Inp_Gate (W)	(~655K) 655360	2560 x 256 x 1 x 1	F16	16.0000
706	blk.9.proj.weight	Block 9 Proj (W)	(~655K) 655360	256 x 2560 x 1 x 1	F16	16.0000
707	blk.9.post_attention_norm.weight	Block 9 Post_Attention_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
708	blk.9.post_ffw_norm.weight	Block 9 Post_Ffw_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
709	blk.9.post_norm.weight	Block 9 Post_Norm (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
710	blk.9.ffn_norm.weight	Block 9 Feed-Forward Network Normalization (W)	( ~3K) 2560	2560 x 1 x 1 x 1	F32	32.0000
711	blk.9.attn_k_norm.weight	Block 9 Attn_K_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
712	blk.9.attn_k.weight	Block 9 Attention Key (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000
713	blk.9.attn_output.weight	Block 9 Attention Output (W)	( ~5M) 5242880	2048 x 2560 x 1 x 1	F16	16.0000
714	blk.9.attn_q_norm.weight	Block 9 Attn_Q_Norm (W)	( 256) 256	256 x 1 x 1 x 1	F32	32.0000
715	blk.9.attn_q.weight	Block 9 Attention Query (W)	( ~5M) 5242880	2560 x 2048 x 1 x 1	F16	16.0000
716	blk.9.attn_v.weight	Block 9 Attention Value (W)	( ~1M) 1310720	2560 x 512 x 1 x 1	F16	16.0000

Total elements in blk.9: (~93M) 93074433
Percentage of total elements: 1.24%
Bits per Weight (BPW) for blk.9: 16.0023 bits

Total BPW for gemma-4-E4B-it-F16.gguf: 16.0012 bits