Hugging Face's logo

eaddario
/
gemma-4-E4B-it-GGUF

Any-to-Any
GGUF
English
quant
experimental
target_bpw
conversational
Model card Files
xet
 Community
eaddario commited on
Commit
ad3ad13
·
verified ·
1 Parent(s): 2c7e1e2

Generate imatrix statistics

Browse files
Files changed (1) hide show
  1. scores/gemma-4-E4B-it.itx +394 -0
scores/gemma-4-E4B-it.itx ADDED
@@ -0,0 +1,394 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Computing tensor statistics for gemma-4-E4B-it-WIP/imatrix/imatrix-gemma-4-E4B-it-medium.gguf (342 tensors)
2
+
3
+ Layer | Tensor | Mean StdDev Skew Kurt H Norm | ∑ E[A²] Gain | L2 Dist PCC Cov
4
+ ---------------------------------------------------------------------------------------------------------------------------------------------------------------------
5
+ 0 | blk.0.attn_k.weight | 0.1432 6.1973 3.0547 70.7529 85.06% | 413953.1162 nan | nan nan nan
6
+ 0 | blk.0.attn_output.weight | 0.0249 0.4554 17.1194 368.2099 70.14% | 870.8786 nan | nan nan nan
7
+ 0 | blk.0.attn_q.weight | 0.1432 6.1973 3.0547 70.7529 85.06% | 413953.1162 nan | nan nan nan
8
+ 0 | blk.0.attn_v.weight | 0.1432 6.1973 3.0547 70.7529 85.06% | 413953.1162 nan | nan nan nan
9
+ 0 | blk.0.ffn_down.weight | -0.0013 0.1092 -1.5771 115.6580 73.34% | 13434.0009 nan | nan nan nan
10
+ 0 | blk.0.ffn_gate.weight | 0.0031 0.1570 1.3752 23.2950 96.02% | 844.0763 nan | nan nan nan
11
+ 0 | blk.0.ffn_up.weight | 0.0031 0.1570 1.3752 23.2950 96.02% | 844.0763 nan | nan nan nan
12
+ 0 | blk.0.inp_gate.weight | 0.1417 13.7151 9.9348 1002.0613 42.99% | 694591.8906 nan | nan nan nan
13
+ 0 | blk.0.proj.weight | -0.0741 0.6905 -6.6319 54.7073 43.67% | 228.3953 nan | nan nan nan
14
+ 1 | blk.1.attn_k.weight | -0.0597 4.3599 -29.4275 1178.2009 47.64% | 73371.6724 0.42 | 386.0879 -0.0143 -0.3856
15
+ 1 | blk.1.attn_output.weight | 0.0197 0.3899 12.7815 299.7932 83.52% | 844.0839 0.98 | 24.9238 0.1580 0.0281
16
+ 1 | blk.1.attn_q.weight | -0.0597 4.3599 -29.4275 1178.2009 47.64% | 73371.6724 0.42 | 386.0879 -0.0143 -0.3856
17
+ 1 | blk.1.attn_v.weight | -0.0597 4.3599 -29.4275 1178.2009 47.64% | 73371.6724 0.42 | 386.0879 -0.0143 -0.3856
18
+ 1 | blk.1.ffn_down.weight | 0.0017 0.1305 0.3669 70.6668 86.61% | 19145.4896 1.19 | 17.1809 0.0050 0.0001
19
+ 1 | blk.1.ffn_gate.weight | 0.0022 0.4086 3.1440 101.6843 94.45% | 2132.2496 1.59 | 21.7678 0.0511 0.0033
20
+ 1 | blk.1.ffn_up.weight | 0.0022 0.4086 3.1440 101.6843 94.45% | 2132.2496 1.59 | 21.7678 0.0511 0.0033
21
+ 1 | blk.1.inp_gate.weight | 0.3123 11.8598 47.5066 2335.5068 6.80% | 372764.3601 0.73 | 852.1015 0.1388 22.5826
22
+ 1 | blk.1.proj.weight | 0.0198 0.4654 7.1535 90.0353 37.12% | 117.7983 0.72 | 12.1726 0.1921 0.0617
23
+ 2 | blk.2.attn_k.weight | 0.0123 0.5668 6.6426 154.1010 90.72% | 7000.0145 0.31 | 217.7052 0.1662 0.4108
24
+ 2 | blk.2.attn_output.weight | 0.0180 0.2294 1.0125 8.1031 98.41% | 707.5469 0.92 | 20.3560 0.0127 0.0011
25
+ 2 | blk.2.attn_q.weight | 0.0123 0.5668 6.6426 154.1010 90.72% | 7000.0145 0.31 | 217.7052 0.1662 0.4108
26
+ 2 | blk.2.attn_v.weight | 0.0123 0.5668 6.6426 154.1010 90.72% | 7000.0145 0.31 | 217.7052 0.1662 0.4108
27
+ 2 | blk.2.ffn_down.weight | 0.0018 0.0666 1.3602 56.5009 87.43% | 6152.1189 0.57 | 14.8404 -0.0021 -0.0000
28
+ 2 | blk.2.ffn_gate.weight | 0.0059 0.2123 8.7630 232.8203 96.40% | 907.4754 0.65 | 18.8171 0.4251 0.0369
29
+ 2 | blk.2.ffn_up.weight | 0.0059 0.2123 8.7630 232.8203 96.40% | 907.4754 0.65 | 18.8171 0.4251 0.0369
30
+ 2 | blk.2.inp_gate.weight | 0.0647 2.4931 47.5998 2340.8467 13.41% | 17305.0377 0.22 | 474.5549 0.9971 29.4814
31
+ 2 | blk.2.proj.weight | 0.0394 0.4491 9.7082 103.9266 19.93% | 866.9649 2.71 | 9.5216 0.1543 0.0322
32
+ 3 | blk.3.attn_k.weight | 0.0165 0.4460 20.1971 559.6910 87.53% | 2715.7003 0.62 | 22.4986 0.6378 0.1612
33
+ 3 | blk.3.attn_output.weight | -0.0005 0.1318 -0.1626 0.6901 99.06% | 572.1309 0.90 | 11.8911 0.0212 0.0006
34
+ 3 | blk.3.attn_q.weight | 0.0165 0.4460 20.1971 559.6910 87.53% | 2715.7003 0.62 | 22.4986 0.6378 0.1612
35
+ 3 | blk.3.attn_v.weight | 0.0165 0.4460 20.1971 559.6910 87.53% | 2715.7003 0.62 | 22.4986 0.6378 0.1612
36
+ 3 | blk.3.ffn_down.weight | -0.0006 0.0551 1.6898 109.1707 90.21% | 3852.2003 0.79 | 8.7301 0.0039 0.0000
37
+ 3 | blk.3.ffn_gate.weight | 0.0098 0.2894 23.8671 808.5522 90.76% | 1119.8940 1.11 | 14.6005 0.3707 0.0228
38
+ 3 | blk.3.ffn_up.weight | 0.0098 0.2894 23.8671 808.5522 90.76% | 1119.8940 1.11 | 14.6005 0.3707 0.0228
39
+ 3 | blk.3.inp_gate.weight | 0.0704 2.8618 48.3802 2399.8164 20.72% | 24707.9709 1.19 | 20.9633 0.9975 7.1169
40
+ 3 | blk.3.proj.weight | 0.0012 0.4823 0.4762 124.6191 33.31% | 475.7627 0.74 | 10.5414 0.0039 0.0009
41
+ 4 | blk.4.attn_k.weight | 0.0290 0.7527 31.0894 1201.8314 76.40% | 4898.0140 1.34 | 21.3556 0.8750 0.2937
42
+ 4 | blk.4.attn_output.weight | 0.0052 0.2961 9.9373 190.0907 89.76% | 683.3545 1.09 | 14.7767 -0.0194 -0.0008
43
+ 4 | blk.4.attn_q.weight | 0.0290 0.7527 31.0894 1201.8314 76.40% | 4898.0140 1.34 | 21.3556 0.8750 0.2937
44
+ 4 | blk.4.attn_v.weight | 0.0290 0.7527 31.0894 1201.8314 76.40% | 4898.0140 1.34 | 21.3556 0.8750 0.2937
45
+ 4 | blk.4.ffn_down.weight | 0.0007 0.0636 0.9346 70.0188 85.27% | 2459.1311 0.80 | 8.4380 0.0173 0.0001
46
+ 4 | blk.4.ffn_gate.weight | 0.0068 0.2279 11.1322 390.1304 95.04% | 901.3311 0.90 | 10.0672 0.7286 0.0480
47
+ 4 | blk.4.ffn_up.weight | 0.0068 0.2279 11.1322 390.1304 95.04% | 901.3311 0.90 | 10.0672 0.7286 0.0480
48
+ 4 | blk.4.inp_gate.weight | 0.0590 3.3217 48.5557 2419.2390 21.85% | 33602.8513 1.17 | 32.6411 0.9892 9.4039
49
+ 4 | blk.4.proj.weight | 0.4371 7.0170 15.9042 250.9642 1.31% | 14954.9509 5.61 | 112.8899 -0.0178 -0.0603
50
+ 5 | blk.5.attn_k.weight | 0.0211 0.4738 17.4190 494.7516 87.43% | 3279.4378 0.82 | 23.8141 0.7985 0.2847
51
+ 5 | blk.5.attn_output.weight | 0.0089 0.2954 0.5311 11.7631 95.41% | 1088.1910 nan | nan nan nan
52
+ 5 | blk.5.attn_q.weight | 0.0211 0.4738 17.4190 494.7516 87.43% | 3279.4378 0.82 | 23.8141 0.7985 0.2847
53
+ 5 | blk.5.attn_v.weight | 0.0211 0.4738 17.4190 494.7516 87.43% | 3279.4378 0.82 | 23.8141 0.7985 0.2847
54
+ 5 | blk.5.ffn_down.weight | 0.0005 0.1442 46.4861 3513.3574 78.95% | 14813.6709 2.45 | 15.8844 0.0113 0.0001
55
+ 5 | blk.5.ffn_gate.weight | 0.0107 0.9526 -3.6917 958.1157 64.96% | 4477.8211 2.23 | 46.3657 0.2756 0.0598
56
+ 5 | blk.5.ffn_up.weight | 0.0107 0.9526 -3.6917 958.1157 64.96% | 4477.8211 2.23 | 46.3657 0.2756 0.0598
57
+ 5 | blk.5.inp_gate.weight | 0.0778 3.5113 49.4751 2481.3577 21.08% | 37374.0243 1.05 | 31.7822 0.9846 11.4845
58
+ 5 | blk.5.proj.weight | 0.0713 3.2987 7.6649 151.4547 18.05% | 3854.2945 0.51 | 124.3308 -0.0028 -0.0648
59
+ 6 | blk.6.attn_k.weight | 0.0156 0.4090 2.4658 108.4221 90.75% | 4248.8560 1.14 | 26.3815 0.3094 0.0600
60
+ 6 | blk.6.attn_output.weight | -0.0066 0.4143 4.6161 148.9217 80.60% | 780.1948 nan | nan nan nan
61
+ 6 | blk.6.attn_q.weight | 0.0156 0.4090 2.4658 108.4221 90.75% | 4248.8560 1.14 | 26.3815 0.3094 0.0600
62
+ 6 | blk.6.attn_v.weight | 0.0156 0.4090 2.4658 108.4221 90.75% | 4248.8560 1.14 | 26.3815 0.3094 0.0600
63
+ 6 | blk.6.ffn_down.weight | 0.0000 0.0732 17.2597 1151.6543 79.87% | 3328.9393 0.47 | 16.3301 0.0059 0.0001
64
+ 6 | blk.6.ffn_gate.weight | 0.0065 0.2673 4.7995 110.3614 97.09% | 1476.9145 0.57 | 44.4375 0.4076 0.1038
65
+ 6 | blk.6.ffn_up.weight | 0.0065 0.2673 4.7995 110.3614 97.09% | 1476.9145 0.57 | 44.4375 0.4076 0.1038
66
+ 6 | blk.6.inp_gate.weight | 0.0953 4.2027 49.6888 2495.2197 14.62% | 50247.4849 1.16 | 40.2136 0.9948 14.6803
67
+ 6 | blk.6.proj.weight | -0.2081 3.1627 -15.7270 247.1759 8.94% | 3268.7248 0.92 | 73.5202 -0.0073 -0.0759
68
+ 7 | blk.7.attn_k.weight | 0.0189 0.6905 6.2355 156.7712 92.31% | 10401.8566 1.56 | 29.2737 0.5477 0.1547
69
+ 7 | blk.7.attn_output.weight | 0.0057 0.2263 7.1240 113.9073 95.53% | 616.6492 0.89 | 21.5047 -0.0151 -0.0014
70
+ 7 | blk.7.attn_q.weight | 0.0189 0.6905 6.2355 156.7712 92.31% | 10401.8566 1.56 | 29.2737 0.5477 0.1547
71
+ 7 | blk.7.attn_v.weight | 0.0189 0.6905 6.2355 156.7712 92.31% | 10401.8566 1.56 | 29.2737 0.5477 0.1547
72
+ 7 | blk.7.ffn_down.weight | -0.0002 0.0459 -1.2193 23.0343 80.56% | 2385.7367 0.85 | 8.7782 -0.0091 -0.0000
73
+ 7 | blk.7.ffn_gate.weight | 0.0012 0.1473 -0.2812 7.5863 99.10% | 704.4833 0.69 | 11.2922 0.5506 0.0217
74
+ 7 | blk.7.ffn_up.weight | 0.0012 0.1473 -0.2812 7.5863 99.10% | 704.4833 0.69 | 11.2922 0.5506 0.0217
75
+ 7 | blk.7.inp_gate.weight | 0.0666 3.0375 49.7630 2500.8787 25.19% | 29473.9277 0.77 | 61.2948 0.9957 12.7110
76
+ 7 | blk.7.proj.weight | -0.0614 0.9310 -13.0942 191.2419 26.35% | 2379.0381 0.85 | 52.8128 -0.0007 -0.0021
77
+ 8 | blk.8.attn_k.weight | 0.0257 1.1911 24.2484 930.9362 85.70% | 17737.8759 1.31 | 42.2872 0.7277 0.5986
78
+ 8 | blk.8.attn_output.weight | 0.0014 0.1924 0.0643 1.1195 98.87% | 726.1231 1.09 | 13.5039 -0.0093 -0.0004
79
+ 8 | blk.8.attn_q.weight | 0.0257 1.1911 24.2484 930.9362 85.70% | 17737.8759 1.31 | 42.2872 0.7277 0.5986
80
+ 8 | blk.8.attn_v.weight | 0.0257 1.1911 24.2484 930.9362 85.70% | 17737.8759 1.31 | 42.2872 0.7277 0.5986
81
+ 8 | blk.8.ffn_down.weight | -0.0005 0.0582 1.0601 103.6317 87.75% | 1710.9171 0.85 | 7.5442 -0.0124 -0.0000
82
+ 8 | blk.8.ffn_gate.weight | -0.0021 0.1312 -1.1754 14.9774 98.35% | 500.3349 0.84 | 5.6421 0.6853 0.0132
83
+ 8 | blk.8.ffn_up.weight | -0.0021 0.1312 -1.1754 14.9774 98.35% | 500.3349 0.84 | 5.6421 0.6853 0.0132
84
+ 8 | blk.8.inp_gate.weight | 0.0331 4.3024 -4.1353 726.5694 55.04% | 83435.4365 1.68 | 201.0983 0.4569 5.9704
85
+ 8 | blk.8.proj.weight | -0.1080 1.7019 -14.6518 225.5258 15.27% | 973.6457 0.64 | 31.0772 -0.0022 -0.0036
86
+ 9 | blk.9.attn_k.weight | 0.0096 0.7456 14.2554 811.4245 80.13% | 4556.8578 0.51 | 39.6396 0.7663 0.6805
87
+ 9 | blk.9.attn_output.weight | 0.0097 0.1980 0.1997 0.7233 99.04% | 709.6762 0.99 | 12.7822 -0.0458 -0.0017
88
+ 9 | blk.9.attn_q.weight | 0.0096 0.7456 14.2554 811.4245 80.13% | 4556.8578 0.51 | 39.6396 0.7663 0.6805
89
+ 9 | blk.9.attn_v.weight | 0.0096 0.7456 14.2554 811.4245 80.13% | 4556.8578 0.51 | 39.6396 0.7663 0.6805
90
+ 9 | blk.9.ffn_down.weight | -0.0001 0.0415 0.1676 17.6353 73.28% | 1020.6390 0.77 | 7.2526 -0.0046 -0.0000
91
+ 9 | blk.9.ffn_gate.weight | 0.0022 0.1242 6.1006 155.6723 96.89% | 344.4768 0.83 | 6.0221 0.5674 0.0092
92
+ 9 | blk.9.ffn_up.weight | 0.0022 0.1242 6.1006 155.6723 96.89% | 344.4768 0.83 | 6.0221 0.5674 0.0092
93
+ 9 | blk.9.inp_gate.weight | -0.0368 3.3030 -44.4185 2174.3069 39.17% | 42796.4621 0.72 | 128.3876 0.8088 11.4931
94
+ 9 | blk.9.proj.weight | -0.5327 6.1029 -11.2476 125.0809 15.86% | 11740.7531 3.47 | 101.7923 -0.0074 -0.0767
95
+ 10 | blk.10.attn_k.weight | 0.0066 0.4932 23.0793 1176.1683 69.68% | 1457.0692 0.57 | 34.6817 0.4477 0.1646
96
+ 10 | blk.10.attn_output.weight | -0.0049 0.2148 -0.5424 4.1410 98.63% | 583.9176 0.91 | 13.3273 -0.0139 -0.0006
97
+ 10 | blk.10.attn_q.weight | 0.0066 0.4932 23.0793 1176.1683 69.68% | 1457.0692 0.57 | 34.6817 0.4477 0.1646
98
+ 10 | blk.10.attn_v.weight | 0.0066 0.4932 23.0793 1176.1683 69.68% | 1457.0692 0.57 | 34.6817 0.4477 0.1646
99
+ 10 | blk.10.ffn_down.weight | -0.0002 0.0209 -1.2414 27.3411 59.42% | 293.1636 0.54 | 4.7132 -0.0053 -0.0000
100
+ 10 | blk.10.ffn_gate.weight | 0.0010 0.0802 4.6823 116.7749 98.23% | 171.4422 0.71 | 4.8209 0.6416 0.0064
101
+ 10 | blk.10.ffn_up.weight | 0.0010 0.0802 4.6823 116.7749 98.23% | 171.4422 0.71 | 4.8209 0.6416 0.0064
102
+ 10 | blk.10.inp_gate.weight | -0.0586 3.2330 -47.5222 2352.4111 30.15% | 35742.3316 0.91 | 33.8693 0.9793 10.4573
103
+ 10 | blk.10.proj.weight | 0.0895 1.6987 15.5098 243.2489 9.03% | 3958.1397 0.58 | 101.6269 0.0084 0.0869
104
+ 11 | blk.11.attn_k.weight | 0.0025 0.2571 8.2842 384.2798 93.04% | 1188.5096 0.90 | 24.0287 0.3306 0.0419
105
+ 11 | blk.11.attn_output.weight | -0.0096 0.2089 -0.9036 7.4232 97.15% | 875.7404 nan | nan nan nan
106
+ 11 | blk.11.attn_q.weight | 0.0025 0.2571 8.2842 384.2798 93.04% | 1188.5096 0.90 | 24.0287 0.3306 0.0419
107
+ 11 | blk.11.attn_v.weight | 0.0025 0.2571 8.2842 384.2798 93.04% | 1188.5096 0.90 | 24.0287 0.3306 0.0419
108
+ 11 | blk.11.ffn_down.weight | 0.0002 0.0375 1.1739 45.6206 61.36% | 750.9759 1.60 | 4.3294 0.0089 0.0000
109
+ 11 | blk.11.ffn_gate.weight | 0.0018 0.1638 19.1971 702.1772 93.42% | 381.9109 1.49 | 6.3048 0.6752 0.0089
110
+ 11 | blk.11.ffn_up.weight | 0.0018 0.1638 19.1971 702.1772 93.42% | 381.9109 1.49 | 6.3048 0.6752 0.0089
111
+ 11 | blk.11.inp_gate.weight | -0.0781 4.0157 -45.6540 2226.7124 28.51% | 51526.0657 1.20 | 54.2290 0.9794 12.7151
112
+ 11 | blk.11.proj.weight | -1.6660 22.1828 -14.7687 223.5913 4.15% | 137798.0296 5.90 | 356.9707 0.0037 0.1394
113
+ 12 | blk.12.attn_k.weight | -0.0040 0.6611 -12.1533 313.6144 86.45% | 5150.9533 2.08 | 32.7433 0.2482 0.0422
114
+ 12 | blk.12.attn_output.weight | 0.0222 0.4193 8.6007 105.3485 84.15% | 904.7852 nan | nan nan nan
115
+ 12 | blk.12.attn_q.weight | -0.0040 0.6611 -12.1533 313.6144 86.45% | 5150.9533 2.08 | 32.7433 0.2482 0.0422
116
+ 12 | blk.12.attn_v.weight | -0.0040 0.6611 -12.1533 313.6144 86.45% | 5150.9533 2.08 | 32.7433 0.2482 0.0422
117
+ 12 | blk.12.ffn_down.weight | -0.0002 0.0423 -1.1648 53.9130 75.05% | 652.4520 0.93 | 5.7084 0.0053 0.0000
118
+ 12 | blk.12.ffn_gate.weight | 0.0029 0.1359 -0.5807 9.6358 98.96% | 520.9008 1.17 | 8.5985 0.3691 0.0082
119
+ 12 | blk.12.ffn_up.weight | 0.0029 0.1359 -0.5807 9.6358 98.96% | 520.9008 1.17 | 8.5985 0.3691 0.0082
120
+ 12 | blk.12.inp_gate.weight | -0.0619 3.8753 -47.6551 2360.7539 18.01% | 43417.2093 0.92 | 39.9831 0.9806 15.2601
121
+ 12 | blk.12.proj.weight | 0.5477 8.9910 15.8125 249.0729 1.81% | 23150.9028 0.41 | 383.9742 0.0047 0.9469
122
+ 13 | blk.13.attn_k.weight | 0.0052 0.6434 1.6731 145.8232 63.62% | 9962.9419 1.39 | 34.6256 0.4500 0.1914
123
+ 13 | blk.13.attn_output.weight | -0.0030 0.2573 3.7852 84.4612 92.36% | 590.4395 0.81 | 22.4768 -0.0185 -0.0020
124
+ 13 | blk.13.attn_q.weight | 0.0052 0.6434 1.6731 145.8232 63.62% | 9962.9419 1.39 | 34.6256 0.4500 0.1914
125
+ 13 | blk.13.attn_v.weight | 0.0052 0.6434 1.6731 145.8232 63.62% | 9962.9419 1.39 | 34.6256 0.4500 0.1914
126
+ 13 | blk.13.ffn_down.weight | 0.0006 0.0431 10.5200 519.4040 74.19% | 479.4971 0.86 | 6.0854 0.0095 0.0000
127
+ 13 | blk.13.ffn_gate.weight | 0.0018 0.1152 0.2551 13.3696 99.27% | 436.3253 0.92 | 4.8782 0.7170 0.0112
128
+ 13 | blk.13.ffn_up.weight | 0.0018 0.1152 0.2551 13.3696 99.27% | 436.3253 0.92 | 4.8782 0.7170 0.0112
129
+ 13 | blk.13.inp_gate.weight | -0.0704 3.7529 -48.1604 2394.3054 20.69% | 41894.8022 0.98 | 30.0611 0.9884 14.3747
130
+ 13 | blk.13.proj.weight | 0.0771 8.3846 7.8217 136.7354 15.24% | 19762.3166 0.92 | 196.9086 -0.0006 -0.0473
131
+ 14 | blk.14.attn_k.weight | -0.0040 0.7911 -1.9224 48.9330 93.71% | 12004.5224 1.10 | 39.2341 0.4309 0.2194
132
+ 14 | blk.14.attn_output.weight | 0.0191 0.3226 6.7196 93.7479 91.87% | 706.5962 1.09 | 19.6565 -0.1076 -0.0089
133
+ 14 | blk.14.attn_q.weight | -0.0040 0.7911 -1.9224 48.9330 93.71% | 12004.5224 1.10 | 39.2341 0.4309 0.2194
134
+ 14 | blk.14.attn_v.weight | -0.0040 0.7911 -1.9224 48.9330 93.71% | 12004.5224 1.10 | 39.2341 0.4309 0.2194
135
+ 14 | blk.14.ffn_down.weight | 0.0000 0.0193 1.7816 145.3103 63.85% | 96.6140 0.45 | 4.8125 -0.0157 -0.0000
136
+ 14 | blk.14.ffn_gate.weight | 0.0010 0.0681 1.4369 23.0203 99.28% | 151.1048 0.59 | 4.9446 0.5328 0.0042
137
+ 14 | blk.14.ffn_up.weight | 0.0010 0.0681 1.4369 23.0203 99.28% | 151.1048 0.59 | 4.9446 0.5328 0.0042
138
+ 14 | blk.14.inp_gate.weight | -0.0576 3.7243 -43.8551 2156.2798 25.09% | 42398.4993 1.01 | 50.5058 0.9644 13.4794
139
+ 14 | blk.14.proj.weight | 0.0160 0.7167 -2.9977 93.0625 13.45% | 4877.7195 0.50 | 130.1729 0.3851 2.3141
140
+ 15 | blk.15.attn_k.weight | -0.0123 0.6456 -2.8955 42.4079 92.77% | 8601.4774 0.85 | 35.7800 0.5312 0.2713
141
+ 15 | blk.15.attn_output.weight | 0.0086 0.4256 3.3312 132.4224 81.76% | 818.9910 1.08 | 24.1571 0.0015 0.0002
142
+ 15 | blk.15.attn_q.weight | -0.0123 0.6456 -2.8955 42.4079 92.77% | 8601.4774 0.85 | 35.7800 0.5312 0.2713
143
+ 15 | blk.15.attn_v.weight | -0.0123 0.6456 -2.8955 42.4079 92.77% | 8601.4774 0.85 | 35.7800 0.5312 0.2713
144
+ 15 | blk.15.ffn_down.weight | -0.0001 0.0204 0.0678 45.7919 68.54% | 102.3100 1.03 | 2.8601 -0.0109 -0.0000
145
+ 15 | blk.15.ffn_gate.weight | 0.0003 0.0799 -4.8878 115.2211 97.59% | 172.4628 1.07 | 3.7587 0.5055 0.0027
146
+ 15 | blk.15.ffn_up.weight | 0.0003 0.0799 -4.8878 115.2211 97.59% | 172.4628 1.07 | 3.7587 0.5055 0.0027
147
+ 15 | blk.15.inp_gate.weight | -0.0085 3.8380 -17.1029 1177.9822 37.66% | 47464.0062 1.06 | 84.0354 0.9040 12.9226
148
+ 15 | blk.15.proj.weight | -0.3291 11.1721 -10.0562 160.5682 12.74% | 33497.4007 2.62 | 183.0821 -0.3426 -2.7430
149
+ 16 | blk.16.attn_k.weight | 0.0190 1.2291 15.7845 523.7653 79.59% | 11754.0414 1.17 | 61.1025 0.2962 0.2351
150
+ 16 | blk.16.attn_output.weight | -0.0037 0.2802 -4.5911 71.4249 94.19% | 619.0127 0.87 | 23.4521 -0.0366 -0.0044
151
+ 16 | blk.16.attn_q.weight | 0.0190 1.2291 15.7845 523.7653 79.59% | 11754.0414 1.17 | 61.1025 0.2962 0.2351
152
+ 16 | blk.16.attn_v.weight | 0.0190 1.2291 15.7845 523.7653 79.59% | 11754.0414 1.17 | 61.1025 0.2962 0.2351
153
+ 16 | blk.16.ffn_down.weight | -0.0001 0.0126 0.4786 43.3025 73.50% | 30.6093 0.55 | 2.4204 0.0047 0.0000
154
+ 16 | blk.16.ffn_gate.weight | 0.0005 0.0653 -0.0054 14.0082 98.10% | 114.6125 0.82 | 3.7621 0.4903 0.0026
155
+ 16 | blk.16.ffn_up.weight | 0.0005 0.0653 -0.0054 14.0082 98.10% | 114.6125 0.82 | 3.7621 0.4903 0.0026
156
+ 16 | blk.16.inp_gate.weight | 0.0036 3.1922 14.4605 1114.7560 41.63% | 34797.0787 0.86 | 86.9400 0.8965 10.9843
157
+ 16 | blk.16.proj.weight | 0.0938 1.1348 14.3371 213.4307 8.44% | 6298.4733 0.43 | 179.8327 -0.0018 -0.0229
158
+ 17 | blk.17.attn_k.weight | 0.0095 1.0179 -2.5008 374.6284 88.56% | 10666.2648 0.95 | 52.0208 0.5954 0.7449
159
+ 17 | blk.17.attn_output.weight | -0.0115 0.2779 -8.7114 218.6682 90.61% | 1083.5076 nan | nan nan nan
160
+ 17 | blk.17.attn_q.weight | 0.0095 1.0179 -2.5008 374.6284 88.56% | 10666.2648 0.95 | 52.0208 0.5954 0.7449
161
+ 17 | blk.17.attn_v.weight | 0.0095 1.0179 -2.5008 374.6284 88.56% | 10666.2648 0.95 | 52.0208 0.5954 0.7449
162
+ 17 | blk.17.ffn_down.weight | -0.0001 0.0098 -0.3753 31.4306 77.16% | 18.1313 0.77 | 1.6134 0.0030 0.0000
163
+ 17 | blk.17.ffn_gate.weight | -0.0005 0.0635 2.5042 66.9977 97.89% | 98.5582 0.93 | 3.0259 0.5688 0.0024
164
+ 17 | blk.17.ffn_up.weight | -0.0005 0.0635 2.5042 66.9977 97.89% | 98.5582 0.93 | 3.0259 0.5688 0.0024
165
+ 17 | blk.17.inp_gate.weight | -0.0096 2.8340 7.3454 450.2230 69.38% | 37095.2485 1.03 | 119.1594 0.7006 6.3379
166
+ 17 | blk.17.proj.weight | 0.2001 6.8239 -3.6073 91.9840 22.81% | 12572.8890 1.41 | 122.7016 -0.7068 -5.4734
167
+ 18 | blk.18.attn_k.weight | 0.0243 2.8944 13.0265 606.5632 68.14% | 40983.1325 1.96 | 123.8749 0.5804 1.7098
168
+ 18 | blk.18.attn_output.weight | -0.0025 0.2548 -0.7025 6.2934 97.13% | 499.4808 nan | nan nan nan
169
+ 18 | blk.18.attn_q.weight | 0.0243 2.8944 13.0265 606.5632 68.14% | 40983.1325 1.96 | 123.8749 0.5804 1.7098
170
+ 18 | blk.18.attn_v.weight | 0.0243 2.8944 13.0265 606.5632 68.14% | 40983.1325 1.96 | 123.8749 0.5804 1.7098
171
+ 18 | blk.18.ffn_down.weight | -0.0001 0.0223 -9.0093 485.4416 84.52% | 51.0535 1.68 | 2.4579 0.0096 0.0000
172
+ 18 | blk.18.ffn_gate.weight | -0.0011 0.1257 -3.3278 211.4994 94.10% | 186.3609 1.38 | 5.3638 0.5379 0.0043
173
+ 18 | blk.18.ffn_up.weight | -0.0011 0.1257 -3.3278 211.4994 94.10% | 186.3609 1.38 | 5.3638 0.5379 0.0043
174
+ 18 | blk.18.inp_gate.weight | 0.0037 2.0983 25.1273 1170.4200 65.41% | 21578.8391 0.76 | 111.5388 0.6369 3.7875
175
+ 18 | blk.18.proj.weight | -0.1020 1.5297 -10.8858 147.4053 22.72% | 13201.0156 1.02 | 117.9793 -0.2574 -2.6874
176
+ 19 | blk.19.attn_k.weight | -0.0066 1.6508 17.2662 826.8350 69.85% | 14156.4714 0.59 | 87.0099 0.8525 4.0731
177
+ 19 | blk.19.attn_output.weight | 0.0088 0.3340 4.4298 58.5086 91.98% | 645.8673 1.14 | 19.6565 -0.0708 -0.0060
178
+ 19 | blk.19.attn_q.weight | -0.0066 1.6508 17.2662 826.8350 69.85% | 14156.4714 0.59 | 87.0099 0.8525 4.0731
179
+ 19 | blk.19.attn_v.weight | -0.0066 1.6508 17.2662 826.8350 69.85% | 14156.4714 0.59 | 87.0099 0.8525 4.0731
180
+ 19 | blk.19.ffn_down.weight | -0.0000 0.0171 0.5420 53.9450 83.69% | 39.9341 0.88 | 2.8536 -0.0082 -0.0000
181
+ 19 | blk.19.ffn_gate.weight | -0.0001 0.1122 -2.6898 67.6206 95.89% | 182.9092 0.99 | 4.9243 0.6706 0.0095
182
+ 19 | blk.19.ffn_up.weight | -0.0001 0.1122 -2.6898 67.6206 95.89% | 182.9092 0.99 | 4.9243 0.6706 0.0095
183
+ 19 | blk.19.inp_gate.weight | -0.0130 1.9287 -25.9991 1284.3584 69.30% | 21260.0594 0.99 | 100.1560 0.5195 2.1023
184
+ 19 | blk.19.proj.weight | -0.0754 1.4823 -8.3624 106.3750 37.50% | 3911.6144 0.54 | 39.2094 -0.3236 -0.7337
185
+ 20 | blk.20.attn_k.weight | -0.0068 0.5664 -6.2891 317.4878 85.82% | 2678.0337 0.43 | 71.3188 0.5664 0.5296
186
+ 20 | blk.20.attn_output.weight | 0.0149 0.2819 0.3111 4.6330 96.04% | 678.5902 1.03 | 20.1334 -0.0365 -0.0034
187
+ 20 | blk.20.attn_q.weight | -0.0068 0.5664 -6.2891 317.4878 85.82% | 2678.0337 0.43 | 71.3188 0.5664 0.5296
188
+ 20 | blk.20.attn_v.weight | -0.0068 0.5664 -6.2891 317.4878 85.82% | 2678.0337 0.43 | 71.3188 0.5664 0.5296
189
+ 20 | blk.20.ffn_down.weight | 0.0000 0.0151 -0.5080 51.4298 72.33% | 38.8302 0.99 | 2.2988 0.0050 0.0000
190
+ 20 | blk.20.ffn_gate.weight | 0.0015 0.0951 0.6099 34.7511 96.70% | 163.1943 0.94 | 4.7754 0.5967 0.0064
191
+ 20 | blk.20.ffn_up.weight | 0.0015 0.0951 0.6099 34.7511 96.70% | 163.1943 0.94 | 4.7754 0.5967 0.0064
192
+ 20 | blk.20.inp_gate.weight | -0.0626 2.8095 -30.6728 1289.4878 61.14% | 36137.6671 1.30 | 91.6103 0.7693 4.1687
193
+ 20 | blk.20.proj.weight | 0.7017 11.3296 15.8371 249.5611 9.06% | 37963.8217 3.12 | 183.2385 0.0001 0.0016
194
+ 21 | blk.21.attn_k.weight | -0.0073 0.3984 -8.9849 298.8808 86.91% | 1478.4928 0.74 | 23.9661 0.5655 0.1276
195
+ 21 | blk.21.attn_output.weight | 0.0004 0.2357 -0.3850 4.2862 98.03% | 560.6140 0.91 | 16.4768 0.0201 0.0013
196
+ 21 | blk.21.attn_q.weight | -0.0073 0.3984 -8.9849 298.8808 86.91% | 1478.4928 0.74 | 23.9661 0.5655 0.1276
197
+ 21 | blk.21.attn_v.weight | -0.0073 0.3984 -8.9849 298.8808 86.91% | 1478.4928 0.74 | 23.9661 0.5655 0.1276
198
+ 21 | blk.21.ffn_down.weight | -0.0001 0.0178 0.2287 40.1923 79.06% | 56.1921 1.20 | 2.3622 -0.0005 -0.0000
199
+ 21 | blk.21.ffn_gate.weight | -0.0002 0.1183 -1.5671 48.1133 95.74% | 216.9898 1.15 | 5.2595 0.5437 0.0061
200
+ 21 | blk.21.ffn_up.weight | -0.0002 0.1183 -1.5671 48.1133 95.74% | 216.9898 1.15 | 5.2595 0.5437 0.0061
201
+ 21 | blk.21.inp_gate.weight | -0.1013 3.4024 -33.1676 1308.4121 47.49% | 44072.6630 1.10 | 126.6059 0.6910 6.6049
202
+ 21 | blk.21.proj.weight | -0.0468 0.9122 -12.0263 187.5818 25.32% | 9866.0989 0.51 | 182.6897 -0.0301 -0.3107
203
+ 22 | blk.22.attn_k.weight | -0.0052 0.1409 -10.0954 221.0564 75.53% | 350.5319 0.49 | 15.8459 0.7171 0.0402
204
+ 22 | blk.22.attn_output.weight | 0.0137 0.1863 0.8052 8.9140 96.95% | 406.3267 0.85 | 12.9199 0.1018 0.0045
205
+ 22 | blk.22.attn_q.weight | -0.0052 0.1409 -10.0954 221.0564 75.53% | 350.5319 0.49 | 15.8459 0.7171 0.0402
206
+ 22 | blk.22.attn_v.weight | -0.0052 0.1409 -10.0954 221.0564 75.53% | 350.5319 0.49 | 15.8459 0.7171 0.0402
207
+ 22 | blk.22.ffn_down.weight | 0.0002 0.0218 57.9790 4865.4102 68.58% | 29.7151 0.73 | 2.8347 0.0124 0.0000
208
+ 22 | blk.22.ffn_gate.weight | -0.0037 0.0833 -3.8080 70.3747 96.92% | 129.9188 0.77 | 4.9607 0.5748 0.0057
209
+ 22 | blk.22.ffn_up.weight | -0.0037 0.0833 -3.8080 70.3747 96.92% | 129.9188 0.77 | 4.9607 0.5748 0.0057
210
+ 22 | blk.22.inp_gate.weight | -0.0756 5.6362 -2.6815 505.0829 60.40% | 125944.4503 1.69 | 292.5060 0.2587 4.9609
211
+ 22 | blk.22.proj.weight | -0.0287 0.7199 -13.9242 216.1355 3.64% | 14112.6564 1.20 | 17.8103 0.0849 0.0557
212
+ 23 | blk.23.attn_k.weight | -0.0100 0.3262 -5.5340 113.5953 95.02% | 2580.2972 2.71 | 16.6630 0.1938 0.0089
213
+ 23 | blk.23.attn_output.weight | -0.0006 0.1831 -0.8259 15.1741 98.40% | 1095.9179 nan | nan nan nan
214
+ 23 | blk.23.attn_q.weight | -0.0100 0.3262 -5.5340 113.5953 95.02% | 2580.2972 2.71 | 16.6630 0.1938 0.0089
215
+ 23 | blk.23.attn_v.weight | -0.0100 0.3262 -5.5340 113.5953 95.02% | 2580.2972 2.71 | 16.6630 0.1938 0.0089
216
+ 23 | blk.23.ffn_down.weight | -0.0003 0.0241 -0.1378 46.9759 94.40% | 43.2144 1.21 | 3.3522 -0.0386 -0.0000
217
+ 23 | blk.23.ffn_gate.weight | -0.0016 0.1236 0.2944 11.0305 97.57% | 202.1315 1.25 | 7.0403 0.1387 0.0014
218
+ 23 | blk.23.ffn_up.weight | -0.0016 0.1236 0.2944 11.0305 97.57% | 202.1315 1.25 | 7.0403 0.1387 0.0014
219
+ 23 | blk.23.inp_gate.weight | -0.2267 17.9635 -9.9955 564.9338 73.19% | 1582610.5247 3.54 | 946.6456 0.0218 2.2108
220
+ 23 | blk.23.proj.weight | -0.1853 3.1601 -9.3566 125.9113 20.95% | 3027.6879 0.46 | 51.8686 0.0043 0.0098
221
+ 24 | blk.24.attn_output.weight | 0.0115 0.2198 -0.0209 16.4977 93.95% | 407.7129 nan | nan nan nan
222
+ 24 | blk.24.attn_q.weight | -0.0671 2.9141 -4.3877 223.6229 74.65% | 47977.2210 4.31 | 148.0839 0.0187 0.0178
223
+ 24 | blk.24.ffn_down.weight | 0.0005 0.0244 0.2161 37.2038 92.77% | 48.6580 1.06 | 3.4714 -0.0001 -0.0000
224
+ 24 | blk.24.ffn_gate.weight | 0.0008 0.1191 0.0907 7.9563 97.71% | 205.9826 1.01 | 7.7044 0.2135 0.0031
225
+ 24 | blk.24.ffn_up.weight | 0.0008 0.1191 0.0907 7.9563 97.71% | 205.9826 1.01 | 7.7044 0.2135 0.0031
226
+ 24 | blk.24.inp_gate.weight | -0.0208 2.8950 -12.6419 615.6168 63.87% | 34355.5561 0.15 | 802.6675 0.7638 39.7198
227
+ 24 | blk.24.proj.weight | 1.1463 18.2350 15.8934 250.7356 0.29% | 98527.4086 5.70 | 296.6788 0.0039 0.2268
228
+ 25 | blk.25.attn_output.weight | 0.0171 0.2050 1.4719 12.7187 96.79% | 424.0233 1.02 | 7.0104 0.7365 0.0332
229
+ 25 | blk.25.attn_q.weight | -0.0573 3.5229 -7.8949 262.0383 71.10% | 71509.0947 1.22 | 156.7030 0.5509 5.6553
230
+ 25 | blk.25.ffn_down.weight | 0.0000 0.0251 -14.9074 885.0902 88.93% | 43.5843 0.95 | 3.5745 -0.0187 -0.0000
231
+ 25 | blk.25.ffn_gate.weight | -0.0014 0.1443 -3.9198 144.7133 93.31% | 198.5308 0.98 | 6.6953 0.5091 0.0088
232
+ 25 | blk.25.ffn_up.weight | -0.0014 0.1443 -3.9198 144.7133 93.31% | 198.5308 0.98 | 6.6953 0.5091 0.0088
233
+ 25 | blk.25.inp_gate.weight | -0.0281 2.2220 -4.0947 354.0560 62.90% | 19467.6219 0.75 | 79.5895 0.8429 5.4220
234
+ 25 | blk.25.proj.weight | 1.5859 20.5988 14.5423 217.7344 4.86% | 130524.5426 1.15 | 441.4176 -0.0055 -2.0554
235
+ 26 | blk.26.attn_output.weight | 0.0160 0.2145 1.2669 13.3630 95.40% | 395.9365 0.97 | 4.8510 0.8704 0.0383
236
+ 26 | blk.26.attn_q.weight | -0.1000 3.3011 -6.6674 226.8098 68.14% | 52860.1555 0.86 | 100.3668 0.8330 9.6877
237
+ 26 | blk.26.ffn_down.weight | 0.0002 0.0286 -10.8838 826.9594 89.13% | 57.6844 1.15 | 3.8306 0.0097 0.0000
238
+ 26 | blk.26.ffn_gate.weight | -0.0071 0.2011 -3.4508 149.4439 88.92% | 300.3366 1.23 | 5.8002 0.8297 0.0241
239
+ 26 | blk.26.ffn_up.weight | -0.0071 0.2011 -3.4508 149.4439 88.92% | 300.3366 1.23 | 5.8002 0.8297 0.0241
240
+ 26 | blk.26.inp_gate.weight | -0.0258 2.4619 -2.3582 298.6275 59.26% | 22043.1603 1.06 | 65.2799 0.8531 4.6668
241
+ 26 | blk.26.proj.weight | 0.9146 13.1989 15.8106 248.9434 2.24% | 51382.6714 0.63 | 392.4739 -0.0050 -1.3652
242
+ 27 | blk.27.attn_output.weight | 0.0149 0.2028 1.4385 13.6670 95.94% | 430.6240 1.04 | 4.1590 0.9045 0.0394
243
+ 27 | blk.27.attn_q.weight | -0.1635 3.5895 -17.4641 466.5713 57.02% | 50201.0399 0.97 | 127.3695 0.7363 8.7246
244
+ 27 | blk.27.ffn_down.weight | -0.0004 0.0272 -1.6186 79.5744 91.81% | 69.2508 1.10 | 3.9877 0.0023 0.0000
245
+ 27 | blk.27.ffn_gate.weight | -0.0016 0.1998 11.8224 423.7152 85.93% | 290.4075 0.98 | 9.8667 0.5271 0.0212
246
+ 27 | blk.27.ffn_up.weight | -0.0016 0.1998 11.8224 423.7152 85.93% | 290.4075 0.98 | 9.8667 0.5271 0.0212
247
+ 27 | blk.27.inp_gate.weight | -0.0095 2.6138 13.6002 737.3785 53.14% | 24670.7795 1.06 | 70.1409 0.8525 5.4858
248
+ 27 | blk.27.proj.weight | 1.0055 10.9920 11.1188 122.0789 15.35% | 37849.8086 0.86 | 276.2931 -0.0109 -1.5758
249
+ 28 | blk.28.attn_output.weight | 0.0146 0.2047 0.9992 12.9558 96.30% | 404.4499 0.97 | 4.6201 0.8745 0.0363
250
+ 28 | blk.28.attn_q.weight | -0.0741 2.3971 3.2279 331.6606 67.03% | 28700.2334 0.76 | 193.7590 0.2309 1.9869
251
+ 28 | blk.28.ffn_down.weight | -0.0005 0.0410 1.7742 93.4172 90.78% | 141.5363 1.43 | 4.9916 -0.0060 -0.0000
252
+ 28 | blk.28.ffn_gate.weight | -0.0040 0.1632 -1.4683 71.7490 96.11% | 347.4671 1.09 | 9.1565 0.5184 0.0169
253
+ 28 | blk.28.ffn_up.weight | -0.0040 0.1632 -1.4683 71.7490 96.11% | 347.4671 1.09 | 9.1565 0.5184 0.0169
254
+ 28 | blk.28.inp_gate.weight | 0.0434 3.0706 40.7253 1932.4574 41.98% | 33286.9508 1.16 | 85.8108 0.8340 6.6935
255
+ 28 | blk.28.proj.weight | -1.0697 14.0299 -10.8831 120.6274 17.03% | 57192.6623 1.23 | 286.1408 0.0069 1.0691
256
+ 29 | blk.29.attn_output.weight | -0.0018 0.2186 -0.3002 5.4282 94.70% | 627.6415 nan | nan nan nan
257
+ 29 | blk.29.attn_q.weight | -0.0009 1.2114 1.6213 83.6776 86.10% | 12932.2475 0.67 | 93.4549 0.6556 1.9037
258
+ 29 | blk.29.ffn_down.weight | 0.0003 0.0211 2.1326 63.8563 91.96% | 42.5352 0.55 | 4.6447 0.0127 0.0000
259
+ 29 | blk.29.ffn_gate.weight | -0.0032 0.0921 -0.1095 14.3892 97.35% | 151.4243 0.66 | 7.3467 0.4671 0.0070
260
+ 29 | blk.29.ffn_up.weight | -0.0032 0.0921 -0.1095 14.3892 97.35% | 151.4243 0.66 | 7.3467 0.4671 0.0070
261
+ 29 | blk.29.inp_gate.weight | 0.0561 4.9030 44.8215 2175.2073 35.14% | 81743.0990 1.57 | 121.6071 0.9197 13.8459
262
+ 29 | blk.29.proj.weight | 1.2928 15.6698 14.6598 221.6435 5.77% | 69998.7596 1.11 | 337.1475 0.0090 1.9716
263
+ 30 | blk.30.attn_output.weight | 0.0137 0.2341 0.9840 13.4447 94.08% | 459.2626 nan | nan nan nan
264
+ 30 | blk.30.attn_q.weight | -0.0237 1.2408 -20.1436 594.9559 71.51% | 9668.2971 0.86 | 81.2413 0.1428 0.2147
265
+ 30 | blk.30.ffn_down.weight | -0.0004 0.0337 -12.7213 532.5463 89.64% | 88.5463 1.44 | 4.0274 0.0003 0.0000
266
+ 30 | blk.30.ffn_gate.weight | -0.0004 0.0986 2.8164 66.5360 97.63% | 221.9058 1.21 | 4.7847 0.5104 0.0046
267
+ 30 | blk.30.ffn_up.weight | -0.0004 0.0986 2.8164 66.5360 97.63% | 221.9058 1.21 | 4.7847 0.5104 0.0046
268
+ 30 | blk.30.inp_gate.weight | 0.0471 3.0186 47.0673 2322.0962 33.63% | 31678.8146 0.62 | 106.7703 0.9695 14.3494
269
+ 30 | blk.30.proj.weight | 2.7003 32.4533 13.0262 176.6259 8.60% | 352836.3079 2.25 | 578.5716 -0.0067 -3.4279
270
+ 31 | blk.31.attn_output.weight | 0.0044 0.3539 -1.4473 22.4312 83.77% | 550.7249 1.10 | 10.2971 0.7746 0.0642
271
+ 31 | blk.31.attn_q.weight | 0.0230 1.7247 29.9740 1342.0023 70.25% | 19157.2613 1.41 | 115.5892 -0.1643 -0.3515
272
+ 31 | blk.31.ffn_down.weight | -0.0003 0.0364 -4.0925 988.9019 84.93% | 72.3968 0.90 | 5.0162 0.0020 0.0000
273
+ 31 | blk.31.ffn_gate.weight | -0.0019 0.0880 -2.8112 51.5003 98.39% | 211.0560 0.98 | 3.6887 0.7001 0.0061
274
+ 31 | blk.31.ffn_up.weight | -0.0019 0.0880 -2.8112 51.5003 98.39% | 211.0560 0.98 | 3.6887 0.7001 0.0061
275
+ 31 | blk.31.inp_gate.weight | 0.0474 3.1028 47.7649 2368.2239 27.32% | 31256.0760 0.99 | 19.7490 0.9922 9.2934
276
+ 31 | blk.31.proj.weight | 0.6098 23.4242 1.2467 92.5060 18.72% | 183429.7764 0.72 | 641.7915 -0.0018 -1.3439
277
+ 32 | blk.32.attn_output.weight | 0.0090 0.3056 -0.6962 19.1410 88.75% | 494.4854 0.95 | 6.9717 0.9012 0.0974
278
+ 32 | blk.32.attn_q.weight | -0.0045 1.3772 17.4332 768.6943 75.95% | 14363.3221 0.87 | 36.3069 0.9172 2.1786
279
+ 32 | blk.32.ffn_down.weight | 0.0012 0.0799 49.0121 3881.0945 73.29% | 206.2244 1.69 | 8.8640 0.0060 0.0000
280
+ 32 | blk.32.ffn_gate.weight | -0.0013 0.1116 -0.5039 8.8918 99.00% | 332.5310 1.26 | 3.0109 0.8482 0.0083
281
+ 32 | blk.32.ffn_up.weight | -0.0013 0.1116 -0.5039 8.8918 99.00% | 332.5310 1.26 | 3.0109 0.8482 0.0083
282
+ 32 | blk.32.inp_gate.weight | 0.0577 3.4260 48.5065 2417.3062 22.38% | 36135.1881 1.08 | 25.8086 0.9927 10.5522
283
+ 32 | blk.32.proj.weight | -1.7670 19.9254 -11.2042 123.7154 12.64% | 151975.0341 0.91 | 492.9240 0.0024 1.1226
284
+ 33 | blk.33.attn_output.weight | 0.0056 0.3841 -2.2858 40.9012 81.96% | 560.6254 1.06 | 8.5483 0.8743 0.1026
285
+ 33 | blk.33.attn_q.weight | 0.0019 1.1315 11.8064 429.3420 81.45% | 11380.3785 0.89 | 36.5442 0.8520 1.3277
286
+ 33 | blk.33.ffn_down.weight | -0.0005 0.0741 -10.2930 2015.8590 74.11% | 188.5063 0.96 | 11.0532 -0.0047 -0.0000
287
+ 33 | blk.33.ffn_gate.weight | -0.0019 0.1082 -0.6097 8.5248 98.99% | 327.4236 0.99 | 2.9424 0.8605 0.0104
288
+ 33 | blk.33.ffn_up.weight | -0.0019 0.1082 -0.6097 8.5248 98.99% | 327.4236 0.99 | 2.9424 0.8605 0.0104
289
+ 33 | blk.33.inp_gate.weight | 0.0660 3.6595 48.7817 2435.4797 20.55% | 40524.3480 1.06 | 21.0025 0.9953 12.4784
290
+ 33 | blk.33.proj.weight | 0.1590 17.0921 2.1837 126.5212 12.96% | 99200.1786 0.81 | 420.9830 0.0008 0.2888
291
+ 34 | blk.34.attn_output.weight | 0.0046 0.3250 -1.8721 28.2445 83.66% | 463.0119 0.91 | 9.2241 0.8476 0.1058
292
+ 34 | blk.34.attn_q.weight | -0.0510 1.8398 -18.4478 559.6673 73.10% | 22161.8976 1.40 | 106.9336 0.0483 0.1006
293
+ 34 | blk.34.ffn_down.weight | 0.0006 0.0817 4.8848 2146.7761 68.98% | 197.0414 1.02 | 11.1555 0.0012 0.0000
294
+ 34 | blk.34.ffn_gate.weight | -0.0021 0.1133 -4.5673 77.2940 98.02% | 288.0805 0.94 | 3.9503 0.7524 0.0092
295
+ 34 | blk.34.ffn_up.weight | -0.0021 0.1133 -4.5673 77.2940 98.02% | 288.0805 0.94 | 3.9503 0.7524 0.0092
296
+ 34 | blk.34.inp_gate.weight | 0.0622 3.8442 48.8433 2439.6812 19.02% | 43945.2486 1.04 | 24.2845 0.9930 13.9698
297
+ 34 | blk.34.proj.weight | 1.6208 25.8603 15.9059 250.9990 0.11% | 248972.5353 1.58 | 496.6483 -0.0005 -0.2409
298
+ 35 | blk.35.attn_output.weight | -0.0048 0.3508 -0.2302 5.6749 89.36% | 937.9399 nan | nan nan nan
299
+ 35 | blk.35.attn_q.weight | -0.0036 2.6454 30.0201 1364.5314 55.21% | 32082.0742 1.20 | 183.9270 -0.2907 -1.4147
300
+ 35 | blk.35.ffn_down.weight | 0.0000 0.0326 5.3149 981.8243 78.31% | 54.7414 0.53 | 8.8634 0.0119 0.0000
301
+ 35 | blk.35.ffn_gate.weight | -0.0011 0.0782 -6.6573 168.9560 97.23% | 139.0019 0.69 | 4.5932 0.6044 0.0054
302
+ 35 | blk.35.ffn_up.weight | -0.0011 0.0782 -6.6573 168.9560 97.23% | 139.0019 0.69 | 4.5932 0.6044 0.0054
303
+ 35 | blk.35.inp_gate.weight | 0.0685 3.9535 49.0306 2452.1257 17.77% | 46038.0880 1.02 | 22.3483 0.9940 15.1066
304
+ 35 | blk.35.proj.weight | 1.3197 14.9363 11.3417 127.8168 12.54% | 79003.1005 0.56 | 478.9923 -0.0055 -2.1429
305
+ 36 | blk.36.attn_output.weight | -0.0037 0.5127 -2.6012 32.6781 73.32% | 722.0482 nan | nan nan nan
306
+ 36 | blk.36.attn_q.weight | -0.0322 1.4680 -5.4127 163.5842 87.09% | 21842.5538 0.83 | 155.4818 -0.0372 -0.1446
307
+ 36 | blk.36.ffn_down.weight | -0.0006 0.0351 -70.6423 6068.5776 64.02% | 33.7140 0.78 | 4.8314 0.0053 0.0000
308
+ 36 | blk.36.ffn_gate.weight | -0.0016 0.0533 -0.1516 8.5997 98.78% | 97.7354 0.84 | 2.4590 0.7909 0.0033
309
+ 36 | blk.36.ffn_up.weight | -0.0016 0.0533 -0.1516 8.5997 98.78% | 97.7354 0.84 | 2.4590 0.7909 0.0033
310
+ 36 | blk.36.inp_gate.weight | 0.0644 3.9675 49.2336 2465.6443 16.24% | 45635.3198 1.00 | 13.1338 0.9979 15.6519
311
+ 36 | blk.36.proj.weight | -1.4291 22.7940 -15.9061 251.0034 0.12% | 184512.3079 1.53 | 437.1291 0.0056 1.9013
312
+ 37 | blk.37.attn_output.weight | -0.0015 0.4906 -2.2482 27.1660 75.81% | 720.9361 1.00 | 5.1168 0.9756 0.2454
313
+ 37 | blk.37.attn_q.weight | -0.1092 3.1571 -12.5035 364.4264 80.81% | 75055.6639 1.85 | 98.2724 0.9014 4.1778
314
+ 37 | blk.37.ffn_down.weight | -0.0003 0.0250 -47.2863 3138.7102 65.14% | 16.7681 0.71 | 4.3612 -0.0001 -0.0000
315
+ 37 | blk.37.ffn_gate.weight | -0.0010 0.0474 -0.8784 15.7096 99.02% | 79.5378 0.90 | 1.1518 0.9043 0.0023
316
+ 37 | blk.37.ffn_up.weight | -0.0010 0.0474 -0.8784 15.7096 99.02% | 79.5378 0.90 | 1.1518 0.9043 0.0023
317
+ 37 | blk.37.inp_gate.weight | 0.0655 3.7048 49.1369 2459.3616 16.57% | 39899.8460 0.94 | 17.5028 0.9983 14.6734
318
+ 37 | blk.37.proj.weight | -0.5175 12.0202 -14.8202 230.5330 4.53% | 48949.9889 0.52 | 413.0014 -0.0026 -0.7028
319
+ 38 | blk.38.attn_output.weight | 0.0020 0.4148 -2.0636 27.7910 77.85% | 589.4480 0.90 | 6.1621 0.9686 0.1971
320
+ 38 | blk.38.attn_q.weight | -0.0102 2.6377 14.1705 598.8671 68.22% | 35991.8565 0.69 | 190.4991 0.1656 1.3794
321
+ 38 | blk.38.ffn_down.weight | -0.0003 0.0242 -52.0965 3602.9846 64.81% | 15.6696 0.97 | 3.5137 0.0036 0.0000
322
+ 38 | blk.38.ffn_gate.weight | -0.0010 0.0482 -1.2021 21.7779 98.96% | 81.2111 1.01 | 0.8463 0.9389 0.0021
323
+ 38 | blk.38.ffn_up.weight | -0.0010 0.0482 -1.2021 21.7779 98.96% | 81.2111 1.01 | 0.8463 0.9389 0.0021
324
+ 38 | blk.38.inp_gate.weight | 0.0642 3.4601 49.0997 2456.8760 17.53% | 35026.7356 0.94 | 15.4283 0.9987 12.8026
325
+ 38 | blk.38.proj.weight | -0.6408 13.4004 -12.4409 191.8305 9.71% | 62831.7927 1.13 | 288.3255 -0.0021 -0.3311
326
+ 39 | blk.39.attn_output.weight | 0.0050 0.3829 -1.6753 25.6331 80.07% | 548.4570 0.96 | 7.9533 0.9060 0.1439
327
+ 39 | blk.39.attn_q.weight | -0.1002 3.6544 -23.0952 860.0178 66.59% | 66355.3545 1.36 | 234.0325 -0.0558 -0.5375
328
+ 39 | blk.39.ffn_down.weight | -0.0005 0.0270 -40.8802 2744.7595 66.01% | 19.9347 1.13 | 3.6699 -0.0000 -0.0000
329
+ 39 | blk.39.ffn_gate.weight | -0.0004 0.0497 -0.9002 19.0104 98.72% | 76.8133 0.97 | 1.0211 0.9155 0.0022
330
+ 39 | blk.39.ffn_up.weight | -0.0004 0.0497 -0.9002 19.0104 98.72% | 76.8133 0.97 | 1.0211 0.9155 0.0022
331
+ 39 | blk.39.inp_gate.weight | 0.0679 3.1204 48.6805 2428.9492 19.80% | 28891.7524 0.91 | 22.7432 0.9960 10.7536
332
+ 39 | blk.39.proj.weight | -1.0683 11.7517 -11.5975 136.2594 13.95% | 47929.9578 0.87 | 285.8672 -0.0043 -0.6818
333
+ 40 | blk.40.attn_output.weight | 0.0072 0.3608 -1.1929 20.4109 82.11% | 472.5990 0.93 | 6.5226 0.9266 0.1280
334
+ 40 | blk.40.attn_q.weight | 0.1321 7.8186 45.1859 2208.5186 22.19% | 179828.3797 1.65 | 524.4500 -0.5758 -16.4507
335
+ 40 | blk.40.ffn_down.weight | 0.0002 0.0542 30.9903 2234.5730 58.71% | 64.9566 1.81 | 6.1533 -0.0103 -0.0000
336
+ 40 | blk.40.ffn_gate.weight | 0.0019 0.1552 35.3120 1580.9548 78.68% | 178.0369 1.52 | 7.0329 0.4702 0.0036
337
+ 40 | blk.40.ffn_up.weight | 0.0019 0.1552 35.3120 1580.9548 78.68% | 178.0369 1.52 | 7.0329 0.4702 0.0036
338
+ 40 | blk.40.inp_gate.weight | 0.0508 2.6877 46.7286 2299.7717 25.08% | 22013.4275 0.87 | 38.3183 0.9770 8.1936
339
+ 40 | blk.40.proj.weight | -0.8112 23.2214 -8.8265 154.5981 12.19% | 169699.7548 1.88 | 415.5581 0.0052 1.4179
340
+ 41 | blk.41.attn_output.weight | -0.0037 0.3036 -0.4906 6.8290 88.11% | 703.6792 nan | nan nan nan
341
+ 41 | blk.41.attn_q.weight | 0.1984 12.5113 46.5407 2304.5955 10.98% | 418343.2603 1.53 | 260.6160 0.9770 95.5679
342
+ 41 | blk.41.ffn_down.weight | -0.0005 0.0695 -25.9098 1496.6333 66.10% | 138.8901 1.46 | 8.9380 -0.0050 -0.0000
343
+ 41 | blk.41.ffn_gate.weight | 0.0026 0.1568 21.3870 815.2978 84.42% | 221.2918 1.11 | 4.6721 0.8249 0.0201
344
+ 41 | blk.41.ffn_up.weight | 0.0026 0.1568 21.3870 815.2978 84.42% | 221.2918 1.11 | 4.6721 0.8249 0.0201
345
+ 41 | blk.41.inp_gate.weight | 0.0521 2.5287 40.9693 1930.4489 29.30% | 21696.9643 0.99 | 54.6007 0.9162 6.2268
346
+ 41 | blk.41.proj.weight | 0.9382 19.1347 15.4470 242.0418 2.27% | 103585.0605 0.78 | 654.1016 -0.8584 -381.4285
347
+
348
+
349
+ Computing layer statistics for gemma-4-E4B-it-WIP/imatrix/imatrix-gemma-4-E4B-it-medium.gguf (42 layers)
350
+
351
+ Layer | ∑ E[A²] Gain | L2 Dist CosSim PCC Cov
352
+ ----------------------------------------------------------------------------
353
+ 0 | 1952672.7500 nan | nan nan nan nan
354
+ 1 | 617251.2500 0.56 | 1084.1038 0.0875 0.0875 1.9689
355
+ 2 | 47846.6602 0.28 | 607.3075 0.8067 0.8067 2.8249
356
+ 3 | 39994.9531 0.91 | 52.0874 0.9401 0.9400 0.7015
357
+ 4 | 68196.9922 1.31 | 125.1799 0.8180 0.8184 0.9518
358
+ 5 | 75924.1406 1.05 | 150.7364 0.7385 0.7386 1.2329
359
+ 6 | 73325.7422 0.98 | 115.4398 0.8565 0.8567 1.4911
360
+ 7 | 67469.8906 0.96 | 99.5572 0.9100 0.9100 1.2126
361
+ 8 | 141060.4219 1.45 | 216.9651 0.4922 0.4921 0.7148
362
+ 9 | 70627.0547 0.71 | 178.4586 0.6955 0.6960 1.2426
363
+ 10 | 45291.6445 0.80 | 123.8142 0.7978 0.7989 1.0066
364
+ 11 | 195280.1562 2.09 | 363.5921 0.4675 0.4690 1.2745
365
+ 12 | 84620.0078 0.66 | 390.4250 0.3837 0.3869 1.5344
366
+ 13 | 93488.5391 1.05 | 209.4356 0.6368 0.6369 1.3729
367
+ 14 | 84395.2031 0.95 | 156.7552 0.7609 0.7608 1.3183
368
+ 15 | 108032.0625 1.13 | 212.2301 0.6313 0.6314 1.2356
369
+ 16 | 77236.5312 0.85 | 227.3391 0.5652 0.5656 1.0724
370
+ 17 | 82965.6875 1.03 | 193.3754 0.5240 0.5240 0.7950
371
+ 18 | 158652.5000 1.39 | 269.1813 0.3987 0.3989 0.8571
372
+ 19 | 68692.7109 0.66 | 186.3432 0.7463 0.7464 1.3085
373
+ 20 | 83179.3984 1.10 | 240.1753 0.3527 0.3534 0.5291
374
+ 21 | 59425.0234 0.85 | 226.8482 0.4270 0.4275 0.6394
375
+ 22 | 141804.5781 1.54 | 294.7108 0.2587 0.2584 0.4681
376
+ 23 | 1594922.6250 3.36 | 948.5629 0.0222 0.0220 0.2219
377
+ 24 | 181728.5312 0.34 | 868.5350 0.3115 0.3120 4.9094
378
+ 25 | 222365.9375 1.11 | 475.2804 0.2018 0.1991 1.2265
379
+ 26 | 127340.2812 0.76 | 410.4586 0.3168 0.3146 1.6064
380
+ 27 | 113802.3125 0.95 | 312.5836 0.4276 0.4257 1.5873
381
+ 28 | 120420.7656 1.03 | 356.3658 0.2611 0.2650 0.9944
382
+ 29 | 165647.1406 1.17 | 370.5672 0.3771 0.3817 1.9705
383
+ 30 | 395175.0312 1.55 | 593.9757 0.1904 0.1867 1.7569
384
+ 31 | 234888.3438 0.77 | 652.5378 0.1017 0.1002 0.9967
385
+ 32 | 203839.3125 0.93 | 495.0794 0.2128 0.2152 1.4537
386
+ 33 | 152508.8906 0.86 | 423.3389 0.2858 0.2872 1.5660
387
+ 34 | 316315.9062 1.44 | 508.8465 0.2311 0.2310 1.5898
388
+ 35 | 158393.9375 0.71 | 513.6954 0.2208 0.2181 1.6652
389
+ 36 | 252941.4219 1.27 | 464.1814 0.2760 0.2802 1.9388
390
+ 37 | 164802.2812 0.81 | 424.9492 0.3670 0.3662 2.1328
391
+ 38 | 134617.9219 0.90 | 345.9932 0.3804 0.3798 1.6080
392
+ 39 | 143899.0781 1.03 | 370.2531 0.2786 0.2773 1.1536
393
+ 40 | 372435.1875 1.61 | 670.3611 -0.1175 -0.1189 -0.8995
394
+ 41 | 544910.3750 1.21 | 706.3104 0.4070 0.4076 7.8632