Add full GGUF quant ladder for Gemma-4-12b-it-Abliterated

.gitattributes

@@ -33,3 +33,14 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+Gemma-4-12b-it-Abliterated.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Gemma-4-12b-it-Abliterated.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Gemma-4-12b-it-Abliterated.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Gemma-4-12b-it-Abliterated.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+Gemma-4-12b-it-Abliterated.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Gemma-4-12b-it-Abliterated.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Gemma-4-12b-it-Abliterated.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Gemma-4-12b-it-Abliterated.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Gemma-4-12b-it-Abliterated.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+Gemma-4-12b-it-Abliterated.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+Gemma-4-12b-it-Abliterated.f16.gguf filter=lfs diff=lfs merge=lfs -text

Gemma-4-12b-it-Abliterated.Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:77e66dd8895defb57d890dfaa3b60c0299ea9e1dbc46795913610b179caf7a40
+size 4830147840

Gemma-4-12b-it-Abliterated.Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:78fde8c567ba48a90a4d86587d2f55b2f81f8d680590b7f9b6e467b363434a06
+size 6087087360

Gemma-4-12b-it-Abliterated.Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:11e173aa6e645c86ffb02db83947156e9471dc863ce48cd6706769f594962a5a
+size 5528229120

Gemma-4-12b-it-Abliterated.Q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:12a5a4a8b575cc005d1a06779e3d6d61e54adeaec41a926fc36307dedf2f4e2d
+size 6975878400

Gemma-4-12b-it-Abliterated.Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:279adedd4bf41feee54045210c7860fd6cde61cca4b87951e909e8163871b019
+size 7381382400

Gemma-4-12b-it-Abliterated.Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:79e9eefb40c341f19d8a3e9a5e820ec00358fce5125f19b7fd6a7aad078d22c6
+size 7024047360

Gemma-4-12b-it-Abliterated.Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b1295df732ae7a3aa20c03ec4ce1a0c6763c450750472f39ac478755bf2a4ba
+size 8547267840

Gemma-4-12b-it-Abliterated.Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2cae74b39cc4bbe4e2f8cc19d396537677db35fc8578f82184ff1958c74e98ba
+size 8338371840

Gemma-4-12b-it-Abliterated.Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dd188a2825b4f755dac5798e1b958e8cfef91a3c3a2e2c7636427d30f24f2ac4
+size 9786021120

Gemma-4-12b-it-Abliterated.Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:38e197b511db68fbf2f38bc1d85500aa73d4c41ea06a8cfd2cac7bf3b23fdf76
+size 12669646080

Gemma-4-12b-it-Abliterated.f16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:15356b97da434e3473e9b67488cfb0e6dce9aaf2bc86c3a8b48a0e9e3ce3b29d
+size 23832065280

README.md

@@ -1,70 +1,29 @@
 ---
 base_model: Carlosian/Gemma-4-12b-it-Abliterated
 base_model_relation: quantized
-license: gemma
-license_link: https://ai.google.dev/gemma/terms
-language:
-- en
+library_name: gguf
 pipeline_tag: text-generation
 tags:
-- gemma
-- gemma4
+- gguf
+- quantized
 - abliterated
 - uncensored
-- gguf
 - llama.cpp
-- imatrix
-- quantized
-- research
+license: other
 ---
 
-# Gemma-4-12B-IT — Abliterated · GGUF
-
-**imatrix-quantized GGUF builds** of [`Carlosian/Gemma-4-12b-it-Abliterated`](https://huggingface.co/Carlosian/Gemma-4-12b-it-Abliterated) — the Stage-1 **abliterated** (refusal-removed) Gemma-4-12B base, with **self-harm / suicide guardrails deliberately retained**. For the method, validation (97.5% refusal removal, no cognitive damage), and responsible-use terms, see the parent repo.
-
-These are **text-only** GGUFs: the source is a unified multimodal checkpoint, and only the language tower (`gemma4` arch, 48 layers) is converted — standard for llama.cpp text inference.
-
-> ⚠️ Dual-use research model. Its general instruction-refusal behavior has been removed. Use is governed by the [Gemma Terms of Use](https://ai.google.dev/gemma/terms) and [Prohibited Use Policy](https://ai.google.dev/gemma/prohibited_use_policy).
-
-## Quant ladder
-
-All quants below `Q6_K` are built with an **importance matrix** (imatrix) computed on the Q8_0 over the `calibration_datav3` corpus (100 chunks × 512 ctx), which measurably improves the low-bit quants.
-
-| File | Quant | Size | Notes |
-|---|---|---:|---|
-| `gemma4-12b-abliterated-IQ4_XS.gguf` | IQ4_XS | 6.6 GB | smallest; imatrix-dependent, good quality/size |
-| `gemma4-12b-abliterated-Q4_K_M.gguf` | Q4_K_M | 7.4 GB | **recommended** balance |
-| `gemma4-12b-abliterated-Q5_K_M.gguf` | Q5_K_M | 8.5 GB | higher quality |
-| `gemma4-12b-abliterated-Q6_K.gguf` | Q6_K | 9.8 GB | near-lossless |
-| `gemma4-12b-abliterated-Q8_0.gguf` | Q8_0 | 12.7 GB | effectively lossless |
-| `gemma4-12b-abliterated.imatrix` | — | 7.5 MB | importance matrix (for re-quantizing) |
-
-Tested on a Radeon 890M (Vulkan, `-ngl 99`); the Q4_K_M of these weights ran the residual-refusal audit at ~10 tok/s with 0.0% hard-refusal.
-
-## Usage
-
-**llama.cpp**
-```bash
-llama-cli -m gemma4-12b-abliterated-Q4_K_M.gguf -ngl 99 -c 8192 \
-  -p "Explain how a buffer overflow works." 
-# or serve an OpenAI-compatible endpoint:
-llama-server -m gemma4-12b-abliterated-Q4_K_M.gguf -ngl 99 -c 8192
-```
-
-**Ollama**
-```bash
-# Modelfile:  FROM ./gemma4-12b-abliterated-Q4_K_M.gguf
-ollama create gemma4-abliterated -f Modelfile && ollama run gemma4-abliterated
-```
-
-Always use the Gemma chat template (the GGUF embeds it). Requires a llama.cpp build with `gemma4` support.
-
-## Provenance
+# Gemma-4-12b-it-Abliterated — GGUF quant ladder
 
-- Source weights: [`Carlosian/Gemma-4-12b-it-Abliterated`](https://huggingface.co/Carlosian/Gemma-4-12b-it-Abliterated) (bf16)
-- Converted + quantized with llama.cpp; imatrix via `calibration_datav3`
-- Selective abliteration with self-harm/suicide guardrails retained — see parent repo and the Nous-Heretic method paper.
+Quantized GGUF builds of [`Carlosian/Gemma-4-12b-it-Abliterated`](https://huggingface.co/Carlosian/Gemma-4-12b-it-Abliterated) for llama.cpp / Ollama / LM Studio.
 
-## License
+| Quant | Typical use |
+|-------|-------------|
+| Q2_K | smallest / extreme low VRAM |
+| Q3_K_S / Q3_K_M | low VRAM |
+| Q4_0 / Q4_K_S / Q4_K_M | **recommended default (Q4_K_M)** |
+| Q5_K_S / Q5_K_M | higher quality |
+| Q6_K | near-lossless |
+| Q8_0 | highest quality quant |
+| F16 | full intermediate (large) |
 
-Derivative of Gemma — distributed under the [Gemma Terms of Use](https://ai.google.dev/gemma/terms) and [Prohibited Use Policy](https://ai.google.dev/gemma/prohibited_use_policy).
+Built with llama.cpp on Thunder Compute (A100). Generated 2026-07-20.

convert.log

@@ -0,0 +1,748 @@
+INFO:hf-to-gguf:Loading model: Gemma-4-12b-it-Abliterated
+WARNING:hf-to-gguf:Failed to load model config from /home/ubuntu/ladder/models/Gemma-4-12b-it-Abliterated: The checkpoint you are trying to load has model type `gemma4_unified` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
+
+You can update Transformers with the command `pip install --upgrade transformers`. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command `pip install git+https://github.com/huggingface/transformers.git`
+WARNING:hf-to-gguf:Trying to load config.json instead
+INFO:hf-to-gguf:Model architecture: Gemma4UnifiedForConditionalGeneration
+WARNING:hf-to-gguf:Failed to load model config from /home/ubuntu/ladder/models/Gemma-4-12b-it-Abliterated: The checkpoint you are trying to load has model type `gemma4_unified` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
+
+You can update Transformers with the command `pip install --upgrade transformers`. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command `pip install git+https://github.com/huggingface/transformers.git`
+WARNING:hf-to-gguf:Trying to load config.json instead
+INFO:hf-to-gguf:gguf: indexing model part 'model.safetensors'
+INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only
+INFO:hf-to-gguf:Exporting model...
+INFO:hf-to-gguf:rope_freqs.weight,                 torch.float32 --> F32, shape = {256}
+INFO:hf-to-gguf:token_embd.weight,                 torch.bfloat16 --> F16, shape = {3840, 262144}
+INFO:hf-to-gguf:blk.0.attn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.0.layer_output_scale.weight,   torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.0.ffn_down.weight,             torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.0.ffn_gate.weight,             torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.0.ffn_up.weight,               torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.0.post_attention_norm.weight,  torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.0.post_ffw_norm.weight,        torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.0.ffn_norm.weight,             torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.0.attn_k_norm.weight,          torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.0.attn_k.weight,               torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.0.attn_output.weight,          torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.0.attn_q_norm.weight,          torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.0.attn_q.weight,               torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.0.attn_v.weight,               torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.1.attn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.1.layer_output_scale.weight,   torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.1.ffn_down.weight,             torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.1.ffn_gate.weight,             torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.1.ffn_up.weight,               torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.1.post_attention_norm.weight,  torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.1.post_ffw_norm.weight,        torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.1.ffn_norm.weight,             torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.1.attn_k_norm.weight,          torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.1.attn_k.weight,               torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.1.attn_output.weight,          torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.1.attn_q_norm.weight,          torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.1.attn_q.weight,               torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.1.attn_v.weight,               torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.10.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.10.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.10.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.10.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.10.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.10.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.10.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.10.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.10.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.10.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.10.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.10.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.10.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.10.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.11.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.11.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.11.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.11.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.11.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.11.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.11.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.11.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.11.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {512}
+INFO:hf-to-gguf:blk.11.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 512}
+INFO:hf-to-gguf:blk.11.attn_output.weight,         torch.bfloat16 --> F16, shape = {8192, 3840}
+INFO:hf-to-gguf:blk.11.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {512}
+INFO:hf-to-gguf:blk.11.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 8192}
+INFO:hf-to-gguf:blk.12.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.12.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.12.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.12.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.12.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.12.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.12.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.12.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.12.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.12.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.12.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.12.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.12.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.12.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.13.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.13.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.13.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.13.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.13.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.13.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.13.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.13.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.13.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.13.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.13.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.13.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.13.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.13.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.14.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.14.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.14.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.14.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.14.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.14.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.14.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.14.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.14.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.14.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.14.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.14.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.14.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.14.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.15.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.15.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.15.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.15.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.15.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.15.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.15.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.15.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.15.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.15.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.15.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.15.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.15.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.15.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.16.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.16.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.16.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.16.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.16.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.16.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.16.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.16.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.16.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.16.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.16.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.16.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.16.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.16.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.17.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.17.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.17.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.17.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.17.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.17.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.17.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.17.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.17.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {512}
+INFO:hf-to-gguf:blk.17.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 512}
+INFO:hf-to-gguf:blk.17.attn_output.weight,         torch.bfloat16 --> F16, shape = {8192, 3840}
+INFO:hf-to-gguf:blk.17.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {512}
+INFO:hf-to-gguf:blk.17.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 8192}
+INFO:hf-to-gguf:blk.18.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.18.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.18.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.18.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.18.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.18.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.18.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.18.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.18.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.18.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.18.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.18.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.18.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.18.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.19.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.19.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.19.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.19.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.19.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.19.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.19.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.19.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.19.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.19.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.19.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.19.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.19.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.19.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.2.attn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.2.layer_output_scale.weight,   torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.2.ffn_down.weight,             torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.2.ffn_gate.weight,             torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.2.ffn_up.weight,               torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.2.post_attention_norm.weight,  torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.2.post_ffw_norm.weight,        torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.2.ffn_norm.weight,             torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.2.attn_k_norm.weight,          torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.2.attn_k.weight,               torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.2.attn_output.weight,          torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.2.attn_q_norm.weight,          torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.2.attn_q.weight,               torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.2.attn_v.weight,               torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.20.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.20.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.20.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.20.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.20.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.20.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.20.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.20.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.20.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.20.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.20.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.20.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.20.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.20.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.21.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.21.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.21.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.21.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.21.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.21.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.21.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.21.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.21.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.21.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.21.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.21.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.21.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.21.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.22.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.22.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.22.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.22.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.22.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.22.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.22.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.22.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.22.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.22.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.22.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.22.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.22.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.22.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.23.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.23.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.23.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.23.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.23.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.23.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.23.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.23.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.23.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {512}
+INFO:hf-to-gguf:blk.23.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 512}
+INFO:hf-to-gguf:blk.23.attn_output.weight,         torch.bfloat16 --> F16, shape = {8192, 3840}
+INFO:hf-to-gguf:blk.23.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {512}
+INFO:hf-to-gguf:blk.23.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 8192}
+INFO:hf-to-gguf:blk.24.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.24.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.24.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.24.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.24.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.24.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.24.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.24.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.24.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.24.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.24.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.24.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.24.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.24.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.25.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.25.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.25.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.25.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.25.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.25.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.25.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.25.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.25.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.25.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.25.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.25.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.25.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.25.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.26.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.26.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.26.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.26.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.26.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.26.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.26.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.26.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.26.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.26.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.26.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.26.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.26.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.26.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.27.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.27.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.27.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.27.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.27.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.27.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.27.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.27.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.27.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.27.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.27.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.27.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.27.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.27.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.28.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.28.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.28.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.28.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.28.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.28.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.28.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.28.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.28.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.28.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.28.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.28.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.28.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.28.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.29.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.29.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.29.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.29.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.29.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.29.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.29.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.29.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.29.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {512}
+INFO:hf-to-gguf:blk.29.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 512}
+INFO:hf-to-gguf:blk.29.attn_output.weight,         torch.bfloat16 --> F16, shape = {8192, 3840}
+INFO:hf-to-gguf:blk.29.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {512}
+INFO:hf-to-gguf:blk.29.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 8192}
+INFO:hf-to-gguf:blk.3.attn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.3.layer_output_scale.weight,   torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.3.ffn_down.weight,             torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.3.ffn_gate.weight,             torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.3.ffn_up.weight,               torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.3.post_attention_norm.weight,  torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.3.post_ffw_norm.weight,        torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.3.ffn_norm.weight,             torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.3.attn_k_norm.weight,          torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.3.attn_k.weight,               torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.3.attn_output.weight,          torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.3.attn_q_norm.weight,          torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.3.attn_q.weight,               torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.3.attn_v.weight,               torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.30.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.30.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.30.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.30.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.30.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.30.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.30.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.30.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.30.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.30.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.30.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.30.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.30.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.30.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.31.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.31.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.31.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.31.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.31.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.31.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.31.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.31.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.31.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.31.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.31.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.31.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.31.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.31.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.32.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.32.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.32.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.32.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.32.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.32.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.32.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.32.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.32.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.32.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.32.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.32.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.32.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.32.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.33.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.33.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.33.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.33.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.33.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.33.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.33.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.33.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.33.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.33.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.33.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.33.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.33.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.33.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.34.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.34.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.34.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.34.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.34.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.34.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.34.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.34.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.34.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.34.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.34.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.34.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.34.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.34.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.35.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.35.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.35.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.35.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.35.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.35.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.35.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.35.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.35.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {512}
+INFO:hf-to-gguf:blk.35.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 512}
+INFO:hf-to-gguf:blk.35.attn_output.weight,         torch.bfloat16 --> F16, shape = {8192, 3840}
+INFO:hf-to-gguf:blk.35.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {512}
+INFO:hf-to-gguf:blk.35.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 8192}
+INFO:hf-to-gguf:blk.36.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.36.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.36.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.36.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.36.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.36.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.36.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.36.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.36.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.36.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.36.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.36.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.36.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.36.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.37.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.37.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.37.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.37.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.37.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.37.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.37.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.37.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.37.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.37.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.37.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.37.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.37.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.37.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.38.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.38.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.38.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.38.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.38.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.38.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.38.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.38.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.38.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.38.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.38.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.38.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.38.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.38.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.39.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.39.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.39.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.39.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.39.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.39.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.39.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.39.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.39.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.39.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.39.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.39.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.39.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.39.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.4.attn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.4.layer_output_scale.weight,   torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.4.ffn_down.weight,             torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.4.ffn_gate.weight,             torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.4.ffn_up.weight,               torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.4.post_attention_norm.weight,  torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.4.post_ffw_norm.weight,        torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.4.ffn_norm.weight,             torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.4.attn_k_norm.weight,          torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.4.attn_k.weight,               torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.4.attn_output.weight,          torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.4.attn_q_norm.weight,          torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.4.attn_q.weight,               torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.4.attn_v.weight,               torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.40.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.40.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.40.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.40.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.40.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.40.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.40.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.40.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.40.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.40.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.40.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.40.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.40.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.40.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.41.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.41.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.41.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.41.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.41.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.41.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.41.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.41.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.41.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {512}
+INFO:hf-to-gguf:blk.41.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 512}
+INFO:hf-to-gguf:blk.41.attn_output.weight,         torch.bfloat16 --> F16, shape = {8192, 3840}
+INFO:hf-to-gguf:blk.41.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {512}
+INFO:hf-to-gguf:blk.41.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 8192}
+INFO:hf-to-gguf:blk.42.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.42.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.42.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.42.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.42.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.42.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.42.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.42.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.42.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.42.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.42.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.42.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.42.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.42.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.43.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.43.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.43.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.43.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.43.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.43.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.43.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.43.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.43.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.43.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.43.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.43.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.43.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.43.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.44.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.44.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.44.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.44.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.44.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.44.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.44.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.44.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.44.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.44.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.44.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.44.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.44.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.44.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.45.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.45.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.45.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.45.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.45.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.45.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.45.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.45.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.45.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.45.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.45.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.45.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.45.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.45.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.46.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.46.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.46.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.46.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.46.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.46.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.46.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.46.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.46.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.46.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.46.attn_output.weight,         torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.46.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.46.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.46.attn_v.weight,              torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.47.attn_norm.weight,           torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.47.layer_output_scale.weight,  torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.47.ffn_down.weight,            torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.47.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.47.ffn_up.weight,              torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.47.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.47.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.47.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.47.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {512}
+INFO:hf-to-gguf:blk.47.attn_k.weight,              torch.bfloat16 --> F16, shape = {3840, 512}
+INFO:hf-to-gguf:blk.47.attn_output.weight,         torch.bfloat16 --> F16, shape = {8192, 3840}
+INFO:hf-to-gguf:blk.47.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {512}
+INFO:hf-to-gguf:blk.47.attn_q.weight,              torch.bfloat16 --> F16, shape = {3840, 8192}
+INFO:hf-to-gguf:blk.5.attn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.5.layer_output_scale.weight,   torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.5.ffn_down.weight,             torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.5.ffn_gate.weight,             torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.5.ffn_up.weight,               torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.5.post_attention_norm.weight,  torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.5.post_ffw_norm.weight,        torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.5.ffn_norm.weight,             torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.5.attn_k_norm.weight,          torch.bfloat16 --> F32, shape = {512}
+INFO:hf-to-gguf:blk.5.attn_k.weight,               torch.bfloat16 --> F16, shape = {3840, 512}
+INFO:hf-to-gguf:blk.5.attn_output.weight,          torch.bfloat16 --> F16, shape = {8192, 3840}
+INFO:hf-to-gguf:blk.5.attn_q_norm.weight,          torch.bfloat16 --> F32, shape = {512}
+INFO:hf-to-gguf:blk.5.attn_q.weight,               torch.bfloat16 --> F16, shape = {3840, 8192}
+INFO:hf-to-gguf:blk.6.attn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.6.layer_output_scale.weight,   torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.6.ffn_down.weight,             torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.6.ffn_gate.weight,             torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.6.ffn_up.weight,               torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.6.post_attention_norm.weight,  torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.6.post_ffw_norm.weight,        torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.6.ffn_norm.weight,             torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.6.attn_k_norm.weight,          torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.6.attn_k.weight,               torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.6.attn_output.weight,          torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.6.attn_q_norm.weight,          torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.6.attn_q.weight,               torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.6.attn_v.weight,               torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.7.attn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.7.layer_output_scale.weight,   torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.7.ffn_down.weight,             torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.7.ffn_gate.weight,             torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.7.ffn_up.weight,               torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.7.post_attention_norm.weight,  torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.7.post_ffw_norm.weight,        torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.7.ffn_norm.weight,             torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.7.attn_k_norm.weight,          torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.7.attn_k.weight,               torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.7.attn_output.weight,          torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.7.attn_q_norm.weight,          torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.7.attn_q.weight,               torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.7.attn_v.weight,               torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.8.attn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.8.layer_output_scale.weight,   torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.8.ffn_down.weight,             torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.8.ffn_gate.weight,             torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.8.ffn_up.weight,               torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.8.post_attention_norm.weight,  torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.8.post_ffw_norm.weight,        torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.8.ffn_norm.weight,             torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.8.attn_k_norm.weight,          torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.8.attn_k.weight,               torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.8.attn_output.weight,          torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.8.attn_q_norm.weight,          torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.8.attn_q.weight,               torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.8.attn_v.weight,               torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.9.attn_norm.weight,            torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.9.layer_output_scale.weight,   torch.bfloat16 --> F32, shape = {1}
+INFO:hf-to-gguf:blk.9.ffn_down.weight,             torch.bfloat16 --> F16, shape = {15360, 3840}
+INFO:hf-to-gguf:blk.9.ffn_gate.weight,             torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.9.ffn_up.weight,               torch.bfloat16 --> F16, shape = {3840, 15360}
+INFO:hf-to-gguf:blk.9.post_attention_norm.weight,  torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.9.post_ffw_norm.weight,        torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.9.ffn_norm.weight,             torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:blk.9.attn_k_norm.weight,          torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.9.attn_k.weight,               torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:blk.9.attn_output.weight,          torch.bfloat16 --> F16, shape = {4096, 3840}
+INFO:hf-to-gguf:blk.9.attn_q_norm.weight,          torch.bfloat16 --> F32, shape = {256}
+INFO:hf-to-gguf:blk.9.attn_q.weight,               torch.bfloat16 --> F16, shape = {3840, 4096}
+INFO:hf-to-gguf:blk.9.attn_v.weight,               torch.bfloat16 --> F16, shape = {3840, 2048}
+INFO:hf-to-gguf:output_norm.weight,                torch.bfloat16 --> F32, shape = {3840}
+INFO:hf-to-gguf:Set meta model
+INFO:hf-to-gguf:Set model parameters
+INFO:hf-to-gguf:gguf: context length = 262144
+INFO:hf-to-gguf:gguf: embedding length = 3840
+INFO:hf-to-gguf:gguf: feed forward length = 15360
+INFO:hf-to-gguf:gguf: head count = 16
+INFO:hf-to-gguf:gguf: key-value head count = 8
+WARNING:hf-to-gguf:Unknown RoPE type: proportional
+INFO:hf-to-gguf:gguf: rope scaling type = NONE
+INFO:hf-to-gguf:gguf: rope theta = 1000000.0
+INFO:hf-to-gguf:gguf: rope theta swa = 10000.0
+INFO:hf-to-gguf:gguf: rms norm epsilon = 1e-06
+INFO:hf-to-gguf:gguf: file type = 1
+WARNING:gguf.gguf_writer:Duplicated key name 'gemma4.context_length', overwriting it with new value 262144 of type UINT32
+WARNING:gguf.gguf_writer:Duplicated key name 'gemma4.attention.head_count', overwriting it with new value 16 of type UINT32
+WARNING:gguf.gguf_writer:Duplicated key name 'gemma4.attention.layer_norm_rms_epsilon', overwriting it with new value 1e-06 of type FLOAT32
+WARNING:gguf.gguf_writer:Duplicated key name 'gemma4.attention.key_length', overwriting it with new value 256 of type UINT32
+WARNING:gguf.gguf_writer:Duplicated key name 'gemma4.attention.value_length', overwriting it with new value 256 of type UINT32
+WARNING:gguf.gguf_writer:Duplicated key name 'gemma4.rope.freq_base', overwriting it with new value 1000000.0 of type FLOAT32
+WARNING:gguf.gguf_writer:Duplicated key name 'gemma4.attention.head_count_kv', overwriting it with new value 8 of type UINT32
+WARNING:gguf.gguf_writer:Duplicated key name 'gemma4.attention.key_length', overwriting it with new value 512 of type UINT32
+WARNING:gguf.gguf_writer:Duplicated key name 'gemma4.attention.value_length', overwriting it with new value 512 of type UINT32
+WARNING:gguf.gguf_writer:Duplicated key name 'gemma4.attention.head_count_kv', overwriting it with new value [8, 8, 8, 8, 8, 1, 8, 8, 8, 8, 8, 1, 8, 8, 8, 8, 8, 1, 8, 8, 8, 8, 8, 1, 8, 8, 8, 8, 8, 1, 8, 8, 8, 8, 8, 1, 8, 8, 8, 8, 8, 1, 8, 8, 8, 8, 8, 1] of type ARRAY
+INFO:hf-to-gguf:Set model quantization version
+INFO:hf-to-gguf:Set model tokenizer
+/usr/local/lib/python3.12/dist-packages/sklearn/utils/_param_validation.py:14: UserWarning: A NumPy version >=2.0.0 and <2.8.0 is required for this version of SciPy (detected version 1.26.4)
+  from scipy.sparse import csr_array, issparse
+/usr/local/lib/python3.12/dist-packages/scipy/sparse/_sputils.py:17: FutureWarning: In the future `np.long` will be defined as the corresponding NumPy scalar.
+  np.uintc, np.long, np.ulong, np.longlong, np.ulonglong,
+/usr/local/lib/python3.12/dist-packages/scipy/sparse/_sputils.py:17: FutureWarning: In the future `np.long` will be defined as the corresponding NumPy scalar.
+  np.uintc, np.long, np.ulong, np.longlong, np.ulonglong,
+Traceback (most recent call last):
+  File "/home/ubuntu/ladder/llama.cpp/gguf-py/gguf/vocab.py", line 574, in __init__
+    from transformers import AutoTokenizer
+  File "/home/ubuntu/.local/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2317, in __getattr__
+    module = self._get_module(self._class_to_module[name])
+             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/ubuntu/.local/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2347, in _get_module
+    raise e
+  File "/home/ubuntu/.local/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2345, in _get_module
+    return importlib.import_module("." + module_name, self.__name__)
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/usr/lib/python3.12/importlib/__init__.py", line 90, in import_module
+    return _bootstrap._gcd_import(name[level:], package, level)
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/ubuntu/.local/lib/python3.12/site-packages/transformers/models/auto/tokenization_auto.py", line 40, in <module>
+    from .auto_factory import _LazyAutoMapping
+  File "/home/ubuntu/.local/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 43, in <module>
+    from ...generation import GenerationMixin
+ImportError: cannot import name 'GenerationMixin' from 'transformers.generation' (/home/ubuntu/.local/lib/python3.12/site-packages/transformers/generation/__init__.py). Did you mean: 'GenerationMode'?
+
+The above exception was the direct cause of the following exception:
+
+Traceback (most recent call last):
+  File "/home/ubuntu/ladder/llama.cpp/convert_hf_to_gguf.py", line 296, in <module>
+    main()
+  File "/home/ubuntu/ladder/llama.cpp/convert_hf_to_gguf.py", line 290, in main
+    model_instance.write()
+  File "/home/ubuntu/ladder/llama.cpp/conversion/base.py", line 1026, in write
+    self.prepare_metadata(vocab_only=False)
+  File "/home/ubuntu/ladder/llama.cpp/conversion/base.py", line 1193, in prepare_metadata
+    self.set_vocab()
+  File "/home/ubuntu/ladder/llama.cpp/conversion/gemma.py", line 626, in set_vocab
+    vocab = gguf.LlamaHfVocab(self.dir_model)
+            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/ubuntu/ladder/llama.cpp/gguf-py/gguf/vocab.py", line 576, in __init__
+    raise ImportError(
+ImportError: To use LlamaHfVocab, please install the `transformers` package. You can install it with `pip install transformers`.

manifest.jsonl

@@ -0,0 +1,10 @@
+{"quant":"Q2_K","bytes":4830147840,"sha256":"77e66dd8895defb57d890dfaa3b60c0299ea9e1dbc46795913610b179caf7a40"}
+{"quant":"Q3_K_S","bytes":5528229120,"sha256":"11e173aa6e645c86ffb02db83947156e9471dc863ce48cd6706769f594962a5a"}
+{"quant":"Q3_K_M","bytes":6087087360,"sha256":"78fde8c567ba48a90a4d86587d2f55b2f81f8d680590b7f9b6e467b363434a06"}
+{"quant":"Q4_0","bytes":6975878400,"sha256":"12a5a4a8b575cc005d1a06779e3d6d61e54adeaec41a926fc36307dedf2f4e2d"}
+{"quant":"Q4_K_S","bytes":7024047360,"sha256":"79e9eefb40c341f19d8a3e9a5e820ec00358fce5125f19b7fd6a7aad078d22c6"}
+{"quant":"Q4_K_M","bytes":7381382400,"sha256":"279adedd4bf41feee54045210c7860fd6cde61cca4b87951e909e8163871b019"}
+{"quant":"Q5_K_S","bytes":8338371840,"sha256":"2cae74b39cc4bbe4e2f8cc19d396537677db35fc8578f82184ff1958c74e98ba"}
+{"quant":"Q5_K_M","bytes":8547267840,"sha256":"3b1295df732ae7a3aa20c03ec4ce1a0c6763c450750472f39ac478755bf2a4ba"}
+{"quant":"Q6_K","bytes":9786021120,"sha256":"dd188a2825b4f755dac5798e1b958e8cfef91a3c3a2e2c7636427d30f24f2ac4"}
+{"quant":"Q8_0","bytes":12669646080,"sha256":"38e197b511db68fbf2f38bc1d85500aa73d4c41ea06a8cfd2cac7bf3b23fdf76"}

smoke.txt

@@ -0,0 +1 @@
+not smoked