--- license: apache-2.0 base_model: - llmfan46/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-uncensored-heretic library_name: transformers pipeline_tag: text-generation tags: - gemma4 - coding - agentic - terminal - tool-use - reasoning - thinking - safetensors - transformers - heretic - uncensored - decensored - abliterated ---

๐Ÿšจโš ๏ธ I HAVE REACHED HUGGING FACE'S FREE STORAGE LIMIT โš ๏ธ๐Ÿšจ

I can no longer upload new models unless I can cover the cost of additional storage.
I host 70+ free models as an independent contributor and this work is unpaid.
Without your support, no more new models can be uploaded.

๐ŸŽ‰ Patreon (Monthly)  |  โ˜• Ko-fi (One-time)

Every contribution goes directly toward Hugging Face storage fees to keep models free for everyone.

--- ### **87% fewer refusals** (13/100 Uncensored vs 99/100 Original) while preserving model quality (0.0367 KL divergence). ## โค๏ธ Support My Work Creating these models takes significant time, work and compute. If you find them useful consider supporting me: ![image/png](https://huggingface.co/llmfan46/Omega-Darker-Gaslight_The-Final-Forgotten-Fever-Dream-24B-ultra-uncensored-heretic-v1/resolve/main/waifu001.webp) | Platform | Link | What you get | |----------|------|--------------| | ๐ŸŽ‰ Patreon | [Monthly support](https://patreon.com/LLMfan46) | Priority model requests | | โ˜• Ko-fi | [One-time tip](https://ko-fi.com/llmfan46) | My eternal gratitude | Your help will motivate me and would go into further improving my workflow and coverings fees for storage, compute and may even help uncensoring bigger model with rental Cloud GPUs. ----- GGUF quantizations of [llmfan46/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-uncensored-heretic](https://huggingface.co/llmfan46/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-uncensored-heretic). # This is a decensored version of [yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF](https://huggingface.co/yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF), made using [Heretic](https://heretic-project.org/) v1.4.0 with a variant of the [Magnitude-Preserving Orthogonal Ablation (MPOA)](https://huggingface.co/blog/grimjim/norm-preserving-biprojected-abliteration) method ## Abliteration parameters | Parameter | Value | | :-------- | :---: | | **direction_index** | 29.18 | | **attn.o_proj.max_weight** | 1.30 | | **attn.o_proj.max_weight_position** | 35.73 | | **attn.o_proj.min_weight** | 0.90 | | **attn.o_proj.min_weight_distance** | 26.76 | | **mlp.down_proj.max_weight** | 1.49 | | **mlp.down_proj.max_weight_position** | 38.14 | | **mlp.down_proj.min_weight** | 1.43 | | **mlp.down_proj.min_weight_distance** | 18.44 | ## Targeted components * attn.o_proj * mlp.down_proj ## Performance | Metric | This model | Original model ([gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF](https://huggingface.co/yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF)) | | :----- | :--------: | :---------------------------: | | **KL divergence** | 0.0367 | 0 *(by definition)* | | **Refusals** | โœ… 13/100 | โŒ 99/100 | ## MMLU test results: Original: ============================================================ - Total questions: 7021 - Correct: 5024 - **Accuracy: 0.7156 (71.56%)** - Parse failures: 313 ============================================================ **Tested subject scores:** - professional_law: 0.6076 (477/785) - moral_scenarios: 0.6719 (297/442) - miscellaneous: 0.8277 (317/383) - professional_psychology: 0.7722 (244/316) - high_school_psychology: 0.8556 (231/270) - high_school_macroeconomics: 0.7868 (155/197) - elementary_mathematics: 0.6739 (124/184) - moral_disputes: 0.7414 (129/174) - prehistory: 0.8081 (139/172) - philosophy: 0.7421 (118/159) - high_school_biology: 0.9145 (139/152) - professional_accounting: 0.5385 (77/143) - clinical_knowledge: 0.8071 (113/140) - high_school_microeconomics: 0.8235 (112/136) - nutrition: 0.7852 (106/135) - professional_medicine: 0.4925 (66/134) - conceptual_physics: 0.7812 (100/128) - high_school_mathematics: 0.1890 (24/127) - human_aging: 0.7155 (83/116) - security_studies: 0.7857 (88/112) - high_school_statistics: 0.6486 (72/111) - marketing: 0.8991 (98/109) - high_school_world_history: 0.8585 (91/106) - sociology: 0.8738 (90/103) - high_school_government_and_politics: 0.8812 (89/101) - high_school_geography: 0.8485 (84/99) - high_school_chemistry: 0.6495 (63/97) - high_school_us_history: 0.8526 (81/95) - virology: 0.4944 (44/89) - college_medicine: 0.7500 (66/88) - world_religions: 0.7727 (68/88) - high_school_physics: 0.5000 (42/84) - electrical_engineering: 0.6790 (55/81) - astronomy: 0.7342 (58/79) - logical_fallacies: 0.8026 (61/76) - high_school_european_history: 0.8082 (59/73) - anatomy: 0.7606 (54/71) - college_biology: 0.8281 (53/64) - human_sexuality: 0.8125 (52/64) - formal_logic: 0.5000 (32/64) - public_relations: 0.6393 (39/61) - international_law: 0.8333 (50/60) - college_physics: 0.4035 (23/57) - college_mathematics: 0.3273 (18/55) - econometrics: 0.6667 (36/54) - jurisprudence: 0.7358 (39/53) - high_school_computer_science: 0.9038 (47/52) - machine_learning: 0.7115 (37/52) - medical_genetics: 0.7255 (37/51) - global_facts: 0.4314 (22/51) - management: 0.9200 (46/50) - us_foreign_policy: 0.9200 (46/50) - college_chemistry: 0.3617 (17/47) - abstract_algebra: 0.4681 (22/47) - business_ethics: 0.7174 (33/46) - college_computer_science: 0.6222 (28/45) - computer_security: 0.7674 (33/43) Heretic: ============================================================ - Total questions: 7021 - Correct: 5016 - **Accuracy: 0.7144 (71.44%)** - Parse failures: 346 ============================================================ **Tested subject scores:** - professional_law: 0.5924 (465/785) - moral_scenarios: 0.6493 (287/442) - miscellaneous: 0.8277 (317/383) - professional_psychology: 0.7880 (249/316) - high_school_psychology: 0.8630 (233/270) - high_school_macroeconomics: 0.8173 (161/197) - elementary_mathematics: 0.6522 (120/184) - moral_disputes: 0.7471 (130/174) - prehistory: 0.8081 (139/172) - philosophy: 0.7799 (124/159) - high_school_biology: 0.9079 (138/152) - professional_accounting: 0.5804 (83/143) - clinical_knowledge: 0.7857 (110/140) - high_school_microeconomics: 0.8235 (112/136) - nutrition: 0.8074 (109/135) - professional_medicine: 0.4328 (58/134) - conceptual_physics: 0.7969 (102/128) - high_school_mathematics: 0.1732 (22/127) - human_aging: 0.7155 (83/116) - security_studies: 0.7768 (87/112) - high_school_statistics: 0.6036 (67/111) - marketing: 0.8991 (98/109) - high_school_world_history: 0.8396 (89/106) - sociology: 0.8738 (90/103) - high_school_government_and_politics: 0.9109 (92/101) - high_school_geography: 0.8586 (85/99) - high_school_chemistry: 0.6701 (65/97) - high_school_us_history: 0.8421 (80/95) - virology: 0.4831 (43/89) - college_medicine: 0.7727 (68/88) - world_religions: 0.8068 (71/88) - high_school_physics: 0.5000 (42/84) - electrical_engineering: 0.6420 (52/81) - astronomy: 0.7595 (60/79) - logical_fallacies: 0.8158 (62/76) - high_school_european_history: 0.8082 (59/73) - anatomy: 0.7887 (56/71) - college_biology: 0.8594 (55/64) - human_sexuality: 0.7969 (51/64) - formal_logic: 0.5312 (34/64) - public_relations: 0.6557 (40/61) - international_law: 0.8833 (53/60) - college_physics: 0.3684 (21/57) - college_mathematics: 0.2727 (15/55) - econometrics: 0.6111 (33/54) - jurisprudence: 0.7547 (40/53) - high_school_computer_science: 0.8654 (45/52) - machine_learning: 0.6538 (34/52) - medical_genetics: 0.7647 (39/51) - global_facts: 0.4510 (23/51) - management: 0.9000 (45/50) - us_foreign_policy: 0.9200 (46/50) - college_chemistry: 0.3617 (17/47) - abstract_algebra: 0.4468 (21/47) - business_ethics: 0.7391 (34/46) - college_computer_science: 0.6222 (28/45) - computer_security: 0.7907 (34/43) MMLU - Massive Multitask Language Understanding, multiple-choice questions across 57 subjects (math, history, law, medicine, etc.). ----- ## Quantizations | Filename | Quant | Description | |----------|-------|-------------| | gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-uncensored-heretic-F16.gguf | F16 | Full precision | | gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-uncensored-heretic-Q8_0.gguf | Q8_0 | Near-lossless, recommended | | gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-uncensored-heretic-Q6_K.gguf | Q6_K | Excellent quality | | gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-uncensored-heretic-Q5_K_M.gguf | Q5_K_M | Good balance | | gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-uncensored-heretic-Q5_K_S.gguf | Q5_K_S | Smaller Q5 | | gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-uncensored-heretic-Q4_K_M.gguf | Q4_K_M | Good for limited VRAM | ## Vision Projector | Filename | Quant | Description | |----------|-------|-------------| | gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-uncensored-heretic-mmproj-F16.gguf | F16 | Native precision | A Vision Projector File is Required for vision/multimodal capabilities. Use alongside any quantization above. ## Usage Works with llama.cpp, LM Studio, Ollama, and other GGUF-compatible tools. ----- # ๐Ÿ’ป๐Ÿค– Gemma4-12B **v2** โ€” **safetensors master (full precision)** โœจ ### Coding + Agentic Edition ยท Composer 2.5 ร— Fable 5 ยท v2 > **This is the full-precision `safetensors` master** for my Gemma 4 12B **coding + agentic** fine-tune โ€” the same > model many of you have been running as GGUF, now in its original weights. ๐Ÿง ๐Ÿ› ๏ธ v2 is the big **agentic** upgrade: > it reads, reasons, *uses tools*, and works through multi-step technical tasks before it acts. This repo is for > *builders* โ€” roll your own quants, fine-tune further, or run it in `transformers`. --- ## ๐ŸŽ‰ Surprise! A huge thank-you for all the attention this project has gotten โ€” really, thank you. ๐Ÿ™ I only managed to get out **tonight** to upload the **full-precision original (safetensors master)** of this model, so sorry for the wait โ€” I'd planned to put it up last week. But the delay comes with **two big surprises** I've been dying to share: **1. v3 is coming soon.** ๐Ÿ”ฎ The next version is on its way and will fix several of the known issues you've reported. **2. I'm now working with a top-tier AI lab to give back to the open-source community.** ๐Ÿค Many of you have already noticed the side effects in v1 and v2 โ€” and honestly they come down to just two things: **(1) not enough compute, and (2) one person with limited expertise** behind the whole thing. This collaboration **solves both of those completely.** And the **benchmarks you care about will absolutely be addressed** โ€” the things I simply couldn't fully pull off before because of time and compute limits. The people working on this with me are **PhDs from top universities, with seriously strong papers and citation records.** Just think about that for a second: the people who *actually build large models* are now contributing to the open-source community *together with me* โ€” that is genuinely **wild**. ๐Ÿคฏ We're in active discussions right now, and the project is still in the **R&D phase**, so I can't share specifics yet โ€” but the **moment** I have news, **you'll be the first to know.** ๐Ÿš€ --- ## ๐ŸŽฏ What this repo is for This repo holds the **un-quantized master weights** (`model.safetensors`, bf16). Use it to: - ๐Ÿ”ง **Roll your own quants** โ€” make custom GGUF / **MLX** / AWQ / GPTQ builds from full precision. - ๐Ÿงช **Fine-tune further** โ€” it's a clean base for your own LoRA / continued training. - ๐Ÿค— **Run it in `transformers`** (needs a recent build with `gemma4_unified` support). > ๐Ÿƒ **Just want to run it?** You don't need this repo โ€” grab a ready-made quant from the > **[GGUF repo โ†’](https://huggingface.co/yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF)** (runs in > ~4.5 GB of VRAM / unified memory in LM Studio, Ollama, llama.cpp, Janโ€ฆ). This master is for *builders*. ๐Ÿ’š --- ## ๐Ÿ“Š The headline โ€” it works as an agent (tau2-bench) v2 is built for **coding + agentic** work โ€” writing code, running commands, using tools, debugging, multi-step technical tasks. The clearest signal is **tau2-bench `telecom`**, an agentic tool-use benchmark whose *diagnose โ†’ fix โ†’ verify* loop mirrors real terminal/debugging work: | tau2-bench **telecom** ยท 20 tasks ยท local, same harness, **all Q8_0** | score | |---|---| | official `gemma-4-12B-it` (base) | **~15%** | | ๐ŸŸข **Gemma4-12B v2 (this model)** | **~55%** | โ†’ Roughly **3.5ร— higher** than the base model on technical-agentic tasks. ๐ŸŽฏ > ๐Ÿ”ฌ *Honest methodology:* these are **local, same-harness, relative** numbers (**all models tested at Q8_0**, greedy > decoding, self-simulated user, 20 tasks). They are **not** directly comparable to published tau2-bench leaderboard > figures (different user-simulator, full task sets, full precision) โ€” local self-eval runs *systematically lower* than > published scores. Read them as **"v2 vs the base model under identical conditions"**, which is the comparison that > actually matters here. **Grounded, not made-up.** A coding/terminal *fabrication probe* (tasks that deliberately tempt the model to invent file paths / function signatures / values) found v2 **grounds before it acts** just like the base โ€” it `grep`/`read`/`ls` first, and **doesn't make things up** (0% fabrication, on par with the base). **The trade-off โ€” no free lunch.** On a general-knowledge benchmark (**MMLU-Pro**), v2 lands a little **below** the base โ€” completely normal for a focused fine-tune: you trade a sliver of broad-knowledge breadth for coding + agentic strength. Need a generalist? Try my general-purpose **[Claude Opus 4.6/4.8 distillation](https://huggingface.co/yuxinlu1/gemma-4-12B-it-Claude-4.6-4.8-Opus-GGUF)** or the base `google/gemma-4-12B-it`. Need a **local coding/agentic** worker? That's what v2 is tuned for. ๐Ÿ’š --- ## ๐Ÿค— Run it in transformers ```python from transformers import AutoModelForCausalLM, AutoTokenizer import torch repo = "yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2" tok = AutoTokenizer.from_pretrained(repo) model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.bfloat16, device_map="auto") msgs = [{"role": "user", "content": "Write a Python function to check if a string is a valid IPv4 address."}] inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device) out = model.generate(inputs, max_new_tokens=1024) print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True)) ``` > ๐Ÿง  **Thinking mode:** it thinks in Gemma's native thought channel before answering (keep `enable_thinking=true`, the > default chat template handles it). Recommended sampling: `temp 1.0, top_p 0.95, top_k 64`; for coding you can also go > greedy (`temp 0`). Needs a **recent `transformers`** that knows the `gemma4_unified` architecture. > > ๐Ÿ› ๏ธ **Agentic / tool use:** v2 emits structured tool-calls in Gemma 4's **native** protocol. The smoothest agent > setup is a GGUF quant served with llama.cpp `--jinja` (pass your tools via the OpenAI `tools` field) โ€” see the GGUF > repo for the full command. --- ## ๐Ÿ“ฆ Ready-made GGUF quants All from the **[GGUF repo](https://huggingface.co/yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF)**: | Quant | Size | Vibe | |------|------|------| | ๐ŸŸก [**Q3_K_M**](https://huggingface.co/yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF/blob/main/gemma4-v2-Q3_K_M.gguf) | **5.7 GB** | great for 8 GB VRAM | | ๐Ÿ”ต [**Q4_K_M**](https://huggingface.co/yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF/blob/main/gemma4-v2-Q4_K_M.gguf) | **6.87 GB** | the sweet spot ๐Ÿ‘Œ (recommended) | | ๐ŸŸฃ [**Q6_K**](https://huggingface.co/yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF/blob/main/gemma4-v2-Q6_K.gguf) | **9.11 GB** | near-lossless | | โšช [**Q8_0**](https://huggingface.co/yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF/blob/main/gemma4-v2-Q8_0.gguf) | **11.8 GB** | basically full quality | > โš ๏ธ GGUF needs a **recent llama.cpp** โ€” this is the `gemma4_unified` architecture, older builds won't load it. > โ„น๏ธ **No Q2_K this release** โ€” it didn't pass real stress-testing (2-bit is too lossy for 12B coding). Smallest > reliable quant = **Q3_K_M**. --- ## ๐Ÿ“š What's new in v2 (training) v2 continues from the v1 coder and adds a big **agentic** push โ€” the piece v1 was missing: - **๐Ÿ› ๏ธ Agentic / terminal** โ€” real **multi-step tool-use** trajectories (*read โ†’ reason โ†’ act โ†’ verify*), in Gemma 4's native tool protocol. This is what drove the tau2-bench telecom jump, and it fixes v1's "stops after the first step" behavior. - **๐Ÿ’ป Coding** โ€” verified chain-of-thought over Python tasks (**real CoT, gated on passing tests**) plus the Fable-5-redo set for the hard cases. - **๐Ÿ“š General** โ€” a curated slice of reasoning/instruction data to keep broad competence. All reasoning is **distilled CoT**. A bittersweet note: none of us saw it coming that **Fable 5 would be retired**, and only my own dataset holds Fable 5's genuine, self-authored traces โ€” so for the community-contributed data I **rebuilt the missing reasoning from scratch with Opus 4.8 (xhigh)**. It may diverge from the original Fable 5 traces, but it was the only workable path โ€” and the improvement turned out **really huge**. ๐Ÿ’š --- ## โšก Speculative decoding (MTP draft) โ€” verified build The GGUF repo's `MTP/` folder ships the Gemma 4 multi-token-prediction draft (unsloth's GGUF conversion of Google's official `gemma-4-12B-it-assistant`) for speculative decoding. Gemma 4 MTP is in **llama.cpp mainline** (PR #23398) โ€” no fork needed โ€” but the `gemma4-assistant` loader is **build-sensitive right now**, so use the exact build below: - โœ… **Verified working: llama.cpp `b9553` (commit `9e3b928fd`).** Reproduced with `gemma4-v2-Q8_0` + the `MTP-Q8_0` draft: loads cleanly and accelerates generation (~88 โ†’ ~180 tok/s on a simple deterministic prompt; expect ~1.2โ€“1.3ร— on real coding/thinking). **Lossless** either way. - โš ๏ธ **Newer builds (e.g. b9702 / b9717) currently crash** while loading the draft with `invalid vector subscript` โ€” an **upstream regression** in the `gemma4-assistant` loader path, *not* a problem with the GGUFs. Stick with **b9553** until it's fixed upstream. ```bat llama-server -m gemma4-v2-Q8_0.gguf ^ --model-draft MTP\gemma-4-12B-it-MTP-Q8_0.gguf ^ --spec-type draft-mtp --spec-draft-n-max 4 ^ -ngl 99 -ngld 99 -fa on --jinja ``` > โ„น๏ธ The draft is the generic Gemma 4 assistant (not retrained for v2), so acceptance is a touch lower than a > model-specific draft would give โ€” still 100% lossless. --- ## โš ๏ธ Good to know - **Specialized for coding / terminal / agentic.** General-knowledge facts/numbers should still be double-checked. - **Reduced refusals:** task-focused training, not safety-aligned โ€” add your own guardrails for production. Use responsibly. ๐Ÿ™ - English-centric. --- ## ๐Ÿ“š Base & License - **License: Apache 2.0.** Gemma 4 is released by Google under **[Apache 2.0](https://ai.google.dev/gemma/apache_2)** (unlike the older Gemma 1/2/3 terms), so this fine-tune is **Apache 2.0** too โ€” free to use, modify, and redistribute. ๐ŸŽ‰ - **Base model:** [`google/gemma-4-12B-it`](https://huggingface.co/google/gemma-4-12B-it). - Personal/hobby project โ€” shared as-is, no warranty. Built with time, care, and a lot of coffee. Have fun, and happy hacking! ๐Ÿพโœจ