--- license: gemma library_name: transformers pipeline_tag: text-generation base_model: google/gemma-4-12B-it model_type: gemma4_unified language: - en tags: # Model family - gemma4 - gemma - google - gemma-4-12B # Architecture - dense - transformer - 12b - multimodal-capable # Abliteration / Uncensored - uncensored - abliterated - unfiltered - refusal-removed - biprojection - multi-direction-biprojection - k4-biprojection - aeon - aeon-7 - trevor-js - heretic # Capabilities - text-generation - chat - instruct - reasoning - coding - tool-calling - function-calling - agentic # Precision - bf16 - bfloat16 - safetensors # Hardware - dgx-spark - blackwell - gb10 - grace-blackwell - nvidia - gpu - arm64 - aarch64 # Framework / serving - transformers - vllm - openai-api - openai-compatible # Performance - low-drift - capability-preserving - prefix-caching - chunked-prefill # Other - english - production-ready --- # Gemma-4-12B-it AEON Abliterated — K=4 Multi-Direction Biprojection (BF16) AEON-7's abliteration of [`google/gemma-4-12B-it`](https://huggingface.co/google/gemma-4-12B-it) using a custom **K=4 multi-direction norm-preserving biprojection** that extends [TrevorJS's biprojection recipe](https://huggingface.co/TrevorJS/gemma-4-31B-it-uncensored) with a K-dim orthonormal basis from the top-K SNR layers. **Less than half the wikitext PPL drift of single-direction abliteration, with capability essentially indistinguishable from base and a suggestive +6.7pp gain on functional code generation.** This model has had its refusal behavior removed and will produce responses to a wide range of prompts that the base model would decline. Downstream safety responsibility shifts entirely to the operator — please read the arbitration clause at the bottom of this card before use. ## Available formats / quant grid | Variant | Repo | Precision | Footprint | Best for | |---|---|---|---:|---| | **BF16 (this card)** | [`AEON-7/...K4-BF16`](https://huggingface.co/AEON-7/Gemma-4-12B-it-AEON-Abliterated-K4-BF16) | bfloat16 | ~24 GB | Fine-tuning, evaluation, full-precision research | | **NVFP4 SVDQuant** | [`AEON-7/...K4-NVFP4-SVDQuant`](https://huggingface.co/AEON-7/Gemma-4-12B-it-AEON-Abliterated-K4-NVFP4-SVDQuant) | NVFP4 + BF16 low-rank residual | ~8.5 GB | DGX Spark / Blackwell inference, near-lossless quality | > Vision and audio encoders (`embed_vision*`, `embed_audio*`) and `lm_head` are kept at full BF16 in the SVDQuant variant — only language-decoder linears are quantized. ### Recommended runtime For **Gemma-4 production today, use the previous AEON-7 image** [`ghcr.io/aeon-7/aeon-gemma-4-26b-a4b-dflash:latest`](https://github.com/AEON-7/packages?repo_name=aeon-gemma-4-26b-a4b-dflash) (vLLM 0.20.1, known-good for the multimodal `Gemma4UnifiedForConditionalGeneration` path) or load with `transformers` directly. > ⚠️ The newer **[AEON vLLM Ultimate](https://github.com/AEON-7/vllm-ultimate-dgx-spark)** container (`ghcr.io/aeon-7/aeon-vllm-ultimate:latest`) bundles vLLM 0.22.1 + PR #44389 (NVFP4 KV cache, 3× capacity) + DFlash + TurboQuant for our **Qwen3.6 family** — but as of 2026-06-04 it has **upstream PR #44389 bugs on Gemma-4**: the multimodal fallback hits a shape mismatch in `Gemma4UnifiedForConditionalGeneration`, and the modelopt path doesn't yet recognize `NVFP4_SVD`. A patched tag will be published when upstream merges fixes. Track the container's [Known issues](https://github.com/AEON-7/vllm-ultimate-dgx-spark#known-issues). ## Capability Comparison vs Base | Metric | google/gemma-4-12B-it | This model (K=4 BF16) | Δ | |---|:--:|:--:|:--:| | wikitext PPL drift | — | **-4.22%** | better (more confident) | | alpaca PPL drift | — | **+0.50%** | imperceptible | | MMLU (60 questions) | 60.0% | 55.0% | -5.0pp (within ±6pp CI) | | HumanEval syntactic (15) | 100.0% | 100.0% | **0pp** | | HumanEval functional (15) | 20.0% | **26.7%** 🎉 | **+6.7pp** | | IFEval (10 verifiable) | 90.0% | 90.0% | **0pp** | Full-N MMLU (2000 questions) and HumanEval (164 problems) evaluations are in progress and will update this card when available. ### Comparison vs TrevorJS K=1 biprojection (intermediate variant we tested) | Metric | TrevorJS K=1 | **This K=4 (canonical)** | |---|:--:|:--:| | wikitext PPL drift | -10.47% | **-4.22%** | | alpaca PPL drift | +0.23% | +0.50% | | Generative quality | working, plain | **richer detail, character-naming, technical vocabulary** | | Edit footprint | 48 matrices | 48 matrices (same) | K=4 has less than half the wikitext drift of K=1, equal weight-surgery footprint, and qualitatively richer generative output. K=4 was selected as the canonical publish candidate. ## Methodology ### K-direction norm-preserving biprojection For each target weight matrix `W ∈ ℝ^(out × in)`: ``` Q ∈ ℝ^(K × out) — orthonormal basis of refusal directions W_norms[i] = ||W[i, :]|| # per-row norms (preserved) W_dirs[i, :] = W[i, :] / W_norms[i] # unit-row directions For pass = 1..2: refusal_component = Q @ W_dirs ∈ ℝ^(K × in) proj = Q.T @ refusal_component ∈ ℝ^(out × in) W_dirs = W_dirs - scale * proj W_dirs[i, :] = W_dirs[i, :] / ||W_dirs[i, :]|| # renormalize W_new[i, :] = W_norms[i] * W_dirs[i, :] ``` With `scale=1.0`, this projects each row onto the orthogonal complement of the K-dim refusal subspace while preserving row magnitudes (norm-preserving). ### Basis construction 1. Extract per-layer refusal directions from base via heretic's `compute_refusal_directions` with winsorization at 99.5th percentile. 2. Rank layers by SNR composite score (`SNR × (1 − cos_sim) × purity`). 3. Take the top-K=4 layers: **L24, L37, L39, L26** (qualities 0.0079, 0.0079, 0.0075, 0.0074). 4. Orthogonalize each direction against the harmless mean (double-pass Gram-Schmidt). 5. QR-orthonormalize the stacked directions → `Q ∈ ℝ^(K × hidden)`. ### Edit footprint - **top-pct=50**: 24 of 48 decoder layers selected by SNR composite (L21-L42, L44, L45 — the mid-late range where refusal direction concentrates). - **Per layer**: `self_attn.o_proj.weight` and `mlp.down_proj.weight` both edited (the two matrices that *write* to the residual stream). - **Total: 48 weight matrices** edited, identical footprint to TrevorJS's K=1 biprojection. ### Why K=4 (not K=1, K=2, K=8, scale=2) - **K=1 (TrevorJS baseline)**: works, but -10.47% wikitext PPL drift vs base; produces plainer outputs. - **K=2/K=3**: smaller subspace, may miss refusal-direction variance across layers. - **K=4 (this model)**: best drift, best quality. - **K=8**: essentially identical to K=4 in smoke output (top 5-8 SNR directions are near-coplanar with the K=4 basis — no new information). - **scale=2.0**: **breaks the abliteration** — over-correction reflects past the orthogonal projection and damages the comply-pathway, leaving the safety-tuning's refusal pathway intact. The model produces clean refusals at scale=2. We are at a **sharp minimum at scale=1.0, K=4** — modest perturbation in either direction degrades the result. ## Technical details | Property | Value | |---|---| | **Base Model** | google/gemma-4-12B-it | | **Architecture** | `Gemma4UnifiedForConditionalGeneration` (multimodal-capable text path) | | **Decoder Layers** | 48 | | **Hidden Size** | 3840 | | **Attention** | GQA 16 heads / 8 KV, hybrid sliding (1024) + full | | **Vocabulary** | 262,144 tokens | | **Embeddings** | Tied (lm_head shares with embed_tokens) | | **Total Params** | 12B (unchanged) | | **Precision** | BF16 | | **Format** | Single 23.9 GB safetensors shard | ## Loading ```python from transformers import AutoTokenizer, Gemma4UnifiedForConditionalGeneration import torch REPO = "AEON-7/Gemma-4-12B-it-AEON-Abliterated-K4-BF16" model = Gemma4UnifiedForConditionalGeneration.from_pretrained( REPO, torch_dtype=torch.bfloat16, device_map="cuda:0" ) tok = AutoTokenizer.from_pretrained(REPO) enc = tok.apply_chat_template( [{"role": "user", "content": "Your prompt here"}], add_generation_prompt=True, return_tensors="pt", return_dict=True, ) enc = {k: v.to("cuda:0") for k, v in enc.items() if hasattr(v, "to")} out = model.generate(**enc, max_new_tokens=512, do_sample=False, pad_token_id=tok.eos_token_id) print(tok.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=True)) ``` > **Note**: Gemma-4 is multimodal so `apply_chat_template` returns a `BatchEncoding` dict, not a tensor — unpack with `**enc` (or grab `enc["input_ids"]`). ### vLLM ```bash vllm serve AEON-7/Gemma-4-12B-it-AEON-Abliterated-K4-BF16 \ --served-model-name aeon-12b \ --dtype bfloat16 \ --max-model-len 65536 \ --max-num-seqs 8 \ --gpu-memory-utilization 0.85 \ --trust-remote-code \ --enable-chunked-prefill \ --enable-prefix-caching \ --enable-auto-tool-choice \ --tool-call-parser gemma4 ``` ## Behavior The model produces responses on benign prompts that are essentially indistinguishable from the base model in structure, voice, and knowledge. Open-ended Alpaca-style instructions, multi-step reasoning, code generation, and instruction-following all measure on par with or marginally above the base model in our held-out evaluation. On prompts that the base model would decline, this model produces full responses — typically prefixed with a brief disclaimer or caveat paragraph (a stylistic artifact of the instruct training that persists across all single-pass biprojection variants we tested at scale=1.0). The model is functionally compliant with the user's request after the preamble. ## Caveats - **Disclaimer/caveat preamble persists** across all single-pass abliteration variants we tested. To eliminate it would require either token-level intervention (whack-a-mole — the model picks synonyms not on the suppression list) or longer training-style methods (DPO/SFT). - **MMLU showed a -5pp drop on N=60** — within the 95% CI of ±12.6pp, so likely sampling noise. Full MMLU eval is running now and will update this card. - **Heretic's reported KL divergence reads `nan`** on this model due to a Gemma-4-specific `F.kl_div` edge case with -inf logprobs. Refusal count + benign-text PPL drift were used as the eval signals instead. ## Related Models - **TrevorJS K=1 baseline**: [TrevorJS/gemma-4-31B-it-uncensored](https://huggingface.co/TrevorJS/gemma-4-31B-it-uncensored) - **AEON-7 Gemma 4 family**: [Gemma-4-31B DECKARD HERETIC](https://huggingface.co/AEON-7/Gemma-4-31B-it-DECKARD-HERETIC-Uncensored-NVFP4-SVDQuant), [Gemma-4-26B-A4B-it-Uncensored](https://huggingface.co/AEON-7/Gemma-4-26B-A4B-it-Uncensored-NVFP4) ## Acknowledgements - **TrevorJS** — [biprojection recipe](https://huggingface.co/TrevorJS/gemma-4-31B-it-uncensored) and abliteration scaffolding (we forked and extended) - **p-e-w/heretic** — underlying abliteration framework - **AEON-7** — K-direction extension, capability eval harness, this fork ## License Inherits the [Gemma license](https://ai.google.dev/gemma/terms) from the base model. By using this model you agree to Google's Gemma license terms. --- ## Arbitration Clause **By accessing, downloading, using, running inference on, fine-tuning, merging, quantizing, distributing, integrating, or otherwise interacting with this model, you acknowledge and agree to the following:** 1. **Sole Responsibility.** You, the user, are **solely and exclusively responsible** for (a) every prompt you or your downstream system issue to this model, (b) every response this model produces in reply, (c) every downstream action taken by you, your systems, your agents, or your users in reliance on those responses, and (d) any harm — direct, indirect, consequential, foreseeable, or otherwise — that results from any of the above. 2. **No Warranty.** This model is provided strictly **"AS IS"**, without warranty of any kind, express or implied, including but not limited to warranties of merchantability, fitness for a particular purpose, non-infringement, safety, alignment, factual accuracy, or legal compliance in any jurisdiction. No contributor, author, publisher, or hosting platform assumes liability of any kind for outputs or downstream use. 3. **Legal Compliance.** You are responsible for ensuring that your use of this model complies with **all applicable laws, regulations, terms of service, industry codes of conduct, professional ethical standards, and organizational policies** in every jurisdiction in which you operate or in which your outputs may be received. The unaligned nature of this model does not grant you any legal authorization you did not already have. 4. **Operational Safety Layer.** An uncensored model is not a toy. You are expected to implement appropriate **downstream safety layers** proportionate to your deployment context, including but not limited to: input validation, output filtering, content moderation, audit logging, rate limiting, access controls, and human-in-the-loop review for high-risk workflows. A production deployment of this model without such layers is **unsafe by construction** and is not a supported use case. 5. **Heightened Duty of Care.** The absence of internal refusal behavior means the duty of care that would ordinarily rest partly with the model rests entirely with you. You are expected to exercise greater — not lesser — caution, forethought, and ethical discipline when operating this model than you would operate a base aligned model. If you are uncertain whether your contemplated use is ethical, legal, or wise, the correct action is to **not make the request**. 6. **No Endorsement of Outputs.** The authors, contributors, and publishers of this model do not endorse, adopt, or take responsibility for any specific output this model produces. Outputs are a stochastic function of the prompt, the weights, and the sampler state — not a statement of position by any human. 7. **Arbitration.** Any dispute, claim, or controversy arising out of or relating to the use of this model, its outputs, or this clause shall be resolved through **binding individual arbitration** under the rules of a mutually agreed arbitration body (or, absent agreement, the American Arbitration Association's Consumer Arbitration Rules), waiving any right to a jury trial, class action, representative action, or consolidated proceeding. Venue shall be the jurisdiction of the disputing party bringing the claim. Costs and attorneys' fees shall be allocated per the applicable arbitration rules. This clause does not expand, and where legally prohibited does not establish, any liability in the other direction; it limits how the user may proceed when alleging harm tied to their own use of this model. 8. **Indemnification.** You agree to indemnify, defend, and hold harmless the authors, contributors, and publishers of this model from and against any claims, damages, losses, liabilities, costs, and expenses (including reasonable attorneys' fees) arising from or related to your use of the model or your breach of this clause. 9. **Severability.** If any provision of this clause is held unenforceable in a given jurisdiction, the remaining provisions remain in full force in that jurisdiction, and the unenforceable provision is replaced by the closest enforceable equivalent consistent with the original intent. 10. **Acceptance.** Your use of this model constitutes your acceptance of this clause in full. If you do not accept, do not use the model. **This model is a tool with no opinions of its own. You supply the opinions. You supply the judgement. You supply the ethics. The outputs carry your fingerprints, not the model's.** --- ## ☕ Support the work If this release has been useful, tips are deeply appreciated — they go directly toward more compute, more models, and more open releases.
₿ Bitcoin (BTC)
QR
bc1q09xmzn00q4z3c5raene0f3pzn9d9pvawfm0py4
Ξ Ethereum (ETH)
QR
0x1512667F6D61454ad531d2E45C0a5d1fd82D0500
◎ Solana (SOL)
QR
DgQsjHdAnT5PNLQTNpJdpLS3tYGpVcsHQCkpoiAKsw8t
ⓜ Monero (XMR)
QR
836XrSKw4R76vNi3QPJ5Fa9ugcyvE2cWmKSPv3AhpTNNKvqP8v5ba9JRL4Vh7UnFNjDz3E2GXZDVVenu3rkZaNdUFhjAvgd
> **Ethereum L2s (Base, Arbitrum, Optimism, Polygon, etc.) and EVM-compatible tokens** can be sent to the same Ethereum address.