gemma-4-12B-it-abliterated-uncensored-exl3-4bpw

Overview

EXL3 4bpw quantized version of gemma-4-12B-it-abliterated-uncensored — an abliterated, uncensored variant of google/gemma-4-12B-it (Gemma 4 12B Unified, dense, ~11.95B parameters).

Important loading note

This repository contains an EXL3 4.0 bpw quantized checkpoint intended for EXL3-compatible runtimes such as ExLlamaV3 / TabbyAPI.

The Hugging Face model page may automatically display a generic Transformers loading snippet such as AutoProcessor / AutoModelForMultimodalLM. That snippet is not the recommended loading path for this EXL3 quantized checkpoint.

For normal use, load this model with an EXL3-compatible runtime and point the runtime at this repository or the downloaded model folder.

Metadata note

Hugging Face's automatic sidebar metadata may not accurately represent EXL3 quantized checkpoints. In particular, automatically displayed parameter counts or tensor-type summaries may be misleading for this format.

The source model is Gemma 4 12B dense from OpenYourMind/gemma-4-12B-it-abliterated-uncensored; this repository contains a 4.0 bpw EXL3 quantized export, not the original full-precision Transformers checkpoint.

Quantization

EXL3, 4.0 bits per weight (4bpw)

Intended runtime: ExLlamaV3 / TabbyAPI-compatible

Local Validation

• Exported and loaded locally on RTX 4090 24GB
• Clean-load VRAM observed around 7220 MiB
• Text generation smoke tested
• enable_thinking=false tested with valid JSON and no channel-marker leakage
• enable_thinking=true tested with valid JSON and correct answer
• Image input not tested
• Audio input not tested
• This model does not provide TTS/audio output

Compatibility Notes

• This is not a full-precision Transformers checkpoint.
• Gemma 4 unified architecture may require runtime compatibility patches in ExLlamaV3/TabbyAPI.
• Output config was patched for ExLlamaV3/TabbyAPI compatibility.
• chat_template.jinja was aligned with validated Gemma 4 12B EXL3 template to avoid thought/channel marker leakage.

Architecture

Property	Value
Architecture	Gemma4ForConditionalGeneration (model_type: gemma4)
Total Parameters	~11.95B (dense)
Decoder Layers	48
Hidden Size	3840
Attention	16 heads / 8 KV heads, hybrid sliding-window + global attention, p-RoPE
Vocabulary	262,144
Context Length	up to 131072 tokens (configured)

Files

File	Description
model.safetensors	EXL3 quantized weights (4bpw)
config.json	Model config
processor_config.json	Processor config
tokenizer.json, tokenizer_config.json, chat_template.jinja, generation_config.json	Standard

Usage

import requests

url = "http://host:5000/v1/chat/completions"
headers = {"Authorization": "Bearer YOUR_API_KEY"}

payload = {
    "model": "gemma-4-12B-it-abliterated-uncensored-exl3-4bpw",
    "messages": [{"role": "user", "content": "Hello!"}],
    "max_tokens": 512,
    "enable_thinking": False,
}

response = requests.post(url, headers=headers, json=payload)

Notes

• License: Gemma (inherits the Gemma 4 license from the base model)
• Base Model: google/gemma-4-12B-it
• Modality: Text + Image + Audio (encoder-free / unified)
• Architecture: Gemma 4 12B Unified (dense, ~11.95B)

Disclaimer

Use is the responsibility of the user. Ensure your usage complies with applicable laws, platform rules, the Gemma 4 license terms, and your deployment requirements.