Instructions to use Jiunsong/supergemma4-26b-uncensored-gguf-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Jiunsong/supergemma4-26b-uncensored-gguf-v2 with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="Jiunsong/supergemma4-26b-uncensored-gguf-v2",
	filename="supergemma4-26b-uncensored-fast-v2-Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Jiunsong/supergemma4-26b-uncensored-gguf-v2 with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf Jiunsong/supergemma4-26b-uncensored-gguf-v2:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf Jiunsong/supergemma4-26b-uncensored-gguf-v2:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf Jiunsong/supergemma4-26b-uncensored-gguf-v2:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf Jiunsong/supergemma4-26b-uncensored-gguf-v2:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Jiunsong/supergemma4-26b-uncensored-gguf-v2:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Jiunsong/supergemma4-26b-uncensored-gguf-v2:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Jiunsong/supergemma4-26b-uncensored-gguf-v2:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Jiunsong/supergemma4-26b-uncensored-gguf-v2:Q4_K_M

Use Docker

docker model run hf.co/Jiunsong/supergemma4-26b-uncensored-gguf-v2:Q4_K_M

LM Studio
Jan

vLLM

How to use Jiunsong/supergemma4-26b-uncensored-gguf-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Jiunsong/supergemma4-26b-uncensored-gguf-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jiunsong/supergemma4-26b-uncensored-gguf-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Jiunsong/supergemma4-26b-uncensored-gguf-v2:Q4_K_M

Ollama
How to use Jiunsong/supergemma4-26b-uncensored-gguf-v2 with Ollama:
```
ollama run hf.co/Jiunsong/supergemma4-26b-uncensored-gguf-v2:Q4_K_M
```

Unsloth Studio

How to use Jiunsong/supergemma4-26b-uncensored-gguf-v2 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Jiunsong/supergemma4-26b-uncensored-gguf-v2 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Jiunsong/supergemma4-26b-uncensored-gguf-v2 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Jiunsong/supergemma4-26b-uncensored-gguf-v2 to start chatting

How to use Jiunsong/supergemma4-26b-uncensored-gguf-v2 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf Jiunsong/supergemma4-26b-uncensored-gguf-v2:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Jiunsong/supergemma4-26b-uncensored-gguf-v2:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Jiunsong/supergemma4-26b-uncensored-gguf-v2 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf Jiunsong/supergemma4-26b-uncensored-gguf-v2:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Jiunsong/supergemma4-26b-uncensored-gguf-v2:Q4_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use Jiunsong/supergemma4-26b-uncensored-gguf-v2 with Docker Model Runner:
```
docker model run hf.co/Jiunsong/supergemma4-26b-uncensored-gguf-v2:Q4_K_M
```

Lemonade

How to use Jiunsong/supergemma4-26b-uncensored-gguf-v2 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Jiunsong/supergemma4-26b-uncensored-gguf-v2:Q4_K_M

Run and chat with the model

lemonade run user.supergemma4-26b-uncensored-gguf-v2-Q4_K_M

List all available models

lemonade list

The model quality is not very good

#10

by SDDian - opened Apr 16

Discussion

SDDian

Apr 16

First of all, I would like to thank the author for their work in modifying the model into an uncensored version, but in practice, the model's performance is not very good. First, the model's lack of censorship seems insufficient; it still doesn't respond to some prompts. Second, during use, I encountered situations where the responses were sometimes garbled and the process seemed unstoppable. Below is my startup command, and the image shows the problem I encountered. But to be honest, this model is much better than supergemma4-26b-abliterated-multimodal-Q4_K_M.gguf. llama-server.exe -m F:\llamacpp_workspace\gemma4-26B\supergemma4-26b-uncensored-fast-v2-Q4_K_M.gguf --port 48080 -c 131072 --temp 1.0 --top-p 0.95 --top-k 64

redaizo

Apr 17

https://ollama.com/library/gemma4:26b-a4b-it-q4_K_M

use this one. this supergemma is broken. had the same for a week unworkable.

GideonWyeth

Apr 17

•

edited Apr 17

https://ollama.com/library/gemma4:26b-a4b-it-q4_K_M

use this one. this supergemma is broken. had the same for a week unworkable.

This is not a recommendation to give in a discussions of UNCENSORED model.

As a real solution, i'm currently using "coder3101/gemma-4-26B-A4B-it-heretic" in Apex I-Balanced quants. It is less compliant, but i'm getting almost no repetition/infinite cycle bugs (I didn't say no bugs, but ALMOST no bugs, coz i noticed they are present in all quants, but their intensity varies). But as far as Gemma 4 goes, even most popular Unsloth quants, had some bugs at the initial release. Maybe this issue will be fixed with time.

SDDian

Apr 18

Thank you. Perhaps you're right; when using an uncensored version, one shouldn't expect the model to have full functionality. However, I have used some very high-quality uncensored models. I'm just pointing out some bugs; it would be great if the author could fix them when they have time.

redaizo

Apr 19

https://ollama.com/library/gemma4:26b-a4b-it-q4_K_M

use this one. this supergemma is broken. had the same for a week unworkable.

This is not a recommendation to give in a discussions of UNCENSORED model.

As a real solution, i'm currently using "coder3101/gemma-4-26B-A4B-it-heretic" in Apex I-Balanced quants. It is less compliant, but i'm getting almost no repetition/infinite cycle bugs (I didn't say no bugs, but ALMOST no bugs, coz i noticed they are present in all quants, but their intensity varies). But as far as Gemma 4 goes, even most popular Unsloth quants, had some bugs at the initial release. Maybe this issue will be fixed with time.

Absolutely right, not a good recommandation given the topic, thanks for bringing that to my attention, i'll keep that in mind

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment