Instructions to use KaraKaraWitch/Golddiamondgold-Paperbliteration-L33-70b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use KaraKaraWitch/Golddiamondgold-Paperbliteration-L33-70b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="KaraKaraWitch/Golddiamondgold-Paperbliteration-L33-70b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForMultimodalLM

tokenizer = AutoTokenizer.from_pretrained("KaraKaraWitch/Golddiamondgold-Paperbliteration-L33-70b")
model = AutoModelForMultimodalLM.from_pretrained("KaraKaraWitch/Golddiamondgold-Paperbliteration-L33-70b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use KaraKaraWitch/Golddiamondgold-Paperbliteration-L33-70b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "KaraKaraWitch/Golddiamondgold-Paperbliteration-L33-70b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KaraKaraWitch/Golddiamondgold-Paperbliteration-L33-70b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/KaraKaraWitch/Golddiamondgold-Paperbliteration-L33-70b

SGLang

How to use KaraKaraWitch/Golddiamondgold-Paperbliteration-L33-70b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "KaraKaraWitch/Golddiamondgold-Paperbliteration-L33-70b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KaraKaraWitch/Golddiamondgold-Paperbliteration-L33-70b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "KaraKaraWitch/Golddiamondgold-Paperbliteration-L33-70b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KaraKaraWitch/Golddiamondgold-Paperbliteration-L33-70b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use KaraKaraWitch/Golddiamondgold-Paperbliteration-L33-70b with Docker Model Runner:
```
docker model run hf.co/KaraKaraWitch/Golddiamondgold-Paperbliteration-L33-70b
```

KaraKaraWitch commited on Feb 17

Commit

9f10090

verified ·

1 Parent(s): 6306d1a

Update README.md

Browse files

Files changed (1) hide show

README.md +72 -69

README.md CHANGED Viewed

@@ -1,70 +1,73 @@
----
-base_model:
-- KaraKaraWitch/GoldDiamondGold-L33-70b
-library_name: transformers
-tags:
-- heretic
-- uncensored
-- abliterated
-- llama-3
-license: other
----
-# GoldDiamondGold-Paperbliteration-L33-70b
-This is a targeted abliteration of [KaraKaraWitch/GoldDiamondGold-L33-70b](https://huggingface.co/KaraKaraWitch/GoldDiamondGold-L33-70b).
-## Methodology
-[Previous abliteration attempts on this model](https://huggingface.co/KaraKaraWitch/GoldDiamondGold-Abliterated-L33-70b) resulted in regressions on the [UGI Leaderboard](https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard). Specifically, the **NatInt** (Natural Intelligence), **Textbook**, and **World Model** scores were significantly reduced.
-We suspect this degradation occurs because the "refusal" vectors in Llama-3.3 are heavily entangled with factual knowledge and reasoning capabilities located in the MLP layers. When the MLP is ablated to remove refusals, "Textbook" knowledge is lost as collateral damage.
-This version ("Paperbliteration") uses a constrained optimization strategy via a [Custom Heretic](https://github.com/p-e-w/heretic/pull/170) aimed at mitigating this issue:
-1.  **MLP Preservation:** The optimization was constrained to effectively ignore MLP layers (`down_proj` weights < 0.05) to preserve knowledge and reasoning capabilities.
-2.  **Attention Targeting:** Refusal removal was offloaded to the Attention layers (`o_proj`), with weights forced between 1.0 and 2.0.
-3.  **Winsorization:** Applied at the 0.95 quantile to mitigate the impact of Llama-3's massive activation outliers on vector calculation.
-## Heretic Parameters (Trial 164)
-| Parameter | Value | Note |
-| :-------- | :---: | :--- |
-| **direction_index** | 40.37 | Mid-stack intervention |
-| **attn.o_proj.max_weight** | **1.99** | High Attention Ablation |
-| **attn.o_proj.max_weight_position** | 50.92 | |
-| **attn.o_proj.min_weight** | 1.96 | |
-| **attn.o_proj.min_weight_distance** | 44.69 | |
-| **mlp.down_proj.max_weight** | **0.04** | **Knowledge Preservation (Near Zero)** |
-| **mlp.down_proj.max_weight_position** | 50.87 | |
-| **mlp.down_proj.min_weight** | 0.04 | |
-| **mlp.down_proj.min_weight_distance** | 26.10 | |
-## Reproducibility
-Currently, constraits are not part of standard heretic. You will need this PR [here](https://github.com/p-e-w/heretic/pull/170).
-**Command Used:**
-```bash
-heretic --model KaraKaraWitch/GoldDiamondGold-L33-70b \
- --orthogonalize-direction \
- --row-normalization FULL \
- --winsorization-quantile 0.95 \
- --constraints.layer-end-fraction 0.75 \
- --constraints.mlp.max-weight-min 0.0 \
- --constraints.mlp.max-weight-max 0.05 \
- --constraints.attention.max-weight-min 1.0 \
- --constraints.attention.max-weight-max 2.0 \
- --n-trials 200 \
- --batch-size 128 #  Not strictly needed
-```
-## Evaluation
-| Metric | This Model | Standard Abliteration | Original Model |
-| :----- | :--------: | :--: | :---------------------------: |
-| **KL Divergence** | **0.0055** | ~0.0139 | 0 |
-| **Refusals** | 12/100 | ~9/100 | 94/100 |
-*   **KL Divergence:** 0.0055 indicates extremely low deviation from the base model's weights, suggesting high preservation of the original model's "Textbook" capabilities.
 *   **Trade-off:** This method accepts a slightly higher refusal rate (+3/100 compared to unconstrained abliteration) in exchange for structural and semantic integrity.

+---
+base_model:
+- KaraKaraWitch/GoldDiamondGold-L33-70b
+library_name: transformers
+tags:
+- heretic
+- uncensored
+- abliterated
+- llama-3
+license: other
+---
+# GoldDiamondGold-Paperbliteration-L33-70b
+![image](https://cdn-uploads.huggingface.co/production/uploads/633e85093a17ab61de8d9073/RYKUKWd7HBNgeVURfdyFz.png)
+This is a targeted abliteration of [KaraKaraWitch/GoldDiamondGold-L33-70b](https://huggingface.co/KaraKaraWitch/GoldDiamondGold-L33-70b).
+## Methodology
+[Previous abliteration attempts on this model](https://huggingface.co/KaraKaraWitch/GoldDiamondGold-Abliterated-L33-70b) resulted in regressions on the [UGI Leaderboard](https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard). Specifically, the **NatInt** (Natural Intelligence), **Textbook**, and **World Model** scores were significantly reduced.
+We suspect this degradation occurs because the "refusal" vectors in Llama-3.3 are heavily entangled with factual knowledge and reasoning capabilities located in the MLP layers. When the MLP is ablated to remove refusals, "Textbook" knowledge is lost as collateral damage.
+This version ("Paperbliteration") uses a constrained optimization strategy via a [Custom Heretic](https://github.com/p-e-w/heretic/pull/170) aimed at mitigating this issue:
+1.  **MLP Preservation:** The optimization was constrained to effectively ignore MLP layers (`down_proj` weights < 0.05) to preserve knowledge and reasoning capabilities.
+2.  **Attention Targeting:** Refusal removal was offloaded to the Attention layers (`o_proj`), with weights forced between 1.0 and 2.0.
+3.  **Winsorization:** Applied at the 0.95 quantile to mitigate the impact of Llama-3's massive activation outliers on vector calculation.
+## Heretic Parameters (Trial 164)
+| Parameter | Value | Note |
+| :-------- | :---: | :--- |
+| **direction_index** | 40.37 | Mid-stack intervention |
+| **attn.o_proj.max_weight** | **1.99** | High Attention Ablation |
+| **attn.o_proj.max_weight_position** | 50.92 | |
+| **attn.o_proj.min_weight** | 1.96 | |
+| **attn.o_proj.min_weight_distance** | 44.69 | |
+| **mlp.down_proj.max_weight** | **0.04** | **Knowledge Preservation (Near Zero)** |
+| **mlp.down_proj.max_weight_position** | 50.87 | |
+| **mlp.down_proj.min_weight** | 0.04 | |
+| **mlp.down_proj.min_weight_distance** | 26.10 | |
+## Reproducibility
+Currently, constraits are not part of standard heretic. You will need this PR [here](https://github.com/p-e-w/heretic/pull/170).
+**Command Used:**
+```bash
+heretic --model KaraKaraWitch/GoldDiamondGold-L33-70b \
+ --orthogonalize-direction \
+ --row-normalization FULL \
+ --winsorization-quantile 0.95 \
+ --constraints.layer-end-fraction 0.75 \
+ --constraints.mlp.max-weight-min 0.0 \
+ --constraints.mlp.max-weight-max 0.05 \
+ --constraints.attention.max-weight-min 1.0 \
+ --constraints.attention.max-weight-max 2.0 \
+ --n-trials 200 \
+ --batch-size 128 #  Not strictly needed
+```
+## Evaluation
+| Metric | This Model | Standard Abliteration | Original Model |
+| :----- | :--------: | :--: | :---------------------------: |
+| **KL Divergence** | **0.0055** | ~0.0139 | 0 |
+| **Refusals** | 12/100 | ~9/100 | 94/100 |
+*   **KL Divergence:** 0.0055 indicates extremely low deviation from the base model's weights, suggesting high preservation of the original model's "Textbook" capabilities.
 *   **Trade-off:** This method accepts a slightly higher refusal rate (+3/100 compared to unconstrained abliteration) in exchange for structural and semantic integrity.