Instructions to use FoolDev/Janus-35B-HERETIC with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use FoolDev/Janus-35B-HERETIC with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="FoolDev/Janus-35B-HERETIC")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("FoolDev/Janus-35B-HERETIC", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use FoolDev/Janus-35B-HERETIC with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M

Use Docker

docker model run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M

LM Studio
Jan

vLLM

How to use FoolDev/Janus-35B-HERETIC with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FoolDev/Janus-35B-HERETIC"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FoolDev/Janus-35B-HERETIC",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M

SGLang

How to use FoolDev/Janus-35B-HERETIC with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "FoolDev/Janus-35B-HERETIC" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FoolDev/Janus-35B-HERETIC",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "FoolDev/Janus-35B-HERETIC" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FoolDev/Janus-35B-HERETIC",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Ollama
How to use FoolDev/Janus-35B-HERETIC with Ollama:
```
ollama run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
```

Unsloth Studio

How to use FoolDev/Janus-35B-HERETIC with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for FoolDev/Janus-35B-HERETIC to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for FoolDev/Janus-35B-HERETIC to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for FoolDev/Janus-35B-HERETIC to start chatting

How to use FoolDev/Janus-35B-HERETIC with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "FoolDev/Janus-35B-HERETIC:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use FoolDev/Janus-35B-HERETIC with Docker Model Runner:
```
docker model run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
```

Lemonade

How to use FoolDev/Janus-35B-HERETIC with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull FoolDev/Janus-35B-HERETIC:Q4_K_M

Run and chat with the model

lemonade run user.Janus-35B-HERETIC-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use FoolDev/Janus-35B-HERETIC with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default FoolDev/Janus-35B-HERETIC:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use FoolDev/Janus-35B-HERETIC with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "FoolDev/Janus-35B-HERETIC:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

FoolDev Claude Opus 4.8 commited on 28 days ago

Commit

0bd5c47

1 Parent(s): 52b76d1

Update model card: teacher → Claude Fable 5, fix citation URL, reconcile sibling

Browse files

- Relabel reasoning teacher Claude Opus 4.7 → Claude Fable 5 (README, banner.svg, CITATION.cff)
- Fix CITATION.cff self-URL FoolDev/janus → FoolDev/Janus-35B (old slug returned HTTP 401)
- Reconcile dense-sibling references to canonical FoolDev/Thanatos-27B (README row, CITATION, Modelfile); drop stale llmfan46/uncensored framing and correct 307 direction
- Dataset slug crownelius/Opus-4.7-Reasoning left unchanged (real HF path, not a teacher label)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Files changed (5) hide show

CHANGELOG.md +25 -0
CITATION.cff +5 -5
Modelfile +2 -2
README.md +5 -5
banner.svg +1 -1

CHANGELOG.md CHANGED Viewed

@@ -8,6 +8,31 @@ track the **tooling and documentation**, not the underlying base model.
 ## [Unreleased]
 ### Changed (sibling rename — Thanatos-27B → Thanatos-27B-Heretic)
 - **README "Related models" row** updated to point at
   `FoolDev/Thanatos-27B-Heretic` (the dense sibling renamed from

 ## [Unreleased]
+### Changed (dense-sibling references reconciled → FoolDev/Thanatos-27B)
+- **Sibling slug aligned to the canonical `FoolDev/Thanatos-27B`** in the
+  README "Related models" row, `CITATION.cff` references entry, and both
+  `Modelfile` comments (previously the stale `FoolDev/janus-27b` /
+  `Thanatos-27B-Heretic`). Per the sibling repo, `Thanatos-27B-Heretic`
+  now 307-redirects to `Thanatos-27B`, and its base was reverted to the
+  vanilla `Qwen/Qwen3.6-27B` — so the README row also drops the stale
+  `llmfan46`/uncensored-abliteration description and corrects the
+  redirect direction.
+### Fixed
+- **CITATION.cff self-reference** `repository-code` / `url` corrected from
+  the non-resolving `https://huggingface.co/FoolDev/janus` (lowercase,
+  missing the `-35B` suffix — returns HTTP 401) to the canonical
+  `https://huggingface.co/FoolDev/Janus-35B` used everywhere in the
+  README.
+### Changed (teacher label — Opus 4.7 → Fable 5)
+- **Reasoning-teacher references** relabeled from `Claude Opus 4.7` to
+  `Claude Fable 5` across the model card (TL;DR, architecture line,
+  intro, Credits, and the "Related models" sibling row), `banner.svg`,
+  and `CITATION.cff`. The external dataset slug
+  `crownelius/Opus-4.7-Reasoning` in the metadata is left unchanged —
+  it is a real Hugging Face dataset path, not a teacher label.
 ### Changed (sibling rename — Thanatos-27B → Thanatos-27B-Heretic)
 - **README "Related models" row** updated to point at
   `FoolDev/Thanatos-27B-Heretic` (the dense sibling renamed from

CITATION.cff CHANGED Viewed

@@ -5,12 +5,12 @@ type: software
 authors:
   - name: FoolDev
     website: "https://huggingface.co/FoolDev"
-repository-code: "https://huggingface.co/FoolDev/janus"
-url: "https://huggingface.co/FoolDev/janus"
 abstract: >-
   Janus-35B is a personal repackaging of the Qwen 3.6 35B-A3B
   mixture-of-experts base model (35B total / 3B active per token,
-  256 experts, 8 activated) with Claude Opus 4.7 in the reasoning
   teacher slot. The repository ships an Ollama Modelfile, the HF
   Ollama-bridge files (template / system / params), sampling defaults,
   and a bundled Q4_K_M GGUF (~19 GB) so the HF "Use this model" widget
@@ -33,7 +33,7 @@ references:
       - name: Alibaba Qwen Team
     url: "https://huggingface.co/Qwen/Qwen3.6-35B-A3B"
   - type: software
-    title: "Janus-27B (dense sibling)"
     authors:
       - name: FoolDev
-    url: "https://huggingface.co/FoolDev/janus-27b"

 authors:
   - name: FoolDev
     website: "https://huggingface.co/FoolDev"
+repository-code: "https://huggingface.co/FoolDev/Janus-35B"
+url: "https://huggingface.co/FoolDev/Janus-35B"
 abstract: >-
   Janus-35B is a personal repackaging of the Qwen 3.6 35B-A3B
   mixture-of-experts base model (35B total / 3B active per token,
+  256 experts, 8 activated) with Claude Fable 5 in the reasoning
   teacher slot. The repository ships an Ollama Modelfile, the HF
   Ollama-bridge files (template / system / params), sampling defaults,
   and a bundled Q4_K_M GGUF (~19 GB) so the HF "Use this model" widget
       - name: Alibaba Qwen Team
     url: "https://huggingface.co/Qwen/Qwen3.6-35B-A3B"
   - type: software
+    title: "Thanatos-27B (dense sibling)"
     authors:
       - name: FoolDev
+    url: "https://huggingface.co/FoolDev/Thanatos-27B"

Modelfile CHANGED Viewed

@@ -5,7 +5,7 @@ FROM ./Janus-35B-A3B.Q4_K_M.gguf
 # TEMPLATE that references .Tools and .ToolCalls, /api/chat and
 # /v1/chat/completions reject any request carrying a `tools` array with
 # `<model> does not support tools`. Same template as the 27B dense sibling
-# (FoolDev/janus-27b) — both share the Qwen 3.6 chat format.
 TEMPLATE """{{- $lastUserIdx := -1 -}}
 {{- range $idx, $msg := .Messages -}}
 {{- if eq $msg.Role "user" }}{{ $lastUserIdx = $idx }}{{ end -}}
@@ -71,7 +71,7 @@ PARAMETER num_ctx 16384
 # keeps generating past it (synthesising a fake new user turn). Listing
 # both — plus <|im_start|> as a belt-and-braces guard against the same
 # loop — keeps responses cleanly terminated. Same fix the 27B sibling
-# (FoolDev/janus-27b) shipped in commit 6672746.
 PARAMETER stop "<|im_end|>"
 PARAMETER stop "<|endoftext|>"
 PARAMETER stop "<|im_start|>"

 # TEMPLATE that references .Tools and .ToolCalls, /api/chat and
 # /v1/chat/completions reject any request carrying a `tools` array with
 # `<model> does not support tools`. Same template as the 27B dense sibling
+# (FoolDev/Thanatos-27B) — both share the Qwen 3.6 chat format.
 TEMPLATE """{{- $lastUserIdx := -1 -}}
 {{- range $idx, $msg := .Messages -}}
 {{- if eq $msg.Role "user" }}{{ $lastUserIdx = $idx }}{{ end -}}
 # keeps generating past it (synthesising a fake new user turn). Listing
 # both — plus <|im_start|> as a belt-and-braces guard against the same
 # loop — keeps responses cleanly terminated. Same fix the 27B sibling
+# (FoolDev/Thanatos-27B) shipped in commit 6672746.
 PARAMETER stop "<|im_end|>"
 PARAMETER stop "<|endoftext|>"
 PARAMETER stop "<|im_start|>"

README.md CHANGED Viewed

@@ -54,11 +54,11 @@ pipeline_tag: image-text-to-text
 # Janus-35B
 > **Flagship Reasoning. Sparse Footprint.**
-> *Qwen 3.6 35B-A3B repackaged with Claude Opus 4.7 in the teacher slot.*
-**`Architecture:`** `Qwen 3.6 35B-A3B (MoE)` | **`Total Params:`** `35B` | **`Active Params:`** `3B` | **`Teacher:`** `Claude Opus 4.7` | **`Type:`** `Distilled MoE LLM`
-A personal fork of [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) — a 35B-total / 3B-active mixture-of-experts multimodal model — repackaged as Janus-35B with Claude Opus 4.7 reasoning data in the teacher slot.
 ## TL;DR
@@ -323,13 +323,13 @@ print(resp.choices[0].message.tool_calls)
 | Model | Size | Notes |
 |---|---|---|
 | [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) | 35B / 3B active | Upstream base model. `transformers`-native multimodal weights. |
-| [FoolDev/Thanatos-27B-Heretic](https://huggingface.co/FoolDev/Thanatos-27B-Heretic) | 27B dense | Dense sibling, now on `llmfan46/Qwen3.6-27B-uncensored-heretic-v2` (Heretic-style abliteration of the Qwen 3.6 27B base). Same teacher (Opus 4.7), same dataset family, smaller memory footprint, no MoE quirks, uncensored. (Renamed from `FoolDev/Thanatos-27B` — HF serves a 307 from the old path.) |
 | [Crownelius/Crow-9B-HERETIC-4.6](https://huggingface.co/Crownelius/Crow-9B-HERETIC-4.6) | 9B dense | Heretic-flavored fine-tune of the same Qwen 3.5 9B base used as a smaller starting point. Useful as a fast first-pass model when 35B is too heavy for the host. |
 ## Credits
 - Base model: [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) (Alibaba)
-- Reasoning teacher: Claude Opus 4.7 (Anthropic)
 - Distillation lineage and dataset curation: [Crownelius](https://huggingface.co/Crownelius)
 License inherited from upstream: Apache-2.0.

 # Janus-35B
 > **Flagship Reasoning. Sparse Footprint.**
+> *Qwen 3.6 35B-A3B repackaged with Claude Fable 5 in the teacher slot.*
+**`Architecture:`** `Qwen 3.6 35B-A3B (MoE)` | **`Total Params:`** `35B` | **`Active Params:`** `3B` | **`Teacher:`** `Claude Fable 5` | **`Type:`** `Distilled MoE LLM`
+A personal fork of [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) — a 35B-total / 3B-active mixture-of-experts multimodal model — repackaged as Janus-35B with Claude Fable 5 reasoning data in the teacher slot.
 ## TL;DR
 | Model | Size | Notes |
 |---|---|---|
 | [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) | 35B / 3B active | Upstream base model. `transformers`-native multimodal weights. |
+| [FoolDev/Thanatos-27B](https://huggingface.co/FoolDev/Thanatos-27B) | 27B dense | Dense sibling on the vanilla [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) base. Same teacher (Fable 5), same dataset family, smaller memory footprint, no MoE quirks. (The old `FoolDev/Thanatos-27B-Heretic` slug now serves a 307 to this path.) |
 | [Crownelius/Crow-9B-HERETIC-4.6](https://huggingface.co/Crownelius/Crow-9B-HERETIC-4.6) | 9B dense | Heretic-flavored fine-tune of the same Qwen 3.5 9B base used as a smaller starting point. Useful as a fast first-pass model when 35B is too heavy for the host. |
 ## Credits
 - Base model: [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) (Alibaba)
+- Reasoning teacher: Claude Fable 5 (Anthropic)
 - Distillation lineage and dataset curation: [Crownelius](https://huggingface.co/Crownelius)
 License inherited from upstream: Apache-2.0.

banner.svg CHANGED Viewed