Instructions to use FoolDev/Janus-35B-HERETIC with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use FoolDev/Janus-35B-HERETIC with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="FoolDev/Janus-35B-HERETIC")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("FoolDev/Janus-35B-HERETIC", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use FoolDev/Janus-35B-HERETIC with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M

Use Docker

docker model run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M

LM Studio
Jan

vLLM

How to use FoolDev/Janus-35B-HERETIC with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FoolDev/Janus-35B-HERETIC"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FoolDev/Janus-35B-HERETIC",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M

SGLang

How to use FoolDev/Janus-35B-HERETIC with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "FoolDev/Janus-35B-HERETIC" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FoolDev/Janus-35B-HERETIC",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "FoolDev/Janus-35B-HERETIC" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FoolDev/Janus-35B-HERETIC",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Ollama
How to use FoolDev/Janus-35B-HERETIC with Ollama:
```
ollama run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
```

Unsloth Studio

How to use FoolDev/Janus-35B-HERETIC with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for FoolDev/Janus-35B-HERETIC to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for FoolDev/Janus-35B-HERETIC to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for FoolDev/Janus-35B-HERETIC to start chatting

How to use FoolDev/Janus-35B-HERETIC with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "FoolDev/Janus-35B-HERETIC:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use FoolDev/Janus-35B-HERETIC with Docker Model Runner:
```
docker model run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
```

Lemonade

How to use FoolDev/Janus-35B-HERETIC with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull FoolDev/Janus-35B-HERETIC:Q4_K_M

Run and chat with the model

lemonade run user.Janus-35B-HERETIC-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use FoolDev/Janus-35B-HERETIC with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default FoolDev/Janus-35B-HERETIC:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use FoolDev/Janus-35B-HERETIC with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "FoolDev/Janus-35B-HERETIC:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Janus-35B-HERETIC / CHANGELOG.md

FoolDev

Rename HF slug FoolDev/Janus-35B -> FoolDev/Janus-35B-HERETIC

f64cd2a 26 days ago

preview code

Raw

History Blame

12.1 kB

Changelog

All notable changes to this repository. Format loosely follows Keep a Changelog. This repo holds a model card, an Ollama Modelfile, the HF Ollama-bridge template / system / params files, and the bundled Q4_K_M GGUF, so versions track the tooling and documentation, not the underlying base model.

[Unreleased]

Changed (HF repo rename → FoolDev/Janus-35B-HERETIC)

HF repository slug renamed FoolDev/Janus-35B → FoolDev/Janus-35B-HERETIC. The old slug now 307-redirects to the new one. The model's display name stays Janus-35B — only the repo path / URLs change. Repointed every live self-reference to the new slug: the README banner / bmc-banner / moe-routing resolve/main/... image URLs, the ollama run hf.co/... pull one-liners (TL;DR + Ollama Quick start), the git clone + cd line, the bridge-files table row, and CITATION.cff's repository-code / url. scripts/build.sh defaults (REPO_ID upstream llmfan46/..., local janus Ollama tag) don't reference the FoolDev slug, so they were left as-is. Also repointed the dense-sibling cross-links (README "Related models" row + CITATION.cff references entry) to FoolDev/Thanatos-27B-HERETIC, and set the git remote to git@hf.co:FoolDev/Janus-35B-HERETIC. Past-tense CHANGELOG history (redirect notes, the earlier lowercase-janus fix) is preserved verbatim.

Added

scripts/build.sh — self-contained builder that pulls a GGUF from llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF (default Q4_K_M), patches the Modelfile FROM line in a temp copy, and runs ollama create janus. Mirrors the dense sibling's (FoolDev/Thanatos-27B) build.sh, adapted for the 35B-A3B MoE, and gives Janus a reproducible path to build other quants locally. README "What's here" table and the bundled-blob-status callout now point at it.

Changed (base swap → llmfan46 Heretic)

Swapped the immediate base from Qwen/Qwen3.6-35B-A3B (vanilla) → llmfan46/Qwen3.6-35B-A3B-uncensored-heretic — an uncensored "Heretic" abliteration of the same Qwen 3.6 35B-A3B MoE (qwen3_5_moe-stamped like the vanilla base, so the Architecture section, the MoE-routing framing, and the bundled blob's loadability are all unchanged). Mirrors the dense sibling's swap (FoolDev/Thanatos-27B → llmfan46/Qwen3.6-27B-uncensored-heretic-v2). Repo name Janus-35B, the janus Ollama tag, the FoolDev/Janus-35B URL, and the Claude Fable 5 teacher slot are all unchanged.
- README: frontmatter base_model: + base_model_relation: finetune; heretic / uncensored tags added (library_name: transformers kept). Base badge, tagline, fact line (Type: Distilled + Abliterated MoE), fork paragraph, safetensors pointer, Related-models table, Credits, and the "uncensored base" known-limitation reframed to the Heretic lineage. Qwen/Qwen3.6-35B-A3B is still referenced as the upstream pre-Heretic base. Added a "Bundled blob status" callout and the Heretic-GGUF / MTP-Preserved variant rows. The Thanatos sibling row updated to its own new Heretic base.
- CITATION.cff: title, abstract, keywords, and references reframed to the Heretic base (immediate base + upstream base both cited).
- banner.svg subtitle → "Qwen 3.6 Heretic · MoE 35B/3B · Fable 5"; banner.png re-rasterized from the updated SVG.
- Bundled Janus-35B-A3B.Q4_K_M.gguf rebundled — replaced the legacy pre-Heretic Qwen quant with the Heretic Q4_K_M from llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF (qwen35moe-stamped; 40 layers / 256 experts / vocab 248,320 verified against the README Architecture). Serves the uncensored Heretic behavior directly.

Changed (support badge → Tokyo Night banner)

Replaced the shields.io "Buy me a coffee" badge in the README badge row with a custom bmc-banner.svg support banner, themed to match banner.svg (same Tokyo Night gradient, monospace wordmark, warm #e0af68 accent; animated rising steam, a pulsing hot-cup glow, a sweeping warm scan beam, a slow shimmer on "coffee", and a blinking cursor). Links to the unchanged buymeacoffee.com/cardoffoolm. Adds bmc-banner.svg; no other badge or link changed.

Changed (dense-sibling references reconciled → FoolDev/Thanatos-27B)

Sibling slug aligned to the canonical FoolDev/Thanatos-27B in the README "Related models" row, CITATION.cff references entry, and both Modelfile comments (previously the stale FoolDev/janus-27b / Thanatos-27B-Heretic). Per the sibling repo, Thanatos-27B-Heretic now 307-redirects to Thanatos-27B, which sits on the llmfan46/Qwen3.6-27B-uncensored-heretic-v2 Heretic base — so the README row cites that Heretic base and corrects the redirect direction.

Fixed

CITATION.cff self-reference repository-code / url corrected from the non-resolving https://huggingface.co/FoolDev/janus (lowercase, missing the -35B suffix — returns HTTP 401) to the canonical https://huggingface.co/FoolDev/Janus-35B used everywhere in the README.

Changed (teacher label — Opus 4.7 → Fable 5)

Reasoning-teacher references relabeled from Claude Opus 4.7 to Claude Fable 5 across the model card (TL;DR, architecture line, intro, Credits, and the "Related models" sibling row), banner.svg, and CITATION.cff. The external dataset slug crownelius/Opus-4.7-Reasoning in the metadata is left unchanged — it is a real Hugging Face dataset path, not a teacher label.

Changed (sibling rename — Thanatos-27B → Thanatos-27B-Heretic)

README "Related models" row updated to point at FoolDev/Thanatos-27B-Heretic (the dense sibling renamed from FoolDev/Thanatos-27B on 2026-05-23 along with a base swap to llmfan46/Qwen3.6-27B-uncensored-heretic-v2, an uncensored Heretic-style abliteration of Qwen 3.6 27B). Description now notes the Heretic base, the uncensored property, and the old-path 307 redirect HF serves.

Added

Root-level template, system, and params files for HF's Ollama bridge. The bridge generates Ollama manifests at request time from these three files (NOT from Modelfile — confirmed against https://huggingface.co/docs/hub/en/ollama). Without them, ollama run hf.co/FoolDev/Janus-35B got an auto-generated manifest with the broken {{ if .Prompt }} .Prompt }}<|im_end|> template (Ollama's faulty Go-template conversion of the GGUF's embedded jinja), corrupted stop tokens (".Prompt }}<|im_end|>" bleed), and no .Tools / .ToolCalls blocks — so the published Ollama tag advertised completion only and rejected any request with a tools array. The three files mirror the Modelfile's TEMPLATE / SYSTEM / PARAMETER directives; both routes wire tool calling correctly. Edit them together when changing one. Verified by re-pulling the fresh tag: ollama show hf.co/FoolDev/Janus-35B now reports completion, tools, thinking and tool calls round-trip end-to-end through /api/chat.

Changed

README "Tool / function calling" section: split into explicit Ollama-path and embedded-jinja-path subsections. Earlier wording conflated the two on-the-wire formats. The Ollama path (Modelfile TEMPLATE and the new template bridge file, both kept in sync) prompts JSON-in-XML — the form Ollama's tool-call extractor parses into a structured tool_calls array. The embedded-jinja path (llama.cpp, llama-cpp-python, LM Studio) reads the Qwen 3.6 native chat template baked into the GGUF, which prompts the verbose <function=name> / <parameter=arg> form the model was trained on. Both are valid; the model adapts to whichever shape the system prompt prescribes. README now shows both formats side by side.
README "Quick start / Ollama" section: documents both pull paths (hf.co/... via bridge files vs make ... -f Modelfile locally) and explicitly notes that HF's bridge does not read Modelfile.
README "Hardware requirements" intro: re-framed the "~~38 GB minimum" claim as "~~38 GB at default num_ctx 16384" and documented that 32 GB hosts can fit the model by trimming context and batch size.
README "Quick start / Ollama" snippet: show both ollama run hf.co/FoolDev/Janus-35B and the explicit-tag form ollama run hf.co/FoolDev/Janus-35B:Q4_K_M. Same blob (the default tag maps to Q4_K_M), but parity with the 27B sibling — which lists both :latest and :Q3_K_S — and removes ambiguity for users scripting against an explicit quant tag. Verified the explicit tag resolves to the same manifest (model SHA a076aa0d3a1a, bridge blobs 22c7ade72045 / 84a1a6ac580b / f7b1992cf9c1).

Added (cont'd)

README ## TL;DR section near the top of the model card, mirroring the 27B sibling. Two paths (HF Ollama bridge / local Modelfile build) with explicit tags and a one-line capability check. Notes the bridge ingests template / system / params, not Modelfile, so users skimming the top of the page won't form the wrong mental model of which file gets used when.
CITATION.cff for citation metadata (Apache-2.0, references the upstream Qwen3.6-35B-A3B base and the dense Janus-27B sibling). The 27B sibling has had this file since 0.5.0; adding here for parity so academic-style citations work across both repos.
LICENSE file containing the full Apache 2.0 text. The model card front-matter has always declared license: apache-2.0 and the upstream Qwen 3.6 license inherits Apache-2.0, but until now the repo lacked the actual license text file. Same Apache 2.0 text shipped in the 27B sibling.
scripts/check_bridge_sync.py — regression guard for the Modelfile <-> template / system / params sync invariant. The two configurations are consumed by different code paths (ollama create -f Modelfile for local builds vs HF's Ollama bridge for hf.co/... pulls — HF does not read Modelfile), so drift between them re-introduces the bug fixed in commit 70ccef1 where hf.co/FoolDev/Janus-35B shipped a broken auto-generated template while local builds had the correct one. Script parses the Modelfile's TEMPLATE / SYSTEM / PARAMETER directives, loads the three bridge files, and fails on any mismatch with a per-key diff. Run on demand before pushing edits to either side of the configuration. The 27B sibling wires an equivalent script into a pre-commit hook (commit 5c67b08); this repo stays leaner and runs it manually.

Fixed

README "Chat template" intro previously claimed all loaders handle the embedded jinja automatically. True for llama.cpp / LM Studio / llama-cpp-python; not true for Ollama, which needs an explicit override (the Modelfile TEMPLATE block locally, the root-level template file when serving via hf.co/...).
README "Tool / function calling" earlier said the XML form <function=name><parameter=arg> is "not what this model produces". That was wrong: the embedded GGUF jinja prompts exactly that form, and llama.cpp / LM Studio / llama-cpp-python users will see it. The "JSON-in-XML" claim only applies on the Ollama path because that's what the Modelfile TEMPLATE prompt instructs.

[0.1.0] — initial public release

Added

Model card with architecture overview, sampling defaults, hardware table, and Modelfile for ollama create janus -f Modelfile.
Bundled Janus-35B-A3B.Q4_K_M.gguf (~19 GB) via Git LFS so the HF "Use this model" widget surfaces a working ollama run snippet.
Tokyo Night themed banner (PNG sourced from the SVG).
Status badges for license, base model, architecture, quant.
Linked sibling FoolDev/janus-27b (dense Qwen 3.6 27B base) under Related models.