Image-Text-to-Text
Transformers
GGUF
qwen36
Mixture of Experts
conversational
multimodal
agent
heretic
uncensored
reasoning
distillation
Instructions to use FoolDev/Janus-35B-HERETIC with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use FoolDev/Janus-35B-HERETIC with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="FoolDev/Janus-35B-HERETIC") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("FoolDev/Janus-35B-HERETIC", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use FoolDev/Janus-35B-HERETIC with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M # Run inference directly in the terminal: llama cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M # Run inference directly in the terminal: llama cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf FoolDev/Janus-35B-HERETIC:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf FoolDev/Janus-35B-HERETIC:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Use Docker
docker model run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use FoolDev/Janus-35B-HERETIC with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "FoolDev/Janus-35B-HERETIC" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FoolDev/Janus-35B-HERETIC", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
- SGLang
How to use FoolDev/Janus-35B-HERETIC with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "FoolDev/Janus-35B-HERETIC" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FoolDev/Janus-35B-HERETIC", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "FoolDev/Janus-35B-HERETIC" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FoolDev/Janus-35B-HERETIC", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Ollama
How to use FoolDev/Janus-35B-HERETIC with Ollama:
ollama run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
- Unsloth Studio
How to use FoolDev/Janus-35B-HERETIC with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for FoolDev/Janus-35B-HERETIC to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for FoolDev/Janus-35B-HERETIC to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for FoolDev/Janus-35B-HERETIC to start chatting
- Pi
How to use FoolDev/Janus-35B-HERETIC with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "FoolDev/Janus-35B-HERETIC:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use FoolDev/Janus-35B-HERETIC with Docker Model Runner:
docker model run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
- Lemonade
How to use FoolDev/Janus-35B-HERETIC with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull FoolDev/Janus-35B-HERETIC:Q4_K_M
Run and chat with the model
lemonade run user.Janus-35B-HERETIC-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use FoolDev/Janus-35B-HERETIC with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default FoolDev/Janus-35B-HERETIC:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use FoolDev/Janus-35B-HERETIC with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "FoolDev/Janus-35B-HERETIC:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
File size: 12,100 Bytes
64b629a f64cd2a 192be81 b302576 192be81 8ae1e81 b302576 8ae1e81 b558907 0bd5c47 b302576 0bd5c47 52b76d1 64b629a b558907 64b629a b558907 64b629a b558907 64b629a b558907 64b629a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 | # Changelog
All notable changes to this repository. Format loosely follows
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/). This repo holds
a model card, an Ollama Modelfile, the HF Ollama-bridge `template` /
`system` / `params` files, and the bundled Q4_K_M GGUF, so versions
track the **tooling and documentation**, not the underlying base model.
## [Unreleased]
### Changed (HF repo rename β FoolDev/Janus-35B-HERETIC)
- **HF repository slug renamed `FoolDev/Janus-35B` β `FoolDev/Janus-35B-HERETIC`.**
The old slug now 307-redirects to the new one. The model's **display name
stays `Janus-35B`** β only the repo path / URLs change. Repointed every
live self-reference to the new slug: the README banner / `bmc-banner` /
`moe-routing` `resolve/main/...` image URLs, the `ollama run hf.co/...`
pull one-liners (TL;DR + Ollama Quick start), the `git clone` +
`cd` line, the bridge-files table row, and `CITATION.cff`'s
`repository-code` / `url`. `scripts/build.sh` defaults (`REPO_ID`
upstream `llmfan46/...`, local `janus` Ollama tag) don't reference the
FoolDev slug, so they were left as-is. Also repointed the dense-sibling
cross-links (README "Related models" row + `CITATION.cff` references entry)
to `FoolDev/Thanatos-27B-HERETIC`, and set the git remote to
`git@hf.co:FoolDev/Janus-35B-HERETIC`. Past-tense CHANGELOG history
(redirect notes, the earlier lowercase-`janus` fix) is preserved verbatim.
### Added
- **`scripts/build.sh`** β self-contained builder that pulls a GGUF from
[`llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF`](https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF)
(default `Q4_K_M`), patches the `Modelfile` `FROM` line in a temp copy, and
runs `ollama create janus`. Mirrors the dense sibling's (`FoolDev/Thanatos-27B`)
`build.sh`, adapted for the 35B-A3B MoE, and gives Janus a reproducible path to
build other quants locally. README "What's here" table and the
bundled-blob-status callout now point at it.
### Changed (base swap β llmfan46 Heretic)
- **Swapped the immediate base from `Qwen/Qwen3.6-35B-A3B` (vanilla) β
[`llmfan46/Qwen3.6-35B-A3B-uncensored-heretic`](https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic)**
β an uncensored "Heretic" abliteration of the same Qwen 3.6 35B-A3B MoE
(`qwen3_5_moe`-stamped like the vanilla base, so the Architecture section,
the MoE-routing framing, and the bundled blob's loadability are all
unchanged). Mirrors the dense sibling's swap (`FoolDev/Thanatos-27B` β
`llmfan46/Qwen3.6-27B-uncensored-heretic-v2`). Repo name `Janus-35B`, the
`janus` Ollama tag, the `FoolDev/Janus-35B` URL, and the Claude Fable 5
teacher slot are all unchanged.
- README: frontmatter `base_model:` + `base_model_relation: finetune`;
`heretic` / `uncensored` tags added (`library_name: transformers` kept).
Base badge, tagline, fact line (`Type: Distilled + Abliterated MoE`), fork
paragraph, safetensors pointer, Related-models table, Credits, and the
"uncensored base" known-limitation reframed to the Heretic lineage.
`Qwen/Qwen3.6-35B-A3B` is still referenced as the upstream pre-Heretic base.
Added a "Bundled blob status" callout and the Heretic-GGUF /
MTP-Preserved variant rows. The Thanatos sibling row updated to its own
new Heretic base.
- `CITATION.cff`: title, abstract, keywords, and references reframed to the
Heretic base (immediate base + upstream base both cited).
- `banner.svg` subtitle β "Qwen 3.6 Heretic Β· MoE 35B/3B Β· Fable 5";
`banner.png` re-rasterized from the updated SVG.
- **Bundled `Janus-35B-A3B.Q4_K_M.gguf` rebundled** β replaced the legacy
pre-Heretic Qwen quant with the Heretic Q4_K_M from
`llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF` (`qwen35moe`-stamped;
40 layers / 256 experts / vocab 248,320 verified against the README
Architecture). Serves the uncensored Heretic behavior directly.
### Changed (support badge β Tokyo Night banner)
- **Replaced the shields.io "Buy me a coffee" badge** in the README badge
row with a custom `bmc-banner.svg` support banner, themed to match
`banner.svg` (same Tokyo Night gradient, monospace wordmark, warm
`#e0af68` accent; animated rising steam, a pulsing hot-cup glow, a
sweeping warm scan beam, a slow shimmer on "coffee", and a blinking
cursor). Links to the
unchanged `buymeacoffee.com/cardoffoolm`. Adds `bmc-banner.svg`; no
other badge or link changed.
### Changed (dense-sibling references reconciled β FoolDev/Thanatos-27B)
- **Sibling slug aligned to the canonical `FoolDev/Thanatos-27B`** in the
README "Related models" row, `CITATION.cff` references entry, and both
`Modelfile` comments (previously the stale `FoolDev/janus-27b` /
`Thanatos-27B-Heretic`). Per the sibling repo, `Thanatos-27B-Heretic`
now 307-redirects to `Thanatos-27B`, which sits on the
`llmfan46/Qwen3.6-27B-uncensored-heretic-v2` Heretic base β so the
README row cites that Heretic base and corrects the redirect
direction.
### Fixed
- **CITATION.cff self-reference** `repository-code` / `url` corrected from
the non-resolving `https://huggingface.co/FoolDev/janus` (lowercase,
missing the `-35B` suffix β returns HTTP 401) to the canonical
`https://huggingface.co/FoolDev/Janus-35B` used everywhere in the
README.
### Changed (teacher label β Opus 4.7 β Fable 5)
- **Reasoning-teacher references** relabeled from `Claude Opus 4.7` to
`Claude Fable 5` across the model card (TL;DR, architecture line,
intro, Credits, and the "Related models" sibling row), `banner.svg`,
and `CITATION.cff`. The external dataset slug
`crownelius/Opus-4.7-Reasoning` in the metadata is left unchanged β
it is a real Hugging Face dataset path, not a teacher label.
### Changed (sibling rename β Thanatos-27B β Thanatos-27B-Heretic)
- **README "Related models" row** updated to point at
`FoolDev/Thanatos-27B-Heretic` (the dense sibling renamed from
`FoolDev/Thanatos-27B` on 2026-05-23 along with a base swap to
`llmfan46/Qwen3.6-27B-uncensored-heretic-v2`, an uncensored
Heretic-style abliteration of Qwen 3.6 27B). Description now
notes the Heretic base, the uncensored property, and the
old-path 307 redirect HF serves.
### Added
- Root-level `template`, `system`, and `params` files for HF's Ollama
bridge. The bridge generates Ollama manifests at request time from
these three files (NOT from `Modelfile` β confirmed against
https://huggingface.co/docs/hub/en/ollama). Without them, `ollama
run hf.co/FoolDev/Janus-35B` got an auto-generated manifest with the
broken `{{ if .Prompt }} .Prompt }}<|im_end|>` template (Ollama's
faulty Go-template conversion of the GGUF's embedded jinja),
corrupted stop tokens (`".Prompt }}<|im_end|>"` bleed), and no
`.Tools` / `.ToolCalls` blocks β so the published Ollama tag
advertised `completion` only and rejected any request with a
`tools` array. The three files mirror the `Modelfile`'s `TEMPLATE`
/ `SYSTEM` / `PARAMETER` directives; both routes wire tool calling
correctly. Edit them together when changing one. Verified by
re-pulling the fresh tag: `ollama show hf.co/FoolDev/Janus-35B` now
reports `completion`, `tools`, `thinking` and tool calls round-trip
end-to-end through `/api/chat`.
### Changed
- README "Tool / function calling" section: split into explicit
Ollama-path and embedded-jinja-path subsections. Earlier wording
conflated the two on-the-wire formats. The Ollama path (Modelfile
`TEMPLATE` and the new `template` bridge file, both kept in sync)
prompts JSON-in-XML β the form Ollama's tool-call extractor parses
into a structured `tool_calls` array. The embedded-jinja path
(llama.cpp, llama-cpp-python, LM Studio) reads the Qwen 3.6 native
chat template baked into the GGUF, which prompts the verbose
`<function=name>` / `<parameter=arg>` form the model was trained
on. Both are valid; the model adapts to whichever shape the system
prompt prescribes. README now shows both formats side by side.
- README "Quick start / Ollama" section: documents both pull paths
(`hf.co/...` via bridge files vs `make ... -f Modelfile` locally)
and explicitly notes that HF's bridge does not read `Modelfile`.
- README "Hardware requirements" intro: re-framed the "~38 GB
minimum" claim as "~38 GB at default `num_ctx 16384`" and
documented that 32 GB hosts can fit the model by trimming context
and batch size.
- README "Quick start / Ollama" snippet: show both
`ollama run hf.co/FoolDev/Janus-35B` and the explicit-tag form
`ollama run hf.co/FoolDev/Janus-35B:Q4_K_M`. Same blob (the default
tag maps to Q4_K_M), but parity with the 27B sibling β which lists
both `:latest` and `:Q3_K_S` β and removes ambiguity for users
scripting against an explicit quant tag. Verified the explicit tag
resolves to the same manifest (model SHA `a076aa0d3a1a`, bridge
blobs `22c7ade72045` / `84a1a6ac580b` / `f7b1992cf9c1`).
### Added (cont'd)
- README `## TL;DR` section near the top of the model card, mirroring
the 27B sibling. Two paths (HF Ollama bridge / local Modelfile
build) with explicit tags and a one-line capability check. Notes
the bridge ingests `template` / `system` / `params`, not
`Modelfile`, so users skimming the top of the page won't form the
wrong mental model of which file gets used when.
- `CITATION.cff` for citation metadata (Apache-2.0, references the
upstream Qwen3.6-35B-A3B base and the dense Janus-27B sibling).
The 27B sibling has had this file since 0.5.0; adding here for
parity so academic-style citations work across both repos.
- `LICENSE` file containing the full Apache 2.0 text. The model card
front-matter has always declared `license: apache-2.0` and the
upstream Qwen 3.6 license inherits Apache-2.0, but until now the
repo lacked the actual license text file. Same Apache 2.0 text
shipped in the 27B sibling.
- `scripts/check_bridge_sync.py` β regression guard for the
`Modelfile` <-> `template` / `system` / `params` sync invariant.
The two configurations are consumed by different code paths
(`ollama create -f Modelfile` for local builds vs HF's Ollama
bridge for `hf.co/...` pulls β HF does not read `Modelfile`), so
drift between them re-introduces the bug fixed in commit 70ccef1
where `hf.co/FoolDev/Janus-35B` shipped a broken auto-generated
template while local builds had the correct one. Script parses
the Modelfile's `TEMPLATE` / `SYSTEM` / `PARAMETER` directives,
loads the three bridge files, and fails on any mismatch with a
per-key diff. Run on demand before pushing edits to either side
of the configuration. The 27B sibling wires an equivalent script
into a pre-commit hook (commit 5c67b08); this repo stays leaner
and runs it manually.
### Fixed
- README "Chat template" intro previously claimed all loaders handle
the embedded jinja automatically. True for llama.cpp / LM Studio /
llama-cpp-python; not true for Ollama, which needs an explicit
override (the `Modelfile` TEMPLATE block locally, the root-level
`template` file when serving via `hf.co/...`).
- README "Tool / function calling" earlier said the XML form
`<function=name><parameter=arg>` is "not what this model produces".
That was wrong: the embedded GGUF jinja prompts exactly that form,
and llama.cpp / LM Studio / llama-cpp-python users will see it.
The "JSON-in-XML" claim only applies on the Ollama path because
that's what the Modelfile TEMPLATE prompt instructs.
## [0.1.0] β initial public release
### Added
- Model card with architecture overview, sampling defaults, hardware
table, and `Modelfile` for `ollama create janus -f Modelfile`.
- Bundled `Janus-35B-A3B.Q4_K_M.gguf` (~19 GB) via Git LFS so the HF
"Use this model" widget surfaces a working `ollama run` snippet.
- Tokyo Night themed banner (PNG sourced from the SVG).
- Status badges for license, base model, architecture, quant.
- Linked sibling `FoolDev/janus-27b` (dense Qwen 3.6 27B base) under
Related models.
|