Image-Text-to-Text
Transformers
GGUF
qwen36
Mixture of Experts
conversational
multimodal
agent
heretic
uncensored
reasoning
distillation
Instructions to use FoolDev/Janus-35B-HERETIC with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use FoolDev/Janus-35B-HERETIC with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="FoolDev/Janus-35B-HERETIC") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("FoolDev/Janus-35B-HERETIC", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use FoolDev/Janus-35B-HERETIC with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M # Run inference directly in the terminal: llama cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M # Run inference directly in the terminal: llama cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf FoolDev/Janus-35B-HERETIC:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf FoolDev/Janus-35B-HERETIC:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Use Docker
docker model run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use FoolDev/Janus-35B-HERETIC with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "FoolDev/Janus-35B-HERETIC" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FoolDev/Janus-35B-HERETIC", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
- SGLang
How to use FoolDev/Janus-35B-HERETIC with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "FoolDev/Janus-35B-HERETIC" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FoolDev/Janus-35B-HERETIC", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "FoolDev/Janus-35B-HERETIC" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FoolDev/Janus-35B-HERETIC", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Ollama
How to use FoolDev/Janus-35B-HERETIC with Ollama:
ollama run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
- Unsloth Studio
How to use FoolDev/Janus-35B-HERETIC with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for FoolDev/Janus-35B-HERETIC to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for FoolDev/Janus-35B-HERETIC to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for FoolDev/Janus-35B-HERETIC to start chatting
- Pi
How to use FoolDev/Janus-35B-HERETIC with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "FoolDev/Janus-35B-HERETIC:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use FoolDev/Janus-35B-HERETIC with Docker Model Runner:
docker model run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
- Lemonade
How to use FoolDev/Janus-35B-HERETIC with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull FoolDev/Janus-35B-HERETIC:Q4_K_M
Run and chat with the model
lemonade run user.Janus-35B-HERETIC-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use FoolDev/Janus-35B-HERETIC with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default FoolDev/Janus-35B-HERETIC:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use FoolDev/Janus-35B-HERETIC with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "FoolDev/Janus-35B-HERETIC:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
| # Changelog | |
| All notable changes to this repository. Format loosely follows | |
| [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). This repo holds | |
| a model card, an Ollama Modelfile, the HF Ollama-bridge `template` / | |
| `system` / `params` files, and the bundled Q4_K_M GGUF, so versions | |
| track the **tooling and documentation**, not the underlying base model. | |
| ## [Unreleased] | |
| ### Changed (HF repo rename β FoolDev/Janus-35B-HERETIC) | |
| - **HF repository slug renamed `FoolDev/Janus-35B` β `FoolDev/Janus-35B-HERETIC`.** | |
| The old slug now 307-redirects to the new one. The model's **display name | |
| stays `Janus-35B`** β only the repo path / URLs change. Repointed every | |
| live self-reference to the new slug: the README banner / `bmc-banner` / | |
| `moe-routing` `resolve/main/...` image URLs, the `ollama run hf.co/...` | |
| pull one-liners (TL;DR + Ollama Quick start), the `git clone` + | |
| `cd` line, the bridge-files table row, and `CITATION.cff`'s | |
| `repository-code` / `url`. `scripts/build.sh` defaults (`REPO_ID` | |
| upstream `llmfan46/...`, local `janus` Ollama tag) don't reference the | |
| FoolDev slug, so they were left as-is. Also repointed the dense-sibling | |
| cross-links (README "Related models" row + `CITATION.cff` references entry) | |
| to `FoolDev/Thanatos-27B-HERETIC`, and set the git remote to | |
| `git@hf.co:FoolDev/Janus-35B-HERETIC`. Past-tense CHANGELOG history | |
| (redirect notes, the earlier lowercase-`janus` fix) is preserved verbatim. | |
| ### Added | |
| - **`scripts/build.sh`** β self-contained builder that pulls a GGUF from | |
| [`llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF`](https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF) | |
| (default `Q4_K_M`), patches the `Modelfile` `FROM` line in a temp copy, and | |
| runs `ollama create janus`. Mirrors the dense sibling's (`FoolDev/Thanatos-27B`) | |
| `build.sh`, adapted for the 35B-A3B MoE, and gives Janus a reproducible path to | |
| build other quants locally. README "What's here" table and the | |
| bundled-blob-status callout now point at it. | |
| ### Changed (base swap β llmfan46 Heretic) | |
| - **Swapped the immediate base from `Qwen/Qwen3.6-35B-A3B` (vanilla) β | |
| [`llmfan46/Qwen3.6-35B-A3B-uncensored-heretic`](https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic)** | |
| β an uncensored "Heretic" abliteration of the same Qwen 3.6 35B-A3B MoE | |
| (`qwen3_5_moe`-stamped like the vanilla base, so the Architecture section, | |
| the MoE-routing framing, and the bundled blob's loadability are all | |
| unchanged). Mirrors the dense sibling's swap (`FoolDev/Thanatos-27B` β | |
| `llmfan46/Qwen3.6-27B-uncensored-heretic-v2`). Repo name `Janus-35B`, the | |
| `janus` Ollama tag, the `FoolDev/Janus-35B` URL, and the Claude Fable 5 | |
| teacher slot are all unchanged. | |
| - README: frontmatter `base_model:` + `base_model_relation: finetune`; | |
| `heretic` / `uncensored` tags added (`library_name: transformers` kept). | |
| Base badge, tagline, fact line (`Type: Distilled + Abliterated MoE`), fork | |
| paragraph, safetensors pointer, Related-models table, Credits, and the | |
| "uncensored base" known-limitation reframed to the Heretic lineage. | |
| `Qwen/Qwen3.6-35B-A3B` is still referenced as the upstream pre-Heretic base. | |
| Added a "Bundled blob status" callout and the Heretic-GGUF / | |
| MTP-Preserved variant rows. The Thanatos sibling row updated to its own | |
| new Heretic base. | |
| - `CITATION.cff`: title, abstract, keywords, and references reframed to the | |
| Heretic base (immediate base + upstream base both cited). | |
| - `banner.svg` subtitle β "Qwen 3.6 Heretic Β· MoE 35B/3B Β· Fable 5"; | |
| `banner.png` re-rasterized from the updated SVG. | |
| - **Bundled `Janus-35B-A3B.Q4_K_M.gguf` rebundled** β replaced the legacy | |
| pre-Heretic Qwen quant with the Heretic Q4_K_M from | |
| `llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF` (`qwen35moe`-stamped; | |
| 40 layers / 256 experts / vocab 248,320 verified against the README | |
| Architecture). Serves the uncensored Heretic behavior directly. | |
| ### Changed (support badge β Tokyo Night banner) | |
| - **Replaced the shields.io "Buy me a coffee" badge** in the README badge | |
| row with a custom `bmc-banner.svg` support banner, themed to match | |
| `banner.svg` (same Tokyo Night gradient, monospace wordmark, warm | |
| `#e0af68` accent; animated rising steam, a pulsing hot-cup glow, a | |
| sweeping warm scan beam, a slow shimmer on "coffee", and a blinking | |
| cursor). Links to the | |
| unchanged `buymeacoffee.com/cardoffoolm`. Adds `bmc-banner.svg`; no | |
| other badge or link changed. | |
| ### Changed (dense-sibling references reconciled β FoolDev/Thanatos-27B) | |
| - **Sibling slug aligned to the canonical `FoolDev/Thanatos-27B`** in the | |
| README "Related models" row, `CITATION.cff` references entry, and both | |
| `Modelfile` comments (previously the stale `FoolDev/janus-27b` / | |
| `Thanatos-27B-Heretic`). Per the sibling repo, `Thanatos-27B-Heretic` | |
| now 307-redirects to `Thanatos-27B`, which sits on the | |
| `llmfan46/Qwen3.6-27B-uncensored-heretic-v2` Heretic base β so the | |
| README row cites that Heretic base and corrects the redirect | |
| direction. | |
| ### Fixed | |
| - **CITATION.cff self-reference** `repository-code` / `url` corrected from | |
| the non-resolving `https://huggingface.co/FoolDev/janus` (lowercase, | |
| missing the `-35B` suffix β returns HTTP 401) to the canonical | |
| `https://huggingface.co/FoolDev/Janus-35B` used everywhere in the | |
| README. | |
| ### Changed (teacher label β Opus 4.7 β Fable 5) | |
| - **Reasoning-teacher references** relabeled from `Claude Opus 4.7` to | |
| `Claude Fable 5` across the model card (TL;DR, architecture line, | |
| intro, Credits, and the "Related models" sibling row), `banner.svg`, | |
| and `CITATION.cff`. The external dataset slug | |
| `crownelius/Opus-4.7-Reasoning` in the metadata is left unchanged β | |
| it is a real Hugging Face dataset path, not a teacher label. | |
| ### Changed (sibling rename β Thanatos-27B β Thanatos-27B-Heretic) | |
| - **README "Related models" row** updated to point at | |
| `FoolDev/Thanatos-27B-Heretic` (the dense sibling renamed from | |
| `FoolDev/Thanatos-27B` on 2026-05-23 along with a base swap to | |
| `llmfan46/Qwen3.6-27B-uncensored-heretic-v2`, an uncensored | |
| Heretic-style abliteration of Qwen 3.6 27B). Description now | |
| notes the Heretic base, the uncensored property, and the | |
| old-path 307 redirect HF serves. | |
| ### Added | |
| - Root-level `template`, `system`, and `params` files for HF's Ollama | |
| bridge. The bridge generates Ollama manifests at request time from | |
| these three files (NOT from `Modelfile` β confirmed against | |
| https://huggingface.co/docs/hub/en/ollama). Without them, `ollama | |
| run hf.co/FoolDev/Janus-35B` got an auto-generated manifest with the | |
| broken `{{ if .Prompt }} .Prompt }}<|im_end|>` template (Ollama's | |
| faulty Go-template conversion of the GGUF's embedded jinja), | |
| corrupted stop tokens (`".Prompt }}<|im_end|>"` bleed), and no | |
| `.Tools` / `.ToolCalls` blocks β so the published Ollama tag | |
| advertised `completion` only and rejected any request with a | |
| `tools` array. The three files mirror the `Modelfile`'s `TEMPLATE` | |
| / `SYSTEM` / `PARAMETER` directives; both routes wire tool calling | |
| correctly. Edit them together when changing one. Verified by | |
| re-pulling the fresh tag: `ollama show hf.co/FoolDev/Janus-35B` now | |
| reports `completion`, `tools`, `thinking` and tool calls round-trip | |
| end-to-end through `/api/chat`. | |
| ### Changed | |
| - README "Tool / function calling" section: split into explicit | |
| Ollama-path and embedded-jinja-path subsections. Earlier wording | |
| conflated the two on-the-wire formats. The Ollama path (Modelfile | |
| `TEMPLATE` and the new `template` bridge file, both kept in sync) | |
| prompts JSON-in-XML β the form Ollama's tool-call extractor parses | |
| into a structured `tool_calls` array. The embedded-jinja path | |
| (llama.cpp, llama-cpp-python, LM Studio) reads the Qwen 3.6 native | |
| chat template baked into the GGUF, which prompts the verbose | |
| `<function=name>` / `<parameter=arg>` form the model was trained | |
| on. Both are valid; the model adapts to whichever shape the system | |
| prompt prescribes. README now shows both formats side by side. | |
| - README "Quick start / Ollama" section: documents both pull paths | |
| (`hf.co/...` via bridge files vs `make ... -f Modelfile` locally) | |
| and explicitly notes that HF's bridge does not read `Modelfile`. | |
| - README "Hardware requirements" intro: re-framed the "~38 GB | |
| minimum" claim as "~38 GB at default `num_ctx 16384`" and | |
| documented that 32 GB hosts can fit the model by trimming context | |
| and batch size. | |
| - README "Quick start / Ollama" snippet: show both | |
| `ollama run hf.co/FoolDev/Janus-35B` and the explicit-tag form | |
| `ollama run hf.co/FoolDev/Janus-35B:Q4_K_M`. Same blob (the default | |
| tag maps to Q4_K_M), but parity with the 27B sibling β which lists | |
| both `:latest` and `:Q3_K_S` β and removes ambiguity for users | |
| scripting against an explicit quant tag. Verified the explicit tag | |
| resolves to the same manifest (model SHA `a076aa0d3a1a`, bridge | |
| blobs `22c7ade72045` / `84a1a6ac580b` / `f7b1992cf9c1`). | |
| ### Added (cont'd) | |
| - README `## TL;DR` section near the top of the model card, mirroring | |
| the 27B sibling. Two paths (HF Ollama bridge / local Modelfile | |
| build) with explicit tags and a one-line capability check. Notes | |
| the bridge ingests `template` / `system` / `params`, not | |
| `Modelfile`, so users skimming the top of the page won't form the | |
| wrong mental model of which file gets used when. | |
| - `CITATION.cff` for citation metadata (Apache-2.0, references the | |
| upstream Qwen3.6-35B-A3B base and the dense Janus-27B sibling). | |
| The 27B sibling has had this file since 0.5.0; adding here for | |
| parity so academic-style citations work across both repos. | |
| - `LICENSE` file containing the full Apache 2.0 text. The model card | |
| front-matter has always declared `license: apache-2.0` and the | |
| upstream Qwen 3.6 license inherits Apache-2.0, but until now the | |
| repo lacked the actual license text file. Same Apache 2.0 text | |
| shipped in the 27B sibling. | |
| - `scripts/check_bridge_sync.py` β regression guard for the | |
| `Modelfile` <-> `template` / `system` / `params` sync invariant. | |
| The two configurations are consumed by different code paths | |
| (`ollama create -f Modelfile` for local builds vs HF's Ollama | |
| bridge for `hf.co/...` pulls β HF does not read `Modelfile`), so | |
| drift between them re-introduces the bug fixed in commit 70ccef1 | |
| where `hf.co/FoolDev/Janus-35B` shipped a broken auto-generated | |
| template while local builds had the correct one. Script parses | |
| the Modelfile's `TEMPLATE` / `SYSTEM` / `PARAMETER` directives, | |
| loads the three bridge files, and fails on any mismatch with a | |
| per-key diff. Run on demand before pushing edits to either side | |
| of the configuration. The 27B sibling wires an equivalent script | |
| into a pre-commit hook (commit 5c67b08); this repo stays leaner | |
| and runs it manually. | |
| ### Fixed | |
| - README "Chat template" intro previously claimed all loaders handle | |
| the embedded jinja automatically. True for llama.cpp / LM Studio / | |
| llama-cpp-python; not true for Ollama, which needs an explicit | |
| override (the `Modelfile` TEMPLATE block locally, the root-level | |
| `template` file when serving via `hf.co/...`). | |
| - README "Tool / function calling" earlier said the XML form | |
| `<function=name><parameter=arg>` is "not what this model produces". | |
| That was wrong: the embedded GGUF jinja prompts exactly that form, | |
| and llama.cpp / LM Studio / llama-cpp-python users will see it. | |
| The "JSON-in-XML" claim only applies on the Ollama path because | |
| that's what the Modelfile TEMPLATE prompt instructs. | |
| ## [0.1.0] β initial public release | |
| ### Added | |
| - Model card with architecture overview, sampling defaults, hardware | |
| table, and `Modelfile` for `ollama create janus -f Modelfile`. | |
| - Bundled `Janus-35B-A3B.Q4_K_M.gguf` (~19 GB) via Git LFS so the HF | |
| "Use this model" widget surfaces a working `ollama run` snippet. | |
| - Tokyo Night themed banner (PNG sourced from the SVG). | |
| - Status badges for license, base model, architecture, quant. | |
| - Linked sibling `FoolDev/janus-27b` (dense Qwen 3.6 27B base) under | |
| Related models. | |