Instructions to use Jackrong/Qwopus3.6-35B-A3B-v1-MTP-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Jackrong/Qwopus3.6-35B-A3B-v1-MTP-GGUF with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Jackrong/Qwopus3.6-35B-A3B-v1-MTP-GGUF")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Jackrong/Qwopus3.6-35B-A3B-v1-MTP-GGUF", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Jackrong/Qwopus3.6-35B-A3B-v1-MTP-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Jackrong/Qwopus3.6-35B-A3B-v1-MTP-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jackrong/Qwopus3.6-35B-A3B-v1-MTP-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Jackrong/Qwopus3.6-35B-A3B-v1-MTP-GGUF

SGLang

How to use Jackrong/Qwopus3.6-35B-A3B-v1-MTP-GGUF with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Jackrong/Qwopus3.6-35B-A3B-v1-MTP-GGUF" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jackrong/Qwopus3.6-35B-A3B-v1-MTP-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Jackrong/Qwopus3.6-35B-A3B-v1-MTP-GGUF" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jackrong/Qwopus3.6-35B-A3B-v1-MTP-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Unsloth Studio

How to use Jackrong/Qwopus3.6-35B-A3B-v1-MTP-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Jackrong/Qwopus3.6-35B-A3B-v1-MTP-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Jackrong/Qwopus3.6-35B-A3B-v1-MTP-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Jackrong/Qwopus3.6-35B-A3B-v1-MTP-GGUF to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="Jackrong/Qwopus3.6-35B-A3B-v1-MTP-GGUF",
    max_seq_length=2048,
)

Docker Model Runner
How to use Jackrong/Qwopus3.6-35B-A3B-v1-MTP-GGUF with Docker Model Runner:
```
docker model run hf.co/Jackrong/Qwopus3.6-35B-A3B-v1-MTP-GGUF
```

Jackrong commited on 4 days ago

Commit

8643694

0 Parent(s):

Super-squash GGUF cleanup history

Browse files

Files changed (8) hide show

.gitattributes +48 -0
Qwopus3.6-35B-A3B-v1-MTP-Q3_K_M.gguf +3 -0
Qwopus3.6-35B-A3B-v1-MTP-Q4_K_M.gguf +3 -0
Qwopus3.6-35B-A3B-v1-MTP-Q5_K_M.gguf +3 -0
Qwopus3.6-35B-A3B-v1-MTP-Q6_K.gguf +3 -0
Qwopus3.6-35B-A3B-v1-MTP-Q8_0.gguf +3 -0
README.md +69 -0
mmproj-F32.gguf +3 -0

.gitattributes ADDED Viewed

	@@ -0,0 +1,48 @@

+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+Qwopus3.6-35B-A3B-v1-MTP-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Qwopus3.6-35B-A3B-v1-MTP-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwopus3.6-35B-A3B-v1-MTP-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwopus3.6-35B-A3B-v1-MTP-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Qwopus3.6-35B-A3B-v1-MTP-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Qwopus3.6-35B-A3B-v1-MTP-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwopus3.6-35B-A3B-v1-MTP-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwopus3.6-35B-A3B-v1-MTP-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwopus3.6-35B-A3B-v1-MTP-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwopus3.6-35B-A3B-v1-MTP-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+Qwopus3.6-35B-A3B-v1-MTP-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+Qwopus3.6-35B-A3B-v1-MTP-BF16.gguf filter=lfs diff=lfs merge=lfs -text
+mmproj-F32.gguf filter=lfs diff=lfs merge=lfs -text

Qwopus3.6-35B-A3B-v1-MTP-Q3_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7c150d81a409d2be6beacbdd7e1b933998b3874e0725443fef1742c31250dc2b
+size 17165606656

Qwopus3.6-35B-A3B-v1-MTP-Q4_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:54256ec02e6f4798b088e07232ef4bcd2f070fb8014799b7840281ae3bc5aa02
+size 21712410368

Qwopus3.6-35B-A3B-v1-MTP-Q5_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:137592fe30e8470cebcd3581fd5d240ecd2fb7cc559b7728c4b6b31bae89ec14
+size 25346479872

Qwopus3.6-35B-A3B-v1-MTP-Q6_K.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:66ec6ca41c71f40b112ab346eb8ed67cbdf43cea1297acfa1402e4180759308d
+size 29207678720

Qwopus3.6-35B-A3B-v1-MTP-Q8_0.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5c43eed390935e26b1851973d3cd5bb929cb22393dc7e6186636a759b7f77627
+size 37801096960

README.md ADDED Viewed

	@@ -0,0 +1,69 @@

+---
+library_name: transformers
+base_model:
+- unsloth/Qwen3.6-35B-A3B
+tags:
+- gguf
+- llama.cpp
+- image-text-to-text
+- vision
+- multimodal
+- text-generation-inference
+- transformers
+- unsloth
+- conversational
+- mtp
+- multi-token-prediction
+- speculative-decoding
+- qwen3_6
+- moe
+- reasoning
+- chain-of-thought
+- lora
+- sft
+- tool-use
+- function-calling
+- long-context
+license: apache-2.0
+language:
+- en
+- zh
+- es
+- ru
+- ja
+pipeline_tag: image-text-to-text
+datasets:
+- Jackrong/Claude-opus-4.7-TraceInversion-5000x
+- Jackrong/Claude-opus-4.6-TraceInversion-9000x
+---
+# Jackrong/Qwopus3.6-35B-A3B-v1-MTP-GGUF
+<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; border: 1px solid #93c5fd; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02); margin-bottom: 20px;">
+  <div style="background: linear-gradient(135deg, #3b82f6 0%, #1d4ed8 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;">
+    <span>⚡</span> What is MTP (Multi-Token Prediction)?
+  </div>
+  <div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.6;">
+    MTP (Multi-Token Prediction) is a technique introduced in the Qwen3.6 architecture that enables the model to predict multiple future tokens simultaneously. By leveraging dedicated MTP heads, this model supports <b>speculative decoding</b>, where a draft model predicts multiple tokens at once and the target model verifies them in parallel, resulting in significant inference speedups without sacrificing output quality.
+    <br><br>
+    This GGUF release preserves the MTP heads from <code>unsloth/Qwen3.6-35B-A3B</code>, making it compatible with mainstream inference frameworks that support MTP-based speculative decoding (such as llama.cpp and its derivatives). For optimal throughput, pair this MTP-enabled GGUF with a corresponding draft model.
+  </div>
+</div>
+Source model: `Jackrong/Qwopus3.6-35B-A3B-v1`
+MTP source: `unsloth/Qwen3.6-35B-A3B`
+Uploaded GGUF variants:
+- `Qwopus3.6-35B-A3B-v1-MTP-Q2_K.gguf`
+- `Qwopus3.6-35B-A3B-v1-MTP-Q3_K_S.gguf`
+- `Qwopus3.6-35B-A3B-v1-MTP-Q3_K_M.gguf`
+- `Qwopus3.6-35B-A3B-v1-MTP-Q3_K_L.gguf`
+- `Qwopus3.6-35B-A3B-v1-MTP-IQ4_XS.gguf`
+- `Qwopus3.6-35B-A3B-v1-MTP-Q4_K_S.gguf`
+- `Qwopus3.6-35B-A3B-v1-MTP-Q4_K_M.gguf`
+- `Qwopus3.6-35B-A3B-v1-MTP-Q5_K_S.gguf`
+- `Qwopus3.6-35B-A3B-v1-MTP-Q5_K_M.gguf`
+- `Qwopus3.6-35B-A3B-v1-MTP-Q6_K.gguf`
+- `Qwopus3.6-35B-A3B-v1-MTP-Q8_0.gguf`
+- `Qwopus3.6-35B-A3B-v1-MTP-BF16.gguf`
+This release was prepared by validating or injecting Qwen MTP/nextn tensors before GGUF conversion.

mmproj-F32.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:56c89f1ca1547a8a15066642f54b94e4911e3c86cccb3d88163d823e8b6b8799
+size 902822560