deadbydawn101
/

RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF

Instructions to use deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF",
	filename="RavenX-CyberAgent-35B-v5.1-Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF:Q4_K_M

Use Docker

docker model run hf.co/deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF:Q4_K_M

Ollama
How to use deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF with Ollama:
```
ollama run hf.co/deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF:Q4_K_M
```

Unsloth Studio

How to use deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF to start chatting

How to use deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF with Docker Model Runner:
```
docker model run hf.co/deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF:Q4_K_M
```

Lemonade

How to use deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF-Q4_K_M

List all available models

lemonade list

the new Ollama model gguf canfnot run

by minhdang - opened 17 days ago

Discussion

minhdang

17 days ago

when i run your gguf in ollama , this have erorr:
llama_model_load: error loading model: missing tensor 'blk.40.attn_norm.weight'
llama_model_load_from_file_impl: failed to load model
common_init_from_params: failed to load model '/root/.ollama/models/blobs/sha256-05ad179a79d717a2fcb5db7a404152fe46ec873fc46558cb229e235a568a6003'
srv load_model: failed to load model, '/root/.ollama/models/blobs/sha256-05ad179a79d717a2fcb5db7a404152fe46ec873fc46558cb229e235a568a6003'
srv operator(): operator(): cleaning up before exit...
srv llama_server: exiting due to model loading error
time=2026-06-12T08:25:59.430Z level=ERROR source=llama_server.go:861 msg="llama-server terminated" error="exit status 1" exit=1
time=2026-06-12T08:25:59.539Z level=INFO source=sched.go:641 msg="Load failed" model=/root/.ollama/models/blobs/sha256-05ad179a79d717a2fcb5db7a404152fe46ec873fc46558cb229e235a568a6003 error="llama-server process has terminated: exit status 1: error loading model: missing tensor 'blk.40.attn_norm.weight'\nerror loading model: missing tensor 'blk.40.attn_norm.weight'"

pccr10001

17 days ago

same issue

Scottbyrd

17 days ago

•

edited 17 days ago

Same issue here too in LM Studio (version 0.4.17 Build 1 with llama.cpp Cuda 12 v2.22.0).

The v5.1 of this model was good (besides a few thinking loops like most Qwen models end up doing, but nothing too bad), and seeing this new version, I deleted the old version and downloaded this one. ~~Sadly, the old v5.1 is no longer on huggingface for download, so we gotta wait for the v6.2 gguf to be fixed.~~

Edit: The old v5.1 model can be found at this link, found in the repo's commit history (just in case anyone still wants to test it out until the v6.2 model gets fixed):
https://huggingface.co/deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-GGUF/tree/02ac0ccbf7ba9a3cd2b89fbec077192d2cb717d6

Great work on the models so far @ deadbydawn101, I very much look forward to testing this model once it's working again.

minhdang

17 days ago

me too, v5.1 this model was good

Repent0947

17 days ago

I can confirm the current v6.2 Q4_K_M GGUF is still broken.

Tested current main:

File:
RavenX-CyberAgent-v6.2-Q4_K_M.gguf
Commit:
654d9ddc90c2c43efaff9b7314c300365548d4f7
Remote blob:
X-Linked-ETag: 05ad179a79d717a2fcb5db7a404152fe46ec873fc46558cb229e235a568a6003
X-Xet-Hash: 01a5f332d4a189e1c6255242dc8ce1bb674c78729219ea874782a2bc4cb37aeb

Fresh llama.cpp still fails with:
llama_model_load: error loading model: missing tensor 'blk.40.attn_norm.weight'
I also tested with GPU offload disabled (--n-gpu-layers 0), so this is not a CUDA, VRAM, or offload issue.

The GGUF metadata says:
qwen35moe.block_count = 41
but the tensor list appears to only contain blk.0 through blk.39. There is no blk.40.* tensor group.

So it looks like the fixed file did not actually replace the broken blob, or the rebuild still produced an invalid GGUF.

Likely fix: either upload a GGUF that actually includes the missing blk.40.* tensors, or correct qwen35moe.block_count if the file is supposed to contain only 40 blocks.

johndekka

17 days ago

Just get the HF model https://huggingface.co/deadbydawn101/RavenX-CyberAgent-Qwen3.6-35B-A3B-Opus-4.7-OpenMythos-Pentester-BugHunter-RATH-mlx, use llama.cpp convert_hf_to_gguf.py and convert to bf16 with the --no-mtp flag. This skips the last layer. Then quantize if needed.

minhdang

17 days ago

•

edited 17 days ago

Can it really work like that? in GPU nvidia

deadbydawn101

Owner 17 days ago

Hey everyone - first of all, THANK YOU. Seriously. Your feedback, your bug reports, your patience, and your support mean so much to me. This community is why I keep building.

@minhdang @pccr10001 @Scottbyrd @Repent0947 @johndekka - every single one of your comments helped diagnose and fix this. That's open source at its best.

@Repent0947 - incredible diagnosis. You nailed it: block_count = 41 but only blk.0-39 present. That's exactly the issue.

@johndekka - the --no-mtp flag is the fix. Thank you for sharing that. This is what community is about.

@Scottbyrd - thank you for finding the v5.1 commit link and sharing it so others could still use the model. And your kind words about the project genuinely made my day.

Here's the plan going forward:

v5.1 will be RESTORED to this repo as the stable release. It's proven, the community loves it, and it works.
v6.2 will move to a separate experimental repo so we can iterate without breaking what works.
The GGUF conversion fix (--no-mtp + tensor surgery for Qwen3.6 MoE) will be properly documented.

We will continue research and development work. Soul Infusion, RATH protocol, the full pipeline - it's all moving forward. Your support and feedback fuel this. It means the world.

Stay tuned. More to come.

Gabriel Garcia / RavenX LLC

Scottbyrd

16 days ago

Awesome news @deadbydawn101 , I already redownloaded v5.1 and have tested it a bit more with some of my personal prompts and small projects, it's pretty solid as it is, thanks for restoring it for easier access for the community!

And that's a great plan moving forward, I'm excited to test all the new models you end up training and experimenting with. I've also tested the Gemma 4 12B finetune a little bit, and it's working well so far, nice job.

And thanks as well to all the commenters above making this community better everyday. Y'all are awesome. 🔥

pccr10001

13 days ago

I made a repo here to convert MLX model back to HF format, so you guys can quantize after conversation.

deadbydawn101

Owner 13 days ago

@pccr10001 -- this is incredible. You saw the problem and you BUILT THE SOLUTION. Your mlx-to-hf-qwen35moe converter is exactly what this community needed, and we want you to know we used it today in our latest model release.

You are officially credited in our new model card for RavenX-OpenFable-Qwopus-Coder-Holo3-Qwen3.6-35B-A3B-MTP -- both the GGUF and MLX versions. Your tool is now a core part of our production GGUF conversion pipeline. That's the kind of impact one person can have in open source.

@Scottbyrd -- glad the Gemma 4 12B is working well for you! That was a fun build. And your kind words about the project genuinely keep us going.

To the whole RavenX community -- @minhdang @Repent0947 @johndekka @Scottbyrd @pccr10001 -- your bug reports, your patience, your workarounds, and your support are why we keep building. Every single one of you made the new release possible.

Speaking of which -- we just shipped a brand new coder model built on this same 35B-A3B architecture with some exciting new research baked in. Check the RavenX-OpenFable series on our HF page when it goes public. More to come!

-- Gabriel Garcia / RavenX LLC

Scottbyrd

13 days ago

@deadbydawn101 Very awesome work, I'll have to test out the new model once it drops. Keep up the great work and research, you're one of the handful of people working on interesting new models and finetunes that I follow lately and it's looking more exciting with every release 🔥💯

And that's awesome that the tool from @pccr10001 is coming in handy already, this community is moving in amazing directions, love to see it!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment