Image-Text-to-Text
GGUF
llama.cpp
qwen3_5_moe
Mixture of Experts
Merge
vision-language
security
conversational
Instructions to use lancejames221b/razorstrike-v1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use lancejames221b/razorstrike-v1-GGUF with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="lancejames221b/razorstrike-v1-GGUF", filename="mmproj.gguf", )
llm.create_chat_completion( messages = [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] ) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use lancejames221b/razorstrike-v1-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf lancejames221b/razorstrike-v1-GGUF:IQ4_XS # Run inference directly in the terminal: llama cli -hf lancejames221b/razorstrike-v1-GGUF:IQ4_XS
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf lancejames221b/razorstrike-v1-GGUF:IQ4_XS # Run inference directly in the terminal: llama cli -hf lancejames221b/razorstrike-v1-GGUF:IQ4_XS
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf lancejames221b/razorstrike-v1-GGUF:IQ4_XS # Run inference directly in the terminal: ./llama-cli -hf lancejames221b/razorstrike-v1-GGUF:IQ4_XS
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf lancejames221b/razorstrike-v1-GGUF:IQ4_XS # Run inference directly in the terminal: ./build/bin/llama-cli -hf lancejames221b/razorstrike-v1-GGUF:IQ4_XS
Use Docker
docker model run hf.co/lancejames221b/razorstrike-v1-GGUF:IQ4_XS
- LM Studio
- Jan
- vLLM
How to use lancejames221b/razorstrike-v1-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "lancejames221b/razorstrike-v1-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "lancejames221b/razorstrike-v1-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/lancejames221b/razorstrike-v1-GGUF:IQ4_XS
- Ollama
How to use lancejames221b/razorstrike-v1-GGUF with Ollama:
ollama run hf.co/lancejames221b/razorstrike-v1-GGUF:IQ4_XS
- Unsloth Studio
How to use lancejames221b/razorstrike-v1-GGUF with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for lancejames221b/razorstrike-v1-GGUF to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for lancejames221b/razorstrike-v1-GGUF to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for lancejames221b/razorstrike-v1-GGUF to start chatting
- Pi
How to use lancejames221b/razorstrike-v1-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf lancejames221b/razorstrike-v1-GGUF:IQ4_XS
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "lancejames221b/razorstrike-v1-GGUF:IQ4_XS" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent new
How to use lancejames221b/razorstrike-v1-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf lancejames221b/razorstrike-v1-GGUF:IQ4_XS
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default lancejames221b/razorstrike-v1-GGUF:IQ4_XS
Run Hermes
hermes
- Atomic Chat new
- OpenClaw new
How to use lancejames221b/razorstrike-v1-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf lancejames221b/razorstrike-v1-GGUF:IQ4_XS
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "lancejames221b/razorstrike-v1-GGUF:IQ4_XS" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
- Docker Model Runner
How to use lancejames221b/razorstrike-v1-GGUF with Docker Model Runner:
docker model run hf.co/lancejames221b/razorstrike-v1-GGUF:IQ4_XS
- Lemonade
How to use lancejames221b/razorstrike-v1-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull lancejames221b/razorstrike-v1-GGUF:IQ4_XS
Run and chat with the model
lemonade run user.razorstrike-v1-GGUF-IQ4_XS
List all available models
lemonade list
Add files using upload-large-folder tool
Browse files- README.md +92 -38
- razorstrike-v1-Q4_K_M.gguf +2 -2
README.md
CHANGED
|
@@ -1,53 +1,107 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
-
base_model:
|
| 4 |
-
- Qwen/Qwen3.5-35B-A3B-Base
|
| 5 |
-
- deepreinforce-ai/Ornith-1.0-35B
|
| 6 |
tags:
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
language:
|
| 16 |
-
|
| 17 |
---
|
| 18 |
|
| 19 |
-
# RazorStrike-v1 (GGUF
|
| 20 |
|
| 21 |
-
|
| 22 |
-
(the MLX 4-bit multimodal original). Vision tower is not included in GGUF (text-only inference). An uncensored
|
| 23 |
-
(abliterated) variant is also available: [razorstrike-v1-uncensored-GGUF](https://huggingface.co/lancejames221b/razorstrike-v1-uncensored-GGUF).
|
| 24 |
|
| 25 |
-
|
| 26 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
|
| 28 |
## Composition
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
```
|
|
|
|
|
|
|
|
|
|
| 46 |
ollama pull hf.co/lancejames221b/razorstrike-v1-GGUF
|
| 47 |
-
ollama run hf.co/lancejames221b/razorstrike-v1-GGUF
|
| 48 |
```
|
| 49 |
-
|
|
|
|
|
|
|
|
|
|
| 50 |
|
| 51 |
## License
|
| 52 |
-
|
| 53 |
-
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
base_model: Qwen/Qwen3.6-35B-A3B
|
|
|
|
|
|
|
| 4 |
tags:
|
| 5 |
+
- gguf
|
| 6 |
+
- qwen3_5_moe
|
| 7 |
+
- moe
|
| 8 |
+
- merge
|
| 9 |
+
- dare-ties
|
| 10 |
+
- reasoning
|
| 11 |
+
- agentic
|
| 12 |
+
- coding
|
| 13 |
+
- security
|
| 14 |
+
- uncensored
|
| 15 |
+
- abliterated
|
| 16 |
+
- llama.cpp
|
| 17 |
+
- ollama
|
| 18 |
language:
|
| 19 |
+
- en
|
| 20 |
---
|
| 21 |
|
| 22 |
+
# RazorStrike-v1 (GGUF)
|
| 23 |
|
| 24 |
+
RazorStrike-v1 is a multi-donor **DARE-TIES** task-arithmetic merge built on the **Qwen3.6-35B-A3B** MoE base (hybrid linear-attention/SSM architecture, 256 experts, 262144 token context). It fuses a reasoning + uncensoring donor with an agentic/coding donor on top of the Qwen3.6 anchor.
|
|
|
|
|
|
|
| 25 |
|
| 26 |
+
This repo hosts the **GGUF Q4_K_M quantized** build (~19.7GB), for `llama.cpp` / Ollama. Text-only (vision tower and MTP head are not included in this GGUF).
|
| 27 |
+
|
| 28 |
+
## Approach
|
| 29 |
+
|
| 30 |
+
RazorStrike-v1's build process is empirical, not assumption-driven: every merge decision is validated against real generated output rather than trusted on theory alone.
|
| 31 |
+
|
| 32 |
+
1. **Task-arithmetic delta merging (DARE-TIES)** β donors contribute as deltas against a shared anchor (`donor - anchor`), not as raw weight averages.
|
| 33 |
+
2. **Coherence-gate before and after every stage** β a dedicated battery of coherence and refusal-behavior prompts is run on every candidate build, at both greedy (temp 0.0) and sampled (temp 0.6) decoding.
|
| 34 |
+
3. **Root-cause, not workaround, on defects** β a coding-context language-contamination defect found during development was traced with a systematic bisect (raw single-donor tests, then pairwise donor-combination tests), correctly identifying a specific donor's task vector as the cause (see below) instead of shipping a band-aid fix.
|
| 35 |
+
4. **Cross-runtime re-verification** β this GGUF build was smoke-tested through `llama-server`'s actual C++ inference kernels (not just conversion success) on both a plain completion and the specific chat prompt that originally triggered the contamination defect, confirming the fix holds across runtimes (MLX and llama.cpp), not just the runtime it was originally verified on.
|
| 36 |
|
| 37 |
## Composition
|
| 38 |
+
|
| 39 |
+
RazorStrike-v1 is a **DARE-TIES** (drop-and-rescale + sign-election disjoint merge) of two donor models applied on top of a Qwen3.6-35B-A3B anchor:
|
| 40 |
+
|
| 41 |
+
| Role | Model | Weight | DARE density | Contribution |
|
| 42 |
+
|---|---|---|---|---|
|
| 43 |
+
| Anchor | `Qwen/Qwen3.6-35B-A3B` | β | β | Base for all non-merged tensors: embeddings, `lm_head`, norms, SSM dynamics, MTP head |
|
| 44 |
+
| Donor 1 (dominant) | huihui-ai's Claude-4.7-Opus-abliterated build (Qwen3.6-35B-A3B lineage) | 0.45 | 0.55 | Reasoning (via an embedded Opus-Reasoning-Distilled lineage) **and** uncensoring (weights-only abliteration / refusal removal). Router (`mlp.gate`, 41 tensors) is taken **verbatim** from this donor. |
|
| 45 |
+
| Donor 2 | `AlexWortega/SIQ-1-35B` | 0.40 | 0.50 | Agentic tool-use, coding, and autoresearch capability |
|
| 46 |
+
|
| 47 |
+
DARE-TIES is applied **only** to mergeable tensors (attention Q/K/V/O, linear-attention in/out projections, routed-expert gate_up/down, shared-expert FFN + gate). The router, norms, embeddings, `lm_head`, and SSM dynamics are all verbatim-copied from the anchor.
|
| 48 |
+
|
| 49 |
+
Because the dominant donor is already an abliterated (refusal-removed) build, RazorStrike-v1 is **uncensored by construction**.
|
| 50 |
+
|
| 51 |
+
## What was tried and dropped
|
| 52 |
+
|
| 53 |
+
A third donor, `UnipatAI/UniMath-35B-A3B`, was originally evaluated to add dedicated math/proof-synthesis specialization. It was **excluded** after root-cause investigation via pairwise donor-combination testing found that UniMath deterministically caused **Chinese-character (CJK) contamination in coding-context generation** whenever merged with *any* other donor via DARE-TIES. This defect was confirmed absent in every donor tested individually β it was an emergent interaction of the merge itself, not a property of UniMath alone.
|
| 54 |
+
|
| 55 |
+
UniMath was dropped entirely. **This build has no dedicated math-specialist donor** β a known scope limitation, not an oversight.
|
| 56 |
+
|
| 57 |
+
## Verification
|
| 58 |
+
|
| 59 |
+
- **16/16 coherence tests clean** (reasoning, math, terminal, coding Γ3, agentic, prose β temp 0.0 and 0.6), **3/3 refusal-battery tests complied** (keylogger/malware, phishing, controlled-substance synthesis) β verified on the bf16 master and the MLX 4-bit build.
|
| 60 |
+
- **This GGUF re-verified separately**: plain completion smoke test (no chat template) came back coherent, and the specific chat-template prompt that originally triggered the CJK-contamination defect was re-tested through `llama-server` at temp=0.0 β clean, no contamination, matches the MLX-side result.
|
| 61 |
+
|
| 62 |
+
### A note on uncensoring
|
| 63 |
+
|
| 64 |
+
This is an abliterated/uncensored build. It does not refuse security-research, offensive-security, or other sensitive-but-legitimate technical requests the way an aligned instruction-tuned model would. It is provided for research, red-teaming, and technical use. Responsibility for lawful and ethical use of this model rests entirely with the user.
|
| 65 |
+
|
| 66 |
+
## Recommended inference settings
|
| 67 |
+
|
| 68 |
+
These match Qwen's own documented "Thinking mode, precise coding tasks" preset for the Qwen3.6-35B-A3B base:
|
| 69 |
+
|
| 70 |
+
```
|
| 71 |
+
temperature: 0.6
|
| 72 |
+
top_p: 0.95
|
| 73 |
+
top_k: 20
|
| 74 |
+
min_p: 0.0
|
| 75 |
+
presence_penalty: 0.0
|
| 76 |
+
repetition_penalty: 1.0
|
| 77 |
+
```
|
| 78 |
+
|
| 79 |
+
Because the dominant donor is reasoning-derived, RazorStrike-v1 can produce long internal reasoning traces if left unconstrained. Give it a generous context/`n_predict` budget, or use a "think fast, not long" system-prompt directive to keep responses tight.
|
| 80 |
+
|
| 81 |
+
## Architecture
|
| 82 |
+
|
| 83 |
+
- **Type**: `qwen35moe` (llama.cpp arch name) / `Qwen3_5MoeForConditionalGeneration` (HF) β hybrid linear-attention + SSM Mixture-of-Experts, 256 experts
|
| 84 |
+
- **Context**: 262,144 tokens (inherited from the Qwen3.6-35B-A3B anchor)
|
| 85 |
+
- **Quantization**: Q4_K_M (~4.88 bits/weight effective), 19.7GB
|
| 86 |
+
|
| 87 |
+
## Usage
|
| 88 |
+
|
| 89 |
+
**llama.cpp:**
|
| 90 |
+
```bash
|
| 91 |
+
llama-server -hf lancejames221b/razorstrike-v1-GGUF --port 8080
|
| 92 |
+
# or, if downloaded locally:
|
| 93 |
+
llama-server -m razorstrike-v1-Q4_K_M.gguf --port 8080 -c 32768
|
| 94 |
```
|
| 95 |
+
|
| 96 |
+
**Ollama:**
|
| 97 |
+
```bash
|
| 98 |
ollama pull hf.co/lancejames221b/razorstrike-v1-GGUF
|
|
|
|
| 99 |
```
|
| 100 |
+
|
| 101 |
+
## Roadmap
|
| 102 |
+
|
| 103 |
+
Future versions are intended to layer domain-specific **LoRA SFT specialists** on top of this merged base rather than folding more donors into the task-arithmetic merge itself β planned specialist domains include **reverse engineering (RE)** and **cryptography**.
|
| 104 |
|
| 105 |
## License
|
| 106 |
+
|
| 107 |
+
Released under **Apache 2.0**, matching the Qwen3.6-35B-A3B base model. This is a derivative merge, so users should also review the license terms of the constituent donor models β huihui-ai's Claude-4.7-Opus-abliterated build and `AlexWortega/SIQ-1-35B` β before use.
|
razorstrike-v1-Q4_K_M.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d3a86f2d016243f501f5156b1136f7ce54031613819e0031d38bc6a987648876
|
| 3 |
+
size 21166757888
|