GGUF
English
Chinese
multilingual
qwen3
qwen3.6
reasoning
coding
coding-agent
academic-writing
uncensored
rys
lora
iq4_nl
bf16
conversational
Instructions to use jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16 # Run inference directly in the terminal: llama cli -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16 # Run inference directly in the terminal: llama cli -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16 # Run inference directly in the terminal: ./llama-cli -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16 # Run inference directly in the terminal: ./build/bin/llama-cli -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
Use Docker
docker model run hf.co/jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
- LM Studio
- Jan
- Ollama
How to use jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF with Ollama:
ollama run hf.co/jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
- Unsloth Studio
How to use jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF to start chatting
- Pi
How to use jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent new
How to use jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
Run Hermes
hermes
- OpenClaw new
How to use jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
- Docker Model Runner
How to use jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF with Docker Model Runner:
docker model run hf.co/jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
- Lemonade
How to use jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
Run and chat with the model
lemonade run user.Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF-BF16
List all available models
lemonade list
- Atomic Chat
Upload PATCHCODE_TESTING_PROCESS.html with huggingface_hub
Browse files
PATCHCODE_TESTING_PROCESS.html
CHANGED
|
@@ -179,6 +179,7 @@ li{margin:.25em 0}
|
|
| 179 |
</tbody>
|
| 180 |
</table>
|
| 181 |
<p>At λ=1.0 the adapter was net-neutral-to-harmful (one checkpoint fell <em>below</em> the un-adapted base). The mechanism: an over-loud LoRA delta pushes activations into regimes that hurt calibrated behaviour (preamble returns, over-claiming). λ=0.5 (effective alpha/r = 1.0) keeps the style direction but respects base calibration. So the merge was done at <strong>λ=0.5 onto SignalLatch (ckpt386-s010)</strong>, then exported to BF16 GGUF. (A future v2 could bake the good strength in by training at alpha=r=32, removing the inference-time knob.)</p>
|
|
|
|
| 182 |
<h2>Why the final testing moved to merged IQ4_NL</h2>
|
| 183 |
<p>The key question was not "best adapter in BF16" — it was "what we would actually deploy". The deploy target was a merged GGUF, <code>IQ4_NL</code>, imatrix-quantized, on the custom ik-llama runtime (Jinja + DeepSeek reasoning format + flash attention + graph split, temp <code>0.7</code>).</p>
|
| 184 |
<p>Live LoRA loading is not the production path for this release (the tested serving profile uses flash attention, which conflicts with live LoRA on this runtime). So the long-term path became: <strong>merge the adapter first, then export + quantize a full GGUF.</strong> That is why the upload is a merged GGUF, not an adapter.</p>
|
|
@@ -271,10 +272,14 @@ li{margin:.25em 0}
|
|
| 271 |
<p><strong>Rule:</strong> 3-seed differences <0.13 on this suite are meaningless. Use <strong>5+ seeds, same-condition head-to-head</strong> before any ship call. Only non-noise axes (size, recipe methodology/safety, long-context at ceiling) reliably tiebreak. HumanEval was rejected — it saturates on Qwen and is the wrong mode for an agent.</p>
|
| 272 |
<p>This is exactly how a 3-seed pass almost shipped the <em>weaker</em> model.</p>
|
| 273 |
<h2>The ship decision</h2>
|
| 274 |
-
<p>With build, discipline, long-context, and autonomy all <strong>tied within noise</strong>, the decision fell to non-noise axes, where plain <code>IQ4_NL</code> wins all three:
|
| 275 |
-
|
| 276 |
-
|
| 277 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 278 |
<p>Ship: <strong>plain <code>IQ4_NL</code> (reasoning-imatrix)</strong>. The mixed-recipe <code>c76</code> is retained on disk as the build-heavy fallback if a future, harder build-gate ever discriminates beyond the noise floor (use 5+ seeds).</p>
|
| 279 |
<h2>What the testing says and does not say</h2>
|
| 280 |
<p><strong>Does say:</strong>
|
|
|
|
| 179 |
</tbody>
|
| 180 |
</table>
|
| 181 |
<p>At λ=1.0 the adapter was net-neutral-to-harmful (one checkpoint fell <em>below</em> the un-adapted base). The mechanism: an over-loud LoRA delta pushes activations into regimes that hurt calibrated behaviour (preamble returns, over-claiming). λ=0.5 (effective alpha/r = 1.0) keeps the style direction but respects base calibration. So the merge was done at <strong>λ=0.5 onto SignalLatch (ckpt386-s010)</strong>, then exported to BF16 GGUF. (A future v2 could bake the good strength in by training at alpha=r=32, removing the inference-time knob.)</p>
|
| 182 |
+
<p><img alt="Merge strength sweep — λ=0.5 wins on all three checkpoints; trained default λ=1.0 is over-applied." src="assets/lambda_sweep.png" /></p>
|
| 183 |
<h2>Why the final testing moved to merged IQ4_NL</h2>
|
| 184 |
<p>The key question was not "best adapter in BF16" — it was "what we would actually deploy". The deploy target was a merged GGUF, <code>IQ4_NL</code>, imatrix-quantized, on the custom ik-llama runtime (Jinja + DeepSeek reasoning format + flash attention + graph split, temp <code>0.7</code>).</p>
|
| 185 |
<p>Live LoRA loading is not the production path for this release (the tested serving profile uses flash attention, which conflicts with live LoRA on this runtime). So the long-term path became: <strong>merge the adapter first, then export + quantize a full GGUF.</strong> That is why the upload is a merged GGUF, not an adapter.</p>
|
|
|
|
| 272 |
<p><strong>Rule:</strong> 3-seed differences <0.13 on this suite are meaningless. Use <strong>5+ seeds, same-condition head-to-head</strong> before any ship call. Only non-noise axes (size, recipe methodology/safety, long-context at ceiling) reliably tiebreak. HumanEval was rejected — it saturates on Qwen and is the wrong mode for an agent.</p>
|
| 273 |
<p>This is exactly how a 3-seed pass almost shipped the <em>weaker</em> model.</p>
|
| 274 |
<h2>The ship decision</h2>
|
| 275 |
+
<p>With build, discipline, long-context, and autonomy all <strong>tied within noise</strong>, the decision fell to non-noise axes, where plain <code>IQ4_NL</code> wins all three:</p>
|
| 276 |
+
<p><img alt="No candidate clears BOTH build and discipline (≥0.90) — promotion destroys discipline; precision does not fix build." src="assets/bothquest.png" /></p>
|
| 277 |
+
<p><img alt="Ship scoreboard (5-seed): IQ4_NL ties the field within noise on build/long-context/discipline, and wins on size." src="assets/ship_scoreboard.png" /></p>
|
| 278 |
+
<ul>
|
| 279 |
+
<li><strong>smaller</strong> (16.6 G vs 20–29 G)</li>
|
| 280 |
+
<li><strong>marginal long-context</strong> edge (0.975 vs 0.935–0.969)</li>
|
| 281 |
+
<li><strong>plain-quant recipe</strong> — the fleet's proven pattern; promotion/mixed recipes carry evidence-harmful risk (discipline collapse) for zero measured benefit</li>
|
| 282 |
+
</ul>
|
| 283 |
<p>Ship: <strong>plain <code>IQ4_NL</code> (reasoning-imatrix)</strong>. The mixed-recipe <code>c76</code> is retained on disk as the build-heavy fallback if a future, harder build-gate ever discriminates beyond the noise floor (use 5+ seeds).</p>
|
| 284 |
<h2>What the testing says and does not say</h2>
|
| 285 |
<p><strong>Does say:</strong>
|