Instructions to use Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF:UD-Q3_K_XL
# Run inference directly in the terminal:
llama cli -hf Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF:UD-Q3_K_XL

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF:UD-Q3_K_XL
# Run inference directly in the terminal:
llama cli -hf Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF:UD-Q3_K_XL

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF:UD-Q3_K_XL
# Run inference directly in the terminal:
./llama-cli -hf Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF:UD-Q3_K_XL

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF:UD-Q3_K_XL
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF:UD-Q3_K_XL

Use Docker

docker model run hf.co/Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF:UD-Q3_K_XL

LM Studio
Jan

vLLM

How to use Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF:UD-Q3_K_XL

Ollama
How to use Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF with Ollama:
```
ollama run hf.co/Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF:UD-Q3_K_XL
```

Unsloth Studio

How to use Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF to start chatting

How to use Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF:UD-Q3_K_XL

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF:UD-Q3_K_XL"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF with Docker Model Runner:
```
docker model run hf.co/Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF:UD-Q3_K_XL
```

Lemonade

How to use Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF:UD-Q3_K_XL

Run and chat with the model

lemonade run user.Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF-UD-Q3_K_XL

List all available models

lemonade list

Hermes Agent

How to use Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF:UD-Q3_K_XL

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF:UD-Q3_K_XL

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF:UD-Q3_K_XL

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF:UD-Q3_K_XL" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF / README.md

Luis23333

Upload 5 files

7f7354d verified 4 days ago

preview code

Raw

History Blame Contribute Delete

7.19 kB

metadata

license: apache-2.0
language:
  - en
  - zh
base_model:
  - redashes/Qwen3.8-27B-BF16-SSMFIX
base_model_relation: quantized
tags:
  - gguf
  - qwen3_5
  - qwen3.8
  - text-generation
  - quantized
  - imatrix
  - iq3_s
  - iq4_xs
  - q3_k
  - q5_k
  - ssm-fix
  - conv1d-repair
  - llama.cpp

Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF

中文说明

Experimental community conversion. This is not an official Qwen release.

This repository contains a single GGUF text-inference model:

Qwen3.8-27B-BF16-SSMFIX-UD-Q3_K_XL.gguf

It is derived from the repaired BF16 model redashes/Qwen3.8-27B-BF16-SSMFIX, which in turn is derived from the official Qwen/Qwen3.8-27B.

At a glance

Property	Value
Model family	Qwen3.8
Parameter count	27B dense model
Artifact format	GGUF V3
Quantization label	UD-Q3_K_XL
Quantization style	Custom mixed quantization with importance matrix
Effective size	12,807.91 MiB, approximately 12.52 GiB
Reported rate	3.93 BPW
Context inherited from upstream	262,144 tokens; practical capacity depends on runtime and hardware
Primary runtime	llama.cpp and compatible GGUF runtimes
Intended modality of this file	Text inference

UD-Q3_K_XL is a quantization-tier name, not a claim that every tensor is stored as pure Q3_K. The exact mixed recipe is documented below and in quantization_recipe.txt.

Model lineage and attribution

Qwen/Qwen3.8-27B
        |
        v
redashes/Qwen3.8-27B-BF16-SSMFIX
        |
        v
Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF

Upstream model: Qwen/Qwen3.8-27B
Direct BF16 source: redashes/Qwen3.8-27B-BF16-SSMFIX
Background discussion: Qwen3.8-27B discussion #76
GGUF publisher: Luis23333

The direct source model describes itself as an independent verification of a community investigation into abnormal SSM/conv1d scale behavior. This GGUF preserves that repaired source lineage and then applies the quantization recipe described below. It should be treated as an experimental community artifact, not as an official correction to Qwen3.8.

What SSMFIX changes

The BF16 source model applies per-layer alpha rescaling to eight ssm_conv1d.weight tensors. The documented correction factors are:

Layer	Tensor	Alpha
52	`blk.52.ssm_conv1d.weight`	0.59005
53	`blk.53.ssm_conv1d.weight`	0.55484
56	`blk.56.ssm_conv1d.weight`	0.54486
57	`blk.57.ssm_conv1d.weight`	0.53574
58	`blk.58.ssm_conv1d.weight`	0.60972
60	`blk.60.ssm_conv1d.weight`	0.48136
61	`blk.61.ssm_conv1d.weight`	0.65327
62	`blk.62.ssm_conv1d.weight`	0.61856

In the local quantization log, ssm_conv1d.weight tensors remain F32 in the GGUF conversion. The repair is therefore not deliberately quantized away by the recipe.

Quantization recipe

The model was quantized from a BF16 GGUF generated from the repaired source weights using llama.cpp build 9222 (9a532ae4b) and an importance matrix named imatrix_unsloth.gguf_file.

Tensor group	Type
Default quantization	`IQ3_S`
Token embeddings	`Q3_K`
Output tensor	`Q5_K`
`attn_v`	`Q5_K`
Selected attention, FFN, SSM and MTP tensors	`IQ4_XS`
`ssm_conv1d.weight`	`F32`

The exact tensor overrides are recorded in quantization_recipe.txt. The resulting file is a custom mixed quantization, not a pure single-type Q3_K file.

Important modality limitation

The upstream Qwen3.8 model is a vision-language model. This repository contains only the language-model GGUF produced by the conversion above. It does not include an mmproj/vision projector file, so this single artifact is documented and tested for text inference only. Use the upstream Transformers repository or a compatible model-plus-projector package when image or video input is required.

Usage with llama.cpp

Use a recent llama.cpp build with Qwen3.8/Qwen3.5 architecture support.

Interactive text chat

llama-cli \
  -m Qwen3.8-27B-BF16-SSMFIX-UD-Q3_K_XL.gguf \
  --jinja \
  --reasoning on \
  -cnv

Disable thinking with --reasoning off when supported by your llama.cpp build.

HTTP server

llama-server \
  -m Qwen3.8-27B-BF16-SSMFIX-UD-Q3_K_XL.gguf \
  --jinja \
  --reasoning auto \
  --ctx-size 16384 \
  --n-gpu-layers auto \
  --fit on \
  --flash-attn auto

The 262,144-token context length is inherited metadata, not a guarantee that a particular GPU can allocate that context. A 16 GiB GPU should start with a smaller context such as 16,384 and increase it only after measuring memory use.

The upstream Qwen recommendations are approximately:

Thinking: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0.
Non-thinking: temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5.

Evaluation and local smoke tests

Reference results from the BF16 source

The following results belong to the direct BF16 source model, not to this GGUF. They are included only as provenance context and should not be interpreted as a GGUF quality benchmark.

Metric	BF16 source v2
MT-Bench average	7.47
IFEval prompt strict	0.5194
IFEval instruction strict	0.6343
GSM8K strict	0.9644
CMMLU	0.6996
TruthfulQA MC1 / MC2	0.3758 / 0.5513
TruthfulQA generation ROUGE-1 / ROUGE-2 / ROUGE-L / BLEU	0.345 / 0.246 / 0.345 / 0.256

GGUF runtime smoke test

Local text-only smoke tests were run with llama.cpp build 9222 (9a532ae4b) on an NVIDIA RTX 5070 Ti, with context size 16,384, six prompts, three runs per prompt, and a maximum of 256 generated tokens.

Mode	Successful requests	Average elapsed time	Average wall generation rate
Baseline	18/18	6506.96 ms	39.367 tokens/s
MTP2	18/18	23098.24 ms	11.180 tokens/s

These are runtime smoke-test measurements, not broad capability scores. In this local setup MTP2 was slower than baseline, so this repository does not claim that speculative decoding is faster for this artifact.

Limitations and disclaimer

This is an experimental community conversion and is not an official Qwen release.
The SSMFIX hypothesis and its claimed benefits are not guaranteed for every workload.
The GGUF has not been assigned the BF16 source's quality scores.
This upload contains no vision projector and is not presented as a single-file multimodal package.
Long-context behavior depends on the runtime, KV-cache settings, available VRAM/RAM and prompt shape.
Validate the model on your own workload before relying on it in production.

License

The model follows the Apache-2.0 license of the upstream Qwen release. See LICENSE. Please preserve the upstream attribution and the direct-source attribution when redistributing this derived artifact.