Instructions to use Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF
# Run inference directly in the terminal:
llama cli -hf Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF
# Run inference directly in the terminal:
llama cli -hf Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF
# Run inference directly in the terminal:
./llama-cli -hf Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF

Use Docker

docker model run hf.co/Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF

LM Studio
Jan

vLLM

How to use Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF

Ollama
How to use Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF with Ollama:
```
ollama run hf.co/Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF
```

Unsloth Studio

How to use Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF to start chatting

How to use Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF with Docker Model Runner:
```
docker model run hf.co/Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF
```

Lemonade

How to use Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF

Run and chat with the model

lemonade run user.Qwen3.8-27B-MixQ4A6E8-GGUF-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Alogotron/Qwen3.8-27B-MixQ4A6E8-GGUF" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3.8-27B-MixQ4A6E8-v0 (experimental rc1)

Experimental candidate — not optimized, recommended, or generally validated. This immutable candidate exists to make a measured mixed-quant experiment reproducible. No quality-equivalence, speed-superiority, memory, or maximum-context claim is made beyond the sealed evidence cited below.

This community release is not affiliated with or endorsed by Qwen or Alibaba Cloud.

Artifact

File	Bytes	SHA256
`Qwen3.8-27B-MixQ4A6E8-v0.gguf`	19,021,793,888	`5cd87bedac2cec27d2b16203ef35b4fa861cf3a6e409a28a7133b37237e49509`

Verify the download with sha256sum -c SHA256SUMS. MANIFEST.json is the canonical machine-readable provenance record.

Recipe and immutable provenance

Upstream model: Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 (Apache-2.0).
BF16 input: ggml-org/Qwen3.8-27B-GGUF@0669b98607d47046c7c2b3f801011d54a08cfccf, SHA256 5a3eedc837bcbd1365cdbf5b71e698df3122e76586ba872f07ce3ed4a9bfa97e.
Quantizer: llama.cpp b10442, commit 9b0a2ce859d3884252705e9ac7c93c98616bb238.
External imatrix: Bartowski Qwen3.8-27B-imatrix.gguf@f0eec4a4bb4975114a030d048952d83c0a53c034, SHA256 aaa933d4b9ce23e1f65c548ad34f16956d8af44a51b5c15bf4f393ba59508cd8.
Quantization: Q4_K_M fallback; Q6_K for matched .*attn.* and .*ssm.* tensors; Q8_0 token embedding and output. Tensor rules SHA256 4000c7ce7f467c4972df085e0ce115bb1f79cc252be8dbc2dd28b14eea2599e4.

The external importance matrix was not created by this project. See ATTRIBUTION.md. Build evidence (dry-run validation, quantize log, commands, environment): results/build-mixq4a6e8-v0-20260818T231742Z.

Sealed Q4_K_M comparison (quality lane)

Full WikiText-2 test set (602 chunks, ctx/batch/ubatch 512, pinned corpus, BF16-relative KL against freshly generated BF16 base logits), pinned b10442 Vulkan lane, identical protocol across artifacts. Lower is better.

Artifact	BF16-relative KL	Full PPL	ΔPPL vs BF16 (6.648031)
MixQ4A6E8-v0	0.007835 ± 0.00005	6.6830 ± 0.0415	+0.0350
Q4_K_M	0.009584 ± 0.00008	6.6921 ± 0.0416	+0.0441

Within this sealed two-artifact comparison, the candidate's BF16-relative KL is 18.2492% lower than Q4_K_M (0.007835 versus 0.009584); the gap is ~35× the combined standard error. Its full PPL is 0.0091 lower (6.6830 versus 6.6921); given the ±0.04 uncertainties this supports the KL claim but only a weak PPL direction. Evidence: results/ppl-full-bf16ngl48-mixq4a6e8-v0-vulkan-b10442-20260819T063633Z.

Sealed Q4_K_M comparison (speed lanes)

Identical llama-bench protocol on both lanes: pp512/pp2048/tg128, 5 repetitions, pinned b10442 CUDA build, medians in tok/s. Higher is better.

Single RTX 3090 24 GiB (one GPU, tensor split 1):

Artifact	`pp512`	`pp2048`	`tg128`
Q4_K_M	1,399.42	1,396.11	41.53
MixQ4A6E8-v0	1,366.28	1,363.71	39.50

Candidate deltas: −2.3681% (pp512), −2.3206% (pp2048), −4.8880% (tg128). Evidence: results/llama-bench-mixq4a6e8-v0-single-gpu-cuda-b10442-20260818T233255Z.

Dual RTX 3090 (layer split 1/1):

Artifact	`pp512`	`pp2048`	`tg128`
Q4_K_M	1,406.90	2,103.70	42.47
MixQ4A6E8-v0	1,368.37	2,011.01	40.49

Candidate deltas: −2.7386% (pp512), −4.4062% (pp2048), −4.6733% (tg128). Evidence: results/llama-bench-frontier-q4toq8-cuda-b10442-20260819T094256Z.

Net measured trade within these sealed comparisons: the candidate is 7.0291% larger than Q4_K_M, 18.2492% lower in BF16-relative KL, and 2.3–4.9% slower across the three workloads. Whether that trade serves a given use case is left to the reader; no recommendation is made.

Measured Q4–Q8 frontier (CUDA lane, dual RTX 3090)

Six artifacts, identical sealed protocol; KL/PPL from the pinned Vulkan perplexity lane (Q8_0 not KL/PPL-measured here). Medians.

Artifact	Bytes	BF16-relative KL ↓	Full PPL ↓	`pp512` ↑	`pp2048` ↑	`tg128` ↑
Q4_K_M	17,772,537,440	0.009584	6.6921	1,406.90	2,103.70	42.47
MixQ4A6E8-v0 (this artifact)	19,021,793,888	0.007835	6.6830	1,368.37	2,011.01	40.49
Q5_K_M	19,231,099,520	0.003949	6.6715	1,353.92	1,980.47	39.49
MixQ5A6E8-v0 (sibling)	20,804,373,120	0.003083	6.6674	1,336.72	1,963.61	37.43
Q6_K	22,082,529,920	0.002099	6.6553	1,251.84	1,816.35	34.56
Q8_0	29,116,388,960	not measured	not measured	1,515.64	2,186.57	28.58

Within this sealed set no artifact is simultaneously higher-fidelity (KL) and faster (tg128) than another: every point is nondominated. MixQ4A6E8-v0 occupies the measured intermediate position between Q4_K_M and Q5_K_M — 1.0883% smaller and +1.0672%/+1.5420%/+2.5225% faster (pp512/pp2048/ tg128) than Q5_K_M at worse fidelity (KL 0.007835 versus 0.003949).

This does not establish a broad optimization, best, recommendation, VRAM benefit, Q8/BF16 equivalence, or industry-benchmark quality claim.

Validation scope: what is NOT covered

MTP (multi-token prediction draft heads): not validated for this artifact; do not assume the sibling's MTP parity results transfer.
Multimodal projector: not validated for this artifact.
Text integration smoke suite (chat template, thinking controls, tool calling): passed for this artifact (exact tokenizer/architecture metadata parity plus five deterministic integration tests; sealed run text-validation-mixq4a6e8-v0-vulkan-b10442-20260819T131943Z). It is a reproducibility smoke, not a quality benchmark.
Downstream task-quality suites: not run. Perplexity/KL are distributional proxies, not task benchmarks.
Downstream task-quality suites: not run. Perplexity/KL are distributional proxies, not task benchmarks.

Compatibility and limitations

Evidence was collected on a dedicated validation host with 2× NVIDIA RTX 3090 24 GiB GPUs: pinned b10442 CUDA sm_86 build (single- and dual-GPU speed lanes) and pinned b10442 Vulkan tools (quality lane). Other runtimes and hardware are unverified.
At 19,021,793,888 bytes the artifact fits a single 24 GiB card with context headroom; exact maximum context per runtime is unmeasured and not claimed.
The tensor-selection rules remain experimental even though this exact artifact was built and experimentally checked.
Review MANIFEST.json, NOTICE, and ATTRIBUTION.md before redistribution.