---
license: apache-2.0
base_model: dealignai/Qwen3.8-27B-MXFP8-CRACK
base_model_relation: quantized
library_name: ninfer
pipeline_tag: image-text-to-text
tags:
- ninfer
- qwen3.8
- groupwise-int
- artifact
- rtx-5090
---

# Qwen3.8-27B "MXFP8-CRACK" — NInfer artifact

A `.ninfer` single-file artifact that lets the [NInfer](https://github.com/Neroued/ninfer) engine
(single-GPU C++/CUDA inference, RTX 5090) run the community fine-tune
[dealignai/Qwen3.8-27B-MXFP8-CRACK](https://huggingface.co/dealignai/Qwen3.8-27B-MXFP8-CRACK).

| | |
|---|---|
| Artifact identity | `qwen3.8-27b / groupwise-int` (registered engine identity) |
| File | `crack.ninfer` — 18,210,531,328 bytes |
| SHA-256 | `260ada97269a05e3296af72c3602ed49e72e8558c477eec9144d41765bfa9ffa` |
| Objects | 1124 (1118 tensors, 6 frontend resources) |
| Built with | ninfer `tools.convert.qwen3_8_27b`, recipe `qwen3_8_27b-v1` |
| Verified | smoke-run on the stock NInfer engine (master, CUDA 13.1, RTX PRO 4500 Blackwell): prefill 530 tok/s, decode 43 tok/s, coherent output |

## Download & run

```bash
hf download WaveCut/Qwen3.8-27B-MXFP8-CRACK-NInfer crack.ninfer --local-dir models

# CLI, one request:
./build/apps/ninfer models/crack.ninfer --prompt "Summarize the difference between prefill and decode." --max-context 16384

# OpenAI/Anthropic-compatible server:
./build/apps/ninfer-serve models/crack.ninfer --max-context 16384 --kv-capacity auto
```

**Hardware.** Two engines run this artifact:

- Upstream [NInfer](https://github.com/Neroued/ninfer) — primary target one RTX 5090 (32 GB); in practice any single Blackwell sm_120 card with 32 GB works (the verification smoke above ran on an RTX PRO 4500).
- [Don-Chad/ninfer-3090](https://github.com/Don-Chad/ninfer-3090) (v0.6.0+) — RTX 3090 (24 GB) edition where **Qwen3.8-27B is a first-class, tested target**: the native SM86 runtime loads the official groupwise `.ninfer` artifact directly (same `qwen3.8-27b / groupwise-int` identity as this file), ships `run-qwen38-*` launch scripts, image understanding, and sustains up to a 171K-token INT8 context on one 3090 (226K with the optional RotorQuant `rk8v4` KV cache).

## How it was built

The source checkpoint stores weights in two packed 8-bit schemes, so it was first dequantized to BF16 and re-named into the official Qwen3.8-27B tensor layout, then converted with the unmodified stock converter:

1. **Text + Vision (582 tensors) — MXFP8**: 4×FP8-E4M3 packed per `U32` word, one `U8` UE8M0 scale (`2^(b-127)`) per 32-element group → BF16.
2. **MTP draft layer (8 tensors) — MLX affine int8**: `uint8` codes with per-128-group `F16` scale and bias, `w = q*scale + bias` → BF16. The formula was verified against the official `Qwen/Qwen3.8-27B` MTP weights (cosine similarity ≈ 1.0, σ matches to 5 decimals).
3. **Renames**: `language_model.model.* → model.language_model.*`, `language_model.lm_head.* → lm_head.*`, `vision_tower.* → model.visual.*`; the vision patch embedding was permuted from the MLX conv layout `[O,T,H,W,I]` to torch `[O,I,T,H,W]`. `config.json` of the fork passes the converter's registered validation unchanged.
4. **Frontend resources** (tokenizer, chat template, preprocessor configs, `generation_config.json`) are the official `Qwen/Qwen3.8-27B` files — the NInfer converter pins them by SHA-256. The fork's tokenizer/chat-template files are byte-identical; its `generation_config.json` differs only by a no-op key, so official sampling defaults apply unless overridden per request.
5. Standard converter: `python3 -m tools.convert.qwen3_8_27b.convert --model <bf16-dir> --out crack.ninfer` (82 s on one RTX PRO 4500).

`crack.ninfer.conversion.json` (included) is the converter's own report for this exact file.

## Caveats

- **Double quantization.** The source is already 8-bit (MXFP8 / affine int8); this artifact re-quantizes into NInfer's groupwise-int profile (Q4/Q5/Q6/W8 groupwise + BF16). Expect marginally lower fidelity than an artifact built from a native BF16 checkpoint of the same tune.
- The engine binds this file to the registered `qwen3.8-27b / groupwise-int` profile: architecture, tokenizer and chat template are identical to the official model; only the weights carry the fine-tune.
- Upstream NInfer is specialized for one RTX 5090-class device and a single CUDA device; for RTX 3090 use the [ninfer-3090](https://github.com/Don-Chad/ninfer-3090) fork (see Hardware above).
- Per the upstream card: set your sampler explicitly (`temperature=1.0, top_p=0.95, top_k=20`) if you rely on the tune's intended behavior.

## Lineage

```
Qwen/Qwen3.8-27B
  └─ dealignai/Qwen3.8-27B-MXFP8-CRACK   (fine-tune + MXFP8/MLX-affine quant, Apache-2.0)
       └─ BF16 dequant + official-layout rename (adapter described above)
            └─ crack.ninfer  (this repo, NInfer groupwise-int profile)
```

Licensed Apache-2.0, same as both upstreams.