# Qwen3.8-27B Heretic ARA FP8 validation

## Result

**PASS**

- Source: [`heretic-org/Qwen3.8-27B-heretic-ara`](https://huggingface.co/heretic-org/Qwen3.8-27B-heretic-ara)
- FP8 derivative: `magiccodingman/Qwen3.8-27B-heretic-ara-fp8`
- Format: native Qwen fine-grained FP8 (`float8_e4m3fn`) weights
- Weight scaling: 128 x 128 blocks with FP32 `weight_scale_inv` tensors
- Activation scheme in model metadata: dynamic FP8
- Quantized matrix tensors: 407
- Tensor payload: 30,869,678,304 bytes (30.87 GB decimal / 28.75 GiB)

## Structural and numerical checks

- Source tensor entries: 1,199
- Candidate tensor entries: 1,606 (407 added scale tensors)
- Non-quantized tensors preserved byte-for-byte: 792
- Aggregate weight RMSE after dequantization: 0.00034144
- Aggregate relative RMSE: 0.0264554
- FP8 tensor type, scale type, positive scale values, block geometry, index closure, and safetensors headers: all passed

## Forward-pass KLD check

Exact forward KL was measured as `D_KL(P_BF16 || P_FP8)` in FP32 across a diverse 12-prompt corpus. The two runs used the same tokenizer, BF16 compute, sequence settings, and device map.

- Scored next-token positions: 638
- Mean KL: 0.00315279
- Median KL: 0.00185845
- P95 KL: 0.0104304
- P99 KL: 0.0222929
- Maximum single-position KL: 0.0977299
- BF16 reference NLL: 2.78304195
- FP8 candidate NLL: 2.78553748
- NLL delta: +0.00249553
- Top-1 token agreement: 96.5517%
- Acceptance gate: mean KL < 0.01
- Result: PASS

The corpus covered code, mathematics, science, JSON/tool syntax, speculative-decoding concepts, Spanish, Arabic, proof writing, inference engineering, SQL, and FP8 technical reasoning.

## Important hardware limitation

RTX 3090 (SM 8.6) cannot execute this model's native W8A8 FP8 path. Transformers therefore dequantized the stored FP8 weights to BF16 for these forward comparisons. The KLD result directly validates the stored weight quantization and shows no material degradation from the conversion. It does **not** benchmark native dynamic-FP8 activation kernels or FP8 KV cache behavior.

On the target Radeon Pro R9700 system, perform a short native-runtime smoke test using the intended serving engine and explicitly enable its FP8 KV-cache option. KV-cache precision is normally a runtime setting rather than baked into the weight checkpoint.

## Provenance

The checkpoint layout and exclusion list were matched to the official `Qwen/Qwen3.8-27B-FP8` native format. Conversion used Python 3.12.3, PyTorch 2.13.0+cu130, Transformers 5.14.1, and Safetensors 0.8.0. The source directory was not modified.