RedHatAI/Qwen3.8-2.4T-A95B-NVFP4-REAP-25

This is a quantized version of Qwen/Qwen3.8-2.4T-A95B with MoE layers quantized to NVFP4 and 25% uniform expert sparsity. The model was calibrated using 1024 samples from perfectblend.

Usage

This model is intended for deployment with vLLM. You can serve the model using

vllm serve RedHatAI/Qwen3.8-2.4T-A95B-NVFP4-REAP-25 \
    --tensor-parallel-size 8 \
    --enable-expert-parallel 8 \
    --reasoning-parser qwen3

Evaluation

inspect eval hf/Idavidrein/gpqa/diamond \
  --model vllm/RedHatAI/Qwen3.8-2.4T-A95B-NVFP4-REAP-25 \
  --reasoning-effort xhigh \
  --model-base-url http://localhost:8000/v1 \
  -M client_timeout=2400 \
  --token-limit 100000 \
  --retry-on-error=2

from inspect_ai import eval
from inspect_ai.model import set_model_info, ModelInfo

set_model_info(
    "vllm/qwen_model",  # model names with `-A` are badly sanitized through docker jobs
    ModelInfo(
        context_length=262144,
        reasoning=True,
    ),
)

eval(
    "inspect_harbor/datacurve_deep_swe",
    model="vllm/qwen_model",
    reasoning_effort="xhigh",
    model_base_url="http://localhost:8000/v1",
    max_connections=8,
    retry_on_error=2,
)

Benchmark	Qwen/Qwen3.8-2.4T-A95B	RedHatAI/Qwen3.8-2.4T-A95B-NVFP4-REAP-50	RedHatAI/Qwen3.8-2.4T-A95B-NVFP4-REAP-25	RedHatAI/Qwen3.8-2.4T-A95B-NVFP4
GPQA Diamond	92.6	90.7	91.5	92.9
DeepSWE 1.1	56.6	50.4	56.6	56.6