---
license: other
base_model:
  - mistralai/Leanstral-1.5-119B-A6B
pipeline_tag: image-text-to-text
library_name: transformers
tags:
  - leanstral
  - deepseek_v3
  - moe
  - nvfp4
  - fp4
  - quantized
  - vllm
  - blackwell
---

# Leanstral-1.5-119B-A6B — NVFP4

**NVFP4 (4-bit)** quantization of Leanstral-1.5-119B-A6B for Blackwell-class NVIDIA
GPUs (native FP4 tensor cores). Derived from the BF16 HF conversion
[`sahilchachra/Leanstral-1.5-119B-A6B-BF16`](https://huggingface.co/sahilchachra/Leanstral-1.5-119B-A6B-BF16),
itself an unofficial community conversion of
[`mistralai/Leanstral-1.5-119B-A6B`](https://huggingface.co/mistralai/Leanstral-1.5-119B-A6B).

## Architecture

Composed [`Mistral3`](https://huggingface.co/docs/transformers/model_doc/mistral3)
model = **DeepSeek-V3** text backbone (MLA attention + shared-expert MoE: 128
routed experts, 4 active/token, 1 shared) **+ Pixtral** vision encoder. (llama.cpp
also detects the text stack as `deepseek2` = DeepSeek-V3.)

## What was quantized

- **NVFP4 (W4A4), `nvfp4-pack-quantized`, group 16, FP8-E4M3 scales:** all
  language-model `Linear` layers **and all MoE experts** — the fused DeepSeek-V3
  experts are auto-linearized by llm-compressor and packed. **13,932 expert tensors
  packed** (verified — experts were NOT skipped).
- **Kept full precision:** vision tower, multimodal projector, MoE router
  (`mlp.gate`), `lm_head`, token embeddings.
- Calibration: `HuggingFaceH4/ultrachat_200k`, 128 samples, seq 512, text-only.

## Provenance & precision

Pipeline: original **FP8** consolidated → dequant to BF16 (lossless) → **NVFP4**.
The 4-bit weights derive from an already-8-bit source, not full-precision weights.

## ✅ Verified / ⚠️ Not yet verified  (tested 2026-07-04 on 1× B200, CUDA 13)

- ✅ Conversion is **structurally exact**: strict key+shape match against a real
  `Mistral3(deepseek_v3+pixtral)` model (801 tensors).
- ✅ NVFP4 produced with **all experts packed** (13,932 tensors); valid
  compressed-tensors format; loads in `transformers`.
- ✅ vLLM 0.23 **resolves the architecture** (`Mistral3ForConditionalGeneration`).
- ⚠️ **End-to-end generation not yet confirmed on a single B200.** Two engine-side
  gaps for this *new composed arch* (deepseek_v3 text inside Mistral3):
  - **vLLM 0.23** raises `KeyError: ...mlp.gate.e_score_correction_bias` when
    loading weights (its DeepSeek-V3 loader in the Mistral3 wrapper expects a
    different key path). Needs a vLLM build that supports this composition.
  - **transformers** loads it but **dequantizes NVFP4 to BF16** (no FP4 compute
    path) → ~159 GB + the naive MoE loop → OOM/too slow on one 183 GB B200.
- ⚠️ **Vision** path is converted (Pixtral tensors + `preprocessor_config.json`
  present) but **not runtime-validated**.
- ⚠️ RoPE scaling is **approximate** (YARN mapped; source also carries
  `llama_4_scaling`). Long-context (>8k) not guaranteed.

**In short: the artifact is correct; serving it needs engine support for the
composed arch that isn't in the pinned stack yet.** Track vLLM support for
DeepSeek-V3-backed Mistral3.

## Intended usage (mirrors the base model)

Tokenizer is **Tekken** (`tekken.json`, use `mistral_common`); chat format is the
Mistral instruct format (handled by `mistral_common`, no manual `[INST]`). The base
model recommends **temperature 1.0**. Once engine support lands, the intended path:

```bash
vllm serve sahilchachra/Leanstral-1.5-119B-A6B-NVFP4 \
  --tokenizer-mode mistral --reasoning-parser mistral \
  --tool-call-parser mistral --enable-auto-tool-choice
```

Files included: `config.json`, weights + index, `tekken.json`, `params.json`,
`chat_template.jinja`, `preprocessor_config.json`.

## License

Derives from `mistralai/Leanstral-1.5-119B-A6B`; upstream license/terms apply.
Community artifact, not affiliated with Mistral AI.