---
license: apache-2.0
library_name: mlx
base_model: mistralai/Mistral-Medium-3.5-128B
pipeline_tag: image-text-to-text
tags:
  - jangtq
  - mxtq
  - jang
  - mlx
  - apple-silicon
  - mistral3
  - ministral3
  - pixtral
  - vision
  - multimodal
  - quantized
---

![Osaurus](osaurus-x-banner.png)

# OsaurusAI/Mistral-Medium-3.5-128B-JANGTQ

Quantized **mistralai/Mistral-Medium-3.5-128B** for Apple Silicon (MLX) — dense 128B parameter multimodal LLM with image input + 256K context.

| | |
|--|--|
| **Source** | [mistralai/Mistral-Medium-3.5-128B](https://huggingface.co/mistralai/Mistral-Medium-3.5-128B) |
| **Architecture** | `mistral3` wrapper — `ministral3` text decoder (88L × 12288, GQA 96/8) + `pixtral` vision encoder (48L × 1664) |
| **Quant format** | JANGTQ (TurboQuant 2-bit + Hadamard, group_size=64) |
| **Bundle size on disk** | **40.76 GB** (37 safetensors shards) |
| **License** | Apache-2.0 (inherits from upstream) |
| **Modalities** | Text + image in / text out (no audio, no video) |
| **Context** | 262 144 tokens (YaRN, factor=64 from orig=4096) |

## What's quantized

- Text decoder linears (88 layers × 7 projections = q/k/v/o + gate/up/down) → **TurboQuant 2-bit** with Hadamard pre-rotation (group_size=64)
- `embed_tokens` → affine 8-bit (`mx.quantize`)
- All RMSNorms (input/post-attention) → fp16 passthrough
- `vision_tower.*`, `multi_modal_projector.*`, `lm_head` → fp16 passthrough (matches upstream `modules_to_not_convert`)


**Vision tower (`model.vision_tower.*`, 48 layers, 1664 hidden, patch=14, image_size=1540, spatial_merge=2)** — kept **bf16 → fp16 passthrough**, matching upstream `quantization_config.modules_to_not_convert`. Images dispatch to the same `pixtral` encoder; embeddings are folded into the LM via a 2-layer GELU multimodal projector (also fp16 passthrough).

**`lm_head` is fp16 passthrough** (matches upstream ignored set) — no quantization noise on the final logits.

## Codec round-trip validation (this bundle)

7/7 PASS at cosine ≥ 0.94 across L0 → L87 covering attn + MLP + GQA k/v projections — the expected 2-bit TurboQuant noise floor. Source FP8 e4m3 dequant uses `weight_scale_inv` per-tensor scale; vision/projector/lm_head pass through unchanged.

## Run on Apple Silicon

```bash
pip install mlx safetensors transformers pillow
python -m jang_tools.mistral3.runtime \
    --src ~/.mlxstudio/models/OsaurusAI/Mistral-Medium-3.5-128B-JANGTQ \
    --prompt "Describe this image." \
    --image /path/to/photo.jpg \
    --max-new 64
```

The runtime auto-detects `weight_format` and dispatches; image preprocessing matches the pixtral spec (`jang_tools.vl.pixtral.PixtralImageProcessor`).

## Build

```bash
python -m jang_tools.convert_mistral3_jangtq \
    ~/.mlxstudio/models/_sources/Mistral-Medium-3.5-128B \
    ~/.mlxstudio/models/JANGQ-AI/Mistral-Medium-3.5-128B-JANGTQ JANGTQ2
```

## Credits

Quantized by **Jinho Jang** ([eric@osaurus.ai](mailto:eric@osaurus.ai)). MLX-native pipeline; 88 dense decoder layers + 48 pixtral vision layers run on M-series Macs.