**QAT-optimized weights preserved at Q4_0, overhead tensors quantized to NVFP4.**
## What Makes This Different
This is a **hybrid quantization** of Google official QAT (Quantization-Aware Training) model. Instead of requantizing the Q4_0 weights (which breaks QAT benefits and vision quality), we:
1. **Kept all weight tensors at Q4_0** - attention, FFN, embeddings - exactly as Google trained them
2. **Quantized only the F32 norm/bias tensors to NVFP4** - these are the overhead tensors (layer norms, RMS norms, etc.)
3. **Used Google QAT mmproj** - the vision projector trained alongside the QAT model
### Why Standard NVFP4 from QAT Breaks Vision
Google QAT model was specifically trained to be resilient to Q4_0 quantization patterns. The weight values learned during QAT compensate for Q4_0 rounding. When you requantize Q4_0 -> F32 -> NVFP4, a second round of quantization error is introduced that QAT training did **not** account for. Vision tokens flow through the same attention/FFN layers - precision loss disproportionately degrades vision.
### How the Hybrid Approach Works
Using llama-quantize --tensor-type-file with --allow-requantize:
The tensor-type-file lists all Q4_0/Q4_K tensors to keep at their current type. When the quantizer sees cur_type == new_type, it copies the tensor data as-is - **zero precision loss**. Only the remaining F32 tensors are quantized to NVFP4.
