@Zesales Good request — and the timing's right, because NVFP4 weight-only (W4A16) is exactly the path vLLM now supports on Blackwell (RTX 50-series), with FP16 KV left untouched. So the format you're asking for is feasible on consumer Blackwell today.
Two honest caveats before I commit to anything:
vLLM compatibility for this checkpoint. vLLM added native Gemma 4 support in 0.19, but this is a fine-tune with a custom chat template and a few non-standard config bits, so I need to validate vLLM compatibility for this checkpoint first — confirm it loads and generates correctly at FP16 before quantizing. I don't want to ship an NVFP4 build that silently misbehaves.
It's a from-master job. NVFP4 has to be calibrated from the fp16 safetensors master with nvidia-modelopt, not reconverted from a GGUF — so it's a separate artifact to build and upload, not a quick repack.
Net: I'm genuinely interested and it's on th (dropping in the next day or two) and the27B first, then do the vLLM/NVFP4 path properly with a real end-to-end check on a 50-series card. I'll post here once I've verified it loads + runs clean. And if you're on a 5090/5080 and want to try quantizing from the master weights yourself in the meantime, happy to point you.
As a user with a single RTX 5080 (16 GB VRAM), I’d really like to try your model with NVFP4 weights. There are currently very few such options available on Hugging Face.
I believe an NVFP4 version of your model would be highly relevant and in demand among users like me who are running it locally on Blackwell GPUs.
I would really appreciate it if you could share the current status of this request and whether you have any plans to release an NVFP4 version.
