unsloth

/

Devstral-2-123B-Instruct-2512-GGUF

like 55

Follow

Unsloth AI 27.6k

Currently llama.cpp only supports text only. Vision is been worked on I think

Appreciate the response @danielhanchen but it's a bit confusing because I've been running --mmproj with Qwen3-VL-30B-A3B-Instruct-UD-Q8_K_XL. Using the param --mmproj /mnt/data/models/Qwen3-VL-30B-A3B-Instruct-UD-Q8_K_XL/mmproj-BF16.gguf the model does load correctly and even in the stripped down llama.cpp web ui, I can paste in images to the chat with no issues.

@aaron-newsome are you in the right repo? Or you mean you used the Qwen3-VL mmproj for Devstral-2-123B?

my question was, where is the mmproj file for Devstral-2-123B-Instruct-2512-GGUF

then danielhanchen said llama.cpp doesn't support images

to which i responded, sure it does because i've used images with llama.cpp and other image models

i'm in the right place

Oh I see! No he was meaning not supported yet for this specific model. Sorry for the misunderstanding! I thought you tried to load the Qwen-VL mmproj alongside Devstral-2-123b 😅

If someone can provide instructions for extracting the encoder and converting it to .mproj, I’m willing to execute them. Let me know the steps.

Maybe we should ask the guys who made the Qwen3-VL-30B-A3B-Instruct-UD-Q8_K_XL/mmproj-BF16.gguf file and see how they made it? I'd be happy to give it a try myself but I have no idea where to start. I also think that @danielhanchen is suggesting that it's not really about making the mmproj file itself, but that llama.cpp needs to support it. I can definitely say the Qwen3-VL works but I have no idea whether llama.cpp would support the mmproj file for this model if it existed. Looking at the llama.cpp guides, it doesn't really say that only mmproj files are supported for certain models, it seems generic like any model that has an mmproj file would be supported. Again, I have no idea.

@dr-e deepwiki is really cool you should try it, it helps really much when you need to dig into documentation ;)
https://deepwiki.com/search/how-to-create-the-mmprojgguf-f_461ddcaa-3ec8-41c3-b19c-cbac84264874

According to unsloth own website the mmproj file should exist in this repository, but it does not exists! … https://unsloth.ai/docs/models/devstral-2

In my opinion this a mistake on unsloth side of things. If its a documentation issue or an issue with the upload to the repo is on unsloth to figure out.

Yep, I think the unsloth docs are incorrect, copied from another model (like devstral small 2).

It seems that only devstral small 2 supports vision. That SORT OF makes sense as it's smaller and faster for ongoing work like checking lots of screen updates or processing lots of images, and you'd normally use a larger model for more "thoughtful" stuff.