Jackrong

/

Qwopus3.5-27B-v3

like 249

I will

Is there any chance we could a get a qwopus3.5 v3 0.8B so we could use it as a draft model for the 27B? Also just qwopus v3 tunes of the smaller models would be nice in general.

How do you draft qwen3.5? MTP?
I am using llama.cpp, it does not support MTP yet.. Old methods simply fail to enable speculative encoding.

Is there any chance we could a get a qwopus3.5 v3 0.8B so we could use it as a draft model for the 27B? Also just qwopus v3 tunes of the smaller models would be nice in general.

How do you draft qwen3.5? MTP?
I am using llama.cpp, it does not support MTP yet.. Old methods simply fail to enable speculative encoding.

I am using vllm not lama.cpp. I have gotten ngram and running a dedicated model separately as its own deployment works as well (does require building a harness for it).

Yes, that would honestly be incredible! A Qwopus3.5 v3 0.8B would be a dream for draft-model / speculative decoding use cases, and a 2B variant on top of that would be even more amazing — perfect size for local/edge deployment!

Jackrong u the man!

Thanks everyone for the support! I’ve been a bit busy lately, but I should have some time today. I’ll optimize the 2B and 0.8B models as soon as I can!

Thanks a lot! And also thanks for the comprehensive PDF guide and the whole repo — super helpful, really cool!

Thanks a lot for the support — really appreciate it!

I’ll gradually put together all the guides and a complete notebook once I have more time😄