A crazy thinking model
This is a preview model i think, it always thinking and thinking, can it stop?
This is a preview model i think, it always thinking and thinking, can it stop?
"--reasoning-budget N",
N equals to thinking tokens
This is a preview model i think, it always thinking and thinking, can it stop?
there are three thinking levels for a reason. Try to change them
The default shouldn't be an unusable amount of thinking.
This was clearly chosen for test maxing because without it Qwen 3.8 isn't broadly better than Qwen 3.6. If anything, it's broadly worse. Alibaba, like the rest of the industry, is masking the fact that they're hit a ceiling. They've been trading broad abilities for marginal gains in select domains, especially coding and agentic work, for more than a year. And now they're adding an unusable amount of latency (>10x longer) to squeeze out a tiny bit more performance in only a handful of domains.
The default shouldn't be an unusable amount of thinking.
This was clearly chosen for test maxing because without it Qwen 3.8 isn't broadly better than Qwen 3.6. If anything, it's broadly worse. Alibaba, like the rest of the industry, is masking the fact that they're hit a ceiling. They've been trading broad abilities for marginal gains in select domains, especially coding and agentic work, for more than a year. And now they're adding an unusable amount of latency (>10x longer) to squeeze out a tiny bit more performance in only a handful of domains.
idk man. I am only using the xhigh level now and it's been pretty good so far. Not using gguf quants though, but exl3
@vonamakitsune That's good to hear. A lot of people are saying Qwen 3.8 is doing better at long coding tasks. But I only test models without thinking, and most are regressing. Even with thinking on Arena across the various domains model families are regressing. Such as Opus 5 doing worse than Opus 4.7, which is doing worse than 4.6. Or Grok 4.6 doing worse than 4.5, which is doing worse than 4.2. And OpenAI's GPT series is stagnated (e.g. GPT 4.5/4.6). And Qwen 3.6 is broadly worse than Qwen 3.5.
The reason why is clear. The underlying LLMs are the same (e.g. same information density), as is the training. The primary change is grossly overtraining for agentic and long context coding work, so once they're done the weights have been scrambled more, making the models generally inferior (e.g. more factual hallucinations and lower quality creative writing with more contradictions).
In short, the industry seems to be giving up making AI models generally stronger, and are instead trying to maximize performance in a handful of select domains, which includes throwing a lot more tokens at them.
This is a preview model i think, it always thinking and thinking, can it stop?
there are three thinking levels for a reason. Try to change them
I have set to low, but nothing happend, still thinking without stop.
Load with chat-template-kwargs "{\"preserve-thinking\": true,\"reasoning_effort\": \"medium\"}" is better.
Load with chat-template-kwargs "{\"preserve-thinking\": true,\"reasoning_effort\": \"medium\"}" is better.
sadly even on medium it hits the wall exactly in a same way. Even exceeding the max output thinking length... its crazy annoying. Hope for some fine tunes of the model. the base on is not impressive when it comes to efficiency sofar
...still thinking
The default shouldn't be an unusable amount of thinking.
This was clearly chosen for test maxing because without it Qwen 3.8 isn't broadly better than Qwen 3.6. If anything, it's broadly worse. Alibaba, like the rest of the industry, is masking the fact that they're hit a ceiling. They've been trading broad abilities for marginal gains in select domains, especially coding and agentic work, for more than a year. And now they're adding an unusable amount of latency (>10x longer) to squeeze out a tiny bit more performance in only a handful of domains.
I beg to differ. It has been a huge improvement for me compared to Qwen3.6.
@hermanbrood Many people are sharing your opinion that Qwen 3.8 is a huge improvement over Qwen 3.6 on large coding projects. But how is it at everything else?
Qwen 3.5 was generally stronger than Qwen 3.6 (more broad knowledge, better stories, fewer boneheaded mistakes, and so on). This is almost certainly why Qwen 3.6's score was never added to the Arena. And my guess is Qwen 3.8 is performing generally much worse than Qwen 3.6, and its score will also never be added to Arena.
It seem Alibaba is pushing coding and agentic really hard, and at the expense of broad knowledge and abilities. Plus if it takes ~10x longer to respond with thinking than without it's less than useless in real-time scenarios, such as customer service. This model really appears to be a tool (agentic coding) rather than a general purpose AI model.
This model really appears to be a tool (agentic coding) rather than a general purpose AI model.
Yeah, exactly. As i always understood, it was never intended to be a general-purpose model. And my guess - if it gained so much in agentic/coding benchmarks compared to the 3.6 version, it certainly got worse in other fields. But tbh that was the expectation for many people (probably developers). We want something that can reason strongly, knows how to work with info and understand all the right patterns rather than having up-to-date knowledge or being creative.
Out of curiosity, have you tried gemma 4 31b or any fine-tune of it? I found it extremely better in terms of creative tasks and all of that
@vonamakitsune Thanks for the recommendation. Yes, I've tried Gemma 4 and it's notably better at most things, including creative writing. For example, Qwen's stories are more repetitive, include more boneheaded mistakes, like contradictions to itself and the user's prompt, and so on.
But since Qwen 3.8 max is balanced and broadly capable you'd think Alibaba would make a balanced general purpose Qwen 3.8 27b and a separate Qwen 3.8 27b Coder for maximum coding/agentic performance.
Alibaba focusing on coding/agentic tasks wouldn't be so bad if most other companies weren't doing the same, including Nvidia and Meta, likely because >90% of the users of small OS AI models are coders so the entire ecosystem is moving away from small general purpose AI models in favor of coding/agentic tools.
if you actually have enough KV to give it a full 256k context window the thinking pays off. It eventually does stop and starts to execute, and because it thought so much it does an excellent job.
you'd think Alibaba would make a balanced general purpose Qwen 3.8 27b and a separate Qwen 3.8 27b Coder for maximum coding/agentic performance.
You know, you got me thinking about making just a very strong base and adding LORA heads to it. That'd be really cool. Your base model is always in VRAM. And then you just switching heads depending on a task. Fast switches, strong capabilities. And i know it was already kinda tried by the HG community, but i never heard about it in terms of a big corporate environment, if you know what i mean. Like actually proved and tested performance, not someone's pet project.
Try to disable preserve_thinking. Seems KV cache wall is more distant in this way.
Before this model I used ThinkingCap-Qwen3.6-27B extensively for programming and agentic flows, and I was really impressed with that finetune. It felt really snappy and responsive and didn't need to think much.
I was very hyped for Qwen3.8-27B, but wasn't prepared for how much slower it would be in practice. Even with effort set to low, it thinks A LOT. I gave it a very simple task of fixing a bug in my project. It managed to identify the problem within a minute or two, but then it went off on a tangent where it read lots of unrelated files and wasted more than half an hour doing something completely unrelated to the task at hand before I had to stop it. This is an issue I've never experienced with Qwen3.6 27B. The latter never derails like that. It gets straight to work and it doesn't waste my time.
I feel like a lot of the positive news about this model is overblown. It does appear to be better at invoking skills in Claude Code automatically though, which Qwen3.6 27B wasn't good at, but aside from that it's just too slow to be useable on my RTX 5090.
@29sushi since you have 32gb of vram i strongly suggest you to try an exl3 quant (you can use tabbyapi for inference, but pay attention!! it is GPU only, no ram offloading). It is stronger than ggufs for the same size and you will probably not have such a problem with 30 minutes thinking. 4 or 5 bits would be enough to feel the power .
If you try it, let me know how it goes. I use exl3 since the release of qwen 3.6 27b and never been happier
In case you need exact params to run it, here is what i use right now (modify it for you gpu):
#!/bin/bash
/home/<YOUR_USER>/tabbyAPI/start.sh \
--model-name Qwen3.8-27B-exl3-5.0bpw \
--max-seq-len 131072 \
--cache-size 262144 \
--cache-mode "6,6" \
--override-preset qwen38 \
--vision true \
--tool-format qwen3_coder \
--max-batch-size 2 \
--tensor-parallel true \
--draft-mode mtp
Also pay attention to the override-preset parameter. It is just a file where you specify sample params like temp, top-p and so on. I put numbers from this official repo
@vonamakitsune I figured out why Qwen3.8 27B was so slow on my machine. Turns out I had a bug in my Jinja template that caused regular cache misses so that I had to do a 70k prefill on every turn, which takes like 50+ seconds. A cache hit takes just over a second. As soon as I resolved that it got much quicker. It's still roughly 2.7x slower than ThinkingCap-Qwen3.6-27B, but the increase in capability more than makes up for it. Now I'm much happier with it. I'm running it on low or medium effort. I still haven't found a use case for xhigh.
Hi @29sushi how did you fixed the prompt issue? I'm having good token generation on my 2 RTX 4090 ~100 T/s, but the MTFT is very high on Claude Code that makes it almost unusable ~40s. I think that mostly depends on wasted cache reuse:
47.48.852.100 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, f_sim_best = 0.397 (> 0.100 thold), f_keep = 0.399
36.57.624.174 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 0.612 (> 0.100 thold), f_keep = 0.614
25.30.365.315 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 0.544 (> 0.100 thold), f_keep = 0.545
10.50.593.196 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, f_sim_best = 0.503 (> 0.100 thold), f_keep = 0.503
820.26.333.256 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, f_sim_best = 0.634 (> 0.100 thold), f_keep = 0.638
6.02.643.370 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, f_sim_best = 0.995 (> 0.100 thold), f_keep = 1.000
6.16.887.422 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, f_sim_best = 0.212 (> 0.100 thold), f_keep = 0.141
13.24.340.610 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 0.153 (> 0.100 thold), f_keep = 0.129
14.31.896.036 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 0.175 (> 0.100 thold), f_keep = 0.174
15.29.837.683 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 0.158 (> 0.100 thold), f_keep = 0.126
16.26.952.182 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 0.295 (> 0.100 thold), f_keep = 0.341
thank you
I don’t know about Claude Code specifically, I don’t use it. But I do use VS Code and Cursor very successfully with this model. I assume Claude Code suffers from the same issue as those other two proprietary harnesses: No effort flags are sent for local models.
Because they want to reserve effort flags and advanced features for their own models only, local models are invoked with default flags.
The issue is that by default, Qwen 3.8 27B responds to all prompts with “xhigh” reasoning effort. This leads to the exact thing you are describing, 40+ second TTFT and long, repetitive reasoning sessions that can drag out for several minutes before it does anything.
Fine for a pelican riding a bike one shot, not fine for chaining hundreds of tool calls in an IDE…
The effort setting that works best for VS Code and Cursor (and I suspect Claude Code as well) is “Low”.
What? Why would anyone ever use Low? Are you crazy? No way xhigh 4 lyfe!!!1!
Trust me. On this model “low” is the same as “High” on Opus or Sol.
Only use “medium” for your hardest tasks and only use “xhigh” for building plans and writing documentation. Never tool-calling.
I use “LiteLLM Proxy” as a shim between my qwen vLLM box and cursor/vs code. LiteLLM serves up 4 “virtual models” for me, all from the same vLLM instance, named: “Qwen-xhigh”, “Qwen-Med”, “Qwen-Low”, and “Qwen-NoThink”.
So when I point Cursor at “Qwen-Low” on my LiteLLM proxy, it intercepts the prompt, injects the effort kwarg set to “Low” and forwards the request to my vLLM box. As far as VS Code or Cursor is concerned, I just have four different models to choose from on the same API, and it will let me pick them in the local model picker.
This is the best workaround I have found for being able to set reasoning effort in apps that don’t support or send effort kwargs.
This has been transformative. My output has tripled or more (since last week when I was stuck on xhigh for everything). Low is amazing. Qwen 3.8 27B isn’t unusable at xhigh effort, it actually does an excellent job, but it does burn about 60K tokens thinking every single turn, and it is very, very slow to act and extremely prone to overthinking. On Low, it behaves just like Opus 4.6 on High.
@Nonplay Try this fixed template instead and see if you get better results. My template is based on this one https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates