Honest thoughts, this is my new daily driver.
I usually don't leave these posts but given how fast-paced LLM developments are and the people also wanting a reason to try something new and most important that *IT WORKS I found myself pleasantly surprised using this model.
Background: I'm just yet another software developer. I own and heavily use the OpenAI Codex $200 subscription. I own a small server with RTX 3090 while trying to see if vulkan/ROCM will ever catch up in performance but I don't know much about that area. Why try local LLMs? Well... as a linux user, open source supporter, etc, etc. Why not? But mostly the fact that these models can continuously work on the background while you sleep researching or coding or troubleshooting is something that really interests me as a developer. I become more of a product owner/creator than just laying out all the tedious architecture/etc. Most of my life has been about creating solutions and trying to breakthrough issues. LLMs feels like an extension, like waking up realizing you have been missing an arm and now it's there and fully functional.
So yes, I do have a wide-range of usage for local LLMs and even use openrouter.ai to try out the new ones. I only dabble a bit.
Main usage so far: Web development, mainly creating wordpress plugins and tightening security.
Observations: This model is more concise and instruction tuned than the base model of Qwen 27B. It feels firmly-rooted in following instructions. It even knows when to stop itself and say, "oh, I'm looping, let me break out and see this differently"
Obviously, being 27B it's not going to be frontier-level. But, guess what? What most of us want from a model is generally to be instruction-tuned, and that it knows how to troubleshoot things. It needs concise, multi-step, pivotal troubleshooting. This model, for whatever reason, delivers that.
I gave it a couple of instructions, create wordpress plugin that supports custom CPTs where you can manage it with metaboxes for the really custom-driven one. And create widgets in the block editor I can pick from the CPT entries. More over, create template-parts that allows me to further customize the CPT entries in a flexibly and re-usable way. It should support desktop and mobile views.
And it delivered. Also, amusingly the model doesn't like PHP which left me chuckling because it was the language I started with back with PHP3.
Main CLI tool: Hermes TUI (previous opencode, openclaw, but those things are awful I'm sorry).
Anyhow, thought I would share even if it doesn't mean much.
Llama.cpp setup
Models:
ThinkingCap-Qwen3.6-27B-Q4_K_M.gguf
mmproj-ThinkingCap-Qwen3.6-27B-f16.gguf
-ngl auto
-c 114688
-ctk q8_0
-ctv q8_0
-n 8192
-np 1
--reasoning on
--reasoning-preserve
--reasoning-budget 2048
--reasoning-format deepseek
--jinja
--chat-template-kwargs '{"preserve_thinking":true}'
--temp 0.6
--top-p 0.95
--top-k 20
--min-p 0.00
--presence-penalty 1.0
--repeat-penalty 1.0
--image-min-tokens 1024
--image-max-tokens 4096
--host 0.0.0.0
--port 8000
--alias assistant
--timeout 7200
--sse-ping-interval 15
--sleep-idle-seconds -1
--no-webui
Thanks for sharing your experience and setup. Quantized kvcache ctk q_8 and ctv q_8
are super important options for long context reasoning and save memory.
I have an industrial passive cooling computer with AMD 7840HS + 780M igpu + 64GB RAM + cachyos + llama-server with Vulkan + Hermes Agent.
This model seems pretty nice. I tested it with a task in chat (create a one pager breakout game) and comparing to the original 27b, with mtp=4 I got a very decent speed on PP and decode.
I run the same task with two mtp settings (2 and 4).
--spec-type draft-mtp
--spec-draft-n-max 4 --> and 2 in an other test
--spec-draft-p-min 0.85
--kv-unified
--cache-type-k q8_0
--cache-type-v q8_0
On the X axis you can see the generated tokens, while the Y axis is the tg/s.
One line is one continuous n_decode run.
In overall, it beat unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_XL in speed and quality.
I also tried unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:Q4_K_XL . This one was painfully slow.
My limited experience is, this qwen3.6 27b just beats everything in my setup.
Of course, I will need to check the same performance parameters during marketing research tasks and other kind of projects. For coding, it seems awesome.
I have an industrial passive cooling computer with AMD 7840HS + 780M igpu + 64GB RAM + cachyos + llama-server with Vulkan + Hermes Agent.
This model seems pretty nice. I tested it with a task in chat (create a one pager breakout game) and comparing to the original 27b, with mtp=4 I got a very decent speed on PP and decode.
I run the same task with two mtp settings (2 and 4).
--spec-type draft-mtp
--spec-draft-n-max 4 --> and 2 in an other test
--spec-draft-p-min 0.85
--kv-unified
--cache-type-k q8_0
--cache-type-v q8_0On the X axis you can see the generated tokens, while the Y axis is the tg/s.
One line is one continuous n_decode run.
In overall, it beat unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_XL in speed and quality.
I also tried unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:Q4_K_XL . This one was painfully slow.
My limited experience is, this qwen3.6 27b just beats everything in my setup.
Of course, I will need to check the same performance parameters during marketing research tasks and other kind of projects. For coding, it seems awesome.
I don't know much about the state of AMD GPUs but try turning on flash attention it can honestly make a huge difference. I basically have it on on my RTX 3090 because devstral would stall a lot.
I don't know much about the state of AMD GPUs but try turning on flash attention it can honestly make a huge difference. I basically have it on on my RTX 3090 because devstral would stall a lot.
Thank you for your suggestion, in fact, the integrated GPU like 780M suffers from low memory bandwidth (dual channel, DDR5, 5600MHz -> 70-90GB/sec maximum), so, these numbers are awesome :)
Normally a non-mtp dense version (qwen3.6 27b - q8-q4) could achieve 1-2 tg/s.
This is my actual parameters, any hints are welcome (please note, igpu is very different than a dedicated GPU):
example: --chat-template-kwargs '{"enable_thinking":false}' -> deprecated, instead will be used: --reasoning on / --reasoning off
llama-server --port ${PORT} --host 0.0.0.0
-hf bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF:Q4_K_M
--temp 0.6
--top-p 0.95
--top-k 20
--min-p 0.00
--repeat-penalty 1.05
--presence-penalty 1.0
-np 1
--ctx-size 262144
--flash-attn on
--n-gpu-layers 99
-b 1024
-ub 256
--spec-type draft-mtp
--spec-draft-n-max 4
--spec-draft-p-min 0.85
--chat-template-kwargs '{"enable_thinking":false}'
--reasoning-format deepseek
--kv-unified
--cache-type-k q8_0
--cache-type-v q8_0
--cache-reuse 256
--cache-ram 16384
--ctx-checkpoints 16
--checkpoint-min-step 1024
--jinja
--no-mmproj
--no-ui
-t 6
--no-warmup
--timeout 0
--log-file /home/pongraczi/llama-thinkingcap-qwen3.6-27-q4-lv5-1.log
-lv 5
As I hit this bug: https://github.com/NousResearch/hermes-agent/issues/20866 , I still running in debug mode, I need to find a solution to avoid this annoying http 400 error, which blocks the "flow" :)
As I hit this bug: https://github.com/NousResearch/hermes-agent/issues/20866 , I still running in debug mode, I need to find a solution to avoid this annoying http 400 error, which blocks the "flow" :)
Not exactly this bug, but very similar, at this moment in Kanban task.
41.09.953.109 D Parsed message: {"role":"assistant","content":"","tool_calls":[{"type":"function","function":{"name":"write_file","arguments":"{"path":"/home/pongraczi/Projektek/WMS_basic/.worktrees/t_e1f323ea/app/src/main/java/com/wm
s/basic/data/database/StockEntity.kt","content":"package com.wms.basic.data.database\n\nimport androidx.room.Entity\nimport androidx.room.PrimaryKey\n\n@Entity(tableName = \"stock\")\ndata class StockEntity(\n @PrimaryK
ey val id: Long = 0,\n val sku: String = \"\",\n val locationCode: String = \"\",\n val quantity: Int = 0\n)\n"}"}}]}
41.09.953.373 D srv operator(): http: streamed chunk: data: {"choices":[{"finish_reason":"tool_calls","index":0,"delta":{}}],"created":1785948005,"id":"chatcmpl-SJmIbCumCh4wpaRrZKXxk2u7kwQup9fY","model":"bottlecapai/ThinkingCap-Qwen3.
6-27B-GGUF:Q4_K_M","system_fingerprint":"b10216-06be260","object":"chat.completion.chunk"}
data: {"choices":[],"created":1785948005,"id":"chatcmpl-SJmIbCumCh4wpaRrZKXxk2u7kwQup9fY","model":"bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF:Q4_K_M","system_fingerprint":"b10216-06be260","object":"chat.completion.chunk","usage":{"completi
on_tokens":146,"prompt_tokens":46098,"total_tokens":46244,"prompt_tokens_details":{"cached_tokens":45804}},"timings":{"cache_n":45804,"prompt_n":294,"prompt_ms":14368.073,"prompt_per_token_ms":48.87099659863946,"prompt_per_second":20.462
034122460263,"predicted_n":146,"predicted_ms":34868.026,"predicted_per_token_ms":238.82209589041094,"predicted_per_second":4.187217251702176,"draft_n":117,"draft_n_accepted":113}}
41.09.953.376 D srv operator(): all results received, terminating stream
41.09.953.387 D srv operator(): http: streamed chunk: data: [DONE]
41.09.953.402 D srv operator(): http: stream ended
41.09.953.413 D res remove_waiti: remove task 1526 from waiting list. current waiting = 1 (before remove)
41.09.953.414 D srv stop: all tasks already finished, no need to cancel
41.42.566.511 D common_chat_templates_apply_jinja: using differential autoparser
41.42.566.514 D ESC[1mESC[38;5;126m=== Starting differential analysis ===
ESC[0m41.42.566.514 D ESC[1mESC[38;5;214mPhase 1: Reasoning analysis
ESC[0m41.42.567.714 D ESC[1mESC[38;5;214mPhase 2: Content analysis
ESC[0m41.42.568.674 D ESC[1mESC[38;5;214mPhase 3: Tool call analysis
ESC[0m41.42.569.926 D ESC[1mESC[38;5;214mPhase 3a: Function call analysis
ESC[0m41.42.570.560 D ESC[1mESC[38;5;214mPhase 3b: Argument analysis
ESC[0m41.42.572.928 D ESC[1mESC[38;5;214mPhase 3c: Call id analysis
ESC[0m41.42.573.853 D Template application failed:
While executing CallExpression at line 43, column 24 in source:
... not messages %}β΅ {{- raise_exception('No messages provided.') }}β΅{%- endif %...
^
Error: Jinja Exception: No messages provided.
41.42.573.960 D ESC[1mESC[38;5;214mdetect_user_start_marker: Template application failed, unsupported empty messages? trying complex variant
ESC[0m41.42.574.342 D
--- Reasoning & Content Structure ---
41.42.574.344 D user_msg_start: <|im_start|>user
41.42.574.344 D assistant_msg_start: <|im_start|>assistant
41.42.574.344 D reasoning_mode: TAG_BASED