THE TRUE GOD-TIER MASTERPIECE: UNMATCHED QUALITY UNLEASHED
Holy shit, after a few tests I've gotta say this is the absolute best Qwen 27B I've ever used.
The results were quite surprising.
My hardware can only manage IQ4_XS Low, but it completely destroys every other IQ4_XS variant on HuggingFace.
It's the real deal.
Thanks, bro 😀
Thank you so much! ;
More are coming... but they need some cookin'
What were you testing?
Holy shit, after a few tests I've gotta say this is the absolute best Qwen 27B I've ever used.
The results were quite surprising.
My hardware can only manage IQ4_XS Low, but it completely destroys every other IQ4_XS variant on HuggingFace.
It's the real deal.Thanks, bro 😀
Hello, I have a few observations after using IQ4_XS-Low for two days. I mainly used it with Pi coding agent.
Fable-Fusion-711 is very compliant with instructions, proactively uses tools and skills, and seems more like a top-tier model. Local models are very lazy in using tools and skills unless thoroughly instructed.
It can perform a task with a single prompt without errors, without having to repeat error correction operations multiple times. With the same prompt, output is often better than expected compared to base version or some other variants (only comparison on IQ4_XS, not on higher quantization.)
I compared it to Ornith 35B in UD-Q8_K_XL without compressing KV cache. Ornith ran faster but had significantly more errors, used more tokens, and its tool usage seemed like that of a young, careless person, while Fable-Fusion-711 was like a mature user.
Please cook UD-IQ3-XXS
Are you doing some kind of QAT + OPD for each level of quantization ? would explain the delay ;)
@DavidAU
What am i doing wrong? standard qwen27b-mtp runs at full gpu speed, but this model uses about 80% gpu and 20%cpu or something like that... in task manager. iq_4xs quant, windows vulkan llama.cpp. Identical parameters/flags. Just same as i use for standard model. -np 1 --no-mmap --spec-type draft-mtp --spec-draft-n-max 2 --cache-type-k q8_0 --cache-type-v q8_0.
For my igpu system this means almost 2x lower speed than standard qwen27b-mtp!
Any other model on my system always uses 100%gpu with vulkan
This is the best 27B model I have tested, I'm recommending it for my AI system:
https://github.com/Kenshiro-28/MAGI
Thank you! :)
Wait, as of today, I feel like something is missing. It seems to have a higher hallucination rate than the base version. Because its writing style is so smooth, maybe it’s just really good at tricking us?
I'm not entirely sure—it might be better than the base model in some aspects at the expense of others. Or could it just be because I'm running a low quant 😄
@toold
Output will vary between quants, and NEO Imatrix will have inverse effect the lower in quants you go too.
Likewise with "Q" vs "IQ" quants ; different mix and slightly different processing.
There seems to be an issue with BF16 Output tensor VS Vulkan/AMD ; try the IQ4_XS "LOW" and see if that addresses the CPU issue.
a bit of a surprise, but it works now, thanks
file size is much smaller though, hope it's no too crucial in terms of output quality
Perfect; I will make a test quant with F16 for the OT shortly ; this will mirror quality in most respects and should load fully on GPU.
[repeat from another thread]
CPU offload issues - Vulcan and/or AMD related:
I am uploading(ed) TWO quants:
IQ4XS and Q6 with "AMD" in the name.
This changes the output tensor to f16 (from bf16).
This should give you close to max performance, without CPU offloading.
For uses with the cpu offload issue, can you post feedback and see if this addresses the cpu offload issue(s)?
These will be uploaded about 1/2 hr or so from this post's timestamp.
Thank you ;
I still do have similar problems with IQ4XS (https://huggingface.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF/blob/main/Qwen3.6-27B-Fable-Fus-711-UnHeretic-NM-DAU-NEO-MAX-NEO-AMD-MTP-IQ4_XS.gguf) with Llamacpp, 2x 9070XT. Generation is very slow and I do see a lot of cpu usage. Prompt processing is fine (on par with Qwopus for example)
Thank you ; we are checking into other solution(s).
I still do have similar problems with IQ4XS (https://huggingface.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF/blob/main/Qwen3.6-27B-Fable-Fus-711-UnHeretic-NM-DAU-NEO-MAX-NEO-AMD-MTP-IQ4_XS.gguf) with Llamacpp, 2x 9070XT. Generation is very slow and I do see a lot of cpu usage. Prompt processing is fine (on par with Qwopus for example)
Llama + gfx12xx == bad times. I have sister cards and have done a ton of testing with them. Testing this quant now to verify speeds.
Switch to my vllm image (mainline vllm will perform terribly) would instantly double PP speed and likely 4x output speed using this model, the llama.cpp mtp is nascent and performs notably worse per predicted token than vllm.
Will publish the RFA quant for it tonight.
Are you running via ROCM or Vulkan, windows or linux?
If ROCM are you on 7.2.3 or 7.2.4 if on linux?
7.2.3 paired with latest llama.cpp compiled on my system with correct AMD packages is the first time I've seen ROCM correctly perform in llama and outrun vulkan on wall clock time during actual work.
Wait, as of today, I feel like something is missing. It seems to have a higher hallucination rate than the base version. Because its writing style is so smooth, maybe it’s just really good at tricking us?
I'm not entirely sure—it might be better than the base model in some aspects at the expense of others. Or could it just be because I'm running a low quant 😄
That is a characteristic of subtle but real quant damage, confidently lying because it can't tell the difference, it gets a signal on a fact it should know, but cant recall it, so it makes up a plausible replacement since the confidence in the "i should know this" stream gives a high score.
Traditionally, ablation makes this worse, I haven't evaluated multiple quants of 711 yet to determine if it does so, but it's a known quantity when compressing models with no internal or a weak internal rejection signal.
Keen of you to note it, a lot of people just quietly plod along assuming their quants are roughly equal to full size. They are not.
I still do have similar problems with IQ4XS (https://huggingface.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF/blob/main/Qwen3.6-27B-Fable-Fus-711-UnHeretic-NM-DAU-NEO-MAX-NEO-AMD-MTP-IQ4_XS.gguf) with Llamacpp, 2x 9070XT. Generation is very slow and I do see a lot of cpu usage. Prompt processing is fine (on par with Qwopus for example)
Based on results, this is consistent with what I've seen 27B do on 2xgfx12xx GPUs, its the current state of llama.cpp on AMD, nothing untoward, its not great but its what I see across all 27Bs.
System state during inference, I have another small model loaded across all 4 gpus but it's completely idle, ignore that little bit MTP2:
qwen36-27b-fable-711 | 3.01.683.073 I statistics draft-mtp: #calls(b,g,a) = 8 1602 1602, #gen drafts = 1602, #acc drafts = 1357, #gen tokens = 3204, #acc tokens = 2536, #mean acc len = 2.58, #acc rate/pos = (0.847, 0.736), dur(b,g,a) = 0.007, 16079.675, 0.802 ms
qwen36-27b-fable-711 | 2.00.450.251 I slot create_check: id 0 | task 606 | created context checkpoint 10 of 32 (pos_min = 26981, pos_max = 26981, n_tokens = 26982, size = 255.539 MiB)
qwen36-27b-fable-711 | 2.03.033.063 I slot print_timing: id 0 | task 606 | n_decoded = 100, tg = 39.53 t/s
qwen36-27b-fable-711 | 2.06.034.122 I slot print_timing: id 0 | task 606 | n_decoded = 230, tg = 41.59 t/s
qwen36-27b-fable-711 | 2.09.086.955 I slot print_timing: id 0 | task 606 | n_decoded = 367, tg = 42.76 t/s
qwen36-27b-fable-711 | 2.12.142.599 I slot print_timing: id 0 | task 606 | n_decoded = 506, tg = 43.47 t/s
qwen36-27b-fable-711 | 2.15.184.783 I slot print_timing: id 0 | task 606 | n_decoded = 624, tg = 42.50 t/s
qwen36-27b-fable-711 | 2.16.099.884 I reasoning-budget: deactivated (natural end)
qwen36-27b-fable-711 | 2.18.242.097 I slot print_timing: id 0 | task 606 | n_decoded = 747, tg = 42.11 t/s
qwen36-27b-fable-711 | 2.21.297.236 I slot print_timing: id 0 | task 606 | n_decoded = 870, tg = 41.84 t/s
qwen36-27b-fable-711 | 2.24.324.087 I slot print_timing: id 0 | task 606 | n_decoded = 1026, tg = 43.07 t/s
qwen36-27b-fable-711 | 2.27.332.335 I slot print_timing: id 0 | task 606 | n_decoded = 1169, tg = 43.57 t/s
qwen36-27b-fable-711 | 2.30.349.093 I slot print_timing: id 0 | task 606 | n_decoded = 1305, tg = 43.72 t/s
qwen36-27b-fable-711 | 2.33.376.831 I slot print_timing: id 0 | task 606 | n_decoded = 1459, tg = 44.38 t/s
qwen36-27b-fable-711 | 2.36.396.799 I slot print_timing: id 0 | task 606 | n_decoded = 1586, tg = 44.19 t/s
qwen36-27b-fable-711 | 2.39.435.485 I slot print_timing: id 0 | task 606 | n_decoded = 1735, tg = 44.56 t/s
qwen36-27b-fable-711 | 2.42.468.419 I slot print_timing: id 0 | task 606 | n_decoded = 1878, tg = 44.75 t/s
qwen36-27b-fable-711 | 2.45.508.585 I slot print_timing: id 0 | task 606 | n_decoded = 2017, tg = 44.82 t/s
qwen36-27b-fable-711 | 2.48.555.210 I slot print_timing: id 0 | task 606 | n_decoded = 2173, tg = 45.22 t/s
qwen36-27b-fable-711 | 2.51.586.615 I slot print_timing: id 0 | task 606 | n_decoded = 2288, tg = 44.79 t/s
qwen36-27b-fable-711 | 2.54.632.563 I slot print_timing: id 0 | task 606 | n_decoded = 2436, tg = 45.00 t/s
qwen36-27b-fable-711 | 2.57.672.315 I slot print_timing: id 0 | task 606 | n_decoded = 2566, tg = 44.88 t/s
qwen36-27b-fable-711 | 2.59.945.064 I slot print_timing: id 0 | task 606 | prompt eval time = 3827.23 ms / 3361 tokens ( 1.14 ms per token, 878.18 tokens per second)
qwen36-27b-fable-711 | 2.59.945.068 I slot print_timing: id 0 | task 606 | eval time = 59441.71 ms / 2659 tokens ( 22.35 ms per token, 44.73 tokens per second)
qwen36-27b-fable-711 | 2.59.945.068 I slot print_timing: id 0 | task 606 | total time = 63268.95 ms / 6020 tokens
Llama + gfx12xx == bad times. I have sister cards and have done a ton of testing with them. Testing this quant now to verify speeds.
Are you running via ROCM or Vulkan, windows or linux?
I'm running on Windows + Vulkan. I plan to install a separate Linux distribution to test if things are indeed faster (I believe they might be so because AMD does not really see Windows as "compute" platform and I do feel that their focus is mostly Gaming).
But that said the problem here is not llamacpp + gfx12xx overall but a low generation speed vs base model or other fine tunes.
For example, I do have around 740 pts and 30-50 (depending on acceptance) gts with clean prompt with other Q4 versions while this one does 3-10 gts due to cpu offloading.
Your pp numbers of 878 are certainly intriguing.
New quants from mradermacher. I'm sharing my testllama-perplexity wikitext-2-raw (8K context)
| GGUF | Quant | Perplexity (PPL) |
|---|---|---|
| DAU Fable-Fus-711-GGUF | IQ4_XS-LOW | 5.5779 ± 0.03366 |
| DAU Fable-Fus-711-GGUF | IQ4_XS-HIGH | 5.5681 ± 0.03359 |
| mradermacher Fable-Fus-711 GGUF | i1.IQ4_XS | 5.5623 ± 0.03355 |
| mradermacher Fable-Fus-711 GGUF | i1.Q4_K_S | 5.5583 ± 0.03346 |
whats the latest on this model? it shows updated, but is that just the readme?
New quants from mradermacher. I'm sharing my test
But is it even the same model? No NEO-MAX in the name.
New quants from mradermacher. I'm sharing my test
But is it even the same model? No NEO-MAX in the name.
Same model, any is fine, you choose one that suits your hardware and feels suitable for your work, from Q5 and above I think maybe the quality is same.