---
license: apache-2.0
base_model:
- Vortex5/Shadow-Siren-26B-A4B
library_name: llama.cpp
pipeline_tag: text-generation
tags:
- gguf
- quantized
- llama.cpp
- roleplay
- apex
- custom apex
quantized_by: Koshkasa
base_model_relation: quantized
---

## What's that?  

As denoted [by its creators](https://github.com/localai-org/apex-quant), APEX is "a novel MoE-aware mixed-precision quantization technique for llama.cpp".  I suggest reading their information on the how and why of the method.  
    
These two GGUFs are APEX quantizations of [Vortex5/Shadow-Siren-26B-A4B](https://huggingface.co/Vortex5/Shadow-Siren-26B-A4B), using EmanuelOverride's custom APEX recipe, [HarmonicMinus](https://huggingface.co/EmanuelOverride/gemma-4-26B-A4B-it-heretic-APEX-GGUF/blob/main/configs/HarmonicMinus.txt), and a minor customization of that edit to cap shared ffn at Q6_K and replace Q4_K and Q4_0 (which was a fallback for ffn_down_exps from Q3_K) sparse expert quants with MXFP4. Both recipes use **mradermacher**'s imatrix found [here](https://huggingface.co/mradermacher/Shadow-Siren-26B-A4B-i1-GGUF)

To quote EmanuelOverride regarding the philosophy behind HarmonicMinus: 
  
--------------------------------
### 🎵 The Harmonic Series (Pulsed Attention)
Following on from the *Harmonic Resonance* branch, the Harmonic layouts further optimize layer contiguity for performance, by following strict precision tiers across all 30 layers (High/Medium/Low), while priorizing high-precision tiers for **Global Attention (GA)** layers, preventing aggressive compression in order to anchor deep-context tracking.

* **[i-Harmonic.Minus](https://huggingface.co/EmanuelOverride/gemma-4-26B-A4B-it-heretic-APEX-GGUF/resolve/main/gemma-4-26B-A4B-it-heretic.APEX.i.Harmonic.Minus.gguf):** `14,369.74 MiB` (4.78 BPW) — *High-performance configuration.* Condenses the architecture into 3 strict tiers (`5/15/10`). Shifts high-precision blocks to coincide with global attention pulses.   
--------------------------------
My modification, aptly named **HarmonicMinusMinus**, retains most of the original recipe and adheres to the same philosophy, sacrificing some precision on sparse experts FFN layers for extra VRAM savings, while retaining global attention-focused attn distribution. A secondary effect is minor speedup on GPUs with native fp4 support. The HarmonicMinusMinus modification APEX recipe is provided.

## Benchmarks  
    
|Measurement|HarmonicMinus|HarmonicMinusMinus|Delta|
|---|---|---|---|
|pp512|2764.56|3233.05|+16.97%|
|tg128|106.81|112.63|+5.45%|
|pp32768|2378.52|2543.86|+6.95%|
|tg512|102.54|103.86|+1.29%|
|pp32768 + tg512 wall time (s)|18.76|17.81|-5.06%|
|hellaswag% (s=42, n=400, greedy)|52.25%|52.25%|==|
|hellaswag (s=42, n=400, greedy)|209|209|==|
|winogrande% (n=1267, greedy)|55.09%|57.06%|+1.97% (abs)|
|winogrande (n=1267, greedy)|698|723|+25|
|Size (bytes)|15083595360|14157258080|-6.14%|
|Bits per weight|4.78|4.48|-0.3|

pp32768/tg512 tests were run at q4_0/q4_0 cache quantization to ensure both models fully fit in VRAM with KV cache on my GPU.  

## Conclusion
WYSIWYG. MXFP4 is lower precision than Q4_K and can underrepresent outliers. For certain ffn_down_exps it may have had an impact that had not been identified during my surface testing. Feedback welcome.  

## Disclosure  
My only contribution is compute and a little tinkering with an already existing custom APEX recipe. This is neither my merge, nor my quantization methodology. Have fun.  
  
## Cheers  
**[Google](https://huggingface.co/google)** - the base model.  
**[Vortex5](https://huggingface.co/Vortex5)** - for the merge effort.  
**Everyone whose finetunes were included in the merge!**  
**[mudler](https://github.com/localai-org/apex-quant)** - for the APEX quantization methodology.  
**[EmanuelOverride](https://huggingface.co/EmanuelOverride)** - for the custom APEX experiments and the base recipe.  
**[mradermacher](https://huggingface.co/mradermacher)** - for the imatrix + myriads of quants we all benefit from.