---
language:
  - en
license: mit
library_name: llama.cpp
pipeline_tag: text-generation
base_model:
  - Qwen/Qwen3.6-35B-A3B
  - bartowski/Qwen_Qwen3.6-35B-A3B-GGUF
tags:
  - umox
  - qwen3.6
  - snapdragon-x2-elite
  - qnn
  - npu
  - heterogeneous-compute
  - coding
  - openai-compatible
  - research-preview
---

# UMoX for Qwen3.6-35B-A3B on Snapdragon X2

> **Historical snapshot:** The current canonical UMoX project has moved to
> [Wilsontj44/UMoX](https://huggingface.co/Wilsontj44/UMoX). This repository
> preserves the earlier Qwen3.6 layer/output-head research and does not contain
> the later admitted utility-concurrency architecture or benchmark.

> **Friendly warning:** UMoX is a research prototype. The first end-to-end
> result is promising, but this is not a finished inference platform. Please
> try it, measure it carefully, tell us what breaks, and help us make it better.

**UMoX (Unified Mixture of Executors)** explores a simple idea: keep one model
resident and let the CPU, GPU, and NPU behave like hardware experts. Each
accelerator receives only work that fits its strengths and passes explicit
latency, precision, synchronization, and stability gates. UMoX does not ask all
three processors to recompute the model, and it does not change the model's
learned MoE routing.

This repository packages the design, current measurements, contribution rules,
and reproducibility tools for the first Qwen3.6 production-facing slice on a
Snapdragon X2 Elite Extreme Windows laptop. It contains **no model weights,
Qualcomm SDK files, compiled QNN contexts, or proprietary binaries**.

## Why UMoX exists

Local LLM runtimes commonly choose one primary accelerator. That is sensible,
but it can leave useful compute idle when a model contains subgraphs with very
different shapes and numerical needs. Unified memory makes heterogeneous
placement interesting because the model can remain resident while the runtime
assigns selected subgraphs to the most suitable engine.

Unified memory is not magic: engines still need compatible layouts, explicit
fences, coherent buffers, and enough work to repay dispatch overhead. UMoX
therefore makes placement an evidence-based scheduling decision. A route is
accepted only when it is faster than a matched control and preserves the
required output quality.

## Current implementation

The accepted Qwen3.6 path keeps the transformer and KV cache on the Adreno GPU.
It moves the regular, stateless, full-vocabulary output projection to the
Hexagon v81 NPU:

```text
Qwen transformer + KV cache (Adreno OpenCL)
                    |
            final hidden state [2048]
                    |
       full-vocabulary W8A16 projection (Hexagon NPU)
                    |
     if top-logit margin <= 0.15, refine candidate shards
                    |
        selected W16A16 projection shards (Hexagon NPU)
                    |
                  logits
```

The W8A16 pass covers all 248,320 output rows using two persistent sessions.
Only uncertain near-ties trigger W16A16 refinement, using eight finer shards.
This preserves exact top-1 selection on the captured calibration states while
avoiding the cost of running the entire high-precision head every token.

This is a **GPU+NPU production slice**, not yet the complete CPU+GPU+NPU
scheduler envisioned by UMoX. CPU orchestration, recovery, and fallback are
present, while useful CPU model compute remains a future qualified route.

## Measured result

Hardware: Snapdragon X2 Elite Extreme laptop, Adreno X2-90 GPU, Hexagon v81
NPU, Windows 11, AC power, ASUS Performance mode. Model:
Qwen3.6-35B-A3B `Q5_K_M`. Decode tests used batch 1.

| Path | Decode | Quality status |
|---|---:|---|
| GPU control A | 21.392 tok/s | reference |
| GPU control B | 23.591 tok/s | reference |
| UMoX adaptive W8 + W16 refinement | **24.257 tok/s** | accepted |
| Static W16-lower / W8-upper split | 23.810 tok/s | accepted component |
| Full W8 performance profile | 25.580 tok/s | **rejected**: 32/33 top-1 |

The adaptive path is **2.82% faster than the faster trailing control** and
7.85% faster than the two-control mean. Control drift was material, so the
2.82% result is the conservative claim. In a separate 192-token greedy test,
the GPU control and UMoX output matched byte-for-byte. A server smoke test
reached 24.392 tok/s through an OpenAI-compatible endpoint.

Component evidence for the accepted adaptive projection:

| Metric | Result |
|---|---:|
| Calibration states | 33 |
| Top-1 match | 33/33 |
| Mean top-5 overlap | 1.000000 |
| Mean top-10 overlap | 0.987879 |
| Cosine similarity | 0.999956645 |
| NRMSE | 0.00933829 |
| Mean KL divergence | 0.000194641 |
| Median component latency | 6.8199 ms |
| Live NPU-head median | 4.551 ms |
| Refined decode calls | 59/193 |

These are early single-machine measurements, not universal performance claims.
The 33-state numerical set and one deterministic generation are insufficient
to establish broad task-level equivalence. See [BENCHMARKS.md](BENCHMARKS.md)
for the larger qualification plan and
[the machine-readable result](benchmarks/results/umox-qwen36-x2-2026-07-27.json).

## What UMoX does—and does not—claim

- UMoX is a runtime architecture, not a fine-tune or a new set of weights.
- It aims to preserve the base model's capability while improving local
  execution. It cannot make the underlying model more intelligent.
- The current result demonstrates a quality-gated NPU output-head offload.
- It does not yet demonstrate a faster full NPU transformer, zero-copy mutable
  GPU/NPU tensors, broad benchmark parity, or generalization to other machines.
- “80 TOPS” is a peak arithmetic figure, not an expected token rate. Batch-1
  decoding is also constrained by memory traffic, small serial operations,
  graph launches, synchronization, supported operators, and quantization.

## Base-model capability context

The upstream Qwen model card reports the following scores. They describe the
base model under Qwen's evaluation setup; **we have not rerun or verified them
under UMoX**.

| Upstream benchmark | Qwen-published score | UMoX rerun |
|---|---:|---|
| SWE-bench Verified | 73.4 | Not run |
| SWE-bench Multilingual | 67.2 | Not run |
| SWE-bench Pro | 49.5 | Not run |
| Terminal-Bench 2.0 | 51.5 | Not run |
| MCPMark | 37.0 | Not run |
| MCP-Atlas | 62.8 | Not run |
| MMLU-Pro | 85.2 | Not run |
| GPQA | 86.0 | Not run |
| LiveCodeBench v6 | 80.4 | Not run |
| AIME 2026 | 92.7 | Not run |

Source: [official Qwen3.6-35B-A3B model card](https://huggingface.co/Qwen/Qwen3.6-35B-A3B).
The evaluation scaffold, prompt format, sampling, context, and runtime version
must be recorded before a UMoX score can be compared with any published value.

## Recommended generation settings

Follow the base model's official defaults unless a benchmark specifies its own
decoding protocol:

| Mode | Temperature | Top-p | Top-k | Min-p | Presence penalty | Repetition penalty |
|---|---:|---:|---:|---:|---:|---:|
| Thinking, general | 1.0 | 0.95 | 20 | 0 | 1.5 | 1.0 |
| Thinking, precise/coding | 0.6 | 0.95 | 20 | 0 | 0 | 1.0 |
| Instruct/non-thinking | 0.7 | 0.8 | 20 | 0 | 1.5 | 1.0 |
| Deterministic parity tests | 0 | n/a | n/a | n/a | 0 | 1.0 |

Temperature alone does not define “coding” versus “creative” quality. The
official chat template, thinking mode, stopping rules, and task-specific
evaluation protocol matter too.

## Try it and help us polish it

We would genuinely value help from runtime engineers, NPU/compiler specialists,
evaluation researchers, and curious local-AI users. A useful contribution can
be as small as reproducing one benchmark, testing another Snapdragon device,
documenting a failed graph, or reviewing the precision gate.

Please be patient with the prototype and kind to fellow experimenters. Negative
results are welcome—they keep the project honest and often reveal the next good
idea. Read [CONTRIBUTING.md](CONTRIBUTING.md), open a Hugging Face Discussion
for questions, or propose a focused change through a Pull Request.

## Repository map

- [ARCHITECTURE.md](ARCHITECTURE.md): design, dataflow, scheduler rules, and limitations.
- [BENCHMARKS.md](BENCHMARKS.md): performance and capability qualification plan.
- [REPRODUCIBILITY.md](REPRODUCIBILITY.md): current environment and exact test procedure.
- [CONTRIBUTING.md](CONTRIBUTING.md): contributor workflow and evidence requirements.
- [docs/FABLE_HANDOFF_ROUND3.md](docs/FABLE_HANDOFF_ROUND3.md): complete second-opinion handoff.
- [scripts/benchmark_openai.py](scripts/benchmark_openai.py): lightweight endpoint benchmark.
- [scripts/validate_release.py](scripts/validate_release.py): release consistency checks.

## License and acknowledgements

Original UMoX documentation and helper scripts in this package are MIT licensed.
Qwen3.6 is Apache-2.0; the referenced GGUF follows its upstream repository's
license. llama.cpp, ONNX Runtime, QAIRT/QNN, device drivers, and model weights
retain their own licenses and are not relicensed or redistributed here. See
[NOTICE.md](NOTICE.md).

UMoX is independent community research and is not affiliated with or endorsed
by Qualcomm, Qwen, Microsoft, Hugging Face, or the llama.cpp project.