Wilsontj44/UMoX-Qwen3.6-35B-A3B-X2

UMoX for Qwen3.6-35B-A3B on Snapdragon X2

Historical snapshot: The current canonical UMoX project has moved to Wilsontj44/UMoX. This repository preserves the earlier Qwen3.6 layer/output-head research and does not contain the later admitted utility-concurrency architecture or benchmark.

Friendly warning: UMoX is a research prototype. The first end-to-end result is promising, but this is not a finished inference platform. Please try it, measure it carefully, tell us what breaks, and help us make it better.

UMoX (Unified Mixture of Executors) explores a simple idea: keep one model resident and let the CPU, GPU, and NPU behave like hardware experts. Each accelerator receives only work that fits its strengths and passes explicit latency, precision, synchronization, and stability gates. UMoX does not ask all three processors to recompute the model, and it does not change the model's learned MoE routing.

This repository packages the design, current measurements, contribution rules, and reproducibility tools for the first Qwen3.6 production-facing slice on a Snapdragon X2 Elite Extreme Windows laptop. It contains no model weights, Qualcomm SDK files, compiled QNN contexts, or proprietary binaries.

Why UMoX exists

Local LLM runtimes commonly choose one primary accelerator. That is sensible, but it can leave useful compute idle when a model contains subgraphs with very different shapes and numerical needs. Unified memory makes heterogeneous placement interesting because the model can remain resident while the runtime assigns selected subgraphs to the most suitable engine.

Unified memory is not magic: engines still need compatible layouts, explicit fences, coherent buffers, and enough work to repay dispatch overhead. UMoX therefore makes placement an evidence-based scheduling decision. A route is accepted only when it is faster than a matched control and preserves the required output quality.

Current implementation

The accepted Qwen3.6 path keeps the transformer and KV cache on the Adreno GPU. It moves the regular, stateless, full-vocabulary output projection to the Hexagon v81 NPU:

Qwen transformer + KV cache (Adreno OpenCL)
                    |
            final hidden state [2048]
                    |
       full-vocabulary W8A16 projection (Hexagon NPU)
                    |
     if top-logit margin <= 0.15, refine candidate shards
                    |
        selected W16A16 projection shards (Hexagon NPU)
                    |
                  logits

The W8A16 pass covers all 248,320 output rows using two persistent sessions. Only uncertain near-ties trigger W16A16 refinement, using eight finer shards. This preserves exact top-1 selection on the captured calibration states while avoiding the cost of running the entire high-precision head every token.

This is a GPU+NPU production slice, not yet the complete CPU+GPU+NPU scheduler envisioned by UMoX. CPU orchestration, recovery, and fallback are present, while useful CPU model compute remains a future qualified route.

Measured result

Hardware: Snapdragon X2 Elite Extreme laptop, Adreno X2-90 GPU, Hexagon v81 NPU, Windows 11, AC power, ASUS Performance mode. Model: Qwen3.6-35B-A3B Q5_K_M. Decode tests used batch 1.

Path	Decode	Quality status
GPU control A	21.392 tok/s	reference
GPU control B	23.591 tok/s	reference
UMoX adaptive W8 + W16 refinement	24.257 tok/s	accepted
Static W16-lower / W8-upper split	23.810 tok/s	accepted component
Full W8 performance profile	25.580 tok/s	rejected: 32/33 top-1

The adaptive path is 2.82% faster than the faster trailing control and 7.85% faster than the two-control mean. Control drift was material, so the 2.82% result is the conservative claim. In a separate 192-token greedy test, the GPU control and UMoX output matched byte-for-byte. A server smoke test reached 24.392 tok/s through an OpenAI-compatible endpoint.

Component evidence for the accepted adaptive projection:

Metric	Result
Calibration states	33
Top-1 match	33/33
Mean top-5 overlap	1.000000
Mean top-10 overlap	0.987879
Cosine similarity	0.999956645
NRMSE	0.00933829
Mean KL divergence	0.000194641
Median component latency	6.8199 ms
Live NPU-head median	4.551 ms
Refined decode calls	59/193

These are early single-machine measurements, not universal performance claims. The 33-state numerical set and one deterministic generation are insufficient to establish broad task-level equivalence. See BENCHMARKS.md for the larger qualification plan and the machine-readable result.

What UMoX does—and does not—claim

• UMoX is a runtime architecture, not a fine-tune or a new set of weights.
• It aims to preserve the base model's capability while improving local execution. It cannot make the underlying model more intelligent.
• The current result demonstrates a quality-gated NPU output-head offload.
• It does not yet demonstrate a faster full NPU transformer, zero-copy mutable GPU/NPU tensors, broad benchmark parity, or generalization to other machines.
• “80 TOPS” is a peak arithmetic figure, not an expected token rate. Batch-1 decoding is also constrained by memory traffic, small serial operations, graph launches, synchronization, supported operators, and quantization.

Base-model capability context

The upstream Qwen model card reports the following scores. They describe the base model under Qwen's evaluation setup; we have not rerun or verified them under UMoX.

Upstream benchmark	Qwen-published score	UMoX rerun
SWE-bench Verified	73.4	Not run
SWE-bench Multilingual	67.2	Not run
SWE-bench Pro	49.5	Not run
Terminal-Bench 2.0	51.5	Not run
MCPMark	37.0	Not run
MCP-Atlas	62.8	Not run
MMLU-Pro	85.2	Not run
GPQA	86.0	Not run
LiveCodeBench v6	80.4	Not run
AIME 2026	92.7	Not run

Source: official Qwen3.6-35B-A3B model card. The evaluation scaffold, prompt format, sampling, context, and runtime version must be recorded before a UMoX score can be compared with any published value.

Recommended generation settings

Follow the base model's official defaults unless a benchmark specifies its own decoding protocol:

Mode	Temperature	Top-p	Top-k	Min-p	Presence penalty	Repetition penalty
Thinking, general	1.0	0.95	20	0	1.5	1.0
Thinking, precise/coding	0.6	0.95	20	0	0	1.0
Instruct/non-thinking	0.7	0.8	20	0	1.5	1.0
Deterministic parity tests	0	n/a	n/a	n/a	0	1.0

Temperature alone does not define “coding” versus “creative” quality. The official chat template, thinking mode, stopping rules, and task-specific evaluation protocol matter too.

Try it and help us polish it

We would genuinely value help from runtime engineers, NPU/compiler specialists, evaluation researchers, and curious local-AI users. A useful contribution can be as small as reproducing one benchmark, testing another Snapdragon device, documenting a failed graph, or reviewing the precision gate.

Please be patient with the prototype and kind to fellow experimenters. Negative results are welcome—they keep the project honest and often reveal the next good idea. Read CONTRIBUTING.md, open a Hugging Face Discussion for questions, or propose a focused change through a Pull Request.

Repository map

• ARCHITECTURE.md: design, dataflow, scheduler rules, and limitations.
• BENCHMARKS.md: performance and capability qualification plan.
• REPRODUCIBILITY.md: current environment and exact test procedure.
• CONTRIBUTING.md: contributor workflow and evidence requirements.
• docs/FABLE_HANDOFF_ROUND3.md: complete second-opinion handoff.
• scripts/benchmark_openai.py: lightweight endpoint benchmark.
• scripts/validate_release.py: release consistency checks.

License and acknowledgements

Original UMoX documentation and helper scripts in this package are MIT licensed. Qwen3.6 is Apache-2.0; the referenced GGUF follows its upstream repository's license. llama.cpp, ONNX Runtime, QAIRT/QNN, device drivers, and model weights retain their own licenses and are not relicensed or redistributed here. See NOTICE.md.

UMoX is independent community research and is not affiliated with or endorsed by Qualcomm, Qwen, Microsoft, Hugging Face, or the llama.cpp project.