# Reproducibility guide

## Current scope

This package is a pre-publication research release. It provides the public
contract, sanitized evidence, benchmark tooling, and contribution process. The
prototype llama.cpp integration exists and is qualified locally, but its source
patch and QNN graph-production pipeline still need a clean-room packaging pass
before they can be uploaded without local assumptions or licensed SDK content.

Do not interpret this document as a one-command install. Reproduction currently
requires a compatible Snapdragon X2 Windows machine, supported Qualcomm runtime,
the base GGUF, locally generated QNN graphs, and the UMoX-enabled runtime build.

## Qualified environment

| Item | Qualified value |
|---|---|
| OS | Windows 11 on ARM64 |
| SoC class | Snapdragon X2 Elite Extreme |
| GPU | Adreno X2-90 through the qualified OpenCL runtime |
| NPU | Hexagon v81 through QNN Execution Provider |
| ONNX Runtime | 1.24.4 |
| Model | Qwen3.6-35B-A3B Q5_K_M GGUF |
| Model SHA-256 | `EC438D1326EB11D8BB94785C54739CD4B60B01AE32BD8532E232A046B19D296D` |
| UMoX libllama SHA-256 | `D15D0734E9D1718E6D3D85F02B06DF7E39CA72B0EF8284ED591F9F0611FFBDF9` |
| Power | AC connected, battery 95–97%, ASUS Performance mode |
| Workload | batch-1 decode |

Drivers and QAIRT/QNN SDK components are not redistributed. Record their exact
versions from your own machine and obey their licenses.

## Runtime structure

The adaptive runtime expects two manifest tables:

1. two ordered W8A16 primary shards that cover the complete vocabulary; and
2. eight ordered W16A16 refinement shards covering the same rows at finer
   granularity.

Each table is tab-separated and declares shard order, starting row, row count,
and model path. Paths are deliberately excluded from the public result because
they are machine-specific and the compiled contexts may be runtime/device bound.

At startup, UMoX verifies full contiguous coverage and loads ten persistent NPU
sessions with CPU fallback disabled. The model server should remain resident;
the measured cold path took about 42.6 seconds for QNN sessions and roughly
2 minutes 15 seconds to endpoint readiness.

## Environment contract

The local prototype uses the following configuration keys:

```text
UMOX_MODE=lm_head
UMOX_LM_HEAD_TABLE=<absolute path to W8 primary table>
UMOX_LM_HEAD_REFINE_TABLE=<absolute path to W16 refinement table>
UMOX_LM_HEAD_REFINE_MARGIN=0.15
UMOX_ORT_DLL=<absolute path to ONNX Runtime 1.24.4>
UMOX_QNN_BACKEND_DLL=<QNN HTP backend library>
UMOX_LM_HEAD_LOG=<optional per-call timing/refinement CSV>
```

Names may change before a stable release. Never put credentials in a runtime
table, log, benchmark prompt, or Hugging Face commit.

## Reproduce the measurement

1. Reboot or quiesce competing AI workloads.
2. Connect AC power and select the same Windows and OEM performance profiles.
3. Record model, runtime, graph, driver, and manifest hashes.
4. Start the server and wait for its health endpoint before measuring.
5. Warm the model and all QNN sessions.
6. Run a GPU-control / UMoX / GPU-control sequence with identical prompt,
   context, seed, thread count, GPU placement, and generated-token count.
7. Repeat across fresh processes and retain individual samples.
8. Run the numerical corpus and deterministic behavioral parity test.
9. Inspect provider logs for graph spill or CPU fallback.
10. Publish raw sanitized evidence with the median and p95, not only the best run.

For an OpenAI-compatible server, the included helper can collect endpoint
latency and token usage:

```powershell
python .\scripts\benchmark_openai.py `
  --base-url http://127.0.0.1:8080/v1 `
  --model qwen3.6-35b-a3b-umox `
  --iterations 5 `
  --max-tokens 256 `
  --output .\benchmarks\results\local-endpoint.json
```

The helper does not grade reasoning or coding. Use the official harnesses and
paired baseline/UMoX protocol in [BENCHMARKS.md](BENCHMARKS.md).

## Artifact rules

- Do not upload model weights unless the repository and model licenses permit it.
- Do not upload Qualcomm SDKs, driver binaries, device-bound compiled contexts,
  private API configuration, or Hugging Face tokens.
- Replace local absolute paths and usernames with logical artifact identifiers.
- Hash every binary and graph that materially affects a reported result.
- Keep rejected experiments. A fast result that misses a quality gate is useful
  evidence, but it must remain labeled rejected.

## Known reproducibility gaps

- Only one Snapdragon X2 laptop has run the final adaptive profile.
- Public source integration and graph-build recipes are not packaged yet.
- Driver and SDK hashes still need a portable collection script.
- The calibration corpus cannot yet be published as a stable versioned dataset.
- Energy and thermal telemetry were not captured in the final A/B sequence.
- Capability suites listed in BENCHMARKS.md remain unrun under UMoX.

These gaps are the first public roadmap items, not details to hide.