---
license: other
license_name: nvidia-open-model-license
base_model: nvidia/Llama-3.1-Nemotron-Nano-8B-v1
tags:
- qualcomm
- hexagon
- npu
- qnn
- qhexrt
- v81
- llama
---

# Llama-3.1-Nemotron-Nano-8B-v1 — Hexagon NPU (v81) bundle

A **QHexRT**-runnable, **arch-pinned** QNN context-binary bundle of
[`nvidia/Llama-3.1-Nemotron-Nano-8B-v1`](https://huggingface.co/nvidia/Llama-3.1-Nemotron-Nano-8B-v1)
for the **Qualcomm Hexagon v81** NPU (SM8850 / Snapdragon 8 Elite Gen 5, `soc_model 87`).

**The first 8B dense LLM on QHexRT.** An 8B W8 decode (~7 GB) exceeds the v81 cDSP per-context serialize
ceiling, so the 32 decoder layers are split into **8 chained context "parts"** (`decode_parts=8`,
`llama_generate_sharded` host-op): part 0 takes the embedding, the hidden state threads part→part, and the
last part applies the final norm before the lm-head. llama3-scaled RoPE (θ=500000, factor 8), 32 q-heads →
materialized MHA at **MAXCTX 512**.

## Validation (v81, SM8850)
- **Export gate:** decode 8/8 match the HF greedy gold; the 8-part split equals the monolithic decode
  bit-for-bit (`max_err = 0`).
- **On-device:** **W8 greedy-EXACT vs HF gold**, ~3.7 tok/s. `"The capital of France is"` →
  `" Paris. The capital of Germany is Berlin. The capital of Italy is Rome. The capital of Spain is Madrid…"`
  (token-for-token identical to Hugging Face).

## Contents (`v81/`)
- `nemotron-nano-8b.json` — the QHexRT manifest (plan + params).
- `nemotronnano8b_decode_p0..p7_w8.bin` — the 8 W8 decode parts.
- `nemotronnano8b_lmhead_w8.bin` — the W8 lm-head.
- `nemotronnano8b_embed_f16.bin` — the fp16 embedding table (host lookup).
- `tokenizer.json`.

## Run (QHexRT)
```bash
adb push v81 /data/local/tmp/wq/nemotron
adb shell "cd /data/local/tmp/wq && export ADSP_LIBRARY_PATH='/data/local/tmp/wq;/vendor/dsp/cdsp'; \
  LD_LIBRARY_PATH=. ./qhx_generate nemotron/nemotron-nano-8b.json libQnnHtp.so libQnnSystem.so nemotron \
  64 'The capital of France is'"
```
Built with [QHexRT](https://github.com/RunanywhereAI) `forge` — recipe `forge/recipes/nemotron-nano-8b/`
(spec-driven; `convert.sh v81` re-builds; the dense decode-split is `forge` + the `llama_generate_sharded`
runtime host-op). Arch-pinned: a v81 binary will not load on another Hexagon arch.