Sanitize public model card metadata
Browse files
README.md
CHANGED
|
@@ -1,80 +1,18 @@
|
|
| 1 |
---
|
| 2 |
-
license: mit
|
| 3 |
tags:
|
| 4 |
-
-
|
| 5 |
-
- hexagon
|
| 6 |
-
- npu
|
| 7 |
-
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
pipeline_tag: automatic-speech-recognition
|
| 11 |
---
|
| 12 |
|
| 13 |
-
#
|
| 14 |
|
| 15 |
-
Prebuilt
|
| 16 |
-
speech-to-text **on-device on the Qualcomm Hexagon NPU** via the [QHexRT](https://github.com/RunanywhereAI/QHexRT)
|
| 17 |
-
runtime. **Arch: `v81`** (SM8850 / Snapdragon 8 Elite Gen 5). Device-validated at **WER = 0** vs HF.
|
| 18 |
|
| 19 |
-
|
| 20 |
-
on another Hexagon arch. Other arches are sibling `<arch>/` dirs (re-converted), added to this same repo.
|
| 21 |
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
Moonshine is an encoder–decoder ASR model on the **raw 16 kHz waveform** (no mel spectrogram). The pipeline:
|
| 25 |
-
- **Host** runs the 3-conv raw-audio stem (`conv1 k127/s64 + tanh → GroupNorm → conv2 k7/s3 + gelu →
|
| 26 |
-
conv3 k3/s2 + gelu`) — the conv is HTP-hostile so it stays on the CPU; the encoder graph starts at the
|
| 27 |
-
post-conv features.
|
| 28 |
-
- **NPU encoder** (bidirectional, partial interleaved RoPE) → per-decoder-layer cross-attention K/V.
|
| 29 |
-
- **NPU decoder** (one autoregressive step: causal self-attn + RoPE in-graph, cross-attn to the cached
|
| 30 |
-
encoder states, gated SwiGLU; tied lm-head) → tokens, detokenized on the host.
|
| 31 |
-
|
| 32 |
-
Variable-length audio is handled on a fixed graph (`n_audio = 415`, ~10 s window): the host pads/truncates
|
| 33 |
-
the features and masks the padding (`encoder_mask` / `cross_mask`). Precision **fp16** (encoder + decoder).
|
| 34 |
-
|
| 35 |
-
## Files (`v81/`)
|
| 36 |
-
|
| 37 |
-
| file | role | size |
|
| 38 |
-
|---|---|---|
|
| 39 |
-
| `moonshine-tiny.json` | QHexRT manifest (declarative run plan; `moonshine_transcribe` host-op) | ~1 KB |
|
| 40 |
-
| `moonshinetiny_enc_f16.bin` | encoder context binary | 17.5 MB |
|
| 41 |
-
| `moonshinetiny_dec_f16.bin` | decoder context binary | 56.5 MB |
|
| 42 |
-
| `moonshine_conv_stem.bin` | host raw-audio conv-stem weights `[c1w,c2w,c2b,c3w,c3b,gnw,gnb]` | 6.8 MB |
|
| 43 |
-
| `tokenizer.json` | SentencePiece-style BPE (byte_fallback; metaspace detok) | 3.8 MB |
|
| 44 |
-
|
| 45 |
-
The QNN runtime libs (`libQnnHtp.so` / `libQnnSystem.so` + the v81 HTP skel) come from the **QAIRT SDK**, not
|
| 46 |
-
this repo. The `qhx_asr` tool comes from a QHexRT build.
|
| 47 |
-
|
| 48 |
-
## Run
|
| 49 |
-
|
| 50 |
-
```bash
|
| 51 |
-
hf download runanywhere/moonshine_tiny_HNPU --local-dir moonshine_tiny_HNPU
|
| 52 |
-
# Windows: adb push from PowerShell with native paths.
|
| 53 |
-
adb push moonshine_tiny_HNPU/v81 /data/local/tmp/wq/moonshine
|
| 54 |
-
adb push my_audio_16k_mono.wav /data/local/tmp/wq/moonshine/
|
| 55 |
-
adb shell "cd /data/local/tmp/wq && export ADSP_LIBRARY_PATH='/data/local/tmp/wq/dsp;/data/local/tmp/wq;/vendor/dsp/cdsp'; \
|
| 56 |
-
LD_LIBRARY_PATH=. ./qhx_asr moonshine/moonshine-tiny.json libQnnHtp.so libQnnSystem.so moonshine moonshine/my_audio_16k_mono.wav"
|
| 57 |
-
```
|
| 58 |
-
|
| 59 |
-
Tool arg order is invariant: `qhx_asr <manifest> libQnnHtp.so libQnnSystem.so <artifacts_root> <audio16k.wav>`.
|
| 60 |
-
Input audio must be **16 kHz mono** (PCM16 or float32).
|
| 61 |
-
|
| 62 |
-
## Measured (v81, SM8850, soc_model 87, QAIRT 2.47)
|
| 63 |
-
|
| 64 |
-
| metric | value |
|
| 65 |
-
|---|---|
|
| 66 |
-
| Parity | **WER = 0.0000** vs HF `UsefulSensors/moonshine-tiny` (LibriSpeech sample) |
|
| 67 |
-
| Latency | 26 tokens in **551 ms** for a 5.9 s clip |
|
| 68 |
-
| Precision | fp16 encoder + decoder |
|
| 69 |
-
|
| 70 |
-
Example: a 5.9 s clip → *"Mr. Quilter is the apostle of the middle classes, and we are glad to welcome his
|
| 71 |
-
gospel."* (matches HF exactly).
|
| 72 |
-
|
| 73 |
-
## Caveats
|
| 74 |
-
|
| 75 |
-
- **v81 only** here (arch-pinned). **fp16** weights. Audio window ~10 s (`n_audio = 415`); longer clips are
|
| 76 |
-
truncated to the window.
|
| 77 |
-
- Parity is greedy (temperature 0) vs the HF reference. WER measured on a standard LibriSpeech sample.
|
| 78 |
-
|
| 79 |
-
Converted + device-validated with the [QHexRT `forge`](https://github.com/RunanywhereAI/QHexRT) pipeline
|
| 80 |
-
(`recipes/moonshine-tiny`).
|
|
|
|
| 1 |
---
|
| 2 |
+
license: "mit"
|
| 3 |
tags:
|
| 4 |
+
- "hnpu"
|
| 5 |
+
- "hexagon"
|
| 6 |
+
- "npu"
|
| 7 |
+
- "asr"
|
| 8 |
+
base_model: "UsefulSensors/moonshine-tiny"
|
| 9 |
+
pipeline_tag: "automatic-speech-recognition"
|
|
|
|
| 10 |
---
|
| 11 |
|
| 12 |
+
# moonshine tiny HNPU
|
| 13 |
|
| 14 |
+
Prebuilt HNPU artifacts for [UsefulSensors/moonshine-tiny](https://huggingface.co/UsefulSensors/moonshine-tiny), a public automatic speech recognition model.
|
|
|
|
|
|
|
| 15 |
|
| 16 |
+
For model behavior, license, intended use, and limitations, see the [upstream model card](https://huggingface.co/UsefulSensors/moonshine-tiny).
|
|
|
|
| 17 |
|
| 18 |
+
Artifacts are architecture-pinned. Available artifact directories: `v75/`, `v79/`, `v81/`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|