File size: 1,695 Bytes
37c4768 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 | # DiariZen Speaker Segmentation - AX650 Deployment
CPU+NPU hybrid speaker diarization segmentation for AX650 NPU.
## Model
- Source: [BUT-FIT/diarizen-wavlm-large-s80-md](https://huggingface.co/BUT-FIT/diarizen-wavlm-large-s80-md)
- Architecture: WavLM-Large (pruned) + Conformer
- Task: Frame-level speaker activity segmentation (11 classes, 4s @ 16kHz)
## Pipeline
```
Audio (16kHz mono, any length)
→ CPU: resample → 4s sliding window → LayerNorm
→ AX650 NPU: CNN feature extractor (7 conv, U16, 17.7ms)
→ CPU: WavLM Transformer (24L) + Conformer (4L) + Classifier (251ms)
→ Log-probabilities (1, 199, 11) per window
```
## Performance
| Stage | Time | Hardware |
|-------|------|----------|
| CNN | 17.7 ms | AX650 NPU @1GHz |
| Backend | 251 ms | CPU ONNX Runtime |
| Total | 269 ms | 14.9x real-time |
## Accuracy
End-to-end cosine: 0.9997 vs full FP32 reference.
## Directory
```
models/ cnn_features.axmodel + backend.onnx
python/ Python SDK (diarizen_sdk)
cpp/ C++ SDK (diarizen_segmenter)
model_convert/ ONNX export + Pulsar2 compile config
reports/ SDK and simulation reports
```
## Quick Start
```bash
pip install -r python/requirements.txt
python python/diarizen_sdk/example.py audio.wav \
--cnn-model models/cnn_features.axmodel \
--backend-model models/backend.onnx
```
## Known Limitations
- 4s fixed window; longer audio requires sliding window stitching.
- CNN requires U16 quantization (U8 insufficient accuracy).
- Transformer + Conformer run on CPU (NPU backend limitation with WavLM attention ops).
- pos_conv_embed Conv (kernel=128, NLC layout) incompatible with NPU; included in CPU backend.
|