README.md

---
license: apache-2.0
language:
- ar
- en
pipeline_tag: automatic-speech-recognition
tags:
- audio
- speech-recognition
- transcription
- onnx
- directml
base_model:
- CohereLabs/cohere-transcribe-arabic-07-2026
---

# Cohere Transcribe Arabic 07-2026 — DirectML-optimized ONNX

ONNX export of [CohereLabs/cohere-transcribe-arabic-07-2026](https://huggingface.co/CohereLabs/cohere-transcribe-arabic-07-2026)
(2B, Arabic + English) restructured to run **fast on the ONNX Runtime DirectML EP**. Same
architecture as [cohere-transcribe-03-2026](https://huggingface.co/Masterx/cohere-transcribe-03-2026-ONNX),
hand-exported from the reference PyTorch weights (stock `optimum` can't export `cohere_asr`).

## What changed (`tools/onnx/cohere_decompose_attention.py` in [WinSTT](https://github.com/dahshury/WinSTT))

1. **Loop-invariant cross-KV hoisted into the encoder** — cross-attention K/V (which depend only on
   the audio) are computed once per utterance in `encoder_model*.onnx`, not per decoded token.
2. **Static self-KV decode** — the self-attention KV cache is a fixed-length buffer updated by a
   masked write, so the DirectML EP compiles the decoder **once** per utterance instead of re-fusing
   its graph on every autoregressive step. Measured ~3.5 ms/token on DirectML (RTX 3080 Ti) vs
   ~16 ms/token on the CPU EP — **~4.6× faster**, entirely on the GPU.

Two decoders per precision (both fed by the hoisted encoder, sharing the decoder sidecar):
`decoder_model_merged*.onnx` (**static self-KV**, marker `winstt_static_kv`, fastest on DirectML) and
`decoder_model_merged*_dyn.onnx` (**growing self-KV**, faster on the CPU EP). WinSTT picks per device.

Numerically verified against the source on CPU (autoregressive logits parity: bit-exact fp32/q4).

**Notes vs the multilingual export:** this torch-traced graph uses a `(B, S, nh, hd)` head layout, so
the cross-attention is left dynamic (not padded to a fixed bucket) — perfectly fast for the ≤10 s
segments a dictation VAD produces; only unrealistically long single clips would benefit from
bucketing. The **int8** decoder ships **dynamic-only** (`decoder_model_merged_int8.onnx` is the
growing-self graph): its `DynamicQuantizeLinear` scales interact badly with the fixed-KV write, so it
runs the hybrid path (encoder-DML / decoder-CPU). fp32 and q4 get the full static decode.

## Files

- `onnx/encoder_model[_int8|_q4].onnx` (+ external data) — hoisted encoders (cross-KV outputs).
- `onnx/decoder_model_merged[_q4].onnx` (+ `_dyn`) — static / dynamic decoders (fp32, q4).
- `onnx/decoder_model_merged_int8.onnx` — dynamic-only (int8).
- Tokenizer / configs — unchanged.