Add files using upload-large-folder tool

README.md

@@ -1,45 +1,51 @@
----
-license: apache-2.0
-language:
-- ar
-- en
-pipeline_tag: automatic-speech-recognition
-tags:
-- audio
-- speech-recognition
-- transcription
-- onnx
-base_model:
-- CohereLabs/cohere-transcribe-arabic-07-2026
-library_name: onnx
----
-
-# Cohere Transcribe Arabic (07-2026) — ONNX
-
-ONNX export of [`CohereLabs/cohere-transcribe-arabic-07-2026`](https://huggingface.co/CohereLabs/cohere-transcribe-arabic-07-2026),
-a 2B-parameter Arabic + English dedicated ASR model (Conformer encoder + lightweight Transformer decoder).
-
-Exported to the same merged encoder / KV-cache decoder graph layout as
-[`onnx-community/cohere-transcribe-03-2026-ONNX`](https://huggingface.co/onnx-community/cohere-transcribe-03-2026-ONNX)
-for use with [WinSTT](https://github.com/dahshury/WinSTT).
-
-## Graphs (`onnx/`)
-
-| File | Inputs | Outputs |
-|------|--------|---------|
-| `encoder_model*.onnx` | `input_features [B,T,128]` | `last_hidden_state [B,S,1024]` |
-| `decoder_model_merged*.onnx` | `input_ids`, `attention_mask`, `position_ids`, `num_logits_to_keep`, `encoder_hidden_states`, `past_key_values.{0..7}.{decoder,encoder}.{key,value}` | `logits`, `present.{0..7}.{decoder,encoder}.{key,value}` |
-
-## Precisions
-
-- `` (default) — fp32. Best accuracy; also the fastest on the CPU EP this model runs on.
-- `q4` — 4-bit (`MatMulNBitsQuantizer`, block_size=32, symmetric), ~2.2 GB.
-
-fp16 / q4f16 are not shipped: the model is CPU-only (its attention kernel is not DirectML-compatible)
-and the ONNX Runtime CPU EP up-casts fp16→fp32, so they'd be slower and larger with no benefit.
-
-Verified numerically against the original PyTorch model (encoder/decoder parity ~1e-5 in fp32) and by
-end-to-end greedy decoding. See `export_merged.py` / `quantize.py` to reproduce.
-
-
-License: Apache-2.0 (inherited from the base model).
+---
+license: apache-2.0
+language:
+- ar
+- en
+pipeline_tag: automatic-speech-recognition
+tags:
+- audio
+- speech-recognition
+- transcription
+- onnx
+- directml
+base_model:
+- CohereLabs/cohere-transcribe-arabic-07-2026
+---
+
+# Cohere Transcribe Arabic 07-2026 — DirectML-optimized ONNX
+
+ONNX export of [CohereLabs/cohere-transcribe-arabic-07-2026](https://huggingface.co/CohereLabs/cohere-transcribe-arabic-07-2026)
+(2B, Arabic + English) restructured to run **fast on the ONNX Runtime DirectML EP**. Same
+architecture as [cohere-transcribe-03-2026](https://huggingface.co/Masterx/cohere-transcribe-03-2026-ONNX),
+hand-exported from the reference PyTorch weights (stock `optimum` can't export `cohere_asr`).
+
+## What changed (`tools/onnx/cohere_decompose_attention.py` in [WinSTT](https://github.com/dahshury/WinSTT))
+
+1. **Loop-invariant cross-KV hoisted into the encoder** — cross-attention K/V (which depend only on
+   the audio) are computed once per utterance in `encoder_model*.onnx`, not per decoded token.
+2. **Static self-KV decode** — the self-attention KV cache is a fixed-length buffer updated by a
+   masked write, so the DirectML EP compiles the decoder **once** per utterance instead of re-fusing
+   its graph on every autoregressive step. Measured ~3.5 ms/token on DirectML (RTX 3080 Ti) vs
+   ~16 ms/token on the CPU EP — **~4.6× faster**, entirely on the GPU.
+
+Two decoders per precision (both fed by the hoisted encoder, sharing the decoder sidecar):
+`decoder_model_merged*.onnx` (**static self-KV**, marker `winstt_static_kv`, fastest on DirectML) and
+`decoder_model_merged*_dyn.onnx` (**growing self-KV**, faster on the CPU EP). WinSTT picks per device.
+
+Numerically verified against the source on CPU (autoregressive logits parity: bit-exact fp32/q4).
+
+**Notes vs the multilingual export:** this torch-traced graph uses a `(B, S, nh, hd)` head layout, so
+the cross-attention is left dynamic (not padded to a fixed bucket) — perfectly fast for the ≤10 s
+segments a dictation VAD produces; only unrealistically long single clips would benefit from
+bucketing. The **int8** decoder ships **dynamic-only** (`decoder_model_merged_int8.onnx` is the
+growing-self graph): its `DynamicQuantizeLinear` scales interact badly with the fixed-KV write, so it
+runs the hybrid path (encoder-DML / decoder-CPU). fp32 and q4 get the full static decode.
+
+## Files
+
+- `onnx/encoder_model[_int8|_q4].onnx` (+ external data) — hoisted encoders (cross-KV outputs).
+- `onnx/decoder_model_merged[_q4].onnx` (+ `_dyn`) — static / dynamic decoders (fp32, q4).
+- `onnx/decoder_model_merged_int8.onnx` — dynamic-only (int8).
+- Tokenizer / configs — unchanged.

onnx/decoder_model_merged.onnx

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3b00b8c5744fa39b79d215bc9a53ebdc3c9d0a3ba7e4fa8272cfa6d5c0fba6b4
-size 147878
+oid sha256:5800d1290f24c78f312a57fb79e40cc7ef873df09e3c2870c66cd6744e52405a
+size 134525

onnx/decoder_model_merged_dyn.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b00b8c5744fa39b79d215bc9a53ebdc3c9d0a3ba7e4fa8272cfa6d5c0fba6b4
+size 147878

onnx/decoder_model_merged_q4.onnx

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3efcd8bc7ac501e6991fb9bb9677ad7143b88870aeac85a14b350cc4c88e62d7
-size 165682
+oid sha256:5e9a47d683b5d89925c61ba4de6b69f142a08fa2b7fa877618aca9e5b5d7482e
+size 152329

onnx/decoder_model_merged_q4_dyn.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3efcd8bc7ac501e6991fb9bb9677ad7143b88870aeac85a14b350cc4c88e62d7
+size 165682