File size: 1,456 Bytes
88d996e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 | ---
license: apache-2.0
tags:
- directml
- onnx
- embedding
- qwen3
- sentence-transformers
- feature-extraction
pipeline_tag: feature-extraction
---
# Octen-Embedding-0.6B — DirectML ONNX
Patched ONNX export of [Octen/Octen-Embedding-0.6B](https://huggingface.co/Octen/Octen-Embedding-0.6B) that runs on **DirectML** (Windows GPU via ONNX Runtime).
## What was fixed
The original `torch.onnx.export` (dynamo) produces `val_41 = [-1]` used in Reshape shapes for multi-head attention (GQA: 16 Q heads, 8 KV heads). DirectML's execution provider cannot resolve symbolic `-1` at graph-capture time.
**Fix**: Replace `[-1]` with four concrete head-count constants (16 for Q, 8 for K, 8 for V, 2048 for attention output) and reconnect 84 Reshape consumer nodes.
See `fix_octen_dml.py` for the full patch script.
## Files
- `model.fp16.onnx` — ONNX graph proto (4 MB)
- `model.fp16.onnx.data` — external weights (1.1 GB, fp16)
- `tokenizer.json` — Qwen2 tokenizer
- `config.json` — model config (max_position_embeddings=32768)
- `fix_octen_dml.py` — reproduction script
## Usage
```python
import onnxruntime as ort
session = ort.InferenceSession(
"model.fp16.onnx",
providers=["DmlExecutionProvider"],
)
```
## Quality
| Dataset | R@5 | R@10 | MRR |
|---------|-----|------|-----|
| esp32 (smoke) | 0.930 | 0.950 | 0.810 |
| autosar | 0.678 | 0.774 | 0.552 |
Identical to CPU fp16 reference — patch preserves quality exactly.
|