---
license: apache-2.0
tags:
- directml
- onnx
- embedding
- qwen3
- sentence-transformers
- feature-extraction
pipeline_tag: feature-extraction
---

# Octen-Embedding-0.6B — DirectML ONNX

Patched ONNX export of [Octen/Octen-Embedding-0.6B](https://huggingface.co/Octen/Octen-Embedding-0.6B) that runs on **DirectML** (Windows GPU via ONNX Runtime).

## What was fixed

The original `torch.onnx.export` (dynamo) produces `val_41 = [-1]` used in Reshape shapes for multi-head attention (GQA: 16 Q heads, 8 KV heads). DirectML's execution provider cannot resolve symbolic `-1` at graph-capture time.

**Fix**: Replace `[-1]` with four concrete head-count constants (16 for Q, 8 for K, 8 for V, 2048 for attention output) and reconnect 84 Reshape consumer nodes.

See `fix_octen_dml.py` for the full patch script.

## Files

- `model.fp16.onnx` — ONNX graph proto (4 MB)
- `model.fp16.onnx.data` — external weights (1.1 GB, fp16)
- `tokenizer.json` — Qwen2 tokenizer
- `config.json` — model config (max_position_embeddings=32768)
- `fix_octen_dml.py` — reproduction script

## Usage

```python
import onnxruntime as ort

session = ort.InferenceSession(
    "model.fp16.onnx",
    providers=["DmlExecutionProvider"],
)
```

## Quality

| Dataset | R@5 | R@10 | MRR |
|---------|-----|------|-----|
| esp32 (smoke) | 0.930 | 0.950 | 0.810 |
| autosar | 0.678 | 0.774 | 0.552 |

Identical to CPU fp16 reference — patch preserves quality exactly.