README.md

metadata

license: apache-2.0
tags:
  - directml
  - onnx
  - embedding
  - qwen3
  - sentence-transformers
  - feature-extraction
pipeline_tag: feature-extraction

Octen-Embedding-0.6B — DirectML ONNX

Patched ONNX export of Octen/Octen-Embedding-0.6B that runs on DirectML (Windows GPU via ONNX Runtime).

What was fixed

The original torch.onnx.export (dynamo) produces val_41 = [-1] used in Reshape shapes for multi-head attention (GQA: 16 Q heads, 8 KV heads). DirectML's execution provider cannot resolve symbolic -1 at graph-capture time.

Fix: Replace [-1] with four concrete head-count constants (16 for Q, 8 for K, 8 for V, 2048 for attention output) and reconnect 84 Reshape consumer nodes.

See fix_octen_dml.py for the full patch script.

Files

• model.fp16.onnx — ONNX graph proto (4 MB)
• model.fp16.onnx.data — external weights (1.1 GB, fp16)
• tokenizer.json — Qwen2 tokenizer
• config.json — model config (max_position_embeddings=32768)
• fix_octen_dml.py — reproduction script

Usage

import onnxruntime as ort

session = ort.InferenceSession(
    "model.fp16.onnx",
    providers=["DmlExecutionProvider"],
)

Quality

Dataset	R@5	R@10	MRR
esp32 (smoke)	0.930	0.950	0.810
autosar	0.678	0.774	0.552

Identical to CPU fp16 reference — patch preserves quality exactly.