NghiaNguyen1529 commited on
Commit
88d996e
·
verified ·
1 Parent(s): e337819

Add README with usage and quality info

Browse files
Files changed (1) hide show
  1. README.md +51 -0
README.md ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - directml
5
+ - onnx
6
+ - embedding
7
+ - qwen3
8
+ - sentence-transformers
9
+ - feature-extraction
10
+ pipeline_tag: feature-extraction
11
+ ---
12
+
13
+ # Octen-Embedding-0.6B — DirectML ONNX
14
+
15
+ Patched ONNX export of [Octen/Octen-Embedding-0.6B](https://huggingface.co/Octen/Octen-Embedding-0.6B) that runs on **DirectML** (Windows GPU via ONNX Runtime).
16
+
17
+ ## What was fixed
18
+
19
+ The original `torch.onnx.export` (dynamo) produces `val_41 = [-1]` used in Reshape shapes for multi-head attention (GQA: 16 Q heads, 8 KV heads). DirectML's execution provider cannot resolve symbolic `-1` at graph-capture time.
20
+
21
+ **Fix**: Replace `[-1]` with four concrete head-count constants (16 for Q, 8 for K, 8 for V, 2048 for attention output) and reconnect 84 Reshape consumer nodes.
22
+
23
+ See `fix_octen_dml.py` for the full patch script.
24
+
25
+ ## Files
26
+
27
+ - `model.fp16.onnx` — ONNX graph proto (4 MB)
28
+ - `model.fp16.onnx.data` — external weights (1.1 GB, fp16)
29
+ - `tokenizer.json` — Qwen2 tokenizer
30
+ - `config.json` — model config (max_position_embeddings=32768)
31
+ - `fix_octen_dml.py` — reproduction script
32
+
33
+ ## Usage
34
+
35
+ ```python
36
+ import onnxruntime as ort
37
+
38
+ session = ort.InferenceSession(
39
+ "model.fp16.onnx",
40
+ providers=["DmlExecutionProvider"],
41
+ )
42
+ ```
43
+
44
+ ## Quality
45
+
46
+ | Dataset | R@5 | R@10 | MRR |
47
+ |---------|-----|------|-----|
48
+ | esp32 (smoke) | 0.930 | 0.950 | 0.810 |
49
+ | autosar | 0.678 | 0.774 | 0.552 |
50
+
51
+ Identical to CPU fp16 reference — patch preserves quality exactly.