dsaad68/LFM2.5-Embedding-350M-ONNX-fp16

LFM2.5-Embedding-350M — ONNX (fp16)

fp16 ONNX export of LiquidAI/LFM2.5-Embedding-350M, a 354M-parameter bidirectional LFM2 embedding model (hybrid short-conv + attention, CLS pooling, 1024-dimensional, query: / document: prompt prefixes).

Runs on ONNX Runtime (CPU and GPU), onnxruntime-web (WASM and WebGPU), with no PyTorch dependency. Parity vs the PyTorch reference: min cosine 0.99976. Looking for a smaller model? See the int8 variant: dsaad68/LFM2.5-Embedding-350M-ONNX-int8. Export tooling, parity tests, and precision rationale: dsaad68/liquid-embedding-onnx.

Files

File	Notes
onnx/model.onnx	fp16 graph, self-contained single file (~711 MB; exported directly in fp16, not post-hoc converted)

The graph is a single model with two outputs (one weight copy serves both):

Output	Shape	Use
sentence_embedding	(batch, 1024)	CLS pooling + L2 normalization baked in — use this for retrieval
last_hidden_state	(batch, sequence, 1024)	raw token embeddings — pool externally (used by the sentence-transformers ONNX backend)

Usage

Prompt prefixes are required (asymmetric retrieval model): prepend query: to queries and document: to passages. Max sequence length is 512 tokens. Outputs are fp16 — upcast to fp32 if your pipeline expects it.

sentence-transformers (ONNX backend)

Verified against the PyTorch reference (min cosine ≥ 0.9996). Requires optimum[onnxruntime]; onnx/model.onnx is found automatically:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("dsaad68/LFM2.5-Embedding-350M-ONNX-fp16", backend="onnx", trust_remote_code=True)
q = model.encode(["How do I reset my password?"], prompt_name="query",
                 normalize_embeddings=True)
d = model.encode(["Click 'Forgot password' on the sign-in page."],
                 prompt_name="document", normalize_embeddings=True)
print(q @ d.T)

ONNX Runtime (Python)

import numpy as np, onnxruntime as ort
from huggingface_hub import snapshot_download
from transformers import AutoTokenizer

repo = snapshot_download("dsaad68/LFM2.5-Embedding-350M-ONNX-fp16")
tok = AutoTokenizer.from_pretrained(repo)
sess = ort.InferenceSession(f"{repo}/onnx/model.onnx")

texts = ["query: How do I reset my password?",
         "document: Click 'Forgot password' on the sign-in page."]
enc = tok(texts, padding=True, truncation=True, max_length=512, return_tensors="np")
emb = sess.run(["sentence_embedding"],
               {"input_ids": enc["input_ids"].astype(np.int64),
                "attention_mask": enc["attention_mask"].astype(np.int64)})[0]
# emb: (2, 1024) fp16 unit vectors; cosine similarity = dot product
print(emb.astype(np.float32) @ emb.astype(np.float32).T)

Browser (onnxruntime-web)

Runs under both the WASM and WebGPU execution providers (fp16 is the right choice for WebGPU); a single self-contained file, no external-data sidecar to wire up. fp16 tensor data arrives as Uint16Array (JS has no native float16). See the web-test harness for a worked example.

Export details

• Exported with torch.onnx.export (opset 17, dynamic batch + sequence axes) from the upstream remote-code Lfm2BidirectionalModel with attn_implementation="eager", which reproduces the exact training-time masking behavior.
• Parity gate (run in CI on every publish): per-vector cosine vs SentenceTransformer(..., trust_remote_code=True) ≥ 0.999 on both outputs, plus retrieval-ranking equality.
• Why fp32 / bf16 / fp8 / int4 were not shipped: learning notes.

License

The model weights are redistributed under the LFM Open License v1.0, unchanged from the original LiquidAI/LFM2.5-Embedding-350M release. All credit for the model itself goes to Liquid AI; this repository only converts it to ONNX.