Spaces:

---

mlx-community

/

README

like 15

Running

Hello MLX Community maintainers,

I converted Octen/Octen-Embedding-4B to a standard MLX-LM affine 4-bit checkpoint and would like to request review for inclusion or transfer into mlx-community.

Model: https://huggingface.co/davied-he/Octen-Embedding-4B-MLX-4bit

Key details:

• Upstream: Octen/Octen-Embedding-4B at revision fea468fae3f0caffbae8a12ba792d1c394b6277d
• License: Apache-2.0
• Official mlx_lm.convert 0.31.3, affine 4-bit, group size 64
• Standard built-in mlx_lm.models.qwen3.Model; no custom Python model code or model_file
• Encoder hidden states: [batch, sequence, 2560]
• Last-token pooling with float32 L2 normalization, matching the upstream prompt contract

The upstream encoder checkpoint stores keys as embed_tokens.* / layers.*. Before running mlx_lm.convert, I deterministically prefixed those keys with model. so they match the built-in MLX-LM Qwen3 wrapper. Tensor values were unchanged during key normalization.

Validation performed on Apple silicon:

• Public Hub repository loads directly with mlx_lm.load
• Runtime classes: built-in Model / Qwen3Model
• No custom .py model files
• Output dimension: 2560; normalized vector norms: 1.0
• Three English and Traditional Chinese samples had BF16-to-MLX-4bit embedding cosine 0.9748–0.9782
• Bilingual retrieval smoke test returned the expected Top-1 documents

The model card includes conversion details, usage, limitations, attribution, checksums, and validation evidence. I am happy to address review feedback or transfer the repository if accepted.

Thank you.