Sentence Similarity
GGUF
Chinese
llama.cpp
ollama
embedding
retrieval
rag
traditional-chinese
zh-tw
taiwan
Instructions to use xCloudinfo/xVITA-Embed-300M-zhTW-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use xCloudinfo/xVITA-Embed-300M-zhTW-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf xCloudinfo/xVITA-Embed-300M-zhTW-GGUF:Q8_0 # Run inference directly in the terminal: llama cli -hf xCloudinfo/xVITA-Embed-300M-zhTW-GGUF:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf xCloudinfo/xVITA-Embed-300M-zhTW-GGUF:Q8_0 # Run inference directly in the terminal: llama cli -hf xCloudinfo/xVITA-Embed-300M-zhTW-GGUF:Q8_0
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf xCloudinfo/xVITA-Embed-300M-zhTW-GGUF:Q8_0 # Run inference directly in the terminal: ./llama-cli -hf xCloudinfo/xVITA-Embed-300M-zhTW-GGUF:Q8_0
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf xCloudinfo/xVITA-Embed-300M-zhTW-GGUF:Q8_0 # Run inference directly in the terminal: ./build/bin/llama-cli -hf xCloudinfo/xVITA-Embed-300M-zhTW-GGUF:Q8_0
Use Docker
docker model run hf.co/xCloudinfo/xVITA-Embed-300M-zhTW-GGUF:Q8_0
- LM Studio
- Jan
- Ollama
How to use xCloudinfo/xVITA-Embed-300M-zhTW-GGUF with Ollama:
ollama run hf.co/xCloudinfo/xVITA-Embed-300M-zhTW-GGUF:Q8_0
- Unsloth Studio
How to use xCloudinfo/xVITA-Embed-300M-zhTW-GGUF with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for xCloudinfo/xVITA-Embed-300M-zhTW-GGUF to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for xCloudinfo/xVITA-Embed-300M-zhTW-GGUF to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for xCloudinfo/xVITA-Embed-300M-zhTW-GGUF to start chatting
- Atomic Chat new
- Docker Model Runner
How to use xCloudinfo/xVITA-Embed-300M-zhTW-GGUF with Docker Model Runner:
docker model run hf.co/xCloudinfo/xVITA-Embed-300M-zhTW-GGUF:Q8_0
- Lemonade
How to use xCloudinfo/xVITA-Embed-300M-zhTW-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull xCloudinfo/xVITA-Embed-300M-zhTW-GGUF:Q8_0
Run and chat with the model
lemonade run user.xVITA-Embed-300M-zhTW-GGUF-Q8_0
List all available models
lemonade list
v2 model card + dense-layer fix notice
Browse files
README.md
CHANGED
|
@@ -18,37 +18,53 @@ tags:
|
|
| 18 |
|
| 19 |
# xcloud-emb-zhtw — GGUF
|
| 20 |
|
| 21 |
-
[xCloudinfo/xcloud-emb-zhtw](https://huggingface.co/xCloudinfo/xcloud-emb-zhtw)(云碩科技自製繁體中文檢索 embedding,DRCD test hit@1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
|
| 23 |
## 兩種版本,依場景選
|
| 24 |
|
| 25 |
| 版本 | context | 檔案 | 大小 | 適用 |
|
| 26 |
|---|---|---|---|---|
|
| 27 |
-
| **標準版(
|
| 28 |
-
| | | xcloud-emb-zhtw-Q8_0.gguf |
|
| 29 |
-
| **長文版
|
| 30 |
-
| | | xcloud-emb-zhtw-8k-Q8_0.gguf |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
|
| 32 |
-
|
| 33 |
-
- 需要把整份長文件當單一檢索單位、或 chunk 常超過 2048 tokens,用長文版。
|
| 34 |
-
- 兩版皆為 768 維、同一 prompt 格式;Q8_0 幾乎無損、體積小一半,一般首選。
|
| 35 |
|
| 36 |
-
|
| 37 |
|
| 38 |
| 模型 | hit@1 | hit@5 | hit@10 |
|
| 39 |
|---|---|---|---|
|
| 40 |
-
| **
|
| 41 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 42 |
| BAAI/bge-m3(8k 原生) | 77.5 | 90.3 | 93.4 |
|
| 43 |
|
| 44 |
-
長文版在完整文章檢索上勝過 bge-m3 原生 8k 達 7.2 個百分點。
|
| 45 |
|
| 46 |
## llama.cpp
|
| 47 |
|
| 48 |
```bash
|
| 49 |
-
# 標準版
|
| 50 |
llama-server -m xcloud-emb-zhtw-Q8_0.gguf --embeddings --port 8080
|
| 51 |
-
# 長文版
|
| 52 |
llama-server -m xcloud-emb-zhtw-8k-Q8_0.gguf --embeddings --port 8080
|
| 53 |
```
|
| 54 |
|
|
@@ -57,14 +73,6 @@ llama-server -m xcloud-emb-zhtw-8k-Q8_0.gguf --embeddings --port 8080
|
|
| 57 |
- 查詢:`task: search result | query: {你的問題}`
|
| 58 |
- 文檔:`title: none | text: {文檔內容}`
|
| 59 |
|
| 60 |
-
## 短段落檢索評測(DRCD test:3,493 查詢 / 1,000 段落庫)
|
| 61 |
-
|
| 62 |
-
| 模型 | hit@1 | hit@5 | MRR@10 |
|
| 63 |
-
|---|---|---|---|
|
| 64 |
-
| **xcloud-emb-zhtw 標準版 (0.3B)** | **85.4** | **97.4** | **90.7** |
|
| 65 |
-
| BAAI/bge-m3 (0.57B) | 82.2 | 95.6 | 88.1 |
|
| 66 |
-
| multilingual-e5-large (0.56B) | 82.8 | 95.3 | 88.3 |
|
| 67 |
-
|
| 68 |
完整評測、訓練細節與誠實聲明見主 repo model card。
|
| 69 |
|
| 70 |
---
|
|
|
|
| 18 |
|
| 19 |
# xcloud-emb-zhtw — GGUF
|
| 20 |
|
| 21 |
+
[xCloudinfo/xcloud-emb-zhtw](https://huggingface.co/xCloudinfo/xcloud-emb-zhtw)(云碩科技自製繁體中文檢索 embedding,DRCD test hit@1 86.0、超越 bge-m3 / multilingual-e5-large)的 GGUF 量化版,適用 llama.cpp 與 ollama。
|
| 22 |
+
|
| 23 |
+
> **2026-07-18 重要修復**:本日以前的所有 GGUF 檔因轉檔工具與 sentence-transformers 5.6 模組命名不相容,
|
| 24 |
+
> **缺少模型的 Dense 投影層**,向量空間與 safetensors 版不一致(cosine 相似度失效)。請重新下載本日後的檔案。
|
| 25 |
+
> 現版本已逐一與 safetensors 版驗證向量一致(f16 cosine = 1.0、Q8_0 = 0.9998),並已把 mean pooling
|
| 26 |
+
> 寫入 GGUF metadata,llama.cpp 無須再手動指定 `--pooling`。
|
| 27 |
|
| 28 |
## 兩種版本,依場景選
|
| 29 |
|
| 30 |
| 版本 | context | 檔案 | 大小 | 適用 |
|
| 31 |
|---|---|---|---|---|
|
| 32 |
+
| **標準版(v2)** | 8192 | xcloud-emb-zhtw-f16.gguf | 584 MB | 一般 RAG、段落級檢索(**短文分數最佳 86.0**) |
|
| 33 |
+
| | | xcloud-emb-zhtw-Q8_0.gguf | 313 MB | |
|
| 34 |
+
| **長文特化版** | 8192 | xcloud-emb-zhtw-8k-f16.gguf | 584 MB | 長文件整篇檢索、法規/報告全文(**長文分數最佳 84.7**) |
|
| 35 |
+
| | | xcloud-emb-zhtw-8k-Q8_0.gguf | 313 MB | |
|
| 36 |
+
|
| 37 |
+
- 段落級 RAG(chunk ≤ 2048 tokens)用標準版,短文檢索分數最佳。
|
| 38 |
+
- 把整份長文件當單一檢索單位(整篇法規、報告全文)用長文特化版。
|
| 39 |
+
- 兩版皆 768 維、同一 prompt 格式、皆支援 8192 tokens 輸入;差別在訓練取向(標準版短文最佳、長文特化版長文最佳)。
|
| 40 |
+
- Q8_0 幾乎無損(與 f16 cosine 0.9998)、體積小一半,一般首選。
|
| 41 |
|
| 42 |
+
## 評測(DRCD test,與 safetensors 版同 protocol)
|
|
|
|
|
|
|
| 43 |
|
| 44 |
+
短段落檢索(3,493 查詢 / 1,000 段落庫):
|
| 45 |
|
| 46 |
| 模型 | hit@1 | hit@5 | hit@10 |
|
| 47 |
|---|---|---|---|
|
| 48 |
+
| **標準版 v2(0.3B)** | **86.0** | **97.4** | **98.7** |
|
| 49 |
+
| multilingual-e5-large(0.56B) | 82.8 | 95.3 | 97.0 |
|
| 50 |
+
| BAAI/bge-m3(0.57B) | 82.2 | 95.6 | 97.2 |
|
| 51 |
+
|
| 52 |
+
長文整篇檢索(378 篇完整文章 / 3,493 查詢):
|
| 53 |
+
|
| 54 |
+
| 模型 | hit@1 | hit@5 | hit@10 |
|
| 55 |
+
|---|---|---|---|
|
| 56 |
+
| **長文特化版** | **84.7** | **94.5** | **96.6** |
|
| 57 |
+
| 標準版 v2 | 79.2 | 91.2 | 94.3 |
|
| 58 |
| BAAI/bge-m3(8k 原生) | 77.5 | 90.3 | 93.4 |
|
| 59 |
|
| 60 |
+
長文特化版在完整文章檢索上勝過 bge-m3 原生 8k 達 7.2 個百分點。
|
| 61 |
|
| 62 |
## llama.cpp
|
| 63 |
|
| 64 |
```bash
|
| 65 |
+
# 標準版(段落級 RAG)
|
| 66 |
llama-server -m xcloud-emb-zhtw-Q8_0.gguf --embeddings --port 8080
|
| 67 |
+
# 長文特化版(整篇長文件檢索)
|
| 68 |
llama-server -m xcloud-emb-zhtw-8k-Q8_0.gguf --embeddings --port 8080
|
| 69 |
```
|
| 70 |
|
|
|
|
| 73 |
- 查詢:`task: search result | query: {你的問題}`
|
| 74 |
- 文檔:`title: none | text: {文檔內容}`
|
| 75 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 76 |
完整評測、訓練細節與誠實聲明見主 repo model card。
|
| 77 |
|
| 78 |
---
|