Text Generation
MLX
Safetensors
Chinese
English
qwen2
Cantonese
Qwen2
chat
conversational
Eval Results (legacy)
4-bit precision
Instructions to use hyperkit/Qwen2-Cantonese-7B-Instruct-mlx with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use hyperkit/Qwen2-Cantonese-7B-Instruct-mlx with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("hyperkit/Qwen2-Cantonese-7B-Instruct-mlx") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- MLX LM
How to use hyperkit/Qwen2-Cantonese-7B-Instruct-mlx with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "hyperkit/Qwen2-Cantonese-7B-Instruct-mlx"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "hyperkit/Qwen2-Cantonese-7B-Instruct-mlx" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hyperkit/Qwen2-Cantonese-7B-Instruct-mlx", "messages": [ {"role": "user", "content": "Hello"} ] }'
Update README.md
Browse files
README.md
CHANGED
|
@@ -12,8 +12,6 @@ datasets:
|
|
| 12 |
- jed351/cantonese-wikipedia
|
| 13 |
- raptorkwok/cantonese-traditional-chinese-parallel-corpus
|
| 14 |
pipeline_tag: text-generation
|
| 15 |
-
base_model: lordjia/Qwen2-Cantonese-7B-Instruct
|
| 16 |
-
library_name: mlx
|
| 17 |
model-index:
|
| 18 |
- name: Qwen2-Cantonese-7B-Instruct
|
| 19 |
results:
|
|
@@ -30,7 +28,8 @@ model-index:
|
|
| 30 |
value: 54.35
|
| 31 |
name: strict accuracy
|
| 32 |
source:
|
| 33 |
-
url:
|
|
|
|
| 34 |
name: Open LLM Leaderboard
|
| 35 |
- task:
|
| 36 |
type: text-generation
|
|
@@ -45,7 +44,8 @@ model-index:
|
|
| 45 |
value: 32.45
|
| 46 |
name: normalized accuracy
|
| 47 |
source:
|
| 48 |
-
url:
|
|
|
|
| 49 |
name: Open LLM Leaderboard
|
| 50 |
- task:
|
| 51 |
type: text-generation
|
|
@@ -60,7 +60,8 @@ model-index:
|
|
| 60 |
value: 8.76
|
| 61 |
name: exact match
|
| 62 |
source:
|
| 63 |
-
url:
|
|
|
|
| 64 |
name: Open LLM Leaderboard
|
| 65 |
- task:
|
| 66 |
type: text-generation
|
|
@@ -75,7 +76,8 @@ model-index:
|
|
| 75 |
value: 6.04
|
| 76 |
name: acc_norm
|
| 77 |
source:
|
| 78 |
-
url:
|
|
|
|
| 79 |
name: Open LLM Leaderboard
|
| 80 |
- task:
|
| 81 |
type: text-generation
|
|
@@ -90,7 +92,8 @@ model-index:
|
|
| 90 |
value: 7.81
|
| 91 |
name: acc_norm
|
| 92 |
source:
|
| 93 |
-
url:
|
|
|
|
| 94 |
name: Open LLM Leaderboard
|
| 95 |
- task:
|
| 96 |
type: text-generation
|
|
@@ -107,6 +110,66 @@ model-index:
|
|
| 107 |
value: 31.59
|
| 108 |
name: accuracy
|
| 109 |
source:
|
| 110 |
-
url:
|
|
|
|
| 111 |
name: Open LLM Leaderboard
|
|
|
|
| 112 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
- jed351/cantonese-wikipedia
|
| 13 |
- raptorkwok/cantonese-traditional-chinese-parallel-corpus
|
| 14 |
pipeline_tag: text-generation
|
|
|
|
|
|
|
| 15 |
model-index:
|
| 16 |
- name: Qwen2-Cantonese-7B-Instruct
|
| 17 |
results:
|
|
|
|
| 28 |
value: 54.35
|
| 29 |
name: strict accuracy
|
| 30 |
source:
|
| 31 |
+
url: >-
|
| 32 |
+
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=lordjia/Qwen2-Cantonese-7B-Instruct
|
| 33 |
name: Open LLM Leaderboard
|
| 34 |
- task:
|
| 35 |
type: text-generation
|
|
|
|
| 44 |
value: 32.45
|
| 45 |
name: normalized accuracy
|
| 46 |
source:
|
| 47 |
+
url: >-
|
| 48 |
+
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=lordjia/Qwen2-Cantonese-7B-Instruct
|
| 49 |
name: Open LLM Leaderboard
|
| 50 |
- task:
|
| 51 |
type: text-generation
|
|
|
|
| 60 |
value: 8.76
|
| 61 |
name: exact match
|
| 62 |
source:
|
| 63 |
+
url: >-
|
| 64 |
+
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=lordjia/Qwen2-Cantonese-7B-Instruct
|
| 65 |
name: Open LLM Leaderboard
|
| 66 |
- task:
|
| 67 |
type: text-generation
|
|
|
|
| 76 |
value: 6.04
|
| 77 |
name: acc_norm
|
| 78 |
source:
|
| 79 |
+
url: >-
|
| 80 |
+
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=lordjia/Qwen2-Cantonese-7B-Instruct
|
| 81 |
name: Open LLM Leaderboard
|
| 82 |
- task:
|
| 83 |
type: text-generation
|
|
|
|
| 92 |
value: 7.81
|
| 93 |
name: acc_norm
|
| 94 |
source:
|
| 95 |
+
url: >-
|
| 96 |
+
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=lordjia/Qwen2-Cantonese-7B-Instruct
|
| 97 |
name: Open LLM Leaderboard
|
| 98 |
- task:
|
| 99 |
type: text-generation
|
|
|
|
| 110 |
value: 31.59
|
| 111 |
name: accuracy
|
| 112 |
source:
|
| 113 |
+
url: >-
|
| 114 |
+
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=lordjia/Qwen2-Cantonese-7B-Instruct
|
| 115 |
name: Open LLM Leaderboard
|
| 116 |
+
library_name: mlx
|
| 117 |
---
|
| 118 |
+
|
| 119 |
+
> This is a MLX conversion of [lordjia/Qwen2-Cantonese-7B-Instruct](https://huggingface.co/lordjia/Qwen2-Cantonese-7B-Instruct)
|
| 120 |
+
|
| 121 |
+
# Qwen2-Cantonese-7B-Instruct
|
| 122 |
+
|
| 123 |
+
## Model Overview / 模型概述
|
| 124 |
+
|
| 125 |
+
Qwen2-Cantonese-7B-Instruct is a Cantonese language model based on Qwen2-7B-Instruct, fine-tuned using LoRA. It aims to enhance Cantonese text generation and comprehension capabilities, supporting various tasks such as dialogue generation, text summarization, and question-answering.
|
| 126 |
+
|
| 127 |
+
Qwen2-Cantonese-7B-Instruct係基於Qwen2-7B-Instruct嘅粵語語言模型,使用LoRA進行微調。 它旨在提高粵語文本的生成和理解能力,支持各種任務,如對話生成、文本摘要和問答。
|
| 128 |
+
|
| 129 |
+
## Model Features / 模型特性
|
| 130 |
+
|
| 131 |
+
- **Base Model**: Qwen2-7B-Instruct
|
| 132 |
+
- **Fine-tuning Method**: LoRA instruction tuning
|
| 133 |
+
- **Training Steps**: 4572 steps
|
| 134 |
+
- **Primary Language**: Cantonese / 粵語
|
| 135 |
+
- **Datasets**:
|
| 136 |
+
- [jed351/cantonese-wikipedia](https://huggingface.co/datasets/jed351/cantonese-wikipedia)
|
| 137 |
+
- [raptorkwok/cantonese-traditional-chinese-parallel-corpus](https://huggingface.co/datasets/raptorkwok/cantonese-traditional-chinese-parallel-corpus)
|
| 138 |
+
- **Training Tools**: [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory)
|
| 139 |
+
|
| 140 |
+
## Quantized Version / 量化版本
|
| 141 |
+
|
| 142 |
+
A 4-bit quantized version of this model is also available: [qwen2-cantonese-7b-instruct-q4_0.gguf](https://huggingface.co/lordjia/Qwen2-Cantonese-7B-Instruct/blob/main/qwen2-cantonese-7b-instruct-q4_0.gguf).
|
| 143 |
+
|
| 144 |
+
此外,仲提供此模型嘅4位量化版本:[qwen2-cantonese-7b-instruct-q4_0.gguf](https://huggingface.co/lordjia/Qwen2-Cantonese-7B-Instruct/blob/main/qwen2-cantonese-7b-instruct-q4_0.gguf)。
|
| 145 |
+
|
| 146 |
+
## Alternative Model Recommendations / 備選模型舉薦
|
| 147 |
+
|
| 148 |
+
For alternatives, consider the following models, both fine-tuned by LordJia on Cantonese language tasks:
|
| 149 |
+
|
| 150 |
+
揾其他嘅話,可以諗下呢啲模型,全部都係LordJia用廣東話嘅工作調教好嘅:
|
| 151 |
+
|
| 152 |
+
1. [Llama-3-Cantonese-8B-Instruct](https://huggingface.co/lordjia/Llama-3-Cantonese-8B-Instruct) based on Meta-Llama-3-8B-Instruct.
|
| 153 |
+
2. [Llama-3.1-Cantonese-8B-Instruct](https://huggingface.co/lordjia/Llama-3.1-Cantonese-8B-Instruct) based on Meta-Llama-3.1-8B-Instruct.
|
| 154 |
+
|
| 155 |
+
## License / 許可證
|
| 156 |
+
|
| 157 |
+
This model is licensed under the Apache 2.0 license. Please review the terms before use.
|
| 158 |
+
|
| 159 |
+
此模型喺Apache 2.0許可證下獲得許可。 請在使用前仔細閱讀呢啲條款。
|
| 160 |
+
|
| 161 |
+
## Contributors / 貢獻
|
| 162 |
+
|
| 163 |
+
- LordJia [https://ai.chao.cool](https://ai.chao.cool/)
|
| 164 |
+
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
|
| 165 |
+
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_lordjia__Qwen2-Cantonese-7B-Instruct)
|
| 166 |
+
|
| 167 |
+
| Metric |Value|
|
| 168 |
+
|-------------------|----:|
|
| 169 |
+
|Avg. |23.50|
|
| 170 |
+
|IFEval (0-Shot) |54.35|
|
| 171 |
+
|BBH (3-Shot) |32.45|
|
| 172 |
+
|MATH Lvl 5 (4-Shot)| 8.76|
|
| 173 |
+
|GPQA (0-shot) | 6.04|
|
| 174 |
+
|MuSR (0-shot) | 7.81|
|
| 175 |
+
|MMLU-PRO (5-shot) |31.59|
|