Instructions to use kernelpool/Kimi-K3-2bit-UVMAX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use kernelpool/Kimi-K3-2bit-UVMAX with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("kernelpool/Kimi-K3-2bit-UVMAX") config = load_config("kernelpool/Kimi-K3-2bit-UVMAX") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -32,12 +32,13 @@ from measured round-trip quantization error, rather than uniformly.
|
|
| 32 |
|
| 33 |
## Use with mlx
|
| 34 |
|
| 35 |
-
This model requires Kimi K3 support from
|
| 36 |
-
|
| 37 |
-
|
|
|
|
| 38 |
|
| 39 |
```bash
|
| 40 |
-
pip install git+https://github.com/ml-explore/mlx-lm.git@refs/pull/
|
| 41 |
pip install tiktoken
|
| 42 |
```
|
| 43 |
|
|
|
|
| 32 |
|
| 33 |
## Use with mlx
|
| 34 |
|
| 35 |
+
This model requires Kimi K3 support from
|
| 36 |
+
[mlx-lm PR #1626](https://github.com/ml-explore/mlx-lm/pull/1626), which has
|
| 37 |
+
not yet been merged. Until it is included in an mlx-lm release, install
|
| 38 |
+
mlx-lm from the PR branch:
|
| 39 |
|
| 40 |
```bash
|
| 41 |
+
pip install git+https://github.com/ml-explore/mlx-lm.git@refs/pull/1626/head
|
| 42 |
pip install tiktoken
|
| 43 |
```
|
| 44 |
|