Instructions to use unsloth/Kimi-K2.6 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use unsloth/Kimi-K2.6 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="unsloth/Kimi-K2.6", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("unsloth/Kimi-K2.6", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use unsloth/Kimi-K2.6 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "unsloth/Kimi-K2.6"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "unsloth/Kimi-K2.6",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/unsloth/Kimi-K2.6

SGLang

How to use unsloth/Kimi-K2.6 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "unsloth/Kimi-K2.6" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "unsloth/Kimi-K2.6",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "unsloth/Kimi-K2.6" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "unsloth/Kimi-K2.6",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Unsloth Studio

How to use unsloth/Kimi-K2.6 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for unsloth/Kimi-K2.6 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for unsloth/Kimi-K2.6 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for unsloth/Kimi-K2.6 to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="unsloth/Kimi-K2.6",
    max_seq_length=2048,
)

Docker Model Runner
How to use unsloth/Kimi-K2.6 with Docker Model Runner:
```
docker model run hf.co/unsloth/Kimi-K2.6
```

danielhanchen commited on Apr 20

Commit

827e1a8

verified ·

1 Parent(s): 68fa6c2

Add files using upload-large-folder tool

Browse files

Files changed (47) hide show

figures/demo_video.mp4 +3 -0
model-00001-of-000064.safetensors +3 -0
model-00002-of-000064.safetensors +3 -0
model-00003-of-000064.safetensors +3 -0
model-00004-of-000064.safetensors +3 -0
model-00006-of-000064.safetensors +3 -0
model-00009-of-000064.safetensors +3 -0
model-00010-of-000064.safetensors +3 -0
model-00011-of-000064.safetensors +3 -0
model-00014-of-000064.safetensors +3 -0
model-00015-of-000064.safetensors +3 -0
model-00016-of-000064.safetensors +3 -0
model-00017-of-000064.safetensors +3 -0
model-00018-of-000064.safetensors +3 -0
model-00019-of-000064.safetensors +3 -0
model-00020-of-000064.safetensors +3 -0
model-00021-of-000064.safetensors +3 -0
model-00022-of-000064.safetensors +3 -0
model-00023-of-000064.safetensors +3 -0
model-00024-of-000064.safetensors +3 -0
model-00026-of-000064.safetensors +3 -0
model-00027-of-000064.safetensors +3 -0
model-00028-of-000064.safetensors +3 -0
model-00029-of-000064.safetensors +3 -0
model-00032-of-000064.safetensors +3 -0
model-00033-of-000064.safetensors +3 -0
model-00037-of-000064.safetensors +3 -0
model-00039-of-000064.safetensors +3 -0
model-00042-of-000064.safetensors +3 -0
model-00043-of-000064.safetensors +3 -0
model-00044-of-000064.safetensors +3 -0
model-00047-of-000064.safetensors +3 -0
model-00048-of-000064.safetensors +3 -0
model-00049-of-000064.safetensors +3 -0
model-00050-of-000064.safetensors +3 -0
model-00051-of-000064.safetensors +3 -0
model-00052-of-000064.safetensors +3 -0
model-00053-of-000064.safetensors +3 -0
model-00054-of-000064.safetensors +3 -0
model-00056-of-000064.safetensors +3 -0
model-00057-of-000064.safetensors +3 -0
model-00058-of-000064.safetensors +3 -0
model-00059-of-000064.safetensors +3 -0
model-00060-of-000064.safetensors +3 -0
model-00061-of-000064.safetensors +3 -0
model.safetensors.index.json +3 -0
tiktoken.model +3 -0

figures/demo_video.mp4 ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:09b4d925aa0a7c712feef50765355f0625d8f6d46ea302fd98db9609e9070047
+size 270100

model-00001-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cb6e037206c1936876c33f348bc644fd6f9f4d7ac973f8906359977c1eaebd43
+size 995001888

model-00002-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a609fccff2406505ae575850a51470474e8bb3eb825ddd208f84aae31cfb4960
+size 9809047464

model-00003-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f79ae8d220ecbba674c1224bbb36e2ad826ccc365765a89fab57555af47a8540
+size 9809047464

model-00004-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:efa041ff6fd295b3a5b8bc4fe7db502d32e6f887d650353425c3074388ecaa33
+size 9809047464

model-00006-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5cf4c4593e5a7e5b4ababc8991245ab53121be414558099dc12dd01ab00eb920
+size 9809047464

model-00009-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4f8c2dcde7c51a3eca91bb892eacd8bcbcd7724842254a0000bb1c45faf92eb2
+size 9809047464

model-00010-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:311ffdf698fb27c03697598cd2c652a1ec210d9a7351d5c2de1b14d605635195
+size 9809047464

model-00011-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b2a93273e323843ad067e00dc859f63d90f62bd2e81ef0a77ffe49a9f39a3607
+size 9809050936

model-00014-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ee309d11f1c4194db36e961293d6e7b9d3c3c8227da5064ffc32b806ac03cc91
+size 9809050936

model-00015-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4859b4dfe8eb7998bbda5e453f9336b139881d403b0a8c128b98842c9c27800d
+size 9809050936

model-00016-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4345e4706a01e63aef6a43025bc101ebbfb8e8c235ea44604fb9c5ad7038b4cc
+size 9809050936

model-00017-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e0b6e9d578c7f0eaa3e91aa917fb3cbdcaf5637caa3ad97e9dc002bcb35feb22
+size 9809050936

model-00018-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1bef116d936999f347b83403ee13fe479f51214a87592409121614793d7e65e7
+size 9809050936

model-00019-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:71b624faee1df5cc99bad259b018befb0bdbbdd160af3d130487bf1165d48b9a
+size 9809050936

model-00020-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9c6ee52ccb57bd2fdba6077a18faaf3ea6da1b3536597af4fef6c5cf713bec34
+size 9809050936

model-00021-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fb99455eb90ddd9ab343538496c3e90b40868c0029a344d79b0b94f73820dd1f
+size 9809050936

model-00022-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f98118e5b5e700ad57b510f3c80183fc2e5e3dcd1badf43a959ce63140ae4e61
+size 9809050936

model-00023-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0a4f4fb4557c2954dc07ffd8d81af93587ab784e6810eca093fef6b0fab03d3c
+size 9809050936

model-00024-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9fa6f9aba1f02a552a66d0bcf2808fdc9192425b5554684b637d7e9d9312fb3d
+size 9809050936

model-00026-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6b6e19204d6f74af7ff2c78c67a33991f5ee4105e0262d99eda6ba6bc0362630
+size 9809050936

model-00027-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8476d21ab258330c16020124d634d2cd1f22a5a7859d3dd778132dea0271e810
+size 9809050936

model-00028-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ee16b8cf5c8a9efb0c5fd89c282622e5cfbf4f49fb28d5b3638240434dadef7a
+size 9809050936

model-00029-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:87e0e7b5caf5b39314caf4158b666d0b0c9fe00a96dd09ea545563f4a6f408ed
+size 9809050936

model-00032-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:eee21740e4d75ba175965f322dd8d2747411fedc1b59b9a0559630d7c94d06c6
+size 9809050936

model-00033-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cc9a7b5737151ca68fed23cb16308aff1527d8932be59d507318795025a9034e
+size 9809050936

model-00037-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0c7c1080287ab3b2968335b87c0652000b0bfdffa1925be375fca3bda44c307f
+size 9809050936

model-00039-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2e09b40d7cc63d12c721b80a628246d984e39613b8c04c3f17c178d8c2b2b08e
+size 9809050936

model-00042-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1215992a0a15aac21b7100c1f236c8029e361d03e69749d5d3e2d93c94dc8315
+size 9809050936

model-00043-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:170fa46329c380b972269f75ccf01c3a4addeb29dd4ee865a700e1be3a0241c0
+size 9809050936

model-00044-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:46101090a2c0b8c1a656a52ff9f10b71b42235470df68acf496eaaf1907cd4ef
+size 9809050936

model-00047-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:29c023ba04d62faae4d2ffb73b59b5595d849727cb937626366cb512586fd5bb
+size 9809050936

model-00048-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ebbde9208528e898fd82a76825c3149e3054a540824b5ca98e1cd56ca7ecfc11
+size 9809050936

model-00049-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4cb4685422970c3933807e3bd2bd81e2b931a0fb4aa43a900d17bc692a2efc7a
+size 9809050936

model-00050-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f9a2ca8f3adb0e15215be60d9d0b2f11a8aa47cffaee8185c738cb7703fc8a23
+size 9809050936

model-00051-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bc9588957be18f8e10b21e279b1764fa140b316c16a2d09583fcaa23188c8ce0
+size 9809050936

model-00052-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8861c6344eb946121bdc67bb3c0c64f6c600c13d83861399862b2c4176d5d25f
+size 9809050936

model-00053-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7e9ed6ed7757d845262390e645c663be925c5634c1afea5e918a0dec5bd4e034
+size 9809050936

model-00054-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a1d4d2090a4e1d2500a5eb7b64a85b9a2386b50e9081220387b910c2153ec4a4
+size 9809050936

model-00056-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b92fb0d442a6c4c3bf68fcc9d2e88ecb0d35f775d09bbad72096a2f245c92010
+size 9809050936

model-00057-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fbce540ebf43d5161cb69ca732a4d32850c73b051d29cd065d3fd717346b5b47
+size 9809050936

model-00058-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e15bd7e760663b37a5c5eecaed66ac936b3540f5125ba71cc13607a07a26eeca
+size 9809050936

model-00059-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:46dcfd4ef0e87d6ac5fb72275cca67dcc1422adf05aa16a03eab747fdf387aa0
+size 9809050936

model-00060-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3f942b79b835d3eaf96117302adbb7baa8ebea283cc9c3f32c27c04f3772e3a9
+size 9809050936

model-00061-of-000064.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ad295842a722b5cc700fbbf485ec9a9875f0aab756326b6b62ff4fc4bed0bd1b
+size 9809050936

model.safetensors.index.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:00a982d005d6e39b25cfcb379a7e6d57f393fc0346e70892ea4271c259c0d1e3
+size 23597438

tiktoken.model ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b6c497a7469b33ced9c38afb1ad6e47f03f5e5dc05f15930799210ec050c5103
+size 2795286