Instructions to use RadixArk/Kimi-K3-DSpark with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use RadixArk/Kimi-K3-DSpark with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="RadixArk/Kimi-K3-DSpark", trust_remote_code=True)

# Load model directly
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("RadixArk/Kimi-K3-DSpark", trust_remote_code=True)
model = AutoModel.from_pretrained("RadixArk/Kimi-K3-DSpark", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use RadixArk/Kimi-K3-DSpark with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "RadixArk/Kimi-K3-DSpark"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RadixArk/Kimi-K3-DSpark",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/RadixArk/Kimi-K3-DSpark

SGLang

How to use RadixArk/Kimi-K3-DSpark with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "RadixArk/Kimi-K3-DSpark" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RadixArk/Kimi-K3-DSpark",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "RadixArk/Kimi-K3-DSpark" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RadixArk/Kimi-K3-DSpark",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use RadixArk/Kimi-K3-DSpark with Docker Model Runner:
```
docker model run hf.co/RadixArk/Kimi-K3-DSpark
```

Kimi-K3-DSpark / README.md

cm00cm

Add Kimi K3 DSpark model card

eb03982 verified 2 days ago

preview code

Raw

History Blame Contribute Delete

3.01 kB

	---
	library_name: transformers
	pipeline_tag: text-generation
	tags:
	- speculative-decoding
	- dspark
	- dflash
	- specforge
	- sglang
	inference: false
	---

	# Kimi K3 DSpark speculator

	## Overview

	A DSpark speculator for the Kimi K3 target, enabling faster inference through speculative decoding. DSpark extends the DFlash parallel-draft backbone with a Markov logit-bias head and a per-position confidence head. This checkpoint was trained with [SpecForge](https://github.com/sgl-project/SpecForge/) using hidden states from a live SGLang target engine.

	## Model Specifications

	- Base model: `moonshotai/Kimi-K3`.
	- Format: Safetensors (single-file BF16, 2,249,289,601 parameters).
	- Draft: 5 layers (Qwen3-style GQA), hidden 7168, 64 heads / 16 KV heads, head_dim 64, FFN 14336, rope_theta 10000, `block_size=7`.
	- Vocabulary: 163,840 rows; `mask_token_id=163824`, `bos=163584`, `eos=163586`, `pad=163839`.
	- DSpark heads: Markov rank 256 (vanilla) and confidence head (with-Markov).
	- Aux hidden-state layers: `[7, 23, 51, 67, 83]` (full-attention layers of the 93-layer target).
	- Trained context: sequence length 4096.
	- Target weights: target embedding and unembedding weights are not included in this checkpoint.

	## Evaluation Results

	\| Dataset \| Accept length \|
	\|---\|---:\|
	\| GSM8K \| 5.6660 \|
	\| MATH500 \| 4.0803 \|
	\| HumanEval \| 5.3613 \|
	\| MBPP \| 4.9220 \|
	\| MT-Bench \| 3.3425 \|
	\| AIME26 \| 2.6326 \|

	## Serving with SGLang

	```bash
	sglang serve \
	--trust-remote-code \
	--model-path moonshotai/Kimi-K3 \
	--tp-size 8 \
	--dcp-size 8 \
	--mem-fraction-static 0.85 \
	--max-mamba-cache-size 160 \
	--max-running-requests 32 \
	--cuda-graph-max-bs-decode 32 \
	--reasoning-parser kimi_k3 \
	--tool-call-parser kimi_k3 \
	--host 0.0.0.0 \
	--port 30000 \
	--speculative-algorithm DSPARK \
	--speculative-draft-model-path RadixArk/Kimi-K3-DSpark \
	--speculative-dspark-block-size 7
	```

	## Training Details

	- Framework: SpecForge online distillation, with hidden states captured from a frozen Kimi K3 target served by a live SGLang engine. Draft trained from random initialization.
	- Data: 700K regenerated OpenPerfectBlend samples. `thinking_effort=max`, right-truncated at 4096.
	- Schedule: 10 epochs, AdamW (fp32 master weights), peak learning rate `6e-4`, cosine decay to 0, 4% warmup, and gradient clipping at 1.0. Global batch 512, seed 42.
	- Loss: `0.1 CE + 0.9 L1 distillation + 1.0 confidence BCE`, decay gamma 4.0, with 512 sampled anchors per sequence and `block_size=7`.
	- Topology: 4 nodes × 4 GB300 (16 ranks) — 2 × TP8 target replicas, DP2 sampler, FSDP16 `SHARD_GRAD_OP` on the draft, TP-batch scatter. Batch 8 per replica × 32 accumulation steps × 2 replicas = global batch 512.

	## Known Limitations

	- This DSpark checkpoint was trained with a maximum context length of 4,096 tokens, which may lead to reduced acceptance lengths in extreme long-context and agentic use cases. Training for these scenarios is ongoing.