Instructions to use deepseek-ai/DeepSeek-V4-Pro with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use deepseek-ai/DeepSeek-V4-Pro with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="deepseek-ai/DeepSeek-V4-Pro")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-V4-Pro")
model = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-V4-Pro", device_map="auto")

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use deepseek-ai/DeepSeek-V4-Pro with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "deepseek-ai/DeepSeek-V4-Pro"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V4-Pro",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/deepseek-ai/DeepSeek-V4-Pro

SGLang

How to use deepseek-ai/DeepSeek-V4-Pro with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "deepseek-ai/DeepSeek-V4-Pro" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V4-Pro",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "deepseek-ai/DeepSeek-V4-Pro" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V4-Pro",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use deepseek-ai/DeepSeek-V4-Pro with Docker Model Runner:
```
docker model run hf.co/deepseek-ai/DeepSeek-V4-Pro
```

Thank You, DeepSeek! Empower Local Open Source with Smaller Models for Consumer Hardware/感谢 DeepSeek！赋能本地开源，社区强烈呼吁推出适合家用硬件的小型模型 (

#168

by VaLtEc-BoY - opened May 4

Discussion

VaLtEc-BoY

May 4

•

edited May 4

🇺🇸First of all, thank you so much to the DeepSeek-AI team for releasing your models to the world. However, I would like to strongly emphasize the urgent need for smaller models (such as 2B, 4B, 8B, 12B, and 21BA4B). The community critically needs realistic, optimized options for domestic, consumer-grade hardware. Releasing these smaller versions is essential to make local Open Source projects viable and to truly boost independent development. Thank you in advance for the team's hard work and dedication!

🇨🇳 首先，非常感谢 DeepSeek-AI 团队向全世界开源你们的模型。然而，我想强烈强调一下对较小参数模型（例如 2B、4B、8B、12B 和 21BA4B）的迫切需求。社区极其需要能在普通家用硬件上流畅运行的优化版本。发布这些较小的模型对于支持和推动本地开源项目的存活与发展至关重要。提前感谢团队的辛勤付出！

VaLtEc-BoY changed discussion title from hank You, DeepSeek! Empower Local Open Source with Smaller Models for Consumer Hardware/感谢 DeepSeek！赋能本地开源，社区强烈呼吁推出适合家用硬件的小型模型 ( to Thank You, DeepSeek! Empower Local Open Source with Smaller Models for Consumer Hardware/感谢 DeepSeek！赋能本地开源，社区强烈呼吁推出适合家用硬件的小型模型 ( May 4

SeanKa

May 6

DeepSeek is the execution line — it will wipe out those stagnant, overpriced closed-source models.

VaLtEc-BoY

May 6

•

edited May 6

I hope so, and I hope more people see LLADA as a breakthrough, including DeepSeek. LLMs are great, but LLADA could be the revolution, perhaps through a fusion of both models...

Well, nobody is a saint, and nothing is free, but I believe AI is far beyond what is currently being discussed. I've been studying and testing some things in AI, and I can say it is something much bigger than what Big Techs sells us. It should be treated as such by Big Techs, and that means benefiting everyone (humanity), not just a group of technocrat billionaires with unreasonable ambitions.

It should be a global, accessible effort to avoid creating even more social divisions and problems for society. DeepSeek has done a great job fighting this and expanding the power of open-source. They are taking absolute power away from countries and a small group that dictates what tomorrow will look like without caring about the opinion of the vast majority.

Just like other open-source initiatives, I hope people understand that this will influence the next 20 years, and in the short term, the next 6 years... It would be interesting to have more open models and more AI research. It's not easy to fight this—it's a battle against giants—but it is essential that it happens... The more open models, the better!

SeanKa

May 7

thanks for your comment,i am first know some about LLADA

VaLtEc-BoY

May 7

LLADA is a text model that works like an AI image generator. Instead of guessing one word at a time (like 'I + am + curious'), it generates the entire text all at once in seconds. It does this by starting with random noise and 'cleaning' it up to form the words, exactly how image diffusion models work. https://github.com/inclusionAI/LLaDA2.X ,https://github.com/NVlabs/Fast-dLLM,https://deepmind.google/models/gemini-diffusion/,https://arxiv.org/abs/2502.09992.

SeanKa

May 8

3q,that's a new world

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment