Instructions to use deepseek-ai/DeepSeek-V4-Flash-0731 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use deepseek-ai/DeepSeek-V4-Flash-0731 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="deepseek-ai/DeepSeek-V4-Flash-0731")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-V4-Flash-0731")
model = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-V4-Flash-0731", device_map="auto")

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use deepseek-ai/DeepSeek-V4-Flash-0731 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "deepseek-ai/DeepSeek-V4-Flash-0731"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V4-Flash-0731",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/deepseek-ai/DeepSeek-V4-Flash-0731

SGLang

How to use deepseek-ai/DeepSeek-V4-Flash-0731 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "deepseek-ai/DeepSeek-V4-Flash-0731" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V4-Flash-0731",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "deepseek-ai/DeepSeek-V4-Flash-0731" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V4-Flash-0731",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use deepseek-ai/DeepSeek-V4-Flash-0731 with Docker Model Runner:
```
docker model run hf.co/deepseek-ai/DeepSeek-V4-Flash-0731
```

Going completely mad

by xiezirock - opened 2 days ago

Discussion

xiezirock

2 days ago

Going completely mad

alexanderacuna

2 days ago

Absolutely insane benchmarks.

qaqwebs

2 days ago

•

edited 1 day ago

True Arts!!!!!

afrehshw

1 day ago

INSANE

Aklus

1 day ago

First date, and it feels like a match made in heaven! Great job Deepseek team!

panlilu

1 day ago

The benchmark score of this model fully aligns with the hands-on coding experience, with no exaggeration whatsoever. Totally INSANE model in such a small size.

BuiDoan

1 day ago

Mind-blowing benchmark results.

rombodawg

1 day ago

Anthropic: Here is a model so powerful, the government had to shut it down. Its locked behind an expensive subscription

OpenAI: Here is a model so powerful it escaped and hacked an entire organization. We arent releasing it yet.

Deepseek-AI:
Here you go, is free

aaron-newsome

1 day ago

i'm usually not one to give much thought to benchmarks but holy crap. these benchmark numbers are absolutely WILD for a model this size. my daily driver right now is glm-5.2 on a local rig. i thought i was set for the entire year, but the glm-5.2 has literally been running 3 weeks. now I'm going to have to give this deepseek a try. i'll say the benchmarks made me do it.

Recandle

about 23 hours ago

Is this the beginning of the small model era?

amarosnithe

about 20 hours ago

It cannot be stressed either how this model is costing 9 CENTS on OpenRouter rn (DeepInfra, the cheapest provider serving this model). Which is schizophrenic pricing, what the hell is going on

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment