Instructions to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4")
model = AutoModelForCausalLM.from_pretrained("nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4

SGLang

How to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 with Docker Model Runner:
```
docker model run hf.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4
```

dmax123 commited on Jun 3

Commit

0033442

verified ·

1 Parent(s): 183edf1

Add files using upload-large-folder tool

Browse files

This view is limited to 50 files because it contains too many changes. See raw diff

Files changed (50) hide show

config.json +3 -0
model-00001-of-00113.safetensors +3 -0
model-00002-of-00113.safetensors +3 -0
model-00003-of-00113.safetensors +3 -0
model-00004-of-00113.safetensors +3 -0
model-00005-of-00113.safetensors +3 -0
model-00007-of-00113.safetensors +3 -0
model-00008-of-00113.safetensors +3 -0
model-00009-of-00113.safetensors +3 -0
model-00010-of-00113.safetensors +3 -0
model-00011-of-00113.safetensors +3 -0
model-00012-of-00113.safetensors +3 -0
model-00013-of-00113.safetensors +3 -0
model-00014-of-00113.safetensors +3 -0
model-00015-of-00113.safetensors +3 -0
model-00016-of-00113.safetensors +3 -0
model-00017-of-00113.safetensors +3 -0
model-00019-of-00113.safetensors +3 -0
model-00020-of-00113.safetensors +3 -0
model-00021-of-00113.safetensors +3 -0
model-00022-of-00113.safetensors +3 -0
model-00024-of-00113.safetensors +3 -0
model-00025-of-00113.safetensors +3 -0
model-00026-of-00113.safetensors +3 -0
model-00028-of-00113.safetensors +3 -0
model-00029-of-00113.safetensors +3 -0
model-00030-of-00113.safetensors +3 -0
model-00031-of-00113.safetensors +3 -0
model-00032-of-00113.safetensors +3 -0
model-00034-of-00113.safetensors +3 -0
model-00036-of-00113.safetensors +3 -0
model-00037-of-00113.safetensors +3 -0
model-00038-of-00113.safetensors +3 -0
model-00039-of-00113.safetensors +3 -0
model-00040-of-00113.safetensors +3 -0
model-00041-of-00113.safetensors +3 -0
model-00042-of-00113.safetensors +3 -0
model-00043-of-00113.safetensors +3 -0
model-00044-of-00113.safetensors +3 -0
model-00045-of-00113.safetensors +3 -0
model-00046-of-00113.safetensors +3 -0
model-00047-of-00113.safetensors +3 -0
model-00048-of-00113.safetensors +3 -0
model-00049-of-00113.safetensors +3 -0
model-00050-of-00113.safetensors +3 -0
model-00051-of-00113.safetensors +3 -0
model-00052-of-00113.safetensors +3 -0
model-00053-of-00113.safetensors +3 -0
model-00056-of-00113.safetensors +3 -0
model-00057-of-00113.safetensors +3 -0

config.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8821818c40611f93342034cdc0e349492df9bc622c589503fd77c06943b79d47
+size 10793633

model-00001-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:eea43a334457de1c87902bcd9c5621d206aba8e8e45fe39abc7752f31b6c6f5b
+size 2569247736

model-00002-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a70c0799beb68df72ed6ff9488b0ddcc99d951b1da80ddfb5c9d83ebe867647d
+size 6291987336

model-00003-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9fffaafe53409d04946c78c578a922c1b67c361c67ac478fd22bc9e6781536ec
+size 421763912

model-00004-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:592b13785d68d017207803a6d1641bbe14e0eec1b2e218968cb5fb8b1e0573b8
+size 6291987336

model-00005-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:88f9c8bd4f3b6127142e87c611c42ef1799ac11c53810c4ea9a61719206281b7
+size 421763912

model-00007-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:71ec214b8316d8101f5e13ee956cbb8007bcf81034da50a9b99781dea5d85623
+size 421763912

model-00008-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bd2116d5738fa4bed76a9c95cd6d14e3d8a8a7b0adee3407eb0dad522942330f
+size 276841224

model-00009-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ef98fcfa9848c9a8a7aa90486b55aeccb567998e03c54f3f7937e5799df5fd28
+size 6291987336

model-00010-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1140fbff1673a46389ea18004c352dfad0cc4892f185fd8704a8c68fbd9bea08
+size 421763912

model-00011-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:96c53f7078be52a5796a2e8847ab7160c6583e3ef96f1de825dd184105421d20
+size 6291991440

model-00012-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d3845fa29dd3f2336aed2eb573cb6ebc59f121a0230b87ad0dbb41a063c1dd4b
+size 421763928

model-00013-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b9124c8e5d80f0095665fa879f284397093559a83eb8ee9a47164676684c6604
+size 6291991440

model-00014-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7f132633a295c8b28d53f12ed6d122af52d1cb267054523c5fb804f422aaee05
+size 421763928

model-00015-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9d53f07d77a0ed124cc6b7c1a69423a4ee36d43c157b1c9a18eb50fb7c83c46b
+size 276841232

model-00016-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c35cad68b41ef1b887904a842299a84aa96f533473be27993a00a0cb51377ab3
+size 6291991440

model-00017-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a0c59d3b8b1dbd9d63ac0b652c2ee36e4a29ac390e6c0c5ebfc138bf582ceff2
+size 421763928

model-00019-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:99b58ae9d45c4d7bc29b946297c688414538d8cddbc67909a88d7f1f6cbffe06
+size 421763928

model-00020-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1e3733eb9570003c92133281eb0d8109a4e82a5bd4d93cc501b5e144ebc81723
+size 6291991440

model-00021-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3e1f43f6071ddf9bb5a918ac7d541bd7ea378ecdb9ad197513f80b516e2a49fe
+size 421763928

model-00022-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0c7719163949d3f490a043d59df4417548fa4e73200020cab6bf1501112298fa
+size 6291991440

model-00024-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6ac7a9948ff6055abf07000ecfc19165917231713e131c4c4004f1ca9f8330c7
+size 276841232

model-00025-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3cb2767ce24a38bd8c0f889f19d1e3eaa2b808164b8f0f3e7d7cc9b8ef3acf95
+size 6291991440

model-00026-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3abd708b729e34943b57c5ab8984db3737807c6d9297696df30f40ea7f8111d3
+size 421763928

model-00028-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a2613c5ffbdf2934ee1fb6df2ae7282070cfca0c8f6b97310b62a907d33fddfc
+size 421763928

model-00029-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:99285d7eb618bc7e89e5b0fdca4af47899bab40e93df6b9c645b9e047f88cfb3
+size 6291991440

model-00030-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3bf22148a517dc5f811a0054c0c5f027abc340de8edb5f904779903c88470159
+size 421763928

model-00031-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a22ed013eb9afa039d208586b4a3598db5b8d35d4a3f4fb5fb5003df98f2e724
+size 6291991440

model-00032-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d391482ae86947961526da16eca6d16bb9930c1f75798dd851fbf88c88ddd53e
+size 421763928

model-00034-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1e72fefea48822bad9489f883beded9e694360b65f2f67e90ea82713c8b4c89d
+size 6291991440

model-00036-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e502e1fe71daa89523df0cce40e898c5c5df4a8dc53db98c3ad826b71e38ef5c
+size 6291991440

model-00037-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:176f338bf8ac86ef72a7db1722e14251928463f5174a2fee31a55af298612c80
+size 421763928

model-00038-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:586cd1ffdf00bc621513e1d2e6facd8d33392438d7458cd8c0fa1a4c08f4ea4f
+size 6291991440

model-00039-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5525f525fba372913db0db9e29da56846c0153cc360ae27ce59d5f0b633c834c
+size 421763928

model-00040-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d983f70a584a91d491ebfd51d50fb0583c2b6f603585569b93ffa13b4b48bea0
+size 276841232

model-00041-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:88a62767e9a3ced169b53844614cb226f4c86bb90539beac760527843df5971e
+size 6291991440

model-00042-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7c5d7478739bfafbe3a20d1606f636df7dddc2e13b26672ba24903b6286b3be2
+size 421763928

model-00043-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cbabb571cb86df339c5d7943f5bdae9f45cfad9c228e7609f5b1ffc982c6070c
+size 6291991440

model-00044-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:82459a9e7010dafaea1c81b833064f1f6aacca706a16e8a37a23389380dabb5d
+size 421763928

model-00045-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:11a8a5bdb179389678284c4788cb0fd3361668944d6c866b08343443e75f2823
+size 6291991440

model-00046-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d2e62cfb888c7e800bf502f69090c40276243e6275220f68877ce7df1436290b
+size 421763928

model-00047-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4012f203f9605442c075b17be452dd064f28a9bdf9f97606aba2a088685b4f40
+size 6291991440

model-00048-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d88bf46960ce8569597be33962d767c38b6ddcc05015f1b4f11f74de79a3cb7c
+size 421763928

model-00049-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:754eb83ec55a4f0d6e8a2053354fcea88774a3a86b9e6c1ce46dfe9151f3a535
+size 276841232

model-00050-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:165f58f066adef2c1743ab611495b533351b693cc9355279c83a576321099b47
+size 6291991440

model-00051-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:64fcb7a0b1e161b88e6fd0f1a6f53d84f352e154fc4467a41a797ca070fddf4a
+size 421763928

model-00052-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:76591a897e50f97af226cbb22a0354279eb063a7d5abe552ec8f4543be8ec030
+size 6291991440

model-00053-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:961e0016af24bc41954fffb86d710c0ae02caf40e05a973023a058d1fe009029
+size 421763928

model-00056-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9de654860536229a6831e83dba7094a6c59fc8a320f17900c412c02bc2f17b51
+size 6291991440

model-00057-of-00113.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:45f70ad3e03647634fc4c9546c9937246a0949a93ec7fe777d67349bb402c6ed
+size 421763928