Instructions to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16")
model = AutoModelForCausalLM.from_pretrained("nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16

SGLang

How to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 with Docker Model Runner:
```
docker model run hf.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16
```

dmax123 commited on Jun 3

Commit

07a16c7

verified ·

1 Parent(s): 833b941

Add files using upload-large-folder tool

Browse files

This view is limited to 50 files because it contains too many changes. See raw diff

Files changed (50) hide show

model-00003-of-00225.safetensors +3 -0
model-00007-of-00225.safetensors +3 -0
model-00008-of-00225.safetensors +3 -0
model-00010-of-00225.safetensors +3 -0
model-00011-of-00225.safetensors +3 -0
model-00012-of-00225.safetensors +3 -0
model-00013-of-00225.safetensors +3 -0
model-00017-of-00225.safetensors +3 -0
model-00019-of-00225.safetensors +3 -0
model-00020-of-00225.safetensors +3 -0
model-00021-of-00225.safetensors +3 -0
model-00023-of-00225.safetensors +3 -0
model-00024-of-00225.safetensors +3 -0
model-00029-of-00225.safetensors +3 -0
model-00030-of-00225.safetensors +3 -0
model-00033-of-00225.safetensors +3 -0
model-00035-of-00225.safetensors +3 -0
model-00038-of-00225.safetensors +3 -0
model-00042-of-00225.safetensors +3 -0
model-00043-of-00225.safetensors +3 -0
model-00049-of-00225.safetensors +3 -0
model-00055-of-00225.safetensors +3 -0
model-00056-of-00225.safetensors +3 -0
model-00057-of-00225.safetensors +3 -0
model-00063-of-00225.safetensors +3 -0
model-00067-of-00225.safetensors +3 -0
model-00070-of-00225.safetensors +3 -0
model-00072-of-00225.safetensors +3 -0
model-00073-of-00225.safetensors +3 -0
model-00076-of-00225.safetensors +3 -0
model-00080-of-00225.safetensors +3 -0
model-00085-of-00225.safetensors +3 -0
model-00086-of-00225.safetensors +3 -0
model-00087-of-00225.safetensors +3 -0
model-00097-of-00225.safetensors +3 -0
model-00098-of-00225.safetensors +3 -0
model-00100-of-00225.safetensors +3 -0
model-00102-of-00225.safetensors +3 -0
model-00107-of-00225.safetensors +3 -0
model-00109-of-00225.safetensors +3 -0
model-00112-of-00225.safetensors +3 -0
model-00114-of-00225.safetensors +3 -0
model-00115-of-00225.safetensors +3 -0
model-00119-of-00225.safetensors +3 -0
model-00125-of-00225.safetensors +3 -0
model-00132-of-00225.safetensors +3 -0
model-00134-of-00225.safetensors +3 -0
model-00140-of-00225.safetensors +3 -0
model-00145-of-00225.safetensors +3 -0
model-00147-of-00225.safetensors +3 -0

model-00003-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f27079ca3c1df122ff861dcd210420cfd3cca654e87dc4f8aa31271c4b3adc59
+size 4991252600

model-00007-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:828062eb0be70c4894f408189c36f3a8793ca3eab2aa14e04b07b05505c191a4
+size 4991252600

model-00008-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8498d0639fce51f37f2e202830ff0daca2a549042f3cdb0d7b7ccfba0d7abefc
+size 4991252600

model-00010-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ddeb974d14aa7af9c98c37efe8d5b215e2e66af031effb1e32f972f19b8a898b
+size 4987305680

model-00011-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7f550821e33c9fedbdf0f09163d91c302d7991598489194cb6c3ef227dae9591
+size 4991252464

model-00012-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:49d31ba13fa60dacde3910ec5518555ff825cd76c0c46546137d0c65ec27ffdf
+size 4991252600

model-00013-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:755d08e086bb18bde99c4bc0698468e25dca45f7f389bacad07cb0430dc81ef6
+size 4991252600

model-00017-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f34f0675d3e6cf7017c39e2eb8ad2e2feea1d20235a9d8a9575fc046d660620c
+size 4991252600

model-00019-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9f7b332e1bb5e2803a037eb143f4ea13c4ca0bd3accc4555ff8c7b16656be69e
+size 4987305744

model-00020-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:786c72a0bf08c1491dfc8d0e815f10de4b4535a03a4fa5f8b617645614d4f26e
+size 4991252664

model-00021-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5c558785086ec4e8a378b30ae5a4dbb0555a875f9abf27963ee20e8151b65b97
+size 4991252840

model-00023-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8c9ea0756cb2a57e30b1cbd911a8223899c94d626243ea8a5b9723877ad0122b
+size 4991252840

model-00024-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7d7bbe7ed40b73645e514cd09253ea63c7840d34198961c9550e9ace7a9fbf42
+size 4987305808

model-00029-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5bed010ad950fc4c85aea58d0206b8b8805e20ce6110b1ec83acb223bb9f2ee2
+size 4995462192

model-00030-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:514eabb5fc560b16f7f126678d235fe92af823c216688540715e3168b1240d67
+size 4991252840

model-00033-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:741ac67fe56a3ac3565adcc9b508482f8f6ee568759b004e8a8a86b0b7b96763
+size 4987305848

model-00035-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:45e578e598d7aa225fe2804d6688392738fca411e843263eadfc63c94e28e603
+size 4991252840

model-00038-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:212de44a18bac5a10b459eb4371cf9cc1de44e1d5161120ad0ab5eb36d3633ee
+size 4995491376

model-00042-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2c2d9536dbfd7a351e03ec8470c0e98c305d28b984ed094c8b5dcdefcecf9c9c
+size 4987305872

model-00043-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cc6c02a9dc1ee0d973a8e79b5d3b46ebed15dc85244b15fe94ce6243d40fab70
+size 4991252704

model-00049-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9d9711fc8fa1de6fc8476500886699f9a08ff854a1e231129476446c309ab7b3
+size 4991252840

model-00055-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9f7628440631687f1ec21cd968eaada9bb6290a3603cb715378a65e551f1bad4
+size 4991252840

model-00056-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:46488ce71f24b08656d0b3545932e07584003e895484bf981bb155c40c7f9893
+size 4987305808

model-00057-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0ec38513db60731088ef74ab55e55dda2937c96a53a703425d94ee3f038eb9bf
+size 4991252776

model-00063-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:82ca7c5000e0c6a99fc24101324736b669ce2897f24510d03ab746f3e8ee138f
+size 4991252840

model-00067-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:095c8b513128f18e9dab0a466d57586c25e5436ebb5c401dc1f56a0285ff1850
+size 4991252840

model-00070-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3ede875bd0b7057a595b7c04e61a8a223c87f7c1cf6ff6dd7159b2c6e303a949
+size 4995491376

model-00072-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7e0f926a102ffa6dd16ec8b691957e7b7a976190c6860fc3b50f416c0604f0da
+size 4991252840

model-00073-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5940a6ceaf28455de4166d66bdd6f506df3ef974a45977c929397fbec8313c6e
+size 4991252840

model-00076-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8b89fccfabe80762111b674a38a96472925f46275d6df3e9060701d137235648
+size 4991252840

model-00080-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fa7022ce030f00f16ddce167624e02bd245d7425a920c3d366a4f0c5f40f871c
+size 4991252800

model-00085-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7e1e32258b81239dedf241743aaeabe8fc528817ebbbba4cf1a12f3e3fe911b1
+size 4991252840

model-00086-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ffbf701213def3b4f367a2855f020f64fc892c89b85af2e669e07375b38df00e
+size 4991252840

model-00087-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f7061ac816e79db5b72324594a61a694038aad93b26370ef232b746fd89aa6a2
+size 4991252840

model-00097-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:52c6c9335c07a3e6a689e61b80a83f2e487aae9939defb8f3542886d3764a2ef
+size 4991515400

model-00098-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7915b9141427af35cf4bcdb969b842f8c98b4cf6aa460fd1eb213393bb2c9981
+size 4991252736

model-00100-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:dfd3ec9bc827b4e9b43a33c0677b7c6f20415baa1c4210ad13af86eb3de2b664
+size 4991252840

model-00102-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:810a214f73cf43a40587ef5a379749e45259d9f99490cb90708d11f67f9581b8
+size 4995491376

model-00107-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7f73139339b0deebf92021e92ece4cdba2f1247a717cbb40807ea714da4aec1c
+size 4991252704

model-00109-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7724f1b79a069e56632b43e79fe51f27e3dcdb169023f953f8d6060c7e8984c4
+size 4991252840

model-00112-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9ebb542d43f66d2a4926f20c7de37cac8040017970327329ff4afa47d2734109
+size 4991252808

model-00114-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:39e84d4432398775638fbbcd03805fc490cb444c85d8f5d3defca6be5823e8e6
+size 4991252840

model-00115-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:10a719cf9f89def32132e590b91b18a78ed72027c16a0dc11d221c0624fb07a6
+size 4991515464

model-00119-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:612ed7c69b81bbf311f48ec7cbc43293a7860390a5c52e02abc43eef8c5bb7e2
+size 4991252840

model-00125-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e08b9545ab93e5c3f513d0c59cba8cbe95a90652e7deccef847c8c308cb4afd6
+size 4991252624

model-00132-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bb5f5cc38d965c034375979e7e896d154aa83d11f2422445a92a05ce0a74370a
+size 4991252840

model-00134-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:95b3a3f5fa6e794be7e582bbbb45f1bcbf8ccba572c521c98d24ce50f079dbdc
+size 4995491376

model-00140-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b220f6d3009e3f8dce2e43292ae9ae62c15cde92f419689d11b473c55f5cd277
+size 4991252840

model-00145-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:af8a84f0c97a4abdba3f55733dcf5152b70cb8f3fd995d2fb20c7e77b00dfdc3
+size 4991252840

model-00147-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ae6ab53f63e40d89f6bd6809f7686e66342dcd29d9ea84aa5d2b443d9ab919bb
+size 4991515464