Instructions to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16")
model = AutoModelForCausalLM.from_pretrained("nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16

SGLang

How to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 with Docker Model Runner:
```
docker model run hf.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16
```

dmax123 commited on Jun 3

Commit

b3596a8

verified ·

1 Parent(s): 07a16c7

Add files using upload-large-folder tool

Browse files

This view is limited to 50 files because it contains too many changes. See raw diff

Files changed (50) hide show

.gitattributes +1 -0
model-00001-of-00225.safetensors +3 -0
model-00002-of-00225.safetensors +3 -0
model-00004-of-00225.safetensors +3 -0
model-00009-of-00225.safetensors +3 -0
model-00014-of-00225.safetensors +3 -0
model-00015-of-00225.safetensors +3 -0
model-00016-of-00225.safetensors +3 -0
model-00022-of-00225.safetensors +3 -0
model-00026-of-00225.safetensors +3 -0
model-00027-of-00225.safetensors +3 -0
model-00031-of-00225.safetensors +3 -0
model-00032-of-00225.safetensors +3 -0
model-00036-of-00225.safetensors +3 -0
model-00037-of-00225.safetensors +3 -0
model-00041-of-00225.safetensors +3 -0
model-00044-of-00225.safetensors +3 -0
model-00045-of-00225.safetensors +3 -0
model-00046-of-00225.safetensors +3 -0
model-00048-of-00225.safetensors +3 -0
model-00051-of-00225.safetensors +3 -0
model-00058-of-00225.safetensors +3 -0
model-00059-of-00225.safetensors +3 -0
model-00061-of-00225.safetensors +3 -0
model-00062-of-00225.safetensors +3 -0
model-00066-of-00225.safetensors +3 -0
model-00068-of-00225.safetensors +3 -0
model-00069-of-00225.safetensors +3 -0
model-00071-of-00225.safetensors +3 -0
model-00074-of-00225.safetensors +3 -0
model-00075-of-00225.safetensors +3 -0
model-00077-of-00225.safetensors +3 -0
model-00079-of-00225.safetensors +3 -0
model-00081-of-00225.safetensors +3 -0
model-00082-of-00225.safetensors +3 -0
model-00084-of-00225.safetensors +3 -0
model-00089-of-00225.safetensors +3 -0
model-00090-of-00225.safetensors +3 -0
model-00092-of-00225.safetensors +3 -0
model-00093-of-00225.safetensors +3 -0
model-00094-of-00225.safetensors +3 -0
model-00103-of-00225.safetensors +3 -0
model-00104-of-00225.safetensors +3 -0
model-00105-of-00225.safetensors +3 -0
model-00106-of-00225.safetensors +3 -0
model-00108-of-00225.safetensors +3 -0
model-00110-of-00225.safetensors +3 -0
model-00113-of-00225.safetensors +3 -0
model-00117-of-00225.safetensors +3 -0
model-00118-of-00225.safetensors +3 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

model-00001-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9270ac4f126fae66781d382632ed0b54c02ad3a7736d1ca58871cc203a8ca19b
+size 4983097968

model-00002-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:90b5e986bb3820797deebaa757d48271895afb3408d5450d5e4f51be54010395
+size 4991252496

model-00004-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3b28987eb68d32d11bc4237d25ab89d24a288e01b95131646b3d4a76cf7e28b7
+size 4991252600

model-00009-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b7161d484ea6f4fee2152b41755619e706ae00f6eeaadca47eb76dc0b150fe01
+size 4991252600

model-00014-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4b06e119d8707579995082e5de17ee25f938d0ae7f6adac9b283d36caac72c5b
+size 4991252600

model-00015-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5e889b8fbf59760bb39d7f79cfc82adcb77bdd1c872bd181da35800d490470c6
+size 4991515208

model-00016-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:20e036c89b31c0f8139001c0d07358b02c5b3ce1a582c05eafac64e5b84c809c
+size 4991252560

model-00022-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cf00047e63ad9389cb7a9c31aac8eeea1dd4bbebe3df2d73953d49a657c96a6c
+size 4991252840

model-00026-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0faf7585a39a5f0f881c5d6d28b7508bf4a5f326b7184e0b33bcd4948b89aabf
+size 4991252840

model-00027-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:57821988acc46522e5ac9aed1444be561cd34449286ef8864796bbe79ecc75d4
+size 4991252840

model-00031-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:342c0e794469b3ded324e4bc4997e496bcec33352559aec7a83542827c71d277
+size 4991252840

model-00032-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d5af25d8ef54401de27f73c102fac882c929197c24912cf06c7946f0407ad81e
+size 4991252840

model-00036-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:673fa39058b4c97354151599ff2dfc28c1c4940e365bcc1ba7e370d2d62a0020
+size 4991252840

model-00037-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:711c9969e344217cf0b1e4f11393abc95b27454c59795eb558141ff5f6334a5f
+size 4962095640

model-00041-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4a89987d78b12a06f4b082a0483520b1a546ae20b19980e6f30feb1274535c7d
+size 4991252840

model-00044-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6af0faed663ff5a1ec751be83dda4cc535adb48a44cfe7bb578245edfef1f60e
+size 4991252840

model-00045-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a1c6cd14c3ffb5d579a9aa76b2c61a67c9751207a63c7fd338c80273a1f6c047
+size 4991252840

model-00046-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e499ae21ed0d761a460cd30572ab366f43da1a90e2df7811692241fc5e17f200
+size 4991252840

model-00048-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1bde2d46e5aeb896b733332237821ffd62590f92adb223ebbdb88378b8669013
+size 4991252800

model-00051-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f66e52dd4bb53cb36b219de4c7440328688f54d8058dfc0c05d2f1383021251d
+size 4987305912

model-00058-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:24c04db02d67a15f4f4054848b3bef3b39dc6b582152746579b7b7037e51c5e6
+size 4991252840

model-00059-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1997ab84b898faef5d88b319dde62464e6c5c44d5ce13d2b02070eab599f58b4
+size 4991252840

model-00061-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:35176872fafd2a439cda86c007286c963aa2047b9279cdd9e72b38a889ce188d
+size 4991252624

model-00062-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8a77b9a547de126f11dab64c593493b1cf524e8a55b96a6791bdf4ce75b4a66c
+size 4991252840

model-00066-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cef7e5835f6d18012dcc7744dffa0023509a1df7f863a9a95ab07ff1455dfb44
+size 4991252736

model-00068-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cae83c0d72194b77f1ff49114c13796055b84d02371f6dcef309f7d70a5e1eca
+size 4991252840

model-00069-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a507cd6ea157cc48f4d18dfc671c300677026d527f117f23b6bcd36909bfa4e3
+size 4941123984

model-00071-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fe709dcbabbedb8a1f3a450dc580727e14e1c65d7d499e893197f98b24823cd3
+size 4991252840

model-00074-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c4be5c40104fdbac360013c0df6dc8750e1945b20e60cb01ebb30a60e5849a73
+size 4987305872

model-00075-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:884979316c0d314048b9a1e7dfc37eecbbb1cb889805d5eac62251be4233959b
+size 4991252704

model-00077-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b00013e988a63db30661235204636d123bae5aee63724a86d2eb956caf773124
+size 4991252840

model-00079-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bfac40af6fcf5d237dd02a19369ec7fe6a14765b08a18a83efdd81e0daa14603
+size 4991515392

model-00081-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:55e301e88d1eeef3cb152889711e8b33c5755a7ba7043e55e7ae3b28f5115954
+size 4991252840

model-00082-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c947c9134e4b2fb324e706b0d856f37a97876581468706dd0a0b92c9076030a3
+size 4991252840

model-00084-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1e0160913f06874c1c86a10ffd20d75e290b2543fbae4a27ea88643c380c4527
+size 4991252664

model-00089-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fad57551a678c0cb4cde55c2794fcc8641bff294b6146703e6a8fc5f24b110db
+size 4991252776

model-00090-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d8c50a1bfb3a9c56963b7b15d5bbbaec028a64824f8c79c57a5cb510e6e1656a
+size 4991252840

model-00092-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e916d8b0d0995504d2dd24d15dcaeca98e55308aa4f7969867140e9f74cc2744
+size 4987305952

model-00093-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c25f645a539dbb2de69f1bdc504ed321560a9ca62192e670e96754626242d377
+size 4991252624

model-00094-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bd759a3aefd0c789ffd140162886a73f6e991ad5a3a4cdcd500ea92bbb94a5b4
+size 4991252840

model-00103-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:651b4b561a146ef0b4fd7f32d02124e4457a9536772b777028a005015cd87fcb
+size 4991252840

model-00104-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3cb884ca67523841d05c8b5dd7d1491855d08a6e8b021c32b1eec5d4ff487b74
+size 4991252840

model-00105-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d09e5d24d3a301baea32f32d9b9faec02f87dbd00e9fe0f938ef145de9a63177
+size 4991252840

model-00106-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e94e7fddb7ce51d9ca91d40f4fb0eea03403afbabd8cc08a4c9b9d6838ccecef
+size 4987305872

model-00108-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:48409d3cca01d652cf6aae9d05f32b3d6cc4f34e50c47451f714c4a2401b9624
+size 4991252840

model-00110-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ba1634849f441b95feaf044345a9c9ea7873b1bbca57d8e245d01f713f7c02e3
+size 4991252840

model-00113-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a762b67056ef04a789894fe2fd9fb95562602a452906bdef74831866eb1513d3
+size 4991252840

model-00117-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:91347c586dd9350a83178e9d4518a3917c4e22311a51a0c20df78bd6e0fac69b
+size 4991252840

model-00118-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8771f5232ef5dc0ab012a59a5e392453fd39495d756f48752224222aa5e03c61
+size 4991252840