Instructions to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16")
model = AutoModelForCausalLM.from_pretrained("nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16

SGLang

How to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 with Docker Model Runner:
```
docker model run hf.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16
```

dmax123 commited on Jun 3

Commit

833b941

verified ·

1 Parent(s): 2fd5f83

Add files using upload-large-folder tool

Browse files

Files changed (50) hide show

__init__.py +0 -0
chat_template.jinja +195 -0
config.json +180 -0
generation_config.json +13 -0
model-00006-of-00225.safetensors +3 -0
model-00018-of-00225.safetensors +3 -0
model-00025-of-00225.safetensors +3 -0
model-00034-of-00225.safetensors +3 -0
model-00039-of-00225.safetensors +3 -0
model-00040-of-00225.safetensors +3 -0
model-00047-of-00225.safetensors +3 -0
model-00050-of-00225.safetensors +3 -0
model-00052-of-00225.safetensors +3 -0
model-00054-of-00225.safetensors +3 -0
model-00064-of-00225.safetensors +3 -0
model-00065-of-00225.safetensors +3 -0
model-00078-of-00225.safetensors +3 -0
model-00083-of-00225.safetensors +3 -0
model-00088-of-00225.safetensors +3 -0
model-00091-of-00225.safetensors +3 -0
model-00095-of-00225.safetensors +3 -0
model-00096-of-00225.safetensors +3 -0
model-00099-of-00225.safetensors +3 -0
model-00101-of-00225.safetensors +3 -0
model-00111-of-00225.safetensors +3 -0
model-00116-of-00225.safetensors +3 -0
model-00122-of-00225.safetensors +3 -0
model-00126-of-00225.safetensors +3 -0
model-00127-of-00225.safetensors +3 -0
model-00130-of-00225.safetensors +3 -0
model-00133-of-00225.safetensors +3 -0
model-00139-of-00225.safetensors +3 -0
model-00146-of-00225.safetensors +3 -0
model-00150-of-00225.safetensors +3 -0
model-00152-of-00225.safetensors +3 -0
model-00157-of-00225.safetensors +3 -0
model-00158-of-00225.safetensors +3 -0
model-00164-of-00225.safetensors +3 -0
model-00168-of-00225.safetensors +3 -0
model-00172-of-00225.safetensors +3 -0
model-00174-of-00225.safetensors +3 -0
model-00193-of-00225.safetensors +3 -0
model-00197-of-00225.safetensors +3 -0
model-00205-of-00225.safetensors +3 -0
model-00217-of-00225.safetensors +3 -0
model-00225-of-00225.safetensors +3 -0
model.safetensors.index.json +0 -0
special_tokens_map.json +30 -0
tokenizer_config.json +0 -0
ultra_v3_reasoning_parser.py +28 -0

__init__.py ADDED Viewed

File without changes

chat_template.jinja ADDED Viewed

	@@ -0,0 +1,195 @@

+{% macro render_extra_keys(json_dict, handled_keys) %}
+    {%- if json_dict is mapping %}
+        {%- for json_key in json_dict if json_key not in handled_keys %}
+            {%- if json_dict[json_key] is mapping or (json_dict[json_key] is sequence and json_dict[json_key] is not string) %}
+                {{- '\n<' ~ json_key ~ '>' ~ (json_dict[json_key] | tojson | safe) ~ '</' ~ json_key ~ '>' }}
+            {%- else %}
+                {{-'\n<' ~ json_key ~ '>' ~ (json_dict[json_key] | string) ~ '</' ~ json_key ~ '>' }}
+            {%- endif %}
+        {%- endfor %}
+    {%- endif %}
+{% endmacro %}
+{%- set enable_thinking = enable_thinking if enable_thinking is defined else True %}
+{%- set medium_effort = medium_effort if medium_effort is defined else False %}
+{%- set truncate_history_thinking = truncate_history_thinking if truncate_history_thinking is defined else True %}
+{%- set ns = namespace(last_user_idx = -1) %}
+{%- set loop_messages = messages %}
+{%- for m in loop_messages %}
+  {%- if m["role"] == "user" %}
+    {%- set ns.last_user_idx = loop.index0 %}
+  {%- endif %}
+{%- endfor %}
+{%- if messages[0]["role"] == "system" %}
+    {%- set system_message = messages[0]["content"] %}
+    {%- set loop_messages = messages[1:] %}
+{%- else %}
+    {%- set system_message = "" %}
+    {%- set loop_messages = messages %}
+{%- endif %}
+{%- if not tools is defined %}
+    {%- set tools = [] %}
+{%- endif %}
+{%- set ns = namespace(last_user_idx = -1) %}
+{%- for m in loop_messages %}
+  {%- if m["role"] == "user" %}
+    {%- set ns.last_user_idx = loop.index0 %}
+  {%- endif %}
+{%- endfor %}
+{%- if system_message is defined %}
+    {{- "<|im_start|>system\n" + system_message }}
+{%- else %}
+    {%- if tools is iterable and tools | length > 0 %}
+        {{- "<|im_start|>system\n" }}
+    {%- endif %}
+{%- endif %}
+{%- if tools is iterable and tools | length > 0 %}
+    {%- if system_message is defined and system_message | length > 0 %}
+        {{- "\n\n" }}
+    {%- endif %}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n" }}
+    {{- "<tools>" }}
+    {%- for tool in tools %}
+        {%- if tool.function is defined %}
+            {%- set tool = tool.function %}
+        {%- endif %}
+        {{- "\n<function>\n<name>" ~ tool.name ~ "</name>" }}
+        {%- if tool.description is defined %}
+            {{- '\n<description>' ~ (tool.description | trim) ~ '</description>' }}
+        {%- endif %}
+        {{- '\n<parameters>' }}
+        {%- if tool.parameters is defined and tool.parameters is mapping and tool.parameters.properties is defined and tool.parameters.properties is mapping %}
+            {%- for param_name, param_fields in tool.parameters.properties|items %}
+                {{- '\n<parameter>' }}
+                {{- '\n<name>' ~ param_name ~ '</name>' }}
+                {%- if param_fields.type is defined %}
+                    {{- '\n<type>' ~ (param_fields.type | string) ~ '</type>' }}
+                {%- endif %}
+                {%- if param_fields.description is defined %}
+                    {{- '\n<description>' ~ (param_fields.description | trim) ~ '</description>' }}
+                {%- endif %}
+                {%- if param_fields.enum is defined %}
+                    {{- '\n<enum>' ~ (param_fields.enum | tojson | safe) ~ '</enum>' }}
+                {%- endif %}
+                {%- set handled_keys = ['name', 'type', 'description', 'enum'] %}
+                {{- render_extra_keys(param_fields, handled_keys) }}
+                {{- '\n</parameter>' }}
+            {%- endfor %}
+        {%- endif %}
+        {% set handled_keys = ['type', 'properties', 'required'] %}
+        {{- render_extra_keys(tool.parameters, handled_keys) }}
+        {%- if tool.parameters is defined and tool.parameters.required is defined %}
+            {{- '\n<required>' ~ (tool.parameters.required | tojson | safe) ~ '</required>' }}
+        {%- endif %}
+        {{- '\n</parameters>' }}
+        {%- set handled_keys = ['type', 'name', 'description', 'parameters'] %}
+        {{- render_extra_keys(tool, handled_keys) }}
+        {{- '\n</function>' }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+{%- endif %}
+{%- if system_message is defined %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if tools is iterable and tools | length > 0 %}
+        {{- '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in loop_messages %}
+    {%- if message.role == "assistant" %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is string and message.reasoning_content | trim | length > 0 %}
+            {%- set content = "<think>\n" ~ message.reasoning_content ~ "</think>" ~ (message.content | default('', true)) %}
+        {%- else %}
+            {%- set content = message.content | default('', true) %}
+            {%- if content is string -%}
+                {%- if '<think>' not in content and '</think>' not in content -%}
+                    {%- set content = "<think></think>" ~ content -%}
+                {%- endif -%}
+            {%- else -%}
+                {%- set content = content -%}
+            {%- endif -%}
+        {%- endif %}
+        {%- if message.tool_calls is defined and message.tool_calls is iterable and message.tool_calls | length > 0 %}
+            {{- '<|im_start|>assistant\n' }}
+                {%- set include_content = not (truncate_history_thinking and loop.index0 < ns.last_user_idx) %}
+                {%- if content is string and content | trim | length > 0 %}
+                    {%- if include_content %}
+                        {{- (content | trim) ~ '\n' -}}
+                    {%- else %}
+                        {%- set c = (content | string) %}
+                        {%- if '</think>' in c %}
+                            {%- set c = c.split('</think>')[-1] %}
+                        {%- elif '<think>' in c %}
+                            {%- set c = c.split('<think>')[0] %}
+                        {%- endif %}
+                        {%- set c = "<think></think>" ~ c %}
+                        {%- if c | length > 0 %}
+                            {{- c ~ '\n' -}}
+                        {%- endif %}
+                    {%- endif %}
+                {%- else %}
+                    {{- "<think></think>" -}}
+                {%- endif %}
+                {%- for tool_call in message.tool_calls %}
+                    {%- if tool_call.function is defined %}
+                        {%- set tool_call = tool_call.function %}
+                    {%- endif %}
+                    {{- '<tool_call>\n<function=' ~ tool_call.name ~ '>\n' -}}
+                        {%- if tool_call.arguments is defined %}
+                            {%- for args_name, args_value in tool_call.arguments|items %}
+                                {{- '<parameter=' ~ args_name ~ '>\n' -}}
+                                    {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                                {{- args_value ~ '\n</parameter>\n' -}}
+                            {%- endfor %}
+                        {%- endif %}
+                    {{- '</function>\n</tool_call>\n' -}}
+                {%- endfor %}
+                {{- '<|im_end|>\n' }}
+        {%- else %}
+            {%- if not (truncate_history_thinking and loop.index0 < ns.last_user_idx) %}
+                {{- '<|im_start|>assistant\n' ~ (content | default('', true) | string | trim) ~ '<|im_end|>\n' }}
+            {%- else %}
+                {%- set c = (content | default('', true) | string) %}
+                {%- if '<think>' in c and '</think>' in c %}
+                    {%- set c = "<think></think>" ~ c.split('</think>')[-1] %}
+                {%- endif %}
+                {%- set c = c | trim %}
+                {%- if c | length > 0 %}
+                    {{- '<|im_start|>assistant\n' ~ c ~ '<|im_end|>\n' }}
+                {%- else %}
+                    {{- '<|im_start|>assistant\n<|im_end|>\n' }}
+                {%- endif %}
+            {%- endif %}
+        {%- endif %}
+    {%- elif message.role == "user" or message.role == "system" %}
+        {{- '<|im_start|>' + message.role + '\n' }}
+        {%- set content = message.content | string %}
+        {%- if message.role == "user" and loop.index0 == ns.last_user_idx and medium_effort %}
+            {{- content + '\n\n{reasoning effort: efficient}' }}
+        {%- else %}
+            {{- content }}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user\n' }}
+        {%- endif %}
+        {{- '<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>\n' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {%- if enable_thinking %}
+        {{- '<|im_start|>assistant\n<think>\n' }}
+    {%- else %}
+        {{- '<|im_start|>assistant\n<think></think>' }}
+    {%- endif %}
+{%- endif %}

config.json ADDED Viewed

	@@ -0,0 +1,180 @@

+{
+  "architectures": [
+    "NemotronHForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "chunk_size": 128,
+  "conv_kernel": 4,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "expand": 2,
+  "head_dim": 128,
+  "hidden_dropout": 0.0,
+  "hidden_size": 8192,
+  "initializer_range": 0.02,
+  "intermediate_size": 5120,
+  "layer_norm_epsilon": 1e-05,
+  "layers_block_type": [
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "attention",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "attention",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "attention",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "attention",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "attention",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "attention",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "attention",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "attention",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "attention",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "attention",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "attention",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "attention",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe",
+    "mamba",
+    "moe"
+  ],
+  "mamba_head_dim": 64,
+  "mamba_hidden_act": "silu",
+  "mamba_num_heads": 256,
+  "mamba_proj_bias": false,
+  "mamba_ssm_cache_dtype": "float32",
+  "max_position_embeddings": 262144,
+  "mlp_bias": false,
+  "mlp_hidden_act": "relu2",
+  "model_type": "nemotron_h",
+  "moe_intermediate_size": 5120,
+  "moe_latent_size": 2048,
+  "moe_shared_expert_intermediate_size": 10240,
+  "moe_shared_expert_overlap": false,
+  "mtp_layers_block_type": [
+    "attention",
+    "moe"
+  ],
+  "n_group": 1,
+  "n_groups": 8,
+  "n_routed_experts": 512,
+  "n_shared_experts": 1,
+  "norm_eps": 1e-05,
+  "norm_topk_prob": true,
+  "num_attention_heads": 64,
+  "num_experts_per_tok": 22,
+  "num_key_value_heads": 2,
+  "num_logits_to_keep": 1,
+  "num_nextn_predict_layers": 1,
+  "pad_token_id": 0,
+  "partial_rotary_factor": 1.0,
+  "rescale_prenorm_residual": true,
+  "residual_in_fp32": false,
+  "rope_theta": 10000,
+  "routed_scaling_factor": 5.0,
+  "sliding_window": null,
+  "ssm_state_size": 128,
+  "tie_word_embeddings": false,
+  "time_step_floor": 0.0001,
+  "time_step_limit": [
+    0.0,
+    Infinity
+  ],
+  "time_step_max": 0.1,
+  "time_step_min": 0.001,
+  "topk_group": 1,
+  "transformers_version": "4.57.6",
+  "use_bias": false,
+  "use_cache": true,
+  "use_conv_bias": true,
+  "use_mamba_kernels": true,
+  "vocab_size": 131072
+}

generation_config.json ADDED Viewed

	@@ -0,0 +1,13 @@

+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "do_sample": true,
+  "eos_token_id": [
+    2,
+    11
+  ],
+  "pad_token_id": 0,
+  "temperature": 1.0,
+  "top_p": 0.95,
+  "transformers_version": "4.57.6"
+}

model-00006-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:319a9408a84b564ef3689ca6c9141e8a4e28103a4269cd491ab446bfb26a257f
+size 4995491168

model-00018-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e9af5e48c78baf57a51ab1408e09f17d8a4fd0d6a427b3a6eb953b1cba2bb44d
+size 4991252600

model-00025-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6025f91271129e193bd4b2c0509a60d01533286349103d794e3fab88c155c373
+size 4991252776

model-00034-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7185a2149133c63b3bfd8b07e23bc2f2423c90ece859fb3c0c21e0a9d9d5b0fd
+size 4991252736

model-00039-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:67646caa4b60c5de98d8bc356df9c44a7111fe4b375238b28e8322773c4ac3fa
+size 4991252840

model-00040-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:779e171252824eb706ed882b6efe8d31093d6556807c98d646ef2427be3008d6
+size 4991252840

model-00047-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6f9137068f7bccd8d532a21b48f8aa9285569df10ce74722f690103fda64e95a
+size 4991515392

model-00050-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5bc2e621890f34bb425be2b83582870a51c7a762e4efe58a6364e3642f9be55a
+size 4991252840

model-00052-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7bf7fed796bb204d4218ebe08f23dbc71079a8fb2b3e0e05cf843cc228c46773
+size 4991252664

model-00054-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7262b47af4d740dd41fb135051f97a851259676452f622cc75ba333e4fc02d2f
+size 4991252840

model-00064-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8b1861e95a6b4bceed1e7b85edf3122b107a9cd2aefa715bde59ce0a0958c8e1
+size 4991252840

model-00065-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:81df978797e268583bb59ffc32a81eb6ed8e8c6a0870ac0b4968dd64605f8c54
+size 4991515400

model-00078-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b4e7dbfb487e34d7af04f4035ade1b08e4e60f3f3ce9b99b1d9c89b4407bcf50
+size 4991252840

model-00083-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fdff633053a68c6fecf8f3401177e56d6e8cdba2c39029c853dd646715b06d8d
+size 4987305912

model-00088-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7029e622c0e8fae86a4c098504ea0289ee274dd6271a5e38152faa8dddd6b415
+size 4987305808

model-00091-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8c094961fef375b72afc345babe98638c8c74fe5e1891a65ecd894ad990d0327
+size 4991252840

model-00095-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:62ab3b512b57a73c9c3ec81f4da26dda82787870ea9e7a0ffdfc440cb4f13dfd
+size 4991252840

model-00096-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cb11da5e84c0fa018158aaddef1cbbcffd0e024a02bfefc110c62d68643b7c81
+size 4991252840

model-00099-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a43250768d5ce856b323cc936210d20f1b5d17ec1a74191f2beb6cbff467b5c6
+size 4991252840

model-00101-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:657eee2c7223268d14d4e40571d3835f5f976d34ba0059defad900132db4119c
+size 4941123984

model-00111-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8a8b84f1855c2b9c777350e06f56be5a7a6a8fb2cd54f316f0adfd6debd96283
+size 4987305824

model-00116-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ec30f5418bece29fb0c97925e418974380c03742fcc478ded80267ba8edef8c4
+size 4991252664

model-00122-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bb593db2391122ab0a80d0819e2891481cdfec58f62e89dcf140f0ea53f7b618
+size 4991252840

model-00126-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c69b61ec9eaac7964b375a751b41cfe444c64e70b2392f5563dc93a7ae2cd4c7
+size 4991252840

model-00127-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:91ddf90c768774fcf954f34a3b715fabcd0c0d8d46aa8b44d7eeccda40ac1e1b
+size 4991252840

model-00130-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6d1db40df631f1bea9d6e6ef1a41403ab4c799a358a58b111e29e370a1f51673
+size 4991252736

model-00133-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ed9d4da503b91de11fcc92e16f4c354757860295ad7f44ce83f520644371baba
+size 4941123984

model-00139-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0c44ca65803b016ed4a82266e07f80ce21144380dd2fd9ec8fc06132d1616b65
+size 4991252704

model-00146-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:19c66a1f0eae58cd0f7ebd58c76c221bf1c781caf36ae2d58a026f1141d52767
+size 4991252840

model-00150-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3f58ccf8d780d38e6754367ce523cc22ad1f80858410f073a1bd34a40ebcb3db
+size 4991252840

model-00152-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:abdfaa85e1235e3efe5b709e1fd9281c77bb5955ea2142e4912ec851b771ad23
+size 4987305808

model-00157-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d2f61db13515db18119bc05c882e9631c99f644a603c9f12b9345d296d8586c7
+size 4991252624

model-00158-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:379007ec6008c6745f75040054ad78a713e106e7ef8afc6bc56bd6602f9f8a16
+size 4991252840

model-00164-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cbe513a8a266c31a7c8188eca763c1dfa82610ba20aede6f458643e705c34ade
+size 4991252840

model-00168-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f25819de47ee6a97ff3a4cef748e4f60df20140050db53291d0d60593ea2c235
+size 4991252840

model-00172-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3a61fc51c38f5b5d452b25804c036d1d383df4c65f2f6871ead7f7cb7fcf256b
+size 4991252840

model-00174-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fe74963526f40353c5d12a0239a11b631f497ef37807830ac63e8d6323140fb6
+size 4991252840

model-00193-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7f5ae48a5eb8c08cc7ab5dddc5d2777501befb7fc4e4514982ae8e51fc86656a
+size 4987305848

model-00197-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:70fa27ffd962d6e67ce839bbb362d38ef722205853c09e77deb4692cff276763
+size 4962095640

model-00205-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d53be34e4784d5d9e4598df3902eb93854b5b9320359377065e2750adb7634db
+size 4991253080

model-00217-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:30119d9e3375f2dc6ced76533b9ba1d8b36e484c2f31264c2e7e8d3dc89d1788
+size 4991253008

model-00225-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4bc4634ae9e4ef06ae73896428267364147de05dad5079ec25b3f30f0fbbac98
+size 4605389936

model.safetensors.index.json ADDED Viewed

The diff for this file is too large to render. See raw diff

special_tokens_map.json ADDED Viewed

	@@ -0,0 +1,30 @@

+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json ADDED Viewed

The diff for this file is too large to render. See raw diff

ultra_v3_reasoning_parser.py ADDED Viewed

	@@ -0,0 +1,28 @@

+from vllm.reasoning.abs_reasoning_parsers import ReasoningParserManager
+from vllm.reasoning.deepseek_r1_reasoning_parser import DeepSeekR1ReasoningParser
+@ReasoningParserManager.register_module("ultra_v3")
+class UltraV3ReasoningParser(DeepSeekR1ReasoningParser):
+    def extract_reasoning(self, model_output, request):
+        reasoning_content, final_content = super().extract_reasoning(
+            model_output, request
+        )
+        if (
+            hasattr(request, "chat_template_kwargs")
+            and request.chat_template_kwargs
+            and (
+                request.chat_template_kwargs.get("enable_thinking") is False
+                or request.chat_template_kwargs.get("force_nonempty_content") is True
+            )
+            and final_content is None
+        ):
+            """
+            The original `deepseek_r1` reasoning parser this inherits from will automatically put everything in the reasoning content when it cannot parse out reasoning. This was fine for the DeepSeek R1 model that was not intended to be used without reasoning.
+            1. Since the Nemotron 3 Nano and Super both have thinking off modes modulated by "enable_thinking=false" in the chat template kwargs, this change instead which will properly place the content in cases where there is no thinking enabled via config.
+            2. There are rare cases where the model will output only reasoning without an end-think token `</think>` (e.g. reasoning exceeds max length), which results in empty content returned. End users may want to unilaterally avoid such cases and always have a content response even if the model does not finish its reasoning.
+            """
+            # Put all nonempty content into the content, rather than return content
+            reasoning_content, final_content = None, reasoning_content
+        return reasoning_content, final_content