Text Generation
Transformers
Safetensors
PyTorch
nemotron_h_puzzle
nvidia
nemotron-3
latent-moe
mtp
conversational
custom_code
8-bit precision
modelopt
Instructions to use nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4
- SGLang
How to use nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4 with Docker Model Runner:
docker model run hf.co/nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4
| [PRIMARY MODEL LICENSE] | |
| Use of this model is governed by the OpenMDWv1.1 License. | |
| A copy of the OpenMDWv1.1 License is included below for convenience. | |
| Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | |
| OpenMDW License Agreement, version 1.1 (OpenMDW-1.1) | |
| By exercising rights granted to you under this agreement, you accept and agree | |
| to its terms. | |
| As used in this agreement, "Model Materials" means the materials provided to | |
| you under this agreement, consisting of: (1) one or more machine learning | |
| models (including architecture and parameters); and (2) all related artifacts | |
| (including associated data, documentation and software) that are provided to | |
| you hereunder. | |
| Subject to your compliance with this agreement, permission is hereby granted, | |
| free of charge, to deal in the Model Materials without restriction, including | |
| under all copyright, patent, database, and trade secret rights included or | |
| embodied therein. | |
| If you distribute any portion of the Model Materials, you shall retain in your | |
| distribution (1) a copy of this agreement, and (2) all copyright notices and | |
| other notices of origin included in the Model Materials that are applicable to | |
| your distribution. | |
| If you file, maintain, or voluntarily participate in a lawsuit against any | |
| person or entity asserting that the Model Materials directly or indirectly | |
| infringe any patent or copyright, then all rights and grants made to you | |
| hereunder are terminated, unless that lawsuit was in response to a | |
| corresponding lawsuit first brought against you. | |
| This agreement does not impose any restrictions or obligations with respect to | |
| any use, modification, or sharing of any outputs generated by using the Model | |
| Materials. | |
| THE MODEL MATERIALS ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS | |
| OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| FITNESS FOR A PARTICULAR PURPOSE, TITLE, NONINFRINGEMENT, ACCURACY, OR THE | |
| ABSENCE OF LATENT OR OTHER DEFECTS OR ERRORS, WHETHER OR NOT DISCOVERABLE, ALL | |
| TO THE GREATEST EXTENT PERMISSIBLE UNDER APPLICABLE LAW. | |
| YOU ARE SOLELY RESPONSIBLE FOR (1) CLEARING RIGHTS OF OTHER PERSONS THAT MAY | |
| APPLY TO THE MODEL MATERIALS OR ANY USE THEREOF, INCLUDING WITHOUT LIMITATION | |
| ANY PERSON'S COPYRIGHTS OR OTHER RIGHTS INCLUDED OR EMBODIED IN THE MODEL | |
| MATERIALS; (2) OBTAINING ANY NECESSARY CONSENTS, PERMISSIONS OR OTHER RIGHTS | |
| REQUIRED FOR ANY USE OF THE MODEL MATERIALS; OR (3) PERFORMING ANY DUE | |
| DILIGENCE OR UNDERTAKING ANY OTHER INVESTIGATIONS INTO THE MODEL MATERIALS OR | |
| ANYTHING INCORPORATED OR EMBODIED THEREIN. | |
| IN NO EVENT SHALL THE PROVIDERS OF THE MODEL MATERIALS BE LIABLE FOR ANY CLAIM, | |
| DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR | |
| OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE MODEL MATERIALS, THE | |
| USE THEREOF OR OTHER DEALINGS THEREIN. | |
| ---------------------------------------------------------------------- | |
| Files under different licenses | |
| ---------------------------------------------------------------------- | |
| The files `configuration_nemotron_h.py` and `modeling_nemotron_h.py` are licensed separately under the Apache License 2.0. | |
| A copy of the Apache-2.0 license is included below for convenience. | |
| The Apache-2.0 license applies ONLY to `configuration_nemotron_h.py` and `modeling_nemotron_h.py` and not to the rest of | |
| the repository contents. | |
| ---------------------------------------------------------------------- | |
| Apache License 2.0 (applies only to `configuration_nemotron_h.py` and `modeling_nemotron_h.py`) | |
| ---------------------------------------------------------------------- | |
| Apache License | |
| Version 2.0, January 2004 | |
| http://www.apache.org/licenses/ | |
| TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION | |
| 1. Definitions. | |
| "License" shall mean the terms and conditions for use, reproduction, | |
| and distribution as defined by Sections 1 through 9 of this document. | |
| "Licensor" shall mean the copyright owner or entity authorized by | |
| the copyright owner that is granting the License. | |
| "Legal Entity" shall mean the union of the acting entity and all | |
| other entities that control, are controlled by, or are under common | |
| control with that entity. For the purposes of this definition, | |
| "control" means (i) the power, direct or indirect, to cause the | |
| direction or management of such entity, whether by contract or | |
| otherwise, or (ii) ownership of fifty percent (50%) or more of the | |
| outstanding shares, or (iii) beneficial ownership of such entity. | |
| "You" (or "Your") shall mean an individual or Legal Entity | |
| exercising permissions granted by this License. | |
| "Source" form shall mean the preferred form for making modifications, | |
| including but not limited to software source code, documentation | |
| source, and configuration files. | |
| "Object" form shall mean any form resulting from mechanical | |
| transformation or translation of a Source form, including but | |
| not limited to compiled object code, generated documentation, | |
| and conversions to other media types. | |
| "Work" shall mean the work of authorship, whether in Source or | |
| Object form, made available under the License, as indicated by a | |
| copyright notice that is included in or attached to the work | |
| (an example is provided in the Appendix below). | |
| "Derivative Works" shall mean any work, whether in Source or Object | |
| form, that is based on (or derived from) the Work and for which the | |
| editorial revisions, annotations, elaborations, or other modifications | |
| represent, as a whole, an original work of authorship. For the purposes | |
| of this License, Derivative Works shall not include works that remain | |
| separable from, or merely link (or bind by name) to the interfaces of, | |
| the Work and Derivative Works thereof. | |
| "Contribution" shall mean any work of authorship, including | |
| the original version of the Work and any modifications or additions | |
| to that Work or Derivative Works thereof, that is intentionally | |
| submitted to Licensor for inclusion in the Work by the copyright owner | |
| or by an individual or Legal Entity authorized to submit on behalf of | |
| the copyright owner. For the purposes of this definition, "submitted" | |
| means any form of electronic, verbal, or written communication sent | |
| to the Licensor or its representatives, including but not limited to | |
| communication on electronic mailing lists, source code control systems, | |
| and issue tracking systems that are managed by, or on behalf of, the | |
| Licensor for the purpose of discussing and improving the Work, but | |
| excluding communication that is conspicuously marked or otherwise | |
| designated in writing by the copyright owner as "Not a Contribution." | |
| "Contributor" shall mean Licensor and any individual or Legal Entity | |
| on behalf of whom a Contribution has been received by Licensor and | |
| subsequently incorporated within the Work. | |
| 2. Grant of Copyright License. Subject to the terms and conditions of | |
| this License, each Contributor hereby grants to You a perpetual, | |
| worldwide, non-exclusive, no-charge, royalty-free, irrevocable | |
| copyright license to reproduce, prepare Derivative Works of, | |
| publicly display, publicly perform, sublicense, and distribute the | |
| Work and such Derivative Works in Source or Object form. | |
| 3. Grant of Patent License. Subject to the terms and conditions of | |
| this License, each Contributor hereby grants to You a perpetual, | |
| worldwide, non-exclusive, no-charge, royalty-free, irrevocable | |
| (except as stated in this section) patent license to make, have made, | |
| use, offer to sell, sell, import, and otherwise transfer the Work, | |
| where such license applies only to those patent claims licensable | |
| by such Contributor that are necessarily infringed by their | |
| Contribution(s) alone or by combination of their Contribution(s) | |
| with the Work to which such Contribution(s) was submitted. If You | |
| institute patent litigation against any entity (including a | |
| cross-claim or counterclaim in a lawsuit) alleging that the Work | |
| or a Contribution incorporated within the Work constitutes direct | |
| or contributory patent infringement, then any patent licenses | |
| granted to You under this License for that Work shall terminate | |
| as of the date such litigation is filed. | |
| 4. Redistribution. You may reproduce and distribute copies of the | |
| Work or Derivative Works thereof in any medium, with or without | |
| modifications, and in Source or Object form, provided that You | |
| meet the following conditions: | |
| (a) You must give any other recipients of the Work or | |
| Derivative Works a copy of this License; and | |
| (b) You must cause any modified files to carry prominent notices | |
| stating that You changed the files; and | |
| (c) You must retain, in the Source form of any Derivative Works | |
| that You distribute, all copyright, patent, trademark, and | |
| attribution notices from the Source form of the Work, | |
| excluding those notices that do not pertain to any part of | |
| the Derivative Works; and | |
| (d) If the Work includes a "NOTICE" text file as part of its | |
| distribution, then any Derivative Works that You distribute must | |
| include a readable copy of the attribution notices contained | |
| within such NOTICE file, excluding those notices that do not | |
| pertain to any part of the Derivative Works, in at least one | |
| of the following places: within a NOTICE text file distributed | |
| as part of the Derivative Works; within the Source form or | |
| documentation, if provided along with the Derivative Works; or, | |
| within a display generated by the Derivative Works, if and | |
| wherever such third-party notices normally appear. The contents | |
| of the NOTICE file are for informational purposes only and | |
| do not modify the License. You may add Your own attribution | |
| notices within Derivative Works that You distribute, alongside | |
| or as an addendum to the NOTICE text from the Work, provided | |
| that such additional attribution notices cannot be construed | |
| as modifying the License. | |
| You may add Your own copyright statement to Your modifications and | |
| may provide additional or different license terms and conditions | |
| for use, reproduction, or distribution of Your modifications, or | |
| for any such Derivative Works as a whole, provided Your use, | |
| reproduction, and distribution of the Work otherwise complies with | |
| the conditions stated in this License. | |
| 5. Submission of Contributions. Unless You explicitly state otherwise, | |
| any Contribution intentionally submitted for inclusion in the Work | |
| by You to the Licensor shall be under the terms and conditions of | |
| this License, without any additional terms or conditions. | |
| Notwithstanding the above, nothing herein shall supersede or modify | |
| the terms of any separate license agreement you may have executed | |
| with Licensor regarding such Contributions. | |
| 6. Trademarks. This License does not grant permission to use the trade | |
| names, trademarks, service marks, or product names of the Licensor, | |
| except as required for reasonable and customary use in describing the | |
| origin of the Work and reproducing the content of the NOTICE file. | |
| 7. Disclaimer of Warranty. Unless required by applicable law or | |
| agreed to in writing, Licensor provides the Work (and each | |
| Contributor provides its Contributions) on an "AS IS" BASIS, | |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or | |
| implied, including, without limitation, any warranties or conditions | |
| of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A | |
| PARTICULAR PURPOSE. You are solely responsible for determining the | |
| appropriateness of using or redistributing the Work and assume any | |
| risks associated with Your exercise of permissions under this License. | |
| 8. Limitation of Liability. In no event and under no legal theory, | |
| whether in tort (including negligence), contract, or otherwise, | |
| unless required by applicable law (such as deliberate and grossly | |
| negligent acts) or agreed to in writing, shall any Contributor be | |
| liable to You for damages, including any direct, indirect, special, | |
| incidental, or consequential damages of any character arising as a | |
| result of this License or out of the use or inability to use the | |
| Work (including but not limited to damages for loss of goodwill, | |
| work stoppage, computer failure or malfunction, or any and all | |
| other commercial damages or losses), even if such Contributor | |
| has been advised of the possibility of such damages. | |
| 9. Accepting Warranty or Additional Liability. While redistributing | |
| the Work or Derivative Works thereof, You may choose to offer, | |
| and charge a fee for, acceptance of support, warranty, indemnity, | |
| or other liability obligations and/or rights consistent with this | |
| License. However, in accepting such obligations, You may act only | |
| on Your own behalf and on Your sole responsibility, not on behalf | |
| of any other Contributor, and only if You agree to indemnify, | |
| defend, and hold each Contributor harmless for any liability | |
| incurred by, or claims asserted against, such Contributor by reason | |
| of your accepting any such warranty or additional liability. | |
| END OF TERMS AND CONDITIONS | |
| APPENDIX: How to apply the Apache License to your work. | |
| To apply the Apache License to your work, attach the following | |
| boilerplate notice, with the fields enclosed by brackets "[]" | |
| replaced with your own identifying information. (Don't include | |
| the brackets!) The text should be enclosed in the appropriate | |
| comment syntax for the file format. We also recommend that a | |
| file or class name and description of purpose be included on the | |
| same "printed page" as the copyright notice for easier | |
| identification within third-party archives. | |
| Copyright [yyyy] [name of copyright owner] | |
| Licensed under the Apache License, Version 2.0 (the "License"); | |
| you may not use this file except in compliance with the License. | |
| You may obtain a copy of the License at | |
| http://www.apache.org/licenses/LICENSE-2.0 | |
| Unless required by applicable law or agreed to in writing, software | |
| distributed under the License is distributed on an "AS IS" BASIS, | |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| See the License for the specific language governing permissions and | |
| limitations under the License. | |