Update vLLM usage docs: remove config_vllm.json overwrite, relax version pin, and clarify minimal required flags

README.md

@@ -167,37 +167,57 @@ print(scores.tolist())
 
 #### vLLM Usage
 
-1. Ensure you are using `vllm==0.16.0`.
-2. Clone [this model's repository](https://huggingface.co/nvidia/llama-nemotron-embed-1b-v2/tree/main).
-3. Overwrite `config.json` with `config_vllm.json`.
-4. Start the vLLM server with the following command (replace the `<path_to_the_cloned_repository>` and `<num_gpus_to_use>` with your values):
+1. Ensure you are using `vllm>=0.14.0`.
+2. Start the vLLM server with the following command:
+
+Minimal command (required):
 ```
 vllm serve \
-    <path_to_the_cloned_repository> \
-    --trust-remote-code \
-    --runner pooling \
-    --model-impl vllm \
-    --pooler-config '{\"pooling_type\": \"MEAN\"}' \
-    --data-parallel-size <num_gpus_to_use> \
-    --dtype float32 \
-    --port 8000
+    nvidia/llama-nemotron-embed-1b-v2 \
+    --trust-remote-code
 ```
 
-You can now access the model using the OpenAI sdk, for instance:
+If you already have a local copy of the model, you can also pass the local
+path instead of the HF repo ID.
+
+Optional flags:
+
+- `--dtype <float32|bfloat16|float16>` to force precision (the default is `auto`, which resolves from model config; this model defaults to BF16).
+- `--data-parallel-size <num_gpus_to_use>` for multi-GPU serving.
+- `--port 8000` to set the server port.
+
+Online serving example (OpenAI SDK):
 
 ```
 from openai import OpenAI
-client = OpenAI(base_url="http://localhost:8000/v1")
-models = client.models.list()
-model_name = models.data[0].id
+client = OpenAI(
+    base_url="http://localhost:8000/v1",
+    api_key="EMPTY",  # required by OpenAI SDK; ignored by default in local vLLM
+)
 
 response = client.embeddings.create(
     input=['query: summit define'],
-    model=model_name
+    model="nvidia/llama-nemotron-embed-1b-v2",
 )
 response.data[0].embedding
 ```
 
+Offline inference example (Python API, no server required):
+
+```
+from vllm import LLM
+
+llm = LLM(
+    model="nvidia/llama-nemotron-embed-1b-v2",
+    runner="pooling",
+    trust_remote_code=True,
+)
+
+outputs = llm.embed(["query: summit define", "passage: a summit is a meeting"])
+for output in outputs:
+    print(len(output.outputs.embedding))  # 2048
+```
+
 ### **Software Integration**
 
 **Runtime Engine:** Llama Nemotron embedding NIM
@@ -355,4 +375,3 @@ Visit the [NeMo Retriever docs page](https://docs.nvidia.com/nemo/retriever/inde
 | Describe the physical safety impact (if present). | Not Applicable |
 | Use Case Restrictions: | Use of this model is governed by the [NVIDIA Open Model License Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/). Additional Information: [Llama 3.2 Community Model License Agreement](https://www.llama.com/llama3_2/license/).  |
 | Model and dataset restrictions: | The Principle of least privilege (PoLP) is applied limiting access for dataset generation and model development. Restrictions enforce dataset access during training, and dataset license constraints adhered to. |
-

config_vllm.json

@@ -1,38 +0,0 @@
-{
-  "_name_or_path": "nvidia/llama-3.2-nv-embedqa-1b-v2",
-  "architectures": [
-    "LlamaModel"
-  ],
-  "attention_bias": false,
-  "attention_dropout": 0.0,
-  "bos_token_id": 128000,
-  "eos_token_id": 128001,
-  "head_dim": 64,
-  "hidden_act": "silu",
-  "hidden_size": 2048,
-  "initializer_range": 0.02,
-  "intermediate_size": 8192,
-  "max_position_embeddings": 131072,
-  "is_causal": false,
-  "mlp_bias": false,
-  "model_type": "llama",
-  "num_attention_heads": 32,
-  "num_hidden_layers": 16,
-  "num_key_value_heads": 8,
-  "pooling": "avg",
-  "pretraining_tp": 1,
-  "rms_norm_eps": 1e-05,
-  "rope_scaling": {
-    "factor": 32.0,
-    "high_freq_factor": 4.0,
-    "low_freq_factor": 1.0,
-    "original_max_position_embeddings": 8192,
-    "rope_type": "llama3"
-  },
-  "rope_theta": 500000.0,
-  "tie_word_embeddings": true,
-  "torch_dtype": "bfloat16",
-  "transformers_version": "4.44.2",
-  "use_cache": true,
-  "vocab_size": 128256
-}