Add SGLang usage section to the Cosmos3-Edge model card

README.md

@@ -8,6 +8,10 @@ tags:
   - nvidia
   - cosmos
   - cosmos3
+  - vllm
+  - vllm-omni
+  - sglang
+  - sglang-diffusion
 ---
 
 # **Cosmos 3: Omnimodal World Models for Physical AI**
@@ -173,6 +177,7 @@ Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated sys
 
 - [vLLM-Omni](https://github.com/vllm-project/vllm-omni)
 - [vLLM](https://github.com/vllm-project/vllm)
+- [SGLang](https://github.com/sgl-project/sglang)
 - [PyTorch](https://github.com/nvidia/cosmos3)
 
 **Supported Hardware Microarchitecture Compatibility:**
@@ -837,6 +842,146 @@ Thinking is enabled by default. To disable it, add the following argument to the
 extra_body={"chat_template_kwargs": {"enable_thinking": False}}
 ```
 
+### SGLang
+
+[SGLang Diffusion](https://docs.sglang.io/docs/sglang-diffusion/index) can serve `nvidia/Cosmos3-Edge` through OpenAI-compatible image and video generation endpoints. Install SGLang from the main branch with diffusion dependencies, then start a server:
+
+```bash
+git clone --branch main https://github.com/sgl-project/sglang.git
+cd sglang
+pip install -e "python[diffusion]"
+pip install "cosmos-guardrail==0.3.1"
+
+sglang serve --model-path nvidia/Cosmos3-Edge
+```
+
+Cosmos 3 support in SGLang Diffusion currently requires the SGLang main branch. Switch to a stable SGLang release once Cosmos 3 support is included there.
+
+Cosmos3-Edge is a compact model that fits comfortably on a single GPU. Pass `--dit-cpu-offload false` to keep the transformer resident on device and avoid host transfers:
+
+```bash
+sglang serve \
+  --model-path nvidia/Cosmos3-Edge \
+  --dit-cpu-offload false
+```
+
+Additional GPUs shard the visual sequence with `--num-gpus N --ulysses-degree N`, or split the two classifier-free-guidance branches with `--enable-cfg-parallel`.
+
+Supported SGLang endpoints:
+
+| Mode | Endpoint | Notes |
+| --- | --- | --- |
+| Text to image | `POST /v1/images/generations` | Returns base64 image data by default |
+| Text to video | `POST /v1/videos` | Creates an async job; poll `GET /v1/videos/{id}` and download `/content` |
+| Image to video | `POST /v1/videos` | Upload the conditioning image with `input_reference` |
+
+Cosmos3-Edge is trained at 256p and 480p. SGLang applies these per-mode defaults when the request leaves them unset; generating above the trained range degrades quality sharply.
+
+| Mode | Default size | Guidance | Flow shift |
+| --- | --- | --- | --- |
+| Text to image | `640x640` | `7.0` | `3.0` |
+| Text to video / image to video | `832x480` | `5.0` | `3.0` |
+
+Supported sizes are `832x480`, `480x832`, `640x480`, `480x640`, `640x640`, `480x480`, `448x256`, `256x448`, and `256x256`.
+
+Example image-to-video request, using the conditioning image and JSON-upsampled prompt from this repo's `assets/`:
+
+```python
+import json
+import mimetypes
+import time
+from pathlib import Path
+
+import requests
+
+base_url = "http://localhost:30000"
+
+# 1. Read JSON-upsampled prompt and negative prompt
+json_prompt = json.load(open("assets/example_i2v_prompt.json"))
+negative_prompt = json.load(open("assets/negative_prompt.json"))
+
+# 2. Submit the job; SGLang's video endpoint is asynchronous
+image_path = Path("assets/example_i2v_input.jpg")
+mime_type = mimetypes.guess_type(image_path)[0] or "image/png"
+data = {
+    "prompt": json.dumps(json_prompt),
+    "negative_prompt": json.dumps(negative_prompt),
+    "size": "832x480",
+    "num_frames": "121",
+    "fps": "24",
+    "num_inference_steps": "50",
+    "guidance_scale": "5.0",
+    "flow_shift": "3.0",
+    "max_sequence_length": "4096",
+    "seed": "0",
+    "extra_params": json.dumps({"guardrails": True, "use_duration_template": False}),
+}
+
+with image_path.open("rb") as image_file:
+    files = {"input_reference": (image_path.name, image_file, mime_type)}
+    print("Sending request to server...")
+    response = requests.post(f"{base_url}/v1/videos", data=data, files=files)
+    response.raise_for_status()
+job_id = response.json()["id"]
+
+# 3. Poll until the job completes
+while True:
+    status = requests.get(f"{base_url}/v1/videos/{job_id}").json()["status"]
+    if status == "completed":
+        break
+    if status == "failed":
+        raise RuntimeError(f"Generation failed for job {job_id}")
+    time.sleep(1)
+
+# 4. Download the generated video
+output_path = Path("/tmp/cosmos3_edge_i2v.mp4")
+output_path.write_bytes(requests.get(f"{base_url}/v1/videos/{job_id}/content").content)
+print(f"Saved video to {output_path}")
+```
+
+Text-to-video needs no file upload, so post a JSON body instead of a multipart form:
+
+```python
+body = {
+    "prompt": json.dumps(json_prompt),
+    "negative_prompt": json.dumps(negative_prompt),
+    "size": "832x480",
+    "num_frames": 121,
+    "fps": 24,
+    "num_inference_steps": 50,
+    "guidance_scale": 5.0,
+    "flow_shift": 3.0,
+    "max_sequence_length": 4096,
+    "seed": 0,
+    "extra_params": {"guardrails": True, "use_duration_template": False},
+}
+response = requests.post(f"{base_url}/v1/videos", json=body)
+```
+
+Poll and download the result the same way. Text-to-image uses the synchronous `POST /v1/images/generations` endpoint, which takes a JSON body and returns base64 image data:
+
+```python
+import base64
+
+body = {
+    "prompt": json.dumps(json_prompt),
+    "negative_prompt": json.dumps(negative_prompt),
+    "size": "640x640",
+    "n": 1,
+    "seed": 0,
+    "max_sequence_length": 4096,
+    "extra_params": {"guardrails": True},
+}
+response = requests.post(f"{base_url}/v1/images/generations", json=body)
+response.raise_for_status()
+image_bytes = base64.b64decode(response.json()["data"][0]["b64_json"])
+Path("/tmp/cosmos3_edge_t2i.png").write_bytes(image_bytes)
+```
+
+SGLang accepts Cosmos 3 request options including `max_sequence_length`, `flow_shift`, `extra_params.guardrails`, `extra_params.use_duration_template`, and `extra_params.use_system_prompt`. The SGLang Diffusion endpoints cover image and video generation only, and this checkpoint ships without a sound head, so it does not generate audio.
+
+For complete serving instructions and request examples, see the [Cosmos3 SGLang cookbook](https://docs.sglang.io/cookbook/diffusion/Cosmos/Cosmos3).
+
 ## Limitations
 
 Cosmos3 may produce imperfect outputs in challenging scenarios. Generation artifacts include temporal inconsistency, unstable camera or object motion, imprecise physical interactions, and action-state drift — especially in long-horizon or high-resolution outputs. Reasoning may also be incorrect: object states, causal relationships, spatial geometry, temporal ordering, agent intent, and future outcomes can be misinferred, and complex or long-context inputs may yield hallucinated entities, inconsistent interpretations, or implausible predictions. Because the model lacks an explicit physics simulator, 3D geometry, 4D space-time evolution, object permanence, contact dynamics, and physical laws are only approximated — producing artifacts such as disappearing or morphing objects, unrealistic collisions, and physically implausible motions. Quality further degrades in out-of-distribution environments, safety-critical edge cases, and domains underrepresented in training.
@@ -845,7 +990,7 @@ Cosmos3 outputs should not be treated as physically accurate simulation, reliabl
 
 ## Inference
 
-**Acceleration Engine:** [vLLM-Omni](https://github.com/vllm-project/vllm-omni), [vLLM](https://github.com/vllm-project/vllm), [PyTorch](https://pytorch.org/)
+**Acceleration Engine:** [vLLM-Omni](https://github.com/vllm-project/vllm-omni), [vLLM](https://github.com/vllm-project/vllm), [SGLang](https://github.com/sgl-project/sglang), [SGLang Diffusion](https://docs.sglang.io/docs/sglang-diffusion/index), [PyTorch](https://pytorch.org/)
 
 **Test Hardware:** B200, H100, H20, RTX PRO 6000, DGX Station, DGX Spark, Jetson Thor, Jetson AGX Orin