Instructions to use caiovicentino1/Nemotron-Cascade-2-30B-A3B-HLWQ-Q5 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use caiovicentino1/Nemotron-Cascade-2-30B-A3B-HLWQ-Q5 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="caiovicentino1/Nemotron-Cascade-2-30B-A3B-HLWQ-Q5", trust_remote_code=True)

# Load model directly
from transformers import AutoTokenizer, AutoModelForMultimodalLM

tokenizer = AutoTokenizer.from_pretrained("caiovicentino1/Nemotron-Cascade-2-30B-A3B-HLWQ-Q5", trust_remote_code=True)
model = AutoModelForMultimodalLM.from_pretrained("caiovicentino1/Nemotron-Cascade-2-30B-A3B-HLWQ-Q5", trust_remote_code=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use caiovicentino1/Nemotron-Cascade-2-30B-A3B-HLWQ-Q5 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "caiovicentino1/Nemotron-Cascade-2-30B-A3B-HLWQ-Q5"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "caiovicentino1/Nemotron-Cascade-2-30B-A3B-HLWQ-Q5",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/caiovicentino1/Nemotron-Cascade-2-30B-A3B-HLWQ-Q5

SGLang

How to use caiovicentino1/Nemotron-Cascade-2-30B-A3B-HLWQ-Q5 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "caiovicentino1/Nemotron-Cascade-2-30B-A3B-HLWQ-Q5" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "caiovicentino1/Nemotron-Cascade-2-30B-A3B-HLWQ-Q5",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "caiovicentino1/Nemotron-Cascade-2-30B-A3B-HLWQ-Q5" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "caiovicentino1/Nemotron-Cascade-2-30B-A3B-HLWQ-Q5",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use caiovicentino1/Nemotron-Cascade-2-30B-A3B-HLWQ-Q5 with Docker Model Runner:
```
docker model run hf.co/caiovicentino1/Nemotron-Cascade-2-30B-A3B-HLWQ-Q5
```

caiovicentino1 commited on Apr 10

Commit

35b5989

verified ·

1 Parent(s): 6837d50

docs: add HLWQ rebrand notice (cite Han et al. PolarQuant prior art)

Browse files

Files changed (1) hide show

README.md +17 -7

README.md CHANGED Viewed

@@ -3,17 +3,27 @@ license: other
 license_name: nvidia-open-model-license
 base_model: nvidia/Nemotron-Cascade-2-30B-A3B
 tags:
-  - polarquant
-  - moe
-  - expert-offloading
-  - nemotron
-  - mamba
-  - consumer-gpu
-  - vllm
 library_name: transformers
 pipeline_tag: text-generation
 ---
 # Nemotron-Cascade-2-30B-A3B — Expert Offloading + PolarQuant Q5
 **30B MoE model at 7.6 GB VRAM, 15+ tok/s, correct output.**

 license_name: nvidia-open-model-license
 base_model: nvidia/Nemotron-Cascade-2-30B-A3B
 tags:
+- hlwq
+- polarquant
+- moe
+- expert-offloading
+- nemotron
+- mamba
+- consumer-gpu
+- vllm
 library_name: transformers
 pipeline_tag: text-generation
 ---
+> [!IMPORTANT]
+> **Naming notice (2026-04-10).** The "PolarQuant" technique used in this model is being rebranded to **HLWQ (Hadamard-Lloyd Weight Quantization)**. The change is only the name; the algorithm and the weights in this repository are unchanged.
+>
+> The rebrand resolves a name collision with an unrelated, earlier KV cache quantization method also named PolarQuant ([Han et al., arXiv:2502.02617, 2025](https://arxiv.org/abs/2502.02617)). HLWQ addresses **weight** quantization with a **deterministic Walsh-Hadamard rotation** and Lloyd-Max scalar codebook; Han et al.'s PolarQuant addresses **KV cache** quantization with a **random polar rotation**. The two methods are technically distinct.
+>
+> Existing loaders that load this repository by ID continue to work without changes. Future model uploads will use the HLWQ name.
+>
+> Reference paper for this technique: [arXiv:2603.29078](https://arxiv.org/abs/2603.29078) (v2 in preparation; v1 still uses the old name).
 # Nemotron-Cascade-2-30B-A3B — Expert Offloading + PolarQuant Q5
 **30B MoE model at 7.6 GB VRAM, 15+ tok/s, correct output.**