Instructions to use briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Cosmos
How to use briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF with Cosmos:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF:Q4_K_M
Use Docker
docker model run hf.co/briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF:Q4_K_M
- Ollama
How to use briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF with Ollama:
ollama run hf.co/briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF:Q4_K_M
- Unsloth Studio
How to use briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF to start chatting
- Pi
How to use briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent new
How to use briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat new
- OpenClaw new
How to use briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
- Docker Model Runner
How to use briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF with Docker Model Runner:
docker model run hf.co/briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF:Q4_K_M
- Lemonade
How to use briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Cosmos-Reason2-8B-Q4_K_M-GGUF-Q4_K_M
List all available models
lemonade list
license: other
license_name: nvidia-open-model-license
license_link: >-
https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license
library_name: cosmos
tags:
- nvidia
- cosmos
- conversational
- llama-cpp
- gguf-my-repo
extra_gated_prompt: >-
# NVIDIA Open Model License Agreement
Version Release Date: September 23, 2025
This NVIDIA Open Model License Agreement (the “Agreement”) is a legal
agreement between the Legal Entity You represent, or if no entity is
identified, You and NVIDIA Corporation and its Affiliates (“NVIDIA”) and
governs Your use of the Models that NVIDIA provides to You under this
Agreement. NVIDIA and You are each a “party” and collectively the “parties.”
NVIDIA models released under this Agreement are intended to be used
permissively and enable the further development of AI technologies. Subject
to the terms of this Agreement, NVIDIA confirms that:
- Models are commercially usable. - You are free to create and distribute
Derivative Models. - NVIDIA does not claim ownership to any outputs generated
using the Models or Model Derivatives.
By using, reproducing, modifying, distributing, performing or displaying any
portion or element of the Model or Derivative Model, or otherwise accepting
the terms of this Agreement, you agree to be bound by this Agreement.
## 1. Definitions
1.1. **Derivative Model** means all (a) modifications to the Model, (b) works
based on the Model, and (c) any other derivative works of the Model. An
output is not a Derivative Model.
1.2. **Legal Entity** means the union of the acting entity and all other
entities that control, are controlled by, or are under common control with
that entity. For the purposes of this definition, “control” means (a) the
power, direct or indirect, to cause the direction or management of such
entity, whether by contract or otherwise, or (b) ownership of fifty percent
(50%) or more of the outstanding shares, or (c) beneficial ownership of such
entity.
1.3. **Model** means the machine learning model, software, checkpoints, learnt
weights, algorithms, parameters, configuration files and documentation shared
under this Agreement.
1.4. **NVIDIA Cosmos Model** means a multimodal Model shared under this
Agreement.
1.5. **Special-Purpose Model** means a Model that is only competent in a
narrow set of purpose-specific tasks and should not be used for unintended or
general-purpose applications.
1.6. **You** or **Your** means an individual or Legal Entity exercising
permissions granted by this Agreement.
## 2. Conditions for Use, License Grant, AI Ethics and IP Ownership
### 2.1. Conditions for Use - The Model and any Derivative Model are subject
to additional terms as described in Section 2 and Section 3 of this
Agreement. - If You institute copyright or patent litigation against any
entity alleging that the Model or a Derivative Model constitutes infringement,
then any licenses granted will terminate as of the date such litigation is
filed. - If You bypass or disable any technical limitation, safety
guardrail, encryption, DRM, or authentication mechanism contained in the Model
without a substantially similar Guardrail, your rights will terminate. -
NVIDIA may designate a Model as a Special-Purpose Model. - NVIDIA may update
this Agreement to comply with legal and regulatory requirements.
### 2.2. License Grant NVIDIA grants You a perpetual, worldwide,
non-exclusive, no-charge, royalty-free, revocable license to publicly perform,
publicly display, reproduce, use, create derivative works of, make, have made,
sell, offer for sale, distribute, and import the Model.
### 2.3. AI Ethics Use of the Models must be consistent with NVIDIA’s
[Trustworthy AI
terms](https://www.nvidia.com/en-us/agreements/trustworthy-ai/terms/).
### 2.4. IP Ownership - NVIDIA owns the Model and any Model Derivatives it
creates. - You own your Model Derivatives. - NVIDIA claims no ownership
rights in outputs. - Except as expressly granted, NVIDIA reserves all
rights.
## 3. Redistribution
You may reproduce and distribute copies of the Model or Derivative Models in
any medium, with or without modifications, provided that:
- **3.1.** You must provide recipients with a copy of this Agreement and
include this attribution in a “Notice” text file:
*“Licensed by NVIDIA Corporation under the NVIDIA Open Model License”*
- **3.2.** If distributing or making available a NVIDIA Cosmos Model, or
products/services derived from it, you must include:
*“Built on NVIDIA Cosmos”*
- **3.3.** You may add your own copyright statements and license terms for
your modifications, provided use still complies with this Agreement.
## 4. Separate Components The Models may include components licensed under
separate legal notices (e.g., Open Source Software Licenses). These terms
apply, except where overridden by this Agreement unless required by
third-party license terms.
## 5. Trademarks No permission is granted to use NVIDIA’s trade names,
trademarks, or product names, except for reasonable descriptive use.
## 6. Disclaimer of Warranty The Model is provided **“AS IS”**, without
warranties of any kind, including title, non-infringement, merchantability, or
fitness for purpose. You assume risks associated with its use.
## 7. Limitation of Liability NVIDIA is not liable for damages (direct,
indirect, incidental, or consequential) arising from use of the Model, unless
required by law.
## 8. Indemnity You will indemnify and hold NVIDIA harmless against claims
from third parties arising from your use or distribution of the Model,
derivatives, or outputs.
## 9. Feedback NVIDIA may use any feedback you provide without restriction or
compensation.
## 10. Governing Law This Agreement is governed by U.S. and Delaware law.
Courts in Santa Clara County, California, have exclusive jurisdiction, except
for urgent injunctive relief.
## 11. Trade and Compliance You must comply with all export, import, trade,
and sanctions laws, including U.S. Export Administration Regulations and OFAC
rules.
extra_gated_fields:
By clicking Submit below, I accept the terms of the NVIDIA Open Model License Agreement and acknowledge that I am an adult of legal age of majority in the country in which the Cosmos Models will be used and have authority to accept this Agreement: checkbox
extra_gated_description: >-
The information you provide will be collected, stored, processed and shared in
accordance with the [NVIDIA Privacy
Policy](https://www.nvidia.com/en-us/about-nvidia/privacy-policy/).
extra_gated_button_content: Submit
base_model: nvidia/Cosmos-Reason2-8B
pipeline_tag: image-text-to-text
briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF
This model was converted to GGUF format from nvidia/Cosmos-Reason2-8B using llama.cpp via the ggml.ai's GGUF-my-repo space.
Refer to the original model card for more details on the model.
Use with llama.cpp
Install llama.cpp through brew (works on Mac and Linux)
brew install llama.cpp
Invoke the llama.cpp server or the CLI.
CLI:
llama-cli --hf-repo briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF --hf-file cosmos-reason2-8b-q4_k_m.gguf -p "The meaning to life and the universe is"
Server:
llama-server --hf-repo briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF --hf-file cosmos-reason2-8b-q4_k_m.gguf -c 2048
Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well.
Step 1: Clone llama.cpp from GitHub.
git clone https://github.com/ggerganov/llama.cpp
Step 2: Move into the llama.cpp folder and build it with LLAMA_CURL=1 flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
cd llama.cpp && LLAMA_CURL=1 make
Step 3: Run inference through the main binary.
./llama-cli --hf-repo briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF --hf-file cosmos-reason2-8b-q4_k_m.gguf -p "The meaning to life and the universe is"
or
./llama-server --hf-repo briliantnugraha/Cosmos-Reason2-8B-Q4_K_M-GGUF --hf-file cosmos-reason2-8b-q4_k_m.gguf -c 2048