Instructions to use CycloneDX/cdx1-14B-BF16-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use CycloneDX/cdx1-14B-BF16-GGUF with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="CycloneDX/cdx1-14B-BF16-GGUF")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("CycloneDX/cdx1-14B-BF16-GGUF")
model = AutoModelForCausalLM.from_pretrained("CycloneDX/cdx1-14B-BF16-GGUF")

llama-cpp-python

How to use CycloneDX/cdx1-14B-BF16-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="CycloneDX/cdx1-14B-BF16-GGUF",
	filename="cdx1-14B-bf16.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use CycloneDX/cdx1-14B-BF16-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf CycloneDX/cdx1-14B-BF16-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf CycloneDX/cdx1-14B-BF16-GGUF:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf CycloneDX/cdx1-14B-BF16-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf CycloneDX/cdx1-14B-BF16-GGUF:BF16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf CycloneDX/cdx1-14B-BF16-GGUF:BF16
# Run inference directly in the terminal:
./llama-cli -hf CycloneDX/cdx1-14B-BF16-GGUF:BF16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf CycloneDX/cdx1-14B-BF16-GGUF:BF16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf CycloneDX/cdx1-14B-BF16-GGUF:BF16

Use Docker

docker model run hf.co/CycloneDX/cdx1-14B-BF16-GGUF:BF16

LM Studio
Jan

vLLM

How to use CycloneDX/cdx1-14B-BF16-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "CycloneDX/cdx1-14B-BF16-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CycloneDX/cdx1-14B-BF16-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/CycloneDX/cdx1-14B-BF16-GGUF:BF16

SGLang

How to use CycloneDX/cdx1-14B-BF16-GGUF with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "CycloneDX/cdx1-14B-BF16-GGUF" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CycloneDX/cdx1-14B-BF16-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "CycloneDX/cdx1-14B-BF16-GGUF" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CycloneDX/cdx1-14B-BF16-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use CycloneDX/cdx1-14B-BF16-GGUF with Ollama:
```
ollama run hf.co/CycloneDX/cdx1-14B-BF16-GGUF:BF16
```

Unsloth Studio

How to use CycloneDX/cdx1-14B-BF16-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for CycloneDX/cdx1-14B-BF16-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for CycloneDX/cdx1-14B-BF16-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for CycloneDX/cdx1-14B-BF16-GGUF to start chatting

How to use CycloneDX/cdx1-14B-BF16-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf CycloneDX/cdx1-14B-BF16-GGUF:BF16

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "CycloneDX/cdx1-14B-BF16-GGUF:BF16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use CycloneDX/cdx1-14B-BF16-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf CycloneDX/cdx1-14B-BF16-GGUF:BF16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default CycloneDX/cdx1-14B-BF16-GGUF:BF16

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use CycloneDX/cdx1-14B-BF16-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf CycloneDX/cdx1-14B-BF16-GGUF:BF16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "CycloneDX/cdx1-14B-BF16-GGUF:BF16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use CycloneDX/cdx1-14B-BF16-GGUF with Docker Model Runner:
```
docker model run hf.co/CycloneDX/cdx1-14B-BF16-GGUF:BF16
```

Lemonade

How to use CycloneDX/cdx1-14B-BF16-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull CycloneDX/cdx1-14B-BF16-GGUF:BF16

Run and chat with the model

lemonade run user.cdx1-14B-BF16-GGUF-BF16

List all available models

lemonade list

prabhuat commited on Aug 8, 2025

Commit

2073663

verified ·

1 Parent(s): f0bbba3

Update README.md

Browse files

Files changed (1) hide show

README.md +82 -42

README.md CHANGED Viewed

@@ -51,48 +51,88 @@ Model responses were scored using a combination of automated evaluation by a hig
 ### Logic Category Comparison
-The Logic category tests reasoning and problem-solving skills. The table below compares the accuracy of nine models on these tasks.
-| Model | Accuracy (%) |
-| :--- | :--- |
-| **cdx1-mlx-8bit** | **46.04** |
-| **cdx1-pro-mlx-8bit** | **73.17** |
-| gemini-2.5-pro | 93.60 |
-| o4-mini-high | 67.99 |
-| qwen3-coder-480B | 48.48 |
-| deepthink-r1 | 89.63 |
-| deepseek-r1 | 82.92 |
-| gpt-oss-120b | 80.49 |
-| gpt-oss-20b | 79.27 |
-**Summary of Results:**
-- **Top Performer:** `gemini-2.5-pro` achieved the highest accuracy at 93.6%.
-- **High Performers:** A group of models demonstrated strong reasoning, including `deepthink-r1` (89.6%), `deepseek-r1` (82.9%), and `gpt-oss-120b` (80.5%).
-- **Specialized Model Performance:** `cdx1-pro` (30B parameters) performed competitively at 73.2%. The score for `cdx1` (14B parameters) was 46.0%, a result attributed primarily to context length limitations rather than a fundamental deficiency in logic.
-- **Performance Tiers:** The results indicate distinct performance tiers, with a significant gap between the top-performing models (>80%) and others.
-### Specification Category Comparison
-The Spec category tests the recall of factual information from technical specifications.
-| Model | Accuracy (%) |
-| :--- | :--- |
-| **cdx1-mlx-8bit** | **83.52** |
-| **cdx1-pro-mlx-8bit** | **98.3** |
-| gemini-2.5-pro | 100 |
-| o4-mini-high | 0 |
-| qwen3-coder-480B | 90.34 |
-| deepthink-r1 | 12.36 |
-| deepseek-r1 | 98.58 |
-| gpt-oss-120b | 89.2 |
-| gpt-oss-20b | 9.09 |
-**Summary of Results:**
-- **Near-Perfect Recall:** `gemini-2.5-pro` (100%), `deepseek-r1` (98.6%), and `cdx1-pro` (98.3%) demonstrated exceptional performance.
-- **Behavioral Failures:** Three models scored poorly due to operational issues rather than a lack of knowledge. `o4-mini-high` (0%) refused to answer, while `deepthink-r1` (12.4%) and `gpt-oss-20b` (9.1%) answered only a small fraction of questions.
-- **`cdx1` Performance:** The smaller `cdx1` model scored 83.5%. Its performance was negatively affected by a systematic misunderstanding of certain technical terms, highlighting the challenge of ensuring factual accuracy in highly specialized domains.
 ### Other Categories

 ### Logic Category Comparison
+| Model               | Accuracy (%) |
+| :------------------ | :----------- |
+| `gemini-2.5-pro`    | 93.60        |
+| `deepthink-r1`      | 89.63        |
+| `gpt-5`             | 83.23        |
+| `deepseek-r1`       | 82.92        |
+| `gpt-oss-120b`      | 80.49        |
+| `gpt-oss-20b`       | 79.27        |
+| `cdx1-pro-mlx-8bit` | 73.17        |
+| `o4-mini-high`      | 67.99        |
+| `qwen3-coder-480B`  | 48.48        |
+| `cdx1-mlx-8bit`     | 46.04        |
+This table compares the accuracy of **ten** different AI models on a logic benchmark designed to assess reasoning and problem-solving skills. The results highlight a clear hierarchy of performance, with the newly added `gpt-5` debuting as a top-tier model.
+**Key Findings from the Chart:**
+- **Dominant Leader:** `gemini-2.5-pro` is the undisputed leader, achieving the highest accuracy of **93.6%**, placing it in a class of its own.
+- **Top-Tier Competitors:** A strong group of models follows, led by `deepthink-r1` at **89.63%**. The newly introduced **`gpt-5`** makes a powerful debut, securing the third-place spot with **83.23%** accuracy. It slightly outperforms `deepseek-r1` (82.92%) and `gpt-oss-120b` (80.49%).
+- **Strong Mid-Tier:** The `gpt-oss-20b` model performs impressively well for its size at **79.27%**, outscoring several larger models and leading the middle pack, which also includes `cdx1-pro-mlx-8bit` (73.17%) and `o4-mini-high` (67.99%).
+- **Lower Performers:** `qwen3-coder-480B` (48.48%) and `cdx1-mlx-8bit` (46.04%) score the lowest. It is noted that the score for `cdx1-mlx-8bit` is artificially low due to context length limitations, which caused it to miss questions.
+- **Efficiency and Performance:** The results from the `gpt-oss` models, particularly the 20B variant, demonstrate that highly optimized, smaller models can be very competitive on logic tasks.
+### Performance Tiers
+The models can be grouped into four clear performance tiers:
+- **Elite Tier (>90%):**
+  - `gemini-2.5-pro` (93.6%)
+- **High-Performing Tier (80%-90%):**
+  - `deepthink-r1` (89.63%)
+  - `gpt-5` (83.23%)
+  - `deepseek-r1` (82.92%)
+  - `gpt-oss-120b` (80.49%)
+- **Mid-Tier (65%-80%):**
+  - `gpt-oss-20b` (79.27%)
+  - `cdx1-pro-mlx-8bit` (73.17%)
+  - `o4-mini-high` (67.99%)
+- **Lower Tier (<50%):**
+  - `qwen3-coder-480B` (48.48%)
+  - `cdx1-mlx-8bit` (46.04%)
+### Spec Category Comparison
+| Model               | Accuracy (%) |
+| :------------------ | :----------- |
+| `gemini-2.5-pro`    | 100.00       |
+| `deepseek-r1`       | 98.58        |
+| `cdx1-pro-mlx-8bit` | 98.30        |
+| `gpt-5`             | 95.17        |
+| `qwen3-coder-480B`  | 90.34        |
+| `gpt-oss-120b`      | 89.20        |
+| `cdx1-mlx-8bit`     | 83.52        |
+| `deepthink-r1`      | 12.36        |
+| `gpt-oss-20b`       | 9.09         |
+| `o4-mini-high`      | 0.00         |
+This table evaluates **ten** AI models on the "Spec Category," a test of factual recall on 352 technical specification questions. The results starkly illustrate that a model's reliability and cooperative behavior are as crucial as its underlying knowledge. Several models, including the newly added `gpt-5`, achieved high scores only after overcoming significant behavioral hurdles.
+**Key Findings from the Chart:**
+- **Elite Factual Recall:** A top tier of models demonstrated near-perfect knowledge retrieval. **`gemini-2.5-pro`** led with a perfect **100%** score and superior answer depth. It was closely followed by **`deepseek-r1`** (98.58%) and **`cdx1-pro-mlx-8bit`** (98.3%).
+- **High Score with Major Caveats (`gpt-5`):** The newly added **`gpt-5`** achieved a high accuracy of **95.17%**, placing it among the top performers. However, this result required a significant compromise:
+  - The model initially refused to answer the full set of questions, only offering to respond in small batches that required six separate user confirmations. This compromise was accepted to prevent an outright failure.
+  - A related variant, `gpt-5-thinking`, refused the test entirely after a minute of processing.
+- **Complete Behavioral Failures:** Three models effectively failed the test not due to a lack of knowledge, but because they refused to cooperate:
+  - **`o4-mini-high`** scored **0%** after refusing to answer, citing too many questions.
+  - **`deepthink-r1`** (12.36%) and **`gpt-oss-20b`** (9.09%) also failed, answering only a small fraction of the questions without acknowledging the limitation.
+- **Strong Mid-Tier Performers:** `qwen3-coder-480B` (90.34%) and `gpt-oss-120b` (89.2%) both demonstrated strong and reliable factual recall without the behavioral issues seen elsewhere.
+- **Impact of Scale and Systematic Errors:** The contrast between the two `cdx1` models is revealing. The larger `cdx1-pro-mlx-8bit` (98.3%) performed exceptionally well, while the smaller `cdx1-mlx-8bit` (83.52%) was hampered by a single systematic error (misunderstanding "CBOM"), which cascaded into multiple wrong answers.
+### Summary of Key Themes
+1.  **Reliability is Paramount:** This test's most important finding is that knowledge is useless if a model is unwilling or unable to share it. The failures of `o4-mini-high`, `deepthink-r1`, `gpt-oss-20b`, and the behavioral friction from `gpt-5` highlight this critical dimension.
+2.  **Scores Don't Tell the Whole Story:** The 95.17% score for `gpt-5` obscures the significant user intervention required to obtain it. Similarly, the near-identical scores of `cdx1-pro` and `gemini-2.5-pro` don't capture Gemini's superior answer quality.
+3.  **Scale Can Overcome Flaws:** The dramatic performance leap from the 14B to the 30B `cdx1` model suggests that increased scale can help correct for specific knowledge gaps and improve overall accuracy.
 ### Other Categories