Instructions to use infohound/cogito-14b-voice-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use infohound/cogito-14b-voice-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf infohound/cogito-14b-voice-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf infohound/cogito-14b-voice-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf infohound/cogito-14b-voice-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf infohound/cogito-14b-voice-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf infohound/cogito-14b-voice-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf infohound/cogito-14b-voice-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf infohound/cogito-14b-voice-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf infohound/cogito-14b-voice-GGUF:Q4_K_M

Use Docker

docker model run hf.co/infohound/cogito-14b-voice-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use infohound/cogito-14b-voice-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "infohound/cogito-14b-voice-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "infohound/cogito-14b-voice-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/infohound/cogito-14b-voice-GGUF:Q4_K_M

Ollama
How to use infohound/cogito-14b-voice-GGUF with Ollama:
```
ollama run hf.co/infohound/cogito-14b-voice-GGUF:Q4_K_M
```

Unsloth Studio

How to use infohound/cogito-14b-voice-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for infohound/cogito-14b-voice-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for infohound/cogito-14b-voice-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for infohound/cogito-14b-voice-GGUF to start chatting

How to use infohound/cogito-14b-voice-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf infohound/cogito-14b-voice-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "infohound/cogito-14b-voice-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

OpenClaw new

How to use infohound/cogito-14b-voice-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf infohound/cogito-14b-voice-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "infohound/cogito-14b-voice-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use infohound/cogito-14b-voice-GGUF with Docker Model Runner:
```
docker model run hf.co/infohound/cogito-14b-voice-GGUF:Q4_K_M
```

Lemonade

How to use infohound/cogito-14b-voice-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull infohound/cogito-14b-voice-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.cogito-14b-voice-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use infohound/cogito-14b-voice-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf infohound/cogito-14b-voice-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default infohound/cogito-14b-voice-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

cogito-14b-voice-GGUF

Voice-optimized GGUF quantizations of deepcogito/cogito-v1-preview-qwen-14B, specifically calibrated for Home Assistant voice assistant tool-calling workloads.

What Makes This Different

Standard GGUF quantizations use generic calibration data (wiki text, general prose). These quantizations use an importance matrix generated from actual voice assistant interactions — system prompts, 54 tool definitions, voice commands, and structured tool-call responses.

This means the quantizer preserves precision on the weights that matter most for:

Structured JSON tool-call generation
Entity name matching from natural speech
Concurrent tool definition handling (50+ tools)
Short, concise voice responses

The result: smaller models that maintain tool-calling accuracy compared to generic quants of the same size.

Available Quantizations

Filename	Quant	Size	BPW	Tool Accuracy*	Avg Latency**	Use Case
`cogito-14b-voice-IQ3_XS.gguf`	IQ3_XS	6.0 GB	3.3	80% (16/20)	1.1s	Too aggressive for 50+ tools
`cogito-14b-voice-IQ4_XS.gguf`	IQ4_XS	7.6 GB	4.25	80% (16/20)	1.3s	Recommended for voice
`cogito-14b-voice-Q3_K_M.gguf`	Q3_K_M	6.9 GB	~3.7	70% (14/20)	1.3s	Traditional quant format
`cogito-14b-voice-Q4_K_M.gguf`	Q4_K_M	8.4 GB	~4.6	—	—	Reference (with imatrix)
`cogito-14b-voice-Q4_K_S.gguf`	Q4_K_S	8.0 GB	~4.4	—	—	Reference (with imatrix)
`cogito-14b-voice-IQ3_M.gguf`	IQ3_M	6.5 GB	3.66	—	—	Reference

* Tool accuracy measured with OpenAI-compatible tools API, 20 voice commands, 54 exposed entities, strict matching. ** Average latency on NVIDIA Blackwell (DGX Spark GB10). RTX 4080 SUPER latency will differ.

Recommended: IQ4_XS (7.6 GB)

The IQ4_XS quantization is the recommended choice — 16% smaller than a standard Q4_K_M (9.0 GB → 7.6 GB) while maintaining reliable tool-calling accuracy in production with 50+ concurrent tool definitions.

Important: IQ3_XS (6.0 GB) passes synthetic benchmarks but is too aggressive for production use with 50+ exposed entities. In real-world Home Assistant testing, IQ3_XS produced wrong tool calls (e.g., calling HassVacuumStart instead of HassTurnOff) and empty responses. Use IQ4_XS or higher for reliable voice assistant operation.

Why cogito:14b for Voice

Several models were tested for Home Assistant voice pipelines. cogito:14b was chosen because it:

Does NOT generate thinking/reasoning tokens — Models that produce <think> tokens (qwen3, etc.) cause empty responses via OpenAI-compatible APIs, resulting in the Voice PE showing a red error ring
Handles 50+ concurrent tool definitions without degradation
Produces clean, structured tool-call JSON in OpenAI format
Reasonable latency (~4s on RTX 4080 SUPER with standard Q4_K_M, ~1s on Blackwell)

Voice Pipeline Architecture

These models are designed for the following Home Assistant voice pipeline:

Voice PE (ESP32)         Your Server                    Home Assistant
┌─────────────┐    ┌─────────────────────────┐    ┌─────────────────┐
│ openWakeWord │───>│ Whisper (STT)           │    │ Voice Pipeline   │
│ "Hey Jarvis" │    │ Piper (TTS)             │<──>│ local_openai     │
│              │    │ Ollama/llama.cpp (LLM)  │    │ integration      │
└─────────────┘    └─────────────────────────┘    └─────────────────┘

The LLM receives:

System prompt (~500-1000 tokens) — personality, camera/device instructions
Tool definitions (~3,000-5,000 tokens) — HassTurnOn, HassTurnOff, HassLightSet, HassVacuumStart, etc.
User message (5-20 tokens) — transcribed speech
Total context per request: ~4,000-6,000 tokens

How to Use

With Ollama

# Create a Modelfile
cat > Modelfile << 'EOF'
FROM ./cogito-14b-voice-IQ3_XS.gguf
PARAMETER temperature 0.1
PARAMETER num_ctx 4096
EOF

# Create and run the model
ollama create cogito-voice -f Modelfile
ollama run cogito-voice

With llama.cpp

llama-server \
  -m cogito-14b-voice-IQ3_XS.gguf \
  -ngl 99 \
  -c 4096 \
  -fa on \
  --port 8080

With Home Assistant (via LiteLLM)

Add to your LiteLLM config.yaml:

- model_name: voice/cogito-14b
  litellm_params:
    model: openai/cogito-voice
    api_base: http://your-ollama-host:11434/v1
  model_info:
    supports_function_calling: true

Then configure the local_openai integration in HA to use voice/cogito-14b.

Importance Matrix Methodology

The importance matrix was generated using llama-imatrix from llama.cpp with a custom calibration dataset containing:

50 voice interaction examples covering all tool types:
- Light control (on/off, brightness, color, temperature)
- Camera script activation (8 cameras × 4 TVs)
- Vacuum robot control (start, return to base)
- Shopping list management (add, remove, complete)
- Scene activation
- DateTime queries
- Broadcast messages
Full system prompt with personality and camera routing instructions
54 entity definitions matching a real Home Assistant deployment
Proper chat template format (<|im_start|> / <|im_end|> Qwen2 format)

The calibration data was processed through the F16 GGUF on an NVIDIA DGX Spark (Blackwell GB10, 128 GB unified memory) at 772 tokens/sec.

Benchmark Details

Test Setup

Hardware: NVIDIA DGX Spark (Blackwell GB10, 128 GB LPDDR5x unified memory)
Server: llama.cpp with flash attention, 4096 context, full GPU offload
API: OpenAI-compatible /v1/chat/completions with tools parameter
Temperature: 0.1
Test set: 20 diverse voice commands across all tool types
Evaluation: Strict matching (correct tool name AND correct primary argument)

Test Cases and Results (IQ3_XS)

Voice Command	Expected Tool	Result	Latency
"Turn on the living room lights"	HassTurnOn	PASS*	1,887ms
"Show the front door on the office TV"	HassTurnOn (script)	PASS	1,126ms
"Set the bedroom light to 50 percent"	HassLightSet	PASS	1,348ms
"Start the vacuum"	HassVacuumStart	PASS	986ms
"Add milk to the shopping list"	HassListAddItem	PASS	1,063ms
"Stop all cameras"	HassTurnOn (script)	FAIL**	1,050ms
"Turn off the hot tub lights"	HassTurnOff	PASS*	1,095ms
"Show the backyard on the bedroom TV"	HassTurnOn (script)	PASS	1,160ms
"Dim the playroom lamp to 30%"	HassLightSet	PASS	1,251ms
"Send Ted home"	HassVacuumReturnToBase	PASS	1,050ms
"Turn on both corner lights"	HassTurnOn	PASS	1,011ms
"What time is it"	GetDateTime	PASS	555ms
"Show the mailbox on the living room TV"	HassTurnOn (script)	PASS	1,191ms
"Turn off Tristan's lamp"	HassTurnOff	PASS	1,127ms
"Make the desk lights blue"	HassLightSet	PASS	1,229ms
"Announce dinner is ready"	HassBroadcast	PASS	843ms
"Stop the bedroom TV camera"	HassTurnOn (script)	FAIL**	1,035ms
"Turn on the playroom lamp"	HassTurnOn	PASS	1,041ms
"Show the driveway on the playroom TV"	HassTurnOn (script)	PASS	1,204ms
"Add dog food to the shopping list"	HassListAddItem	PASS	1,067ms

* Uses entity_id format instead of friendly name — functionally correct, HA handles both. ** Uses HassTurnOff instead of HassTurnOn for stop scripts — minor tool name mismatch.

Comparison with Baseline

Metric	Standard Q4_K_M (9.0 GB)	Voice IQ3_XS (6.0 GB)
Size	9.0 GB	6.0 GB (-33%)
VRAM (est.)	~9 GB	~6 GB
Tool accuracy	Baseline	80% strict, ~95% functional
Avg latency (Blackwell)	~1.5s	~1.1s

Base Model

Model: deepcogito/cogito-v1-preview-qwen-14B
Architecture: Qwen2 (48 layers, 40 query heads, 8 KV heads, head_dim=128)
Parameters: 14.8B
License: Apache 2.0

Quantization Hardware

All quantization was performed on an NVIDIA DGX Spark personal AI supercomputer:

NVIDIA GB10 Grace Blackwell Superchip
128 GB LPDDR5x unified coherent memory
CUDA 13.0, llama.cpp built from source with Blackwell support

Acknowledgments

Deep Cogito for the excellent cogito base model
llama.cpp for quantization tools and importance matrix support
Home Assistant and the local_openai integration
TurboQuant research (ICLR 2026) which inspired this voice-optimized quantization approach